A Micron/Dstillery Case Study
Dstillery, formerly Media6Degrees, is an advertising technology pioneer that was founded in 2008. Headquartered in New York, the company has rapidly grown to offices in Chicago, San Francisco, Detroit, Los Angeles and Boston. Its team of more than 100 employees helps big brands like Verizon, Hertz, and Angie’s List, deliver the right ads to the right customers with performance that dramatically exceeds that of conventional advertising.
Traditional Advertising Has Failed
“Traditional advertising methods just don’t work in the fragmented world of digital media,” says Amit Gupta, vice president of infrastructure at Dstillery. “Advertisers used to buy space in widely distributed print magazines or newspapers or on television networks, broadcasting to a mass audience and hoping to reach a small segment who might be interested. That’s just not a viable approach now.”
The Internet Helped—Sort of
“In the late 90s, we saw that any company in any field could flood the online marketplace with ads, but there was no intelligent targeting behind it,” said Gupta. Although potential customers could be reached more efficiently online, the lack of intelligence—or data science—to match the right customers to the respective products had been missing until recent years.
A Targeted Approach
Dstillery’s revolutionary approach to targeting captures the complete customer journey—from web browsing to app usage to physical visits—to deliver marketing campaigns with unprecedented scale and efficacy. This customer journey gathers mountains of raw, anonymous data, which is distilled to find custom audiences scientifically proven to most likely care about a given brand. The window of opportunity to connect these potential customers with the right ads is microscopic and fleeting, so Gupta needed a solution that offered speed and agility.
Challenges of Scale and Speed
To support a company that ingests billions of data points on a daily basis, in a world where content consumption continues to grow rapidly, Gupta’s infrastructure must not only satisfy current performance demands, but also establish the throughput for future growth. And keeping up with the future growth of big data is a huge task. According to Smithsonian. com, 5 billion gigabytes of data were generated from the beginning of human history until 2003; and now, the human race is generating that much data every 10 minutes.
“We bought a lot of SAS hard disk
drives but kept hitting performance
bottlenecks. We had to find a solution to
scale quickly without buying hundreds of
– Amit Gupta, Vice President of
Infrastructure at Dstillery
“We needed to solve what seems to be a simple problem: how do we deliver the best ROI for our clients now and in the future?” said Gupta. Dstillery started with more traditional IT design processes by building SAS hard drive arrays with plans to scale them up and out. “We bought a lot of SAS hard disk drives but kept hitting performance bottlenecks. HDDs could take us only so far,” said Gupta. “We had to determine the best solution to scale in a short amount of time without buying hundreds of additional servers.”
The solution not only had to be cost-effective, it also required extremely low latency and high throughput because Dstillery’s matching algorithms, which are a cornerstone of the company’s technology, rely on quick processing of vast amounts of data for online advertising exchanges. “We bid on inventory across several exchanges—such as MAX by Microsoft, Right Media by Yahoo, and DoubleClick by Google—on behalf of our clients and serve the marketer’s ad if we win the bid.” Easy enough, right? Not exactly. Gupta went on to explain, “The process of receiving the opportunity, bidding, winning, parsing our data set to locate the optimal ad, and placing that ad online, must all take place in less than 100 milliseconds—often in less than 60 milliseconds.”
Solving a Massive Data Problem
Dstillery uses proprietary algorithms and a robust computing architecture with Micron SSDs to make billions of complex data connections each day, connecting their customers with exactly the right audiences.
A Solid Solution Needed
After thoroughly evaluating the alternatives, Micron’s broad solid state drive (SSD) portfolio was the clear choice for Dstillery’s diverse needs and growing infrastructure.
Different Applications Require Different SSDs
“We perform 15 billion+ web transactions daily, many of which must be read from reliable storage media like Micron’s PCIe SSDs.”
– Amit Gupta
Because Dstillery is a relatively new company in a highly competitive business, cost/benefit analysis is a key part of selecting the right SSD. For example, when Dstillery needed a simple boot drive, Micron’s P400e MLC SATA SSD made more sense than a higher-end PCIe application accelerator. “For booting, I don’t really need all of the features of a higher-end drive like the P400m,” said Gupta. “For a boot drive, the P400e was my optimal choice.”
For Dstillery applications that write a significant amount of data daily (like Apache Kafka or applications used for data relevance scoring), Gupta chose higher-endurance, mainstream SATA SSDs. Starting with the P300 SSD and then moving to the P400m, Dstillery found an optimal balance between application requirements and drive cost. “Our drives have to last two to three years to justify the cost, so anything that delivers less just doesn’t make sense.”
Gupta further noted, “Dstillery performs over 15 billion web transactions daily, many of which must be read from reliable storage media, like Micron’s PCIe SSDs, so that other internal processes can use that data to make intelligent decisions.”
“Micron's PCIe SSDs ranked first in our speed test.”
– Amit Gupta
When Gupta factored in the “trip time” (the actual time spent sending content to and from the Dstillery sites), the response window was reduced even further, from 100 milliseconds to less than 40 milliseconds. “For example, imagine a data packet traveling from Atlanta to our New York data center. We then need to calculate some information, formulate a response, and deliver the content. The transit time is quick, and a significant factor in the overall system response, so it must be factored into the total time. Because our systems have to respond incredibly quickly, system performance is critical. Micron’s P420m PCIe SSDs, with their exceptional read speed, ranked first in our speed test.” Thanks to the speed of PCIe SSDs, Gupta was able to continue to scale the company’s capability without adding more HDDs or servers.
More Than Just Better Performance: Lower Power and Hosting Fees
Since Dstillery pays a hosting service based on power and rack space, optimizing each system and each application is key. Gupta noted, “If Dstillery had stayed with SAS HDDs, we would have had to scale to 2X or 4X the server count,” and pay commensurate hosting space and power fees.
A Winning Partnership
“Among the key factors in choosing Micron was their keen understanding of our costs and requirements, their success with other top-tier firms in the industry, and their suite of solutions. Once I started talking to Micron about our company’s specific goals, they showed a genuine interest in working with us to help us achieve those goals.”
Gupta knew that Micron was a key provider of enterprise SSDs to both tier-one system OEMs and Crucial SSDs, which he was already familiar with and thus helped instill a great level of confidence in Micron’s SSD products. Micron’s broad product selection—from entry-level and mainstream SATA SSDs, to PCIe application accelerators—was critical to meeting Dstillery’s diverse, application-specific needs.
“Come to think of it, it really is that simple,” Gupta said. “Our confidence in the company, great support from the Micron team, and Micron’s broad product line is what created a successful partnership.”
Download this case study