Netflix powers through 2 billion content requests per day with Java-driven architecture.
Andy Glover and Jon Schneider check processes.
Putting members in control and serving up targeted content based on their preferences has made Netflix the world’s leading internet television network. Just about everyone is familiar with its flexible, on-demand model. Some consumers are even ditching their traditional satellite and cable services in favor of an "all-you-can-eat" TV service. For about US$10 per month, Netflix members can watch as many movies and shows as they want—anytime, anywhere, on nearly any internet-connected screen. They can play, pause, and resume watching, all without commercials or commitments.
Java's innate scalability has played a key role in enabling Netflix to expand its service to more than 57 million members in 50 countries. Collectively, these viewers enjoy more than 1 billion hours of shows and movies per month, including original series, documentaries, and feature films.
"The vast majority of the services running within our architecture are built on Java and the Java Virtual Machine [JVM]," says Andrew Glover, delivery engineering manager at Netflix. "Netflix uses a stateless architecture, so as we bring in more customers we are able to bring up more instances relatively easily. We have thousands of Java processes running all the time, yet as we grow we don’t have huge infrastructure challenges. We also have a lot of open source tools that are Java-based, which makes it easy to monitor, upgrade, and scale our services."
Glover (right) and Adam Jordens confer. “The vast majority of the services running within our architecture are built on Java and the Java Virtual Machine," Glover says.
Glover has a distinguished history of executive leadership, yet his passion for writing code hasn’t wavered. Before joining Netflix, he held senior positions at Vanward Technologies, Stelligent, and App47. Today, he manages a team of seven engineers who are responsible for the continuous delivery and deployment of new software within the vast, worldwide Netflix infrastructure. "Rapid turnaround is an important differentiator for IT, and it is certainly a competitive advantage for Netflix," he emphasizes. "Thanks to our continuous delivery platform, if a business unit wants a new feature pushed into production, we can do it within a couple hours."
Glover is echoing one of the core mantras at Netflix: move fast, make decisions fast, and do it all on a solid architecture. "Because we are a data company we need to be able to facilitate rapid innovation across the globe," he adds. "We need to make decisions fast, based on the data we collect, so we can provide the viewing experiences that our customers want—and stay ahead of competitors."
"We have thousands of Java processes running all the time, yet as we grow we don't have huge infrastructure challenges," Glover says.
Netflix members love being able to browse, select, and play movies with a few clicks of a remote control or mouse, but not many of them stop to think about the sophisticated hardware and software infrastructure that delivers the content. Whenever you launch Netflix to queue up Caddyshack, Some Like it Hot, Breaking Bad, or any other movie or television show, Netflix first executes some core Java services to determine who you are, where you live, if you paid your bill, and what display device you are using. Then its content delivery network determines the closest cache from which to stream your content, buffered to the optimum audio and video quality for your connection at that moment.
"When a customer launches Netflix, behind the scenes the system kicks off about a dozen different processes to authorize that person, figure out what device he or she is using, make sure the account is current, and look at their recent activity," explains Glover. "That kind of orchestration is done with Eureka, an open source tool based on Java."
We need to make decisions fast, based on the data we collect.
Netflix stores master copies of digital films in Amazon Web Services and Amazon Simple Storage Service clouds. Each film is encoded into more than 50 versions based on video resolution and audio quality—amounting to more than a petabyte of data. From there it is distributed to content delivery networks such as Akamai, Limelight, and Level 3, which in turn feed it to local internet service providers, and on to viewers.
The infrastructure is based on a service-oriented architecture (SOA) that handles approximately 2 billion content requests per day. "We have thousands of microservices running all the time," explains Glover. "This configuration offers lots of independence to our developers as they deploy new software."
Glover, Shelley Bower, Brian Johnson, and Mike McGarr strategize. Netflix uses its continuous delivery system to schedule deployments based on up-to-the-second traffic statistics.
Netflix engineers constantly deliver new features to the company's worldwide user base via continuous delivery. It's the cornerstone of all the company's business initiatives, with engineers using continuous delivery processes to rapidly push enhancements and bug fixes to customers, at low risk and with minimal overhead. According to Glover, this automated pipeline enables engineering teams to move concise feature sets from development and testing into a stable production environment. "The Netflix software environment changes a little bit every day," he adds. "The continuous delivery environment is the conduit for speed and innovation. We don't have the luxury of downtime. We can't tell customers to come back in three hours after we have completed a deployment."
Performance problems can cause buffering errors for viewers, so Netflix uses its continuous delivery system to schedule each deployment based on up-to-the-second traffic statistics. For example, if Europe is in its peak viewing time, the delivery team might elect to delay a deployment until off-peak hours. “One of the core metrics that guides the business is the number of streaming starts per second," continues Glover. "We watch this closely in every region. If there is a dip, we are alerted immediately."
Streaming video content over the internet isn’t difficult. Technically, there is a low barrier to entry. What differentiates Netflix among video-on-demand services is all the data it collects about what customers are watching, how often they use the service, what choices they make before and after each show is completed, which genres they prefer, and many other interrelated data points. Java programs capture about a petabyte of data each day for the recommendation engine, which suggests other shows and movies that it thinks viewers will like.
Netflix also uses this data to determine what content it should buy versus produce—as well as just how to produce original content to maximize viewership. For example, when Netflix purchased the licensing rights to House of Cards, it was comfortable with the big investment because David Fincher, the showrunner, and Kevin Spacey, its star, were very popular on Netflix.
"The bread and butter of Netflix involves using data to recommend shows that customers might like," Glover says. “Everyone at Netflix plays a role in either capturing that data or making use of that data. Our business units can make changes quickly, partly because the entire infrastructure is built on Java."
The bread and butter of Netflix involves using data to recommend shows that customers might like.
Netflix Open Connect includes Java processes that are installed in third-party data centers to govern how content is exchanged, cached, and streamed. According to Glover, this SOA includes thousands of instances that communicate with one another to execute business processes. "As Sun [Microsystems] used to say, 'The network is the computer.' It's all commodity hardware running Linux Ubuntu and Java EE 7. If a node fails, we can bring up a new one easily."
While the infrastructure runs on Linux computers, video encoding is often done on Windows machines, facilitated by a Java-based framework. Java's easy portability among platforms permits flexibility, now and in the future. It also makes it easy to hire skilled engineers. "When it comes to recruiting, it is easy to find Java developers," Glover notes. "We can always find top talent because we are not chasing some brand-new technology or language that only hipsters know."
Cameron Fieber, Glover, and Zan Thrash analyze results. Netflix uses commodity hardware running Linux Ubuntu.
Glover believes Java 8 has revolutionized Java, calling it "the most significant release of Java in the last 10 years." He especially likes default methods, lambdas, and the java.util.stream API, which enables functional-style operations on streams of elements, such as Hadoop MapReduce transformations on collections. "Functional-style programming as a core language feature is a powerful construct that will have a big impact on the developmental landscape," he predicts. "Java 8 dramatically changes the Java language for the better."
Netflix uses Spring Boot as the basis of its SOA because it offers the scalability and maturity of the JVM. "Netflix is a giant SOA," Glover says. "Java platform services make it possible for a developer to quickly come up to speed and write a service that works in our architecture. Our operational footprint on the JVM is tried and true. Spring Boot is a good example of a web framework that you can get up and running in short order. There is still a tremendous amount of innovation within Java and in the Java ecosystem. It's solid. It's not going anywhere. That's why there is so much infrastructure being built on Java, such as Hadoop."
Java provides both the flexibility and scalability Netflix needs. "Our technology infrastructure is not a house of cards," he concludes. "In order to continue to be the number one streaming video content provider on the internet, we have to continually innovate. Java is the bedrock of our software architecture. The core operational infrastructure of Netflix, including the monitoring tools and platform services, has all been built in Java."
Based in Santa Barbara, California, David Baum writes about innovative businesses and emerging technologies.
Article originally written for Java Magazine