How Netflix modularized their JavaScript Codebase

Making the move to JavaScript

This approach helped Netflix to create a UI, but it did not work well on older devices. Creating applications that were consistent across devices was hard because operations like DOM manipulation had to traverse a giant tree to insert UI elements. So Netflix had to decide whether they needed the entire DOM and all of the baggage that came with it, Purewal said. 

The way they worked around this was to build a custom rendering engine. This architecture works sort of like Node.js, but on the client rather than the server, Purewal said. This approach allows the main rendering application to access system resources for the Netflix platform across the various devices from high performance gaming platforms to basic TVs.

It needed to handle system related functions like video decoding and playback, networking, logging, security, content control and caching. This approach allows Netflix to innovate on all of the other features and services rendered across this common platform. For example, they have tried out new ways to make movies start faster, secure information, adaptively stream and log.

JavaScript's scope problem

"Our code was leaner, more organized and more testable."

Semmy Purewal, Senior software engineer, Netflix

The problem was that much of these functions had existed in C++ and once this code got to devices, it was not always updated in the field. The goal was to move more of the non-performance critical stuff to JavaScript to allow more innovation. They implemented the Netflix Message Security Layer, MSL JS as an extensible and flexible secure messaging protocol for communicating data.

This led to some interesting differences in scope between variables in JavaScript and C++. The issues came up when two supposedly independent variables also affected a third variable that could be accessed by both. These supposedly independent variables ended up affecting each other, even though they were not intended to be connected. As a result the code suffered a lot, Purewal said.

The perils of too much connection

As a result of the monolithic nature of this code base, Netflix was not doing units tests. It left all testing up to the integration tests at the end. It was hard to mock out the global state that new code was dependent on. Also Netflix was not able to share code across teams. When other JavaScript teams needed the same functionality they would have to bring in all of the code that was required.

This is the same problem that can occur with object-oriented programs with a giant class hierarchy. Developers have to include a giant code base to get to one class. These are all anti-patterns relating to wide scope, Purewal observed. The real problem was that developers did not have access to the same language features that existed in C++. All of the other modern programming languages have an idea of maintaining independence.

Make JavaScript object-oriented

In the past, JavaScript lacked familiar language primitives that supported hiding data and functionality like classes. Netflix turned to modules to make class-like constructs to limit scope. In this case, programmers internally define objects, give them an API and at the end export them. The idea of modules was popularized in Node.js, and is more widely available now. ECMAScript 6 also has modules built in.

Modular programming is a superset of class based object oriented programming. It is possible to make a module export a specific class. Modular programming is also a subset of procedural programming. Procedures should be collected in terms of modules. The benefit of using this style of programming is that developers can do development independently of systems. They are small, concrete and can be put into package managers.

This approach supports two of the good ideas of OO programming around programming by contract and programming to an interface, Purewal said. This makes it easier for a developer to change one module without affecting others and allows the use of package managers like npm.

Gradual transition

Netflix refactored its code into modules over the course of a year. They moved from CMake to Grunt to make the JavaScript artifacts. Browserify helped Netflix efficiently build JavaScript bundles. Jasmine simplified the process of unit testing new modules. The first module was the EventEmitter, which was roughly modeled after the Node.js API.

New functions were implemented as Common JavaScript modules. Netflix also took all of the old singleton subsystems and refactored those into instance based subsystems in JavaScript. Dependency injection was used when required to share these subsystems instead of using global properties.

Purewal advocates other enterprises making a similar transition to JavaScript use bundling tools like Browserify or webpack. Also it is a good idea to start small by exporting the API of one or two modules and then establish a policy that new features are implemented as modules. “For the most part, our code evolves so quickly that a lot of that stuff goes away anyways,” Purewal noted.

After this transition, Netflix moved from three week cycles, to daily deployment. “It was kind of a game changer for us,” Purewal said. “Our code was leaner, more organized and more testable.”