How we use ES modules to speed up frontend development in a large application

Earlier this year, we turned to Webpack to help us refactor our entire codebase to use ES modules. Some of the advantages we were hoping to gain from modules included better management of our code and the ability to prune unused code from the javascript that we were sending to our users. We also wanted to be able to write unit tests that required only the code necessary to run a single module. Our existing test suite required the entire application bundle to be built and loaded for each test. We thought that being able to write faster, simpler tests would help us improve our test coverage.

So we were disappointed when we finished our refactor and saw that resolving our hundreds of module dependencies in Webpack would slow down our development cycles a lot: it took between 4 and 6 minutes to bundle modules on startup!

Refreshing the package after changes with a watch script wasn’t much better. We knew there were steps we could take to improve our build performance, but it seemed like we had a long way to go. We worried that we’d have to abandon our whole module refactor.

Fortunately, with Firefox’s announcement earlier this year, all of the major browsers have landed support for ES modules. Since our problems with Webpack were only in development mode, we wondered if we could just skip the build step entirely while developing, and load our modules directly in the browser.

It turns out, we could!

In brief, by making our Javascript source files publicly available and using only relative paths in our modules, we were able to load our Webpack entry.js file via a <script type=”module” > tag, and the browser loaded the rest of our application from there. This worked so well that it was actually faster to start up our development server than it was before we moved to modules, because we were actually serving our application without any Javascript build step at all!

Along the way we learned a few lessons about using ES modules in a large application, as well as some gotchas for apps that have been built with NodeJS or Webpack in mind.

Network latency

In production web applications, delivering code as fast as possible over the network is always a high priority, and it’s one of the biggest blockers for building an app based on browser modules. Such apps can produce a large number of parallel requests and subsequent round trips while resolving nested dependencies. Indeed, at present, loading a single large javascript file is generally faster than loading a large number of smaller files.

In development mode, however, none of this is a concern. On the local filesystem, round trips are resolved very quickly. It takes us about 600 ms to resolve our entire dependency tree of about 300 modules on a typical page.

Even optimized Webpack builds can take much longer.

Older browsers

Another obvious obstacle to shipping ES modules is that they're still pretty new. Users with IE 11 or Firefox version 59, for example, won’t be able to load your JS. As with the network latency issues above, depending on your target audience, this may be more or less of an issue. In our case, as with the network issues, it prevents us from shipping modules in production, but isn't an issue on our developers' machines. If we need to test older browsers, we can always build a production version of the app to do that.

Public paths

One of the thorniest areas for making your modules work in browsers is path resolution.

To load ES modules directly in browsers, they have to be available at a public path that the browser can understand.

Your modules may be written using niceties supported by Node.js and Webpack such as aliases and "bare paths". import * as _ from 'lodash'; simply won't work when loaded in a browser, unless you serve a file called `lodash` at that route. The same is true for aliases, like import * as MyModule from '@myAliasedDirectory/path/to/myModule.js';. You might have @myAliasedDirectory defined in your Webpack config, but the browser doesn't know what to do with aliases.

To make your modules work with browsers, you'll have to resolve these path issues somehow. The simplest way to do this is to use relative paths in your module imports.

(Note that if you must rely on bare module names and other features of Node.js, you can try to implement support for them in your development server, or even in a service worker.)

Solution: Use relative paths

Our solution was to symlink our entire JS directory to the public directory in development mode, and use relative paths in our modules. Relative paths are the only format that work the same in the browser and the Node environment. Unfortunately, relative paths are pretty ungainly when it comes to importing node_modules into your application code. In our case, this wasn’t an issue because we use vendor JS via global references, and don’t need to import node_modules packages directly into our application modules.

Module type mismatches

node_modules present a second problem for module browsers: vendor modules that aren't in the ES6 module format. A lot of modules are likely to be written for NodeJS using CommonJS syntax. These don't work in browsers, so if you’re using modules in that format, you'll need to convert or wrap them in some way with es6 modules, and make them publicly accessible as well.

A gotcha: reloading of files

One difference between ES modules and regular Javascript modules that caused us some brief trouble is that modules are only executed once, regardless of how many times they are required. This means that if you have current Javascript code that relies on being loaded multiple times, it will break when converted to modules. This could be a problem with hot reloading strategies such as turbolinks.

Publishing modules

Our browser implementation was pretty simple. We had to publish our modules, so we decided to symlink them into the public directory. We could also have handled this at the routing level in our development server. Symlinking allows us to keep from complicating our server code with concerns from our development setup.

We use Gulp for this because it has a great utility for copying whole directories and symlinking their contents using the glob syntax, for example:

gulp.symlink('src/\*\*/\*.js', 'public');

Repurposing our Webpack entry file

We maintain an entry.js file for each Webpack build we use. Since we're using relative module paths, and all of our vendor JS is referenced via global variables, we can simply load entry.js as a module:

{ if env === "development" }
 <script type="module" src="/path/to/entry.js"></script>
{ else }
 <script type="text/javascript" src="/build/app.js"></script>
{ / }

And that's pretty much it! Now, in spite of having hundreds of modules in our dependency tree, when we make changes in code, we simply refresh the browser, and see our changes! There’s no watch script, and no JS build process.

Then, when we’re ready to deploy, the same entry.js file that we’ve been using to load modules in our browsers while developing can be used with Webpack to generate our production build.

Do you have any experiences to share about using ES modules? We'd love to hear about them on Twitter!

‍