Do you use Browserify, WebPack, jspm, StealJS, or SystemJS? This article will provide a peak under the hood at the layer on top of which those libraries are implemented.
Modules are a way to organize your application. Simply break your functionality into small pieces, focusing on how they will work with each other, and then assemble them together. A module could be seen as a black box with a clear, simple API. Modules commonly depend on other modules.
In today's modern browsers, there is support for not only writing and loading modules, but performing various low-level tasks around loading and executing the module. This article will explain the current standard for module loaders - their lifecycle and their hooks. A future article will show a detailed example for how to use these hooks.
Pop quiz! If you have modules written in CoffeeScript and CommonJS, is it possible to use them both within an ES6 application?
The answer is “Yes to both”. This article will explain how this is possible using Module Loaders.
Table of Contents:
1. Module Loaders
For modern web development, the following module standards are available:
- AMD - Asynchronous Module Definition, good for loading modules asynchronously (dynamic import).
- CommonJS is widely known for being used in NodeJS. It is good for synchronous module loading (static import) which works well for server-side scripting.
They have different but similar APIs and serve the following purposes:
- define a module (module syntax);
- load a module.
In this article we will focus on how a module is loaded and get a gist of what all module loaders do.
A module system aims to simplify your development: you can focus on your current module and only have to care about what modules you directly depend on. The module loader does all the heavy lifting:
- performs the loading task,
- acts as a dependency manager
- and maintains a Module Registry (an object that keeps track of all modules and stores theirs source code along with other meta data)
Let’s look at how WHATWG specification describes what a module loader should do:
The loader is a system for loading and executing modules, and there is a way to participate in the process. There are several Loader hooks which are called at various points in the process of loading a module. The default hooks are implemented on the Loader.prototype, and thus could be overridden/extended.
2. Loader Pipeline
In the diagram you can see the different stages that the Loader passes through:
Note: WHATWG (ES6) module standard defines four stages: “Resolve” replaces “Normalize” and “Locate”.
During the Normalize phase the Loader converts the provided name into a Module Identifier that will be used as a key to store the module’s data in the Module Registry. The given name could be a relative path to the resource, it also could contain a shorthand mapping to a certain path, or any other logic that a particular Loader implementation provides.
The Locate phase serves to determine the final resource address that the Loader will use to fetch the resource from. It is either a URL (if the host is the browser), or a path (if the host is a NodeJS server).
During the Fetch phase Loader fetches the resource by provided address. It could be that module’s body is provided to the Loader directly, in which case this phase will be skipped. The result of this phase is a string with the source code of the module.
During the Instantiate phase module’s dependencies are loaded and linked together, then the module gets evaluated.
3. Loading hooks
Now let’s see how the process could be customized. For each of the stages there is a hook, which is a method that will be called with certain arguments. A hook can either return an immediate result or a promise.
When you override the loader’s hook method you can also call the original method. In this case you will have to pass it the parameters as defined by the hook’s signature. Alternatively, you can just return the expected result.
For an example we will look at how module my.js imports module math.js. Both are saved in the same directory called "utils" (look here for ES6 module syntax):
Normalize: (name, referrerName, referrerAddress) → normalizedModuleName
The Module Loader calls this hook by passing three arguments:
referrerName (the normalized name of the module that initiated the import),
referrerAddress. The result of the call should be a string, which is a normalized module name. It is usually a path to the module file or folder from the root of the project. This way it uniquely identifies a module within the project.
Locate: loadRequest → loadRequest
This hook receives the
loadRequest object, in which the
name property is a normalized module name. It adds the
address property to the object, which represents the resource address. It is called immediately after normalize unless the module is already loaded or loading (the same applies to the rest of the hooks).
Fetch: loadRequest → sourceCodeString
loadRequest object with
address property, and returns a string containing the source code of the module.
Translate: loadRequest → ecmaCompliantSourceCodeString
loadRequest object with a
source property, which is a result of the previous step. The purpose of this hook is to translate the source code into ECMAScript. If the code is in another language (CoffeeScript for example), this is when the transpiling would happen.
Instantiate: loadRequest → instantiationRequest
In this hook, the translated source is instantiated. It receives
loadRequest with the
source property as a translated source. It returns an
instantiationRequest object, which has two required properties. The value of the
deps property is an array of strings. Each string is the name of module dependencies. The value of the
execute property is a function which the loader will use to create the module.
A module is evaluated during the linking process. First, all of the modules it depends upon are linked and evaluated, and then passed to the execute function. Then the resulting module is linked with the downstream dependencies.
It is worth mentioning that the current draft of ECMA-262 does not include the specification for module loaders since it was removed in 2014. You can still find it in the archive. It is a very useful resource - SystemJS' and StealJS' implementation was based on that draft. The new draft is now being developed by WHATWG and is not completed yet.
To recap, we looked at what a module system is, what standards are available for modern web development, then we dove into the loader pipeline and saw how it could be extended. In the next post we will write a simple loader’s plugin for translating CoffeeScript on the fly (no need to precompile, and you can even debug in browser against the original source)!