Reliably building frontend assets with NVM and Yarn (or NPM)

For better or worse, most web applications these days require a build step to bundle, transpile, minimize, version, split, or otherwise prepare your Javascript and CSS files for consumption in the browser.

There are a lot of tools available to build & bundle assets:

In Laravel, we use Laravel Mix, which is just a lovely wrapper around Webpack, to handle all of this for us. I don't know enough about all of these tools to tell you which one to use. I use Mix because it does the job well and I don't have to futz about with it.

When developing locally, we just call npm run dev and Mix does it thing and packs up all our assets for use.

But what about when it comes time to deploy to production?

How do we ensure that we're building the same assets, with the same settings, with the same dependencies, for production?

How do we handle this across many team members?

Committing Built Files to Version Control (don't)

One option you have is to run your production build locally and then commit that to git and just push them out to the production server.

This can work, but it has a few downsides.

The first is a huge amount of nonsense changes in your history. You'll get all the changes in your source files, and then all the changes again in your built files. This can also lead to merge conflicts in your built files if you have more than one person trying to merge frontend changes at once.

You can tell git to treat built files as binary, so that you don't have the diff cluttering up your PR review screen, but this still doesn't solve the problem of merge conflicts.

Finally, you have to make sure you always remember to build the assets before you commit! You can set up git hooks or other automation methods, but this can be a real pain when developing locally. You end up with a lot of "did you build the assets?" comments on your pull requests, and a lot of "building assets" commits in your history.

Or worse, deploying code that depends on newly built assets, without having actually built the assets!

Requiring that your built assets always be up to date in version control is a real waste of developer time and energy.

I'd recommend against building locally and committing to version control.

Building as a Deploy Step

If you can't (or shouldn't) build locally and commit, then the other option is to build as a part of your deploy process.

This lets us ignore all built files from version control, so we're only ever comparing changes to source files, it reduces the chances of a merge conflict, and it eliminates the chances of forgetting to build the newest assets.

The trick here is how do you ensure that you're building the exact thing you think you are?

Managing Node Versions With NVM

The first thing you'll want to ensure is that you're using the same version of Node on the server as you are locally. If we're using different versions locally and in production, we'll end up with differing assets. In fact, you may even not be able to build the assets at all if your versions are too far off. You'll get a message like this:

The engine "node" is incompatible with this module. Expected version ">=12 <14". Got "10.0.0"

Code highlighting powered by torchlight.dev (A service I created!)

This means that some of the packages that you're requiring can't even be installed because they require a different Node version than the one you're using.

To ensure we're using a consistent version of Node everywhere, we can use the fantastic NVM library.

NVM (short for Node Version Manager) lets you seamlessly switch between versions of Node on a project basis. Even if you don't follow any of the other advice in this article, install NVM locally. You'll thank me later!

One of the reasons NVM is great is because you can create a .nvmrc file in the root of your project that specifies what version of Node is required. All you need to do is put one of the recognized NVM versions in the file:

v12.20.1

You can specify a specific point release like I have above, or you can use one of the codenamed versions:

lts/fermium

Now NVM will use the Fermium LTS release of Node, which is major version 14. You can see all available versions by running nvm ls-remote. There are... a lot of them.

Once you have a .nvmrc file you can switch your shell to run that version by calling nvm use.

$ cd aaronfrancis.com/
$ nvm use
# Found '/Users/aaron/Code/aaronfrancis.com/.nvmrc' with version <lts/fermium>
# Now using node v14.15.5 (npm v6.14.11)

Anytime I cd into a new project, the first thing I do is run nvm use. (If a particular version of Node hasn't been installed yet, you can run nvm install && nvm use.)

Ok, now you've pegged Node to a specific version, which ensures that everyone on the team and all the servers are aligned on the version of Node they're using.

Installing Dependencies with Yarn or NPM

Before you build the assets, you'll need to install all of the required packages. Most of the time you do this by running yarn install or npm install. However, when you're building your assets for production, this isn't always a good idea.

By default yarn install will try to install based on your yarn.lock file, but according to the docs, there are times when yarn install could modify your lockfile:

If yarn.lock is absent, or is not enough to satisfy all the dependencies listed in package.json (for example, if you manually add a dependency to package.json), Yarn looks for the newest versions available that satisfy the constraints in package.json. The results are written to yarn.lock.

This is bad news because now you're potentially using different package versions locally than you are in production.

To get around this, yarn allows you to pass a --frozen-lockfile flag.

It's true that if your lockfile is up to date then --frozen-lockfile has no effect whatsoever (according to the Yarn maintainer himself!) But of course there are times when your lockfile may become out of date for any number of reasons. Bad merges, manually updated package.json files, etc.

We'd always rather get an explicit failure for someone to fix than an unexpected upgrade or installation into our production environment.

If you need reproducible dependencies, which is usually the case with the continuous integration systems, you should pass --frozen-lockfile flag.

NPM actually operates the same way. If you call npm install it might modify your package-lock.json.

NPM too has a specific command for this exact scenario: npm ci. From their docs:

This command is similar to npm install, except it's meant to be used in automated environments such as test platforms, continuous integration, and deployment -- or any situation where you want to make sure you're doing a clean install of your dependencies.

It will never write to package.json or any of the package-locks: installs are essentially frozen.

Building Your Assets

Now that you have 1) a consistent version of Node and 2) identical versions of your dependencies, you can (finally) build your assets!

You're finally ready to run npm run build to reliably build your final asset bundles.

Putting it All Together

This process will work across all kinds of deployment setups. Build servers, serverless, local packaging, etc.

So far I've used it to build assets on Laravel Forge, in a GitHub workflow that deploys Laravel Vapor, and on my local machine for deployment to Vapor.

To keep things portable, I have a file called bin/build that puts it all together.

The first thing it does is installs NVM and loads it. After that it uses NVM to install the correct version of Node. Then it uses NPM to install Yarn, then uses Yarn to install the dependencies. Then it builds the dependencies.

(It really does sound insane when you write it all out.)

#!/bin/sh
 
# Exit on errors
set -e
 
# Install NVM
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.37.2/install.sh | bash
 
# Load NVM after it's installed
export NVM_DIR="$HOME/.nvm"
[ -s "$NVM_DIR/nvm.sh" ] && . "$NVM_DIR/nvm.sh"
 
# Install and use the version of node that is in .nvmrc
nvm install && nvm use
 
# Install Yarn
npm install --global yarn
 
# Install dependencies
yarn install --frozen-lockfile
 
# Build assets
npm run build

If you're using Laravel Vapor, your asset step is now just the bin/build command.

environments:
  production:
    domain: aaronfrancis.com
    memory: 512
    build: 
      - 'bin/build'

Code highlighting powered by torchlight.dev (A service I created!)

If you're deploying from GitHub Actions, your build step is similarly just ./bin/build

(Of course you could add your composer install and other commands to that file as well.)

Only Building When Necessary

Building assets for production can be a relatively expensive operation. It can take a long time to tree-shake, minimize, purge CSS, etc. Some asset build processes can take up to 5 minutes or longer.

If you're deploying multiple backend-only changes per day, you really don't want to be building these assets over and over again when literally nothing has changed.

I've built a Laravel package to solve this very problem called Laravel Airdrop. Airdrop will stash your built assets on a remote filesystem and if a build is not required on the next deploy, it will pull the assets and put them right where they need to go.

Airdrop calculates a hash of all inputs required to build your asset bundle, and if nothing has changed from the last build it will pull the build assets down and put them in place.

Take a look at the docs for a full explanation, but if you use this approach then you can skip many of those expensive steps and make your deploys super fast.

Here's the bin/build command to take Airdrop into account.

php artisan airdrop:download 
 
# Skip building assets 
if [ ! -f ".airdrop_skip" ]; then 
    # Install NVM
    curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.37.2/install.sh | bash
 
    # Load NVM after it's installed
    export NVM_DIR="$HOME/.nvm"
    [ -s "$NVM_DIR/nvm.sh" ] && . "$NVM_DIR/nvm.sh"
 
    # Install and use the version of node that is in .nvmrc
    nvm install && nvm use
 
    # Install Yarn
    npm install --global yarn
 
    # Install dependencies
    yarn install --frozen-lockfile
 
    # Build assets
    npm run build
fi 
 
php artisan airdrop:upload