How to Simplify Drupal Builds with Yarn and Webpack

GWW Opinion

February 11, 2020

Drupal Development

When a Drupal site grows organically, with multiple contributors, the front-end configuration can get out of hand. What starts as a single theme and module, over time can evolve into a Frankenstein mashup of modules and themes and JavaScript (JS) and Cascading Style Sheets (CSS) that no one understands. As a result, the deployment script has to work double time to put everything together. Ugh.

Because the front-end workflow is outside of Drupal’s scope, it needs to be managed separately. It makes no difference to Drupal if you use vanilla JS and CSS, or you build in TypeScript and Sass. All it requires is that you provide it with the JS and CSS that the module or theme needs to function in the browser. Without it, the site can’t execute enhanced user experiences such as animation and complex interactions.

What everyone needs is a plan to assure the front-end build configuration setup is as clean as possible. With that in mind, we’ve come up with a process for a consistent setup for JS and CSS across modules and themes. Luckily, there are some simple tools that can help.

Let’s start from the beginning…

On Drupal projects, we spend most of our time building custom modules and themes, along with the JS and CSS that enhances them. There are a lot of efficiencies that can be gained by using tools such as preprocessors, transpilers, bundlers, and more. However, setting them up right the first time can be a challenging task.

For example, you might start off only using the Compass CSS compiler for the theme. It’s a simple workflow. Developers call compass watch during development, and compass compileon deploy.

That is until the theme gains JS, which necessitates the use of the Gulp toolkit. A little later, Compass folds into Gulp but keeps the original Compass binary due to compatibility. Then someone tries to use ES modules to split up code, and Babel to transpile code, but not everyone writes JS in the newer syntax. Add to that, the custom modules might not follow the same setup depending on who wrote it and/or how badly they needed the setup. Eventually, devs might skip the entire toolset altogether and write in some vanilla syntax to avoid the overhead.

In the end, you have a Frankenstein build setup that no one understands.

We’d like to propose an alternative…

Enter Yarn

Sometimes you need a helping hand from Yarn.

Yarn is a front-end package manager. One of its neat features is workspaces support. This feature allows developers to manage multiple packages under a single codebase. You can install dependencies, run scripts, and more on multiple packages within the same codebase with just one command.

Now if you think about it, any Drupal site codebase is essentially a monorepo. While the codebase looks like one monolithic blob of files, you usually be working on multiple custom modules and themes. Each of these can be their own project, requiring their own dependencies and build routines. That sounds like a monorepo to me, and something Yarn can help with.

To use Yarn on Drupal, you’ll need a package.json at the root of the project containing the JSON below. What this snippet does is tell Yarn to look for workspaces in the modules and themes directories. This will be used whenever you launch commands with the workspace or workspaces subcommand.

{
  "private": true,
  "workspaces":
    "web/modules/custom/*",
    "web/themes/custom/*"
  ]
}

Next, to treat each module and theme as a workspace, simply add a package.json. Add the required name and version fields. Note that the name field in the package.json will serve as the workspace name. Then add scripts for automation. In most cases, you’ll need a watch script to compile on save during development, and a build script to compile for production. We’ll leave the script fields empty for now.

{
  "name": "my-drupal-module",
  "version": "1.0.0",
  "scripts": {
    "watch": "",
    "build": ""
  }
}

And that’s it! Each module and theme are now workspaces. This allows developers to manage multiple packages at once.

Enter Webpack

Now it’s time to sprinkle in some front-end magic with Webpack.

With the workspaces set up, we’ll need to assign functionality to the scripts. We’ll be using Webpack, specifically an opinionated tool that builds on top of Webpack: Laravel Mix. This allows us to use Webpack with the most common compilation setup without having to deal with all the configuration that’s needed.

First, let’s add a webpack.mix.js file at the root of every workspace containing the following snippet of code:

// webpack.mix.js

const mix = require('laravel-mix');

mix.disableNotifications()
  .js('src/app.js', 'dist')
  .sass('src/app.scss', 'dist')
  .sourceMaps()
  .webpackConfig({
    devtool: "source-map",
    externals: {
      jquery: "jQuery"
    }
  })

This file might look intimidating so let me break it down:

js() and sass() accept a module’s or theme’s JS and CSS entry points, and the destination directory of the output. The output of the build will use the same file name and the appropriate extension. You may add one or more of these depending on the module or theme.
sourceMaps() and devtool: "source-map" ensure that you’ll see the source files when you debug the files from a browser debugger, as Drupal will be serving the bundled files.
externals define modules that need to be excluded from the bundle and mapped to global variables. In the example, any imports of jquery will be mapped to window.jQuery so that it uses the jQuery that’s loaded by Drupal.

Next, in the same directories that you added webpack.mix.js, add a webpack.config.js to initialize Webpack with a predefined config. This is required because in the documentation, --config points to a config file in a module in the node_modules of the workspace. However, we cannot use it that way because Yarn hoists dependencies and the file needed will not be in the workspace node_modules anymore.

const mix = require("laravel-mix/src/index")
const ComponentFactory = require("laravel-mix/src/components/ComponentFactory")
new ComponentFactory().installAll()
require(Mix.paths.mix())
Mix.dispatch("init", Mix)
const WebpackConfig = require("laravel-mix/src/builder/WebpackConfig")
module.exports = new WebpackConfig().build()

Lastly, update your workspace’s package.json to contain the snippet below. What this does is assign commands to the scripts. When you execute any of the scripts using the workspaces subcommand, they’ll execute the commands assigned to them.

{
  "name": "my-drupal-module",
  "version": "1.0.0",
  "scripts": {
    "watch": "npm run development -- --watch",
    "build": "npm run production",
    "development": "cross-env NODE_ENV=development webpack --progress --hide-modules --config=webpack.config.js",
    "production": "cross-env NODE_ENV=production webpack --no-progress --hide-modules --config=webpack.config.js"
  },
  "dependencies": {
    "cross-env": "^6.0.3",
    "laravel-mix": "^5.0.0",
    "sass": "^1.23.0",
    "sass-loader": "^8.0.0",
    "vue-template-compiler": "^2.6.11"
  }
}

Now simply add src/app.js and src/app.scss in your modules and themes, add in your JS and CSS, and ta-da, your front-end setup is complete!

Put it together in Drupal

The last thing is to plug all that work in to Drupal.

To do just that, we register the outputs defined in js() and sass() of webpack.mix.js as libraries in Drupal. To do this, define a libraries.yml file in the workspaces, and add the following snippet:

my_library:
  js:
    dist/app.js: { minified: true }
  css:
    dist/app.css: { minified: true }

And that’s pretty much it! Now you can add this library to your module or theme!

Last minute checks!

It’s always important to check everything twice.

In the end, your Drupal structure, with all of the needed files included, should look somewhat like the following. I’ve added the usual suspects in a typical Drupal project setup for a bit of guidance on where things go. Highlighted in red are the files affected.

composer.json
package.json
web/
  core/
  libraries/
  modules/
    contrib/
      custom/
        my_drupal_module/
          src/
            app.js
            app.scss
          my_drupal_module.info.yml
          my_drupal_module.libraries.yml
          my_drupal_module.module
          package.json
          webpack.config.js
          webpack.mix.js
  profiles/
  sites/
  themes/
    contrib/
      custom/
        my_drupal_theme/
          src/
            app.js
            app.scss
          my_drupal_theme.info.yml
          my_drupal_theme.libraries.yml
          my_drupal_theme.theme
          package.json
          webpack.config.js
          webpack.mix.js
  vendor/

Don’t forget to define your dependencies in each workspace. It’s as easy as doing yarn workspace $WORKSPACE_NAME add $DEPENDENCY_NAME. For example, installing Lodash in our example module my_drupal_module, whose workspace name is my-drupal-module would look like the following:

yarn workspace my-drupal-module add lodash

To check if everything runs as expected, run yarn workspaces run build. What this does is run the build command for each and every workspace found in the project, as defined in the root package.json file. So as long as the workspace has that script defined, it will run whatever command was defined in that script. In our case, our Webpack compilation commands.

Conclusion

And there you have it! An example of how to use Yarn and Webpack in Drupal. Now you can take advantage of all the features Yarn and Webpack offer in your Drupal projects. From transpilation, to bundling, and most importantly, automating the management of multiple packages.

Of course, there’s always room for improvement. You can probably write a node module or Drush script to automate the addition of these files to the workspaces. But that’s a story for another time.