UPDATE: We've updated our process. Read all about it here.
At Foster Made, we work on a wide variety of projects, everything from small Statamic sites to complex ExpressionEngine sites to large-scale Angular and Symfony applications. These projects vary by technology, size and complexity. But they do share some fundamental needs as modern web projects. And those needs are met by writing automated scripts. As the project grows in complexity, these task runners morph and expand. The build process for a complex application can be vastly different from a straightforward flat-file cms build. But all benefit from a baseline build process.
These tasks are necessary for every project:
We wanted to universalize these core tasks so we could easily use them as a starting place across projects. NOTE: We've deleted the original Github repo in favor of our new npm package repo: https://github.com/fostermadeco/fm-site-build
Currently, we use Gulp to create our build workflow. We like it because we can make small, composable tasks and takes advantage of node streams. It can be frustrating when it comes to debugging. Also sometimes 'too' asynchronous. Errors are sometimes cryptic and it can difficult to track down the source. But no process is perfect and we are continuously working on smoothing out those pain points.
Deciding on a uniform directory structure sets a solid foundation. This does vary slightly from project to project, but all the paths are configurable. Here is an example of a typical structure:
. ├── gulp # <-- Custom gulp tasks can be defined here ├── dev # <-- Generated, pre-production app assets │ ├── app.css # <-- Project compiled, concatenated sass files │ ├── vendor.css # <-- Vendor compiled, concatenated sass files │ ├── main.js # <-- Concatenated js files with source maps ├── dist # <-- Generated, production-ready app assets │ ├── main-min.css # <-- Minified css │ ├── main-min.js # <-- Minified js │ src # <-- Where you edit files │ ├── scss │ │ ├── mixins # <-- Folders for sass partials │ │ └── main.scss │ └── js # <-- Editable js files, add more to subfolders ├── images # <-- Image assets, relative to `/` ├── fonts # <-- Font assets, relative to `/` ├── static # <-- pdfs, etc. Files that need to be relative to `/` ├── vendor # <-- Third-party assets │ ├── bower_components # <-- Specify path in .bowerrc │ ├── css # <-- other vendor css not in Bower │ └── js # <-- other vendor js not in Bower └── public/site/../templates # <-- Wherever your templates are └── index.html # <-- Where your stylesheets and js are included from
In most web applications, it's ideal for the dev and dist folders to contain all the assets needed to run the application. Images, fonts, etc. are copied into the dev and dist folders. ExpressionEngine and Statamic have templates outside these folders and are not setup to be compiled or moved. So we have to take a slightly different approach when organizing assets. The important thing in the above directory structure is that images and css are always located where the path references between the two are preserved. We don't usually copy images anywhere, but reference them from the main images folder.
Bower is standard for managing and updating front-end assets. In theory it's as easy as 'bower install awesome-package --save'. In practice, I've run across a couple gotchas.
An easy way to use Bower files is to simply reference them from the bower_components directory. However, this doesn't scale well. You don't want to reference 8 different files in production. We need a way to take those files out of Bower and make them usable in our projects.
When a Bower package is installed, it grabs the entire git repo. How do we get a list of files we actually want to include on our project? We can get a list of files from each package's "main" property in their bower.json file. However, that doesn't get us a completely usable list. It can include .less files when you want .css or can simply be inaccurate. Yes, you could use the unofficial 'overrides' property in bower.json, but that is another config to keep track of and we are trying to consolidate configuration because we want this process to be super easy to setup on a new project. Also this doesn't solve the possible problem with fonts and images.
I really wanted to automate the whole process: from 'bower install' to creating production ready files. I've tried a bunch of different packages and approaches in hopes of solving the issue, but anomalies continue to crop up. Sometimes giving a little control over configuration goes a long way to smooth out future frustrations. Enter gulp.options.js
As much as possible, gulp.options.js consolidates the configuration for each site. The file is heavily commented and explains what each option does. Hopefully when starting a new project, this is the only configuration that needs to be touched. It includes a list of js and css Bower files to be used. It also specifies font directories and images. Everything in these lists will be included in the dev and dist files.
Font files can pose another issue as their location and thus css path reference vary by package. This puts a kink in automating the use of these files in dev and dist builds. The Bower task normalizes this out by rewriting '../fonts' to './fonts' in the css when it's copied from Bower into the dev and dist folders.
Something like Browserify would solve some of these issues. And we've had the pleasure of using it on some projects. But we work on a lot of existing projects, most of which would be difficult to back-port to use something like Browserify. Our Gulp build process is easier to implement on projects that may contain various existing assets.
Just as the landscape of web technologies is ever-changing, our build process evolves with the times. They are a living document that can be improved and tweaked as new requirements and ideas ebb and flow.
Posted in #Technologies
Everyday I get to interact with and meet incredible companies and organizations that are finding creative ways to make the world a better place. We enable them to make their ideas become a reality. It’s exciting to be a part of.
- Alex Mejias