For example, ngrx provides schematics for creating stores and reducers, and nrwl provides a complete workspace management solution, including the ability to create React apps!
We can also use schematics to customise existing CLI commands, so they better suit our own workflows.
For example, we have an app which is built as a collection of micro-frontends. Each workflow is built as its own app, and each app lives in its own repository. Every time we need to spin up a new app we need to:
- create a new project using
ng new - add .npmrc, .nvmrc and .prettierrc files, with our specific configurations
- update karma.config.js and tslint.config.js to match our common test and linting settings
- install prettier and husky, for formatting and git hooks
- add some additional NPM scripts, for our CI server
- add git hooks via husky
As you can imagine, doing all this manually is fairly tedious and error-prone. To avoid that, we use a custom schematic which runs ng new for us to create the repository, then makes all the necessary changes.
We’re going to recreate this process by creating a schematic named new-project. The new-project schematic is going to do all the things listed above: create a new project, add additional files, make necessary changes to existing files, install dependencies, add scripts and add a git hook.
iJS Newsletter
Keep up with JavaScript’s latest news!
Creating a schematic
We’ll start by creating the new-project schematic. To do that, we’re going to use an NPM package called schematics. You’ll need to install it globally, just like the Angular CLI.
> npm install -g @angular-devkit/schematics-cli
Once we have this package installed, we can use it to create a new schematic, just like we use the Angular CLI to create a new app.
> schematics blank --name=new-project
The blank parameter creates a minimal schematic, with only the bare necessities in it. The output in the console tells us which files have been created.
CREATE new-project/README.md (639 bytes)
CREATE new-project/.gitignore (191 bytes)
CREATE new-project/.npmignore (64 bytes)
CREATE new-project/package.json (568 bytes)
CREATE new-project/tsconfig.json (656 bytes)
CREATE new-project/src/collection.json (234 bytes)
CREATE new-project/src/new-project/index.ts (319 bytes)
CREATE new-project/src/new-project/index_spec.ts (476 bytes)
✔ Packages installed successfully.
README.md, .gitignore, package.json, and tsconfig.json should be pretty self-explanatory. .npmignore tells NPM to ignore the TypeScript files when bundling up the package. This is necessary because of the way the TypeScript compiler is set up in the project. Rather than outputting all the transpiled files into a common dist/ or build/ directory, the transpiled files remain in the directory with the original TypeScript files. So building new-project/src/new-project/index.ts will result in the file new-project/src/new-project/index.js.
Unsurprisingly, the real interesting stuff is in the src directory.
- collection.json is like an index of the schematics in this project. It links the name of each schematic with the code that runs it.
- new-project/index.ts is the code run by our schematic.
- new-project/index_spec.ts contains the tests. We’re not going to worry about writing tests today, but testing your schematics is definitely possible.
Great, we’ve created our first schematic! Let’s check that it works, using the age-old JavaScript debugging tool, console.log(). I’m going to add a log to src/new-project/index.ts, so it now looks like this:
import { Rule, SchematicContext, Tree } from '@angular-devkit/schematics';
export function newRepo(_options: any): Rule {
console.log('Hello from your new schematic!')
return (tree: Tree, _context: SchematicContext) => {
return tree;
};
}
Because everything is in TypeScript, we need to build it before we run it. We can do that by running `npm run build`.
Once the build succeeds, we can run our schematic via `schematics .:new-project`. You should get output like:
Hello from your new schematic!
Nothing to be done.
`Nothing to be done` here indicates that we haven’t made any changes to the file system.
Yes, it works!
One small thing though – running the build script manually like this every time we make a change can get quite tedious. Instead, we can get the TypeScript compiler to watch for changes and build automatically, by adding a watch script to package.json, like so:
"scripts": {
"build": "tsc -p tsconfig.json",
"watch": "tsc -p tsconfig.json --watch",
"test": "npm run build && jasmine src/**/*_spec.js"
},
Now, if you run npm run watch, the build will re-run automatically whenever a change is made.
[10:57:31 AM] File change detected. Starting incremental compilation…
[10:57:31 AM] Found 0 errors. Watching for file changes.
Time to make our schematic actually do something!
Handling User Input
This schematic is going to create a brand new project, using the name passed in by the user. To do that, we’re going to call the existing ng new schematic, passing in the name, and a bunch of default options.
First, we want to tell our schematic that we’re expecting a name to be passed in. We can do this by creating a schema.json file in our new-project directory. The schema.json file is optional, but it improves the user experience. For example, if the user misses a required option, the schema allows the schematic to ask the user for the value. Think of it like adding types in TypeScript.
We’re going to create new-project/schema.json with the following contents.
{
"$schema": "http://json-schema.org/schema",
"id": "NewRepoSchematic",
"title": "ng new options schema",
"type": "object",
"description": "Initialise a new project",
"properties": {
"name": {
"type": "string",
"description": "The name of the project",
"x-prompt": "Name:",
"$default": {
"$source": "argv",
"index": 0
}
}
}
}
The x-prompt value is the prompt that the schematic will use to ask the user for a value, if they don’t supply one. The default object allows you to provide a default value. In this case, the default value will be the first argument the user passes in on the command line.
So we’ll be able to use our new schematic in three ways.
-
- Just call the schematic, without passing anything in. The schematic will ask for a name.
> schematics .:new-project ✔ ? Name: fancy-project
- Call the schematic, passing in the name option
> schematics .:new-project --name=fancy-project
- Call the schematic, passing the name as an argument
> schematics .:new-project fancy-project
- Just call the schematic, without passing anything in. The schematic will ask for a name.
All three of these options will result in fancy-project being passed in as the value of the name option.
However, it won’t work just yet! First, we need to tell collections.json about our schema. We can do that by adding a schema property, which points to our new schema.
{
"$schema": "../node_modules/@angular-devkit/schematics/collection-schema.json",
"extends": ["@schematics/angular"],
"schematics": {
"new-project": {
"description": "A blank schematic.",
"factory": "./new-project/index#newRepo",
"schema": "./new-project/schema.json"
}
}
}
Now, if you build and run the schema without passing in a name value, it should ask you for it, just like above.
If we have a look in new-project/index.ts now, we can get access to the name as a property of the _options object being passed into our newRepo function.
import { Rule, SchematicContext, Tree } from '@angular-devkit/schematics';
export function newRepo(_options: any): Rule {
const name = _options.name
console.log('The name of the repo will be', name)
return (tree: Tree, _context: SchematicContext) => {
return tree;
};
}
> schematics .:new-project ✔ ? Name: fancy-app The name of the repo will be fancy-app Nothing to be done.
Obviously this is all pretty exciting, but our schema still doesn’t actually do anything yet.
Call an External Schematic
The first thing we want to do is call the ng new schematic, passing in our name option, and a few other default options. To do that, we’re finally going to write some TypeScript!
Currently, our index.ts file looks something like this:
import { Rule, SchematicContext, Tree } from '@angular-devkit/schematics';
export function newRepo(_options: any): Rule {
const name = _options.name
console.log('The name of the repo will be', name)
return (tree: Tree, _context: SchematicContext) => {
return tree;
};
}
We have a function, newRepo, which is going to get called when the schematic runs. It’s going to get passed in _options, which we can use to get the name. It’s going to return a function, which takes a Tree and a SchematicContext, and returns a new Tree. A function like this is referred to as a Rule. A function which returns a Rule is a rule factory.
A Tree is a fundamental concept in Schematics. It refers to the schematic’s internal representation of the file system. For those familiar with React, it’s kind of like a virtual DOM, but for the file system. When we make changes, they’re applied to the Tree, rather than the actual files. Once we’ve finished with all our changes, the Tree is written to the file system (or not, if we’re in debug mode).
The SchematicContext object just contains some utility functions and metadata.
So, a schematic is a factory which returns a Rule. The Ruleis applied to a Tree to produce a new Tree.
Currently, our Rule is just passing back the same Tree that was passed in. Instead, we can return the result of calling another Rule factory. In our case, we want to call the externalSchematic Rule factory, which can be found in @angular-devkit/schematics. We need to pass externalSchematic the name of a collection, the name of a schematic in that collection, and an options object to call the schematic with.
import { Rule, SchematicContext, Tree, externalSchematic } from '@angular-devkit/schematics';
export function newRepo(_options: any): Rule {
const name = _options.name
return (_: Tree, _context: SchematicContext) => {
return externalSchematic('@schematics/angular', 'ng-new', {
name,
version: '9.0.0',
directory: name,
routing: false,
style: 'scss',
inlineStyle: false,
inlineTemplate: false
});
};
}
Notice that I had to change the variable tree to _ in the Rule definition. The default linter in a schematics project is super picky, and it won’t compile with unused variables. You can either adjust the linter settings, or just give it what it wants.
Now if we run our schematic, we get an output like this.
> schematics .:new-project fancy-app ✔
CREATE fancy-app/README.md (1025 bytes)
CREATE fancy-app/.editorconfig (274 bytes)
CREATE fancy-app/.gitignore (631 bytes)
CREATE fancy-app/angular.json (3678 bytes)
CREATE fancy-app/package.json (1285 bytes)
CREATE fancy-app/tsconfig.json (489 bytes)
CREATE fancy-app/tslint.json (3125 bytes)
CREATE fancy-app/browserslist (429 bytes)
CREATE fancy-app/karma.conf.js (1021 bytes)
CREATE fancy-app/tsconfig.app.json (210 bytes)
CREATE fancy-app/tsconfig.spec.json (270 bytes)
CREATE fancy-app/src/favicon.ico (948 bytes)
CREATE fancy-app/src/index.html (294 bytes)
CREATE fancy-app/src/main.ts (372 bytes)
CREATE fancy-app/src/polyfills.ts (2835 bytes)
CREATE fancy-app/src/styles.scss (80 bytes)
CREATE fancy-app/src/test.ts (753 bytes)
CREATE fancy-app/src/assets/.gitkeep (0 bytes)
CREATE fancy-app/src/environments/environment.prod.ts (51 bytes)
CREATE fancy-app/src/environments/environment.ts (662 bytes)
CREATE fancy-app/src/app/app.module.ts (314 bytes)
CREATE fancy-app/src/app/app.component.scss (0 bytes)
CREATE fancy-app/src/app/app.component.html (25725 bytes)
CREATE fancy-app/src/app/app.component.spec.ts (951 bytes)
CREATE fancy-app/src/app/app.component.ts (214 bytes)
CREATE fancy-app/e2e/protractor.conf.js (808 bytes)
CREATE fancy-app/e2e/tsconfig.json (214 bytes)
CREATE fancy-app/e2e/src/app.e2e-spec.ts (642 bytes)
CREATE fancy-app/e2e/src/app.po.ts (301 bytes)
However, if we look in our file system, we’ll see that no such files have been created! This is because, by default, schematics runs in debug mode, which is the same as using --dryRun in the CLI. The changes are applied to the Tree, but the Tree isn’t written to the disk.
Turning off Debug Mode
If you want to write the files for real, you need to use the --dryRun=false or --debug=false flag. If you do so, the files will be written, and npm install will be run, just like it would be when a user is using your schematic.
> schematics .:new-project fancy-app –debug=false ✔
CREATE fancy-app/README.md (1025 bytes)
CREATE fancy-app/.editorconfig (274 bytes)
CREATE fancy-app/.gitignore (631 bytes)
CREATE fancy-app/angular.json (3678 bytes)
CREATE fancy-app/package.json (1285 bytes)
CREATE fancy-app/tsconfig.json (489 bytes)
CREATE fancy-app/tslint.json (3125 bytes)
CREATE fancy-app/browserslist (429 bytes)
CREATE fancy-app/karma.conf.js (1021 bytes)
CREATE fancy-app/tsconfig.app.json (210 bytes)
CREATE fancy-app/tsconfig.spec.json (270 bytes)
CREATE fancy-app/src/favicon.ico (948 bytes)
CREATE fancy-app/src/index.html (294 bytes)
CREATE fancy-app/src/main.ts (372 bytes)
CREATE fancy-app/src/polyfills.ts (2835 bytes)
CREATE fancy-app/src/styles.scss (80 bytes)
CREATE fancy-app/src/test.ts (753 bytes)
CREATE fancy-app/src/assets/.gitkeep (0 bytes)
CREATE fancy-app/src/environments/environment.prod.ts (51 bytes)
CREATE fancy-app/src/environments/environment.ts (662 bytes)
CREATE fancy-app/src/app/app.module.ts (314 bytes)
CREATE fancy-app/src/app/app.component.scss (0 bytes)
CREATE fancy-app/src/app/app.component.html (25725 bytes)
CREATE fancy-app/src/app/app.component.spec.ts (951 bytes)
CREATE fancy-app/src/app/app.component.ts (214 bytes)
CREATE fancy-app/e2e/protractor.conf.js (808 bytes)
CREATE fancy-app/e2e/tsconfig.json (214 bytes)
CREATE fancy-app/e2e/src/app.e2e-spec.ts (642 bytes)
CREATE fancy-app/e2e/src/app.po.ts (301 bytes)
✔ Packages installed successfully.
Successfully initialized git.
If you’d rather not create a new project from inside your current project, you can also run your schematic from a different directory, but the syntax is slightly different:
> schematics ./path/to/collection.json:schematic-name
For example, if we wanted to run our schematic from its own parent directory, we could use:
> schematics ./new-project/src/collection.json:new-project fancy-app
If you do this, you’ll need to make sure that you have @schematics/angular installed locally, by running npm install -g @schematics/angular first.
Alright, so this is already useful. Now, whenever we want to create a new project, we can run our schematic, and we don’t have to remember which options to pass into ng new!
We can make it even more useful though. For a start, there are a few files that we have to add to every new repository that we create:
-
-
- .npmrc: points to our internal NPM repository
- .nvmrc: determines which version of Node we’re using
- .prettierrc: code formatting settings
-
There are also a few files that get generated by the CLI, but that we need to change
-
-
- browserslist & polyfills.ts: we need to support IE11
- .karma.conf.js: we need to set up some reporters
- tslint.json: we use a slightly different set of linting rules
-
All of these files are exactly the same in each repository, so it would be handy if the schematic could just add them automatically. Well, it turns out it can!
Call an External Schematic
The first thing we need to do is create a folder that contains all the files that we want to add, using the directory structure that they need to be added in. We can call this folder anything we want, but I’m going to go with ‘files’. So our folder structure needs to look something like this:
— files
— [project-name]
— .npmrc
— .nvmrc
— .prettierrc
— browserslist
— polyfills.ts
— karma.conf.js
— tslint.json
Remember, when we run this schematic, the current directory is going to be the parent of the project directory, so we need to specify the name of the project directory in our directory structure. Otherwise the files would all be added to the parent directory, which would not be useful. Unfortunately, we don’t know the name of the project in advance, so we need some kind of placeholder. Happily, it turns out the schematics package provides us with a handy way to add placeholders which will be replaced by values we provide.
In a filename, we can use __optionName__. So, in our case, the [project-name] folder would be called __name__. This gives us the following directory structure:
— files
— __name__
— .npmrc
— .nvmrc
— .prettierrc
— browserslist
— polyfills.ts
— karma.conf.js
— tslint.json
You can also add placeholders inside files, using <%= optionName %>. Plus, the schematics package provides a bunch of handy functions to do simple transforms on the option values, like camelize (turns my-option to camel case: myOption) and classify (capitalises my-option like a class name: MyOption). Your IDE will absolutely hate you using these, and will give you a bunch of errors, but don’t worry about it.
{
"rulesDirectory": ["codelyzer"],
"extends": ["../../tslint.json"],
"rules": {
"directive-selector": [
true,
"attribute",
"<%= prefix %>",
"camelCase"
],
"component-selector": [
true,
"element",
"<%= prefix %>",
"kebab-case"
]
}
}
Ok, once we’ve got our files, we need a way to add them to the Tree. To do this, we can use another Rule factory, called mergeWith. To use mergeWith, we need to pass in a templateSource and a MergeStrategy.
We can create a templateSource as follows:
const templateSource = apply(url('./files'), [
template({..._options, ...strings}),
]);
'./files' is the location of the files we want to include, _options is the options passed into our schematic (which will be used by the placeholders), and strings is a collection of utility functions (like camelize and classify) provided by schematics. You can import string from @angular-devkit/core.
Once we’ve got our templateSource, we can pass it into the mergeWith factory along with a MergeStrategy.
const merged = mergeWith(templateSource, MergeStrategy.Overwrite)
MergeStrategy.Overwrite means that if the Tree contains a file that’s also in our templateSource, then use the one in our templateSource.
Ok, so now we have two rules – one returned by mergeWith, and one from externalSchematic. We need to apply both of them to our tree, but we can only return a single rule. So we need some way to combine the two.
Call an External Schematic
Enter chain. chain takes in a list of Rule factories, and returns a factory that combines them all.
const rule = chain([ generateRepo(name), merged ]); return rule(tree, _context) as Rule;
(I factored the code for calling externalSchematic out into a function to simplify this).
So, all together, our schematic now looks like this:
import { Rule, SchematicContext, Tree, externalSchematic, apply, url, template, chain, mergeWith, MergeStrategy } from '@angular-devkit/schematics';
import { strings } from '@angular-devkit/core';
export function newRepo(_options: any): Rule {
const name = _options.name;
return (tree: Tree, _context: SchematicContext) => {
const templateSource = apply(url('./files'), [
template({..._options, ...strings}),
]);
const merged = mergeWith(templateSource, MergeStrategy.Overwrite)
const rule = chain([
generateRepo(name),
merged
]);
return rule(tree, _context) as Rule;
}
}
function generateRepo(name: string): Rule {
return externalSchematic('@schematics/angular', 'ng-new', {
name,
version: '9.0.0',
directory: name,
routing: false,
style: 'scss',
inlineStyle: false,
inlineTemplate: false
});
}
And, if we run it, we get something like this:
> schematics .:new-project fancy-app ✔
CREATE fancy-app/README.md (1025 bytes)
CREATE fancy-app/.editorconfig (274 bytes)
CREATE fancy-app/.gitignore (631 bytes)
CREATE fancy-app/angular.json (3678 bytes)
CREATE fancy-app/package.json (1285 bytes)
CREATE fancy-app/tsconfig.json (489 bytes)
CREATE fancy-app/tslint.json (3125 bytes)
CREATE fancy-app/browserslist (429 bytes)
CREATE fancy-app/karma.conf.js (1021 bytes)
CREATE fancy-app/tsconfig.app.json (210 bytes)
CREATE fancy-app/tsconfig.spec.json (270 bytes)
CREATE fancy-app/src/favicon.ico (948 bytes)
CREATE fancy-app/src/index.html (294 bytes)
CREATE fancy-app/src/main.ts (372 bytes)
CREATE fancy-app/src/polyfills.ts (2835 bytes)
CREATE fancy-app/src/styles.scss (80 bytes)
CREATE fancy-app/src/test.ts (753 bytes)
CREATE fancy-app/src/assets/.gitkeep (0 bytes)
CREATE fancy-app/src/environments/environment.prod.ts (51 bytes)
CREATE fancy-app/src/environments/environment.ts (662 bytes)
CREATE fancy-app/src/app/app.module.ts (314 bytes)
CREATE fancy-app/src/app/app.component.scss (0 bytes)
CREATE fancy-app/src/app/app.component.html (25725 bytes)
CREATE fancy-app/src/app/app.component.spec.ts (951 bytes)
CREATE fancy-app/src/app/app.component.ts (214 bytes)
CREATE fancy-app/e2e/protractor.conf.js (808 bytes)
CREATE fancy-app/e2e/tsconfig.json (214 bytes)
CREATE fancy-app/e2e/src/app.e2e-spec.ts (642 bytes)
CREATE fancy-app/e2e/src/app.po.ts (301 bytes)
CREATE fancy-app/.npmrc (71 bytes)
CREATE fancy-app/.nvmrc (7 bytes)
CREATE fancy-app/.prettierrc (228 bytes)
You can see our additional files being added at the end. If you want to check that the existing files were overwritten by our versions, you’ll need to run the schematic with debug mode turned off.
Editing files
The last thing we want to do is make some changes to our package.json.
-
-
- Add
prettier, and husky to thedevDependencies - Add a commit hook using husky
- Add some additional scripts for our CI server
- Add
-
In theory, we could just include a standard package.json file like we did with our other files. However, when we use the CLI to generate a project, it also generates the package.json, so we’d risk our package.json file getting out of sync with the generated one. Instead, we’re going to use the generated one, and alter it. You can use the same technique to alter your angular.json if you need to – for example if you want to add assets or styles.
To do this, we’re going to write our own Rule factory, called updatePackageJson. This factory will need to be passed in the name of the project, and will return a Rule.
function updatePackageJson(name: string): Rule {
return (tree: Tree): Tree => {
}
}
The next thing we need to do is read in the current package.json file. We can use tree.read(path) to fetch the contents of the file as a buffer, then, because we’re dealing with a JSON file, we can use JSON.parse() to parse it.
const path = `/${name}/package.json`;
const file = tree.read(path);
const json = JSON.parse(file!.toString());
Don’t forget to convert the buffer to a string, using the buffer’s toString() method. The ! tells TypeScript that we are certain file won’t be null, so it doesn’t need to worry about it.
Now that we’ve parsed our package.json file, we can manipulate it just like any other object. I’m going to extend the scripts property with some additional scripts, and add a husky property, and our devDependencies.
json.scripts = {
...json.scripts,
'build:prod': 'ng build --prod',
'test:ci': 'ng test --no-watch --code-coverage'
};
json.husky = {
'hooks': {
'pre-commit': 'pretty-quick --staged --pattern \"apps/**/**/*.{ts,scss,html}\"'
}
};
json.devDependencies.prettier = '^2.0.0';
json.devDependencies.husky = '^4.2.0';
The one downside to this approach is that we need to know the version of prettier and husky to use. Or, we could look on this as an upside, as it means all our projects will be using the same version. If you just want to always install the latest version, you can use the value latest instead. If you want to install the latest version at the time the project is created (and then have everyone working on the project use that same version), you can fetch that information from the NPM API before writing the JSON file. That’s a little complicated for this post though!
The final thing we need to do is update the tree, and return it.
tree.overwrite(path, JSON.stringify(json, null, 2)); return tree;
Note that tree.overwrite() does what it says on the tin, and actually updates the current tree object, rather than returning a new, updated object.
So now, we’ve got our completed schematic.
import { Rule, SchematicContext, Tree, externalSchematic, apply, url, template, chain, mergeWith, MergeStrategy } from '@angular-devkit/schematics';
import { strings } from '@angular-devkit/core';
export function newRepo(_options: any): Rule {
const name = _options.name
return (tree: Tree, _context: SchematicContext) => {
const templateSource = apply(url('./files'), [
template({..._options, ...strings}),
]);
const merged = mergeWith(templateSource, MergeStrategy.Overwrite)
const rule = chain([
generateRepo(name),
merged,
updatePackageJson(name)
]);
return rule(tree, _context) as Rule;
}
}
function generateRepo(name: string): Rule {
return externalSchematic('@schematics/angular', 'ng-new', {
name,
version: '9.0.0',
directory: name,
routing: false,
style: 'scss',
inlineStyle: false,
inlineTemplate: false
});
}
function updatePackageJson(name: string): Rule {
return (tree: Tree, _: SchematicContext): Tree => {
const path = `/${name}/package.json`;
const file = tree.read(path);
const json = JSON.parse(file!.toString());
json.scripts = {
...json.scripts,
'build:prod': 'ng build --prod',
test: 'ng test --code-coverage',
lint: 'ng lint --fix',
};
json.husky = {
'hooks': {
'pre-commit': 'pretty-quick --staged --pattern \"apps/**/**/*.{ts,scss,html}\"',
}
};
json.devDependencies.prettier = '^2.0.0';
json.devDependencies.husky = '^4.2.
tree.overwrite(path, JSON.stringify(json, null, 2));
return tree;
}
}
Publishing our Schematic
Now that our schematic is complete, we need to make it available for others to use. This works just like any other NPM package. At my workplace, we have a Jenkins job which runs npm publish whenever we push a version with a tag.
Once your schematic is published, anyone who wants to use it will need to install it globally.
> npm install -g new-project`
It needs to be installed globally like this because we need to use it before we’ve created our project. If you were creating a schematic that, say, generated new components, after the project had been created, you could just install it locally.
Finally, you can use the schematic via:
> ng new fancy-project --collection=new-project
It’s a little cumbersome, but you only have to do it once per project!
Conclusion
So that’s it. We’ve created a new schematic which:
-
-
- calls
ng new - adds some new files
- changes some of the files provided by the CLI
- adds some additional dependencies
- calls
-
We’ve looked at how to create the schematic, how to improve the user experience through the use of a schema, and how to publish and use our schematic. Hopefully you find some of this useful!
