One of the most common uses of Node.js is the development of web APIs. Numerous modules from the community are available for this, covering a whole range of aspects, such as routing, validation, and CORS.

The first part of this series introduced Node.js as a server-side runtime environment for JavaScript and showed how to write a simple web server. In addition, the package management npm was introduced, which allows us to easily install modules written by the community into our own application. So, we already know some of the basics, but the developed application still lacks meaningful functionality.

This will change in this part of the series: The application, which so far only launches a rudimentary web server, is supposed to provide an API that can be used to manage a task list. First, it is necessary to make some technical preliminary considerations, because we must define what exactly the application is supposed to do. For example, the following functions are possible:

It must be possible to write down a new task. In the simplest form, this task consists of only a title, which must not be empty.
It must also be possible to call up a list of all tasks that still need to be done, in order to see what still needs doing.
Last but not least, it must be possible to check off a completed task so that it is removed from the todo list.

These three functions are essential, without them a task list cannot be used meaningfully. All other functions, such as renaming a task or undoing the check-off of a task, are optional. Of course, it would make sense to implement them in order to make the application as user-friendly and convenient as possible – but they are not really necessary. The three functions mentioned above represent the scope of a Minimum Viable Product (MVP), so to speak.

Another restriction should be specified right at the beginning: The task list shall deliberately not have user management in order to keep the example manageable. This means that there will be neither authentication nor authorization, and it will not be possible to manage multiple task lists for different people. This would be essential to use the application in production, but it is beyond the scope of this article and ultimately offers little learning for Node.js.

Current state

The current state of the application we wrote in the first part includes two code files: app. js, which starts the actual server, and lib/getApp.js, which contains the functionality to respond to requests from the outside. In the app.js file, we already used the npm module processenv [1] to be able to set the port to a value other than the default 3000 via an environment variable (Listing 1).

'use strict';
 
const getApp = require('./lib/getApp');
const http = require('http');
const { processenv } = require('processenv');
 
const port = processenv('PORT', 3000);
 
const server = http.createServer(getApp());
 
server.listen(port);

The good news is that at this point, nothing will change in this file. This is because there is already a separation of content in the app.js and getApp.js files: The first file takes care of the HTTP server itself, while the second contains the actual logic of the application. In this part of the article series, only the application logic will be adapted and extended, so the app.js file can remain as it is.

However, the situation is different in the getApp.js file, where we will leave no stone unturned. But, one thing at a time. First, the package.json file must be modified so that the name of the application is more meaningful. For example, instead of my-http-server, the application could be called tasklist:

{
  "name": "tasklist",
  "version": "0.0.1",
  "dependencies": {
    "processenv": "3.0.2"
  }
}

The file and directory structure of the application still looks the same as in the first part:

/
  lib/
    getApp.js
  node_modules/
  app.js
  package.json
  package-lock.json

REST? No thanks!

Now it’s a matter of incorporating routing. As usual with APIs, this is done via different paths in the URLs. In addition, you can fall back on the different HTTP verbs such as GET and POST to map different actions. A common pattern is the so-called REST approach, which specifies that so-called resources are defined via the URL and the HTTP verbs define the actions on these resources. The usual mapping according to REST is as follows:

POST creates a new resource, and corresponds to a Create.
GET retrieves a resource, and represents the classic Read.
PUT updates a resource, and corresponds to an Update.
DELETE finally deletes a resource, and corresponds to a Delete.

As you can see, these four HTTP verbs can be easily mapped to four actions of the so-called CRUD pattern, which in turn corresponds to the common approach of how to access data in (relational) databases. This is one of the most important reasons for the success of REST: It is simple and builds on the already familiar logic of databases. Nevertheless, there are some reasons against using this transfer of CRUD to the API level. The most weighty of these is that the verbs do not conform to the technical language: Users do not talk about creating or updating a task.

Instead, they think in terms of technical processes: They want to make a note of a task or check off a task as completed. This is where a business and a technical view collide. It is obvious that a mapping between these views must take place at some point – but the code of an application should tend to be structured in a domain-oriented rather than a technical way [2]. After all, the application is written to solve a domain-oriented problem, and technology is merely the means to an end. Seen in this light, CRUD is also an antipattern [3].

An alternative approach is provided by the CQRS pattern, which is based on commands and queries [4]. A command is an action that changes the state of the application and reflects a user’s intention. A command is usually in the imperative, since it is a request to the application to do something. In the context of the task list, there are two actions that change the state of the list, noting and checking off a task. If we formulate these actions in the imperative and translate them into English, we get phrases such as “Note a todo.”, “Tick off a todo.”

Analogously, you can formulate a query, i.e. a query that doesn’t change the state of the application, but returns it. This is the difference between a command and a query: A command writes to the application, so to speak, while a query reads from the application. The CQRS pattern states that every interaction with an application should be either a command or a query – but never both at the same time. In particular, this means that Commands should not return the current state of the task list, but that a separate Query is needed for that: For example: “Get pending todos.”

If we abandon the idea that an API must always be structured according to REST and prefer the much simpler pattern of separating writing and reading, the question arises as to how the URLs should be structured and which HTTP verbs should be used. In fact, the answer to this question is surprisingly simple: The URLs are formulated exactly as mentioned above, POST for commands, and GET for queries are used as HTTP verbs – that’s it. This results in the following routes:

POST /note-todo
POST /tick-off-todo
GET /pending-todos

The beauty of this approach is that it is much more self-explanatory than REST. POST /tick-off-todo is much more technical than a PUT /todo. Here, it is clear that an update is executed, but which functional purpose this update has is unclear. When there are different reasons for initiating a (technical) update, the semantically stronger approach gains a lot in comprehensibility and traceability.

Define routes

Now it is necessary to define the appropriate routes. However, this is not done with Node.js’s on-board tools. Instead, we can use the npm module Express [5]:

$ npm install express

The module can now be loaded and used within the getApp.js file. First, an express application has to be defined, for which only the express function has to be called. Then, the get and post functions can be used to define routes, specifying the desired path name and a callback – similar to the one used in the standard Node.js server (Listing 2).

'use strict';
 
const express = require('express');
 
const getApp = function () {
  const app = express();
 
  app.post('/note-todo', (req, res) =&amp;amp;amp;gt; {
    // ...
  });
 
  app.post('/tick-off-todo', (req, res) =&amp;amp;amp;gt; {
    // ...
  });
 
  app.get('/pending-todos', (req, res) =&amp;amp;amp;gt; {
    // ...
  });
 
  return app;
};
 
module.exports = getApp;

With this, the basic framework for the routes is already built. The individual routes can, of course, also be swapped out into independent files, but for the time being, focus should be on implementing functionality. The next step is to implement a task list, which is initially designed as a pure in-memory solution. However, since it will be backed by a database in a future part of this series, it will be designed from the outset to be seamlessly extensible later. Essentially, this means that all functions to access the task list will be created asynchronously, since accesses to databases in Node.js are usually asynchronous. For the same reason, an asynchronous initialize function is also created, which may seem unnecessary at this stage, but will later be used to establish the database connection.

Defining the todo list

The easiest way to do this is to use a class called Todos, to which corresponding methods are attached. Again, these methods should be named functionally and not technically, i.e. their names should be based on the names of the routes of the API. The class is placed in a new file in the lib directory, resulting in lib/Todos.js as the file name. For each task that is noted, an ID should also be generated, and the time of creation should be noted. While accessing the current time is not a problem, generating an ID requires recourse to an external module such as uuid, which can also be installed via npm:

$ npm install uuid

Last but not least, it is advisable to get into the habit from the very beginning of providing every .js file with strict mode, a special JavaScript execution mode in which some dangerous language constructs are not allowed, for example, the use of global variables. To enable the mode, you need to insert the appropriate string at the beginning of a file as a kind of statement. This makes the full contents of the app.js file look like the one shown in Listing 1.

'use strict';
 
const { v4 } = require('uuid');
 
class Todos {
  constructor () {
    this.items = [];
  }
 
  async initialize () {
    // Intentionally left blank.
  }
 
  async noteTodo ({ title }) {
    const id = v4();
    const timestamp = Date.now();
 
    const todo = {
      id,
      timestamp,
      title
    };
 
    this.items.push(todo);
  }
 
  async tickOffTodo ({ id }) {
    const todoToTickOff = this.items.find(item =&amp;amp;amp;gt; item.id === id);
 
    if (!todoToTickOff) {
      throw new Error('Todo not found.');
    }
 
    this.items = this.items.filter(item =&amp;amp;amp;gt; item.id !== id);
  }
 
  async getPendingTodos () {
    return this.items;
  }
}
 
module.exports = Todos;

It is striking in the implementation that the functions representing a command actually contain no return, while the function representing a query consists of only a single return. The separation between writing and reading has become very clear.

Now the file getApp.js can be extended accordingly, so that an instance of the task list is created there and the routes are adapted in such a way that they call the appropriate functions. To prepare the code for later, the initialize function should be called now. However, since this is marked as async, the getApp function must call it with the await keyword, and therefore, must also be marked as asynchronous (Listing 4).

'use strict';
 
const express = require('express');
const Todos = require('./Todos');
 
const getApp = async function () {
  const todos = new Todos();
  await todos.initialize();
 
  const app = express();
 
  app.post('/note-todo', async (req, res) =&amp;amp;amp;gt; {
    const title = // ...
 
    await todos.noteTodo({ title });
  });
 
  app.post('/tick-off-todo', async (req, res) =&amp;amp;amp;gt; {
    const id = // ...
 
    await todos.tickOffTodo({ id });
  });
 
  app.get('/pending-todos', async (req, res) =&amp;amp;amp;gt; {
    const pendingTodos = await todos.getPendingTodos();
 
    // ...
  });
 
  return app;
};
 
module.exports = getApp;

Before the application can be executed, three things have to be done:

First, the title and id parameters must be determined from the request body.
Second, the query route must return the read tasks to the client as a JSON array.
Finally, the app.js file must be modified so that the getApp function is called asynchronously there.

Input and output with JSON

Fortunately, all three tasks are easy to accomplish. For the first task, it is first necessary to determine what a request from the client looks like, i.e. what form it takes. In practice, it has proven useful to send the payload as part of a JSON object in the request body. For the server, this means that it must read this object from the request body and parse it. A suitable module called body-parser [6] is available in the community for this purpose and can be easily installed using npm:

$ npm install body-parser

It should be noted that the version number must always consist of three parts and follow the concept of semantic versioning [6]. In addition, however, dependencies can also be stored in this file, whereby required third-party modules are explicitly added. This makes it much easier to restore a certain state later or to get an overview of which third-party modules an application depends on. To install a module, call npm as follows:

$ npm install processenv

It can then be loaded with require:

const bodyParser = require('body-parser');

Since the parser will be available for several routes, it is implemented as so-called middleware. In the context of Express, middleware is a type of plug-in that provides functionality for all routes and therefore only needs to be registered once instead of individually for each route. This is done in Express via the app.use function. Therefore, it is important to insert the following line directly after creating the Express application: app.use(bodyParser.json());

Now the property body of the req object can be accessed within the routes, which was not available before. Provided a valid JSON object was submitted, this property now contains that very object. This allows the two command routes to be extended, as shown in Listing 5.

app.post('/note-todo', async (req, res) =&amp;amp;gt; {
  const { title } = req.body;
 
  await todos.noteTodo({ title });
});
 
app.post('/tick-off-todo', async (req, res) =&amp;amp;gt; {
  const { id } = req.body;
 
  await todos.tickOffTodo({ id });
});

When implementing the tick-off-todo route, it is noticeable that error handling is still missing: If the task to be ticked off is not found, the tickOffTodo function of the Todos class raises an exception – but this is not caught at the moment. So it is still necessary to provide the corresponding call with a try/catch and to return a corresponding HTTP status code in case of an error. In this case, the error code 404, which stands for an element not found (Listing 6), is a good choice.

app.post('/tick-off-todo', async (req, res) =&amp;amp;gt; {
  const { id } = req.body;
 
  try {
    await todos.tickOffTodo({ id });
  } catch {
    res.status(404).end();
  }
});

And finally, in addition to the node_modules directory, npm has also created a file called package-lock.json. It is actually used to lock version numbers despite the roof being specified. However, it has its quirks, so if npm behaves strangely, it’s often a good idea to delete this file and the node_modules directory and run npm install again from scratch. Once a module has been installed via npm, it can be loaded in the same way as a module built into Node.js. In that case, Node.js recognizes that it is not a built-in module and loads the appropriate code from the node_modules directory:

app.get('/pending-todos', async (req, res) =&amp;amp;gt; {
  const pendingTodos = await todos.getPendingTodos();
 
  res.json(pendingTodos);
});

Now, if you start the server by entering node app.js and try to call some routes, you will notice that some of the routes work as desired – but others do not, because they never end. This is where an effect comes into play that is very unusual at first: Node.js is inherently designed to stream data, so an HTTP connection is not automatically closed when a route has been processed. Instead, it has to be done explicitly, as in the case of the 404 error. The json function already does this natively, but the two command routes still lack closing the connection successfully. To indicate that the operation was successful, it is a good idea to send the HTTP status code 200. The getApp.js file now looks like Listing 7.

'use strict';
 
const bodyParser = require('body-parser');
const express = require('express');
const Todos = require('./Todos');
 
const getApp = async function () {
  const todos = new Todos();
  await todos.initialize();
 
  const app = express();
  app.use(bodyParser.json());
 
  app.post('/note-todo', async (req, res) =&amp;amp;gt; {
    const { title } = req.body;
 
    await todos.noteTodo({ title });
    res.status(200).end();
  });
 
  app.post('/tick-off-todo', async (req, res) =&amp;amp;gt; {
    const { id } = req.body;
 
    try {
      await todos.tickOffTodo({ id });
      res.status(200).end();
    } catch {
      res.status(404).end();
    }
  });
 
  app.get('/pending-todos', async (req, res) =&amp;amp;gt; {
    const pendingTodos = await todos.getPendingTodos();
 
    res.json(pendingTodos);
  });
 
  return app;
};
 
module.exports = getApp;

Validate the inputs

What is still missing is a validation of the inputs: At the moment, it is quite possible to call one of the command routes without passing the required parameters in the request body. In practice, it has proven useful to validate JSON objects by using a JSON schema. A JSON schema represents a description of the valid structure of a JSON object. In order to be able to use JSON schemas, a module is again required, for example, validate-value [7] which can be installed via npm:

$ npm install validate-value

Now the module can be loaded in the getApp.js file:

const { Value } = require('validate-value');

The next step is to create two schemas. Since these are always the same, it is advisable not to do this inside the routes, but outside them, so that the code does not have to be executed over and over again, ultimately ending up with the same result each time (Listing 8).

const noteTodoSchema = new Value({
  type: 'object',
  properties: {
    title: { type: 'string', minLength: 1 }
  },
  required: [ 'title' ],
  additionalProperties: false
});
 
const tickOffTodoSchema = new Value({
  type: 'object',
  properties: {
    id: { type: 'string', format: 'uuid' }
  },
  required: [ 'id' ],
  additionalProperties: false
});

Within the two command routes, the only thing left to do is to validate the received data using the respective schema, and in case of an error, return an appropriate HTTP status code, for example, a 400 error (Listing 9).

app.post('/note-todo', async (req, res) =&amp;amp;gt; {
  if (!noteTodoSchema.isValid(req.body)) {
    return res.status(400).end();
  }
 
  const { title } = req.body;
 
  await todos.noteTodo({ title });
  res.status(200).end();
});
 
app.post('/tick-off-todo', async (req, res) =&amp;amp;gt; {
  if (!tickOffTodoSchema.isValid(req.body)) {
    return res.status(400).end();
  }
 
  const { id } = req.body;
 
  try {
    await todos.tickOffTodo({ id });
    res.status(200).end();
  } catch {
    res.status(404).end();
  }
});

CORS and testing

With this the API is almost finished, only a little bit of small stuff is missing. For example, it would be handy to be able to configure CORS – that is, from which clients the server can be accessed. In practice, this topic is a bit more complex than described below, but for development purposes, it is often sufficient to allow access from everywhere. The best way to do this is to use the npm module cors [8], which must first be installed via npm:

$ npm install cors

It must then be loaded, which is again done in the getApp.js file:

const cors = require('cors');

Finally, it must be integrated into the express application in the same way as body-parser, because this module is also middleware. Whether this call is made before or after the body-parser does not really matter – but since access should be denied before the request body is processed, it makes sense to include cors as the first middleware:

// ...
const app = express();
app.use(cors());
app.use(bodyParser.json());
// ...

Now, in order to test the API, a client is still missing. Developing this right now would be too time-consuming, so you can fall back on a tool that is extremely practical for testing HTTP APIs and that is usually pre-installed on macOS and Linux, namely, curl. On Windows, it is also available, at least in the Windows Subsystem for Linux (WSL). First, you can try to retrieve the (initially empty) list of all tasks:

$ curl http://localhost:3000/pending-todos
[]

In the next step, you can now add a task. Make sure that you not only send the required data, but also set the Content-Type header to the correct value – otherwise the body-parser will not be active:

$ curl \
  -X POST \
  -H 'content-type:application/json' \
  -d '{"title":"Develop a Client"}' \
  http://localhost:3000/note-todo

If you retrieve the tasks again, you will get a list with one entry (in fact, the list would be output unformatted in a single line, but for the sake of better readability it is shown formatted in the following):

$ curl http://localhost:3000/pending-todos
[
  {
    "id": "dadd519b-71ec-4d18-8011-acf021e14365",
    "timestamp": 1601817586633,
    "title": "Develop a Client"
  }
]

If you try to check off a task that does not exist, you will notice that this has no effect on the list of all tasks. However, if you use the -i parameter of curl to also output the HTTP headers, you will see that you get the value 404 as the HTTP status code:

$ curl \
  -i \
  -X POST \
  -H 'content-type:application/json' \
  -d '{"id":"43445c25-c116-41ef-9075-7ef0783585cb"}' \
  http://localhost:3000/tick-off-todo

The same applies if you do not pass a UUID as a parameter (or specify an empty title in the previous example). However, in these cases, you get the HTTP status code 400. Last but not least, you can now try to actually check off the noted task by passing the correct ID:

$ curl \
  -X POST \
  -H 'content-type:application/json' \
  -d '{"id":"dadd519b-71ec-4d18-8011-acf021e14365"}' \
  http://localhost:3000/tick-off-todo

If you retrieve the list of all unfinished tasks again, you will get an empty list

– as desired:

$ curl http://localhost:3000/pending-todos
[]

Outlook

This concludes the second part of this series on Node.js. Of course, there is much more to discover in the context of Node.js and Express for writing Web APIs. Another article could be dedicated to the topics of authentication and authorization alone. But now we have a foundation to build upon.

The biggest shortcoming of the application at the moment is that it is not possible to ensure code quality and the code has already become relatively confusing. There is a lack of structure, binding specifications regarding the code style, and automated tests. These topics will be dealt with in the third part of the series – before further functionality can be added.

The author’s company, the native web GmbH, offers a free video course on Node. js [9] with close to 30 hours of playtime. Episodes 4 and 5 of this video course deal with topics covered in this article, such as developing web APIs, using Express, and using middleware. Therefore, this course is recommended for anyone interested in more details.