NodeJS and the Raspberry Pi

After the move to Canada I decided to rebuild my development infrastructure, mostly because it afforded me a good opportunity to play with some technologies I’d not played with before.

For how to install NodeJS on the Raspberry Pi check out this post.

My Infrastructure

My infrastructure is made up of a few Raspberry Pi’s, a generation one and two generation two’s. There’s also the upcoming additions of an ODroid C2 and XU4to replace the existing Raspberry Pi 2 currently running my Plex server.

What’s important here however is the Raspberry Pi 1 running a PostgreSQL database. This was originally set up in order to understand the steps required to get a PostgreSQL database running, as well as continue development in a platform agnostic way – my dev machine runs Windows 10, so whatever I write on my dev machine must also be able to run on debian as my Raspberry Pi’s run a mixture of debian 7, debian 8, devuan 1. It also seemed like a good idea to have a database running remotely so I could get some experience configuring and troubleshooting remote connections for PostgreSQL.

The Problem with Mono

While re-imaging my Raspberry Pi 1 it became apparent that I could no longer install mono via apt. The default repositories were missing a number of dependencies, and even after using the repository set up that Xamarin suggests it still didn’t work: it catastrophically failed, and I had to manually recover the installed broken packages.

After some investigation it appeared that mono was no longer available for the Raspberry Pi 1 in relation to Raspbian for that distribution not supporting hardware floating point. Unfortunately I’m unable to relocate the forum that discusses this, however plugwash alludes to this issue here.

Researching further on this it does appear that there are solutions to the issue, however the path I decided to go down was to port the existing tools for the database server from C# .NET/Mono to NodeJS. Why? Two Reasons: first it doesn’t look like mono is ever going to be an easy install for the Raspberry Pi again and why move forward with something that isn’t being supported properly; second however given the purpose of the database is a self-teaching tool porting to NodeJS is an excellent opportunity to learn.

Porting from C# .NET/Mono to NodeJS

There was two existing jobs written in C# .NET/Mono that needed porting to NodeJS. These two jobs ran on the same server as the database, although bad practice it’s only for development and so mattered very little. The two jobs helped maintain the database, creating rows in a dimension table every day: the PostgreSQL database was a dimension modelled data mart, where one of the jobs maintained the date dimension always had enough date rows.

Porting these jobs was a very good exercise in learning NodeJS practically. The job definitely worked before, it had been observed to work, and the process was explicitly laid out in working C#. Therefore re-writing the job in Javascript should be fairly straight forward.

pg & pg-promise

The biggest hurdle to overcome was how to connect to the database. If there was no way to connect to a PostgreSQL database from Javascript then this port wouldn’t be happening. Fortunately there are two libraries called pg and pg-promise which allow fairly easy access to a PostgreSQL database.

As the database connection was going to be used across both jobs, potentially more later on, I decided to write quick a utility to create the database connection for my project.

var pgp = require('pg-promise')({
  promiseLib: require('promise')
});

module.exports = {
  getConnection: function(connectionObject) {
    return pgp(connectionObject);
  }
};

All this utility does it return a connection to the PostgreSQL database defined in the connectionObject, which for pg could be either a connection string or an actual object defined in the documentation for pg-promise.

module.exports = {
  connection: {
    mart: {
      host: '127.0.0.1',
      port: 5432,
      database: 'mart',
      user: 'user',
      password: 'password'
    },
    admin: {
      host: '127.0.0.1',
      port: 5432,
      database: 'admin',
      user: 'user',
      password: 'password'
    },
  }
};

Above shows an example of what it may look like as a module used in NodeJS.

The Solution

With access to the database resolved the next step was re-creating the job with the correct syntax and code layout for Javascript. The latter part of this was the harder part, first because NodeJS is single threaded and everything is based on promises, second because I am not very experienced in writing Javascript applications from nothing.

The layout chosen was to follow the examples given on Github for pg-promise, and use callbacks for everything via promises, including exception handling. Below shows the resultant Javascript:

var config = require('../config.js');
var dataaccess = require('../utilities/dataAccess.js');
var moment = require('moment');

// Log the progress
console.log('Beginning maintenance on Date Dimension...');

// Connect to the mart database
var connection = false;
exports = module.exports = dataaccess
  .getConnection(config.connection.mart)
  .connect()
  .then(function (obj) {

    // Store the connection
    connection = obj;

    // Return the highest database key
    return connection.one('SELECT MAX("Date") AS date FROM public."Date Dimension";');
  })
  .then(function (data) {

    //  Check whether the row came back successfully
    var minDate = moment('1999-12-31');
    if(data && data.date) {
      minDate = moment(data.date);
    }

    // Calculate the date range
    var maxDate = moment().startOf().add(1, 'years');
    var maxDays = maxDate.diff(minDate, 'days');

    // Populate until the max date
    var promises = [];
    for (var i = 1; i < maxDays; i++) {

      // Add the new date dimension key
      var newDate =  moment(minDate).add(i, 'days');
      console.log('Adding new date...', newDate);
      promises.push(connection
        .proc('public.AddDateDimensionKey', [ newDate ])
        .catch(function (error) {
          console.log(error);
        }));
    }

    // Return all the promises
    return Promise.all(promises);
  })
  .catch(function (error) {

    // Log the error
    console.log(error);
  })
  .then(function () {

    // Close the connection
    if (connection) {
      connection.done();
    }

    // Log the end
    console.log('Finished maintenance on Date Dimension.');
  });

Although this works, it does not lend itself well to unit testing. The previous incarnation written in C# was created using dependency injections, and was very easily unit tested to ensure that the correct number of dates were at least attempted to be created. For this reason, and others that would require me to have further experience with Javascript, means this is not a perfect example of a NodeJS application.

However it is by practical means a successful port from C# .NET/Mono to NodeJS, and now this job runs on both Windows and Linux without an issue, so long as the config.js is set up correctly.