How to support promised predicates in Array find

I’m building ACL support in NodeJS. For example, I’ll allow to write:

{[ .can-example | 1.hilite(=javascript=) ]}

For any given action, my ´Can´ function must check a list of permissions. If one matches, then the action ´Can´ asks for is granted, otherwise (no permission exists) it’s denied. Of course I’m only interested in the first permission to match, and to allow for optimizations, I want permissions to be checked in the order I provide.

It’s easy to see that my problem is solved by the Array.prototype.find method. The problem I have, though, is that it only works with immediate predicates, but my checks can entail both immediate and promised predicates. For example, I allow predicates to access the database.

I googled my problem and found this StackOverflow page. Bergi’s answer gives both recursive and non-recursive solutions. (Side note. There was a time when recursive was opposed to iterative. With promises, that’s no longer the case. In fact, the non-recursive solution is a chain of ´catch´ handlers. An iteration is used to build the chain but promises themselves, throwing exceptions, control the iteration.) Benjamin Gruenbaum’s answer gives a recursive solution.

Here are their issues.

  1. The predicate is hacky because it signals a ´false´ by throwing an exception.
  2. The promise management is hacky because (a) it must cater for the predicate with ´.catch()´, and (b) it signals the “not found” outcome with ´reject()´.
  3. The promise management and the predicate are very coupled.
  4. The contract is different from that of the Array.prototype.find method.

So I came up with this one.

{[ .ArrayFind | 1.hilite(=javascript=) ]}

Apart from being a global function instead of an Array instance method, the contract is exactly the same as that of the Array.prototype.find method.

  1. The only hack I used is to immediately exit when an element is found instead of continuing until the end of the ´.then()´ chain. But how I implemented it is both robust (as in Robustness) and hidden (as in Information Hiding). To make sure I do not mistake a rightful exception with my hack, I throw my own fake exception which is a wrapper around the found element. Thus, my fake exception is caught by the last ´.catch()´ and the element is returned.
  2. The predicate can be both immediate or a promise, thanks to ´Promise.resolve(Predicate…)´. If it’s immediate, it can throw an exception if it has to, not if it doesn’t hold true. If it’s a promise, it can reject() if it has to, not if it doesn’t hold true.

Examples

Here are some examples.

{[ .predicates | 1.hilite(=javascript=) ]}

The result is a promise

Here is how ArrayFind compares to Array.prototype.find when no exceptions are thrown:

[1,13,5,4,7].find(ImmediatePredicate)
VM834:3 -- 1
VM834:3 -- 13
VM834:3 -- 5
VM834:3 -- 4
4

ArrayFind([1,13,5,4,7], ImmediatePredicate)
VM834:3 -- 1
VM834:3 -- 13
VM834:3 -- 5
VM834:3 -- 4
Promise {[[PromiseStatus]]: "pending", [[PromiseValue]]: undefined}

Of course the result is a Promise instead of the found element, but we can append additional handlers, like

ArrayFind([1,13,5,4,7], ImmediatePredicate)
  .then(function(result){
    console.log(result);
  })
  .catch(function(reason){
    console.warn(reason)
  })

VM834:3 -- 1
VM834:3 -- 13
VM834:3 -- 5
VM834:3 -- 4
VM880:4 4
Promise {[[PromiseStatus]]: "pending", [[PromiseValue]]: undefined}

An exception makes the promise reject

Here is how ArrayFind compares to Array.prototype.find when an exception is thrown:

[1,3,5,4,7].find(ImmediatePredicate)
VM834:3 -- 1
VM834:3 -- 3
VM834:4 Uncaught dirty

ArrayFind([1,3,5,4,7], ImmediatePredicate)
  .then(function(result){
    console.log(result);
  })
  .catch(function(reason){
    console.warn(reason)
  })

VM834:3 -- 1
VM834:3 -- 3
VM925:7 dirty
Promise {[[PromiseStatus]]: "pending", [[PromiseValue]]: undefined}

Immediate and promised predicates work equally well

Here you can see how ArrayFind supports at the same time immediate and promised predicates:

ArrayFind([1,13,5,4,7], MixedPredicate)
  .then(function(result){
    console.log(result);
  })
  .catch(function(reason){
    console.warn(reason)
  })

VM834:3 -- 1
Promise {[[PromiseStatus]]: "pending", [[PromiseValue]]: undefined}
VM834:12 -- 13
VM834:3 -- 5
VM834:3 -- 4
VM955:4 4


ArrayFind([1,3,5,4,7], MixedPredicate)
  .then(function(result){
    console.log(result);
  })
  .catch(function(reason){
    console.warn(reason)
  })

VM834:3 -- 1
VM834:3 -- 3
VM974:7 dirty
Promise {[[PromiseStatus]]: "pending", [[PromiseValue]]: undefined}


ArrayFind([1,12,5,4,7], MixedPredicate)
  .then(function(result){
    console.log(result);
  })
  .catch(function(reason){
    console.warn(reason)
  })

VM834:3 -- 1
Promise {[[PromiseStatus]]: "pending", [[PromiseValue]]: undefined}
VM834:12 -- 12
VM975:7 dirty

You can’t appreciate from the output above, but the immediate predicate really outputs immediately. 🙂

Also notice that in the second to last example the warned ´dirty´ comes from the immediate predicate because 3 < 10, while in the last example it comes from the promised predicate because 12 >= 10.

How to customize MorganJS

MorganJS is easy to install and works nicely out of the box.

{[.old-setup | 1.hilite(=javascript=) ]}

Here is what it looks like. Highlighted HTTP status codes are quite useful.

Screen Shot 2015-11-08 at 14.44.07

Thankfully, it’s possible to customize MorganJS by adding tokens, which are template symbols, like this:

{[ .current-user | 1.hilite(=javascript=) ]}

which can be later used like this:

{[ .middle-setup | 1.hilite(=javascript=) ]}

to produce something like this:

Screen Shot 2015-11-08 at 14.52.16

Uh-oh!! Where are my colors?

I delve into MorganJS code…

{[ .morgan | 1.hilite(=javascript=) ]}

As you may have noticed, the above code is hard to understand and quite hard-coded too.

  • Hard-coded because, even if the ´dev´ template is documented as ´:method :url :status :response-time ms – :res[content-length]´, it’s really embedded into the code and mixed up with extraneous bits rather than being declared into some option and used like any other MorganJS template is.
  • Hard to understand because the function object is being used as a cache for its own executions which entail a compilation step whose raison d’être I still have to grasp. I could be wrong, but this one could be a clear example of over-engineering.

However my biggest disappointment was that there is no way of reusing the colored ´:status´ token nor the coloring functionality, neither directly, by calling a method, nor indirectly, by copy-pasting some code. A total fail. 🙁

Googling “terminal colors” I eventually got to this Unix StackExchange answer, which I used to write this:

{[ .color-factory | 1.hilite(=javascript=) ]}

A nice collateral about my ´ColorFactory´ function is that I can use it also in the console like this:

{[ .console-log | 1.hilite(=javascript=) ]}

to get something like this:

Screen Shot 2015-11-08 at 16.12.17

Finally, I was able to customize Morgan with this:

{[ .morgan-factory | 1.hilite(=javascript=) ]}

and use it like this:

{[ .new-setup | 1.hilite(=javascript=) ]}

to get something like this:

Screen Shot 2015-11-08 at 16.51.54

 

How to improve filters with promises

I had been programming a filters setup for the node API of a MEAN stack app.

Having this ´User´ model:

// user.model.js (complete)

var mongoose = require('mongoose');

var schema   = new mongoose.Schema({
    name: String,
    admin: Boolean
});

module.exports = mongoose.model('User', schema);

It allowed a ´User´ controller like this:

// user.controller.js (complete)

var fields = [
    'name', 
    function (admin) { 
        return !!admin.length; 
    }
];

var Item = require('./user.model');
var Controller = require(global.absPath + '/app/shared/CRUD.controller');
module.exports = Controller(Item, fields);

The meaning should be straightforward: copy the ´name´ field as is and make the ´admin´ field a proper boolean. That was made possible by this:

// CRUD.controller.js (excerpt)

module.exports = CRUD_Controller;

function CRUD_Controller(Item, fields) {
    //...
    function Create(req, res) {

        var item = new Item();

        CopyFields(fields, req.body, item);

        item.save(function(err) {

            if (err) {
                return res.send(err);
            }

            res.json({
                message: 'Item created!'
            });

        });

    }


    function CopyFields(fields, data, item) {

        (fields || []).forEach(function(field) {

            switch (typeof field) {

                case 'string':
                    item[field] = data[field];
                    break;

                case 'function':
                    var matches = String(field).match(/^functions*(s*(w+)s*)/);
                    if (!(matches && matches[1])) {
                        console.log('Expected a function with only one argument.');
                        return;
                    }
                    var name = matches[1];
                    item[name] = field(data[name]);
                    break;

            }

        });

    }
    //...
}

Then I wanted to add a ´password´ field to the ´User´ model. For storing it I decided to go with Strong Password Hashing with Node.js Standard Library. Properly translated to JavaScript and slightly tweaked I got this:

// hash.js (complete)

var crypto = require('crypto');

module.exports = Hash;

return;

function Hash(options, callback) {

    // Default options.plaintext to a random 8-character string
    if (!options.plaintext) {
        return crypto.randomBytes(8, function(err, buf) {
            if (err) {
                return callback(err);
            }
            options.plaintext = buf.toString('base64');
            Hash(options, callback);
        });
    }

    // Default options.salt to a random 64-character string (512 bits)
    if (!options.salt) {
        return crypto.randomBytes(64, function(err, buf) {
            if (err) {
                return callback(err);
            }
            options.salt = buf.toString('base64');
            Hash(options, callback);
        });
    } 

    // Default options.iterations to 10k
    if (!options.iterations) {
        options.iterations = 10000;
    }

    // Default options.digest to sha1
    if (!options.digest) {
        options.digest = 'sha1';
    }

    crypto.pbkdf2(options.plaintext, options.salt, options.iterations, 64, options.digest, function(err, key) {
        if (err) {
            return callback(err);
        }
        options.algorithm = 'PBDFK2';
        options.key = key.toString('base64');
        callback(null, options);
    });

}

So my ´User´ model became this:

// user.model.js (complete)

var mongoose = require('mongoose');

var schema   = new mongoose.Schema({
    name: String,
    password: {
        algorithm:  String,
        digest:     String,
        iterations: Number,
        salt:       String,
        key:        String
    },
    admin: Boolean
});

module.exports = mongoose.model('User', schema);

Have you noticed that the ´Hash´ function relies on the asynchronous´crypto.pbkdf2´ function? That’s just standard, so I wasn’t going to use the synchronous version on a second thought.

Then my problem was:

How do I make these filters work with deferred values?

Ta-da! Promises:

// user.controller.js (complete)

var Promise = require('es6-promise').Promise;
var fields = [
    'name', 
    function (password) { 
        return new Promise(function (resolve, reject) {
            var Hash = require(global.absPath + '/app/components/auth/hash');
            Hash({plaintext: password}, function (error, result) {
                if (error) {
                    reject(Error(error));
                } else {
                    delete result.plaintext;
                    resolve(result);
                }
            });
        });
    }, 
    function (admin) { 
        return !!admin.length; 
    }
];

var Item = require('./user.model');
var Controller = require(global.absPath + '/app/shared/CRUD.controller');
module.exports = Controller(Item, fields);

To make that work I had to change a bit the ´CRUD´ controller.

The first change was to separate the filtering from the assignment, so that I could later use the ´Promise.all´ method which allows to synchronize promises and values as well. That implied to pass from a ´CopyFields´ function which filters and assigns each value in turn to a ´FilterFields´ function which filters all values at once, thus making the assignments directly in the ´Create´ function.

// CRUD.controller.js (broken excerpt) 
 
module.exports = CRUD_Controller; 
 
function CRUD_Controller(Item, fields) { 
    //... 
    function Create(req, res) {

        FilterFields(fields, req.body, function (fFields) {
            var item = new Item();

            fFields.forEach(function (fField) {
                item[fField.name] = fField.value;
            });

            item.save(function(err) {

                if (err) {
                    return res.send(err);
                }

                res.json({
                    message: 'Item created!'
                });

            });
        });

    }


    function FilterFields(fields, data, callback) {

        Promise
            .all((fields || []).map(Filter))
            .then(callback)
            .catch(function (error) {
                console.log(error);
            });


        function Filter(field) {
            var result;

            switch (typeof field) {

                case 'string':
                    result = {
                        name: field,
                        value: data[field]
                    };
                    break;

                case 'function':
                    var matches = String(field).match(/^functions*(s*(w+)s*)/);
                    if (!(matches && matches[1])) {
                        console.log('Expected a function with only one argument.');
                        return;
                    }
                    result = {
                        name: matches[1],
                        value: field(data[matches[1]])
                    };
                    break;

            }

            return result;
        }

    }
    //... 
}

The second change was to add a needed special treatment for my promises. You may have noticed that, in the ´case ‘function’:´ above, ´result.value´ can be a promise BUT that won’t make ´result´ a promise itself!! So the code above wouldn’t work yet, because it would complete ´Promise.all´ before getting the hashed password. Finally, I got this:

// CRUD.controller.js (working excerpt)

module.exports = CRUD_Controller; 
 
function CRUD_Controller(Item, fields) { 
    //... 
    function Create(req, res) {

        FilterFields(fields, req.body, function (fFields) {
            var item = new Item();

            fFields.forEach(function (fField) {
                item[fField.name] = fField.value;
            });

            item.save(function(err) {

                if (err) {
                    return res.send(err);
                }

                res.json({
                    message: 'Item created!'
                });

            });
        });

    }


    function FilterFields(fields, data, callback) {

        Promise
            .all((fields || []).map(Filter))
            .then(callback)
            .catch(function (error) {
                console.log(error);
            });


        function Filter(field) {
            var result;

            switch (typeof field) {

                case 'string':
                    result = {
                        name: field,
                        value: data[field]
                    };
                    break;

                case 'function':
                    var matches = String(field).match(/^functions*(s*(w+)s*)/);
                    if (!(matches && matches[1])) {
                        console.log('Expected a function with only one argument.');
                        return;
                    }
                    result = {
                        name: matches[1],
                        value: field(data[matches[1]])
                    };
                    if (stuff.isPromise(result.value)) {
                        var promise = new Promise(function (resolve, reject) {
                            var name = result.name;
                            result.value.then(function (value) {
                                resolve({
                                    name: name,
                                    value: value
                                });
                            }).catch(function (error) {
                                reject(Error(error));
                            });
                        });
                        result = promise;
                    }
                    break;

            }

            return result;
        }

    }
    //...
}

The added lines make ´result´ a promise if ´result.value´ is one: ´result´ will eventually resolve to the expected result. BTW, the ´stuff.isPromise´ method is the classical ´object.then && typeof object.then == ‘function’´.