Beware here may be dragons: this is document is in draft and very much liable to change. Don't take anything seriously here.
Click to hide this warning.

What is Refactoring?

Refactoring is an daily activity for programmers but something that outside of the tech scene seems to be called “reorganising”, “cleaning up” or “putting stuff away”.

Following feedback on the Node-RED forum on my previous post on refactoring in Node-RED, I thought I might explain the term refactoring.

Wikipedia has this to say:

In computer programming and software design, code refactoring is the process of restructuring existing computer code—changing the factoring—without changing its external behavior. Refactoring is intended to improve the design, structure, and/or implementation of the software (its non-functional attributes), while preserving its functionality.

OK when it comes down to changing the structure but not the functionality, but for what benefit? Second sentence at Wikipedia:

Potential advantages of refactoring may include improved code readability and reduced complexity; these can improve the source code’s maintainability and create a simpler, cleaner, or more expressive internal architecture or object model to improve extensibility. Another potential goal for refactoring is improved performance; software engineers face an ongoing challenge to write programs that perform faster or use less memory.

So the intention behind refactoring is the long term maintainability of code. Code is nothing if it becomes incomprehensible with age. Code that never changes does not suffer from this however code is ever changing, being updated, extended and replaced.

How does this apply to Node-RED which is a visual programming environment? After all, what is structure in a visual coding environment? What is maintainability in a visual coding environment? Not to mention understandability and comprehension of code.

Spaghetti

One obvious example of understandability is spaghetti code which visually in Node-RED literally looks as if it were spaghetti:

[{"id":"e3e278445b7bcb21","type":"subflow","name":"BlogPageEndsHere","info":"","category":"","in":[{"x":132,"y":215,"wires":[{"id":"b2f270b15349b67c"}]}],"out":[],"env":[],"meta":{},"color":"#ddeeff","icon":"font-awesome/fa-hand-stop-o"},{"id":"627f98bbc0ea4f8b","type":"link out","z":"e3e278445b7bcb21","name":"link out 77","mode":"return","links":[],"x":1328,"y":306,"wires":[]},{"id":"408189fb7a17ba8b","type":"switch","z":"e3e278445b7bcb21","name":"has _linkSource - is this a link-call action","property":"_linkSource","propertyType":"msg","rules":[{"t":"nnull"},{"t":"else"}],"checkall":"true","repair":false,"outputs":2,"x":1002,"y":403,"wires":[["627f98bbc0ea4f8b"],[]]},{"id":"2a2cc4ef4e7704b4","type":"template","z":"e3e278445b7bcb21","name":"","field":"payload","fieldType":"msg","format":"handlebars","syntax":"mustache","template":"","output":"str","x":545,"y":632,"wires":[["408189fb7a17ba8b"]]},{"id":"693c46a206c026a4","type":"change","z":"e3e278445b7bcb21","name":"","rules":[{"t":"set","p":"template","pt":"msg","to":"payload","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":474,"y":510,"wires":[["2a2cc4ef4e7704b4"]]},{"id":"b2f270b15349b67c","type":"switch","z":"e3e278445b7bcb21","name":"","property":"page.noderedjson","propertyType":"msg","rules":[{"t":"true"},{"t":"else"}],"checkall":"false","repair":false,"outputs":2,"x":374,"y":408,"wires":[["408189fb7a17ba8b"],["693c46a206c026a4"]]},{"id":"82a04eeec5d1ec39","type":"link in","z":"04fab8edd0e8f2df","name":"[blog] backticks-in-markdown-and-node-red","links":[],"x":434,"y":292,"wires":[["cc439de6514717e7"]]},{"id":"cc439de6514717e7","type":"BlogPageInfo","z":"04fab8edd0e8f2df","name":"","image":"1687952972847_Screen_Shot_2023-06-28_at_13.48.43.png","summary":"Codeblocks are one thing but images are further a thousand Json files. Show me your flow!","title":"Replacing Node-RED flow codeblocks with flow images.","publishedAt":"2023-06-28T11:22","updatedAt":"2023-06-28T11:22","incRss":true,"incSitemap":true,"incIndex":true,"shrLinkedIn":true,"supportNodeRedJson":true,"reloadOnEdit":false,"redirectToNode":false,"redirectNodeId":"status-404","redirectStatusCode":"301","x":1075.71435546875,"y":448,"wires":[["be61a774e4c6d75e"]]},{"id":"be61a774e4c6d75e","type":"template","z":"04fab8edd0e8f2df","name":"flow example 1","field":"flowexample1","fieldType":"msg","format":"handlebars","syntax":"plain","template":"","output":"str","x":998,"y":563,"wires":[["db01f2dab4fa12ab"]]},{"id":"ac8306319a302f0c","type":"BlogPageInfo","z":"04fab8edd0e8f2df","name":"","image":"","summary":"Can I replace codeblocks of flows.json with images?","title":"Testing the inclusion and replacement of codeblocks","publishedAt":"2023-06-25T11:22","updatedAt":"2023-06-25T11:22","incRss":false,"incSitemap":false,"incIndex":false,"shrLinkedIn":false,"supportNodeRedJson":true,"reloadOnEdit":false,"redirectToNode":false,"redirectNodeId":"status-404","redirectStatusCode":"301","x":1077.71435546875,"y":544,"wires":[["be61a774e4c6d75e"]]},{"id":"db01f2dab4fa12ab","type":"template","z":"04fab8edd0e8f2df","name":"flowexample2","field":"flowexample2","fieldType":"msg","format":"handlebars","syntax":"plain","template":"","output":"str","x":991,"y":467,"wires":[["440887f5bfa191c9"]]},{"id":"24767f8be515eebe","type":"link in","z":"04fab8edd0e8f2df","name":"[blog] codeblock-test","links":[],"x":432,"y":336,"wires":[["ac8306319a302f0c"]]},{"id":"440887f5bfa191c9","type":"template","z":"04fab8edd0e8f2df","name":"flowexample3","field":"flowexample3","fieldType":"msg","format":"handlebars","syntax":"plain","template":"","output":"str","x":1218,"y":534,"wires":[["03989336aea86433"]]},{"id":"03989336aea86433","type":"template","z":"04fab8edd0e8f2df","name":"flowexample4","field":"flowexample4","fieldType":"msg","format":"handlebars","syntax":"plain","template":"","output":"str","x":1151,"y":491,"wires":[["01dae03d9ec9694f"]]},{"id":"01dae03d9ec9694f","type":"template","z":"04fab8edd0e8f2df","name":"flowexample5","field":"flowexample5","fieldType":"msg","format":"handlebars","syntax":"plain","template":"","output":"str","x":1043,"y":507,"wires":[["33bfba9a9fcd9e58"]]},{"id":"33bfba9a9fcd9e58","type":"template","z":"04fab8edd0e8f2df","name":"flowexample6","field":"flowexample6","fieldType":"msg","format":"handlebars","syntax":"plain","template":"","output":"str","x":1205,"y":472,"wires":[["930cf2c0b7cbc504"]]},{"id":"930cf2c0b7cbc504","type":"template","z":"04fab8edd0e8f2df","name":".md page","field":"payload","fieldType":"msg","format":"markdown","syntax":"mustache","template":"","output":"str","x":1158,"y":550,"wires":[["e6cc075cbd12d4ef"]]},{"id":"e6cc075cbd12d4ef","type":"subflow:e3e278445b7bcb21","z":"04fab8edd0e8f2df","name":"","x":1645,"y":425,"wires":[]}]

This is working and usable code in Node-RED. However I prefer to have a visual overview of what is happening:

[{"id":"41ce544c739b990a","type":"subflow","name":"BlogPageEndsHere (2)","info":"","category":"","in":[{"x":132,"y":215,"wires":[{"id":"64ee16a4e734dd95"}]}],"out":[],"env":[],"meta":{},"color":"#ddeeff","icon":"font-awesome/fa-hand-stop-o"},{"id":"30aab5237a19a97b","type":"link out","z":"41ce544c739b990a","name":"link out 77","mode":"return","links":[],"x":1328,"y":306,"wires":[]},{"id":"d51777c344dbbbbf","type":"switch","z":"41ce544c739b990a","name":"has _linkSource - is this a link-call action","property":"_linkSource","propertyType":"msg","rules":[{"t":"nnull"},{"t":"else"}],"checkall":"true","repair":false,"outputs":2,"x":1002,"y":403,"wires":[["30aab5237a19a97b"],[]]},{"id":"8095b14a17e44eeb","type":"template","z":"41ce544c739b990a","name":"","field":"payload","fieldType":"msg","format":"handlebars","syntax":"mustache","template":"","output":"str","x":545,"y":632,"wires":[["d51777c344dbbbbf"]]},{"id":"6f3076ba44da119e","type":"change","z":"41ce544c739b990a","name":"","rules":[{"t":"set","p":"template","pt":"msg","to":"payload","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":474,"y":510,"wires":[["8095b14a17e44eeb"]]},{"id":"64ee16a4e734dd95","type":"switch","z":"41ce544c739b990a","name":"","property":"page.noderedjson","propertyType":"msg","rules":[{"t":"true"},{"t":"else"}],"checkall":"false","repair":false,"outputs":2,"x":374,"y":408,"wires":[["d51777c344dbbbbf"],["6f3076ba44da119e"]]},{"id":"46eb49536e45e3e8","type":"template","z":"9c4c290bdd06d11d","name":"flowexample3","field":"flowexample3","fieldType":"msg","format":"handlebars","syntax":"plain","template":"","output":"str","x":1231,"y":1038,"wires":[["68249116d8b18bed"]]},{"id":"75b55d3d6ea925ca","type":"template","z":"9c4c290bdd06d11d","name":"flowexample2","field":"flowexample2","fieldType":"msg","format":"handlebars","syntax":"plain","template":"","output":"str","x":1220,"y":901,"wires":[["46eb49536e45e3e8"]]},{"id":"68249116d8b18bed","type":"template","z":"9c4c290bdd06d11d","name":"flowexample4","field":"flowexample4","fieldType":"msg","format":"handlebars","syntax":"plain","template":"","output":"str","x":1458,"y":1038,"wires":[["a4d2ab20fa4c9e38"]]},{"id":"22d0334769763d0d","type":"template","z":"9c4c290bdd06d11d","name":"flow example 1","field":"flowexample1","fieldType":"msg","format":"handlebars","syntax":"plain","template":"","output":"str","x":970,"y":1032,"wires":[["75b55d3d6ea925ca"]]},{"id":"a4d2ab20fa4c9e38","type":"template","z":"9c4c290bdd06d11d","name":"flowexample5","field":"flowexample5","fieldType":"msg","format":"handlebars","syntax":"plain","template":"","output":"str","x":1471,"y":902,"wires":[["9c439e80641959bf"]]},{"id":"b95256b2e4fe55c5","type":"BlogPageInfo","z":"9c4c290bdd06d11d","name":"","image":"1687952972847_Screen_Shot_2023-06-28_at_13.48.43.png","summary":"Codeblocks are one thing but images are further a thousand Json files. Show me your flow!","title":"Replacing Node-RED flow codeblocks with flow images.","publishedAt":"2023-06-28T11:22","incRss":true,"incSitemap":true,"incIndex":true,"shrLinkedIn":true,"supportNodeRedJson":true,"reloadOnEdit":false,"redirectToNode":false,"redirectNodeId":"status-404","redirectStatusCode":"301","x":821.71435546875,"y":904,"wires":[["22d0334769763d0d"]]},{"id":"f257dac6ff537599","type":"BlogPageInfo","z":"9c4c290bdd06d11d","name":"","image":"","summary":"Can I replace codeblocks of flows.json with images?","title":"Testing the inclusion and replacement of codeblocks","publishedAt":"2023-06-25T11:22","incRss":false,"incSitemap":false,"incIndex":false,"shrLinkedIn":false,"supportNodeRedJson":true,"reloadOnEdit":false,"redirectToNode":false,"redirectNodeId":"status-404","redirectStatusCode":"301","x":700.71435546875,"y":1032,"wires":[["22d0334769763d0d"]]},{"id":"9c439e80641959bf","type":"template","z":"9c4c290bdd06d11d","name":"flowexample6","field":"flowexample6","fieldType":"msg","format":"handlebars","syntax":"plain","template":"","output":"str","x":1742,"y":903,"wires":[["00729a3329108bd0"]]},{"id":"e0b52515201734da","type":"link in","z":"9c4c290bdd06d11d","name":"[blog] backticks-in-markdown-and-node-red","links":[],"x":521,"y":904,"wires":[["b95256b2e4fe55c5"]]},{"id":"92ed796c6686af23","type":"link in","z":"9c4c290bdd06d11d","name":"[blog] codeblock-test","links":[],"x":510,"y":1032,"wires":[["f257dac6ff537599"]]},{"id":"00729a3329108bd0","type":"template","z":"9c4c290bdd06d11d","name":".md page","field":"payload","fieldType":"msg","format":"markdown","syntax":"mustache","template":"","output":"str","x":1751,"y":1039,"wires":[["9641fb49345b7551"]]},{"id":"9641fb49345b7551","type":"subflow:41ce544c739b990a","z":"9c4c290bdd06d11d","name":"","x":2011,"y":1039,"wires":[]}]

This is a form of visual refactoring: making a flow visually understandable.

Both of these flows are working flows and Node-RED can work with both. Herein lies a further characteristic of refactoring: it requires discipline, a developer is not forced to refactor since the code works. The computer does not care whether code is human understandable or not, computer only cares whether the code is syntactically correct.

Dotting the 'i’s and crossing the 't’s

At the risk of starting a debate on what can be considered aesthetically pleasing in terms of flow and structure, some pedantic behaviour I find myself doing when I begin my Node-RED day. If I see this flow, I get suspicions:

[{"id":"e5b13199afb96185","type":"switch","z":"9c4c290bdd06d11d","name":"","property":"req.originalUrl","propertyType":"msg","rules":[{"t":"regex","v":"v12\\.html$","vt":"str","case":false},{"t":"regex","v":"v11\\.html$","vt":"str","case":false},{"t":"else"}],"checkall":"false","repair":false,"outputs":3,"x":585,"y":972.2857666015625,"wires":[["f04560239a98d725"],["c0f63c3f60dcd469"],["73958e1655079ca6"]]},{"id":"f04560239a98d725","type":"template","z":"9c4c290bdd06d11d","name":".md page","field":"payload","fieldType":"msg","format":"markdown","syntax":"mustache","template":"","output":"str","x":772,"y":944.2857666015625,"wires":[["7e9fe38682eef18d"]]},{"id":"c0f63c3f60dcd469","type":"template","z":"9c4c290bdd06d11d","name":".md page","field":"payload","fieldType":"msg","format":"markdown","syntax":"mustache","template":"","output":"str","x":773,"y":1022.2857666015625,"wires":[["11ca20053907641d"]]},{"id":"73958e1655079ca6","type":"link out","z":"9c4c290bdd06d11d","name":"link out 83","mode":"return","links":[],"x":1458.4287109375,"y":1180,"wires":[]},{"id":"7e9fe38682eef18d","type":"markdown","z":"9c4c290bdd06d11d","name":"","x":924,"y":891.2857666015625,"wires":[["f32fbb3a3e3ed75d"]]},{"id":"11ca20053907641d","type":"markdown","z":"9c4c290bdd06d11d","name":"","x":937,"y":973.2857666015625,"wires":[["daae7204d83735f5"]]},{"id":"206bf60d60867dfd","type":"switch","z":"9c4c290bdd06d11d","name":"has _linkSource - is this a link-call action","property":"_linkSource","propertyType":"msg","rules":[{"t":"nnull"},{"t":"else"}],"checkall":"true","repair":false,"outputs":2,"x":1240.285888671875,"y":1110.4285888671875,"wires":[["73958e1655079ca6"],[]]},{"id":"f32fbb3a3e3ed75d","type":"template","z":"9c4c290bdd06d11d","name":"v12","field":"payload","fieldType":"msg","format":"javascript","syntax":"mustache","template":"","output":"str","x":1103,"y":937.2857666015625,"wires":[["206bf60d60867dfd"]]},{"id":"daae7204d83735f5","type":"template","z":"9c4c290bdd06d11d","name":"v11","field":"payload","fieldType":"msg","format":"javascript","syntax":"mustache","template":"","output":"str","x":1094,"y":1005.2857666015625,"wires":[["206bf60d60867dfd"]]}]

What I don’t like is that nodes are covering wires, for me this is a sign of code smell. For example, the lower template node labelled .md page, by being above the wire that connects the switch node and the link-out node, seems to be connected to that wire. It’s confusing and hinders understanding. Additionally the second switch is also covering the wire. Again, confusing for me.

So I would do an initial clean up:

[{"id":"e5b13199afb96185","type":"switch","z":"9c4c290bdd06d11d","name":"","property":"req.originalUrl","propertyType":"msg","rules":[{"t":"regex","v":"v12\\.html$","vt":"str","case":false},{"t":"regex","v":"v11\\.html$","vt":"str","case":false},{"t":"else"}],"checkall":"false","repair":false,"outputs":3,"x":585,"y":972.2857666015625,"wires":[["f04560239a98d725"],["c0f63c3f60dcd469"],["73958e1655079ca6"]]},{"id":"f04560239a98d725","type":"template","z":"9c4c290bdd06d11d","name":".md page","field":"payload","fieldType":"msg","format":"markdown","syntax":"mustache","template":"","output":"str","x":750,"y":892.2857666015625,"wires":[["7e9fe38682eef18d"]]},{"id":"c0f63c3f60dcd469","type":"template","z":"9c4c290bdd06d11d","name":".md page","field":"payload","fieldType":"msg","format":"markdown","syntax":"mustache","template":"","output":"str","x":749,"y":953.2857666015625,"wires":[["11ca20053907641d"]]},{"id":"73958e1655079ca6","type":"link out","z":"9c4c290bdd06d11d","name":"link out 83","mode":"return","links":[],"x":1458.4287109375,"y":1180,"wires":[]},{"id":"7e9fe38682eef18d","type":"markdown","z":"9c4c290bdd06d11d","name":"","x":924,"y":891.2857666015625,"wires":[["f32fbb3a3e3ed75d"]]},{"id":"11ca20053907641d","type":"markdown","z":"9c4c290bdd06d11d","name":"","x":937,"y":973.2857666015625,"wires":[["daae7204d83735f5"]]},{"id":"206bf60d60867dfd","type":"switch","z":"9c4c290bdd06d11d","name":"has _linkSource - is this a link-call action","property":"_linkSource","propertyType":"msg","rules":[{"t":"nnull"},{"t":"else"}],"checkall":"true","repair":false,"outputs":2,"x":1409.285888671875,"y":993.4285888671875,"wires":[["73958e1655079ca6"],[]]},{"id":"f32fbb3a3e3ed75d","type":"template","z":"9c4c290bdd06d11d","name":"v12","field":"payload","fieldType":"msg","format":"javascript","syntax":"mustache","template":"","output":"str","x":1103,"y":937.2857666015625,"wires":[["206bf60d60867dfd"]]},{"id":"daae7204d83735f5","type":"template","z":"9c4c290bdd06d11d","name":"v11","field":"payload","fieldType":"msg","format":"javascript","syntax":"mustache","template":"","output":"str","x":1094,"y":1005.2857666015625,"wires":[["206bf60d60867dfd"]]}]

Now all the connections wires are not covered by nodes. But for me, it’s still too curvy and wavy, call me pedantic but I like straight wires:

[{"id":"e5b13199afb96185","type":"switch","z":"9c4c290bdd06d11d","name":"","property":"req.originalUrl","propertyType":"msg","rules":[{"t":"regex","v":"v12\\.html$","vt":"str","case":false},{"t":"regex","v":"v11\\.html$","vt":"str","case":false},{"t":"else"}],"checkall":"false","repair":false,"outputs":3,"x":542,"y":1019.5,"wires":[["f04560239a98d725"],["c0f63c3f60dcd469"],["73958e1655079ca6"]]},{"id":"f04560239a98d725","type":"template","z":"9c4c290bdd06d11d","name":".md page","field":"payload","fieldType":"msg","format":"markdown","syntax":"mustache","template":"","output":"str","x":750,"y":922.2857666015625,"wires":[["7e9fe38682eef18d"]]},{"id":"c0f63c3f60dcd469","type":"template","z":"9c4c290bdd06d11d","name":".md page","field":"payload","fieldType":"msg","format":"markdown","syntax":"mustache","template":"","output":"str","x":749,"y":980.857177734375,"wires":[["11ca20053907641d"]]},{"id":"73958e1655079ca6","type":"link out","z":"9c4c290bdd06d11d","name":"link out 83","mode":"return","links":[],"x":1738.4287109375,"y":1027,"wires":[]},{"id":"7e9fe38682eef18d","type":"markdown","z":"9c4c290bdd06d11d","name":"","x":924,"y":922.2857666015625,"wires":[["f32fbb3a3e3ed75d"]]},{"id":"11ca20053907641d","type":"markdown","z":"9c4c290bdd06d11d","name":"","x":937,"y":980.857177734375,"wires":[["daae7204d83735f5"]]},{"id":"206bf60d60867dfd","type":"switch","z":"9c4c290bdd06d11d","name":"has _linkSource - is this a link-call action","property":"_linkSource","propertyType":"msg","rules":[{"t":"nnull"},{"t":"else"}],"checkall":"true","repair":false,"outputs":2,"x":1436.285888671875,"y":980.857177734375,"wires":[["73958e1655079ca6"],[]]},{"id":"f32fbb3a3e3ed75d","type":"template","z":"9c4c290bdd06d11d","name":"v12","field":"payload","fieldType":"msg","format":"javascript","syntax":"mustache","template":"","output":"str","x":1089,"y":922.2857666015625,"wires":[["206bf60d60867dfd"]]},{"id":"daae7204d83735f5","type":"template","z":"9c4c290bdd06d11d","name":"v11","field":"payload","fieldType":"msg","format":"javascript","syntax":"mustache","template":"","output":"str","x":1094,"y":980.857177734375,"wires":[["206bf60d60867dfd"]]}]

Now I can understand the flow! Straight wires for me define the main flow of data, the main use case or everything goes wright wire. I don’t have a hard or fast rule for this notion but visually straight lines make understanding simpler.

Node-RED provides a collection of alignment shortcuts that all begin with alt(⌥)-a (on a mac keyboard):

⌥-a l - left align
⌥-a c - center align
⌥-a r - right align

⌥-a t - top align
⌥-a m - middle align
⌥-a b - bottom align

All possible arrangement shortcuts can be found in the arrange sub-menu.

img

Perhaps one day there will be style guides for Node-RED similar to those style guides for text-based coding. I can already imagine the discussions whether one should align top or bottom … left or right … - reminiscent of space v. tab, 2 v. 4 spaces indentation, 80 v 120 characters per line discussions.

Dry, Kiss, Fail Fast

Refactorings have guidelines, just as putting things away have rules about where things should go. Refactoring is based on principles that have been developed over the years and that aim to improve understandability. I will not list them all, just a couple to give an impression.

First principle is Don’t Repeat Yourself - DRY which is simply that if the same code appears twice, then put it into an abstraction, i.e., function or method, and call the abstraction in place of repeating the code. This can then lead to more drying since suddenly one recognises that there are other similar bits of code. The abstraction can be extended, for example by adding parameters. This makes these abstractions more general and therefore more applicable to other similar code blocks.

A second main principle is Keep It Simple, Stupid Soldier Sailor Silly! - KISS which is harder to nail down since complexity is a very personal thing. For some, that which is complex is very simple. Complexity also has much to do with context: do I understand the context in which the code is being executed? If I don’t, then the complexity is increased. Alternative is keep it short and simple however short can mean many layers of abstraction, containing their own confusion. In the end, your mileage might well vary.

A third principle I like is the fail fast, fail early principle. This basically is the notion that a system should fail as soon as possible, that is do not do any further processing if failure is inevitable. This can lead to doing certain external activities before internal activities since external dependencies are more likely to failure (due to networks, systems and general unpredictability of systems that are out of ones direct control).

There are many more and everyone has their own favourites. I will now try to illustrate these using Node-RED.

Example 1: DRY

To demonstrate the DRY principle, consider the following contrived example (for brevity assume a has been initialised to some value, assume 10):

b = a + 1;
c = b + 1;
d = c + 2;

The result would be that a is 10, b is 11, c is 12 and d is 14. Refactoring this we could define a increment function:

funct increment(v) {
    return v + 1
}
b = increment(a);
c = increment(b);
d = c + 2;

Having refactored we notice that c + 2 is very similar to increment(..) only it is adding 2, so we can parameterise increment:

funct increment(v,by) {
    return v + by;
}
b = increment(a,1);
c = increment(b,1);
d = increment(c,2);

Now we realise that increment is badly named since increment generally means increasing a value by 1. Renaming the abstractions, i.e. functions or methods, is also a form of refactoring to increase understandability.

We rename the function to add:

funct add(v,by) {
    return v + by;
}
b = add(a,1);
c = add(b,1);
d = add(c,2);

At the end of all this, our values are still the same: a is 10, b is 11, c is 12 and d is 14.

DRYing in Node-RED

Now let us see what the equivalent would be in Node-RED. This is the flow I came up with:

To those unfamiliar with Node-RED, let me explain the flow above.

All flows in Node-RED “flow” from left to right, that is the input is on the left hand side of a node and its output is on the right. What are “nodes”? Nodes are bits of computation, i.e. the code. Nodes are the colourful rectangles connected by the grey lines.

What flows through a “flow”? A data object, i.e., a data structure with named attributes. This data object is modified by nodes. As the msg[1] flows through the flow, it is altered and extended by individual nodes. Each node has complete read-write access to the data object.

Each flow is started by an initial data object, in the case of the flow above, the inject node labelled inject empty msg does exactly that - it pushes and empty msg object into the flow. What is not shown on the diagram is the button for doing that.[2]

The change node following the inject node modifies the data object setting msg.a to 10, i.e., it sets the attribute a to the value of ten. The function nodes that follow execute Javascript code in the context of the Node-RED environment. Each function is given a copy of the data object to do whatever it likes with the data object.

At the end of the flow is a debug node displays the contents of the data object in the debug panel within the Node-RED editor. The result in this case is:

final result

The second step involves the definition of the increment abstraction. For Node-RED that abstraction is based on function nodes:

Not shown are the debug and inject nodes, for brevity I have removed them. This flow also contains two change nodes that set separate attributes on the msg object. Node-RED has a convention that nodes act on the attribute payload on the msg object. This means the value that we want to increment (i.e. a), is first set to the payload attribute on the msg object. Then the increment node adds one to the payload which then also becomes its output.

After the function has done its magic, we take the value of payload and set msg.b with to that value. This is the other convention: nodes update the payload attribute to their final result value.

The increment node appears twice and in fact its a duplication of the same code. So I have actually duplicate the same code - not exactly refactoring!

Here we come to one of the core features of Node-RED: nodes are stateless, there is no knowledge of what happens before a node receives it data and no knowledge of what happens after a node has completed. Nodes should always take their state information from the msg object on which it acts.

This flow is probably the most obvious thing I would do to reuse the increment node. This would cause a major outage of the Node-RED server. It actually produces an endless loop. So what happens here?

The first message comes in to the increment function node and it returns its output. Node-RED sends a message to the set msg.c function node and also to the set msg.b function node. The set msg.b node sets msg.b to the payload value and then sends out its message. This message goes to the increment node - the intention is to set the value of msg.c. The increment node does its computation and sends out its message. Node-RED sends this message to the set msg.c node and also to the set msg.b node. And the cycle repeats.

Node-RED nodes are completely stateless and that is by design so. Nodes do not know to which other nodes they are connected to. All they do is a) receive a message b) return a value. Each node reacts only to the contents of the message, i.e., the data object flowing through the flow. Node-RED is responsible for passing messages to other nodes, the nodes do not do this themselves.

Since Node-RED does the message handling, logic in the increment node that would say if msg.b is set then do not send message to 'set msg.b' node cannot be done. There is nothing that the increment node can do to prevent this infinite loop.

(This is not strictly true but the intention is clear: nodes should have the same behaviour for the same message, they should be predictable. Flows can have state and nodes can react accordingly, but the details are something for another blog post.)

The lesson here: if you see a figure 8 anywhere in your flows, run and hide, only dragons live where there are eights in flows.

Back to the flow, the final step is the definition of the add abstraction, that is this flow:

What is going on? It’s getting worse! The flow has gotten longer. The add node is duplicated three times and there is a further attribute msg.by. The msg.by is required since the add function requires two arguments and add is duplicate three times since we do three additions.

For clarity, this is the Javascript code inside the add function node:

msg.payload = msg.payload + msg.by;

return msg;

Here we see that the node only computes the addition by using msg.by and assuming the msg.payload contains the value to add to. The message is then returned. The altered data object is then passed to all connections of the add node at that position in the flow. This is the same code for all three add nodes.

But our refactoring has not turned out too well, we have only duplicated the code. How can we do better? In a previous post I described Subflows and how they could be used to solve code duplication. Here I will use link in/out/call nodes.

What are link nodes and how do link nodes help to reduce code duplication?

Link in & Link out Nodes

Link nodes come in pairs, a link-out node connects to a corresponding link-in node. As all nodes, they are directional with data flowing from the link-out node to the link-in node. Their naming comes out of the perspective of the data object: at the link-out node the data exits an existing flow to enter a new flow at the link-in node. Think of them as creating tunnels between two points across the Node-RED flows. Via these tunnels, data objects flow between the link nodes. The beginning of the tunnel is the link-out node, the exit of the tunnel is the link-in node.

The following flow has a link-out node to the left and a link in node to the right. When the inject node pushes data into the flow, the two debug nodes display the same data. That is because the inject nodes’ data flows to both the link-out node and the debug 1 node. The link-out node is connected to the link-in node and the same data flows out of the link-in node to the debug 2 node.

The dashed connection is the tunnel along which the data objects flow. The link between the nodes remains regardless where they are located. Also whatever happens after the data object enters the new flow via the link-in node, is of no concern to the link-out node that sent the data. There is no inverse coupling, data flows from link-out to link-in but the results do not flow back. A link-out node has no output connector to make this clear.

Link-call node

The link-call node breaks this paradigm by having an output connector. A link-call node is used in conjunction with link-in and link-out nodes. When data flows into a link-call node, the data is passed to the corresponding link-in node to which the link-call node has been coupled.

The following flow demonstrates the use of a link-call node.

What is happening is that when the data object generated and injected into the link-call node, the data object is passed to the link in node. The link-call node has been coupled with the link-in node - done on the Node-RED editor. From there, the data object flows to the add function node which does a the following computation on the data object:

msg.payload = msg.payload + msg.by;

return msg;

Results of the computation is passed to the link out node. The link-out node passes that result data object back to the link call node. How does the link-out node know to do this? Because the link-out node is set to return mode - shown by its icon. This setting needs to be made so that a link-call node receives the data object from the the link-out node. Node properties can be set within the Node-RED editor.

Finally the link-call node passes the data object to the debug node which then displays the result in the debug console within the Node-RED editor:

30/27/3023, 25:12:07  node: debug
msg.payload : number
11

What happens when …?

The add function node is a abstraction of the addition computation. As such, using the link-call node, we can use this computation anywhere its required. Sometimes though it becomes tempting to use the functionality directly. The following flow is a simple extension of the one above however it demonstrates the direct use of the add function node.

I have added two extra links: one between inject and add and one between add and a new debug node:

When the inject node is triggered, the flow now has four outcomes:

But how does this fit together with the refactoring of the add flow above? The end result is the following flow:

We end up calling the add function node three times via the link-call nodes. If we changed the code in the add function, this would be reflected all three calls to the function via the link-call nodes.

The link-out node is in return mode and returns the result of the add function node to the respective link-call node. The link-out “knows” which link-call node called the link-in node.

With that, the add functionality is isolated in the add function node and can be modified there. Any modification is then reused in three separate places.

Last updated: 2023-08-28T13:04:56.471Z

  1. The naming convention of Node-RED is to call the data object msg. ↩︎

  2. The flow images shown here are not 100% representational of the Node-RED editor. ↩︎


Comments powered by giscus