Quoting Dave - @dceejay:
I have often thought of and described Node-RED to non-programmers as “digital duct tape” - It was always conceived as a way to help make things work and fix things, IE a practical engineering tool rather than a logically rigid programming language. As such there will always need to be adaptations to fit different mindsets, and compromises to allow for simplifications for more general usage across many different application areas.
– @ Slack
Duct taping is an underselling of Node-RED, IMHO.
I have the impression that many folks in the community consider Node-RED as a duct-taping solution provider. A programming environment used to create solutions that are one-hit wonders: quick solutions for some data routing or some data manipulation task, never to be touched again. Node-RED is then used as a one-shot solution, leave it running, and don’t touch it again.
Node-RED is not considered to be a long-term solution designed for application creation or company-core code definition. It is considered to be sort of a glue code between applications, between individual parts of the entire system. I think this is underselling the abilities and features of Node-RED.
I think Node-RED, especially with the recently released version 5, has great potential to be a development environment for “serious” long-term application creation and maintenance.[1]
I say that because with the new release, Node-RED version 5, much has been done to make the editor UI/UX experience far better for developers who create long-term solutions. With version 5, dependency management of flows has been introduced. Sidebars and plugins are more present in the editor, allowing sidebars to provide better support for long-term development and maintenance of code.
It’s another undersell to interpret Node-RED as just being an IoT solution or a home automation solution. I think there’s far more that Node-RED can do if one looks closely. It is just a matter of altering the duct-tape mindset.
Advantage of Node-RED is that it takes a different approach to programming. It doesn’t take a functional approach, a strictly event-based approach, nor an object-oriented approach, nor a list approach. It actually takes a flow-based programming approach, which is a completely different paradigm of programming.
Briefly, flow-based programming sets the focus on data and not computation. Functional programming is focused on functions, thereby focusing on computation. Object-oriented programming combines data and computation and is focused on encapsulation. Flow-based programming is based on data flows, describing where data flows, the computation is done on well defined data messages.
For the perfectionists, true flow-based programming has no context and no state; it’s a completely stateless paradigm. Node-RED bends that. That’s why Node-RED is inspired flow-based programming. Arguably this bending of the rules makes it simpler to engage in flow-based programming.
For the argument’s sake, it makes no difference if it’s inspired or not. The thing is, visually, it looks like flow-based programming as there is a clear focus on data flows; developers design the flow of data between computational units.
That has a set of benefits that aren’t clear just by looking at the flows. Two benefits that are dependency management and concurrency.
If you have libraries that provide the functionality of nodes, those libraries are used only by those nodes. That means if you have supply chain problems with certain packages/libraries being corrupted, there’s a possibility of isolating those problems by deactivating the nodes that depend on those library packages. Once you do that, you can replace them with other nodes that use different libraries so that you can combat supply chain problems without having to rewrite your entire application or deactivate your entire application. That comes for free in flow-based programming because of the isolation between nodes. Nodes don’t depend on each other; they are independent of one another, which means all the dependencies are also independent one another.
True, two or more nodes could be using the same library and might need more than one node need to get deactivated if that library gets compromised. However, there is no dependency of nodes amongst each other.
The other good thing about flow-based programming is because of this computational isolation, you can easily create parallelisation of code. Because you’re just sending data to a computation, that computation could be running anywhere. All that computation needs, because it’s stateless, is the data. You send the data to that computation; it performs its operations and sends back an answer, which you then pass on to further computation downstream. It’s quite easy to parallelise flow-based programs. That comes for free.
The flow-based programming paradigm is perfectly represented by Node-RED development environment. You can take a different approach to application development. You take a completely different approach to it and explore how your application would benefit from a flow-based programming approach. You can prototype that and try it out with Node-RED to see if that works for you.
What is still missing in Node-RED built-in is proper visual version control and visual change management of code. What I think is necessary is no less than a complete visual representation of the functionality provided by Git. This is what is required for long term application maintenance and development.
Node-RED debug panel provides a great mechanism for introspecting messages as they pass through flows. Making it simple to understand issues when they arise.
What is missing here is a dynamic view of what is going. Using a debug node that is not deployed does not work. Redeploying the flow might reset the error being observed. What to do? Add debug node and hope the error occurs again.
A dynamic debugger would allow introspection of the running system without redeploying the flow and within the context of the editor. This might well be possible by attaching a text-based debugger to the running NodeJS process but that is beyond the core users of Node-RED. Therefore a visual, editor integrated solution should be provided.
Node-RED is the top-of-class tool for visual inspired flow-based programming (FBP). It is completely open source and provides a good foundation for exploring a new programming paradigm. A paradigm that, itself, has been undersold over the years[2].
With version 5 of Node-RED, a major step has been made for bring FBP to a larger audience and one that should be acknowledged by the Node-RED community. Referring to Node-RED as a duct-tape solution is underselling it and also the FBP paradigm which can provide genuine advantage in the development of large scale applications.
I have been blogging about Node-RED for a while and some of the ideas that I have tossing around still have relevance:
Breadboard Programming which I came up when talking about prototyping in a visual programming environment. Sometimes it is useful to draw parallels to concepts that might not be directly related but that provide bridges. In this case breadboarding, which is well known amongst folks that work with electronics, could be applied to programming.
The Erlang-Red project is a demonstration of how to implement true parallelisation and concurrency using FBP. As such, it is an experiment with a language, Erlang, that supports message passing, immutable data structures and independent processing out-of-the-box.
I have also explored the missing ideas presented here by implementing FlowHub.org and creating the dynamic debugger. Both are prototypes and explore the possibilities of extending Node-RED beyond the duct-tape stage!
Perhaps some would say that they are building long term solution using Node-RED however how are these solutions maintained and how often do they change? I am talking about daily changes to complicate flow-based code and not long-running flows that change rarely. ↩︎
See https://jpaulm.github.io/fbp/index.html for a history of FBP. ↩︎