Coding in the Browser - Node-RED v. Jupyter

Coding in the browser - Node-RED and Jupyter, a comparison

Node-RED and Jupyter are two tools for coding in the browser. They are two amongst a growing collection of tools that run in the browser. Most tools have a specific purposes, be it design/prototyping using Figma or PenPot, or be it prompt engineering using OpenGPT. There a vast array of tasks that can be done in the browser.

I will concentrate on Node-RED and Jupyter since both aim to be more general purpose. Both provide web frontends to a programming language and allow the user to do anything that the underlying programming language can do. But Node-RED and Jupyter take alternative approaches to coding in the browser. Node-RED takes a visual approach and Jupyter a text-based editor-like approach.

Note:

My agenda in writing this is two-fold: one to demonstrate how flexible Node-RED is and secondly that Node-RED has the potential to bring developers and stakeholders closer together. But more of that later.

Node-RED

Node-REDs visual-programming approach focuses on modelling data flows amongst code fragments hidden within nodes. What? In less technical terms: flows describe pathways between waypoints (i.e. nodes) and these waypoints do the computing, manipulating data as needed. The data flows along the pathways, being modified the data passes through nodes (i.e. the waypoints along the pathways). Eventually each pathway completes and the data falls off the cliff, that is to say, there is no automatic storage of data that exits a flow.

Node-RED is a low code approach whereby the code is abstracted away within nodes that do the computing. It is a fundamentally different approach to programming than most developers are used to. Nodes within Node-RED represent programming concepts rather than code. For example, the switch node diverts data flows according to conditions defined within the node. The user defines the conditions, the switch node does the work. The user does not see the underlying NodeJS code that is being executed.

The program created by Node-RED are the flows. Flows are self-contained and shareable but depended on Node-RED, these are the programs running within Node-RED. So the description of the pathway between waypoints become the code that Node-RED executes. These “programs” are not human understandable as they describe connections between nodes using sixteen-random-character node IDs, X-Y location of the nodes and there is no NodeJS code. These programs cannot be transformed into a self-contained executable applications (i.e., compiled). Flows require a Node-RED engine to be executed.

Jupyter

Jupyter takes a more classic approach by combining Python code with documentation to create interactive documents called notebooks. Notebooks are self-contained and shareable, being the programs executed by Jupyter. Notebooks are also not human understandable however the Python code contained within the notebooks can be easily extracted, for example, by viewing a notebook at GitHub. So notebooks are also not independent from Jupyter however extracting the original Python code is trivial.

Jupyter is a documentation focussed approach to classic coding. I say classic coding since the typing-words-into-an-editor form of programming is how most developers work and Jupyter supports this by having a text document at its core. Classic coding concentrates on code with augmented documentation, Jupyter concentrates on documentation with embedded code.

Comparison

There are definitely more resources available for Jupyter than Node-RED. There are several sites that host and execute notebooks, in comparison there are few sites where a Node-RED flow may be tested or viewed. A quick search on GitHub for Node-RED and Jupyter gave a factor four difference: approximately 20k v. 80k results. (As an aside: GitHub provides visualisation of Jupyter notebooks but not of Node-RED flows.)

Note:

Disclaimer:
This is fairly broad-strokes comparison between the two tools. I have used both but I am not involved in either project. I am very much looking from the outside in and drawing my own conclusions. Also Jupyter is in fact language agnostic, so it’s not only for embedded Python code.

As an avid fan of Node-RED, I ask myself why does Node-RED remain a relatively niche product? Why are there so few projects out there promoting the strengths of Node-RED?

Both products are roughly the same age, both use the browser as their frontends, both run online and both are easily installed and used. Both attempt to make coding more accessible and understandable. Both have the same architecture with a frontend running in the browser and a server doing the code execution. Results are streamed to the frontend.

Where they differ is that Jupyter was a adopted for a specific problem space - data analytics. Because of its integrated visualisation of data, it set the benchmark for all other data analytics products. Jupyter did not choose this specific problem space nor is it limited to it. To an extend, Jupyter piggy-backed on Pythons success in the data analytics space to become the frontend for data analytics.

Node-RED started out with a specific problem space - Internet of Things (IoT) - and hasn’t moved out of that space. Whereas Jupyter was initially a generalised tool for documenting code via interactive notebooks and later found uptake in data analytics (specifically because its core is Python and Python has become the programming language of data analytics), Node-RED remains mainly IoT and corporate. IoT isn’t a mainstream problem space. Few people are interested in automating their home device and few companies have IoT issues that make it to the forefront of their core business.

Why care?

Why do I even care? My reason for being an advocate for Node-RED is that I believe it has the potential to bring people together. Because of its visual nature, non-developers can also understand the underlying logic of applications and programs. I believe that Node-RED can build bridges between stakeholders and developers, just as eXtreme Programming aimed to do, just as the Scrum and Agile processes aim to do, I believe that Node-RED can do the same but on a different level.

In my experimenting with Node-RED, I have created a number of non-IoT projects, so Node-RED can definitely do more than just IoT. The learning curve was certainly not that easy and having many years of coding experience definitely helped. Node-RED is definitely a tool that requires open-mindedness and patiences in the beginning.

I will continue with my comparison of Jupyter and Node-RED and then get into discussing the issues with Node-RED - again all my personal opinions, not to be taken too seriously.

Similarities

Some commonalities between Jupyter and Node-RED from the top of my head:

Same architecture: browser-based frontends with execution-engine servers. Code is modelled in the frontend and sent to the server for execution, with results streamed to the frontend to be displayed there.
Open Source Software: anyone and everyone can try and experiment with Jupyter and Node-RED.
Both have been around for a similar length of time: approximately ten years.
Both attempt to make coding more accessible, one by mixing code and documentation into a single document, the other by abstracting code away from the user.
Flexibility: both have the flexibility to solve many problems. Basically both provide web-frontends to their respect implementation languages. NodeJS in the case of Node-RED and Python in the case of Jupyter.
Code sharing is made easy by each having their on all-inclusive executable modules which can be shared. Notebooks in the case of Jupyter and flows in the case of Node-RED.
Code generated is depended on the product. Notebooks and Flows cannot be transformed into standalone executable applications.

Dissimilarities

Programming languages: Jupyter is Python based and Node-RED is NodeJS based. Python has become the de facto standard for data analytics and with it, Jupyter became popular in that problem space. NodeJS is also a very event-driven language as opposed to Python being a more procedural and straight-forward language. NodeJS code can be difficult to grasp a high-level understanding of what its doing.
Problem space: Jupyter did not begin with a specific problem space, rather it was an attempt to make Python coding more accessible. Later it became the de facto data analytics tool. Node-RED began life in the IoT space and has not moved much from that space even though it has the flexibility enter other problem spaces.
Dimensionality: I would argue that Node-RED is a two-dimensional programming environment. Being a visual-based environment, the user is working in two dimensions, moving rectangles and connecting them with lines. This does not seem like programming, comparisons to a drawing tool could be made. Jupyter is one-dimensional, there is a lengthy notebook with a mixture of code and documentation moving from top to bottom, there is no left nor right.
Low code approach: Node-RED takes a low-code approach to make coding more accessible, Jupyter makes code very much front and centre but adds documentation to make code understandable. One could argue that Node-RED makes code inaccessible to make coding more accessible.
Code reusability: Node-RED makes a lot of effort to make code reusable by allowing the creation of subflows, making node creation easy and by providing a well thought collection of libraries that can easily be installed, all within Node-RED. Jupyter relies on the user doing this in their Python environment and provides no special help for installing third party packages.
Coding concepts not code is the focus of Node-RED while Jupyter is pure code. By using Nodes to represent computational units, Node-RED is abstracting code away from the developer however thereby bring code concepts to the stakeholder.

Why does Node-RED remain an IoT product?

Node-RED has a comparably low profile in compared to Jupyter. Perhaps that’s because of the
approaches the two take. Node-RED simply takes a different approach, which might not be that simple to understand.

If one imagines the holistic idea that programming is about getting a machine to perform tasks that solve specific problems for us, then using an editor and typing lines of instructions is one approach. Perhaps the oldest approach to instructing machines. (In fact not the oldest, the oldest are punch cards!)

Node-RED takes a new approach for doing the same holistic task. Whether this approach is better or worse is comparable to debating the roundness of wheels - singularly pointless. Node-REDs approach has proven to be useful for IoT applications and building blogs.

So what is it that keeps Node-RED from mainstream uptake?

Too flexible

Is Node-RED too flexible?

A Node-RED instance running somewhere in the internet is hard to limit in its functionality. Anyone who can modify the flow, can execute anything on the server. Node-RED has limited mechanisms for limiting functionality of NodeJS within the server.

Hence the proposition that Node-RED might be too flexible for having instance available to the general public and with it the ability, to get people easily interested in Node-RED.

Two dimensionality is hard to comprehend

One comparisons of coding with an editor or using Node-RED is the realisation that editors are one-dimensional columns of characters, words, symbols and numbers. Node-RED is a visual two-dimensional world of boxes and lines, rectangles connected by grey lines - it is code in form of diagrams.

It is this visual aspect that is hard for mainstream developers to appreciate. Somehow we expect coding to be done in an editor using characters, symbols and lines - or at least any serious form of programming.

This is unfortunate since the expressiveness of Node-RED makes for conciser programs than pages and pages of code. Even if that code has syntax highlighting and fancy tab-completion. Or even AI-supported code completion.

One could compare coding using an editor/IDE to reading a book, while using Node-RED is something like watching a film. Ironically the majority would rather watch a film than read a book, but it seems that developers are a bunch of bookworms.

Code hard to view

As the code (in form from flows) is visual, something like GitHub is no use for comparing or understanding code. Flows need to be visualised to be understood. GitHub makes assumption that textual representation of code is human understandable.

GitHub goes a long way to accommodate other formats, for example, visualising Jupyter notebooks. But the equivalent functionality does not exist for Node-RED flows.

Flows are easily shared because of the excellent import/export functionality of Node-RED however the Json representation is less than ideal for human understandability.

However visual flows are bad for existing development processes such as the GitHub flow or other version control based processes.

Programming background required for Node-RED?

Which of the two statements can a non-developer understand:

Jupyter uses the concept of notebooks to embed Python code into documentation.
Node-RED is a visual programming environment that provides tools for the modelling of data flows.

Herein lies the problem of approachability. For someone who has not programmed before, having a notebook with documentation and graphs as end result is far simpler to approach than a flow diagram with nodes and lines.

Node-RED does a great job in hiding code - after all its goal is low-code - however this makes Node-RED difficult to define: is a drawing program, is it an IDE, is it a browser application, what is Node-RED?

Only if you appreciate how boring coding really is, does one understand how grand Node-RED is!

Hence my conclusion is that Node-RED is only approachable (in the beginning) if you have an appreciation for what coding is. Or you have someone who can explain the details to you. In addition to that, you really need to have understanding of the problem space.

Anyone can be given a Jupyter notebook and the documentation - should - explain the problem space and what is happening. Notebooks are self-contained and self-explanatory, flows are not. Flows provide no means to give the reader a context, notebooks contain documentation that provides context for the problem being addressed.

Why bother?

Perhaps its the way it is? Node-RED will remain a niche product for those interested in IoT.

Which is a shame.

As someone who has spent many years working in a dysfunctional industry with a very strong them and us mentality, it is shame that something that could bridge the gap is being ignored.

How do I mean this?

Software development has been and still is a case of the people writing the code and the people using that code. This divide is maintained in an industry that is imploding under its own complexity. Fewer and fewer developers understand the inherent complexity of code.

Node-RED and visual-based programming generally offers a paradigm shift towards more understanding amongst all involved. Much as the eXtreme Programming and later Agile development movements attempted to do. Bring people and wisdom together for the greater good of all.

Added to this, developers now begin to believe that the complexity of programming can be avoided by using AI-based technologies. This unfortunately will only (my opinion) make things worse by introducing subtle errors that copy&paste - aka “prompt engineering” - developers don’t understand. How many developers will fully understand the code blocks they are now including from AI tools?

Edge case debugging and debugging strange unknown exceptions will become the new norm. Subtle logically errors are the worst errors to debug - and the hardest to find.

In the end, the potential for better understanding and knowledge sharing is there but unfortunately the will and desire is lacking.

Postscriptum

I’ve done some dog-fooding by writing this article. This article is the representation of a two-dimensional flow in one dimension, that is, this article is generated by a Node-RED flow.

All the articles on this blog are flows. Hence at the bottom of each article is a link to the source flow. Beware though that flows are best viewed on a desktop, mobile has its limits unfortunately.

This blog site has been created using Node-RED and is hosted on an Node-RED instance. I am experimenting to see what I can do with Node-RED and it seems only logical to create a blog using it! Of course, the codebase of the blog is a flow!

Comments powered by giscus

The author is available for Node-RED development and consultancy.