Click to close
The Deutsch Limit, to quote from Wikipedia:
The problem with visual programming is that you can’t have more than 50 visual primitives on the screen at the same time.
is perhaps a somewhat arbitrary statement especially since in the same article, the authors write:
However, criticisms of the limit include that it is not clear whether a similar limit also exists in textual programming languages […]
Personally, I am not a fan of “hard limits” - everyone understands complexity differently. In fact, I would argue that my ETL pipeline flow is very simple to understand:
Why would I say that it is simple to understand, there are obviously more than 50 visual primitives? In fact there are 1,165 nodes in that flow - the wonders of copy & paste!
I would claim it to be a non-complex flow because the ETL flow does the same thing for multiple providers: for each provider, all the flow does is retrieve a yaml, from that yaml it reads a data url, from that url, the actual data is retrieved and stored to disk. A second functionality loads the data from disk and streams individual messages for each data point found. Finally, each provider is represented by a group.
I would call that flow a shallow flow because it lacks logical depth and it is the logical depth that, for me, makes a flow complex. The ETL flow has no or little logic. There is no failure handling, there is no combining of data points and there is not transformation complexity: retrieve, save and parse. That’s all.[1]
What do I mean by logical depth?
Basically link-out and link-in nodes, or link-call nodes or subflows. If a flow makes use of many links and indirections, then for me, it becomes logically deep: the reader has to mentally maintain a stack of indirections to understand the flow and what is going on.
This also applies to textual code with many function/method calls. The reader needs to maintain a mental model of the indirections. In addition, the reader has to mentally parse the streams of textual information such as keywords, numbers and comments.
In comparison, this flow (the raw data for the crunchy numbers article):
Is for me complex. It maintains and updates many bits of data, it renders a collection of dashboard graphs, each represent a different insight into the data. Yet the flow contains 286 nodes, a fifth of the ETL flow above.
A rudimentary explanation: the groups on the right side maintain and collect the raw data, while the other parts of the flow update the dashboard graphs, massaging the data as necessary. It is difficult to identify which graph is represented by which data source and which graph is generated by which graph node, it becomes guess work in the end.
I would suggest this flow is a deep flow with much complexity, indirection and logical computation. Much of the logical computation is hidden in function nodes, that simplifies the flow to an extent though.
In the end, everyone will have their own personal level of complexity, I do not believe there are hard limits to visual complexity. Something that I have found that helps is creating form in a flow. If the flow looks like what it does, it becomes simpler to understand the prime goal of the flow. I use my decision tree flow as an example of this idea.
Another qwerty that dawned on me was that shapes have traditionally been important to visual programming. Shapes are used to distinguish functionality. I belief that shapes, in fact, add to the complexity of understanding flows.
For example, the DRAKON language has the following shapes:
UML also uses shapes instead of colours and symbolic. In defence of shapes: these things were invented in the days of green-screens and monochrome monitors, there was simply no colour to help. Also the computation power was not up to scratch to render complex SVGs - even if SVGs had been invented!
Node-RED instead of shapes, uses symbols and colours:
I find shapes harder to associated with functionality, that is, I have no mental association with a parallelogram or diamond being an if or switch statement. Nor do I have an association with a rectangle as being an action.
Why is that? Probably because the concept that a shape has computational meaning escapes me. Or perhaps because I see a shape and think of other shapes and not functionality. Or perhaps shapes are such a basic concepts for me that I cannot re-associate them with anything else. Or shapes are just too general and widespread that they have too many meanings. Traffic signs have shapes, windows have shapes, keys on a keyboard have shapes.
Hence I find the symbolic of Node-RED far more intuitive. The crossed paths symbol for a change node or the fork symbol for a switch - a fork in the road, a divide. Or the ‘f’ in form of an integral for a function, a general computational unit, an action. Symbols also have more possibilities to describe meaning since they are more complex than basic shapes.
Those symbols make sense to me and I can freely associate them to specific functionality because I have no other previous associations for those symbols. Or perhaps, much as emoticons, symbols are more complicated than basic shapes and therefore can have multiple associations or have less associations to begin with.
For whatever reason, the symbolisation or iconification of programming might well be a good thing. Just as a picture tells the story of a thousand words, so one day an icon might well hide a thousand lines of code!
Thanks to @donpedro for pointing me to the Deutsch Limit.
Of course, there is a lot of logic hidden in the HTTP request that does the retrieval - logic for 404 pages or for doing the streaming or authentication, etc. Or the parsing of data streams. But this logic is not clattering the flow. ↩︎