Click to close
Four short screencasts presenting the Screaming Streaming node package.
These screencasts present meta programming in Node-RED and an approach to doing ETL pipelines within Node-RED using the NodeJS Stream API.
Please note: These screencasts are not for folks new to Node-RED, if you are looking for an introduction to Node-RED, then check out my blog for a bunch of articles on learning Node-RED.
Why do streaming in Node-RED? How to handle large datasets in Node-RED? The latter is the reason for the initial question.
Links:
This episode demonstrates a simple example of using Streaming to handle a dataset of 250MB. The dataset is compressed JsonL data that is streamed into Node-RED without performance degradation. (As an aside: JsonL has many names - Json Streaming, NDJson and JsonL.)
Links:
When is a flow not a flow? When it is a meta-flow! This describes the use of meta-programming in Node-RED to describe a streaming pipeline using a Node-RED flow.
Links:
Explaining how the package works and how other streamings nodes may be created and used within the streaming package.