Click to close

Streaming Package

Four short screencasts presenting the Screaming Streaming node package.

These screencasts present meta programming in Node-RED and an approach to doing ETL pipelines within Node-RED using the NodeJS Stream API.

Please note: These screencasts are not for folks new to Node-RED, if you are looking for an introduction to Node-RED, then check out my blog for a bunch of articles on learning Node-RED.

Episode 1: Motivation

Why do streaming in Node-RED? How to handle large datasets in Node-RED? The latter is the reason for the initial question.


Episode 2: Example

This episode demonstrates a simple example of using Streaming to handle a dataset of 250MB. The dataset is compressed JsonL data that is streamed into Node-RED without performance degradation. (As an aside: JsonL has many names - Json Streaming, NDJson and JsonL.)


Episode 3: Meta Programming

When is a flow not a flow? When it is a meta-flow! This describes the use of meta-programming in Node-RED to describe a streaming pipeline using a Node-RED flow.


Episode 4: Code Explained

Explaining how the package works and how other streamings nodes may be created and used within the streaming package.

Last updated: 2024-04-24T12:22:10.105Z

Comments powered by giscus

The author is available for Node-RED development and consultancy.