Skip to content

Pipeline Debugging

How to check if a pipeline is working, and what to do if it isn't doing what you expect.

Debugging with stepping mode

The Stepper is Stroom's most powerful debugging feature. Unlike traditional logs, it allows you to visualise the data transformation as it happens, one record at a time.

How to start stepping

There are two ways in, and which you take depends on what you already have in front of you.

From the pipeline, when you know what you want to test:

  1. Open a Pipeline on its Structure tab.
  2. Click the Step button in the toolbar.

From the data, when you have a stream that behaved oddly:

  1. Open a Feed, or the Stream Store.
  2. Select the stream you want to test against.
  3. Click the Step button, on the toolbar or in the context menu.
  4. Choose the Pipeline to step it through.

What the stepper shows

The UI will split into multiple panes, each representing an element in your pipeline (e.g., Source, Parser, XSLTFilter, XMLWriter).

  • Input vs. output: You can click on any element to see exactly what data it received and what data it produced.
  • Navigation: You can move forward or backward through the stream record-by-record using the arrow buttons.
  • Finding errors: If an XSLT transformation fails on record #502, you can jump to that specific record and see the input XML that caused the crash.
  • Markers: Fatal errors and warnings are highlighted with red and yellow markers in the margin, allowing you to quickly identify "bad" data.

Use cases for stepping

  • Developing XSLT: See the immediate effect of your stylesheet changes on real data (see Parsers and translation).
  • Fixing parsers: Identify why a specific log line isn't being split into the correct XML fields.
  • Validating lookups: Verify that reference data lookups are returning the expected values.

Then checking it at scale

The Stepper proves what a pipeline does to one record. It cannot tell you whether the pipeline is keeping up, whether some streams are failing, or whether anything is being processed at all, because it never touches the processing path.

Those are different questions with their own pages:

All documentation