Skip to main content

​
What are Pipelines?

Transformation Pipelines let you filter, reshape, enrich, and transform job data before it’s pushed to an external destination via an Outbound Feed. Build visual flows by dragging nodes onto a canvas, connecting them, and configuring each step — no code required. Once attached to a feed destination, a pipeline runs automatically on every sync cycle: each job passes through your pipeline before being written to the destination.
Pipelines run during Outbound Feed syncs — you cannot trigger them independently, but you can test them against a real job from your feed at any time in the editor’s test panel. The test input is exactly the document your pipeline receives during a live sync.

​
Feature Highlights

Filters

Sync only the jobs you want — match on any field with 12 operators (equals, contains, greater than, regex, and more). Non-matching jobs are skipped and counted, never written.

Field Mapping

Rename and restructure fields to your own schema with simple source → target path mappings. No templates needed for the common case.

Liquid Templates

Full Liquid templating for advanced reshaping: conditionals, loops, date formatting, string manipulation.

AI Transforms

Send data to any OpenRouter model to extract skills, classify jobs, or rewrite descriptions — using your own OpenRouter key.

JSON Merge

Combine AI output with original data using deep or shallow merge with configurable conflict resolution. Preserves all existing fields.

Test with Real Data

Run a random job from your live feed through the pipeline and inspect every node’s input, output, timing, and errors.

​
Node Types

Every pipeline has exactly one Input node and one Output node. In between, you can add any combination of transformation nodes.

Input

Entry point — receives the job document from the Outbound Feed sync engine

Filter

Keep or drop jobs based on field conditions

Field Mapper

Rename and restructure fields with path mappings

Liquid Template

Transform data using Liquid template syntax

AI (OpenRouter)

Process data with any LLM via OpenRouter with structured JSON output

JSON Merge

Combine two JSON inputs (base + patch) with configurable strategies

Output

Exit point — final JSON written to your destination

​
How It Works

Every pipeline is a directed acyclic graph (DAG) of nodes. Data flows from the Input node through transformation nodes and arrives at the Output node.
1

Create a Pipeline

Open the visual editor in Data → Pipelines in the dashboard. Start from a blank canvas or choose a pre-built template — including a zero-cost Filter + Field Mapper template that needs no AI setup.
2

Add & Connect Nodes

Drag nodes from the palette onto the canvas. Connect outputs to inputs to define the data flow.
3

Configure Each Node

Click a node to open its config panel. Set filter conditions, field mappings, Liquid templates, AI prompts and output schemas, or merge strategies.
4

Test with Real Data

Use the built-in test panel to run a real job from your feed through the pipeline and inspect each node’s output before going live.
5

Attach to a Feed Destination

Select the pipeline in the destination’s configuration on the Outbound Feed page. Every sync cycle runs jobs through the pipeline before writing.

​
Attaching to Outbound Feeds

Pipelines don’t run in isolation — they’re attached to a destination on the Outbound Feed page. When the feed syncs, each job passes through the attached pipeline before being written.
  1. Create your pipeline — build and test it in the Pipeline Editor. A pipeline with unresolved configuration errors can be saved, but not attached.
  2. Open your Outbound Feed — edit the destination configuration you want to transform.
  3. Select a pipeline — choose it from the Transformation pipeline dropdown in the destination settings.

​
Supported destinations

Pipelines can change the shape of your documents, so they’re available on document destinations only:
Document identity is preserved. Whatever your pipeline does to a job’s body, the destination document keeps the original Jobo job id as its identifier. Updates and expiry deletions keep working even if you rename every other field.

​
AI nodes and cost

AI nodes call OpenRouter with your own API key (configured in Settings). During a sync, that means one LLM call per synced job for each AI node in your pipeline:
The first sync after attaching a pipeline processes your entire feed — with AI nodes, that’s one LLM call per job in the feed, billed to your OpenRouter account. Subsequent syncs are incremental and only process new and changed jobs. If your OpenRouter key is invalid or out of credits, the sync fails fast with an actionable error instead of burning through the batch.

​
Applying pipeline changes to already-synced jobs

Feed syncs are incremental — they only process jobs that are new or changed since the last sync. When you edit an attached pipeline, the new logic applies to future syncs only. To reapply it to everything, use Full resync on the destination card: it clears the sync watermark and re-runs the whole feed through the pipeline.

​
Next Steps

Node Types Reference

Detailed configuration for each node type

Example Flows

Pre-built pipeline examples for common use cases

Execution Model

How nodes are executed and how errors are handled

Best Practices

Tips for building efficient and reliable pipelines