Skip to content
Back to

Data lineage

Data lineage is Fontana’s record of how a value in a workflow run came to be: which upstream ports supplied it, which transforms and validations touched it, which lookups enriched it, and which nodes on the graph produced the result you inspect.

Lineage is atomic: attached at row and column level, not only as a workflow summary. That gives you a precise answer to how, why, and from where a cell changed, including when conditional logic sent rows down different processing paths.

What lineage records

Section titled “What lineage records”

Processors that derive data emit pointer-based sources. Those sources may themselves have lineage, forming a recursive tree you walk in Flow from a selected cell back through upstream nodes.

Stage What lineage can record
Ingress Origin of rows that a source processor attaches to the run (file, connector, or API)
Transform and validate Operations, lookups, merges, compute steps, schema mapping, and AI-assisted transforms
Conditional processing Filters, validations, and condition operations that include or exclude rows, and branches of the graph that contributed to each output
Egress The same provenance chain on the run, so you can inspect an output cell against the nodes that produced it

How sources are described

Section titled “How sources are described”

Each lineage link carries a usage type that explains why that source contributed to the result:

Usage Meaning
loaded Row or field came from an external source (file upload, connector, API)
transformation Source value was transformed (for example string, date, or decimal operations)
lookup Source was matched and enriched from a lookup dataset
merge Source was combined with other inputs
reference Source passed through unchanged
computed Source used in a function or compute step
aggregated Source contributed to a grouped or rolled-up result
generated Source used to generate new values (Query, subgrid, schema mapping, or AI output)
filtered Source passed through filter or validation context (row included or excluded on a branch)

Together, usage types and node references show what data was used to calculate results and which conditional branches were involved when filters, validations, or graph paths split processing.

View lineage in Flow

Section titled “View lineage in Flow”

You do not need to leave the app to inspect provenance:

  • Overview - select a cell in the data grid. Tree walks that cell back through upstream nodes with usage labels. Sources and Impact list hops in a grid. Graph places the selected cell in the middle, sources above, and impact below by default (toggle to left-right in the graph controls). Each Graph hop box shows that hop’s cell value, matching Tree, so you can see the value change along the chain. When a hop’s value differs from the upstream hop, Graph and Tree mark that value like the data grid: green label text and a green status dot. Identity hops that pass the same value through stay unmarked. The DAG starts fitted to the Overview pane; you can pan and zoom to inspect nodes
  • Writing operations - when Operation or Compute changed the selected cell, Overview walks one hop per writing operation on that field. Tree, Graph, Sources, and Impact share that walk. Graph shows a box per writing operation. Click a hop to select the host node and cell and open that operation in the Operations tab
  • Cell hover - the grid cell tooltip shows audit and lineage for that column (plus row-level events with no column)
  • Row gutter hover - the thin left-hand column shows audit and lineage for the whole row
  • Workflow data view - review port-level datasets per node for the active run
  • Audit panel - complementary audit items (validation outcomes, transform events, manual edits) alongside lineage for the same run

Lineage answers provenance; audit items answer events on the row. Both support the Transparent data principle.

Run-scoped provenance

Section titled “Run-scoped provenance”

Lineage lives with the run. You inspect it in Flow against the same snapshot as your datasets and Save File Export outputs (see Egress). Fontana’s internal pointer model is the source of truth for that inspection.

OpenLineage export

Section titled “OpenLineage export”

You can push run, job, and dataset events to an OpenLineage backend when a workflow run finishes. In Admin → OpenLineage you set one public https endpoint and one write-only bearer token. An empty endpoint turns export off.

  • When events post - the workflow engine sends START, then COMPLETE or FAIL after the run ends
  • What the events contain - jobs and datasets projected from Fontana’s pointer-based lineage. Job and dataset namespace is your tenant domain. Job names lead with the workflow and canvas node titles you see in Flow, with stable ids after a middle dot (·). Marquez stores parented node jobs as {workflowJob}.{nodeJob}; Flow deep links use that full catalog name. Column-level lineage is included where Fontana recorded it, and is empty for columns produced by row-level steps such as AI nodes
  • How Flow relates - Overview and the inspector still read Fontana’s internal lineage model. OpenLineage is an export projection for catalog and observability tools that consume the standard

Why it matters

Section titled “Why it matters”
  • Regulatory and audit review - demonstrate which inputs and transforms produced reported figures
  • Incident analysis - isolate when a value changed and which step introduced an error or exception
  • Operational trust - you see the same provenance tree that a compliance reviewer walks in Overview
  • Downstream integration - OpenLineage export lets your catalog or observability backend ingest Fontana runs as standard events

Lineage is part of Fontana’s Auditable and Transparent data principles. It complements, but is not the same as, the platform immutable security audit trail (sign-in, admin, cluster events in ImmuDB).

Section titled “Related documentation”