`tabular-to-dataservice` (Transform Node)

The Tabular to Dataservice node converts a single tabular dataset (CSV, TSV, or tabular-query JSON) into a $heat-dataservice envelope for the Next dashboard data service. Downstream dashboard-v2 can consume the output without a hand-written json-template JMESPath layer.

Typical pipeline


CSV upload → tabular-to-dataservice → dashboard-v2

Optional upstream steps:

tabular-remap to normalize column names
tabular-query to filter or aggregate (set inputFormat: "json" and sourceTable on this node)

See the dashboard-v2 upstream contract for canonical payload shapes.

Configuration

Property	Type	Required	Default	Description
`inputFormat`	`auto \| csv \| tsv \| json`	✖	`auto`	`json` for `tabular-query` output; `auto` sniffs CSV/TSV or JSON object
`sourceTable`	`string`	✖*	,	`tables` key when input is JSON; required if multiple tables exist
`timeColumn`	`string`	✔	,	Column used for series `timeMs`
`timeColumnIsMs`	`boolean`	✖	`true`	Values are milliseconds offset from `baseCapturedAt`
`timeColumnIsIso8601`	`boolean`	✖	`false`	Parse ISO8601; `timeMs` = ms since minimum timestamp per realm
`baseCapturedAt`	`string`	✖**	,	ISO8601 base when `timeColumnIsMs=true`
`sortByTime`	`boolean`	✖	`true`	Sort emitted series points by `timeMs`
`defaultRealm`	`string`	✖	`default`	Realm when no `realmColumn` or cell is empty
`realmColumn`	`string`	✖	,	Distinct values become separate `realms[].name`
`discardRowsWithEmptyRealm`	`boolean`	✖	`false`	Drop rows with empty realm cell
`groups`	`array`	✖	one `metrics` group	Channel picker groups (`id`, `name`)
`defaultGroupId`	`string`	✖	`metrics`	Group for auto-discovered channels
`excludeColumns`	`array`	✖	`[]`	Skip these columns (plus time/realm/PK columns)
`includeColumns`	`array`	✖	,	If set, only these columns become channels
`channels`	`array`	✖	`[]`	Explicit channel definitions (override auto)
`treatEmptyAsNone`	`boolean`	✖	`true`	Treat blank cells as empty
`discardEmptyValues`	`boolean`	✖	`true`	Omit series points with empty/non-numeric values
`discardEmptyRows`	`boolean`	✖	`false`	Drop rows where all channel values are empty
`primaryKeyColumns`	`array`	✖	`[]`	Extra dedupe key parts with `timeColumn` per channel
`dedupePolicy`	`last \| first`	✖	`last`	When duplicate time (+ PK), keep last or first row
`dataserviceVersion`	`string`	✖	`1.0`	`$heat-dataservice.version`
`dashboardUsers`	`array`	✖	`[]`	HEAT Auth external user GUIDs for dimension ACL

* Required when JSON input contains more than one table.

** Required when timeColumnIsMs=true and timeColumnIsIso8601=false.

Auto-discovered channels

By default, every column that is not reserved (time, realm, primary keys, excluded) and has at least one numeric value becomes a shape: "series" channel.

Channel id rule: lowercase header, non-alphanumeric characters become _, repeated _ collapsed (e.g. Speed (m/s) → speed_m_s). Layout configuration.channels in dashboard-v2 must use these ids (or define explicit channels[].id overrides).

Example (CSV)


{
  "timeColumn": "time_ms",
  "timeColumnIsMs": true,
  "baseCapturedAt": "2025-01-01T00:00:00Z",
  "realmColumn": "pilot_id",
  "excludeColumns": ["notes"]
}

Example (after tabular-query)


{
  "inputFormat": "json",
  "sourceTable": "filtered",
  "timeColumn": "elapsed_ms",
  "timeColumnIsMs": true,
  "baseCapturedAt": "2025-01-01T00:00:00Z"
}

Output


{
  "$heat-dataservice": {
    "version": "1.0",
    "groups": [{ "id": "metrics", "name": "Metrics" }],
    "realms": [
      {
        "name": "default",
        "channels": [
          {
            "id": "speed_mps",
            "name": "speed_mps",
            "groupId": "metrics",
            "shape": "series",
            "data": [{ "timeMs": 0, "value": 10.0 }]
          }
        ]
      }
    ]
  },
  "dashboard_users": []
}

Limits

The full table is loaded in memory (same as tabular-query / tabular-remap).
dashboard-v2 warns when realm count exceeds 50; avoid high-cardinality realmColumn values.

tabular-to-dataservice (Transform Node)