Skip to content
Documentation Prelude Collector 1.0.0

Transforms

How Prelude Collector reshapes raw collected values before they reach the output — built-in vs. custom transforms, chaining, and error handling.

A transform is a function that takes a single raw collected value and returns a modified version of it. Prelude Collector applies transforms after a value is read from a device and before it is written to an output. After this first mention, the rest of this section refers to Prelude Collector as "the collector".

A typical use case is converting a packed integer into a human-readable string — for example turning the SNMP TimeTicks counter 360000 into "1h 0m 0s", or the packed 32-bit address 0xC0A80001 into "192.0.2.1".

Where transforms fit in the pipeline

Device / Source
      │
      ▼
  Collection  (gNMI, SNMP, NETCONF, CLI, …)
      │
      ▼
  Transforms  ← you are here
      │
      ▼
  Output / Storage

Transforms operate on a single field value at a time. Each field in a data model mapping can list zero or more transform names; the pipeline applies them in order.

Built-in vs. custom

The collector ships with two categories of transforms:

Built-in Custom
Origin Compiled into the collector binary Created at runtime via the API or web UI
Language Go Starlark (Python-like)
Coverage Common networking conversions Anything you need that isn't built in
Editable No (always available under their fixed names) Yes (full CRUD)

Built-in transforms cover IP address formatting, BGP community encoding, SNMP type conversions, unit scaling, MIB enum lookups, and similar operations. See Built-in transforms for the full catalogue.

Custom transforms let you write your own logic in a sandboxed Starlark expression when the built-ins do not cover your case. They can also call any built-in transform internally. See Custom transforms for the language reference and the registration API.

Attaching transforms to a field

Transforms are configured per-field inside a data model mapping. The transforms key on a field accepts an ordered list of registered transform names:

{
  "field": "uptime",
  "oid": "1.3.6.1.2.1.1.3.0",
  "transforms": ["timeticks_to_uptime"]
}

Built-in and custom names may be mixed freely. The collector resolves each name in the registry at collection time, so adding a new custom transform makes it immediately usable in any mapping that references it.

For the full field mapping schema, see Fields & Mappings.

Chaining multiple transforms

You can assign a list of transforms to a single field. They run left to right, each receiving the output of the previous step:

raw value → transform_1 → transform_2 → … → final value

For example, to convert a bytes-per-second counter into Mbps, first multiply by 8 to convert bytes to bits (mul_8), then divide by 1,000,000 (to_mbps):

"transforms": ["mul_8", "to_mbps"]

Order matters. Applying to_mbps before mul_8 would produce a value 8x too small.

Error handling

A transform error never crashes a collection cycle, but the behaviour differs slightly between the parser path (the path used during live collection) and the standalone Apply helper (used when a caller wants pass-through-on-error semantics).

Situation Behaviour during live collection
Transform name not found in registry Field is dropped from the parsed entry; a transform-not-found parse error is recorded for the snapshot.
Transform function returns an error Field is dropped from the parsed entry; a transform-error parse error is recorded for the snapshot.
Transform returns a null / missing value Warning logged; original value kept.

Parse errors surface on the snapshot's errors array so you can diagnose problems without grepping logs. Each entry includes the transform name, the original value, and the underlying error.

Where transforms run inside a mapping

A field-level transforms list (the focus of this page) is the first of three stages a mapping applies to every collected value before the snapshot is written:

raw value
   │
   ▼
1. field-transforms   ← per-field chain documented here (e.g. ["mul_8", "to_mbps"])
   │
   ▼
   value is stringified  (fmt.Sprintf("%v", value))
   │
   ▼
2. value-transforms   ← per-value lookup table {"1":"up","2":"down"}
   │
   ▼
3. ignore-values      ← drop the entry if the resulting string matches
   │
   ▼
final value

Two consequences worth knowing:

  • value-transforms keys are strings. Because the value is stringified before lookup, a key of "42" matches both the integer 42 and the string "42".
  • ignore-values runs last — after both transform stages — so you can normalise a code into a name with value-transforms and then drop unwanted names with ignore-values.

For the full mapping schema and concrete examples of all three stages, see Fields & Mappings — Transforms and value normalisation.

Reference pages in this section

  • Built-in transforms — the full catalogue, grouped by module, with input / output types and a one-line example each.
  • Custom transforms — Starlark language reference, the available globals, the return contract, and the registration API.

For the request and response schemas of every transform endpoint, see the API reference.

Filtering by: