Fields & Mappings

Every data model in Prelude Collector is composed of two types of child object: fields, which define the output schema, and mappings, which bind raw protocol output into those fields. After this first mention, the rest of this page refers to Prelude Collector as "the collector".

Fields

A field is a typed column in the model's output. Fields are ordered by their position value (ascending) and returned in that order from the API.

Field properties

Supported field types

string   int32   int64   uint32   uint64   float32   float64   boolean

Format annotations

Format annotations are applied after type coercion. The collection engine validates the coerced value against the format and drops the value (with a debug-level log entry) if validation fails.

Reordering fields

Send a JSON array of field IDs in the desired order. The engine assigns position values based on array index:

[42, 38, 45, 39]

The endpoint shape is documented in the API reference.

Mappings

A mapping binds a protocol + NetOS combination to the model. Each mapping specifies which protocol paths to collect, how to translate raw source keys into model fields, and any optional transforms.

A single model can carry several mappings — for example, a gNMI mapping for ios-xr and a NETCONF mapping for the same NetOS, or separate gNMI mappings for different firmware version ranges.

Mapping properties

Protocol path fields

Set exactly one set of path fields, matching the chosen protocol:

Field mappings

field-mappings is a JSON object that maps raw source keys onto target field names: { "source_key": "target_field_name" }.

Source keys support two formats:

• Plain key — for example "neighbor-address". Matched against any leaf in the data update.
• Path-qualified key — for example "/some/gnmi/path.leafName". Matched only when the update path equals or extends the qualifier. Path-qualified keys are resolved in a first pass before plain keys, which avoids ambiguity when a leaf name appears in more than one sub-tree.

{
  "neighbor-address": "neighbor-address",
  "peer-as": "peer-as",
  "/state.session-state": "session-state"
}

Transforms

Three transform mechanisms are available per mapping. They apply in the order shown in the pipeline below.

Field transforms

field-transforms is a pipeline of named transform functions applied to the raw value before any other processing. Format:

{
  "session-state": ["uppercase", "trim"]
}

Each entry is the name of a transform — either a built-in transform shipped with the collector or a custom Starlark transform you have registered. Built-in transforms cover common conversions (case folding, trimming, IP and MAC formatting, BGP community encoding, SNMP type coercion, unit scaling, MIB enum lookups). Custom transforms run user-supplied Starlark code in a sandboxed runtime; they receive the raw value, can call any built-in transform internally, and return a new value of the same or a different type.

The pipeline runs left to right — the output of one transform feeds the next. A misbehaving transform never silently drops the field: an unknown name, a runtime error, or a null return all log a warning and pass the original value through unchanged so the rest of the chain still runs. See Transforms for the full reference, the catalogue of built-ins, and the Starlark API.

Value transforms

value-transforms is a static substitution map applied after field transforms. Format:

{
  "session-state": {
    "IDLE":        "idle",
    "ESTABLISHED": "established"
  },
  "peer-type": {
    "INTERNAL": "ibgp",
    "EXTERNAL": "ebgp"
  }
}

Ignore values

ignore-values lists values that cause the row to be dropped (when the field is the key field) or the individual field to be skipped.

{
  "name": ["Null0"]
}

Processing pipeline

For each raw update received from the protocol client, the parser:

1. Applies the optional gnmi-path-filter and skips updates whose path does not match the suffix.
2. Extracts the key field value. Drops the update if the key is empty or appears in ignore-values for the key field.
3. Merges with the cached row for that key, preserving fields collected from earlier sub-path updates.
4. Resolves path-qualified field mappings first, then plain field mappings.
5. For each resolved field: applies field transforms, stringifies the result, applies value transforms, checks ignore-values, coerces to the target type (using both the original raw value and the stringified value, so type-correct inputs skip string parsing), and validates the format annotation.
6. Returns a parsed row with key, data, and any per-field parse errors.

Mapping uniqueness

The combination (model, protocol, netos, version-constraints) is unique. Two mappings with the same protocol + NetOS + version range on the same model are not allowed — the second POST returns 409 Conflict.

Complete example

The full JSON for a model with one gNMI mapping looks like this:

{
  "id": 1,
  "name": "oc-bgp-neighbor",
  "description": "OpenConfig BGP neighbor session state",
  "version": "1.0.0",
  "is-active": true,
  "fields": [
    {"id": 1, "name": "neighbor-address", "field-type": "string", "format": "ip-address", "required": true,  "position": 0},
    {"id": 2, "name": "peer-as",          "field-type": "string", "required": false, "position": 1},
    {"id": 3, "name": "session-state",    "field-type": "string", "required": false, "position": 2},
    {"id": 4, "name": "local-as",         "field-type": "string", "required": false, "position": 3},
    {"id": 5, "name": "peer-type",        "field-type": "string", "required": false, "position": 4},
    {"id": 6, "name": "description",      "field-type": "string", "required": false, "position": 5}
  ],
  "mappings": [
    {
      "id": 1,
      "protocol": "gnmi",
      "netos": "ios-xr",
      "version-constraints": [],
      "key-field": "neighbor-address",
      "gnmi-paths": [
        "openconfig-network-instance:network-instances/network-instance/protocols/protocol/bgp/neighbors/neighbor/state"
      ],
      "gnmi-path-filter": "/state",
      "field-mappings": {
        "neighbor-address": "neighbor-address",
        "peer-as":          "peer-as",
        "session-state":    "session-state",
        "local-as":         "local-as",
        "peer-type":        "peer-type",
        "description":      "description"
      },
      "value-transforms": {
        "session-state": {
          "IDLE": "idle", "CONNECT": "connect", "ACTIVE": "active",
          "OPENSENT": "opensent", "OPENCONFIRM": "openconfirm", "ESTABLISHED": "established"
        },
        "peer-type": {
          "INTERNAL": "ibgp",
          "EXTERNAL": "ebgp"
        }
      }
    }
  ]
}

See also

• Custom Models overview
• Workflow
• API reference