Documentation Index
Fetch the complete documentation index at: https://docs.siftstack.com/llms.txt
Use this file to discover all available pages before exploring further.
Ingestion-config-based streaming uses IngestionConfigService.CreateIngestionConfig to register the schema of your telemetry, then streams compact structured messages to Sift over gRPC. This approach reduces packet size and improves throughput compared to schemaless methods.
CreateIngestionConfigRequest
message CreateIngestionConfigRequest {
string asset_name = 1 [(google.api.field_behavior) = REQUIRED];
repeated FlowConfig flows = 2;
string organization_id = 3 [(google.api.field_behavior) = OPTIONAL];
string client_key = 4 [(google.api.field_behavior) = OPTIONAL];
}
| Field | Description |
|---|
asset_name | The name of the asset to create. Required. |
flows | One or more flow configurations defining the schema. See FlowConfig. |
organization_id | Your organization ID. Only required if you belong to multiple organizations. |
client_key | An arbitrary string you choose to uniquely identify this ingestion config. Optional but strongly recommended; it simplifies lookups. |
FlowConfig
A flow is a named group of channels whose values are sent together in one request.
message FlowConfig {
string name = 1 [(google.api.field_behavior) = REQUIRED];
repeated ChannelConfig channels = 2;
}
ChannelConfig
message ChannelConfig {
string name = 1 [(google.api.field_behavior) = REQUIRED];
string component = 2;
string unit = 3;
string description = 4;
sift.common.type.v1.ChannelDataType data_type = 5 [(google.api.field_behavior) = REQUIRED];
repeated sift.common.type.v1.ChannelEnumType enum_types = 6;
repeated sift.common.type.v1.ChannelBitFieldElement bit_field_elements = 7;
}
| Field | Description |
|---|
name | The channel name. Required. |
component | Optional component label. |
unit | Optional unit string (for example, km/hr). |
description | Optional human-readable description. |
data_type | The data type of the channel. Required. |
enum_types | Enum type definitions for enum-typed channels. |
bit_field_elements | Bit field element definitions for bit-field-typed channels. |
Channel ordering
The order of channels in a FlowConfig must be preserved exactly when sending values in an IngestWithConfigDataStreamRequest. Sift attributes each value to a channel by its position in the list.
If data is available for some channels in a flow but not others, send google.protobuf.Empty in the position of the missing channel to maintain correct ordering.
Example
Given a flow reading with two channels—a double channel followed by a string channel:
from sift_stream_bindings import (
ChannelConfigPy,
ChannelDataTypePy,
FlowConfigPy,
)
flow_config = FlowConfigPy(
name="reading",
channels=[
ChannelConfigPy(
name="mainmotor.velocity",
unit="km/hr",
description="vehicle speed",
data_type=ChannelDataTypePy.Double,
enum_types=[],
bit_field_elements=[],
),
ChannelConfigPy(
name="log",
description="logs",
data_type=ChannelDataTypePy.String,
enum_types=[],
bit_field_elements=[],
),
],
)
from sift_stream_bindings import ChannelValuePy, FlowPy, TimeValuePy, ValuePy
from datetime import datetime, timezone
now = datetime.now(timezone.utc)
await ingest_client.send(
FlowPy(
flow_name="reading",
timestamp=TimeValuePy.from_timestamp_millis(int(now.timestamp() * 1000)),
values=[
# velocity channel (position 0)
ChannelValuePy(name="mainmotor.velocity", value=ValuePy.Double(10.0)),
# log channel (position 1)
ChannelValuePy(name="log", value=ValuePy.String("example log")),
],
)
)
IngestWithConfigDataStreamRequest
message IngestWithConfigDataStreamRequest {
string ingestion_config_id = 1;
string flow = 2;
google.protobuf.Timestamp timestamp = 3;
repeated IngestWithConfigDataChannelValue channel_values = 4;
string run_id = 5;
bool end_stream_on_validation_error = 6;
string organization_id = 7;
}
| Field | Description |
|---|
ingestion_config_id | The ID returned when you created the ingestion config. |
flow | The name of the flow this request sends data for. |
timestamp | The timestamp for all channel values in this request. |
channel_values | Ordered list of values. Order must match the channel order in the flow definition. |
run_id | Optional. Must be included if this data belongs to a Run. |
end_stream_on_validation_error | When true, the stream terminates if a server-side error occurs. Use only during development; this flag severely impacts production performance. |
organization_id | Optional unless your user belongs to multiple organizations. |
end_stream_on_validation_error is false appear in Sift’s Data Processing dashboard (https://app.siftstack.com/manage/data-processing).
Retrieving an ingestion config by client key
curl -G -H "Authorization: Bearer $API_TOKEN" -d "filter=client_key=='example_client_key'" $SIFT_REST_URL/api/v1/ingestion-configs
Updating an ingestion config
To add new flows after creation, send a CreateIngestionConfigFlowRequest to IngestionConfigService. Adding flows and channels is backwards compatible. Modifying or removing existing flows or channels is not.
