Skip to main content
After creating your ingestion config from the previous section, you are ready to start streaming data into Sift over gRPC.

​
Creating a data ingestion request

Streaming is straightforward. To begin, create an instance of IngestWithConfigDataStreamRequest, which has the following shape: 
message IngestWithConfigDataStreamRequest {
  string ingestion_config_id = 1;
  string flow = 2;
  google.protobuf.Timestamp timestamp = 3;
  repeated IngestWithConfigDataChannelValue channel_values = 4;
  string run_id = 5;
  bool end_stream_on_validation_error = 6;
  string organization_id = 7;
}
  • ingestion_config_id: The ID of the ingestion config that was created in a previous section.
  • flow: The name of the flow where you want to send data.
  • timestamp: The timestamp to use for this flow.
  • channel_values: An order-specific list of Channel values that matches the order of Channel configs for this particular flow. Read more on ordering.
  • run_id: Gernally, an optional Run ID to associate with the data. The run_id must be included if this data is part of a Run. Read more on creating and using Runs.
  • end_stream_on_validation_error: When true, terminate the stream if an error occurs Sift-side.
  • organization_id: The ID of your organization. Optional unless your user belongs to multiple organizations.
Impact of Validation on PerformanceThe end_stream_on_validation_error flag is useful during development to debug and verify streaming setup. It should not be used during production because it can severely impact performance. Any errors that occur while this flag is set to false can be found in Sift’s Data Processing errors (`https://app.siftstack.com/manage/data-processing).

​
Sending data

The IngestWithConfigDataStreamRequest you created can now be passed to IngestService.IngestWithConfigDataStream. If the request is processed successfully, you will see data in Sift under the Asset associated with the ingestion config.