Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.siftstack.com/llms.txt

Use this file to discover all available pages before exploring further.

This topic explains how to verify that your data is arriving correctly in Sift, identify records that failed to ingest, and track ingestion rates using sift_app, a built-in system Asset that surfaces real-time telemetry about your data pipeline.

​
Before you begin

​
How ingestion monitoring works

sift_app is a built-in system Asset that provides real-time telemetry about your data pipeline’s performance and stability. It surfaces internal metrics through the same infrastructure you use for your own data, allowing you to build dashboards and set up alerts for specific Channels using familiar Sift tools. sift_app is organized into four subsections, each representing a different layer of the ingestion journey:
  • data_import: Metrics for data imported via file upload. Use this to confirm an import completed successfully or to understand why it failed.
  • dlq_ingestion: Metrics for records that failed to ingest. Use this if data is not appearing in Sift as expected.
  • ingest_grpc: Metrics for data streamed through the gRPC APIs. Use this to confirm data is actively flowing into Sift.
  • stream: Metrics for the Rust sift-stream client. Use this to monitor client-side ingestion behavior.

​
Access sift_app

  1. In Sift’s homepage, click the Assets tab.
  2. In the Asset name search box, enter sift_app.
  3. In the Assets table, click sift_app.
  4. Click Explore.

​
Monitor ingestion metrics

Once in Explore, the sift_app subsections appear in the Channels tab.
  1. Expand the subsection you want to monitor.
  2. Click a Channel to plot it.
  3. Review the plot to understand the current state of that part of your pipeline.

​
Reference