Rules
Overview
Rules in Sift define logical conditions used to automatically evaluate telemetry data from one or more Channels during a Run. Built using the Common Expression Language (CEL), Rules enable users to identify behaviors, anomalies, or thresholds in live or historical data streams.
Rules support both real-time and retrospective evaluations. They are continuously evaluated as data is received from Assets, and can also be re-evaluated on historical data when a specific Run is selected. This allows Rules to drive analysis, diagnostics, and automation at scale. Sift provides a rich set of built-in functions to define complex logic within Rules, enabling users to model sophisticated behaviors and system checks. In addition to built-in functions, users can also create and leverage User-defined Functions to encapsulate reusable logic, improve readability, and ensure consistency across their Rules and Calculated Channels.
Additionally, Rules can generate Annotations or send notifications when their logical condition evaluates to true, helping users capture important events and trigger downstream workflows automatically. Each Rule benefits from versioning, allowing teams to track changes over time, maintain historical accuracy, and iterate safely as monitoring requirements evolve. Rules also support previews, enabling users to test their expressions against existing Run data before applying them to live or future telemetry streams, reducing risk and ensuring higher confidence in production rule behavior. Rules can be created, previewed, and managed either through the intuitive Sift UI or programmatically using the Sift API.
Rule configuration modes
While all Rules in Sift share the same structure and evaluation engine, they can be configured to behave differently depending on how they're created. These configurations are commonly referred to as Internal Rules and External Rules. Each mode affects how a Rule is evaluated, managed, and integrated into workflows.
Internal Rules
Internal Rules in Sift are created and managed directly through the Sift UI or API and are fully visible in the Rules tab of the Sift web application. These Rules can be edited, previewed, versioned, and managed over time, providing a persistent and flexible framework for ongoing telemetry monitoring, automated diagnostics, and data quality checks across Runs. When evaluated during a Run, internal Rules can generate Annotations and populate Reports, enabling teams to standardize analysis workflows and surface insights consistently across their datasets. Internal Rules are ideal when teams need reusable, centrally managed logic embedded directly into their telemetry observability workflows.
External Rules
External Rules in Sift are designed for programmatic, ad hoc usage, typically within automated workflows such as CI/CD pipelines. External Rules are created using the Sift API by calling the BatchUpdateRules endpoint with the isExternal flag set to true. Unlike internal Rules, External Rules are not visible in the Rules tab of the Sift web application and cannot be edited once created. They exist to allow flexible, dynamic evaluation of logic without permanently altering the set of Rules managed through the UI.
Reports that involve External Rules still display the External Rule's logic, and any Annotations generated by their evaluation are accessible through the Annotations tab and Explore views. To preview an External Rule before committing results, users can simulate its evaluation using the EvaluateRulesPreview endpoint, which previews the Annotations that would be generated without saving them. To apply an External Rule and create persistent Annotations and a Report, users can use the EvaluateRules endpoint.
External Rules provide flexibility for scenarios where teams need to evaluate custom logic on demand without managing those Rules permanently inside the Sift UI.
Live Rules: recommendations
Follow these best practices to configure your Live Rules in Sift for optimal performance minimizing data loss, improving evaluation accuracy, and ensuring consistent results.
| Recommendation | Description |
|---|---|
| Use batch rules for full data coverage | Use Batch Rules when you need to evaluate all available data at once using event timestamps. This approach is ideal for high-volume or mixed-frequency Channels. |
| Monitor data rates | Keep the total data rate across all Channels at or below 1,000 data points per second (1 kHz) to avoid silent data loss in Live Rules. |
| Combine low- and high-frequency Channels with care | Avoid pairing low-frequency Channels with high-frequency ones in Live Rules, as slower Channels may delay or suppress faster ones. |
Live Rules: limitations
When using live Rules in Sift, it's important to be aware of their current constraints to ensure accurate and reliable event processing. Below are the key limitations to consider:
| Limitation | Description |
|---|---|
| Data timing | Out-of-order or late-arriving data may not be processed correctly, leading to incomplete or inaccurate Rule evaluations. |
| Rule edits | Modifying a Rule while it is processing live data can cause mismatches between Rule versions, resulting in inconsistent evaluations. |
| Throughput limit | Live Rules support a maximum of 1,000 data points per second (1 kHz) across all Channels. Exceeding this limit may result in silent data loss. |
| Channel frequency | Using Channels with different update rates can delay or block evaluation. A slower Channel may throttle faster ones. |
Versioned Rules
Rules are stored as immutable versions. Every time a Rule is published or promoted, a new version is created. This enables full traceability of changes and supports reviewing or reverting to any previous state.
Promoting Rules
Promoting a Rule version means designating a prior version as the new current version. The system does not overwrite the current version; instead, it duplicates the selected version as a new version and sets it as the latest. This process maintains the integrity of the version history and ensures consistent tracking of all modifications.