`, for example `sensor_accel_0.x`. Logged status text is imported as `log_messages` Channels.
The ULog dialog has no Channel preview or per-Channel configuration. Every Channel is imported automatically. To rename, retype, or select Channels, see **Programmatic Channel selection** below.
#### Programmatic Channel selection
To select specific Channels via the REST API or Python client, use `UlogDataConfig` entries in the `data` field of `UlogConfig`.
When `data` is empty, Sift imports every detected Channel with its default name and type; when `data` is non-empty, only the listed Channels import, and each requires a full Channel config. Use `info_keys` and `param_keys` to import ULog info and parameter values as Run metadata; both require a Run. See [UlogConfig](/api/reference/protocol-buffers/data_imports#ulogconfig) in the protocol buffers reference.
The Python client requires the `ulog` extra: `pip install sift-stack-py[ulog]`.
### Timestamp formats
Supported timestamp formats for the Time Format setting.
**Relative timestamps**: If your timestamp column counts from a device boot time instead of epoch, select a relative format (such as `milliseconds`) and set **Relative Start time (UTC)** to the wall-clock time when the counter was at zero. Selecting an absolute format instead anchors the Run at the Unix epoch (January 1, 1970).
### Channel configuration table
#### CSV and Parquet
Shown for CSV and Parquet (Flat Dataset). Each non-timestamp column appears as one row.
#### HDF5
Each row maps one dataset to a Channel. The columns available are the same across all three HDF5 schemas.
### Data types
Supported data types for the Data Type field in the Channel configuration table.
**float vs. double**: `float` (32-bit) cannot represent every decimal value exactly, introducing small precision errors (for example, storing `170.9` as `170.89999...`). Use `double` for values where exact decimal precision matters.
## Export file formats
The Sift UI supports the following file formats for export:
## Export file settings
### Channel name display
Controls how Channel names appear in column headers.
### File splitting
Controls how exported data is partitioned across multiple files.
### Channel data grouping
Determines how Channels are organized within the exported files.
## Embedded Channel configurations
An embedded Channel configuration is a Channel and time configuration that Sift writes directly into an exported CSV or Parquet file.
On import, Sift reads this configuration and pre-fills the import configuration automatically, so a file exported from Sift can be re-imported with no manual setup. This is useful for moving data between Assets or environments.
You can also construct your own CSV or Parquet files using the shapes below and import them with no additional configuration.
### Enable on export
Enable **Embed Channel Configs** when exporting a file. See [Export data to a file](/documentation/export/export-data-from-a-file).
### CSV format
Sift inserts a configuration row directly after the header row. Column 0 holds the time configuration JSON; each data column holds a `ChannelConfig` JSON.
### Parquet format
Sift writes the configuration to Arrow field metadata. The `sift_time_config` key holds the time configuration JSON on the timestamp field; the `sift_channel_config` key holds the `ChannelConfig` JSON on each data field.
### Channel configuration examples
A Channel configuration is a JSON representation of the [ChannelConfig](/api/reference/protocol-buffers/channel_config) protobuf message.
```json theme={null}
{
"name": "engine.rpm",
"units": "rpm",
"description": "Engine speed",
"data_type": "CHANNEL_DATA_TYPE_DOUBLE",
"enum_types": [],
"bit_field_elements": [],
"metadata": []
}
```
For an enum Channel, `enum_types` maps each integer value to a name:
```json theme={null}
{
"name": "gearbox.state",
"units": "",
"description": "",
"data_type": "CHANNEL_DATA_TYPE_ENUM",
"enum_types": [
{
"name": "PARK",
"key": 0,
"is_signed": false
},
{
"name": "DRIVE",
"key": 1,
"is_signed": false
}
],
"bit_field_elements": [],
"metadata": []
}
```
### Time configuration examples
The time configuration is the time format and relative start time as JSON.
```json theme={null}
{
"time_format": "TIME_FORMAT_ABSOLUTE_RFC3339"
}
```
Relative formats set `relative_start_time`, the absolute time that offsets are measured from:
```json theme={null}
{
"time_format": "TIME_FORMAT_RELATIVE_NANOSECONDS",
"relative_start_time": "2026-07-24T00:00:00Z"
}
```
### Import behavior
Uploading a file that contains an embedded Channel configuration auto-populates the [Channel configuration table](#channel-configuration-table). Enum and bit field columns expand to reveal their editors if changes are needed.
# User-Defined Functions reference
Source: https://docs.siftstack.com/documentation/reference/user-defined-functions-settings
Settings, options, and behaviors for User-Defined Functions in Sift.
User-Defined Functions are reusable CEL expressions that can be called by name inside the expressions of Calculated Channels, Rules, and other User-Defined Functions.
They allow teams to define shared logic in one place and reference it across multiple expressions, ensuring consistency and reducing duplication.
When a User-Defined Function is updated, the change applies everywhere it is referenced without requiring updates to individual expressions.
## Settings
The following table describes each setting available when creating or editing a User-Defined Function.
## Input data types
Each input in a User-Defined Function must be assigned one of the following data types.
## Lifecycle actions
The following table describes the actions available for managing a User-Defined Function after it has been created.
## Behavior
The following table describes known constraints and behaviors to be aware of when working with User-Defined Functions.
# Detect and review issues in a Run
Source: https://docs.siftstack.com/documentation/review/detect-and-review-issues-in-a-run
Generate a Report, review flagged issues, and close them out
After completing this topic, you can generate a Report for a Run, investigate the issues it flags, and close them out from start to finish.
New to Detect and Review? Read the [Overview](/documentation/review/overview) to understand how Rules, Reports, Annotations, and Campaigns work together before proceeding.
## When to use this workflow
Use this workflow when you have a completed Run and want to evaluate it against a set of Rules, examine any flagged issues, and mark them as resolved. This is the core daily loop for anyone doing data review in Sift.
## Before you begin
* You have at least one Run in Sift. If you need to import or stream telemetry first, see [Import](/documentation/ingest/data-import/import-data-from-a-file) or [Stream](/documentation/ingest/stream).
* At least one Rule exists in your workspace. If no Rules have been created yet, see [Detect deviations automatically using Rules](/documentation/review/detect-deviations-automatically-using-rules).
## Complete the workflow
### Step 1: Generate a Report
1. In **Sift**, select **Runs** in the navigation.
2. In the **Runs** table, click the Run you want to review.
3. Click **Create report**.
4. Click the **Rules** tab.
5. In the **Rules** table, select the Rules you want to evaluate.
6. Click **Evaluate**.
Sift evaluates the selected Rules against the Run's telemetry and opens the Report when complete.
If your team has already set up a Report Template for this type of Run, use the **Report Template** tab in step 5 to select it. This applies a predefined set of Rules in one click. See [Set up a repeatable review checklist](/documentation/review/set-up-a-repeatable-review-checklist).
**Python client**: To generate Reports across many Runs, such as every Run in an Asset's history, use the [Python client](/documentation/reference/stream/client-library-reference) instead of repeating this workflow manually for each Run.
If the Report includes a Family Rule, Sift prompts you to resolve the alignment for the Run before evaluation. Map the Run's alignment configurations to the Family's T-0 so the comparison is time-synchronized correctly. See [Detect statistical deviations using Family Rules](/documentation/analyze/families/detect-deviations-from-a-historical-baseline).
### Step 2: Read the Rule summary panel
The Report opens with a list of Rules that were evaluated and violated. This means each Rule generated at least one Annotation. Rules that passed without generating any Annotations are not shown by default.
Each Rule in the list shows its current state:
* **Open**: no Annotations have been reviewed yet. This is the default.
* **Failed**: one or more Annotations were marked **Failed**.
* **Accepted**: all Annotations were marked **Accepted**.
Click any Rule to expand it and see its Annotations and the chart of the channel data that triggered it.
For a full definition of each state, see [Annotations reference](/documentation/reference/annotations-reference).
### Step 3: Open an Annotation
1. In the **Rules** section, click the Rule you want to investigate.
2. Click the Annotation listed under that Rule.
The Annotation details panel opens on the right. It shows the triggered channel values, the Rule expression that fired, Asset metadata, and a comment thread.
### Step 4: Update the Annotation state
In the Annotation details panel, use the status control to change the state from **Open** to either **Failed** or **Accepted**:
* Mark **Failed** if the telemetry represents a real problem that must be addressed.
* Mark **Accepted** if the finding was reviewed and determined to be valid or not requiring action.
State changes are recorded in the activity log.
### Step 5: Leave a comment
In the **Enter a comment** box at the bottom of the details panel, type your notes and click **Post**. Comments support rich text and appear alongside state changes in the activity log, giving the full context of the review decision.
### Step 6: Assign the Annotation (optional)
If the issue needs follow-up from a specific team member, use **Assigned to** to assign it. Assignment changes are logged alongside comments and state updates.
## Verify the result
After reviewing all Annotations in the Report:
* Every Rule in the Report should show a state of **Failed** or **Accepted**. None should remain **Open** if the review is complete.
* Each Annotation you investigated should have a state, a comment explaining the decision, and an assignee if follow-up is needed.
## Next steps
* To set up Rules that detect issues automatically in future Runs, see [Detect deviations automatically using Rules](/documentation/review/detect-deviations-automatically-using-rules).
* To create a template so you do not need to pick Rules manually next time, see [Set up a repeatable review checklist](/documentation/review/set-up-a-repeatable-review-checklist).
* To triage a large set of Annotations across multiple reviewers, see [Triage and close out flagged issues](/documentation/review/triage-and-close-out-flagged-issues).
## Reference
* [Annotations reference](/documentation/reference/annotations-reference)
* [Report chart display options](/documentation/reference/report-chart-display-options)
# Detect deviations automatically using Rules
Source: https://docs.siftstack.com/documentation/review/detect-deviations-automatically-using-rules
Automatically flag anomalies in your telemetry without manual inspection.
After completing this topic, you can write a CEL-based Rule that flags conditions in your telemetry data. Sift evaluates the Rule against a Run when you generate a Report and flags any deviations automatically.
**New to detect and review?** Read the [Overview](/documentation/review/overview) to understand how Rules, Reports, Annotations, and Campaigns work together before proceeding.
## When to use this workflow
Use this workflow when you want Sift to flag a specific condition in your telemetry data every time a Run is evaluated. For example, flagging when a channel exceeds a threshold, drops below a minimum, or a log message contains a known error string.
Rules are reusable. Once created, a Rule can be applied to any Run and included in Report Templates so your team does not need to reconfigure detection logic from scratch.
If your signal's acceptable range varies across tests rather than sitting at a fixed limit, use Family statistics as dynamic thresholds instead. See [Detect statistical deviations using Family Rules](/documentation/analyze/families/detect-deviations-from-a-historical-baseline).
## Before you begin
* You have at least one Asset and at least one Channel ingested into Sift. See [Channels](/documentation/reference/channels-reference) and [Assets](/documentation/ingest/stream/organize-streamed-data-into-assets-and-runs).
* Optional: You have an existing Run to use for previewing the Rule before saving it.
## Complete the workflow
This walkthrough uses a concrete example: flagging when a channel called `motor_temp` exceeds 120 degrees.
### Step 1: Create a new Rule
1. In **Sift**, select **Rules** in the navigation.
2. Click **New Rule**.
3. In **Rule name**, enter a name. For example, `Motor temp over threshold`.
4. Optional: In **Rule description**, enter a description.
5. In **Asset name**, enter or select the Asset that contains the channel you want to monitor.
6. Optional: In **Asset tag name**, enter a tag name to apply this Rule to all Assets with that tag rather than a single Asset.
### Step 2: Select input Channels
In the **Input Channels** list, select the channel(s) your Rule expression will reference. For this example, select `motor_temp`.
Sift assigns each selected Channel a shorthand variable in selection order: the first Channel becomes `$1`, the second `$2`, and so on. You use these variables in the expression in the next step.
You can select both raw telemetry channels and [Calculated Channels](/documentation/transform/create-a-derived-signal).
### Step 3: Write the expression
In the **Expression** box, enter a CEL expression that evaluates to `true` when the condition you want to flag is met. Reference input Channels using their assigned shorthand variables.
For this example, `motor_temp` was selected first, so it is `$1`:
```
$1 > 120
```
When this expression evaluates to `true` during a Run, Sift creates an Annotation at that point in time.
Keep the expression focused on a single logical condition.
Expression resources:
* For syntax help, operators, and Sift's built-in functions, see [Expression syntax](/documentation/reference/expression-syntax).
* For reusable logic shared across Rules, see [User-Defined Functions](/documentation/transform/reuse-expression-logic).
### Step 4: Preview the Rule against a Run
Before saving, preview the Rule against existing telemetry to confirm it behaves as expected.
1. In **Rule preview**, click **Select Run**.
2. In the **Runs** table, select one or more Runs that contain the channel you are monitoring.
3. Click **Preview N Runs**.
Sift evaluates the expression against the selected Runs and shows the Annotations that would be generated, without saving them.
4. Review the output to confirm the expression fires when and only when you expect it to.
**Preview timeout handling**: Rule preview has a 60-second execution limit and can time out on long Runs with high-rate Channels. If preview times out, save the Rule and evaluate it against the full Run instead of relying on preview to validate it first.
### Step 5: Decide between Rules and Ad Hoc Rules
Before publishing, decide whether this Rule should be a standard Rule or an Ad Hoc Rule:
* **Rules** are visible by clicking **Rules** in the navigation, can be edited and versioned over time, and are the right choice for ongoing telemetry monitoring.
* **Ad Hoc Rules** are created via the API for ad hoc or CI/CD use, are not visible in the UI, and cannot be edited after creation.
For most use cases, use a Rule. For CI/CD pipeline integration, see [Rules vs Ad Hoc Rules](/documentation/reference/rule-settings#rules-vs-ad-hoc-rules).
### Step 6: Save and publish the Rule
1. Click **Save**.
2. Optional: In **Version notes**, enter a note describing this version.
3. Click **Publish**.
Sift saves the Rule as version 1. Every subsequent edit creates a new immutable version, preserving the full change history.
## Verify the result
After publishing:
* The Rule appears under **Rules** in the navigation with the Asset and Channels you configured.
* You can click the Rule to view its expression, version history, and preview output.
* The Rule is now available to select when generating a Report or building a Report Template.
## Next steps
* To apply this Rule without manually selecting it each time, add it to a Report Template. See [Set up a repeatable review checklist](/documentation/review/set-up-a-repeatable-review-checklist).
* To review a Run using this Rule now, see [Detect and review issues in a Run](/documentation/review/detect-and-review-issues-in-a-run).
* To notify an external system the moment this Rule fires during live ingestion, see [Get alerted when a limit is breached](/documentation/manage/get-alerted-when-a-limit-is-breached).
* To manage Rule versions or promote an older version to current, see the version history section on the Rule's detail page.
* To write a Rule that flags deviations from a historical baseline rather than a fixed value, see [Detect statistical deviations using Family Rules](/documentation/analyze/families/detect-deviations-from-a-historical-baseline).
## Reference
* [Expression syntax](/documentation/reference/expression-syntax)
# Overview
Source: https://docs.siftstack.com/documentation/review/overview
Learn how Rules, Reports, Annotations, and Campaigns work together when reviewing a Run
After completing this topic, you can understand how Rules, Reports, Annotations, and Campaigns work together and know where to start when reviewing a Run.
## How Review works
Review in Sift is organized around a four-object pipeline:
**Rule → Report → Annotation → Campaign**
Each object has a distinct role. Understanding what each one does and why they are separate is the fastest way to orient yourself before starting a review.
## The review pipeline
### Rules
A Rule defines what to look for in your telemetry.
You write a logical condition using the [Common Expression Language (CEL)](https://cel.dev). For example, flagging when a channel exceeds a threshold or a log contains a specific string. Rules are reusable across Runs, versioned over time, and shared across your team.
Rules are the starting point for anyone setting up automated detection.
### Reports
A Report evaluates one or more Rules against a specific Run and collects the results in one place. When you generate a Report, Sift runs each Rule's expression against the Run's telemetry and shows you which Rules passed and which generated issues.
Reports are the primary workspace for reviewing a single Run. Most reviewers spend the majority of their time here.
### Annotations
When a Rule's condition evaluates to true during a Run, Sift creates an Annotation: a timestamped marker linked to the specific channel values that triggered the Rule. Annotations can also be created manually in Explore.
There are two types:
* **Phase Annotations**: informational markers for milestones such as "Engine Ignition" or "Max-Q". They have no status and cannot be assigned.
* **Data Review Annotations**: issue-tracking entries with a status workflow (`Open`, `Failed`, `Accepted`), an assignee field, and a comment thread. Use these when data needs investigation.
Use a Phase Annotation to mark a point of interest for reference, such as a milestone to align on or a boundary between sub-tests in a Run. Use a Data Review Annotation when the finding needs to be tracked to resolution by an assignee. See [Annotations reference](/documentation/reference/annotations-reference) for the full field and behavior differences between the two types.
Annotations represent the actual work of review: triaging findings, assigning ownership, and tracking resolution.
### Campaigns
A Campaign groups Reports from multiple Runs into a single workspace. Use a Campaign when you need to coordinate a review effort that spans several Runs. For example, all qualification tests for a hardware release.
Campaigns are not required for reviewing a single Run. Start with a Campaign only when you are managing a multi-run effort.
## Who uses each object
## Where to start
# Set up a repeatable review checklist
Source: https://docs.siftstack.com/documentation/review/set-up-a-repeatable-review-checklist
Create a standard set of automated checks that applies to every new dataset automatically.
After completing this topic, you can create a Report Template that applies a consistent set of Rules to every Run automatically.
New to Detect and Review? Read the [Overview](/documentation/review/overview) to understand how Rules, Reports, Annotations, and Campaigns work together before proceeding.
## When to use this workflow
Every time you generate a Report without a template, you manually select which Rules to evaluate. For a one-off review this process is fine, but for a recurring test scenario, hardware configuration, or standard review process, it becomes repetitive and error-prone.
A **Report Template** solves this by saving a named group of Rules. When you generate a Report, you select the template instead of individual Rules and Sift evaluates all the Rules in the template automatically.
Use this workflow when:
* You review the same type of Run repeatedly and always apply the same set of Rules.
* You want your team to apply a consistent set of checks without remembering which Rules to include.
* You are building a standard review process across a program or hardware configuration.
## Before you begin
* The Rules you want to include in the template already exist in Sift. See [Detect deviations automatically using Rules](/documentation/review/detect-deviations-automatically-using-rules) if you need to create them first.
## Complete the workflow
### Step 1: Create the template
1. In **Sift**, select **Report Templates** in the navigation.
2. Click **New Report Template**.
3. In **Untitled Report Template**, enter a name that describes the review scenario; for example, `Motor qualification checklist` or `Thermal system review`.
4. Optional: In the **Report Template tags** list, select tags to make the template easier to find.
5. Optional: In **Report Template description**, describe what this template checks and when to use it.
### Step 2: Add Rules to the template
1. Click **Add Rules**.
2. In the **Rules** table, select each Rule you want to include.
3. Click **Confirm selection**.
4. Click **Save new Report Template**.
The template is now available to anyone in your workspace when generating a Report.
### Step 3: Apply the template when generating a Report
1. In **Sift**, select **Runs** in the navigation.
2. In the **Runs** table, click the Run you want to review.
3. Click **Create report**.
4. Click the **Report Template** tab.
5. Click the template you want to apply.
6. Click **Evaluate**.
Sift evaluates all Rules in the template against the Run and opens the Report.
## When to create a new version of a template
Report Templates are not automatically versioned the way Rules are, so keep a consistent naming or tagging convention when your review requirements change.
Create a new template (or update the existing one) when:
* New Rules have been added to your workspace that belong in the standard checklist.
* A hardware revision changes which channels or thresholds are relevant.
* A Rule in the template has been archived or replaced with a newer version.
When you rerun an existing Report, Sift uses the same Rule versions that were originally evaluated; it does not pick up template changes automatically. To evaluate with updated Rules or a revised template, generate a new Report on the Run instead.
## Verify the result
After creating the template:
* It appears under **Report Templates** in the navigation.
* When generating a Report, the template appears in the **Report Template** tab and applies all its Rules in one step.
## Next steps
* To review a Run using your new template, see [Detect and review issues in a Run](/documentation/review/detect-and-review-issues-in-a-run).
* To triage the Annotations generated by the Report, see [Triage and close out flagged issues](/documentation/review/triage-and-close-out-flagged-issues).
## Reference
* [Annotations reference](/documentation/reference/annotations-reference)
* [Report chart display options](/documentation/reference/report-chart-display-options)
# Campaigns: track a multi-Run review effort
Source: https://docs.siftstack.com/documentation/review/track-a-multi-run-review-campaign
Organize Reports from multiple Runs into a Campaign and track overall review progress
After completing this topic, you can organize Reports from multiple Runs into a Campaign and track overall review progress in one place.
New to Detect and Review? Read the [Overview](/documentation/review/overview) to understand how Rules, Reports, Annotations, and Campaigns work together before proceeding.
## When to use this workflow
A **Campaign** groups Reports from multiple Runs into a single workspace. Use a Campaign when:
* You are reviewing all Runs from a test program. For example, all qualification tests for a new hardware release.
* Multiple reviewers are working in parallel across different Runs and you need a single view of overall progress.
* You need to filter and triage Annotations across Runs by assignee, status, Asset, or Rule without opening each Report individually.
**Use a single Report instead** when you are reviewing one Run or a small number of Runs where a shared workspace adds no meaningful benefit.
## Before you begin
* Two or more Reports already exist for the Runs you want to include. If not, see [Detect and review issues in a Run](/documentation/review/detect-and-review-issues-in-a-run) to generate Reports.
* You know the scope of the Campaign (which Runs and Reports are in scope).
## Complete the workflow
### Step 1: Create the Campaign
1. In **Sift**, select **Campaigns** in the navigation.
2. Click **New Campaign**.
3. In the **Campaign name** box, enter a name that describes the review effort, for example, `Thermal system qualification Q2` or `Motor drive endurance test series`.
4. Replace **No description** with a description of the Campaign scope.
5. Click **Add report**.
6. In the **Reports** table, click the Reports you want to include.
7. Click **Confirm selection**.
8. Click **Save**.
### Step 2: Add Reports from additional Runs
As new Runs complete and Reports are generated, add them to the Campaign:
1. In **Campaigns**, click the Campaign.
2. Click **Edit reports**.
3. In the **Reports** table, click the Reports you want to add.
4. Click **Confirm selection**.
**Adding Multiple Runs**: Adding Reports to a Campaign requires manually selecting each Report in the UI. To add a large number of Runs, such as an Asset's entire Run history, use the [Python client](https://sift-stack.github.io/sift/python/latest/) to add Runs to the Campaign instead of adding them one by one.
### Step 3: Track progress using Annotation filters
Inside the Campaign, use the **View Annotations** option (in the left navigation) to see all Annotations across every included Report in one place. Apply filters to monitor progress:
* Filter by **Status** to see how many Annotations remain Open across the entire Campaign.
* Filter by **Assignee** to see each reviewer's outstanding work.
* Filter by **Run**, **Asset**, or **Rule** to isolate a specific area of concern.
The **View/edit Campaign** section provides a summary of Annotation status that can be grouped by Run, Report, Assignee, or Rule, giving a high-level view of where the review stands.
### Step 4: Decide when the Campaign is done
A Campaign has no automatic completion state. Use the following criteria to determine when the effort is complete:
* All Annotations in the Campaign have a state of **Failed** or **Accepted**, and none remain **Open**.
* Any Annotations marked **Failed** have been assigned and have a documented next action.
* Stakeholders have been notified or reports shared as needed.
Once complete, the Campaign serves as a historical record of the review effort.
## Verify the result
After completing the Campaign setup:
* The Campaign appears under **Campaigns** in the navigation.
* The **View Annotations** section shows all Annotations from all included Reports in a single filterable list.
* Team members can navigate to the Campaign to see their assigned issues without opening individual Reports.
## Next steps
* To triage specific Annotations within the Campaign, see [Triage and close out flagged issues](/documentation/review/triage-and-close-out-flagged-issues).
* To share a Report with stakeholders outside the Campaign, open the Report, hover over its title, and click **Share** to copy a link.
* To tag the Campaign with a requirement, program, or hardware identifier so it is easy to find later, see [Organize resources with Metadata](/documentation/manage/organize-and-filter-resources-with-metadata).
## Reference
* [Annotations reference](/documentation/reference/annotations-reference)
# Triage and close out flagged issues
Source: https://docs.siftstack.com/documentation/review/triage-and-close-out-flagged-issues
Work through a Report's Annotations: assign, update, and resolve issues to drive a review to completion
After completing this topic, you can work through a Report's Annotations systematically: assigning, updating, and closing out every flagged issue.
New to Detect and Review? Read the [Overview](/documentation/review/overview) to understand how Rules, Reports, Annotations, and Campaigns work together before proceeding.
## When to use this workflow
Use this workflow when a Report has been generated and you need to work through its Annotations systematically:
1. Filtering by status or owner
2. Distributing work across teammates
3. Adding evidence
4. Driving every issue to a final state
This is the day-to-day workflow for reviewers. It picks up where [Detect and review issues in a Run](/documentation/review/detect-and-review-issues-in-a-run) leaves off and covers the full range of tools available for managing Annotations at scale.
## Before you begin
* At least one Report exists for the Run you are reviewing. If not, see [Detect and review issues in a Run](/documentation/review/detect-and-review-issues-in-a-run).
* You have the team members' names available if you plan to assign Annotations.
## Complete the workflow
### Filter Annotations by state or assignee
**Annotations** in the navigation provides a workspace-level view of all Annotations across all Runs. Use the filters at the top to narrow down to the issues you want to work on:
* Filter by **Status** (Open, Failed, Accepted) to see only unreviewed or in-progress items.
* Filter by **Assignee** to see only the issues assigned to a specific team member.
* Filter by **Asset**, **Run**, or **Rule** to focus on a specific scope.
You can also access Annotations directly within a Report by clicking a Rule and then an Annotation in the Rules section.
### Assign issues to teammates
1. In **Sift**, select **Annotations** in the navigation.
2. In the **Annotations** table, click the Annotation you want to assign.
3. In the **Assigned to** selector, choose a team member.
Assignment changes are recorded in the activity log alongside comments and state transitions, so the full history of ownership is preserved.
### Add a comment
1. In the Annotation details panel, locate the comment thread.
2. Type your notes in the comment box. Comments support rich text.
3. Click **Post**.
Use comments to document your reasoning when making a review decision, flag follow-up actions, or provide context for the next reviewer.
### Attach a file to an Annotation
To attach a file directly to an Annotation:
1. In **Sift**, select **Annotations** in the navigation.
2. In the **Annotations** table, click the Annotation you want to attach a file to.
3. In the **File attachments** section, click **Attach**.
4. Upload the file.
To attach a file to a specific comment on an Annotation:
1. In the comment, click **Attach**.
2. Upload the file.
3. Click **Post**.
For supported file types and size limits, see [Annotations reference](/documentation/reference/annotations-reference).
### Download an attached file
To download a file attached to an Annotation:
1. In the **File attachments** section, expand it.
2. Hover over the file you want to download.
3. Click **Download**.
To download a file attached to a comment, locate the comment, hover over the file, and click **Download**.
### Move an Annotation through its states
In the Annotation details panel, use the status control to transition the Annotation:
* **Open** → **Failed**: the telemetry represents a real problem that must be addressed.
* **Open** → **Accepted**: the finding was reviewed and determined to be valid or not requiring action.
* **Failed** or **Accepted** → **Open**: reopen the Annotation if new information changes the assessment.
For the full list of valid state transitions, see [Annotations reference](/documentation/reference/annotations-reference).
Phase Annotations do not support state changes or assignment. Only Data Review Annotations have the status workflow described above.
### Archive a resolved Annotation
Once an Annotation is fully resolved and no longer needs to appear in the active view, archive it:
1. In the Annotation details panel, scroll to the bottom and click **Archive**.
2. Confirm in the dialog.
Archived Annotations are soft-deleted. They no longer appear in the tree or on charts but can be restored. To restore an archived Annotation, locate it in the archived view and click **Restore**.
## Verify the result
A triage pass is complete when:
* Every Data Review Annotation in the scope of your review has a state of **Failed** or **Accepted**, and none remain **Open**.
* Issues requiring follow-up have an assignee and a comment explaining the next action.
* Supporting evidence (screenshots, logs, or other files) is attached to Annotations where context is needed.
## Next steps
* To track progress across a multi-Run effort, see [Track a multi-run review campaign](/documentation/review/track-a-multi-run-review-campaign).
* To share a Report with a stakeholder, open the Report, hover over its title, and click **Share** to copy a link to your clipboard.
## Reference
* [Annotations reference](/documentation/reference/annotations-reference)
* [Annotations reference](/documentation/reference/annotations-reference)
# Calculated Channels: create a derived signal
Source: https://docs.siftstack.com/documentation/transform/create-a-derived-signal
Use a Calculated Channel to compute a new derived signal without modifying the original telemetry data.
After completing this workflow, you can create a Calculated Channel that derives a new signal from one or more existing Channels, without modifying the original telemetry.
## Before you begin
* You are familiar with how Sift organizes data into Assets, Channels, and Runs.
* If not, see [Understand Assets, Channels, and Runs](/documentation/get-started/data-model).
* You have at least one Asset with ingested telemetry data and know which Channels you want to use as inputs.
* You have at least one Run that includes the input Channels you plan to use, so you can verify the derived signal after creating it.
## How Calculated Channels work
A derived signal is created using a Calculated Channel. A Calculated Channel is a new signal computed from one or more existing Channels using a [CEL expression](/documentation/reference/expression-syntax). Sift evaluates the expression against each incoming data point and stores the result as a separate Channel, leaving the original telemetry intact.
Each input Channel is assigned a shorthand variable (`$1`, `$2`, and so on) based on the order you select them. You use these variables to write the expression. Input Channels can be raw Channels or other Calculated Channels (up to 10 levels of nesting are supported).
For example, say you want to compute the power draw of a system from its voltage and current Channels. Instead of exporting the raw data and computing it manually, you select both Channels and write `$1 * $2` as the expression.
## Create a Calculated Channel
1. Select **Calculated Channels** in the navigation.
2. Click **New Calculated Channel**.
3. In the **Calculated Channel name** box, enter a name for the derived signal.
4. In the **Asset name** list, select the Asset associated with your input Channels.
5. In the **Input Channels** list, select one or more Channels to use in the expression.
* To use raw telemetry, select one or more existing Channels.
* To build on a previously derived signal, select an existing Calculated Channel.
* You can mix raw Channels and Calculated Channels as inputs.
6. In the **Query** box, write a [CEL expression](/documentation/reference/expression-syntax) using the assigned input variables.
* To call shared logic, reference a User-Defined Function directly in the expression.
7. Optional: In the **Units** box, enter the unit of measurement for the derived signal.
8. Optional: In the **Calculated Channel(s) Preview** section, click **Select Run** to select a Run, then review the plot to confirm the derived signal behaves as expected against the input Channels.
9. Click **Save**.
**Asset scope**: A Calculated Channel is scoped to the Asset you select in step 4. It does not appear when you open a different Asset in Explore. To apply the same Calculated Channel to multiple Assets, set the **Asset tag name** field. Any Asset tagged with that value can use the channel. For all available settings, see [Calculated Channel reference](/documentation/reference/calculated-channel-settings).
## Verify the created Calculated Channel
You can confirm the Calculated Channel is working correctly in two ways:
* **During creation**: In the **Calculated Channel(s) Preview** section, select a Run to confirm the expression produces the expected values against the input Channels.
* **After saving**: Select **Runs** in the navigation, then search for and select the Run that contains the input Channels used in the expression, click **Explore**, then in the **Calculated Channels** tab, click your new Calculated Channel to plot it alongside the source Channels.
If the values look unexpected, return to the Calculated Channel, review the expression, and use the preview to validate changes before saving.
## Next steps
* [Reuse expression logic](/documentation/transform/reuse-expression-logic)
* Once you have created a derived signal, learn how to define expression logic once as a User-Defined Function and reference it across multiple Calculated Channels and Rules to avoid rewriting the same expression each time.
## Reference
* [Calculated Channels reference](/documentation/reference/calculated-channel-settings)
* [CEL expression](/documentation/reference/expression-syntax)
# Reference files in expressions
Source: https://docs.siftstack.com/documentation/transform/reference-files-in-expressions
Use file data from Assets and Runs in Rule and Calculated Channel expressions instead of hardcoding values.
After completing this workflow, you can reference external file data directly in Rule and Calculated Channel expressions to compare telemetry against thresholds, apply calibration values, or use configuration data that changes between Runs without hardcoding values in your expressions.
## Before you begin
* You have an Asset or a Run with at least one file attached in a [supported format](/documentation/reference/expression-syntax#supported-file-formats).
* You have an existing Rule or Calculated Channel to edit, or you are ready to create a new one.
## How file references work
Rule and Calculated Channel expressions can access data from files attached to an Asset or a Run using two functions: `assetFile()` and `runFile()`.
**Live Rules**: Rules that reference files can't be used in live Rule evaluation. File functions are not supported in live Rule evaluation.
* `assetFile("filename.format")` reads a file attached to the Asset the Rule or Calculated Channel is configured for.
* The file is verified when the expression is created or edited.
* Use this for data that stays the same across Runs, such as hardware limits, calibration offsets, or hardware specifications.
* `runFile("filename.format")` reads a file attached to the Run being evaluated.
* The file is resolved at evaluation time, when a Rule is evaluated against a Run, or when a Calculated Channel is evaluated against a Run.
* Use this for data that changes between Runs, such as test parameters, environmental conditions, or per-session acceptance criteria.
Once a file is loaded, you access specific values using [bracket notation](/documentation/reference/expression-syntax#bracket-notation). The syntax depends on the file format. For supported formats and syntax, see [File functions](/documentation/reference/expression-syntax#file-functions).
## Reference a file in an expression
1. Attach a file to the Asset or the Run you want the expression to reference.
2. Create a new Rule or Calculated Channel, or open an existing one to edit.
3. In the **Expression** editor, use `assetFile("filename")` or `runFile("filename")` to reference the file by name.
4. Use [bracket notation](/documentation/reference/expression-syntax#bracket-notation) to index into the specific value.
5. Verify the expression:
* **Rules**: Preview the Rule against an existing Run to confirm the expression resolves correctly.
* **Calculated Channels**: Preview the Calculated Channel against a Run to confirm the expression resolves correctly.
## Examples
### Rules
### Calculated Channels
## Reference
* [CEL expression](/documentation/reference/expression-syntax#file-functions)
# Reuse expression logic
Source: https://docs.siftstack.com/documentation/transform/reuse-expression-logic
Define expression logic once and reference it across multiple Calculated Channels, Rules, and User-Defined Functions.
After completing this workflow, you can define a CEL expression once as a User-Defined Function and call it by name in Calculated Channels, Rules, and other User-Defined Functions, keeping logic consistent and reducing duplication across your workspace.
## Before you begin
* You are familiar with how Sift organizes data into Assets, Channels, and Runs.
* If not, see [Understand Assets, Channels, and Runs](/documentation/get-started/data-model).
* You know the CEL expression logic you want to reuse. If you need help writing expressions, see [Expression syntax](/documentation/reference/expression-syntax).
## How reusable expression logic works
A **User-Defined Function** is a reusable CEL expression that can be called by name inside the expressions of Calculated Channels, Rules, and other User-Defined Functions. It does not produce a signal on its own.
It acts as a building block that encapsulates a piece of logic, such as a threshold calculation or a normalization formula, that you want to apply consistently across multiple places in your workspace.
Each input in a User-Defined Function is assigned a shorthand variable (`$1`, `$2`, and so on) based on the order you define them. You use these variables to write the expression, just as you would in a Calculated Channel.
### Example
For example, say you want to convert temperature readings from Celsius to Fahrenheit across multiple assets.
Instead of writing `($1 * 9/5) + 32` in every Calculated Channel and Rule separately, you define it once as a User-Defined Function called `celsius_to_fahrenheit` with the expression `($1 * 9/5) + 32`.
You can then reference it in Calculated Channels, Rules, and other User-Defined Functions:
* In a Calculated Channel: `celsius_to_fahrenheit($1)`
* In a Rule: `celsius_to_fahrenheit($1) > 200`
If the conversion logic changes, you update the function once and the change applies everywhere it is referenced automatically.
## Create a User-Defined Function
1. Select **User-Defined Functions** in the navigation.
2. Click **New Function**.
3. In the **User-defined function name** box, enter a name for the function. For naming requirements, see [User-Defined Functions settings](/documentation/reference/user-defined-functions-settings#settings).
4. Optional: In the **Description** box, enter a description of what the function does.
5. In the **Input Channels** box, select a data type for the first input (`$1`).
* To add more inputs, click **Add input Channel** and select a data type for each additional input.
6. In the **Expression** box, write the CEL expression using the defined input variables. For supported syntax, see [Expression syntax](/documentation/reference/expression-syntax).
7. Optional: In the **Preview** section, click **Select Run** to select a Run, then review the plot to confirm the function produces the expected values against the input Channels.
8. Click **Save**.
## Verify the function
You can confirm the User-Defined Function is working correctly in two ways:
* **During creation**: In the **Preview** section, select a Run, and review the output to confirm the expression produces the expected values before saving.
* **After saving**: The function appears in the **User-Defined Functions** section of the Expression Syntax panel. Call it in a Rule, Calculated Channel, or other User-Defined Function to confirm it behaves as expected.
## Reference
* [User-Defined Functions reference](/documentation/reference/user-defined-functions-settings)
* [CEL expression](/documentation/reference/expression-syntax)
# Troubleshooting
Source: https://docs.siftstack.com/documentation/troubleshoot/troubleshooting
Learn how to fix common configuration issues.
This guide covers the common issues users encounter and provides steps to resolve them quickly.
If your issue is not listed here, please contact support with details about your browser, operating system, and the steps taken before the issue occurred.
To contact support in Sift, click your profile, and then click **Customer Support**.
If the issue occured during a Run, include a sharelink of the Run or Asset/Run ID.
## Plot and visualization issues
These are the most common types of issues and are typically related to rendering, zooming, or large datasets.
### Zoom does not work (X or XY zoom is unresponsive)
This usually happens when:
* The dataset being plotted is very large
* The plot has not fully finished loading
* The browser is temporarily overloaded
* FFT or high-resolution rendering is enabled
In some cases, the browser tab might appear frozen while processing data.
1. Wait 10–15 seconds to allow the plot to fully load.
2. Refresh the browser tab.
3. Reduce the time range being plotted.
4. Plot fewer Channels at once.
5. Disable FFT (if enabled).
6. Close other heavy browser tabs and try again.
If the issue persists, provide support with:
* Browser and OS version
* Time range plotted
* Number of Channels
* Whether FFT was enabled
### Plot freezes or browser tab locks up
This typically occurs when:
* Plotting a very large time range
* Viewing high-sample-rate data
* Plotting many Channels simultaneously
* Rendering FFT on large datasets
* The browser might become temporarily unresponsive while processing.
1. Refresh the page.
2. Reduce the plotted time range.
3. Plot fewer Channels.
4. Disable FFT (if enabled).
5. Try a different browser to confirm whether the issue is browser-specific.
If reproducible, share the exact plot configuration or link with support.
### Error after removing a Channel from a plot
This might occur when:
* A removed Channel is still referenced in the plot state
* A shared link contains outdated Channel references
* A rendering state mismatch occurs after dynamic updates
1. Refresh the page.
2. Re-add the Channel and then remove it again.
3. Recreate the plot from scratch.
4. If using a shared link, regenerate the link.
5. Clear browser cache and retry.
If the error continues, provide a share link and reproduction steps to support.
### Annotations obscure plotted data
Annotations might render above data layers, which can cause labels or markers to visually block parts of the timeseries.
1. Toggle annotation visibility off (if available).
2. Zoom in to reduce visual overlap.
3. Reduce annotation density where possible.
4. Capture a screenshot and report if this significantly affects usability.
### Metadata box values appear swapped or incorrect
In rare cases, the UI rendering state might temporarily mismatch labels and values, particularly after zooming or rapid interactions.
1. Zoom slightly in or out.
2. Refresh the page.
3. Reopen the plot in a new tab.
4. Clear browser cache if the issue persists.
If the issue consistently reproduces, report with:
* Screenshot
* Browser version
* Plot configuration
## Channel and layout issues
### Long Channel names overlap or break layout
Very long Channel names might exceed the available UI width, leading to:
* Overlapping text
* Truncated names
* Misaligned layout elements
1. Resize your browser window.
2. Hover over truncated names to view the full Channel name (if tooltip is available).
3. Rename Channels to shorter aliases (if appropriate).
4. Report cases where the layout breaks completely.
See Asset [naming conventions](/documentation/ingest/stream/organize-streamed-data-into-assets-and-runs#naming-conventions) and [Channel naming restrictions](/documentation/reference/channels-reference#naming-restrictions).
### Channel labels appear duplicated or formatted incorrectly
This might occur when:
* Channel metadata includes overlapping naming attributes
* Asset or Run information is appended multiple times
* A UI formatting issue is triggered
1. Refresh the page.
2. Remove and re-add the Channel.
3. Verify the Channel's metadata configuration.
4. If persistent, provide the Channel name and Asset details to support.
## Rule configuration issues
### Rule shows "Error" instead of "Failure"
This occurs when the Rule itself is misconfigured, rather than the data violating a correctly written Rule.
* Rules are written as CEL expressions. When Rule logic is discussed in this section, it is referring to CEL expression validity. See [Rules overview](/documentation/review/detect-deviations-automatically-using-rules).
* See [live Rule limitations](/documentation/review/detect-deviations-automatically-using-rules#live-rules-limitations).
There are two distinct outcomes when a Rule Runs:
* **Failure**: The Rule logic is valid, but the data violates it.
* **Error**: The Rule configuration is invalid or cannot execute properly.
An error state is typically caused by:
* Invalid or incomplete Rule logic
* Incorrect threshold expressions
* Referencing fields or Channels that do not exist
* Invalid arguments in Rule conditions
* Unsupported comparison types
In these cases, the system cannot properly evaluate the Rule, so it enters an error state rather than reporting a failure.
1. Review the Rule configuration carefully.
2. Confirm all referenced Channels or fields exist and are spelled correctly.
3. Verify thresholds and comparison operators are valid.
4. Ensure the Rule logic matches the intended data type (numeric vs boolean vs string).
5. Save the Rule and test it on a small, known dataset.
6. If the error persists, copy the error message and provide:
* Rule configuration
* Asset or Run used
### Rule never evaluates or shows no results
If a Rule appears to never run, or produces no failures or results, it is often due to configuration scope rather than a system issue.
Common causes include:
* The Rule is attached to the wrong Asset
* No data exists in the selected time range
* The Rule is scoped to Runs that do not contain matching data
* Filters or conditions prevent the Rule from executing
* The Rule is not enabled or not associated with an active evaluation context
### Rule errors when a Run is missing a referenced Channel
Sift requires every Channel referenced in a Rule to exist on the Run being evaluated. If a Channel only reports data under certain conditions, such as a mode that is not always active, Runs where that Channel never reports have no Channel to reference, and the Rule cannot evaluate.
Split the logic into two Rules:
1. A general Rule that does not reference the conditional Channel, so it evaluates on every Run.
2. A Rule that adds the conditional Channel as an extra input, so it only evaluates on Runs where that Channel is present.
Run both Rules via a Report. On Runs where the conditional Channel is absent, the second Rule shows a per-Rule error; this is expected, not a failure of the first Rule.
Preview the two Rules one at a time. Previewing both together against a Run that is missing the conditional Channel can hide the other Rule's result in the Preview panel.
## Python client (sift-stack-py) issues
### Import fails in a PyInstaller windowed EXE
When `sift-stack-py` is imported inside a PyInstaller windowed build (compiled with `--noconsole` or `--windowed`), the process has no attached console, so Python sets `sys.stdout` and `sys.stderr` to `None`. An internal dependency used by `sift-stack-py` attempts to write to these streams on import and raises an error before any Sift code runs.
This affects all `sift-stack-py` versions prior to the fix released in response to this issue.
**Recommended**: Upgrade to the latest version of `sift-stack-py`, which handles `None` streams internally.
```bash theme={null}
pip install --upgrade sift-stack-py
```
**Workaround for older versions**: Add the following snippet at the top of your entry-point module, before any `sift-stack-py` import:
```python theme={null}
import sys
import os
if sys.stdout is None:
sys.stdout = open(os.devnull, "w", encoding="utf-8")
if sys.stderr is None:
sys.stderr = open(os.devnull, "w", encoding="utf-8")
```
This redirects the missing streams to `/dev/null` so the dependency initializes without error. Place this code before any `from sift...` or `import sift` statements.
## Streaming issues
### Ingestion fails with a bytes value size limit error
A single `bytes` (binary) Channel value is limited to **2 MB** (2,097,152 bytes). Sift rejects any larger value during ingestion with an error such as `bytes value is too large: received bytes, limit is 2097152`. This limit applies to each individual data point and is independent of the overall ingestion message size.
For binary payloads larger than 2 MB, store the data as a [file attachment](/documentation/review/triage-and-close-out-flagged-issues#attach-a-file-to-an-annotation) rather than streaming it as a `bytes` Channel value.
## File import issues
### Import fails with gRPC RESOURCE\_EXHAUSTED error
This error occurs when importing a Parquet file that contains a large number of Channels. The import configuration Sift generates for the file can exceed the 10 MiB gRPC message size limit even when the data file itself is smaller than that threshold.
```
StatusCode.RESOURCE_EXHAUSTED
grpc: received message larger than max (X vs. 10485760)
```
The error is thrown by the [CreateDataImportFromUpload](/api-reference/dataimportservice/createdataimportfromupload#createdataimportfromupload) endpoint. The configuration size scales with the number of Channels defined in the file, not the volume of data.
Split the Parquet file into multiple smaller files, each containing a subset of the Channels, then import each file separately targeting the same Asset and Run.
For example, if your file contains 500 channels:
1. Create **File A** with channels 1–250.
2. Create **File B** with channels 251–500.
3. Import File A and File B as separate imports, both mapped to the same Asset and Run.
Sift is working on increasing the gRPC message size limit to accommodate files with large channel counts in a future release.
# Learning path
Source: https://docs.siftstack.com/learning-path/overview
Begin your Sift journey and grow your confidence with step-by-step tutorials
Get your data into Sift from files or a live stream.
Explore different methods for ingesting historical data.
Explore the different streaming options available for live data ingestion.
Use Explore's Panels to spot patterns and dig into the details that matter.
Discover Sift's tools to explore, navigate, inspect, and compare your telemetry.
Detect issues automatically and get notified the moment they happen.
Use Sift's Rules and Reports features to automatically detect anomalies and summarize insights.
Enable real-time notifications by connecting Rule triggers to external systems using Webhooks.
Derive and reuse logic across your telemetry.
Use Calculated Channels to generate new signals and metrics from your existing telemetry.
Centralize reusable transformation logic using User-Defined Functions.
Organize your workspace so resources stay easy to find.
Use Metadata to add structured context to your workspace resources for faster searching and better organization.
# Conclusion
Source: https://docs.siftstack.com/learning-path/tutorial-1/conclusion
In this tutorial, you learned how to import and visualize historical data in Sift. You explored how Sift organizes time-series telemetry and developed an understanding of how historical data can be ingested.
You then imported a CSV file using one of these approaches and visualized the data in Sift's Explore, learning how to explore and analyze historical measurements once they are available in Sift.
# Tutorial 1: Import and visualize historical data in Sift
Source: https://docs.siftstack.com/learning-path/tutorial-1/overview
Explore different methods for ingesting historical data
By the end of this tutorial, you'll be able to import historical data into Sift using the method that best fits your workflow and visualize that data to uncover trends and insights. We'll walk through the different import options Sift offers, then put them into practice by ingesting a dataset and exploring it with Sift's visualization tools.
## Objectives
* Understand the different options Sift offers to import historical data into Sift
* Learn how to visualize imported data
## Prerequisites
* Download the following CSV file: [tutorial-1-data.csv](https://drive.google.com/uc?export=download\&id=1fLNXV0bM1AbO9Z9T4DJUH2YJo1Qz6SNE)
# Step 1: Understand how Sift organizes telemetry data
Source: https://docs.siftstack.com/learning-path/tutorial-1/step-1
## Overview
Before covering the different import options, let's review how Sift organizes time-series data.
## Assets, Channels, and Runs
Sift uses three core building blocks: [Assets](../../documentation/ingest/stream/organize-streamed-data-into-assets-and-runs), [Channels](../../documentation/reference/channels-reference), and [Runs](../../documentation/reference/runs-reference). These building blocks help structure telemetry data, making it easier to explore, analyze, and reuse it across different experiments or observation periods.
* **Assets**: These represent physical or virtual systems that generate data.
* **Channels**: These represent individual time-series signals recorded from an Asset.
* **Runs**: These represent time windows or observation sessions during which data is collected from one or more Channels.
# Step 2: Discover methods to import historical data
Source: https://docs.siftstack.com/learning-path/tutorial-1/step-2
## Overview
Before learning how to visualize historical data in Sift, this step introduces the different methods available for **importing historical (backfill) datasets**. Historical data refers to data that was collected in the past and uploaded to Sift after the fact, rather than streamed in real time.
## Options
Sift supports multiple methods for importing historical data. Each method is suited to different workflows and supports specific file formats, as shown below:
# Step 3: Understand how to visualize imported historical data
Source: https://docs.siftstack.com/learning-path/tutorial-1/step-3
## Overview
Now that you better understand the different options Sift offers for importing historical data, in this step, let's import a CSV file and explore how to visualize it in Sift. The CSV contains data from a low Earth orbit (LEO) satellite.
## Import a CSV
Let's import a CSV:
1. Select **Runs** in the navigation, then select **Import data**.
2. Upload the [tutorial-1-data.csv](https://drive.google.com/uc?export=download\&id=1fLNXV0bM1AbO9Z9T4DJUH2YJo1Qz6SNE) file.
3. Click the **Asset** toggle to the **New** position.
4. In the **Asset name** box, enter:
```
fl_tutorial_1_asset
```
5. In the **tutorial-1-data.csv** box, edit the Run name with the following:
```
fl_tutorial_1_run
```
6. Replace `fl` (in `fl_tutorial_1_asset` and `fl_tutorial_1_run`) with your initials (first and last name). This helps avoid naming conflicts if others are also completing the tutorial on the same Sift instance.
7. Click **Upload**.
8. Click **Close**
## Visualize data
Now that we have imported the dataset, let's visualize the imported CSV.
1. In the **Run name or description** box, enter the name of the created Run (for example, **fl\_tutorial\_1\_run**) and select it.
2. Click **Explore**.
Explore is Sift's workspace for working through high-rate timeseries data to uncover root causes, spot trends, and get answers faster.
It gives engineers one place to visualize telemetry across Panels, perform calculations, compare telemetry across tests, and share a snapshot view with colleagues. Engineers go from “something happened” to “here’s what happened and here’s the proof” without exporting telemetry or switching tools.
3. In the **Channels** tab, click the following Channels:
1. Under the **motor\_b** section, click the following Channels:
* **current**
* **encoder**
# Conclusion
Source: https://docs.siftstack.com/learning-path/tutorial-2/conclusion
In this tutorial, you reviewed the different ways to stream live data into Sift and learned how to reason about which streaming method best fits a given use case.
You also ran a live streaming example using one of these methods and observed the incoming data in the Sift interface. By visualizing live telemetry as it was sent to Sift, you learned how to explore and monitor real-time data, preparing you to confidently work with live streams and apply these concepts to your own systems.
# Tutorial 2: Stream and visualize live data in Sift
Source: https://docs.siftstack.com/learning-path/tutorial-2/overview
Explore the different streaming options available for live data ingestion
By the end of this tutorial, you will be able to stream live data into Sift using the approach that best fits your use case and begin exploring that data in real time. We will walk through the different streaming options Sift provides, then put one into practice so you can see your data flowing into Sift and ready for analysis.
## Objectives
* Understand the different options Sift offers to live-stream data into Sift
* Learn how to observe live-streamed data in Sift
## Prerequisites
* Basic understanding of Python
* A Sift API key and your Sift gRPC and REST base URLs
* This tutorial describes how to create an API key and obtain your URLs.
# Step 1: Discuss options to live stream data
Source: https://docs.siftstack.com/learning-path/tutorial-2/step-1
## Overview
Before we learn how to observe live-streamed data in Sift, let's explore in this step the different options Sift offers for live-streaming data into Sift.
## Options
Sift supports multiple options for ingesting data as it is generated in real time. The following table summarizes each live-streaming method, including the type of data it accepts and the scenarios in which it is typically used.
# Step 2: Understand how to observe live-streamed data
Source: https://docs.siftstack.com/learning-path/tutorial-2/step-2
## Overview
Now that you understand the different methods available for live-streaming data into Sift, let's explore how to observe live-streamed data in Sift.
For this tutorial, we'll simulate live-streaming by running a Python script that sends robotic vehicle telemetry (such as velocity and internal temperature) to Sift via the `sift_client`. This data is generated every 0.5 seconds and is used to demonstrate how streaming data can be observed in Sift.
## Live-streamed data
Follow the steps below to live-stream robotic vehicle telemetry data.
1. Clone the Sift repository and navigate to the ingestion tutorial directory:
```bash theme={null}
git clone https://github.com/sift-stack/sift.git
cd sift/python/examples/ingestion-tutorial
```
2. Create and activate a Python virtual environment, then install the required packages:
```bash theme={null}
python3.11 -m venv venv
source venv/bin/activate
pip install -r requirements.txt
```
3. Create a `.env` file and provide your Sift instance URLs and API key:
```bash theme={null}
touch .env
```
```bash theme={null}
SIFT_API_KEY=your_api_key_here
SIFT_GRPC_URL=https://your-grpc-url
SIFT_REST_URL=https://your-rest-url
```
To obtain your instance URLs and an API key:
1. In **Sift**, click your profile icon, which shows the first initial of your account name.
2. Select **Manage**.
3. In the **Manage** navigation menu, click **API Keys**.
4. Click **+ Create API Key**.
5. In the **Name** box, enter a name for the API key.
6. In the **User** list, select your email to link to the API key.
7. Click **Create**.
8. Click **Copy**.
9. Click **Close**.
10. In the **REST API URL** field, click **Copy**.
11. In the **gRPC API URL** field, click **Copy**.
4. Run the following command to live-stream data to Sift:
```bash theme={null}
python stream.py
```
* Check the terminal for the generated Run name
* For example, **robot\_vehicle\_…\_run**
* You will use this Run name to access the live-streamed data shortly
## Observe live-streamed data
Now that data is being streamed to Sift, let's see how we can observe that data live.
1. Open the **Run** that the data is being streamed to. In particular, open the Run using **Explore**.
2. In **Explore**, click **Live**.
3. In the **Channels** tab, select the following Channels:
1. **temperature**
2. **velocity**
4. Before concluding this step, stop the streaming with the following command:
```bash theme={null}
Ctrl+C
```
# Conclusion
Source: https://docs.siftstack.com/learning-path/tutorial-3/conclusion
In this tutorial, you explored how Sift visualizes and analyzes telemetry using Panels in Explore.
You learned what different Panel types are designed for, how they help render and interpret time-series data, and how to arrange Panels into a multi-panel layout.
Together, these steps showed how Panels in Explore helps you uncover patterns, relationships, and cross-subsystem effects in telemetry data.
# Tutorial 3: Visualize and analyze data with interactive telemetry charts
Source: https://docs.siftstack.com/learning-path/tutorial-3/overview
Discover Sift's tools to explore, navigate, inspect, and compare your telemetry
By the end of this tutorial, you'll be able to use Explore's Panels to visualize, navigate, and interpret your telemetry data in Sift. We'll walk through each Panel type available in Explore and show you how to use them to render your data in meaningful ways, spot patterns, and dig into the details that matter most.
# Step 1: Learn about Panels
Source: https://docs.siftstack.com/learning-path/tutorial-3/step-1
## Overview
So far, you have been using only one of the many Panels available in Explore. In this step, let's explore all available Panels to analyze an array of different telemetry data types.
## Panels
For a full list of available panels and what each one does, see [Panels](/documentation/reference/explore-settings#panels).
# Step 2: Create a multi-Panel layout
Source: https://docs.siftstack.com/learning-path/tutorial-3/step-2
## Overview
Now that you understand the available Panels in Explore, in this step you will learn how to arrange them in a multi-Panel layout.
## Multi-Panel layout
Let's create a multi-Panel layout.
1. Using **Explore**, open the Run created in [Tutorial 1](../tutorial-1/overview).
2. Create a few **Timeseries** Panels:
* Panel 1:
1. Click the **Timeseries** tile.
2. In the **Channels** tab, under the **camera\_1** section, click **current**.
* Panel 2:
1. Click **Plus**, then select **Timeseries**.
2. In the **Channels** tab, under the **camera\_2** section, click **temperature**.
* Panel 3:
1. Click **Plus**, then select **Timeseries**.
2. In the **Channels** tab, under the **motor\_a** section, click **encoder**.
3. Create a **Table** Panel:
1. Click **Plus**, then select **Table**.
2. In the **Channels** tab, under the **motor\_b** section, click the following Channels:
* **current**
* **encoder**
* **temperature**
4. Create a **multi-Panel** layout:
1. Click and hold the **Table 1** tab, then drag it to the bottom of the screen. Release when that area highlights.
2. Click and hold the **Timeseries 3** tab, then drag it to the right side of the top Panel area. Release when the area highlights.
3. Click and hold the **Timeseries 2** tab, then drag it to the right of the **Timeseries 2** Panel. Release when the right-side highlight appears.
# Conclusion
Source: https://docs.siftstack.com/learning-path/tutorial-4/conclusion
In this tutorial, you learned what Rules are, how to create one using a CEL-based expression, and how to preview it against an existing Run to verify it behaves as expected.
You also generated a Report to review the Rule's evaluations, inspect the Annotations it triggered, and assess system behavior during a Run.
# Tutorial 4: Automatically detect and summarize issues in telemetry data
Source: https://docs.siftstack.com/learning-path/tutorial-4/overview
Use Sift's Rules and Reports features to automatically detect anomalies and summarize insights
By the end of this tutorial, you'll be able to create logic-based expressions in Sift that automatically evaluate your telemetry and surface conditions of interest, helping you detect anomalies as they happen. We'll walk through how to build these expressions, apply them to your data, and use them to flag the moments that need your attention.
# Step 1: Create a Rule
Source: https://docs.siftstack.com/learning-path/tutorial-4/step-1
## Overview
Let's discuss and create a Rule in Sift, which enables you to create logical conditions to evaluate your telemetry.
## What are Rules
Rules in Sift are logical conditions, defined using the Common Expression Language (CEL), that evaluate telemetry across one or more Channels from an Asset to detect behaviors, anomalies, and threshold breaches in live and historical data.
* When a Rule evaluates to true, it can create Annotations, send notifications, or be assigned to a user for follow-up
* Rules support versioning and previewing against existing Run data
## Preview and create a Rule
To better understand Rules, let's create one that captures a specific behavior in a Channel from the low Earth orbit satellite (LEO) we explored in [Tutorial 1](../tutorial-1/overview).
Specifically, we'll create a Rule for the temperature Channel in the **camera\_1** subsystem that triggers an Annotation when the temperature exceeds **30** degrees Celsius.
This ensures future datasets are automatically checked for similar anomalies and flagged for review.
1. Select **Rules** in the navigation.
2. Click **New Rule**.
3. In the **Rule name** box, enter the following:
```
fl_tutorial_4_rule
```
4. Replace `fl` (in `fl_tutorial_4_rule`) with your own initials.
5. In the **Asset name**, select the Asset created in [Tutorial 1](../tutorial-1/overview).
* **For example**: fl\_tutorial\_1\_asset
6. In the **Input Channels** list, select the following Channel:
* **camera\_1.temperature**
7. In the **Enter an expression** box, enter:
```
$1 > 30
```
* **Channel references**: Selected Channels are referenced in the expression using `$1`, `$2`, `$3`, and so on, in the order they appear in the **Input Channels** list. In this case, `$1` refers to **camera\_1.temperature**.
* **Functions**: Expressions support a variety of built-in function types for different kinds of calculations. To browse all available functions, observe the **Expression Syntax** section.
8. Click **Select Run(s) to preview**.
9. Click the Run created in [Tutorial 1](../tutorial-1/overview).
* **For example**: fl\_tutorial\_1\_run
10. Click **Preview 1 Runs**.
11. Click **Save**.
# Step 2: Generate a Report
Source: https://docs.siftstack.com/learning-path/tutorial-4/step-2
## Overview
Now that you've created a Rule, let's explore Reports, a built-in way to review how your Rules evaluated telemetry during a Run, inspect the Annotations they generated, and assess the overall health and behavior of your system.
## What are Reports
Reports provide a consolidated view of how the Rules configured for an Asset evaluated telemetry during a Run, including the [Annotations](../../documentation/reference/annotations-reference) they generated and their current status.
Annotations are markers that Sift creates when a Rule condition is met. They highlight an exact moment in your telemetry data.
## Create a Report
Let's create a Report for our created Run.
1. Select **Runs** in the navigation, then in the **Run name or description** box, enter the name of the Run created in [Tutorial 1](../tutorial-1/overview).
* **For example**: fl\_tutorial\_1\_run
2. In the **Runs** table, click your Run.
3. Click **Create Report**.
4. Click the **Rules** tab.
5. In the **Rules** table, click the Rule you created.
* **For example**: fl\_tutorial\_4\_rule
6. Click **Evaluate**.
## Open generated Report
After a few seconds, the report should be ready. Let's open it.
1. Select **Reports** in the navigation.
2. In the **Report name** box, enter the name of the Run created in [Tutorial 1](../tutorial-1/overview).
* **For example**: fl\_tutorial\_1\_run
3. In the **Reports** table, click the name of the created Run.
In a Report, you can assign trigger Annotations to specific users, or you can comment on them to enable further discussion with others. For example:
1. In the **Rules** section, click **14:21:36**.
2. Optional: In the **Assigned to** list, assign a user to the generated Annotation.
3. Optional: In the **Comments and logs** box, enter a comment for the generated Annotation.
# Conclusion
Source: https://docs.siftstack.com/learning-path/tutorial-5/conclusion
In this tutorial, you learned what Webhooks are, how to create one from the Manage page, and how to connect one to a Rule.
By attaching the Webhook to the Rule from the previous tutorial, Sift is now prepared to notify external systems whenever the Rule enters a violated or resolved state during live data evaluation.
# Tutorial 5: Trigger real-time alerts from live data events
Source: https://docs.siftstack.com/learning-path/tutorial-5/overview
Enable real-time notifications by connecting Rule triggers to external systems using Webhooks.
By the end of this tutorial, you'll be able to set up Webhooks in Sift that deliver real-time notifications the moment a Rule detects an important condition in your telemetry. You'll create a Webhook, configure its payload, and connect it to the Rule you built in Tutorial 4 so that it's ready to fire notifications as soon as live telemetry is evaluated.
## Prerequisites
* Only users with an Admin or Editor role can create Webhooks. You must be granted access to Webhooks before continuing. If you do not have the required permissions, contact your administrator.
* Basic understanding of webhooks
# Step 1: Understand Webhooks
Source: https://docs.siftstack.com/learning-path/tutorial-5/step-1
## Overview
Webhooks allow Sift to notify external systems the moment a Rule transitions into a violated or resolved state during live evaluation, actively pushing information instead of requiring another system to poll for changes.
Each Webhook sends an outbound request to a destination system with a customizable payload that describes what happened and can include context from the Rule that triggered the event.
Note that Webhooks only fire during live Rule evaluations and do not run when Rules are evaluated against historical data.
# Step 2: Create a Webhook
Source: https://docs.siftstack.com/learning-path/tutorial-5/step-2
## Overview
In this step, you will create a Webhook that will later be connected to the Rule you created in the previous tutorial. Webhooks in Sift are created and managed from the **Manage** page.
## Create a Webhook
Let's create a Webhook.
1. Click your profile menu.
2. Select **Manage**, then open **Webhooks** from the navigation menu.
3. Click **Create webhook**.
4. In the **Webhook name** box, enter the following:
```
fl_tutorial_5_webhook
```
5. Replace `fl` (in `fl_tutorial_5_webhook`) with your initials (first and last name).
6. In **Trigger event type**, select **Rule violation**.
7. In **Destination URL**, enter the the following:
```
https://example.com/webhook
```
* This is an example destination URL. It is provided so you can continue through the Webhook configuration flow. In a real environment, this would be replaced with the URL of a system that can accept Webhooks.
8. Click **Next**.
On this page, you can specify custom HTTP headers and send a test notification to the selected destination URL. You can also define the payload that will be sent when the Webhook fires, either by using [built-in variables](../../documentation/manage/get-alerted-when-a-limit-is-breached#built-in-variables) or by starting from one of the available templates.
For the purposes of this tutorial, you will use one of the templates to send a payload to Slack when the Rule created in the previous tutorial enters a violated or resolved state during live data evaluation.
9. In the **Payload config** list, select **Slack**.
The Slack template you selected uses built-in variables associated with the [Rule Violation](../../documentation/manage/get-alerted-when-a-limit-is-breached#rule-violation) trigger event type. These variables allow the payload to include useful context about the Rule and where it was evaluated.
* `{{.RuleName}}` and `{{.RuleVersion}}` identify the Rule that triggered the Webhook
* `{{.Status}}` indicates whether the Rule entered a violated or resolved state
* `{{.AssetName}}` and `{{.RunId}}` provide context about the Asset and Run involved in the evaluation
Sift replaces these placeholders automatically when sending the Webhook.
10. Click **Save**.
# Step 3: Connect the Webhook to the Rule
Source: https://docs.siftstack.com/learning-path/tutorial-5/step-3
## Overview
Now that the Webhook is created, let's connect it to the Rule from the previous tutorial. Once connected, the Webhook will fire whenever the Rule enters a violated or resolved state during live data evaluation.
This means that if live data is streamed to the Channels of the Asset associated with the Rule and the Rule's condition is met, Sift will automatically send a notification to the configured destination URL.
## Connect Webhook
Let's connect the created Webhook.
1. Select **Rules** in the navigation.
2. Search for the Rule created in the previous tutorial and click **Edit**.
3. Click the **Enable live Rule evaluation** checkbox.
4. In the **Webhooks** section, click **+ Configure Webhook**.
5. Enter the Webhook name, select the created Webhook, and click **Select**.
6. Click **Save**, then click **Publish**.
# Conclusion
Source: https://docs.siftstack.com/learning-path/tutorial-6/conclusion
In this tutorial, you learned how Calculated Channels work, created a derived signal using a CEL expression, and plotted it alongside the original Channel to compare the results.
These concepts provide a foundation for building more advanced analyses and deeper insight into system behavior through transformed and derived data.
# Tutorial 6: Create new signals from original data
Source: https://docs.siftstack.com/learning-path/tutorial-6/overview
Use Calculated Channels to generate new signals and metrics from your existing telemetry
By the end of this tutorial, you'll be able to use Calculated Channels in Sift to derive new signals from your existing telemetry, all without modifying the underlying source data. We'll walk through how to create and configure Calculated Channels so you can transform and enrich your data to suit your analysis needs.
# Step 1: Understand Calculated Channels
Source: https://docs.siftstack.com/learning-path/tutorial-6/step-1
## Overview
**Calculated Channels** in Sift let you define custom metrics and derive new signals from existing Channels using [CEL](https://cel.dev) expressions, without modifying the original data. Sift supports two types of transformations:
* **Stateless**: Processes each data point independently, ideal for real-time computations like unit conversions or threshold checks. For example, converting a temperature Channel from Celsius to Fahrenheit.
* **Stateful**: Retains context over time, enabling trend analysis and detection of gradual changes. For example, computing a rolling average over a defined time window.
# Step 2: Create a Calculated Channel
Source: https://docs.siftstack.com/learning-path/tutorial-6/step-2
## Overview
To better understand how Calculated Channels work, let's create one using the temperature Channel from the LEO dataset in Tutorial 1. Specifically, we will create a Calculated Channel that converts the **camera\_1.temperature** Channel from Celsius to Fahrenheit.
## Create a Calculated Channel
1. Select **Calculated Channels** in the navigation.
2. Click **New Calculated Channel**.
3. In the **Calculated Channel name** box, enter the following:
```
fl_tutorial_6_calculated_channel
```
4. Replace `fl` (in `fl_tutorial_6_calculated_channel`) with your initials.
5. In the **Asset** list, select the Asset you created in Tutorial 1.
6. In the **Input Channels** list, select the following Channel:
* **camera\_1.temperature**
7. In the **Query (expression)** box, enter the following:
```
($1 * 9/5) + 32
```
Expressions define how a Calculated Channel derives its values from input data. They can reference input Channels (for example, `$1`), other Calculated Channels, constants, operators, and built-in functions. Before saving, you can preview the output to verify the expression behaves as expected.
8. Click the **Select Run(s) to preview** tile.
9. In the **Runs** table, click the name of the Run created in Tutorial 1.
10. Click **Preview 1 Runs**.
11. Click **Save**.
# Step 3: Plot a Calculated Channel
Source: https://docs.siftstack.com/learning-path/tutorial-6/step-3
## Overview
Calculated Channels can be plotted alongside the original Channels of an Asset in Explore.
## Plot Calculated Channel
1. Select **Runs** in the navigation, then open the Run created in Tutorial 1 in **Explore**.
2. In **Explore**, in the **Channels** tab, click the following Channel in the **camera\_1** section:
* **temperature**
3. Click the **Calculated Channels** tab, then click the name of the created Calculated Channel.
* **For example**: fl\_tutorial\_6\_calculated\_channel
# Conclusion
Source: https://docs.siftstack.com/learning-path/tutorial-7/conclusion
## Summary
In this tutorial, you learned how User-Defined Functions work, created one using the **camera\_1.temperature** Channel from the LEO dataset to convert temperature from Celsius to Fahrenheit, and saw how it can be reused across Rules and Calculated Channels.
# Tutorial 7: Build reusable logic for consistent data transformation
Source: https://docs.siftstack.com/learning-path/tutorial-7/overview
Centralize reusable transformation logic using User-Defined Functions
By the end of this tutorial, you'll be able to create User-Defined Functions in Sift that encapsulate custom logic you can reuse across Calculated Channels and Rules. We'll walk through how to define these functions and apply them so you can keep your logic consistent, reduce duplication, and streamline your workflows.
# Step 1: Understand User-Defined Functions
Source: https://docs.siftstack.com/learning-path/tutorial-7/step-1
## Overview
User-Defined Functions let you define custom logic once as a named expression and reuse it wherever expressions are supported in Sift, including Calculated Channels, Rules, and other User-Defined Functions. They behave like built-in functions, can use built-in functions within their definitions, support optional inputs and constant values, and are version-controlled so logic can evolve safely.
# Step 2: Create a User-Defined Function
Source: https://docs.siftstack.com/learning-path/tutorial-7/step-2
## Overview
To better understand User-Defined Functions, let's create one using the LEO dataset from Tutorial 1. This function will use the **camera\_1.temperature** Channel to convert the temperature from Celsius to Fahrenheit, allowing it to be reused consistently across a Calculated Channel and a Rule.
## Create a User-Defined Function
1. Select **User-Defined Functions** in the navigation.
2. Click **New Function**.
3. In the **User-Defined Function name** box, enter:
```
fl_tutorial_7_user_defined_function
```
* User-Defined Function names must meet the following requirements:
* No spaces
* Must be 1 to 253 characters long
* Must start with a letter
* Use only letters, numbers, and underscores
4. Replace `fl` (in `fl_tutorial_7_user_defined_function`) with your initials.
5. In the **Enter an expression** box, enter the following:
```
($1 * 9/5) + 32
```
* In this expression, `$1` is a placeholder that refers to the first input Channel assigned to the function, in this case **camera\_1.temperature**. Each input Channel can be assigned a data type, including
* **Number**,
* **String**,
* **Boolean**, or
* their constant equivalents (**Constant Number**, **Constant String**, **Constant Boolean**).
* Constant types use fixed values defined in the function that do not change between evaluations, while non-constant types receive their values from Channels at the time the function is called.
* For this function, `$1` is defined as a **Number** since **camera\_1.temperature** is a numeric Channel.
* When a function needs to operate on more than one Channel, you can add additional inputs and reference them as `$2`, `$3`, and so on.
* For example, if you wanted to calculate the temperature difference between two cameras in the LEO dataset, you could add **camera\_1.temperature** as `$1` and **camera\_2.temperature** as `$2`, and write the expression as
* `$1 - $2`.
6. In the **Preview** section, click **Select Run to preview**.
7. In the **Run name or description** box, enter the name of the created Run in Tutorial 1.
8. Click **Next**.
9. In the **Channel name** list, select **camera\_1.temperature**.
10. Click **Preview**.
11. Click **Save**.
# Step 3: Apply User-Defined Functions
Source: https://docs.siftstack.com/learning-path/tutorial-7/step-3
## Overview
Once created, a User-Defined Function becomes available anywhere expressions are used in Sift, including Rules, Calculated Channels, and other User-Defined Functions. In the expression editor, User-Defined Functions appear in the **Expression Syntax** panel under the **User-Defined Functions** section.
## Using User-Defined Functions in expressions
When defining an expression, you can reference a User-Defined Function just like a built-in function. In the examples below, `$1` refers to **camera\_1.temperature**. For example, in a Calculated Channel:
```
fl_tutorial_7_user_defined_function($1)
```
Or in a Rule, as part of a logical condition:
```
fl_tutorial_7_user_defined_function($1) > 86
```
Because User-Defined Functions are defined once and reused across expressions, updating the function's logic applies the change centrally without requiring updates everywhere it is used.
# Conclusion
Source: https://docs.siftstack.com/learning-path/tutorial-8/conclusion
## Summary
In this tutorial, you learned how to create a Metadata key, assign a predefined value to it, attach it to the LEO dataset Run from Tutorial 1, and use it to search for that Run.
The same approach can be applied to other resources such as Assets, Rules, Calculated Channels, Reports, and User-Defined Functions, helping your workspace remain organized and easy to navigate as it grows.
# Tutorial 8: Tag and search workspace resources with Metadata
Source: https://docs.siftstack.com/learning-path/tutorial-8/overview
Use Metadata to add structured context to your workspace resources for faster searching and better organization.
By the end of this tutorial, you'll be able to use Metadata in Sift to improve discoverability and organization across your workspace. Using the Run from the LEO dataset in Tutorial 1, you'll create a Metadata key, attach it to the Run, and then use it to quickly search for and locate that Run.
# Step 1: Understand Metadata
Source: https://docs.siftstack.com/learning-path/tutorial-8/step-1
## Overview
Metadata in Sift consists of organization-defined key-value pairs with fixed data types, including string, number, and boolean, that provide a structured way to describe and organize workspace resources beyond their names. Each key is defined once and applied consistently across the workspace, making resources easier to search and filter.
## Where Metadata can be applied
Metadata can be attached to Runs, Assets, Annotations, Rules, Reports, Campaigns, Calculated Channels, and User-Defined Functions, allowing the same organizational structure to be shared across different types of resources.
# Step 2: Create a Metadata key
Source: https://docs.siftstack.com/learning-path/tutorial-8/step-2
## Overview
In this step, you will create a string Metadata key called `fl_data_source` to help identify the Run from the LEO dataset in Tutorial 1. The key name represents the attribute you want to track, in this case the source of the data. You will also assign a predefined value to the key, which will appear as a selectable option when the key is applied to a resource.
## Create a Metadata key
1. Click your profile icon, which shows the first initial of your account name.
2. Select **Manage**, then open **Metadata** from the navigation menu.
3. Click **+ Create Metadata key**.
4. In the **Name** box, enter:
```
fl_data_source
```
5. Replace `fl` (in `fl_data_source`) with your initials.
6. Click **Create**.
* Next, let's assign a predefined value to the key.
7. In the **Metadata key name** box, enter the name of the created Metadata key.
8. In the **Metadata keys** table, click **Options**, then select **View values**.
9. In the **Enter string value** box, enter the following:
```
fl_option_1_data_source
```
10. Replace `fl` (in `fl_option_1_data_source`) with your initials.
11. Click **Add value**.
12. Click **Close**.
# Step 3: Use Metadata to improve discoverability
Source: https://docs.siftstack.com/learning-path/tutorial-8/step-3
## Overview
In this step, you will attach the Metadata key you created to the Run from the LEO dataset in Tutorial 1 and use it to search for and locate that Run.
## Attach Metadata key to a Run
1. Select **Runs** in the navigation, then in the **Run name or description** box, enter the Run name from Tutorial 1.
* **For example**: fl\_tutorial\_1\_run
2. In the **Runs** table, click the name of the Run.
* **For example**: fl\_data\_source
3. Click **Edit**.
4. Click **Add Metadata**.
5. In the **Select or create** list, select the name of the created Metadata key.
6. In the **Enter or select value** list, select the created option for the Metadata.
* **For example**: fl\_option\_1\_data\_source
7. Click **Save Changes**.
## Search for a Run using its attached Metadata key
1. Select **Runs** in the navigation, then click **Metadata**.
2. In the **Select key** list, select the name of the Metadata key created in the previous step.
3. In the **Enter or select value** list, select the created option for the created Metadata.
Each resource in Sift's navigation, such as **Runs**, **Assets**, **Annotations**, **Rules**, **Reports**, and **Calculated Channels**, includes a Metadata filter that lets you search for resources using attached Metadata, making it easier to locate specific data as your workspace grows.
# April 2026
Source: https://docs.siftstack.com/release-notes/2026/april-2026
New ways to investigate telemetry and build on your calculated metrics
April 2026 ships a rebuilt Explore workspace, the ability to use Calculated Channels in Rule logic, and support for referencing Calculated Channels within other Calculated Channel calculations.
## New Investigate telemetry in one workspace
Skip the back-and-forth between tools when investigating telemetry. Open the new [Explore](/documentation/reference/explore-settings) as Sift's default workspace, shaped around your everyday workflows.
![Investigate telemetry in one workspace]()
## New Alert on every Calculated Channel
Catch problems in every metric your team calculates. Feed any [Calculated Channel](/documentation/reference/calculated-channel-settings) into a Rule and reuse the same alert logic you already rely on.
## New Build calculations on top of calculations
Cut duplicate work and keep every dependent metric in sync. Drop any existing [Calculated Channel](/documentation/transform/create-a-derived-signal#create-a-derived-signal) into a new calculation, and updates flow through automatically.
# Import flight logs, reuse investigations, and shape your Channel tree
Source: https://docs.siftstack.com/release-notes/2026/august-2026
Release notes · August 2026
Bring PX4 flight logs into Sift in their native format, reopen any Explore session exactly as you left it, and control how Channel names build your Channel tree.
## Event Sift Edge demo: capture telemetry offline at the test stand
Sift Edge is now available. Join Benjamin Nguyen, Lead Engineer for Sift Edge, and Zach Plunkett, Solutions Architect, for a 30-minute demo on Wednesday, August 19 at 10 AM PT.
See Sift Edge in action: capture telemetry locally and view it live at the test stand, even without an internet connection. You'll also see how Sift Edge uploads selected runs to Sift once you reconnect. Expect roughly 20 minutes of demo and 10 minutes of Q\&A.
* Watch a [two-minute preview](https://youtu.be/rSRJR0R9wnY) of what to expect
* [Register here](https://www.siftstack.com/edge-webinar#register-webinar) to save your spot
## New Import PX4 flight logs (ULog) straight into Sift
Bring [ULog files](/documentation/ingest/data-import/import-data-from-a-file) into Sift through the UI, CLI, Python client, or API (REST and gRPC).
* Analyze PX4 flight telemetry in its native format, ready the moment it lands
* Correlate vehicle parameters and system messages alongside your Channels
* Line up flight events with the rest of your Run on one timeline, anchored to a start time you supply or GPS time when none is set
![ULog import]()
## New Reopen any investigation exactly as you left it
Save the full state of an Explore session, then reopen it to pick up where you left off. [Saved Explorations](/documentation/reference/explore-settings#saved-explorations) are now generally available to every team.
* Capture Panels, layout, data sources, Channels, time range, and compare settings in one saved state
* Start a recurring investigation from a known baseline
* Share a Saved Exploration as a template for onboarding or operation-specific analysis
![Saved Explorations]()
## Beta Shape your Channel tree with two new delimiter settings
Control how Channel names resolve into the Channel tree with two new settings that build on the existing [Channel Name Delimiter](/documentation/reference/naming-rules#channel-name-structure) setting.
* Add a Channel name prefix delimiter to split off folder-style paths, such as the slash in `telemetry/motor.rpm`
* Set Channel leaf keywords to trim repetitive trailing segments that recur across many Channel names
* Keep source Channel names as they are and shape the Channel tree in Sift
![Delimiter settings]()
# February 2026
Source: https://docs.siftstack.com/release-notes/2026/february-2026
Scalable data analysis in context
This release makes Sift more capable and easier to use by helping engineers analyze telemetry in context. Teams can now calculate statistics over any time window, align signals with images and video, build reusable derived metrics, and navigate dense multi-axis data with greater precision, moving faster from raw signals to confident insight.
## Beta Measure your data over any time range
Select any section of your timeline in Explore v2 (Beta), and the Metrics Panel gives you the numbers: minimum, maximum, mean, and more. No need to export anything or switch tools.
* Pick a time range, and see stats for every signal in that window.
* Turn on "Follow selections" to update the numbers as you move through your data.
* Choose which stats you care about, lock a range for repeated checks, and export to CSV when you need a report.
![Calculate statistics for any data window]()
## New Play back video, images, and audio synced to your telemetry
Explore critical moments with precise time alignment in Explore v2 (Beta) using the File Viewer Panel to correlate images, video, and audio with your telemetry data. Review media inline and anchor it directly to your data, enabling seamless movement between signal behavior and real world context within a single, unified view.
* Set a start timestamp for accurate time correlation,
* inspect images with built in zoom and pan, and
* stay oriented during playback with hover mirroring across Panels.
![Align data with critical visual moments]()
Video footage courtesy of SpaceX. Used here for educational and analytical purposes to demonstrate Sift's telemetry tools.
## Beta Reuse existing calculations to build new metrics, without starting from scratch
Build new calculations using Calculated Channels you've already created. Change one, and every metric that depends on it updates automatically.
* Edit one building block, and every metric that uses it stays up to date.
* Built-in validation catches circular references and missing inputs before they cause any issues.
* Dependency previews show you which calculations feed into which, so you can trace the logic at a glance.
![Reference existing calculations to build complex metrics]()
## New Sort, group, and focus signals across up to eight axes
Organize busy charts and find the data that matters with new legend controls and focused tooltips in Explore v2 (Beta). Drag signals between axes, group them by type or source, and hover to highlight what you care about. Add up to eight independent axes to the Timeseries Panel and cut through visual noise. Tooltips now show only the nearest data point.
* Drag signals between axes to compare them side by side.
* Group legend items by data source, type, or units.
* Hover over a trace to highlight it and dim everything else.
![Navigate dense data with greater precision]()
# January 2026
Source: https://docs.siftstack.com/release-notes/2026/january-2026
Collaborative analysis and data portability
This month, we are launching new features that give you more precise control over your telemetry. These capabilities make it easier to share targeted visual reviews and export data for external analysis.
## New Launch targeted analyses via dynamic URLs
Create custom dynamic URLs that open Explore v2 (Beta) with pre-specified data. This capability allows you to:
* Define exact session parameters, including the specific Run or Asset, Channels, Panel type, and precise start/end times.
* Automate complex visual setups using dynamic links that eliminate the need for manual configuration.
* Ensure team members land on the exact telemetry and timeframe required for their review immediately upon clicking.
![Parameterized URLs]()
[Documentation](../../documentation/reference/explore-settings)
## New Easily recreate and share visual analyses
Standardize your analysis using Panel Configurations, which have moved from Beta to General Availability (GA). This feature allows you to capture and reuse the complete visualization setup of an individual Panel to:
* Save specific Channel selections, axis mappings, and styling preferences for instant recall.
* Share these configurations across your organization to provide the granular control needed to recreate complex visual analyses.
* Standardize your analysis workflow by instantly reproducing saved visual setups.
![Panel Configurations]()
[Documentation](../../documentation/reference/explore-settings)
## New Export data to Parquet
Move your telemetry beyond Sift for custom analysis by exporting Runs or specific Channels into a Parquet format. This ensures your data remains portable and allows you to:
* Transfer high-volume telemetry into any tool that supports the Parquet standard, such as pandas, DuckDB, or Polars.
* Preserve native data types and precision, eliminating the need for manual cleanup.
* Empower your team to perform complex analysis in your preferred data stack.
![Export Run to Parquet]()
[Documentation](../../documentation/reference/supported-file-formats)
# New Panels, faster rendering, and Saved Explorations
Source: https://docs.siftstack.com/release-notes/2026/july-2026
Release notes · July 2026
This release ships Sift CLI v0.2.0 with HDF5 and TDMS imports, Parquet export, and built-in documentation. It also adds two new Panel types and performance improvements to the Timeseries Panel in Explore. Explorations can now be saved and reused across Runs, Assets, and Families.
## New Move telemetry in and out of Sift from the terminal
Bring more telemetry into Sift straight from the terminal with the Sift [CLI](/documentation/cli/sift-cli) v0.2.0.
* Import HDF5 and TDMS files in their native format, with a preview before upload
* Export to Parquet for downstream analysis alongside existing export formats
* Confirm authentication and connectivity before starting a long-running upload
* Access reference documentation directly in the terminal
## Beta Pick up any investigation where you left off and reuse it across the team
Save the full state of an Explore session as a [Saved Exploration](/documentation/reference/explore-settings#saved-explorations), including Panels, layout, selected Runs, Assets, Families, Channels, time range, and compare settings. Open a Saved Exploration to continue an in-progress investigation or to start a recurring one from a known baseline.
* Reuse the same exploration across Runs, Assets, and Families for consistent analysis
* Share explorations with teammates as templates for onboarding or operation-specific analysis
![Saved Explorations]()
## Beta See a system's state and every change across time
Plot enum and boolean Channels as a color-coded timeline with the [Enum Panel](/documentation/reference/explore-settings#panel-types). Each value period appears as a labeled band with its state name, so system behavior reads as named states over time.
![Enum Panel]()
## Beta Read every Channel's current value, live or at any point in time
Display the most recent value for each plotted Channel as a tile with the [Stat Panel](/documentation/reference/explore-settings#panel-types). Values update in real time during Live mode and follow the time cursor when scrubbing through historical telemetry.
* Updates automatically in Live mode or when hovering over another Panel
* Supports numeric, boolean, enum, and bit field Channels
* Works with Timeseries, Enum, Table, Geo Map, and File Viewer Panels
![Stat Panel]()
## Beta Pan, zoom, and render high-rate telemetry at full speed
Enable [WebGL rendering](/documentation/reference/explore-settings#webgl-traces) in the Timeseries Panel for faster panning, zooming, and point rendering. Recommended for Windows and Linux users working with high-volume telemetry or high-rate Channels.
![WebGL traces]()
# March 2026
Source: https://docs.siftstack.com/release-notes/2026/march-2026
New tools for comparison, export, and investigation
This release adds data export, per-Channel downsampling controls, relative-time alignment, data source swapping, and the ability to plot and create Annotations in Explore, plus Calculated Channel support in Rules.
## New: Export data from any Timeseries Panel in CSV, Parquet, or WinPlot
[Export](../../documentation/reference/explore-settings) data directly from any Timeseries Panel in Explore and move it into downstream tools without copying or reformatting.
Choose the format that fits your pipeline:
* **CSV** for broad compatibility,
* **Parquet** for columnar analysis, or
* **WinPlot** for signal-specific tooling.
![Export data from any Timeseries Panel in CSV, Parquet, or WinPlot]()
## New: Control how each Channel is downsampled in a Timeseries Panel
Select any Channel in a Timeseries Panel to [set](../../documentation/reference/explore-settings) its downsampling method:
* **LTTB** for preserving overall signal shape,
* **Min/Max** for retaining true peaks and valleys, or
* **Changed Only** for capturing state transitions in enum and boolean Channels.
![Control how each Channel is downsampled in a Timeseries Panel]()
## New: Swap a Run or Asset in Explore without rebuilding your Panels
Replace the data source behind an investigation, **Run** or **Asset**, while keeping every Panel, layout, and setting intact. When new test data arrives, [swap](../../documentation/reference/explore-settings) the source and continue where you left off.
![Swap a Run or Asset in Explore without rebuilding your Panels]()
## Beta: Align Runs on a shared timeline to compare data across tests
Offset Runs that happened at different times so they share a common reference point.
* [Align](../../documentation/analyze/align-runs-comparison) to the **start or end of a Run**,
* a specific **timestamp**, or
* an **Annotation** within the Run, then compare behavior side by side on one timeline.
![Align Runs on a shared timeline to compare data across tests]()
## Beta: Plot and create Annotations directly on a Timeseries Panel
[Plot](../../documentation/analyze/analyze-a-run) Annotations from an **Asset** or **Run** onto any Timeseries Panel and [create](../../documentation/analyze/analyze-a-run) new ones without leaving Explore. Mark a phase, flag a review, or pin an exact moment so the full context stays attached to the data.
![Plot and create Annotations directly on a Timeseries Panel]()
## Beta: Trigger Rules on Calculated Channels to alert on derived metrics
Rules now accept Calculated Channels as inputs. Define thresholds and trigger alerts on derived metrics, not just raw telemetry Channels.
# Control access, catch drift, and automate your thresholds
Source: https://docs.siftstack.com/release-notes/2026/may-2026
Release notes · May 2026
This release puts you in control of who sees your telemetry, helps you catch Run drift with statistical baselines built from related Runs, and lets Rules and Calculated Channels use values from Asset and Run files.
HDF5 and TDMS files now upload directly into Sift, and every new Panel in Explore opens with your preferred settings.
## New Give everyone exactly the right access to your telemetry
Control telemetry access with [Attribute-Based Access Controls](/documentation/manage/set-up-data-access-governance). Tie view, edit, and share permissions on Assets, Channels, and Runs to the attributes your team already manages in your identity provider.
* Share curated Assets, Channels, and Runs with your teammates
* Give each team and region the right level of access to the same data
* Access updates everywhere it applies when group membership changes in your identity provider
![Data Access Governance (DAG)]()
## New Start analyzing HDF5 and TDMS data the moment it lands in Sift
Upload test data with HDF5 and TDMS imports. Both formats use the same upload flow, progress reporting, and error handling as CSV and Parquet.
* Upload from the surface you already work in: UI, Python client, or CLI
* HDF5 files upload as-is and are ready to analyze right away
* All TDMS data types come through, including complex numbers, with TDMS file properties imported as metadata
![Import TDMS an HDF% files]()
## Beta Catch any Run that drifts from your baseline
Compare related Runs with [Families](/documentation/analyze/families/overview). Align members to a shared reference event, such as an Annotation or Run start, and build a statistical baseline from their aggregated signal history.
* Plot every Family member on a shared timeline in Explore to compare signal shapes and spot outliers
* Compute aggregate statistics across member Runs, including average, minimum, maximum, and standard deviation
* Build Rules that use Family statistics as dynamic thresholds, with results surfaced as Annotations in Reports
![Families]()
## Beta Configure and open every new Panel the way you work
Set personal defaults with [User Settings](/documentation/reference/explore-settings#user-settings). Configure your display preferences once, and every new Panel in Explore applies them automatically.
* Set global preferences such as default timezone and live data refresh
* Choose defaults for each of the Panel types, including Timeseries, Table, and FFT
* Pick from Channel Name Delimiter options to match how your team separates hierarchy in Channel names
* Reset defaults per Panel type or across all types at once
![User settings]()
## Beta Make Rules and Calculated Channels adapt to every Run
Replace hardcoded thresholds with [file references](/documentation/transform/reference-files-in-expressions) in Rules and Calculated Channel expressions. Read values directly from files attached to an Asset or Run, and each expression picks up the right constants for that Asset or Run.
* Attach files to an Asset for values that hold steady across Runs, such as hardware limits or calibration offsets
* Attach files to a Run for values that change between tests, such as test parameters or acceptance criteria
* Works with CSV, JSON, YAML, and TXT formats
![Reference files in CEL expressions]()
# Analyze with agents, keep tests visible, and follow every state
Source: https://docs.siftstack.com/release-notes/2026/september-2026
Release notes · September 2026
This release connects your coding agent to Sift for ingestion, investigation, and approved write actions. It also keeps telemetry visible beside the hardware through network drops so you can move the Runs you choose into Sift when you're ready. You can also follow state changes, read current values, and mark important moments directly in Explore.
## Analyze telemetry from your coding agent
Connect Claude Code, Codex, Cursor, or OpenCode to your Sift environment with the [Sift MCP](/documentation/cli/sift-mcp) server. Ask your agent to ingest and discover telemetry, run analyses and transformations, or create Sift resources from the interface you already use.
* Install skills and register the Model Context Protocol (MCP) server with supported clients using a single Sift CLI command.
* Keep tools that overwrite or archive existing disabled resources until you choose to enable them.
* Protect sessions with context limits, filtered queries, and automatic retry backoff.
![Analyze telemetry from your coding agent]()
## Keep test telemetry visible through network drops
Capture and inspect telemetry beside your hardware with [Sift Edge](https://www.siftstack.com/use-case/sift-edge). Your Runs are saved locally during an interruption, letting you upload the ones you choose to Sift once your connection returns.
* Watch live and recent telemetry with low latency from a local desktop app.
* Translate protocols on site, buffer telemetry, and replay it after reconnecting.
* Continue the analysis in Sift beside your Rules, Reports, saved views, and Run history.
![Keep test telemetry visible through network drops]()
## Compare every state change on one shared timeline
Plot enum and boolean Channels as labeled, color-coded periods with the [Enum Panel](/documentation/reference/explore-settings#panel-types). Give each Channel its own row so changes across the system stay aligned in time.
* Scan labeled bands to see how long each Channel held a state.
* Zoom into striped Multiple Values bands to resolve changes that share one pixel.
![Compare every state change on one shared timeline]()
## Track live values and scrub back through a test
Display each Channel's latest value as a tile with the [Stat Panel](/documentation/reference/explore-settings#panel-types). Values update every second in Live mode and follow a shared time cursor during historical review.
* Sync with a Timeseries Panel to read values at the hovered or focused time.
* Look back up to 5 minutes to find the latest value for each Channel.
![Track live values and scrub back through a test]()
## Keep every review marker attached to its telemetry
Mark a point or time range in a Run with [Annotations](/documentation/reference/explore-settings#annotations). Plot those markers on a Timeseries Panel or use one as the shared T-0 for Run comparison.
* Create an Annotation from the telemetry and time range you are reviewing.
* Align related Runs to the same event so behavior reads against one reference point.
![Keep every review marker attached to its telemetry]()
# August 2025
Source: https://docs.siftstack.com/release-notes/august-2025
Accelerating the journey from data to decisions
Each of this month's updates stands strong on its own, but together they tell a story about where Sift is headed.
1. If your pipelines produce data in [Parquet](#new%3A-parquet-file-support-for-telemetry-ingestion-in-the-ui-and-rest-api-beta), you can now bring it directly into Sift.
2. Once that data is inside, [Metadata](#new%3A-user-defined-metadata-for-consistent-organization%2Ffiltering-across-sift-resources), a new Sift feature, ensures it can be organized, managed, and easily found later, making reviews more automated and efficient.
3. From there, Explore v2 (Beta) becomes the place to work, transforming Parquet data into insights. Now, in Explore v2 (Beta), with [Live Mode, Sync Mode, and Views](#new%3A-explore-2-beta-expands-with-sync-mode%2C-live-mode%2C-and-views), teams can track signals in real time and uncover root causes faster.
Individually, these updates add speed, structure, and efficiency. Together, they show Sift as the platform that turns raw data into organized review, live analysis, and shared insight without friction.
Beyond these highlights, you will also find several other New and Improved updates described below.
## New: Explore 2 (Beta) expands with Sync mode, Live mode, and Views
* **Overview**: Explore 2 (Beta) now expands with Live mode to stream data directly into the workspace and Sync mode to align a multi-panel view containing different Channels.
When Sync mode is active, zooming into one panel automatically adjusts the others to the same time range.
An early implementation of Views is also available in Explore 2 (Beta): you can apply Views created in Explore 1, with support for creating and editing Views planned for a future release (in Explore 2).
These updates build on the foundation of Explore 2 (Beta), which delivers speed, precision, and control for high-frequency telemetry analysis.
![Live + Sync mode]()
## New: User-Defined Metadata for consistent organization/filtering across Sift resources
* **Overview**: Sift now supports user-defined Metadata,
allowing you to add structured key-value pairs to all resources including
Runs,
Assets,
Annotations,
Rules,
Reports,
Report Templates,
Campaigns,
Calculated Channels, and
User-defined Functions.
This provides a consistent way to describe and categorize resources beyond unstructured tags.
* See the documentation on User-Defined Metadata: [Tutorial](../tutorials/manage/tutorial-2), [Reference](../documentation/manage/organize-and-filter-resources-with-metadata), and [How-to guide](../documentation/manage/get-alerted-when-a-limit-is-breached).
![Metadata]()
## New: Identity provider (IdP) (Beta) integration for scalable user management
* **Overview**: Sift now integrates with external Identity Providers (IdPs) that support push provisioning.
User and group changes in your IdP automatically synchronize to Sift, ensuring account information remains current without manual intervention.
* See the documentation on Identity provider (IdP): [Tutorial](../tutorials/manage/tutorial-1), [Reference](../documentation/manage/connect-an-identity-provider), [How-to guide](../documentation/manage/connect-an-identity-provider).
![IdP]()
## New: A dedicated Jobs page for easier monitoring and results tracking
* **Overview**: Sift now includes a dedicated Jobs page at `/manage/jobs`.
This page provides a central location to view all Jobs in your organization as well as your own Jobs, making it easier to monitor progress, manage execution, review results, and take actions such as canceling or retrying Jobs.
![Jobs]()
## New: NaN and infinity handling for state-aware telemetry analysis
* **Overview**: Sift now supports ingestion and analysis of `NaN`, `+Infinity`, and `-Infinity` values in float and double Channels.
These values were previously filtered out and sent to the dead-letter queue (DLQ) but are now available in Rules, visualizations, and Calculated Channels.
To ensure safe handling, new built-in functions `isnan`, `isinf`, and `isfinite` provide precise control.
![NaN]()
## New: Parquet file support for telemetry ingestion in the UI and REST API (Beta)
* **Overview**: Sift now supports importing Parquet files through both the UI and the REST API.
Parquet files can be ingested directly, with each Channel represented as a column in a flat schema.
This makes it easier to bring telemetry from modern data pipelines into Sift.
* See the Parquet ingestion docs: [Parquet uploads](../documentation/ingest/data-import/import-data-from-a-file).
![Parquet]()
## Improved: Support for 10 GiB exports with background notifications
* **Overview**: You can now export large volumes of data directly from the Explore page, whether viewing a Run or an Asset.
Exports are conducted in the background, and progress is visible in the Notifications dropdown and the Jobs page at `/manage/jobs`.
Once the export is complete, you will receive a notification and can download the results without needing to monitor the process.
At this time, exports are supported up to 10 GiB per Job. This safeguard can be adjusted for specific customer needs.
![Export 1]()
![Export 2]()
## Onboarding tutorial
* **Overview**: New to Sift, or onboarding a new teammate? Sift now includes a beginner tutorial that introduces the basics of the platform and walks through an end-to-end workflow using real telemetry data. This tutorial is designed to help new users understand Sift quickly and make onboarding easier for your team. [Learn more](../learning-path/overview).
# December 2025
Source: https://docs.siftstack.com/release-notes/december-2025
Clear correlation finding and modular logic
This month's updates make it easier to analyze, reuse, and deepen telemetry insights.
* You can now use the new Scatterplot Panel to analyze correlations between signals and more easily identify anomalies by visualizing multiple signals in a single view.
* You can also now reference existing calculations (Calculated Channels) inside other Calculated Channels, a capability (Beta) that lets you build modularized logic with greater clarity.
## Uncover deep correlations with multi-dimensional visualization
The Scatterplot Panel in Explore v2 (Beta) helps you discover relationships between signals by plotting multiple dimensions in a single view. This enables faster validation of correlations and clearer identification of outliers. If you regularly compare multiple signals or investigate unexpected changes, consider using the Scatterplot Panel in Explore v2 (Beta) to explore those relationships.
![Scatterplot Panel]()
## Beta Reference existing calculations to build more complex derived metrics
You can now reference existing Calculated Channels as inputs to new ones, enabling more complex expressions while maintaining readability and modularity. This enables greater consistency and reduces the potential for errors when performing complex, repeated data transformations. If you regularly build derived metrics, consider referencing an existing Calculated Channel to create cleaner, more modular expressions.
![Calculated Channel]()
# July 2025
Source: https://docs.siftstack.com/release-notes/july-2025
## Overview
This month's release notes introduce foundational upgrades across the Sift workspace. Explore 2 (Beta) is now live, delivering a rebuilt exploration experience with sharper visuals, faster performance, and more flexible workflows. We've also rolled out protective rate limiting for high-volume data access and brought back flexible Channel search for faster navigation. Together, these updates give teams deeper visibility, more reliable performance, and a smoother path from raw data to answers.
## New: Explore v2 (Beta)
* **Overview**: Explore v2 (Beta) is a full rebuild of Sift's Explore workspace for telemetry analysis, designed for speed, precision, and control. Now available in Beta, this update enhances several aspects of the exploration workflow to help teams move faster, compare signals more effectively, and uncover root causes with less friction. It introduces capabilities such as multi-chart layouts, cross-source plotting, smarter Channel selection, advanced styling, and X+Y zoom, along with performance improvements including nanosecond resolution and significantly faster rendering, especially on Linux. The redesign is grounded in direct input from teams working with high-frequency, mission-critical data.
* To learn more, see Explore v2 (Beta)'s [Tutorial](../tutorials/analyze/explore/tutorial-1).
* **Unlocked**: Explore v2 (Beta) gives teams greater control and clarity when working with high-frequency telemetry. Whether you're troubleshooting a system fault, comparing behavior across Assets, or surfacing trends across large datasets, the new workspace is built to accelerate that process. The result: faster root cause analysis, clearer comparisons, and less time spent wrangling data.
![Explore 2 (Beta)]()
## New: GetData rate limiting
* **Overview**: Sift now includes rate limiting for GetData requests across many of our interfaces, including the API, Grafana plugin, and Explore v1.
This change is designed to protect system stability without disrupting normal usage, especially for users running automated or large-scale queries.
* **Unlocked**: This feature introduces a protective layer that limits excessively parallel queries or long-running scripts that could affect performance for others.
The rate limiter is configured with generous thresholds based on real usage data and has been running in dry-run mode for validation.
Importantly, this does not apply to Explore v2 (Beta), which uses a separate data path.
Teams relying on GetData via these interfaces should expect no impact under normal usage but will benefit from improved overall reliability.
## Restored: Keyword search in the Channel selector
* **Overview**: Explore's Channel selector now supports keyword search using multiple partial terms. This restores a popular search behavior previously referred to as "fuzzy Channel search" by users. You can now locate Channels more efficiently, even when you don't remember the exact name or formatting.
* **Unlocked**: You can search using multiple space-separated keywords, and each will be matched independently within the Channel name, regardless of order. For example, typing `max_ground temp` will successfully match `max_ground_temp(C)`. This improves discoverability for long or inconsistently named Channels and streamlines navigation across large datasets.
![Search]()
## Announcements
* **Manifesto**: Ever wonder why Sift was created? It wasn't to build more software; it was to fix the fragile infrastructure that slows down the teams building our most complex machines. We've published a manifesto that lays out what we're building and why it matters. [Download manifesto](https://www.siftstack.com/manifesto).
* **Webinar recording**: Traceability isn't just a compliance checkbox anymore; it's now mission-critical infrastructure. In our latest webinar, we break down how leading aerospace and defense teams are unifying hardware genealogy across build, test, and operations to move faster and reduce risk. Whether you missed the live session or want to revisit key insights, the full recording is now available [here](https://www.siftstack.com/mission-critical/payload-webinar).
## Recent blogs
How Sift transforms hardware telemetry chaos into a trusted foundation for analysis
}
>
“Sift gives engineering teams a single foundation to work with hardware telemetry by delivering a \[Single Source of Truth (SSOT)] and a \[Single Pane of Glass (SPOG)].”
# June 2025
Source: https://docs.siftstack.com/release-notes/june-2025
## Overview
This month's release notes introduce new ways to bring data into Sift and view it across your tools. You can now send data from LabVIEW and FlexLogger directly to Sift. The Sift Grafana plugin is also now available in the public plugin catalog for easy installation. Sift's workspace (Explore) now supports a logarithmic Y-axis scale, making it easier to inspect signals that span multiple orders of magnitude.
## New: Visualize Sift telemetry in Grafana via official plugin
* **Overview**: The [Sift Grafana plugin](https://grafana.com/grafana/plugins/sift-grafana-datasource/) is now publicly available in the Grafana plugin catalog. It supports both simple Channel queries and Calculated Channel expressions, enabling teams to visualize Sift telemetry directly in Grafana dashboards without any manual installation.
* **Unlocked**: With public catalog availability, teams no longer need to distribute or install the plugin manually. This update makes it easier to integrate Sift into existing dashboards and observability workflows, while continuing to pull structured telemetry from Sift's Single Source of Truth (SSOT) with full consistency and governance.
![Grafana]()
## New: Send data to Sift from LabVIEW and FlexLogger
* **Overview**: Sift now offers official plugins for [LabVIEW](https://github.com/sift-stack/sift-labview/tree/main) and [FlexLogger](https://github.com/sift-stack/sift-labview/tree/main/plugins/flexlogger), enabling teams to send or upload engineering data from National Instruments environments directly into Sift for centralized analysis and observability.
* **Unlocked**: Engineers can now send or upload data from LabVIEW and FlexLogger into Sift to unify telemetry from test and validation workflows. This closes the gap between data collection and analysis, making results easier to visualize, review, and trace all within a single platform.
![LabVIEW, FlexLogger, and Sift]()
## New: Log scale to view signals across multiple magnitudes
* **Overview**: Explore now supports a logarithmic Y-axis scale, giving users a clearer view of signals (Channels) that span multiple orders of magnitude.
* **Unlocked**: Log scale helps surface patterns and trends in high-dynamic-range data that might be flattened or hidden on a linear scale.
![Log]()
## Webinar: Track component history across systems
Join us on **July 15 at 1:00 PM ET** for a live Payload webinar on closing the traceability gap in aerospace and defense programs. Industry experts including **Karthik Gollapudi**, CEO and Co-founder of Sift, will discuss how organizations are unifying genealogy tracking across test, production, and sustainment to reduce compliance risk and improve mission readiness. [Learn more and register](https://us06web.zoom.us/webinar/register/WN_i9yi-RggRVG-f3w3xheGJA#/registration).
# May 2025
Source: https://docs.siftstack.com/release-notes/may-2025
## Overview
This month's release notes introduce a suite of impactful updates designed to enhance visibility, automation, and workspace organization in Sift. Highlights include the new Table Viewer, which provides a high-fidelity view of telemetry data; support for programmatic logic with external Rules; webhooks to trigger external workflows; the ability to archive Assets and Runs; and improved file uploads. These updates help streamline your workflows and enhance your ability to work with data efficiently.
## New: Trigger external workflows with webhooks (Beta)
* **Overview**: Webhooks allow you to automatically trigger external workflows whenever a Rule condition is violated, enabling seamless integration between Sift and your operational stack. Built-in support for tools like Slack, PagerDuty, and more helps streamline responses to issues as they arise. To learn more, see [Webhooks (Beta)](../documentation/manage/get-alerted-when-a-limit-is-breached).
* **Unlocked**: Rule violations can now trigger outbound webhooks, enabling automated connections between Sift and external systems, such as Microsoft Teams, Slack or PagerDuty.
![Webhooks]()
## New: View high-fidelity data in the Table Viewer (Beta)
* **Overview**: The Table Viewer displays raw telemetry in a high-fidelity table format, allowing users to inspect and filter exact values aligned with time series charts. This structured view is ideal for pinpoint analysis and precise data review.
* **Unlocked**: You can now perform detailed, time-aligned reviews of telemetry data, identify anomalies with greater precision, and correlate events across Channels using an interactive, filterable table view.
![Table Viewer]()
## New: Integrate external Rules into CI/CD workflows
* **Overview**: External Rules let you evaluate custom logic on demand via the Sift API, making them ideal for automated and programmatic workflows like CI/CD. These Rules operate outside the UI, are temporary by design, and can produce Annotations that appear in Reports.
* **Unlocked**: You can now automate Rule evaluation in pipelines, generate annotations programmatically, and integrate Sift's logic engine into external systems without relying on the UI.
## New: Archive Assets and Runs
* **Overview**: You can now archive Assets and Runs in Sift to declutter your workspace without losing data. Archived items are hidden from the main view but remain accessible via the archive filter, preserving historical records while maintaining a clear day-to-day focus.
* **Unlocked**: You can now streamline your workspace by hiding inactive or outdated items, stay focused on current work, and still retain access to complete historical context when needed.
![Archived Runs]()
![Archived Assets]()
## New: Monitor upload status and errors
* **Overview**: Uploading CSV and TDMS files in Sift now shows real-time progress indicators and more explicit error messages. These enhancements enhance transparency, making it easier to diagnose and resolve issues during data import.
* **Unlocked**: You can now confidently track file upload progress, quickly identify problems, and ensure smoother, more reliable data ingestion.
![Monitor upload status and errors]()
## Improvement: View documentation by domain and content type
* **Overview**: The public-facing documentation has been reorganized into two main sections: *UI* and *API*. Each section is now structured around four content types: *Tutorials*, *How-to guides*, *Reference*, and *Explanations*. At launch, only the How-to guides and Reference materials are available, with Tutorials and Explanations to be added incrementally.
* **Unlocked**: You can now find documentation more easily, with a clearer separation between UI and API content and a structure that aligns with best practices for accessibility and discoverability.
## Recent blogs
Engineering at the speed of flight
}
>
"We call it the new Sift world,” Blair said. “As soon as someone sees what it can do, they want in."
Test like you fly starts with the data, not the dashboard
}
>
"When observability and execution work together, CI/CD becomes more than a software metaphor. It becomes real."
Why frequent, small releases strengthen hardware development
}
>
"Frequent releases allow engineers to validate performance at every stage, reducing the likelihood of major failures right before launch."
Why Impulse Space chose to build on Sift
}
>
"Impulse is deploying Sift across their engineering workflows, allowing teams to work from a centralized source of truth and streamline their ability to review, analyze, and act on high-frequency mission data."
# November 2025
Source: https://docs.siftstack.com/release-notes/november-2025
Clearer ways to understand telemetry changes
## Overview
This month's updates reinforce our mission to make it easy to analyze, recreate, and act on the insights in your telemetry.
* Explore v2 (Beta) now offers [improved performance and clearer interactions](#new:-accelerate-your-analysis-with-a-faster-explore-v2-beta), giving you a smoother experience when investigating live or historical data.
* Within Explore v2 (Beta), the new Panel Configurations (Beta) feature lets you capture the precise setup of any Panel so you can [quickly recreate analyses](#new:-easily-recreate-and-share-visual-analyses) or apply them to new data.
* And when you are not directly analyzing your telemetry, Webhooks (now generally available) notify you the moment [important changes](#new:-trigger-external-workflows-the-instant-something-important-happens-in-your-telemetry) occur in your live data.
## New Accelerate your analysis with a faster Explore v2 (Beta)
Explore v2 (Beta) continues to evolve with upgrades that make it more responsive, intuitive, and stable across both live and historical analysis. Improvements span smoother real-time analysis, a clearer and more useful Table Panel, more flexibility in Charts, and simplified time range controls. [See the Changelog for full details](../changelog).
![Explore v2 (Beta)]()
## New Easily recreate and share visual analyses
The new Panel Configurations (Beta) feature in Explore v2 (Beta) builds on and expands the capabilities previously offered by Views in Explore v1, providing more granular control, greater flexibility, and improved reusability across analysis workflows. They let you capture and reuse the complete visualization setup of an individual Panel, including Channel selections, axis mappings, color scales, hidden states, and styling preferences.
[Learn more](../documentation/reference/explore-settings).
![Panel Configurations]()
Contact us to join the Beta.
## New Trigger external workflows the instant something important happens in your telemetry
Webhooks are now generally available, making it easy to integrate Sift with your external tools and trigger actions based on telemetry. By automatically sending real-time events when a live Rule is violated and resolved, Webhooks connect Sift directly with your existing software tools, such as Microsoft Teams, Slack, Jira, OpsGenie, and PagerDuty. [Learn more](../documentation/manage/get-alerted-when-a-limit-is-breached).
![Webhooks]()
## Recent blogs
### Working at Sift: Alexa Agnew
“Before transitioning into engineering, \[at Sift, Alexa] led analytics and infrastructure products at ABL Space Systems and Carta, and earlier worked as a quantum engineer and deployment lead at Rigetti Computing. With a PhD in physics from Columbia, where she conducted research at the Nobel Prize–winning LIGO lab, … Alexa recently made the leap from product management to full-time software development at Sift to build the tools that accelerate some of the most ambitious hardware programs in the world.” [Read blog](https://www.siftstack.com/mission-critical/working-at-sift-alexa-agnew).
### Working at Sift: Mike Casey
"Before joining Sift, Mike spent nearly a decade as an intelligence analyst at the Defense Intelligence Agency, where he focused on Chinese space, satellite, and AI-enabled military capabilities. His Mandarin-language research and C4ISR expertise directly informed national security decisions and technology investments. At Sift, he’s traded policy reports for launch pads, working side by side with engineers to help hardware teams move faster and smarter." [Read blog](https://www.siftstack.com/mission-critical/working-at-sift-mike-casey).
## Looking ahead
We're excited to deliver these improvements and features to you and your team.
We're looking forward to hearing how these capabilities shape the way you work.
# October 2025
Source: https://docs.siftstack.com/release-notes/october-2025
Connecting data, insight, and support
## Overview
This month's release strengthens the connection between how data enters Sift and how teams stay supported while working in it.
Parquet file ingestion, previously in beta, is now generally available, making it easier to move high-volume, structured data from collection to analysis in [Explore v2 (Beta)](./july-2025#new%3A-explore-v2-beta).
When something in that process needs attention, the new Sift Support experience makes it easy to get help right where you work. Teams can now report and track issues directly within the app, creating a single source of truth for communication and visibility.
The Changelog is updated regularly with smaller fixes and performance improvements. Visit anytime to explore what's new between releases.
## New: Simplify data ingestion with Parquet file support, now generally available
Parquet file ingestion, previously in beta, is now generally available in both the Sift UI and REST API. This update makes it easier to bring high-volume, structured Parquet data directly into Sift for faster analysis. Once ingested, you can use Explore v2 (Beta) to dive deeper into your data. [Learn more](../documentation/ingest/data-import/import-data-from-a-file).
![Parquet]()
## New: Obtain direct in-app support with full issue tracking
We've introduced Sift Support as the centralized way to manage support within the Sift app. It serves as the single source of truth for reporting issues, tracking progress, and communicating with our team. Users can follow their own requests, and an assigned admin can view all open issues across the organization to maintain visibility and ensure timely resolution.
![Pylon]()
## Recent blogs
* [Bringing automation to orbit determination with Sift](https://www.siftstack.com/mission-critical/orbit-determination-with-sift)
* “The challenge \[with orbit determination] is \[the need] to transform \[its] data into actionable insight. Instead of manually combing through plots or spreadsheets, engineers use Sift to ingest, visualize, and automatically evaluate orbit determination performance. The result is a repeatable workflow that allows teams to verify positioning accuracy with confidence and speed.”
* [Why I joined Sift: Russ Parrish](https://www.siftstack.com/mission-critical/russ-parrish)
* "Russ Parrish is the Head of Design at Sift, where he leads product and brand execution for tools that help hardware engineers test, build, and operate advanced hardware. Over the last 15 years, he's worked across aerospace, robotics, and AI, including roles at SpaceX, Intuitive Surgical, and IBM.”
# Release notes
Source: https://docs.siftstack.com/release-notes/overview
Stay up to date with the latest major montly updates
### Analyze with agents, keep tests visible, and follow every state
This release connects your coding agent to Sift for ingestion, investigation, and approved write actions. It also keeps telemetry visible beside the hardware through network drops so you can move the Runs you choose into Sift when you're ready. You can also follow state changes, read current values, and mark important moments directly in Explore. [Learn more](../release-notes/2026/september-2026).
### Import flight logs, reuse investigations, and shape your Channel tree
Bring PX4 flight logs into Sift in their native format, reopen any Explore session exactly as you left it, and control how Channel names build your Channel tree. [Learn more](../release-notes/2026/august-2026).
### New Panels, faster rendering, and Saved Explorations
This release ships Sift CLI v0.2.0 with HDF5 and TDMS imports, Parquet export, and built-in documentation. It also adds two new Panel types and performance improvements to the Timeseries Panel in Explore.
Explorations can now be saved and reused across Runs, Assets, and Families. [Learn more](../release-notes/2026/july-2026).
### Control access, catch drift, and automate your thresholds
This release puts you in control of who sees your telemetry, helps you catch Run drift with statistical baselines built from related Runs, and lets Rules and Calculated Channels use values from Asset and Run files.
HDF5 and TDMS files now upload directly into Sift, and every new Panel in Explore opens with your preferred settings. [Learn more](../release-notes/2026/may-2026).
April 2026 ships a rebuilt Explore workspace, the ability to use Calculated Channels in Rule logic, and support for referencing Calculated Channels within other Calculated Channel calculations. [Learn more](../release-notes/2026/april-2026).
This release adds data export, per-Channel downsampling controls, relative-time alignment, data source swapping, and the ability to plot and create Annotations in Explore, plus Calculated Channel support in Rules. [Learn more](../release-notes/2026/march-2026).
This release makes Sift more capable and easier to use by helping engineers analyze telemetry in context. [Learn more](../release-notes/2026/february-2026).
This month, we are launching new features that give you more precise control over your telemetry. These capabilities make it easier to share targeted visual reviews and export data for external analysis. [Learn more](../release-notes/2026/january-2026).
2025
This month's updates make it easier to analyze, reuse, and deepen telemetry insights. [Learn more](../release-notes/december-2025).
This month's updates reinforce our mission to make it easy to analyze, recreate, and act on the insights in your telemetry. [Learn more](../release-notes/november-2025).
This month's release strengthens the connection between how data enters Sift and how teams stay supported while working in it. [Learn more](../release-notes/october-2025).
This month's updates build on last month's momentum, bringing faster data ingestion, richer insights, and simpler ways to access your data once it's saved. Together, they highlight how Sift continues to streamline the path from collection to insight.
[Learn more](../release-notes/september-2025).
Each of this month's updates stands strong on its own, but together they tell a story about where Sift is headed. [Learn more](../release-notes/august-2025).
This month's release notes introduce foundational upgrades across the Sift workspace. [Learn more](../release-notes/july-2025).
This month's release notes introduce new ways to bring data into Sift and view it across your tools. [Learn more](../release-notes/june-2025).
This month's release notes introduce a suite of impactful updates designed to enhance visibility, automation, and workspace organization in Sift. [Learn more](../release-notes/may-2025).
# September 2025
Source: https://docs.siftstack.com/release-notes/september-2025
Faster data ingestion, rich insights: simpler access
## Overview
This month's updates build on last month's momentum, bringing faster data ingestion, richer insights, and simpler ways to access your data once it's saved. Together, they highlight how Sift continues to streamline the path from collection to insight.
1. Faster ingestion. Telemetry points are now stored in the database faster. Tail latency for point ingestion has been [cut in half](#improved%3A-faster-telemetry-point-ingestion), making data available more quickly for analysis and downstream processing.
2. Richer insights. Once your data is stored, Explore v2 (Beta) lets you convert it into actionable insights using [new panels](#new%3A-explore-v2-beta-expands-with-geomap%2C-histogram%2C-and-fft-panels%2C-bulk-assignments%2C-and-table-enhancements). The additions of GeoMaps, Histograms, and FFT (Fast Fourier Transform) panels provide ways to analyze telemetry, whether you're tracking location, studying distributions, or examining signals in the frequency domain.
3. Simpler access. After your data is stored in a Run, accessing it is easier. You can now [export a Run's Channels](#new%3A-export-a-run’s-channels-from-its-overview-page) directly from its Overview page for provenance and custom analysis, reducing unnecessary steps while maintaining transparent data access.
## New: Explore v2 (Beta) expands with GeoMap, Histogram, and FFT panels, Bulk Assignments, and Table enhancements
* GeoMap (Beta) panel: In Explore v2 (Beta), you can now use GeoMap panels to explore telemetry by location, enabling spatial analysis of data across regions.
* Histogram panel: You can now create Histogram panels to analyze distribution patterns in telemetry signals.
* FFT (Fast Fourier Transform) panel: Explore v2 (Beta) now includes an FFT panel for frequency-domain analysis, helping identify signal characteristics and periodic behaviors.
* Bulk Assignments: You can now configure multiple Channels at once, reducing repetitive work.
* Int64 and Uint64 support: Explore v2 (Beta) supports int64 and uint64 data types for high-range values.
* Performance improvements: Channel List and Tree views are now faster, improving navigation and workflows.
* UTC offset: You can display UTC offsets in Time Series and Table panels, ensuring consistent time alignment.
* Table panel enhancements: Tables now include text wrapping for text Channels, smart auto-scaling of column widths, and a fix to the Show Carried Values toggle.
* Context Menu in a Table panel: You can now right-click column values in the Table panel to remove a Channel or copy a cell value to the clipboard.
* Date Range Picker updates (Table panel only): The Date Range Picker in Table panel settings has been fixed so that selected ranges apply correctly after “save and close.” It now also includes a Set to Full Range option for easier configuration.
![demo]()
The primary distinction is that v2 builds on v1's foundation with a redesigned experience that introduces:
* **New capabilities**: Multi-chart layouts, cross-source plotting (Runs and Assets), smarter Channel selection, advanced styling, and X+Y zoom.
* **Performance improvements**: Nanosecond resolution, faster rendering, and can plot significantly more Channels and data points.
* **User-centered design**: Built from feedback of teams working with high-frequency, mission-critical data.
## New: Export a Run's Channels from its Overview page
* **Overview**: You can now export Channel data straight from a Run's Overview page, eliminating the need to open Explore and plot Channels first. This streamlines access to Run data and reduces unnecessary steps, enabling faster workflows and simpler selection of all Channels in a Run.
![demo]()
## New: Download a Run's original data file from its Overview page
* **Overview**: When a Run is created by importing a CSV, TDMS, or Parquet file, the original file is now available as an attachment. This improves data provenance, makes it easier to validate the source of your Runs, and provides convenient access for off-Sift analysis. You can download the file directly from the Runs Overview page or from the Run details in Explore v1/v2 (Beta).
![demo]()
## Improved: Faster telemetry point ingestion
* **Overview**: Telemetry points are now stored in the database faster. Tail latency for point ingestion has been cut in half (from \~1s to under 500ms at the 99.99th percentile), making data available more quickly for analysis and downstream processing.
## Improved: Simplified notifications menu
* **Overview**: The Notifications menu was redesigned to provide a consistent experience: where applicable, notifications include a direct link to their destination, while purely informational notifications appear as text only. For example, being tagged in an Annotation will now give you a direct link to open it. The Notifications menu was also cleared to only include the most relevant information, and unnecessary details were removed to make notifications simpler and easier to scan. In-progress and completed Jobs were also removed and are now accessible from the Jobs page.
## Recent blog
* [Scale now or pay later](https://www.siftstack.com/mission-critical/scale-now-or-pay-later) “Sift was purpose-built to address precisely what these generic solutions \[like ClickHouse, InfluxDB, TimescaleDB] overlook: nested structures, evolving schemas, extreme cardinality, and consistent performance throughout the entire telemetry lifecycle.”
# Getting started with Explore (legacy)
Source: https://docs.siftstack.com/tutorials/analyze/explore-v1/tutorial-1
Learn how to work with Explore (legacy)
In this tutorial, you'll learn how to organize and explore telemetry data using Explore v1.
## Prerequisites
* Download the [rover.csv](https://drive.google.com/file/d/1_8MXmOcT2vLh1zXE6uThX-bYytN4XGBS/view?usp=sharing) file.
## Step 1: Import data
1. Select **Runs** in the navigation, then click **Import data**.
2. Upload the **rover.csv** file.
3. Click **New Asset**.
4. In the **Asset** box, enter:
```
fl_rover-asset
```
Replace `fl` with your initials (first and last name). This helps avoid naming conflicts if others are also completing the tutorial on the same Sift instance.
5. In the **Run** box (New), edit the Run name with the following:
```
fl_rover-run
```
Again, replace `fl` with your initials to ensure the Run name is unique within your workspace.
6. Click **Upload**.
7. Click **Close**
## Step 2: Plot data
1. In the **Runs name or description** box, enter the following:
```
fl_rover-run
```
2. In the **Runs** table, click **fl\_rover-run**.
3. Click **Explore Legacy**.
4. In the **Channels** section, click the following Channels:
* **battery\_charge**
* **vehicle\_state**
5. Click **Other charts**.
6. Select **Table Viewer**.
7. In the **Channels** section, under **camera\_2**, click the following Channels:
* **current**
* **temperature**
## Conclusion
In this tutorial, you learned how to organize and explore telemetry data using Explore v1.
# Getting started with Explore
Source: https://docs.siftstack.com/tutorials/analyze/explore/tutorial-1
Build your first multi Panel workspace to visualize and understand your data
In this tutorial, you will get started with [Explore](/documentation/reference/explore-settings) by using [Panels](/documentation/reference/explore-settings#panel-types) to analyze telemetry data from a Low Earth Orbit (LEO) satellite. You will visualize Channels, build a flexible, [multi-Panel layout](/documentation/reference/explore-settings#layout), and learn how to share your analysis to collaborate with others.
## Prerequisites
* Import [this](https://drive.google.com/uc?export=download\&id=1FWTLYMzxRNRNUESuLgHWBU0HHJsfXlFk) CSV file (`leo.csv`) to Sift, creating a new Asset named `leo_satellite` and a Run named `leo_satellite_simulation`
* If you are unsure how to import a CSV, see [Import a CSV](/documentation/ingest/data-import/import-data-from-a-file) for step by step instructions.
## Step 1: Open your data
Open your dataset in Explore:
1. Select **Runs** in the navigation, then search for and select **leo\_satellite\_simulation**.
2. Click **Explore**.
## Step 2: Build a multi-Panel layout
Panels let you visualize and analyze your data in Explore. Use a **Timeseries** Panel to view trends and a **Table** Panel to inspect values, then create a multi-Panel layout to better understand your data. Other Panel types are also available. See [Panels](/documentation/reference/explore-settings#panel-types).
1. Click the **Timeseries** Panel.
2. In the **Channels** tab, under the **EPS** section, click:
* **Battery\_Current**
* **Solar\_Array\_Current**
3. Click **Add**, then select **Timeseries**.
4. In the **Channels** tab, under the **Payload** section, click:
* **Payload\_Power\_Draw**
5. Click **Add**, then select **Table**.
6. In the **Channels** tab, under the **Comms** section, click the following Channels:
* **Data\_Backlog**
* **Downlink\_Data\_Rate**
* **Downlink\_SNR**
* **Tx\_Power**
7. Drag the **Table 1** tab to the bottom of the workspace.
8. Drag the **Timeseries 2** tab to the bottom right.
## Step 3: Share your workspace
Share your workspace to let others view the same Panels and layout:
1. Click **Share**.
2. Click **Copy**.
## Conclusion
In this tutorial, you used Explore to build a multi-Panel layout to analyze telemetry data. You learned how to visualize Channels, use Panels to explore your data, and share your workspace to collaborate with others.
# Getting started with streaming data to Sift using Python
Source: https://docs.siftstack.com/tutorials/ingest/tutorial-1
Learn how to send live telemetry from a Python script to Sift and view it in real time
This tutorial shows how to stream telemetry from a Python script to Sift using the [Sift Python client library](https://sift-stack.github.io/sift/python/latest/).
You will install the client library, configure authentication, define telemetry signals, and send time-series data to Sift from a Python script.
## Prerequisites
* Basic understanding of how [Assets](../../documentation/ingest/stream/organize-streamed-data-into-assets-and-runs), [Channels](../../documentation/reference/channels-reference), and [Runs](../../documentation/reference/runs-reference) relate to each other in Sift
* Working knowledge of Python and familiarity with `async` / `await`
* A Sift [API key](../../documentation/manage/set-up-api-access) and your Sift [gRPC and REST base URLs](../../documentation/manage/set-up-api-access)
## Scenario
Imagine a robotic vehicle that continuously reports telemetry such as velocity and internal temperature. In this tutorial, a Python script simulates the system by generating these measurements every 0.5 seconds.
## Step 1: Obtain the Python example project
Clone the Sift repository and navigate to the directory that contains the Python streaming ingestion example used in this tutorial, including the script and a `requirements.txt` file that lists the dependencies required to run it.
```bash theme={null}
git clone https://github.com/sift-stack/sift.git
cd sift/python/examples/ingestion-tutorial
```
## Step 2: Install the Python dependencies
From a Python environment, run the following command to install the dependencies required for the Python streaming ingestion example:
```bash theme={null}
pip install -r requirements.txt
```
## Step 3: Configure authentication
Create a `.env` file and add your Sift API key along with your Sift gRPC and REST URLs:
```bash id="v9s3dk" theme={null}
touch .env
```
```bash id="8b2c6p" theme={null}
SIFT_API_KEY=your_api_key_here
SIFT_GRPC_URL=https://your-grpc-url
SIFT_REST_URL=https://your-rest-url
```
## Step 4: Run the example and view streamed data in Sift
Run the Python script to start streaming telemetry to Sift. The script runs continuously until you stop it (for example, with Ctrl+C), and the telemetry appears live in [Explore](/documentation/reference/explore-settings):
```bash theme={null}
python stream.py
```
1. Check the terminal for the generated Run name (for example, **robot\_vehicle\_…\_run**). Then, in **Sift**, locate the **Run name or description** field and enter that name.
2. Click **Explore**.
3. Click **Live**.
4. In the **Channels** tab, select the following Channels:
* **temperature**
* **velocity**
## Step 5: Understand the ingestion workflow
The ingestion process follows a clear sequence. The steps below summarize how telemetry is structured, configured, and streamed to Sift.
* At a high level, streaming telemetry to Sift involves defining structure first, then sending timestamped data that conforms to that structure.
* In this script, authentication is configured using `SiftConnectionConfig`, and a `SiftClient` is created to communicate with your Sift environment.
* A `FlowConfigPy` defines the telemetry schema, and each `ChannelConfigPy` declares an individual signal with its name, unit, and data type.
* These types are used directly from `sift_stream_bindings` to avoid the CPU-bound overhead of the ergonomic Python type conversions.
**Performance**: Using `sift_stream_bindings` types directly avoids the CPU-bound bottlenecks that occur when converting between `Flow` and `FlowPy` types.
* An `IngestionConfigFormPy` associates that schema with an Asset, and a `RunCreate` defines the session that will group all incoming telemetry.
* Once this ingestion context is established, the script opens a streaming ingestion session over gRPC and begins sending timestamped flows in real time.
* Each flow is constructed directly as a `FlowPy` object using `ChannelValuePy`, `ValuePy`, and `TimeValuePy` from `sift_stream_bindings`, ensuring the data matches the defined schema. These flows are transmitted over the open stream and immediately appear in Sift under the specified Run.
**Performance:** `ingest_client` should be opened once and reused for the entire duration of your program. Recreating it per batch or per iteration creates a new ingestion config each time, which introduces significant performance overhead.
## Conclusion
In this tutorial, you cloned a Python example project, installed its dependencies, configured authentication, and streamed telemetry to Sift using the Sift Python client library.
You also viewed the live telemetry in Explore and saw how streaming ingestion sends structured time-series data to Sift in real time.
# Getting started with streaming data to Sift using Rust
Source: https://docs.siftstack.com/tutorials/ingest/tutorial-2
Learn how to send live telemetry from a Rust application to Sift and view it in real time
This tutorial shows how to stream telemetry from a Rust application to Sift using the [Sift Stream Rust library](https://crates.io/crates/sift_stream).
You will install the library, configure authentication, define telemetry signals, and send time-series data to Sift from a Rust application.
## Prerequisites
* Basic understanding of how [Assets](../../documentation/ingest/stream/organize-streamed-data-into-assets-and-runs), [Channels](../../documentation/reference/channels-reference), and [Runs](../../documentation/reference/runs-reference) relate to each other in Sift
* Working knowledge of Rust and familiarity with async programming
* A Sift [API key](../../documentation/manage/set-up-api-access) and your Sift [gRPC base URL](../../documentation/manage/set-up-api-access)
## Scenario
Imagine a robotic vehicle that continuously reports telemetry during operation. The vehicle produces two signals: its current velocity and its internal temperature. In this tutorial, a Rust application simulates this system by generating measurements every 0.5 seconds and streaming them to Sift.
## Step 1: Obtain the Rust example project
Clone the Sift repository and navigate to the `sift_stream` crate directory, which contains the `Cargo.toml` file for the streaming client library and the Rust example used in this tutorial:
```bash theme={null}
git clone https://github.com/sift-stack/sift.git
cd sift/rust/crates/sift_stream
```
## Step 2: Configure authentication
Create a `.env` file so the Rust example can authenticate with your Sift environment. Add your Sift API key and gRPC URL as environment variables:
```bash theme={null}
touch .env
```
```bash theme={null}
SIFT_API_KEY=your_api_key_here
SIFT_GRPC_URL=https://your-grpc-url
```
## Step 3: Run the application and view streamed data in Sift
Run the Rust example to start streaming telemetry to Sift. The example runs for about 10 minutes before stopping automatically.
1. From the `sift_stream` directory (the folder containing `Cargo.toml`), run the example:
```bash theme={null}
cargo run --example ingestion-tutorial
```
2. Check the terminal for the generated Run name (for example, **robot\_vehicle\_...\_run**). Then, in **Sift**, locate the **Run name or description** field and enter that name.
3. In the **Runs** table, click the Run name.
4. Click **Explore**.
5. Click **Live**.
6. In the **Channels** tab, select the following Channels:
* **temperature**
* **velocity**
## Step 4: Understand the ingestion workflow
The ingestion process follows a sequence that defines the telemetry structure, establishes an ingestion session, and streams timestamped data to Sift.
* The application begins by loading authentication credentials from the `.env` file using `dotenvy`. These values are used to create a `Credentials` configuration that allows the client to authenticate with your Sift environment.
* A streaming client is then initialized using `SiftStreamBuilder`, which establishes the ingestion connection and prepares the client to send telemetry.
* Telemetry sent to Sift must follow a defined schema. In this example, a `FlowConfig` defines the telemetry structure, and each `ChannelConfig` represents an individual signal.
* Two Channels are defined: velocity and temperature. These Channels are grouped together within a Flow named `vehicle_metrics`.
* An `IngestionConfigForm` associates the telemetry schema with an Asset and a unique client key. A `RunForm` then defines the session that groups incoming telemetry.
* The Run is created as part of the same streaming ingestion session initialized by `SiftStreamBuilder`. This means the Run is established on the same connection used to send telemetry data.
* Once the ingestion context is established, the application begins generating mock telemetry values and sending them to Sift using `sift_stream.send()`.
Each message is sent as a `Flow` containing:
* a timestamp
* the Flow name
* values for each Channel
* After the loop completes, the program calls `finish()` to ensure any queued telemetry is transmitted before the streaming session closes.
## Conclusion
In this tutorial, you streamed telemetry from a Rust application to Sift using the Sift Stream Rust library. You configured authentication, defined telemetry signals, and sent timestamped data to Sift in real time. You also viewed the live telemetry in Sift to understand how streamed data appears during ingestion.
# Integrate an Identity Provider (IdP) with Sift
Source: https://docs.siftstack.com/tutorials/manage/tutorial-1
Learn how to integrate an external IdP with Sift using Microsoft Entra ID
This tutorial shows how to integrate an external Identity Provider (IdP) with Sift using Microsoft Entra ID as an example. The same process applies to other IdPs that support push provisioning. After setup, your IdP can synchronize external groups and manage their permissions in Sift.
## Prerequisites
* Administrator access to Sift.
* Administrator access to your Identity Provider (IdP).
* IdP integration enabled for your Sift account by your Sift account representative.
## Step 1: Understand IdP integration in Sift
Sift integrates with Identity Providers (IdPs) that support **push provisioning** to keep groups synchronized. After setup, Sift automatically syncs with your IdP every 24 hours, and you can also run a manual sync.
Groups provisioned from an IdP appear in Sift as **external groups**. You can manage their permissions in Sift, but membership must be managed in the IdP. IdPs that require pull provisioning, such as Google Workspace, require custom integration.
## Step 2: Generate the SCIM endpoint URL and access token in Sift
After IdP integration is enabled for your account, generate the SCIM credentials needed to connect your IdP to Sift.
1. Click your profile icon, which shows the first initial of your account name.
2. Select **Manage**.
3. Click **Manage Identity Provider**.
4. Click **Generate Token**.
5. Copy URL: In the **SCIM Server URL** section, click **Copy**.
6. Copy token: In the **Token** section, click **Copy**.
7. Click **Close**.
## Step 3: Configure your IdP (Microsoft Entra ID)
Use the SCIM credentials generated in Sift to configure your IdP. In Microsoft Entra ID, enter the **SCIM Server URL** in the **Tenant URL** field and the **Token** in the **Secret Token** field.
Save the configuration, test the connection, and start provisioning. After provisioning begins, changes from your IdP will sync to Sift during the next automatic or manual sync.
## Step 4: Sync and verify external groups in Sift
Rather than waiting 24 hours for the automatic sync, run a manual sync to import external groups into Sift immediately.
1. Click **Manage Identity Provider**.
2. Click **Sync Organization**.
## Conclusion
You integrated an Identity Provider (IdP) with Sift using Microsoft Entra ID. External groups can now sync to Sift, where you can manage their permissions. Group membership must be managed in the IdP and will sync to Sift automatically or through a manual sync.
The same process can be used for other IdPs that support push provisioning.
## Resources
* [Identity Provider (IdP) settings](/documentation/reference/manage/idp-settings)
* [Connect an Identity Provider (IdP) to Sift](/documentation/manage/connect-an-identity-provider)
# Getting started with Metadata in Sift
Source: https://docs.siftstack.com/tutorials/manage/tutorial-2
Learn how to create Metadata keys and filter resources in Sift
In this tutorial, you'll create and apply Metadata in Sift, then use it to filter a Run. Metadata helps you organize resources using consistent key-value pairs across your workspace.
## Prerequisites
* At least one Run in your workspace
## Step 1: Understand Metadata in Sift
Metadata in Sift uses key-value pairs to organize and filter resources across your workspace. Each Metadata key is defined once and assigned a data type to ensure consistent values. Metadata values support three data types: **string**, **number**, and **boolean**.
You can apply Metadata to the following resources: **Runs**, **Assets**, **Annotations**, **Rules**, **Reports**, **Report Templates**, **Campaigns**, **Calculated Channels**, and **User-Defined Functions**.
## Step 2: Create a Metadata key
Let's create a numeric Metadata key.
1. Click your profile icon, which shows the first initial of your account name.
2. Select **Manage**.
3. In the **Manage** navigation menu, click **Metadata**.
4. Click **+ Create Metadata Key**.
5. In the **Name** box, enter the following:
```
priority_level
```
6. In the **Type** list, select **number**.
7. Click **Create**.
## Step 3: Add a Metadata key to a Run
Now, let's add the created Metadata key to a Run.
1. Select **Runs** in the navigation.
2. In the **Runs** table, click the name of a Run to add the Metadata key you just created.
Later in [Step 5](#step-5-optional-remove-metadata-key), you will have the option to remove the Metadata key. For now, we will use this Run as an example to demonstrate how Runs, like all Sift resources, can support Metadata.
3. Click **Edit**.
4. Click **+ Add Metadata**.
5. In the **Select or create key** list, select **priority\_level**.
6. In the **Enter number** list, enter the following (where `1` on the 1-5 scale refers to high priority):
```
1
```
7. Click **Save changes**.
## Step 4: Locate a Run by its Metadata key
Let's locate the Run using the **priority\_level** Metadata key.
1. Select **Runs** in the navigation, then click **Metadata**.
2. In the **Select key** list, select **priority\_level**.
3. In the **Enter number** box, enter **1**.
## Step 5 (optional): Remove Metadata key
To remove the Metadata key added to the Run, consider the following steps:
1. In the Run's overview page, click **Edit**.
2. In the **Metadata** section, locate the **priority\_level** key.
3. Click **Remove**.
4. Click **Save changes**.
## Conclusion
You created a Metadata key, applied it to a Run, and used it to filter resources in Sift. Metadata helps you organize your workspace and quickly locate the data you need.
## Resources
* Reference: [Metadata](../../documentation/manage/organize-and-filter-resources-with-metadata)
* How-to guide: [Manage Metadata](../../documentation/manage/organize-and-filter-resources-with-metadata)
# Tutorials
Source: https://docs.siftstack.com/tutorials/overview
Explore step-by-step tutorials that show you how to use Sift's full range of features and capabilities
Send telemetry to Sift from a Python script or a Rust application.
Send live telemetry from a Python script to Sift and view it in real time.
Send live telemetry from a Rust application to Sift and view it in real time.
Explore telemetry data using Explore, Sift's visualization workspace.
Explore telemetry data using Explore (legacy).
Build a multi-Panel workspace to visualize and understand your telemetry.
Control access to your workspace and organize resources for faster searching.
Integrate an external Identity Provider (IdP) with Sift using Microsoft Entra ID.
Create Metadata keys and use them to filter resources in Sift.
Use attribute-based access control (ABAC) to restrict access to specific resources.