Ingestion
Overview
In the UI, Sift allows you to upload supported files for analysis.
File types
Sift allows you to upload certain file types directly through the web interface. This is the fastest way to bring data into Sift for review, visualization, or analysis without any setup or code. Sift allows you to upload the following file types through the UI:
| File type | Details |
|---|---|
.csv | Learn more |
.tdms | |
.parquet | Learn more |
General configuration
Right after uploading a supported file type, Sift requires basic configuration to define the associated Asset and Run, specify the starting row of data, and determine how timestamps should be interpreted.
| Field | Description |
|---|---|
| Asset | The system that generated the data. Select an existing Asset or create a new one. |
| Run | A specific data collection session. Defaults to the file name but can be edited. |
| First data row | Row number where time-series data begins. Used to skip extra header rows. This setting is available for CSV files but not for Parquet files. |
| Timestamp column | The column containing timestamps. Auto-detected but editable. |
| Timestamp format | The format of the timestamp column. Must be one of the supported types. |
Timestamp formats
The following table lists the timestamp formats supported by Sift. These formats must match the structure of the selected timestamp column during a supported file type upload.
| Format | Description |
|---|---|
| rfc3339 | 2023-01-02T15:04:05Z |
| datetime | 2023-01-02 15:04:05 |
| UNIX | Seconds since epoch |
| unix_millis | Milliseconds since epoch |
| unix_micros | Microseconds since epoch |
| unix_nanos | Nanoseconds since epoch |
| nanoseconds | Relative time in nanoseconds |
| microseconds | Relative time in microseconds |
| milliseconds | Relative time in milliseconds |
| seconds | Relative time in seconds |
| minutes | Relative time in minutes |
| hours | Relative time in hours |
Channel configuration
During a supported file type upload, each non-timestamp column is treated as a Channel representing a stream of time-series data. Sift displays a configuration table where each Channel can be reviewed and customized before ingestion.
| Column | Description |
|---|---|
| Checkbox | Determines whether the Channel is included in the upload. |
| Name | Autodetected Channel name from the source file. Editable if incorrect. |
| Data type | Autodetected supported data type. Editable if incorrect. |
| Units | Optional. Unit of the Channel (for example, °C). |
| Description | Optional. Description of the Channel. |
Data type
After uploading a supported file type (in the UI), each Channel must be assigned a data type. The following data types are supported:
| Data type | Description |
|---|---|
| double | 64-bit floating point number |
| float | 32-bit floating point number |
| int32 | 32-bit signed integer |
| uint32 | 32-bit unsigned integer |
| int64 | 64-bit signed integer |
| uint64 | 64-bit unsigned integer |
| bytes | Sequence of raw bytes |
| bool | Boolean value (true or false) |
| string | Text or alphanumeric values |
| enum | Categorical value with a fixed set of possible strings |
| bit field | Integer where each bit represents a distinct flag or condition |
CSV
Format
A CSV to be uploaded to Sift must include a header row, a timestamp column, and one or more telemetry Channels, each formatted according to supported conventions.
| Requirement | Description |
|---|---|
| First row | It must contain column headers. |
| Timestamp column | One column must contain timestamps, and the recommended name for this column is timestamp. |
| Data columns | All other columns are treated as telemetry Channels. |
Parquet
Format
Sift supports Parquet files with a flat schema, where each telemetry Channel is represented as an individual column.
| Requirement | Description |
|---|---|
| Timestamp column | One column must contain timestamps. |
| Channel columns | Each additional column is treated as a telemetry Channel. |
Maps, lists, and structs are not supported and are ignored during upload.