Import data from a file

After completing this workflow, you can import data from a file into Sift and have its Channels available for investigation in Explore.

Before you begin

You are familiar with how Sift organizes data into Assets, Channels, and Runs.
Your file is in a supported import format.
Your file includes a timestamp column in a supported timestamp format.

How data file import works

Importing a file adds its data to an Asset and a Run. For each, you select an existing one or create a new one. Each import file can be up to 80 GB. Sift offers the following options for importing a file.

Import a file

Select Runs in the navigation, then click Import data.
Select your file.
Based on your file format, configure the import settings.
Skip manual Channel setup: If your CSV or Parquet file has an embedded Channel configuration, such as a file exported from Sift or one you built with the configuration added, Sift pre-fills the Channel Configuration table during import. Learn more about embedded Channel configurations.
Click Upload.

Import by upload

Send a request to CreateDataImportFromUpload at your REST base URL with your API key and a configuration object for your file format.
curl -X POST "$SIFT_REST_URL/api/v2/data-imports:upload" \ -H "Authorization: Bearer $SIFT_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "csvConfig": { "assetName": "my-asset", "firstDataRow": 2, "timeColumn": {"columnNumber": 1, "format": "TIME_FORMAT_ABSOLUTE_RFC3339"}, "dataColumns": {"2": {"name": "my-channel", "dataType": "CHANNEL_DATA_TYPE_DOUBLE"}} } }'
The response returns the uploadUrl for step 2 and the dataImportId for step 3.
Map each column: In dataColumns, add one entry per column you want to import. Each key is the column number, starting at 1, and each value sets the Channel name and dataType. For files with many columns, save the request body to a JSON file and pass it with -d @config.json.

Skip manual Channel setup: To use a file’s embedded Channel configuration instead of timeColumn and dataColumns, set useEmbeddedConfig to true. For CSV, set it in csvConfig. For Parquet, set it in parquetConfig.flatDataset, and set footerOffset and footerLength in parquetConfig. Learn more about embedded Channel configurations.
Upload the file in a POST request to the uploadUrl from the CreateDataImportFromUpload response.
curl -X POST "$UPLOAD_URL" \ -H "Authorization: Bearer $SIFT_API_KEY" \ -H 'Content-Disposition: attachment; filename="test-data.csv"' \ --data-binary @test-data.csv
Upload timeouts: Sift queues the import as soon as it receives the request, so a client-side timeout on the upload does not mean the import failed. To confirm, check the status in step 3.

Optional: Check the import status by sending the dataImportId from the step 1 response to GetDataImport.

curl "$SIFT_REST_URL/api/v2/data-imports/$DATA_IMPORT_ID" \
  -H "Authorization: Bearer $SIFT_API_KEY"

Import from a URL

Send a request to CreateDataImportFromUrl at your REST base URL with your API key, the file’s url, and a configuration object for your file format. Sift supports HTTP and S3 URLs.
curl -X POST "$SIFT_REST_URL/api/v2/data-imports:url" \ -H "Authorization: Bearer $SIFT_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "url": "https://example.com/test-data.csv", "csvConfig": { "assetName": "my-asset", "firstDataRow": 2, "timeColumn": {"columnNumber": 1, "format": "TIME_FORMAT_ABSOLUTE_RFC3339"}, "dataColumns": {"2": {"name": "my-channel", "dataType": "CHANNEL_DATA_TYPE_DOUBLE"}} } }'
The response returns the dataImportId for step 2.
Map each column: In dataColumns, add one entry per column you want to import. Each key is the column number, starting at 1, and each value sets the Channel name and dataType. For files with many columns, save the request body to a JSON file and pass it with -d @config.json.

Skip manual Channel setup: To use a file’s embedded Channel configuration instead of timeColumn and dataColumns, set useEmbeddedConfig to true. For CSV, set it in csvConfig. For Parquet, set it in parquetConfig.flatDataset, and set footerOffset and footerLength in parquetConfig. Learn more about embedded Channel configurations.

Optional: Check the import status by sending the dataImportId from the step 1 response to GetDataImport.

curl "$SIFT_REST_URL/api/v2/data-imports/$DATA_IMPORT_ID" \
  -H "Authorization: Bearer $SIFT_API_KEY"

Add file data to an existing Run

You can import data from a file in any supported format, such as Parquet or CSV, into a Run that already exists. For example, if your test system streams data to Sift during a test and exports high-rate data afterward, import the high-rate data file into the same Run. A Run can hold the same signal at more than one rate. Sift matches an imported Channel to a Channel already in the Run when both have the same name, data type, and unit.

For Asset, select Existing, then select the Run’s Asset.
For Run, select Existing, then select the Run.

In the configuration object for your file format (for example, csvConfig), set assetName to the Run’s Asset.
Set runId to the Run’s ID.

Verify

The imported Channels appear in the chosen Run under the chosen Asset.
The Channels plot as expected when you open the Run in Explore.
If you imported through the API, GetDataImport returns DATA_IMPORT_STATUS_SUCCEEDED.

Get started

Ingest telemetry

Investigate

Detect and review

Transform

Export

Sift CLI

Integrate

Manage workspace

Reference

Import data from a file

Before you begin

How data file import works

Import a file

Add file data to an existing Run

Verify

Reference

​Before you begin

​How data file import works

​Import a file

​Add file data to an existing Run

​Verify

​Reference

Before you begin

How data file import works

Import a file

Add file data to an existing Run

Verify

Reference