sift/data_imports/v2/data_imports.proto
Ch10Config
| Field | Type | Label | Description |
|---|---|---|---|
| asset_name | string | ||
| run_name | string | ||
| scale_values | bool |
CreateDataImportFromUploadRequest
| Field | Type | Label | Description |
|---|---|---|---|
| csv_config | CsvConfig | ||
| ch10_config | Ch10Config | ||
| tdms_config | TDMSConfig | ||
| parquet_config | ParquetConfig |
CreateDataImportFromUploadResponse
| Field | Type | Label | Description |
|---|---|---|---|
| upload_url | string | ||
| data_import_id | string |
CreateDataImportFromUrlRequest
| Field | Type | Label | Description |
|---|---|---|---|
| url | string | The url to import. HTTP and S3 urls are supported. If you need to import non-public S3 objects, please contact Sift to set that up. | |
| csv_config | CsvConfig | ||
| ch10_config | Ch10Config | ||
| tdms_config | TDMSConfig | ||
| parquet_config | ParquetConfig |
CreateDataImportFromUrlResponse
| Field | Type | Label | Description |
|---|---|---|---|
| data_import_id | string |
CsvConfig
| Field | Type | Label | Description |
|---|---|---|---|
| asset_name | string | ||
| run_name | string | ||
| run_id | string | The id of the run to add this data to. If set, run_name is ignored. | |
| first_data_row | uint32 | The first row to start reading as data. Can be used to skip header rows. The first row in the file is 1. | |
| time_column | CsvTimeColumn | optional | |
| data_columns | CsvConfig.DataColumnsEntry | repeated | A map from column number (1-indexed) to the channel configuration for that column. |
| num_rows | uint64 | optional | This will be read on upload from the file if not set. |
CsvConfig.DataColumnsEntry
| Field | Type | Label | Description |
|---|---|---|---|
| key | uint32 | ||
| value | sift.common.type.v1.ChannelConfig |
CsvTimeColumn
| Field | Type | Label | Description |
|---|---|---|---|
| column_number | uint32 | The column number (1-indexed) of the time column. | |
| format | TimeFormat | ||
| relative_start_time | google.protobuf.Timestamp | optional |
DataImport
| Field | Type | Label | Description |
|---|---|---|---|
| data_import_id | string | ||
| source_url | string | ||
| status | DataImportStatus | ||
| error_message | string | ||
| created_date | google.protobuf.Timestamp | ||
| modified_date | google.protobuf.Timestamp | ||
| csv_config | CsvConfig | ||
| ch10_config | Ch10Config | ||
| tdms_config | TDMSConfig | ||
| parquet_config | ParquetConfig | ||
| run_id | string | optional | The run id will be set if the data import ingests to a run once the run is available. |
| report_id | string | optional | The report id will be set if the data import creates a report once the report is available. |
| asset_id | string | optional | |
| data_start_time | google.protobuf.Timestamp | optional | |
| data_stop_time | google.protobuf.Timestamp | optional |
DetectConfigRequest
| Field | Type | Label | Description |
|---|---|---|---|
| data | bytes | ||
| type | DataTypeKey |
DetectConfigResponse
| Field | Type | Label | Description |
|---|---|---|---|
| csv_config | CsvConfig | ||
| parquet_config | ParquetConfig |
GetDataImportRequest
| Field | Type | Label | Description |
|---|---|---|---|
| data_import_id | string |
GetDataImportResponse
| Field | Type | Label | Description |
|---|---|---|---|
| data_import | DataImport |
ListDataImportsRequest
| Field | Type | Label | Description |
|---|---|---|---|
| page_size | uint32 | The maximum number of data imports to return. The service may return fewer than this value. If unspecified, at most 50 data imports will be returned. The maximum value is 1000; values above 1000 will be coerced to 1000. Optional. | |
| page_token | string | A page token, received from a previous ListDataImports call. Provide this to retrieve the subsequent page. When paginating, all other parameters provided to ListDataImports must match the call that provided the page token. Optional. | |
| filter | string | A Common Expression Language (CEL) filter string. Available fields to filter by are data_import_id, source_url, status. For further information about how to use CELs, please refer to this guide. | |
| order_by | string | How to order the retrieved data imports. Formatted as a comma-separated string i.e. “FIELD_NAME[ desc],…”. Available fields to order_by are created_date and modified_date. If left empty, items are ordered by created_date in ascending order (oldest-first). For more information about the format of this field, read this Example: “created_date desc,modified_date” |
ListDataImportsResponse
| Field | Type | Label | Description |
|---|---|---|---|
| data_imports | DataImport | repeated | |
| next_page_token | string |
ParquetConfig
| Field | Type | Label | Description |
|---|---|---|---|
| asset_name | string | ||
| run_name | string | ||
| run_id | string | The id of the run to add this data to. If set, run_name is ignored. | |
| oneof config.flat_dataset | ParquetFlatDatasetConfig | ||
| footer_offset | uint64 | ||
| footer_length | uint32 | ||
| complex_types_import_mode | ParquetComplexTypesImportMode |
ParquetDataColumn
| Field | Type | Label | Description |
|---|---|---|---|
| path | string | ||
| channel_config | sift.common.type.v1.ChannelConfig |
ParquetFlatDatasetConfig
| Field | Type | Label | Description |
|---|---|---|---|
| time_column | ParquetTimeColumn | ||
| data_columns | ParquetDataColumn | repeated |
ParquetTimeColumn
| Field | Type | Label | Description |
|---|---|---|---|
| path | string | ||
| format | TimeFormat | ||
| relative_start_time | google.protobuf.Timestamp | optional |
RetryDataImportRequest
| Field | Type | Label | Description |
|---|---|---|---|
| data_import_id | string | data_import_id is the id of the data import to retry. You can only retry an import that is a “url” based import (created with CreateDataImportFromUrl) and is in a failed state. |
RetryDataImportResponse
TDMSConfig
| Field | Type | Label | Description |
|---|---|---|---|
| asset_name | string | ||
| run_name | string | ||
| start_time_override | google.protobuf.Timestamp | Override the wf_start_time metadata field for all channels. Useful if your waveform channels have wf_increment but no wf_start_time (Veristand is guilty of this). | |
| file_size | uint64 | optional | The file size in bytes. If the file has truncated chunks, this will be required to pass validation. |
DataImportStatus
| Name | Number | Description |
|---|---|---|
| DATA_IMPORT_STATUS_UNSPECIFIED | 0 | |
| DATA_IMPORT_STATUS_PENDING | 1 | |
| DATA_IMPORT_STATUS_IN_PROGRESS | 2 | |
| DATA_IMPORT_STATUS_SUCCEEDED | 3 | |
| DATA_IMPORT_STATUS_FAILED | 4 |
DataTypeKey
| Name | Number | Description |
|---|---|---|
| DATA_TYPE_KEY_UNSPECIFIED | 0 | |
| DATA_TYPE_KEY_CSV | 1 | |
| DATA_TYPE_KEY_TDMS | 2 | |
| DATA_TYPE_KEY_CH10 | 3 | |
| DATA_TYPE_KEY_PARQUET_FLATDATASET | 4 |
ParquetComplexTypesImportMode
| Name | Number | Description |
|---|---|---|
| PARQUET_COMPLEX_TYPES_IMPORT_MODE_UNSPECIFIED | 0 | |
| PARQUET_COMPLEX_TYPES_IMPORT_MODE_IGNORE | 1 | Ignore complex types and do not ingest them. |
| PARQUET_COMPLEX_TYPES_IMPORT_MODE_BOTH | 2 | Import complex types as both Arrow bytes and JSON strings. |
| PARQUET_COMPLEX_TYPES_IMPORT_MODE_STRING | 3 | Import complex types as only JSON strings. |
| PARQUET_COMPLEX_TYPES_IMPORT_MODE_BYTES | 4 | Import complex types as only Arrow bytes. |
TimeFormat
| Name | Number | Description |
|---|---|---|
| TIME_FORMAT_UNSPECIFIED | 0 | |
| TIME_FORMAT_RELATIVE_NANOSECONDS | 1 | |
| TIME_FORMAT_RELATIVE_MICROSECONDS | 2 | |
| TIME_FORMAT_RELATIVE_MILLISECONDS | 3 | |
| TIME_FORMAT_RELATIVE_SECONDS | 4 | |
| TIME_FORMAT_RELATIVE_MINUTES | 5 | |
| TIME_FORMAT_RELATIVE_HOURS | 6 | |
| TIME_FORMAT_ABSOLUTE_RFC3339 | 10 | |
| TIME_FORMAT_ABSOLUTE_DATETIME | 11 | |
| TIME_FORMAT_ABSOLUTE_UNIX_SECONDS | 12 | |
| TIME_FORMAT_ABSOLUTE_UNIX_MILLISECONDS | 13 | |
| TIME_FORMAT_ABSOLUTE_UNIX_MICROSECONDS | 14 | |
| TIME_FORMAT_ABSOLUTE_UNIX_NANOSECONDS | 15 |
DataImportService
| Method Name | Request Type | Response Type | Description |
|---|---|---|---|
| CreateDataImportFromUrl | CreateDataImportFromUrlRequest | CreateDataImportFromUrlResponse | |
| CreateDataImportFromUpload | CreateDataImportFromUploadRequest | CreateDataImportFromUploadResponse | |
| DetectConfig | DetectConfigRequest | DetectConfigResponse | |
| ListDataImports | ListDataImportsRequest | ListDataImportsResponse | |
| RetryDataImport | RetryDataImportRequest | RetryDataImportResponse | |
| GetDataImport | GetDataImportRequest | GetDataImportResponse |