> ## Documentation Index > Fetch the complete documentation index at: https://docs.siftstack.com/llms.txt > Use this file to discover all available pages before exploring further. # CreateDataImportFromUrl > Creates a data import from a url. ## OpenAPI ````yaml /openapi.json post /api/v2/data-imports:url openapi: 3.0.0 info: title: Sift API version: '1.0' servers: - url: https://api.siftstack.com description: Production - url: https://gov.api.siftstack.com description: Gov security: - BearerAuth: [] tags: - name: TagService description: Service to programmatically interact with [tags](/glossary#tag). - name: ProtobufDescriptorService description: >- Service to programmatically interact with protobuf desciptors used for protobuf ingestion. externalDocs: description: Read more about what protobuf ingestion. url: /ingestion/protobuf_ingestion - name: ApiKeyService - name: UnitService - name: ChannelSchemaService description: Service to programmatically interact with channel schemas - name: PingService - name: SavedSearchService - name: IngestService - name: UserDefinedFunctionService - name: RoleService - name: DataImportService - name: AnnotationLogService description: >- Service to programmatically interact with [annotation logs](/glossary#annotation). externalDocs: description: Read more about annotations. url: >- https://customer.support.siftstack.com/servicedesk/customer/portal/2/article/265486685 - name: ExportService - name: AnnotationService description: Service to programmatically interact with annotations. externalDocs: description: Read more about annotations. url: >- https://customer.support.siftstack.com/servicedesk/customer/portal/2/article/265486685 - name: RuleEvaluationService description: Service to evaluate rules. externalDocs: description: Read more about what rules are. url: >- https://customer.support.siftstack.com/servicedesk/customer/portal/2/article/265421102 - name: CalculatedChannelsService description: Service to programmatically interact with calculated channels. externalDocs: description: Read more about calculated channels. url: >- https://customer.support.siftstack.com/servicedesk/customer/portal/2/article/265421153 - name: CalculatedChannelService - name: MeService - name: UserService description: Service to programmatically interact with user objects. - name: RuleService description: Service to programmatically interact with rules. externalDocs: description: Read more about what rules are. url: >- https://customer.support.siftstack.com/servicedesk/customer/portal/2/article/265421102 - name: RemoteFileService - name: CampaignService - name: ReportTemplateService - name: JobService - name: DlqErrorsService - name: WebhookService - name: DataService description: Service to query data - name: ViewService description: Service to programmatically interact with views. externalDocs: description: Read more about what views are. url: >- https://customer.support.siftstack.com/servicedesk/customer/portal/2/article/298188809 - name: AssetService description: Service to programmatically interact with [assets](/glossary#asset). externalDocs: description: Read more about what assets are. url: /data-model - name: UserGroupService - name: RunService description: Service to programmatically interact with [runs](/glossary#run). externalDocs: description: Read more about what runs are. url: >- https://customer.support.siftstack.com/servicedesk/customer/portal/2/article/265454053 - name: MetadataService - name: NotificationService description: Service to programmatically interact with in-app notifications. - name: PanelConfigurationService description: Service to programmatically interact with panel configurations. - name: IngestionConfigService description: >- Service to programmatically interact with [ingestion configs](/glossary#ingestion-config). externalDocs: description: Read more about what ingestion configs are. url: /ingestion/creating-amend-ingestion-config - name: ReportService - name: ExternalSyncService - name: ChannelService description: Service to programmatically interact with [channels](/glossary#channel). externalDocs: description: Read more about what channels are. url: >- https://customer.support.siftstack.com/servicedesk/customer/portal/2/article/265453943 paths: /api/v2/data-imports:url: post: tags: - DataImportService summary: CreateDataImportFromUrl description: Creates a data import from a url. operationId: DataImportService_CreateDataImportFromUrlV2 requestBody: content: application/json: schema: $ref: '#/components/schemas/v2CreateDataImportFromUrlRequest' required: true responses: '200': description: A successful response. content: application/json: schema: $ref: '#/components/schemas/v2CreateDataImportFromUrlResponse' default: description: An unexpected error response. content: application/json: schema: $ref: '#/components/schemas/rpcStatus' components: schemas: v2CreateDataImportFromUrlRequest: type: object properties: url: type: string description: >- The url to import. HTTP and S3 urls are supported. If you need to import non-public S3 objects, please contact Sift to set that up. csvConfig: $ref: '#/components/schemas/v2CsvConfig' ch10Config: $ref: '#/components/schemas/v2Ch10Config' tdmsConfig: $ref: '#/components/schemas/v2TDMSConfig' parquetConfig: $ref: '#/components/schemas/v2ParquetConfig' required: - url v2CreateDataImportFromUrlResponse: type: object properties: dataImportId: type: string rpcStatus: type: object properties: code: type: integer format: int32 message: type: string details: type: array items: $ref: '#/components/schemas/protobufAny' v2CsvConfig: type: object properties: assetName: type: string runName: type: string runId: type: string description: >- The id of the run to add this data to. If set, `run_name` is ignored. firstDataRow: type: integer format: int64 description: >- The first row to start reading as data. Can be used to skip header rows. The first row in the file is 1. timeColumn: $ref: '#/components/schemas/v2CsvTimeColumn' dataColumns: type: object additionalProperties: $ref: '#/components/schemas/commontypev1ChannelConfig' description: >- A map from column number (1-indexed) to the channel configuration for that column. numRows: type: string format: uint64 description: This will be read on upload from the file if not set. v2Ch10Config: type: object properties: assetName: type: string runName: type: string scaleValues: type: boolean v2TDMSConfig: type: object properties: assetName: type: string runName: type: string startTimeOverride: type: string format: date-time description: >- Override the wf_start_time metadata field for all channels. Useful if your waveform channels have wf_increment but no wf_start_time (Veristand is guilty of this). fileSize: type: string format: uint64 description: >- The file size in bytes. If the file has truncated chunks, this will be required to pass validation. v2ParquetConfig: type: object properties: assetName: type: string runName: type: string runId: type: string description: >- The id of the run to add this data to. If set, `run_name` is ignored. flatDataset: $ref: '#/components/schemas/v2ParquetFlatDatasetConfig' footerOffset: type: string format: uint64 footerLength: type: integer format: int64 complexTypesImportMode: $ref: '#/components/schemas/v2ParquetComplexTypesImportMode' protobufAny: type: object properties: '@type': type: string description: >- A URL/resource name that uniquely identifies the type of the serialized protocol buffer message. This string must contain at least one "/" character. The last segment of the URL's path must represent the fully qualified name of the type (as in `path/google.protobuf.Duration`). The name should be in a canonical form (e.g., leading "." is not accepted). In practice, teams usually precompile into the binary all types that they expect it to use in the context of Any. However, for URLs which use the scheme `http`, `https`, or no scheme, one can optionally set up a type server that maps type URLs to message definitions as follows: * If no scheme is provided, `https` is assumed. * An HTTP GET on the URL must yield a [google.protobuf.Type][] value in binary format, or produce an error. * Applications are allowed to cache lookup results based on the URL, or have them precompiled into a binary to avoid any lookup. Therefore, binary compatibility needs to be preserved on changes to types. (Use versioned type names to manage breaking changes.) Note: this functionality is not currently available in the official protobuf release, and it is not used for type URLs beginning with type.googleapis.com. As of May 2023, there are no widely used type server implementations and no plans to implement one. Schemes other than `http`, `https` (or the empty scheme) might be used with implementation specific semantics. additionalProperties: {} description: >- `Any` contains an arbitrary serialized protocol buffer message along with a URL that describes the type of the serialized message. Protobuf library provides support to pack/unpack Any values in the form of utility functions or additional generated methods of the Any type. Example 1: Pack and unpack a message in C++. Foo foo = ...; Any any; any.PackFrom(foo); ... if (any.UnpackTo(&foo)) { ... } Example 2: Pack and unpack a message in Java. Foo foo = ...; Any any = Any.pack(foo); ... if (any.is(Foo.class)) { foo = any.unpack(Foo.class); } // or ... if (any.isSameTypeAs(Foo.getDefaultInstance())) { foo = any.unpack(Foo.getDefaultInstance()); } Example 3: Pack and unpack a message in Python. foo = Foo(...) any = Any() any.Pack(foo) ... if any.Is(Foo.DESCRIPTOR): any.Unpack(foo) ... Example 4: Pack and unpack a message in Go foo := &pb.Foo{...} any, err := anypb.New(foo) if err != nil { ... } ... foo := &pb.Foo{} if err := any.UnmarshalTo(foo); err != nil { ... } The pack methods provided by protobuf library will by default use 'type.googleapis.com/full.type.name' as the type URL and the unpack methods only use the fully qualified type name after the last '/' in the type URL, for example "foo.bar.com/x/y.z" will yield type name "y.z". JSON ==== The JSON representation of an `Any` value uses the regular representation of the deserialized, embedded message, with an additional field `@type` which contains the type URL. Example: package google.profile; message Person { string first_name = 1; string last_name = 2; } { "@type": "type.googleapis.com/google.profile.Person", "firstName": , "lastName": } If the embedded message type is well-known and has a custom JSON representation, that representation will be embedded adding a field `value` which holds the custom JSON in addition to the `@type` field. Example (for message [google.protobuf.Duration][]): { "@type": "type.googleapis.com/google.protobuf.Duration", "value": "1.212s" } v2CsvTimeColumn: type: object properties: columnNumber: type: integer format: int64 description: The column number (1-indexed) of the time column. format: $ref: '#/components/schemas/v2TimeFormat' relativeStartTime: type: string format: date-time commontypev1ChannelConfig: type: object properties: name: type: string units: type: string description: type: string dataType: $ref: '#/components/schemas/v1ChannelDataType' enumTypes: type: array items: $ref: '#/components/schemas/v1ChannelEnumType' bitFieldElements: type: array items: $ref: '#/components/schemas/v1ChannelBitFieldElement' metadata: type: array items: $ref: '#/components/schemas/v1MetadataValue' v2ParquetFlatDatasetConfig: type: object properties: timeColumn: $ref: '#/components/schemas/v2ParquetTimeColumn' dataColumns: type: array items: $ref: '#/components/schemas/v2ParquetDataColumn' v2ParquetComplexTypesImportMode: type: string enum: - PARQUET_COMPLEX_TYPES_IMPORT_MODE_UNSPECIFIED - PARQUET_COMPLEX_TYPES_IMPORT_MODE_IGNORE - PARQUET_COMPLEX_TYPES_IMPORT_MODE_BOTH - PARQUET_COMPLEX_TYPES_IMPORT_MODE_STRING - PARQUET_COMPLEX_TYPES_IMPORT_MODE_BYTES default: PARQUET_COMPLEX_TYPES_IMPORT_MODE_UNSPECIFIED description: |2- - PARQUET_COMPLEX_TYPES_IMPORT_MODE_IGNORE: Ignore complex types and do not ingest them. - PARQUET_COMPLEX_TYPES_IMPORT_MODE_BOTH: Import complex types as both Arrow bytes and JSON strings. - PARQUET_COMPLEX_TYPES_IMPORT_MODE_STRING: Import complex types as only JSON strings. - PARQUET_COMPLEX_TYPES_IMPORT_MODE_BYTES: Import complex types as only Arrow bytes. v2TimeFormat: type: string enum: - TIME_FORMAT_UNSPECIFIED - TIME_FORMAT_RELATIVE_NANOSECONDS - TIME_FORMAT_RELATIVE_MICROSECONDS - TIME_FORMAT_RELATIVE_MILLISECONDS - TIME_FORMAT_RELATIVE_SECONDS - TIME_FORMAT_RELATIVE_MINUTES - TIME_FORMAT_RELATIVE_HOURS - TIME_FORMAT_ABSOLUTE_RFC3339 - TIME_FORMAT_ABSOLUTE_DATETIME - TIME_FORMAT_ABSOLUTE_UNIX_SECONDS - TIME_FORMAT_ABSOLUTE_UNIX_MILLISECONDS - TIME_FORMAT_ABSOLUTE_UNIX_MICROSECONDS - TIME_FORMAT_ABSOLUTE_UNIX_NANOSECONDS default: TIME_FORMAT_UNSPECIFIED v1ChannelDataType: type: string enum: - CHANNEL_DATA_TYPE_UNSPECIFIED - CHANNEL_DATA_TYPE_DOUBLE - CHANNEL_DATA_TYPE_STRING - CHANNEL_DATA_TYPE_ENUM - CHANNEL_DATA_TYPE_BIT_FIELD - CHANNEL_DATA_TYPE_BOOL - CHANNEL_DATA_TYPE_FLOAT - CHANNEL_DATA_TYPE_INT_32 - CHANNEL_DATA_TYPE_UINT_32 - CHANNEL_DATA_TYPE_INT_64 - CHANNEL_DATA_TYPE_UINT_64 - CHANNEL_DATA_TYPE_BYTES default: CHANNEL_DATA_TYPE_UNSPECIFIED v1ChannelEnumType: type: object properties: name: type: string key: type: integer format: int64 isSigned: type: boolean required: - name - key v1ChannelBitFieldElement: type: object properties: name: type: string index: type: integer format: int32 description: >- The index of this element's first bit in the logical bit field array. bitCount: type: integer format: int64 required: - name - index - bitCount v1MetadataValue: type: object properties: key: $ref: '#/components/schemas/v1MetadataKey' stringValue: type: string numberValue: type: number format: double booleanValue: type: boolean archivedDate: type: string format: date-time required: - key v2ParquetTimeColumn: type: object properties: path: type: string format: $ref: '#/components/schemas/v2TimeFormat' relativeStartTime: type: string format: date-time v2ParquetDataColumn: type: object properties: path: type: string channelConfig: $ref: '#/components/schemas/commontypev1ChannelConfig' v1MetadataKey: type: object properties: name: type: string type: $ref: '#/components/schemas/v1MetadataKeyType' archivedDate: type: string format: date-time required: - name - type v1MetadataKeyType: type: string enum: - METADATA_KEY_TYPE_UNSPECIFIED - METADATA_KEY_TYPE_STRING - METADATA_KEY_TYPE_NUMBER - METADATA_KEY_TYPE_BOOLEAN default: METADATA_KEY_TYPE_UNSPECIFIED description: |- Metadata key type. - METADATA_KEY_TYPE_STRING: string - METADATA_KEY_TYPE_NUMBER: number - METADATA_KEY_TYPE_BOOLEAN: boolean securitySchemes: BearerAuth: type: http scheme: bearer bearerFormat: JWT ````