Files API

Use the Files API to upload files to a staging area for ingestion. See Upload files by API for more details and examples.

You can accomplish the following tasks with the Files API:

All URIs are relative to https://api.imply.io/v1

Upload a file

URL

POST /files

Parameters

Query parameters

  • dataFormat   type: String
    If specified, the format of the data. Otherwise, the data format is determined by the filename or content type associated with the file in the multipart form request. Supported options:

    • nd-json - newline-delimited JSON (one JSON record per line)
    • csv - CSV or TSV

  • compressionFormat   type: String
    If specified, the compression used for the file. Otherwise, the compression is determined by the filename or content type associated with the file in the multipart form request. Supported options:

Request body

Send the file as form data in the request. File names must be unique in Polaris. You cannot upload a file if the staging already holds a file with the same name.

The media type of the HTTP request is multipart/form-data; however, you should not set the Content-Type: multipart/form-data header explicitly. Allow your client to assign the header automatically because it will also properly set the associated boundary directive.

You can upload a file up to 10 GB in size. This limit refers to the size of the file transmitted by the browser or HTTP client. You may upload a file that's larger than 10 GB on disk if your browser or client compresses the file in transit to below 10 GB.

Responses

A successful request returns an HTTP 2XX status code and a JSON response body of type FileMetadata.


Created.


Bad Request.


Conflict.


Payload Too Large.

Sample request

curl --location --request POST 'https://api.imply.io/v1/files' \
--header 'Content-Type: multipart/form-data' \
--header 'Authorization: Bearer <Bearer Token>' \
--form 'file=@"mydata.csv"'

Sample response

{
    "id":"0600a548-5a66-40af-a93d-079d35a66fc5",
    "name":"mydata.csv",
    "sizeBytes":5573348,
    "dataFormat":"nd-json",
    "compressionFormat":"none",
    "digest":{
        "algo":"md5",
        "hash":"CDA9B678171E24BD956574E78659E0AD"
    },
    "uploadedByUserId":"dbb63d7c-3b35-4268-b351-17e0a98a7bfb",
    "uploadedOnDatetime":"2021-09-29T00:16:47.083004121Z"
}

List available files

URL

GET /files

Parameters

Query parameters

  • limit   type: Integer
    The maximum number of files to return at a time (max: 100)

  • cont   type: Integer
    An opaque continuation token (default: 0)

  • pattern   type: String
    A wildcard (glob) to match files

Responses

A successful request returns an HTTP 2XX status code and a JSON response body with the following properties:

  • files: An array of FileMetadata objects for every matching file.
  • links: Holds next and previous pagination links for paginated responses.
  • space: Information about the file repository, with the following fields:
    • allocatedByes: The total space allocated for the repository in bytes; sum of usedBytes and remainingBytes.
    • usedBytes: The amount of space used by files in the repository in bytes.
    • remainingBytes: The amount of unused space in the repository in bytes.


OK.


Invalid request payload.

Sample request

curl --location --request GET 'https://api.imply.io/v1/files' \
--header 'Authorization: Bearer <Bearer Token>'

Sample response

{
    "files": [
        {
            "id": "1645ac6e-5095-49bf-90ff-171dce77e66c",
            "name": "kttm-2019-08-19.json.gz",
            "sizeBytes": 8213401,
            "dataFormat": "nd-json",
            "compressionFormat": "gz",
            "digest": {
                "algo": "md5",
                "hash": "E370CAC6A8D6A938A08279200883D7B3"
            },
            "uploadedOnDatetime": "2021-09-29T23:20:24Z"
        },
        {
            "id": "55ef893a-4a53-4249-8fa6-b361eba31c39",
            "name": "wikipedia.zip",
            "sizeBytes": 2017143,
            "dataFormat": "nd-json",
            "compressionFormat": "none",
            "digest": {
                "algo": "md5",
                "hash": "B40356B48BF586043DA7F810027C4DFE"
            },
            "uploadedOnDatetime": "2021-10-07T05:52:59Z"
        }
    ],
    "space": {
        "allocatedBytes": 1000000000000,
        "usedBytes": 4972427805,
        "remainingBytes": 995027572195
    }
}

Get a file's metadata

URL

GET /files/{name}

Parameters

Path parameters

  • name   type: String   required
    Name of the file (may contain slashes)

Responses

A successful request returns an HTTP 2XX status code and a JSON response body of type FileMetadata.


OK.


Not Found.

Sample request

curl --location --request GET 'https://api.imply.io/v1/files/mydata.csv' \
--header 'Authorization: Bearer <Bearer Token>'

Sample response

{
    "id":"0600a548-5a66-40af-a93d-079d35a66fc5",
    "name":"mydata.csv",
    "sizeBytes":5573348,
    "dataFormat":"nd-json",
    "compressionFormat":"none",
    "digest":{
        "algo":"md5",
        "hash":"CDA9B678171E24BD956574E78659E0AD"
    },
    "uploadedByUserId":"dbb63d7c-3b35-4268-b351-17e0a98a7bfb",
    "uploadedOnDatetime":"2021-09-29T00:16:47.083004121Z",
    "modifiedByUserId":"dbb63d7c-3b35-4268-b351-17e0a98a7bfb",
    "modifiedOnDatetime":"2021-09-30T00:16:47.083004121Z"
}

Delete a file from the staging area

URL

DELETE /files/{name}

Parameters

Path parameters

  • name   type: String   required
    Name of the file (may contain slashes)

Responses


No Content.

Sample request

curl --location --request DELETE 'https://api.imply.io/v1/files/mydata.csv' \
--header 'Authorization: Bearer <Bearer Token>'
← Events APIIngestion Jobs API →