JobProgressResponse

Information on the progress of an ingestion job.

Properties

jobStatus

String   required

Possible values: CREATED, SCHEDULED, RUNNING, SUCCESS, SUCCESS_WITH_WARNING, NO_ROWS_INGESTED, FAILED, CANCELED, UNKNOWN

A job can take one of the following statuses:

  • CREATED: The job has been created but has not yet been scheduled to run. Jobs may be in this state if they must wait for another job to complete before being scheduled.

  • SCHEDULED: The job has been scheduled but not yet started. Jobs may be in this state if they are waiting for resources, such as compute or locks, to become available. For jobs waiting on a compute resources, Polaris automatically scales up as necessary (within your account limits) to run the job.

  • RUNNING: The job is running.

  • SUCCESS: The job succeeded.

  • SUCCESS_WITH_WARNING: The job succeeded but encountered non-fatal errors. This state is most often seen on ingestion jobs where data was ingested but some rows were malformed. The job details for a job in this state contain a description of the non-fatal errors.

  • NO_ROWS_INGESTED: The ingestion job finished without failures, but no rows were ingested.

  • FAILED: The job failed. Check the job details for the cause of the failure. This state is most often seen on ingestion jobs if the data cannot be parsed.

  • CANCELED: The job was canceled by a user.

currentPhase

String

Possible values: DETERMINE_PARTITIONS, BUILD_SEGMENTS

Druid-side task phase; only applicable if the task is currently running.

progress

Integer

Minimum: 0

Maximum: 100

Percentage of Druid subtasks that have completed.

numRowsProcessed

Integer

Number of rows processed.

numRowsProcessedWithError

Integer

Number of processed rows containing an error.

numRowsThrownAway

Integer

Number of rows discarded.

numRowsUnparseable

Integer

Number of rows unable to be parsed.

unparseableRowReports

Array

An array of parse exception reports from Druid ingestion. Each exception report is a JSON object having the following properties:

  • errorType: Type of the parse exception. Possible values: unparseable, processedWithError.
  • input: String representation of the input that failed to parse.
  • details: List of detailed parsing error messages.
  • timeOfExceptionMillis: Timestamp in milliseconds when the parse exception occurred.

duration

Integer

Duration of the ingestion job in milliseconds.

submittedOn

DateTime   required

Timestamp of when the ingestion job was submitted.

errorMsg

String

An error message from the TaskStatus response from Druid, if any.