Skip to main content

Release notes

These release notes provide information about the latest features, improvements, and fixes for Imply Polaris.
For older release notes, see the archive.
For information about Imply Enterprise, see Imply Enterprise and Hybrid release notes.
For a comprehensive list of deprecated and sunset features, refer to Deprecations and sunsets.

Polaris version

The release notes for Polaris are organized by release date. It may take a few days from the release date for the new Polaris version to become available for your projects.

You can verify the current Polaris version identifier for your projects in the Polaris UI or the Projects API.

April 24, 2024​

Late April release | Polaris version 9401d804

Restore or permanently delete data​

Polaris now supports a soft delete option through the UI or API. Soft deleted data can be restored within a 30 day grace period. Note that Imply reserves the right to delete data before the 30 day grace period is up if necessary. The default deletion method through the API is still to permanently delete data. Include "softDelete": True in your payload to soft delete data.

After you soft delete data or your data expires based on the retention period, you can restore the deleted data using the UI or API.

(id: 40088)

Intervals for retention and cache policies​

You can now use intervals or intervals with an additional ISO 8601 duration when you define your retention and cache policies through the API. For more information, see Set a storage policy by API.

Access keys for S3​

Using access keys to authenticate S3 connections is now generally available.

(id: 39243)

Other improvements​

Admin​

  • Improved the UI for cases when a project is not in a running state (id: 40659)
  • Improved the New project dialog (id: 60139)

Analytics​

  • You can now configure an embedding link to allow users to download All rows (id: 43129)
  • Improved custom date time formatting (id: 40619)

Ingestion​

  • You can now specify a parse schema for Avro and Protobuf keys in the Polaris UI (id: 43071)
  • Improved the performance of Tables and Jobs APIs when a project has a large number of tables. This also improves the performance of the table list and SQL editor in the Polaris UI (id: 60283)

Security​

  • Improved the user experience of configuring an IP allowlist (id: 60312)

Fixes​

  • Fixed an issue where dashboards with mixed tile types didn't apply filters to some visualizations (id: 41787)
  • Fixed an issue with removing late message rejection period in the UI (id: 56025)
  • Fixed an issue where hiding the overall option for a column dimension also hid the dimension name (id: 40649)
  • Fixed an issue where the ingestion wizard displayed an error if you entered and then removed text from the Protobuf message type field (id: 60305)
  • Fixed an issue that prevented the All rows async download option from working in the records table visualization (id: 60110)
  • Fixed an issue with alert webhook URLs missing protocol (id: 60383)
  • Fixed an issue when creating embedded visualizations with the API (id: 43281)

April 10, 2024​

Early April release | Polaris version 40796e41

Imply Polaris on Microsoft Azure (beta)​

Polaris on Azure is a beta feature available to select customers. For information about joining the beta, contact Polaris support

Once you join the beta program, you can deploy Polaris projects in the Azure East US region. The beta also enables batch ingestion from Azure Blob Storage and streaming ingestion from Apache Kafka on Azure Event Hubs. These capabilities are in addition to existing support for batch ingestion from files, tables, and Amazon S3, and streaming ingestion from Confluent Cloud and Apache Kafka (id: 38029)

A guide for ingesting data from Azure Blob Storage by API is coming at a later date. For now, you can use the sample request in the S3 ingestion guide to create a batch ingestion job from Azure, with the following changes:

  • Change source.type from s3 to azure
  • Set connectionName to an Azure connection
  • If your source uses uris, change s3:// to azureStorage:// for each item in uris

Other improvements​

Analytics​

  • You can now download the maximum number of rows permitted by your permission through an embedded link. When there are no restrictions on the data cube, users with the AccessDownloadLargeData permission can download all rows. When restrictions are present, users with the AccessDownloadLargeData permission see download limits smaller than or equal to the limit on the data cube. Users with the AccessDownloadData permission see the download limits less than or equal to 5000 or the configured fast download threshold (id: 43129)
  • You can now use asymmetric number range filters with flat table, gauge, time series, and overall (beta) visualizations (id: 58507)
  • Added persistence to query cache and approximate queries settings (id: 43251)
  • Added FilterWithRegex permission which allows users to use the regex filter for string dimensions (id: 42840)

Ingestion​

  • Polaris now shows an error message when a job is submitted with an empty string for an input expression (id: 42757)

Metrics​

  • Changed the icon and renamed Server to Server metrics in the monitoring UI (id: 60205)

Fixes​

  • Fixed an issue where any timezone information added under the Time settings section of the __time column was omitted from sampling requests in the Polaris UI (id: 42936, 42886)
  • Fixed an issue where unavailable or partially available tables were reported as available in the Polaris UI (id: 40093)
  • Fixed an issue with table columns not respecting the time format (id: 43181)
  • Fixed an issue with timestamps not formatted in flat table async downloads (id: 42922)
  • Fixed an issue where dashboard filters reset from Greater than or equal and Less than or equal to Greater than and Less than (id: 40777)
  • Fixed an issue with the street map visualization including all data cube Latitude/Longitude dimensions, even the ones not in the visualization (id: 58505)
  • Fixed an issue with the line chart visualization causing dashboards to crash (id: 58506)
  • Fixed an issue with the y-axis extending above the line chart visualization boundary for very small values (id: 43300)
  • Fixed an issue where users were not redirected to the Alerts page after deleting an alert (id: 60160)
  • Fixed an issue where flexible tables didn't declare new columns if clustering columns were present (id: 58511)
  • Fixed an issue where the average Kinesis lag in the Streaming monitoring tab was being reported incorrectly (id: 60145)

March 27, 2024​

Late March release | Polaris version 0df7af44

IP allowlist​

You can now configure a network policy to control API access to your Polaris projects using the Network policy API. The only supported type of network policy is IP allowlist. For information on how to configure a network policy using the API, see Set a network policy by API. For information on how to set a network policy in the Polaris UI, see IP allowlist.

Imply must enable this feature for you. Contact Imply Polaris support for more information (id: 34075)

Data cube query API change​

In the Data cube query API, the MEASURE_BY_NAME and DIMENSION_BY_NAME functions have been replaced by MEASURE_BY_ID and DIMENSION_BY_ID. This is not a breaking change. The behavior remains the same and the old functions still work. We recommend that you use MEASURE_BY_ID and DIMENSION_BY_ID in your API requests going forward (id: 41639)

Other improvements​

Analytics​

  • Measures can now reference other measures in flat table, gauge, time series, and overall (beta) visualizations (id: 42973)
  • You can now create an embedding link for flat table, gauge, time series, and overall (beta) visualizations from a data cube (id: 42470)

Ingestion​

  • You can now apply the EARLIEST_BY and LATEST_BY aggregations on numeric input fields (id: 41628)
  • You can now provide the Protobuf message type when inserting data in the UI. This ensures that the parser uses the desired schema for processing (id: 42759)
  • Improved the user experience when changing screen sizes (id: 37805)

Metrics​

  • Added the Server page that provides a data cube for monitoring JVM memory and jetty metrics (id: 41698)
  • Added the dimensions Polaris Job Id, Polaris Job Type, Partition, and Query Status to the Detailed metrics data cube (id: 41695)
  • Added the Message Gap measure to the Detailed metrics data cube (id: 43022)

Fixes​

  • Fixed an issue where filtering by a number dimension required both Greater than and Less than fields to have inputs (id: 43253)
  • Fixed an issue with embedding links where non-showable and non-filterable dimensions didn't work as expected for dashboards. Polaris now properly propagates non-showable and non-filterable dimensions to the explore view allowing flat table, gauge, time series, and overall (beta) visualizations to work (id: 43030)
  • Fixed an issue where some measure types were showing up as options on detailed flex tables (id: 42857)
  • Fixed an issue where editing a dashboard tab would take you to the first tab after saving (id: 41367)

March 13, 2024​

Mid-March release | Polaris version 63b8c063

Time series visualization​

You can now use time series functions to generate a line or bar chart showing the rate of change in your data. See the Visualizations reference for more information (id: 40670)

Visualize data from table​

When you view a table in the Polaris UI you can now click Visualize data to display a sample view of your data as an overall, line chart, or vertical bar chart visualization. You can also create a data cube from the Visualize data dialog (id: 40568)

Other improvements​

Admin​

  • Added permission AccessDownloadAsync to allow users to access the async download (alpha) feature when the feature is enabled for your organization (id: 42274)

Analytics​

  • Added an optional parameter exactResultsOnly to the Data cube query API to perform exact rather than approximate SQL queries (id: 42521)
  • Improved pop-up notifications and errors (ids: 42501, 42500)

Ingestion​

  • Improved UI performance when handling rows that contain very large data (id: 41388)
  • You can now change the query granularity to finer granularity than table partitioning in the UI (id: 39454)

Querying​

  • Added DOWNSAMPLED_SUM_TIMESERIES, TIMESERIES_ATTACH_META, and TIMESERIES_CLEAR_META time series functions (id: 42103)

Fixes​

  • Fixed a rounding issue in the display of dimensions (id: 42654)
  • Fixed downloads limited to 5,000 rows for flat table, gauge, and overall (beta) visualizations (id: 42600)
  • Fixed failed async downloads producing a truncated file instead of an error (id: 42595)
  • Fixed changes not propagated from flat table data cube to dashboard (id: 42275)
  • Fixed a display issue in the async download feature (id: 42249)
  • Fixed async downloads not working with "previous period" comparisons (id: 42247)
  • Fixed query precision issues (ids: 42227, 42230)
  • Fixed an issue where switching to the SQL code editor in a replace flow takes you to INSERT mode (id: 41908)
  • Fixed lack of indication when data cube is refreshed (id: 40260)

Deprecations and sunsets​

The bucketMillis parameter is deprecated​

The bucketMillis parameter in the output of the time series object has been deprecated and will be removed at the end of March 2024. Instead of the bucketMillis parameter, use the timeProperties object. It provides more detailed information about the time properties, including bucket period, timezone, and origin.

February 28, 2024​

Late February release | Polaris version e21efcf7

The Embedding links API replaces the Embedding v1 API. (id: 38743)

info

The Embedding v1 API is now deprecated and will be removed by the end of September 2024. Use the Embedding links API to update your existing embedding links and create embedding links going forward.

Other improvements​

Analytics​

  • When you set a data cube's defaults, you can now set the default view to be a gauge, flat table, time series, or overall (beta) visualization (id: 41373)
  • In the table visualization, you can now choose whether or not to display the year in time values (id: 40988)
  • In the advanced data cube options, you can now set the Latest data strategy to Query the latest timestamp from the data source, relative to the latest full day (id: 39634)

Ingestion​

  • You can now right click to open Polaris resources in a new tab, including tables, jobs, and certain menu options (id: 41864)
  • Added the response property totalCount to the Files API to return the total number of files when listing all files (id: 41901)

Metrics​

  • You can now monitor Amazon Kinesis lag from the Streaming view of the Monitoring page (id: 40229)

Querying​

  • Added the MAP_TIMESERIES time series function (id: 41518)
  • Added the PIVOT_LEFT_JOIN function for use when creating data cube dimensions (id: 42193)
  • Improved error message when setting query context (id: 23820)

Security​

  • Improved the user experience of updating existing connections to Confluent Cloud (id: 35388)

Fixes​

  • Fixed an issue with filters on expression virtual column indexes incorrectly considering values null in some cases for expressions which translate null values into not null values (id: 42448)
  • Fixed an error when using embedded links to fetch data from another web app (id: 42273)
  • Fixed an issue with sorting by source in the job list (id: 41803)
  • Fixed Polaris showing an error for some time comparisons in a data cube (id: 41013)
  • Fixed an issue where filters and shown dimensions and measures were not preserved when switching to some visualization types (id: 41059)

Deprecations and sunsets​

Embedding API​

The Embedding v1 API has been deprecated and will be removed by the end of September 2024. Use the Embedding links API to update your existing embedding links and create embedding links going forward.

February 13, 2024​

Mid-February release | Polaris version ffe71477

Analytics APIs​

You can now use the following APIs to create and manage Polaris assets:

New overall visualization (beta)​

A new overall visualization includes a trend line and an updated properties panel.

To enable this beta feature, contact Polaris support. Once enabled, the new overall visualization replaces the standard overall visualization. See Visualizations reference for more information. (ids: 40562, 41090)

Time series functions (beta)​

The time series functions feature has been promoted to beta.

Additionally, improved the feature as follows:

  • Renamed the existing SUM_TIMESERIES function to SUM_OVER_TIMESERIES
  • Added new TIMESERIES and SUM_TIMESERIES aggregator functions
  • Added new time series processing functions

See Time series functions for more information. (id: 41518)

Other improvements​

Ingestion​

  • Added a progress bar to async downloads UI (id: 41229)
  • Improved error handling for ingestion jobs (id: 38453)
  • Improved error messages when mapping a source to a table (id: 37695)

Querying​

  • Improved query fetch operations to reduce the likelihood of timeouts (id: 41570)

Fixes​

  • Fixed discrepancies when applying custom UI colors (id: 40266)
  • Fixed an issue with the records visualization not supporting async download (id: 41289)
  • Fixed a display issue with undeclared columns (id: 40318)
  • Fixed certain failures when a drop table job fails (id: 41618)
  • Fixed an issue with populating the Amazon S3 ingestion dialog (id: 39087)
  • Fixed an issue when duplicating a dimension as a measure (id: 39500)
  • Fixed an issue with displaying column filters (id: 38468)
  • Fixed an issue with filtering on a dimension with Set/String type that contains nulls (id: 41459)
  • Fixed an issue with the Insert data UI ignoring the second timestamp column when there are multiple candidates in the sampler (id: 41794)
  • Fixed an issue with the Insert data UI attempting to save the table with an empty schema when switching to SQL editor without choosing any input sources (id: 41912)
  • Fixed an issue with detail tables missing the dimension icon in column headers (id: 40845)

February 9, 2024​

Early February hotfix | Polaris version e2797837

Fixes​

  • Fixed an issue where DataSketches HLL sketches would erroneously be considered empty. For details, see the following Imply Knowledge Base article (id: 41916)
  • Fixed an issue with the handling of NULL array elements with MV_CONTAINS and MV_OVERLAP

January 30, 2024​

Late January release | Polaris version 837a89bf

Polaris version identifier​

You can access the version identifier for your project through the Polaris UI or the Projects API. (id: 32810, 40902)

EARLIEST and LATEST functions​

You can use EARLIEST and LATEST aggregation functions on numeric types in ingestion queries. (id: 40615)

Audit logs​

Polaris now logs user authentication events. You can use audit logs to detect anomalous activities and investigate potential misuse of privileges or security incidents. Users with the AccessAuditLogs permission can access audit logs through the Polaris UI or the Audit API. (id: 17543)

Other improvements​

Analytics​

  • Improved the overall visualization by adding a properties panel to the right side of the page and updating property labels and descriptions (ids: 40562, 41090)
  • Improved time filter handling in dashboard tiles (id: 41171)
  • Improved the display of comparison values in visualizations, by adding the ability to sort by delta and percentage (id: 38539)

Ingestion​

  • Improved the UI for table-to-table ingestion by hiding buttons related to adding inputs (id: 37671)

Metrics​

Other​

  • Improved the UI for cases where no projects are available, either because an organization doesn't have any projects or a user lacks the necessary permissions (id: 40478)
  • Improved the UI for creating your first project in Polaris (id: 40477)

Fixes​

  • Fixed an issue with the records visualization not supporting async download (id: 41289)
  • Fixed an issue with the embedding visualizations feature where the data cube view setup page was displaying only some visualizations (id: 41095)
  • Fixed an issue where the flat table visualization was only accepting time bucketing by PT1M (id: 40953)
  • Fixed an issue where the incorrect time bucket label was displayed for the America/Mexico_City timezone during Daylight Saving Time (DST) (id: 39749)
  • Fixed an issue with time filters in dashboard tiles (id: 41171)
  • Fixed an issue with the More link in the data cube header (id: 40815)
  • Fixed text wrapping issue with long error message in ingestion wizard (id: 37695)
  • Fixed missing S3 description in connection details dialog in ingestion wizard (id: 39087)

January 17, 2024​

Mid-January release

Added system fields to input sources​

Polaris now supports ingesting data from system fields, including S3 metadata for an object’s URI, bucket name, and key. This allows for easier ingestion of metadata from batch sources. For example, if you ingest data from an S3 object with the URI s3://bucket-name/file.json.gz, you can include a column in the table for bucket-name, a column for file.json.gz, and a column for s3://bucket-name/file.json.gz. For more information, see Ingest object metadata. (id: 40064)

Project capacity warning​

Added a warning icon to the project selector drop-down that signals when a project reaches capacity. This can help you scale up to a bigger project size or prompt you to delete underused data and tables. (id: 40479)

Import and export data cube and dashboard definitions​

You can now manually import and export JSON definitions for data cubes and dashboards. This can be useful to move data cubes and dashboards between projects and to create manual backups. See Manage data cubes and Dashboards overview for more information. (id: 39610)

SQL PIVOT and UNPIVOT (beta)​

Polaris now supports SQL PIVOT and UNPIVOT operators. (id: 37598)

The PIVOT operator carries out an aggregation and transforms rows into columns in the output. The following is the general syntax for the PIVOT operator:

PIVOT (aggregation_function(column_to_aggregate)
  FOR column_with_values_to_pivot
  IN (pivoted_column1 [, pivoted_column2 ...])
)

The UNPIVOT operator transforms existing column values into rows. The following is the general syntax for the UNPIVOT operator:

UNPIVOT (values_column
  FOR names_column
  IN (unpivoted_column1 [, unpivoted_column2 ... ])
)

Other improvements​

Admin​

  • Added the Project info dialog to the project selector (id: 40099)
  • Improved the Usage and Billing > Usage tab by adding cloud provider information to the View details dialog and creating collapsible sections for each project (id: 40100)

Ingestion​

  • Added the ability to sort table data by dimension by clicking the dimension column header (id: 31363)

Querying​

  • Added the ability to save a measure when the preview query fails (id: 38763)

Fixes​

  • Fixed an issue where dimensions with periods in their IDs were displaying as undefined in the records table visualization (id: 41009)
  • Fixed an issue where column changes weren't saved in the browser session (id: 39865)
  • Fixed an issue where the input field mapped to __time was ingested as its own column (id: 37550)
  • Fixed an issue with the flat table visualization where reloading the Explore view reset the filter to the default (id: 40339)
  • Fixed an issue where visualization for new data cubes was set to None instead of Overall (id: 40663)
  • Fixed an issue where clicking a checkbox in the Engine drop-down didn't toggle the state (id: 40617)
  • Fixed an issue where user profiles couldn't be accessed if the project wasn't in an active state (id:40636)
  • Fixed an issue with the Percent of root measure causing downloads from deep storage to fail (id: 40673)
  • Fixed an issue where users were redirected to the Home view when their session expired instead of resuming on the same page after logging in (id: 40843)
  • Fixed an issue where exporting data for a data cube would fail if it referenced a measure that had been removed (id: 40565)
  • Fixed an issue with the SQL editor where updating a table would also change the table's partitioning granularity to day (id: 40932)

Deprecations and sunsets​

This section provides information about deprecated and sunset features.

Deprecations​

Deprecated features continue to be available but receive no ongoing maintenance or additional support.

Apache Druid two-value logic for native filters removed​

Unless you have requested otherwise, Polaris uses the following ANSI SQL-compliant behavior as of February 29, 2024:

  • Polaris differentiates between empty records and null records. For example, for queries evaluating x is not equal to 'some value', Polaris will include all rows not matching "x='some value'" but exclude null values.
  • Polaris stores boolean values as 0 or 1.
  • Polaris uses SQL three-valued logic for native query filters: true, false, and unknown.

If these changes impact your query patterns and you need more time to update your queries, please contact Polaris customer support to request an extension. For projects that have extensions, you must update all queries to use the new behavior by November 2024, when the aforementioned Druid legacy operations will be removed.

bucketMillis time series parameter​

The bucketMillis parameter in the output of the time series object has been deprecated and will be removed at the end of March 2024. Instead of the bucketMillis parameter, use the timeProperties object. It provides more detailed information about the time properties, including bucket period, timezone, and origin.

Embedding v1 API​

The Embedding v1 API has been deprecated and will be removed by the end of September 2024. Use the Embedding links API to update your existing embedding links and create embedding links going forward.

Project-less regional API resources​

Project-less regional API resources have been deprecated and will be removed by the end of September 2024. You must include the project ID in the base URL for all regional API calls in projects created after September 29, 2023.

For example, update your requests to the following APIs to include the project ID in the URL:

  • /v0/pivot/api/v1/embed is deprecated. Use /v0/projects/PROJECT_ID/pivot/api/v1/embed instead.
  • /v1/events/CONNECTION_NAME is deprecated. Use /v1/projects/PROJECT_ID/events/CONNECTION_NAME instead.
  • /v1/files is deprecated. Use /v1/projects/PROJECT_ID/files instead.

For detailed instructions on how to migrate to new API resources, see the migration guide for deprecated APIs.

API v2 version identifier​

All API requests must reference the v1 version identifier in the URL path, even if the previous API version had a higher version identifier. For example, update your requests to the following APIs to use the v1 version identifier and include the project ID in the URL:

  • /v2/connections is deprecated. Use /v1/projects/PROJECT_ID/connections instead.
  • /v2/jobs is deprecated. Use /v1/project/PROJECT_ID/jobs instead.
  • /v2/tables is deprecated. Use /v1/projects/PROJECT_ID/tables instead.

Support for v2 APIs will be removed by the end of September 2024. For detailed instructions on how to migrate to new API resources, see the migration guide for deprecated APIs.

Project API​

The Project v1 API has been deprecated and will be removed by the end of September 2024. Use the Projects v1 API instead.

Project-less JDBC connection string​

Applications that integrate with Polaris using JDBC, including BI tools such as Tableau and Looker, must specify the project ID in the connection URL. Support for previously established JDBC connections will be removed by the end of September 2024. For information on how to update your connection string, see the migration guide for deprecated APIs.

Files API query parameters​

The following Files API query parameters are now deprecated: cont, limit, and pattern. We recommend using paginationOffset, paginationLimit, and searchString query parameters instead.

Even though Polaris does not currently enforce a limit to the number of files returned for the GET /v1/projects/PROJECT_ID/files API call, we will enforce a limit in the future. Please verify your usage of the GET /v1/projects/PROJECT_ID/files API call and use the paginationOffset, paginationLimit, and searchString query parameters to limit the number of files returned in the response.

Week time partitioning granularity​

The week time partitioning granularity is deprecated. Polaris no longer supports week partitioning for new tables or changing time partitioning to week for existing tables. We recommend using hour, day, or month partitioning instead. The week partitioning does not properly align with other segment sizes, causing occasional bugs.

Sunsets​

The following features have been phased out and removed from Polaris.

Tables API properties​

We have removed the following deprecated properties on API Tables objects:

  • createdBy. Use createdByUser instead.
  • createdTimestamp. Use createdOnTimestamp instead.
  • lastModifiedBy. Use modifiedByUser instead.
  • lastUpdateTimestamp. Use modifiedOnTimestamp instead.