Skip to main content

Release notes

These release notes provide information about the latest features, improvements, and fixes for Imply Polaris.
For older release notes, see the archive.
For information about Imply Enterprise, see Imply Enterprise and Hybrid release notes.
For a comprehensive list of deprecated and sunset features, refer to Deprecations and sunsets.

Polaris version

The release notes for Polaris are organized by release date. It may take a few days from the release date for the new Polaris version to become available for your projects.

You can verify the current Polaris version identifier for your projects in the Polaris UI or the Projects API.

July 24, 2024​

Late July release | Polaris version 65b6697e

Lookups (beta)​

Polaris now offers lookups as a beta feature so that you can join information between a table and a lookup source. For details, see Lookups (id: 39037)

Suspend a job​

You can suspend or resume a streaming ingestion job in the UI. When you suspend a job, the job execution status changes to Suspended. For details, see Pause and resume jobs (id: 61929)

Suspend a job

AWS Asia Pacific (Seoul) region​

You can now deploy Polaris projects in the AWS Asia Pacific (Seoul) region, ap-northeast-2. For more information, see Cloud providers and regions (id: 61763)

Improvement to alert creation request​

Creating an alert using the Alerts v1 API now returns exampleQuery in the response. This is the query string Polaris uses to evaluate the alert, with the time boundary fixed to the alert creation time. This can be useful if you want to create the same alert in another system (id: 40980)

You can now create an embedding link to flat table, overall (beta), gauge, time series, and multi-axis line chart visualizations in the UI and using the Embedding links v1 API.

When creating embedding links for these visualizations in the API, complete the exploreView property instead of viewDescription, select the appropriate moduleName, and set the display parameters (id: 61989)

Improvements to data cube downloads​

Added the following Numerical call formatting options when downloading data from flat table, overall (beta), gauge, time series, and multi-axis line chart visualizations:

  • Decimal formatting: Enforce decimal precision.
  • Abbreviations: Enforce the same formatting you see in the UI (id: 61988)

Daily cost breakdown​

The Billing page now features a daily cost bar chart for each project. Each bar represents a single day's charges. Hover over a bar to see the detailed charges accrued for that day (id: 60675)

Daily cost chart

Other improvements​

Analytics​

  • Users with the AdministerEmbedLinks permission can now enable standard and async downloads when configuring an embedding link (id: 62293)
  • Query monitoring and server-side logging now includes data cube and dashboard IDs, names, and tiles (id: 39741)

Ingestion​

  • Improved error message when passing a malformed period to TIME_FLOOR during ingestion (id: 61771)
  • You can create up to 2000 tables for a Polaris project (id: 8742)

Monitoring​

  • Added query metrics to the Streaming metrics dashboard (id: 62014)
  • Updated quantiles measures to be faster and more accurate (id: 61675)

Fixes​

  • Fixed an issue where the download output for HLL sketch columns included decimal points (id: 60859)
  • Fixed an issue where async downloads failed when using PIVOT_NESTED_AGG in the filter (id: 61040)
  • Fixed an issue where the Auto label for the line chart visualization displayed a value different from the actual limit (id: 62015)
  • Fixed an issue where the loader element persisted for paused projects (id: 62342)
  • Fixed an issue where the dashboard download button disappeared when the Show title bar option was disabled (id: 61964)
  • Fixed an issue with the Embedding links v1 API returning an incorrect link in the response Location header (id: 61952)

July 10, 2024​

Early July release | Polaris version 05f613fd

Jobs v1 API improvements​

The Jobs API now includes a /status endpoint for retrieving and updating the execution status of an ingestion job. You can suspend or resume a streaming ingestion job using the desiredExecutionStatus parameter (id: 61188, 61471)

Improved data cube configuration UI​

Improved the UI experience when creating and editing data cubes. Click the tabs across the top of the data cube edit view to access the configuration sections. In addition to the new look & feel, you can now:

  • Use the auto-suggest feature when editing SQL in data cubes, dimensions, and measures.
  • Use dedicated controls for advanced options that weren't previously available.

See Manage data cubes for more information (id: 39621)

Improved monitoring experience​

The streaming ingestion page has been renamed to Streaming metrics. It now includes more detailed information and allows you to expand and drill into each dashboard tile. Tiles are now split by table so that you can track metrics per stream (id: 62013, 61679)

Expand tile to view detailed metrics

Detailed query metrics are now available in a dedicated data cube. To access the Detailed Query Metrics data cube, go to the User queries page and click the Expand tile button on any tile (id: 61724)

Detailed query metrics

Other improvements​

Analytics​

Ingestion​

  • You can now specify one or more intervals and an optional period when configuring cache and retention policies (id: 41768)
  • Streaming parse exceptions now include the column's name (id: 60231)
  • The Source field of the Job details dialog now displays All objects if an entire S3 bucket is selected for ingestion (id: 61884)
  • You can now group by complex columns, such as JSON (id: 60440)

Metrics​

  • Improved the accuracy of the Queries executed panel on the User queries page and the Total Queries measure on the Detailed metrics page (id: 61892)

Fixes​

  • Fixed an issue where switching to the JSON job spec in the ingestion flow before selecting a target table resulted in an error message (id: 60133)
  • Fixed an issue where changing the Y-axis title in the multi-axis line chart displayed the new title alongside the old one (id: 61555)
  • Fixed an issue where alerts and reports stopped running after Polaris restarted the service responsible for coordinating distributed systems (id: 61954)
  • Fixed an issue where you couldn't edit or remove newly created columns while ingesting to a table that has data (id: 61703)
  • Fixed the Get embedding links endpoint to include the viewDescription property in its response (id: 61853)
  • Fixed an issue where filters weren't applied to off-screen dashboard tiles (id: 61722)
  • Fixed an issue where the table visualization incorrectly displayed "Other" values (id: 61704)
  • Fixed a problem rendering table visualizations with a comparison by removing comparison sorts, sort by delta, and sort by percent (id: 62176)

June 26, 2024​

Late June release | Polaris version f6f3d662

Azure Marketplace​

You can now sign up for a Polaris account through Azure Marketplace (id: 36112)

Collections v1 API improvements​

You must now specify the assetType when you remove an asset from a collection or favorites. The old endpoints are now deprecated and will be removed by the end of September 2024.

New endpoints:

  • DELETE /v1/projects/{projectId}/collections/{id}/assets/{assetType}/{assetId} removes a specified asset from a collection.
  • DELETE /v1/projects/{projectId}/favorites/{assetType}/{assetId} removes a specified asset from the authenticated user's favorites.

Deprecated endpoints:

  • DELETE /v1/projects/{projectId}/collections/{id}/assets/{assetId}
  • DELETE /v1/projects/{projectId}/favorites/{assetId}

See Collections v1 API for more information (id: 60863)

Other improvements​

Admin​

  • The auto kill feature is now enabled for all projects. This feature helps clean up data after the 30-day grace period discussed in Restore or permanently delete data. Auto kill was previously enabled for most projects (id: 61789)

Ingestion​

  • When ingesting into a flexible table, you can override the table's partitioning and clustering in the ingestion job itself (id: 39725)
  • You no longer need to declare columns before using them for clustering (id: 39725)
  • Polaris now also supports ingestion from Azure Blob Storage for projects hosted on AWS (id: 61848)

Metrics​

  • Added a new Distinct Hosts measure to the Server metrics data cube. After updating to Polaris version f6f3d662, Distinct Hosts measure will include historical data (id: 60619)
  • Added a new set of CPU and memory usage metrics to the Server metrics data cube (id: 61392)

Fixes​

June 17, 2024​

Azure GA | Polaris version f05d1f17

Polaris on Azure GA​

Polaris on Azure is now generally available.

Note that if you subscribe to Polaris through the AWS marketplace, the Azure cloud option is not available.

Azure Germany West Central region​

You can now deploy Polaris projects in the Azure Germany West Central region, germanywestcentral. For more information, see Cloud providers and regions (id: 40971)

Fixes​

SAS token authentication for Azure Blob Storage connections no longer requires the write permission (id: 61371)

June 12, 2024​

Mid-June release | Polaris version 75961ecc

Multi-axis line chart visualization (beta)​

The new multi-axis line chart visualization is optimized for the display of multiple axes—you can display up to 10 measures on a single chart. To enable this beta feature, contact Polaris support. Once enabled, you can continue to access the original line chart as well (id: 39612)

Delete and recover data by version​

You can now manage deleted data by versions in the UI. For example, if you have three versions of deleted data for a given interval and your table doesn't already contain data for that interval, you can restore any of the three deleted versions (id: 60870)

Enhanced schema editing​

You can now use IP address and IP prefix data types when editing your schema using the UI. For more details, see Data types (id: 61027)

Window functions in custom measures​

You can now use window functions when defining custom measures in data cubes. See Custom measure examples for details (id: 41427)

In the Embedding links v1 API you can now use enableDownloads and enableAsyncDownloads to set the download properties of embedded visualizations (id: 61141)

Other improvements​

Analytics​

  • Improved the user experience for time range truncation in line chart visualizations when using a non-UTC timezone (id: 61173)
  • Improved how the Embedding links v1 API validates download limit and filter clauses (id: 61143)

Ingestion​

  • Added the option to replace all data in the UI (id: 38969)
  • Improved error messages for Protobuf and Avro key input formats (id: 60786)

Fixes​

  • Fixed an issue where downloads from the records table visualization failed (id: 61396)
  • Fixed an issue with sorting by Added measure in some visualizations (id: 61359)
  • Fixed an issue where attempting a table-to-table ingestion job immediately after populating a flexible table showed a cached version of the source table in the UI, omitting the newly added columns (id: 60945)
  • Fixed an issue where the All rows limit was incorrectly interpreted as the current setting in a flat table visualization (id: 61045)
  • Fixed an issue with non-filterable and non-showable dimensions not working when editing embedding links (id: 61142)
  • Corrected the reference type definition for defaultFacetSplits in the Data cubes v1 API payload schema (id: 61039)

June 3, 2024​

Azure beta | Polaris version 7fabd8ff

Polaris on Azure (beta) is now available to all customers. You can now select Azure as a cloud provider when creating a project.

May 29, 2024​

Late May release | Polaris version 7fabd8ff

You can now establish a unidirectional secure connection from your Azure virtual network (VNet) to Imply Polaris over a private endpoint using Azure Private Link to query and publish data.

When you query the Polaris endpoints through Azure Private Link, traffic between your Azure VNet and Polaris travels the Microsoft backbone network and doesn't traverse the public internet. For more information, see Azure Private Link for querying and publishing data (id: 38954)

Replace all mode for batch jobs​

When replacing data using the Polaris API, you can now include the replaceAll parameter to replace all data in the table. Previously, time intervals were required for data replacement jobs (id: 41716)

Other improvements​

Admin​

  • Improved layout of billing UI (id: 60662)

Analytics​

  • If you change the time zone in a data cube's settings, you can now apply the same time zone the next time you access the data cube—as either a one-time or persistent change (id: 39603)
  • The View raw data option in a data cube now opens the query in SQL view for all visualizations (id: 60544)
  • You can now add pinnedDimensions as a query parameter in a Polaris URL to pin a specified dimension to the side bar—for example:
    https://ORGANIZATION_NAME.REGION.CLOUD_PROVIDER.api.imply.io/f683410c-0935-4d2a-ab74-9a584f2727b7/pivot/i/f382102f34471ef5c1/Earthquake_sample_day?visualization=table&pinnedDimensions=Geometry_type
    (id: 60664)

Ingestion​

  • You can now switch an S3 connection between different authentication types (id: 60294)
  • Step 3 of the Insert data flow now contains a link to the SQL filter documentation (id: 42736)
  • Improved the user experience of configuring a Kafka connection (id: 60739)

Metrics​

  • Added the Datacube dimension to Detailed metrics (id: 60914)

Fixes​

  • Fixed an error when navigating from the records or records table visualizations to the overall visualization (id: 60787)
  • Fixed an issue with enabling and disabling non-showable and non-filterable dimensions in embedding links (id: 61142)
  • Fixed an issue with intervals not being displayed for replace data jobs (id: 42236)
  • Fixed an issue with the Download button missing from embedded flat table visualizations (id: 60509)
  • Fixed an issue with the Parse Kafka metadata toggle not working for CSV and TSV formats (id: 61063)
  • Fixed an error caused by null job source URIs (id: 61137)
  • Fixed a timeout exception when listing Azure objects (id: 60691)
  • Fixed an issue with the table detail view failing to render when parsing invalid time columns (id: 60747)
  • Fixed an issue where alert creation fails when the first data cube in the list is missing a primary time dimension (id: 40948)
  • Fixed an issue where calling GET /dashboards/{id} with default filters returns filters with name properties instead of id properties, causing them to fail a validation check upon re-sending (id: 61094)
  • Fixed an issue in the UI that prevented column edits in an ingestion job when the input data contains __time (id: 60843)
  • Fixed an issue with query failures during Polaris project upgrades (id: 60925)
  • Fixed ability to add and edit dimensions in Detailed metrics which is a read-only view (id: 60320)

May 8, 2024​

Early May release | Polaris version f0de541f

Administrative events​

Polaris audit logs now include administrative events. You can view administrative actions performed on your Polaris account, including who performed the action and when. See Administrative events for more information (id: 41684)

Manage specific versions of soft deleted data​

If you have multiple versions of soft deleted data for a given time interval, you can now use the API to restore or permanently delete a specific version of the data. For example, if you replaced data multiple times for January 2024, you may want to restore an older version of the data rather than the most recently deleted one. Polaris may generate different versions of data when you perform a replace job or when you soft delete data (manually or by a retention policy) and reingest into that interval.

For more information, see Restore an older version of data that's in use and Permanently delete data by time interval and version (id: 60654)

Other improvements​

Admin​

  • Improved layout of project management UI (id: 60135)

Analytics​

  • Added a warning to data cube downloads with a large time filter (id: 60445)
  • Added All rows option to the Enable downloads configuration for embedding links (id: 43129)
  • Improved the flat table, gauge, time series, and overall (beta) visualizations to allow users to drag-and-drop dimensions, measures, and comparisons into the visualization. This functionality is already available in other visualizations (id: 60621)
  • Improved the experience of switching from the overall visualization to the overall (beta) visualization (id: 60607)
  • Improved performance when creating restricted embedding links (id: 60676)

Ingestion​

  • Added option to specify input format and key schema for Protobuf and Avro data (id: 60639)

Fixes​

  • Fixed ability to set multiple time ranges in visualizations (id: 60502)
  • Fixed a problem with downloading data from a records table visualization (id: 41415)
  • Fixed an issue that prevented the All rows async download option from working in the records table visualization (id: 60110)
  • Fixed missing parse metadata options in the UI when ingesting from Azure Blob Storage (id: 60241)
  • Fixed optional SAS token in Azure Blob Storage connections (id: 60297)
  • Fixed saving retention rules when set from the tables list (id: 42449)
  • Fixed ingestion being blocked on fully uploaded files when another file upload is in progress (id: 41532)
  • Fixed switching to native query tab causing the SQL IDE to crash (id: 60671)
  • Fixed handling JSON source columns in table-to-table ingestion jobs (id: 60401)
  • Fixed a problem with opening a report from a report notification email (id: 60229)
  • Fixed a problem with embedding links when switching between the overall visualization and the overall (beta) visualization (id: 60649)
  • Fixed report owner's inability to see a report when they are removed from the recipients list (id: 42841)
  • Fixed missing async download options from embedding links feature (id: 60836)
  • Fixed a problem with the overall (beta) visualization when using a LONG column as primary time dimension (id: 43293)
  • Fixed crash when opening restricted embed links a second time (id: 60676)

April 24, 2024​

Late April release | Polaris version 9401d804

Restore or permanently delete data​

Polaris now supports a soft delete option through the UI or API. Soft deleted data can be restored within a 30 day grace period. Note that Imply reserves the right to delete data before the 30 day grace period is up if necessary. The default deletion method through the API is still to permanently delete data. Include "softDelete": True in your payload to soft delete data.

After you soft delete data or your data expires based on the retention period, you can restore the deleted data using the UI or API (id: 40088)

Intervals for retention and cache policies​

You can now use intervals or intervals with an additional ISO 8601 duration when you define your retention and cache policies through the API. For more information, see Set a storage policy by API.

Access keys for S3​

Using access keys to authenticate S3 connections is now generally available (id: 39243)

Other improvements​

Admin​

  • Improved the UI for cases when a project is not in a running state (id: 40659)
  • Improved the New project dialog (id: 60139)

Analytics​

  • You can now configure an embedding link to allow users to download All rows (id: 43129)
  • Improved custom date time formatting (id: 40619)

Ingestion​

  • You can now specify a parse schema for Avro and Protobuf keys in the Polaris UI (id: 43071)
  • Improved the performance of Tables and Jobs APIs when a project has a large number of tables. This also improves the performance of the table list and SQL editor in the Polaris UI (id: 60283)

Security​

  • Improved the user experience of configuring an IP allowlist (id: 60312)

Fixes​

  • Fixed an issue where dashboards with mixed tile types didn't apply filters to some visualizations (id: 41787)
  • Fixed an issue with removing late message rejection period in the UI (id: 56025)
  • Fixed an issue where hiding the overall option for a column dimension also hid the dimension name (id: 40649)
  • Fixed an issue where the ingestion wizard displayed an error if you entered and then removed text from the Protobuf message type field (id: 60305)
  • Fixed an issue that prevented the All rows async download option from working in the records table visualization (id: 60110)
  • Fixed an issue with alert webhook URLs missing protocol (id: 60383)
  • Fixed an issue when creating embedded visualizations with the API (id: 43281)

April 10, 2024​

Early April release | Polaris version 40796e41

Imply Polaris on Microsoft Azure (beta)​

Polaris on Azure is a beta feature available to select customers. For information about joining the beta, contact Polaris support

Once you join the beta program, you can deploy Polaris projects in the Azure East US region. The beta also enables batch ingestion from Azure Blob Storage and streaming ingestion from Apache Kafka on Azure Event Hubs. These capabilities are in addition to existing support for batch ingestion from files, tables, and Amazon S3, and streaming ingestion from Confluent Cloud and Apache Kafka (id: 38029)

A guide for ingesting data from Azure Blob Storage by API is coming at a later date. For now, you can use the sample request in the S3 ingestion guide to create a batch ingestion job from Azure, with the following changes:

  • Change source.type from s3 to azure
  • Set connectionName to an Azure connection
  • If your source uses uris, change s3:// to azureStorage:// for each item in uris

Other improvements​

Analytics​

  • You can now download the maximum number of rows permitted by your permission through an embedded link. When there are no restrictions on the data cube, users with the AccessDownloadLargeData permission can download all rows. When restrictions are present, users with the AccessDownloadLargeData permission see download limits smaller than or equal to the limit on the data cube. Users with the AccessDownloadData permission see the download limits less than or equal to 5000 or the configured fast download threshold (id: 43129)
  • You can now use asymmetric number range filters with flat table, gauge, time series, and overall (beta) visualizations (id: 58507)
  • Added persistence to query cache and approximate queries settings (id: 43251)
  • Added FilterWithRegex permission which allows users to use the regex filter for string dimensions (id: 42840)

Ingestion​

  • Polaris now shows an error message when a job is submitted with an empty string for an input expression (id: 42757)

Metrics​

  • Changed the icon and renamed Server to Server metrics in the monitoring UI (id: 60205)

Fixes​

  • Fixed an issue where any timezone information added under the Time settings section of the __time column was omitted from sampling requests in the Polaris UI (id: 42936, 42886)
  • Fixed an issue where unavailable or partially available tables were reported as available in the Polaris UI (id: 40093)
  • Fixed an issue with table columns not respecting the time format (id: 43181)
  • Fixed an issue with timestamps not formatted in flat table async downloads (id: 42922)
  • Fixed an issue where dashboard filters reset from Greater than or equal and Less than or equal to Greater than and Less than (id: 40777)
  • Fixed an issue with the street map visualization including all data cube Latitude/Longitude dimensions, even the ones not in the visualization (id: 58505)
  • Fixed an issue with the line chart visualization causing dashboards to crash (id: 58506)
  • Fixed an issue with the y-axis extending above the line chart visualization boundary for very small values (id: 43300)
  • Fixed an issue where users were not redirected to the Alerts page after deleting an alert (id: 60160)
  • Fixed an issue where flexible tables didn't declare new columns if clustering columns were present (id: 58511)
  • Fixed an issue where the average Kinesis lag in the Streaming monitoring tab was being reported incorrectly (id: 60145)

March 27, 2024​

Late March release | Polaris version 0df7af44

IP allowlist​

You can now configure a network policy to control API access to your Polaris projects using the Network policy API. The only supported type of network policy is IP allowlist. For information on how to configure a network policy using the API, see Set a network policy by API. For information on how to set a network policy in the Polaris UI, see IP allowlist.

Imply must enable this feature for you. Contact Imply Polaris support for more information (id: 34075)

Data cube query API change​

In the Data cube query API, the MEASURE_BY_NAME and DIMENSION_BY_NAME functions have been replaced by MEASURE_BY_ID and DIMENSION_BY_ID. This is not a breaking change. The behavior remains the same and the old functions still work. We recommend that you use MEASURE_BY_ID and DIMENSION_BY_ID in your API requests going forward (id: 41639)

Other improvements​

Analytics​

  • Measures can now reference other measures in flat table, gauge, time series, and overall (beta) visualizations (id: 42973)
  • You can now create an embedding link for flat table, gauge, time series, and overall (beta) visualizations from a data cube (id: 42470)

Ingestion​

  • You can now apply the EARLIEST_BY and LATEST_BY aggregations on numeric input fields (id: 41628)
  • You can now provide the Protobuf message type when inserting data in the UI. This ensures that the parser uses the desired schema for processing (id: 42759)
  • Improved the user experience when changing screen sizes (id: 37805)

Metrics​

  • Added the Server page that provides a data cube for monitoring JVM memory and jetty metrics (id: 41698)
  • Added the dimensions Polaris Job Id, Polaris Job Type, Partition, and Query Status to the Detailed metrics data cube (id: 41695)
  • Added the Message Gap measure to the Detailed metrics data cube (id: 43022)

Fixes​

  • Fixed an issue where filtering by a number dimension required both Greater than and Less than fields to have inputs (id: 43253)
  • Fixed an issue with embedding links where non-showable and non-filterable dimensions didn't work as expected for dashboards. Polaris now properly propagates non-showable and non-filterable dimensions to the explore view allowing flat table, gauge, time series, and overall (beta) visualizations to work (id: 43030)
  • Fixed an issue where some measure types were showing up as options on detailed flex tables (id: 42857)
  • Fixed an issue where editing a dashboard tab would take you to the first tab after saving (id: 41367)

March 13, 2024​

Mid-March release | Polaris version 63b8c063

Time series visualization​

You can now use time series functions to generate a line or bar chart showing the rate of change in your data. See the Visualizations reference for more information (id: 40670)

Visualize data from table​

When you view a table in the Polaris UI you can now click Visualize data to display a sample view of your data as an overall, line chart, or vertical bar chart visualization. You can also create a data cube from the Visualize data dialog (id: 40568)

Other improvements​

Admin​

  • Added permission AccessDownloadAsync to allow users to access the async download (alpha) feature when the feature is enabled for your organization (id: 42274)

Analytics​

  • Added an optional parameter exactResultsOnly to the Data cube query API to perform exact rather than approximate SQL queries (id: 42521)
  • Improved pop-up notifications and errors (ids: 42501, 42500)

Ingestion​

  • Improved UI performance when handling rows that contain very large data (id: 41388)
  • You can now change the query granularity to finer granularity than table partitioning in the UI (id: 39454)

Querying​

  • Added DOWNSAMPLED_SUM_TIMESERIES, TIMESERIES_ATTACH_META, and TIMESERIES_CLEAR_META time series functions (id: 42103)

Fixes​

  • Fixed a rounding issue in the display of dimensions (id: 42654)
  • Fixed downloads limited to 5,000 rows for flat table, gauge, and overall (beta) visualizations (id: 42600)
  • Fixed failed async downloads producing a truncated file instead of an error (id: 42595)
  • Fixed changes not propagated from flat table data cube to dashboard (id: 42275)
  • Fixed a display issue in the async download feature (id: 42249)
  • Fixed async downloads not working with "previous period" comparisons (id: 42247)
  • Fixed query precision issues (ids: 42227, 42230)
  • Fixed an issue where switching to the SQL code editor in a replace flow takes you to INSERT mode (id: 41908)
  • Fixed lack of indication when data cube is refreshed (id: 40260)

Deprecations and sunsets​

The bucketMillis parameter is deprecated​

The bucketMillis parameter in the output of the time series object has been deprecated and will be removed at the end of March 2024. Instead of the bucketMillis parameter, use the timeProperties object. It provides more detailed information about the time properties, including bucket period, timezone, and origin.

February 28, 2024​

Late February release | Polaris version e21efcf7

The Embedding links API replaces the Embedding v1 API. (id: 38743)

info

The Embedding v1 API is now deprecated and will be removed by the end of September 2024. Use the Embedding links API to update your existing embedding links and create embedding links going forward.

Other improvements​

Analytics​

  • When you set a data cube's defaults, you can now set the default view to be a gauge, flat table, time series, or overall (beta) visualization (id: 41373)
  • In the table visualization, you can now choose whether or not to display the year in time values (id: 40988)
  • In the advanced data cube options, you can now set the Latest data strategy to Query the latest timestamp from the data source, relative to the latest full day (id: 39634)

Ingestion​

  • You can now right click to open Polaris resources in a new tab, including tables, jobs, and certain menu options (id: 41864)
  • Added the response property totalCount to the Files API to return the total number of files when listing all files (id: 41901)

Metrics​

  • You can now monitor Amazon Kinesis lag from the Streaming view of the Monitoring page (id: 40229)

Querying​

  • Added the MAP_TIMESERIES time series function (id: 41518)
  • Added the PIVOT_LEFT_JOIN function for use when creating data cube dimensions (id: 42193)
  • Improved error message when setting query context (id: 23820)

Security​

  • Improved the user experience of updating existing connections to Confluent Cloud (id: 35388)

Fixes​

  • Fixed an issue with filters on expression virtual column indexes incorrectly considering values null in some cases for expressions which translate null values into not null values (id: 42448)
  • Fixed an error when using embedded links to fetch data from another web app (id: 42273)
  • Fixed an issue with sorting by source in the job list (id: 41803)
  • Fixed Polaris showing an error for some time comparisons in a data cube (id: 41013)
  • Fixed an issue where filters and shown dimensions and measures were not preserved when switching to some visualization types (id: 41059)

Deprecations and sunsets​

Embedding API​

The Embedding v1 API has been deprecated and will be removed by the end of September 2024. Use the Embedding links API to update your existing embedding links and create embedding links going forward.

February 13, 2024​

Mid-February release | Polaris version ffe71477

Analytics APIs​

You can now use the following APIs to create and manage Polaris assets:

New overall visualization (beta)​

A new overall visualization includes a trend line and an updated properties panel.

To enable this beta feature, contact Polaris support. Once enabled, the new overall visualization replaces the standard overall visualization. See Visualizations reference for more information. (ids: 40562, 41090)

Time series functions (beta)​

The time series functions feature has been promoted to beta.

Additionally, improved the feature as follows:

  • Renamed the existing SUM_TIMESERIES function to SUM_OVER_TIMESERIES
  • Added new TIMESERIES and SUM_TIMESERIES aggregator functions
  • Added new time series processing functions

See Time series functions for more information. (id: 41518)

Other improvements​

Ingestion​

  • Added a progress bar to async downloads UI (id: 41229)
  • Improved error handling for ingestion jobs (id: 38453)
  • Improved error messages when mapping a source to a table (id: 37695)

Querying​

  • Improved query fetch operations to reduce the likelihood of timeouts (id: 41570)

Fixes​

  • Fixed discrepancies when applying custom UI colors (id: 40266)
  • Fixed an issue with the records visualization not supporting async download (id: 41289)
  • Fixed a display issue with undeclared columns (id: 40318)
  • Fixed certain failures when a drop table job fails (id: 41618)
  • Fixed an issue with populating the Amazon S3 ingestion dialog (id: 39087)
  • Fixed an issue when duplicating a dimension as a measure (id: 39500)
  • Fixed an issue with displaying column filters (id: 38468)
  • Fixed an issue with filtering on a dimension with Set/String type that contains nulls (id: 41459)
  • Fixed an issue with the Insert data UI ignoring the second timestamp column when there are multiple candidates in the sampler (id: 41794)
  • Fixed an issue with the Insert data UI attempting to save the table with an empty schema when switching to SQL editor without choosing any input sources (id: 41912)
  • Fixed an issue with detail tables missing the dimension icon in column headers (id: 40845)

February 9, 2024​

Early February hotfix | Polaris version e2797837

Fixes​

  • Fixed an issue where DataSketches HLL sketches would erroneously be considered empty. For details, see the following Imply Knowledge Base article (id: 41916)
  • Fixed an issue with the handling of NULL array elements with MV_CONTAINS and MV_OVERLAP

January 30, 2024​

Late January release | Polaris version 837a89bf

Polaris version identifier​

You can access the version identifier for your project through the Polaris UI or the Projects API. (id: 32810, 40902)

EARLIEST and LATEST functions​

You can use EARLIEST and LATEST aggregation functions on numeric types in ingestion queries. (id: 40615)

Audit logs​

Polaris now logs user authentication events. You can use audit logs to detect anomalous activities and investigate potential misuse of privileges or security incidents. Users with the AccessAuditLogs permission can access audit logs through the Polaris UI or the Audit API. (id: 17543)

Other improvements​

Analytics​

  • Improved the overall visualization by adding a properties panel to the right side of the page and updating property labels and descriptions (ids: 40562, 41090)
  • Improved time filter handling in dashboard tiles (id: 41171)
  • Improved the display of comparison values in visualizations, by adding the ability to sort by delta and percentage (id: 38539)

Ingestion​

  • Improved the UI for table-to-table ingestion by hiding buttons related to adding inputs (id: 37671)

Metrics​

Other​

  • Improved the UI for cases where no projects are available, either because an organization doesn't have any projects or a user lacks the necessary permissions (id: 40478)
  • Improved the UI for creating your first project in Polaris (id: 40477)

Fixes​

  • Fixed an issue with the records visualization not supporting async download (id: 41289)
  • Fixed an issue with the embedding visualizations feature where the data cube view setup page was displaying only some visualizations (id: 41095)
  • Fixed an issue where the flat table visualization was only accepting time bucketing by PT1M (id: 40953)
  • Fixed an issue where the incorrect time bucket label was displayed for the America/Mexico_City timezone during Daylight Saving Time (DST) (id: 39749)
  • Fixed an issue with time filters in dashboard tiles (id: 41171)
  • Fixed an issue with the More link in the data cube header (id: 40815)
  • Fixed text wrapping issue with long error message in ingestion wizard (id: 37695)
  • Fixed missing S3 description in connection details dialog in ingestion wizard (id: 39087)

January 17, 2024​

Mid-January release

Added system fields to input sources​

Polaris now supports ingesting data from system fields, including S3 metadata for an object’s URI, bucket name, and key. This allows for easier ingestion of metadata from batch sources. For example, if you ingest data from an S3 object with the URI s3://bucket-name/file.json.gz, you can include a column in the table for bucket-name, a column for file.json.gz, and a column for s3://bucket-name/file.json.gz. For more information, see Ingest object metadata. (id: 40064)

Project capacity warning​

Added a warning icon to the project selector drop-down that signals when a project reaches capacity. This can help you scale up to a bigger project size or prompt you to delete underused data and tables. (id: 40479)

Import and export data cube and dashboard definitions​

You can now manually import and export JSON definitions for data cubes and dashboards. This can be useful to move data cubes and dashboards between projects and to create manual backups. See Manage data cubes and Dashboards overview for more information. (id: 39610)

SQL PIVOT and UNPIVOT (beta)​

Polaris now supports SQL PIVOT and UNPIVOT operators. (id: 37598)

The PIVOT operator carries out an aggregation and transforms rows into columns in the output. The following is the general syntax for the PIVOT operator:

PIVOT (aggregation_function(column_to_aggregate)
  FOR column_with_values_to_pivot
  IN (pivoted_column1 [, pivoted_column2 ...])
)

The UNPIVOT operator transforms existing column values into rows. The following is the general syntax for the UNPIVOT operator:

UNPIVOT (values_column
  FOR names_column
  IN (unpivoted_column1 [, unpivoted_column2 ... ])
)

Other improvements​

Admin​

  • Added the Project info dialog to the project selector (id: 40099)
  • Improved the Usage and Billing > Usage tab by adding cloud provider information to the View details dialog and creating collapsible sections for each project (id: 40100)

Ingestion​

  • Added the ability to sort table data by dimension by clicking the dimension column header (id: 31363)

Querying​

  • Added the ability to save a measure when the preview query fails (id: 38763)

Fixes​

  • Fixed an issue where dimensions with periods in their IDs were displaying as undefined in the records table visualization (id: 41009)
  • Fixed an issue where column changes weren't saved in the browser session (id: 39865)
  • Fixed an issue where the input field mapped to __time was ingested as its own column (id: 37550)
  • Fixed an issue with the flat table visualization where reloading the Explore view reset the filter to the default (id: 40339)
  • Fixed an issue where visualization for new data cubes was set to None instead of Overall (id: 40663)
  • Fixed an issue where clicking a checkbox in the Engine drop-down didn't toggle the state (id: 40617)
  • Fixed an issue where user profiles couldn't be accessed if the project wasn't in an active state (id:40636)
  • Fixed an issue with the Percent of root measure causing downloads from deep storage to fail (id: 40673)
  • Fixed an issue where users were redirected to the Home view when their session expired instead of resuming on the same page after logging in (id: 40843)
  • Fixed an issue where exporting data for a data cube would fail if it referenced a measure that had been removed (id: 40565)
  • Fixed an issue with the SQL editor where updating a table would also change the table's partitioning granularity to day (id: 40932)

Deprecations and sunsets​

This section provides information about deprecated and sunset features.

Deprecations​

Deprecated features continue to be available but receive no ongoing maintenance or additional support.

Collections v1 API endpoints​

The following Collections v1 API endpoints to remove an asset from a collection or favorite have been deprecated and will be removed by the end of September 2024:

  • DELETE /v1/projects/{projectId}/collections/{id}/assets/{assetId}
  • DELETE /v1/projects/{projectId}/favorites/{assetId}

Update your requests to use the following replacement endpoints going forward:

  • DELETE /v1/projects/{projectId}/collections/{id}/assets/{assetType}/{assetId}
  • DELETE /v1/projects/{projectId}/favorites/{assetType}/{assetId}

Apache Druid two-value logic for native filters removed​

Unless you have requested otherwise, Polaris uses the following ANSI SQL-compliant behavior as of February 29, 2024:

  • Polaris differentiates between empty records and null records. For example, for queries evaluating x is not equal to 'some value', Polaris will include all rows not matching "x='some value'" but exclude null values.
  • Polaris stores boolean values as 0 or 1.
  • Polaris uses SQL three-valued logic for native query filters: true, false, and unknown.

If these changes impact your query patterns and you need more time to update your queries, please contact Polaris customer support to request an extension. For projects that have extensions, you must update all queries to use the new behavior by November 2024, when the aforementioned Druid legacy operations will be removed.

bucketMillis time series parameter​

The bucketMillis parameter in the output of the time series object has been deprecated and will be removed at the end of March 2024. Instead of the bucketMillis parameter, use the timeProperties object. It provides more detailed information about the time properties, including bucket period, timezone, and origin.

Embedding v1 API​

The Embedding v1 API has been deprecated and will be removed by the end of September 2024. Use the Embedding links API to update your existing embedding links and create embedding links going forward.

Project-less regional API resources​

Project-less regional API resources have been deprecated and will be removed by the end of September 2024. You must include the project ID in the base URL for all regional API calls in projects created after September 29, 2023.

For example, update your requests to the following APIs to include the project ID in the URL:

  • /v0/pivot/api/v1/embed is deprecated. Use /v0/projects/PROJECT_ID/pivot/api/v1/embed instead.
  • /v1/events/CONNECTION_NAME is deprecated. Use /v1/projects/PROJECT_ID/events/CONNECTION_NAME instead.
  • /v1/files is deprecated. Use /v1/projects/PROJECT_ID/files instead.

For detailed instructions on how to migrate to new API resources, see the migration guide for deprecated APIs.

API v2 version identifier​

All API requests must reference the v1 version identifier in the URL path, even if the previous API version had a higher version identifier. For example, update your requests to the following APIs to use the v1 version identifier and include the project ID in the URL:

  • /v2/connections is deprecated. Use /v1/projects/PROJECT_ID/connections instead.
  • /v2/jobs is deprecated. Use /v1/project/PROJECT_ID/jobs instead.
  • /v2/tables is deprecated. Use /v1/projects/PROJECT_ID/tables instead.

Support for v2 APIs will be removed by the end of September 2024. For detailed instructions on how to migrate to new API resources, see the migration guide for deprecated APIs.

Project API​

The Project v1 API has been deprecated and will be removed by the end of September 2024. Use the Projects v1 API instead.

Project-less JDBC connection string​

Applications that integrate with Polaris using JDBC, including BI tools such as Tableau and Looker, must specify the project ID in the connection URL. Support for previously established JDBC connections will be removed by the end of September 2024. For information on how to update your connection string, see the migration guide for deprecated APIs.

Files API query parameters​

The following Files API query parameters are now deprecated: cont, limit, and pattern. We recommend using paginationOffset, paginationLimit, and searchString query parameters instead.

Even though Polaris does not currently enforce a limit to the number of files returned for the GET /v1/projects/PROJECT_ID/files API call, we will enforce a limit in the future. Please verify your usage of the GET /v1/projects/PROJECT_ID/files API call and use the paginationOffset, paginationLimit, and searchString query parameters to limit the number of files returned in the response.

Week time partitioning granularity​

The week time partitioning granularity is deprecated. Polaris no longer supports week partitioning for new tables or changing time partitioning to week for existing tables. We recommend using hour, day, or month partitioning instead. The week partitioning does not properly align with other segment sizes, causing occasional bugs.

Sunsets​

The following features have been phased out and removed from Polaris.

Tables API properties​

We have removed the following deprecated properties on API Tables objects:

  • createdBy. Use createdByUser instead.
  • createdTimestamp. Use createdOnTimestamp instead.
  • lastModifiedBy. Use modifiedByUser instead.
  • lastUpdateTimestamp. Use modifiedOnTimestamp instead.