Release notes

The Imply release notes provide information on features, improvements, and bug fixes in each release. Be sure to read these release notes carefully before upgrading to the 2020.12 release.

Imply evaluation

New to Imply? Get started with an Imply Cloud Free Trial or start a self-hosted trial at Get started with Imply!

With Imply Cloud, the Imply team manages your clusters in AWS, while you control the infrastructure and own the data. With self-hosted Imply, you can run Imply on *NIX systems in your own environment or cloud provider.

2020.12 Release

2020.12 Highlights

Druid highlights

Imply 2020.12 includes the following Druid changes:

• Grouping function: The new grouping function is now supported for computing multiple grouping sets using GROUPING SETS or CUBE in SQL. The new function returns a number for each output row to indicate whether the dimension was included in the sub-grouping to compute that row. As a result, you can tell which grouping set was used to compute each row by looking at the result of this function. See Grouping aggregator for more details.

• Auto interval detection: Support for native batch ingestion with hash/range partitioning. Native batch ingestion no longer requires intervals to be set in the granularitySpec. Instead, the batch task can automatically detect the intervals to ingest by scanning the whole input data. This is useful when the intervals are unknown at ingestion time. Note that scanning the whole input data can increase ingestion time.

• Increased accuracy for memory usage estimation during ingestion: Ingestion tasks, for both batch and streaming ingestion, support maxBytesInMemory which controls how much data the task can hold in memory. Imply now provides more accurate memory usage estimation, especially for string type columns. This reduces OutOfMemory errors during ingestion.

• New Coordinator metrics: The coordinator periodically executes multiple duties, such as polling information of new segments from the metadata store, overshadowing old segments, etc. It now supports the following new metrics to give an idea of how long those duties take:

  • coordinator/time: the time for an individual coordinator duty to execute.
  • coordinator/global/time: the time for all duties to execute. See Coordination Metrics for more details.

See Druid Changes for other Druid updates.

Platform Changes

• An additional AWS operating region has been introduced for Imply Cloud, us-west-1 (N. California).
• Direct access mode for Pivot is now generally available. It was previously available on a limited basis, and enabled only by customer request. For more information, see Pivot Direct Access.
• For usability reasons, the Disk utilization chart no longer appears at the top of the cluster overiew page. Disk utilization information still appears for each cluster in the Used column of the clusters list page. It has been removed from the top of the overview page to avoid confusion due to insufficient context.

Changes in 2020.12

Pivot Changes

• Fix Excel downloads are corrupted and can't be opened
• Fix "There are no connections" prompt appearing in settings when connections are still loading
• Fix issue where Zookeeper can enter an unresolvable state causing alerts to not be evaluated
• Fix expanding columns in the Crosstab visualization is broken
• Improve performance and stability of the Crosstab visualization
• Fix reset password links should not be shown when passwords are not managed by Pivot
• Fix auto refresh options not being applied or persisted in the Crosstab visualization

Druid Changes

• Auto interval detection support for native batch ingestion with hash/range partitioning
• Increased accuracy for memory usage estimation during ingestion
• New coordinator metrics

Known issues

For a full list of open Druid issues, please see https://github.com/apache/druid/labels/Bug

2020.11 Release

2020.11 Highlights

Version number scheme change

The version numbering scheme for Imply has changed in this release. The version numbering scheme reflects a shift to a monthly release cadence.

2020.11 is a Short Term Support release. As of this release, Imply releases fall into one of two categories: Long Term Support (LTS) and Short Term Support (STS).

Imply Product Long Term Support (LTS) releases are complete, stable versions of our products. They are subject to general bug fixes for one year and to security bug fixes for two years after release, but do not receive features enhancements. They are best suited to production environments that require stable operation and do not require the latest features.

Releases that are not subject to long term support — that is, Short Term Support (STS) releases — include the latest features, including experimental features. STS releases occur on a regular basis. They are not subject to bug fixes, but may receive critical security patches. In general, to receive the latest bug fixes, you should upgrade to the latest monthly release.

Note the following additional highlights in 2020.11.

Druid highlights

Imply 2020.11 includes updates for Druid that include:

• A refreshed style for the web console
• The ability to use an expression for the timestamp column during ingestion. For example, you can concatenate multiple fields to comprise the timestamp
• Improved SQL query error handling

See Druid Changes for other Druid updates.

Platform highlights

Imply Cloud now offers additional AWS EC2 m5 series types as deployment options for master nodes, including m5.xlarge, m5.2xlarge, and m5.4xlarge.

Changes in 2020.11

Pivot Changes

• Add support for exposing headersTimeout as a Pivot config property
• Fix client query requests do not retry correctly on failure
• Add support for matching defaultRole against external role id when using OIDC
• Fix users can not enter custom dimension or measure values
• Add support for controlling feature flag visibility by stage
• Fix streaming of export response column headers can fail due to GZIP compression
• Fix raw data view does not render all columns on Pivot SQL data cubes
• Improve workflow for migrating from native user management to OIDC with local role authority
• Improve UX for updating Plywood-based data cubes to add support for Pivot SQL
• Fix Y axis labels are still shown and overlap graph contents when dashboard tile "Show axes if possible" option is unchecked
• Security updates

Druid Changes

• Allow column reordering during ingestion schema setup UI
• Allow point-and-click to add dimensions to dimensionExclusion list
• Added repartitionTransitionDuration
• Added helpful warnings for non-newline delimited JSON
• Roll up is now disabled by default in data loader UI
• Disallow submitting of an unfinished spec
• Added ability to AutoForm to fold things into a Show more folder
• Fixed bug where totalNumMergeTasks and maxNumSegmentsToMerge were not showing up in the form
• Fix issue where all the tables were constantly being re-mounted, losing scroll position/column sizing
• Queries containing nullable columns for numeric types can now be executed by vectorized groupBy engine
• Web console now correctly renders when there are datasources without loaded segments
• Fix auto-compaction task slot computation
• Fix Avro support for Web console
• Add segment-level caching support for join queries
• Added shuffle metrics for parallel indexing
• shard_spec, dimensions, metrics and last_compaction_state in the segments table are now serialized as JSON consistently, where previously they were sometimes serialized as Java objects
• New option added to disable all hardcoded Zookeeper use
• Security updates

Upgrading from previous releases

When upgrading from earlier versions, see the release notes for all relevant intermediate versions.

Also note the following considerations.

Docker deployment not supported

Imply no longer recommends nor supports Docker-based deployments for new installations. Existing Docker-based deployments are supported through July 1, 2021. If you are currently using a Docker-based deployment, you should migrate to one of the following deployment modes before that date:

• Imply Cloud
• Imply Private on Kubernetes
• Imply Private on Linux

End of support

As of July 15, 2020, Imply version 2.x is no longer supported. If you still have active deployments that use Imply version 2.x, you are strongly encouraged to upgrade to the current version as soon as possible. See Subscription Support Maintenance Terms for more information about supported versions.