2023.03

2023.03

  • Imply
  • Ingest
  • Query
  • Visualize
  • Administer
  • Deploy

›View Manager

Overview

  • About Imply administration

Manager

  • Using Imply Manager
  • Managing Imply clusters
  • Imply Manager security
  • Extensions

Users

  • Imply Manager users
  • Druid API access
  • Authentication and Authorization

    • Get started with Imply Hybrid Auth
    • Authentication
    • Local users
    • User roles
    • User groups
    • User sessions
    • Brute force attack detection
    • Identity provider integration
    • Okta OIDC integration
    • Okta SAML integration
    • LDAP integration
    • OAuth client authentication

Clarity

  • Monitoring
  • Set up SaaS Clarity
  • Cloudwatch monitoring
  • Metrics

Druid administration

  • Configuration reference
  • Logging
  • API reference
  • Druid design

    • Design
    • Segments
    • Processes and servers
    • Deep storage
    • Metadata storage
    • ZooKeeper

    Data management

    • Overview
    • Data updates
    • Data deletion
    • Schema changes
    • Compaction
    • Automatic compaction

    Security

    • Security overview
    • User authentication and authorization
    • LDAP auth
    • Dynamic Config Providers
    • Password providers
    • Authentication and Authorization
    • TLS support
    • Row and column level security

    Performance tuning

    • Basic cluster tuning
    • Segment size optimization
    • Mixed workloads
    • HTTP compression
    • Automated metadata cleanup

    View Manager

    • View Manager
    • View Manager API
    • Create a view
    • List views
    • Delete a view
    • Inspect view load status
  • Rolling updates
  • Using rules to drop and retain data
  • Alerts
  • Java runtime
  • Working with different versions of Apache Hadoop
  • Misc

    • dump-segment tool
    • reset-cluster tool
    • pull-deps tool
    • Deep storage migration
    • Export Metadata Tool
    • Metadata Migration

View Manager extension

> Imply View Manager is an [alpha feature](/latest/preview-features) that should be considered preview and subject to change or removal at any time. Alpha features are provided "as is," and are not subject to Imply SLAs.

The Imply View Manager extension lets you define views that expose only a subset of data for a datasource. The view manager is a key feature for Druid's Row and column security model.

The View Manager provides a set of View Manager APIs for view management. To manage permissions to views, use the Coordinator Security API. For an example, see Row and column level security.

The Druid Coordinator acts as the "view manager" for SQL view definitions and stores the definitions in the SQL metadata store in the druid_views table. Druid Brokers periodically refresh a snapshot of the current view state from the Coordinators. Brokers also listen for updates pushed from the Coordinator. When they receive an update, the Brokers modify the schema to match the current set of view definitions, making them available for querying. You can use the API to check the freshness and availability of views on all Brokers.

Unsupported use cases

The View Manager provides row and column level security by filtering the data in rows and columns to prevent users access to data. The feature is not designed to provide a generic logic abstraction layer for Druid.

The following query features are not supported in view definitions:

  • Subqueries of any kind.
  • Joins of any kind.
  • UNION ALL.
  • Aggregations.
  • LIMIT, OFFSET, ORDER BY operations.
  • Queries on anything that is not a base datasource: lookups, INFORMATION_SCHEMA, sys.* tables, or other views.
  • Transforms on the projected columns. Note: transforms within filters are allowed.

Configuration

Enable and configure the View Manager in the common.runtime.properties.

  • Load the "imply-view-manager" extension. For example:
druid.extensions.loadList=["druid-hdfs-storage", "druid-kafka-indexing-service", "druid-datasketches", "druid-basic-security", "imply-view-manager"]
  • Specify the view manager:
# Specify the view manager
druid.sql.viewmanager.type=imply

Managing view access

You can manage access to views similar to the way you manage access to datasourcs using the Coordinator Security API. The VIEW resource type controls access to views as follows:

  • To query a view, you need READ access to the view. For example, to access a view named "english":
    {
      "resource": {
        "name": "english",
        "type": "VIEW"
        },
    "action": "READ"
    }
    
  • To create or delete a view, you need WRITE access to the CONFIG type resource. For example:
       {
         "resource": {
           "name": ".*",
           "type": "CONFIG"
         },
           "action": "READ"
       },
       {
         "resource": {
           "name": ".*",
           "type": "CONFIG"
         },
            "action": "WRITE"
       }
    

Advanced configuration and tuning properties

The following properties are for tuning the View Manager behavior. In general you shouldn't need to change these properties without guidance:

propertydescriptiondefault
druid.sql.viewmanager.imply.pollingPeriodFrequency in milliseconds that Brokers poll Coordinators to refresh view state60000
druid.sql.viewmanager.imply.maxRandomDelayRandom delay in milliseconds between Broker polling period to prevent herd effects from overwhelming Coordinators.6000
druid.sql.viewmanager.imply.maxSyncRetriesNumber of retry attempts before Brokers abandon a view state refresh from Coordinators5
druid.sql.viewmanager.imply.cacheDirectoryLocal disk path where Brokers can store a snapshot of viewstate to allow cold startup when Coordinators are not available.None.
druid.sql.viewmanager.imply.enableCacheNotificationsAllow Coordinators to propagate view state changes to Brokers.true
druid.sql.viewmanager.imply.cacheNotificationTimeoutTimeout period in milliseconds for view state change notifications.5000

Learn more

See the following topics for more information:

  • Row and column level security to learn how to use views to secure data.
  • View Manager APIs for a list of APIs and links to examples.
  • Coordinator Security API for details of the security API.
Last updated on 11/17/2022
← Automated metadata cleanupView Manager API →
  • Unsupported use cases
  • Configuration
  • Managing view access
  • Advanced configuration and tuning properties
  • Learn more
2023.03
Key links
Try ImplyApache Druid siteImply GitHub
Get help
Stack OverflowSupportContact us
Learn more
Apache Druid forumsBlog
Copyright © 2023 Imply Data, Inc