Skip to main content

Manage data

By default, Imply Lumi copies all ingested data to the hot tier to deliver low latency and high concurrency. To manage storage costs and control retention for data you ingest into Lumi, use the following:

  • Virtual tier is a cost-efficient alternative to the hot tier that loads data and provisions compute resources on demand at a lower cost. Tiering rules control when data moves out of the hot tier and becomes accessible through the virtual tier. A virtual compute pool spins up when a query targets data outside the hot tier and spins down after reaching the maximum idle time you configure.
  • Deletion rules determine how long Lumi retains data before deletion. Configure deletion rules to automatically remove data you no longer need, helping you manage storage costs and comply with data retention policies.

For data you want to keep in your own Amazon S3 storage instead, connect Lumi to S3 to use it as an external data store. When connected, Lumi indexes your data and writes the index to your S3 bucket. Queries are served on demand through the virtual tier, which can greatly reduce ingestion costs and the expense of keeping data in the hot tier.