Skip to main content

Manage data

By default, Imply Lumi copies all ingested data to the hot tier to deliver low latency and high concurrency. For data you ingest into Lumi, you can use the following resources to manage storage costs and control retention:

  • Virtual tier is a cost-efficient alternative to the hot tier that loads data and spins up compute resources on demand at a lower cost. Tiering rules control when data moves out of the hot tier and becomes accessible through the virtual tier. Virtual compute pools spin up when a query targets data outside the hot tier and spin down after reaching the maximum idle time you configure.
  • Deletion rules determine how long Lumi retains data before deletion. You configure deletion rules to automatically remove data you no longer need, helping you manage storage costs and comply with data retention policies.

For data you want to keep in your own Amazon S3 storage instead, you can connect Lumi to S3 to use it as an external data store. When connected, Lumi indexes your data and writes the index to your S3. Queries are served on demand through the virtual tier, which can greatly reduce ingestion costs and the expense of keeping data in the hot tier.