2.5 KiB
Two independent MinIO expiry mechanisms
This platform expires objects in MinIO through two mechanisms that were built independently, for different features, and remain independent today. This note exists so a reader who finds one does not assume it is the whole story.
ModelImport.ensure_import_bundle_retention
laborious/activities/model_import.py. Applies a bucket lifecycle rule (a MinIO/S3-native
Expiration rule, ENABLED, keyed by a fixed rule_id) to the import bundle prefix
(import_config['prefix']), expiring objects after import_config['retention_days'] days. The
rule is declarative: once set, MinIO itself removes the object when it ages out — the workflow does
not scan for or delete anything. The activity's job is only to ensure the rule exists and matches
the configured days/prefix, which is what makes "lifecycle active" a criterion the code can
verify rather than an assumption about a Helm value.
Storage.cleanup_minio_objects_expired
laborious/activities/storage.py. An older, unrelated activity that scans and deletes: given a
list of key prefixes and a retention window (retention_hours), it lists objects under each prefix,
compares their age against now() - retention_hours, and deletes the ones that have aged out. There
is no bucket-side rule — the deletion happens in this code, on whatever schedule calls the activity.
Why they are not unified today
They expire different prefix families, on different units (days vs. hours), through different
mechanisms (declarative rule vs. active scan), and were written for different features at different
times. Neither mechanism depends on or interferes with the other; a bucket can carry an
ensure_import_bundle_retention lifecycle rule for one prefix and still have
cleanup_minio_objects_expired scanning a different prefix on its own cadence.
What unifying them would require
Either retiring the scan-and-delete path in favor of a lifecycle rule for every prefix it currently handles (only possible if every one of its callers can tolerate a fixed number of retention days, since S3-style lifecycle rules do not support hour-granularity windows), or teaching the lifecycle-rule path to manage a per-prefix table of rules with arbitrary retention windows instead of the one fixed rule it manages today. Either direction touches callers and configuration this change does not, and neither is a refactor — this is tracked as a separate, larger change, not a task of this one.