<#6994 [V2] Add TaskAction garbage collector for t...
# flyte-github
a
#6994 [V2] Add TaskAction garbage collector for terminal CRDs Pull request opened by pingsutw Tracking issue Closes #6995 Why are the changes needed? Terminal TaskAction CRDs (Succeeded/Failed) remain in the cluster indefinitely after completion. This wastes etcd storage and slows down list operations. We need a garbage collector that periodically deletes terminal TaskActions after a configurable TTL. What changes were proposed in this pull request? Adds a label-based TTL garbage collector for terminal TaskActions, modeled after the propeller FlyteWorkflow GC (
flytepropeller/pkg/controller/garbage_collector.go
): •
executor/pkg/config/config.go
— Added
GCConfig
struct with
Interval
and
MaxTTL
fields. •
executor/pkg/controller/taskaction_controller.go
— Added
ensureTerminalLabels()
method that stamps terminal TaskActions with
<http://flyte.org/termination-status=terminated|flyte.org/termination-status=terminated>
and
<http://flyte.org/completed-time=<UTC|flyte.org/completed-time=<UTC> hour>
labels. Called from both the terminal short-circuit path and after status updates that result in terminal state. Idempotent — no-op if labels already set. •
executor/pkg/controller/garbage_collector.go
(NEW) —
GarbageCollector
implementing
manager.Runnable
. Runs on a ticker, lists TaskActions with terminated label, filters by completed-time using lexicographic string comparison, deletes expired ones. •
executor/setup.go
— Conditionally adds GC to the controller manager when
cfg.GC.Interval.Duration > 0
•
executor/pkg/config/config_flags.go
/
config_flags_test.go
— Regenerated via
go generate
How was this patch tested? •
executor/pkg/controller/garbage_collector_test.go
(NEW) — Ginkgo/envtest tests for: expired deletion, recent retention, non-terminated retention, empty list handling, and
ensureTerminalLabels
idempotency •
executor/pkg/controller/taskaction_controller_test.go
— Extended with test verifying GC labels are set when reconciling a terminal TaskAction •
go build ./executor/...
compiles •
go test ./executor/pkg/config/...
passes Labels • added: New TaskAction garbage collector feature Check all the applicable boxes • I updated the documentation accordingly. • All new and existing tests passed. • All commits are signed-off. •
main
• #6583 • #6994 👈 flyteorg/flyte