#7119 perf: parallel worker pool for TaskAction watch event processing
Pull request opened by
AdilFayyaz
Why are the changes needed?
Watch events for TaskActions were processed serially in the watch loop, causing status update delays under load. This adds a configurable worker pool with consistent sharding to parallelize processing while preserving per-resource ordering.
What changes were proposed in this pull request?
• Added WatchWorkers config field (default: 4) controlling the number of parallel event-processing goroutines.
• ActionsClient now owns a pool of worker channels; StartWatching spawns N workers on init.
• dispatchEvent routes events to workers via FNV-32a hash of TaskAction.Name, ensuring all events for the same resource go to the same worker (ordering preserved).
• Full channels drop events with a warning log rather than blocking the watch loop.
How was this patch tested?
• Run new unit tests in actions/k8s/client_test.go:
• TestDispatchEvent_ConsistentSharding — same name always routes to the same shard
• TestDispatchEvent_DifferentNamesCanLandOnDifferentShards — events spread across shards
• TestDispatchEvent_DropsErrorEvents / DropsNonTaskActionObjects — invalid events are discarded
• TestDispatchEvent_FullChannelDropsWithoutBlocking — full channel doesn't block
• TestWorker_ExitsOnStopCh / ExitsOnContextCancel — worker lifecycle
This is important to improve the readability of release notes.
Setup process
Screenshots
Check all the applicable boxes
• I updated the documentation accordingly.
• All new and existing tests passed.
• All commits are signed-off.
Related PRs
Docs link
•
main
•
#6583
•
perf: parallel worker pool for TaskAction watch event processing 👈
flyteorg/flyte