<#6988 Track ResourceVersion in TaskAction watcher...
# flyte-github
a
#6988 Track ResourceVersion in TaskAction watcher to prevent duplicate ADDED events Issue created by pingsutw Problem The
ActionsClient.doWatch
in
actions/k8s/client.go
does not track or pass a
ResourceVersion
when re-establishing the watch. When the watch channel closes (server-side timeout, network blip, etc.),
watchLoop
retries by calling
doWatch
again, which starts a fresh watch without a ResourceVersion. This causes the API server to replay all existing CRs as
ADDED
events, resulting in duplicate
RecordAction
gRPC calls. This happens even without a server restart — Kubernetes watches have a server-side timeout (typically 5-10 minutes), after which the watch channel closes and reconnects. Current Mitigation An
OppoBloomFilter
was added in #6987 to deduplicate
RecordAction
calls at the application layer, which prevents the duplicate DB writes. However, the root cause (unnecessary event replays) remains. Proposed Fix Track the
ResourceVersion
from the last seen event and pass it when restarting the watch via
ListOptions
. On reconnect, the API server will only send events that occurred after that version. Handle
410 Gone
(resource version too old / compacted) by resetting and doing a full relist. References •
actions/k8s/client.go
—
doWatch
and
watchLoop
methods • K8s API Concepts: Efficient detection of changes flyteorg/flyte