<#7250 InternalAppService.Watch can drop events du...
# flyte-github
a
#7250 InternalAppService.Watch can drop events during initial snapshot replay Issue created by pingsutw What In
app/internal/service/internal_app_service.go
(around lines 217-251),
InternalAppService.Watch
subscribes via
s.k8s.Watch(...)
before calling
s.k8s.List(...)
, which is the right ordering. However, the watch channel (
ch
) is not drained until after the entire initial snapshot has been streamed back to the client: ch, err := s.k8s.Watch(ctx, project, domain, appName) // ... snapshot, _, err := s.k8s.List(ctx, project, domain, appName, 0, "") // ... for _, app := range snapshot { stream.Send(...) // can take arbitrary time; ch is not being read } for { select { case <-ctx.Done(): ... case event, ok := <-ch: ... } } If
s.k8s.Watch
returns an unbuffered or small-buffered channel, events produced while the snapshot is being sent can block the producer or be dropped, defeating the purpose of starting the watch early. Why
RunService.WatchRuns
(
runs/service/run_service.go:1143
) handles this correctly: it starts a goroutine that pushes updates into a buffered channel (size 50) before listing, so events are queued during the initial list + send phase and consumed afterward.
InternalAppService.Watch
should follow the same pattern so no events are lost between subscribing and finishing the initial snapshot replay. Proposed change • Ensure the watch producer writes into a sufficiently buffered channel (or a goroutine-backed buffer) started before
List
. • Drain/forward events from that buffer after the snapshot is sent. • Mirror the
WatchRuns
structure for consistency. References •
app/internal/service/internal_app_service.go:197-252
— current
InternalAppService.Watch
•
runs/service/run_service.go:1143-1199
—
RunService.WatchRuns
reference pattern flyteorg/flyte