orange-arm-76433
02/07/2023, 1:39 PMorange-arm-76433
02/07/2023, 1:43 PME0207 13:40:41.336097 1 reflector.go:138] pkg/mod/k8s.io/client-go@v0.24.1/tools/cache/reflector.go:167: Failed to watch *v1alpha1.FlyteWorkflow: failed to list *v1alpha1.FlyteWork
flow: Get "https://10.54.32.1:443/apis/flyte.lyft.com/v1alpha1/flyteworkflows?labelSelector=termination-status+notin+%28terminated%29&limit=500&resourceVersion=0": context deadline excee
ded
W0207 13:42:02.338334 1 reflector.go:324] pkg/mod/k8s.io/client-go@v0.24.1/tools/cache/reflector.go:167: failed to list *v1alpha1.FlyteWorkflow: Get "https://10.54.32.1:443/apis/fl
yte.lyft.com/v1alpha1/flyteworkflows?labelSelector=termination-status+notin+%28terminated%29&limit=500&resourceVersion=0": context deadline exceeded
I0207 13:42:02.338403 1 trace.go:205] Trace[1771930903]: "Reflector ListAndWatch" name:pkg/mod/k8s.io/client-go@v0.24.1/tools/cache/reflector.go:167 (07-Feb-2023 13:41:32.337) (tot
al time: 30000ms):
Trace[1771930903]: ---"Objects listed" error:Get "https://10.54.32.1:443/apis/flyte.lyft.com/v1alpha1/flyteworkflows?labelSelector=termination-status+notin+%28terminated%29&limit=500&res
ourceVersion=0": context deadline exceeded 30000ms (13:42:02.338)
Trace[1771930903]: [30.000971812s] [30.000971812s] ENDorange-arm-76433
02/07/2023, 1:52 PMflyteadmin [260.964ms] [rows:25] SELECT * FROM "executions" WHERE executions.execution_project = '<redacted>' AND executions.execution_domain = 'development' AND executions.state = '
EXECUTION_ACTIVE' ORDER BY created_at desc LIMIT 25
the query actually takes 200ms ish - i verified by going directly to the database
executions table holdes 370k records atm:
flyteadmin=# select count(*) from executions
flyteadmin-# ;
count
--------
373647
(1 row)freezing-airport-6809
freezing-airport-6809
orange-arm-76433
02/07/2023, 3:44 PMorange-arm-76433
02/07/2023, 3:45 PMorange-arm-76433
02/07/2023, 3:45 PMfreezing-airport-6809
freezing-airport-6809
freezing-airport-6809
orange-arm-76433
02/07/2023, 3:46 PMorange-arm-76433
02/07/2023, 3:46 PMfreezing-airport-6809
freezing-airport-6809
orange-arm-76433
02/07/2023, 3:47 PMfreezing-airport-6809
freezing-airport-6809