hey everyone, we are seeing a lot of re-starts in ...
# flyte-connectors
g
hey everyone, we are seeing a lot of re-starts in agent pod that do not happen to any of our other pods, we are curious if you guys have seen this before or have any idea why this might happen?
Copy code
yaou@yaou-mn1 ~ % k get pods
NAME                               READY   STATUS    RESTARTS       AGE
ambassador-69d4998797-2rf9r        1/1     Running   0              4d23h
datacatalog-864796d889-ngqx2       1/1     Running   0              4d23h
flyte-pod-webhook-76d88c67-6x7rb   1/1     Running   0              4d23h
flyteadmin-5cf46b95c7-8dnnk        1/1     Running   0              4d23h
flyteagent-56bd59b5d4-9vjcg        1/1     Running   37 (33m ago)   6d3h
flyteconsole-69c4c8795d-hnhg8      1/1     Running   0              67d
d
I think
maybe its OOM KILLED?
or the flytekit code in agent will some how crash
you have to print logs as much as possible to ensure that you didn't crash by the flytekit
only these 2 possibility
g
let me check
@mysterious-knife-69764 lolcry
d
I can schedule a call with you
but not today, have other priority
g
no worries, we just set it to log as much as possible, will reach back out if there’s findings / difficulty , thanks !
you can take a look at this
this can help you create your custom agent more easily
cc @gray-ram-51379 @mysterious-knife-69764