https://flyte.org logo
#contribute
Title
# contribute
y

Yubo Wang

06/08/2023, 6:57 PM
hi team, is there any ongoing RFCs or work related to flyte backend logging? For example, improve on propeller logs
d

Dan Rammer (hamersaw)

06/08/2023, 7:09 PM
can you elaborate? just cleaning up messages and making more accessible?
y

Yubo Wang

06/08/2023, 7:11 PM
so, the logs are not accessible unless look into flytepropeller pod logs
I am imagining
fltyectl log execution...
and can get all the system logs that’s related to that execution
similar to what
kubectl logs
can do
d

Dan Rammer (hamersaw)

06/08/2023, 7:21 PM
interesting, is the idea that this parses logs from all flyte components (ie. admin, propeller, datacatalog, etc)?
y

Yubo Wang

06/08/2023, 7:21 PM
ideally
so collecting logs from different components actually is annoying when we are trying to debug
often have to jump back and forth between different backend components
I guess it is not an issue if people use datadog integration, but at Linkedin, we don’t have that
@Dan Rammer (hamersaw) I can do some kind of poc on this if you think this is interesting. I can try deploy fluentd sidecar and try some log streaming with flytectl
d

Dan Rammer (hamersaw)

06/08/2023, 11:00 PM
i think this would be really cool! my only concern is the diversity in deployments, it would be impossible to create a
flytectl log ...
command that will work in all scenarios. i'm wondering if this is something baked into flytectl, or a separate debugging script somewhere? thoughts?