https://flyte.org logo
#flyte-deployment
Title
# flyte-deployment
f

Felix Ruess

01/09/2023, 11:12 AM
Hi, wondering what people are using for viewing/archiving container logs on self managed on prem k8s... did anyone find a nice setup yet, since flyte can't collect and archive task logs for you?
r

Rahul Mehta

01/09/2023, 4:27 PM
A common thing to consider for on-prem logging infra is the "ELK" stack (ElasticSearch, LogStash, & Kibana), or the alternative "EFK" stack (which replaces LogStash w/ fluentd). The drawback here is needing to manage an on-prem elasticsearch deployment, which can be a bit onerous. https://www.digitalocean.com/community/tutorials/how-to-set-up-an-elasticsearch-fluentd-and-kibana-efk-logging-stack-on-kubernetes If you're comfortable with using a hosted service for this, https://logz.io/ is a pretty good solution and can integrate w/ on prem environments.
f

Felix Ruess

01/09/2023, 4:53 PM
Thanks! The ELK stack is too heavy weight for me. Right now I'm using parts of the Grafana LGTM stack (so Loki for logs), but was wondering if there is an even simpler solution e.g. via a sidecar that uploads the logs to S3 (e.g. Argo workflows already has something like that integrated).
r

Rahul Mehta

01/09/2023, 6:06 PM
I rigged up a thing with fluentd that write JSON logs to S3 that uses https://docs.fluentd.org/output/s3
Spent a while fiddling with the config so happy to answer any questions if you decide to go down that route. https://vector.dev/ may also be of interest
f

Felix Ruess

01/09/2023, 6:10 PM
cool, thanks! I'll go with grafana agent operator (also creates daemonsets to scrape logs and metrics) and see how it goes...
124 Views