Any example of how to configure fluentd or fluent-bit helm chart to save k8s logs to s3 in a the url format that Flyte expects for logging links to work?

Hey Paul The closest is to generate a logging link that takes you to Cloudwatch. You want Flyte to send the logs to S3?

yeah, i thought then i could just link to a text file in s3 that has the logs. at least it looks like in the docs ( https://www.union.ai/docs/flyte/deployment/flyte-configuration/configuring-logging-links-in-the-ui/ ) that's basically what cloudwatch is doing-- just serving up a text file with a name keyed by some params from the k8s container name

I'm using the Cloudwatch EKS add-on which uses Fluent-bit under the hood. No configuration required there at all. The only trick was finding out the default link format and the right template to pass to the Flyte helm chart for where to look.

logging: {
  level: 5,
  plugins: {
    cloudwatch: {
      enabled: true,
      templateUri: `<https://console.aws.amazon.com/cloudwatch/home?region=${this.cluster.env.region}#logEventViewer:group=/aws/containerinsights/${this.cluster.clusterName}/application;stream={{> .nodeName }}-application.var.log.containers.{{ .podName }}_{{ .namespace }}_{{ .containerName }}-{{ .containerId }}.log`,
    },
  },
},

got it, thanks! looked at EKS configs a little, maybe they can help

The Cloudwatch EKS add-on has a helm chart where you can see how they configured fluent-bit to get the logs to cloudwatch. You could swap that to the fluent bit s3 output, and probably configure a dynamic link from the Flyte UI.

Re: the right configuration for Flyte this is yet to be updated in the docs https://github.com/flyteorg/flyte/issues/5760

Thank you, especially @brainy-cartoon-98942 for the citation! fluent-bit can indeed upload logs, but trouble is once they're in S3 they can be multiple files, so that doesn't work with flyte's URL-based API. maybe my

s3_key_format

was wrong tho? surprised i can't find any straightforward blogs on this