Hi all I m hoping to add logging to our `flyte binary` deplo Flyte #flyte-deployment

Hi all, I'm hoping to add logging to our `flyte-bi...

echoing-carpenter-92090

06/22/2023, 8:30 PM

Hi all, I'm hoping to add logging to our

flyte-binary

deployment on AWS EKS. Is there any change in AWS role permissions to allow Cloudwatch writing? If so, it looks like I have 4 active roles in IAM related to flyte (nodeInstance, cluster service, load balancer, and

flyte-system-role

echoing-carpenter-92090

06/22/2023, 8:59 PM

While I'm able to link out to Cloudwatch, when I click through, I get "log stream does not exist" in the AWS console.

echoing-carpenter-92090

06/22/2023, 8:59 PM

message has been deleted

tall-lock-23197

06/23/2023, 5:21 AM

How does your

task_logs

configuration look like? https://docs.flyte.org/projects/cookbook/en/latest/auto/deployment/configure_logging_links.html

echoing-carpenter-92090

06/23/2023, 6:50 PM

Copy code

logging:
    level: 1
    plugins:
      cloudwatch:
        enabled: true
        templateUri: |-
          <https://console.aws.amazon.com/cloudwatch/home?region=us-west-2#logEventViewer:group=/eks/opta-development/cluster;stream=var.log.containers.{{> .podName }}_{{ .namespace }}_{{ .containerName }}-{{ .containerId }}.log

freezing-boots-56761

06/26/2023, 1:27 PM

Logs are assumed to be sent to cloudwatch via fluentd or fluentbit or equivalent. Flyte task containers simply log to stdout/err. The k8s cluster needs to be configured to correctly handle the container logs. https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Container-Insights-EKS-logs.html

gratitude thank you 1

freezing-boots-56761

06/26/2023, 1:29 PM

Assuming that is already set up correctly, you might want to view the log groups available on Cloudwatch and see if you can navigate to the container logs in question. From there work backwards to the correct logs link template. Are you sure “/eks/opta-development” is right?

171 Views

Open in Slack

Previous Next