Edgar Trujillo
06/27/2022, 9:56 PMlog
plugin is defined as well
task_logs.yaml: |
plugins:
logs:
cloudwatch-enabled: true
cloudwatch-log-group: 'bv-ml-pipelines'
cloudwatch-region: 'us-east-1'
kubernetes-enabled: false
Yee
Haytham Abuelfutuh
<https://console.aws.amazon.com/cloudwatch/home?region=%s#logEventViewer:group=%s;stream=var.log.containers.{{> .podName }}_{{ .namespace }}_{{ .containerName }}-{{ .containerId }}.log
You can, alternatively, use cloudwatch-template-uri
to specify any other log stream url you configured…
task_logs.yaml: |
plugins:
logs:
cloudwatch-template-uri: '<https://console.aws.amazon.com/cloudwatch/home?region=us-east-1#logsV2:log-groups/log-group/bv-ml-pipelines$3FlogStreamNameFilter=var.log.containers.{{> .podName }}_{{ .namespace }}_{{ .containerName }}'
kubernetes-enabled: false
I just filed this issue to change the default template to work with FluentD OOB.
If you also want array tasks to use the same config, you will, at least as of now, configure them separately like this:
task_logs.yaml: |
plugins:
logs:
cloudwatch-template-uri: '<https://console.aws.amazon.com/cloudwatch/home?region=us-east-1#logsV2:log-groups/log-group/bv-ml-pipelines$3FlogStreamNameFilter=var.log.containers.{{> .podName }}_{{ .namespace }}_{{ .containerName }}'
kubernetes-enabled: false
k8s-array:
logs:
config:
cloudwatch-template-uri: '<https://console.aws.amazon.com/cloudwatch/home?region=us-east-1#logsV2:log-groups/log-group/bv-ml-pipelines$3FlogStreamNameFilter=var.log.containers.{{> .podName }}_{{ .namespace }}_{{ .containerName }}'
kubernetes-enabled: false
Yee
Edgar Trujillo
06/28/2022, 4:04 PM/aws/containerinsights/ml-pipelines/*
but with the updated config map matching
task_logs.yaml: |
plugins:
logs:
cloudwatch-template-uri: '<https://console.aws.amazon.com/cloudwatch/home?region=us-east-1#logsV2:log-groups/log-group/bv-ml-pipelines$3FlogStreamNameFilter=var.log.containers.{{> .podName }}_{{ .namespace }}_{{ .containerName }}'
kubernetes-enabled: false
k8s-array:
logs:
config:
cloudwatch-template-uri: '<https://console.aws.amazon.com/cloudwatch/home?region=us-east-1#logsV2:log-groups/log-group/bv-ml-pipelines$3FlogStreamNameFilter=var.log.containers.{{> .podName }}_{{ .namespace }}_{{ .containerName }}'
kubernetes-enabled: false
FlyteConsole is showing no logs... can this be a permissions issue? Either flyte-user/system role not having the permissions?fluent-bit-config
to: log_stream_prefix var.
as the default was log_stream_prefix ${HOST_NAME}-
Yee
cloudwatch-template-uri
, could you set cloudwatch-enabled
to true
as well?Haytham Abuelfutuh