Carsten Klaus
08/24/2023, 6:01 PMvalues.yaml
we added the following segment, but with no success.
(I am awere that we want to log the entire namespace)
task_logs:
plugins:
logs:
templates:
- displayName: Grafana Loki
templateUris:
- '<http://grafana-loki.loki.svc.cluster.local:3100/loki/api/v1/query_range?start=1692873712&end=1692880912&query={namespace=>"our-namespace"}'
messageFormat: 2
scheme: TaskExecution
What do we need to do in order to see the task logs in the UI.Yee
plugins:
logs:
kubernetes-enabled: false
cloudwatch-enabled: true
cloudwatch-template-uri: <https://your-region.console.aws.amazon.com/cloudwatch/home?region=us-east-2#logsV2:log-groups/log-group/$252Faws$252Feks$252Fmy-eks-cluster$252Fworkloads/log-events/fluentbit-kube.var.log.containers.{{.podName}}_{{.namespace}}_{{.containerName}}-{{.containerID}}.log>
stackdriver-enabled: false
is what we have internally.Carsten Klaus
08/24/2023, 8:20 PMYee
Carsten Klaus
08/25/2023, 6:35 AMvalues.yaml
like this:. Still no logs in the UI
task_logs:
plugins:
logs:
kubernetes-enabled: true
cloudwatch-enabled: false
stackdriver-enabled: false
templates:
- displayName: Grafana Loki
templateUris:
- '<http://grafana-loki.loki.svc.cluster.local:3100/loki/api/v1/query_range?start=1692873712&end=1692880912&query={namespace=>"our-namespace"}'
messageFormat: 2
scheme: TaskExecution
Can you please tell me what lines of configuration I need to put into my values.yaml
for my helm
installation?Yee
templates
but no longer isCarsten Klaus
08/25/2023, 9:32 PMtask_logs:
plugins:
logs:
kubernetes-enabled: true
cloudwatch-enabled: false
stackdriver-enabled: false
kubernetes-template-uri: '<http://grafana-loki.loki.svc.cluster.local:3100/loki/api/v1/query_range?start=1692873712&end=1692880912&query={namespace=>"our-namespace"}'
Yee
Carsten Klaus
08/27/2023, 7:32 PMYee
Ketan (kumare3)
Haytham Abuelfutuh
@task()
def my_task():
...
Or is it a plugin (e.g. Spark task)? because those are configured in a different sectionconfigmap:
task_logs:
plugins:
logs:
...
Ketan (kumare3)
Carsten Klaus
08/30/2023, 10:49 PMManuel Hoffmann
08/31/2023, 12:35 PMconfigmap
key and had configuration.logging.plugins.kubernetes.enabled: false
.
I was wondering, if these details were documented somewhere and we just didn't find the right resource?
Anyway, huge thanks again 😉
Greets from Germany
Manuel
cc: @Carsten KlausHaytham Abuelfutuh
flyte.configmap.task_logs
Would you be open to writing up a doc on configuring log links perhaps? we can include that as part of docs.flyte.org so the next person wouldn't have to go through the same struggle 🙂Manuel Hoffmann
09/04/2023, 6:21 AM