strong-carpet-69168
08/24/2023, 6:01 PMvalues.yaml
we added the following segment, but with no success.
(I am awere that we want to log the entire namespace)
task_logs:
plugins:
logs:
templates:
- displayName: Grafana Loki
templateUris:
- '<http://grafana-loki.loki.svc.cluster.local:3100/loki/api/v1/query_range?start=1692873712&end=1692880912&query={namespace=>"our-namespace"}'
messageFormat: 2
scheme: TaskExecution
What do we need to do in order to see the task logs in the UI.thankful-minister-83577
plugins:
logs:
kubernetes-enabled: false
cloudwatch-enabled: true
cloudwatch-template-uri: <https://your-region.console.aws.amazon.com/cloudwatch/home?region=us-east-2#logsV2:log-groups/log-group/$252Faws$252Feks$252Fmy-eks-cluster$252Fworkloads/log-events/fluentbit-kube.var.log.containers.{{.podName}}_{{.namespace}}_{{.containerName}}-{{.containerID}}.log>
stackdriver-enabled: false
is what we have internally.thankful-minister-83577
thankful-minister-83577
strong-carpet-69168
08/24/2023, 8:20 PMstrong-carpet-69168
08/24/2023, 8:33 PMthankful-minister-83577
thankful-minister-83577
thankful-minister-83577
strong-carpet-69168
08/25/2023, 6:35 AMvalues.yaml
like this:. Still no logs in the UI
task_logs:
plugins:
logs:
kubernetes-enabled: true
cloudwatch-enabled: false
stackdriver-enabled: false
templates:
- displayName: Grafana Loki
templateUris:
- '<http://grafana-loki.loki.svc.cluster.local:3100/loki/api/v1/query_range?start=1692873712&end=1692880912&query={namespace=>"our-namespace"}'
messageFormat: 2
scheme: TaskExecution
Can you please tell me what lines of configuration I need to put into my values.yaml
for my helm
installation?thankful-minister-83577
thankful-minister-83577
thankful-minister-83577
templates
but no longer isstrong-carpet-69168
08/25/2023, 9:32 PMtask_logs:
plugins:
logs:
kubernetes-enabled: true
cloudwatch-enabled: false
stackdriver-enabled: false
kubernetes-template-uri: '<http://grafana-loki.loki.svc.cluster.local:3100/loki/api/v1/query_range?start=1692873712&end=1692880912&query={namespace=>"our-namespace"}'
thankful-minister-83577
thankful-minister-83577
strong-carpet-69168
08/27/2023, 7:32 PMthankful-minister-83577
thankful-minister-83577
thankful-minister-83577
thankful-minister-83577
freezing-airport-6809
freezing-airport-6809
high-park-82026
@task()
def my_task():
...
Or is it a plugin (e.g. Spark task)? because those are configured in a different sectionhigh-park-82026
configmap:
task_logs:
plugins:
logs:
...
freezing-airport-6809
strong-carpet-69168
08/30/2023, 10:49 PMbulky-machine-98870
08/31/2023, 12:35 PMconfigmap
key and had configuration.logging.plugins.kubernetes.enabled: false
.
I was wondering, if these details were documented somewhere and we just didn't find the right resource?
Anyway, huge thanks again 😉
Greets from Germany
Manuel
cc: @strong-carpet-69168high-park-82026
flyte.configmap.task_logs
Would you be open to writing up a doc on configuring log links perhaps? we can include that as part of docs.flyte.org so the next person wouldn't have to go through the same struggle 🙂bulky-machine-98870
09/04/2023, 6:21 AM