Hello all.. so I have a flyte setup where logs go to splunk and it works properly. However for map tasks it looks like the logging is not setup correctly and shows a non working k8s link in the flyte UI. Is there some additional config required?

Hey @Rupsha Chaudhuri, it looks like maptask has a separate config for logs. This can be set under the

k8s-array

specific plugin configuration. I believe the nested keys are

log

and

config

. So something like this:

plugins:
  # All k8s plugins default configuration
  k8s:
    # k8s plugin configuration
  k8s-array:
    logs:
      config:
        kubernetes-enabled: true
        kubernetes-url: "<http://localhost:30082>"
  # Logging configuration
  logs:
    kubernetes-enabled: true
    kubernetes-url: "<http://localhost:30082>"

thanks.. so basically k8s-array

yeah, regular tasks use the logs at the plugin config level, k8s-array defines it's own separate, similar to spark. I'm wondering if it would make sense to have a flag allowing these to use the global config rather than their own default - it may be more useful.

+1 for common config

It would be under

plugins

and it looks like it requires another layer

config

so:

task_logs:
    plugins:
        logs: # existing configuration
            templates:
        k8s-array: # adding configuration
            logs:
                config:
                    templates:

I can test this locally to be sure, but I don't have time right now. Probably in an hour or so.

⭐ Create a new Flyte Core Feature issue: https://github.com/flyteorg/flyte/issues/new?assignees=&labels=enhancement%2Cuntriaged&template=feature_request.yaml&title=%5BCore+feature%5D+

Sure.. I'll add it!