Are there existing plugins or tools for performing metrics queries on workflows/tasks? I don't find anything scanning docs and google. I'd love to ask for available metrics for workflow X for the past 10 days, e.g. for each task tell me the mem/cpu requests and the execution time (and maybe pluck key attributes from the Input).. I'd also really like to know peak cpu and memory utilization, but I suspect I need to query DataDog or similar to get this. I use Flyte Console a lot to look at what's available for individual workflow executions, but this is arduous for looking at even 10 or 20 jobs to understand distribution of execution times etc.

No such tools- union has this built in, but otherwise you will Have to use a metrics / APM system like datadog. Be warned it’s very expensive as the cardinality is very high

Thanks @Ketan (kumare3)