i just submitted my first pr and would greatly appreciate a review https://github.com/flyteorg/flyte/pull/7473, @clean-glass-36808 @wide-vegetable-51116 @powerful-horse-58724 @average-finland-92144

I am wondering if we should be using prometheus metrics at all because I prefer using OTEL as a frameworking for tracing + metrics