Hello folks! Curious what UX patterns are you using for Flyte <> Spark integration? Particularly, 1. Is there cap on number of concurrent Spark jobs can be started from Flyte job 2. Do you allow users to set priority on the jobs? 3. What do you use of Spark infra? (in house/EMR etc)

@creamy-policeman-62284, sorry for the delay. Answers in order: 1. there's no cap. 2. can you expand on that? 3. the Flyte spark plugin is agnostic to the Spark infra. We've seen people run it in-house, EMR, etc.