Late to the party, I think what you, @Jon, have built is just fine! You are using KEDA to scale up based on pod events, right? an integration with flyte isn't strictly required but of course it can speed up scaling the cluster so it's ready for when the pods land.
Are there concerns about partial failures? What if a pod starts, reads a message, processes a few files before some catastrophic event happens (e.g. the spot machine disappears or the pod crashes)? Are the operations that pod perform idempotent (ideally)?