Another question, but related - we right now use scheduled launchplans in order to process incoming new data. We execute the pipeline every 4h or so, which checks if there is any new data in the bucket. This works, but is not perfect. Ideally we would have a cloud storage based trigger, which executes a pipeline. Has anyone implemented this in the past and can share some code, experience and/or best practices?

Triggers are available in union. https://docs.union.ai/developing-workflows/triggers They are attached to launchplans. Happy to demo union. Will also solve some other issues