New RunLLM response generated:
>>>*# Question*
Hey folks! We have a use cases to trigger Flyte workflows when some external dependency is ready, for example, a Iceberg/Parquet dataset was updated or there's a new event.
# Answer
To trigger Flyte workflows based on external dependencies like an updated Iceberg/Parquet dataset or a new event, you can use Flyte's sensor plugins or external event triggers.
1.
Sensor Plugins: Flyte supports sensor plugins that can wait for specific events, such as a file being updated in an object store. You can create a custom sensor plugin to wait for the dataset update.
2.
External Event Triggers: You can use external services like AWS Lambda to trigger Flyte workflows when an event occurs, such as a new file upload or a message in a queue.
For more details, refer to the
Flyte sensor plugin example and the
event-driven workflow example.
Would you like more detail on either of these approaches?
Sources:
-
Flyte sensor plugin example
Slack Link
https://flyte-org.slack.com/archives/CP2HDHKE1/p1724791406.658189 👍 0 button 👎 0 button 💡
Hint: Mention @runllm in the thread for followups.