Hi all! :wave: Our team is currently evaluating a ...
# ask-the-community
j
Hi all! 👋 Our team is currently evaluating a migration from Airflow to Flyte, and we’re excited about the possibilities Flyte has to offer. However, we have a couple of questions/were hoping for some guidance. Slack Notifications: In our current Airflow setup, we heavily rely on Slack notifications triggered by callbacks at both the DAG and task levels. Looking through the documentation, we couldn’t find information on setting up Slack notifications for a Flyte deployment on GCP. Could anyone provide resources on how to achieve similar functionality with Flyte? Params: currently we pass a function that loads a YAML file into a dictionary to the
params
argument of the DAG when creating DAGs. This allows us to template our SQL files. We’re curious if there’s a similar mechanism or recommended practice in Flyte for achieving this parameterization for SQL files. Our best solution, so far, has been to use the
string.Template(sql_path).substitute(params)
to add parameters and pass that to the BigQueryTask, however, any advice, examples, or pointers to better solutions would be incredibly helpful.
k
Welcome to the community and the timing might be great. We are about to launch agents and especially airflow agents ( @Kevin Su ) can share pointers. Slack notifications (it uses emails) so you can use smtp if you want. Another option is to continue using your airflow operators for slack notifications through Flyte agents ir write new agents. There is work for webhooks coming soon too Would love to understand more about Params. Seems you want to generate the workflow- if so use - imperative workflows. If you want runtime parameters then just use template substitution - Cc @Yee
g
+1 for webhooks so I can use MS teams based notifications
k
I know @Haytham Abuelfutuh / @Fabio Grätz / @Kevin Su
j
with the
params
, it allows us to pass a dictionary to a
dag
, where the keys value can be accessed in tasks (rendered at run-time) using
params.key
with jinja templates. Is there a way to pass a dictionary as the input to the
SQLTask
?
k
Not today but possible to add - especially with agent should be trivial