Hello,
Do you know if there is a flytekit plugins ...
# ask-the-communityf
Frank Shen
03/27/2023, 8:32 PMHello,
Do you know if there is a flytekit plugins for airflow?
I need to detect if an Airflow dag run is completed from Flyte.
Thanks.
n
Niels Bantilan
03/27/2023, 8:36 PMf
Frank Shen
03/27/2023, 8:44 PM@Niels Bantilan, thank you.
However, that package allows an Airflow dag to trigger a Flyte workfl.
What I want is different.
I want Flyte to check the status of an Airflow dag run, so that Flyte can trigger the dependent downstream tasks.
Frank Shen
03/27/2023, 8:44 PMAny idea?
e
Evan Sadler
03/27/2023, 8:51 PMIs there a python client for airflow that lets you list workflow executions?
If that existed then you could write a task that checks if an execution succeeded in the last X hours. It could sleep and retry every so often.
Evan Sadler
03/27/2023, 8:53 PMMaybe there is an API?
f
Frank Shen
03/27/2023, 9:32 PMYes, @Evan Sadler, I am working on it.
k
Ketan (kumare3)
03/27/2023, 10:59 PM
also folks - @Frank Shen as promised this is in the works and landing soon -
https://github.com/flyteorg/flytekit/pull/1524
Ketan (kumare3)
03/27/2023, 11:00 PM
This will make it simple to add all kinds of backend plugins, that are cheap to run, easy to write and very efficient
f
Frank Shen
03/27/2023, 11:04 PM@Ketan (kumare3), awesome!
150 Views