Attention Airflow users, <@U01J90KBSU9> recently ...
# announcements
f
Attention Airflow users, @tall-lock-23197 recently created a FlyteOperator in Airflow, and would love if some of you can upvote this operator - https://github.com/apache/airflow/pull/22646
πŸŽ‰ 1
πŸ‘ 1
cc @lemon-fish-7011 / @gorgeous-iron-33877
πŸ‘ 1
t
I’ve started a discussion in the devlist: https://lists.apache.org/thread/b55g3gydgmqmhow6f7xzzbm5t0gmhs2x. It’d be nice if some of you could share your thoughts. Only on getting some upvotes will the operator be merged into the Airflow repository. Thanks in advance!
b
@tall-lock-23197 to clarify, do folks need to add β€œ+1s” or specific comments to the discussion thread you started?
f
To the issue itself
g
I was reading through the linked discussion thread in the issue: https://lists.apache.org/thread/t1k3d0518v4kxz1pqsprdc78h0wxobg0 and I noticed they talked about testing the provider
Copy code
So I would say all future providers (if they are using some external
service) will have to have system tests implemented. That means few
things: first of all, there should be systems tests, secondly, there
should be some way (credits, free accounts with enough capacity to run
regular testing in an automated way - donated to Airflow in order to
be able to support a provider). We have not yet discussed the last
point (credits and system tests being condition) but this seems like
we are heading towards - with Amazon and Google providers leading the
effort of implementing system tests (we already got credits and
commitments from both).
which I think is a good point before an operator is added to airflow. Is there currently a hosted flyte option that could be used to enable this?
f
Flyte is an open source, linux foundation project, good thing is the test can be run on any machine with docker
πŸ‘ 1
g
So they could be run with the credits provided by AWS/GCP. Sounds good
πŸ‘ 1
t
@tall-lock-23197 to clarify, do folks need to add β€œ+1s” or specific comments to the discussion thread you started?
@broad-monitor-993, either one works.
168 Views