https://flyte.org logo
h

honnix

06/21/2022, 10:04 AM
k

Ketan (kumare3)

06/21/2022, 2:22 PM
@honnix so the notebook task can be any task - if I would redo it I will create a new task base called wrapper tasks
The idea is to allow a new facade for any other task type - like for example spark
And allow notebooks to execute spark tasks
h

honnix

06/21/2022, 2:29 PM
Right, that makes sense! I was wondering why not generalizing that “hacky” part. Will that wrapper task fit https://github.com/flyteorg/flytekit/blob/07299d0b0e8d172e33ad48d719f9014effb5d96b/plugins/flytekit-greatexpectations/flytekitplugins/great_expectations/task.py as well? Why is there no hacky part?
k

Ketan (kumare3)

06/21/2022, 2:33 PM
Ya I think we should - up for a contribution 🤩
3 Views