@cool-lifeguard-49380 / @brief-window-55364 / @agreeable-flower-8989 What do you think about this PR - https://github.com/flyteorg/flytekit/pull/2579 Would love some help in testing things out in real world

in general, i think this ability to run on cluster from notebook code is super useful! However, not clear to me how this would work with say an existing workflow that i have checked-in source code. Currently, we have our code in git repo mirrored in databricks notebook (previusly we had it synced to our version of jupyter notebook). My understanding is fast registration works on python file abstractions, here you are working at Flyte entities level right? If we update multiple python files, once mirrored to notebook, and we load those updated modules, execution on workflow relying on changes outside notebook cells will not work in the proposed impl?

You can import tasks from Python files and that should work but will test - can you provide me a structure

from mycompany.team_a.module_b import taskA

@workflow
def foo():
   taskA()

taskA is updated outside the notebook UI, but the

mycompany/team_a/module_b.py

is available in notebook filesystem

Yup imo this should work, but this helps a lot. It will need a small change

many of my users don't use notebook for flyte related tasks, fast-registering and launching the ui is convenient enough, or using

pyflyte run--remote

is good enough

This is awesome to hear

I haven't touched notebooks in quite some time and also our developers don't really use them ^^ That being said, I do believe that many data scientists will really love this - the UX shown in the video looks super smooth.

Ohh ya that was debugging - already fixed @cool-lifeguard-49380

Yeah, 5 years ago me using notebooks more often would have absolutely loved this.

Would it be possible to find some folks to try this / my goal also to make Flyte more accessible to more folks

I need to think about who.

Ya

jupyter nbconvert

Yup and basically ignore a few tagged elements

But that wouldn't remove the cells which start the execution, they would have to be stripped.

I was thinking of tagging using a Jupyter magics

Yes

%...

Ya that would be cool right

catching up here, do we still need users who might test this?