:wave: hi all! have you ever wanted to run flyteki...
# contributeb
broad-monitor-993
07/20/2023, 9:02 PMđź‘‹ hi all! have you ever wanted to run flytekit code directly from within a jupyter notebook?
Then I have the RFC for you! https://github.com/flyteorg/flyte/discussions/3878
Please add your comments to the RFC incubator post with ideas on what the ideal UX would be in your day-to-day work in jupyter notebooks.
🚀 8
broad-monitor-993
07/20/2023, 9:02 PMHere’s a working example that can be found in the PR referenced in that post:
broad-monitor-993
07/20/2023, 9:03 PMThe notebook is effectively treated as a full flytekit script, and the
%notebook_runpyflyte runbroad-monitor-993
07/20/2023, 9:04 PMThis is probably not the ideal UX, so please add your ideas!
f
freezing-airport-6809
07/21/2023, 1:24 AM
This is awesome
freezing-airport-6809
07/21/2023, 1:24 AM
Cc @agreeable-flower-8989 / @most-gold-65483
m
most-gold-65483
07/21/2023, 1:31 AM@broad-train-34581 ^ cool stuff here, will improve our onboarding experience quite a lot.
a
agreeable-flower-8989
07/21/2023, 9:30 PMoh nice! yah this will be super helpful to for quick iteration. So the idea here is that you prototype quickly on local notebook and get interactive feedback.. once you are happy and want to scale up you can luanch on cluster.
agreeable-flower-8989
07/21/2023, 9:31 PMthis is helpful when workflow lives in a self-contained code.. some notebooking system (ours, DB) have a notebook living as if a file in directory which has reference to local modules. So this change could be next step after treating notebook as "full flyte script"
b
broad-monitor-993
07/21/2023, 11:24 PMSo the idea here is that you prototype quickly on local notebook and get interactive feedbackyeah, so the idea is that I can interleave code that runs locally in my notebook runtime and send it up to run on a Flyte cluster when I need to scale. My question would be: would it make sense to support registering/running tasks/workflows in cells that are tagged “flyte” (or something), then be able to run those on a Flyte cluster? Basically the UX would be E.g.
Copy code
# runs on the notebook server
print("hello world") Copy code
{tags: "flyte"}
# this cell is special: it MUST NOT share state with regular cells
@task
def t1(x: int): ...
@workflow
def wf(x: int) -> int:
return t1(x=x) Copy code
%% notebook_run --config ... wf --x 5 Copy code
Running "wf" on <remote_execution_urL>/<execution_id>
Retrieve this execution with
remote = FlyteRemote(...)
execution = remote.fetch_execution(<exection_id>)4 Views