https://flyte.org logo
Join Slack
Powered by
Hey :wave::skin-tone-2: , Has anyone made the flyt...
# flyte-support
s
Hey 👋🏻 , Has anyone made the flyte deck work for notebook tasks? I got a 404 after a successful run Also, is there a way to copy the executed notebook ipynb or html to GCS ? I got file not found when I tried to use the implicit output of notebook task
Copy code
No such file or directory: '/v3/f25eaabfb23924c25946-n1-0/d1a84d1c0cea75f237d21a70dcde6810/model1-out.ipynb'
using this flag on flytectl register files
Copy code
--outputLocationPrefix
and setting it to gcs bucket did the trick for me
f
So notebook should automatically show up in deck. No need of raw? @shy-holiday-15500 infact contributed this
s
no the flyte deck still doesn't work but the executed notebooks are stored in gcs bucket which is what I wanted, but making flyte deck work would be awesome too
s
It's intended that it should show up as a deck without any work. Note that there is a kwarg you must pass to get it to show up (please check the docstring, I'm not at my computer right this second). I have used this a good bit on AWS.
❤️ 1
s
Yes I set render_deck=True but it doesn't work
s
Huh interesting. Could this be a GCS specific issue?
Ah!
😄 1
Do you get an error when you navigate to see the deck? I can investigate more in a few hours (commuting right now)
My current guess is that I need to tweak something to better support custom output location prefixes... My company doesn't use those in our registrations which could explain why it isn't showing up (when you are using this setting). Will verify this root cause first when I'm at a laptop.
f
It shouldn’t be gcs specific. But I can help look at the code. The output should be rendered into the deck else it won’t work
s
No I don't get any error when I'm getting to flyte deck tab, I get a 404 file not found error and then in the console logs I find that the payload is pointing to this bucket
Copy code
<gs://a-production-storage/metadata/propeller/notebooks-development-f7dfe709de49846358de/n1/data/0/deck.html>
but the one I get from my notebook tasks point out to my outputLocationPrefix gcs path, maybe flyte deck should point to this bucket as well ?
👍 1
s
@straight-ghost-77338 do the default decks for other tasks work for you? e.g., a python task's deck that shows the inputs/outputs? I'm realizing the code that is specific to the shell task doesn't have anything that "knows about" where the deck_uri is pointing... knowing that this impacts multiple decks will help pin down root cause
s
I can see inputs and outputs for every task including notebookTasks
👍 1
thx 1
s
Hmm, looking at the difference between what I did and whats in the deck example, I suspect I needed to append to the default deck, rather than appending to the .decks list. I haven't traced through why that would matter, and I don't know how to write a unit test for this yet. Will try to work through that today.
To keep you moving, let me quickly put up a quick branch with the change to see if it fixes things for you. If you haven't done this before, Flytekit is easy to tweak without changing your whole deployment because it gets installed into your container, you can pip install a specific git commit to test out a change.
s
Sure would love to test it thank you for that , these are the version im currently running :
Copy code
flytekit                      ==1.2.0
flytekitplugins-papermill     ==1.2.0
s
I'll give this a quick spin as well on my end
Copy code
flytekit==1.2.0
git+<https://github.com/ProjectAussie/flytekit.git@cl/bugfix-custom-output-dir-and-papermill-deck#subdirectory=plugins/flytekit-papermill>
🙏 1
^ what I'm testing
👍 1
Hmm I'm getting a seemingly unrelated error on my test environment with a number of test tasks
Will be a bit longer today before I can smoke test this change
Hmm well appending it to default_deck makes it no longer show up for me either! 😆 I think I'll need to read the inner workings of how the deck uri gets passed around before I can confidently resolve this
f
You don’t need to append to default deck. Each deck is just a tab
Is this some weird bug
157 Views
Open in Slack
PreviousNext
Powered by