Hey :wave::skin-tone-2: , Has anyone made the flyt...
# ask-the-communityh
Hamza Azelmat
10/06/2022, 11:58 AMHey 👋🏻 , Has anyone made the flyte deck work for notebook tasks? I got a 404 after a successful run
Also, is there a way to copy the executed notebook ipynb or html to GCS ? I got file not found when I tried to use the implicit output of notebook task
Copy code
No such file or directory: '/v3/f25eaabfb23924c25946-n1-0/d1a84d1c0cea75f237d21a70dcde6810/model1-out.ipynb'Hamza Azelmat
10/06/2022, 12:59 PMusing this flag on flytectl register files
Copy code
--outputLocationPrefixk
Ketan (kumare3)
10/06/2022, 2:49 PM
So notebook should automatically show up in deck. No need of raw? @Calvin Leather infact contributed this
h
Hamza Azelmat
10/06/2022, 2:51 PMno the flyte deck still doesn't work but the executed notebooks are stored in gcs bucket which is what I wanted, but making flyte deck work would be awesome too
c
Calvin Leather
10/06/2022, 2:51 PMIt's intended that it should show up as a deck without any work. Note that there is a kwarg you must pass to get it to show up (please check the docstring, I'm not at my computer right this second). I have used this a good bit on AWS.
h
Hamza Azelmat
10/06/2022, 2:52 PMYes I set render_deck=True but it doesn't work
c
Calvin Leather
10/06/2022, 2:52 PMHuh interesting. Could this be a GCS specific issue?
Calvin Leather
10/06/2022, 2:53 PMCalvin Leather
10/06/2022, 2:54 PMDo you get an error when you navigate to see the deck? I can investigate more in a few hours (commuting right now)
Calvin Leather
10/06/2022, 3:00 PMMy current guess is that I need to tweak something to better support custom output location prefixes... My company doesn't use those in our registrations which could explain why it isn't showing up (when you are using this setting). Will verify this root cause first when I'm at a laptop.
k
Ketan (kumare3)
10/06/2022, 3:09 PM
It shouldn’t be gcs specific. But I can help look at the code. The output should be rendered into the deck else it won’t work
h
Hamza Azelmat
10/06/2022, 3:17 PMNo I don't get any error when I'm getting to flyte deck tab, I get a 404 file not found error and then in the console logs I find that the payload is pointing to this bucket
Copy code
<gs://a-production-storage/metadata/propeller/notebooks-development-f7dfe709de49846358de/n1/data/0/deck.html>c
Calvin Leather
10/06/2022, 6:36 PM@Hamza Azelmat do the default decks for other tasks work for you? e.g., a python task's deck that shows the inputs/outputs? I'm realizing the code that is specific to the shell task doesn't have anything that "knows about" where the deck_uri is pointing... knowing that this impacts multiple decks will help pin down root cause
h
Hamza Azelmat
10/07/2022, 11:54 AMI can see inputs and outputs for every task including notebookTasks
c
Calvin Leather
10/07/2022, 12:03 PMHmm, looking at the difference between what I did and whats in the deck example, I suspect I needed to append to the default deck, rather than appending to the .decks list. I haven't traced through why that would matter, and I don't know how to write a unit test for this yet. Will try to work through that today.
Calvin Leather
10/07/2022, 12:03 PMTo keep you moving, let me quickly put up a quick branch with the change to see if it fixes things for you. If you haven't done this before, Flytekit is easy to tweak without changing your whole deployment because it gets installed into your container, you can pip install a specific git commit to test out a change.
h
Hamza Azelmat
10/07/2022, 12:06 PMSure would love to test it thank you for that , these are the version im currently running :
Copy code
flytekit ==1.2.0
flytekitplugins-papermill ==1.2.0c
Calvin Leather
10/07/2022, 12:07 PMCalvin Leather
10/07/2022, 12:07 PMI'll give this a quick spin as well on my end
Calvin Leather
10/07/2022, 12:09 PM Copy code
flytekit==1.2.0
git+<https://github.com/ProjectAussie/flytekit.git@cl/bugfix-custom-output-dir-and-papermill-deck#subdirectory=plugins/flytekit-papermill>Calvin Leather
10/07/2022, 12:09 PM^ what I'm testing
Calvin Leather
10/07/2022, 12:40 PMHmm I'm getting a seemingly unrelated error on my test environment with a number of test tasks
Calvin Leather
10/07/2022, 12:40 PMWill be a bit longer today before I can smoke test this change
Calvin Leather
10/07/2022, 1:24 PMHmm well appending it to default_deck makes it no longer show up for me either! 😆
I think I'll need to read the inner workings of how the deck uri gets passed around before I can confidently resolve this
k
Ketan (kumare3)
10/07/2022, 7:21 PM
You don’t need to append to default deck. Each deck is just a tab
Ketan (kumare3)
10/07/2022, 7:21 PM
Is this some weird bug
156 Views