https://flyte.org logo
#announcements
Title
# announcements
d

David Espejo (he/him)

02/20/2023, 3:56 PM
📣 📣 📣 Hi Flyte community! Join us tomorrow for the bi-weekly Community meeting, including: • 📖 New community-contributed content🎯 New Governance model proposedkubernetes Lighting Talk: K8s Ingress: why?, what? and how?📄 New Getting Started guide💡*Guest presentation by @Sören Brunk on how USU AI Services uses Flyte for production and experimental setups* 🕐 Tuesday Feb 21 at 9am PT 🚨 According to community feedback, we're going back to Zoom: join here 📆 Subscribe to the calendar (import it to your calendar app): Flyte calendar ICS file
g

Greg Gydush

02/22/2023, 5:43 AM
Was this recorded by chance?
d

David Espejo (he/him)

02/22/2023, 3:11 PM
@Greg Gydush it was! we'll let you all know once recordings are up in YouTube. Anything particular in the agenda caught your attention?
g

Greg Gydush

02/22/2023, 3:45 PM
Thank you! I heard that debugpy was mentioned for debugging Flyte workflows and it's also fun to hear other updates!
d

David Espejo (he/him)

02/22/2023, 3:47 PM
Right! It's mentioned in a recent blog post by @Fabio Grätz : https://medium.com/@fabiograetz/attach-a-visual-debugger-to-ml-training-jobs-on-kubernetes-eb9678389f1f During the meeting, Fabio shared more details. Hope you can catch the recording!
g

Greg Gydush

02/22/2023, 3:48 PM
yep! I tried it myself after reading the blog and it worked great!
f

Fabio Grätz

02/22/2023, 3:48 PM
Happy to hear 🙂
g

Greg Gydush

02/22/2023, 3:49 PM
Thanks for sharing that!
d

Dan Corbiani

02/22/2023, 5:06 PM
This is pretty slick
g

Greg Gydush

02/22/2023, 5:07 PM
@Fabio Grätz did you get this working well with pod selector? Going by pod IP is less convenient, so I’ve opted to just port forward with kubectl for now, and use local host IP, if that makes sense
Trying to get a solid user experience, as this is mostly for ML scientists/bioinformaticians debugging remote workflow executions
f

Fabio Grätz

02/22/2023, 5:08 PM
I do
kubectl port-forward pod/<pod name>
Works well
g

Greg Gydush

02/22/2023, 5:09 PM
Yep, that’s my move as well
d

David Espejo (he/him)

02/23/2023, 3:21 PM
Highlights of this week's meeting: • Community updates ◦ Logistics update: agenda doc is now open including attendance list and voluntary note taker for each session ◦ Blog posts authored by Dr Fabio Graetz and Bernhard Stadlbauer • Soeren Brunk presentation and further discussion 📖 Read the notes 📽️

Community Updates

📽️ Presentation:

"The right GPU in Flyte for you"

➡️ Subscribe to the calendar We hope to see you in the next one!
g

Greg Gydush

02/26/2023, 7:02 PM
@Fabio Grätz @Ketan (kumare3) Could we possibly do this without any modifications to the workflow (notably the debugpy statement, which requires a re-register) by adding this as a task/workflow override? I’m imagining something like, before running pyflyte run from within the container, we would run
Copy code
python -m debugpy --listen 0.0.0.0:5678" <task/workflow>
Then we could add a debug boolean field to the UI, and the process would look like, run workflow on remote, place breakpoint on local Vscode, run debugger (which connects to remote process)
This shows that you can set breakpoints in the gutter (no need for explicit breakpoint() call) - so as long as we can listen, we can attach!
k

Ketan (kumare3)

02/26/2023, 8:08 PM
But you need to pause the process
Granted, pyflyte execute can pause when enabled
g

Greg Gydush

02/26/2023, 8:11 PM
What do you mean by that? The breakpoint in the gutter pauses the process after connecting
so if you had this on the task, and the first line was a breakpoint, it would pause at the breakpoint (provided you’ve connected the debugpy debugger)
k

Ketan (kumare3)

02/26/2023, 8:12 PM
But after connecting
g

Greg Gydush

02/26/2023, 8:12 PM
yep
k

Ketan (kumare3)

02/26/2023, 8:12 PM
But the task has to start a debug server and wait for connecting right
g

Greg Gydush

02/26/2023, 8:13 PM
It would
Both are potentially possible?
7 Views