Hey folks! Potentially silly question: Suppose I have an extremely capable headless desktop with a bunch of GPUs and memory. What would be the best way to execute a flyte pipeline while developing from a remote laptop? In other words, does flyte include a native way to set up my headless desktop to receive a workflow, or should I focus on remote execution via SSH?

There has to be connectivity

Thank Ketan! Yes, Sandbox caught my eye but I wasn’t sure if that was the correct approach.

It stores state on a local disk so decent

No worries! I’ll be sure to grab some extra SSDs. While working with other workflow orchestrators in the past, I found it useful to set up an automated job that deletes files older than the cache staleness requirement.

Thank you @Kevin Su , this is extremely insightful. Yes, I would want to be able to have select tasks utilize my GPUs for model training within Flyte Sandbox. It seems that this is not yet possible, but is currently a WIP?

@L godlike / @Daniel Farrell were working on it, it supposedly works, but this is purely a community effort

it is possible now

This is awesome, thank you! I might reach out with some questions later.

and please remeber to rebase with latest branch