https://flyte.org logo
#ask-the-community
Title
# ask-the-community
t

Tushar Gaurav

02/07/2024, 5:20 AM
Hi guys, I'm trying to execute a workflow on a remote Flyte cluster (Sandbox) using "remote.execute()" function. I have also registered the workflows & tasks via functions remote.register_task() & remote.register_workflow(). The Workflow & Tasks are visible on the Flyte Console. The tasks are defined in a python module called tasks.py. The Workflow however, is generated dynamically using ImperativeWorkflow. However, upon running "remote.execute()", I get an error in the first task itself saying "ModuleNoFoundError: No module named 'tasks'", and the Workflow fails to execute. I'm wondering if this is due to constraints around directory structure or something else entirely? Any help would be appreciated.
k

Kevin Su

02/07/2024, 5:32 AM
how do you register the workflows
could you show me the output of
tree
in your project root
t

Tushar Gaurav

02/07/2024, 5:33 AM
I register them using FlyteRemote.register_worflow() function
k

Kevin Su

02/07/2024, 5:35 AM
could you share the code snippet or minimum repro
t

Tushar Gaurav

02/07/2024, 5:38 AM
Directory structure: | flyte-tutorials | -- my_project | ---- init.py | ---- tasks.py | ---- workflow_config.yaml | ---- build_imperative_workflow.py | ---- register_imperative_workflow.py | ---- execute_imperative_workflow_remote.py
k

Kevin Su

02/07/2024, 5:40 AM
ah, did you built a image by yourself? default image wouldn’t have your code
t

Tushar Gaurav

02/07/2024, 5:42 AM
No, I didn't build an image myself.. I've used ImageConfig.from_images("ghcr.io/flyteorg/flytekit:py3.11-1-10.2") while registering the tasks & workflow.
k

Kevin Su

02/07/2024, 5:44 AM
you have to build an image and copy the code to that image by yourself.
t

Tushar Gaurav

02/07/2024, 5:46 AM
Ahh, thank you.. any docs on how to build an image ourselves? I'm using the sandbox as of now, and working with Windows OS
cc @Eduardo Apolinario (eapolinario) We discussed this earlier today.
@Tushar Gaurav we’re going to support fast-register in flytekit remote
t

Tushar Gaurav

02/07/2024, 6:02 AM
Thank you Kevin! Will try this out
Follow up question regarding pushing Docker image.. We have a private container registry that needs authentication before we're able to pull our docker image from there.. That becomes a problem since Flyte Sandbox is a docker-in-docker container, and I don't see how to authenticate to our registry. I also tried pushing my image to localhost:3000, but I got an error there. What's the simplest way to test custom images in Flyte Sandbox?
k

Kevin Su

02/07/2024, 6:53 PM
push to local registry
what error did you see
t

Tushar Gaurav

02/08/2024, 4:07 AM
Getting this error:
Copy code
docker push localhost:30000/my_image:latest

The push refers to repository [localhost:30000/my_image]
2ae9700cb251: Waiting
failed to do request: Head "<https://localhost:30000/v2/my_image/blobs/sha256:ddbe9722373d36aa251707bb0ed42ea0216e44e75024b808581534a4b5325de1>": dial tcp [::1]:30000: connect: connection refused
The "envd context create" command won't work here since I'm using a Windows machine..
I'm now trying this out on WSL2 Ubuntu.. will check if it works over there
I was using "flyte sandbox start". Was advised by Samhita to use "flyte demo start". Pushing to local docker registry works now! Also, running register_script() works for my use case. Thank you!
k

Kevin Su

02/08/2024, 10:41 AM
awesome, good to hear that. we’re going to remove sandbox command, it’s deprecated for a while
j

Jonatan Eriksson

02/09/2024, 2:26 PM
@Kevin Su are you saying that the code under the "my_project" directory needs to copied to the image? Is there a complete example that shows how to use "remote.register_task" or "remote.register_workflow" to upload and run local tasks remotely? If not, could you provide one?