Hi guys, I'm trying to execute a workflow on a rem...
# ask-the-community
Hi guys, I'm trying to execute a workflow on a remote Flyte cluster (Sandbox) using "remote.execute()" function. I have also registered the workflows & tasks via functions remote.register_task() & remote.register_workflow(). The Workflow & Tasks are visible on the Flyte Console. The tasks are defined in a python module called tasks.py. The Workflow however, is generated dynamically using ImperativeWorkflow. However, upon running "remote.execute()", I get an error in the first task itself saying "ModuleNoFoundError: No module named 'tasks'", and the Workflow fails to execute. I'm wondering if this is due to constraints around directory structure or something else entirely? Any help would be appreciated.
how do you register the workflows
could you show me the output of
in your project root
I register them using FlyteRemote.register_worflow() function
could you share the code snippet or minimum repro
Directory structure: | flyte-tutorials | -- my_project | ---- init.py | ---- tasks.py | ---- workflow_config.yaml | ---- build_imperative_workflow.py | ---- register_imperative_workflow.py | ---- execute_imperative_workflow_remote.py
ah, did you built a image by yourself? default image wouldn’t have your code
No, I didn't build an image myself.. I've used ImageConfig.from_images("ghcr.io/flyteorg/flytekit:py3.11-1-10.2") while registering the tasks & workflow.
you have to build an image and copy the code to that image by yourself.
Ahh, thank you.. any docs on how to build an image ourselves? I'm using the sandbox as of now, and working with Windows OS
cc @Eduardo Apolinario (eapolinario) We discussed this earlier today.
@Tushar Gaurav we’re going to support fast-register in flytekit remote
Thank you Kevin! Will try this out
Follow up question regarding pushing Docker image.. We have a private container registry that needs authentication before we're able to pull our docker image from there.. That becomes a problem since Flyte Sandbox is a docker-in-docker container, and I don't see how to authenticate to our registry. I also tried pushing my image to localhost:3000, but I got an error there. What's the simplest way to test custom images in Flyte Sandbox?
push to local registry
what error did you see
Getting this error:
Copy code
docker push localhost:30000/my_image:latest

The push refers to repository [localhost:30000/my_image]
2ae9700cb251: Waiting
failed to do request: Head "<https://localhost:30000/v2/my_image/blobs/sha256:ddbe9722373d36aa251707bb0ed42ea0216e44e75024b808581534a4b5325de1>": dial tcp [::1]:30000: connect: connection refused
The "envd context create" command won't work here since I'm using a Windows machine..
I'm now trying this out on WSL2 Ubuntu.. will check if it works over there
I was using "flyte sandbox start". Was advised by Samhita to use "flyte demo start". Pushing to local docker registry works now! Also, running register_script() works for my use case. Thank you!
awesome, good to hear that. we’re going to remove sandbox command, it’s deprecated for a while
@Kevin Su are you saying that the code under the "my_project" directory needs to copied to the image? Is there a complete example that shows how to use "remote.register_task" or "remote.register_workflow" to upload and run local tasks remotely? If not, could you provide one?