https://flyte.org
Join Slack
Hi guys, I'm trying to execute a workflow on a remote Flyte cluster (Sandbox) using "remote.execute(...
# flyte-support
c
Hi guys, I'm trying to execute a workflow on a remote Flyte cluster (Sandbox) using "remote.execute()" function. I have also registered the workflows & tasks via functions remote.register_task() & remote.register_workflow(). The Workflow & Tasks are visible on the Flyte Console. The tasks are defined in a python module called tasks.py. The Workflow however, is generated dynamically using ImperativeWorkflow. However, upon running "remote.execute()", I get an error in the first task itself saying "ModuleNoFoundError: No module named 'tasks'", and the Workflow fails to execute. I'm wondering if this is due to constraints around directory structure or something else entirely? Any help would be appreciated.
g
how do you register the workflows
could you show me the output of 
tree
in your project root
c
I register them using FlyteRemote.register_worflow() function
g
could you share the code snippet or minimum repro
c
Directory structure: | flyte-tutorials | -- my_project | ---- init.py | ---- tasks.py | ---- workflow_config.yaml | ---- build_imperative_workflow.py | ---- register_imperative_workflow.py | ---- execute_imperative_workflow_remote.py
g
ah, did you built a image by yourself? default image wouldn’t have your code
c
No, I didn't build an image myself.. I've used ImageConfig.from_images("ghcr.io/flyteorg/flytekit:py3.11-1-10.2") while registering the tasks & workflow.
g
you have to build an image and copy the code to that image by yourself.
c
Ahh, thank you.. any docs on how to build an image ourselves? I'm using the sandbox as of now, and working with Windows OS
g
cc @high-accountant-32689 We discussed this earlier today.
@creamy-ice-33999 we’re going to support fast-register in flytekit remote
👍 1
c
Thank you Kevin! Will try this out
Follow up question regarding pushing Docker image.. We have a private container registry that needs authentication before we're able to pull our docker image from there.. That becomes a problem since Flyte Sandbox is a docker-in-docker container, and I don't see how to authenticate to our registry. I also tried pushing my image to localhost:3000, but I got an error there. What's the simplest way to test custom images in Flyte Sandbox?
g
push to local registry
what error did you see
c
Getting this error:
Copy code
docker push localhost:30000/my_image:latest

The push refers to repository [localhost:30000/my_image]
2ae9700cb251: Waiting
failed to do request: Head "<https://localhost:30000/v2/my_image/blobs/sha256:ddbe9722373d36aa251707bb0ed42ea0216e44e75024b808581534a4b5325de1>": dial tcp [::1]:30000: connect: connection refused
The "envd context create" command won't work here since I'm using a Windows machine..
I'm now trying this out on WSL2 Ubuntu.. will check if it works over there
I was using "flyte sandbox start". Was advised by Samhita to use "flyte demo start". Pushing to local docker registry works now! Also, running register_script() works for my use case. Thank you!
g
awesome, good to hear that. we’re going to remove sandbox command, it’s deprecated for a while
👍 1
j
@glamorous-carpet-83516 are you saying that the code under the "my_project" directory needs to copied to the image? Is there a complete example that shows how to use "remote.register_task" or "remote.register_workflow" to upload and run local tasks remotely? If not, could you provide one?
23 Views
Open in Slack
PreviousNext
Powered by

Flyte

Flyte enables production-grade orchestration for machine learning workflows and data processing created to accelerate local workflows to production.

Powered by