https://flyte.org logo
#ask-the-community
Title
# ask-the-community
m

Matheus Moreno

03/11/2022, 6:43 PM
Hey, everyone! I need some help understanding what I'm doing wrong. I'm trying out Flyte using the sandbox and flytectl. I'm following the workflow: build the container, serialize tasks, register the tasks. I'm avoiding fast serialization because I'm using a custom Dockerfile that sets the user as non-root, and apparently that breaks the fast registration system (by the way, is this expected? Will this also happen in a remote cluster?). Now, I just created a new task to test Google Cloud integration, called
test_google_cloud
, but the system won't find it: if I try to run it, I'm met with the error
AttributeError: module 'src.flyte.tasks' has no attribute 'test_google_cloud'
. I ran
docker build
again, serialized the tasks again, registered them again, and still nothing. The task is registered, since I can see it on the UI, but the worker can't find it. What is happening?
k

Ketan (kumare3)

03/11/2022, 6:54 PM
Firstly, fast serialization breaks in some case, we would love to know that. That is a bug. Non root users should be able to install the fast serialized tarball. I think when you fast serialize you have to provide the alternate path. As you can see here, default is
/root
you can just do
pyflyte package --in-container-source-path /…
is
src.flyte.tasks
right?
so, important thing to note is that at runtime, from the
working
directory the function should be loadable
m

Matheus Moreno

03/11/2022, 7:13 PM
Just tried using the
-d
option and the error persists. Before, it was just a
Forbidden
error coming from trying to copy the file to
/root
. Now, this appears:
Copy code
flytekit.exceptions.user.FlyteAssertion: Failed to get data from <s3://my-s3-bucket/fast/v3-fastb17df494435c35acc740957bbca00389.tar.gz> to /home/flyte/app (recursive=False).

Original exception: Called process exited with error code: 1.  Stderr dump:

b'fatal error: An error occurred (403) when calling the HeadObject operation: Forbidden\n'
The root of the source code is
/home/flyte/app
And yes,
src.flyte.tasks
is right. It deleted the sandbox and started from scratch, and now the task can be executed. It seems that the problem happens when I create new tasks after the first build/serialization
They are all in this same file, src/flyte/tasks.py
g

George Snelling

03/24/2022, 6:51 PM
Hey @Matheus Moreno did you ever get past this?
m

Matheus Moreno

03/25/2022, 3:19 PM
Hi @George Snelling! I was able to resolve the problem by: • Passing the
chown
argument when copying the directory to the container (
COPY --chown=flyte:flyte src/ /home/flyte/src
on the Dockerfile, for a non-root user
flyte
) • Passing the
--in-container-source-path
when (fast) registering the tasks (
--in-container-source-path /home/flyte
in my case)
👍 1
❤️ 1
27 Views