Hello, It happens to me from time to time that the command pyflyte register …. either takes a long time to complete.

pyflyte register --project examples --image <http://xxx.dkr.ecr.us-east-1.amazonaws.com/yyy:latest|xxx.dkr.ecr.us-east-1.amazonaws.com/yyy:latest> train/train_wf.py

And when it finally failed with the error:

OverflowError: string longer than 2147483647 bytes
...
flytekit.exceptions.user.FlyteAssertion: Failed to put data from /var/folders/xn/j7gcmr5j12b7jy0nm2kfykhm0000gp/T/tmp58m18z2p/fast53e0b9edd2101668d22c8cd5fe99d0b8.tar.gz to <https://dev-wm-max-flyte-us-east-1.s3.amazonaws.com/examples/development/>....... (recursive=False).

Original exception: string longer than 2147483647 bytes

2147483647 bytes is 2000+ MB. I did du -h . and the total size of my project folder is only 380+ KB. Why does flyte fast register needs to send 2000+ MB data to S3?

because flytekit will serialize workflow to workflow.proto, and upload it to s3.

just one

hmm, pyflyte register will always package all the files from the parent folder.

I executed pyflyte at the root folder of my workflow project.

which flytekit version did you use before?

1.2.4

so there is a regression, which flytekit version are you using now?

I have always used 1.2.4, before and now.

did you upgrade flyteadmin server?

Say a/b/train is my folder structure. /b is my workflow project root. In /b, I executed command:

pyflyte register --project examples train/train_wf.py

if b has _`__init__`_ file, it will package /a

I see. It does and I have removed the _`__init__`_ file. And that fixed the issue.

for now it will package venv folder. you could add a VenvIgnore here

Yes, that will work.

you have to update the relative import to

.b.sql.loal_input_data import ...

@glamorous-carpet-83516, for that to work, I need to have a init py file in /b. I am in a catch 22 situation.