Frank Shen
06/06/2023, 10:13 PMpyflyte register --project examples --image <http://xxx.dkr.ecr.us-east-1.amazonaws.com/yyy:latest|xxx.dkr.ecr.us-east-1.amazonaws.com/yyy:latest> train/train_wf.py
And when it finally failed with the error:
OverflowError: string longer than 2147483647 bytes
...
flytekit.exceptions.user.FlyteAssertion: Failed to put data from /var/folders/xn/j7gcmr5j12b7jy0nm2kfykhm0000gp/T/tmp58m18z2p/fast53e0b9edd2101668d22c8cd5fe99d0b8.tar.gz to <https://dev-wm-max-flyte-us-east-1.s3.amazonaws.com/examples/development/>....... (recursive=False).
Original exception: string longer than 2147483647 bytes
2147483647 bytes is 2000+ MB.
I did du -h .
and the total size of my project folder is only 380+ KB.
Why does flyte fast register needs to send 2000+ MB data to S3?Kevin Su
06/06/2023, 11:19 PMFrank Shen
06/06/2023, 11:20 PMKevin Su
06/06/2023, 11:23 PMFrank Shen
06/06/2023, 11:23 PMKevin Su
06/06/2023, 11:23 PMFrank Shen
06/06/2023, 11:23 PMKevin Su
06/06/2023, 11:24 PMFrank Shen
06/06/2023, 11:25 PMKevin Su
06/06/2023, 11:25 PMFrank Shen
06/06/2023, 11:26 PMpyflyte register --project examples train/train_wf.py
Kevin Su
06/06/2023, 11:28 PMFrank Shen
06/06/2023, 11:37 PMKevin Su
06/06/2023, 11:42 PMpyflyte register --ignore venv/ wf
Frank Shen
06/06/2023, 11:46 PMFile "/Users/fshen/hbo-code/dai-mle-paid-renewal/train/train_wf.py", line 22, in <module>
from ..sql.load_input_data import load_input_data
ValueError: attempted relative import beyond top-level package
Kevin Su
06/07/2023, 12:45 AM.b.sql.loal_input_data import ...
Frank Shen
06/07/2023, 1:00 AM