#5568 [BUG] Raise an error if fast-package.tar.gz is too big
Issue created by
pingsutw
Describe the bug
request failed to upload the tar file if it's too big.
Expected behavior
We should check the file size before uploading and raise a nicer error.
Additional context to reproduce
import random
import string
from flytekit import task, workflow
@task(enable_deck=True)
def t1(a: int) -> int:
return a + 1
@workflow
def wf() -> int:
return t1(a=3)
if
name == '__main__':
def create_large_file(filename, size_in_mb):
# Size of the file in megabytes
size_in_bytes = size_in_mb * 1024 * 1024
chunk = bytes(''.join(random.choices(string.ascii_uppercase + string.digits, k=1024 * 1024)), 'utf-8')
with open(filename, 'wb') as f:
for _ in range(size_in_bytes // len(chunk)):
f.write(chunk)
# Write the remaining bytes if any
remaining_bytes = size_in_bytes % len(chunk)
if remaining_bytes:
f.write(b'0' * remaining_bytes)
create_large_file('/Users/kevin/git/flytekit/flyte-example/test/dummy', 1024 * 5)
Screenshots
No response
Are you sure this issue hasn't been raised already?
• Yes
Have you read the Code of Conduct?
• Yes
flyteorg/flyte