Channels
announcements
ask-the-community
auth
conference-talks
contribute
databricks-integration
datahub-flyte
deployment
ecosystem-unionml
engineeringlabs
events
feature-discussions
flyte-bazel
flyte-build
flyte-console
flyte-deployment
flyte-documentation
flyte-github
flyte-on-gcp
flyte-ui-ux
flytekit
flytekit-java
flytelab
great-content
hacktoberfest-2022
helsing-flyte
in-flyte-conversations
integrations
introductions
jobs
konan-integration
linkedin-flyte
random
ray-integration
ray-on-flyte
release
scipy-2022-sprint
show-and-tell
sig-large-models
torch-elastic
workflow-building-ui-proj
writing-w-sfloris
Title
a
Alexey Kharlamov
03/10/2023, 3:26 PMHi there!
Do we have any options to increase limits?
failed at Node[n1]. BindingResolutionError: Error binding Var [wf].[chunk_list], caused by: failed at Node[n0]. CausedByError: Failed to GetPrevious data from outputDir [<s3://flyte-data/metadata/propeller/flytesnacks-development-a52pb87sdr8blxsbzgpp/n3/data/0/n0/0/outputs.pb>], caused by: path:<s3://flyte-data/metadata/propeller/flytesnacks-development-a52pb87sdr8blxsbzgpp/n3/data/0/n0/0/outputs.pb>: [LIMIT_EXCEEDED] limit exceeded. 5mb > 2mb.k
Ketan (kumare3)
03/10/2023, 4:10 PMYes check out propeller config. But preferably offload large lists etc
a
Alexey Kharlamov
03/10/2023, 4:13 PMIs it working in binary setup with:
inline:
storage:
cache:
max_size_mbs: 512
target_gc_percent: 70
limits:
maxDownloadMBs: 10k
Ketan (kumare3)
03/10/2023, 4:13 PMMaxDownload
That’s the one
a
Alexey Kharlamov
03/10/2023, 4:15 PMCurrently I’m using this setup:
storage:
cache:
max_size_mbs: 5120
target_gc_percent: 70
limits:
maxDownloadMBs: 1000[LIMIT_EXCEEDED] limit exceeded. 5mb > 2mb.I found the reason for the incorrect limits. The config was incorrectly aligned.