Hi there! How to increase the allowed output size...
# ask-the-community
a
Hi there! How to increase the allowed output size of the node?
Copy code
RuntimeExecutionError: failed during plugin execution, caused by: failed to execute handle for plugin [k8s-array]: output file @[] is too large [255540150] bytes, max allowed [10485760] bytes
v
I ran a search for
10485760
in https://github.dev/flyteorg/flyte, seems like it is the default value for
max-output-size-bytes
for flyteadmin, flytepropeller, and scheduler_config… After searching through discuss.flyte.org for mentions of
max-output-size-bytes
I found this thread where someone mentioned the exact problem you’re having: https://discuss.flyte.org/t/2714230/hi-all-wave-i-have-a-problem-of-sharing-data-between-tasks-i In their case adding
max-output-size-bytes
to the
flyte-propeller-config
configmap in the namespace where you installed flyte, followed by a restart of the flytepropeller pod, let them increase the max output size. If you installed it without helm, just edit the configmap. If you installed with helm. keep reading for how to configure this in helm: In the flyteorg/flyte github repo, there’s the values file for flyte-core, and there’s this section:
Copy code
# -- Core propeller configuration
  core:
    # -- follows the structure specified [here](<https://pkg.go.dev/github.com/flyteorg/flytepropeller/manager/config#Config>).
    manager:
      pod-application: "flytepropeller"
      pod-template-container-name: "flytepropeller"
      pod-template-name: "flytepropeller-template"
    # -- follows the structure specified [here](<https://pkg.go.dev/github.com/flyteorg/flytepropeller/pkg/controller/config>).
    propeller:
      rawoutput-prefix: <s3://my-s3-bucket/>
      metadata-prefix: metadata/propeller
      workers: 4
      max-workflow-retries: 30
      workflow-reeval-duration: 30s
      downstream-eval-duration: 30s
      limit-namespace: "all"
      prof-port: 10254
I checked how it’s read by the helm templates, and it is read as-is:
Copy code
{{- if .Values.flytepropeller.enabled }}
apiVersion: v1
kind: ConfigMap
metadata:
  name: flyte-propeller-config
  namespace: {{ template "flyte.namespace" . }}
  labels: {{ include "flyteadmin.labels" . | nindent 4 }}
data:
...
{{- with .Values.configmap.core }}
  core.yaml: | {{ tpl (toYaml .) $ | nindent 4 }}
{{- end }}
...
So you should be able to just add the helm value
.configmap.core.propeller.max-output-size-bytes
with the desired max output size value and it should work
s
Thanks for the detailed response, @Victor Churikov. Really appreciate it. If this solution proves effective, would it be possible for you to add it to our troubleshooting guide? It could be very helpful for many of our users. https://docs.flyte.org/en/latest/community/troubleshoot.html
100 Views