https://flyte.org
Join Slack
our flyte-sandbox container often crashes when using multiple heavy containers. The hardware for the...
# flyte-support
c
our flyte-sandbox container often crashes when using multiple heavy containers. The hardware for the sandbox is a fairly powerful node with enough RAM and CPU resources. Any pointers on debuging this will be really helpful!
👍 1
f
Did you change the docker daemon to have more cpu / mem
Cc @freezing-boots-56761
a
@creamy-knife-86303 currently the minimum requirements for the Docker daemon are 4CPUs and 3GB RAM https://docs.flyte.org/projects/cookbook/en/latest/index.html#running-workflows-in-a-flyte-cluster Please let us know if this answers your question
f
the docker daemon resources is a mac thing because of the virtualization i believe. is this running on linux?
ah this might be because of the resources:
Copy code
resources:
      limits:
        memory: 1Gi
      requests:
        cpu: "1"
        memory: 1Gi
@creamy-knife-86303 this will need a fix in the chart and sandbox configuration. i think we should just drop the resource limits entirely. in the meantime, you can try manually editing the deployment to see if that helps you.
Copy code
kubectl edit deploy flyte-sandbox -n flyte
c
I am running on a Linux server with 88 cores and ~800 GB RAM. I have not yet played around with resource limits. @freezing-boots-56761 thanks for your help. I will try this out and let you know if it helps!
f
@thankful-minister-83577: cc
we can push out a fix for this in the next release
🙏 1
@creamy-knife-86303: 1.4.0 release should fix this.
154 Views
Open in Slack
PreviousNext
Powered by

Flyte

Flyte enables production-grade orchestration for machine learning workflows and data processing created to accelerate local workflows to production.

Powered by