James Evers
09/14/2022, 6:38 PM/dev/shm
. what's the best approach to bumping this default in the sandbox deployment?
/usr/local/lib/python3.8/dist-packages/paramiko/transport.py:236: CryptographyDeprecationWarning: Blowfish has been deprecated
"class": algorithms.Blowfish,
2022-09-14 18:10:43,190 WARNING services.py:1882 -- WARNING: The object store is using /tmp instead of /dev/shm because /dev/shm has only 67108864 bytes available. This will harm performance! You may be able to free up space by deleting files in /dev/shm. If you are inside a Docker container, you can increase /dev/shm size by passing '--shm-size=0.10gb' to 'docker run' (or add it to the run_options list in a Ray cluster config). Make sure to set this to more than 30% of available RAM.
2022-09-14 18:10:46,611 INFO worker.py:1509 -- Started a local Ray instance. View the dashboard at [1m[32m<http://127.0.0.1:8265> [39m[22m
[2022-09-14 18:10:57,864 E 1 1] <http://core_worker.cc:149|core_worker.cc:149>: Failed to register worker 01000000ffffffffffffffffffffffffffffffffffffffffffffffff to Raylet. IOError: [RayletClient] Unable to register worker with raylet. No such file or directory
Ketan (kumare3)
James Evers
09/14/2022, 8:08 PMflytectl
?Ketan (kumare3)
James Evers
09/14/2022, 8:13 PMError: rpc error: code = InvalidArgument desc = Requested MEMORY default [3500Mi] is greater than current limit set in the platform configuration [1Gi]. Please contact Flyte Admins to change these limits or consult the configuration
flyteadmin
, is this doable in the sandbox?config-sandbox.yaml
, but i'm not seeing a full list of parameters for the config anywhereEduardo Apolinario (eapolinario)
09/14/2022, 8:35 PM$ kubectl -n flyte edit cm flyte-admin-base-config
...
task_resource_defaults.yaml: |
task_resources:
defaults:
cpu: 100m
memory: 500Mi
storage: 500Mi
limits:
cpu: 2
gpu: 1
memory: 1Gi <- change this value
storage: 20Mi
James Evers
09/14/2022, 8:35 PMkubectl
rather than flytectl
?Eduardo Apolinario (eapolinario)
09/14/2022, 8:41 PMflytectl
is more about interacting with a flyte deployment, more specifically with its api (via flyteadmin)James Evers
09/14/2022, 8:42 PMEduardo Apolinario (eapolinario)
09/14/2022, 8:53 PMJames Evers
09/14/2022, 9:03 PMEduardo Apolinario (eapolinario)
09/14/2022, 9:06 PMKetan (kumare3)
James Evers
09/14/2022, 9:34 PM