acoustic-carpenter-78188
08/22/2023, 9:55 PMflytectl demo sandbox cluster.
The flytectl demo sandbox cluster is a great way onboard new Flyte users and debug workflows in a light-weight Flyte cluster. It also has a docker registry that you can push images to so that you can build and push images in a completely local environment. For example, the following code will create a flytekit dev image, assuming you're in the top-level directory of the flytekit repo:
docker build . -f Dockerfile.dev -t localhost:30000/flytekit:dev --build-arg PYTHON_VERSION=3.9
docker push localhost:30000/flytekit:dev
However, there are reports that, under certain conditions, the sandbox cluster will fail with OOM errors, or the cluster crashes. This latter case is notable when trying to perform integration tests with a sandbox cluster on a github action runner (see this unionml GH action run, where the flyte cluster died when trying to push a docker image).
Goal: What should the final outcome look like, ideally?
• Running workflows on a local sandbox should "just work" within some reasonably small images/workload sizes.
• We should be able to build/push the flytekit base image in a github action runner and run workflows on them so we can do basic integration tests with the minimal flyte image.
Describe alternatives you've considered
• Use custom action runners for doing integration tests
Propose: Link/Inline OR Additional context
No response
Are you sure this issue hasn't been raised already?
☑︎ Yes
Have you read the Code of Conduct?
☑︎ Yes
flyteorg/flyte