Flyte enables production-grade orchestration for machine learning workflows and data processing created to accelerate local workflows to production.

Flyte

Hello,

I’m using `fast register` to develop in the `sandbox` environment, and I’d like to configure it to use a storage backend other than `awscli`. Is there any documentation for doing this?

We run exclusively in GCP and install `gsutil` in each of our Flyte-ready Docker images. I’m hoping to avoid installing `awscli` in our images just to support running in `sandbox`.

Thanks!

<https://docs.flyte.org/en/latest/concepts/data_management.html#data-movement>

also if you user fast register, better to switch to the new system - used by `pyflyte run`

also, since you mentioned the sandbox, keep in mind in that environment that we use <https://min.io/|minio> as the blob storage (i.e. it's not really writing to s3, but a drop-in replacement for that).

gotcha, is there any way to use the existing minio in sandbox without `awscli`, say via the HTTP data persistence plugin included in flytekit?

as for `pyflyte run`, I was under the impression that I couldn’t use that to test launch plans with a schedule

<@U0464MSVA80>, let's take a step back here. Essentially you don't want to maintain two sets of images, one for the sandbox deployment and one for the production deployment (which is gcp-based). This would not be an issue if we could have a gcp-enabled sandbox, which we currently do not have. The sandbox was supposed to be used as  a quick iteration/experimentation tool, not as part of a larger deployment. That said, it is interesting that you're considering that.

As for the question about launchplans and `pyflyte run`, you're correct, we do not support launchplans in `pyflyte run` yet. I opened <https://github.com/flyteorg/flyte/issues/2998> to track the work to add support for it.

Thanks for opening <https://github.com/flyteorg/flyte/issues/2998>! That would be very helpful.

Yes, you captured my intent exactly. I’d like to avoid either sandbox-specific images or bloating our production images.

Sandbox is important to my development workflow as it supports the native scheduler. It’s much faster than testing on GCP.

What do you think about supporting this use case?

<@U0464MSVA80>, let me discuss this with the team. I created <https://github.com/flyteorg/flyte/issues/3000> to track this issue for now. How about you link to this issue and install both binaries to unblock you?

I’ll do that for the time being. Thanks for opening the ticket.