https://flyte.org logo
#ask-the-community
Title
# ask-the-community
c

Chris Hairfield

10/18/2022, 7:21 PM
Hello, I’m using
fast register
to develop in the
sandbox
environment, and I’d like to configure it to use a storage backend other than
awscli
. Is there any documentation for doing this? We run exclusively in GCP and install
gsutil
in each of our Flyte-ready Docker images. I’m hoping to avoid installing
awscli
in our images just to support running in
sandbox
. Thanks!
also if you user fast register, better to switch to the new system - used by
pyflyte run
e

Eduardo Apolinario (eapolinario)

10/18/2022, 7:29 PM
also, since you mentioned the sandbox, keep in mind in that environment that we use minio as the blob storage (i.e. it's not really writing to s3, but a drop-in replacement for that).
c

Chris Hairfield

10/18/2022, 8:51 PM
gotcha, is there any way to use the existing minio in sandbox without
awscli
, say via the HTTP data persistence plugin included in flytekit?
as for
pyflyte run
, I was under the impression that I couldn’t use that to test launch plans with a schedule
e

Eduardo Apolinario (eapolinario)

10/18/2022, 9:42 PM
@Chris Hairfield, let's take a step back here. Essentially you don't want to maintain two sets of images, one for the sandbox deployment and one for the production deployment (which is gcp-based). This would not be an issue if we could have a gcp-enabled sandbox, which we currently do not have. The sandbox was supposed to be used as a quick iteration/experimentation tool, not as part of a larger deployment. That said, it is interesting that you're considering that. As for the question about launchplans and
pyflyte run
, you're correct, we do not support launchplans in
pyflyte run
yet. I opened https://github.com/flyteorg/flyte/issues/2998 to track the work to add support for it.
c

Chris Hairfield

10/18/2022, 9:50 PM
Thanks for opening https://github.com/flyteorg/flyte/issues/2998! That would be very helpful. Yes, you captured my intent exactly. I’d like to avoid either sandbox-specific images or bloating our production images. Sandbox is important to my development workflow as it supports the native scheduler. It’s much faster than testing on GCP. What do you think about supporting this use case?
e

Eduardo Apolinario (eapolinario)

10/18/2022, 11:05 PM
@Chris Hairfield, let me discuss this with the team. I created https://github.com/flyteorg/flyte/issues/3000 to track this issue for now. How about you link to this issue and install both binaries to unblock you?
c

Chris Hairfield

10/19/2022, 2:09 PM
I’ll do that for the time being. Thanks for opening the ticket.
24 Views