https://flyte.org logo
#ask-the-community
Title
# ask-the-community
c

Chris Grass

10/25/2023, 4:08 PM
Howdy I am working to bring Azure AD authentication in stow (here) into flyte. As part of that, I'm adding a
RemoteURLInterface
for Azure (example). It seems the interface is only called when
remoteDataConfig.SignedURL.Enabled == true
, is that right? Also, I noticed that the aws implementation uses the s3 client directly to fetch a signed URL from a created Request object which does not appear to be used (beyond fetching the URL). I found that surprising; does anyone know why
RawStore.CreateSignedURL
isn't used for that purpose? It feels like offloading that responsibility to the store implementation (likely Stow) would work, while avoiding having to re-implement the logic in each concrete RemoteURL type.
y

Yee

10/28/2023, 1:49 AM
hey are you still working on this? I can reach out internally to find out more of the history of the code there.
you are correct that this appears to use the s3 client directly and not go through stow. can’t speak to why that is.
as for the url that’s returned however, that is used. this is part of the fast-registration/upload of offloaded data types flow used in flytekit.
this is how your updated code gets sent to blob store.
c

Chris Grass

10/30/2023, 2:40 PM
as for the url that’s returned however, that is used. this is part of the fast-registration/upload of offloaded data types flow used in flytekit.
ok, that makes sense. but the URL returned from s3 client and the URL returned from stow should be the same, right? i think stow has an s3 implementation, which would likely use the s3 client in the same manner as the flyte code
as for the url that’s returned however, that is used. this is part of the fast-registration/upload of offloaded data types flow used in flytekit.
right, the URL is used to fetch data from the store. but only when
remoteDataConfig.SignedURL.Enabled == true
, correct?
@Yee - what are your thoughts on me adding presigned url support for the azure use case? is that a feature that the community wants to continue to support, or is relying on a workload identity reasonable for the azure integration?
y

Yee

11/30/2023, 2:02 AM
that would be awesome, but i don’t think we have the resources to do/test that.
is that something you could help with?
c

Chris Grass

11/30/2023, 2:55 PM
yeah, i can work on that and try to put together some tests