Howdy I am working to bring Azure AD authentication in stow (here) into flyte. As part of that, I'm adding a

RemoteURLInterface

for Azure (example). It seems the interface is only called when

remoteDataConfig.SignedURL.Enabled == true

, is that right? Also, I noticed that the aws implementation uses the s3 client directly to fetch a signed URL from a created Request object which does not appear to be used (beyond fetching the URL). I found that surprising; does anyone know why

RawStore.CreateSignedURL

isn't used for that purpose? It feels like offloading that responsibility to the store implementation (likely Stow) would work, while avoiding having to re-implement the logic in each concrete RemoteURL type.

hey are you still working on this? I can reach out internally to find out more of the history of the code there.

as for the url that’s returned however, that is used. this is part of the fast-registration/upload of offloaded data types flow used in flytekit.

ok, that makes sense. but the URL returned from s3 client and the URL returned from stow should be the same, right? i think stow has an s3 implementation, which would likely use the s3 client in the same manner as the flyte code

that would be awesome, but i don’t think we have the resources to do/test that.

yeah, i can work on that and try to put together some tests