Join Slack
Communities
Powered by
<#791 [Flytekit] FlyteFile and FlyteDirectory rand...
# flytekit
a
acoustic-carpenter-78188
11/02/2023, 9:28 PM
#791 [Flytekit] FlyteFile and FlyteDirectory randomness needs to be fixed
Issue created by
wild-endeavor
This is for
@cosmicBboy
, whom I cannot assign. The Flyte IDL has a
type
/
corresponding literal
that's supposed to represent offloaded binary data. These are weird names for users. Instead, flytekit takes a single part blob to mean a file and a multi-part blob to mean a directory. There is a
transformer
that will help translate between a file on the filesystem and Blob type. The contents are uploaded to S3 (or gcs or whatever) when converting to a Literal and they're downloaded to the os FS when converting to a Python file. In both cases, a call to get a random path is used
https://github.com/flyteorg/flytekit/blob/master/flytekit/types/file/file.py#L244
and
https://github.com/flyteorg/flytekit/blob/master/flytekit/types/file/file.py#L261
. This is fine but if a user repeatedly calls one or the other, new paths should not be returned... we should memoize the path in the object the first time each of these is used.
flyteorg/flyte
acoustic-carpenter-78188
11/02/2023, 9:28 PM
#791 [Flytekit] FlyteFile and FlyteDirectory randomness needs to be fixed
Issue reopened by
eapolinario
flyteorg/flyte
Open in Slack
Previous
Next