Hi all, new here. I didn't see a beginner's channel, so I'll ask what seems like a silly question he...
g

Guy Arad

almost 2 years ago
Hi all, new here. I didn't see a beginner's channel, so I'll ask what seems like a silly question here. I followed the instructions described here. I used Python 3.10 on MacBook Pro with M2 Pro chipset. I'm getting the following error:
Failed with Unknown Exception <class 'TypeError'> Reason: Encountered error while executing workflow 'example.training_workflow':
Error encountered while executing 'training_workflow':
Failed to convert outputs of task 'example.get_data' at position 0:
[Errno 2] Failed to open local file '/var/folders/vs/v9fd0vyx6b735n5_l607sq440000gn/T/flyte-nl8jmq53/raw/2bb31bdcb6e0b0733d1bad9b87a3e886/00000'. Detail: [errno 2] No such file or directory
So I did a little digging. From the exception stacktrace it seems like the error happens when attempting to write the output. While debugging I found that indeed this folder isn't created. The temp folder
/var/folders/vs/v9fd0vyx6b735n5_l607sq440000gn/T/flyte-nl8jmq53/
does exist, but the
raw
sub folder isn't. I kept debugging until I noticed several things (in reversed order 🙂). The method
flytekit.core.data_persistence.FileAccessProvider.get_random_remote_path
- the default remote is expectedly
LocalFileSystem
.
self._default_protocol
is
"file"
but in that method, this following line returns `('file', 'local')`:
default_protocol = self._default_remote.protocol
This causes
get_random_remote_path
to eventually return a path that starts with
file:///...
. Right after getting the random path, in
flytekit.types.structured.basic_dfs.PandasToParquetEncodingHandler.encode
these lines happen:
if not ctx.file_access.is_remote(uri):
    Path(uri).mkdir(parents=True, exist_ok=True)
is_remote
returns
True
but apparently
Path.mkdir
when operating on
file://...
path, does NOTHING. So the necessary folder hierarchy does not exist and the code fails. Potential root causes: 1.
fsspec
for some reason, returns
('file', 'local')
instead of
file
. 2. Another option, that
Flyte
code in
flytekit.core.data_persistence.FileAccessProvider.get_random_remote_path
if type(default_protocol) == list:
            default_protocol = default_protocol[0]
was supposed to test
tuple
condition as well - this would have probably solved the issue 3. Another possiblity is my Mac... maybe
Path
behaves weirdly on Mac