Hi I am running a workflow locally on my machine using `pyfl

Question

Hi I am running a workflow locally on my machine using `pyflyte run` or `python m` but I am getting this error ```2023 06 20 17 04 51 354876 ERROR asctime 2023 06 20 17 04 51 354 name flytekit levelname base task py 587 ERROR message Failed to convert outputs of task read dataset at position 0 n Errno 28 Error writing bytes to file Detail errno 28 No space left on device ``` Is there a virtual directory or disk flyte creates when running on a local machine which I can purge The data for training is being pulled from s3

Chandrashekhar Kotekar · Answer

I ve seen this error when I tried `flytectl demo start` command then I purged all the unused docker images containers volumes etc then error disappeared

Nandakumar Raghu · Answer

I am not using any docker image at the moment I mean I am not passing in an image with ` image` I am just running with something like `pyflyte run sample py sample wf`

Kevin Su · Answer

hmm are you using image spec in the workflow

Nandakumar Raghu · Answer

No no image spec in my workflow

Nandakumar Raghu · Answer

Looks like this is happening when creating a StructuredDataset ``` 865 lv = transformer to literal ctx python val python type expected sd = StructuredDataset dataframe=python val metadata=meta 600 return self encode ctx sd python type protocol fmt sdt 627 sd model = handler encode ctx sd structured literal type 53 df to parquet 54 path 55 coerce timestamps= us 56 allow truncated timestamps=False 2976 return to parquet 2977 self 2978 path 2979 engine 430 impl write 431 df 432 path or buf 433 compression=compression 204 self api parquet write table 205 table path or handle compression=compression kwargs 206 2985 writer write table table row group size=row group size 1054 self writer write table table row group size=row group size ```

Nandakumar Raghu · Answer

I think the StructuredDataset was being created multiple times i e the writing to parquet files So I moved that to a separate task and added a `cache=True` But now I am getting this error ` FlyteScopedUserException database or disk is full` What local database does flyte use and is there a way to flush it I am running on my mac and I don t think my disk is full slightly smiling face

Kevin Su · Answer

are you able to share your workflow code

Kevin Su · Answer

we didn t run any database when you run workflow locally