New RunLLM response generated: >>>*# Question* Also, tried rerunning an old task that used to run succesfully back in April on flyte-binary 1.10 but now is failing on 1.13, so I'm sure it's due to a backend change. I assume I'm just missing a new dependency here. This is the stacktrace when trying to output a pandas dataframe from a task. Which is recommended here, pyarrow or fastparquet?

TypeError: Failed to convert outputs of task 'jpg_workflow.lpr_service.main.lpr_task' at position 0:
  Unable to find a usable engine; tried using: 'pyarrow', 'fastparquet'.
A suitable version of pyarrow or fastparquet is required for parquet support.
Trying to import the above resulted in these errors:
 - Missing optional dependency 'pyarrow'. pyarrow is required for parquet support. Use pip or conda to install pyarrow.
 - Missing optional dependency 'fastparquet'. fastparquet is required for parquet support. Use pip or conda to install fastparquet.

# Answer The error indicates that your task requires either

pyarrow

or

fastparquet

to handle Parquet files, but neither is installed. Both libraries are optional dependencies for Flyte, and you need to install at least one of them. Recommendation: Install

pyarrow

as it is more commonly used and supported. To resolve the issue, you can install

pyarrow

using pip:

pip install pyarrow

Alternatively, if you prefer

fastparquet

, you can install it using:

pip install fastparquet

Would you like more details on how to configure Flyte to use these dependencies? Sources: - Flyte Issue #4479 - Flyte Issue #4418 Slack Link https://flyte-org.slack.com/archives/CP2HDHKE1/p1723027827.197919 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.