Flyte enables production-grade orchestration for machine learning workflows and data processing created to accelerate local workflows to production.

Flyte

Hey, does anyone have a code for *HuggingFace/Transformers-based workflows using Flyte* and can share it so that I do not reinvent the wheel?
Specifically I am interesting in best practices for e.g. data handling (reading data using `datasets` library, tokenization and returning encoded data from a task etc).

I'm interested in that too :slightly_smiling_face:

<https://github.com/unionai-oss/llm-fine-tuning|https://github.com/unionai-oss/llm-fine-tuning>

<@U0679CLE5UY> already using that :sunglasses:

Awesome! I’m working on folding some of the code in this repo into flytekit proper / flyte docs