Hey, how to use CuDF acceleration for Pandas in Flyte tasks?
👀 1
f
freezing-airport-6809
11/13/2023, 3:46 PM
This is something we were just discussing off adding a new type transformer
freezing-airport-6809
11/13/2023, 3:47 PM
Cc @brash-river-80189 / @broad-monitor-993
b
broad-monitor-993
11/13/2023, 5:24 PM
Hi @rich-artist-45699 you can use these inside a Flyte task today, with the current considerations:
• need to make sure the cluster is properly configured to use GPUs so you can request them with
requests=Resources(gpu="1")
• You’ll have to manually convert the results to a type that Flyte understands (e.g. FlyteFile or pandas StructuredDataset)
Do you mind opening up a feature request for adding support for CuDF type transformer?
[flyte-core] 👇