Ryan Delgado
09/08/2022, 11:07 PMKetan (kumare3)
09/09/2022, 12:55 AMKevin Su
09/09/2022, 4:54 AMNiels Bantilan
09/09/2022, 2:07 PMkwargs
of the flyte @task
decorator:
• Dataset.reader
• Model.trainer
So you can do flytekit.Resources(cpu="4")
for the dataset reading step and another configuration for the training step.
FYI, the dataset.loader
, parser
, splitter
, are all invoked within the training task to reduce overhead of spinning up pods.