astonishing-ocean-79318
09/08/2022, 11:07 PMfreezing-airport-6809
freezing-airport-6809
freezing-airport-6809
freezing-airport-6809
glamorous-carpet-83516
09/09/2022, 4:54 AMbroad-monitor-993
09/09/2022, 2:07 PMkwargs
of the flyte @task
decorator:
• Dataset.reader
• Model.trainer
So you can do flytekit.Resources(cpu="4")
for the dataset reading step and another configuration for the training step.
FYI, the dataset.loader
, parser
, splitter
, are all invoked within the training task to reduce overhead of spinning up pods.