astonishing-ocean-79318
09/08/2022, 11:07 PMfreezing-airport-6809
freezing-airport-6809
freezing-airport-6809
freezing-airport-6809
glamorous-carpet-83516
09/09/2022, 4:54 AMbroad-monitor-993
09/09/2022, 2:07 PMkwargs of the flyte @task decorator:
• Dataset.reader
• Model.trainer
So you can do flytekit.Resources(cpu="4") for the dataset reading step and another configuration for the training step.
FYI, the dataset.loader, parser, splitter, are all invoked within the training task to reduce overhead of spinning up pods.