@ancient-wolf-19325 map tasks vs dynamic workflow

Map tasks in Flyte are designed for running the same task over a list of inputs in parallel, within a single workflow node, providing performance gains and efficient metadata storage. They are best for simple, homogeneous operations like batch processing or hyperparameter sweeps, and support features like concurrency limits and minimum success ratios. In contrast, dynamic workflows allow you to programmatically construct arbitrary workflow DAGs at runtime, enabling more complex, flexible, and nested execution patterns, such as recursion or variable numbers/types of tasks and subworkflows, but with more overhead and less efficient metadata handling for large fan-outs. Map tasks are more efficient for large-scale, uniform parallelism, while dynamic workflows offer greater flexibility for complex, runtime-dependent logic and workflow composition. For example, use map tasks when you need to apply a single task to many inputs efficiently, and use dynamic workflows when the workflow structure itself depends on runtime data or you need to compose tasks and subworkflows dynamically. For more details and code examples, see Flyte map tasks documentation and dynamic workflows documentation. Would you like a step-by-step breakdown or code samples? Sources: - Dynamic workflows documentation - Flyte map tasks documentation 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.

@ancient-wolf-19325 how to trigger this wf from cli?

def *train_by_key_list_wf*(_model_ids_: list[str], _train_: str, _test_: str):

You can trigger the

train_by_key_list_wf

workflow from the CLI using

pyflyte run

, passing the required arguments. For a list input, provide it as a JSON array string. For example:

pyflyte run your_script.py train_by_key_list_wf --model-ids '["id1","id2"]' --train train.csv --test test.csv

Replace

your_script.py

with the actual filename. For more, see Flyte workflow CLI usage and input argument formats. Would you like more detail or a step-by-step example? Sources: - Flyte workflow CLI usage - Input argument formats 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.