@ancient-wolf-19325 I have a series of tasks i would like to run in parallel, say task A -> B -> C in sequence, but I could have one to many of those threads, how would you write that into flyte

To run multiple independent sequences (A → B → C) in parallel in Flyte 2, define each step as an async task and use an async workflow to launch each sequence as a coroutine. Use

asyncio.gather

to execute all sequences in parallel. Each sequence can be parameterized for different inputs. Example:

import asyncio
import flyte

env = flyte.TaskEnvironment("parallel_sequences")

@env.task
async def task_a(x: int) -> int:
    return x + 1

@env.task
async def task_b(y: int) -> int:
    return y * 2

@env.task
async def task_c(z: int) -> int:
    return z - 3

@env.task
async def sequence(start: int) -> int:
    a = await task_a(start)
    b = await task_b(a)
    c = await task_c(b)
    return c

@env.task
async def main(starts: list[int]) -> list[int]:
    tasks = [sequence(start) for start in starts]
    results = await asyncio.gather(*tasks)
    return results

This pattern allows you to run any number of independent A→B→C sequences in parallel, each with its own input, leveraging Flyte's distributed execution and true parallelism according to Flyte 2 async model and async example. Would you like a step-by-step breakdown or more detail? Sources: - Parallelism in Flyte 1 vs Flyte 2 - llms-full.txt 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.

@ancient-wolf-19325 how would you do it in flyte 1

It seems likely that in Flyte 1, you would define each step (A, B, C) as a Flyte task and compose them into a workflow using the workflow DSL. To run multiple independent A→B→C sequences in parallel, you would use the

map

operator to fan out the initial input, then chain B and C for each mapped result. The workflow DSL automatically parallelizes tasks that aren't dependent on each other, and

map

allows running a task multiple times in parallel with different inputs. However, explicit async/await syntax and Python-native concurrency are not available in Flyte 1; parallelism is expressed via the workflow DSL and

map

operator, not Python async constructs. See Parallelism in Flyte 1 vs Flyte 2. Would you like a code example or more detail? Sources: - Parallelism in Flyte 1 vs Flyte 2 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.