https://flyte.org logo
Join Slack
Powered by
<@U06RKAMK0UC> I want to understand more about `ea...
# ask-ai
m
@ancient-wolf-19325 I want to understand more about 
eager
workflows: • What version are they available on • How do I choose specific version of a task run by eager workflow (or is it always latest registered) • How do I observe the task progress, number of tasks spawned etc. • How does the authentication works? Does each eager task need auth separately?
a
Eager workflows in Flyte allow you to define workflows using Python's native async/await syntax, enabling you to materialize task and subworkflow outputs as real Python values at runtime. This approach provides flexibility to use Python constructs like conditionals, loops, and exception handling, which are not possible in static or dynamic workflows. Eager workflows are defined with the 
@eager
decorator and executed as async functions, leveraging the 
asyncio
API for parallelism and concurrency. However, you lose compile-time type safety and first-class data lineage tracking compared to static workflows. This feature is still in beta and has some limitations, such as all outputs being fully materialized and limited UI support in Flyte Console. For more, see Flyte eager workflows documentation and Flyte 1.10 release blog. Would you like more detail or code examples? Sources: - Flyte eager workflows documentation - Flyte 1.10 release blog 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.
m
@ancient-wolf-19325 Yes I want more details and answers to my questions • What version are they available on • How do I choose specific version of a task run by eager workflow (or is it always latest registered) • How do I observe the task progress, number of tasks spawned etc. • How does the authentication works? Does each eager task need auth separately?
a
Certainly! Here’s a step-by-step breakdown of eager workflows in Flyte: 1. What are eager workflows? Eager workflows are defined using the 
@eager
decorator on async Python functions. Unlike static (
@workflow
) or dynamic (
@dynamic
) workflows, eager workflows are not compiled into a workflow spec. Instead, they execute tasks and subworkflows immediately, materializing their outputs as native Python values. This allows you to use any Python control flow (if/else, loops, try/except) and operate on outputs directly, rather than dealing with Flyte Promises. See Flyte eager workflows documentation. 2. How do you define and use them? You decorate an async function with 
@eager
. Inside, you can 
await
Flyte tasks, subworkflows, or other eager workflows. For parallel execution, use 
asyncio.gather
. Here’s a minimal example:
Copy code
from flytekit import task, eager

@task
def add_one(x: int) -> int:
    return x + 1

@task
def double(x: int) -> int:
    return x * 2

@eager
async def simple_eager_workflow(x: int) -> int:
    out = await add_one(x=x)
    if out < 0:
        return -1
    return await double(x=out)
You can run this locally with 
asyncio.run(simple_eager_workflow(x=1))
or register and run it remotely using 
pyflyte register
and 
pyflyte run
(docs). 3. What can you do with eager workflows? - Use Pythonic control flow (if/else, loops, try/except). - Gather outputs from multiple tasks in parallel with 
asyncio.gather
. - Invoke static workflows and nest eager subworkflows. - Catch exceptions using 
EagerException
. - Materialize outputs as Python objects for further computation (examples). 4. Limitations and caveats: - No compile-time type safety or static DAG analysis. - All outputs are fully materialized (not streamed). - Limited UI support: eager workflows appear as tasks in Flyte Console, and their execution graph is only visible via Flyte Decks. - Context managers only work for locally executed functions. - All entities invoked must be registered in the same project/domain as the eager workflow (limitations). 5. How does parallelism work? Eager workflows leverage Python’s 
asyncio
for concurrency. When you use 
await
or 
asyncio.gather
, Flyte executes tasks in parallel across distributed infrastructure (e.g., Kubernetes pods), not just within a single process (Flyte 2 async model). If you have specific questions about eager workflows, please ask! Sources: - Flyte eager workflows documentation - Flyte eager workflow code examples - Flyte 1.10 release blog - Flyte 2 async model 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.
m
@ancient-wolf-19325 Can I select what version of task do I use or are eager workflows always use latest
a
You can select which version of a task to use in eager workflows. By default, eager workflows execute the latest version of tasks and subworkflows registered under the same project and domain, but you can specify a particular version if needed. This is managed via Flyte's versioning system, and the 
FlyteRemote
object allows you to reference tasks by specific version or use 
auto_version="latest"
or 
auto_version="current"
as needed. See Flyte eager workflows documentation and Remote tasks versioning. Would you like a code example or more detail? Sources: - Flyte eager workflows documentation - Remote tasks versioning 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.
m
@ancient-wolf-19325 I found out that eager workflow requires me to use authentication with the admin. Can I somehow use the same credentials used to submit eager workflow to submit the tasks? Or I have to reauthenticate
a
Yes, you can use the same credentials used to submit the eager workflow to submit the tasks within it. Eager workflows use the FlyteRemote object, which is initialized with the current authentication context, so as long as your environment variables or config provide the necessary credentials (e.g., client secret), all tasks and subworkflows invoked by the eager workflow will use the same authentication context—no need to reauthenticate. See Flyte eager workflows remote access. Would you like more detail or a code example? Sources: - Flyte eager workflows documentation 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.
m
@ancient-wolf-19325 but the initial workflow was run from users machine, so then I need to use Service Account to authenticate? I cant pass the idnentity of the user who submitted eager to the sub-tasks?
a
You cannot automatically pass the user's identity from a local machine execution to the sub-tasks running remotely in an eager workflow; the remote tasks will use the credentials (such as a service account or client secret) configured in the environment where the workflow is executed. FlyteRemote uses the authentication context available in the pod or environment, not the local user's identity, for all remote executions within the eager workflow. See Flyte eager workflows remote access. Would you like more detail on authentication models or possible workarounds? Sources: - Flyte eager workflows documentation 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.
3 Views
Open in Slack
PreviousNext
Powered by