👋 We're in the process of updating our flyte components to support

@eager

and I had a few questions: • It seems

client_secret_group

and

client_secret_key

are required — but we use a different authentication strategy with

authType: ExternalCommand

that passes an access token that is validated on a proxy that sits in front of flyte-admin. Are these secrets truly required in our auth scenario? ie., we do not use secrets in our

flytectl

usage • For all

@eager

usage that will execute on the backend, will users need to provide the

FlyteRemote

configuration for all of them? This feels pretty toilsome:

@eager(
    remote=FlyteRemote(
        config=Config.auto(config_file="config.yaml"),
        default_project="flytesnacks",
        default_domain="development",
    ),
    client_secret_group="<my_client_secret_group>",
    client_secret_key="<my_client_secret_key>",
)

◦ We likely can provide predefined configs, but were also curious if there were recommendations from the team on how to better manage this

there is an issue to automatically pass the auth down

Ah — I see. Would you be able to share more specifics or if there's already an open issue you could link me to

@John Votta @Ketan (kumare3) Feedback about

@eager

from our user:

from Limitations: https://docs.flyte.org/projects/cookbook/en/latest/auto_examples/advanced_composition/eager_workflows.html#limitations

• "You cannot invoke dynamic workflows, map tasks, or launch plans inside an eager workflow."

◦ 🟢/🟡 this seems ~fine i think? no real need for

@dynamic

or

@workflow

if you're adopting

@eager

• "Context managers will only work on locally executed functions within the eager workflow, i.e. using a context manager to modify the behavior of a task or subworkflow will not work because they are executed on a completely different pod."

◦ 🟢 seems fine

• "All exceptions raised by Flyte tasks or workflows will be caught and raised as an EagerException at runtime."

◦ 🟢 weird, but fine

• "All task/subworkflow outputs are materialized as Python values, which includes offloaded types like FlyteFile, FlyteDirectory, StructuredDataset, and pandas.DataFrame will be fully downloaded into the pod running the eager workflow. This prevents you from incrementally downloading or streaming very large datasets in eager workflows."

◦ 🟢 totally fine, e.g. FDE passes all datasets around as s3 paths anyway instead of using flyte's df handling magic

• "Flyte entities that are invoked inside of an eager workflow must be registered under the same project and domain as the eager workflow itself. The eager workflow will execute the latest version of these entities."

◦ 🟡/🔴 iiuc what this means it seems whack to me (see msg above)

• "Flyte console currently does not have a first-class way of viewing eager workflows, but it can be accessed via the task list view and the execution graph is viewable via Flyte Decks."

◦ 🟡/🔴 also problematic, since clicking through tasks in the flyte UI is a very useful way to introspect what's happening, monitor task runtimes, monitor overall progress, grab intermediate inputs/outputs/paths, etc.