Hi, guys! I am running my workflows using airflow....
# ask-the-community
m
Hi, guys! I am running my workflows using airflow. And I want to read outputs of the previous execution before running a new one. How to properly do that using flytekit python api? • How to get recent execution for a particular workflow (latest version)? I know that you can fetch_execution but only with known id • How to check the outputs of this executions? I’ve tried the following:
Copy code
hook = FlyteHook(flyte_conn_id=FLYTE_CONN_ID, project=flyte_project, domain=flyte_domain)
remote = hook.create_flyte_remote()

# workflow = remote.fetch_workflow(flyte_project, flyte_domain, flyte_task)
executions = remote.recent_executions(flyte_project, flyte_domain)
executions_filtered = list(
    filter(
        lambda e: e.spec.launch_plan.name == flyte_task and e.is_done,
        executions,
    )
)
if executions_filtered:
    execution = executions_filtered[0]
    <http://logging.info|logging.info>(f"{execution.id}")
    outputs = execution.outputs
    <http://logging.info|logging.info>(f"{outputs}")
But filtering of executions is not really convenient and the outputs are empty (however they shouldn’t be so)
e
@mykyta luzan, two things: 1.
recent_executions
has a
limit
parameter. 2. make sure to call sync on the execution prior to checking its outputs / nodes.
m
Sync works! Thanks
limit
- not very useful in my case. It just set the number of recent executions Is there any better way to filter executions that match a certain workflow/task? I’ve tried to run
execution.flyte_workflow.id == workflow.id
, but
flyte_workflow
returns None
158 Views