Hi -- Is there a way to use

flytectl

to retrieve the outputs from a task? For some background, I'd like to generate a list of task runs that includes some of the arguments passed to the task as well as the s3 location of the output dataframe. I plan on doing further analysis on this data in a notebook. I've looked at

flytectl get

and the list of available nouns, but I don't see anything that provides the output of the tasks. Is there a way to get this information from the command line or via the python api?

have you tried fetching an execution? i would expect output to be part of that payload but i haven't confirmed

I have tried

flytectl get execution ...

but I assumed "execution" meant the specific instance of a workflow. I think I want something like a "task execution". The output of

flytectl get execution ...

does include a

closure.outputs

field that has a

uri

pointing to an s3 object (a protobuf .pb file which I haven't tried to inspect) but there is nothing indicating a specific task run.

interesting. the admin api allows for

/executionId/taskNumber

to fetch the specific details you're looking for (i think). let me poke around a bit and see if that's called by any of the flytectl code

I see in the docs for flytekit get execution information about

--nodeID

, as well as

--details

. I'm trying to figure out what "oeh94k9r2r" in the docs might mean in my context.

if you want to use the CLI, this works to get output from all nodes related to an execution id:

flytectl get execution -p flyteproject -d development executionId --details

Ah, that "oeh94k9r2r" is an execution id. If I pass a specific execution id copied from the Flyte UI, I get a lot more information.

if you are trying to do interesting stuff with the results, it might be easier to use the FlyteRemote python sdk

Oh, interesting. I'm not familiar with that. I'll look it up. For this one-off project, parsing the json might be just fine. But good to know there is another interface available.

just in case, heh https://github.com/flyteorg/flytekit

I've been looking into

FlyteRemote

a bit, and while I find it easy to get the latest execution or a specific execution, I don't see a simple way to get a list of executions that fit some constraints. Specifically, I'd like to filter on the launch plan name and the fixed parameters passed to the workflow. The only method in the

FlyteRemote

class that accepts filters is

list_signals

, which works for a specific execution name if I understand it correctly. I see the same question asked in . At the time, it was marked as to-do, but this is from two years ago, so maybe this has been implemented? So I think I'll focus on retrieving this information from

flytectl get execution

and just parse the pages of json.

you might be able to use something like

list_executions_paginated

, but i haven't used it myself. and if the cli is working for ya - 🚀