https://flyte.org
Join Slack
Hi everyone! Does anyone know if there is a way to get the `node id` of a Flyte task execution from ...
# flyte-support
c
Hi everyone! Does anyone know if there is a way to get the 
node id
of a Flyte task execution from within that task? I have been looking into 
current_context
from 
flytekit
but it is of type 
ExecutionParameters
which only has 
execution_id
(workflow id) and 
task_id
. The 
task_id
is of type 
Identifier
not 
TaskExecutionIdentifier
, so it does not have 
node_execution_id
Thank you !!
t
not that i know of easily. what are you trying to accomplish ?
it’s embedded in the raw output prefix that propeller sets, but that’s not a good thing to rely on.
c
from within a task execution, I am trying to export a custom message to a pubsub topic. I need the message to be digested in a similar manner to Flyte internal messages. In order to link the custom message to messages published by flyte, I was planning on using execution_id and node_id to identify unique task executions
I guess this brings up another question. Given that I can have multiple executions of the same task within a single workflow, is there a way for me to keep track of unique task executions without using the node id?
t
just curious (not that there’s support for this or anything) but does this need to be done while the task is executing, or would it be okay for the events to be queued and sent only after the task is complete?
short of passing a counter through, no. tasks are supposed to be agnostic right?
the raw output prefix is unique btw
the raw output prefix set by propeller today looks like
Copy code
<s3://my-s3-bucket/data/o4/feda4e266c748463a97d-n0-0>
that’s node and retry
but again i think this is a bad thing to rely on
i think propeller should publish a formal unique identifier that’s unique per proj/domain/execname/node/retry
c
hmm. I store some custom metrics about the task (collected during the execution) within the task and I want to export them to pubsub at the end of the execution so I think it needs to be during the execution since I may lose the info after the task is done running
right, the raw output prefix has the node id so it should be unique and I have access to it from within 
current_context
, i dont think I see the same within the 
TaskExecutionEventRequest
which gets published by Flyte by default
and I need to find a way to link messages sent by flyte and messages that I explicitly send within tasks
is there anyway to use propeller to get the specific node id from within a task
t
like a rpc of some sort going from task back to propelelr?
no there is not
can you cut an issue to add a new unique identifier?
👍 1
you don’t actually care about node id right? you just want something that matches
c
right
t
make an issue for it?
222 Views
Open in Slack
PreviousNext
Powered by

Flyte

Flyte enables production-grade orchestration for machine learning workflows and data processing created to accelerate local workflows to production.

Powered by