https://flyte.org logo
#ask-the-community
Title
# ask-the-community
a

Alykhan Tejani

12/12/2023, 2:58 PM
Hi I have a flyte task that runs a subprocess that runs a binary. I want to redirect the output of the subprocess s.t. it displays in flyte logs. Is this possible?
k

Ketan (kumare3)

12/12/2023, 3:14 PM
Use shell task - all of this is handled for you
a

Alykhan Tejani

12/12/2023, 3:15 PM
I am using a TFJob in the task config
will this still work with shell task?
or should I call the shellTask from within the other task
i.e. my task looks like:
Copy code
@task(
    task_config=TfJob(
        chief=Chief(replicas=1),
        ps=PS(replicas=1),
        worker=Worker(replicas=1),
    ),
)
def some_task_name(...):
   <download binary>
   subprocess.run(binary)
k

Ketan (kumare3)

12/12/2023, 3:39 PM
No , in that case just redirect stdout and stderr
a

Alykhan Tejani

12/12/2023, 3:39 PM
so just by doing
subprocess.run(binary, stdout=subprocess.PIPE)?
if
binary
is a long running binary (such as a training service) will this pipe logs
y

Yi Chiu

12/12/2023, 6:09 PM
I think you can get the stdout and stderr with
Copy code
process = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
stdout, stderr = process.communicate()
k

Kevin Su

12/12/2023, 7:10 PM
if
binary
is a long running binary (such as a training service) will this pipe logs
yes, it will.
2 Views