Hi I have a flyte task that runs a subprocess that...
# flyte-supportl
late-traffic-56555
12/12/2023, 2:58 PMHi I have a flyte task that runs a subprocess that runs a binary. I want to redirect the output of the subprocess s.t. it displays in flyte logs. Is this possible?
f
freezing-airport-6809
12/12/2023, 3:14 PM
Use shell task - all of this is handled for you
l
late-traffic-56555
12/12/2023, 3:15 PMI am using a TFJob in the task config
late-traffic-56555
12/12/2023, 3:15 PMwill this still work with shell task?
late-traffic-56555
12/12/2023, 3:15 PMor should I call the shellTask from within the other task
late-traffic-56555
12/12/2023, 3:15 PMi.e. my task looks like:
late-traffic-56555
12/12/2023, 3:16 PM Copy code
@task(
task_config=TfJob(
chief=Chief(replicas=1),
ps=PS(replicas=1),
worker=Worker(replicas=1),
),
)
def some_task_name(...):
<download binary>
subprocess.run(binary)f
freezing-airport-6809
12/12/2023, 3:39 PM
No , in that case just redirect stdout and stderr
l
late-traffic-56555
12/12/2023, 3:39 PMso just by doing
subprocess.run(binary, stdout=subprocess.PIPE)?late-traffic-56555
12/12/2023, 3:40 PMif
binaryf
flaky-daybreak-42119
12/12/2023, 6:09 PMI think you can get the stdout and stderr with
Copy code
process = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
stdout, stderr = process.communicate()g
glamorous-carpet-83516
12/12/2023, 7:10 PMifyes, it will.is a long running binary (such as a training service) will this pipe logsbinary
17 Views