https://flyte.org
Join Slack
Hi I have a flyte task that runs a subprocess that runs a binary. I want to redirect the output of t...
# flyte-support
l
Hi I have a flyte task that runs a subprocess that runs a binary. I want to redirect the output of the subprocess s.t. it displays in flyte logs. Is this possible?
f
Use shell task - all of this is handled for you
l
I am using a TFJob in the task config
will this still work with shell task?
or should I call the shellTask from within the other task
i.e. my task looks like:
Copy code
@task(
    task_config=TfJob(
        chief=Chief(replicas=1),
        ps=PS(replicas=1),
        worker=Worker(replicas=1),
    ),
)
def some_task_name(...):
   <download binary>
   subprocess.run(binary)
f
No , in that case just redirect stdout and stderr
l
so just by doing 
subprocess.run(binary, stdout=subprocess.PIPE)?
if 
binary
is a long running binary (such as a training service) will this pipe logs
f
I think you can get the stdout and stderr with
Copy code
process = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
stdout, stderr = process.communicate()
g
if 
binary
is a long running binary (such as a training service) will this pipe logs
yes, it will.
17 Views
Open in Slack
PreviousNext
Powered by

Flyte

Flyte enables production-grade orchestration for machine learning workflows and data processing created to accelerate local workflows to production.

Powered by