Noah Jackson
02/28/2024, 12:31 AMpyflyte run remote-workflow workflow.simple_input_flow --input=input.txt
The issue is that flytekit doesn't upload input.txt
to s3 the way it would if I were creating the workflow via
pyflyte run --remote workflow.py simple_input_flow --input=input.txt
In the former example flytekit will use a path like /tmp/flyte-3x5f57__/raw/6b1dee242a380ced7fe1e44af4f3a36b/input.txt
to refer to the file, which will cause the task to fail. In contrast, the latter case correctly uploads the file to s3 and the file will be referenced by a path like <s3://noahfly37041-flyte-data/flytesnacks/development/WTIDKJAQCPBKDYDRFRUHFJPKBM======/input.txt>
I'm not sure if I'm going about this in the wrong way or if this is a bug that impacts my specific use caseKetan (kumare3)
pyflyte run remote-workflow workflow.simple_input_flow --input=input.txt --help
Ketan (kumare3)
Ketan (kumare3)
Ketan (kumare3)
is_remote
is set only when --remote
is passed
https://github.com/flyteorg/flytekit/blob/e394af0be9f904fbf8be675eaa8b8cdc24311ced/flytekit/clis/sdk_in_container/run.py#L372
we should just set
run_level_params.remote = True
here - https://github.com/flyteorg/flytekit/blob/e394af0be9f904fbf8be675eaa8b8cdc24311ced/flytekit/clis/sdk_in_container/run.py#L639Ketan (kumare3)
Ketan (kumare3)
L godlike
02/28/2024, 5:33 AML godlike
02/28/2024, 5:33 AML godlike
02/28/2024, 5:34 AMSamhita Alla
Ketan (kumare3)
Noah Jackson
02/28/2024, 4:58 PMSamhita Alla
Ketan (kumare3)