bitter-photographer-94114
12/05/2024, 8:25 PM<http://ghcr.io/flyteorg/rawcontainers-python:v2|ghcr.io/flyteorg/rawcontainers-python:v2>
) it was ok with default inputs and outputs (e.x. a:float, b:float, area:float, metadata:str
), BUT when I change them to FlyteFile or something else. it throws an error that "No Such File or Directory called area
or metadata
etc.
I was thinking it's due to the image build.
Is there anybody who has experience on this? or facing the same issues?
Thank you.average-finland-92144
12/05/2024, 8:28 PMFlyteDirectory
as input to RawContainer Tasks is not supported yet (it will be in the 1.14 release going out tomorrow)average-finland-92144
12/05/2024, 8:29 PMbitter-photographer-94114
12/05/2024, 8:36 PM--step definitiion--
save_to_csv = ContainerTask(
name="save_to_csv_container",
input_data_dir="/var/inputs",
output_data_dir="/var/outputs",
inputs=kwtypes(a=float, b=float),
outputs=kwtypes(area=float, metadata=str),
image="<http://ghcr.io/flyteorg/rawcontainers-python:v2|ghcr.io/flyteorg/rawcontainers-python:v2>",
command=[
"python",
"main.py",
"{{.inputs.a}}",
"{{.inputs.b}}",
"/var/outputs",
],
)
--main.py--
import sys
def write_output(output_dir, output_file, v):
with open(f"{output_dir}/{output_file}", "w") as f:
f.write(str(v))
def main(a, b, output_dir):
write_output(output_dir, "area", str(float(5.0)))
write_output(output_dir, "metadata", "[from python rawcontainer]")
if __name__ == "__main__":
input_path = sys.argv[1]
output_file = sys.argv[2]
output_dir = sys.argv[3]
main(input_path, output_file, output_dir)
This works but the below code doesn't work.
--step definitiion--
save_to_csv = ContainerTask(
name="save_to_csv_container",
input_data_dir="/var/inputs",
output_data_dir="/var/outputs",
inputs=kwtypes(a=str, b=float), # a changed to str type
outputs=kwtypes(area=float, metadata=str),
image="<http://ghcr.io/flyteorg/rawcontainers-python:v2|ghcr.io/flyteorg/rawcontainers-python:v2>",
command=[
"python",
"main.py",
"{{.inputs.a}}",
"{{.inputs.b}}",
"/var/outputs",
],
)
--main.py--
import sys
def write_output(output_dir, output_file, v):
with open(f"{output_dir}/{output_file}", "w") as f:
f.write(str(v))
def main(a, b, output_dir):
write_output(output_dir, "area", str(float(5.0)))
write_output(output_dir, "metadata", "[from python rawcontainer]")
if __name__ == "__main__":
input_path = sys.argv[1]
output_file = sys.argv[2]
output_dir = sys.argv[3]
main(input_path, output_file, output_dir)
The code might look weird since I removed some parts.
Of course I passed the a
input to both code as each variable type.bitter-photographer-94114
12/05/2024, 11:16 PMdamp-lion-88352
12/06/2024, 12:52 AMdamp-lion-88352
12/06/2024, 4:19 PMdocker run --it <http://ghcr.io/flyteorg/rawcontainers-python:v2|ghcr.io/flyteorg/rawcontainers-python:v2> /bin/sh
and ls
to me?damp-lion-88352
12/06/2024, 4:20 PMmain.py
really existdamp-lion-88352
12/06/2024, 4:23 PM(dev) future@outlier ~ % pyflyte run build/flytesnacks/examples/customizing_dependencies/customizing_dependencies/raw_container.py calculate_ellipse_area_python --a 1 --b 1.0
00:23:00.454852 INFO file.py:252 - Using flytectl/YAML config /Users/future-outlier/.flyte/config-sandbox.yaml
Running Execution on local.
DefaultNamedTupleOutput(area=3.141592653589793, metadata='[from python rawcontainer]')
damp-lion-88352
12/06/2024, 4:23 PMcalculate_ellipse_area_python = ContainerTask(
name="ellipse-area-metadata-python",
input_data_dir="/var/inputs",
output_data_dir="/var/outputs",
inputs=kwtypes(a=str, b=float),
outputs=kwtypes(area=float, metadata=str),
image="<http://ghcr.io/flyteorg/rawcontainers-python:v2|ghcr.io/flyteorg/rawcontainers-python:v2>",
command=[
"python",
"calculate-ellipse-area.py",
"{{.inputs.a}}",
"{{.inputs.b}}",
"/var/outputs",
],
metadata=TaskMetadata(cache=True, cache_version="1.0"),
)
damp-lion-88352
12/06/2024, 4:23 PMechoing-account-76888
12/06/2024, 4:31 PMContainerTask(
name="save_to_csv_container",
input_data_dir="/var/inputs",
output_data_dir="/var/outputs",
inputs=kwtypes(a=float, b=float),
outputs=kwtypes(area=float, metadata=str),
image=ImageSpec(
registry="localhost:30000",
base_image="<http://ghcr.io/flyteorg/rawcontainers-python:v2|ghcr.io/flyteorg/rawcontainers-python:v2>",
copy=["main.py"],
)
bitter-photographer-94114
12/09/2024, 8:16 PMbitter-photographer-94114
12/09/2024, 8:25 PMaverage-finland-92144
12/10/2024, 12:15 PM'flytekit.types.file.file.FlyteFile'>. Expected a file, but
hello.txt is not a file.
average-finland-92144
12/10/2024, 12:16 PMflytekit==1.14.0
bitter-photographer-94114
12/10/2024, 4:38 PM