I'm trying to use `ImageSpec` with `@task(containe...
# ask-the-community
a
I'm trying to use
ImageSpec
with
@task(container_image=...)
Here's my code:
Copy code
from flytekit import task, ImageSpec

my_task_image = ImageSpec(packages=["torch"],)

@task(
    container_image=my_task_image
) 
def hello(name: str) -> str:
    try:
        import torch
    except ImportError:
        print("no torch")

    msg = f"Hello {name}"
    return msg


@workflow
def hello_world_workflow(
    input_name: str = "World",
) -> str:
    hello_greeting: str = hello(name=input_name)

    return hello_greeting
I'm using
flytekit==1.9.1
and
flytekitplugins-envd==1.9.1
. When I execute my workflow, I get the following error:
Copy code
Failed with Unknown Exception <class 'Exception'> Reason: failed to run command envd build --path /tmp/flyte-m0o9qb2q/sandbox/local_flytekit/0f01355716b8894a0f943a8fc1db30fe  --platform linux/amd64 with error b'time="2023-08-30T20:02:29Z" level=fatal msg="failed to interpret: failed to exec starlark file /tmp/flyte-m0o9qb2q/sandbox/local_flytekit/0f01355716b8894a0f943a8fc1db30fe/build.envd: Exception when exec build func: io.copy: unexpected keyword argument \\"host_path\\""\n'
Inspecting the
build.envd
file that Flyte creates, this is what I see:
Copy code
# syntax=v1

def build():
    base(image="<http://cr.flyte.org/flyteorg/flytekit:py3.8-1.9.1|cr.flyte.org/flyteorg/flytekit:py3.8-1.9.1>", dev=False)
    install.python_packages(name=["torch"])
    install.apt_packages(name=[])
    runtime.environ(env={'PYTHONPATH': '/root', '_F_IMG_ID': 'flytekit:P1CReKTnTRkuvn7554aqfA..'})
    config.pip_index(url="<https://pypi.org/simple>")
    io.copy(host_path="./", envd_path="/root")
and the error seems to be at the last line
io.copy(host_path=...)
. Any leads?
k
looking? are you using non-fast register?
a
can you elaborate on non-fast register? I'm using an internal tool that handles registration:
Copy code
sh.flytectl(
        flatten(
            [
                "--logger.formatter.type",
                "text",
                "register",
                "files",
                "--project",
                workflow["flyteExecConf"]["flyteIdentifier"]["project"],
                "--domain",
                domain,
                "--version",
                version,
                "--archive",
                os.path.join(work_dir, FLYTE_PACKAGE),
                ["--files.outputLocationPrefix", output_location_prefix]
                if output_location_prefix
                else [],
            ],
        ),
@Kevin Su any ideas?
I think I understand the error. Flytekit adds
syntax=v1
to the
build.envd
, whereas in
v1
io takes
source
and
target
as args. Flytekit should add
syntax=v0
here see here
Here the flytekit adds both
syntax=v1
and
io.copy(host_path=..., envd_path=...)
to
build.envd
. I believe it should add
source
and
target
if syntax=v1
k
Oh, I think envd just changed the API, so it breaks.
we should update it to source and target.
downgrading envd can fix it
a
what version should I downgrade to?
flytekit / flytekitplugins-envd~=1.9.1
resolves to
envd==0.3.39
k
try 0.3.34