Tom Szumowski
07/29/2022, 6:11 PMbasic_workflow.py
, but with the t2
task decorator changed to:
@task(container_image="python:3.7")
When I run it, I get this error:
[f902b0296c1a94ed4ade-n1-0] terminated with exit code (128). Reason [StartError]. Message:
failed to create containerd task: failed to create shim: OCI runtime create failed: container_linux.go:380: starting container process caused: exec: "pyflyte-fast-execute": executable file not found in $PATH: unknown.
Is it possible to use any arbitrary image as a task there? Or does the image need to follow a specific build process that includes pyflyte-fast-execute
?
Thank you!pyflyte run --image <http://gcr.io/urbn-data-science/flytekit-test-wrapper:latest|gcr.io/urbn-data-science/flytekit-test-wrapper:latest> --remote workflows/basic_workflow_custom.py my_wf --a 10 --b foobar
The <http://gcr.io/urbn-data-science/flytekit-test-wrapper:latest|gcr.io/urbn-data-science/flytekit-test-wrapper:latest>
image is a default image I built due to GCP workflow identity errors discussed here. Not sure if that is conflicting with my desired goal above or not.Kevin Su
07/29/2022, 6:43 PMTom Szumowski
07/29/2022, 6:49 PMWORKDIR /root
ENV PYTHONPATH /root
RUN pip install awscli
RUN pip install gsutil
ARG VERSION
ARG DOCKER_IMAGE
# Pod tasks should be exposed in the default image
RUN pip install -U flytekit==$VERSION flytekitplugins-pod==$VERSION
ENV FLYTE_INTERNAL_IMAGE "$DOCKER_IMAGE"
If I wish to use a custom image, do I need to create a new image that runs all of the above as well? Otherwise I was getting GCP permission errors.
And if so, does that mean this should be applied to every custom Dockerfile I wish to have?Ketan (kumare3)
Tom Szumowski
07/29/2022, 10:34 PMKetan (kumare3)
Tom Szumowski
07/29/2022, 10:35 PMKetan (kumare3)
Tom Szumowski
07/29/2022, 10:39 PMKetan (kumare3)
pyflyte run / register
what we call fast register
Eli Bixby
01/25/2023, 4:56 PMflytekit
on e.g. a ShellTask
that doesn't have anything to do with python, has anyone tried making a sidecar pattern work?
We have a lot of weird tools we package up (bringing in legacy tools from another industry) in containers, and I'm a little worried about this pulling us into dependency hell.Felix Ruess
01/25/2023, 5:32 PMEli Bixby
01/25/2023, 5:36 PMDan Rammer (hamersaw)
01/25/2023, 5:37 PMEli Bixby
01/25/2023, 5:37 PMKetan (kumare3)
Slackbot
01/25/2023, 7:18 PMNiels Bantilan
01/25/2023, 9:54 PMKetan (kumare3)