helpful-church-28990
01/21/2025, 4:27 PMRunContainerError: failed to create containerd task: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: exec: \"pyflyte-execute\": executable file not found in $PATH: unknown" logger="UnhandledError"
But when we try to run the same image from local we can see that pyflyte-execute
is available but when we run in dev server we get error.
For ref I have attached our docker file :
FROM <http://dockerhubproxy.schibsted.io/python:3.9.18-slim|dockerhubproxy.schibsted.io/python:3.9.18-slim>
# The installer requires curl (and certificates) to download the release archive
RUN apt-get update && apt-get install -y --no-install-recommends curl ca-certificates
# Download the latest installer
ADD <https://astral.sh/uv/install.sh> /uv-installer.sh
# TODO: Specify a version
#ADD <https://astral.sh/uv/0.5.#/install.sh> /uv-installer.sh
# Run the installer then remove it
RUN sh /uv-installer.sh && rm /uv-installer.sh
# Ensure the installed binary is on the `PATH`
ENV PATH="/root/.local/bin/:$PATH"
COPY /flyte /flyte
COPY /tests /tests
ENV WORKDIR=/app
WORKDIR $WORKDIR
ENV PYTHONPATH="$WORKDIR:$WORKDIR/flyte"
ENV PATH="/app/.venv/bin:$PATH"
COPY pyproject.toml /pyproject.toml
COPY create_config.sh /create_config.sh
COPY .github/scripts /.github/scripts
RUN chmod +x /.github/scripts/build_and_push_image.sh
# Run the shell script to create the config-remote.yaml file with an argument
# dev is the default value. Can be overridden during the build process using --build-arg.
ARG FLYTE_ENVIRONMENT=dev
RUN chmod +x /create_config.sh && /create_config.sh ${FLYTE_ENVIRONMENT}
# Install a specific version of flytectl
ENV FLYTECTL_VERSION=v0.9.2
RUN curl -sLo flytectl_Linux_x86_64.tar.gz <https://github.com/flyteorg/flyte/releases/download/flytectl/${FLYTECTL_VERSION}/flytectl_Linux_x86_64.tar.gz> && \
tar -xzf flytectl_Linux_x86_64.tar.gz && \
install flytectl /usr/local/bin/ && \
# Clean up downloaded and extracted files
rm -rf flytectl_Linux_x86_64.tar.gz flytectl
RUN uv sync --inexact --no-cache-dir
So can anyone help us on how to address this issue. Your input and help will be much appreciatedtall-lock-23197
helpful-church-28990
01/22/2025, 7:12 AMpyflyte-execute --help
command from anywhere in the imagehelpful-church-28990
01/22/2025, 7:14 AM[project]
name = "my-flyte-workflows"
version = "0.1.0"
description = "My workflow"
readme = "README.md"
requires-python = "~=3.9.17"
dependencies = [
"flytekit~=1.8.1",
"kubernetes~=28.1.0",
"pandas~=1.5.3",
"numpy~=1.26.4",
"toml~=0.10.2",
"datadog~=0.48.0",
"ddtrace~=2.5.0",
"pytest>=8.3.3",
"fire>=0.7.0",
"ruff>=0.9.0",
]
[project.optional-dependencies]
dbt = [
"flytekitplugins-dbt~=1.12.0",
"dbt-core~=1.8.7",
"dbt-snowflake~=1.8.3",
]
optuna = [
"psycopg2-binary~=2.9.9"
]
redis = [
"redis[hiredis]~=5.0.4"
]
s3 = [
"s3fs~=2024.6.1",
"boto3~=1.34.70"
]
snowflake = [
"snowflake-connector-python[pandas]~=3.12.2",
"PySocks~=1.7.1",
]
visualization = [
"plotly~=5.20.0",
"flytekitplugins-deck-standard~=1.11.0",
]
[tool.uv]
dev-dependencies = [
"jupyterlab>=4.3.1",
]
tall-lock-23197
helpful-church-28990
01/22/2025, 7:25 AMtall-lock-23197
tall-lock-23197
helpful-church-28990
01/22/2025, 7:27 AMhelpful-church-28990
01/22/2025, 7:29 AMroot@fc2cd75f1d7c:/app# which pyflyte-execute
/app/.venv/bin/pyflyte-execute
root@fc2cd75f1d7c:/app# which flytectl
/usr/local/bin/flytectl
root@fc2cd75f1d7c:/app#
helpful-church-28990
01/22/2025, 7:30 AM/usr/local/bin/flytectl
in here, while /app/.venv/bin/pyflyte-execute
is in different pathtall-lock-23197
tall-lock-23197
/app/.venv/bin
is in PATH.helpful-church-28990
01/22/2025, 7:32 AMPATH=/root/.local/bin:/app/.venv/bin:/root/.local/bin/:/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
tall-lock-23197
helpful-church-28990
01/22/2025, 7:59 AMbrave-machine-85911
01/22/2025, 10:08 AMENTRYPOINT
or another command it does? Also, I thought that the containers themselves do not need to have pyflyte
installed?
I’ve tried to search for docs about this, but have not found anything relevant unfortunately.brave-machine-85911
01/22/2025, 10:08 AMbrave-machine-85911
01/22/2025, 10:10 AMdocker run
command with the relevant image:
docker run -it uv-test-arm64 pyflyte-execute --help
Usage: pyflyte-execute [OPTIONS] [RESOLVER_ARGS]...
Options:
--inputs TEXT [required]
--output-prefix TEXT [required]
--raw-output-data-prefix TEXT
--checkpoint-path TEXT
--prev-checkpoint TEXT
--test
--dynamic-addl-distro TEXT
--dynamic-dest-dir TEXT
--resolver TEXT
--help Show this message and exit.
brave-machine-85911
01/22/2025, 10:10 AM@task
in a pod?average-finland-92144
01/22/2025, 4:31 PMaverage-finland-92144
01/22/2025, 4:32 PMpyflyte-execute
should be foundaverage-finland-92144
01/22/2025, 4:33 PMflytekit~=1.8.1
?brave-machine-85911
01/23/2025, 1:55 PMWORKDIR
in our Dockerfile
. Yup, we need to upgrade to a higher flytekit
version, most def 😊 .average-finland-92144
01/23/2025, 2:39 PM