Hello, when I am packaging using

pyflyte package

command (following this guide basically) I am running into a strange issue where the serialized task definitions end up with a resolver path like

--resolver site-packages.flytekit.core.python_auto_container.default_task_resolver

Note the leading

site-packages

. Anyone know why this might be getting added / how to resolve it?

That is indeed weird

hmm yes

Hmm how did that get added

Just narrowed it down to

pip install sagemaker-experiments

What

repro:

❯ cat Dockerfile
FROM python:3.8-buster

WORKDIR /root
ENV VENV /opt/venv
ENV LANG C.UTF-8
ENV LC_ALL C.UTF-8
ENV PYTHONPATH /root

RUN apt-get update && apt-get install -y build-essential

RUN ls /usr/local/lib/python3.8/site-packages/__init__.py && echo "found" || echo "not found"
RUN pip install sagemaker-experiments
RUN ls /usr/local/lib/python3.8/site-packages/__init__.py && echo "found" || echo "not found"

❯ docker build --no-cache  --progress=plain .
#1 [internal] load build definition from Dockerfile
#1 sha256:3385e9be81db97c5e0e5ce54090f31ba550f4b07548d4c65ae02179933b6e332
#1 transferring dockerfile: 37B done
#1 DONE 0.0s

#2 [internal] load .dockerignore
#2 sha256:ebcfe1edb48fe24feab40f488611095f6a4d0518af01db20aab1011a0d483920
#2 transferring context: 2B done
#2 DONE 0.0s

#3 [internal] load metadata for <http://docker.io/library/python:3.8-buster|docker.io/library/python:3.8-buster>
#3 sha256:88586e40fdfcc6826fc75a7a9653c2727184be908b30c80bf7580390d6713f00
#3 DONE 0.3s

#4 [1/6] FROM <http://docker.io/library/python:3.8-buster@sha256:0a5ce3959e92cf12188a2f5482ca05297318e526d49956cbdfbb290d3da2e0e5|docker.io/library/python:3.8-buster@sha256:0a5ce3959e92cf12188a2f5482ca05297318e526d49956cbdfbb290d3da2e0e5>
#4 sha256:9f7dc4f5bfd550282d1b5910194759a207640d3086465bf212ff8d48789dea3b
#4 resolve <http://docker.io/library/python:3.8-buster@sha256:0a5ce3959e92cf12188a2f5482ca05297318e526d49956cbdfbb290d3da2e0e5|docker.io/library/python:3.8-buster@sha256:0a5ce3959e92cf12188a2f5482ca05297318e526d49956cbdfbb290d3da2e0e5> done
#4 DONE 0.0s

#5 [2/6] WORKDIR /root
#5 sha256:ca2d7269c2982d0f8e3e99c442a6ce6e9371cce1c83740287de2b48f37bf21c2
#5 CACHED

#6 [3/6] RUN apt-get update && apt-get install -y build-essential
#6 sha256:612de9715de36d9d860ed1ad266eaea7a8df903f5a117a210208761ee78d1eb2
#6 0.282 Get:1 <http://deb.debian.org/debian> buster InRelease [122 kB]
#6 0.327 Get:2 <http://deb.debian.org/debian-security> buster/updates InRelease [73.5 kB]
#6 0.348 Get:3 <http://deb.debian.org/debian> buster-updates InRelease [56.6 kB]
#6 0.451 Get:4 <http://deb.debian.org/debian> buster/main amd64 Packages [7911 kB]
#6 0.706 Get:5 <http://deb.debian.org/debian-security> buster/updates/main amd64 Packages [338 kB]
#6 0.713 Get:6 <http://deb.debian.org/debian> buster-updates/main amd64 Packages [8788 B]
#6 1.616 Fetched 8509 kB in 1s (6095 kB/s)
#6 1.616 Reading package lists...
#6 2.186 Reading package lists...
#6 2.737 Building dependency tree...
#6 2.850 Reading state information...
#6 2.964 The following NEW packages will be installed:
#6 2.965   build-essential
#6 3.021 0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
#6 3.021 Need to get 7576 B of archives.
#6 3.021 After this operation, 20.5 kB of additional disk space will be used.
#6 3.021 Get:1 <http://deb.debian.org/debian> buster/main amd64 build-essential amd64 12.6 [7576 B]
#6 3.154 debconf: delaying package configuration, since apt-utils is not installed
#6 3.178 Fetched 7576 B in 0s (167 kB/s)
#6 3.195 Selecting previously unselected package build-essential.
(Reading database ... 24614 files and directories currently installed.)
#6 3.209 Preparing to unpack .../build-essential_12.6_amd64.deb ...
#6 3.211 Unpacking build-essential (12.6) ...
#6 3.240 Setting up build-essential (12.6) ...
#6 DONE 3.3s

#7 [4/6] RUN ls /usr/local/lib/python3.8/site-packages/__init__.py && echo "found" || echo "not found"
#7 sha256:120a2156b92714e98bf5e48adad3b9311ced7dba3d1480c3167b42da145a5a8e
#7 0.211 ls: cannot access '/usr/local/lib/python3.8/site-packages/__init__.py': No such file or directory
#7 0.212 not found
#7 DONE 0.2s

#8 [5/6] RUN pip install sagemaker-experiments
#8 sha256:7378e0f8ae19cf099f0e340437565ba37834cd35a7648b9c64160f4564f65c38
#8 1.591 Collecting sagemaker-experiments
#8 1.718   Downloading sagemaker_experiments-0.1.35-py3-none-any.whl (42 kB)
#8 1.738      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 42.2/42.2 KB 2.0 MB/s eta 0:00:00
#8 2.284 Collecting boto3>=1.16.27
#8 2.304   Downloading boto3-1.24.44-py3-none-any.whl (132 kB)
#8 2.331      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 132.5/132.5 KB 5.3 MB/s eta 0:00:00
#8 2.374 Collecting s3transfer<0.7.0,>=0.6.0
#8 2.395   Downloading s3transfer-0.6.0-py3-none-any.whl (79 kB)
#8 2.403      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 79.6/79.6 KB 14.1 MB/s eta 0:00:00
#8 2.448 Collecting jmespath<2.0.0,>=0.7.1
#8 2.466   Downloading jmespath-1.0.1-py3-none-any.whl (20 kB)
#8 3.121 Collecting botocore<1.28.0,>=1.27.44
#8 3.142   Downloading botocore-1.27.44-py3-none-any.whl (9.0 MB)
#8 3.656      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 9.0/9.0 MB 17.6 MB/s eta 0:00:00
#8 3.770 Collecting urllib3<1.27,>=1.25.4
#8 3.788   Downloading urllib3-1.26.11-py2.py3-none-any.whl (139 kB)
#8 3.801      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 139.9/139.9 KB 14.5 MB/s eta 0:00:00
#8 3.848 Collecting python-dateutil<3.0.0,>=2.1
#8 3.868   Downloading python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
#8 3.885      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 247.7/247.7 KB 16.8 MB/s eta 0:00:00
#8 4.115 Collecting six>=1.5
#8 4.138   Downloading six-1.16.0-py2.py3-none-any.whl (11 kB)
#8 4.228 Installing collected packages: urllib3, six, jmespath, python-dateutil, botocore, s3transfer, boto3, sagemaker-experiments
#8 5.035 Successfully installed boto3-1.24.44 botocore-1.27.44 jmespath-1.0.1 python-dateutil-2.8.2 s3transfer-0.6.0 sagemaker-experiments-0.1.35 six-1.16.0 urllib3-1.26.11
#8 5.035 WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: <https://pip.pypa.io/warnings/venv>
#8 5.173 WARNING: You are using pip version 22.0.4; however, version 22.2.1 is available.
#8 5.173 You should consider upgrading via the '/usr/local/bin/python -m pip install --upgrade pip' command.
#8 DONE 5.5s

#9 [6/6] RUN ls /usr/local/lib/python3.8/site-packages/__init__.py && echo "found" || echo "not found"
#9 sha256:c41b0fbb49ddcbc7a10f2e66ab534680f125340949e01be1a09a211ff6aea0a7
#9 0.215 /usr/local/lib/python3.8/site-packages/__init__.py
#9 0.215 found
#9 DONE 0.2s

#10 exporting to image
#10 sha256:e8c613e07b0b7ff33893b694f7759a10d42e180f2b4dc349fb57dc6b71dcab00
#10 exporting layers
#10 exporting layers 0.5s done
#10 writing image sha256:a749ccca340ffe432a4459b822693dea909298456aacd6a13e2ea80f15980032 done
#10 DONE 0.5s

not sure what you mean by - restrict traversing to flytekit for core packages ketan?

@Yee I mean in the package resolution if the package path encounters flytekit we stop there