Hi everyone! I've recently started trying Flyte in...
# flyte-support
o
Hi everyone! I've recently started trying Flyte in order to deploy some ML pipelines. Since i use some privately developed libraries in my code i figured out that i must dockerize and push it to the registry in order to execute the flows w/ pyflyte . However, i haven't use Docker extensively so i have trouble pushing and using. Is there any sort of guide on how to extend the demo image or a base dockerfile to begin with? Thank you very much!
f
Copy code
FROM <http://ghcr.io/flyteorg/flytekit:tag|ghcr.io/flyteorg/flytekit:tag> # whichever tag you want to build on

USER root

# Upgrade pip, git not needed if you don't want it
RUN apt-get -y install git && pip install --upgrade pip

# Install python packages
RUN pip install ........ \
     .......

# If we want to install custom git packages we do something like this:
# RUN pip install git+<https://gitrepolinkhere>

USER flytekit

CMD [] # Not needed
This is a basic Dockerfile structure we use to build custom images You can also use ImageSpec from 1.6.0 onwards I believe, but this is more explicit and manual @orange-author-51398
🔥 1
In terms of registry that would be something you set up on something like AWS ECR or Dockerhub etc
then you would build the images on your local machine or in a pipeline with: docker build, then you docker tag then you docker push after authenticating to your registry
o
Thanks for the quick reply! One question more. If my modules are local .py files can i import them directly or must i setup a private repository?
f
You can just use the inbuilt
COPY
command for Docker to copy the modules from your local build path into the Docker container's working directory. By default it's /root or $HOME So If you want to use relative paths with "./"
Copy code
COPY ./path/to/your/modules/on/local ./path/to/docker/container/directory
Or absolute paths in terms of PATH
Copy code
COPY /absolute/path/here/on/local /absolute/path/inside/docker/container
then pip install or just refer to it in your code and it should work as long as you've copied it into a PATH that your docker container + code can find
o
Thank you very much, going to try it!
Hello again. While toying with docker and the dockerfiles i wondered if i could extend the demo or sandbox environment in order to have the libraries and local files i want
147 Views