https://flyte.org logo
#ask-the-community
Title
# ask-the-community
f

Fotis Kalioras

06/28/2023, 9:06 AM
Hi everyone! I've recently started trying Flyte in order to deploy some ML pipelines. Since i use some privately developed libraries in my code i figured out that i must dockerize and push it to the registry in order to execute the flows w/ pyflyte . However, i haven't use Docker extensively so i have trouble pushing and using. Is there any sort of guide on how to extend the demo image or a base dockerfile to begin with? Thank you very much!
t

Tommy Nam

06/28/2023, 9:12 AM
Copy code
FROM <http://ghcr.io/flyteorg/flytekit:tag|ghcr.io/flyteorg/flytekit:tag> # whichever tag you want to build on

USER root

# Upgrade pip, git not needed if you don't want it
RUN apt-get -y install git && pip install --upgrade pip

# Install python packages
RUN pip install ........ \
     .......

# If we want to install custom git packages we do something like this:
# RUN pip install git+<https://gitrepolinkhere>

USER flytekit

CMD [] # Not needed
This is a basic Dockerfile structure we use to build custom images You can also use ImageSpec from 1.6.0 onwards I believe, but this is more explicit and manual @Fotis Kalioras
In terms of registry that would be something you set up on something like AWS ECR or Dockerhub etc
then you would build the images on your local machine or in a pipeline with: docker build, then you docker tag then you docker push after authenticating to your registry
f

Fotis Kalioras

06/28/2023, 9:17 AM
Thanks for the quick reply! One question more. If my modules are local .py files can i import them directly or must i setup a private repository?
t

Tommy Nam

06/28/2023, 9:21 AM
You can just use the inbuilt
COPY
command for Docker to copy the modules from your local build path into the Docker container's working directory. By default it's /root or $HOME So If you want to use relative paths with "./"
Copy code
COPY ./path/to/your/modules/on/local ./path/to/docker/container/directory
Or absolute paths in terms of PATH
Copy code
COPY /absolute/path/here/on/local /absolute/path/inside/docker/container
then pip install or just refer to it in your code and it should work as long as you've copied it into a PATH that your docker container + code can find
f

Fotis Kalioras

06/28/2023, 9:31 AM
Thank you very much, going to try it!
Hello again. While toying with docker and the dockerfiles i wondered if i could extend the demo or sandbox environment in order to have the libraries and local files i want
69 Views