Hey all, I'm trying to experiment with "fast registration" via pyflyte but am unsuccessful and having a hard time finding docs that work. The docs tell me to use

--fast

in conjunction with

--image

to do a fast register of local code that will be "overlayed" onto to the specified image (as I understand it). But in my trials I find that: 1. --fast is not a recognized option, eventually realized it is a command (without dashes), are the docs are wrong? 2. Then it complains "No such option --image". Again, are docs are wrong? Or I'm misreading something. I was trying to modify a command that works without "fast" into one that runs with "fast" to see if I can avoid building and pushing a new image (the image I specify in this case already exists). The command looks like this, where I've inserted $FAST_REG into a command that otherwise works to serialize against a given image:

pyflyte --pkgs plaster.genv2 serialize $FAST_REG --image <my-ecr-registry-path>/<my-image-name>:<tag> --local-source-root . workflows -f /tmp/_pb_outputs

...where the $FAST_REG is where I'm attempting to insert "fast" and run my local code overlayed with what is in the image I'm specifying. I'm probably misunderstanding something, and the docs aren't helping me. Thanks for any tips!

@Thomas Blom let me message

1.3.1

I am not close to a computer but cc @Eduardo Apolinario (eapolinario) / @Alireza anyone helps or

Ok, no worries, I don't have to fix this tonight, I was just exploring and wanting to get involved in this community on Slack. I'll probably have a look in the morning. Thanks!

I can try and answer a little later

Hi Thomas, you can certainly use your image. Can you try

-i

then image parameters?

Thanks Alireza, but I don't see anything on that page about serialization via pyflyte like what I'm attempting -- I'm trying to add a fast-registration option into a fairly complete process we have for building/pushing an image, serializing tasks/workflows against that image, and then registering those serialized outputs with the backend. I'm just trying to insert that "fast" option so that I can refer to an existing image to use as a base for the subsequent overlay of current code. My high-level understanding is that this will essentially gzip my source and overlay this onto the image I specify so that I don't have to build and push a new image with every local change. Anyway, I'm turning in the for evening; I'll resume investigation in the morning. Thanks!

Thank you Thomas for the feedback. Fast registration handles all serialization and packaging behind the scenes. If your changes only involve the code and your other configurations remain unchanged, fast registration should be able to handle this without worrying about serialization. But in case I am missing something here, I'll make sure to update the documentation based on the outcome of this discussion. 🙂

@Thomas Blom you are right - --fast registration uses a special api from flyteadmin to upload code and then apply that at runtime to the images. thus skipping an image build

I'm assuming it's safe to delete this page because all the registration-related content is available on https://docs.flyte.org/projects/cookbook/en/latest/getting_started/package_register.html page. @Thomas Blom, please refer to the linked page.

Ok, thanks all. I'm a bit confused b/c the last page linked by @Samhita Alla only mentions "fast registration" once in a sidebar, and based on this it sounds like fast registration is done by default when using the "register" command. Elsewhere it says to use --fast with the "package" command, and in the --help for the command line it says to use the command "fast". I'm working with Union to host the backend so my current workflow is to use pyflyte to serialize, and then uctl to register, so I'm going to take this up with them, and I'll report back results here for clarity and in case it can help someone else.

@Samhita Alla let’s preserve this page and fix it… the getting started info is surface-level stuff, I think https://docs.flyte.org/projects/cookbook/en/stable/auto/deployment/deploying_workflows.html can be improved to provide more detail. Also @Thomas Blom I think

pyflyte serialize

is an old CLI command:

pyflyte package

is the recommended way of getting a portable zip file of flyte tasks/workflows at this point/

With either

pyflyte register

or

pyflyte package + flytectl register

, does the docker image need to contain the actual workflow/task definition logic? The existing Dockerfile generated by

pyflyte init

has code to copy the workflow/task logic, which seems to imply that the workflow/task logic should be in the Docker image, but from the discussion in this thread and some of the documentation, it doesn't seem like this copy is necessary. Are there benefits to copying the code into the Docker image (would there be lower latency/steps skipped)? Logic in Dockerfile that copies workflow/task code:

# Copy the actual code
COPY . /root

@Jimmy Du great question, so there are 2 flows 1. Fast register / pyflyte run / pyflyte register (defaults to fast). In this flow, the code can be in the docker image, but a new copy of the code is supplied at runtime and overriden. Important note: This needs all your dependencies to be installed inside the container. This is recommended for quick iteration, development 2. regular register. It assumes the code is in the docker image. This is recommended for production and makes everything highly traceable, and reproducible. the copy into

/root

is that code is copied there in a default “regular” register flow

Thank you for the clarification, @Ketan (kumare3)!