Hello :wave: community! Appreciate your great work...
# ask-the-community
x
Hello đź‘‹ community! Appreciate your great work! I have a potentially quick question, can the flyte serialization settings take a image digest (with
@sha256
) in anyway for example:
Copy code
image_config = ImageConfig(default_image=Image(name=container_image_url))

    ss = SerializationSettings(
        image_config=image_config,
        project=execution_options.project,
        domain=execution_options.domain,
        version=version,
    )
where
contaienr_image_url
is an image digest, and it would give me such error
Copy code
TypeError: __init__() missing 2 required positional arguments: 'fqn' and 'tag'
I checked the source code and it seems like the image full path can only be name-tag but not digest?
Copy code
class Image(object):
    """
    Image is a structured wrapper for task container images used in object serialization.

    Attributes:
        name (str): A user-provided name to identify this image.
        fqn (str): Fully qualified image name. This consists of
            #. a registry location
            #. a username
            #. a repository name
            For example: `hostname/username/reponame`
        tag (str): Optional tag used to specify which version of an image to pull
    """

    name: str
    fqn: str
    tag: str

    @property
    def full(self) -> str:
        """ "
        Return the full image name with tag.
        """
        return f"{self.fqn}:{self.tag}"
k
yes, we don’t support using digest in the image name. contributions are welcome. we add a new class called
imageSpec
in flytekit, you can pass it to the task. then flytekit will help you build the image without writing a dockerfile. feel free to give it a try. here is an example. https://github.com/flyteorg/flytekit/pull/1555#issuecomment-1506384435
x
Awesome, I will try
imageSpec
. Thanks!
@Kevin Su One more question. If I define image in
@task
, how should I register the task using
remote.register_task
? Can I skip image config in
SerializationSettings
somehow? Because currently,
SerializationSettings
requires
image_config
k
yes, you can skip it. IIRC, The
image_config
in
SerializationSettings
will override the container image in the @task if you specify it.
x
ok, that’s great! thanks!
x
does the image need to pre-package the task code? for example
Copy code
@task(container_image=<container_image_digest>)
def t1():
    df = pd.DataFrame({"Name": ["Tom", "Joseph"], "Age": [20, 22]})
    print(df)
Will
container_image
need to contain
t1
code?
k
if you use fast-register (like pyflyte run), the image doesn’t need to contain code. flytekit will upload to s3 while registering, and download it when pod is running
x
got it. any example on using FastSerializationSettings?
k
you could just specify the
enabled
. others args aren’t necessarily need it.
x
Slightly different question, but still about registration, if i only set serialization settings for tasks but not workflow, workflow would error out. Is this expected? Example:
Copy code
...
def my_workflow(a: int, b: int, c: int) -> int:
    x = my_task(a=a, b=b)
    y = my_task(a=c, b=c)
    z = my_task(a=x, b=y)
    return my_other_task(m=x, n=z)
...
    remote.register_task(my_task, serialization_settings=ss, version=version)
    remote.register_task(my_other_task, serialization_settings=ss, version=version)

    registered_workflow = remote.register_workflow(my_workflow, version=version)
and error i got
Copy code
File "/data/home/xinzhou/.cache/bazel/_bazel_xinzhou/009814b602246296742ad72f1ace15d8/execroot/__main__/bazel-out/k8-fastbuild/bin/atomwise/feather/worker/run_my_workflow.runfiles/pip_pypi__flytekit/flytekit/remote/remote.py", line 626, in _serialize_and_register
    f"No serialization settings set, but workflow contains entities that need to be registered. {cp_entity.id.name}",
AttributeError: 'TaskSpec' object has no attribute 'id'
121 Views