@Sören Brunk have you started working on KerasCheckpoint wrapper? cc @Niels Bantilan?

Hello everyone! I'm testing out flyte and running it locally in sandbox mode. I have created a simple task that loads a 2 GB file in memory and prints some useful information about the file. At this point, I'm just loading the file in memory and not doing anything with it. As I try to run the task from the console, my pod gets killed with OOMKilled. How can I increase the resources available for pod created by Flyte? Thank you!

Hi! Is there an endpoint to list all executions across projects/domains? Like this, but without the project/domain parameters.

When using ImageSpec, can we do the image building remotely? Can we launch builtkitd in a K8s pod and send our build requests there? Possibly with

envd context

Hello Is there any way to use partial functions with map tasks, because it gives an error when I run the script on flyte cluster

ValueError: Only Flyte python task types are supported in map tasks currently, received <class 'functools.partial'>

hi team, I have this PR to support Union type in flyteidl. https://github.com/flyteorg/flyteidl/pull/401 However, i do have a few questions related to what I should generate when making the default value for Union types. Currently I am generating it as a Union Scalar with the first type available. However, I can also just generate the the literal for the type itself. Wondering if that make a difference when user creating the task using simple types but task is defined as Union type. An example:

@task
def task(a: Union[Integer, Float]):

and then I am current generating default value as Union scalar literal:

x:
        value:
            value:
                scalar:
                    value:
                        primitive:
                            value:
                                integer: 0
                            xxx_nounkeyedliteral: {}
                            xxx_unrecognized: []
                            xxx_sizecache: 0
                    xxx_nounkeyedliteral: {}
                    xxx_unrecognized: []
                    xxx_sizecache: 0
            hash: ""
            xxx_nounkeyedliteral: {}
            xxx_unrecognized: []
            xxx_sizecache: 0
        type:
            type:
                simple: 1
            metadata: null
            annotation: null
            structure:
                tag: int
                xxx_nounkeyedliteral: {}
                xxx_unrecognized: []
                xxx_sizecache: 0
            xxx_nounkeyedliteral: {}
            xxx_unrecognized: []
            xxx_sizecache: 0
        xxx_nounkeyedliteral: {}
        xxx_unrecognized: []
        xxx_sizecache: 0

or I can generate default as simple integer literal like this:

"x": 0

@Dan Rammer (hamersaw) @Ketan (kumare3) @Kevin Su

Hi team, I get the following error when I try to pass an array (which is a List[DataClass]) of length 15000 to a task.

TooLarge: Event message exceeds maximum gRPC size limit, caused by [rpc error: code = ResourceExhausted desc = grpc: received message larger than max (6020936 vs. 4194304)].

How can I address this? where can I increase this size limit and how large is the recommended size be?

Is there a straightforward way to either override or set a template for the pod name for dynamic steps? It'd be very helpful to minimally be able to name the pods associated w/ a dynamic step with a common prefix for the workflow (rather than a random set of characters)

Hi Folks, We need to be able to provide different images for different tasks in a workflow, so I am testing the Multiple Container Images in a Single Workflow feature. I am using the Whylogs example. It won't work as is in a remote cluster. So, I added the flytecoobook whylogs container image to each task to be able to run the workflow successfully in a remote cluster with

pyflyte run --remote whylogs_example wf

@task(container_image="<http://ghcr.io/flyteorg/flytecookbook:whylogs_examples-latest|ghcr.io/flyteorg/flytecookbook:whylogs_examples-latest>")
def get_reference_data() -> pd.DataFrame:
    ...

@task(container_image="<http://ghcr.io/flyteorg/flytecookbook:whylogs_examples-latest|ghcr.io/flyteorg/flytecookbook:whylogs_examples-latest>")
def get_target_data() -> pd.DataFrame:
    ...

@task(container_image="<http://ghcr.io/flyteorg/flytecookbook:whylogs_examples-latest|ghcr.io/flyteorg/flytecookbook:whylogs_examples-latest>")
def create_profile_view(df: pd.DataFrame) -> DatasetProfileView:
    ...

@task(container_image="<http://ghcr.io/flyteorg/flytecookbook:whylogs_examples-latest|ghcr.io/flyteorg/flytecookbook:whylogs_examples-latest>")
def constraints_report(profile_view: DatasetProfileView) -> bool:
    ...

However, the

get_reference_data

and

get_target_data

should not need whylogs. They just work with pandas and scikit-learn. We should be able to run those tasks with the

@task(container_image="<http://ghcr.io/flyteorg/flytecookbook:core-latest|ghcr.io/flyteorg/flytecookbook:core-latest>")

image. I did try that but it fails, k8s logs say:

File "/opt/venv/lib/python3.8/site-packages/flytekit/core/python_auto_container.py", line 279, in load_task
    task_module = importlib.import_module(name=task_module)  # type: ignore
  ...
  File "/root/whylogs_example.py", line 17, in <module>
    import whylogs as why
ModuleNotFoundError: No module named 'whylogs'
Traceback (most recent call last):

Every task container is trying to parse the entire whyogs_example.py file and since

fytecookbook:core-latest

does not have whylogs it is failing. What is the best design pattern or strategy to be followed in such cases ? How can I make it work remotely ? I read the containerization/multi_images.html, that example has two methods

svm_trainer

and

svm_predictor

but both end up using the same image. All the examples I see in https://github.com/flyteorg/flytesnacks/tree/master/cookbook/case_studies/ml_training also have only one custom docker file. Is there a production grade example workflow with tasks taking different images which are significantly different with each other ? Looking for a reference complex workflow that talks about these nuances on how to organize the pieces together with different custom images for each task.

Hi, 🙂 I am trying to configure a private container registry. What worked for me was to add the imagePullSecret to the service account as described here: https://docs.flyte.org/projects/cookbook/en/latest/auto/core/containerization/private_images.html It works when I add the imagePullSecret to the project-specific service account. However, I want to use a global (flyte-wide) service account instead so that I do not have to patch the service account on the cluster for every new Flyte project. I tried using the --k8sServiceAccount parameter with flytectl register. However, it always preprends the project-specific service account. E.g. if I set '--k8sServiceAccount flyte-backend-flyte-binary', then Flyte reports ''[1/1] currentAttempt done. Last Error: USER::pods "apqwf5zfxrj8sbpnlhtv-n0-0" is forbidden: error looking up service account flyte-poc-development/flyte-backend-flyte-binary: serviceaccount "flyte-backend-flyte-binary" not found" i.e., Flyte preprends the project-specific service account to my custom servicer account and, consequently, cannot find it. Is there a way to get this done with --k8sServiceAccount? Or will I have to use custom pod templates?