@ancient-wolf-19325 What could be the root cause of the following error (related to

CreateLaunchPlan

gpc call)?

../pip_flytekit/site-packages/flytekit/remote/remote.py:788: in _serialize_and_register
    self.raw_register(cp_entity, serialization_settings, version, og_entity=entity)
../pip_flytekit/site-packages/flytekit/remote/remote.py:720: in raw_register
    self.client.create_launch_plan(lp_entity.id, lp_entity.spec)
../pip_flytekit/site-packages/flytekit/clients/friendly.py:336: in create_launch_plan
    super(SynchronousFlyteClient, self).create_launch_plan(
../pip_flytekit/site-packages/flytekit/clients/raw.py:282: in create_launch_plan
    return self._stub.CreateLaunchPlan(launch_plan_create_request, metadata=self._metadata)
../pip_grpcio/site-packages/grpc/_interceptor.py:247: in __call__
    response, ignored_call = self._with_call(request,
../pip_grpcio/site-packages/grpc/_interceptor.py:287: in _with_call
    call = self._interceptor.intercept_unary_unary(continuation,
../pip_flytekit/site-packages/flytekit/clients/grpc_utils/wrap_exception_interceptor.py:46: in intercept_unary_unary
    raise e
../pip_flytekit/site-packages/flytekit/clients/grpc_utils/wrap_exception_interceptor.py:42: in intercept_unary_unary
    self._raise_if_exc(request, e)
Type mismatch for Parameter train_test_holdout_split__partitions in default_inputs has type union_type:{variants:{simple:INTEGER  structure:{tag:"int"}}  variants:{collection_type:{simple:FLOAT}  structure:{tag:"Typed List"}}}, expected simple:NONE"

For context, the code was trying to register an imperative workflow including a task `train_test_holdout_split`(alias of

random

).

@aw_task(cache=True, cache_version="1.0")
def random(
    dfs: list[pd.DataFrame],
    partitions: Union[int, list[float]],
    stratified_columns: list[str] = [],
    with_replacement: bool = False,
    random_seed: int = 528491,
) -> list[CacheableDataFrame]:
   ...

flytekit version:

1.13.7

, flyte version:

1.13.1

The root cause of the error is a type mismatch for the parameter

train_test_holdout_split__partitions

in the default inputs of the

CreateLaunchPlan

gRPC call. The error message indicates that the parameter has a type of

union_type

with variants

simple:INTEGER

and

collection_type:simple:FLOAT

, but the expected type is

simple:NONE

. This issue is likely due to the way the

partitions

parameter is defined in the

random

task. The

partitions

parameter is defined as

Union[int, list[float]]

, which results in a union type that Flyte is having trouble handling correctly during the launch plan creation. To resolve this issue, you can try the following steps: 1. Ensure Consistent Types: Make sure that the types used in the task definition are consistent and supported by Flyte. You might need to avoid using

Union

types directly and instead handle the type logic within the task. 2. Explicit Type Handling: Modify the task to handle the type logic explicitly within the task code, rather than relying on

Union

types in the function signature. Here is an example of how you might modify the task to handle the type logic explicitly:

from typing import List, Union
import pandas as pd
from flytekit import task

@task(cache=True, cache_version="1.0")
def random(
    dfs: List[pd.DataFrame],
    partitions: Union[int, List[float]],
    stratified_columns: List[str] = [],
    with_replacement: bool = False,
    random_seed: int = 528491,
) -> List[CacheableDataFrame]:
    if isinstance(partitions, int):
        # Handle integer partitions
        pass
    elif isinstance(partitions, list):
        # Handle list of float partitions
        pass
    # Task logic here
    ...

By handling the type logic within the task, you can avoid the type mismatch issue during the launch plan creation. Would you like more details on how to implement this or any other aspect of Flyte? Sources: - Flyte Issue #4889 👍 0 button 👎 1 button 💡 Hint: Mention @ancient-wolf-19325 in the thread for followups.

The suggested solution didn’t work. The workaround I discovered is to wrap the Union type param (`partitions`in this case) with

Optional

. @ancient-wolf-19325