Curious if there are plans/interest to support callables are arguments? What I am really trying to do: I have a generic preprocessing task which is shared by many other teams, and based on the type of ML model we are training, it needs to invoke different preprocessing functions A simplified example:

class ExperimentType(Enum):
    FRAUD: "fraud"
    SIMILARITY: "similarity"

@task
def preprocess(data_path: str, experiment_type: ExperimentType):
    # common preprocessing
    data_path = common_preprocessing(data_path)

    # experiment specific preprocessing
    if experiment_type == ExperimentType.FRAUD:
        preprocessed_path = fraud_preprocessing(data_path)
    elif experiment_type == ExperimentType.SIMILARITY:
        preprocessed_path = similarity_preprocessing(data_path)

    # another step of processing
    return final_preprocessing(preprocessed_path)

The hard thing right now is to maintain the enums and threading those through the entire pipeline + if-else ladders. If we can pass a function to the preprocess task, that would make the code quite a lot simpler.

We definitely are interested but not prioritized

We are excited about eager mode! But even in eager mode, flyte needs a way to serde the callable, right?

Ohh you want it to be user passable

Yes, Flyte already knows how to find tasks using the task resolver, I was wondering if something similar could be used for passing vanilla python functions as task arguments • It would be useful even if the passable functions are restricted to not have references to globals or closures

You should be able to use pickle already

able to use pickle already

Say more? You mean use the PickleTransformer?

Types that aren't Flyte-native are pickled by default.