https://flyte.org logo
#ask-the-community
Title
# ask-the-community
j

Joe Hartshorn

09/28/2023, 12:08 PM
Hi team, I was having some trouble with
map_task
and solved it so I thought I’d share the solution in case anyone else does too. I was using tasked wrapped in
partial
and the curried argument was a
FlyteFile
. Getting an error like
Copy code
TypeError: object of type 'FlyteFile' has no len()
Here’s an simplified example (task bodies omitted for brevity
Copy code
@task()
def download_files() -> list[FlyteFile]:
    ...
    return file_list


@task()
def get_checkpoint() -> FlyteFile:
    ...
    return checkpoint_file


@task()
def process_file(file: FlyteFile, model: FlyteFile) -> dict:
    ...
    return result


@workflow
def process_all_files() -> list[dict]:
    file_list = download_files()
    model = get_checkpoint()

    process_using_model = partial(
        process_file,
        model=model,
    )
    result = map_task(process_using_model)(file=file_list)

    return result
The error was in the task definition,
def process_file(model: FlyteFile, file: FlyteFile)
should have been
process_file(file: FlyteFile, model: FlyteFile)
. Notice how the arguments for the map task are defined after the argument which is supplied from the list. This was a tough one to work out and I think should either be changed to make it more generic, or mentioned in the docs.
k

Ketan (kumare3)

09/28/2023, 2:39 PM
Ohh are you saying this is an ordering or arguments problem
If so this is a bug should not be the case, reason why we use keywords args
n

Niels Bantilan

09/28/2023, 4:22 PM
Thanks for reporting this @Joe Hartshorn, this seems like a bug… can you open up an issue? [flyte-bug]
n

Niels Bantilan

09/29/2023, 6:56 PM
@Yee @Eduardo Apolinario (eapolinario)
@Joe Hartshorn just to confirm, you’re seeing this error in a local execution?
j

Joe Hartshorn

10/03/2023, 7:57 AM
Happy to open a bug! It’s not local, only remote, and it’s definitely an ordering issue. Edit: I have a minimal reproducible example I can post in a bug.