https://flyte.org logo
Powered by
Title
n

Nan Qin

04/12/2023, 8:27 PM
Hi I am trying to register a task using the FlyteRemote API. The task was successfully registered. However the 
task-module
is missing from the generated command args (see below). So executing the task leads to 
ValueError: Empty module name
args:[24 items
0:"pyflyte-fast-execute"
1:"--additional-distribution"
2:"<s3://protopia-stained-glass-shop-stage-01/flytesnacks/development/JAWX3GGV5URP7U>..."
3:"--dest-dir"
4:"{{ .dest_dir }}"
5:"--"
6:"pyflyte-execute"
7:"--inputs"
8:"{{.input}}"
9:"--output-prefix"
10:"{{.outputPrefix}}"
11:"--raw-output-data-prefix"
12:"{{.rawOutputDataPrefix}}"
13:"--checkpoint-path"
14:"{{.checkpointOutputPrefix}}"
15:"--prev-checkpoint"
16:"{{.prevCheckpointPrefix}}"
17:"--resolver"
18:"flytekit.core.python_auto_container.default_task_resolver"
19:"--"
20:"task-module"
21:""
22:"task-name"
23:"my_task"
]
here is the code to register the task. Am I doing something wrong here? 
import flytekit
import flytekit.remote
import flytekit.configuration as flyte_config
from flytekit.tools import repo
import uuid

@flytekit.task
def my_task():
    print('my task')


def main(remote_endpoint='dns:///localhost:8089'):
    remote = flytekit.remote.FlyteRemote(
        config=flyte_config.Config.for_endpoint(endpoint=remote_endpoint, insecure=True),
        default_project="flytesnacks",
        default_domain="development",
    )
    detected_root = repo.find_common_root(["."])
    print(f"detected root: {detected_root}")
    _, native_url = remote.fast_package(detected_root)
    fast_serialization_settings = flyte_config.FastSerializationSettings(
        enabled=True,
        destination_dir=".",
        distribution_location=native_url,
    )
    task = remote.register_task(
        my_task,
        version=str(uuid.uuid4()).lower().replace("-", ""),
        serialization_settings=flyte_config.SerializationSettings(
            image_config=flyte_config.ImageConfig.auto_default_image(),
            fast_serialization_settings=fast_serialization_settings,
        ),
    )
    print(task)


if __name__ == "__main__":
    main()
k

Ketan (kumare3)

04/12/2023, 9:03 PM
Weird
y

Yee

04/12/2023, 10:10 PM
how are you running this @Nan Qin 
python file.py
?
n

Nan Qin

04/12/2023, 10:11 PM
yes
y

Yee

04/12/2023, 10:11 PM
can you move that if block into another file? and try to import this original file there?
just wondering something.
n

Nan Qin

04/12/2023, 10:15 PM
yes that works
so the flyte entities have to be in a different python module when registering with the remote API?
y

Yee

04/12/2023, 10:31 PM
i thought i saw something about this recently… @Niels Bantilan do you remember something about this?
it’s just that when you run something directly, as the main module, the logic that detects what a task should be called gets messed up. we should properly log though.
normally when you run 
pyflyte run/register
right, user modules are loaded according to the python load paths.
and the names are detected based on how they were found.
but if you just start with main then we don’t know. we can just use “” also. not sure what the right behavior is here, but definitely need to be cleaner about it.
n

Nan Qin

04/13/2023, 1:38 AM
yeah make sense. Thank you for explaining
n

Niels Bantilan

04/13/2023, 12:59 PM
I don’t recall if we’ve seen this already
should we make a ticket to address this @Yee?
k

Ketan (kumare3)

04/13/2023, 1:41 PM
I do not remember too
This looks like a bug
y

Yee

04/13/2023, 4:51 PM
yeah would you mind cutting an issue for this @Nan Qin?
we will add to it what we know and take a deeper look as well
n

Nan Qin

04/14/2023, 4:13 PM
https://github.com/flyteorg/flyte/issues/3595
e

Erik Dao

05/29/2023, 8:01 AM
I have this issue as well. Hope the bug will soon be triaged.
k

Ketan (kumare3)

05/29/2023, 2:24 PM
This is odd, cc @Yee @Kevin Su . We will take a look
3 Views
#ask-the-communityJoin Slack
Powered by