https://flyte.org logo
Join Slack
Powered by
New RunLLM response generated: >>>*# Question* Fa...
# runllm-flyte-trial
a
New RunLLM response generated: >>>*# Question* Facing an issue with registering the workflow via FlyteRemote. This issue only appears with flytekit versions 
> 1.11
Trace:
Copy code
(Pdb) remote.register_workflow(main_workflow, serialization_settings=settings, version=pipeline_version)
*** flytekit.exceptions.user.FlyteInvalidInputException: USER:BadInputToAPI: error=None, cause=<_InactiveRpcError of RPC that terminated with:
        status = StatusCode.INVALID_ARGUMENT
        details = "failed to compile workflow for [resource_type:WORKFLOW project:"default" domain:"development" name:"some_wf.workflow.pipeline" version:"miha-flytekit-bugfix-ff46438-dirty-A7NQV"] with err failed to compile workflow with err Collected Errors: 3
        Error 0: Code: TaskReferenceNotFound, Node Id: start-node, Description: Referenced Task [resource_type:TASK name:"some_wf.workflow.create_config" version:"miha-flytekit-bugfix-ff46438-dirty-A7NQV"] not found.
However, I can see that all the tasks are successfully registered:
Copy code
❯ flytectl get tasks -p default -d development --filter.fieldSelector "version=miha-flytekit-bugfix-ff46438-dirty-A7NQV"
 ------------------------------------------ --------------------------------- ------------- --------------------------- --------------------------- -------------- ------------------- ----------------------------- 
| VERSION                                  | NAME                            | TYPE        | INPUTS                    | OUTPUTS                   | DISCOVERABLE | DISCOVERY VERSION | CREATED AT                  |
 ------------------------------------------ --------------------------------- ------------- --------------------------- --------------------------- -------------- ------------------- ----------------------------- 
| miha-flytekit-bugfix-ff46438-dirty-A7NQV | some_wf.workflow.train_task     | python-task | cfg: trainer config       | o0: uri of the model      |              | 1.0               | 2024-08-01T08:06:46.567848Z |
 ------------------------------------------ --------------------------------- ------------- --------------------------- --------------------------- -------------- ------------------- ----------------------------- 
| miha-flytekit-bugfix-ff46438-dirty-A7NQV | some_wf.workflow.create_config  | python-task | cfg: [description]. De... | o0: [description]         |              | 1.0               | 2024-08-01T08:06:46.511291Z |
|                                          |                                 |             |                           | o1: [description]         |              |                   |                             |
 ------------------------------------------ --------------------------------- ------------- --------------------------- --------------------------- -------------- ------------------- -----------------------------
>>```
| miha-flytekit-bugfix-ff46438-dirty-A7NQV | some_wf.workflow.evaluator_task | python-task | cfg: evaluator config | o0: result of the eval... | | 1.0 | 2024-08-01T080646.453174Z | | | | | model_uri: uri of the ... | | | | | ------------------------------------------ --------------------------------- ------------- --------------------------- --------------------------- -------------- ------------------- -----------------------------
Copy code
```

*# Answer*
The error you're encountering, `TaskReferenceNotFound`, indicates that the workflow is referencing a task that FlyteAdmin cannot find, even though the tasks appear to be registered. This issue can occur due to a mismatch in the task version or an issue with the registration process.

### Steps to Resolve:

1. **Ensure Consistent Versions**:
   Make sure that the version specified during the registration of tasks and workflows is consistent. Any mismatch can lead to the `TaskReferenceNotFound` error.

2. **Register Tasks and Workflows Separately**:
   Register tasks and workflows separately to ensure that all tasks are correctly registered before the workflow registration.

3. **Use `pyflyte` for Registration**:
   Instead of using `FlyteRemote.register_workflow`, try using `pyflyte` for registration. This method ensures that all dependencies are correctly packaged and registered.

### Example Commands:

1. **Register Tasks**:
FLYTE_SDK_LOGGING_LEVEL=20 pyflyte register --non-fast -p default --domain development --version miha-flytekit-bugfix-ff46438-dirty --tasks-only
Copy code
2. **Register Workflows**:
FLYTE_SDK_LOGGING_LEVEL=20 pyflyte register --non-fast -p default --domain development --version miha-flytekit-bugfix-ff46438-dirty --workflows-only
Copy code
3. **Register Launch Plans**:
FLYTE_SDK_LOGGING_LEVEL=20 pyflyte register --non-fast -p default --domain development --version miha-flytekit-bugfix-ff46438-dirty --launchplans-only ``` ### Additional Context: According to [this GitHub issue](https://github.com/flyteorg/flyte/issues/3658), tasks within dynamic workflows are not automatically registered. Ensure that all tasks, including those within dynamic workflows, are explicitly registered. If the issue persists, it might be related to a bug in the Flyte version you're using. Consider downgrading to a version prior to 
1.11
or checking for any updates or patches that address this issue. Would you like more detailed steps or further assistance? ### Sources: - [GitHub Issue #3658](https://github.com/flyteorg/flyte/issues/3658) - [GitHub Issue #2508](https://github.com/flyteorg/flyte/issues/2508) - [Flyte Documentation on Registering Workflows](https://github.com/flyteorg/flyte/blob/master/docs/flyte_fundamentals/registering_workflows.md)
>>Slack Link
https://flyte-org.slack.com/archives/CP2HDHKE1/p1722500851.260079 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.
Open in Slack
PreviousNext
Powered by