Hello, I have a problem with using FlyteRemote to ...
# ask-the-community
e
Hello, I have a problem with using FlyteRemote to register my workflow where Flyte complains that there is no nodes in my workflow. My codebase is on a Jupyter Notebook. The structure of my code is as follow:
Copy code
my_package
|-- __init__.py
|-- data.py
|-- workflow.py
my_notebook.ipynb
My
workflow.py
is basically like this
Copy code
import pandas as pd
from flyte import task, workflow

from data import generate_data, normalize_data


@task
def load_data() -> pd.DataFrame:
    return generate_data()

@task
def preprocess_data(data: pd.DataFrame) -> pd.DataFrame:
    return normalize_data(data)

@workflow
def simple_workflow():
    data = load_data()
    preprocess_data(data)
In my notebook, I first add the path to my local package to my system path, then create a FlyteRemote instance and try to register the workflow
Copy code
import os
import sys
sys.path.append(os.getcwd())
sys.path.append(os.path.join(os.getcwd(), "my_package"))

from flytekit.remote import FlyteRemote
from flytekit.configuration import Config, PlatformConfig, ImageConfig, SerializationSettings
from flytekit.configuration import DataConfig, S3Config

remote = FlyteRemote(
    config=Config(
        platform=PlatformConfig(
            endpoint=f"dns:///{os.environ['FLYTE_ENDPOINT']}",
            insecure=True,
            insecure_skip_verify=True,
        ),
        data_config=DataConfig(s3=S3Config(
            endpoint=os.environ['AWS_S3_ENDPOINT'],
            enable_debug=True,
            access_key_id=os.environ['AWS_ACCESS_KEY_ID'],
            secret_access_key=os.environ['AWS_SECRET_ACCESS_KEY']
        ))
    ),
    default_project="my_project",
    default_domain="development",
    data_upload_location=os.environ['FLYTE_S3_BUCKET'],
)

from my_package.workflow import simple_workflow

flyte_workflow = remote.register_script(
    simple_workflow,
    image_config=ImageConfig.auto_default_image(),
    version="v1",
    module_name="my_package",
    source_path="./"
)

remote.execute(flyte_workflow, inputs={})
The error I've been facing is
Copy code
_InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
        status = StatusCode.INTERNAL
        details = "failed to compile workflow for [resource_type:WORKFLOW 
project:"7b33c23287ae4d2481160448171b307c" domain:"development" 
name:"my_package.worklow.simple_workflow" version:"v1" ] with err failed to compile workflow with 
err Collected Errors: 1
        Error 0: Code: NoNodesFound, Node Id: resource_type:WORKFLOW project:"7b33c23287ae4d2481160448171b307c" 
domain:"development" name:""my_package.worklow.simple_workflow" version:"v1" , Description: Can't 
find any nodes in workflow [resource_type:WORKFLOW project:"7b33c23287ae4d2481160448171b307c" domain:"development" 
name:""my_package.worklow.simple_workflow" version:"v1" ].
"
        debug_error_string = "UNKNOWN:Error received from peer  {grpc_message:"failed to compile workflow for 
[resource_type:WORKFLOW project:\"7b33c23287ae4d2481160448171b307c\" domain:\"development\" 
name:\""my_package.worklow.simple_workflow\" version:\"v1\" ] with err failed to compile workflow 
with err Collected Errors: 1\n\tError 0: Code: NoNodesFound, Node Id: resource_type:WORKFLOW 
project:\"7b33c23287ae4d2481160448171b307c\" domain:\"development\" 
name:\""my_package.worklow.simple_workflow\" version:\"v1\" , Description: Can\'t find any nodes in 
workflow [resource_type:WORKFLOW project:\"7b33c23287ae4d2481160448171b307c\" domain:\"development\" 
name:\"my_package.worklow.simple_workflow\" version:\"v1\" ].\n", grpc_status:13, 
created_time:"2023-06-08T10:45:49.81363907+00:00"}"
Any idea on the cause of this problem and how to resolve it? Flyte seems to require proper structure of python modules, which might not be the case in the Jupyter notebook. Thanks,
v
from flyte import task, workflow
Did you mean ‘flytekit’?
e
Yeah. It was meant to be flytekit
s
The error you're seeing crops up only when there are no tasks in the workflow.
103 Views