sparse-dream-26689
05/15/2024, 11:03 AMremote.register_script(...)
but I keep getting a TaskReferenceNotFound
error saying that workflows.s3_ingest.initialize_resources was not found. Similarly, when using remote.register_workflow(...)
I get an error on execution time that module workflows was not found. At the moment, I am attempting to run these workflows from a monorepo to utilize a local library imported with a relative import in poetry. Do I have to register each task on its own before registering the workflow with remote.register_script()
?sparse-dream-26689
05/15/2024, 11:25 AMsparse-dream-26689
05/15/2024, 12:27 PMregister_script()
. If I use register_workflow()
, registration works, but when executing the workflow, I get the ModuleNotFoundError
mentioned earlier.tall-lock-23197
register_workflow
?sparse-dream-26689
05/16/2024, 9:57 AMroot
|->other projects
|->...
|->backend
|->other python projects
|->common library
|->flyte-worker
|->worker
| |->workflows
| |-><workflow files>
|->main.py <-registered and executed here
sparse-dream-26689
05/16/2024, 10:03 AMtall-lock-23197
tall-lock-23197
from flytekit.remote import FlyteRemote
from flytekit.configuration import Config, ImageConfig
from wf.test_json import jsons_iter_wf
from flytekit.configuration import SerializationSettings, FastSerializationSettings
remote = FlyteRemote(config=Config.auto(), default_project="flytesnacks", default_domain="development")
_, native_url = remote.fast_package(root=".")
flyte_workflow = remote.register_workflow(
entity=jsons_iter_wf,
serialization_settings=SerializationSettings(
image_config=ImageConfig.auto(img_name="..."),
project="flytesnacks",
domain="development",
version="v1",
fast_serialization_settings=FastSerializationSettings(enabled=True, destination_dir=".", distribution_location=native_url),
),
)
^^^ register.pysparse-dream-26689
05/17/2024, 3:28 PMModuleNotFoundError
when executing the workflow. It's probably an error I made when setting up the package that I'm not seeing. I will circle back to this as I said previously, but for now I'm registering from the CLIprehistoric-kangaroo-27449
08/01/2024, 8:15 AM>v1.11