Peter Klingelhofer
04/12/2023, 9:00 PM\n
after `@workflow`(unsurprising as Jupyter Notebooks are typically run in the browser obviously), not sure if that could be causing the problem.Samhita Alla
04/13/2023, 6:24 AMPeter Klingelhofer
04/13/2023, 12:14 PMrepo
, and I have the ray_example.ipynb
file in the workflows
folder, and I'm trying to add the workflow to the development
domain.
I was adding a new separate cell to the bottom of the Jupyter Notebook ray_example.ipynb
file like so:
from flytekit.remote import FlyteRemote
from flytekit.configuration import Config, SerializationSettings, ImageConfig
# Using image pushed to local registry at localhost:30000
img = ImageConfig.from_images(
"localhost:30000/repo:latest", {"repo": "localhost:30000/repo:latest"}
)
# FlyteRemote object is the main entrypoint to API
remote = FlyteRemote(
config=Config.for_sandbox(),
default_project="repo",
default_domain="development",
)
# Get Task
# flyte_task = remote.fetch_task(name="workflows.ray_example", version="v1")
flyte_task = remote.fetch_task(
name="workflows.ray_example",
version="v1",
project="repo",
domain="development",
)
flyte_task = remote.register_task(
entity=flyte_task,
serialization_settings=SerializationSettings(image_config=None),
version="v2",
)
flyte_workflow = remote.register_workflow(
entity=flyte_task,
serialization_settings=SerializationSettings(image_config=None),
version="v1",
)
flyte_launch_plan = remote.register_launch_plan(entity=flyte_task, version="v1")
Yet I still receive the FlyteEntityNotExistException
. Apologies if the answer is obvious. Thank you again so much for any help/assistance you can provide!Samhita Alla
04/14/2023, 10:08 AMPeter Klingelhofer
04/15/2023, 1:11 PMflyte_task = remote.register_task(
entity=flyte_task,
serialization_settings=SerializationSettings(image_config=None),
version="v2",
)
Interestingly, this user suggests that registering workflows inside Jupyter notebooks is not possible: https://github.com/flyteorg/flyte/issues/3588#issuecomment-1509599891
If it is indeed possible, I would be happy to work on an MR to add an example Jupyter Notebook file to the Ray example (https://docs.flyte.org/projects/cookbook/en/latest/auto/integrations/kubernetes/ray_example/ray_example.html), just need to figure out how to get an example workflow working via a Jupyter Notebook. I'm just pushing the Docker image to the local registry at localhost:30000
, which is what I would think would be the simplest implementation possible to run a workflow.Samhita Alla
04/17/2023, 4:55 AMpyflyte run
or pyflyte register
. You need to use FlyteRemote to register your code. Can you try registering by following the example I've sent earlier?Peter Klingelhofer
04/17/2023, 1:45 PMpyflyte run
or pyflyte register
, and I don't really see in the documentation regarding FlyteRemote what the equivalent commands would be to register workflows via FlyteRemote would be. If I'm using FlyteRemote, what command would need to be run to register workflows, since we can't use pyflyte register
? Apologies again for the confusion on my part.Samhita Alla
04/17/2023, 3:18 PMregister_task
/ register_workflow
/ register_launch_plan
/ register_script
function. FlyteRemote is a Python API. You can use it to programmatically register your code.
https://github.com/flyteorg/flytekit/blob/e865db57d3bfbb7fb997417b052a05bc871cb0ed/flytekit/remote/remote.pyPeter Klingelhofer
04/17/2023, 3:58 PM