https://flyte.org logo
#flytekit
Title
# flytekit
m

Matheus Moreno

07/20/2022, 8:24 PM
Hi, everyone. I was wondering: is there a way to programatically register a package of tasks/workflows? Using
FlyteRemote
and not
flytectl
, that is. My use case is that I want to develop a Python script that executes the entire deployment process, but I do not want to call subprocesses, since that would require the user to install
flytectl
(and some members of our team use Windows!). I know that
FlyteRemote
can register individual tasks and workflows, but is there a way to register an entire package with it?
k

Kevin Su

07/20/2022, 9:57 PM
we don’t support that. if you want to register all the tasks and workflows in the folder, you can do something like below
Copy code
from flytekit.clis.sdk_in_container.run import get_entities_in_file, load_naive_entity
import os

dev_dir = "development/workflow/"
for file in os.listdir(dev_dir):
    file_name = dev_dir + file
    rel_path = os.path.relpath(file_name)
    entities = get_entities_in_file(file_name)
    for wf in entities.workflows:
        module = os.path.splitext(rel_path)[0].replace(os.path.sep, ".")
        exe_entity = load_naive_entity(module, wf)
        remote.register_workflow(exe_entity)
e

Eduardo Apolinario (eapolinario)

07/20/2022, 10:06 PM
@Matheus Moreno, we ship windows binaries for
flytectl
as well.
Having said that, extending flyteremote to cover this case seems reasonable. Mind filing an issue?
y

Yee

07/20/2022, 10:49 PM
what’s a package?
like a python package? a folder with an init file?
can you make a ticket for this @Matheus Moreno?
it’s basically just this right?
but in programmatic form
not in click form
i don’t know if the entire logic should be encapsulated in the FlyteRemote class though, something to think about.
👍 2
k

Ketan (kumare3)

07/21/2022, 1:52 AM
Why can't this be done programmatically
This is sad @Yee
Our goal was to make everything programmatic
y

Yee

07/21/2022, 1:54 AM
We just need to factor out the function.
It can be
k

Ketan (kumare3)

07/21/2022, 1:55 AM
Let's do It @Matheus Moreno open to contributing
y

Yee

07/21/2022, 2:06 AM
yeah it shouldn’t be too hard, also related to this issue which we already have.
basically just moving most of the logic from the click function into a FlyteRemote function, and then invoking it from the click function
m

Matheus Moreno

07/21/2022, 1:29 PM
Yeah, it's pretty much the
register
command @Yee mentioned. I didn't know about it!
I could take a look and try to contribute. I'd just need to look how FlyteRemote works under the hood
k

Ketan (kumare3)

07/21/2022, 6:58 PM
ya flyteremote just wraps all the grpc apis
27 Views