Hi, I an trying to do scheduling workflows by foll...
# ask-the-community
r
Hi, I an trying to do scheduling workflows by following this https://docs.flyte.org/projects/cookbook/en/latest/auto/core/scheduled_workflows/lp_schedules.html From

this youtube

,
kubectl get po -n flyte
shall show scheduler pod, but for me I cannot find such pods. So I get stacked now. Could you give me some advice?
Copy code
$ kubectl get po -n flyte 
NAME                                                  READY   STATUS    RESTARTS      AGE
flyte-sandbox-proxy-d95874857-m8hlt                   1/1     Running   0             46m
flyte-sandbox-kubernetes-dashboard-6757db879c-867nz   1/1     Running   0             46m
flyte-sandbox-docker-registry-78686dccc-vd666         1/1     Running   0             46m
flyte-sandbox-postgresql-0                            1/1     Running   0             46m
flyte-sandbox-c744b94c5-5mp68                         1/1     Running   1 (45m ago)   46m
flyte-sandbox-minio-645c8ddf7c-4vwxr                  1/1     Running   0             46m
k
Current sandbox is actually built from single Flyte binary. Flyte scheduler is rolled into the same binary
Are schedules not running
Did you register
r
Did you register
Sorry I cannot get the meaning. How can I register? And what I should register?
k
Please share a code snippet
Also AFK, will try to answer once at the desk. Maybe @Samhita Alla / or some other community member can help
r
Copy code
# <https://docs.flyte.org/projects/cookbook/en/latest/auto/core/scheduled_workflows/lp_schedules.html#activating-a-schedule>

from datetime import datetime

from flytekit import task, workflow


@task
def format_date(run_date: datetime) -> str:
    return run_date.strftime("%Y-%m-%d %H:%M")


@workflow
def date_formatter_wf(kickoff_time: datetime):
    formatted_kickoff_time = format_date(run_date=kickoff_time)
    print(formatted_kickoff_time)

from flytekit import CronSchedule, LaunchPlan  # noqa: E402

# creates a launch plan that runs every minute.
cron_lp = LaunchPlan.get_or_create(
    name="my_cron_scheduled_lp",
    workflow=date_formatter_wf,
    schedule=CronSchedule(
        # Note that the ``kickoff_time_input_arg`` matches the workflow input we defined above: kickoff_time
        # But in case you are using the AWS scheme of schedules and not using the native scheduler then switch over the schedule parameter with cron_expression
        schedule="*/1 * * * *",  # Following schedule runs every min
        kickoff_time_input_arg="kickoff_time",
    ),
)
Then I run,
Copy code
$ flytectl update launchplan -p flyteexamples -d development {{ cron_lp }} --version v1 --activate


INFO[0000] [0] Couldn't find a config file []. Relying on env vars and pflags. 
Error: launch plan name wasn't passed

{"json":{},"level":"error","msg":"launch plan name wasn't passed\n","ts":"2023-03-12T13:08:28+09:00"}
k
You have to run pyflyte register
Or package and register
r
Which module?
Copy code
$ pyflyte register
Usage: pyflyte register [OPTIONS] [PACKAGE_OR_MODULE]...

  This command is similar to package but instead of producing a zip file, all
  your Flyte entities are compiled, and then sent to the backend specified by
  your config file. Think of this as combining the pyflyte package and the
  flytectl register step in one command. This is why you see switches you'd
  normally use with flytectl like service account here.

  Note: This command runs "fast" register by default. This means that a zip is
  created from the detected root of the packages given, and uploaded. Just
  like with pyflyte run, tasks registered from this command will download and
  unzip that code package before running.

  Note: This command only works on regular Python packages, not namespace
  packages. When determining       the root of your project, it finds the
  first folder that does not have an __init__.py file.

Options:
  -p, --project TEXT          Project to register and run this workflow in
  -d, --domain TEXT           Domain to register and run this workflow in
  -i, --image TEXT            A fully qualified tag for an docker image, e.g.
                              <http://somedocker.com/myimage:someversion123|somedocker.com/myimage:someversion123>. This is a
                              multi-option and can be of the form --image
                              <http://xyz.io/docker:latest|xyz.io/docker:latest> --image
                              my_image=<http://xyz.io/docker2:latest|xyz.io/docker2:latest>. Note, the
                              `name=image_uri`. The name is optional, if not
                              provided the image will be used as the default
                              image. All the names have to be unique, and thus
                              there can only be one --image option with no
                              name.
  -o, --output DIRECTORY      Directory to write the output zip file
                              containing the protobuf definitions
  -D, --destination-dir TEXT  Directory inside the image where the tar file
                              containing the code will be copied to
  --service-account TEXT      Service account used when creating launch plans
  --raw-data-prefix TEXT      Raw output data prefix when creating launch
                              plans, where offloaded data will be stored
  -v, --version TEXT          Version the package or module is registered with
  --deref-symlinks            Enables symlink dereferencing when packaging
                              files in fast registration
  --non-fast                  Enables to skip zipping and uploading the
                              package
  --dry-run                   Execute registration in dry-run mode. Skips
                              actual registration to remote
  --help                      Show this message and exit.
k
Pyflyte register file.py is ok too
r
Thank you but I see
This launch plan has no fixed inputs
in the condole. I missed something?
k
You launchplan declaration does not declare fixed inputs - so you are ok
r
schedule="*/1 * * * *" triggers workflows every one minute. But I cannot see running workflow. Is it okay?
s
Oh. You should be able to see them in the Workflows tab. Have you activated your schedule, @Ryo M?
r
$ flytectl update launchplan -p flyteexamples -d development {{ my_cron_scheduled_lp }} --version v1 --activate
INFO[0000] [0] Couldn't find a config file []. Relying on env vars and pflags.
Error: launch plan name wasn't passed
{"json":{},"level":"error","msg":"launch plan name wasn't passed\n","ts":"2023-03-12T173152+09:00"}
I got the error. From the error message, I missed something?
s
Can you pass the name of the launch plan to that command? Since your launch plan name is
my_cron_scheduled_lp
, the command would be
flytectl update launchplan -p flyteexamples -d development my_cron_scheduled_lp --version v1 --activate
r
Thank you I tried.
Copy code
$ flytectl update launchplan -p flyteexamples -d development my_cron_scheduled_lp --version v1 --activate
INFO[0000] [0] Couldn't find a config file []. Relying on env vars and pflags. 
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1ade323]

goroutine 1 [running]:
<http://github.com/flyteorg/flytectl/cmd/core.CommandContext.AdminClient(...)|github.com/flyteorg/flytectl/cmd/core.CommandContext.AdminClient(...)>
        /home/runner/work/flytectl/flytectl/cmd/core/cmd_ctx.go:57
<http://github.com/flyteorg/flytectl/cmd/update.updateLPFunc({0x2561ab8|github.com/flyteorg/flytectl/cmd/update.updateLPFunc({0x2561ab8>, 0xc0000540d0}, {0xc0000e1200, 0x1, 0x2561760?}, {0x0, {0x0, 0x0}, {0x0, 0x0}, ...})
        /home/runner/work/flytectl/flytectl/cmd/update/launch_plan.go:60 +0x1e3
<http://github.com/flyteorg/flytectl/cmd/core.generateCommandFunc.func1(0xc0009c3180|github.com/flyteorg/flytectl/cmd/core.generateCommandFunc.func1(0xc0009c3180>?, {0xc0000e1200, 0x1, 0x8})
        /home/runner/work/flytectl/flytectl/cmd/core/cmd.go:70 +0x93d
<http://github.com/spf13/cobra.(*Command).execute(0xc0009c3180|github.com/spf13/cobra.(*Command).execute(0xc0009c3180>, {0xc0000e1180, 0x8, 0x8})
        /home/runner/go/pkg/mod/github.com/spf13/cobra@v1.4.0/command.go:856 +0x67c
<http://github.com/spf13/cobra.(*Command).ExecuteC(0xc000922a00)|github.com/spf13/cobra.(*Command).ExecuteC(0xc000922a00)>
        /home/runner/go/pkg/mod/github.com/spf13/cobra@v1.4.0/command.go:974 +0x3bd
<http://github.com/spf13/cobra.(*Command).Execute(...)|github.com/spf13/cobra.(*Command).Execute(...)>
        /home/runner/go/pkg/mod/github.com/spf13/cobra@v1.4.0/command.go:902
<http://github.com/flyteorg/flytectl/cmd.ExecuteCmd()|github.com/flyteorg/flytectl/cmd.ExecuteCmd()>
        /home/runner/work/flytectl/flytectl/cmd/root.go:137 +0x1e
main.main()
        /home/runner/work/flytectl/flytectl/main.go:12 +0x1d
s
Can you share the flyte config.yaml config?
d
@Ryo M the config.yaml is usually located in
$HOME/.flyte/
r
Can you share the flyte config.yaml config?
Copy code
ryo@ryo:~/.flyte$ cat config-sandbox.yaml 
admin:
  # For GRPC endpoints you might want to use dns:///flyte.myexample.com
  endpoint: localhost:30080
  authType: Pkce
  insecure: true
console:
  endpoint: <http://localhost:30080>
logger:
  show-source: true
k
When you started sandbox it will ask you to export an env car
So that it can find the config file. As the name of config is sandbox-config.yaml. This is done so that you can use a default config - maybe you have a Flyte installed on cloud etc
export FLYTECTL_CONFIG=~/.flyte/config-sandbox.yaml
153 Views