How to use multiple images with a config file in `pyflyte register`command? I am running, `pyflyte ...
s

Sanjay Chouhan

about 3 years ago
How to use multiple images with a config file in `pyflyte register`command? I am running,
pyflyte --config config.yaml register test1.py --version 1.0.1
I have created the code as mentioned in the doc, https://docs.flyte.org/projects/cookbook/en/latest/auto/core/containerization/multi_images.html# The config.yaml is,
admin:
  # For GRPC endpoints you might want to use dns:///flyte.myexample.com
  endpoint: dns:///##############-16##<http://26454.us-west-1.elb.amazonaws.com:80|26454.us-west-1.elb.amazonaws.com:80>
  authType: Pkce
  insecure: true
logger:
  show-source: true
  level: 0
images:
  trainer: <http://ghcr.io/flyteorg/flytecookbook:core-latest|ghcr.io/flyteorg/flytecookbook:core-latest>
  predictor: moulee31/sample:1.0
The error is,
Traceback (most recent call last):
  File "/home/sanjaychouhan/.local/bin/pyflyte", line 8, in <module>
    sys.exit(main())
  File "/usr/lib/python3/dist-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/usr/lib/python3/dist-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/lib/python3/dist-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/lib/python3/dist-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/sanjaychouhan/.local/lib/python3.8/site-packages/flytekit/clis/sdk_in_container/register.py", line 174, in register
    registerable_entities = load_packages_and_modules(
  File "/home/sanjaychouhan/.local/lib/python3.8/site-packages/flytekit/tools/repo.py", line 225, in load_packages_and_modules
    registrable_entities = serialize(pkgs_and_modules, ss, str(project_root), options)
  File "/home/sanjaychouhan/.local/lib/python3.8/site-packages/flytekit/tools/repo.py", line 54, in serialize
    registrable_entities = get_registrable_entities(ctx, options=options)
  File "/home/sanjaychouhan/.local/lib/python3.8/site-packages/flytekit/tools/serialize_helpers.py", line 75, in get_registrable_entities
    get_serializable(new_api_serializable_entities, ctx.serialization_settings, entity, options=options)
  File "/home/sanjaychouhan/.local/lib/python3.8/site-packages/flytekit/tools/translator.py", line 578, in get_serializable
    cp_entity = get_serializable_task(entity_mapping, settings, entity)
  File "/home/sanjaychouhan/.local/lib/python3.8/site-packages/flytekit/tools/translator.py", line 173, in get_serializable_task
    container = entity.get_container(settings)
  File "/home/sanjaychouhan/.local/lib/python3.8/site-packages/flytekit/core/python_auto_container.py", line 164, in get_container
    image=get_registerable_container_image(self.container_image, settings.image_config),
  File "/home/sanjaychouhan/.local/lib/python3.8/site-packages/flytekit/core/python_auto_container.py", line 235, in get_registerable_container_image
    raise AssertionError(f"Image Config with name {name} not found in the configuration")
AssertionError: Image Config with name trainer not found in the configuration
It was working with pyflyte run command,
pyflyte --config config.yaml run --remote test1.py test_workflow
Hi All, I am following the flyte deployment from Github and I am facing an issue setting up. Below i...
v

Vijay Venugopal

over 3 years ago
Hi All, I am following the flyte deployment from Github and I am facing an issue setting up. Below is the error from docker:
Timed out while waiting for the datacatalog rollout to be created
Terminated
Timed out while waiting for dockerd to start
Terminated
found several pods were in ContainerCreating status, when i logged in one of them - found some "MountVolume" error as shown below: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Scheduled 21m default-scheduler Successfully assigned flyte/flytepropeller-6f5c985cd4-kflzv to a7b7d18b2fc5 Warning FailedMount 21m (x2 over 21m) kubelet MountVolume.SetUp failed for volume "auth" : failed to sync secret cache: timed out waiting for the condition Warning FailedMount 21m (x2 over 21m) kubelet MountVolume.SetUp failed for volume "config-volume" : failed to sync configmap cache: timed out waiting for the condition Warning FailedMount 21m kubelet MountVolume.SetUp failed for volume "kube-api-access-jcdxv" : failed to sync configmap cache: timed out waiting for the condition Normal Pulling 21m kubelet Pulling image "cr.flyte.org/flyteorg/flytepropeller-release:v0.19.2" Warning FailedMount 5m19s kubelet MountVolume.SetUp failed for volume "config-volume" : failed to sync configmap cache: timed out waiting for the condition Warning FailedMount 5m12s (x4 over 5m19s) kubelet MountVolume.SetUp failed for volume "auth" : failed to sync secret cache: timed out waiting for the condition Normal Pulling 5m6s kubelet Pulling image "cr.flyte.org/flyteorg/flytepropeller-release:v0.19.2" Any pointer to fix such issues? I've tried removing image and redeploying without any luck.