I am trying to provide a tls certificate to hook with Ingress using https://github.com/flyteorg/flyte/blob/ca704e24daad628535a4414e0a7cb1164621105d/charts/flyte-core/values.yaml#L344-L347 While installing flyte via helm3 I get this error:

rror: unable to build kubernetes objects from release manifest: error validating "": error validating data: ValidationError(Ingress.spec.tls[0].hosts): unknown object type "nil" in Ingress.spec.tls[0].hosts[0]

When I render the template for helm I see:

tls:
    - secretName: my-secret-name
      hosts:
        - null

Why is it generating this hosts as null? How can I override/correct this hosts as there is no option to do that in the values-eks.yaml.

Hi, our team is trying to use fast registration using

pyflyte register

but hitting a wall. The registration passes and it's worklows and launchplans even shows up in Flyte Console however when executing it we get error:

ModuleNotFoundError: No module named 'flyte'

When looking at the resulted generated file on the S3 minio we find that it is completely empty. Our full command is:

pyflyte --config ~/.flyte/cloud-config.yaml register flyte/workflows -p {{.PROJECT}} --domain {{.DOMAIN}} -v {{add .DOCKER_VERSION 1}} -i "{{.CONTAINER_REGISTRY}}/{{.PROJECT}}:{{.DOCKER_VERSION}}"

Folder structure see image Note that using the

pyflyte package

and

flytectl register files

in the same project does work as expected. Are we missing something?

seeing a weird issue with flyteadmin when upgrading from 0.6.44 -> 0.6.104:

time="2022-02-28T20:30:40Z" level=info msg="Using config file: [/etc/flyte/config/allowed_origins.yaml /etc/flyte/config/cluster_resources.yaml /etc/flyte/config/core.yaml /etc/flyte/config/db.yaml /etc/flyte/config/domain.yaml /etc/flyte/config/logger.yaml /etc/flyte/config/server.yaml /etc/flyte/config/storage-limits.yaml /etc/flyte/config/storage.yaml /etc/flyte/config/task_resource_defaults.yaml]"
{"json":{"src":"migrate.go:107"},"level":"fatal","msg":"failed to connect to `host=cloud-sql-proxy user=flyte database=flyte`: server error (ERROR: SSL required (SQLSTATE 08P01))","ts":"2022-02-28T20:30:40Z"}

2022/02/28 20:30:40 /go/src/github.com/flyteorg/flyteadmin/pkg/repositories/database.go:102
[error] failed to initialize database, got error failed to connect to `host=cloud-sql-proxy user=flyte database=flyte`: server error (ERROR: SSL required (SQLSTATE 08P01))

is it possible to delete a project using

flytectl

? The documentation says that

flytectl delete

can do that (here), but then I run the tool, project is not listed in available commands:

flytectl delete --help

Delete a resource; if an execution:
::

 flytectl delete execution kxd1i72850  -d development  -p flytesnacks

Usage:
  flytectl delete [command]

Available Commands:
  cluster-resource-attribute Deletes matchable resources of cluster attributes.
  execution                  Terminates/deletes execution resources.
  execution-cluster-label    Deletes matchable resources of execution cluster label.
  execution-queue-attribute  Deletes matchable resources of execution queue attributes.
  plugin-override            Deletes matchable resources of plugin overrides.
  task-resource-attribute    Deletes matchable resources of task attributes.
  workflow-execution-config  Deletes matchable resources of workflow execution config.

I am evaluating Flyte for data engineering. When is Flyte not a good choice (compared to Dagster - another candidate)? The data science group uses Metaflow and they are actively advocating for us to adopt it too. Is Flyte comparable to data science tools like Metaflow apart from the common workflow orchestration bit?

Hi team.. I need some help. I have a flyte task that needs to read a parquet file stored in s3 (about 2 GB in size, 100k rows) and return 1000 records from it… I’m using awswrangler for reading the file into a pandas dataframe and then return the first 1000 records from it (list of dicts). Each task node has upto 8 GB in memory… but the pod keeps getting OOM killed with no useful info in the stack trace. Not sure what’s going on…

Hi, I’m getting the following error (failed task with a dataclass/dataclass_json as an input and tuple of numpy arrays as an output) but I can’t seem to make sense of it. Looks like something to do with the task input, but any help with what might be causing this error would be appreciated:

Traceback (most recent call last):

      File "/opt/venv/lib/python3.10/site-packages/flytekit/exceptions/scopes.py", line 165, in system_entry_point
        return wrapped(*args, **kwargs)
      File "/opt/venv/lib/python3.10/site-packages/flytekit/core/base_task.py", line 518, in dispatch_execute
        native_inputs = TypeEngine.literal_map_to_kwargs(exec_ctx, input_literal_map, self.python_interface.inputs)
      File "/opt/venv/lib/python3.10/site-packages/flytekit/core/type_engine.py", line 868, in literal_map_to_kwargs
        return {k: TypeEngine.to_python_value(ctx, lm.literals[k], python_types[k]) for k, v in lm.literals.items()}
      File "/opt/venv/lib/python3.10/site-packages/flytekit/core/type_engine.py", line 868, in <dictcomp>
        return {k: TypeEngine.to_python_value(ctx, lm.literals[k], python_types[k]) for k, v in lm.literals.items()}
      File "/opt/venv/lib/python3.10/site-packages/flytekit/core/type_engine.py", line 832, in to_python_value
        return transformer.to_python_value(ctx, lv, expected_python_type)
      File "/opt/venv/lib/python3.10/site-packages/flytekit/core/type_engine.py", line 606, in to_python_value
        return self._fix_dataclass_int(expected_python_type, self._deserialize_flyte_type(dc, expected_python_type))
      File "/opt/venv/lib/python3.10/site-packages/flytekit/core/type_engine.py", line 589, in _fix_dataclass_int
        dc.__setattr__(f.name, self._fix_val_int(f.type, val))
      File "/opt/venv/lib/python3.10/site-packages/flytekit/core/type_engine.py", line 568, in _fix_val_int
        ktype, vtype = DictTransformer.get_dict_types(t)

Message:

    not enough values to unpack (expected 2, got 0)

Hello, I have some python dependencies including custom packages to use in my flyte workflow. I want to first try them out on a local flyte cluster. How do I install them on the local flytesnack or demo cluster ? e.g. I have the dependencies scikit-learn, xgboost and some custom packages in a local wheel file I can do pip install -r requirements.txt --index-url https://maven.homebox.com/repository/max-pypi-snapshots/simple --extra-index-url https://maven.homebox.com/repository/max-pypi-releases/simple --use-pep517

Hey everyone 🙂 I'm currently deploying a flyte sandbox at my new company and am struggling with getting auth to work. I followed this guide. Flyte is deployed, I am redirected to the SSO login screen, after that I can view the flyte console 👍 I can also run

flytectl get projects

, am redirected to the browser, and then retrieve the projects.

pyflyte register ...

works as well. So far so good. When I manually launch a registered workflow from the console, it doesn't start however.

k -n development describe <http://flyteworkflows.flyte.lyft.com|flyteworkflows.flyte.lyft.com> ah9ghn272q7v22dbsvvd

gives:

Status:
  Failed Attempts:  9
  Message:          Workflow[] failed. ErrorRecordingError: failed to publish event, caused by: EventSinkError: Error sending event, caused by [rpc error: code = Unavailable desc = connection error: desc = "transport: authentication handshake failed: tls: first record does not look like a TLS handshake"]
  Phase:            0
Tasks:
  resource_type:TASK project:"3dod" domain:"development" name:"project.workflows.workflow.deploy_model" version:"2"

Details about config in 🧵 A pointer in the right direction would be much appreciated.

Can anyone help me debug why my task is stuck in Queueing state? When I look at the status of the pods it says they have Completed - including the pod corresponding to the task that is stuck Queueing in the dashboard - but the next tasks do not kick off.