Hi everyone. Ive been working off the PR that adds a transformer for pydantic basemodels (https://github.com/flyteorg/flytekit/pull/1620) running into an unfortunate issue with flytepaths/directories though. These inherit from os.PathLike which falls outside of pydantic's supported json serialization/deserialization schemas. my understanding is also that flytefiles/directories need to be initialized in a particular way to work (below iv'e implemented a flytedirectory initialized inspired from the dataclass transformer). This works but seems a bit hacky so wanted to hear if anyone else encountered similar challenges. TLDR; how to serialize-deserialize flytedirectories/files in pydantic basemodels (or other objects)

from data_engine import (
    pydantic_transformer,
)  # code from <https://github.com/flyteorg/flytekit/pull/1620>


class Rax(pydantic.BaseModel):
    flytedir: str

    @pydantic.validator("flytedir")
    def validate_hax(cls, v: str) -> str:
        flytedir = make_flytedir(v)
        return str(flytedir.path)

    @classmethod
    def from_json(cls, json: str) -> "Self":
        self = cls.parse_raw(json)
        return self


@flytekit.workflow
def test_wf(rax: Rax) -> str:
    return test(rax=rax)  # type: ignore


@flytekit.task
def test(rax: Rax) -> str:
    return str(os.listdir(rax.flytedir))


def make_flytedir(path: Union[str, os.PathLike]) -> directory.FlyteDirectory:
    context = context_manager.FlyteContext.current_context()
    dimensionality = core_types.BlobType.BlobDimensionality.MULTIPART
    literal = make_literal(uri=path, dimensionality=dimensionality)
    transformer = directory_types.FlyteDirToMultipartBlobTransformer()
    out_dir = transformer.to_python_value(context, literal, directory.FlyteDirectory)
    os.listdir(out_dir)  # the dir isnt synced if we dont do this
    return out_dir


def make_literal(
    uri: Union[str, os.PathLike],
    dimensionality,
) -> literals.Literal:
    scalar = make_scalar(uri, dimensionality)
    return literals.Literal(scalar=scalar)  # type: ignore


def make_scalar(
    uri: Union[str, os.PathLike],
    dimensionality,
) -> literals.Scalar:
    blobtype = core_types.BlobType(format="", dimensionality=dimensionality)
    blob = literals.Blob(metadata=literals.BlobMetadata(type=blobtype), uri=uri)

return literals.Scalar(blob=blob) # type: ignore

Hi, While running pyflyte with java 19 from my local pc I'm getting java error. Can anyone tell me whether flyte is java 19 compatible?

Anyone able to set up debugger for flytekit in VScode? I kept seeing this:

Hello. Thanks to the Flyte team for their work on this framework. Real quick, using Flyte-Binary as a helm chart onto AWS EKS. Trying to set up OIDC and/or OAuth2 via Auth0 and have been messing around with some configs but have been unsuccessful getting the

/console

endpoint to query the browser for login with variations off of the following values.yaml config:

auth: 
      enabled: true
      enableAuthServer: false
      oidc:
        baseUrl: https://{DOMAIN}.<http://auth0.com/.well-known/openid-configuration|auth0.com/.well-known/openid-configuration>
        clientId: {CLIENT ID}
        clientSecret: {CLIENT SECRET}
      authorizedUris:
        - https://{LOAD BALANCED ADDRESS}.com
        - https://{LOAD BALANCED ADDRESS}.com

Bit unsure how to transpose the documentation from the official site into

flyte-binary

as the documentation seems to only go over

flyte-core

or

flyte

. The documentation doesn't seem to really exist for flyte-binary and there's scant discussion surrounding the topic unfortunately. I understand on some level that there are three components to the Flyte deployment but how does one access those configurations through the Helm chart within the

flyte-binary

deployment? Specifically, how are you supposed to affect the various OIDC/auth config values when the templates for binary don't seem to match the documentation? Also, we wish to use Auth0 as our external authorization server and don't need the internal server, but there have been conflicting resources suggesting that internal still needs to be defined. Any help in the matter would be most appreciated.

Hello I'm encountering error while running py script using pyflyte

Encountered error while executing workflow '<http://ss.wf|ss.wf>':
  Error encountered while executing 'wf':
  Failed to convert inputs of task 'ss.split_preprocess':
  [Errno 2] No such file or directory: '/tmp/flyte-2tev7275/raw/b8cd37c42c4279d44bbc8f2b4626f7c3'

This seems like a bug with

pyflyte package

-

pyflyte run

seems to work fine when I add the project root path to

sys.path

I’m trying to execute a simple test workflow and getting this error:

Request failed with status code 500 failed to create workflow in propeller namespaces <project-name> not found

. The project already shows up in the flyte console and I was able to package/register the workflow, but I can’t seem to execute it?

My subworkflow inside a flyte conditional receives inputs from these 5 tasks, but it doesn’t appear to depend on them on graph UI. I think that’s because the “conditional” itself doesn’t depend on them, but the workflow it calls does. Is it possible to configure dependencies for a conditional, or assign this dependency explicitly? This is mostly a visual issue, but I’m also getting an error locally which I suspect may be related

AttributeError: 'FlyteBranchNode' object has no attribute 'interface'

(it does not abort my run on flyte remote, so it is tolerable)

Has anyone been able to have multiple resource-tolerations when requesting gpus for a job? I have 2 node pools for single gpus and multi-gpus, and I want to be able to only use the different gpu machine types when needed: a job wants 4 gpus on the same instance. But I noticed that when requesting 1 gpu flyte defaults to the last toleration when requesting resources. And furthermore when launching a task I don't see any place to determine the tolerations associated with that task.

k8s:
    plugins:
      # -- Configuration section for all K8s specific plugins [Configuration structure](<https://pkg.go.dev/github.com/lyft/flyteplugins/go/tasks/pluginmachinery/flytek8s/config>)
      k8s:
        default-env-vars: []
        #  DEFAULT_ENV_VAR: VALUE
        default-cpus: 100m
        default-memory: 100Mi
        resource-tolerations:
          - <http://nvidia.com/gpu|nvidia.com/gpu>:
            - key: "num-gpus"
              operator: "Equal"
              value: 1
              effect: "NoSchedule"
          - <http://nvidia.com/gpu|nvidia.com/gpu>:
            - key: "num-gpus"
              operator: "Equal"
              value: 4
              effect: "NoSchedule"

Hello Team! I have some problems, I made a workflow that has one shelltask. This shelltask have pod task config. But workflow gives errors that

failed at Node[n0]. RuntimeExecutionError: failed during plugin execution, caused by: failed to execute
│  handle for plugin [sidecar]: [BadTaskSpecification] invalid TaskSpecification [<nil>], Err: [nil Struct Object passed]

How can I handle it?