• GitHub

    GitHub

    4 months ago
    message has been deleted
  • j

    jeev

    4 months ago
    do we already have a workaround or fix for this? https://github.com/flyteorg/flyte/issues/2508
    j
    Ketan (kumare3)
    +3
    17 replies
    Copy to Clipboard
  • Alex Bain

    Alex Bain

    4 months ago
    Flyte Team, I have been working on using
    FlyteRemote
    and now I understand how to ask my question lol. My remote workflow looks like this:
    @dataclass_json
    @dataclass
    class WrappedList:
        s: str
    
    @flytekit.workflow
    def count_files_test_workflow(
        in: WrappedList,
    ):
    How do I call this workflow with
    FlyteRemote
    ? If I try to do:
    remote = FlyteRemote(...)
    lp = remote.fetch_launch_plan(...)
    remote.execute(lp, inputs={'in': WrappedList("foo")})
    Then I get
    FlyteTypeException: Type error! Received: <class '__main__.WrappedList'> with value: WrappedList(s='foo'), Expected: <class 'types.WrappedlistSchema'>
    . I looked into the
    type_hints
    argument to
    remote.execute
    , but I don't think that's the right thing here. What I really need to be able to do is specify
    inputs={'in': x}
    where x is an instance of types.WrappedListSchema.
    Alex Bain
    Kevin Su
    +1
    5 replies
    Copy to Clipboard
  • Stephan Gref

    Stephan Gref

    4 months ago
    Scratching my head here a bit... Does anyone use flytekit + flytekitplugins-data-fsspec + fast register? It seems that fsspec does not handle the the default destination
    .
    (dot, like current directory) well and does not download the tar.gz at all. Reproduced this with this minimal example:
    import os
    
    from flytekit.tools.fast_registration import download_distribution as _download_distribution
    
    os.environ["AWS_ACCESS_KEY_ID"] = "key_id"
    os.environ["AWS_SECRET_ACCESS_KEY"] = "secret_access_key"
    
    destination = "."
    
    _download_distribution(
        additional_distribution="<s3://bucket/fast-register.tar.gz>",
        destination=".")
    It works if I replace
    destination
    with is
    ./
    in this example. The issue is, that
    .
    is set by propeller (this is at least my understanding). Is there any way I can configure that?
    Stephan Gref
    s
    +1
    3 replies
    Copy to Clipboard
  • Greg Gydush

    Greg Gydush

    4 months ago
    Jeev helped me find a very interesting bug this morning when using dictionaries on Flyte. The order of the keys seems to be non-deterministic, issue is here!
    Greg Gydush
    Ketan (kumare3)
    +2
    11 replies
    Copy to Clipboard
  • GitHub

    GitHub

    4 months ago
    message has been deleted
  • Alex Bain

    Alex Bain

    4 months ago
    Hey Flyte, I am on Flyte backend system
    v1.0.0
    with
    flytekit==0.26.0
    and Spark tasks are fine. I just moved up (last night) to
    flytekit==1.0.1
    with
    flytekitplugins-spark==1.0.1
    (with the same backend system) and now Spark tasks are broken with the error
    can't open file '/usr/bin/entrypoint.py': [Errno 2] No such file or directory
    . I took a look and the old (good) tasks have the following set:
    "mainApplicationFile": {
        "stringValue": "local:///usr/local/lib/python3.8/dist-packages/flytekit/bin/entrypoint.py"
    }
    and that file exists and looks fine. However, my new (broken) Spark task shows the following:
    "mainApplicationFile": {
        "stringValue": "local:///usr/bin/entrypoint.py"
    },
    "executorPath": {
        "stringValue": "/usr/bin/python3.8"
    }
    but I have no
    /usr/bin/entrypoint.py
    in the container (which explains the error message). This seems 99% like a bug, can you take a look? I do have a
    v1.0.1
    Flyte backend system up and running... I'll try the same thing there.
    Alex Bain
    Ketan (kumare3)
    26 replies
    Copy to Clipboard
  • GitHub

    GitHub

    4 months ago
    message has been deleted
  • GitHub

    GitHub

    4 months ago
    message has been deleted
  • GitHub

    GitHub

    4 months ago
    message has been deleted