https://flyte.org logo
Join the conversationJoin Slack
Channels
announcements
ask-the-community
auth
conference-talks
contribute
databricks-integration
datahub-flyte
deployment
ecosystem-unionml
engineeringlabs
events
feature-discussions
flyte-bazel
flyte-build
flyte-console
flyte-deployment
flyte-documentation
flyte-github
flyte-ui-ux
flytekit
flytekit-java
flytelab
great-content
hacktoberfest-2022
helsing-flyte
in-flyte-conversations
introductions
jobs
konan-integration
linkedin-flyte
random
ray-integration
ray-on-flyte
release
scipy-2022-sprint
sig-large-models
workflow-building-ui-proj
writing-w-sfloris
Powered by Linen
flytekit
  • h

    Hampus Rosvall

    05/02/2022, 12:30 PM
    How does Flyte handle Pod clean up? I currently have ~30-40 pods in one of my namespaces. Should I do that manually or is this configured else where?
    p
    • 2
    • 2
  • d

    Dylan Wilder

    05/02/2022, 3:35 PM
    making this a top level thread for visibility 🙂 is it possible to inject env vars into workflows at launch time? or to extend flyte context with user info?
    y
    g
    • 3
    • 34
  • d

    Dylan Wilder

    05/02/2022, 5:16 PM
    sorry for so much spam today 😅 for context this is all around upgrading to 1.0. Getting the following issue when registering with pyflyte. see 🧵
    e
    k
    y
    • 4
    • 47
  • d

    Dylan Wilder

    05/03/2022, 10:09 PM
    noticed something a bit buggy in 1.0.0.
    flytekit.current_context().execution_id
    has a few issues • during compile time it is a string of the form
    ex:local:local:local
    • during execution time it is a struct of the form project, domain, name and the name portion is empty
    k
    y
    • 3
    • 21
  • g

    GitHub

    05/04/2022, 12:52 AM
    message has been deleted
  • g

    GitHub

    05/04/2022, 3:20 PM
    message has been deleted
  • b

    Babis Kiosidis

    05/05/2022, 10:18 AM
    Hey I am trying to pass a list of string arguments in a ContainerTask and add all of them in the
    arguments
    field but struggling a bit with understanding if this is possible. 🧵
    k
    • 2
    • 24
  • d

    Dylan Wilder

    05/05/2022, 8:33 PM
    is there a flyte literal parser utility to produce the json that gets surfaced in the flyte UI?
    y
    n
    +2
    • 5
    • 33
  • g

    GitHub

    05/05/2022, 9:52 PM
    message has been deleted
  • g

    GitHub

    05/06/2022, 12:28 AM
    message has been deleted
  • g

    GitHub

    05/06/2022, 8:28 PM
    message has been deleted
  • y

    Yee

    05/06/2022, 9:26 PM
    @Han apologies for the delay, but i owe you the sample commands for specifying multiple images
    👍 1
    s
    k
    s
    • 4
    • 8
  • r

    Robert Everson

    05/10/2022, 11:35 PM
    I see above that
    flyte-cli parse-proto
    would work to print out the raw values of a proto, is there a way to get the proto from the flyte metastore directly? I’m trying to add an architecture feature to flytekit, and flyteidl and flyteadmin have both been updated, but the value isn’t making it from my register step all the way to the pod runtime, so I’m trying to debug where the breakdown is happening. I’ve copied the closure field from the tasks table, but the proto parsing isn’t outputting any information for me.
    y
    • 2
    • 23
  • g

    GitHub

    05/11/2022, 1:58 AM
    message has been deleted
  • s

    Sonja Ericsson

    05/11/2022, 1:25 PM
    Hi! 👋 We are passing the environment variable
    FLYTE_SDK_WORKFLOW_PACKAGES
    to
    pyflyte workflows serialize
    . Looks like that functionality was removed in this PR. Was this intended?
    k
    • 2
    • 8
  • g

    GitHub

    05/11/2022, 3:29 PM
    message has been deleted
  • g

    GitHub

    05/11/2022, 10:13 PM
    message has been deleted
  • g

    GitHub

    05/13/2022, 11:37 PM
    message has been deleted
    🎉 1
  • g

    Greg Gydush

    05/17/2022, 6:00 PM
    Added a ticket requesting ability to await first task execution success before exiting pyflyte! Thanks for the great demo today!
    🙏 2
  • g

    Greg Gydush

    05/17/2022, 10:56 PM
    Anyone using latest
    mypy
    and have this issue with
    NamedTuple
    (for named outputs). The example in the docs (shown below) fails mypy 0.910 Code from Flyte docs:
    import typing
    
    from flytekit import task
    
    hello_output = typing.NamedTuple("OP", greet=str)
    
    
    @task
    def say_hello() -> hello_output:
        return hello_output("hello world")
    Error:
    mint/workflows/tmp.py:5:16: error: Unexpected arguments to namedtuple()  [misc]
        hello_output = typing.NamedTuple("OP", greet=str)
                       ^
    mint/workflows/tmp.py: note: In function "say_hello":
    mint/workflows/tmp.py:10:12: error: Too many arguments for "hello_output"  [call-arg]
            return hello_output("hello world")
                   ^
    s
    • 2
    • 2
  • g

    GitHub

    05/18/2022, 7:15 AM
    message has been deleted
  • j

    jeev

    05/18/2022, 9:26 PM
    do we already have a workaround or fix for this? https://github.com/flyteorg/flyte/issues/2508
    k
    y
    +2
    • 5
    • 17
  • a

    Alex Bain

    05/19/2022, 10:09 AM
    Flyte Team, I have been working on using
    FlyteRemote
    and now I understand how to ask my question lol. My remote workflow looks like this:
    @dataclass_json
    @dataclass
    class WrappedList:
        s: str
    
    @flytekit.workflow
    def count_files_test_workflow(
        in: WrappedList,
    ):
    How do I call this workflow with
    FlyteRemote
    ? If I try to do:
    remote = FlyteRemote(...)
    lp = remote.fetch_launch_plan(...)
    remote.execute(lp, inputs={'in': WrappedList("foo")})
    Then I get
    FlyteTypeException: Type error! Received: <class '__main__.WrappedList'> with value: WrappedList(s='foo'), Expected: <class 'types.WrappedlistSchema'>
    . I looked into the
    type_hints
    argument to
    remote.execute
    , but I don't think that's the right thing here. What I really need to be able to do is specify
    inputs={'in': x}
    where x is an instance of types.WrappedListSchema.
    👀 1
    k
    k
    e
    • 4
    • 5
  • s

    Stephan Gref

    05/19/2022, 5:15 PM
    Scratching my head here a bit... Does anyone use flytekit + flytekitplugins-data-fsspec + fast register? It seems that fsspec does not handle the the default destination
    .
    (dot, like current directory) well and does not download the tar.gz at all. Reproduced this with this minimal example:
    import os
    
    from flytekit.tools.fast_registration import download_distribution as _download_distribution
    
    os.environ["AWS_ACCESS_KEY_ID"] = "key_id"
    os.environ["AWS_SECRET_ACCESS_KEY"] = "secret_access_key"
    
    destination = "."
    
    _download_distribution(
        additional_distribution="<s3://bucket/fast-register.tar.gz>",
        destination=".")
    It works if I replace
    destination
    with is
    ./
    in this example. The issue is, that
    .
    is set by propeller (this is at least my understanding). Is there any way I can configure that?
    👀 1
    s
    k
    • 3
    • 3
  • g

    Greg Gydush

    05/19/2022, 5:32 PM
    Jeev helped me find a very interesting bug this morning when using dictionaries on Flyte. The order of the keys seems to be non-deterministic, issue is here!
    k
    j
    e
    • 4
    • 11
  • g

    GitHub

    05/20/2022, 10:05 PM
    message has been deleted
  • a

    Alex Bain

    05/22/2022, 2:25 AM
    Hey Flyte, I am on Flyte backend system
    v1.0.0
    with
    flytekit==0.26.0
    and Spark tasks are fine. I just moved up (last night) to
    flytekit==1.0.1
    with
    flytekitplugins-spark==1.0.1
    (with the same backend system) and now Spark tasks are broken with the error
    can't open file '/usr/bin/entrypoint.py': [Errno 2] No such file or directory
    . I took a look and the old (good) tasks have the following set:
    "mainApplicationFile": {
        "stringValue": "local:///usr/local/lib/python3.8/dist-packages/flytekit/bin/entrypoint.py"
    }
    and that file exists and looks fine. However, my new (broken) Spark task shows the following:
    "mainApplicationFile": {
        "stringValue": "local:///usr/bin/entrypoint.py"
    },
    "executorPath": {
        "stringValue": "/usr/bin/python3.8"
    }
    but I have no
    /usr/bin/entrypoint.py
    in the container (which explains the error message). This seems 99% like a bug, can you take a look? I do have a
    v1.0.1
    Flyte backend system up and running... I'll try the same thing there.
    k
    m
    • 3
    • 26
  • g

    GitHub

    05/24/2022, 12:58 AM
    message has been deleted
  • g

    GitHub

    05/25/2022, 8:28 AM
    message has been deleted
  • g

    GitHub

    05/26/2022, 2:31 AM
    message has been deleted
Powered by Linen
Title
g

GitHub

05/26/2022, 2:31 AM
message has been deleted
View count: 1