https://flyte.org logo
Join the conversationJoin Slack
Channels
announcements
ask-the-community
auth
conference-talks
contribute
databricks-integration
datahub-flyte
deployment
ecosystem-unionml
engineeringlabs
events
feature-discussions
flyte-bazel
flyte-build
flyte-console
flyte-deployment
flyte-documentation
flyte-github
flyte-ui-ux
flytekit
flytekit-java
flytelab
great-content
hacktoberfest-2022
helsing-flyte
in-flyte-conversations
introductions
jobs
konan-integration
linkedin-flyte
random
ray-integration
ray-on-flyte
release
scipy-2022-sprint
sig-large-models
workflow-building-ui-proj
writing-w-sfloris
Powered by Linen
announcements
  • t

    Thomas Addison

    06/15/2022, 2:30 PM
    Anyone have any experience adding caching to
    map_task
    ? I've tried by adding the caching stuff to the
    @task
    decorator and in the "mapification" like this:
    map_task(my_task)(input=input).with_overrides(cache=True, cache_version="1.0")
    but I'm not seeing in the flyte console the little swirly that indicates the task either wrote to or read from the cache. Is caching a map task supported?
    d
    n
    • 3
    • 10
  • a

    allen

    06/15/2022, 3:32 PM
    Hey guys, noob question here, but does Flyte have an automatic or configurable retry mechanism to use more resources in the case of OOM pods or something like that? Couldn’t find any info about it from here. TY!
    d
    k
    • 3
    • 4
  • s

    Sandra Youssef

    06/16/2022, 12:40 AM
    Hi Flyers, Thank you for attending the Flyte Community Sync yesterday. Special thanks to our guest appearances and presenters! We had a busy agenda with: • Community Update, featuring guest appearance by @George Odette and @Mihai Todor, UnionML launch recap, blogs, and roadmap (current and upcoming releases).

    Recording▾

    • Flyte Decks, by Union.ai Software Engineer @Eduardo Apolinario (eapolinario).

    Recording▾

    • Pterodactyl, Javascript SDK for Flyte, by @Matthew Griffin.

    Recording▾

    Slides are in the Meeting notes. Mark your calendars for the next community sync on June 28th! Flyte Team
    🎉 7
  • s

    SeungTaeKim

    06/16/2022, 5:50 AM
    does any one know how to stop launchplan? this command does not work on the flyte I guess...
    flytectl update launchplan -p flytesnacks -d development --version <version> <launchplan name> --archived
    s
    p
    • 3
    • 13
  • d

    David G. Simmons

    06/16/2022, 5:49 PM
    👋 Hey folks!
    👋 6
    a
    s
    • 3
    • 3
  • a

    Alex Pozimenko

    06/16/2022, 9:26 PM
    hi flyte team, we recently upgraded the console to the latest and now it doesn't show dynamic subtasks. Is there a switch/config setting for that?
    h
    n
    +4
    • 7
    • 127
  • l

    Liang Yao

    06/21/2022, 12:32 AM
    Hello, Team, Nice to meet you.
    👋 3
  • k

    Ketan (kumare3)

    06/21/2022, 4:56 AM
    Folks please upvote ecosystem project - https://www.producthunt.com/posts/unionml
    👍 7
  • s

    SeungTaeKim

    06/21/2022, 5:39 AM
    Hi, I'd like to ask a question of flyte type. I am using custom class module, Kirin with flyte. Flyte does not return any type error before when I run sample code of flyte, but I have no idea why flyte make this error.
    ValueError: Type <class 'kirin.api.Kirin'> not supported currently in Flytekit. Please register a new transformer
    
    The above exception was the direct cause of the following exception:
    would you give any idea to solve this happening?
    p
    k
    k
    • 4
    • 6
  • s

    SeungTaeKim

    06/22/2022, 4:20 AM
    Dear Flyte users, I have a question to pull private docker images. According to this link, we can specify V1Container per task. This is the point I want to ask how I can pull a private image from dockerhub. I have setup token & name on "Secret" namespace, but flyte console UI does not recognize it.
    k
    • 2
    • 11
  • s

    Sam

    06/22/2022, 7:50 AM
    👋 Hi everyone!
    🖖 1
    👋 3
  • g

    George Odette

    06/22/2022, 4:03 PM
    HI all, I am trying to pass a file locally (not running the workflow in the console) but getting the error "Error converting /home/sgodette/Downloads/GCF_000013925.1_ASM1392v2_genomic.fna because it's not a file". What could I not be possibly inputting right?
    k
    • 2
    • 6
  • s

    SeungTaeKim

    06/23/2022, 8:23 AM
    I have a question to use
    CronSchedule
    method when I setup launch plan this is my example of launchplan below.
    if __name__ == "__main__":
        remote = FlyteRemote(config=Config.auto())
        remote.register_launch_plan(
            entity=LaunchPlan.get_or_create(
                name="daily-test",
                workflow=<http://workflows.wf|workflows.wf>,
                # Minutes, Hours, Day-of-month, Month, Day-of-week, Year
                # minute, hour, day-of-month, month, day-of-week, year
                # cron_expression -> aws cron scheme
                schedule=CronSchedule(cron_expression="0 3 * * ? *"),
            ),
            project="flytesnacks", #flyte-aistudio
            domain="development",
            version="51Dwe74FqGRzs2v1WPwrcg==",
        )
    It returns the error below
    launch plan daily failed to update due to rpc error: code = Internal desc = failed adding schedule for unknown schedule expression type &{0 3 * * ? *}
    Is this bug fixed to use
    cron_expression with 6 field type
    in the latest version?
    p
    • 2
    • 1
  • k

    Kim Junil

    06/23/2022, 9:21 AM
    Hi, It's related to the chat above I'm trying to schedule launchplan with cron_expression but error occurs
    launch plan daily failed to update due to rpc error: code = Internal desc = failed adding schedule for unknown schedule expression type &{0 0 * * ? *}
    I found this code in flyteadmin github repo while debugging (https://github.com/flyteorg/flyteadmin/blob/master/scheduler/dbapi/event_scheduler_impl.go)
    func (s *eventScheduler) AddSchedule(ctx context.Context, input interfaces.AddScheduleInput) error {
    	...
    	switch v := input.ScheduleExpression.GetScheduleExpression().(type) {
    	case *admin.Schedule_Rate:
    		fixedRateValue = v.Rate.Value
    		fixedRateUnit = v.Rate.Unit
    	case *admin.Schedule_CronSchedule:
    		cronString = v.CronSchedule.Schedule
    	default:
    		return fmt.Errorf("failed adding schedule for unknown schedule expression type %v", v)
    	}
    	...
    }
    and this code in flyteidl repo (https://github.com/flyteorg/flyteidl/blob/master/gen/pb-go/flyteidl/admin/schedule.pb.go)
    type Schedule struct {
    	// Types that are valid to be assigned to ScheduleExpression:
    	//	*Schedule_CronExpression
    	//	*Schedule_Rate
    	//	*Schedule_CronSchedule
    	ScheduleExpression isSchedule_ScheduleExpression `protobuf_oneof:"ScheduleExpression"`
    	// Name of the input variable that the kickoff time will be supplied to when the workflow is kicked off.
    	KickoffTimeInputArg  string   `protobuf:"bytes,3,opt,name=kickoff_time_input_arg,json=kickoffTimeInputArg,proto3" json:"kickoff_time_input_arg,omitempty"`
    	XXX_NoUnkeyedLiteral struct{} `json:"-"`
    	XXX_unrecognized     []byte   `json:"-"`
    	XXX_sizecache        int32    `json:"-"`
    }
    It seems that the error is caused by
    AddSchedule
    function not processing ths
    Schedule_CronExpression
    type. Can someone help me?
    p
    n
    • 3
    • 6
  • j

    Jonathan Lamiel

    06/24/2022, 9:23 AM
    Hello there, I would like to get events equivalent to what is generated by
    flytectl get execution -p flytesnacks -d development -o json
    in the logs or somewhere for post-processing is there a way to do that? (I already have External events configured and processed by a lambda but I missing some context from it)
    s
    k
    y
    • 4
    • 18
  • s

    Samhita Alla

    06/24/2022, 10:40 AM
    📣 Contributors of the Month 📣 Hi everyone! For the month of June, we want to give shout-outs and send swag to the following list of contributors who are sweating it out to make Flyte even better! • @Matthew Griffin built ✨ Pterodactyl ✨, Javascript SDK for Flyte. Check out the repo & watch the

    recording▾

    to see what Matthew has to say about the library. • @David Przybilla wrote an article that differentiates Flyte and Kubeflow. He has also worked on compiling a doc of all his learnings detailing how Flyte works — highly recommended for first-time contributors. • @Robert Everson worked on adding ARM architecture support to Flytekit for Lyft internal use; the changes would hopefully be merged into the master branch very soon. We hope to have many more of your contributions, Matthew, David, and Robert! All contributors of the month should be available on flyte.org.
    ❤️ 13
  • s

    Sandra Youssef

    06/24/2022, 4:11 PM
    Hi Flyers, Learn more about Flyte's upcoming v1.1.0 Hawk release featuring FlyteDecks, how Flyte compares to other popular orchestration tools, and meet some of Flyte's contributors in “Flyte Monthly - Issue # 9”. https://lnkd.in/g2DDzpEZ Subscribe here: https://lnkd.in/gQyBmi2c
    👍 1
    🔥 1
  • m

    Mike Ossareh

    06/24/2022, 8:30 PM
    We've a dataclass that has a parameter that is of type:
    typing.List[typing.List[float]]
    ; it appears to fail to serialize. Is this a known issue / limitation?
    y
    • 2
    • 24
  • s

    Sandra Youssef

    06/27/2022, 3:54 PM
    Hi Flyers, Join the Flyte Community Sync tomorrow to learn more about
    pyflyte register
    and plans for how the
    pyflyte register, run, package
    commands are evolving. Also, get a preview of the upcoming
    flyte x ray
    and
    flyte signals
    features, as well as the all-new Flyte Airflow Provider! June 28 at 9 am PT Calendar invite and Zoom link See you there! Flyte Team
    🚀 4
    👍 2
    💯 3
  • v

    Vijay Saravana

    06/27/2022, 6:15 PM
    Hello Team, I want to specify a GPU pool for my map task. I went through this documentation and it is not very clear. Could you please help with this ?
    🖖 1
    k
    v
    s
    • 4
    • 14
  • r

    Robert Everson

    06/27/2022, 7:23 PM
    Does anyone else use the
    namespaceTemplate
    option for the k8s-array plugin in flytepropeller? We’re on a rather old version, but I’m seeing that flyteadmin is only using the parent namespace for the resources it creates, so when we override the
    namespaceTemplate
    in our clusterresource templates in flyteadmin, we are now seeing an error that the namespace is incorrect (our templates have the namespaceTemplate manually applied to them, so the verification of creating the resource fails as the namespace on the Template and the namespace on the object differ). This might just be an issue with using 2 different versions of flyteadmin and flytepropeller, but wanted to check if this is expected.
    k
    • 2
    • 3
  • r

    Robin Kahlow

    06/29/2022, 11:06 AM
    Anyone know a good way to do non-grid (eg. some kind of bayesian) hyperparameter optimization with Flyte, with multiple trials in parallel? (ie. is there some library that makes this easy or do I have to implement most of the optimization stuff myself? eg. a library which spits out parameters to try and I give it back the results would be pretty easy to use with Flyte, rather than the library calling an objective function like hyperopt does)
    n
    k
    s
    • 4
    • 32
  • s

    Sonja Ericsson

    06/29/2022, 12:30 PM
    Hey! Is there an easy way to see all workflow using a particular task?
    k
    y
    • 3
    • 3
  • k

    Ketan (kumare3)

    06/29/2022, 9:27 PM
    Invitations sent to 12 outstanding contributors to get officially added to the Flyte contributors family. Thank you for your contributions and making Flyte what it is
    💯 5
    😛artyparrot: 6
    :flyte: 9
    🏅 8
    ❤️ 4
    🙌 7
  • s

    Sandra Youssef

    06/29/2022, 9:38 PM
    Hi Flyers, Thank you for attending the Flyte Community Sync yesterday. Special thanks to our guest appearances and presenters! On the agenda were: • Community Updates, featuring Contributors of the Month (June), guest appearance by @Bernhard Stadlbauer, Flyte Ecosystem Project UnionML update by @Niels Bantilan, and roadmap details by @Ketan (kumare3).

    Recording▾

    • PyFlyte Register, a combination of PyFlyte Package and PyFlyte Run, by @Yee.

    Recording▾

    • Flyte Airflow Provider, which can construct Flyte ETL pipelines in Airflow and then ML pipelines in Flyte, triggering Flyte ML pipelines from within Airflow, by @Samhita Alla.

    Recording▾

    Slides are in the Meeting notes. Mark your calendars for the next community sync on July 12th! Flyte Team
    ❤️ 7
  • a

    Andrew Achkar

    06/30/2022, 3:54 PM
    Hello, I’m wondering how involved it would be to extend the sagemaker plugin to support additional task types like batch transforms and creating/updating endpoint configurations. Is this something that can be done purely in python (flytekit)?
    k
    e
    • 3
    • 18
  • f

    Felix Ruess

    06/30/2022, 5:06 PM
    Hi all, just evaluating flyte right now and looks promising for automating and scaling our container based workflows! I'm wondering if there are plans to add support for local execution of raw containers?
    🙏 2
    k
    k
    • 3
    • 16
  • v

    varsha Parthasarathy

    06/30/2022, 6:26 PM
    Hey team, What is the recommended way to get the execution Id of a workflow? We used the following, are there better ways to do it?
    flytekit.current_context().execution_id.name or os.environ['FLYTE_INTERNAL_EXECUTION_ID']
    k
    a
    • 3
    • 11
  • r

    Robin Kahlow

    07/01/2022, 11:18 AM
    Hey, a colleague is having an issue: he's running a workflow locally, and Flyte stores all the inputs/outputs in /tmp, but he's using tmpfs mounted on /tmp. Is there a way to change the folder? // solved by setting TMPDIR for the command 🙂
    :excellent: 3
  • v

    Vijay Saravana

    07/02/2022, 5:18 AM
    Hi team, While specifying resources for a task, what does CPU = '2' actually mean? whether cpu means different instances or threads in the same instance?
    s
    • 2
    • 2
Powered by Linen
Title
v

Vijay Saravana

07/02/2022, 5:18 AM
Hi team, While specifying resources for a task, what does CPU = '2' actually mean? whether cpu means different instances or threads in the same instance?
s

Samhita Alla

07/02/2022, 10:50 AM
@Vijay Saravana, cpu = “2” means 2 CPU cores; it’s as per the Kubernetes terminology. Here’s a detailed guide on it: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/#resource-units-in-kubernetes
👍 2
v

Vijay Saravana

07/05/2022, 11:51 PM
Thank you!
View count: 2