Anyone have any experience adding caching to

map_task

? I've tried by adding the caching stuff to the

@task

decorator and in the "mapification" like this:

map_task(my_task)(input=input).with_overrides(cache=True, cache_version="1.0")

but I'm not seeing in the flyte console the little swirly that indicates the task either wrote to or read from the cache. Is caching a map task supported?

Hey guys, noob question here, but does Flyte have an automatic or configurable retry mechanism to use more resources in the case of OOM pods or something like that? Couldn’t find any info about it from here. TY!

Hi Flyers, Thank you for attending the Flyte Community Sync yesterday. Special thanks to our guest appearances and presenters! We had a busy agenda with: • Community Update, featuring guest appearance by @George Odette and @Mihai Todor, UnionML launch recap, blogs, and roadmap (current and upcoming releases). • Flyte Decks, by Union.ai Software Engineer @Eduardo Apolinario (eapolinario). • Pterodactyl, Javascript SDK for Flyte, by @Matthew Griffin. Slides are in the Meeting notes. Mark your calendars for the next community sync on June 28th! Flyte Team

does any one know how to stop launchplan? this command does not work on the flyte I guess...

flytectl update launchplan -p flytesnacks -d development --version <version> <launchplan name> --archived

👋 Hey folks!

hi flyte team, we recently upgraded the console to the latest and now it doesn't show dynamic subtasks. Is there a switch/config setting for that?

Hello, Team, Nice to meet you.

Folks please upvote ecosystem project - https://www.producthunt.com/posts/unionml

Hi, I'd like to ask a question of flyte type. I am using custom class module, Kirin with flyte. Flyte does not return any type error before when I run sample code of flyte, but I have no idea why flyte make this error.

ValueError: Type <class 'kirin.api.Kirin'> not supported currently in Flytekit. Please register a new transformer

The above exception was the direct cause of the following exception:

would you give any idea to solve this happening?

Dear Flyte users, I have a question to pull private docker images. According to this link, we can specify V1Container per task. This is the point I want to ask how I can pull a private image from dockerhub. I have setup token & name on "Secret" namespace, but flyte console UI does not recognize it.

👋 Hi everyone!

HI all, I am trying to pass a file locally (not running the workflow in the console) but getting the error "Error converting /home/sgodette/Downloads/GCF_000013925.1_ASM1392v2_genomic.fna because it's not a file". What could I not be possibly inputting right?

I have a question to use

CronSchedule

method when I setup launch plan this is my example of launchplan below.

if __name__ == "__main__":
    remote = FlyteRemote(config=Config.auto())
    remote.register_launch_plan(
        entity=LaunchPlan.get_or_create(
            name="daily-test",
            workflow=<http://workflows.wf|workflows.wf>,
            # Minutes, Hours, Day-of-month, Month, Day-of-week, Year
            # minute, hour, day-of-month, month, day-of-week, year
            # cron_expression -> aws cron scheme
            schedule=CronSchedule(cron_expression="0 3 * * ? *"),
        ),
        project="flytesnacks", #flyte-aistudio
        domain="development",
        version="51Dwe74FqGRzs2v1WPwrcg==",
    )

It returns the error below

launch plan daily failed to update due to rpc error: code = Internal desc = failed adding schedule for unknown schedule expression type &{0 3 * * ? *}

Is this bug fixed to use

cron_expression with 6 field type

in the latest version?

Hi, It's related to the chat above I'm trying to schedule launchplan with cron_expression but error occurs

launch plan daily failed to update due to rpc error: code = Internal desc = failed adding schedule for unknown schedule expression type &{0 0 * * ? *}

I found this code in flyteadmin github repo while debugging (https://github.com/flyteorg/flyteadmin/blob/master/scheduler/dbapi/event_scheduler_impl.go)

func (s *eventScheduler) AddSchedule(ctx context.Context, input interfaces.AddScheduleInput) error {
	...
	switch v := input.ScheduleExpression.GetScheduleExpression().(type) {
	case *admin.Schedule_Rate:
		fixedRateValue = v.Rate.Value
		fixedRateUnit = v.Rate.Unit
	case *admin.Schedule_CronSchedule:
		cronString = v.CronSchedule.Schedule
	default:
		return fmt.Errorf("failed adding schedule for unknown schedule expression type %v", v)
	}
	...
}

and this code in flyteidl repo (https://github.com/flyteorg/flyteidl/blob/master/gen/pb-go/flyteidl/admin/schedule.pb.go)

type Schedule struct {
	// Types that are valid to be assigned to ScheduleExpression:
	//	*Schedule_CronExpression
	//	*Schedule_Rate
	//	*Schedule_CronSchedule
	ScheduleExpression isSchedule_ScheduleExpression `protobuf_oneof:"ScheduleExpression"`
	// Name of the input variable that the kickoff time will be supplied to when the workflow is kicked off.
	KickoffTimeInputArg  string   `protobuf:"bytes,3,opt,name=kickoff_time_input_arg,json=kickoffTimeInputArg,proto3" json:"kickoff_time_input_arg,omitempty"`
	XXX_NoUnkeyedLiteral struct{} `json:"-"`
	XXX_unrecognized     []byte   `json:"-"`
	XXX_sizecache        int32    `json:"-"`
}

It seems that the error is caused by

AddSchedule

function not processing ths

Schedule_CronExpression

type. Can someone help me?

Hello there, I would like to get events equivalent to what is generated by

flytectl get execution -p flytesnacks -d development -o json

in the logs or somewhere for post-processing is there a way to do that? (I already have External events configured and processed by a lambda but I missing some context from it)

📣 Contributors of the Month 📣 Hi everyone! For the month of June, we want to give shout-outs and send swag to the following list of contributors who are sweating it out to make Flyte even better! • @Matthew Griffin built ✨ Pterodactyl ✨, Javascript SDK for Flyte. Check out the repo & watch the to see what Matthew has to say about the library. • @David Przybilla wrote an article that differentiates Flyte and Kubeflow. He has also worked on compiling a doc of all his learnings detailing how Flyte works — highly recommended for first-time contributors. • @Robert Everson worked on adding ARM architecture support to Flytekit for Lyft internal use; the changes would hopefully be merged into the master branch very soon. We hope to have many more of your contributions, Matthew, David, and Robert! All contributors of the month should be available on flyte.org.

Hi Flyers, Learn more about Flyte's upcoming v1.1.0 Hawk release featuring FlyteDecks, how Flyte compares to other popular orchestration tools, and meet some of Flyte's contributors in “Flyte Monthly - Issue # 9”. https://lnkd.in/g2DDzpEZ Subscribe here: https://lnkd.in/gQyBmi2c

We've a dataclass that has a parameter that is of type:

typing.List[typing.List[float]]

; it appears to fail to serialize. Is this a known issue / limitation?

Hi Flyers, Join the Flyte Community Sync tomorrow to learn more about

pyflyte register

and plans for how the

pyflyte register, run, package

commands are evolving. Also, get a preview of the upcoming

flyte x ray

and

flyte signals

features, as well as the all-new Flyte Airflow Provider! June 28 at 9 am PT Calendar invite and Zoom link See you there! Flyte Team

Hello Team, I want to specify a GPU pool for my map task. I went through this documentation and it is not very clear. Could you please help with this ?

Does anyone else use the

namespaceTemplate

option for the k8s-array plugin in flytepropeller? We’re on a rather old version, but I’m seeing that flyteadmin is only using the parent namespace for the resources it creates, so when we override the

namespaceTemplate

in our clusterresource templates in flyteadmin, we are now seeing an error that the namespace is incorrect (our templates have the namespaceTemplate manually applied to them, so the verification of creating the resource fails as the namespace on the Template and the namespace on the object differ). This might just be an issue with using 2 different versions of flyteadmin and flytepropeller, but wanted to check if this is expected.

Anyone know a good way to do non-grid (eg. some kind of bayesian) hyperparameter optimization with Flyte, with multiple trials in parallel? (ie. is there some library that makes this easy or do I have to implement most of the optimization stuff myself? eg. a library which spits out parameters to try and I give it back the results would be pretty easy to use with Flyte, rather than the library calling an objective function like hyperopt does)

Hey! Is there an easy way to see all workflow using a particular task?

Invitations sent to 12 outstanding contributors to get officially added to the Flyte contributors family. Thank you for your contributions and making Flyte what it is

Hi Flyers, Thank you for attending the Flyte Community Sync yesterday. Special thanks to our guest appearances and presenters! On the agenda were: • Community Updates, featuring Contributors of the Month (June), guest appearance by @Bernhard Stadlbauer, Flyte Ecosystem Project UnionML update by @Niels Bantilan, and roadmap details by @Ketan (kumare3). • PyFlyte Register, a combination of PyFlyte Package and PyFlyte Run, by @Yee. • Flyte Airflow Provider, which can construct Flyte ETL pipelines in Airflow and then ML pipelines in Flyte, triggering Flyte ML pipelines from within Airflow, by @Samhita Alla. Slides are in the Meeting notes. Mark your calendars for the next community sync on July 12th! Flyte Team

Hello, I’m wondering how involved it would be to extend the sagemaker plugin to support additional task types like batch transforms and creating/updating endpoint configurations. Is this something that can be done purely in python (flytekit)?

Hi all, just evaluating flyte right now and looks promising for automating and scaling our container based workflows! I'm wondering if there are plans to add support for local execution of raw containers?

Hey team, What is the recommended way to get the execution Id of a workflow? We used the following, are there better ways to do it?

flytekit.current_context().execution_id.name or os.environ['FLYTE_INTERNAL_EXECUTION_ID']

Hey, a colleague is having an issue: he's running a workflow locally, and Flyte stores all the inputs/outputs in /tmp, but he's using tmpfs mounted on /tmp. Is there a way to change the folder? // solved by setting TMPDIR for the command 🙂