When using `FlyteFile` we end up returning something like th

Question

When using `FlyteFile` we end up returning something like this when we want to upload a file to specific GCS path This is fine if you have one file you are returning but can be verbose when working with a handful of output files ```FlyteFile random local path remote path=os path join gcs outdir os path basename random local path ``` Does anyone have recommendations to simplify this I was thinking a small PR adding an additional parameter to FlyteFile could be nice but thought I d ask here in case anyone has other ideas to simplify ```FlyteFile random local path remote dir=gcs outdir ```

Greg Gydush · Answer

Above outlines a simple case but it can get unwieldy when you start thinking about defining tempfile in line like so ```FlyteFile os path join current context working directory message txt remote path= ```

Greg Gydush · Answer

This example may help show what I think I want and am wondering if anyone else feels the same I d be happy to PR if we think this would be useful or if you have suggestions they d be greatly appreciated

Yee · Answer

< Greg Gydush> why don t you set the raw output prefix

Yee · Answer

then you shouldn t have to do any of that

Greg Gydush · Answer

What if I want it used for some files permanent but not for others temporary file output from a task Some tasks may have 5 outputs that are meant to be uploaded to specific gcs location 1 output that is temporary where random path is fine

Greg Gydush · Answer

gt why don t you set the raw output prefix Could you maybe elaborate on this too This can be set on the context

Greg Gydush · Answer

Anyone else have thoughts on this

Yee · Answer

oh oops sorry i was on vacation last week

Yee · Answer

was just answering when i had time

Yee · Answer

ping ketan or eduardo or just repost in the future in the channel

Yee · Answer

see <https docs flyte org en latest concepts data management html raw data prefix>

Greg Gydush · Answer

No apologies necessary < Yee> Hope you had a nice vacation I can try to jump on office hours this week to chat about it with Ketan

Yee · Answer

<https github com flyteorg flyte blob master deployment eks flyte helm generated yaml L501> for the default as set by propeller

Yee · Answer

<https github com flyteorg flyte issues 2070> for how we hope to have this be a project domain level default in the future instead of just a propeller default

Greg Gydush · Answer

Yeah I think this is still globally For a single task I may have some outputs that need to be uploaded to an explicit location others where the randomly generated path using `raw data prefix` is fine Does that make sense I can show you some examples of why I would need something like that it s extremely common in bioinfo e g intermediate files upload to ephemeral location so next task can use but keep the logs in permanent GCS location

Greg Gydush · Answer

That s where having `remote dir` would be handy but may be best to discuss more in OH

Yee · Answer

oh yeah i remember thinking when i first saw this that it might be configurable via environment variables but I think it s not actually

Yee · Answer

yeah maybe something to discuss in OH