https://flyte.org logo
#ask-the-community
Title
# ask-the-community
v

Vitali

12/22/2023, 2:29 PM
Hi, is there a way how I could define the
working_dir
on task level? Per default its
/root
, but in my container image this folder does not exist and would like to use another directory instead. I am aware that I could define it via CLI of the workflow, but I would rather have it set for this specific task/container in code.
k

Ketan (kumare3)

12/22/2023, 4:06 PM
Do you mean when you use pyflyte run?
v

Vitali

12/22/2023, 4:12 PM
Yes I use pyflyte run. I would love to see it Independent of the cli tool. Rather be part of the code. Something like this:
@task(working_dir="/home/user_a", container_image="a")
def do_a():
@task(working_dir="/home/user_b", container_image="a")
def do_b():
@workflow
def wf():
do_a()
do_b()
k

Ketan (kumare3)

12/22/2023, 4:16 PM
?
In imagespec? Or dockerfile
v

Vitali

12/22/2023, 4:17 PM
pardon updated the the message
it would be fine if you could that via imagespec
k

Ketan (kumare3)

12/22/2023, 4:18 PM
The problem with putting it there is we cannot guarantee that dir 1. In local and 2. in docker images it’s harder to change as you have to update code
This is why it should be available in every cli
Package run and register
v

Vitali

12/22/2023, 4:26 PM
I understand the issue with local. But would it be possible to set it for a container image. Since I set the container image I also know if the directory exist in the docker image or not. (and the default does not exist in many of ours.) I could imagen:
@task(container_working_dir="/home/a", container_image="a")
but also
@task(container_image=ImageSpec(working_dir="/home/a", base_image="a"))
I may miss some understanding, but the downside of the CLI is: • How do I set different working_dirs for task_a and task_b inside the same workflow? • The working_dir depends on the "env" (local or docker image) you execute it and therefore should also be specified where your env is specified. Or people would need to also type in the working dir for every task on the CLI.
k

Ketan (kumare3)

12/22/2023, 6:38 PM
@Vitali you are right
i think maybe a better ux is
Copy code
@task(container_image=Container(image="....", working_dir="..."))
def foo():
  ...
As this is more extensible and it is obvious you are controlling working dir
though i really hope we can eventually move most folks to imagespec
take this pain away
@Vitali if you agree, with the ux can you file an issue
v

Vitali

12/23/2023, 11:39 AM
@Ketan (kumare3) thanks! Here is the link to the issue: https://github.com/flyteorg/flyte/issues/4638
k

Ketan (kumare3)

12/23/2023, 4:52 PM
Actually I had some conversations with the team, would installing in the worksite solve the problem
Can we not simply set the right workdir in docker and then let it install in the right way
This way the code remains relatively error proof
@Vitali
v

Vitali

12/24/2023, 7:45 AM
Good point. I think this should work for everybody
k

Ketan (kumare3)

12/24/2023, 4:18 PM
Ok then I will just make the default as current dir • awesome 👏
v

Vitali

01/08/2024, 9:58 AM
@Ketan (kumare3) and @Eduardo Apolinario (eapolinario) since the ticket was closed, I wanted to understand what are the next steps to resolve the described issue? Do have another ticket for this? Or do I just miss howto implement it on our side?
k

Ketan (kumare3)

01/08/2024, 4:13 PM
Ohh sorry never realized it was closed, let me check if we changed the default
I will follow up
v

Vitali

01/15/2024, 10:26 AM
@Ketan (kumare3) Any updates?
k

Ketan (kumare3)

01/15/2024, 4:04 PM
It will be part of next flytekit release.
4 Views