https://flyte.org logo
Powered by
#ask-the-community
Title
# ask-the-community
f

Frank Shen

04/05/2023, 5:33 PM
It looks like this in the UI, with many sub tasks named with xgbse_train. Is there a way to make the task names to show the different values of tenure, e.g. xgbse_train_1, xgbse_train_2, …xgbse_train_24?
y

Yee

04/05/2023, 5:47 PM
Copy code
@dynamic
def train_foreach_tenure(df, ...)
    ...
        training_task(df=data).with_overrides(name=f"train-tenure-{tenure}")
f

Frank Shen

04/05/2023, 6:02 PM
Thanks @Yee!
I got this error when I do that.
Copy code
return object.__getattribute__(self, name)
AttributeError: 'DataFrame' object has no attribute 'with_overrides'
this task returns a pd.DataFrame
y

Yee

04/05/2023, 6:18 PM
you mean in local execution right?
f

Frank Shen

04/05/2023, 6:18 PM
yes
I tried xgbse_train(…).with_overrides(name=f”train-tenure-{tenure}“) in flyte remote , it ran fine but I don’t see the task names being changed.
@Yee, the dynamic workflow was inside another workflow if that makes the difference.
s

Samhita Alla

04/07/2023, 8:36 AM
@Frank Shen, would you mind trying 
with_overrides
on a simple task and see if that works?
f

Frank Shen

04/07/2023, 11:54 PM
@Samhita Alla, I tried with a simple task, and nothing works.
s

Samhita Alla

04/10/2023, 4:53 AM
It works for me.
Copy code
@dynamic
def dynamic_wf():
    t1(x=x).with_overrides(name="Hello-hello")
Which flytekit version have you installed?
f

Frank Shen

04/10/2023, 5:05 PM
@karthikraj, what flyte version is running in Flyte WM server ?
@Samhita Alla, thanks for verifying. The only thing I did differently was I have a param in the name of the task.
Copy code
@workflow
def wf_train_parallel():
    for tenure in range(1, 25, 1):
        xgbse_training(
            tenure=tenure,
            df=df
        ).with_overrides(name=f"xgbse_training_{tenure}")
In this case, neither workflow nor dynamic works.
k

karthikraj

04/10/2023, 5:22 PM
@Samhita Alla Does that mean which version is available in the propeller pod's image? This is the image & tag we are using for flytepropeller. Or, Do you want the local version of the flytekit where we are running pyflyte run commands to run our workflows?
Copy code
repository: pingsutw/flytepropeller 
    tag: fcfce5e490382297e88e8f8c5ca02084acf6a4d8
s

Samhita Alla

04/11/2023, 5:58 AM
Yeah, I'm asking for the flytekit version.
@Frank Shen, I'm assuming the workflow you're referring to is a dynamic workflow since the code snippet you attached won't work. 
with_overrides
is working in a dynamic workflow but not in a workflow. cc @Eduardo Apolinario (eapolinario)
f

Frank Shen

04/11/2023, 4:07 PM
@Samhita Alla, I tried both workflow and dynamic, neither has worked.
s

Samhita Alla

04/14/2023, 9:05 AM
It works for me. I installed flytekit 1.5
f

Frank Shen

04/14/2023, 3:39 PM
@Samhita Alla, in my custom docker image, I am using flytekit==1.2.4. Is this the place to upgrade the flytekit to 1.5? Or the Flyte admin server’s flytekit version needs to be upgraded? CC: @karthikraj
s

Samhita Alla

04/14/2023, 3:46 PM
Yeah, please update the flytekit version in your docker image.
f

Frank Shen

04/14/2023, 4:37 PM
@Samhita Alla, I also have other flyte packages like below in the docker image. Do I have to upgrade them all with the flytekit upgraded?
Copy code
flytekitplugins-snowflake==1.2.4
flytekitplugins-spark==1.2.4
I have a long dependency list. That will for sure cause conflicts.
s

Samhita Alla

04/14/2023, 4:48 PM
Right. It'd be great if you can bump the versions coz 1.2.x is quite old.
Powered by