It looks like this in the UI, with many sub tasks ...
# ask-the-community
f
It looks like this in the UI, with many sub tasks named with xgbse_train. Is there a way to make the task names to show the different values of tenure, e.g. xgbse_train_1, xgbse_train_2, …xgbse_train_24?
y
Copy code
@dynamic
def train_foreach_tenure(df, ...)
    ...
        training_task(df=data).with_overrides(name=f"train-tenure-{tenure}")
f
Thanks @Yee!
I got this error when I do that.
Copy code
return object.__getattribute__(self, name)
AttributeError: 'DataFrame' object has no attribute 'with_overrides'
this task returns a pd.DataFrame
y
you mean in local execution right?
f
yes
I tried xgbse_train(…).with_overrides(name=f”train-tenure-{tenure}“) in flyte remote , it ran fine but I don’t see the task names being changed.
@Yee, the dynamic workflow was inside another workflow if that makes the difference.
s
@Frank Shen, would you mind trying
with_overrides
on a simple task and see if that works?
f
@Samhita Alla, I tried with a simple task, and nothing works.
s
It works for me.
Copy code
@dynamic
def dynamic_wf():
    t1(x=x).with_overrides(name="Hello-hello")
Which flytekit version have you installed?
f
@karthikraj, what flyte version is running in Flyte WM server ?
@Samhita Alla, thanks for verifying. The only thing I did differently was I have a param in the name of the task.
Copy code
@workflow
def wf_train_parallel():
    for tenure in range(1, 25, 1):
        xgbse_training(
            tenure=tenure,
            df=df
        ).with_overrides(name=f"xgbse_training_{tenure}")
In this case, neither workflow nor dynamic works.
k
@Samhita Alla Does that mean which version is available in the propeller pod's image? This is the image & tag we are using for flytepropeller. Or, Do you want the local version of the flytekit where we are running pyflyte run commands to run our workflows?
Copy code
repository: pingsutw/flytepropeller 
    tag: fcfce5e490382297e88e8f8c5ca02084acf6a4d8
s
Yeah, I'm asking for the flytekit version.
@Frank Shen, I'm assuming the workflow you're referring to is a dynamic workflow since the code snippet you attached won't work.
with_overrides
is working in a dynamic workflow but not in a workflow. cc @Eduardo Apolinario (eapolinario)
f
@Samhita Alla, I tried both workflow and dynamic, neither has worked.
s
It works for me. I installed flytekit 1.5
f
@Samhita Alla, in my custom docker image, I am using flytekit==1.2.4. Is this the place to upgrade the flytekit to 1.5? Or the Flyte admin server’s flytekit version needs to be upgraded? CC: @karthikraj
s
Yeah, please update the flytekit version in your docker image.
f
@Samhita Alla, I also have other flyte packages like below in the docker image. Do I have to upgrade them all with the flytekit upgraded?
Copy code
flytekitplugins-snowflake==1.2.4
flytekitplugins-spark==1.2.4
I have a long dependency list. That will for sure cause conflicts.
s
Right. It'd be great if you can bump the versions coz 1.2.x is quite old.
149 Views