It looks like this in the UI, with many sub tasks ...
# flyte-supports
salmon-refrigerator-32115
04/05/2023, 5:33 PMIt looks like this in the UI, with many sub tasks named with xgbse_train.
Is there a way to make the task names to show the different values of tenure, e.g. xgbse_train_1, xgbse_train_2, …xgbse_train_24?
t
thankful-minister-83577
04/05/2023, 5:47 PM
Copy code
@dynamic
def train_foreach_tenure(df, ...)
...
training_task(df=data).with_overrides(name=f"train-tenure-{tenure}")s
salmon-refrigerator-32115
04/05/2023, 6:02 PMThanks @thankful-minister-83577!
salmon-refrigerator-32115
04/05/2023, 6:14 PMI got this error when I do that.
Copy code
return object.__getattribute__(self, name)
AttributeError: 'DataFrame' object has no attribute 'with_overrides'salmon-refrigerator-32115
04/05/2023, 6:14 PMthis task returns a pd.DataFrame
t
thankful-minister-83577
04/05/2023, 6:18 PM
you mean in local execution right?
s
salmon-refrigerator-32115
04/05/2023, 6:18 PMyes
salmon-refrigerator-32115
04/05/2023, 7:29 PMI tried xgbse_train(…).with_overrides(name=f”train-tenure-{tenure}“) in flyte remote , it ran fine but I don’t see the task names being changed.
salmon-refrigerator-32115
04/05/2023, 9:42 PM@thankful-minister-83577, the dynamic workflow was inside another workflow if that makes the difference.
t
tall-lock-23197
04/07/2023, 8:36 AM
@salmon-refrigerator-32115, would you mind trying
with_overridess
salmon-refrigerator-32115
04/07/2023, 11:54 PM@tall-lock-23197, I tried with a simple task, and nothing works.
t
tall-lock-23197
04/10/2023, 4:53 AM
It works for me.
Copy code
@dynamic
def dynamic_wf():
t1(x=x).with_overrides(name="Hello-hello")tall-lock-23197
04/10/2023, 4:53 AM
Which flytekit version have you installed?
s
salmon-refrigerator-32115
04/10/2023, 5:05 PM@delightful-lion-57360, what flyte version is running in Flyte WM server ?
salmon-refrigerator-32115
04/10/2023, 5:07 PM@tall-lock-23197, thanks for verifying.
The only thing I did differently was I have a param in the name of the task.
Copy code
@workflow
def wf_train_parallel():
for tenure in range(1, 25, 1):
xgbse_training(
tenure=tenure,
df=df
).with_overrides(name=f"xgbse_training_{tenure}")salmon-refrigerator-32115
04/10/2023, 5:07 PMIn this case, neither workflow nor dynamic works.
d
delightful-lion-57360
04/10/2023, 5:22 PM@tall-lock-23197 Does that mean which version is available in the propeller pod's image?
This is the image & tag we are using for flytepropeller.
Or, Do you want the local version of the flytekit where we are running pyflyte run commands to run our workflows?
Copy code
repository: pingsutw/flytepropeller
tag: fcfce5e490382297e88e8f8c5ca02084acf6a4d8t
tall-lock-23197
04/11/2023, 5:58 AM
Yeah, I'm asking for the flytekit version.
tall-lock-23197
04/11/2023, 6:45 AM
@salmon-refrigerator-32115, I'm assuming the workflow you're referring to is a dynamic workflow since the code snippet you attached won't work.
with_overridess
salmon-refrigerator-32115
04/11/2023, 4:07 PM@tall-lock-23197, I tried both workflow and dynamic, neither has worked.
t
tall-lock-23197
04/14/2023, 9:05 AM
It works for me. I installed flytekit 1.5
s
salmon-refrigerator-32115
04/14/2023, 3:39 PM@tall-lock-23197, in my custom docker image, I am using flytekit==1.2.4. Is this the place to upgrade the flytekit to 1.5?
Or the Flyte admin server’s flytekit version needs to be upgraded?
CC: @delightful-lion-57360
t
tall-lock-23197
04/14/2023, 3:46 PM
Yeah, please update the flytekit version in your docker image.
s
salmon-refrigerator-32115
04/14/2023, 4:37 PM@tall-lock-23197,
I also have other flyte packages like below in the docker image.
Do I have to upgrade them all with the flytekit upgraded?
Copy code
flytekitplugins-snowflake==1.2.4
flytekitplugins-spark==1.2.4salmon-refrigerator-32115
04/14/2023, 4:38 PMI have a long dependency list. That will for sure cause conflicts.
t
tall-lock-23197
04/14/2023, 4:48 PM
Right. It'd be great if you can bump the versions coz 1.2.x is quite old.
151 Views