https://flyte.org
Join Slack
It looks like this in the UI, with many sub tasks named with xgbse_train. Is there a way to make the...
# flyte-support
s
It looks like this in the UI, with many sub tasks named with xgbse_train. Is there a way to make the task names to show the different values of tenure, e.g. xgbse_train_1, xgbse_train_2, …xgbse_train_24?
t
Copy code
@dynamic
def train_foreach_tenure(df, ...)
    ...
        training_task(df=data).with_overrides(name=f"train-tenure-{tenure}")
s
Thanks @thankful-minister-83577!
I got this error when I do that.
Copy code
return object.__getattribute__(self, name)
AttributeError: 'DataFrame' object has no attribute 'with_overrides'
this task returns a pd.DataFrame
t
you mean in local execution right?
s
yes
I tried xgbse_train(…).with_overrides(name=f”train-tenure-{tenure}“) in flyte remote , it ran fine but I don’t see the task names being changed.
@thankful-minister-83577, the dynamic workflow was inside another workflow if that makes the difference.
t
@salmon-refrigerator-32115, would you mind trying 
with_overrides
on a simple task and see if that works?
s
@tall-lock-23197, I tried with a simple task, and nothing works.
t
It works for me.
Copy code
@dynamic
def dynamic_wf():
    t1(x=x).with_overrides(name="Hello-hello")
Which flytekit version have you installed?
s
@delightful-lion-57360, what flyte version is running in Flyte WM server ?
@tall-lock-23197, thanks for verifying. The only thing I did differently was I have a param in the name of the task.
Copy code
@workflow
def wf_train_parallel():
    for tenure in range(1, 25, 1):
        xgbse_training(
            tenure=tenure,
            df=df
        ).with_overrides(name=f"xgbse_training_{tenure}")
In this case, neither workflow nor dynamic works.
d
@tall-lock-23197 Does that mean which version is available in the propeller pod's image? This is the image & tag we are using for flytepropeller. Or, Do you want the local version of the flytekit where we are running pyflyte run commands to run our workflows?
Copy code
repository: pingsutw/flytepropeller 
    tag: fcfce5e490382297e88e8f8c5ca02084acf6a4d8
t
Yeah, I'm asking for the flytekit version.
@salmon-refrigerator-32115, I'm assuming the workflow you're referring to is a dynamic workflow since the code snippet you attached won't work. 
with_overrides
is working in a dynamic workflow but not in a workflow. cc @high-accountant-32689
s
@tall-lock-23197, I tried both workflow and dynamic, neither has worked.
t
It works for me. I installed flytekit 1.5
s
@tall-lock-23197, in my custom docker image, I am using flytekit==1.2.4. Is this the place to upgrade the flytekit to 1.5? Or the Flyte admin server’s flytekit version needs to be upgraded? CC: @delightful-lion-57360
t
Yeah, please update the flytekit version in your docker image.
s
@tall-lock-23197, I also have other flyte packages like below in the docker image. Do I have to upgrade them all with the flytekit upgraded?
Copy code
flytekitplugins-snowflake==1.2.4
flytekitplugins-spark==1.2.4
I have a long dependency list. That will for sure cause conflicts.
t
Right. It'd be great if you can bump the versions coz 1.2.x is quite old.
150 Views
Open in Slack
PreviousNext
Powered by

Flyte

Flyte enables production-grade orchestration for machine learning workflows and data processing created to accelerate local workflows to production.

Powered by