Yubo Wang
02/07/2023, 11:16 PMConditions are limited to certain binary and logical operators and can only be performed on primitive values.
Dan Rammer (hamersaw)
02/07/2023, 11:17 PMYubo Wang
02/07/2023, 11:18 PMAnkit Goyal
02/07/2023, 11:20 PMDan Rammer (hamersaw)
02/07/2023, 11:21 PMYubo Wang
02/07/2023, 11:21 PM@dataclass_json
@dataclass
class TFTrainerConfig(object):
dataset_config: DatasetLoaderConfig
hyperparameters: Hyperparameters
column_names: List[str]
model_code: str
batch_size: int
num_training_steps: int
num_eval_steps: int
num_eval_frequency: int
model_output_names_map: Optional[dict]
@workflow
def tf2_trainer_wf(trainer_config: TFTrainerConfig):
res = conditonal("batch_size_greater")
.if_(trainer_config.batch_size > 10)
.then()
@workflow
def tf2_trainer_wf(trainer_config: TFTrainerConfig):
res = conditonal("batch_size_greater")
.if_(trainer_config.model_output_names_map not None)
.then()
Dan Rammer (hamersaw)
02/07/2023, 11:28 PM@task
def process_success(...):
# foo
@dynamic
def tf2_trainer_wf(trainer_config: TFTrainerConfig):
trainer_config.batch_size > 10:
process_success()
of course this spins up a separate Pod to process this and compiles / injects a new DAG. So there is some level of overhead.@task
def process(...) bool:
return trainer_config.model_output_names_map == None
@workflow
def tf2_trainer_wf(trainer_config: TFTrainerConfig):
condition = process(trainer_config)
res = conditonal("batch_size_greater")
.if_(condition)
.then()
which also has some level of overhead obviously.Yubo Wang
02/07/2023, 11:31 PMDan Rammer (hamersaw)
02/07/2023, 11:32 PMYubo Wang
02/07/2023, 11:32 PMAnkit Goyal
02/07/2023, 11:35 PMtrainer_config
is a dataclass? If it was just an int, it would work; right?Yubo Wang
02/07/2023, 11:36 PMuse_model_bundle_creator
@workflow
def tf2_trainer_wf(use_model_bundle_creator: bool):
trainer_res = trainer_task()
res = conditional("use_model_bundle_creator").if_(use_model_bundle_creator.is_true()).then(model_bundle_creator())
trainer_res >> res
I got AttributeError: 'Condition' object has no attribute 'ref'
Dan Rammer (hamersaw)
02/07/2023, 11:42 PMdataclasses
stored as literals? If there was some way to encode the specific field of the dataclass in a Literal
proto then in propeller when we evaluate in the BranchNode you could look up the field and return the primitive value.Yubo Wang
02/07/2023, 11:45 PMDan Rammer (hamersaw)
02/07/2023, 11:46 PMEduardo Apolinario (eapolinario)
02/08/2023, 1:34 AMdataclasses
as literals. We lean pretty heavily on protobuf Struct.Yubo Wang
02/08/2023, 1:39 AMEduardo Apolinario (eapolinario)
02/08/2023, 1:42 AMYubo Wang
02/08/2023, 1:47 AMconditional("use_model_bundle_creator").if_(use_model_bundle_creator.is_true()).then(model_bundle_creator()).else_()
Samhita Alla
else
isn't supported. I think it makes sense to mandatorily input what needs to be done in case if
isn't traversed.Yubo Wang
02/08/2023, 7:16 AMSamhita Alla
fail
Niels Bantilan
02/08/2023, 3:12 PMconditional
construct is more like
x = 0 if cond else 1
as opposed to
if cond:
x = 0
... # technically don't need an else here
i.e. all conditions must be specified.
You could use a noop
task in the else clause to workaround thisYubo Wang
02/08/2023, 5:00 PMNiels Bantilan
02/08/2023, 5:02 PM@task
def noop(): ...
Yubo Wang
02/08/2023, 5:03 PMend()
function similar to the use of fail() that allows us to do nothing on a branch.Samhita Alla
Niels Bantilan
02/13/2023, 3:07 PMend()
use case? If I understand correctly, this would be for cases where you don’t want to do any thing in one of the conditional branches, right?Slackbot
02/13/2023, 3:07 PMDan Rammer (hamersaw)
02/13/2023, 3:48 PMYubo Wang
02/13/2023, 4:17 PM