https://flyte.org logo
Join Slack
Powered by
Hi, I have a strange error where I am creating sev...
# flyte-support
f
Hi, I have a strange error where I am creating several LaunchPlans with different names using 
flytekit.LaunchPlan.get_or_create
. However, when I try to register this archive with the launchplans, it fails on a particular one with the name: 
a-cst-old
, giving the error: 
Error: rpc error: code = InvalidArgument desc = missing name
It’s just this name and the fixed input (string) that’s changed, nothing else, if I prefix this name with something else like, it works. And on a related note, is there a way to delete, not just archive, all launchplans?
FWIW I have attached the protobuf here:
17_a-cst-old_3.pb
v
But @glamorous-carpet-83516 or @icy-agent-73298 could add to it
i
Hi @faint-monitor-96441 thats a very strange issue . Will need to check more on it . Also regarding deletion, currently we dont have a delete functionality in flyte and every object in flyte can be archived . This is by design i think . You can always choose to delete the archived ones from the DB .
So strange comes due to unmarshalling this proto works for both TaskSpec and Launchplan object. ``````
Following test demonstrates it. I am assuming you are using flytectl to register where you see this error.
Copy code
func TestUnmarshalProtoLaunchPlan(t *testing.T) {
	pbBytes, err := os.ReadFile("/Users/praful/Downloads/17_a-cst-old_3.pb")
	lp := &admin.LaunchPlan{}
	err = proto.Unmarshal(pbBytes, lp)
	assert.Nil(t, err)
	assert.Equal(t, "a-cst-old", lp.Id.Name)
}

func TestUnmarshalProtoTaskSpec(t *testing.T) {
	pbBytes, err := os.ReadFile("/Users/praful/Downloads/17_a-cst-old_3.pb")
	lp := &admin.TaskSpec{}
	err = proto.Unmarshal(pbBytes, lp)
	assert.Nil(t, err)
	assert.Equal(t, "a-cst-old", lp.Template.Id.Name)
}
@faint-monitor-96441 may be you can share the updated proto where this works for you and we can run both test on it. I bet it fails the 2nd test and hence it works for you.
flytectl register relies on the fact the object will be one of the following types and not multiple • admin.LaunchPlan • admin.WorkflowSpec • admin.TaskSpec In your case the proto is able to be unmarshalled without error into Launchplan and TaskSpec type and taskSpec type gets precedence in flytectl code and thats the one thats gets sent to admin for registration and fails since the Name field in the unmarshalled taskSpec proto is empty . If you unrmarshal it with Launchplan it has the correct Id.Name field
Cc: @high-park-82026
h
I downloaded an example from flytesnacks releases. And I do not see a way to distinguish between types based on the file. I believe the protocol based on such convention is not good enough… I can think of two ways to improve the package -> register behavior: 1. Change the naming of these files to include a prefix informing flytectl/pyflyte register about how to parse these (e.g. `1_tsk_blast`… `2_wf_blast`… etc.).. This change will need to happen in all SDKs (python, java, scala)… the register behavior will need to be backward compatible… 2. Create an envelope message for the data serialized in these files… something like below. And update the registration logic to always only unmarshal into that envelope type.
Copy code
message RegistrationEntity {
  oneof item {
    TaskTemplate task = 1;
    WorkflowTemplate workflow = 2;
...
  }
}
There is no way around backward compatibility ugliness here, we will have to fallback to existing logic. Both options require changes in flytekit. I do think the second option a tad better since it doesn’t rely on string comparisons and doesn’t assume the existence of “files” to begin with (this logic works just fine with REST APIs and programmatic APIs)… If we don’t have one already, we should file an issue on this and list these (and other?) options considered and maybe move the discussion there.. CC @thankful-minister-83577 @freezing-airport-6809 @high-accountant-32689
i
156 Views
Open in Slack
PreviousNext
Powered by