Rupsha Chaudhuri
04/26/2023, 6:38 PM{
"id": {
"resourceType": "LAUNCH_PLAN",
"project": "my_project",
"domain": "production",
"name": "my_lp",
"version": "20949aa7bbea5421dfef9f96d4248c7a26a6f5d5"
},
"spec": {
"workflowId": {
"resourceType": "WORKFLOW",
"project": "my_project",
"domain": "production",
"name": "some_wf",
"version": "20949aa7bbea5421dfef9f96d4248c7a26a6f5d5"
},
"entityMetadata": {
"schedule": {
"rate": {
"value": 8,
"unit": "HOUR"
}
}
},
"defaultInputs": {},
"fixedInputs": {},
"labels": {},
"annotations": {},
"rawOutputDataConfig": {}
},
"closure": {
"state": "ACTIVE",
"expectedInputs": {},
"expectedOutputs": {},
"createdAt": "2023-04-26T18:23:39.503710Z",
"updatedAt": "2023-04-26T18:23:39.503710Z"
}
}
Ketan (kumare3)
04/27/2023, 4:18 AMPrafulla Mahindrakar
04/27/2023, 4:35 AMRupsha Chaudhuri
04/27/2023, 4:54 AMflytectl update launchplan --admin.endpoint $(FLYTE_ADMIN_HOST):443 -p $(FLYTE_PROJECT_NAME) -d $(FLYTE_PROJECT_DOMAIN) \
$(name) --activate --version $(IMAGE_VERSION)"
---------------------- ------------------- ------------------------------------------ ------------- ----------- ---------------------- -------------------------------- ----------------- -------------------- --------------------
| NAME | LAUNCH PLAN NAME | VERSION | TYPE | PHASE | SCHEDULED TIME | STARTED | ELAPSED TIME | ABORT DATA (TRUNC) | ERROR DATA (TRUNC) |
---------------------- ------------------- ------------------------------------------ ------------- ----------- ---------------------- -------------------------------- ----------------- -------------------- --------------------
| f9618b52b437ae85d000 | my_pipeline_lp | b699495922ea41c0849e7bd03f190783475552d3 | LAUNCH_PLAN | SUCCEEDED | 2023-04-26T18:21:44Z | 2023-04-26T18:21:49.735450044Z | 3565.693426035s | | |
---------------------- ------------------- ------------------------------------------ ------------- ----------- ---------------------- -------------------------------- ----------------- -------------------- --------------------
---------------------- ------------------- ------------------------------------------ ------------- --------- ---------------------- -------------------------------- ---------------- -------------------- --------------------
| NAME | LAUNCH PLAN NAME | VERSION | TYPE | PHASE | SCHEDULED TIME | STARTED | ELAPSED TIME | ABORT DATA (TRUNC) | ERROR DATA (TRUNC) |
---------------------- ------------------- ------------------------------------------ ------------- --------- ---------------------- -------------------------------- ---------------- -------------------- --------------------
| f9318b52b437ae85d000 | my_pipeline_lp | b699495922ea41c0849e7bd03f190783475552d3 | LAUNCH_PLAN | ABORTED | 2023-04-26T18:21:41Z | 2023-04-26T18:21:47.419096843Z | 170.893382172s | Terminated from UI | |
---------------------- ------------------- ------------------------------------------ ------------- --------- ---------------------- -------------------------------- ---------------- -------------------- --------------------Prafulla Mahindrakar
04/27/2023, 5:29 AMRupsha Chaudhuri
04/27/2023, 5:12 PMPrafulla Mahindrakar
04/27/2023, 7:05 PMRupsha Chaudhuri
04/27/2023, 7:06 PMPrafulla Mahindrakar
04/27/2023, 7:11 PMBut the other pipelines with schedules are working perfectlyThats odd that its just affecting this launchplan
Rupsha Chaudhuri
04/27/2023, 7:12 PMPrafulla Mahindrakar
04/27/2023, 7:14 PMflytectl get launchplan -p project -d domain name -o yaml
Rupsha Chaudhuri
04/27/2023, 7:15 PMPrafulla Mahindrakar
04/27/2023, 7:33 PMRupsha Chaudhuri
04/27/2023, 8:36 PMPrafulla Mahindrakar
04/28/2023, 1:48 AM2023-04-26T10:21:33.678573Z
2023-04-26T18:21:44Z
2023-04-26T18:21:41Z
Aleksei Potov
04/30/2023, 11:11 PMPrafulla Mahindrakar
05/01/2023, 7:11 AMlogger.yaml: |
logger:
level: 6
I also wrote a unit to check for the above params like this
func TestGetScheduleName(t *testing.T) {
ctx := context.Background()
schedule := models.SchedulableEntity{
BaseModel: adminModels.BaseModel{
ID: 2,
UpdatedAt: time.Now(),
},
SchedulableEntityKey: models.SchedulableEntityKey{
Project: "navigation",
Domain: "production",
Name: "daily_pipeline_8hr_lp",
Version: "b699495922ea41c0849e7bd03f190783475552d3",
},
Unit: admin.FixedRateUnit_HOUR * 8,
KickoffTimeInputArg: "kickoff_time",
}
str := GetScheduleName(ctx, schedule)
assert.Equal(t, "17193008309657693892", str)
}
And the GetScheduleName is only dependent on these
Project: "navigation",
Domain: "production",
Name: "daily_pipeline_8hr_lp",
Version: "b699495922ea41c0849e7bd03f190783475552d3",
https://github.com/flyteorg/flyteadmin/blob/master/scheduler/identifier/identifier.go#L26
From the previous conversations for these execution ID’s f9618b52b437ae85d000 and f9318b52b437ae85d000 the above launchplan params needed to generate schedule name are the same so this line should tell if another has been added https://github.com/flyteorg/flyteadmin/blob/master/scheduler/core/gocron_scheduler.go#L117
and add the schedule at this line https://github.com/flyteorg/flyteadmin/blob/master/scheduler/core/gocron_scheduler.go#L139
But for some wierd reason we are seeing L139 being executed twice for the same launchplan params . Suspecting some race if thats the case but still would be great to pin point from the logs this exact issue. Having the scheduler logs at debug level would help lotAleksei Potov
05/01/2023, 4:54 PMPrafulla Mahindrakar
05/01/2023, 5:17 PMAleksei Potov
05/01/2023, 6:34 PM