1 new commit pushed to

<https://github.com/flyteorg/flyte-attendant/tree/main|main>

by cosmicBboy

<https://github.com/flyteorg/flyte-attendant/commit/c4b199293d270a55cb5cbd44787ff4c345056cc7|c4b19929>

- add loading text flyteorg/flyte-attendant

#211 Add registerLaunchPlan with action to SimpleSdkLaunchPlanRegistry Pull request opened by pablocasares Read then delete this section TL;DR Added a new method in SimpleSdkLaunchPlanRegistry to allow to register launchplans by loading all and executing an action per each one of them. Type ☐ Bug Fix ☑︎ Feature ☐ Plugin Are all requirements met? ☑︎ Code completed ☐ Smoke tested ☑︎ Unit tests added ☐ Code documentation added ☐ Any pending items have an associated Issue Complete description How did you fix the bug, make the feature etc. Link to any design docs etc Added a new method in SimpleSdkLaunchPlanRegistry to allow to register launchplans by loading all and executing an action per each one of them. Tracking Issue _Remove the '_fixes_' keyword if there will be multiple PRs to fix the linked issue_ fixes https://github.com/flyteorg/flyte/issues/ Follow-up issue NA OR https://github.com/flyteorg/flyte/issues/ flyteorg/flytekit-java GitHub Actions: build-snapshot DCO: DCO ✅ 1 other check has passed 1/3 successful checks

6 new commits pushed to

<https://github.com/flyteorg/flyte-attendant/tree/main|main>

by cosmicBboy

<https://github.com/flyteorg/flyte-attendant/commit/fbda3ee1d8b599abd9442fd791b5c7d1c17514f6|fbda3ee1>

- ignore katacoda tutorials

<https://github.com/flyteorg/flyte-attendant/commit/af976bda2e7330202c2da3899e19b1bd9b8ac9a8|af976bda>

- update cache version

<https://github.com/flyteorg/flyte-attendant/commit/94de578bd8c0b861a972b90112f14962fe75c36b|94de578b>

- update app

<https://github.com/flyteorg/flyte-attendant/commit/b1c5a3df691d7b3acb27165b6dc5c19cab6b234f|b1c5a3df>

- app typos

<https://github.com/flyteorg/flyte-attendant/commit/ba43b47aeaa04860763d0f47a671c66e91fe4492|ba43b47a>

- update app

<https://github.com/flyteorg/flyte-attendant/commit/5973e823a0cfe7b0d1d95de238b99ea06260e6e1|5973e823>

- update app copy flyteorg/flyte-attendant

1 new commit pushed to

<https://github.com/flyteorg/flytesnacks/tree/master|master>

by cosmicBboy

<https://github.com/flyteorg/flytesnacks/commit/878271d118e96862ceaf34c4e0293be9c1e6d5b9|878271d1>

- Add gate node example to the control flow section of flytesnacks (#968) flyteorg/flytesnacks

Release - flytesnacks v0.3.181 New release published by flyte-bot Changelog • 878271d Add gate node example to the control flow section of flytesnacks (#968) • 3f33555 add gitignore/dockerignore section to fast registration (#969) flyteorg/flytesnacks

#3473 [BUG] Secret name should be validated on flytekit side and propeller should handle auto aborting/cancelling it Issue created by pmahindrakar-oss Describe the bug Currently workflows using non-conformed secrets cause error when propeller tries to create pods using the created CRD

flytepropeller-1 flytepropeller E0315 17:26:22.419072       1 workers.go:102] error syncing 'flyte-attendant-development/fbf61da4211b749988f0': failed at Node[n1]. RuntimeExecutionError: failed during plugin execution, caused by: failed to execute handle for plugin [container]: [Invalid] failed to create resource, caused by: Pod "fbf61da4211b749988f0-n1-0" is invalid: [spec.containers[0].env[26].name: Invalid value: "_FSEC_ARN:AWS:SECRETSMANAGER:US-EAST-2:356633062068:SECRET:_OPENAI_API_KEY-QA0K0U": a valid environment variable name must consist of alphabetic characters, digits, '_', '-', or '.', and must not start with a digit (e.g. 'my.env-name',  or 'MY_ENV.NAME',  or 'MyEnvName1', regex used for validation is '[-._a-zA-Z][-._a-zA-Z0-9]*'), spec.containers[0].env[26].valueFrom.secretKeyRef.name: Invalid value: "arn:aws:secretsmanager:us-east-2:356633062068:secret:": a lowercase RFC 1123 subdomain must consist of lower case alphanumeric characters, '-' or '.', and must start and end with an alphanumeric character (e.g. '<http://example.com|example.com>', regex used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*')]

There is no progress that propeller can make with this workflow . This ticket tracks. • Adding validation in flytekit to disallow non-conforming secret names to be registered • Propeller should auto abort/cancel such tasks and send the validation error over so that admin can showcase the issue and the UI can render it • Existing workflows which bypassed this check should some how be flagged Expected behavior Disallow incorrect workflows and propeller should abort if any such incorrect workflows fall through the cracks and flyteadmin and UI should be able to show the right issue in the workflow Additional context to reproduce No response Screenshots No response Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

#3474 drag-and-pull in graph view doesn't work for very large dynamic workflows Issue created by cosmicBboy Update: This issue is now for adding drag-and-pull support over nested graph nodes. Currently, users can drag graphs if they click on the canvas but not on any expanded nested nodes. Original: Workflow example: https://playground.hosted.unionai.cloud/console/projects/flytesnacks/domains/development/executions/ar5t8chs5kbnbg69njmw This workflow is composed of two dynamic workflows and a bunch of other tasks, which produces a very large graph in the graph view. When dragging and pulling within the dynamic graph the view doesn't slide in the direction as expected. flyteorg/flyte

#3475 Cached Dynamic Task No Cache Icon in the execution detail page Issue created by eugenejahn There are 2 problems: 1. the first time execute the dynamic task, it won't show cache icon (should show the cache_populated icon) because the api payload doesn;t return any cache status 2. the second time execute the dynamic task, it will show the cache icon, but it can't expand. because the api payload doesn't return children tasks. First time executed the dynamic task : (don't have the cache icon) https://demo.nuclyde.io/console/projects/uxsandbox/domains/development/executions/apn7n4p8b8rgbgdccm6r?duration=all Second time executed the Dyanmic Task: (have the cache icon) https://demo.nuclyde.io/console/projects/uxsandbox/domains/development/executions/arlnv6p6zmqvc8rgxq92?duration=all Slack Discussion: https://flyte-org.slack.com/archives/CP2HDHKE1/p1655227632603389 flyteorg/flyte

#526 Improvement: Refactor entity search Issue created by jsonporter Currently the entity search (workflows, tasks, launch plans, etc) is a simple string search that requires loading the full search space. We want to refactor this experience so that the search doesn't need to pre-fetch all entities and instead and submit a search query to the backend. Related: #385 flyteorg/flyteconsole

#3476 [BUG] Flyte Console UI does not update workflow signature on change Issue created by myz540 Describe the bug The

flyteconsole

UI for workflows typically shows a preview of the workflow's inputs. However, when changes are made the workflow's signature and the registration is updated, the UI continues to show the outdated workflow signature rather than the latest, most recent version's signature. This is a pretty minor bug since clicking on

Launch Workflow

does bring up the correct workflow inputs. Expected behavior The

flyteconsole

UI should show the latest version of a workflow's signature. Additional context to reproduce 1. Create a dummy project and workflow

from flytekit import task, workflow


@task
def addition_task(a: int, b: int) -> int:
    return a + b


@workflow
def dummy_wf(a: int, b: int) -> int:
    return addition_task(a=a, b=b)

2. package and register the project + workflow. The UI should look as follows: 3. Make change to the workflow's signature

from flytekit import task, workflow


@task
def addition_task(a: int, b: int) -> int:
    return a + b


@workflow
def dummy_wf(a: int, b: int, c: int, d: str) -> int:
    return addition_task(a=a, b=b)

4. Package and register the updated workflow. Note the UI still shows the older version's signature despite there being a newer / more recent version Actually launching the workflow shows the updated signature Screenshots No response Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

#332 [WIP] Add nil check in databricks plugin Pull request opened by pingsutw TL;DR Add nil check in databricks plugin. should fix https://flyte-org.slack.com/archives/CP2HDHKE1/p1678980405562029?thread_ts=1678831311.261609&cid=CP2HDHKE1 Type ☑︎ Bug Fix ☐ Feature ☐ Plugin Are all requirements met? ☐ Code completed ☐ Smoke tested ☐ Unit tests added ☐ Code documentation added ☐ Any pending items have an associated Issue Complete description ^^^ Tracking Issue ^^^ Follow-up issue NA flyteorg/flyteplugins

#505 Bug: Nullref in node executions Issue created by ursucarina Repro: • open any workflow with multiple task executions • start clicking several executions in a row. flyteorg/flyteconsole

#466 [Graph] Bug: Workflow Versions Graph not centered Issue created by jsonporter When viewing workflow versions, the static graph that is rendered is not correctly centered/sized on initial load. We need to fix this so that the graph is centered and correctly sized/zoomed to fit into the container on initial load. Example: https://demo.nuclyde.io/console/projects/cluster-deploy/domains/development/workflows/workflows.driver.create_org/version/4edd875cbe683f20023415de25c0cacd264f1f7e flyteorg/flyteconsole

#393 [CleanUp] Flyteconsole cleanups task Issue created by anrusina This task is created to cover set of clean-ups which don't have bug attached and mainky needed to support our code than to fix any issues. This Ticket number should be mentioned in code as:

TODO FC#393: what's need to be done

whenever cleanup is needed. Current list of cleanups: Simplify

WorkflowTabs

usage in NodeExecutionDetailsPanelContent Remove inputs/outputs fields from

ExecutionData

interface, ensure that they are not used anymore Unify

showOnHoverClass

usage - move it to commonStyles, for both parent, and child style Move all Core. and Admin. enums to one file:

src/models/enums.ts

(could be found by

It's an ENUM exports

string) move

createWorkflowName

into src/components/Project/test/ProjectWorkflows.test.tsx Storybooks for all olf flyteconsole components should start with

Legacy/

path in stories location(eg.

const stories = storiesOf('Navigation/NavBar', module);

) to ensure that we now that the y should be reimplemented/updated up to new standart flyteorg/flyteconsole

#385 [Improvement] Too many workflows initially loaded when person opens Workflows tab Issue created by anrusina Motivation Update: this ticket will need to be part of a broader refactor #385 Currently when person opens Workflow tab we immediately are requesting all workflows with the limit of 10000, and then to show

Last 10 executions

on each of returned workflow we perform executions call It leads to a lot of redundant calls, information from which user most probably won't even see. Solution Request paginated workflow items for example by using:

useWorkflowNameList

from

src/components/hooks/useNamedEntity.ts

, which already have limit set to

limits.DEFAULT === 25

Load more items when user have already scrolled through have of the list. flyteorg/flyteconsole

#382 [Project Dashboard] Add Global Project level settings to the Project Dashboard page Issue created by anrusina Update: Default case will be showing Global config and if we detect that overrides are present then we want to show both the global and overrides separately (as shown in image). However what is not captured in the image is that we want each section to be (1) collapsable and (2) in the case where an override is present we want to render the Global settings component as collapsed. Acceptance Criteria: • Component for global settings will always be present • Component for project settings will be shown only if there are overrides • Both components are collapsable • If overrides are present, Global component init render should be collapsed. Motivation If current Project/Domain doesn't have it's own uniques project level settings than we hide whole Domain Settings sections. It can lead to a wrong assumption that none of the fields are used in this case, or that there is no setup possible. However when default Project/Domain setting is empty we always use Global/Domain settings instead. They should be shown in the app to ensure that users know what they are working with. Goal Show globall Project/Domain settings Provide Amount of Workflows / Executions / Tasks without need to request their details Blockers We need an API from BackEnd to access such settings. It is covered by: flyteorg/flyte#2322 Ideally we would also want an API to get total number of workflows / executions per period / tasks to show this numbers in the header. Currently we are requesting a whole list of Workflows and Tasks with 1000 limit, and count amount of items inside flyteorg/flyteconsole

#379 [Graph]For specific Workflow Version Graph view shows 404 error Issue created by anrusina Issue For one specific workflow version a 404 is shown for Graph views . Repro steps. • Open https://demo.nuclyde.io/console/projects/flytesnacks/domains/development/executions/f536157ef1e4f459c84d?duration=all Obesrve: • Navigate to - > View All workflow versions and try to select

lC0y/z2YQl3FJ5qO7c441w==

• Filter executions by

lC0y/z2YQl3FJ5qO7c441w==

version, open execution and navigate to Graph tab Goal There are few possible solutions, the one should be picked depending on why issue happens. If Workflow version data is not accessible /valid (not present any more): • hide it from versions list OR • a more relevant message with error/ info would be shown If Workflow Version could be accessed: • EnsureThe workflow version data would be rendered properly flyteorg/flyteconsole

#375 [DAG v2] The specific dynamic workflow doesn't show child tasks in Graph and in Timeline Issue created by anrusina Issue Some of the dynamic workflows aren't showing as such in Graphic and Timeline view. Example: https://demo.nuclyde.io/console/projects/flytesnacks/domains/development/executions/ejln5s2l03?duration=all Observe: • Node tab includes childs under

my_subwf

• Graph and timeline show it up as a regular python task Most probably the issue is connected to DagV2 parsing. Goal Show al child tasks in Timeline and Graph view flyteorg/flyteconsole

#3477 [BUG] [flytekit] When enter a number with a lot of decimal places in parameter of type List[float] in Flyte Console the number change Issue created by zm8597 Describe the bug When we input float numbers with a bigger number of decimal places into launch plan parameter of type List[float] and we run the launch plan the numbers change. This is only the case with numbers that have more decimal places. When run locally everything works fine. Expected behavior Numbers in a list don't change. Additional context to reproduce No response Screenshots No response Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

#332 [BUG][Timeline] Timeline is crashing for one specific workflow, with failed NodeExecution Issue created by anrusina Failing Workflow The Crash for this workflow was fixed. However, the fix was mainly an overcome of corner case than a proper fix. Workflow execution in question: https://demo.nuclyde.io/console/projects/cluster-deploy/domains/development/executions/f76101f1a820a4dda8c4?duration=all Looking closer into it I see that even though node execution is failed, it looks like one of its child is still running, for 7400h+, see screenshot. Other Workflows with similar behavior: From 10 month ago: run fca7d008dc9ab42f2856 From a week ago: https://demo.nuclyde.io/console/projects/flytesnacks/domains/development/executions/ad4j68tk2wrx7bjw6s9h?duration=all Where wan of a map child tasks are still in the "Running state", while parent is FAILED BE Conversation in Slack: https://unionai.slack.com/archives/C01H0FN1NJX/p1647536794593289 What we currently have The latest solution in this PR Includes check that if task is running longer than 1 week (WEEK_DURATION_SEC) we would assume that it wasn't properly finished. In such case we will mark it with state ABORTED and set its length to

allegedDurationSec

- longest currently available duration happened earlier in the Dag tree. Proper solutions Ideally we should: • Mark status as ABORTED if parent item has FAILED state (which is not that easy to do with current DAG system) • Fix backend to ensure that status is updating properly flyteorg/flyteconsole

#319 [Timeline] Populate info only for open timeline items ( up to 1 level deep) Issue created by anrusina Currently in Timeline we request status of all node executions in a specific workflow execution. As result if Workflow execution has 50+ items, it can take a long time for first update to occur. It is very visible with dynamic tasks were currently only parent item could be visible. To improve the behavior, we want to populate nodeExecution info only for items which are already visible ( root nodes), when user expands node - we want to populate nodeExecution info for first child of expanded node. In this manner we will increase/ request nodes depending on user interest in them. flyteorg/flyteconsole

#3479 Timeline: Search bar Issue created by jsonporter Motivation: Why do you think this is important? Provide timeline view a search bar that searches across values in the "Task Name" section of the timeline. Goal: What should the final outcome look like, ideally? Acceptance Criteria: • Should do a string search across NodeId and Task Name. • Should have a debounce protection on key events - set at 300 ms • Should have a clear/reset functionality • detect key event "escape" • render a small "x" on right side of text input (note: not included in comps) Propose: Link/Inline OR Additional context No response Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

#3480 [UI Feature] Show all launchplan associated with a workflow in workflow details page Issue created by pradithya Motivation: Why do you think this is important? It's currently not possible from flyte console to see all available launchplan associated with a workflow. Goal: What should the final outcome look like, ideally? Launch plan is one resource that doesn't have much visibility in flyte console. Having no visibility of launch plan is not critical as long as users trigger a workflow via deliberate action (either using flyte remote or create execution from UI itself), however it become inconvenience when users have scheduled workflow. With scheduled workflow users set the schedule and launch plan's inputs in advance. There also can be multiple launch plan for a given workflow with different schedule and different inputs. And since the execution happens later than the launch plan creation itself, it is important for the users to be able to visually validate all launch plan that's associated with a workflow, their configuration (i.e. schedule, inputs) and whether they are active to ensure that their scheduled launch plans are ready. Propose: Link/Inline OR Additional context Ideally, launchplan information is available within workflow details page (i.e /projects/`project`/domains/`domain`/workflows/`workflow`). The information can be served as a table similar to the list of execution and the content is similar to what

flytectl get launchplan

command provides e.g.

--------- ------------------------------------------------ ------ ------- ------------------------------------------- -------------- ---------
| VERSION | NAME                                           | TYPE | STATE | SCHEDULE                                  | INPUTS       | OUTPUTS |
 --------- ------------------------------------------------ ------ ------- ------------------------------------------- -------------- ---------
| aa8d820 | awesome_workflow.workflows.launchplan.schedule |      |       | map[cronSchedule:map[schedule:0 10 * * 1] | kickoff_time | o0      |
|         |                                                |      |       | kickoffTimeInputArg:kickoff_time]         |              |         |
 --------- ------------------------------------------------ ------ ------- ------------------------------------------- -------------- ---------
| fe791e5 | awesome_workflow.workflows.launchplan.schedule |      |       | map[cronSchedule:map[schedule:0 10 * * 1] | a            | o0      |
|         |                                                |      |       | kickoffTimeInputArg:kickoff_time]         | b            |         |
|         |                                                |      |       |                                           | kickoff_time |         |
 --------- ------------------------------------------------ ------ ------- ------------------------------------------- -------------- ---------

and clicking one of the launch plan will lead to opening launchplan details page. (related to flyteorg/flyteconsole#407) Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

#1683 [UI Feature] Use doc strings to populate task description Issue created by jsonporter Motivation: Why do you think this is important? We want to surface any doc string documentation that users provide in the UI. The response from get tasks will include a "description" value when these data are present. see: https://docs.google.com/document/d/1m_-_QMloU7pLcnOQU8OUqfyGhPaBwB4XxYZiZIMOgq0/edit# Goal: What should the final outcome look like, ideally? Propose: Link/Inline OR Additional context No response Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

#294 [UI Feature] Use doc strings to populate description on Workflow search Ux Issue created by jsonporter Motivation: Why do you think this is important? The new Workflow search UX now displays various metadata about a workflow - included in that is a field for description. We want to surface any "description" values user included in their documentation. see: https://docs.google.com/document/d/1m_-_QMloU7pLcnOQU8OUqfyGhPaBwB4XxYZiZIMOgq0/edit# Goal: What should the final outcome look like, ideally? Propose: Link/Inline OR Additional context No response Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes On Fix notify threads: https://flyte-org.slack.com/archives/CNMKCU6FR/p1645732559319929 flyteorg/flyteconsole

#3481 [Core] Better execution search capabilities (search by inputs) Issue created by michaels-lyft Motivation: Why do you think this is important? This would allow us to quickly find a past execution based on either inputs, outputs, sub-task/workflow names, launch plans, or logs. In the past, we've run into situations were we would like to restart a workflow from a few days/weeks ago. But we had to resort to a linear search in order to find the right one. Goal: What should the final outcome look like, ideally? A search bar somewhere on the UI that either allows for more complex search languages (like Kibana or Sourcegraph) or have more attributes that can be used to search (eg. one attribute each for input, output, sub-task name, sub-workflow name, logs). Propose: Link/Inline OR Additional context No response Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

1 new commit pushed to

<https://github.com/flyteorg/flyte-attendant/tree/main|main>

by cosmicBboy

<https://github.com/flyteorg/flyte-attendant/commit/3d01b39b3bfabfdd9ea3f47e500f1866c4ff5d1e|3d01b39b>

- debug auth flyteorg/flyte-attendant

1 new commit pushed to

<https://github.com/flyteorg/flyte-attendant/tree/main|main>

by cosmicBboy

<https://github.com/flyteorg/flyte-attendant/commit/e77600ec4ec88d4f1c48899673e525313968412f|e77600ec>

- revert authType to clientSecret flyteorg/flyte-attendant

#3483 [BUG] flytectl fails to unmarshal execution spec with certain input key Issue created by honnix Describe the bug If execution spec has input parameter named as

n

,

y

,

no

,

yes

, etc.

flytectl

fails to unmarshal the spec correctly. As an example:

inputs:
  n: 42

flytectl

wil fail with

Error: no matching type for [false]
{"json":{},"level":"error","msg":"no matching type for [false]","ts":"2023-03-14T10:48:56-04:00"}

After a quick investigation, this is because of using sigs.k8s.io/yaml in flytectl: https://github.com/flyteorg/flytectl/blob/cea39d9bdf2476f9a5313d1bf19bf08b3923237a/cmd/create/execution_util.go#L13. This yaml library depends on

go-yaml

v2, which does not support YAML 1.2 specification, see go-yaml/yaml#214. If an input argument is named as

n

, when creating an execution via a execution spec file, that argument must be quoted as

"n"

, otherwise

flytectl

would fail because when unmarshalling

n

is parsed as

false

as the input parameter name. Expected behavior

flytectl

should correctly unmarshal the execution spec that is compliant with YAML 1.2 specification. Additional context to reproduce No response Screenshots No response Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

#541 Pick up flytepropeller compiler changes for generic structured datasets Pull request opened by katrogan TL;DR Picks up changes from flyteorg/flytepropeller#536 Type ☑︎ Bug Fix ☐ Feature ☐ Plugin Are all requirements met? ☐ Code completed ☐ Smoke tested ☐ Unit tests added ☐ Code documentation added ☐ Any pending items have an associated Issue Complete description How did you fix the bug, make the feature etc. Link to any design docs etc Tracking Issue NA Follow-up issue NA flyteorg/flyteadmin GitHub Actions: End2End Test / End to End tests GitHub Actions: Integration Test / Integration tests ✅ 8 other checks have passed 8/10 successful checks