GitHub
01/19/2023, 3:23 PMGitHub
01/19/2023, 3:55 PMGitHub
01/19/2023, 4:14 PM<https://github.com/flyteorg/flytekit-java/tree/master|master>
by narape
<https://github.com/flyteorg/flytekit-java/commit/ebc38e24bf9004d86d9bcb897974e88ebcdd4e97|ebc38e24>
- Make Build run on all branches (#172)
flyteorg/flytekit-javaGitHub
01/19/2023, 4:33 PMGitHub
01/19/2023, 4:33 PMGitHub
01/19/2023, 4:44 PMDynamicJobSpec
from previous the previous DynamicTask
execution during node recovery.
Type
☑︎ Bug Fix
☐ Feature
☐ Plugin
Are all requirements met?
☑︎ Code completed
☑︎ Smoke tested
☑︎ Unit tests added
☑︎ Code documentation added
☐ Any pending items have an associated Issue
Complete description
^^^
Tracking Issue
fixes flyteorg/flyte#3121
Follow-up issue
NA
flyteorg/flytepropeller
GitHub Actions: Goreleaser
GitHub Actions: Build & Push Flytepropeller Image
GitHub Actions: Bump Version
✅ 11 other checks have passed
11/14 successful checksGitHub
01/19/2023, 5:15 PMVALUE_IN
filter operation to get all executions/tasks
Tracking Issue
_Remove the '_fixes_' keyword if there will be multiple PRs to fix the linked issue_
flyteorg/flyte#661
flyteorg/flyteconsole
DCO: DCO
✅ 1 other check has passed
1/2 successful checksGitHub
01/19/2023, 5:26 PM<https://github.com/flyteorg/flyteconsole/tree/master|master>
by ursucarina
<https://github.com/flyteorg/flyteconsole/commit/924bb7e2fdab3c306de32a03924cf01a09f1c050|924bb7e2>
- chore: Refactor flyteconsole (#654)
flyteorg/flyteconsoleGitHub
01/19/2023, 5:26 PMGitHub
01/19/2023, 5:26 PMGitHub
01/19/2023, 6:22 PMGitHub
01/19/2023, 6:48 PM<https://github.com/flyteorg/flyteconsole/tree/master|master>
by ursucarina
<https://github.com/flyteorg/flyteconsole/commit/ab91dfc09a072aaa1eac7063b675d4ff92800c15|ab91dfc0>
- chore: update flyteidl version @1.3.4 (#673)
flyteorg/flyteconsoleGitHub
01/19/2023, 7:48 PM<https://github.com/flyteorg/flytepropeller/tree/master|master>
by pmahindrakar-oss
<https://github.com/flyteorg/flytepropeller/commit/20146304738db96e8a15cf9a824233676a7560ea|20146304>
- Update flyteidl version (#490)
flyteorg/flytepropellerGitHub
01/19/2023, 7:48 PMGitHub
01/19/2023, 9:08 PM<https://github.com/flyteorg/flytekit/tree/master|master>
by wild-endeavor
<https://github.com/flyteorg/flytekit/commit/5dd887cf5cbe817a35d682a32886f304a97fc910|5dd887cf>
- Flip the settings for channel and logger (#1415)
flyteorg/flytekitGitHub
01/19/2023, 11:34 PMAudience
in its handshake process. When using Flyte with Authentication enabled, Flyte Scheduler and Flyte Propeller talk to Auth0 server with ClientSecret flow. Having no option to set Audience parameter in ClientSecret flow has halted our deployment as we are not able to use Flyte with Authentication.
Read more in Audience: https://auth0.com/docs/secure/tokens/access-tokens/get-management-api-access-tokens-for-production#get-access-tokens
The reason this is important is it expands the use case of Flyte with users taking extra precaution by using Audience. It does not change anything for the users who do not want to use Audience, so it is a non-breaking change to make.
Goal: What should the final outcome look like, ideally?
Flytepropeller, Flyte Scheduler and any other backend service that uses ClientSecret flow for authentication should have the option to enable Audience via values.yaml configs.
An example for setting Audience within config is already present at Flyteadmin config:
flyte/charts/flyte-core/values-keycloak-idp-flyteclients-without-browser.yaml
Line 527 in </flyteorg/flyte/commit/01f7cd5c5fa7b713062fc8338821a956422639ba|01f7cd5>
Describe alternatives you've considered
I am not sure of an alternative solution in this case, the company policies are pretty tight to use Audience in every Authentication flow.
Propose: Link/Inline OR Additional context
Since ClientSecret flow is using Auth2 golang SDK so I dug a little bit into it and probably something that can help Flyte implement is this: golang/oauth2@01b79d9
Are you sure this issue hasn't been raised already?
☑︎ Yes
Have you read the Code of Conduct?
☑︎ Yes
flyteorg/flyteGitHub
01/19/2023, 11:39 PMGitHub
01/20/2023, 8:20 AMPhaseNotStarted
)
• Webapi plugin try to allocate token
2 round (state: PhaseAllocationTokenAcquired
, phase: queued
):
• Plugin fail to create a task, and will not update the task state. The state is PhaseAllocationTokenAcquired
in this round
• Task handler send the task event (queued) to admin
• However we transition the phase from queued to running here
• And we will record the node event (running) in the node executor
3 round (state: PhaseAllocationTokenAcquired
, phase: running
):
• Will try to create a task, and fail again
• In the screenshot below, you can see the node phase is running
, but the task state is queued
4+ round
• Repeat 3 round
Type
☑︎ Bug Fix
☐ Feature
☐ Plugin
Are all requirements met?
☑︎ Code completed
☑︎ Smoke tested
☑︎ Unit tests added
☐ Code documentation added
☐ Any pending items have an associated Issue
Complete description
Before
image▾
image▾
GitHub
01/20/2023, 8:29 AMGitHub
01/20/2023, 9:12 AMGitHub
01/20/2023, 1:36 PMGitHub
01/20/2023, 2:28 PMSdkBindingData
to a typed version SdkBindingData<?>
. This new version allows the user to know the inner type during the normal development and also helps to ensure that you are setting the current input to a task because if not you get a type mismatch error at compile time.
Describe alternatives you've considered
no alternatives
Propose: Link/Inline OR Additional context
flyteorg/flytekit-java#164
Are you sure this issue hasn't been raised already?
☑︎ Yes
Have you read the Code of Conduct?
☑︎ Yes
flyteorg/flyteGitHub
01/20/2023, 2:36 PMgetOutput("<attributeName>")
. This is difficult during the development because you need to check the output class or docs to know what are the available types, also writing the name manually produces typos errors that only appear at registration time. The proposal here is to have some way to use the output class during the workflow development on that way you can access directly to the output class attributes.
Goal: What should the final outcome look like, ideally?
SumTaskOutput class
@AutoValue
public abstract static class SumOutput {
public abstract SdkBindingData<Long> c();
public static SumOutput create(SdkBindingData<Long> c) {
return new AutoValue_SumTask_SumOutput(c);
}
}
Current status:
SdkBindingData fib3 = builder.apply("fib-3", SumTask.of(fib1, fib2)).getOutput("c");
Desire status:
SdkBindingData fib3 = builder.apply("fib-3", SumTask.of(fib1, fib2)).getOutputs().c();
This allows the user to auto-discover the output attributes using an IDE, and also. check that you choose only available attributes at compile time.
Describe alternatives you've considered
no alternatives
Propose: Link/Inline OR Additional context
flyteorg/flytekit-java#164
Are you sure this issue hasn't been raised already?
☑︎ Yes
Have you read the Code of Conduct?
☑︎ Yes
flyteorg/flyteGitHub
01/20/2023, 3:09 PMGitHub
01/20/2023, 4:11 PM<https://github.com/flyteorg/flytekit-java/tree/master|master>
by narape
<https://github.com/flyteorg/flytekit-java/commit/cd1026bda94387184864c6969b644c8be8df12dc|cd1026bd>
- Change SdkBindingData to be typed and add typed transform outputs #minor (#164)
flyteorg/flytekit-javaGitHub
01/20/2023, 6:24 PMGitHub
01/20/2023, 6:29 PMflytekit.clients.friendly.SynchronousFlyteClient
and flytekit.clients.raw.RawSynchronousFlyteClient
Tracking Issue
flyteorg/flyte#3095
Follow-up issue
NA
flyteorg/flytekit
GitHub Actions: build-plugins (3.8, flytekit-pandera)
GitHub Actions: build-plugins (3.8, flytekit-onnx-scikitlearn)
GitHub Actions: build-plugins (3.8, flytekit-onnx-pytorch)
GitHub Actions: build-plugins (3.8, flytekit-modin)
GitHub Actions: build-plugins (3.8, flytekit-kf-tensorflow)
GitHub Actions: build-plugins (3.8, flytekit-kf-pytorch)
GitHub Actions: build-plugins (3.8, flytekit-kf-mpi)
GitHub Actions: build-plugins (3.8, flytekit-k8s-pod)
GitHub Actions: build-plugins (3.8, flytekit-hive)
GitHub Actions: build-plugins (3.8, flytekit-greatexpectations)
GitHub Actions: build-plugins (3.8, flytekit-dolt)
GitHub Actions: build-plugins (3.8, flytekit-deck-standard)
GitHub Actions: build-plugins (3.8, flytekit-dbt)
GitHub Actions: build-plugins (3.8, flytekit-data-fsspec)
GitHub Actions: build-plugins (3.8, flytekit-dask)
GitHub Actions: build-plugins (3.8, flytekit-bigquery)
GitHub Actions: build-plugins (3.8, flytekit-aws-sagemaker)
GitHub Actions: build-plugins (3.8, flytekit-aws-batch)
GitHub Actions: build-plugins (3.8, flytekit-aws-athena)
GitHub Actions: build (windows-latest, 3.9)
GitHub Actions: build (windows-latest, 3.8)
GitHub Actions: build (windows-latest, 3.7, -spark2)
GitHub Actions: build (windows-latest, 3.7)
GitHub Actions: build (ubuntu-latest, 3.10)
GitHub Actions: build (ubuntu-latest, 3.9)
GitHub Actions: build (ubuntu-latest, 3.8)
GitHub Actions: build (ubuntu-latest, 3.7, -spark2)
GitHub Actions: build (ubuntu-latest, 3.7)
GitHub Actions: lint
✅ 1 other check has passed
1/30 successful checksGitHub
01/20/2023, 6:33 PMfrom flytekit import ContainerTask, kwtypes, workflow
hello_task = ContainerTask(
name="hello",
image="ubuntu:20.04",
inputs=kwtypes(name=str),
command=["echo", "hello {{.inputs.name}}"]
)
@workflow
def say_hello(name: str):
hello_task(name=name)
Screenshots
No response
Are you sure this issue hasn't been raised already?
☑︎ Yes
Have you read the Code of Conduct?
☑︎ Yes
flyteorg/flyteGitHub
01/20/2023, 7:20 PMflyte
repo.
In order to centralize the community-related information like Governance, communication methods, Code of Conduct, contribution guides, among others, a flyteorg/community repo would be useful. Also it would help with issue tracking, avoiding having community-related issues stored alongside a number of issues related to the codebase.
The overarching goal is to make easier for new and established community members to discover and track any progress on making the Flyte community even more welcoming for users and contributors.
Also, if there would be additional OSS projects in the future hosted under flyteorg (even in a monorepo), /community could still be the place to centralize all the procedures for the flyteorg community
What if we do not do this?
With the use of labels we could filter out the community-related issues, but we would have to create new labels for this to happen.
If someone new to the project explores current issues, it would probably not be evident that community-related discussions/ideas are also located there
Related component(s)
No response
Are you sure this issue hasn't been raised already?
☑︎ Yes
Have you read the Code of Conduct?
☑︎ Yes
flyteorg/flyteGitHub
01/20/2023, 7:48 PM<https://github.com/flyteorg/flytekit/tree/master|master>
by kumare3
<https://github.com/flyteorg/flytekit/commit/310aea3296170126433f3e7b2949c6a6ea1203c0|310aea32>
- Preserving Exception in the LazyEntity fetch (#1412)
flyteorg/flytekit