GitHub
03/10/2023, 8:06 PM<https://github.com/flyteorg/flyteadmin/tree/master|master>
by hamersaw
<https://github.com/flyteorg/flyteadmin/commit/f6d949aa2efb8ebd8324f5a595118c0d7eb69984|f6d949aa>
- Tracking DynamicJobSpecUri for DynamicWorkflows (#513)
flyteorg/flyteadminGitHub
03/10/2023, 8:06 PMDynamicJobSpecUri
in the NodeExecutionClosure
so that we can correctly recover partially completed DynamicWorkflows
.
Type
☑︎ Bug Fix
☐ Feature
☐ Plugin
Are all requirements met?
☑︎ Code completed
☑︎ Smoke tested
☑︎ Unit tests added
☑︎ Code documentation added
☐ Any pending items have an associated Issue
Complete description
^^^
Tracking Issue
flyteorg/flyte#3121
Follow-up issue
NA
flyteorg/flyteadmin
GitHub Actions: Build & Push Flytescheduler Image
GitHub Actions: Build & Push Flyteadmin Image
GitHub Actions: Goreleaser
GitHub Actions: Bump Version
✅ 10 other checks have passed
10/14 successful checksGitHub
03/10/2023, 8:08 PMGitHub
03/10/2023, 8:47 PMGitHub
03/10/2023, 8:48 PMGitHub
03/10/2023, 10:18 PMGitHub
03/10/2023, 10:20 PMgate_nodes.py
example (see flyteorg/flytesnacks#968) for using gate nodes in a dynamic workflow.
Are you sure this issue hasn't been raised already?
☑︎ Yes
Have you read the Code of Conduct?
☑︎ Yes
flyteorg/flyteGitHub
03/10/2023, 10:24 PM<https://github.com/flyteorg/flyteconsole/tree/master|master>
by jsonporter
<https://github.com/flyteorg/flyteconsole/commit/be0e533280927b3aec4ebff6203cb3a8946e991b|be0e5332>
- feat: differentiate between cache disabled and cache put failure (#719)
flyteorg/flyteconsoleGitHub
03/10/2023, 10:44 PMGitHub
03/10/2023, 11:52 PM$ make build
=> => exporting manifest sha256:fd5ffa0a5cbfaf72dd4bc89f70be185c856e26656bc6b7456a87600636c0fdac 0.0s
=> => exporting config sha256:83112bc87339d549419c587008ed89b886ce3b34432ccb8a371bf58744eed15b 0.0s
=> => sending tarball 1.9s
mkdir -p manifests
kustomize build \
--enable-helm \
--load-restrictor=LoadRestrictionsNone \
kustomize/complete > manifests/complete.yaml
Error: Error: An error occurred while checking for chart dependencies. You may need to run `helm dependency build` to fetch missing dependencies: found in Chart.yaml, but missing in charts/ directory: docker-registry, flyte-binary, kubernetes-dashboard, minio, postgresql
: unable to run: 'helm template flyte-sandbox --namespace flyte /Users/kevin/git/flyte/charts/flyte-sandbox --values /var/folders/by/42kc70lj1bb77klbhf2fmxx00000gn/T/kustomize-helm-4205539172/flyte-sandbox-kustomize-values.yaml' with env=[HELM_CONFIG_HOME=/var/folders/by/42kc70lj1bb77klbhf2fmxx00000gn/T/kustomize-helm-4205539172/helm HELM_CACHE_HOME=/var/folders/by/42kc70lj1bb77klbhf2fmxx00000gn/T/kustomize-helm-4205539172/helm/.cache HELM_DATA_HOME=/var/folders/by/42kc70lj1bb77klbhf2fmxx00000gn/T/kustomize-helm-4205539172/helm/.data] (is 'helm' installed?)
make: *** [manifests] Error 1
should run helm dependency update
first.
btw, we should expose port 6443 and kubeconfig, so we will be able to connect to the k8s cluster.
Check all the applicable boxes
☐ I updated the documentation accordingly.
☐ All new and existing tests passed.
☑︎ All commits are signed-off.
Screenshots
(base) ➜ sandbox-bundled git:(backend-plugin-system-grpc) ✗ make start
[ -n "flyte-sandbox" ] || \
docker volume create flyte-sandbox
[ -n "" ] || \
docker run --detach --rm --privileged --name flyte-sandbox \
--add-host "host.docker.internal:host-gateway" \
--env FLYTE_DEV=False \
--env K3S_KUBECONFIG_OUTPUT=/.kube/kubeconfig \
--volume /Users/kevin/git/flyte/docker/sandbox-bundled/.kube:/.kube \
--volume /Users/kevin/.flyte/sandbox:/var/lib/flyte/config \
--volume flyte-sandbox:/var/lib/flyte/storage \
--publish "6443":6443 \
--publish "30000:30000" \
--publish "30001:30001" \
--publish "30002:30002" \
--publish "30080:30080" \
flyte-sandbox:latest
2cb1c659bbaad0cb1f8bbeb5a6d1e53b569f3dabb0fab8c3170abcc77d89dcdb
export KUBECONFIG=.kube/kubeconfig
(base) ➜ sandbox-bundled git:(backend-plugin-system-grpc) ✗ k get pods
No resources found in default namespace.
Note to reviewers
flyteorg/flyte
✅ All checks have passed
12/12 successful checksGitHub
03/11/2023, 12:14 AMGitHub
03/11/2023, 1:56 AMtelemetryutils
package that adapts the opentelemetry functionality for use in Flyte repos. This includes support for multiple TracerProviders
allowing logical partitioning of tracers within single binary. Also instruments the blobstore and provides a wrapper for instrumenting k8s client.
Type
☐ Bug Fix
☑︎ Feature
☐ Plugin
Are all requirements met?
☑︎ Code completed
☑︎ Smoke tested
☐ Unit tests added
☐ Code documentation added
☐ Any pending items have an associated Issue
Complete description
^^^
Tracking Issue
flyteorg/flyte#3304
Follow-up issue
NA
flyteorg/flytestdlib
✅ All checks have passed
5/5 successful checksGitHub
03/11/2023, 2:39 AMScreen Shot 2023-03-10 at 12 18 18 PM▾
GitHub
03/11/2023, 4:37 PMpyflyte run --remote example.py training_workflow --hyperparameters "{\"C\": 0.1}"
Error:
{"asctime": "2023-03-11 11:32:20,895", "name": "flytekit", "levelname": "WARNING", "message": "FlyteSchema is deprecated, use Structured Dataset instead."} Failed with Exception: Reason: SYSTEM:Unknown Underlying Exception: cannot find context for 'fork'
The example is taken from https://docs.flyte.org/projects/cookbook/en/latest/index.html.
The error appears to be caused by this line:
https://github.com/flyteorg/flytekit/blob/master/flytekit/clients/auth/auth_client.py#L308
The python multiprocessing docs indicate only "spawn" is supported on Windows, and as of 3.8 is also recommended for macOS.
Version Info
Windows 10
Python 3.10.10
flytekit-1.4.1
The same example works for me on macOS.
flyteorg/flyteGitHub
03/12/2023, 10:24 PM<https://github.com/flyteorg/flyte/tree/master|master>
by pingsutw
<https://github.com/flyteorg/flyte/commit/b82eaa5c507640a551f942de5c789198d076a491|b82eaa5c>
- Update helm dependency before kustomize build (#3449)
flyteorg/flyteGitHub
03/12/2023, 11:23 PM<https://github.com/flyteorg/flytekit/tree/master|master>
by pingsutw
<https://github.com/flyteorg/flytekit/commit/152080fa27c6d07edda9fcc2a602ca3ca9f4739a|152080fa>
- Less strict docker dependency versions (#1536)
flyteorg/flytekitGitHub
03/13/2023, 8:50 AMtype Input struct {
HostName string `json:"hostname"`
PodName string `json:"podName"`
Namespace string `json:"namespace"`
ContainerName string `json:"containerName"`
ContainerID string `json:"containerId"`
LogName string `json:"logName"`
PodUnixStartTime int64 `json:"podUnixStartTime"`
PodUnixFinishTime int64 `json:"podUnixFinishTime"`
PodUID string `json:"podUID"`
}
expose execution id, project, domain, task name, workflow name, version and anything that may be relevant.
Describe alternatives you've considered
/
Propose: Link/Inline OR Additional context
Slack discussion: https://flyte-org.slack.com/archives/CP2HDHKE1/p1676538205902059
Are you sure this issue hasn't been raised already?
☑︎ Yes
Have you read the Code of Conduct?
☑︎ Yes
flyteorg/flyteGitHub
03/13/2023, 10:25 AMgroup
field to allow using both (and other Vault secret backends) dynamically.
This also allows using the Database Secrets Engine, which effectively means noop since we want to read the whole credential and not just a specific key.
Type
☐ Bug Fix
☑︎ Feature
☐ Plugin
Are all requirements met?
☑︎ Code completed
☑︎ Smoke tested
☑︎ Unit tests added
☐ Code documentation added
☐ Any pending items have an associated Issue
Complete description
The Secret protobuf defines a group field which we have previously ignored. Leveraging that field allows to simplify the secret manager config and allows supporting multiple Vault secret backends dynamically by deciding what secret template to inject based on the optional group
parameter.
This is a breaking change for people using the previous version of the Vault Secret Manager. To migrate they will need to remove the kvVersion
key from their propeller configs and specify the appropriate group version in their secret requests, like for example so:
@task(
secret_requests=[
Secret(group="foo", key="bar", group_version="kv2"),
]
)
Follow-up issue
NA
flyteorg/flytepropeller
GitHub Actions: Goreleaser
GitHub Actions: Build & Push Flytepropeller Image
GitHub Actions: Bump Version
GitHub Actions: End2End Test / End to End tests
✅ 10 other checks have passed
10/14 successful checksGitHub
03/13/2023, 1:51 PMJacksonSdkType.of(Input.class)
doesn't get the fields from the parent class properly.that
Expected behavior
@AutoValue
public abstract static class Input extends ParentInput {
abstract SdkBindingData<String> c();
public static Input create(
SdkBindingData<String> a,
SdkBindingData<String> b,
SdkBindingData<String> c) {
return new AutoValue_Input(a, b, c);
}
}
public abstract static class Input extends ParentInput {
abstract SdkBindingData<String> a();
abstract SdkBindingData<String> b();
}
This autovalue class is generated well but the JacksonSdkType.of(Input.class)
only identifies the c
as a field. It must identify a
, b
& c
.
Additional context to reproduce
No response
Screenshots
No response
Are you sure this issue hasn't been raised already?
☑︎ Yes
Have you read the Code of Conduct?
☑︎ Yes
flyteorg/flyteGitHub
03/13/2023, 3:03 PMGitHub
03/13/2023, 3:29 PM<https://github.com/flyteorg/flyteadmin/tree/master|master>
by hamersaw
<https://github.com/flyteorg/flyteadmin/commit/22189711d9524e03dff47121b5da92c416ffa6ed|22189711>
- Bump github.com/emicklei/go-restful (#520)
flyteorg/flyteadminGitHub
03/13/2023, 3:29 PMChange history of go-restful
[v3.10.1] - 2022-11-19
• fix broken 3.10.0 by using path package for joining paths
[v3.10.0] - 2022-10-11 - BROKEN
• changed tokenizer to match std route match behavior; do not trimright the path (#511)
• Add MIME_ZIP (#512)
• Add MIME_ZIP and HEADER_ContentDisposition (#513)
• Changed how to get query parameter issue #510
[v3.9.0] - 2022-07-21
• add support for http.Handler implementations to work as FilterFunction, issue #504 (thanks to https://github.com/ggicci)
[v3.8.0] - 2022-06-06
• use exact matching of allowed domain entries, issue #489 (#493)
• this changes fixes [security] Authorization Bypass Through User-Controlled Key by changing the behaviour of the AllowedDomains setting in the CORS filter. To support the previous behaviour, the CORS filter type now has a AllowedDomainFunc callback mechanism which is called when a simple domain match fails.
• add test and fix for POST without body and Content-type, issue #492 (#496)
• [Minor] Bad practice to have a mix of Receiver types. (#491)
[v3.7.2] - 2021-11-24
• restored FilterChain (#482 by SVilgelm)
[v3.7.1] - 2021-10-04
• fix problem with contentEncodingEnabled setting (#479)
[v3.7.0] - 2021-09-24
• feat(parameter): adds additional openapi mappings (#478)
[v3.6.0] - 2021-09-18
• add support for vendor extensions (#477 thx erraggy)
[v3.5.2] - 2021-07-14
• fix removing absent route from webservice (#472)
[v3.5.1] - 2021-04-12... (truncated) Commits • `ac666c0` update changes • `9266625` use exact matching of allowed domain entries, issue #489 (#493) (#503) • `d9c71e1` support multipart/form-data (#502) • `1e8c63c` add access to Route from Request, issue #459 (#462) • `0d68a53` fix typo (#465) • `a22b51d` add check for wildcard (#463) • `8dd9eb8` update chg • `e5d3175` add options shortcut (#455) • `1f7f1de` fix link to example • `601692b` examples use v3 • Additional commits viewable in compare view Dependabot compatibility score Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting
@dependabot rebase
.
* * *
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
• @dependabot rebase
will rebase this PR
• @dependabot recreate
will recreate this PR, overwriting any edits that have been made to it
• @dependabot merge
will merge this PR after your CI passes on it
• @dependabot squash and merge
will squash and merge this PR after your CI passes on it
• @dependabot cancel merge
will cancel a previously requested merge and block automerging
• @dependabot reopen
will reopen this PR if it is closed
• @dependabot close
will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
• @dependabot ignore this major version
will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this minor version
will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this dependency
will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
• @dependabot use these labels
will set the current labels as the default for future PRs for this repo and language
• @dependabot use these reviewers
will set the current reviewers as the default for future PRs for this repo and language
• @dependabot use these assignees
will set the current assignees as the default for future PRs for this repo and language
• @dependabot use this milestone
will set the current milestone as the default for future PRs for this repo and language
You can disable automated security fix PRs for this repo from the Security Alerts page.
flyteorg/flyteadmin
GitHub Actions: Build & Push Flytescheduler Image
GitHub Actions: Goreleaser
GitHub Actions: Build & Push Flyteadmin Image
GitHub Actions: Bump Version
✅ 10 other checks have passed
10/14 successful checksGitHub
03/13/2023, 3:32 PM<https://github.com/flyteorg/flyteidl/tree/master|master>
by hamersaw
<https://github.com/flyteorg/flyteidl/commit/e1a667ef2536bbfc61331490b5417404f45ddc0b|e1a667ef>
- added DataLoadingConfig to K8sPod message (#368)
flyteorg/flyteidlGitHub
03/13/2023, 3:33 PMGitHub
03/13/2023, 3:37 PMpod_template
field in the ContainerTask
.
Type
☐ Bug Fix
☑︎ Feature
☐ Plugin
Are all requirements met?
☑︎ Code completed
☑︎ Smoke tested
☐ Unit tests added
☐ Code documentation added
☐ Any pending items have an associated Issue
Complete description
^^^
Tracking Issue
flyteorg/flyte#3123
Follow-up issue
NA
flyteorg/flyteplugins
Codecov: 62.83% (-0.08%) compared to 5981c35
Codecov: 30.00% of diff hit (target 62.90%)
✅ 4 other checks have passed
4/6 successful checksGitHub
03/13/2023, 3:50 PMGitHub
03/13/2023, 3:57 PM<https://github.com/flyteorg/flytekit-java/tree/master|master>
by sonjaer
<https://github.com/flyteorg/flytekit-java/commit/63dcc29f0f616b81a9a9959092fe644b4d8051c1|63dcc29f>
- Add factory method for remote task with fixed version (#210)
flyteorg/flytekit-javaGitHub
03/13/2023, 4:00 PMGitHub
03/13/2023, 4:00 PMGitHub
03/13/2023, 4:00 PM