GitHub
05/31/2023, 4:44 PMGitHub
05/31/2023, 4:45 PMGitHub
05/31/2023, 5:19 PMalpine:3
, as this image has only 0 known vulnerabilities. To do this, merge this pull request, then verify your application still works as expected.
* * *
Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.
For more information: https://camo.githubusercontent.com/e4510f38536c4f9c5ddde6d596c5e2b2515376cef2cfa87b17dec96c17bdf34e/68747470733a2f2f6170692e7365676d656e742e696f2f76312f706978656c2f747261636b3f646174613d65794a33636d6c305a55746c65534936496e4a79576d785a634564485932527954485a7362306c596430645563566734576b4652546e4e434f5545774969776959573576626e6c746233567a535751694f694a6d5a5441354f4463335a4330354d7a52684c54526959574974596a426a4e69316a597a51344e3249794e544d334d5451694c434a6c646d567564434936496c425349485a705a58646c5a434973496e42796233426c636e52705a584d694f6e736963484a4a5a434936496d5a6c4d446b344e7a646b4c546b7a4e4745744e474a68596931694d474d324c574e6a4e446733596a49314d7a63784e434a3966513d3d
🧐 View latest project report
🛠️ Adjust project settings
* * *
Learn how to fix vulnerabilities with free interactive lessons:
🦉 Learn about vulnerability in an interactive lesson of Snyk Learn.
flyteorg/flyteadmin
GitHub Actions: End2End Test / End to End tests
GitHub Actions: Integration Test / Integration tests
DCO: DCO
✅ 7 other checks have passed
7/10 successful checksGitHub
05/31/2023, 5:48 PM<https://github.com/flyteorg/flytekit/tree/master|master>
by wild-endeavor
<https://github.com/flyteorg/flytekit/commit/484a221024dd894673a5bfd3d913c37893a77b82|484a2210>
- FLYTECTL_CONFIG env var higher precedence, config flag respected in pyflyte package (#1662)
flyteorg/flytekitGitHub
05/31/2023, 6:33 PMflyteidl/admin/agent
and rename it (TaskCreateRequest -> CreateTaskRequest), because TaskCreateRequest already exists in the task.proto
Type
☐ Bug Fix
☐ Feature
☐ Plugin
Are all requirements met?
☐ Code completed
☐ Smoke tested
☐ Unit tests added
☐ Code documentation added
☐ Any pending items have an associated Issue
Complete description
^^^
Tracking Issue
NA
Follow-up issue
NA
flyteorg/flyteidl
✅ All checks have passed
13/13 successful checksGitHub
05/31/2023, 8:36 PMGitHub
05/31/2023, 9:17 PMadminflyterole
Google Service Account (GSA) bound to Kubernetes Service Account (KSA) for Flyte control plane, including FlytePropeller.
2. userflyterole
GSA bound to KSA for task pods.
Running this workflow results in the following error:
[3/3] currentAttempt done. Last Error: SYSTEM::Traceback (most recent call last):
File "/usr/local/lib/python3.10/site-packages/flytekit/exceptions/scopes.py", line 165, in system_entry_point
return wrapped(*args, **kwargs)
File "/usr/local/lib/python3.10/site-packages/flytekit/core/base_task.py", line 530, in dispatch_execute
raise type(exc)(msg) from exc
Message:
403 Failed to convert inputs of task 'flytesnacks.convert_bq_table_to_pandas_dataframe':
403 Access Denied: Dataset mygcpproject:_042b1dd57b11c054c884727b94f55d11775727e6: User does not have permission to access results of another user's job.
SYSTEM ERROR! Contact platform administrators
Expected behavior
Output from BigQueryTask
can be successfully passed as an input into the subsequent python task, and workflow completes successfully
Additional context to reproduce
No response
Screenshots
No response
Are you sure this issue hasn't been raised already?
☑︎ Yes
Have you read the Code of Conduct?
☑︎ Yes
flyteorg/flyteGitHub
05/31/2023, 10:40 PM<https://github.com/flyteorg/flytekit/tree/master|master>
by eapolinario
<https://github.com/flyteorg/flytekit/commit/894c260e84a83add793408e82ea68f3c8e60b404|894c260e>
- Change flytekit Pytorch, TFJob and MPI plugins to use new kubeflow config (#1627)
flyteorg/flytekitGitHub
05/31/2023, 10:47 PMWithDeadline
with an optionally configurable deadline set in config
What if we do not do this?
See https://grpc.io/blog/deadlines/
In general, when you don’t set a deadline, resources will be held for all in-flight requests, and all requests can potentially reach the maximum timeout. This puts the service at risk of running out of resources, like memory, which would increase the latency of the service, or could crash the entire process in the worst case.Related component(s) flyteidl Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte
GitHub
06/01/2023, 12:00 AMflake.nix
{
inputs = {
nixpkgs.url = "github:nixos/nixpkgs/nixpkgs-unstable";
};
outputs = {
self,
nixpkgs,
flake-utils
}:
flake-utils.lib.eachDefaultSystem
(system:
let
pkgs = import nixpkgs { inherit system; };
in
with pkgs;
{
devShells.default = mkShell {
buildInputs = [ pkgs.flytectl ];
};
}
);
}
Are you sure this issue hasn't been raised already?
☑︎ Yes
Have you read the Code of Conduct?
☑︎ Yes
flyteorg/flyteGitHub
06/01/2023, 4:33 PMGitHub
06/01/2023, 4:37 PMGitHub
06/01/2023, 5:15 PM<https://github.com/flyteorg/flytekit/tree/master|master>
by wild-endeavor
<https://github.com/flyteorg/flytekit/commit/d76dceb8ffe835cc591b6456255a8e63cb50e325|d76dceb8>
- Root cert should be byte string when loading from caCertFilePath (#1669)
flyteorg/flytekitGitHub
06/01/2023, 5:39 PMalpine:3
, as this image has only 0 known vulnerabilities. To do this, merge this pull request, then verify your application still works as expected.
* * *
Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.
For more information: https://camo.githubusercontent.com/e13f3c226645686f5ad3eedfe86e3a20c85bb45cc288187e7d41af33a5523297/68747470733a2f2f6170692e7365676d656e742e696f2f76312f706978656c2f747261636b3f646174613d65794a33636d6c305a55746c65534936496e4a79576d785a634564485932527954485a7362306c596430645563566734576b4652546e4e434f5545774969776959573576626e6c746233567a535751694f694933597a68685a574e6a4d6930304e7a417a4c54526a4d324974595745324f5330794e5459784e6a417a4e4441314e5745694c434a6c646d567564434936496c425349485a705a58646c5a434973496e42796233426c636e52705a584d694f6e736963484a4a5a434936496a646a4f47466c59324d794c5451334d444d744e474d7a59693168595459354c5449314e6a45324d444d304d44553159534a3966513d3d
🧐 View latest project report
🛠️ Adjust project settings
* * *
Learn how to fix vulnerabilities with free interactive lessons:
🦉 Learn about vulnerability in an interactive lesson of Snyk Learn.
flyteorg/flytecopilot
GitHub Actions: Goreleaser
GitHub Actions: Build & Push Flyte copilot Image
GitHub Actions: Bump Version
DCO: DCO
✅ 6 other checks have passed
6/10 successful checksGitHub
06/01/2023, 8:15 PM<https://github.com/flyteorg/flytekit/tree/master|master>
by pingsutw
<https://github.com/flyteorg/flytekit/commit/c8433ea2d2ecf362815b1ee63554549ced9acd75|c8433ea2>
- Explicitly set the content type for flyte deck (#1658)
flyteorg/flytekitGitHub
06/01/2023, 8:18 PMGitHub
06/01/2023, 8:35 PMdatasource_raw_data:
execution_engine:
module_name: great_expectations.execution_engine
class_name: SparkDFExecutionEngine
force_reuse_spark_context: true
module_name: great_expectations.datasource
class_name: Datasource
data_connectors:
runtime_data_connector:
class_name: RuntimeDataConnector
module_name: great_expectations.datasource.data_connector
batch_identifiers:
- pipeline_stage
to validate a Spark dataframe, codes returned the following error
│
'Traceback (most recent call last):\n File "/Users/.pyenv/versions/3.9.8/envs/flyte/lib/python3.9/site-packages/great_expectations/execution_engine/execution_engine.py", line 555, in _process_direct_and_bundled_metric_computation_configurations\n ] = metric_computation_configuration.metric_fn( # type: ignore[misc] # F not callable\n File "/Users/.pyenv/versions/3.9.8/envs/flyte/lib/python3.9/site-packages/great_expectations/expectations/metrics/metric_provider.py", line 50, in inner_func\n return metric_fn(*args, **kwargs)\n File "/Users/.pyenv/versions/3.9.8/envs/flyte/lib/python3.9/site-packages/great_expectations/expectations/metrics/table_metrics/table_column_types.py", line 95, in _spark\n df.schema, include_nested=metric_value_kwargs["include_nested"]\n File "/Users/.pyenv/versions/3.9.8/envs/flyte/lib/python3.9/site-packages/pandas/core/generic.py", line 5902, in *getattr*\n return object.*getattribute*(self, name)\nAttributeError: 'DataFrame' object has no attribute 'schema'\n\nThe above exception was the direct cause of the following exception:\n\nTraceback (most recent call last):\n File "/Users/.pyenv/versions/3.9.8/envs/flyte/lib/python3.9/site-packages/great_expectations/validator/validation_graph.py", line 276, in _resolve\n self._execution_engine.resolve_metrics(\n File "/Users/.pyenv/versions/3.9.8/envs/flyte/lib/python3.9/site-packages/great_expectations/execution_engine/execution_engine.py", line 290, in resolve_metrics\n return self._process_direct_and_bundled_metric_computation_configurations(\n File "/Users/.pyenv/versions/3.9.8/envs/flyte/lib/python3.9/site-packages/great_expectations/execution_engine/execution_engine.py", line 559, in _process_direct_and_bundled_metric_computation_configurations\n raise gx_exceptions.MetricResolutionError(\ngreat_expectations.exceptions.exceptions.MetricResolutionError: 'DataFrame' object has no attribute 'schema'\n',
Expected behavior
There should have no error like the one in the description.
Additional context to reproduce
No response
Screenshots
No response
Are you sure this issue hasn't been raised already?
☑︎ Yes
Have you read the Code of Conduct?
☑︎ Yes
flyteorg/flyteGitHub
06/01/2023, 9:17 PMexecution_admin_tags
is a join table, and it contains admin_tags's ID and execution's ID.
admin_tags
saves all the tag names.
To list executions that match specific tags, we will join execution
, execution_admin_tags
, and admin_tags
table.
Type
☐ Bug Fix
☑︎ Feature
☐ Plugin
Are all requirements met?
☑︎ Code completed
☑︎ Smoke tested
☑︎ Unit tests added
☐ Code documentation added
☐ Any pending items have an associated Issue
Complete description
image▾
GitHub
06/02/2023, 1:28 AMalpine:3
, as this image has only 0 known vulnerabilities. To do this, merge this pull request, then verify your application still works as expected.
* * *
Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.
For more information: https://camo.githubusercontent.com/88d04efb7e0d780005f6c0a34b0e3932e3343af6838879dcac9a607960931c54/68747470733a2f2f6170692e7365676d656e742e696f2f76312f706978656c2f747261636b3f646174613d65794a33636d6c305a55746c65534936496e4a79576d785a634564485932527954485a7362306c596430645563566734576b4652546e4e434f5545774969776959573576626e6c746233567a535751694f6949774d6a55335a44526a4f53307a4e5751324c5451344e7a4d745954497a4f5330774e7a526a4d474a6c4e7a67354e5445694c434a6c646d567564434936496c425349485a705a58646c5a434973496e42796233426c636e52705a584d694f6e736963484a4a5a434936496a41794e54646b4e474d354c544d315a4459744e4467334d7931684d6a4d354c5441334e474d77596d55334f446b314d534a3966513d3d
🧐 View latest project report
🛠️ Adjust project settings
* * *
Learn how to fix vulnerabilities with free interactive lessons:
🦉 Learn about vulnerability in an interactive lesson of Snyk Learn.
flyteorg/flytepropeller
GitHub Actions: Goreleaser
GitHub Actions: Build & Push Flytepropeller Image
GitHub Actions: Bump Version
GitHub Actions: Lint / Run Lint
DCO: DCO
✅ 9 other checks have passed
9/14 successful checksGitHub
06/02/2023, 1:42 AMalpine:3
, as this image has only 0 known vulnerabilities. To do this, merge this pull request, then verify your application still works as expected.
* * *
Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.
For more information: https://camo.githubusercontent.com/dbd075cbff7aa8d8763cba68b5e90076cf3227ad8e2b6670eb8dc4a81ca9a1ea/68747470733a2f2f6170692e7365676d656e742e696f2f76312f706978656c2f747261636b3f646174613d65794a33636d6c305a55746c65534936496e4a79576d785a634564485932527954485a7362306c596430645563566734576b4652546e4e434f5545774969776959573576626e6c746233567a535751694f694a684e5451774e544178595330775a5468684c5451334d544174595755785a433035596a566d4d5455334d4467345a5445694c434a6c646d567564434936496c425349485a705a58646c5a434973496e42796233426c636e52705a584d694f6e736963484a4a5a434936496d45314e4441314d4446684c54426c4f4745744e4463784d4331685a54466b4c546c694e5759784e5463774f44686c4d534a3966513d3d
🧐 View latest project report
🛠️ Adjust project settings
* * *
Learn how to fix vulnerabilities with free interactive lessons:
🦉 Learn about vulnerability in an interactive lesson of Snyk Learn.
flyteorg/datacatalog
GitHub Actions: Goreleaser
GitHub Actions: Build & Push Datacatalog Image
GitHub Actions: Bump Version
DCO: DCO
✅ 7 other checks have passed
7/11 successful checksGitHub
06/02/2023, 3:06 AMGitHub
06/02/2023, 4:25 AMGitHub
06/02/2023, 6:34 AMenv:
- name: FLYTE_INTERNAL_EXECUTION_WORKFLOW
value: ...
- name: FLYTE_INTERNAL_EXECUTION_ID
value: ...
- name: FLYTE_INTERNAL_EXECUTION_PROJECT
value: ...
- name: FLYTE_INTERNAL_EXECUTION_DOMAIN
value: ...
- name: FLYTE_ATTEMPT_NUMBER
value: ...
- name: FLYTE_INTERNAL_TASK_PROJECT
value: ...
- name: FLYTE_INTERNAL_TASK_DOMAIN
value: ...
- name: FLYTE_INTERNAL_TASK_NAME
value: ...
- name: FLYTE_INTERNAL_TASK_VERSION
value: ...
- name: FLYTE_INTERNAL_PROJECT
value: ...
- name: FLYTE_INTERNAL_DOMAIN
value: ...
- name: FLYTE_INTERNAL_NAME
value: ...
- name: FLYTE_INTERNAL_VERSION
value: ...
- name: FLYTE_PLATFORM_CLOUD_PROVIDER
value: gcp
# duplicates start from here
- name: FLYTE_INTERNAL_EXECUTION_WORKFLOW
value: ...
- name: FLYTE_INTERNAL_EXECUTION_ID
value: ...
- name: FLYTE_INTERNAL_EXECUTION_PROJECT
value: ...
- name: FLYTE_INTERNAL_EXECUTION_DOMAIN
value: ...
- name: FLYTE_ATTEMPT_NUMBER
value: ...
- name: FLYTE_INTERNAL_TASK_PROJECT
value: ...
- name: FLYTE_INTERNAL_TASK_DOMAIN
value: ...
- name: FLYTE_INTERNAL_TASK_NAME
value: ...
- name: FLYTE_INTERNAL_TASK_VERSION
value: ...
- name: FLYTE_INTERNAL_PROJECT
value: ...
- name: FLYTE_INTERNAL_DOMAIN
value: ...
- name: FLYTE_INTERNAL_NAME
value: ...
- name: FLYTE_INTERNAL_VERSION
value: ...
- name: FLYTE_PLATFORM_CLOUD_PROVIDER
value: gcp
It seems the same set of env vars are attached twice.
Expected behavior
No duplicates of env vars.
Additional context to reproduce
Run any container based task and example the pod.
Screenshots
No response
Are you sure this issue hasn't been raised already?
☑︎ Yes
Have you read the Code of Conduct?
☑︎ Yes
flyteorg/flyteGitHub
06/02/2023, 11:45 AMGitHub
06/02/2023, 3:28 PMGitHub
06/02/2023, 5:04 PM<https://github.com/flyteorg/flyte/tree/master|master>
by eapolinario
<https://github.com/flyteorg/flyte/commit/14171fd84d171a609e619a214fa36695fc79990b|14171fd8>
- Proto-docs: Compile-time and Runtime PodTemplates (#3391)
flyteorg/flyteGitHub
06/02/2023, 5:04 PMPodTemplate
configuration: #3123
Are you sure this issue hasn't been raised already?
☑︎ Yes
Have you read the Code of Conduct?
☑︎ Yes
flyteorg/flyteGitHub
06/02/2023, 5:05 PMGitHub
06/02/2023, 6:52 PM<https://github.com/flyteorg/flyteidl/tree/master|master>
by pingsutw
<https://github.com/flyteorg/flyteidl/commit/e5062062be56dcd12bed8e19c0bbf7bb1ab2bee9|e5062062>
- Add external_plugin_service proto back to the idl (#413)
flyteorg/flyteidlGitHub
06/02/2023, 7:10 PM