GitHub
01/13/2023, 12:52 PMpyflyte run --remote core/flyte_basics/hello_world.py my_wf
, it does not work because no configuration is provided. After calling flytectl config init
, the config file ~/.flyte/config.yaml
is created, but the endpoint port there is not 30080 as in the demo cluster, but 30081. I had to manually change (rewrite) the config file to have endpoint: dns:///localhost:30080
. Then, everything works well.
Expected behavior
Following userguide_setup, I'd expect that following all steps from top to bottom would lead to successful --remote
execution, which didn't happen.
Additional context to reproduce
1. create virtual env
2. git clone <https://github.com/flyteorg/flytesnacks>
3. cd flytesnacks/cookbook
4. pip install -r core/requirements.txt
5. install flytectl
6. flytectl demo start
7. pyflyte run --remote core/flyte_basics/hello_world.py my_wf
<-- this fails
8. flytectl config init
9. edit ~/.flyte/config.yaml
and change edpoint port from 30081 to 30080
10. pyflyte run --remote core/flyte_basics/hello_world.py my_wf
<-- now it works
Steps 8 and 9 are missing in the documentation.
Screenshots
Result of step 8 in _Additional context to reproduce_:
pyflyte run --remote cookbook/core/flyte_basics/hello_world.py my_wf
{"asctime": "2023-01-13 13:48:16,864", "name": "flytekit.cli", "levelname": "ERROR", "message": "Non-auth RPC error <_InactiveRpcError of RPC that terminated with:\n\tstatus = StatusCode.UNAVAILABLE\n\tdetails = \"failed to connect to all addresses\"\n\tdebug_error_string = \"{\"created\":\"@1673614096.864140274\",\"description\":\"Failed to pick subchannel\",\"file\":\"src/core/ext/filters/client_channel/client_channel.cc\",\"file_line\":3260,\"referenced_errors\":[{\"created\":\"@1673614096.864139984\",\"description\":\"failed to connect to all addresses\",\"file\":\"src/core/lib/transport/error_utils.cc\",\"file_line\":167,\"grpc_status\":14}]}\"\n>, sleeping 200ms and retrying"}
{"asctime": "2023-01-13 13:48:17,064", "name": "flytekit.cli", "levelname": "ERROR", "message": "Non-auth RPC error <_InactiveRpcError of RPC that terminated with:\n\tstatus = StatusCode.UNAVAILABLE\n\tdetails = \"failed to connect to all addresses\"\n\tdebug_error_string = \"{\"created\":\"@1673614097.064544493\",\"description\":\"Failed to pick subchannel\",\"file\":\"src/core/ext/filters/client_channel/client_channel.cc\",\"file_line\":3260,\"referenced_errors\":[{\"created\":\"@1673614097.064543902\",\"description\":\"failed to connect to all addresses\",\"file\":\"src/core/lib/transport/error_utils.cc\",\"file_line\":167,\"grpc_status\":14}]}\"\n>, sleeping 400ms and retrying"}
Traceback (most recent call last):
File "/home/michal/miniconda3/envs/flyte/bin/pyflyte", line 8, in <module>
sys.exit(main())
File "/home/michal/miniconda3/envs/flyte/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
return self.main(*args, **kwargs)
File "/home/michal/miniconda3/envs/flyte/lib/python3.10/site-packages/click/core.py", line 1055, in main
rv = self.invoke(ctx)
File "/home/michal/miniconda3/envs/flyte/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/michal/miniconda3/envs/flyte/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/michal/miniconda3/envs/flyte/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/michal/miniconda3/envs/flyte/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/michal/miniconda3/envs/flyte/lib/python3.10/site-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "/home/michal/miniconda3/envs/flyte/lib/python3.10/site-packages/flytekit/clis/sdk_in_container/run.py", line 539, in _run
remote_entity = remote.register_script(
File "/home/michal/miniconda3/envs/flyte/lib/python3.10/site-packages/flytekit/remote/remote.py", line 596, in register_script
upload_location, md5_bytes = fast_register_single_script(
File "/home/michal/miniconda3/envs/flyte/lib/python3.10/site-packages/flytekit/tools/script_mode.py", line 113, in fast_register_single_script
upload_location = create_upload_location_fn(content_md5=md5)
File "/home/michal/miniconda3/envs/flyte/lib/python3.10/site-packages/flytekit/clients/friendly.py", line 998, in get_upload_signed_url
return super(SynchronousFlyteClient, self).create_upload_location(
File "/home/michal/miniconda3/envs/flyte/lib/python3.10/site-packages/flytekit/clients/raw.py", line 41, in handler
return fn(*args, **kwargs)
File "/home/michal/miniconda3/envs/flyte/lib/python3.10/site-packages/flytekit/clients/raw.py", line 854, in create_upload_location
return self._dataproxy_stub.CreateUploadLocation(create_upload_location_request, metadata=self._metadata)
File "/home/michal/miniconda3/envs/flyte/lib/python3.10/site-packages/grpc/_channel.py", line 946, in __call__
return _end_unary_response_blocking(state, call, False, None)
File "/home/michal/miniconda3/envs/flyte/lib/python3.10/site-packages/grpc/_channel.py", line 849, in _end_unary_response_blocking
raise _InactiveRpcError(state)
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "failed to connect to all addresses"
debug_error_string = "{"created":"@1673614097.465232045","description":"Failed to pick subchannel","file":"src/core/ext/filters/client_channel/client_channel.cc","file_line":3260,"referenced_errors":[{"created":"@1673614097.465231304","description":"failed to connect to all addresses","file":"src/core/lib/transport/error_utils.cc","file_line":167,"grpc_status":14}]}"
Are you sure this issue hasn't been raised already?
☑︎ Yes
Have you read the Code of Conduct?
☑︎ Yes
flyteorg/flyteGitHub
01/13/2023, 4:24 PMGitHub
01/13/2023, 8:07 PMGitHub
01/13/2023, 8:07 PMGitHub
01/13/2023, 8:07 PMGitHub
01/13/2023, 8:07 PMGitHub
01/13/2023, 8:32 PM<https://github.com/flyteorg/flytekit-feedstock/tree/main|main>
by cosmicBboy
<https://github.com/flyteorg/flytekit-feedstock/commit/b8c3a793dbf231a70b9b3bc7b973ea761749bc00|b8c3a793>
- add maintainers from flyteorg
flyteorg/flytekit-feedstockGitHub
01/13/2023, 8:34 PM<https://github.com/flyteorg/flytekit-feedstock/tree/main|main>
by cosmicBboy
<https://github.com/flyteorg/flytekit-feedstock/commit/03517601593b9541463a9771ee6b343b791ca2a7|03517601>
- bump build
flyteorg/flytekit-feedstockGitHub
01/13/2023, 8:35 PMGitHub
01/13/2023, 8:35 PMGitHub
01/13/2023, 8:36 PMGitHub
01/13/2023, 8:36 PMGitHub
01/13/2023, 8:36 PMGitHub
01/13/2023, 8:36 PMGitHub
01/13/2023, 9:00 PMPodTemplate
and / or PodTemplate
name. The differences are client vs. server side application respectively.
Type
☐ Bug Fix
☑︎ Feature
☐ Plugin
Are all requirements met?
☑︎ Code completed
☑︎ Smoke tested
☑︎ Unit tests added
☑︎ Code documentation added
☐ Any pending items have an associated Issue
Complete description
Currently supported task types:
☐ container
☐ dask
☑︎ mpi
☑︎ pod
☑︎ python
☑︎ pytorch
☐ ray
☑︎ shell
☐ spark
☑︎ tensorflow
Tracking Issue
flyteorg/flyte#3123
Follow-up issue
NA
flyteorg/flyteplugins
Codecov: 62.94% (-0.09%) compared to 471a262
✅ 5 other checks have passed
5/6 successful checksGitHub
01/13/2023, 9:01 PMTaskTemplate
which contains either a PodTemplate
name or a marshaled PodTemplate
struct.
Type
☐ Bug Fix
☑︎ Feature
☐ Plugin
Are all requirements met?
☑︎ Code completed
☑︎ Smoke tested
☐ Unit tests added
☐ Code documentation added
☐ Any pending items have an associated Issue
Complete description
^^^
Tracking Issue
flyteorg/flyte#3123
Follow-up issue
NA
flyteorg/flyteidl
✅ All checks have passed
13/13 successful checksGitHub
01/13/2023, 9:02 PMPodTemplate
configuration at the task granularity.
Type
☐ Bug Fix
☑︎ Feature
☐ Plugin
Are all requirements met?
☑︎ Code completed
☑︎ Smoke tested
☑︎ Unit tests added
☑︎ Code documentation added
☐ Any pending items have an associated Issue
Complete description
^^^
Tracking Issue
flyteorg/flyte#3123
Follow-up issue
NA
flyteorg/flytepropeller
GitHub Actions: Build & Push Flytepropeller Image
GitHub Actions: Goreleaser
GitHub Actions: Bump Version
✅ 11 other checks have passed
11/14 successful checksGitHub
01/13/2023, 10:27 PM<https://github.com/flyteorg/flytekit/tree/master|master>
by wild-endeavor
<https://github.com/flyteorg/flytekit/commit/4b1675ffb85648dc5742e9a6dea98b94714963e1|4b1675ff>
- Add support for overriding task configurations (#1410)
flyteorg/flytekitGitHub
01/13/2023, 10:49 PMretry
is only used in the sagemaker plugin - it should no longer be a flytekit dependence
Type
☑︎ Bug Fix
☐ Feature
☐ Plugin
Are all requirements met?
☐ Code completed
☐ Smoke tested
☐ Unit tests added
☐ Code documentation added
☐ Any pending items have an associated Issue
Complete description
retry
is only used in the sagemaker flytekit plugin, which means that we only need retry
there.
We're going to release a minor version of flytekit and in a follow up PR we can then update the dependence in the sagemaker plugin's requirements.
Tracking Issue
flyteorg/flyte#3052
Follow-up issue
NA
flyteorg/flytekit
GitHub Actions: build-plugins (3.10, flytekit-vaex)
GitHub Actions: build-plugins (3.10, flytekit-sqlalchemy)
GitHub Actions: build-plugins (3.10, flytekit-spark)
GitHub Actions: build-plugins (3.10, flytekit-snowflake)
GitHub Actions: build-plugins (3.10, flytekit-ray)
GitHub Actions: build-plugins (3.10, flytekit-hive)
GitHub Actions: build-plugins (3.9, flytekit-whylogs)
GitHub Actions: build-plugins (3.9, flytekit-vaex)
GitHub Actions: build-plugins (3.9, flytekit-sqlalchemy)
GitHub Actions: build-plugins (3.9, flytekit-spark)
GitHub Actions: build-plugins (3.9, flytekit-snowflake)
GitHub Actions: build-plugins (3.9, flytekit-ray)
GitHub Actions: build-plugins (3.9, flytekit-polars)
GitHub Actions: build-plugins (3.9, flytekit-papermill)
✅ 16 other checks have passed
16/30 successful checksGitHub
01/14/2023, 1:38 AMGitHub
01/14/2023, 6:07 PMGitHub
01/15/2023, 4:27 PMGitHub
01/16/2023, 1:04 PM<https://github.com/flyteorg/flyte/tree/master|master>
by samhita-alla
<https://github.com/flyteorg/flyte/commit/e69f24542871785c34d2a3c07cd682c22441d9fa|e69f2454>
- Clarify CRD and CR (#3237)
flyteorg/flyteGitHub
01/16/2023, 5:00 PMGitHub
01/16/2023, 5:00 PMGitHub
01/16/2023, 10:10 PMGitHub
01/17/2023, 12:00 AMpip
dependencies of this project.
Changes included in this PR
• Changes to the following files to upgrade the vulnerable dependencies to a fixed version:
• doc-requirements.txt
⚠️ Warning
vaex 4.16.0 requires vaex-core, which is not installed.
vaex-viz 0.5.4 requires vaex-core, which is not installed.
vaex-server 0.8.1 requires vaex-core, which is not installed.
vaex-server 0.8.1 requires fastapi, which is not installed.
vaex-server 0.8.1 requires uvicorn, which is not installed.
vaex-ml 0.18.1 requires vaex-core, which is not installed.
vaex-jupyter 0.8.1 requires vaex-core, which is not installed.
vaex-jupyter 0.8.1 requires xarray, which is not installed.
vaex-hdf5 0.14.1 requires vaex-core, which is not installed.
vaex-astro 0.9.3 requires vaex-core, which is not installed.
protoc-gen-swagger 0.1.0 requires protobuf, which is not installed.
notebook 5.7.16 requires terminado, which is not installed.
nbformat 4.4.0 requires jsonschema, which is not installed.
nbconvert 5.6.1 has requirement mistune<2,>=0.8.1, but you have mistune 2.0.4.
jupyter 1.0.0 requires qtconsole, which is not installed.
jupyter 1.0.0 requires jupyter-console, which is not installed.
ipyvolume 0.5.2 requires pythreejs, which is not installed.
ipython 5.10.0 requires simplegeneric, which is not installed.
ipympl 0.9.2 has requirement matplotlib<4,>=3.4.0, but you have matplotlib 2.2.5.
ipyleaflet 0.17.0 requires xyzservices, which is not installed.
ImageHash 4.3.1 requires scipy, which is not installed.
ImageHash 4.3.1 requires PyWavelets, which is not installed.
google-auth 2.15.0 requires rsa, which is not installed.
flyteidl 1.3.2 requires googleapis-common-protos, which is not installed.
flyteidl 1.3.2 requires protobuf, which is not installed.
docker-image-py 0.1.12 requires regex, which is not installed.
databricks-cli 0.17.4 requires pyjwt, which is not installed.
databricks-cli 0.17.4 requires tabulate, which is not installed.
Vulnerabilities that will be fixed
By pinning:
(*) Note that the real score may have changed since the PR was raised.
Some vulnerabilities couldn't be fully fixed and so Snyk will still find them when the project is tested again. This may be because the vulnerability existed within more than one direct dependency, but not all of the affected dependencies could be upgraded.
Check the changes in this PR to ensure they won't cause issues with your project.
* * *
Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.
For more information: https://camo.githubusercontent.com/ac569e0eb063183e468b6931c596a477a38a9dcf1301df2059231a1917fb0a91/68747470733a2f2f6170692e7365676d656e742e696f2f76312f706978656c2f747261636b3f646174613d65794a33636d6c305a55746c65534936496e4a79576d785a634564485932527954485a7362306c596430645563566734576b4652546e4e434f5545774969776959573576626e6c746233567a535751694f6949344d32466a4e6a426c5a69307a4d6a457a4c5451334e6a637459545977595331685a444e694d32466b4f544535596a55694c434a6c646d567564434936496c425349485a705a58646c5a434973496e42796233426c636e52705a584d694f6e736963484a4a5a434936496a677a59574d324d47566d4c544d794d544d744e4463324e7931684e6a42684c57466b4d32497a595751354d546c694e534a3966513d3d
🧐 View latest project report
🛠️ Adjust project settings
📚 Read more about Snyk's upgrade and patch logic
* * *
Learn how to fix vulnerabilities with free interactive lessons:
🦉 Regular Expression Denial of Service (ReDoS)
flyteorg/flytekit
✅ All checks have passed
30/30 successful checksGitHub
01/17/2023, 6:16 AM<https://github.com/flyteorg/flytesnacks/tree/master|master>
by samhita-alla
<https://github.com/flyteorg/flytesnacks/commit/9a1b05ab7f64cfa7121d690119d04155c7642b22|9a1b05ab>
- Minor typo in Golang (#941)
flyteorg/flytesnacksGitHub
01/17/2023, 9:23 AMGitHub
01/17/2023, 11:53 AM