I have deployed the Flyte Sandbox to a remote machine UI is Flyte #ask-the-community

I have deployed the Flyte Sandbox to a remote mach...

Marcus Isnard

07/15/2022, 6:17 PM

I have deployed the Flyte Sandbox to a remote machine -- UI is accessible and everything is up. I have set the endpoint in ~/.flyte/config.yaml. When I run a workflow using PyFlyte and a custom container in a registry local to my machine, I get the following error:

Copy code

flytekit.exceptions.user.FlyteAssertion: Failed to put data from /tmp/tmpby0anszu/script_mode.tar.gz to <http://localhost:30084/my-s3-bucket/ff/flytesnacks/development/MOAHDD5B6MWAWQCHZQVZWQX6UU%3D%3D%3D%3D%3D%3D/scriptmode.tar.gz?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=minio%2F20220715%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20220715T180620Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=content-md5%3Bhost&X-Amz-Signature=d807a4f73adf126a579aa7dcbe1800e8654559f37c8728500b61d20b067d804e> (recursive=False).

Original exception: HTTPConnectionPool(host='localhost', port=30084): Max retries exceeded with url: /my-s3-bucket/ff/flytesnacks/development/MOAHDD5B6MWAWQCHZQVZWQX6UU%3D%3D%3D%3D%3D%3D/scriptmode.tar.gz?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=minio%2F20220715%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20220715T180620Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=content-md5%3Bhost&X-Amz-Signature=d807a4f73adf126a579aa7dcbe1800e8654559f37c8728500b61d20b067d804e (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f440c5cd910>: Failed to establish a new connection: [Errno 111] Connection refused'))

Niels Bantilan

07/15/2022, 8:10 PM

looks like the

script_mode.tar.gz

is trying to be uploaded to localhost… what does

cat ~/.flyte/config.yaml

look like?

👍 1

Marcus Isnard

07/15/2022, 8:28 PM

Copy code

admin:
  # For GRPC endpoints you might want to use dns:///flyte.myexample.com
  endpoint: dns:///<domainName>:30081
  authType: Pkce
  insecure: true
logger:
  show-source: true
  level: 0
storage:
  connection:
    access-key: minio
    auth-type: accesskey
    disable-ssl: true
    endpoint: http://<domainName>:30084
    region: us-east-1
    secret-key: miniostorage
  type: minio
  container: "my-s3-bucket"
  enable-multicontainer: false

Marcus Isnard

07/15/2022, 8:34 PM

I am also not running the sandbox on localhost it is deployed so I am confused why it references localhost

Ketan (kumare3)

07/15/2022, 9:26 PM

Can you reply sandbox, you also get a s3 like Blob store bundled

Ketan (kumare3)

07/15/2022, 9:26 PM

Since you deployed yo remote you will have to port Forward to It

Ketan (kumare3)

07/17/2022, 3:02 AM

@Marcus Isnard if you have trouble making it work let me know, initially sandbox on a remote machine was not intended but I think we should add docs for it

👀 1

Bryan Yeung

07/20/2022, 5:31 PM

Adding onto this thread. Getting an error related to the local minio installation (Mac M1). Demo env works in browser, just not example deploy.

Copy code

(flyte) (base) btyeung@by-mbp flyte-learning % pyflyte run --remote example.py wf --n 500 --mean 42 --sigma 2
Traceback (most recent call last):
  File "/Users/btyeung/.virtualenvs/flyte/lib/python3.9/site-packages/urllib3/connectionpool.py", line 703, in urlopen
    httplib_response = self._make_request(
  File "/Users/btyeung/.virtualenvs/flyte/lib/python3.9/site-packages/urllib3/connectionpool.py", line 449, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/Users/btyeung/.virtualenvs/flyte/lib/python3.9/site-packages/urllib3/connectionpool.py", line 444, in _make_request
    httplib_response = conn.getresponse()
  File "/opt/homebrew/Cellar/python@3.9/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1377, in getresponse
    response.begin()
  File "/opt/homebrew/Cellar/python@3.9/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 320, in begin
    version, status, reason = self._read_status()
  File "/opt/homebrew/Cellar/python@3.9/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 289, in _read_status
    raise RemoteDisconnected("Remote end closed connection without"
http.client.RemoteDisconnected: Remote end closed connection without response

Bryan Yeung

07/20/2022, 5:32 PM

And config file

Copy code

admin:
  # For GRPC endpoints you might want to use dns:///flyte.myexample.com
  endpoint: localhost:30081
  authType: Pkce
  insecure: true
logger:
  show-source: true
  level: 0
storage:
  connection:
    access-key: minio
    auth-type: accesskey
    disable-ssl: true
    endpoint: http://<domainName>:30084
    region: us-east-1
    secret-key: miniostorage
  type: minio
  container: "my-s3-bucket"
  enable-multicontainer: false

Ketan (kumare3)

07/21/2022, 2:29 AM

@Bryan Yeung on m1 there is a Grpc library bug

Ketan (kumare3)

07/21/2022, 2:29 AM

If you have to explicitly install something

Ketan (kumare3)

07/21/2022, 2:30 AM

Have tried this according to the docs - pip install --no-binary all grpcio --ignore-installed

Ketan (kumare3)

07/21/2022, 2:30 AM

https://docs.flyte.org/en/latest/getting_started/index.html

Bryan Yeung

07/21/2022, 3:28 AM

Will try, thanks!

Bryan Yeung

07/21/2022, 3:37 AM

Outstanding, it worked, thanks for the tip on the cli

❤️ 1

Ketan (kumare3)

07/21/2022, 4:31 AM

Fantastic

Ketan (kumare3)

07/21/2022, 4:31 AM

Please help Document better

172 Views

Open in Slack

Previous Next