Hi everyone, I'm trying to run a flyte workflow remotely using flyte run --remote but this is the er...
j

Jay Phan

over 2 years ago
Hi everyone, I'm trying to run a flyte workflow remotely using flyte run --remote but this is the error I encountered, it was fine when running locally. I ran into the same problem with the very first Getting Started module.
{"asctime": "2023-02-19 10:12:08,645", "name": "flytekit.cli", "levelname": "ERROR", "message": "Non-auth RPC error <_InactiveRpcError of RPC that terminated with:\n\tstatus = StatusCode.UNAVAILABLE\n\tdetails = \"failed to connect to all addresses; last error: UNKNOWN: ipv4:127.0.0.1:30081: Failed to connect to remote host: Connection refused\"\n\tdebug_error_string = \"UNKNOWN:failed to connect to all addresses; last error: UNKNOWN: ipv4:127.0.0.1:30081: Failed to connect to remote host: Connection refused {created_time:\"2023-02-19T10:12:08.645069-05:00\", grpc_status:14}\"\n>, sleeping 200ms and retrying"}
{"asctime": "2023-02-19 10:12:08,846", "name": "flytekit.cli", "levelname": "ERROR", "message": "Non-auth RPC error <_InactiveRpcError of RPC that terminated with:\n\tstatus = StatusCode.UNAVAILABLE\n\tdetails = \"failed to connect to all addresses; last error: UNKNOWN: ipv4:127.0.0.1:30081: Failed to connect to remote host: Connection refused\"\n\tdebug_error_string = \"UNKNOWN:failed to connect to all addresses; last error: UNKNOWN: ipv4:127.0.0.1:30081: Failed to connect to remote host: Connection refused {created_time:\"2023-02-19T10:12:08.846193-05:00\", grpc_status:14}\"\n>, sleeping 400ms and retrying"}
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.10/bin/pyflyte", line 8, in <module>
    sys.exit(main())
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/flytekit/clis/sdk_in_container/run.py", line 552, in _run
    remote_entity = remote.register_script(
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/flytekit/remote/remote.py", line 800, in register_script
    upload_location, md5_bytes = fast_register_single_script(
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/flytekit/tools/script_mode.py", line 111, in fast_register_single_script
    upload_location = create_upload_location_fn(content_md5=md5)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/flytekit/clients/friendly.py", line 998, in get_upload_signed_url
    return super(SynchronousFlyteClient, self).create_upload_location(
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/flytekit/clients/raw.py", line 43, in handler
    return fn(*args, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/flytekit/clients/raw.py", line 871, in create_upload_location
    return self._dataproxy_stub.CreateUploadLocation(create_upload_location_request, metadata=self._metadata)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/grpc/_channel.py", line 946, in __call__
    return _end_unary_response_blocking(state, call, False, None)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/grpc/_channel.py", line 849, in _end_unary_response_blocking
    raise _InactiveRpcError(state)
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
	status = StatusCode.UNAVAILABLE
	details = "failed to connect to all addresses; last error: UNKNOWN: ipv4:127.0.0.1:30081: Failed to connect to remote host: Connection refused"
	debug_error_string = "UNKNOWN:failed to connect to all addresses; last error: UNKNOWN: ipv4:127.0.0.1:30081: Failed to connect to remote host: Connection refused {grpc_status:14, created_time:"2023-02-19T10:12:09.247133-05:00"}"
>
Hi everyone, I'm following along the tutorial in the Getting Started section. I'm trying to run the ...
j

Jay Phan

over 2 years ago
Hi everyone, I'm following along the tutorial in the Getting Started section. I'm trying to run the sample example.py locally but kept running into this error. Any change someone could help me with this?
File "/Library/Frameworks/Python.framework/Versions/3.11/bin/pyflyte", line 5, in <module>
    from flytekit.clis.sdk_in_container.pyflyte import main
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/flytekit/__init__.py", line 165, in <module>
    from flytekit.core.base_sql_task import SQLTask
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/flytekit/core/base_sql_task.py", line 4, in <module>
    from flytekit.core.base_task import PythonTask, TaskMetadata
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/flytekit/core/base_task.py", line 26, in <module>
    from flytekit.configuration import SerializationSettings
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/flytekit/configuration/__init__.py", line 473, in <module>
    @dataclass(init=True, repr=True, eq=True, frozen=True)
     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/dataclasses.py", line 1210, in wrap
    return _process_class(cls, init, repr, eq, order, unsafe_hash,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/dataclasses.py", line 958, in _process_class
    cls_fields.append(_get_field(cls, name, type, kw_only))
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/dataclasses.py", line 815, in _get_field
    raise ValueError(f'mutable default {type(f.default)} for field '
ValueError: mutable default <class 'flytekit.configuration.S3Config'> for field s3 is not allowed: use default_factory
I am trying to serialize and register a workflow in Github Actions. Feels like a pretty standard use...
s

Sebastian

about 3 years ago
I am trying to serialize and register a workflow in Github Actions. Feels like a pretty standard use case. Running the Github Actions workflow
jobs:
  register-flyte-workflows:
    name: Register Flyte workflows
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2

      - name: Setup flytectl
        uses: unionai-oss/flytectl-setup-action@v0.0.1

      - name: Setup pyflyte
        run: pip install flytekit==1.1.*

  - name: Serialize project
        run: |
          mkdir serialized
          pyflyte --pkgs flyte.workflows serialize --local-source-root . --image ${{ env.DOCKER_IMAGE }} workflows -f serialized
gives the following errors. The 'setup pyflyte' job prints
ERROR: googleapis-common-protos 1.56.4 has requirement protobuf<5.0.0dev,>=3.15.0, but you'll have protobuf 3.6.1 which is incompatible.
ERROR: grpcio-status 1.47.0 has requirement protobuf>=3.12.0, but you'll have protobuf 3.6.1 which is incompatible.
ERROR: cookiecutter 2.1.1 has requirement requests>=2.23.0, but you'll have requests 2.22.0 which is incompatible.
ERROR: responses 0.21.0 has requirement urllib3>=1.25.10, but you'll have urllib3 1.25.8 which is incompatible.
but the workflow still continues, then the 'serialize project' job crashes with
Traceback (most recent call last):
  File "/home/runner/.local/bin/pyflyte", line 5, in <module>
    from flytekit.clis.sdk_in_container.pyflyte import main
  File "/home/runner/.local/lib/python3.8/site-packages/flytekit/__init__.py", line 164, in <module>
    from flytekit.core.base_sql_task import SQLTask
  File "/home/runner/.local/lib/python3.8/site-packages/flytekit/core/base_sql_task.py", line 4, in <module>
    from flytekit.core.base_task import PythonTask, TaskMetadata
  File "/home/runner/.local/lib/python3.8/site-packages/flytekit/core/base_task.py", line 28, in <module>
    from flytekit.core.context_manager import ExecutionParameters, FlyteContext, FlyteContextManager, FlyteEntities
  File "/home/runner/.local/lib/python3.8/site-packages/flytekit/core/context_manager.py", line 30, in <module>
    from flytekit.clients import friendly as friendly_client  # noqa
  File "/home/runner/.local/lib/python3.8/site-packages/flytekit/clients/friendly.py", line 20, in <module>
    from flytekit.models import execution as _execution
  File "/home/runner/.local/lib/python3.8/site-packages/flytekit/models/execution.py", line 10, in <module>
    from flytekit.models import security
  File "/home/runner/.local/lib/python3.8/site-packages/flytekit/models/security.py", line 11, in <module>
    class Secret(_common.FlyteIdlEntity):
  File "/home/runner/.local/lib/python3.8/site-packages/flytekit/models/security.py", line 22, in Secret
    class MountType(Enum):
  File "/home/runner/.local/lib/python3.8/site-packages/flytekit/models/security.py", line 23, in MountType
    ANY = _sec.Secret.MountType.ANY
AttributeError: 'EnumTypeWrapper' object has no attribute 'ANY'
Error: Process completed with exit code 1.
Can someone please help me figure out what's going on? The commands work when I run them locally using pyflyte 1.1.1.