I am trying to serialize and register a workflow i...
# ask-the-community
s
I am trying to serialize and register a workflow in Github Actions. Feels like a pretty standard use case. Running the Github Actions workflow
Copy code
jobs:
  register-flyte-workflows:
    name: Register Flyte workflows
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2

      - name: Setup flytectl
        uses: unionai-oss/flytectl-setup-action@v0.0.1

      - name: Setup pyflyte
        run: pip install flytekit==1.1.*

  - name: Serialize project
        run: |
          mkdir serialized
          pyflyte --pkgs flyte.workflows serialize --local-source-root . --image ${{ env.DOCKER_IMAGE }} workflows -f serialized
gives the following errors. The 'setup pyflyte' job prints
Copy code
ERROR: googleapis-common-protos 1.56.4 has requirement protobuf<5.0.0dev,>=3.15.0, but you'll have protobuf 3.6.1 which is incompatible.
ERROR: grpcio-status 1.47.0 has requirement protobuf>=3.12.0, but you'll have protobuf 3.6.1 which is incompatible.
ERROR: cookiecutter 2.1.1 has requirement requests>=2.23.0, but you'll have requests 2.22.0 which is incompatible.
ERROR: responses 0.21.0 has requirement urllib3>=1.25.10, but you'll have urllib3 1.25.8 which is incompatible.
but the workflow still continues, then the 'serialize project' job crashes with
Copy code
Traceback (most recent call last):
  File "/home/runner/.local/bin/pyflyte", line 5, in <module>
    from flytekit.clis.sdk_in_container.pyflyte import main
  File "/home/runner/.local/lib/python3.8/site-packages/flytekit/__init__.py", line 164, in <module>
    from flytekit.core.base_sql_task import SQLTask
  File "/home/runner/.local/lib/python3.8/site-packages/flytekit/core/base_sql_task.py", line 4, in <module>
    from flytekit.core.base_task import PythonTask, TaskMetadata
  File "/home/runner/.local/lib/python3.8/site-packages/flytekit/core/base_task.py", line 28, in <module>
    from flytekit.core.context_manager import ExecutionParameters, FlyteContext, FlyteContextManager, FlyteEntities
  File "/home/runner/.local/lib/python3.8/site-packages/flytekit/core/context_manager.py", line 30, in <module>
    from flytekit.clients import friendly as friendly_client  # noqa
  File "/home/runner/.local/lib/python3.8/site-packages/flytekit/clients/friendly.py", line 20, in <module>
    from flytekit.models import execution as _execution
  File "/home/runner/.local/lib/python3.8/site-packages/flytekit/models/execution.py", line 10, in <module>
    from flytekit.models import security
  File "/home/runner/.local/lib/python3.8/site-packages/flytekit/models/security.py", line 11, in <module>
    class Secret(_common.FlyteIdlEntity):
  File "/home/runner/.local/lib/python3.8/site-packages/flytekit/models/security.py", line 22, in Secret
    class MountType(Enum):
  File "/home/runner/.local/lib/python3.8/site-packages/flytekit/models/security.py", line 23, in MountType
    ANY = _sec.Secret.MountType.ANY
AttributeError: 'EnumTypeWrapper' object has no attribute 'ANY'
Error: Process completed with exit code 1.
Can someone please help me figure out what's going on? The commands work when I run them locally using pyflyte 1.1.1.
s
cc: @Eduardo Apolinario (eapolinario) @Yee
šŸ™ 1
šŸ‘ 1
k
Are you using the standard published GitHub action? Cc @Yuvraj.
n
Copy code
AttributeError: 'EnumTypeWrapper' object has no attribute 'ANY'
Seems like
flyteidl
is out-of-date? @Sebastian can you get at the version of
flyteidl
installed?
s
Thank you for your responses! @Ketan (kumare3) I am using exactly the workflow I posted above. Are there other details that you need? I might not understand what you're asking for. @Niels Bantilan It is installing flyteidl-1.1.12-py3-none-any.whl, which has the same version number as what I run locally (which can run the register command). Do note that there are other errors with the pip install command though, which might be related.
n
can you do a
pip list
after installing flytekit in your workflow and paste the results here? just wanted to check all the versions. Also not sure why those package version errors are happeningā€¦
I also wonder if using
unionai-oss/flytectl-setup-action@master
will make any differenceā€¦
m
FWIW I am getting the exact same problem, hereā€™s my pip list: https://gist.github.com/hwaxxer/c36f01cb4ff2ff5562e944421c4afded
šŸ‘ 1
n
interestingā€¦ @Yuvraj @Eduardo Apolinario (eapolinario) @Yee we should look into why this error is occurring on
unionai-oss/flytectl-setup-action
, it seems like all the correct versions of flytekit and flyteidl are installed.
e
yeah, I'll take a look at this shortly.
@Sebastian, @Martin Hwasser, I was able to repro the error. Can you install another python version by, for example, using https://github.com/actions/setup-python ? The python version that comes with the
ubuntu-latest
is relatively old (it comes with python 3.8.10).
s
Thank you @Eduardo Apolinario (eapolinario), using python 3.10 solves the issue šŸ‘
k
@Eduardo Apolinario (eapolinario) but why does 3.8 does not work?
e
using that gh action to setup python and setting the version to '3.8' also works.
we're working on a template that will include examples of github workflows. I'll make sure to include a comment explaining this there.
163 Views