Hi everyone, is torch a required dependency of flytekit? Or is it possible to install without the

extras

?

File "/home/ttheisen/repos/chariot/py/apps/v2-model-recommender/internal/workflow/secrets.py", line 1, in <module>
    from flytekit import Secret, current_context
  File "/home/ttheisen/.cache/pypoetry/virtualenvs/v2-model-recommender-KgGCWpXS-py3.8/lib/python3.8/site-packages/flytekit/__init__.py", line 187, in <module>
    from flytekit.extras import pytorch
  File "/home/ttheisen/.cache/pypoetry/virtualenvs/v2-model-recommender-KgGCWpXS-py3.8/lib/python3.8/site-packages/flytekit/extras/pytorch/__init__.py", line 18, in <module>
    import torch
  File "/home/ttheisen/.cache/pypoetry/virtualenvs/v2-model-recommender-KgGCWpXS-py3.8/lib/python3.8/site-packages/torch/__init__.py", line 228, in <module>
    _load_global_deps()
  File "/home/ttheisen/.cache/pypoetry/virtualenvs/v2-model-recommender-KgGCWpXS-py3.8/lib/python3.8/site-packages/torch/__init__.py", line 189, in _load_global_deps
    _preload_cuda_deps(lib_folder, lib_name)
  File "/home/ttheisen/.cache/pypoetry/virtualenvs/v2-model-recommender-KgGCWpXS-py3.8/lib/python3.8/site-packages/torch/__init__.py", line 154, in _preload_cuda_deps
    raise ValueError(f"{lib_name} not found in the system path {sys.path}")
ValueError: libcublas.so.*[0-9] not found in the system path ['/home/ttheisen/repos/chariot/py/apps/v2-model-recommender', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/ttheisen/.cache/pypoetry/virtualenvs/v2-model-recommender-KgGCWpXS-py3.8/lib/python3.8/site-packages']
make: *** [Makefile:34: run] Error 1

you don’t need torch in the flytekit.

But a ValueError is being raised (not ImportError or OSError)

this is the key here. We're being too optimistic in https://github.com/flyteorg/flytekit/blob/master/flytekit/extras/pytorch/__init__.py#L17-L22. For some reason in @Tom Theisen’s case the import statement raises a

ValueError

. We should catch that just to be safe.

After upgrading to 1.6.0 I get past the issue