#1511 Remove requirements files Pull request opened by eapolinario TL;DR Remove mentions to requirements.txt and dev-requirements.txt Type ☐ Bug Fix ☑︎ Feature ☐ Plugin Are all requirements met? ☑︎ Code completed ☑︎ Smoke tested ☐ Unit tests added ☐ Code documentation added ☐ Any pending items have an associated Issue Complete description Up to this point we operated under the assumption that maintaining a single set of requirements files would be enough. This would guarantee a semblance of reproducibility (since we would be able to reproduce the environment used for that change). That assumption broke down as we started seeing libraries not being available in all OS+architecture we support, for example,

tensorflow

does not install cleanly on Mac M1's. In this change we trade reproducibility for the maintenance burden of having exhaustive requirements files by environment. jazzband/pip-tools#826 (comment) is the current advice coming from

pip-tools

, basically if we are to do this we need to run

pip-tools

once per-environment. Instead, we're taking one step back in order to figure out what's the right next steps. The idea is that we're going to invest in a real multi-environment solution in the near future, but in the meanwhile, we're going to lose reproducibility in order to decrease the maintenance burden. Note that this change is going to subsume the need for #1424. As a side-note, we removed the legacy

all-spark

extras in #807, but never removed the CI bits related to the spark-2.4 extra. Those are being removed in this PR. Tracking Issue flyteorg/flyte#3284 Follow-up issue NA flyteorg/flytekit ✅ All checks have passed 30/30 successful checks