We are looking at upgrading some key dependencies across our stack, and as part of that effort we are considering updating pandas to >=2.0. I notice flytekit currently has a ceiling pin for pandas <2.0. My question: is this due to a genuine known incompatibility, or can we relax this constraint now that pandas 2 has been on the block for awhile now? If it's the former, is there an issue tracking the changes required to make flytekit compatible? The issue I found (https://github.com/flyteorg/flyte/issues/3928) seems to be just requesting the pin be changed.

Need to double check with @Yee @Kevin Su but I don’t think there’s a fundamental compatibility issue here. However, we will need to think about the CI testing strategy: e.g. do we want to test both pandas 1.x.x (maybe the latest 1.x.x version?) and pandas 2+, or just pandas 2+.

I think we can just remove pandas from setup.py, but we definitely need to install pandas to test both 1.x and 2.x in the ci.

Do we need to do lazy module loading here? https://github.com/flyteorg/flytekit/blob/master/flytekit/types/structured/basic_dfs.py#L6