I’m building a data validation and cleaning tool. Users upload a dataset.csv and validation_rules.csv. I tried using LLMs, but hit rate limits due to large data and free plan limits. Now I’m exploring Pandera for static validation in Python

Reasoning Complete! 🤖 🔍 Searched knowledge base: To help you use Pandera for validating your datasets with user-supplied rules, I'll look up the best ways to implement this in Python, including how to handle large files and dynamic validation rules. • Searching over 10 data sources from the following: File, Github File, Github Issue, Web • Found 57 documents that could be relevant from the following data sources: Github File, Github Issue, Github Pull Request. • Filtered down to the top 30 most relevant documents. • To help you automate the process of turning your validation_rules.csv into a Pandera schema, I'll look for best practices or examples on how to map CSV-based rules into Pandera's validation system. • Searching over 10 data sources from the following: File, Github File, Github Issue, Web • Found 54 documents that could be relevant from the following data sources: Github File, Github Issue, Github Pull Request. • Filtered down to the top 30 most relevant documents. • To give you the most practical advice, I'll look for examples or best practices on how others have built tools to convert a CSV of validation rules into Pandera schemas, so you can automate this step efficiently. • Searching over 10 data sources from the following: File, Github File, Github Issue, Web • Found 55 documents that could be relevant from the following data sources: Github File, Github Issue, Github Pull Request. • Filtered down to the top 30 most relevant documents. ✍️ Generated answer: I now have enough information to give you a clear answer on how to use Pandera for your use case, including how to handle user-uploaded validation rules and what to expect regarding data cleaning.