I am trying to run basic BigQueryTask example prov...
# ask-the-community
m
I am trying to run basic BigQueryTask example provided in bq plugin doc, when it runs the
convert_bq_table_to_pandas_dataframe
task below error is thrown. Is this combination not supported in structure_dataset encoding decoding?
Copy code
flytekit/types/structured/structured_dataset.py", line 351, in _finder
        raise ValueError(f"Failed to find a handler for {df_type}, protocol {protocol}, fmt {format}")
Message:
    Failed to find a handler for <class 'pandas.core.frame.DataFrame'>, protocol bq, fmt parquet
[RESOLVED]
task code
Copy code
@task
def convert_bq_table_to_pandas_dataframe(sd: DogeCoinDataset) -> pd.DataFrame:
    return sd.open(pd.DataFrame).all()
input to the task looks like
Copy code
{
  "sd": {
    "uri": "bq://<proj-id>:shahid.temp_table_flyte",
    "format": "parquet",
    "columns": [
      {
        "hash": "string"
      },
      {
        "size": "integer"
      },
      {
        "block_number": "integer"
      }
    ]
  }
}
k
This seems like a dependency issue. Cc @Yee do you need to install some bq dependency? Is that not included with bq plugin
m
Thanks @Ketan (kumare3) @Kevin Su 🙏 Including
google-cloud-bigquery
and
google-cloud-bigquery-storage
in requirement file resolved the problem
101 Views