https://flyte.org logo
Powered by
#ask-the-community
Title
# ask-the-community
m

Mohd Shahid Khan Afridi

02/27/2023, 1:27 PM
I am trying to run basic BigQueryTask example provided in bq plugin doc, when it runs the 
convert_bq_table_to_pandas_dataframe
task below error is thrown. Is this combination not supported in structure_dataset encoding decoding?
Copy code
flytekit/types/structured/structured_dataset.py", line 351, in _finder
        raise ValueError(f"Failed to find a handler for {df_type}, protocol {protocol}, fmt {format}")
Message:
    Failed to find a handler for <class 'pandas.core.frame.DataFrame'>, protocol bq, fmt parquet
[RESOLVED]
task code
Copy code
@task
def convert_bq_table_to_pandas_dataframe(sd: DogeCoinDataset) -> pd.DataFrame:
    return sd.open(pd.DataFrame).all()
input to the task looks like
Copy code
{
  "sd": {
    "uri": "bq://<proj-id>:shahid.temp_table_flyte",
    "format": "parquet",
    "columns": [
      {
        "hash": "string"
      },
      {
        "size": "integer"
      },
      {
        "block_number": "integer"
      }
    ]
  }
}
k

Ketan (kumare3)

02/27/2023, 3:45 PM
This seems like a dependency issue. Cc @Yee do you need to install some bq dependency? Is that not included with bq plugin
k

Kevin Su

02/27/2023, 4:38 PM
m

Mohd Shahid Khan Afridi

02/27/2023, 6:24 PM
Thanks @Ketan (kumare3) @Kevin Su 🙏 Including 
google-cloud-bigquery
and 
google-cloud-bigquery-storage
in requirement file resolved the problem
Powered by