Mohd Shahid Khan Afridi
02/27/2023, 1:27 PMconvert_bq_table_to_pandas_dataframe
task below error is thrown. Is this combination not supported in structure_dataset encoding decoding?
flytekit/types/structured/structured_dataset.py", line 351, in _finder
raise ValueError(f"Failed to find a handler for {df_type}, protocol {protocol}, fmt {format}")
Message:
Failed to find a handler for <class 'pandas.core.frame.DataFrame'>, protocol bq, fmt parquet
[RESOLVED]@task
def convert_bq_table_to_pandas_dataframe(sd: DogeCoinDataset) -> pd.DataFrame:
return sd.open(pd.DataFrame).all()
{
"sd": {
"uri": "bq://<proj-id>:shahid.temp_table_flyte",
"format": "parquet",
"columns": [
{
"hash": "string"
},
{
"size": "integer"
},
{
"block_number": "integer"
}
]
}
}
Ketan (kumare3)
02/27/2023, 3:45 PMKevin Su
02/27/2023, 4:38 PMMohd Shahid Khan Afridi
02/27/2023, 6:24 PMgoogle-cloud-bigquery
and google-cloud-bigquery-storage
in requirement file resolved the problem