handsome-noon-32363
10/30/2022, 9:22 PM[1/1] currentAttempt done. Last Error: UNKNOWN::Outputs not generated by task execution
Can anyone help me to solve this issue.tall-lock-23197
handsome-noon-32363
10/31/2022, 12:59 PMimport typing
import pandas as pd
import numpy as np
#from databricks import sql
import os
from flytekit import task, workflow
import flyte_db_plugin as fdp
result= fdp.DatabricksTask("","").get_sql("select * from student")
@task
def compute_result(df:pd.DataFrame)-> pd.DataFrame:
return df
@workflow
def wf1()->pd.DataFrame:
# df = pd.DataFrame(result)
return (compute_result(df=result))
#if __name__ == "__main__":
print(wf1())
tall-lock-23197
tall-lock-23197
handsome-noon-32363
11/01/2022, 2:12 PM@task
def generate_normal_df():
l=[]
result= DatabricksTask("","").get_sql(<any sql query>)
print(result)
for i in result:
l=l.append(i)
print(l)
return l
@task
def compute_stats(df:pd.DataFrame) -> pd.DataFrame:
return df
@workflow
def wf():
return generate_normal_df()
if __name__ == "__main__":
print(wf())
handsome-noon-32363
11/01/2022, 2:15 PMtall-lock-23197
handsome-noon-32363
11/01/2022, 2:56 PMhandsome-noon-32363
11/01/2022, 2:57 PMtall-lock-23197
high-accountant-32689
11/02/2022, 5:43 PMDatabricksTask
? It feels a bit weird that we're calling get_sql
and that itself is reaching out to databricks. Ideally DatabricksTask
should behave similarly to SQLite3Task. Note how it's used in a workflow: https://github.com/flyteorg/flytesnacks/blob/master/cookbook/integrations/flytekit_plugins/sql/sqlite3_integration.py