chilly-agent-24526
02/17/2023, 4:07 PMparallel_requests
is my own library, which uses aiohttp and asyncio, to run requests in parallel.
from parallel_requests import parallel_requests
from flytekit import task, workflow
@task
def download(urls:list)->list:
#return requests.get(url, headers={"user-agent":"my-user-agent"}).json()
return parallel_requests(
urls=urls
)
@workflow
def run():
urls = ["<https://query2.finance.yahoo.com/v7/finance/quote?symbols=AAPL>"]*10
res = download(urls=urls)
print(res[0])
if __name__=='__main__':
run()
Running the workflow with pyflyte run test_flyte_with_asyncio.py run
gives me the followin error:
Traceback (most recent call last):
File "/root/mambaforge/envs/flyte/lib/python3.10/site-packages/flytekit/core/type_engine.py", line 914, in get_literal_type
sub_type = TypeEngine.to_literal_type(self.get_sub_type(t))
File "/root/mambaforge/envs/flyte/lib/python3.10/site-packages/flytekit/core/type_engine.py", line 907, in get_sub_type
raise ValueError("Only generic univariate typing.List[T] type is supported.")
ValueError: Only generic univariate typing.List[T] type is supported.
What am I doing wrong here?
Thankswhite-yak-77603
02/17/2023, 4:10 PM