acoustic-carpenter-78188
10/16/2023, 4:08 PMDoTask function
https://user-images.githubusercontent.com/76461262/281728045-0b54f7ae-0d4b-451a-aebd-0c804fb661cd.png▾
TaskExecutor class, a base agent for all plugins with DoTask function
https://user-images.githubusercontent.com/76461262/281728082-5bd24323-6e92-4c5c-9361-aad4fd534b27.png▾
base_agent.py. ASYNC_PLUGIN and SYNC_PLUGIN. They are use in runtime flavor from TaskMetadata
• Use the runtime_flavor in TaskMetadata (It hasn't been used before in flytekit)
• Change the usecase of runtime_flavor by adding comments
Change the usecase to the plugin routing mechanism
• Add runtime_flavor paramters in all agent task and webapi task
Here's the list.
plugins/flytekit-airflow
plugins/flytekit-aws-athena
plugins/flytekit-aws-batch
plugins/flytekit-bigquery
plugins/flytekit-mmcloud
plugins/flytekit-snowflake
plugins/flytekit-spark
flytekit/sensor
I get the list from
1. https://github.com/flyteorg/flyte/tree/master/flyteplugins/go/tasks/plugins/webapi
2. use find . | grep agent.py under the flytekit folder
3. sensor is a special case.
Note: ChatGPT doensn't need it because the external api task has done it for it.
• Add test for the above changes
• Finish above without introducing breaking changes
• Currently, ChatGPT timeout works when retrying the second time in an API call, which means that if the query takes you 40seconds, you have to wait 40seconds.
This is not ideal, we will improve it when a new server integrates into Flyte Backend System. Currently, we can set the timeout settings in the config-map.
Additional Information
Note: the timeout of the ChatGPT API will check the logic when retry,
so I highly recommend users change the timeout of the DoTask function to 40 seconds,
which will probably not exceed the limit, and we will introduce a api_task_server to fix this situation.
Note: If we use aiohttp to prevent the timeout by setting the timeout time, it is doable,
but in my current experiment, openai.acreate is faster than using aiohttp (I haven't figure it out).
Here is the proof:
https://hackmd.io/Fmx3DHGtQBSqYydcrdF_Cg
Type
☐ Bug Fix
☑︎ Feature
☑︎ Plugin
Are all requirements met?
☑︎ Code completed
☐ Smoke tested
☑︎ Unit tests added
☐ Code documentation added
☐ Any pending items have an associated Issue
How to test it?
Configuration Example
tasks:
task-plugins:
enabled-plugins:
- agent-service
default-for-task-types:
api_task: agent-service
plugins:
agent-service:
supportedTaskTypes:
- api_task
defaultAgent:
endpoint: "dns:///localhost:8000"
insecure: true
timeouts:
DoTask: 100s
Run in Remote Environment
You can test the agent with Dockerfile.
FROM python:3.9-slim-buster
USER root
WORKDIR /root
ENV PYTHONPATH /root
RUN apt-get update && apt-get install build-essential -y
RUN apt-get install git -y
RUN pip install -U git+https://github.com/Future-Outlier/flytekit.git@0ecebdfc3a018d888748e4d9adf715c72c761180#subdirectory=plugins/flytekit-openai-chatgpt
RUN pip install -U git+https://github.com/Future-Outlier/flyte.git@b879137d24fe6bfb60d6d1182cd8d9b3f32335cc#subdirectory=flyteidl
RUN pip install -U git+https://github.com/Future-Outlier/flytekit.git@0ecebdfc3a018d888748e4d9adf715c72c761180
from flytekit import task, workflow, ImageSpec
from flytekitplugins.chatgpt import ChatGPTTask
chatgpt_job = ChatGPTTask(
name="chatgpt",
config={
"openai_organization": "org-NayNG68kGnVXMJ8Ak4PMgQv7",
"chatgpt_conf": {
"model": "gpt-3.5-turbo",
# 'messages': [{'role': 'user', 'content': 'Say this is a test!'}],
"temperature": 0.7,
},
},
)
@task()
def t1(s: str) -> str:
s = "Repsonse: " + s
return s
@workflow
def wf() -> str:
message = chatgpt_job(message="hi")
return t1(s=message)
if __name__ == "__main__":
print(wf())
1. start your grpc agent server
pyflyte serve
2. execute it in remote environment
pyflyte run --remote --image futureoutlier/flytekit:chatgpt-v2 chatgpt_example.py wf
Screen Shot
image▾
api_task.
How it works at high level
image▾
acoustic-carpenter-78188
10/16/2023, 4:08 PM