<#1822 Chatgpt Agent> Pull request opened by <Futu...
# flyte-github
a
#1822 Chatgpt Agent Pull request opened by Future-Outlier TL;DR • Create Base class for all external API tasks, a base task for all plugins with
DoTask
function

https://user-images.githubusercontent.com/76461262/281728045-0b54f7ae-0d4b-451a-aebd-0c804fb661cd.png▾

• Create an
TaskExecutor
class, a base agent for all plugins with
DoTask
function

https://user-images.githubusercontent.com/76461262/281728082-5bd24323-6e92-4c5c-9361-aad4fd534b27.png▾

• Create a ChatGPT plugin by OpenAI API • Support local execution for do task function • Do refactor for base agent • Add 2 variable in
base_agent.py
.
ASYNC_PLUGIN
and
SYNC_PLUGIN
. They are use in
runtime flavor
from
TaskMetadata
• Use the
runtime_flavor
in
TaskMetadata
(It hasn't been used before in flytekit) • Change the usecase of
runtime_flavor
by adding comments Change the usecase to the plugin routing mechanism • Add
runtime_flavor
paramters in all agent task and webapi task Here's the list. plugins/flytekit-airflow plugins/flytekit-aws-athena plugins/flytekit-aws-batch plugins/flytekit-bigquery plugins/flytekit-mmcloud plugins/flytekit-snowflake plugins/flytekit-spark flytekit/sensor I get the list from 1. https://github.com/flyteorg/flyte/tree/master/flyteplugins/go/tasks/plugins/webapi 2. use
find . | grep agent.py
under the
flytekit
folder 3. sensor is a special case. Note: ChatGPT doensn't need it because the
external api task
has done it for it. • Add test for the above changes • Finish above without introducing breaking changes • Currently, ChatGPT timeout works when retrying the second time in an API call, which means that if the query takes you 40seconds, you have to wait 40seconds. This is not ideal, we will improve it when a new server integrates into Flyte Backend System. Currently, we can set the timeout settings in the config-map. Additional Information Note: the timeout of the ChatGPT API will check the logic when retry, so I highly recommend users change the timeout of the
DoTask
function to 40 seconds, which will probably not exceed the limit, and we will introduce a
api_task_server
to fix this situation. Note: If we use aiohttp to prevent the timeout by setting the timeout time, it is doable, but in my current experiment,
openai.acreate
is faster than using aiohttp (I haven't figure it out). Here is the proof: https://hackmd.io/Fmx3DHGtQBSqYydcrdF_Cg Type ☐ Bug Fix ☑︎ Feature ☑︎ Plugin Are all requirements met? ☑︎ Code completed ☐ Smoke tested ☑︎ Unit tests added ☐ Code documentation added ☐ Any pending items have an associated Issue How to test it? Configuration Example
Copy code
tasks:
  task-plugins:
    enabled-plugins:
      - agent-service
    default-for-task-types:
      api_task: agent-service
plugins:
  agent-service:
    supportedTaskTypes:
      - api_task
    defaultAgent:
      endpoint: "dns:///localhost:8000"
      insecure: true
      timeouts:
        DoTask: 100s
Run in Remote Environment You can test the agent with Dockerfile.
Copy code
FROM python:3.9-slim-buster
USER root
WORKDIR /root
ENV PYTHONPATH /root
RUN apt-get update && apt-get install build-essential -y
RUN apt-get install git -y
RUN pip install -U git+https://github.com/Future-Outlier/flytekit.git@0ecebdfc3a018d888748e4d9adf715c72c761180#subdirectory=plugins/flytekit-openai-chatgpt

RUN pip install -U git+https://github.com/Future-Outlier/flyte.git@b879137d24fe6bfb60d6d1182cd8d9b3f32335cc#subdirectory=flyteidl

RUN pip install -U git+https://github.com/Future-Outlier/flytekit.git@0ecebdfc3a018d888748e4d9adf715c72c761180
Copy code
from flytekit import task, workflow, ImageSpec
from flytekitplugins.chatgpt import ChatGPTTask


chatgpt_job = ChatGPTTask(
    name="chatgpt",
    config={
        "openai_organization": "org-NayNG68kGnVXMJ8Ak4PMgQv7",
        "chatgpt_conf": {
            "model": "gpt-3.5-turbo",
            # 'messages': [{'role': 'user', 'content': 'Say this is a test!'}],
            "temperature": 0.7,
        },
    },
)

@task()
def t1(s: str) -> str:
    s = "Repsonse: " + s
    return s

@workflow
def wf() -> str:
    message = chatgpt_job(message="hi")
    return t1(s=message)

if __name__ == "__main__":
    print(wf())
1. start your grpc agent server
Copy code
pyflyte serve
2. execute it in remote environment
Copy code
pyflyte run --remote --image futureoutlier/flytekit:chatgpt-v2 chatgpt_example.py wf
Screen Shot

image▾

Note: You can find that the ChatGPT task is an
api_task
. How it works at high level

image▾

Tracking Issue flyteorg/flyte#3936 flyteorg/flytekit GitHub Actions: build (macos-latest, 3.11) GitHub Actions: build (macos-latest, 3.8) GitHub Actions: build (windows-latest, 3.11) GitHub Actions: build (windows-latest, 3.8) GitHub Actions: build (ubuntu-latest, 3.11) GitHub Actions: build-integration (ubuntu-latest, 3.11) GitHub Actions: build (ubuntu-latest, 3.8) GitHub Actions: build-integration (ubuntu-latest, 3.8) GitHub Actions: lint GitHub Actions: docs GitHub Actions: Docs Warnings ✅ 2 other checks have passed 2/13 successful checks