Screenshot from 2024-06-27 15-56-50.png

The bizarre first answer looked like the foundational model disconnected or the embeddings mismatched. the vector db part looked like it found the loaded flyte docs fine. I added my openai_api_key and got correct answers 👍 haven't dug into streamlit/app.py yet fwiw

haha, yeah the default langchain prompt for this confuses even the SOTA models… will need to tweak to prompt to say something like “I only answer flyte questions” if the user asks something out-of-scope

the streamlit piece didn’t quite match. I had to put stuff in the Settings > Secrets (not advanced) which is toml format

strange… you don’t actually need the open ai api key in the streamlit app. Was there an error if you didn’t include it?

• In the first text cell you may want

Copy code

## Optional  1. git clone <https://github.com/unionai-oss/union-rag.git>  2. code union-rag/union_rag

What’s option 2?

re: secrets, there’s an “Advanced Settings” button in the initial app deployment flow:

let me add screenshots to make this clearer. The Settings > Secret flow is to add secrets after initial app deployment

I updated the workshop content, hopefully it’s a little clearer: https://github.com/unionai-oss/union-rag/commit/64875dac955375acf65a6474a28a9555e6f6134d

re: secrets, there’s an “Advanced Settings” button in the initial app deployment flow:

great, I must have just missed it. Re: hallucination. not clear how I got that odd answer, but undoubtedly something on my end since the streamlit part doesn't reference any openai stuff. I had a few bad (old keys generating 429) so perhaps I hit weird timing. Sifting through execution logs I couldn't find an error back from openai. Using the union ui I don't see a way "grep" the logs to hunt down the errors. I suspect that my streamlit needed a reboot and was connected to an older workflow that didn't have my correct openai key.

ps. got the "I'm not able to answer any questions that are not related to Flyte or flytekit.". btw I just use that query as my way to judge inference speed and it's using really slow (like 12s) on my laptop with llama.