Quick feedback. Great to have the workshop, Thank ...
# llm-rag-workshop-june-2024
c
Quick feedback. Great to have the workshop, Thank you! Here are some nits: • update "will that" -> "This step will take about 5-10 minutes" • In the first text cell you may want
Copy code
## Optional  1. git clone <https://github.com/unionai-oss/union-rag.git>  2. code union-rag/union_rag
• the streamlit piece didn't quite match. I had to put stuff in the Settings > Secrets (not advanced) which is toml format, and had to include the openai_api_key 🧵 then restart the app by reloading the app's page.
Copy code
UNIONAI_SERVERLESS_API_KEY="c2Vydxxx"
OPENAI_API_KEY="sk-proj-qxTvSMxxx"
Got mine working after many zoom meeting interruptions 🙂
Screenshot from 2024-06-27 15-56-50.png
The bizarre first answer looked like the foundational model disconnected or the embeddings mismatched. the vector db part looked like it found the loaded flyte docs fine. I added my openai_api_key and got correct answers 👍 haven't dug into streamlit/app.py yet fwiw
n
thanks Charles! I’ll update the guide based on your feedback
haha, yeah the default langchain prompt for this confuses even the SOTA models… will need to tweak to prompt to say something like “I only answer flyte questions” if the user asks something out-of-scope
the streamlit piece didn’t quite match. I had to put stuff in the Settings > Secrets (not advanced) which is toml format
strange… you don’t actually need the open ai api key in the streamlit app. Was there an error if you didn’t include it?
• In the first text cell you may want
Copy code
## Optional  1. git clone <https://github.com/unionai-oss/union-rag.git>  2. code union-rag/union_rag
What’s option 2?
re: secrets, there’s an “Advanced Settings” button in the initial app deployment flow:
let me add screenshots to make this clearer. The Settings > Secret flow is to add secrets after initial app deployment
I updated the workshop content, hopefully it’s a little clearer: https://github.com/unionai-oss/union-rag/commit/64875dac955375acf65a6474a28a9555e6f6134d
c
2. code union-rag/union_rag
this was just to open visual code on the git repo that was just cloned. I'm cmdline / linux so perhaps that doesn't work for everyone, but I think it should in a mac terminal or WSL
re: secrets, there’s an “Advanced Settings” button in the initial app deployment flow:
great, I must have just missed it. Re: hallucination. not clear how I got that odd answer, but undoubtedly something on my end since the streamlit part doesn't reference any openai stuff. I had a few bad (old keys generating 429) so perhaps I hit weird timing. Sifting through execution logs I couldn't find an error back from openai. Using the union ui I don't see a way "grep" the logs to hunt down the errors. I suspect that my streamlit needed a reboot and was connected to an older workflow that didn't have my correct openai key.
ps. got the "I'm not able to answer any questions that are not related to Flyte or flytekit.". btw I just use that query as my way to judge inference speed and it's using really slow (like 12s) on my laptop with llama.