No Data for the Witch’s Cauldron: Using Safe, Open-Source AI Chats systems

Learn how to assemble an open‑source chat system with OpenWeb UI, LiteLLM, Azure/Bedrock models, internal RAG, voice, image, and code integration.

Overview

Stop building chat applications, reuse existing ones and extend them ! Sharing a technical journey to use a safe and compliant solution for chatting with private project data, leveraging local and remote LLMs, internal RAG systems, MCP servers, live voice, image generation, web search, and a code interpreter. The result is similar to ChatGPT but built entirely on open-source technologies and trusted cloud providers LLM APIs.

Using OpenWeb UI as the chat front-end, connected to a foundation model on Azure AI and integrated with internal RAG system for documentation. LiteLLM serves as the OpenAPI layer for both OpenWeb UI and server calls, supporting claude code, codex-cli, and a custom workaround for gemini-cli, as we operate exclusively on AWS Bedrock and Azure AI.

Links

https://github.com/r0mdau/owuilite
OpenWebUI/LiteLLM deployment configurations using Docker Compose, Qdrant vector database, and PostgreSQL.
https://excalidraw.com/
Excalidraw is a drawing canvas managing themes via localStorage and `prefers-color-scheme` media queries.

Tech stack