mirror of
https://github.com/SpudGunMan/meshing-around.git
synced 2026-03-28 17:32:36 +01:00
88 lines
3.3 KiB
Markdown
88 lines
3.3 KiB
Markdown
# How do I use this thing?
|
|
This is not a full turnkey setup yet?
|
|
|
|
|
|
For Ollama to work, the command line `ollama run 'model'` needs to work properly. Ensure you have enough RAM and your GPU is working as expected. The default model for this project is set to `gemma3:270m`. Ollama can be remote [Ollama Server](https://github.com/ollama/ollama/blob/main/docs/faq.md#how-do-i-configure-ollama-server) works on a pi58GB with 40 second or less response time.
|
|
|
|
|
|
# Ollama local
|
|
```bash
|
|
# bash
|
|
curl -fsSL https://ollama.com/install.sh | sh
|
|
# docker
|
|
docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -e OLLAMA_API_BASE_URL=http://host.docker.internal:11434 open-webui/open-webui
|
|
```
|
|
|
|
## Update /etc/systemd/system/ollama.service
|
|
https://github.com/ollama/ollama/issues/703
|
|
```ini
|
|
#service file addition not config.ini
|
|
# [Service]
|
|
Environment="OLLAMA_HOST=0.0.0.0:11434"
|
|
```
|
|
## validation
|
|
http://IP::11434
|
|
`Ollama is running`
|
|
|
|
## Docs
|
|
Note for LLM in docker with [NVIDIA](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/docker-specialized.html). Needed for the container with ollama running?
|
|
|
|
---
|
|
|
|
# OpenWebUI (docker)
|
|
```bash
|
|
## ollama in docker
|
|
docker run -d -p 3000:8080 --gpus all -v open-webui:/app/backend/data --name open-webui ghcr.io/open-webui/open-webui:cuda
|
|
|
|
## external ollama
|
|
docker run -d -p 3000:8080 -e OLLAMA_BASE_URL=https://IP:11434 -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main
|
|
```
|
|
wait for engine to build, update the config.ini for the bot
|
|
|
|
```ini
|
|
# Use OpenWebUI instead of direct Ollama API (enables advanced RAG features)
|
|
useOpenWebUI = False
|
|
# OpenWebUI server URL (e.g., http://localhost:3000)
|
|
openWebUIURL = http://localhost:3000
|
|
# OpenWebUI API key/token (required when useOpenWebUI is True)
|
|
openWebUIAPIKey = sk-xxxx (see below for help)
|
|
```
|
|
|
|
## Validation
|
|
http://IP:3000
|
|
make a new admin user.
|
|
validate you have models imported or that the system is working for query.
|
|
make a new user for the bot
|
|
|
|
## API Key
|
|
- upper right settings for the user
|
|
- settings -> account
|
|
- get/create the API key for the user
|
|
|
|
## Troubleshooting
|
|
- make sure the OpenWebUI works from the bot node and loads (try lynx etc)
|
|
- make sure the model in config.ini is also loaded in OpenWebUI and you can use it
|
|
- make sure **OpenWebUI** can reach **Ollama IP** it should auto import the models
|
|
- I find using IP and not common use names like localhost which may not work well with docker etc..
|
|
|
|
- Check OpenWebUI and Ollama are working
|
|
- Go to Admin Settings within Open WebUI.
|
|
- Connections tab
|
|
- Ollama connection and click on the Manage (wrench icon)
|
|
- download models directly from the Ollama library
|
|
- **Once the model is downloaded or imported, it will become available for use within Open WebUI, allowing you to interact with it through the chat interface**
|
|
|
|
## Docs
|
|
[OpenWebUI Quick Start](https://docs.openwebui.com/getting-started/quick-start/)
|
|
[OpenWebUI API](https://docs.openwebui.com/getting-started/api-endpoints)
|
|
[OpenWebUI Ollama](https://docs.openwebui.com/getting-started/quick-start/starting-with-ollama/)
|
|
[Blog OpenWebUI on Pi](https://pimylifeup.com/raspberry-pi-open-webui/)
|
|
|
|
https://docs.openwebui.com/tutorials/tips/rag-tutorial#tutorial-configuring-rag-with-open-webui-documentation
|
|
https://docs.openwebui.com/features/plugin/
|
|
|
|
|
|
|
|
|
|
|
|
--- |