# How do I use this thing?
This is not a full turnkey setup yet?


For Ollama to work, the command line `ollama run 'model'` needs to work properly. Ensure you have enough RAM and your GPU is working as expected. The default model for this project is set to `gemma3:270m`. Ollama can be remote [Ollama Server](https://github.com/ollama/ollama/blob/main/docs/faq.md#how-do-i-configure-ollama-server) works on a pi58GB with 40 second or less response time.


# Ollama local
```bash
# bash
curl -fsSL https://ollama.com/install.sh | sh
# docker
docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -e OLLAMA_API_BASE_URL=http://host.docker.internal:11434 open-webui/open-webui
```

## Update /etc/systemd/system/ollama.service
https://github.com/ollama/ollama/issues/703
```ini
#service file addition not config.ini
# [Service]
Environment="OLLAMA_HOST=0.0.0.0:11434"
```
## validation
http://IP::11434
`Ollama is running`

## Docs
Note for LLM in docker with [NVIDIA](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/docker-specialized.html). Needed for the container with ollama running? 

---

# OpenWebUI (docker)
```bash
## ollama in docker
docker run -d -p 3000:8080 --gpus all -v open-webui:/app/backend/data --name open-webui ghcr.io/open-webui/open-webui:cuda

## external ollama
docker run -d -p 3000:8080 -e OLLAMA_BASE_URL=https://IP:11434 -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main
```
wait for engine to build, update the config.ini for the bot

```ini
# Use OpenWebUI instead of direct Ollama API (enables advanced RAG features)
useOpenWebUI = False
# OpenWebUI server URL (e.g., http://localhost:3000)
openWebUIURL = http://localhost:3000
# OpenWebUI API key/token (required when useOpenWebUI is True)
openWebUIAPIKey = sk-xxxx (see below for help)
```

## Validation
http://IP:3000
make a new admin user.
validate you have models imported or that the system is working for query.
make a new user for the bot

## API Key
- upper right settings for the user
- settings -> account
- get/create the API key for the user

## Troubleshooting
- make sure the OpenWebUI works from the bot node and loads (try lynx etc)
- make sure the model in config.ini is also loaded in OpenWebUI and you can use it
- make sure **OpenWebUI** can reach **Ollama IP** it should auto import the models
- I find using IP and not common use names like localhost which may not work well with docker etc..

- Check OpenWebUI and Ollama are working
  - Go to Admin Settings within Open WebUI.
  - Connections tab
  - Ollama connection and click on the Manage (wrench icon)
  - download models directly from the Ollama library
  - **Once the model is downloaded or imported, it will become available for use within Open WebUI, allowing you to interact with it through the chat interface**

## Docs
[OpenWebUI Quick Start](https://docs.openwebui.com/getting-started/quick-start/)
[OpenWebUI API](https://docs.openwebui.com/getting-started/api-endpoints)
[OpenWebUI Ollama](https://docs.openwebui.com/getting-started/quick-start/starting-with-ollama/)
[Blog OpenWebUI on Pi](https://pimylifeup.com/raspberry-pi-open-webui/)

https://docs.openwebui.com/tutorials/tips/rag-tutorial#tutorial-configuring-rag-with-open-webui-documentation
https://docs.openwebui.com/features/plugin/





---