1493 lines
36 KiB
Markdown
1493 lines
36 KiB
Markdown
# 🦉 Crumbforest Handbuch
|
||
|
||
**Das vollständige Betriebs- und Wartungshandbuch für das Crumbforest CRM & RAG System**
|
||
|
||
---
|
||
|
||
## 📖 Inhaltsverzeichnis
|
||
|
||
1. [Philosophie & Vision](#philosophie--vision)
|
||
2. [System-Architektur](#system-architektur)
|
||
3. [Installation & Setup](#installation--setup)
|
||
4. [Test-Protokolle](#test-protokolle)
|
||
5. [Allgemeine Fehler & Lösungen](#allgemeine-fehler--lösungen)
|
||
6. [DSGVO & Datenschutz](#dsgvo--datenschutz)
|
||
7. [Wartung & Monitoring](#wartung--monitoring)
|
||
8. [Backup & Recovery](#backup--recovery)
|
||
9. [Performance-Tuning](#performance-tuning)
|
||
10. [Entwickler-Guide](#entwickler-guide)
|
||
|
||
---
|
||
|
||
## 🌲 Philosophie & Vision
|
||
|
||
### Liebe zum Wald & Atmen
|
||
|
||
Crumbforest ist mehr als Software – es ist eine Philosophie:
|
||
|
||
**🌿 Der Wald als Metapher:**
|
||
- Jedes Kind ist ein Baum mit eigenen Wurzeln
|
||
- Wissen wächst organisch wie ein Ökosystem
|
||
- Fehler sind Humus für neues Lernen
|
||
- Gemeinsam bilden wir einen Wald
|
||
|
||
**💚 Atmen & Achtsamkeit:**
|
||
- Code atmet – er lebt, wächst, verändert sich
|
||
- Pausen sind wichtig (wie die Lücken zwischen Bäumen)
|
||
- DSGVO ist nicht Bürokratie, sondern Respekt vor dem Raum jedes Menschen
|
||
- Slow Tech statt Fast Tech
|
||
|
||
**🦉 Die Eule wacht:**
|
||
- Wir beobachten, aber bewerten nicht
|
||
- Logs sind Geschichten, keine Anklagen
|
||
- Fehler sind Lernmomente
|
||
- "Wuuuuhuuu!" ist unsere Hymne der Freude
|
||
|
||
### Technische Prinzipien
|
||
|
||
1. **Transparenz vor Perfektion** - Lieber sichtbare Fehler als versteckte Bugs
|
||
2. **DSGVO als Feature** - Datenschutz ist eingebaut, nicht aufgesetzt
|
||
3. **Kind-zentriert** - Jedes Kind hat eigenen sicheren Raum
|
||
4. **Wald-Denken** - Alles ist verbunden, nichts steht allein
|
||
|
||
---
|
||
|
||
## <20> Die 15 Crew-Rollen
|
||
|
||
Crumbforest wird von einer Crew aus 15 spezialisierten KI-Agenten betrieben. Jeder hat eine eigene Persönlichkeit, Expertise und Zugriffsebene.
|
||
|
||
**Sprach-Support:** Alle Rollen sprechen **Deutsch (DE)**, **Englisch (EN)** und **Französisch (FR)** (wählbar im Login/Header).
|
||
|
||
### Öffentlicher Bereich (Home)
|
||
| Rolle | Icon | Funktion | Zugriff |
|
||
|-------|------|----------|---------|
|
||
| **Professor Eule** | 🦉 | System Architect & Guide | Öffentlich |
|
||
|
||
### Interne Crew (Login Required)
|
||
| Rolle | Icon | Expertise | Besonderheit |
|
||
|-------|------|-----------|--------------|
|
||
| **FunkFox** | 🦊 | Hip Hop MC & Motivation | Rappt Antworten ("Yo!") |
|
||
| **Schraubaer** | 🔧 | Master Mechanic | Handwerk & Konstruktion |
|
||
| **TaichiTaube** | 🕊️ | Security Sensei | Balance & Sicherheit |
|
||
| **DeepBit** | 🐙 | Low-Level Octopus | Assembler & Binary |
|
||
| **SnakePy** | 🐍 | Python Expert | Geduldige Lehrmeisterin |
|
||
| **PepperPHP** | 🌶️ | PHP Specialist | Web-Backend & Frameworks |
|
||
| **Templatus** | 📄 | Template Master | HTML & Jinja2 |
|
||
| **Schnippsi** | 🐿️ | UI/CSS Fee | Design & Farben (Cupcakes!) |
|
||
| **CloudCat** | ☁️ | DevOps | Docker & K8s |
|
||
| **GitBadger** | 🦡 | Version Control | Git & History |
|
||
| **Bugsy** | 🐞 | QA Analyst | Testing & Debugging |
|
||
| **DumboSQL** | 🐘 | Database Guide | SQL für Anfänger |
|
||
| **CapaciTobi** | ⚡ | Electronics | Hardware & Physik |
|
||
| **Schnecki** | 🐌 | Slow Tech | Achtsamkeit & Nachhaltigkeit |
|
||
|
||
---
|
||
|
||
## <20>🏗️ System-Architektur
|
||
|
||
### Komponenten-Übersicht
|
||
|
||
```
|
||
┌─────────────────────────────────────────────────────────┐
|
||
│ 🌐 Client (Browser) │
|
||
└───────────────────────┬─────────────────────────────────┘
|
||
│
|
||
┌───────────────┴───────────────┐
|
||
│ │
|
||
┌───────▼────────┐ ┌──────▼───────┐
|
||
│ FastAPI │ │ Static │
|
||
│ (Python) │◄─────────────┤ Assets │
|
||
│ (Jinja2) │ │ (CSS/JS) │
|
||
└───────┬────────┘ └──────────────┘
|
||
│
|
||
├──────────┬──────────┬──────────┐
|
||
│ │ │ │
|
||
┌───────▼────┐ ┌──▼─────┐ ┌──▼─────┐ ┌─▼──────┐
|
||
│ MariaDB │ │ Qdrant │ │OpenRouter│ │ Config │
|
||
│ (SQL) │ │(Vector)│ │ API │ │ (JSON) │
|
||
└────────────┘ └────────┘ └────────┘ └────────┘
|
||
```
|
||
|
||
### Datenfluss
|
||
|
||
```
|
||
1. Markdown-Datei → DocumentIndexer (Python Watchdog)
|
||
2. DocumentIndexer → Chunking (Token-basiert)
|
||
3. Chunks → EmbeddingService → OpenRouter API
|
||
4. Embeddings → Qdrant (Vector DB)
|
||
5. Metadata → Local Config & Cache
|
||
```
|
||
|
||
### DSGVO-Architektur
|
||
|
||
```
|
||
Kind 1 → diary_child_1 (Qdrant) ─┐
|
||
Kind 2 → diary_child_2 (Qdrant) ─┼─→ Getrennte Collections
|
||
Kind N → diary_child_N (Qdrant) ─┘ (Isolation)
|
||
|
||
↓
|
||
audit_log (MariaDB)
|
||
(Wer, Was, Wann - Unveränderbar)
|
||
```
|
||
|
||
---
|
||
|
||
## 🚀 Installation & Setup
|
||
|
||
### Voraussetzungen prüfen
|
||
|
||
```bash
|
||
# System-Check ausführen
|
||
cat > /tmp/system_check.sh << 'EOF'
|
||
#!/bin/bash
|
||
echo "🔍 Crumbforest System-Check"
|
||
echo "=========================="
|
||
|
||
# Docker
|
||
if command -v docker &> /dev/null; then
|
||
echo "✅ Docker: $(docker --version)"
|
||
else
|
||
echo "❌ Docker nicht gefunden!"
|
||
exit 1
|
||
fi
|
||
|
||
# Docker Compose
|
||
if command -v docker compose &> /dev/null; then
|
||
echo "✅ Docker Compose: $(docker compose version)"
|
||
else
|
||
echo "❌ Docker Compose nicht gefunden!"
|
||
exit 1
|
||
fi
|
||
|
||
# Python (optional)
|
||
if command -v python3 &> /dev/null; then
|
||
echo "✅ Python: $(python3 --version)"
|
||
else
|
||
echo "⚠️ Python nicht gefunden (optional)"
|
||
fi
|
||
|
||
# Freier Speicher
|
||
free_space=$(df -h . | awk 'NR==2 {print $4}')
|
||
echo "💾 Freier Speicher: $free_space"
|
||
|
||
# Ports prüfen
|
||
if lsof -i :8000 &> /dev/null; then
|
||
echo "⚠️ Port 8000 bereits belegt!"
|
||
lsof -i :8000
|
||
else
|
||
echo "✅ Port 8000 frei"
|
||
fi
|
||
|
||
if lsof -i :6333 &> /dev/null; then
|
||
echo "⚠️ Port 6333 bereits belegt!"
|
||
else
|
||
echo "✅ Port 6333 frei"
|
||
fi
|
||
|
||
echo ""
|
||
echo "System-Check abgeschlossen!"
|
||
EOF
|
||
|
||
chmod +x /tmp/system_check.sh
|
||
/tmp/system_check.sh
|
||
```
|
||
|
||
### Minimale Installation
|
||
|
||
```bash
|
||
# 1. API Keys konfigurieren
|
||
cd compose
|
||
cp .env.example .env # Falls vorhanden
|
||
nano .env
|
||
|
||
# Minimum: Ein Provider
|
||
OPENROUTER_API_KEY=sk-or-v1-...
|
||
|
||
# 2. Starten
|
||
docker compose up -d
|
||
|
||
# 3. Warten auf Services
|
||
echo "Warte auf Database..."
|
||
until docker compose exec -T db sh -c 'mariadb -u$MARIADB_USER -p$MARIADB_PASSWORD $MARIADB_DATABASE -e "SELECT 1"' &> /dev/null; do
|
||
sleep 2
|
||
done
|
||
echo "✅ Database bereit"
|
||
|
||
echo "Warte auf App..."
|
||
until curl -s http://localhost:8000/health > /dev/null; do
|
||
sleep 2
|
||
done
|
||
echo "✅ App bereit"
|
||
|
||
# 4. Status prüfen
|
||
docker compose ps
|
||
```
|
||
|
||
---
|
||
|
||
## 🧪 Test-Protokolle
|
||
|
||
### Level 1: Bash/System Tests
|
||
|
||
```bash
|
||
#!/bin/bash
|
||
# test_level1_system.sh
|
||
|
||
echo "🧪 Level 1: System Tests"
|
||
echo "======================="
|
||
|
||
# Test 1: Health Check
|
||
echo -n "Test 1.1 - Health Endpoint... "
|
||
if curl -s http://localhost:8000/health | grep -q '"ok":true'; then
|
||
echo "✅"
|
||
else
|
||
echo "❌ FAILED"
|
||
exit 1
|
||
fi
|
||
|
||
# Test 2: Database Connection
|
||
echo -n "Test 1.2 - Database Connection... "
|
||
if docker compose -f compose/docker-compose.yml exec -T db sh -c \
|
||
'mariadb -u$MARIADB_USER -p$MARIADB_PASSWORD $MARIADB_DATABASE -e "SELECT 1"' &> /dev/null; then
|
||
echo "✅"
|
||
else
|
||
echo "❌ FAILED"
|
||
exit 1
|
||
fi
|
||
|
||
# Test 3: Qdrant
|
||
echo -n "Test 1.3 - Qdrant Connection... "
|
||
if curl -s http://localhost:6333/collections | grep -q '"ok"'; then
|
||
echo "✅"
|
||
else
|
||
echo "❌ FAILED"
|
||
exit 1
|
||
fi
|
||
|
||
# Test 4: File System
|
||
echo -n "Test 1.4 - Document Directories... "
|
||
if [ -d "docs/rz-nullfeld" ] && [ -d "docs/crumbforest" ]; then
|
||
echo "✅"
|
||
else
|
||
echo "❌ FAILED"
|
||
exit 1
|
||
fi
|
||
|
||
# Test 5: Docker Logs (keine Errors)
|
||
echo -n "Test 1.5 - Docker Logs Check... "
|
||
error_count=$(docker compose -f compose/docker-compose.yml logs app | grep -i error | grep -v "ERROR_FOR_DIVISION_BY_ZERO" | wc -l)
|
||
if [ "$error_count" -lt 5 ]; then
|
||
echo "✅ ($error_count Errors)"
|
||
else
|
||
echo "⚠️ ($error_count Errors - prüfe Logs)"
|
||
fi
|
||
|
||
echo ""
|
||
echo "Level 1 Tests abgeschlossen!"
|
||
```
|
||
|
||
### Level 2: cURL/API Tests
|
||
|
||
```bash
|
||
#!/bin/bash
|
||
# test_level2_api.sh
|
||
|
||
echo "🧪 Level 2: API Tests"
|
||
echo "===================="
|
||
|
||
BASE_URL="http://localhost:8000"
|
||
|
||
# Test 2.1: Routes auflisten
|
||
echo -n "Test 2.1 - List Routes... "
|
||
route_count=$(curl -s $BASE_URL/__routes | jq 'length')
|
||
if [ "$route_count" -gt 10 ]; then
|
||
echo "✅ ($route_count routes)"
|
||
else
|
||
echo "❌ FAILED"
|
||
exit 1
|
||
fi
|
||
|
||
# Test 2.2: Provider Status
|
||
echo -n "Test 2.2 - Provider Status... "
|
||
if curl -s "$BASE_URL/admin/rag/providers/status" | grep -q 'available'; then
|
||
echo "✅"
|
||
else
|
||
echo "❌ FAILED"
|
||
exit 1
|
||
fi
|
||
|
||
# Test 2.3: Document Collections
|
||
echo -n "Test 2.3 - Qdrant Collections... "
|
||
collections=$(curl -s http://localhost:6333/collections | jq '.result.collections | length')
|
||
if [ "$collections" -ge 2 ]; then
|
||
echo "✅ ($collections collections)"
|
||
else
|
||
echo "⚠️ ($collections collections - sollten mindestens 2 sein)"
|
||
fi
|
||
|
||
# Test 2.4: Document Search (ohne Auth)
|
||
echo -n "Test 2.4 - Document Search API... "
|
||
# Dieser Test erwartet 401, da Auth erforderlich
|
||
status=$(curl -s -o /dev/null -w "%{http_code}" "$BASE_URL/api/documents/search?q=test")
|
||
if [ "$status" = "401" ]; then
|
||
echo "✅ (Auth erforderlich - korrekt)"
|
||
else
|
||
echo "⚠️ Status: $status"
|
||
fi
|
||
|
||
# Test 2.5: API Hello
|
||
echo -n "Test 2.5 - API Hello... "
|
||
if curl -s "$BASE_URL/api/hello?lang=de" | grep -q 'Hallo Welt'; then
|
||
echo "✅"
|
||
else
|
||
echo "❌ FAILED"
|
||
exit 1
|
||
fi
|
||
|
||
echo ""
|
||
echo "Level 2 Tests abgeschlossen!"
|
||
```
|
||
|
||
### Level 3: Python Integration Tests
|
||
|
||
```python
|
||
#!/usr/bin/env python3
|
||
# test_level3_integration.py
|
||
|
||
"""
|
||
Level 3: Python Integration Tests
|
||
Testet die Python-Komponenten direkt
|
||
"""
|
||
|
||
import sys
|
||
import os
|
||
sys.path.insert(0, 'app')
|
||
|
||
def test_config():
|
||
"""Test 3.1: Config laden"""
|
||
print("Test 3.1 - Config Loading... ", end="")
|
||
try:
|
||
from config import get_settings
|
||
settings = get_settings()
|
||
assert settings.mariadb_database == "crumbcrm"
|
||
print("✅")
|
||
return True
|
||
except Exception as e:
|
||
print(f"❌ FAILED: {e}")
|
||
return False
|
||
|
||
def test_provider_factory():
|
||
"""Test 3.2: Provider Factory"""
|
||
print("Test 3.2 - Provider Factory... ", end="")
|
||
try:
|
||
from services.provider_factory import ProviderFactory
|
||
from config import get_settings
|
||
|
||
settings = get_settings()
|
||
available = ProviderFactory.get_available_providers(settings)
|
||
|
||
assert len(available) > 0, "Keine Provider verfügbar"
|
||
print(f"✅ ({len(available)} providers)")
|
||
return True
|
||
except Exception as e:
|
||
print(f"❌ FAILED: {e}")
|
||
return False
|
||
|
||
def test_embedding_service():
|
||
"""Test 3.3: Embedding Service"""
|
||
print("Test 3.3 - Embedding Service... ", end="")
|
||
try:
|
||
from services.embedding_service import EmbeddingService
|
||
from services.provider_factory import ProviderFactory
|
||
from config import get_settings
|
||
|
||
settings = get_settings()
|
||
providers = ProviderFactory.get_available_providers(settings)
|
||
|
||
if not providers:
|
||
print("⚠️ SKIP (keine Provider)")
|
||
return True
|
||
|
||
provider = ProviderFactory.create_provider(providers[0], settings)
|
||
service = EmbeddingService(provider)
|
||
|
||
# Test chunking
|
||
chunks = service.chunk_text("Test" * 500, chunk_size=100, overlap=20)
|
||
assert len(chunks) > 1, "Chunking failed"
|
||
|
||
print(f"✅ ({len(chunks)} chunks)")
|
||
return True
|
||
except Exception as e:
|
||
print(f"❌ FAILED: {e}")
|
||
return False
|
||
|
||
def test_database_schema():
|
||
"""Test 3.4: Database Schema"""
|
||
print("Test 3.4 - Database Schema... ", end="")
|
||
try:
|
||
from deps import get_db
|
||
from pymysql.cursors import DictCursor
|
||
|
||
conn = get_db()
|
||
with conn.cursor(DictCursor) as cur:
|
||
# Prüfe wichtige Tabellen
|
||
tables = ['users', 'posts', 'post_vectors', 'audit_log']
|
||
cur.execute("SHOW TABLES")
|
||
existing = [row['Tables_in_crumbcrm'] for row in cur.fetchall()]
|
||
|
||
for table in tables:
|
||
assert table in existing, f"Tabelle {table} fehlt"
|
||
|
||
conn.close()
|
||
print(f"✅ ({len(tables)} tables)")
|
||
return True
|
||
except Exception as e:
|
||
print(f"❌ FAILED: {e}")
|
||
return False
|
||
|
||
def main():
|
||
print("🧪 Level 3: Python Integration Tests")
|
||
print("====================================")
|
||
|
||
tests = [
|
||
test_config,
|
||
test_provider_factory,
|
||
test_embedding_service,
|
||
test_database_schema,
|
||
]
|
||
|
||
passed = sum(1 for test in tests if test())
|
||
total = len(tests)
|
||
|
||
print()
|
||
print(f"Level 3 Tests: {passed}/{total} bestanden")
|
||
|
||
return 0 if passed == total else 1
|
||
|
||
if __name__ == "__main__":
|
||
sys.exit(main())
|
||
```
|
||
|
||
### Level 4: PHP Integration Tests
|
||
|
||
```php
|
||
<?php
|
||
// test_level4_php.php
|
||
|
||
/**
|
||
* Level 4: PHP Integration Tests
|
||
* Testet die PHP-Komponenten und FastAPI Integration
|
||
*/
|
||
|
||
echo "🧪 Level 4: PHP Integration Tests\n";
|
||
echo "==================================\n";
|
||
|
||
$base_url = "http://localhost:8000";
|
||
$tests_passed = 0;
|
||
$tests_total = 0;
|
||
|
||
function test_api_call($name, $url, $expected_status = 200) {
|
||
global $tests_passed, $tests_total;
|
||
$tests_total++;
|
||
|
||
echo "Test 4.$tests_total - $name... ";
|
||
|
||
$ch = curl_init($url);
|
||
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
|
||
curl_setopt($ch, CURLOPT_HEADER, true);
|
||
$response = curl_exec($ch);
|
||
$status = curl_getinfo($ch, CURLINFO_HTTP_CODE);
|
||
curl_close($ch);
|
||
|
||
if ($status == $expected_status) {
|
||
echo "✅\n";
|
||
$tests_passed++;
|
||
return true;
|
||
} else {
|
||
echo "❌ FAILED (Status: $status)\n";
|
||
return false;
|
||
}
|
||
}
|
||
|
||
// Test 4.1: Health Check
|
||
test_api_call("Health Check", "$base_url/health");
|
||
|
||
// Test 4.2: API Hello
|
||
test_api_call("API Hello", "$base_url/api/hello?lang=de");
|
||
|
||
// Test 4.3: Routes List
|
||
test_api_call("Routes List", "$base_url/__routes");
|
||
|
||
// Test 4.4: Protected Endpoint (sollte 401 sein)
|
||
test_api_call("Protected Endpoint", "$base_url/admin/rag/providers", 401);
|
||
|
||
// Test 4.5: JSON Response Format
|
||
echo "Test 4.5 - JSON Response Format... ";
|
||
$tests_total++;
|
||
$ch = curl_init("$base_url/api/hello");
|
||
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
|
||
$response = curl_exec($ch);
|
||
curl_close($ch);
|
||
|
||
$json = json_decode($response, true);
|
||
if ($json && isset($json['message'])) {
|
||
echo "✅\n";
|
||
$tests_passed++;
|
||
} else {
|
||
echo "❌ FAILED\n";
|
||
}
|
||
|
||
echo "\n";
|
||
echo "Level 4 Tests: $tests_passed/$tests_total bestanden\n";
|
||
|
||
exit($tests_passed == $tests_total ? 0 : 1);
|
||
?>
|
||
```
|
||
|
||
### Test-Suite ausführen
|
||
|
||
```bash
|
||
#!/bin/bash
|
||
# run_all_tests.sh
|
||
|
||
echo "🦉 Crumbforest Test-Suite"
|
||
echo "========================="
|
||
echo ""
|
||
|
||
# Level 1: System
|
||
bash test_level1_system.sh
|
||
level1=$?
|
||
|
||
# Level 2: API
|
||
bash test_level2_api.sh
|
||
level2=$?
|
||
|
||
# Level 3: Python
|
||
python3 test_level3_integration.py
|
||
level3=$?
|
||
|
||
# Level 4: PHP (falls vorhanden)
|
||
if command -v php &> /dev/null; then
|
||
php test_level4_php.php
|
||
level4=$?
|
||
else
|
||
echo "⚠️ PHP nicht installiert - Level 4 übersprungen"
|
||
level4=0
|
||
fi
|
||
|
||
# Zusammenfassung
|
||
echo ""
|
||
echo "📊 Test-Zusammenfassung"
|
||
echo "======================="
|
||
echo "Level 1 (System): $([ $level1 -eq 0 ] && echo '✅ PASS' || echo '❌ FAIL')"
|
||
echo "Level 2 (API): $([ $level2 -eq 0 ] && echo '✅ PASS' || echo '❌ FAIL')"
|
||
echo "Level 3 (Python): $([ $level3 -eq 0 ] && echo '✅ PASS' || echo '❌ FAIL')"
|
||
echo "Level 4 (PHP): $([ $level4 -eq 0 ] && echo '✅ PASS' || echo '❌ FAIL')"
|
||
|
||
# Exit Code
|
||
if [ $level1 -eq 0 ] && [ $level2 -eq 0 ] && [ $level3 -eq 0 ] && [ $level4 -eq 0 ]; then
|
||
echo ""
|
||
echo "🎉 Alle Tests bestanden! Wuuuuhuuu!"
|
||
exit 0
|
||
else
|
||
echo ""
|
||
echo "⚠️ Einige Tests fehlgeschlagen. Prüfe die Logs."
|
||
exit 1
|
||
fi
|
||
```
|
||
|
||
---
|
||
|
||
## 🐛 Allgemeine Fehler & Lösungen
|
||
|
||
### Fehler 1: Port bereits belegt
|
||
|
||
**Symptom:**
|
||
```
|
||
Error: bind: address already in use
|
||
```
|
||
|
||
**Diagnose:**
|
||
```bash
|
||
# Prüfe welcher Prozess Port 8000 nutzt
|
||
lsof -i :8000
|
||
# oder
|
||
netstat -an | grep 8000
|
||
```
|
||
|
||
**Lösung 1 - Prozess beenden:**
|
||
```bash
|
||
# Finde PID
|
||
lsof -i :8000 | grep LISTEN | awk '{print $2}'
|
||
|
||
# Beende Prozess
|
||
kill -9 <PID>
|
||
```
|
||
|
||
**Lösung 2 - Port ändern:**
|
||
```bash
|
||
# In docker-compose.yml
|
||
ports:
|
||
- "8001:8000" # Nutze 8001 statt 8000
|
||
```
|
||
|
||
### Fehler 2: Database Connection Failed
|
||
|
||
**Symptom:**
|
||
```
|
||
sqlalchemy.exc.OperationalError: (2003, "Can't connect to MySQL server")
|
||
```
|
||
|
||
**Diagnose:**
|
||
```bash
|
||
# Prüfe DB Container Status
|
||
docker compose ps db
|
||
|
||
# Prüfe DB Logs
|
||
docker compose logs db | tail -50
|
||
|
||
# Teste DB Verbindung
|
||
docker compose exec -T db sh -c \
|
||
'mariadb -u$MARIADB_USER -p$MARIADB_PASSWORD -e "SELECT 1"'
|
||
```
|
||
|
||
**Lösung 1 - Warte länger:**
|
||
```bash
|
||
# DB braucht Zeit zum Starten (15-30 Sekunden)
|
||
sleep 30
|
||
|
||
# Health Check bis DB bereit
|
||
until docker compose exec -T db sh -c 'mariadb -u$MARIADB_USER -p$MARIADB_PASSWORD $MARIADB_DATABASE -e "SELECT 1"' &> /dev/null; do
|
||
echo "Warte auf DB..."
|
||
sleep 2
|
||
done
|
||
```
|
||
|
||
**Lösung 2 - DB neu initialisieren:**
|
||
```bash
|
||
# ACHTUNG: Löscht ALLE Daten!
|
||
docker compose down -v
|
||
docker compose up -d
|
||
```
|
||
|
||
### Fehler 3: API Key Invalid
|
||
|
||
**Symptom:**
|
||
```
|
||
Error code: 401 - Incorrect API key provided
|
||
```
|
||
|
||
**Diagnose:**
|
||
```bash
|
||
# Prüfe Environment Variables im Container
|
||
docker compose exec app env | grep API_KEY
|
||
|
||
# Prüfe .env Datei
|
||
cat compose/.env | grep API_KEY
|
||
```
|
||
|
||
**Lösung:**
|
||
```bash
|
||
# 1. Korrekten Key in .env eintragen
|
||
nano compose/.env
|
||
|
||
# 2. Container neu starten (um .env zu laden)
|
||
docker compose restart app
|
||
|
||
# 3. Verifiziere mit Provider Status
|
||
curl http://localhost:8000/admin/rag/providers/status
|
||
```
|
||
|
||
### Fehler 4: Indexing schlägt fehl
|
||
|
||
**Symptom:**
|
||
```
|
||
Error indexing docs/...: OpenRouter embedding API call failed
|
||
```
|
||
|
||
**Diagnose:**
|
||
```bash
|
||
# Logs prüfen
|
||
docker compose logs app | grep -i error
|
||
|
||
# Provider Status prüfen
|
||
curl http://localhost:8000/admin/rag/providers/status | jq
|
||
|
||
# Qdrant Collections prüfen
|
||
curl http://localhost:6333/collections | jq
|
||
```
|
||
|
||
**Lösung 1 - API Credits prüfen:**
|
||
```bash
|
||
# OpenRouter Dashboard: https://openrouter.ai/credits
|
||
# Stelle sicher, dass Guthaben vorhanden ist
|
||
```
|
||
|
||
**Lösung 2 - Manuell re-indexieren:**
|
||
```bash
|
||
curl -X POST "http://localhost:8000/api/documents/index" \
|
||
-H "Content-Type: application/json" \
|
||
-d '{"provider": "openrouter", "force": true}'
|
||
```
|
||
|
||
**Lösung 3 - Anderen Provider nutzen:**
|
||
```bash
|
||
# In compose/.env
|
||
DEFAULT_EMBEDDING_PROVIDER=openai # oder claude
|
||
|
||
# Container neu starten
|
||
docker compose restart app
|
||
```
|
||
|
||
### Fehler 5: Qdrant Point ID Error
|
||
|
||
**Symptom:**
|
||
```
|
||
value 123456_0 is not a valid point ID
|
||
```
|
||
|
||
**Diagnose:**
|
||
```bash
|
||
# Prüfe Qdrant Version
|
||
curl http://localhost:6333/collections | jq
|
||
|
||
# Prüfe rag_service.py
|
||
grep "point_id =" app/services/rag_service.py
|
||
```
|
||
|
||
**Lösung:**
|
||
```bash
|
||
# UUID-basierte Point IDs sind implementiert
|
||
# Falls Fehler auftritt: Container neu bauen
|
||
docker compose up --build -d
|
||
```
|
||
|
||
### Fehler 6: Dateinamen mit Sonderzeichen
|
||
|
||
**Symptom:**
|
||
```
|
||
UnicodeEncodeError: 'ascii' codec can't encode character
|
||
```
|
||
|
||
**Diagnose:**
|
||
```bash
|
||
# Prüfe Dateinamen
|
||
find docs/ -name "*[^a-zA-Z0-9._-]*" -type f
|
||
|
||
# Prüfe UTF-8 Encoding
|
||
file docs/crumbforest/*.md | grep -v UTF-8
|
||
```
|
||
|
||
**Lösung:**
|
||
```bash
|
||
# System unterstützt UTF-8, aber falls Probleme:
|
||
# Dateinamen säubern (optional)
|
||
for file in docs/crumbforest/*; do
|
||
newname=$(echo "$file" | iconv -f UTF-8 -t ASCII//TRANSLIT)
|
||
if [ "$file" != "$newname" ]; then
|
||
mv "$file" "$newname"
|
||
fi
|
||
done
|
||
```
|
||
|
||
### Fehler 7: Docker Out of Memory
|
||
|
||
**Symptom:**
|
||
```
|
||
docker: Error response from daemon: OCI runtime create failed
|
||
```
|
||
|
||
**Diagnose:**
|
||
```bash
|
||
# Docker Memory Limit prüfen
|
||
docker info | grep -i memory
|
||
|
||
# Container Resources
|
||
docker stats
|
||
```
|
||
|
||
**Lösung:**
|
||
```bash
|
||
# In docker-compose.yml Memory Limits erhöhen
|
||
services:
|
||
app:
|
||
deploy:
|
||
resources:
|
||
limits:
|
||
memory: 2G
|
||
reservations:
|
||
memory: 1G
|
||
```
|
||
|
||
### Fehler 8: Session nicht persistent
|
||
|
||
**Symptom:**
|
||
```
|
||
Login klappt, aber bei Reload ausgeloggt
|
||
```
|
||
|
||
**Diagnose:**
|
||
```bash
|
||
# Prüfe SECRET_KEY
|
||
docker compose exec app env | grep SECRET_KEY
|
||
|
||
# Prüfe Session Middleware
|
||
grep SessionMiddleware app/main.py
|
||
```
|
||
|
||
**Lösung:**
|
||
```bash
|
||
# Festen SECRET_KEY in .env setzen
|
||
echo "SECRET_KEY=$(openssl rand -hex 32)" >> compose/.env
|
||
|
||
# Container neu starten
|
||
docker compose restart app
|
||
```
|
||
|
||
### Fehler 9: "Unsupported embedding model" trotz richtiger Config
|
||
|
||
**Symptom:**
|
||
```
|
||
{"detail":"Unsupported embedding model: openai/text-embedding-3-small.
|
||
Supported models: ['text-embedding-3-small', 'text-embedding-3-large', 'text-embedding-ada-002']"}
|
||
```
|
||
|
||
**Ursachen:**
|
||
1. **Falscher Provider in .env** - `DEFAULT_EMBEDDING_PROVIDER=openai` statt `openrouter`
|
||
2. **Code-Änderungen nicht aktiv** - Kein Volume Mount, Image-Rebuild nötig!
|
||
3. **Collection-Namen falsch** - Mit/ohne trailing underscore
|
||
|
||
**Diagnose:**
|
||
```bash
|
||
# 1. Prüfe .env Provider-Einstellungen
|
||
grep "DEFAULT_EMBEDDING_PROVIDER" compose/.env
|
||
|
||
# 2. Prüfe ob Code im Container aktuell ist
|
||
docker compose exec app grep -n "DEBUG" /app/routers/document_rag.py
|
||
|
||
# 3. Prüfe File-Timestamp im Container
|
||
docker compose exec app ls -la /app/routers/document_rag.py
|
||
|
||
# 4. Vergleiche mit Host-Datei
|
||
ls -la app/routers/document_rag.py
|
||
```
|
||
|
||
**Lösung:**
|
||
```bash
|
||
# 1. Korrigiere .env (KRITISCH!)
|
||
nano compose/.env
|
||
|
||
# Setze:
|
||
DEFAULT_EMBEDDING_PROVIDER=openrouter
|
||
DEFAULT_EMBEDDING_MODEL=text-embedding-3-small
|
||
DEFAULT_COMPLETION_PROVIDER=openrouter
|
||
DEFAULT_COMPLETION_MODEL=anthropic/claude-3-5-sonnet
|
||
|
||
# 2. Rebuild Image (KEINE Volume Mounts!)
|
||
cd compose
|
||
docker compose down
|
||
docker compose up --build -d
|
||
|
||
# 3. Verifiziere
|
||
docker compose logs app | grep "provider"
|
||
# Erwartete Ausgabe: "✓ Using provider: openrouter"
|
||
|
||
# 4. Teste Search
|
||
curl "http://localhost:8000/api/documents/search?q=Docker&limit=3"
|
||
```
|
||
|
||
**Wichtige Erkenntnis:**
|
||
```
|
||
⚠️ Es gibt KEIN Volume Mount für App-Code!
|
||
Der Code ist im Docker Image eingebacken.
|
||
|
||
JEDE Code-Änderung erfordert:
|
||
docker compose up --build -d
|
||
|
||
NUR .env Änderungen:
|
||
docker compose restart app
|
||
```
|
||
|
||
---
|
||
|
||
## 🔒 DSGVO & Datenschutz
|
||
|
||
### Compliance-Checkliste
|
||
|
||
#### ✅ Implementiert
|
||
|
||
- [x] **Datentrennung**: Jedes Kind hat eigene Qdrant-Collection
|
||
- [x] **Audit-Logging**: Unveränderbare Logs in `audit_log` Tabelle
|
||
- [x] **File-Hash-Tracking**: Keine Duplikate, Change Detection
|
||
- [x] **Role-Based Access**: Admin/User Trennung
|
||
- [x] **Session-Based Auth**: Keine Token-Leaks
|
||
- [x] **Metadata-Tracking**: Wer, Was, Wann in JSON
|
||
|
||
#### 📋 Datenfluss-Dokumentation
|
||
|
||
```
|
||
┌─────────────────────────────────────────────────────┐
|
||
│ DSGVO Datenfluss-Diagramm │
|
||
└─────────────────────────────────────────────────────┘
|
||
|
||
1. Kind sendet Tagebuch-Eintrag
|
||
↓
|
||
2. PHP Backend → FastAPI (REST)
|
||
↓
|
||
3. FastAPI validiert Token
|
||
↓
|
||
4. Content → Chunking (max 1000 Zeichen)
|
||
↓
|
||
5. Chunks → Embedding API (OpenRouter/OpenAI)
|
||
↓
|
||
6. Embeddings → Qdrant (diary_child_{id})
|
||
│
|
||
└──→ audit_log (MariaDB)
|
||
- action: "diary_indexed"
|
||
- entity_id: entry_id
|
||
- metadata: {timestamp, provider, chunks}
|
||
```
|
||
|
||
#### 🗄️ Audit-Log Schema
|
||
|
||
```sql
|
||
CREATE TABLE audit_log (
|
||
id BIGINT AUTO_INCREMENT PRIMARY KEY,
|
||
action VARCHAR(100) NOT NULL, -- z.B. "diary_indexed"
|
||
entity_type VARCHAR(50) NOT NULL, -- z.B. "diary", "document"
|
||
entity_id INT NOT NULL, -- z.B. entry_id
|
||
user_id INT NULL, -- NULL für System-Actions
|
||
metadata JSON NULL, -- Flexibles Tracking
|
||
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
|
||
INDEX idx_action (action),
|
||
INDEX idx_entity (entity_type, entity_id),
|
||
INDEX idx_created (created_at)
|
||
);
|
||
```
|
||
|
||
#### 🔍 Audit-Log Queries
|
||
|
||
```bash
|
||
# Alle Aktionen für ein Kind
|
||
docker compose exec -T db sh -c 'mariadb -u$MARIADB_USER -p$MARIADB_PASSWORD $MARIADB_DATABASE -e "
|
||
SELECT action, entity_id, created_at, metadata
|
||
FROM audit_log
|
||
WHERE entity_type=\"diary\"
|
||
AND JSON_EXTRACT(metadata, \"$.child_id\") = 1
|
||
ORDER BY created_at DESC
|
||
LIMIT 10;
|
||
"'
|
||
|
||
# Alle Indexierungs-Vorgänge heute
|
||
docker compose exec -T db sh -c 'mariadb -u$MARIADB_USER -p$MARIADB_PASSWORD $MARIADB_DATABASE -e "
|
||
SELECT COUNT(*) as count, action
|
||
FROM audit_log
|
||
WHERE DATE(created_at) = CURDATE()
|
||
GROUP BY action;
|
||
"'
|
||
|
||
# Metadaten-Analyse
|
||
docker compose exec -T db sh -c 'mariadb -u$MARIADB_USER -p$MARIADB_PASSWORD $MARIADB_DATABASE -e "
|
||
SELECT
|
||
JSON_EXTRACT(metadata, \"$.provider\") as provider,
|
||
COUNT(*) as count
|
||
FROM audit_log
|
||
WHERE action LIKE \"%indexed\"
|
||
GROUP BY provider;
|
||
"'
|
||
```
|
||
|
||
### Datenlöschung (Recht auf Vergessen)
|
||
|
||
```bash
|
||
#!/bin/bash
|
||
# delete_child_data.sh - DSGVO Löschung
|
||
|
||
CHILD_ID=$1
|
||
|
||
if [ -z "$CHILD_ID" ]; then
|
||
echo "Usage: $0 <child_id>"
|
||
exit 1
|
||
fi
|
||
|
||
echo "🗑️ DSGVO Datenlöschung für Kind $CHILD_ID"
|
||
echo "=========================================="
|
||
|
||
# 1. Qdrant Collection löschen
|
||
echo "1. Lösche Qdrant Collection..."
|
||
curl -X DELETE "http://localhost:6333/collections/diary_child_${CHILD_ID}"
|
||
|
||
# 2. MariaDB Einträge löschen
|
||
echo "2. Lösche Datenbank-Einträge..."
|
||
docker compose exec -T db sh -c "mariadb -u\$MARIADB_USER -p\$MARIADB_PASSWORD \$MARIADB_DATABASE <<EOF
|
||
-- Lösche Vektoren
|
||
DELETE FROM post_vectors WHERE child_id = $CHILD_ID;
|
||
|
||
-- Lösche Tagebuch-Einträge
|
||
DELETE FROM diary_entries WHERE child_id = $CHILD_ID;
|
||
|
||
-- Kind-Record löschen
|
||
DELETE FROM children WHERE id = $CHILD_ID;
|
||
|
||
-- Audit-Log NICHT löschen (DSGVO-Pflicht: Nachweisbarkeit)
|
||
INSERT INTO audit_log (action, entity_type, entity_id, metadata)
|
||
VALUES ('child_deleted', 'child', $CHILD_ID, JSON_OBJECT('timestamp', NOW()));
|
||
EOF"
|
||
|
||
echo ""
|
||
echo "✅ Datenlöschung abgeschlossen"
|
||
echo "📝 Audit-Log Eintrag erstellt (DSGVO-Nachweis)"
|
||
```
|
||
|
||
### Datenexport (Recht auf Datenportabilität)
|
||
|
||
```bash
|
||
#!/bin/bash
|
||
# export_child_data.sh - DSGVO Export
|
||
|
||
CHILD_ID=$1
|
||
OUTPUT_DIR="exports/child_${CHILD_ID}"
|
||
|
||
mkdir -p "$OUTPUT_DIR"
|
||
|
||
echo "📦 DSGVO Datenexport für Kind $CHILD_ID"
|
||
echo "========================================"
|
||
|
||
# 1. Tagebuch-Einträge
|
||
echo "1. Exportiere Tagebuch-Einträge..."
|
||
docker compose exec -T db sh -c "mariadb -u\$MARIADB_USER -p\$MARIADB_PASSWORD \$MARIADB_DATABASE -e \"
|
||
SELECT * FROM diary_entries WHERE child_id = $CHILD_ID
|
||
\" --xml" > "$OUTPUT_DIR/diary_entries.xml"
|
||
|
||
# 2. Metadaten
|
||
echo "2. Exportiere Metadaten..."
|
||
docker compose exec -T db sh -c "mariadb -u\$MARIADB_USER -p\$MARIADB_PASSWORD \$MARIADB_DATABASE -e \"
|
||
SELECT * FROM children WHERE id = $CHILD_ID
|
||
\" --xml" > "$OUTPUT_DIR/child_info.xml"
|
||
|
||
# 3. Audit-Log
|
||
echo "3. Exportiere Audit-Log..."
|
||
docker compose exec -T db sh -c "mariadb -u\$MARIADB_USER -p\$MARIADB_PASSWORD \$MARIADB_DATABASE -e \"
|
||
SELECT * FROM audit_log
|
||
WHERE entity_type='diary'
|
||
AND JSON_EXTRACT(metadata, '$.child_id') = $CHILD_ID
|
||
\" --xml" > "$OUTPUT_DIR/audit_log.xml"
|
||
|
||
# 4. Qdrant Vektoren (optional)
|
||
echo "4. Exportiere Vektoren-Metadaten..."
|
||
curl -s "http://localhost:6333/collections/diary_child_${CHILD_ID}/points?limit=1000" \
|
||
> "$OUTPUT_DIR/vectors.json"
|
||
|
||
# 5. ZIP erstellen
|
||
echo "5. Erstelle ZIP-Archiv..."
|
||
cd exports
|
||
zip -r "child_${CHILD_ID}_export_$(date +%Y%m%d).zip" "child_${CHILD_ID}/"
|
||
cd ..
|
||
|
||
echo ""
|
||
echo "✅ Export abgeschlossen"
|
||
echo "📁 Datei: exports/child_${CHILD_ID}_export_$(date +%Y%m%d).zip"
|
||
```
|
||
|
||
---
|
||
|
||
## 🔧 Wartung & Monitoring
|
||
|
||
### Tägliche Checks
|
||
|
||
```bash
|
||
#!/bin/bash
|
||
# daily_health_check.sh
|
||
|
||
echo "🏥 Crumbforest Täglicher Health-Check"
|
||
echo "====================================="
|
||
echo "Datum: $(date)"
|
||
echo ""
|
||
|
||
# 1. Container Status
|
||
echo "📦 Container Status:"
|
||
docker compose ps
|
||
|
||
# 2. Disk Space
|
||
echo ""
|
||
echo "💾 Speicherplatz:"
|
||
df -h | grep -E "Filesystem|/var/lib/docker"
|
||
|
||
# 3. Database Size
|
||
echo ""
|
||
echo "🗄️ Database Größe:"
|
||
docker compose exec -T db sh -c 'mariadb -u$MARIADB_USER -p$MARIADB_PASSWORD -e "
|
||
SELECT
|
||
table_schema AS \"Database\",
|
||
ROUND(SUM(data_length + index_length) / 1024 / 1024, 2) AS \"Size (MB)\"
|
||
FROM information_schema.tables
|
||
WHERE table_schema = \"crumbcrm\"
|
||
GROUP BY table_schema;
|
||
"'
|
||
|
||
# 4. Qdrant Stats
|
||
echo ""
|
||
echo "📊 Qdrant Collections:"
|
||
curl -s http://localhost:6333/collections | jq '.result.collections[] | {name, points: .points_count}'
|
||
|
||
# 5. Recent Errors
|
||
echo ""
|
||
echo "⚠️ Letzte Errors (24h):"
|
||
docker compose logs --since 24h app 2>&1 | grep -i error | tail -10
|
||
|
||
# 6. API Response Time
|
||
echo ""
|
||
echo "⏱️ API Response Time:"
|
||
time curl -s http://localhost:8000/health > /dev/null
|
||
|
||
echo ""
|
||
echo "✅ Health-Check abgeschlossen"
|
||
```
|
||
|
||
### Wöchentliche Maintenance
|
||
|
||
```bash
|
||
#!/bin/bash
|
||
# weekly_maintenance.sh
|
||
|
||
echo "🛠️ Crumbforest Wöchentliche Wartung"
|
||
echo "===================================="
|
||
|
||
# 1. Docker Cleanup
|
||
echo "1. Docker Cleanup..."
|
||
docker system prune -f
|
||
|
||
# 2. Database Optimize
|
||
echo "2. Database Optimierung..."
|
||
docker compose exec -T db sh -c 'mariadb -u$MARIADB_USER -p$MARIADB_PASSWORD $MARIADB_DATABASE -e "
|
||
OPTIMIZE TABLE posts;
|
||
OPTIMIZE TABLE post_vectors;
|
||
OPTIMIZE TABLE audit_log;
|
||
"'
|
||
|
||
# 3. Qdrant Optimization
|
||
echo "3. Qdrant Collections optimieren..."
|
||
for collection in $(curl -s http://localhost:6333/collections | jq -r '.result.collections[].name'); do
|
||
echo " - $collection"
|
||
curl -s -X POST "http://localhost:6333/collections/$collection/optimizer" > /dev/null
|
||
done
|
||
|
||
# 4. Logs rotieren
|
||
echo "4. Logs rotieren..."
|
||
docker compose logs --tail=10000 > "logs/crumbforest_$(date +%Y%m%d).log"
|
||
|
||
# 5. Backup erstellen
|
||
echo "5. Backup erstellen..."
|
||
./backup.sh
|
||
|
||
echo ""
|
||
echo "✅ Wartung abgeschlossen"
|
||
```
|
||
|
||
---
|
||
|
||
## 💾 Backup & Recovery
|
||
|
||
### Vollständiges Backup
|
||
|
||
```bash
|
||
#!/bin/bash
|
||
# backup.sh - Vollständiges System-Backup
|
||
|
||
BACKUP_DIR="backups/$(date +%Y%m%d_%H%M%S)"
|
||
mkdir -p "$BACKUP_DIR"
|
||
|
||
echo "💾 Crumbforest Backup"
|
||
echo "===================="
|
||
echo "Ziel: $BACKUP_DIR"
|
||
echo ""
|
||
|
||
# 1. Database Dump
|
||
echo "1. Database Backup..."
|
||
docker compose exec -T db sh -c 'mysqldump -u$MARIADB_USER -p$MARIADB_PASSWORD $MARIADB_DATABASE' \
|
||
| gzip > "$BACKUP_DIR/database.sql.gz"
|
||
|
||
# 2. Qdrant Snapshot
|
||
echo "2. Qdrant Snapshot..."
|
||
curl -X POST "http://localhost:6333/collections/docs_crumbforest_/snapshots" \
|
||
-H "Content-Type: application/json" > "$BACKUP_DIR/qdrant_snapshot.json"
|
||
|
||
# 3. Config Files
|
||
echo "3. Config Backup..."
|
||
cp compose/.env "$BACKUP_DIR/.env.backup"
|
||
cp compose/docker-compose.yml "$BACKUP_DIR/docker-compose.yml"
|
||
|
||
# 4. Dokumente
|
||
echo "4. Dokumente Backup..."
|
||
tar -czf "$BACKUP_DIR/docs.tar.gz" docs/
|
||
|
||
# 5. Metadata
|
||
echo "5. Backup-Metadata..."
|
||
cat > "$BACKUP_DIR/backup_info.json" << EOF
|
||
{
|
||
"timestamp": "$(date -Iseconds)",
|
||
"hostname": "$(hostname)",
|
||
"docker_version": "$(docker --version)",
|
||
"compose_version": "$(docker compose version)",
|
||
"collections": $(curl -s http://localhost:6333/collections | jq '.result.collections')
|
||
}
|
||
EOF
|
||
|
||
# 6. Checksums
|
||
echo "6. Checksums erstellen..."
|
||
cd "$BACKUP_DIR"
|
||
sha256sum * > checksums.sha256
|
||
cd - > /dev/null
|
||
|
||
echo ""
|
||
echo "✅ Backup abgeschlossen"
|
||
echo "📁 Verzeichnis: $BACKUP_DIR"
|
||
echo "📊 Größe: $(du -sh $BACKUP_DIR | cut -f1)"
|
||
```
|
||
|
||
### Restore
|
||
|
||
```bash
|
||
#!/bin/bash
|
||
# restore.sh - System wiederherstellen
|
||
|
||
BACKUP_DIR=$1
|
||
|
||
if [ -z "$BACKUP_DIR" ]; then
|
||
echo "Usage: $0 <backup_directory>"
|
||
echo "Verfügbare Backups:"
|
||
ls -la backups/
|
||
exit 1
|
||
fi
|
||
|
||
echo "♻️ Crumbforest Restore"
|
||
echo "====================="
|
||
echo "Quelle: $BACKUP_DIR"
|
||
echo ""
|
||
|
||
read -p "ACHTUNG: Dies überschreibt ALLE Daten! Fortfahren? (yes/no) " confirm
|
||
if [ "$confirm" != "yes" ]; then
|
||
echo "Abgebrochen."
|
||
exit 0
|
||
fi
|
||
|
||
# 1. Container stoppen
|
||
echo "1. Stoppe Container..."
|
||
docker compose down
|
||
|
||
# 2. Database wiederherstellen
|
||
echo "2. Database Restore..."
|
||
docker compose up -d db
|
||
sleep 10
|
||
gunzip < "$BACKUP_DIR/database.sql.gz" | \
|
||
docker compose exec -T db sh -c 'mariadb -u$MARIADB_USER -p$MARIADB_PASSWORD $MARIADB_DATABASE'
|
||
|
||
# 3. Qdrant Daten löschen
|
||
echo "3. Qdrant Reset..."
|
||
rm -rf data/qdrant/*
|
||
|
||
# 4. Dokumente wiederherstellen
|
||
echo "4. Dokumente Restore..."
|
||
tar -xzf "$BACKUP_DIR/docs.tar.gz"
|
||
|
||
# 5. Config wiederherstellen
|
||
echo "5. Config Restore..."
|
||
cp "$BACKUP_DIR/.env.backup" compose/.env
|
||
|
||
# 6. System starten
|
||
echo "6. Starte System..."
|
||
docker compose up -d
|
||
|
||
# 7. Re-Indexing
|
||
echo "7. Warte auf System..."
|
||
sleep 30
|
||
echo "8. Triggere Re-Indexing..."
|
||
curl -X POST "http://localhost:8000/api/documents/index" \
|
||
-H "Content-Type: application/json" \
|
||
-d '{"provider": "openrouter", "force": true}'
|
||
|
||
echo ""
|
||
echo "✅ Restore abgeschlossen"
|
||
echo "⚠️ Prüfe System mit: ./test_all.sh"
|
||
```
|
||
|
||
---
|
||
|
||
## ⚡ Performance-Tuning
|
||
|
||
### Qdrant Optimierung
|
||
|
||
```bash
|
||
# Qdrant Memory Mapping aktivieren
|
||
curl -X PATCH http://localhost:6333/collections/docs_crumbforest_ \
|
||
-H "Content-Type: application/json" \
|
||
-d '{
|
||
"optimizers_config": {
|
||
"memmap_threshold": 20000
|
||
}
|
||
}'
|
||
|
||
# Indexing optimieren
|
||
curl -X PATCH http://localhost:6333/collections/docs_crumbforest_ \
|
||
-H "Content-Type: application/json" \
|
||
-d '{
|
||
"hnsw_config": {
|
||
"m": 16,
|
||
"ef_construct": 100
|
||
}
|
||
}'
|
||
```
|
||
|
||
### Database Optimierung
|
||
|
||
```sql
|
||
-- Indexes für häufige Queries
|
||
ALTER TABLE post_vectors ADD INDEX idx_collection_type (collection_name, post_type);
|
||
ALTER TABLE audit_log ADD INDEX idx_created_action (created_at, action);
|
||
|
||
-- Query Cache aktivieren
|
||
SET GLOBAL query_cache_size = 67108864; -- 64 MB
|
||
SET GLOBAL query_cache_type = 1;
|
||
```
|
||
|
||
### Docker Resources
|
||
|
||
```yaml
|
||
# docker-compose.yml
|
||
services:
|
||
app:
|
||
deploy:
|
||
resources:
|
||
limits:
|
||
cpus: '2.0'
|
||
memory: 2G
|
||
reservations:
|
||
cpus: '1.0'
|
||
memory: 1G
|
||
|
||
db:
|
||
deploy:
|
||
resources:
|
||
limits:
|
||
memory: 1G
|
||
reservations:
|
||
memory: 512M
|
||
```
|
||
|
||
---
|
||
|
||
## 👨💻 Entwickler-Guide
|
||
|
||
### Development Setup
|
||
|
||
```bash
|
||
# 1. Python Virtual Environment
|
||
python3 -m venv venv
|
||
source venv/bin/activate
|
||
|
||
# 2. Dependencies installieren
|
||
pip install -r app/requirements.txt
|
||
|
||
# 3. Pre-commit Hooks (optional)
|
||
pip install pre-commit
|
||
pre-commit install
|
||
|
||
# 4. Development Server (ohne Docker)
|
||
cd app
|
||
uvicorn main:app --reload --host 0.0.0.0 --port 8000
|
||
```
|
||
|
||
### Code-Style
|
||
|
||
```bash
|
||
# Black Formatter
|
||
black app/ --line-length 100
|
||
|
||
# Flake8 Linter
|
||
flake8 app/ --max-line-length 100
|
||
|
||
# Type Checking
|
||
mypy app/ --ignore-missing-imports
|
||
```
|
||
|
||
### Neue Features hinzufügen
|
||
|
||
```python
|
||
# 1. Neuen Router erstellen
|
||
# app/routers/my_feature.py
|
||
|
||
from fastapi import APIRouter, Depends
|
||
from deps import admin_required
|
||
|
||
router = APIRouter()
|
||
|
||
@router.get("/status")
|
||
async def get_status(user = Depends(admin_required)):
|
||
return {"status": "ok"}
|
||
|
||
# 2. In main.py mounten
|
||
from routers.my_feature import router as my_feature_router
|
||
app.include_router(my_feature_router, prefix="/api/my-feature", tags=["My Feature"])
|
||
|
||
# 3. Tests schreiben
|
||
# tests/test_my_feature.py
|
||
|
||
def test_my_feature_status():
|
||
response = client.get("/api/my-feature/status")
|
||
assert response.status_code == 200
|
||
|
||
# 4. Dokumentieren
|
||
# Docstring mit Examples
|
||
```
|
||
|
||
### Debug-Modus
|
||
|
||
```bash
|
||
# Aktiviere Debug Logging
|
||
export LOG_LEVEL=DEBUG
|
||
|
||
# FastAPI Debug-Modus
|
||
# app/main.py
|
||
app = FastAPI(debug=True)
|
||
|
||
# SQL Query Logging
|
||
# config.py
|
||
echo_queries = True
|
||
```
|
||
|
||
---
|
||
|
||
## 🦉 Wuuuuhuuu! - Schlusswort
|
||
|
||
Dieses Handbuch ist ein lebendes Dokument – wie der Wald selbst wächst und verändert es sich.
|
||
|
||
**Grundregeln:**
|
||
1. **Atme** - Bei Stress: 3x tief durchatmen, dann debuggen
|
||
2. **Logs sind deine Freunde** - Sie urteilen nicht, sie erzählen nur
|
||
3. **DSGVO ist Respekt** - Nicht Bürokratie, sondern Achtung vor jedem Menschen
|
||
4. **Der Wald verzeiht Fehler** - Jeder Bug ist Humus für besseren Code
|
||
|
||
**Bei Problemen:**
|
||
- Lies die Logs: `docker compose logs -f app`
|
||
- Prüfe die Tests: `./run_all_tests.sh`
|
||
- Atme
|
||
- Frag die Eule (Claude/Community)
|
||
|
||
**Weiterentwicklung:**
|
||
- Neue Tests hinzufügen wenn Bugs gefunden werden
|
||
- Dokumentation aktualisieren bei Features
|
||
- DSGVO-Compliance bei jeder Änderung prüfen
|
||
|
||
---
|
||
|
||
**Happy Coding im Crumbforest! 💚**
|
||
|
||
*"Im Wald sind wir alle Lernende. Die Bäume wachsen langsam, aber stetig. Genau wie guter Code."*
|
||
|
||
---
|
||
|
||
Erstellt: 2025
|
||
Version: 1.0
|
||
Lizenz: [Your License]
|
||
Kontakt: [Your Contact]
|