first commit
This commit is contained in:
Branko May Trinkwald
344
README.md
Normal file
344
README.md
Normal file
@@ -0,0 +1,344 @@
|
||||
# Gold Market Analysis System mit Qdrant
|
||||
|
||||
Ein umfassendes Python-System zur Analyse des internationalen Goldmarktes (XAUUSD) mit Vektordatenbank-Integration für langfristige Beziehungsanalysen.
|
||||
|
||||
## 🎯 Projektübersicht
|
||||
|
||||
Dieses System sammelt, analysiert und speichert:
|
||||
- **Stündliche XAUUSD-Kursdaten** von Yahoo Finance
|
||||
- **Technische Indikatoren** (SMA, EMA, RSI, MACD, Bollinger Bands, etc.)
|
||||
- **Markt-News** mit Sentiment-Analyse
|
||||
- **Handelssession-Analysen** (COMEX, London, Shanghai, Tokyo)
|
||||
- **Ereignis-Tracking** für spätere Vergleiche
|
||||
|
||||
Alle Daten werden in **Qdrant** als Vektoren gespeichert, um semantische Suchen und Beziehungsanalysen zu ermöglichen.
|
||||
|
||||
## 🚀 Features
|
||||
|
||||
### Datensammlung
|
||||
- ✅ Yahoo Finance Integration (yfinance)
|
||||
- ✅ XAUUSD, Gold Futures (GC=F), Gold/Silver Index
|
||||
- ✅ Stündliche, tägliche, wöchentliche Daten
|
||||
- ✅ Echtzeit-Updates
|
||||
|
||||
### Technische Analyse
|
||||
- ✅ 15+ technische Indikatoren
|
||||
- ✅ SMA (20, 50, 200), EMA (12, 26)
|
||||
- ✅ RSI, MACD, Bollinger Bands
|
||||
- ✅ ATR, Stochastic, OBV, ADX
|
||||
- ✅ Custom Gold-Indikatoren
|
||||
|
||||
### News & Events
|
||||
- ✅ Alpha Vantage News Integration
|
||||
- ✅ Yahoo Finance News Scraping
|
||||
- ✅ Sentiment-Analyse
|
||||
- ✅ Event-Extraktion (Fed-Entscheidungen, Inflation, etc.)
|
||||
|
||||
### Qdrant Integration
|
||||
- ✅ Vektordatenbank für semantische Suche
|
||||
- ✅ Zeitbasierte Filterung
|
||||
- ✅ Handelssession-Gruppierung
|
||||
- ✅ Ähnlichkeits-Suche für Marktbedingungen
|
||||
|
||||
### Visualisierung & Reporting
|
||||
- ✅ Automatische Marktberichte
|
||||
- ✅ Session-Analysen
|
||||
- ✅ Statistiken & Metriken
|
||||
|
||||
## 📋 Voraussetzungen
|
||||
|
||||
- Python 3.8+
|
||||
- Docker (für Qdrant)
|
||||
- Optional: Alpha Vantage API Key (für News)
|
||||
|
||||
## 🛠️ Installation
|
||||
|
||||
### 1. Repository klonen
|
||||
```bash
|
||||
git clone <repository-url>
|
||||
cd gold-market-analysis
|
||||
```
|
||||
|
||||
### 2. Python-Pakete installieren
|
||||
```bash
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
### 3. Qdrant starten (Docker)
|
||||
```bash
|
||||
# Lokale Installation
|
||||
docker run -p 6333:6333 -p 6334:6334 \
|
||||
-v $(pwd)/qdrant_storage:/qdrant/storage:z \
|
||||
qdrant/qdrant
|
||||
|
||||
# ODER: Qdrant Cloud nutzen (kostenloser Tier verfügbar)
|
||||
# https://cloud.qdrant.io/
|
||||
```
|
||||
|
||||
### 4. Konfiguration (config.py)
|
||||
```python
|
||||
# Optional: API-Keys hinzufügen
|
||||
export ALPHA_VANTAGE_KEY="your_key_here"
|
||||
|
||||
# Optional: Qdrant Cloud
|
||||
export QDRANT_HOST="your-cluster.qdrant.io"
|
||||
export QDRANT_API_KEY="your_api_key"
|
||||
```
|
||||
|
||||
## 📖 Verwendung
|
||||
|
||||
### Schnellstart
|
||||
```bash
|
||||
python main.py
|
||||
```
|
||||
|
||||
Wählen Sie einen Modus:
|
||||
1. **Einmalige Datensammlung** - Sammelt historische Daten
|
||||
2. **Kontinuierlicher Betrieb** - Automatische Updates
|
||||
3. **Analyse-Abfrage** - Suche ähnliche Marktbedingungen
|
||||
4. **Bericht generieren** - Erstellt Marktbericht
|
||||
|
||||
### Beispiele
|
||||
|
||||
#### 1. Historische Daten sammeln
|
||||
```python
|
||||
from main import GoldMarketAnalysisSystem
|
||||
|
||||
system = GoldMarketAnalysisSystem()
|
||||
system.collect_and_store_market_data(
|
||||
ticker_key="XAUUSD",
|
||||
period="1mo", # 1 Monat
|
||||
interval="1h" # Stündlich
|
||||
)
|
||||
```
|
||||
|
||||
#### 2. Ähnliche Marktbedingungen finden
|
||||
```python
|
||||
results = system.analyze_market_conditions(
|
||||
query="hohe Volatilität mit steigendem RSI und bullischem MACD",
|
||||
limit=10
|
||||
)
|
||||
|
||||
for result in results:
|
||||
print(f"Score: {result['score']:.3f}")
|
||||
print(f"Datum: {result['payload']['timestamp']}")
|
||||
print(f"Preis: ${result['payload']['close']:.2f}")
|
||||
```
|
||||
|
||||
#### 3. Session-Analyse
|
||||
```python
|
||||
from datetime import datetime
|
||||
|
||||
analysis = system.get_session_analysis(
|
||||
ticker_key="XAUUSD",
|
||||
date=datetime(2025, 1, 5)
|
||||
)
|
||||
|
||||
for session, stats in analysis.items():
|
||||
print(f"\n{session}:")
|
||||
print(f" Durchschnittspreis: ${stats['avg_price']:.2f}")
|
||||
print(f" Volatilität: {stats['volatility']:.2f}")
|
||||
print(f" Volumen: {stats['total_volume']:,.0f}")
|
||||
```
|
||||
|
||||
#### 4. News & Sentiment
|
||||
```python
|
||||
from news_collector import GoldNewsCollector
|
||||
|
||||
collector = GoldNewsCollector()
|
||||
articles = collector.get_all_news(hours_back=24)
|
||||
sentiment = collector.aggregate_sentiment(articles)
|
||||
|
||||
print(f"Sentiment: {sentiment['sentiment_label']}")
|
||||
print(f"Score: {sentiment['average_score']:.2f}")
|
||||
```
|
||||
|
||||
## 📊 Datenstruktur in Qdrant
|
||||
|
||||
Jeder Datenpunkt in Qdrant enthält:
|
||||
|
||||
```python
|
||||
{
|
||||
"timestamp": "2025-01-05T10:00:00",
|
||||
"ticker": "XAUUSD",
|
||||
"open": 2650.50,
|
||||
"high": 2655.30,
|
||||
"low": 2648.20,
|
||||
"close": 2652.80,
|
||||
"volume": 125000,
|
||||
"trading_session": "LONDON",
|
||||
"indicators": {
|
||||
"RSI_14": 65.4,
|
||||
"MACD_12_26": 5.2,
|
||||
"SMA_20": 2645.0,
|
||||
"BB_upper_20": 2670.0,
|
||||
...
|
||||
},
|
||||
"news_sentiment": 0.15,
|
||||
"related_events": [
|
||||
"Fed: Zinsentscheidung erwartet",
|
||||
"Inflation: Neue Daten veröffentlicht"
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
## 🔍 Handelssessions
|
||||
|
||||
Das System identifiziert automatisch aktive Handelssessions:
|
||||
|
||||
| Session | Zeitzone | Öffnungszeiten (UTC) |
|
||||
|---------|----------|---------------------|
|
||||
| COMEX (New York) | America/New_York | 13:20 - 12:30 (nächster Tag) |
|
||||
| London OTC | Europe/London | 08:00 - 16:30 |
|
||||
| Shanghai Gold Exchange | Asia/Shanghai | 01:30 - 03:30 |
|
||||
| Tokyo (TOCOM) | Asia/Tokyo | 00:00 - 15:00 |
|
||||
|
||||
## 📈 Technische Indikatoren
|
||||
|
||||
### Trend-Indikatoren
|
||||
- **SMA** (20, 50, 200): Simple Moving Average
|
||||
- **EMA** (12, 26): Exponential Moving Average
|
||||
- **ADX** (14): Average Directional Index
|
||||
|
||||
### Momentum-Indikatoren
|
||||
- **RSI** (14): Relative Strength Index
|
||||
- **MACD** (12, 26, 9): Moving Average Convergence Divergence
|
||||
- **Stochastic** (14, 3, 3): Stochastic Oscillator
|
||||
|
||||
### Volatilitäts-Indikatoren
|
||||
- **Bollinger Bands** (20, 2): Volatilitätsbänder
|
||||
- **ATR** (14): Average True Range
|
||||
|
||||
### Volumen-Indikatoren
|
||||
- **OBV**: On Balance Volume
|
||||
|
||||
### Custom-Indikatoren
|
||||
- Price Momentum (ROC)
|
||||
- Volatility (Std Dev)
|
||||
- Support/Resistance Levels
|
||||
- Volume Ratio
|
||||
|
||||
## 🔄 Automatisierung
|
||||
|
||||
### Kontinuierlicher Betrieb
|
||||
```bash
|
||||
python main.py
|
||||
# Wähle Option 2: Kontinuierlicher Betrieb
|
||||
# Gib Update-Intervall ein (z.B. 60 Minuten)
|
||||
```
|
||||
|
||||
Das System führt automatisch aus:
|
||||
- ⏰ **Stündliche Updates**: Echtzeit-Daten
|
||||
- 📅 **Tägliche Sammlung**: Historische Daten (08:00 UTC)
|
||||
- 📊 **Tägliche Berichte**: Marktanalyse (18:00 UTC)
|
||||
|
||||
### Manuelle Zeitplanung (crontab)
|
||||
```bash
|
||||
# Alle 6 Stunden Daten sammeln
|
||||
0 */6 * * * cd /path/to/project && python -c "from main import GoldMarketAnalysisSystem; s = GoldMarketAnalysisSystem(); s.collect_and_store_market_data()"
|
||||
|
||||
# Täglich um 18:00 Bericht generieren
|
||||
0 18 * * * cd /path/to/project && python -c "from main import GoldMarketAnalysisSystem; s = GoldMarketAnalysisSystem(); s.generate_report()"
|
||||
```
|
||||
|
||||
## 🎨 Erweiterte Nutzung
|
||||
|
||||
### Custom-Embeddings
|
||||
```python
|
||||
# In config.py ändern:
|
||||
EMBEDDING_MODEL = "sentence-transformers/all-mpnet-base-v2" # Bessere Qualität
|
||||
EMBEDDING_DIMENSION = 768
|
||||
```
|
||||
|
||||
### Mehrere Tickers gleichzeitig
|
||||
```python
|
||||
from yahoo_collector import YahooFinanceGoldCollector
|
||||
|
||||
collector = YahooFinanceGoldCollector()
|
||||
data = collector.get_multiple_tickers(
|
||||
ticker_keys=["XAUUSD", "GC", "XAU"],
|
||||
period="1mo",
|
||||
interval="1h"
|
||||
)
|
||||
```
|
||||
|
||||
### Zeitbasierte Suche
|
||||
```python
|
||||
from datetime import datetime, timedelta
|
||||
|
||||
start = datetime.now() - timedelta(days=30)
|
||||
end = datetime.now()
|
||||
|
||||
results = system.db.search_by_time_range(
|
||||
start_time=start,
|
||||
end_time=end,
|
||||
ticker="XAUUSD",
|
||||
limit=1000
|
||||
)
|
||||
```
|
||||
|
||||
## 📁 Projektstruktur
|
||||
|
||||
```
|
||||
gold-market-analysis/
|
||||
├── config.py # Konfiguration
|
||||
├── qdrant_client.py # Qdrant Datenbank-Client
|
||||
├── yahoo_collector.py # Yahoo Finance Datensammlung
|
||||
├── technical_indicators.py # Technische Indikatoren
|
||||
├── news_collector.py # News & Sentiment
|
||||
├── main.py # Hauptprogramm
|
||||
├── requirements.txt # Python-Abhängigkeiten
|
||||
├── README.md # Diese Datei
|
||||
└── gold_market_analysis.log # Log-Datei
|
||||
```
|
||||
|
||||
## 🐛 Troubleshooting
|
||||
|
||||
### Qdrant Verbindungsfehler
|
||||
```bash
|
||||
# Prüfe ob Qdrant läuft
|
||||
curl http://localhost:6333/collections
|
||||
|
||||
# Neustart
|
||||
docker restart <qdrant-container-id>
|
||||
```
|
||||
|
||||
### Yahoo Finance Rate Limits
|
||||
- Yahoo Finance hat keine offiziellen Rate Limits
|
||||
- Bei Problemen: Erhöhe Intervall zwischen Requests
|
||||
|
||||
### Alpha Vantage API Limits
|
||||
- Free Tier: 25 Requests/Tag
|
||||
- Upgrade auf kostenpflichtigen Plan für mehr Requests
|
||||
|
||||
## 🔐 Sicherheit
|
||||
|
||||
- API-Keys niemals im Code speichern
|
||||
- Nutze Umgebungsvariablen
|
||||
- `.gitignore` für sensible Daten
|
||||
|
||||
## 📝 Lizenz
|
||||
|
||||
MIT License
|
||||
|
||||
## 🤝 Contributing
|
||||
|
||||
Beiträge sind willkommen! Bitte öffne ein Issue oder Pull Request.
|
||||
|
||||
## 📞 Support
|
||||
|
||||
Bei Fragen oder Problemen:
|
||||
- GitHub Issues öffnen
|
||||
- Dokumentation prüfen
|
||||
- Logs analysieren (`gold_market_analysis.log`)
|
||||
|
||||
## 🎓 Ressourcen
|
||||
|
||||
- [Qdrant Documentation](https://qdrant.tech/documentation/)
|
||||
- [yfinance GitHub](https://github.com/ranaroussi/yfinance)
|
||||
- [Technical Analysis Library](https://github.com/bukosabino/ta)
|
||||
- [Alpha Vantage API](https://www.alphavantage.co/documentation/)
|
||||
|
||||
---
|
||||
|
||||
**Erstellt mit ❤️ für Gold-Markt-Analyse**
|
||||
Reference in New Issue
Block a user