diff --git a/tech_docs/database/sql_roadmap.md b/tech_docs/database/sql_roadmap.md index eaea5fa..3c23b43 100644 --- a/tech_docs/database/sql_roadmap.md +++ b/tech_docs/database/sql_roadmap.md @@ -1,3 +1,186 @@ +# **SQL for Forex Trading - Ultimate Cheat Sheet** + +## **1. Essential Command Structure** +``` +[ACTION] [TARGET] [DETAILS] [CONDITIONS] [MODIFIERS] +``` +Example: +```sql +SELECT bid, ask -- ACTION: what to get +FROM ticks -- TARGET: where from +WHERE symbol = 'EUR/USD' -- CONDITIONS: filters +ORDER BY timestamp DESC -- MODIFIERS: sorting +LIMIT 100; -- MODIFIERS: quantity +``` + +## **2. Core Commands** + +### **Table Operations** +| Command | Example | Purpose | +|---------|---------|---------| +| `CREATE TABLE` | `CREATE TABLE ticks(timestamp TIMESTAMP, bid FLOAT)` | Create new table | +| `ALTER TABLE` | `ALTER TABLE ticks ADD COLUMN volume FLOAT` | Modify structure | +| `DROP TABLE` | `DROP TABLE ticks` | Delete table | + +### **Data Import/Export** +```sql +-- Import CSV +COPY ticks FROM '/data/ticks.csv' (FORMAT CSV); + +-- Export results +COPY (SELECT * FROM ticks) TO '/output.csv' (FORMAT CSV); +``` + +### **Basic Queries** +```sql +-- All columns, limited rows +SELECT * FROM ticks LIMIT 10; + +-- Specific columns +SELECT timestamp, bid, ask FROM ticks; + +-- Filtered data +SELECT * FROM ticks WHERE bid > 1.1000 AND volume > 1000; +``` + +## **3. Time-Series Patterns** + +### **Candlestick Generation** +```sql +SELECT + DATE_TRUNC('1 minute', timestamp) AS minute, + FIRST(bid) AS open, + MAX(bid) AS high, + MIN(bid) AS low, + LAST(bid) AS close, + SUM(volume) AS volume +FROM ticks +GROUP BY minute; +``` + +### **Rolling Calculations** +```sql +-- 30-minute moving average +SELECT + timestamp, + AVG(bid) OVER (ORDER BY timestamp ROWS 29 PRECEDING) AS ma_30 +FROM ticks; + +-- Rolling spread +SELECT + timestamp, + AVG(ask - bid) OVER (ORDER BY timestamp RANGE '5 minutes') AS avg_spread +FROM ticks; +``` + +## **4. Key Metrics** + +### **Spread Analysis** +```sql +-- Basic spread +SELECT timestamp, ask - bid AS spread FROM ticks; + +-- Session averages +SELECT + EXTRACT(HOUR FROM timestamp) AS hour, + AVG(ask - bid) AS avg_spread +FROM ticks +GROUP BY hour; +``` + +### **Order Book Imbalance** +```sql +SELECT + timestamp, + (bid_size - ask_size) / (bid_size + ask_size) AS imbalance +FROM ticks; +``` + +## **5. Advanced Patterns** + +### **Correlation Analysis** +```sql +WITH hourly AS ( + SELECT + DATE_TRUNC('hour', timestamp) AS hour, + AVG(CASE WHEN symbol='EUR/USD' THEN bid END) AS eurusd, + AVG(CASE WHEN symbol='USD/JPY' THEN bid END) AS usdjpy + FROM ticks + GROUP BY hour +) +SELECT CORR(eurusd, usdjpy) FROM hourly; +``` + +### **Event Detection** +```sql +-- Large price jumps +SELECT * +FROM ticks +WHERE ABS(bid - LAG(bid) OVER (ORDER BY timestamp)) > 0.0010; +``` + +## **6. Optimization** + +### **Indexing** +```sql +-- Basic index +CREATE INDEX idx_symbol_time ON ticks(symbol, timestamp); + +-- For time-series +SELECT create_hypertable('ticks', 'timestamp'); -- TimescaleDB +``` + +### **Query Performance** +```sql +-- Explain plan +EXPLAIN ANALYZE SELECT * FROM ticks WHERE symbol = 'EUR/USD'; + +-- Common optimizations: +-- 1. Use WHERE before GROUP BY +-- 2. Limit selected columns (not SELECT *) +-- 3. Use EXISTS instead of IN for large datasets +``` + +## **7. Quick Reference** + +### **Data Types** +| Type | Example | Used For | +|------|---------|----------| +| `TIMESTAMP` | `2024-01-01 12:00:00` | Time data | +| `FLOAT` | `1.12345` | Prices/values | +| `VARCHAR` | `'EUR/USD'` | Text/symbols | +| `BOOLEAN` | `TRUE` | Flags | + +### **Aggregate Functions** +| Function | Example | Purpose | +|----------|---------|---------| +| `AVG()` | `AVG(bid)` | Average | +| `MAX()` | `MAX(ask)` | Highest value | +| `MIN()` | `MIN(bid)` | Lowest value | +| `SUM()` | `SUM(volume)` | Total volume | +| `COUNT()` | `COUNT(*)` | Row count | + +## **8. Common Errors & Fixes** + +| Error | Solution | +|-------|----------| +| Missing comma in column list | Check commas between columns in SELECT/CREATE | +| "GROUP BY" needed | Add `GROUP BY` when using aggregates | +| NULL comparison issues | Use `IS NULL` not `= NULL` | +| Slow queries | Add indexes on filtered/sorted columns | + +## **How to Use This Cheat Sheet** +1. **Daily Practice**: Pick 2-3 patterns to implement each day +2. **Wall Reference**: Print and keep near your workstation +3. **Troubleshooting**: Use the errors section when queries fail + +Want me to: +1. Provide a PDF version of this cheat sheet? +2. Add more forex-specific examples? +3. Or focus on a particular section in more detail? + +--- + Here's a streamlined 8-week roadmap focused purely on practical SQL skills for forex bid/ask analysis, structured for immediate application in cron jobs: ### **Week 1-2: Core Foundations for Tick Data**