Update financial_docs/ml_trading.md

financial_docs/ml_trading.md

@@ -1,3 +1,180 @@
+---
+marp: true
+theme: default
+paginate: true
+header: "Forex Time Series Analysis with AI/ML Models"
+footer: "Prepared for CS and Math PhD Students"
+---
+
+# Forex Time Series Analysis with AI/ML Models
+
+---
+
+## Objective
+- Understand the technical principles and methodologies for analyzing forex time series data
+- Focus on inference topics: trend identification, model training, and model evaluation
+
+---
+
+## Trend Identification
+
+- Moving Averages
+  - Simple Moving Average (SMA):
+    $\text{SMA}(n) = \frac{1}{n} \sum_{i=0}^{n-1} X_{t-i}$
+  - Exponential Moving Average (EMA):
+    $\text{EMA}(t) = \alpha \cdot X_t + (1-\alpha) \cdot \text{EMA}(t-1)$
+- Trend Indicators
+  - MACD (Moving Average Convergence Divergence)
+  - RSI (Relative Strength Index)
+- Visualization of trends using matplotlib or seaborn
+
+---
+
+## Model Training
+
+- ARIMA (AutoRegressive Integrated Moving Average)
+  - Components:
+    - AR: $X_t = \phi_1 X_{t-1} + \phi_2 X_{t-2} + \dots + \phi_p X_{t-p} + \epsilon_t$
+    - I: $Y_t = X_t - X_{t-1}$ (d times differencing)
+    - MA: $X_t = \epsilon_t + \theta_1 \epsilon_{t-1} + \theta_2 \epsilon_{t-2} + \dots + \theta_q \epsilon_{t-q}$
+  - Determining optimal p, d, q parameters
+  - Python implementation using statsmodels library
+
+---
+
+## Model Training (cont.)
+
+- LSTM (Long Short-Term Memory)
+  - Architecture: Input, forget, and output gates control the cell state
+  - Equations:
+    - Forget Gate: $f_t = \sigma(W_f \cdot [h_{t-1}, X_t] + b_f)$
+    - Input Gate: $i_t = \sigma(W_i \cdot [h_{t-1}, X_t] + b_i)$
+    - Output Gate: $o_t = \sigma(W_o \cdot [h_{t-1}, X_t] + b_o)$
+    - Cell State: $C_t = f_t * C_{t-1} + i_t * \tilde{C_t}$
+  - Python implementation using TensorFlow or PyTorch
+
+---
+
+## Model Training (cont.)
+
+- Transformers
+  - Self-attention mechanism
+  - Attention Mechanism: $\text{Attention}(Q, K, V) = \text{softmax}\left( \frac{QK^T}{\sqrt{d_k}} \right) V$
+  - Components: Multi-head attention, feed-forward networks, positional encodings
+  - Python implementation using transformers library
+
+---
+
+## Model Evaluation
+
+- Root Mean Squared Error (RMSE)
+  - Formula: $\text{RMSE} = \sqrt{ \frac{1}{n} \sum_{i=1}^n (Y_i - \hat{Y_i})^2 }$
+  - Lower RMSE indicates better model performance
+- Cross-validation techniques
+  - Rolling window
+  - Time series split
+- Comparison of RMSE values across different models and hyperparameter configurations
+
+---
+
+## Python Code: Trend Identification
+
+```python
+import pandas as pd
+
+def calculate_moving_averages(data, window_sizes):
+    sma = data.rolling(window=window_sizes[0]).mean()
+    ema = data.ewm(span=window_sizes[1]).mean()
+    return pd.concat([sma, ema], axis=1, keys=['SMA', 'EMA'])
+
+# Example usage
+window_sizes = [10, 20]
+moving_averages = calculate_moving_averages(data, window_sizes)
+```
+
+---
+
+## Python Code: Model Training (ARIMA)
+
+```python
+from statsmodels.tsa.arima.model import ARIMA
+
+def train_arima_model(data, p, d, q):
+    model = ARIMA(data, order=(p, d, q))
+    results = model.fit()
+    return results
+
+# Example usage
+p, d, q = 2, 1, 2
+arima_model = train_arima_model(training_data, p, d, q)
+```
+
+---
+
+## Python Code: Model Training (LSTM)
+
+```python
+from tensorflow.keras.models import Sequential
+from tensorflow.keras.layers import LSTM, Dense
+
+def train_lstm_model(data, num_layers, hidden_units, dropout):
+    model = Sequential()
+    model.add(LSTM(hidden_units, return_sequences=True, input_shape=(data.shape[1], 1)))
+    model.add(Dropout(dropout))
+    model.add(LSTM(hidden_units))
+    model.add(Dense(1))
+    model.compile(loss='mean_squared_error', optimizer='adam')
+    model.fit(data, epochs=50, batch_size=32)
+    return model
+
+# Example usage
+num_layers = 2
+hidden_units = 64
+dropout = 0.2
+lstm_model = train_lstm_model(training_data, num_layers, hidden_units, dropout)
+```
+
+---
+
+## Python Code: Model Evaluation
+
+```python
+from sklearn.metrics import mean_squared_error
+import numpy as np
+
+def evaluate_model(model, test_data):
+    predictions = model.predict(test_data)
+    rmse = np.sqrt(mean_squared_error(test_data, predictions))
+    return rmse
+
+# Example usage
+arima_rmse = evaluate_model(arima_model, test_data)
+lstm_rmse = evaluate_model(lstm_model, test_data)
+transformer_rmse = evaluate_model(transformer_model, test_data)
+```
+
+---
+
+## Conclusion
+
+- Trend identification techniques: Moving averages (SMA, EMA) and trend indicators (MACD, RSI)
+- Model training: ARIMA, LSTM, and Transformers
+- Model evaluation using RMSE and cross-validation techniques
+- Python code snippets for each component
+
+Further considerations:
+- Hyperparameter tuning for optimal model performance
+- Ensemble methods for combining multiple models
+- Integration with TimescaleDB for efficient data storage and retrieval
+
+---
+
+# Thank You!
+
+Questions?
+
+---
+
 Certainly! Let's break down the Python code for each component of the forex time series analysis pipeline and highlight the important values and parameters. I'll use pseudocode for unnecessary code to keep the focus on the key aspects.
 
 ### 1. Data Preparation