medusa/the_information_nexus
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

site updates

Browse Source
This commit is contained in:
Whisker Jones
2024-03-08 11:49:16 -07:00
parent 3c576ec2c0
commit 9ea58c0270
6 changed files with 0 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

354
docs/financial_docs/forex_algo_trading.md Normal file
View File

@@ -0,0 +1,354 @@
## Example
```mermaid
graph TD
style A fill:#333,stroke:#FFCC00,stroke-width:2px
style B fill:#333,stroke:#FFCC00,stroke-width:2px
style C fill:#555,stroke:#FFCC00,stroke-width:2px
style D fill:#555,stroke:#FFCC00,stroke-width:2px
style E fill:#555,stroke:#FFCC00,stroke-width:2px
style F fill:#555,stroke:#FFCC00,stroke-width:2px
style G fill:#777,stroke:#FFCC00,stroke-width:2px
style H fill:#777,stroke:#FFCC00,stroke-width:2px
A[Data Ingestion Service] -->|Fetches Data| B[Kubernetes Cluster]
B -->|Stores| C[Data Storage: S3/GlusterFS]
B -->|Analysis| D[Analysis Microservice]
D -->|Generates Signals| E[Signal Generation Microservice]
E -->|Executes Orders| F[Order Execution Service]
F -->|Places Orders| G[Brokerage API: Oanda]
B -->|Monitors System| H[Monitoring & Alerting: Prometheus/Grafana]
linkStyle default interpolate basis
```
## Folder Structure
```mermaid
graph TD
A("/mean-reversion-trading") --> B("/data")
A --> C("/models")
A --> D("/strategies")
A --> E("/backtesting")
A --> F("/trading")
A --> G("/utils")
A --> H("Dockerfile")
A --> I("requirements.txt")
A --> J("README.md")
B --> B1("/raw")
B --> B2("/processed")
C --> C1("/trained")
C --> C2("model_training.py")
D --> D1("mean_reversion_strategy.py")
E --> E1("backtest.py")
F --> F1("live_trade.py")
G --> G1("data_fetcher.py")
G --> G2("feature_engineering.py")
G --> G3("indicators.py")
```
---
```plaintext
/mean-reversion-trading
|-- /data
| |-- /raw # Store raw historical data fetched from Oanda
| `-- /processed # Processed and feature-engineered datasets
|
|-- /models
| |-- /trained # Saved models after training
| `-- model_training.py # Script for ML model training and evaluation
|
|-- /strategies
| |-- mean_reversion_strategy.py # Trading strategy implementation for Backtrader
|
|-- /backtesting
| |-- backtest.py # Script for backtesting strategies using Backtrader
|
|-- /trading
| |-- live_trade.py # Script for live trading on Oanda
|
|-- /utils
| |-- data_fetcher.py # Script for fetching data from Oanda
| |-- feature_engineering.py # Utilities for data cleaning and feature engineering
| `-- indicators.py # Custom indicators for strategy (e.g., Bollinger Bands, RSI)
|
|-- Dockerfile # Dockerfile for containerization
|-- requirements.txt # Python dependencies
`-- README.md # Project documentation
```
---
# Mean Reversion Trading Strategy for EUR/USD with Machine Learning
## Overview
This guide is dedicated to developing a mean reversion trading strategy for the EUR/USD currency pair. It harnesses the power of machine learning (ML) via scikit-learn for strategy development, Backtrader for backtesting, and ultimately, deploying the optimized strategy for live trading on Oanda.
## Step 1: Data Preparation
### Fetch Historical EUR/USD Data
- **Objective**: Use `oandapyV20` to download 5 years of EUR/USD daily data from Oanda.
- **Rationale**: A 5-year period provides a balanced dataset to capture various market phases, essential for training robust mean reversion models.
### Clean and Preprocess Data
- **Tasks**: Eliminate duplicates and handle missing data. Standardize prices to ensure consistency across the dataset.
- **Normalization**: Apply Min-Max scaling to align features on a similar scale, enhancing model training efficiency.
## Step 2: Exploratory Data Analysis (EDA) and Feature Engineering
### Perform EDA
- **Visualization**: Plot price movements with `matplotlib` to identify mean reversion patterns. Analyze price volatility and its correlation with mean reversion points.
### Develop Technical Indicators
- **Indicators for Mean Reversion**: Calculate Bollinger Bands and RSI. These indicators help identify overbought or oversold conditions signaling potential mean reversions.
### Feature Engineering
- **Feature Creation**: Derive features like the distance from moving averages, Bollinger Band width, and RSI levels to capture market states indicative of mean reversion.
## Step 3: Machine Learning Model Development with Scikit-learn
### Choose an ML Model
- **Model Selection**: Start with Logistic Regression to classify potential mean reversion opportunities. Consider Random Forest for a more nuanced understanding of feature relationships.
### Train and Validate the Model
- **Cross-Validation**: Implement cross-validation to assess model performance, minimizing the risk of overfitting.
- **Metrics**: Evaluate models based on accuracy, precision, recall, and the F1 score to ensure a balanced assessment of the model's predictive capabilities.
## Step 4: Backtesting Strategy with Backtrader
### Integrate ML Model into Backtrader Strategy
- **Strategy Implementation**: Embed your scikit-learn model within a custom Backtrader strategy. Use model predictions to drive trade entries and exits based on identified mean reversion signals.
### Execute Backtesting
- **Configuration**: Set up Backtrader with historical EUR/USD data, including transaction costs and initial capital.
- **Analysis**: Utilize Backtrader's analyzers to evaluate the strategy's performance, focusing on net profit, drawdown, and Sharpe ratio.
## Step 5: Live Trading Preparation
### Paper Trading with Oanda Demo
- **Objective**: Validate the strategy under current market conditions using Oanda's demo account.
- **Adjustments**: Fine-tune strategy parameters and risk management settings based on paper trading outcomes.
### Transition to Live Trading
- **Live Account Switch**: Transition the strategy to a live Oanda account for real trading.
- **Capital Management**: Start with conservative capital allocation, gradually scaling based on live performance and risk appetite.
### Continuous Monitoring and Optimization
- **Live Performance Tracking**: Closely monitor trading activity and performance metrics.
- **Strategy Iteration**: Regularly review and adjust the trading model and strategy parameters in response to evolving market conditions and performance insights.
## Conclusion
This guide provides a concise roadmap for creating a mean reversion trading strategy for the EUR/USD pair, leveraging machine learning for signal generation, Backtrader for rigorous backtesting, and Oanda for deployment. It emphasizes a systematic approach from data analysis to live trading, ensuring a well-founded strategy backed by empirical evidence and optimized through practical experience.
---
# Swing Trading Project with EUR/USD Using Oanda and scikit-learn
## Step 1: Environment Setup
### Install Python
Ensure Python 3.8+ is installed.
### Create a Virtual Environment
Navigate to your project directory and run:
```bash
python -m venv venv
source venv/bin/activate # Unix/macOS
venv\Scripts\activate # Windows
deactivate
```
### Install Essential Libraries
Create `requirements.txt` with the following content:
```
pandas
numpy
matplotlib
seaborn
scikit-learn
jupyterlab
oandapyV20
requests
```
Install with `pip install -r requirements.txt`.
## Step 2: Project Structure
Organize your directory as follows:
```
swing_trading_project/
├── data/
├── notebooks/
├── src/
│ ├── __init__.py
│ ├── data_fetcher.py
│ ├── feature_engineering.py
│ ├── model.py
│ └── backtester.py
├── tests/
├── requirements.txt
└── README.md
```
## Step 3: Fetch Historical Data
- Sign up for an Oanda practice account and get an API key.
- Use `oandapyV20` in `data_fetcher.py` to request historical EUR/USD data. Consider H4 or D granularity.
- Save the data to `data/` as CSV.
```python
import os
import pandas as pd
from oandapyV20 import API # Import the Oanda API client
import oandapyV20.endpoints.instruments as instruments
# Set your Oanda API credentials and configuration for data fetching
ACCOUNT_ID = 'your_account_id_here'
ACCESS_TOKEN = 'your_access_token_here'
# List of currency pairs to fetch. Add or remove pairs as needed.
CURRENCY_PAIRS = ['EUR_USD', 'USD_JPY', 'GBP_USD', 'AUD_USD', 'USD_CAD']
TIME_FRAME = 'H4' # 4-hour candles, change as per your analysis needs
DATA_DIRECTORY = 'data' # Directory where fetched data will be saved
# Ensure the data directory exists, create it if it doesn't
if not os.path.exists(DATA_DIRECTORY):
os.makedirs(DATA_DIRECTORY)
def fetch_and_save_forex_data(account_id, access_token, currency_pairs, time_frame, data_dir):
"""Fetch historical forex data for specified currency pairs and save it to CSV files."""
# Initialize the Oanda API client with your access token
api_client = API(access_token=access_token)
for pair in currency_pairs:
# Define the parameters for the data request: time frame and number of data points
request_params = {"granularity": time_frame, "count": 5000}
# Prepare the data request for fetching candle data for the current currency pair
data_request = instruments.InstrumentsCandles(instrument=pair, params=request_params)
# Fetch the data
response = api_client.request(data_request)
# Extract the candle data from the response
candle_data = response.get('candles', [])
# If data was fetched, proceed to save it
if candle_data:
# Convert the candle data into a pandas DataFrame
forex_data_df = pd.DataFrame([{
'Time': candle['time'],
'Open': float(candle['mid']['o']),
'High': float(candle['mid']['h']),
'Low': float(candle['mid']['l']),
'Close': float(candle['mid']['c']),
'Volume': candle['volume']
} for candle in candle_data])
# Construct the filename for the CSV file
csv_filename = f"{pair.lower()}_data.csv"
# Save the DataFrame to a CSV file in the specified data directory
forex_data_df.to_csv(os.path.join(data_dir, csv_filename), index=False)
print(f"Data for {pair} saved to {csv_filename}")
def main():
"""Orchestrates the data fetching and saving process."""
print("Starting data fetching process...")
# Call the function to fetch and save data for the configured currency pairs
fetch_and_save_forex_data(ACCOUNT_ID, ACCESS_TOKEN, CURRENCY_PAIRS, TIME_FRAME, DATA_DIRECTORY)
print("Data fetching process completed.")
if __name__ == '__main__':
# Execute the script
main()
```
## Step 4: Exploratory Data Analysis
- Create a new Jupyter notebook in `notebooks/`.
- Load the CSV with `pandas` and perform initial exploration. Plot closing prices and moving averages.
## Step 5: Basic Feature Engineering
- In the notebook, add technical indicators as features (e.g., SMA 50, SMA 200, RSI) using `pandas`.
- Investigate the relationship between these features and price movements.
## Step 6: Initial Model Training
- In `model.py`, fit a simple `scikit-learn` model (e.g., LinearRegression, LogisticRegression) to predict price movements.
- Split data into training and testing sets to evaluate the model's performance.
## Step 7: Documentation
- Document your project's setup, objectives, and findings in `README.md`.
## Next Steps
- Refine features, try different models, and develop a backtesting framework as you progress.
---
# From Backtesting to Live Trading with Backtrader and Oanda
## Setup and Installation
- **Install Required Packages**
```bash
pip install backtrader oandapyV20
```
- **Oanda API Credentials**
- Obtain API credentials from your Oanda demo account.
## Backtesting
### 1. Data Preparation
- Fetch historical data using Oanda's API for your target currency pairs.
### 2. Strategy Development
- Code your trading strategy within a subclass of `bt.Strategy`.
- Define indicators, entry and exit logic.
### 3. Backtesting Execution
- Initialize a `bt.Cerebro()` engine, adding your strategy and data.
- Set initial capital and other parameters.
- Run backtest and analyze results using Backtrader's built-in analyzers.
## Transition to Paper Trading
### 1. Configure Live Data Feed
- Setup a live data feed from Oanda using the `oandapyV20` package.
### 2. Integrate Oanda Demo as Broker
- Configure Backtrader to use Oanda as the broker with your demo account credentials.
- This simulates order execution in the demo environment.
### 3. Run Paper Trading
- Execute your strategy with Backtrader against the live data feed in simulation mode.
- Monitor performance and make adjustments as necessary.
## Going Live
### 1. Strategy Review and Adjustment
- Fine-tune your strategy based on insights gained from paper trading.
### 2. Switch to Live Account
- Change the API credentials in your script to those of your live Oanda account.
### 3. Start Live Trading
- Begin with the smallest lot sizes.
- Closely monitor the strategy's live trading performance.
## Key Considerations
- **Monitoring**: Keep a close watch on your system's operation during live trading.
- **Incremental Deployment**: Gradually increase your trading size based on the strategy's live performance.
- **Continuous Improvement**: Regularly update your strategy based on live trading data and market conditions.
This markdown guide outlines a focused and actionable path from developing and backtesting trading strategies with Backtrader and Oanda, to paper trading and eventually live trading.
---

125
docs/financial_docs/mean_reversion_algo.md Normal file
View File

@@ -0,0 +1,125 @@
# Mean Reversion Trading System for EUR/USD
Welcome to the Mean Reversion Trading System project, specifically tailored for the EUR/USD forex pair. This project combines the precision of machine learning (ML), the robustness of Backtrader for strategy evaluation, and the real-world applicability of trading via the Oanda platform. It’s structured for scalability, leveraging containerization for development consistency and serverless architecture for operational efficiency.
## Project Overview
This system is designed for traders and developers interested in exploring and deploying automated trading strategies. It focuses on the mean reversion principle, a well-regarded concept in finance that suggests asset prices and returns eventually move back towards the mean or average. This project aims to capitalize on this phenomenon using historical EUR/USD data, predictive modeling, and live execution.
## Project Structure
```
/mean-reversion-trading
|-- /data
| |-- /raw # Store raw historical data from Oanda
| `-- /processed # Processed data, ready for ML
|
|-- /models
| |-- /trained # Location for storing trained models
| `-- model_training.py # Script for ML model training
|
|-- /strategies
| |-- mean_reversion_strategy.py # Strategy implementation for Backtrader
|
|-- /backtesting
| |-- backtest.py # Backtesting script using Backtrader
|
|-- /trading
| |-- live_trade.py # Live trading execution script
|
|-- /utils
| |-- data_fetcher.py # Utility script for data retrieval
| |-- feature_engineering.py # Data processing and feature engineering
| `-- indicators.py # Custom technical indicators
|
|-- Dockerfile # For containerization
|-- requirements.txt # Python package dependencies
`-- README.md # Project documentation
```
### Detailed Component Overview
- **/data**: Contains both raw and processed datasets. Raw data is directly fetched from Oanda, while processed data includes features engineered to enhance model training.
- **/models**: This directory is pivotal for the ML lifecycle, encompassing scripts for training, validation, and serialization of models.
- **/strategies**: Implements trading strategies within the Backtrader framework, enabling a seamless transition from theory to practical testing.
- **/backtesting & /trading**: These separate concerns between historical strategy evaluation and real-world application, ensuring a clear pathway from concept to execution.
- **/utils**: A collection of utility scripts support the data pipeline, from fetching and cleaning to feature engineering, alongside custom indicator development for strategic analysis.
## Getting Started
### Installation
Ensure Python 3.8+ is installed on your system. Clone this repository and navigate into the project directory. Set up a virtual environment and install the required dependencies:
```sh
python -m venv venv
source venv/bin/activate # or venv\Scripts\activate on Windows
pip install -r requirements.txt
```
### Environment Configuration
Create a `.env` file in the project root. Add your Oanda API credentials here:
```plaintext
OANDA_ACCOUNT_ID=<your_account_id>
OANDA_API_KEY=<your_api_key>
```
### Running the Project
#### Data Preparation
Fetch and prepare your data for analysis and model training:
```sh
python utils/data_fetcher.py
```
#### Model Training
Train your model using historical data:
```sh
python models/model_training.py
```
#### Backtesting
Evaluate your strategy with Backtrader:
```sh
python backtesting/backtest.py
```
#### Live Trading
Execute your strategy in the live market:
```sh
python trading/live_trade.py
```
## Deployment
For containerization, build the Docker image with:
```sh
docker build -t mean-reversion-trading .
```
To run the containerized application:
```sh
docker run -it mean-reversion-trading
```
For serverless deployment, refer to your cloud provider's documentation for deploying Docker containers and setting up serverless functions for tasks like scheduled data fetching and model retraining.
## Contributing
We welcome contributions! Whether it's feature requests, bug reports, or code contributions, please feel free to reach out or submit a pull request.
## License
Distributed under the MIT License. See `LICENSE` for more information.

93
docs/financial_docs/wire.md Normal file
View File

@@ -0,0 +1,93 @@
## Primary Combined Briefing
### **Crypto Briefing: [Date]**
#### **Date & Time:** [Insert Date & Time]
---
### **Price Snapshot:**
- **BTC:** $[Insert BTC Price] _(24h: [Insert % Change], 7d: [Insert % Change])_
- **ETH:** $[Insert ETH Price] _(24h: [Insert % Change], 7d: [Insert % Change])_
- **ADA:** $[Insert ADA Price] _(24h: [Insert % Change], 7d: [Insert % Change])_
---
### **Market Highlights:**
- [Bullet Point or Summary]
- [Bullet Point or Summary]
---
### **Key News & Events:**
- [News/Event 1]
- [News/Event 2]
---
### **General Market Sentiment:**
- [Insert Overall Sentiment Overview]
---
### **Closing:**
_Check out our specialized briefing on [Crypto Name] for an in-depth look at today's significant developments._
---
## Specialized Briefing
### **Special [Crypto Name] Briefing: [Date]**
#### **Date & Time:** [Insert Date & Time]
---
### **Price Snapshot:**
- **[Crypto Name]:** $[Insert Crypto Price] _(Support: [Price Level], Resistance: [Price Level])_
---
### **Technical & On-chain Data Insights:**
- **Technical Analysis:** [Insert Technical Analysis Details]
- **On-chain Metric 1:** [Insert Data]
- **On-chain Metric 2:** [Insert Data]
---
### **Detailed News & Events:**
- [Detailed News/Event 1]
- [Detailed News/Event 2]
---
### **Upcoming Milestones & Events:**
- [Milestone/Event 1]
- [Milestone/Event 2]
---
### **Analysis or Comment:**
[Insert Detailed Analysis]
---
### **Closing:**
_For an overview of other cryptos, refer to our primary combined briefing dated [Date]._
---
### **Disclaimer:**
_The information provided in this briefing is for informational purposes only and should not be considered financial advice._