From 64e245406d0e844b21bdf965e9cbe92ce49136f3 Mon Sep 17 00:00:00 2001 From: medusa Date: Tue, 5 Aug 2025 23:51:19 -0500 Subject: [PATCH] Update tech_docs/jinja2_project.md --- tech_docs/jinja2_project.md | 295 ++++++++++++++++++++++++++++++++++++ 1 file changed, 295 insertions(+) diff --git a/tech_docs/jinja2_project.md b/tech_docs/jinja2_project.md index 6ab401d..7f31f19 100644 --- a/tech_docs/jinja2_project.md +++ b/tech_docs/jinja2_project.md @@ -1,3 +1,298 @@ +Based on your Jinja2 mastery goals and the comprehensive use cases in your document, here's the **tactical toolchain** that will make you unstoppable: + +## **Core Stack (Your Foundation)** + +### **1. Python Ecosystem** +```python +# Essential Python libraries for Jinja2 mastery +pip install jinja2 pyyaml requests click rich tabulate +``` + +**Why Each Matters:** +- **PyYAML**: Parse YAML data files (your primary input format) +- **Requests**: Pull data from APIs (NetBox, cloud providers, monitoring systems) +- **Click**: Build CLI tools for your templates +- **Rich**: Beautiful terminal output for your tools +- **Tabulate**: Generate nice tables in outputs + +### **2. Data Sources & APIs** +| Tool | Purpose | Jinja2 Integration | +|------|---------|-------------------| +| **NetBox** | Network documentation SSOT | `pynetbox` → YAML → templates | +| **Nautobot** | NetBox alternative | Same pattern as NetBox | +| **Ansible** | Automation platform | Built-in Jinja2 support | +| **Git** | Version control | Store templates + data | + +--- + +## **Specialized Tools by Use Case** + +### **Network Engineering Stack** +```bash +# Network-specific tools +pip install netmiko nornir napalm textfsm +pip install ipaddress netaddr +``` + +**Power Combo Example:** +```python +from netmiko import ConnectHandler +from jinja2 import Template +import yaml + +# Pull live config from device +device = ConnectHandler(**cisco_device) +running_config = device.send_command("show running-config") + +# Generate new config from template +template = Template(open('templates/bgp.j2').read()) +new_config = template.render(yaml.safe_load(open('data/bgp.yml'))) + +# Deploy with error checking +device.send_config_set(new_config.split('\n')) +``` + +### **Documentation & Visualization** +```bash +# Documentation tools +pip install mkdocs mkdocs-material +pip install graphviz plotly mermaid-cli +npm install -g @mermaid-js/mermaid-cli # For diagram generation +``` + +**Workflow:** +```python +# Generate network diagrams from templates +template = env.get_template('network_diagram.mmd.j2') +mermaid_output = template.render(topology=network_data) + +# Convert to PNG via CLI +subprocess.run(['mmdc', '-i', 'diagram.mmd', '-o', 'diagram.png']) +``` + +### **Sales/Proposal Generation** +```bash +# Document generation +pip install python-docx reportlab pandoc +``` + +**Pro Tip:** +```python +# Jinja2 → LaTeX → PDF pipeline +template = env.get_template('proposal.tex.j2') +latex_content = template.render(customer_data) + +with open('proposal.tex', 'w') as f: + f.write(latex_content) + +subprocess.run(['pdflatex', 'proposal.tex']) +``` + +--- + +## **DevOps/Automation Stack** + +### **Container & Cloud Tools** +```bash +# Cloud automation +pip install boto3 azure-identity google-cloud-storage +pip install kubernetes docker-py +``` + +**Multi-Cloud Template Example:** +```jinja2 +{# Works for AWS, Azure, GCP #} +{% if cloud_provider == 'aws' %} +resource "aws_instance" "{{ instance_name }}" { + ami = "{{ ami_id }}" + instance_type = "{{ instance_type }}" +} +{% elif cloud_provider == 'azure' %} +resource "azurerm_virtual_machine" "{{ instance_name }}" { + name = "{{ instance_name }}" + vm_size = "{{ vm_size }}" +} +{% endif %} +``` + +### **CI/CD Integration Tools** +```bash +# Pipeline tools +pip install github3.py python-gitlab +pip install pre-commit black flake8 +``` + +**GitHub Actions Integration:** +```yaml +# .github/workflows/jinja2-render.yml +- name: Render Templates + run: | + python scripts/render_all.py + git add rendered/ + git commit -m "Auto-rendered configs [skip ci]" +``` + +--- + +## **Advanced Toolchain** + +### **Database Integration** +```python +# Pull data from multiple sources +import sqlite3 +import psycopg2 +from sqlalchemy import create_engine +import pandas as pd + +# SQL → Jinja2 pipeline +df = pd.read_sql("SELECT * FROM network_devices", engine) +devices = df.to_dict('records') +template.render(devices=devices) +``` + +### **Testing & Validation** +```bash +pip install pytest jsonschema yamale +``` + +**Template Testing:** +```python +# tests/test_templates.py +def test_cisco_config_template(): + template = env.get_template('cisco.j2') + result = template.render(test_data) + + assert 'hostname TEST-SW-01' in result + assert 'interface GigabitEthernet0/1' in result + assert 'no shutdown' in result +``` + +### **Performance & Scalability** +```python +# For high-volume templating +from concurrent.futures import ThreadPoolExecutor +from functools import lru_cache + +@lru_cache(maxsize=128) +def get_template(template_name): + return env.get_template(template_name) + +# Parallel rendering +with ThreadPoolExecutor(max_workers=4) as executor: + configs = list(executor.map(render_device_config, devices)) +``` + +--- + +## **Monitoring & Observability** +```bash +pip install prometheus_client grafana-api +pip install loguru structlog +``` + +**Template Metrics:** +```python +from prometheus_client import Counter, Histogram +import time + +TEMPLATE_RENDERS = Counter('jinja2_renders_total', 'Total template renders') +RENDER_TIME = Histogram('jinja2_render_seconds', 'Template render time') + +@RENDER_TIME.time() +def render_with_metrics(template, data): + TEMPLATE_RENDERS.inc() + return template.render(**data) +``` + +--- + +## **IDE & Development Tools** + +### **VS Code Extensions** +- **Jinja2**: Syntax highlighting for `.j2` files +- **YAML**: Better YAML editing +- **Python**: Obviously +- **GitLens**: Track template changes +- **REST Client**: Test APIs that feed your templates + +### **CLI Tools** +```bash +# Essential CLI tools +pip install jinja2-cli j2cli cookiecutter +npm install -g js-yaml # YAML validation +``` + +**Quick CLI Usage:** +```bash +# One-liner template rendering +j2 template.j2 data.yml --format=yaml > output.cfg + +# Template from stdin +echo "Hello {{ name }}" | j2 --undefined -D name=World +``` + +--- + +## **Your 90-Day Mastery Stack** + +### **Month 1: Foundation** +```bash +# Core tools only +pip install jinja2 pyyaml click rich +git clone https://github.com/your-templates/network-templates +``` + +### **Month 2: Specialization** +```bash +# Add your domain-specific tools +pip install netmiko ansible # Network +# OR +pip install boto3 kubernetes # Cloud +# OR +pip install python-docx reportlab # Documentation +``` + +### **Month 3: Automation** +```bash +# Full pipeline +pip install pytest pre-commit +# Set up CI/CD +# Add monitoring +``` + +--- + +## **Tool Selection Decision Tree** + +**For Network Engineering:** +- **Primary**: Jinja2 + PyYAML + Netmiko + Ansible +- **Documentation**: MkDocs + Mermaid +- **Advanced**: NetBox + Nornir + +**For Cloud/DevOps:** +- **Primary**: Jinja2 + PyYAML + Terraform + Helm +- **Multi-Cloud**: Boto3 + Azure SDK + Google Cloud +- **Advanced**: Kubernetes Python client + ArgoCD + +**For Sales/Proposals:** +- **Primary**: Jinja2 + PyYAML + ReportLab/python-docx +- **Advanced**: LaTeX + Pandoc +- **Integration**: CRM APIs (Salesforce, HubSpot) + +--- + +## **Pro Tips for Tool Selection** + +1. **Start minimal**: Jinja2 + PyYAML + Click gets you 80% of the value +2. **Add incrementally**: Don't boil the ocean—add tools as you hit specific limitations +3. **Standardize data formats**: YAML for configs, JSON for APIs, CSV for bulk data +4. **Version everything**: Templates, data, and generated outputs +5. **Test templates like code**: Unit tests prevent production surprises + +**Want to dive deep on any specific toolchain?** Pick your primary use case (network, cloud, sales) and I'll show you the exact setup with working examples. + +--- + Now you have an **interactive Jinja2 playground** that brings your document to life! This lets you experiment with all the concepts from your data dump in real-time. ## **What Makes You Dangerous with Jinja2**