571 lines
17 KiB
Markdown
571 lines
17 KiB
Markdown
Below is a “cheat-sheet” style upgrade to the playbook you already have.
|
||
It is organized by *pain points* I see teams hit once they move past the first 5–10 devices.
|
||
Everything is copy-paste ready (CLI one-liners, tiny Python/Go helpers, Ansible snippets).
|
||
If you only have 15 min right now, jump to the “Tonight” section at the end.
|
||
|
||
────────────────────────────────────────────────────────
|
||
1. YAML → Jinja2 → Device in <10 s
|
||
────────────────────────────────────────────────────────
|
||
Problem: you still open an editor to change a variable file.
|
||
Fix: keep YAML inside Git and treat it as the *single source of truth*.
|
||
|
||
A. Git-based variable loader (avoids “where did I save that file?”)
|
||
```bash
|
||
# vars live in ./inventory/host_vars/$hostname.yml
|
||
git ls-files -z '*.yml' | xargs -0 yamllint # fast lint
|
||
```
|
||
B. One-shot render + diff
|
||
```bash
|
||
#!/usr/bin/env bash
|
||
host=$1
|
||
jinja2 templates/ios_base.j2 \
|
||
inventory/host_vars/${host}.yml \
|
||
--format yaml \
|
||
-o /tmp/${host}.cfg
|
||
colordiff -u <(ssh $host show run) /tmp/${host}.cfg | less -R
|
||
```
|
||
C. Atomic push with Ansible “deploy_if_changed”
|
||
```yaml
|
||
- name: Deploy only if rendered differs
|
||
ios_config:
|
||
src: "/tmp/{{ inventory_hostname }}.cfg"
|
||
vars:
|
||
ansible_check_mode: false
|
||
when: >
|
||
lookup('pipe', 'diff -q /tmp/{{ inventory_hostname }}.cfg
|
||
<(ssh {{ inventory_hostname }} show run)')
|
||
is not search('identical')
|
||
```
|
||
────────────────────────────────────────────────────────
|
||
2. Multi-vendor, one variable set
|
||
────────────────────────────────────────────────────────
|
||
Trick: store *capability* flags instead of vendor syntax.
|
||
|
||
vars/host_vars/leaf01.yml
|
||
```yaml
|
||
interfaces:
|
||
- name: Ethernet1/1
|
||
port_type: access
|
||
vlan: 200
|
||
poe: true
|
||
mtu: 9214
|
||
```
|
||
|
||
templates/shared/interface_capabilities.j2
|
||
```jinja2
|
||
{# Works on IOS-XE, Junos, EOS #}
|
||
{% set vendor = hostvars[inventory_hostname].vendor %}
|
||
{% for i in interfaces %}
|
||
interface {{ i.name | re_replace('^Eth','') if vendor=='arista' else
|
||
i.name | re_replace('^Eth','ge-') if vendor=='juniper' else
|
||
i.name }}
|
||
{% if i.port_type == 'access' %}
|
||
switchport mode access
|
||
switchport access vlan {{ i.vlan }}
|
||
{% endif %}
|
||
{% if i.poe and (vendor == 'cisco' or vendor == 'arista') %}
|
||
power inline never
|
||
{% endif %}
|
||
mtu {{ i.mtu }}
|
||
!
|
||
{% endfor %}
|
||
```
|
||
|
||
────────────────────────────────────────────────────────
|
||
3. NetBox as a live inventory
|
||
────────────────────────────────────────────────────────
|
||
Inventory plugin (ships with Ansible ≥2.10) in `ansible.cfg`:
|
||
```
|
||
[inventory]
|
||
enable_plugins = netbox.netbox.nb_inventory
|
||
```
|
||
Group by site + role:
|
||
```yaml
|
||
# group_vars/netbox_site_edge.yml
|
||
ntp_servers: ["10.0.0.10", "10.0.0.11"]
|
||
```
|
||
Now zero extra YAML for new sites.
|
||
|
||
────────────────────────────────────────────────────────
|
||
4. Rego in CI to block bad configs
|
||
────────────────────────────────────────────────────────
|
||
`.gitlab-ci.yml` (runs in <5 s):
|
||
```yaml
|
||
validate:
|
||
image: openpolicyagent/opa:0.59
|
||
script:
|
||
- opa test policies/
|
||
- >
|
||
find . -name "*.cfg" -print0 |
|
||
xargs -0 -I{} opa eval --fail-defined --format pretty
|
||
--data policies/network.rego --input {} "data.network.deny[_]"
|
||
```
|
||
|
||
Example policy (`policies/network.rego`):
|
||
```rego
|
||
package network
|
||
deny[msg] {
|
||
input.kind == "interface"
|
||
input.mtu > 9216
|
||
msg := sprintf("MTU %d too high on %s", [input.mtu, input.name])
|
||
}
|
||
```
|
||
|
||
────────────────────────────────────────────────────────
|
||
5. Secrets without Ansible Vault pain
|
||
────────────────────────────────────────────────────────
|
||
Use `sops` + age (age is easier than GPG).
|
||
|
||
```bash
|
||
# encrypt vars/secrets.yml
|
||
sops --age $(cat ~/.age/key.txt | grep public | cut -d' ' -f4) \
|
||
--encrypt vars/secrets.yml > vars/secrets.enc.yml
|
||
```
|
||
Ansible automatically decrypts when `ansible_sops_enabled=true`.
|
||
|
||
────────────────────────────────────────────────────────
|
||
6. Tonight → Next week → Go Pro (15 min each)
|
||
────────────────────────────────────────────────────────
|
||
Tonight (literally 15 min):
|
||
```bash
|
||
pipx install jinja2-cli yq
|
||
git init network-auto && cd network-auto
|
||
mkdir templates vars
|
||
cat > vars/lab.yml <<EOF
|
||
vlans:
|
||
- id: 100
|
||
name: STAFF
|
||
EOF
|
||
cat > templates/ios_vlan.j2 <<'EOF'
|
||
{% for v in vlans %}
|
||
vlan {{ v.id }}
|
||
name {{ v.name }}
|
||
{% endfor %}
|
||
EOF
|
||
jinja2 templates/ios_vlan.j2 vars/lab.yml
|
||
```
|
||
You now have version-controlled VLANs.
|
||
|
||
Next week:
|
||
- Replace static `vars/lab.yml` with NetBox inventory plugin.
|
||
- Add a single playbook that runs against “site:lab” and pushes the VLANs.
|
||
|
||
Go Pro:
|
||
- Add Rego policy to reject VLAN 1 or duplicate names.
|
||
- GitLab CI job auto-pushes when MR merges (use Ansible’s `ios_config` with `backup: yes` so you can rollback).
|
||
|
||
────────────────────────────────────────────────────────
|
||
Quick reference card
|
||
────────────────────────────────────────────────────────
|
||
Render: jinja2 tpl.j2 vars.yml --format yaml
|
||
Lint: yamllint vars/
|
||
Test: opa test policies/
|
||
Encrypt: sops -e vars/secrets.yml
|
||
Diff: colordiff -u live.cfg new.cfg
|
||
|
||
Copy the snippets above into your repo and you’re already ahead of most teams.
|
||
|
||
---
|
||
|
||
This is **exactly** the kind of tactical, network-focused Jinja2 guide I wish I had when I started automating configs. Let’s sharpen it further with battle-tested patterns and CLI-ready examples. Here’s your playbook:
|
||
|
||
---
|
||
|
||
### **1. Immediate Wins: Stop Hand-Editing Configs**
|
||
#### **A. Interface Templating (Multi-Vendor)**
|
||
**Problem:** Vendor-specific syntax for the same logical interface.
|
||
**Solution:** One YAML → Cisco/Juniper/Arista templates.
|
||
|
||
**`vars/interface.yml`**
|
||
```yaml
|
||
interfaces:
|
||
- name: Eth1/1
|
||
description: "Server Farm Uplink"
|
||
mode: trunk
|
||
vlans: [100, 200]
|
||
mtu: 9214
|
||
```
|
||
|
||
**`templates/cisco_interface.j2`**
|
||
```jinja
|
||
interface {{ interface.name }}
|
||
description {{ interface.description }}
|
||
{% if interface.mode == 'trunk' %}
|
||
switchport mode trunk
|
||
switchport trunk allowed vlan {{ interface.vlans | join(',') }}
|
||
{% endif %}
|
||
mtu {{ interface.mtu }}
|
||
```
|
||
|
||
**`templates/juniper_interface.j2`**
|
||
```jinja
|
||
interfaces {
|
||
{{ interface.name | replace('Eth','ge-') }} {
|
||
description "{{ interface.description }}";
|
||
{% if interface.mode == 'trunk' %}
|
||
unit 0 {
|
||
family ethernet-switching {
|
||
vlan members [ {{ interface.vlans | join(' ') }} ];
|
||
}
|
||
}
|
||
{% endif %}
|
||
mtu {{ interface.mtu }};
|
||
}
|
||
}
|
||
```
|
||
|
||
**Render Both:**
|
||
```bash
|
||
# Cisco
|
||
jinja2 templates/cisco_interface.j2 vars/interface.yml
|
||
# Juniper
|
||
jinja2 templates/juniper_interface.j2 vars/interface.yml
|
||
```
|
||
|
||
---
|
||
|
||
#### **B. BGP Config Generation (With Error-Prone Logic)**
|
||
**Problem:** Complex BGP configs with neighbor policies.
|
||
**Solution:** Template + YAML with validation.
|
||
|
||
**`vars/bgp.yml`**
|
||
```yaml
|
||
bgp:
|
||
asn: 65001
|
||
neighbors:
|
||
- ip: 10.0.0.2
|
||
remote_as: 65002
|
||
policies: [ "PREVENT_LEAK" ]
|
||
- ip: 192.168.1.1
|
||
remote_as: 65123
|
||
policies: [ "CUSTOMER_ROUTES" ]
|
||
```
|
||
|
||
**`templates/bgp.j2`**
|
||
```jinja
|
||
router bgp {{ bgp.asn }}
|
||
{% for neighbor in bgp.neighbors %}
|
||
neighbor {{ neighbor.ip }} remote-as {{ neighbor.remote_as }}
|
||
{% if "PREVENT_LEAK" in neighbor.policies %}
|
||
neighbor {{ neighbor.ip }} route-map BLOCK_DEFAULT in
|
||
{% endif %}
|
||
{% endfor %}
|
||
```
|
||
|
||
**Key Trick:** Use `| selectattr` to filter neighbors:
|
||
```jinja
|
||
{% for neighbor in bgp.neighbors | selectattr("policies", "contains", "CUSTOMER_ROUTES") %}
|
||
...customer-specific config...
|
||
{% endfor %}
|
||
```
|
||
|
||
---
|
||
|
||
### **2. Advanced: Network-as-Code Patterns**
|
||
#### **A. Generate Device-Specific Configs from NetBox API**
|
||
**Problem:** NetBox has device data, but CLI configs are manual.
|
||
**Solution:** Fetch NetBox data → Jinja2.
|
||
|
||
**`fetch_netbox_data.py`**
|
||
```python
|
||
import requests
|
||
import json
|
||
|
||
devices = requests.get("https://netbox/api/dcim/devices/").json()
|
||
with open('vars/netbox_devices.yml', 'w') as f:
|
||
json.dump(devices, f)
|
||
```
|
||
|
||
**`templates/netbox_cisco.j2`**
|
||
```jinja
|
||
hostname {{ device.name }}
|
||
{% for iface in device.interfaces %}
|
||
interface {{ iface.name }}
|
||
description {{ iface.description }}
|
||
{% endfor %}
|
||
```
|
||
|
||
**Render All Devices:**
|
||
```bash
|
||
python fetch_netbox_data.py
|
||
jinja2 templates/netbox_cisco.j2 vars/netbox_devices.yml
|
||
```
|
||
|
||
---
|
||
|
||
#### **B. Auto-Generate Port-Channel Configs (LACP)**
|
||
**Problem:** Port-channel members change frequently.
|
||
**Solution:** Dynamic YAML + template.
|
||
|
||
**`vars/portchannel.yml`**
|
||
```yaml
|
||
portchannel:
|
||
id: 10
|
||
members: [ "Eth1/1", "Eth1/2" ]
|
||
mode: "active"
|
||
vlan: 100
|
||
```
|
||
|
||
**`templates/portchannel.j2`**
|
||
```jinja
|
||
interface Port-channel{{ portchannel.id }}
|
||
channel-group {{ portchannel.id }} mode {{ portchannel.mode }}
|
||
switchport access vlan {{ portchannel.vlan }}
|
||
|
||
{% for member in portchannel.members %}
|
||
interface {{ member }}
|
||
channel-group {{ portchannel.id }} mode {{ portchannel.mode }}
|
||
{% endfor %}
|
||
```
|
||
|
||
---
|
||
|
||
### **3. Pro Tooling: Network-Specific Tricks**
|
||
#### **A. Ansible + Jinja2 for Zero-Touch Deployment**
|
||
**`ansible_playbook.yml`**
|
||
```yaml
|
||
- name: Push Interface Configs
|
||
hosts: switches
|
||
tasks:
|
||
- name: Generate Config
|
||
template:
|
||
src: templates/cisco_interface.j2
|
||
dest: /tmp/{{ inventory_hostname }}.cfg
|
||
- name: Deploy Config
|
||
cisco.ios.ios_config:
|
||
src: /tmp/{{ inventory_hostname }}.cfg
|
||
```
|
||
|
||
#### **B. Validate Configs Before Deployment**
|
||
```bash
|
||
# Check for missing variables
|
||
jinja2 --format vars:yaml templates/interface.j2 vars/device.yml
|
||
|
||
# Dry-run with Ansible
|
||
ansible-playbook playbook.yml --check
|
||
```
|
||
|
||
#### **C. Secret Management (Ansible Vault)**
|
||
```yaml
|
||
# Encrypted vars.yml
|
||
ansible-vault encrypt vars/secrets.yml
|
||
```
|
||
|
||
**Template:**
|
||
```jinja
|
||
username {{ vaulted_username }}
|
||
password {{ vaulted_password }}
|
||
```
|
||
|
||
---
|
||
|
||
### **4. Your Battle Plan**
|
||
1. **Start Tonight:**
|
||
- Pick 1 repetitive config (VLANs/ACLs) → template it.
|
||
- Render with `jinja2-cli` locally.
|
||
|
||
2. **Next Week:**
|
||
- Integrate with NetBox/Python to auto-generate configs.
|
||
- Use Ansible to push to devices.
|
||
|
||
3. **Go Pro:**
|
||
- Build a GitLab pipeline that auto-generates configs on NetBox changes.
|
||
|
||
**Example Repo Structure:**
|
||
```
|
||
network-automation/
|
||
├── templates/ # .j2 files
|
||
├── vars/ # .yml device vars
|
||
├── scripts/ # Python/Ansible helpers
|
||
└── outputs/ # Rendered configs
|
||
```
|
||
|
||
**Key Takeaway:** Jinja2 turns network configs from **manual CLI work** into **version-controlled code**. The more you template now, the more time you save later.
|
||
|
||
---
|
||
|
||
Great pivot! **Rego** (Open Policy Agent’s policy language) and **Jinja2** serve different but complementary roles in infrastructure automation. Let’s break down how they compare, overlap, and where you’d use each—with gritty examples tailored to infrastructure/security use cases.
|
||
|
||
---
|
||
|
||
### **1. Quick Comparison: Jinja2 vs. Rego**
|
||
| Feature | Jinja2 | Rego |
|
||
|---------|--------|------|
|
||
| **Primary Use** | Templating (generate configs/docs) | Policy enforcement (validate inputs) |
|
||
| **Input** | YAML/JSON/CSV | JSON/YAML (often from APIs) |
|
||
| **Output** | Rendered text (configs, CLI commands) | Policy decisions (allow/deny + detailed reasons) |
|
||
| **Context** | "Generate this network config" | "Is this network config compliant?" |
|
||
| **Key Strength** | Flexibility in text generation | Logic-based evaluation with auditing |
|
||
|
||
---
|
||
|
||
### **2. Where Jinja2 and Rego Overlap**
|
||
Both operate on structured data (YAML/JSON), but solve different problems in the pipeline:
|
||
|
||
#### **Example Workflow: Network Change Automation**
|
||
1. **Jinja2**: Generates a candidate BGP config from YAML variables.
|
||
```jinja
|
||
router bgp {{ bgp.asn }}
|
||
neighbor {{ neighbor.ip }} remote-as {{ neighbor.asn }}
|
||
```
|
||
2. **Rego**: Validates the generated config *before* deployment.
|
||
```rego
|
||
# Prevent BGP peers in untrusted ASNs
|
||
deny[msg] {
|
||
input.kind == "bgp_config"
|
||
not input.neighbor.asn in trusted_asns
|
||
msg := sprintf("BGP peer ASN %v is not trusted", [input.neighbor.asn])
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
### **3. Rego Use Cases (Where It Shines Over Jinja2)**
|
||
#### **A. Pre-Deployment Validation**
|
||
**Problem**: Ensure Jinja2-generated configs meet security/compliance rules.
|
||
**Rego Policy** (`policies/networking.rego`):
|
||
```rego
|
||
package networking
|
||
|
||
# Deny firewall rules that allow SSH from the internet
|
||
deny[msg] {
|
||
input.kind == "firewall_rule"
|
||
input.action == "allow"
|
||
input.port == 22
|
||
input.source == "0.0.0.0/0"
|
||
msg := "SSH must not be open to the internet!"
|
||
}
|
||
|
||
# Require VLAN descriptions for auditability
|
||
deny[msg] {
|
||
input.kind == "vlan_config"
|
||
not input.description
|
||
msg := "VLANs must have a description"
|
||
}
|
||
```
|
||
|
||
**How to Use**:
|
||
```bash
|
||
# Validate a Jinja2-rendered config against Rego
|
||
opa eval --data policies/networking.rego --input rendered_config.json "data.networking.deny"
|
||
```
|
||
|
||
---
|
||
|
||
#### **B. Multi-Cloud Policy Enforcement**
|
||
**Problem**: Enforce tagging standards across AWS/Azure/GCP.
|
||
**Rego Policy** (`policies/cloud.rego`):
|
||
```rego
|
||
package cloud
|
||
|
||
# Require 'CostCenter' tag on all resources
|
||
deny[msg] {
|
||
input.resource.tags["CostCenter"] == ""
|
||
msg := "All resources must have a CostCenter tag"
|
||
}
|
||
|
||
# Block public S3 buckets
|
||
deny[msg] {
|
||
input.kind == "aws_s3_bucket"
|
||
input.acl == "public-read"
|
||
msg := "Public S3 buckets are prohibited"
|
||
}
|
||
```
|
||
|
||
**Input (JSON from Terraform/Cloud API)**:
|
||
```json
|
||
{
|
||
"kind": "aws_s3_bucket",
|
||
"acl": "public-read",
|
||
"tags": { "CostCenter": "" }
|
||
}
|
||
```
|
||
|
||
**Output**:
|
||
```json
|
||
{
|
||
"deny": [
|
||
"All resources must have a CostCenter tag",
|
||
"Public S3 buckets are prohibited"
|
||
]
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
#### **C. Network Topology Auditing**
|
||
**Problem**: Validate that Jinja2-generated network designs follow best practices.
|
||
**Rego Policy** (`policies/topology.rego`):
|
||
```rego
|
||
package topology
|
||
|
||
# Core switches must have redundant links
|
||
deny[msg] {
|
||
input.device.role == "core"
|
||
count(input.device.links) < 2
|
||
msg := "Core devices must have redundant links"
|
||
}
|
||
|
||
# VLAN 1 should not be used (security best practice)
|
||
deny[msg] {
|
||
input.vlan.id == 1
|
||
msg := "VLAN 1 is prohibited for security reasons"
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
### **4. When to Use Jinja2 vs. Rego**
|
||
| Scenario | Tool | Why |
|
||
|----------|------|-----|
|
||
| Generating Cisco/Juniper configs | Jinja2 | Best at text templating. |
|
||
| Validating configs pre-deploy | Rego | Logic-based policy checks. |
|
||
| Creating API payloads | Jinja2 | Structure dynamic JSON/YAML. |
|
||
| Auditing existing infra | Rego | Query live state for violations. |
|
||
| Documenting network designs | Jinja2 | Render Markdown/HTML. |
|
||
| Enforcing security rules | Rego | Evaluate against compliance rules. |
|
||
|
||
---
|
||
|
||
### **5. Pro Tips for Rego in Infrastructure**
|
||
1. **Test Policies Early**: Use `opa test` to unit test your Rego rules.
|
||
```bash
|
||
opa test policies/ -v
|
||
```
|
||
2. **Integrate with CI/CD**:
|
||
```yaml
|
||
# GitLab CI example
|
||
validate_policy:
|
||
image: openpolicyagent/opa
|
||
script:
|
||
- opa eval --data policies/ --input $CHANGES "data.deny"
|
||
```
|
||
3. **Debug with `trace`**:
|
||
```rego
|
||
# Add to your policy to see evaluation steps
|
||
trace(sprintf("Input: %v", [input]))
|
||
```
|
||
|
||
---
|
||
|
||
### **6. Example End-to-End Flow**
|
||
1. **Jinja2 Generates Config**:
|
||
```bash
|
||
jinja2 bgp_template.j2 vars.yml > bgp_config.json
|
||
```
|
||
2. **Rego Validates It**:
|
||
```bash
|
||
opa eval --data policies/ --input bgp_config.json "data.networking.deny"
|
||
```
|
||
3. **Only deploy if Rego returns no denials**.
|
||
|
||
---
|
||
|
||
### **7. Key Takeaway**
|
||
- **Jinja2**: Your "build" tool (creates configs/scripts/docs).
|
||
- **Rego**: Your "guardrail" tool (ensures compliance/safety).
|
||
|
||
Together, they form a **powerful pipeline**:
|
||
`Jinja2 (generate) → Rego (validate) → Ansible/Terraform (deploy)`
|
||
|
||
Want a deep dive on a specific Rego use case (e.g., network security policies or cloud governance)? I can break it down with real code. |