From d5ed99d14d466d319eb606adb03f961c00bdc71a Mon Sep 17 00:00:00 2001 From: medusa Date: Wed, 6 Aug 2025 05:48:11 -0500 Subject: [PATCH] Update tech_docs/docker_compose_guide.md --- tech_docs/docker_compose_guide.md | 186 ++++++++++++++++++++++++++++++ 1 file changed, 186 insertions(+) diff --git a/tech_docs/docker_compose_guide.md b/tech_docs/docker_compose_guide.md index 2bd99fd..65d2383 100644 --- a/tech_docs/docker_compose_guide.md +++ b/tech_docs/docker_compose_guide.md @@ -1,3 +1,189 @@ +```plaintext +# ***The Deterministic Serendipity Guide*** +## Using Jinja2 to Rule a Pure-YAML Kingdom +*(Private reference v1.0 – for your eyes only)* + +--- + +### 0. Executive Purpose +Demonstrate how **Jinja2 becomes the single source of truth** in a closed system where +- **YAML is the only artifact** (input, intermediate, output) +- **Objects are pure data** (no code, no imports) +- **Determinism is absolute** (same seed → same bits) +- **Serendipity is guaranteed** (drop any new `.yaml` and re-render) + +--- + +### 1. Kingdom Layout (single directory tree) + +``` +serendipity/ +├─ kingdom.yml # the royal charter +├─ objects/ # every object you will ever need +│ ├─ _manifest.yml # optional index +│ ├─ unicorn.yaml +│ ├─ dragon.yaml +│ └─ … +└─ rules/ + └─ resolve.j2 # the only executable artifact +``` + +--- + +### 2. The Royal Charter (`kingdom.yml`) + +```yaml +kingdom: + # determinism knobs + seed: 42 + rng: + engine: Random + module: random + magic: + engine: Magic + module: hashlib + + # path conventions + objects: "objects/**/*.yaml" # glob pattern + rules: "rules/resolve.j2" + + # post-processing + exports: + - format: yaml + to: stdout + - format: yaml + to: out/kingdom_rendered.yaml +``` + +Nothing outside this file ever changes the deterministic output except the **seed**. + +--- + +### 3. Object Contract (what your YAML may contain) + +Every object **MUST** be valid YAML and **MAY** include Jinja2 expressions inside **quoted** scalars only. + +```yaml +# objects/unicorn.yaml +id: unicorn +color: "{{ rng.choice(['silver','iridescent']) }}" +blessing: "{{ magic.uuid4() }}" +``` + +Important: +- Expressions are **strings** → safe for YAML parsers. +- No top-level keys named `jinja2` or `__` (reserved for rulebook). + +--- + +### 4. The Single Rulebook (`rules/resolve.j2`) + +```jinja2 +{#- 1. Bootstrap deterministic engines -#} +{% set rng = load_rng(kingdom.rng) %} +{% set magic = load_magic(kingdom.magic) %} + +{#- 2. Load and expand every object -#} +{% set court = [] %} +{% for path in glob(kingdom.objects) %} + {% set raw = read_file(path) %} + {% set yaml = from_yaml(render_str(raw, rng=rng, magic=magic)) %} + {{ court.append(yaml) or '' }} +{% endfor %} + +{#- 3. Emit final deterministic YAML -#} +{{ court | to_yaml }} +``` + +Helper filters (`load_rng`, `load_magic`, `glob`, `read_file`, `render_str`) are registered once at startup; **they never change** and are **not serialisable**, so they cannot leak into objects. + +--- + +### 5. Bootstrap Script (one-time, never edited) + +```python +#!/usr/bin/env python3 +# bootstrap.py (kept outside kingdom/) +import os, yaml, jinja2, glob, hashlib, random + +# deterministic seed from kingdom.yml +meta = yaml.safe_load(open('kingdom.yml')) +random.seed(meta['kingdom']['seed']) + +# Jinja2 env +env = jinja2.Environment( + loader=jinja2.FileSystemLoader('rules'), + finalize=lambda x: '' if x is None else x, +) +env.filters.update({ + 'to_yaml': lambda d: yaml.dump(d, sort_keys=True), +}) +env.globals.update({ + 'load_rng': lambda cfg: random, + 'load_magic': lambda cfg: type('Magic', (), {'uuid4': lambda: hashlib.md5(os.urandom(8)).hexdigest()[:8]})(), + 'glob': lambda p: glob.glob(p, recursive=True), + 'read_file': lambda f: open(f).read(), + 'render_str': lambda s, **kw: jinja2.Template(s, undefined=jinja2.StrictUndefined).render(**kw), +}) + +template = env.get_template('resolve.j2') +print(env.from_string(template.render(kingdom=meta['kingdom'])).render()) +``` + +Run once → deterministic stream of bytes. +Commit the stream if you need reproducibility without Python. + +--- + +### 6. Deep Powers Inside Jinja2 (exploited here) + +| Power | Usage in Kingdom | +|---|---| +| **StrictUndefined** | Every undefined var raises → no silent typos | +| **Custom globals/filters** | Inject rng, magic, filesystem helpers | +| **Macro imports** | `{% from 'macros.j2' import mutate %}` inside object templates | +| **Whitespace control** | `{%- … -%}` keeps emitted YAML clean | +| **Recursive rendering** | Object YAML contains Jinja2 that is re-parsed | +| **Immutable context** | No `{% set %}` leaks between object renders | + +--- + +### 7. Serendipity Workflow (non-breaking) + +1. Drop `objects/new_artifact.yaml` +2. Re-run `python bootstrap.py` +3. Diff the two deterministic outputs → only deltas from the new object appear. +4. Version-control the diff, not the code. + +--- + +### 8. Security & Reproducibility Checklist + +- **No Python code inside YAML** → safe for linters like yamllint. +- **Seed is single source of entropy** → store it in a repo tag for audit. +- **Read-only filesystem** → templates and bootstrap script are immutable at runtime. +- **One-way data flow**: YAML → Jinja2 → expanded YAML → stdout/file. +- **No external network calls** → reproducible in air-gapped CI. + +--- + +### 9. Single-Command Invocation (CI / Makefile) + +```makefile +render: + @python bootstrap.py > kingdom_rendered.yaml + @echo "SHA256: $$(sha256sum kingdom_rendered.yaml)" +``` + +--- + +### 10. Closing Axiom + +> In this kingdom the **template is the constitution**, +> the seed is the crown jewel, +> and every `.yaml` is a citizen whose fate is **deterministically serendipitous**. +``` + Here is a **drop-in Jinja2 template skeleton** that turns the entire service stanza into variables you can override at render-time (`docker compose -f <(jinja2 compose.yml.j2 env.yml)` or `docker compose config`). You only need to set the variables you care about; everything else falls back to safe blanks or defaults.