Update tech_docs/docker_compose_guide.md

tech_docs/docker_compose_guide.md

@@ -1,3 +1,189 @@
+```plaintext
+# ***The Deterministic Serendipity Guide***  
+## Using Jinja2 to Rule a Pure-YAML Kingdom  
+*(Private reference v1.0 – for your eyes only)*
+
+---
+
+### 0. Executive Purpose
+Demonstrate how **Jinja2 becomes the single source of truth** in a closed system where  
+- **YAML is the only artifact** (input, intermediate, output)  
+- **Objects are pure data** (no code, no imports)  
+- **Determinism is absolute** (same seed → same bits)  
+- **Serendipity is guaranteed** (drop any new `.yaml` and re-render)
+
+---
+
+### 1. Kingdom Layout (single directory tree)
+
+```
+serendipity/
+├─ kingdom.yml        # the royal charter
+├─ objects/           # every object you will ever need
+│  ├─ _manifest.yml   # optional index
+│  ├─ unicorn.yaml
+│  ├─ dragon.yaml
+│  └─ …
+└─ rules/
+   └─ resolve.j2      # the only executable artifact
+```
+
+---
+
+### 2. The Royal Charter (`kingdom.yml`)
+
+```yaml
+kingdom:
+  # determinism knobs
+  seed: 42
+  rng:
+    engine: Random
+    module: random
+  magic:
+    engine: Magic
+    module: hashlib
+
+  # path conventions
+  objects: "objects/**/*.yaml"   # glob pattern
+  rules:   "rules/resolve.j2"
+
+  # post-processing
+  exports:
+    - format: yaml
+      to:   stdout
+    - format: yaml
+      to:   out/kingdom_rendered.yaml
+```
+
+Nothing outside this file ever changes the deterministic output except the **seed**.
+
+---
+
+### 3. Object Contract (what your YAML may contain)
+
+Every object **MUST** be valid YAML and **MAY** include Jinja2 expressions inside **quoted** scalars only.
+
+```yaml
+# objects/unicorn.yaml
+id: unicorn
+color: "{{ rng.choice(['silver','iridescent']) }}"
+blessing: "{{ magic.uuid4() }}"
+```
+
+Important:  
+- Expressions are **strings** → safe for YAML parsers.  
+- No top-level keys named `jinja2` or `__` (reserved for rulebook).
+
+---
+
+### 4. The Single Rulebook (`rules/resolve.j2`)
+
+```jinja2
+{#- 1. Bootstrap deterministic engines -#}
+{% set rng   = load_rng(kingdom.rng) %}
+{% set magic = load_magic(kingdom.magic) %}
+
+{#- 2. Load and expand every object -#}
+{% set court = [] %}
+{% for path in glob(kingdom.objects) %}
+  {% set raw  = read_file(path) %}
+  {% set yaml = from_yaml(render_str(raw, rng=rng, magic=magic)) %}
+  {{ court.append(yaml) or '' }}
+{% endfor %}
+
+{#- 3. Emit final deterministic YAML -#}
+{{ court | to_yaml }}
+```
+
+Helper filters (`load_rng`, `load_magic`, `glob`, `read_file`, `render_str`) are registered once at startup; **they never change** and are **not serialisable**, so they cannot leak into objects.
+
+---
+
+### 5. Bootstrap Script (one-time, never edited)
+
+```python
+#!/usr/bin/env python3
+# bootstrap.py  (kept outside kingdom/)
+import os, yaml, jinja2, glob, hashlib, random
+
+# deterministic seed from kingdom.yml
+meta = yaml.safe_load(open('kingdom.yml'))
+random.seed(meta['kingdom']['seed'])
+
+# Jinja2 env
+env = jinja2.Environment(
+    loader=jinja2.FileSystemLoader('rules'),
+    finalize=lambda x: '' if x is None else x,
+)
+env.filters.update({
+    'to_yaml': lambda d: yaml.dump(d, sort_keys=True),
+})
+env.globals.update({
+    'load_rng':   lambda cfg: random,
+    'load_magic': lambda cfg: type('Magic', (), {'uuid4': lambda: hashlib.md5(os.urandom(8)).hexdigest()[:8]})(),
+    'glob':       lambda p: glob.glob(p, recursive=True),
+    'read_file':  lambda f: open(f).read(),
+    'render_str': lambda s, **kw: jinja2.Template(s, undefined=jinja2.StrictUndefined).render(**kw),
+})
+
+template = env.get_template('resolve.j2')
+print(env.from_string(template.render(kingdom=meta['kingdom'])).render())
+```
+
+Run once → deterministic stream of bytes.  
+Commit the stream if you need reproducibility without Python.
+
+---
+
+### 6. Deep Powers Inside Jinja2 (exploited here)
+
+| Power | Usage in Kingdom |
+|---|---|
+| **StrictUndefined** | Every undefined var raises → no silent typos |
+| **Custom globals/filters** | Inject rng, magic, filesystem helpers |
+| **Macro imports** | `{% from 'macros.j2' import mutate %}` inside object templates |
+| **Whitespace control** | `{%- … -%}` keeps emitted YAML clean |
+| **Recursive rendering** | Object YAML contains Jinja2 that is re-parsed |
+| **Immutable context** | No `{% set %}` leaks between object renders |
+
+---
+
+### 7. Serendipity Workflow (non-breaking)
+
+1. Drop `objects/new_artifact.yaml`  
+2. Re-run `python bootstrap.py`  
+3. Diff the two deterministic outputs → only deltas from the new object appear.  
+4. Version-control the diff, not the code.
+
+---
+
+### 8. Security & Reproducibility Checklist
+
+- **No Python code inside YAML** → safe for linters like yamllint.  
+- **Seed is single source of entropy** → store it in a repo tag for audit.  
+- **Read-only filesystem** → templates and bootstrap script are immutable at runtime.  
+- **One-way data flow**: YAML → Jinja2 → expanded YAML → stdout/file.  
+- **No external network calls** → reproducible in air-gapped CI.
+
+---
+
+### 9. Single-Command Invocation (CI / Makefile)
+
+```makefile
+render:
+	@python bootstrap.py > kingdom_rendered.yaml
+	@echo "SHA256: $$(sha256sum kingdom_rendered.yaml)"
+```
+
+---
+
+### 10. Closing Axiom
+
+> In this kingdom the **template is the constitution**,  
+> the seed is the crown jewel,  
+> and every `.yaml` is a citizen whose fate is **deterministically serendipitous**.
+```
+
 Here is a **drop-in Jinja2 template skeleton** that turns the entire service stanza into variables you can override at render-time (`docker compose -f <(jinja2 compose.yml.j2 env.yml)` or `docker compose config`).  
 You only need to set the variables you care about; everything else falls back to safe blanks or defaults.