Update tech_docs/docker_compose_guide.md
This commit is contained in:
@@ -1,3 +1,189 @@
|
||||
```plaintext
|
||||
# ***The Deterministic Serendipity Guide***
|
||||
## Using Jinja2 to Rule a Pure-YAML Kingdom
|
||||
*(Private reference v1.0 – for your eyes only)*
|
||||
|
||||
---
|
||||
|
||||
### 0. Executive Purpose
|
||||
Demonstrate how **Jinja2 becomes the single source of truth** in a closed system where
|
||||
- **YAML is the only artifact** (input, intermediate, output)
|
||||
- **Objects are pure data** (no code, no imports)
|
||||
- **Determinism is absolute** (same seed → same bits)
|
||||
- **Serendipity is guaranteed** (drop any new `.yaml` and re-render)
|
||||
|
||||
---
|
||||
|
||||
### 1. Kingdom Layout (single directory tree)
|
||||
|
||||
```
|
||||
serendipity/
|
||||
├─ kingdom.yml # the royal charter
|
||||
├─ objects/ # every object you will ever need
|
||||
│ ├─ _manifest.yml # optional index
|
||||
│ ├─ unicorn.yaml
|
||||
│ ├─ dragon.yaml
|
||||
│ └─ …
|
||||
└─ rules/
|
||||
└─ resolve.j2 # the only executable artifact
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 2. The Royal Charter (`kingdom.yml`)
|
||||
|
||||
```yaml
|
||||
kingdom:
|
||||
# determinism knobs
|
||||
seed: 42
|
||||
rng:
|
||||
engine: Random
|
||||
module: random
|
||||
magic:
|
||||
engine: Magic
|
||||
module: hashlib
|
||||
|
||||
# path conventions
|
||||
objects: "objects/**/*.yaml" # glob pattern
|
||||
rules: "rules/resolve.j2"
|
||||
|
||||
# post-processing
|
||||
exports:
|
||||
- format: yaml
|
||||
to: stdout
|
||||
- format: yaml
|
||||
to: out/kingdom_rendered.yaml
|
||||
```
|
||||
|
||||
Nothing outside this file ever changes the deterministic output except the **seed**.
|
||||
|
||||
---
|
||||
|
||||
### 3. Object Contract (what your YAML may contain)
|
||||
|
||||
Every object **MUST** be valid YAML and **MAY** include Jinja2 expressions inside **quoted** scalars only.
|
||||
|
||||
```yaml
|
||||
# objects/unicorn.yaml
|
||||
id: unicorn
|
||||
color: "{{ rng.choice(['silver','iridescent']) }}"
|
||||
blessing: "{{ magic.uuid4() }}"
|
||||
```
|
||||
|
||||
Important:
|
||||
- Expressions are **strings** → safe for YAML parsers.
|
||||
- No top-level keys named `jinja2` or `__` (reserved for rulebook).
|
||||
|
||||
---
|
||||
|
||||
### 4. The Single Rulebook (`rules/resolve.j2`)
|
||||
|
||||
```jinja2
|
||||
{#- 1. Bootstrap deterministic engines -#}
|
||||
{% set rng = load_rng(kingdom.rng) %}
|
||||
{% set magic = load_magic(kingdom.magic) %}
|
||||
|
||||
{#- 2. Load and expand every object -#}
|
||||
{% set court = [] %}
|
||||
{% for path in glob(kingdom.objects) %}
|
||||
{% set raw = read_file(path) %}
|
||||
{% set yaml = from_yaml(render_str(raw, rng=rng, magic=magic)) %}
|
||||
{{ court.append(yaml) or '' }}
|
||||
{% endfor %}
|
||||
|
||||
{#- 3. Emit final deterministic YAML -#}
|
||||
{{ court | to_yaml }}
|
||||
```
|
||||
|
||||
Helper filters (`load_rng`, `load_magic`, `glob`, `read_file`, `render_str`) are registered once at startup; **they never change** and are **not serialisable**, so they cannot leak into objects.
|
||||
|
||||
---
|
||||
|
||||
### 5. Bootstrap Script (one-time, never edited)
|
||||
|
||||
```python
|
||||
#!/usr/bin/env python3
|
||||
# bootstrap.py (kept outside kingdom/)
|
||||
import os, yaml, jinja2, glob, hashlib, random
|
||||
|
||||
# deterministic seed from kingdom.yml
|
||||
meta = yaml.safe_load(open('kingdom.yml'))
|
||||
random.seed(meta['kingdom']['seed'])
|
||||
|
||||
# Jinja2 env
|
||||
env = jinja2.Environment(
|
||||
loader=jinja2.FileSystemLoader('rules'),
|
||||
finalize=lambda x: '' if x is None else x,
|
||||
)
|
||||
env.filters.update({
|
||||
'to_yaml': lambda d: yaml.dump(d, sort_keys=True),
|
||||
})
|
||||
env.globals.update({
|
||||
'load_rng': lambda cfg: random,
|
||||
'load_magic': lambda cfg: type('Magic', (), {'uuid4': lambda: hashlib.md5(os.urandom(8)).hexdigest()[:8]})(),
|
||||
'glob': lambda p: glob.glob(p, recursive=True),
|
||||
'read_file': lambda f: open(f).read(),
|
||||
'render_str': lambda s, **kw: jinja2.Template(s, undefined=jinja2.StrictUndefined).render(**kw),
|
||||
})
|
||||
|
||||
template = env.get_template('resolve.j2')
|
||||
print(env.from_string(template.render(kingdom=meta['kingdom'])).render())
|
||||
```
|
||||
|
||||
Run once → deterministic stream of bytes.
|
||||
Commit the stream if you need reproducibility without Python.
|
||||
|
||||
---
|
||||
|
||||
### 6. Deep Powers Inside Jinja2 (exploited here)
|
||||
|
||||
| Power | Usage in Kingdom |
|
||||
|---|---|
|
||||
| **StrictUndefined** | Every undefined var raises → no silent typos |
|
||||
| **Custom globals/filters** | Inject rng, magic, filesystem helpers |
|
||||
| **Macro imports** | `{% from 'macros.j2' import mutate %}` inside object templates |
|
||||
| **Whitespace control** | `{%- … -%}` keeps emitted YAML clean |
|
||||
| **Recursive rendering** | Object YAML contains Jinja2 that is re-parsed |
|
||||
| **Immutable context** | No `{% set %}` leaks between object renders |
|
||||
|
||||
---
|
||||
|
||||
### 7. Serendipity Workflow (non-breaking)
|
||||
|
||||
1. Drop `objects/new_artifact.yaml`
|
||||
2. Re-run `python bootstrap.py`
|
||||
3. Diff the two deterministic outputs → only deltas from the new object appear.
|
||||
4. Version-control the diff, not the code.
|
||||
|
||||
---
|
||||
|
||||
### 8. Security & Reproducibility Checklist
|
||||
|
||||
- **No Python code inside YAML** → safe for linters like yamllint.
|
||||
- **Seed is single source of entropy** → store it in a repo tag for audit.
|
||||
- **Read-only filesystem** → templates and bootstrap script are immutable at runtime.
|
||||
- **One-way data flow**: YAML → Jinja2 → expanded YAML → stdout/file.
|
||||
- **No external network calls** → reproducible in air-gapped CI.
|
||||
|
||||
---
|
||||
|
||||
### 9. Single-Command Invocation (CI / Makefile)
|
||||
|
||||
```makefile
|
||||
render:
|
||||
@python bootstrap.py > kingdom_rendered.yaml
|
||||
@echo "SHA256: $$(sha256sum kingdom_rendered.yaml)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 10. Closing Axiom
|
||||
|
||||
> In this kingdom the **template is the constitution**,
|
||||
> the seed is the crown jewel,
|
||||
> and every `.yaml` is a citizen whose fate is **deterministically serendipitous**.
|
||||
```
|
||||
|
||||
Here is a **drop-in Jinja2 template skeleton** that turns the entire service stanza into variables you can override at render-time (`docker compose -f <(jinja2 compose.yml.j2 env.yml)` or `docker compose config`).
|
||||
You only need to set the variables you care about; everything else falls back to safe blanks or defaults.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user