Episode goal: show process management end to end — the brain/muscle split, declaring processes as data, generating the deployment, the three worker methods you write, the durable loop, exactly-once guarantees, and running it. Built around the process deployment tutorial and the p1 sample.
Cold open — the problem
On screen: one box straining, then fanning into many sub-jobs, then a crash. Voiceover: “Some work doesn’t fit a single request. A long job fans out into many, done only when all of them finish — and the naive way, a crash halfway strands everything. Chenile declares it as data, generates the plumbing, and survives the crash.”
The brain and the muscle
On screen: a calm server box (state machine · outbox · triggers) and a busy worker box (stateless). Voiceover: “Split the work in two. A brain that owns the state and never loses it; a muscle that does the work, stateless, run as many as you like. In p1 that’s two deployables — one server, one worker — with one Kafka topic and a callback between them.”
Declare it as data
On screen:
defs.json — batch (non-leaf) and item (leaf). “Leaf or not decides the worker: a leaf executes, a non-leaf splits and aggregates. Typed fields become typed objects. Each definition names its Kafka topic. Cron lives in its own file, seeded off.”Generate the deployment
Demo:
jgen.sh -g chenile-process-management then the generated output/p1/ reactor. “One blueprint composes smaller ones — a server, a headless Kafka worker-listener, one worker JAR per definition, and two mini-monoliths. The server holds the state and ships no business logic.”Three methods
On screen:
BatchSplitter, ItemExecutor, BatchAggregator. “The only code you write. The splitter makes a typed payload per child; the executor turns input into output; the aggregator folds children into AggregatedResult<ItemOutput>, which Chenile ships. Three methods — no dispatch, no retries, no plumbing.”The durable loop
On screen: the animated schematic, step by step. “A trigger commits durably and returns 202 before any work runs. The batch enters the state machine; its next step is written to the outbox in the same transaction. A poller publishes to Kafka; the worker runs your splitter and calls back through the registry. Three items execute over the same loop, then the batch aggregates, reaches PROCESSED, and emits completion.”
Exactly-once, crash-proof
Lower third: at-least-once in, exactly-once out. “Kafka can deliver twice. Three guards — a unique enqueue key, claim-token fencing, and effect-level receipts — make the effects exactly-once: a replayed child counts once, a replayed completion makes no duplicate. Crash the worker and nothing is lost. State and its consequences commit together, or not at all.”
Run it and watch it
Demo:
./run-server.sh, ./run-workers.sh, then curl -X POST /trigger. “Start the server, then the worker — it joins a consumer group, so more throughput is just more workers. Fire a batch with one call, watch the tree fill in process-ui, and query every causal edge with /trace while the outbox drains in the H2 console.”Recap
“A brain, a muscle, one topic — declared as data, durable by default. The full tutorial and the p1 sample take it further. Go build one.”