Hydrate Dynamic Element With Kindergarten
Design Log #106 — Hydrate Dynamic Elements with Kindergarten
Written for AI agents. See Log Methodology Note below for details.
Background
The element target's e() function uses Kindergarten to manage mixed children — static elements, forEach groups, and conditional groups each get their own KindergartenGroup. The getOffsetFor(group) method sums the sizes of preceding groups to compute the correct DOM insertion index. This ensures that adding a forEach item inserts at the right position among siblings.
The hydrate target's adoptElement does NOT use Kindergarten. It collects child updates/mounts but treats all children as a flat list. When hydrateForEach creates a Kindergarten, it creates it directly on the container element (the forEach wrapper <div class="list">), not on the parent that holds mixed content.
Related
- DL#102 Issue 8 (new) — forEach add item inserts at wrong position
- DL#104 — hydration test plan (6c forEach interactivity)
Problem
Page structure:
<div jay-coordinate="0">
<!-- parent with mixed content -->
<h1>ForEach Headless</h1>
<!-- static child (group 0) -->
<div class="list">...</div>
<!-- forEach item 1 (group 1) -->
<div class="list">...</div>
<!-- forEach item 2 (group 1) -->
<div class="list">...</div>
<!-- forEach item 3 (group 1) -->
<button>Add Item</button>
<!-- static child (group 2) -->
<button>Remove Last</button>
<!-- static child (group 3) -->
</div>
Current hydrate code:
adoptElement('0', {}, [
adoptText('0/0', (vs) => vs.title),
hydrateForEach('0', (vs) => vs.items, '_id', adoptItem, createItem),
adoptElement('0/2', {}, [], refAddButton()),
adoptElement('0/3', {}, [], refRemoveButton()),
]);
hydrateForEach creates a Kindergarten on the container element (<div jay-coordinate="0">), but with newGroup() at index 0. It doesn't account for the <h1> before it. When a new item is added, getOffsetFor(group) returns 0 (no preceding groups), so the item is inserted before the <h1> instead of after the existing forEach items.
Similarly, hydrateConditional inserts an anchor comment and uses insertBefore relative to the adopted element's sibling. If other children are added/removed around it, the anchor position may become incorrect.
Design
Unified approach: adoptDynamicElement with Kindergarten
When the compiler detects that an element's children include hydrateForEach or hydrateConditional, it emits adoptDynamicElement instead of adoptElement. This function creates a Kindergarten on the DOM element and assigns each child position to a KindergartenGroup.
Only elements with dynamic children use this. Static-only adoptElement calls remain unchanged (no Kindergarten overhead).
Accounting for static holes
The SSR DOM may contain static children that have no corresponding hydrate code. For example:
<div jay-coordinate="0">
<h1>Title</h1>
<!-- static, no hydrate code -->
<p>Description</p>
<!-- static, no hydrate code -->
<div class="list">...</div>
<!-- forEach item 1 -->
<div class="list">...</div>
<!-- forEach item 2 -->
<span jay-coordinate="0/4">count</span>
<!-- dynamic text -->
<button jay-coordinate="0/5">Add</button>
<!-- ref -->
</div>
The hydrate script is compiled from the pre-rendered jay-html — at compile time we don't know how many forEach items will exist at runtime. So we can't use absolute DOM position indices.
Solution: STATIC placeholder in children array. Each static child that doesn't need hydration is represented by a STATIC marker. This tells the Kindergarten "there's 1 DOM node here that I don't manage."
adoptDynamicElement('0', {}, [
STATIC, // h1 — 1 DOM node, no hydration
STATIC, // p — 1 DOM node, no hydration
hydrateForEach(...), // forEach — N DOM nodes
adoptText('0/4', ...), // dynamic text — 1 DOM node
adoptElement('0/5', {}, [], refAdd()), // button — 1 DOM node
])
At hydration time, the Kindergarten walks the parent's actual DOM children and assigns them to groups in order:
Template children: [STATIC] [STATIC] [forEach] [adoptText] [adoptElement]
↓ ↓ ↓ ↓ ↓
DOM children: [h1] [p] [item1][item2] [span] [button]
↑ N items ↑
Groups: [g0:1] [g1:1] [g2:2] [g3:1] [g4:1]
Each group type:
STATIC→ group with 1 node, consumed from DOM in orderhydrateForEach→ group with N nodes (N = initial items array length)hydrateConditional→ group with 0 or 1 nodes (based on SSR condition)adoptText/adoptElement→ group with 1 node
How it works — example
When forEach adds item3:
getOffsetFor(g2)= g0.size(1) + g1.size(1) = 2- Insert at position 2 + 2 = 4 (after item2, before span) ✓
When forEach removes item2:
g2.removeNode(item2)→ g2.size becomes 1getOffsetFor(g3)recalculates: 1 + 1 + 1 = 3 → span stays at correct position ✓
When conditional toggles false→true:
g_cond.ensureNode(dom, 0)→ g_cond.size becomes 1- Subsequent groups' offsets shift by +1 ✓
Changes needed
1. New function: adoptDynamicElement
/** Sentinel value representing a static DOM child that doesn't need hydration */
export const STATIC: BaseJayElement<any> = {
dom: null,
update: noopUpdate,
mount: noopMount,
unmount: noopMount,
__static: true,
};
export function adoptDynamicElement<ViewState>(
coordinate: string,
attributes: Attributes<ViewState>,
children: BaseJayElement<ViewState>[], // may include STATIC markers
ref?: PrivateRef<ViewState, BaseJayElement<ViewState>>,
): BaseJayElement<ViewState>;
Implementation:
- Resolve element by coordinate (same as
adoptElement) - Create
Kindergartenon the element - Walk
childrenarray and the parent's actual DOM children in parallel:STATICmarker → create group, consume 1 DOM node, register in group (never changes)hydrateForEachresult → create group, consume N DOM nodes (N = initial items count)hydrateConditionalresult → create group, consume 0 or 1 DOM nodes- Regular
adoptText/adoptElement/childCompHydrate→ create group, consume 1 DOM node
- Wire up updates/mounts/unmounts same as
adoptElement
2. hydrateForEach receives a KindergartenGroup
Instead of creating its own Kindergarten, it receives the group from the parent:
export function hydrateForEach<ViewState, Item>(
accessor: (vs: ViewState) => Item[],
trackBy: string,
adoptItem: () => BaseJayElement<Item>[],
createItem: (item: Item, id: string) => BaseJayElement<Item>,
group: KindergartenGroup, // from parent's Kindergarten
): BaseJayElement<ViewState>;
containerCoordinate parameter removed — the parent adoptDynamicElement already resolved the container and created the group.
3. hydrateConditional receives a KindergartenGroup
export function hydrateConditional<ViewState>(
condition: (vs: ViewState) => boolean,
adoptExisting: () => BaseJayElement<ViewState>,
createFallback: () => BaseJayElement<ViewState>,
group: KindergartenGroup,
): BaseJayElement<ViewState>;
- True at SSR: group has 1 child (the adopted element)
- False at SSR: group has 0 children
- Toggle true→false:
group.removeNode(dom), group.size becomes 0 - Toggle false→true:
group.ensureNode(dom, 0), group.size becomes 1
No anchor comments needed — Kindergarten offset system handles positioning. The old anchor-based approach is removed entirely.
Questions and Answers
Q1: Should all adoptElement calls use Kindergarten?
A: No (Option A). Only when children include dynamic groups. The compiler knows at compile time and emits adoptDynamicElement vs adoptElement accordingly.
Q2: How does the compiler know which children are dynamic?
A: The compiler already knows — forEach attribute → hydrateForEach, if attribute → hydrateConditional. It emits STATIC for template children that have no hydrate code (fully static elements).
Q3: Does this affect coordinate resolution? A: No. Coordinates are resolved via the coordinate map. Kindergarten only manages DOM positions for insertion/removal.
Q4: What about nested dynamic elements? A: A forEach item containing a conditional creates nested Kindergartens. Each level is independent — same pattern as the element target.
Implementation Plan
Phase 1: Runtime
- Add
DynamicChildGrouptype tohydrate.ts - Add
adoptDynamicElementfunction that creates Kindergarten and assigns groups - Modify
hydrateForEachto accept aKindergartenGroupinstead of creating its own Kindergarten - Modify
hydrateConditionalto accept an optionalKindergartenGroupand use it for DOM positioning
Phase 2: Compiler
- Detect when
adoptElementchildren include forEach or conditional - Emit
adoptDynamicElementinstead ofadoptElementfor those cases - Pass group references from parent to forEach/conditional children
Phase 3: Tests
- Update hydration tests for correct forEach add/remove positioning
- Test conditional toggle within mixed-content parent
- Test nested forEach + conditional
Verification Criteria
- forEach "Add Item" inserts at correct position (after existing items, before buttons)
- forEach "Remove Last" removes from correct position
- Conditional toggle doesn't displace siblings
- Nested dynamic elements work (forEach item with conditional)
- Static-only adoptElement unchanged (no Kindergarten overhead)
- All existing hydration tests pass
Implementation Results
Deviations from design
Deferred group assignment via _setGroup callback — The design proposed passing KindergartenGroup as a parameter to hydrateForEach/hydrateConditional. This doesn't work because JS argument evaluation order means these calls execute before adoptDynamicElement can create groups. Solution: hydrateForEach and hydrateConditional return a DynamicChild<ViewState> (extends BaseJayElement with a _setGroup callback). adoptDynamicElement creates groups and calls _setGroup on each dynamic child.
STATIC is a Symbol, not an object — The design proposed a BaseJayElement with __static: true. Implementation uses Symbol('STATIC') which is simpler and allows child === STATIC identity check instead of property access.
Code size reduction via adoptBase / collectChild helpers — adoptElement and adoptDynamicElement share coordinate resolution, ref wiring, and dynamic attribute binding via extracted adoptBase(). Child update/mount/unmount collection extracted into collectChild().
Additional bug fix: buildCoordinatePrefix coordinate mismatch
During integration testing with the fake-shop example, discovered that buildCoordinatePrefix (used by discoverHeadlessInstances to compute __headlessInstances keys) produced coordinates that didn't match assignCoordinates (used by server-element and hydrate targets for lookup).
Root cause: buildCoordinatePrefix computed child indices incorrectly:
- It counted
jay:xxxelements as DOM children, butassignCoordinatesskips them (childCounteris not incremented for headless directives) - When the headless instance was a direct child of a slowForEach div, this produced a spurious
/0index ("1/0/widget:0"instead of"1/widget:0") - When intermediate wrapper elements existed (e.g.,
<div class="product-card">), the child index was needed but relative to the wrapper's real DOM siblings
Fix: buildCoordinatePrefix now:
- Walks up from the element, stopping at the first
jayTrackByancestor - For intermediate elements between the instance and the jayTrackBy scope, adds their positional index (matching
assignCoordinates childCounter) - Skips
jay:xxxelements when computing sibling position (they're directives, not DOM elements) - Uses
jayTrackByvalue as the coordinate base (not a positional index)
Test improvements
waitForHydrationwith diagnostics — UsesPromise.racewith a 6s hard timeout to handle Vite reload loops. Collects page errors and reports them on failure.page.setDefaultTimeout(2000)— Playwright selector calls fail fast (2s) instead of auto-waiting 30s, preventing test timeouts from masking assertion failures.dumpTargetContenton failure — Catches check/interactivity errors, reads#targetinnerHTML viapage.evaluate, and re-throws with DOM content for debugging.afterAllcleanup — Removesbuild/directories created during tests.// @ts-ignorein fixtures — Before} from '...'lines to suppress tsc errors on Vite-style import paths.stripTsDirectivesremoves them before comparison.
Test results
- Runtime: 249 passed (252 total, 3 skipped)
- Compiler: 592 passed (596 total, 4 skipped)
- Dev-server: 42 passed (including all 6a–6d headless tests)
- Full suite: 68 packages, all passing
Removed redundant instanceVs lookup from hydrate render functions
The generated hydrate code had 3 lines per headless instance that looked up HEADLESS_INSTANCES context to override the viewState:
const instanceData = useContext(HEADLESS_INSTANCES);
const instanceKey = '1/widget:0';
const instanceVs = instanceData?.viewStates?.[instanceKey] ?? viewState;
This was redundant — makeHeadlessInstanceComponent's wrapped constructor already resolves the fast ViewState from HEADLESS_INSTANCES and merges it into compCore.render(). By the time the hydrate render function is called (step 3 in makeJayComponent: render(compCore.render())), the viewState parameter already has the correct instance data.
The redundancy also caused a bug for forEach instances: dataIds already contained the instance suffix (added by forInstance in childCompHydrate), and appending it again produced duplicated keys like "3,stock-status:0,stock-status:0".
Also simplified makeHeadlessInstanceComponent — removed old ComponentConstructor parameter support, now only accepts HeadlessComponentDef.
Fix: forEach coordinate $trackBy prefix not stripped in hydrate target
Discovered via fake-shop example: headless instances inside a forEach with intermediate wrapper elements (e.g., <div class="card"><jay:widget>) failed to hydrate — coordinate resolution produced "not found" errors.
Two related issues:
Element coordinates emitted with
$trackByprefix —assignCoordinatesassigns coordinates like"$_id/0/0"inside forEach items. The server-element target compiles these viacompileCoordinateExprwhich resolves the$placeholder to a runtime expression. But the hydrate target emitted the$_id/0/0string literally. At runtime,forItem("1")sets coordinateBase to["1"], thenresolveCoordinate("$_id/0/0")looked up"1/$_id/0/0"— wrong.childCompHydratecoordinate missing intermediate elements — For forEach,coordKeyArgwas justcoordinateSuffix(e.g.,"widget:0").forInstance("widget:0")extended the base to["1", "widget:0"], soadoptElement("0")resolved to"1/widget:0/0". But the DOM element was at"1/0/widget:0/0"— the/0/from the wrapper<div class="card">was missing.
Fix in renderHydrateElementContent: When inside a forEach and the coordinate starts with $, strip the $trackBy/ prefix. forItem already scopes by trackBy value, so resolveCoordinate("0/0") correctly resolves to "1/0/0".
Fix in coordKeyArg: Unified forEach and slowForEach — both use coordSegments.slice(1).join('/') to strip the first segment ($trackBy or jayTrackBy). The remaining path includes intermediate wrapper elements (e.g., "$_id/0/widget:0" → "0/widget:0").
Test: Updated 6c dev-server test (page-headless-foreach) to wrap the headless instance in <div class="card"><strong>{name}</strong><jay:widget> — the same pattern as the fake-shop. Verifies both hydration DOM correctness and interactivity (button click updates widget value).
Fix: dataIds polluted by forInstance segments
ConstructContext.dataIds (used by makeHeadlessInstanceComponent to compute __headlessInstances lookup keys) returned coordinateBase, which serves double duty: coordinate resolution AND forEach trackBy tracking. When forInstance("0/widget:0") extended coordinateBase for resolution, it also polluted dataIds — producing keys like "1,0,widget:0" instead of "1,widget:0".
Fix: Separated _dataIds from coordinateBase in ConstructContext. forItem adds to both (trackBy values). forInstance only adds to coordinateBase (instance coordinate segments are not trackBy values).
Fix: hydrate target missing coordinateSuffix in forEach key function
The hydrate target generated (dataIds) => dataIds.join(',') for the makeHeadlessInstanceComponent coordinate key function — missing the coordinateSuffix. The element target correctly used (dataIds) => [...dataIds, '${coordinateSuffix}'].toString(). Fixed hydrate target to match.
Test: Added new 6e test (page-headless-foreach-nested) — forEach with headless instance inside wrapper div + preceding static sections + no clientDefaults. Reproduces the exact fake-shop pattern where key mismatch caused "undefined is not iterable" crash.
Fix: headless instance counter mismatch between discovery and assignCoordinates
Problem: Two static <jay:widget> instances with different props in separate parent scopes (e.g., each inside its own <div class="card">) both got __headlessInstances key "widget:0". The server stored them correctly as "widget:0" and "widget:1", but the hydrate code looked up "widget:0" for both — the second instance got the first's fast ViewState.
Root cause: assignCoordinates used per-scope headless counters (reset per parent element via newScope()), while discoverHeadlessInstances used a prefix-keyed global counter. Two widgets in different <div> scopes both got counter index 0 in assignCoordinates, but discovery gave them indices 0 and 1.
Why counters are the wrong approach: The real problem is that both systems independently maintain counters that must stay synchronized. Any scoping difference causes mismatches. The secure runtime avoids this: it uses ref names (not counters) for coordinates, and uniqueness comes from the ref being unique within its scope.
Fix: use ref attribute instead of counter index. Each <jay:xxx> tag already has a ref attribute — either explicit in the HTML or auto-generated by discoverHeadlessInstances. The ref is unique within the discovery scope. Both discoverHeadlessInstances and assignCoordinates read the same ref attribute, so they always produce the same coordinate suffix (contractName:ref). No counter synchronization needed.
The contractName: prefix is kept because auto-generated refs are "0", "1", etc. — without the prefix, they'd collide with positional child indices in the coordinate map.
Changes:
assignCoordinates: readreffrom element attribute, usecontractName:refas suffix. RemoveheadlessCountersfromScopeState.discoverHeadlessInstances: no change (already uses ref for coordinate suffix).computeInstanceKey, hydrate compiler: no change (already receive/extract the suffix as-is).
Test: Added 6e-2 test (page-headless-two-instances) — two static widget instances with different props in separate wrapper divs. Verifies each gets its own fast ViewState and interactivity works independently.
Fix: Nested fast forEach coordinate mismatch (Bug G)
When a fast forEach is nested inside another fast forEach (or inside a slow forEach), SSR output produced coordinates missing ancestor prefixes. The hydration runtime accumulates coordinateBase via forItem() at each level, so it looks for g1/a1/0, but SSR generated a1/0.
Root cause — two issues:
walkForEachChildreninassign-coordinates.tshad no nested forEach detection. A forEach inside a forEach was treated as a regular element, producing coordinates in the wrong scope (e.g.,$_id/1/0instead of positional0within the inner forEach).SSR compiler's forEach item root coordinate was always just the trackBy value — no ancestor prefix. The
$_idplaceholder caused variable shadowing when the same trackBy name was used at multiple levels.
Fix in assign-coordinates.ts: walkForEachChildren now detects nested forEach children. When found, it assigns the container coordinate in the current scope, then recursively calls walkForEachChildren with itemPrefix = null — inner children get purely positional coordinates (e.g., 0, 1), avoiding $_id shadowing.
Fix in jay-html-compiler.ts (SSR): Three new ServerContext fields:
forEachAccumulatedPrefix— full prefix chain (all ancestor + current forEach). Prefixes static (positional) coordinates and serves as the forEach item root coordinate.forEachAncestorPrefix— ancestor-only prefix (before current forEach). Prefixes dynamic ($-based) coordinates where$\_idalready resolves to the current item's value.slowForEachCoordPrefix— concrete jayTrackBy string from ancestor slow forEach, consumed by fast forEach handler.
Static coords (no $) → prepend forEachAccumulatedPrefix. Dynamic coords (with $) → prepend forEachAncestorPrefix.
Test infrastructure: hydration warning detection
Added "no hydration warnings" test to testFixtureMode. Captures console.warn messages containing [jay hydration] during hydration and fails if any are found. Runs for every fixture in all 3 modes (SSR disabled, first request, cached).
New test fixtures (10a–10d)
| Fixture | Description |
|---|---|
10a-nested-slow-foreach |
2 categories × 2 items, nested slowForEach |
10b-nested-fast-foreach |
2 groups × 2-3 items, nested fast forEach (Bug G) |
10c-nested-conditional |
Conditional inside forEach, active/inactive per item |
10d-nested-combination |
Slow forEach → fast conditional + fast forEach |
Each has expected-ssr.html and expected-hydrate.ts.
Fix: Nested slowForEach coordinate stripping mismatch (Bug H)
Discovered via real-world golf page (nested slow forEach with interactive content). The hydrate compiler's slowForEachJayTrackBy was set to just the immediate jayTrackBy value (e.g., f999e7a4...), but assignCoordinates uses the accumulated slowForEachPrefix (e.g., e2a1b326.../f999e7a4...). The stripping logic (coordinate.startsWith(slowForEachJayTrackBy + '/')) failed for nested items, leaving the full coordinate unstripped. At runtime, forItem had already accumulated the same segments in coordinateBase, causing doubling: outer/inner/outer/inner.
Fix: Accumulate slowForEachJayTrackBy in the hydrate context, matching assignCoordinates' slowForEachPrefix:
const accumulatedJayTrackBy = context.slowForEachJayTrackBy
? `${context.slowForEachJayTrackBy}/${jayTrackBy}`
: jayTrackBy;
Test: Updated 10a fixture to include count field at phase: fast+interactive inside the inner slow forEach items, forcing the hydrate compiler to generate slowForEachItem calls for nested items. This exposed a further code generation issue (see DL#115).
Test results (after Bug G + H fixes)
- 455/455 hydration tests pass (52 new across 10a–10d)
- 616/616 compiler-jay-html tests pass
- 68/68 packages build successfully
- 10a now validates nested slowForEach with interactive content (hydration warnings + DOM checks)
Fix: Static sibling flattening in adoptDynamicElement (Bug I)
When a parent uses adoptDynamicElement (Kindergarten) because it has conditional children, static sibling elements with dynamic descendants had their content flattened into the parent's children array. This broke the 1-group-per-DOM-child mapping.
Example: <main> with <section> (containing dynamic text) + conditional <div if="showContent"> + <footer>:
- Bug: Kindergarten saw 3 children (adoptText, hydrateConditional, STATIC) but DOM had 3 different elements (section, div, footer) — adoptText claimed the section element instead of navigating inside it
- Symptom: conditional elements inserted at wrong DOM position when toggled true
Root cause: renderHydrateElementContent with !needsAdoption flattens children via mergeHydrateFragments. The adoptDynamicElement builder pushed these flattened fragments as separate children without wrapping.
Fix: In the adoptDynamicElement builder, plain static elements (not components, not non-interactive conditionals) use renderHydrateElementContent with forceAdopt=true, producing adoptElement("coord", {}, [...descendants...]) instead of leaking descendants.
File: packages/compiler/compiler-jay-html/lib/jay-target/jay-html-compiler-hydrate.ts
Test: test/fixtures/conditions/conditions-with-static-sibling/
SSR coordinate gap (Bug I follow-up)
The hydrate fix exposed a second issue: adoptElement("coord", ...) on the static wrapper element requires jay-coordinate="coord" in the SSR DOM, but the server element compiler only emits coordinates on elements it considers dynamic.
Fix: Added parentHasInteractiveChildren flag to ServerContext. When rendering children of an element whose children include conditionals/forEach/async, the flag is set to true. The needsCoordinate check now includes this flag, ensuring static siblings get jay-coordinate in SSR output.
File: packages/compiler/compiler-jay-html/lib/jay-target/jay-html-compiler-server.ts
Test results (after Bug I)
- 660/660 compiler-jay-html tests pass (3 async server-element fixtures updated)
- 678/678 dev-server hydration tests pass (fixtures regenerated via
UPDATE_FIXTURES=1)
Fix: Production server missing __headlessInstances in client init args (Bug J)
Production build works in dev but fails in production with three errors:
[Jay] Headless instance "S0/0/0/0/0/2/login-indicator:AR0" has no server data and no clientDefaults[jay hydration] adoptBase coordinate "S1/0/0" not found in DOM[jay hydration] adoptBase coordinate "S1/0" not found in DOM
Root cause: In fetch-page-handler.ts, the serve-time code correctly constructs fullSlowViewState (with __headlessInstances from carryForward.__instances.slowViewStates) for SSR rendering. But when serializing the client init args, it passed cached.slowViewState (without __headlessInstances) instead of fullSlowViewState.
The SSR uses fullViewState (which includes __headlessInstances), so the server element guard if (vs_login_indicator0) works and the inline template is rendered with coordinates. But the client receives a slowViewState without __headlessInstances. When deepMergeViewStates(slowViewState, fastViewState) runs on the client, it only gets instance data from the fast phase. For components without fastRender (render-only or slow+render), the fast phase stores {} — which may be insufficient for the headless instance context to find meaningful data.
Fix: Changed line 191 in fetch-page-handler.ts from cached.slowViewState to fullSlowViewState.
File: packages/jay-stack/production-server/lib/serve/fetch-page-handler.ts
Note: The DL106 compiler fixes (assignCoordinates, parentHasInteractiveChildren, forceAdopt) are automatically used by the production build since compileRouteServerElement and compileRouteHydrateScript both call generateServerElementFile/generateElementHydrateFile from compiler-jay-html. No changes needed in the build pipeline for the compiler fixes.
Log Methodology Note
Note: These design logs are written primarily for AI agents as part of the Design Log methodology and made accessible here for human readers. The language and structure are optimized for machine consumption — expect precise, specification-style prose rather than narrative documentation.