Skip to content

Documentation / @etherfold/core / ReceivingIndexer

Class: ReceivingIndexer<ABI, ProcessResultType, State> ​

Defined in: packages/core/src/receivingContainer.ts:884

A NAMED INDEXER on the receiving side: several generations, one canonical pointer, and one receiver per LIVE WIRE CONTEXT.

Built through openReceivingIndexer. See the module JSDoc for what it adds over a bare StreamBuilder and which of the chain-facing container's rules deliberately do not come over.

It also ANSWERS what the indexer-server's registry asks a name (IndexerRegistryEntry, @etherfold/server): liveIngestions and canonicalGeneration are exactly the two questions the ingest routes and the feed put to an entry, so a host registers this container itself rather than an adapter that could answer them differently. What a host must add beside it is the DATABASE that name owns (ADR-0053, indexerEntryOn), which is not expressible here: this package knows no database, which is why the state a generation folds into is a type parameter.

Type Parameters ​

ABI ​

ABI extends Abi

ProcessResultType ​

ProcessResultType = unknown

State ​

State = unknown

Implements ​

Constructors ​

Constructor ​

new ReceivingIndexer<ABI, ProcessResultType, State>(registry, options): ReceivingIndexer<ABI, ProcessResultType, State>

Defined in: packages/core/src/receivingContainer.ts:1083

Parameters ​

registry ​

GenerationRegistry

options ​

ReceivingIndexerOptions<ABI, ProcessResultType, State>

Returns ​

ReceivingIndexer<ABI, ProcessResultType, State>

Properties ​

registry ​

readonly registry: GenerationRegistry

Defined in: packages/core/src/receivingContainer.ts:890

Which generations this indexer holds, which one is canonical, and the caps that refuse.


stateMoved ​

protected readonly stateMoved: StateMovedPublisher

Defined in: packages/core/src/receivingContainer.ts:909

THE SIGNAL, and the token it carries: what this container tells the sides that are READING (ADR-0083).

The SAME class the chain-facing container holds and deliberately not a second implementation of it: there are two things in this system that apply blocks, both publish the same signal, and one notification model is the claim being made -- two producers that drift is how that claim dies. So the subscription, the containment of a throwing handler and the token's rotation are all StateMovedPublisher's, and what is here is the half only a container knows: WHICH generation applied the block, and whether it is the one that answers reads.

It matters most HERE rather than on the chain-facing twin: every server and CLI deployment folds through this container, so a reader of a hosted indexer has no other producer to be told by.

Accessors ​

caps ​

Get Signature ​

get caps(): GenerationCaps

Defined in: packages/core/src/receivingContainer.ts:1462

The caps in force, reported so a host can see the bound rather than re-derive the default.

Returns ​

GenerationCaps


fetchedSource ​

Get Signature ​

get fetchedSource(): IndexingSource<ABI> | undefined

Defined in: packages/core/src/receivingContainer.ts:1213

THE SOURCE THIS DEPLOYMENT FETCHES FOR ITS CANONICAL GENERATION: the one it was configured with, or, on a deployment started with NOTHING configured (ADR-0093), the one its first fold carried -- the canonical generation instantiated at open from its stored bundle, or the first generation an arrival added -- and from then on the source of whatever generation a promotion made canonical.

It is the source a fold that names none of its own defaults to, and it is NOT the whole of what is fetched: every stream a held fold reads is fetched, each by its own fetcher through its own writer (fetchedStreams), so a successor on a new stream catches up beside the incumbent rather than waiting for this to move.

undefined is a real answer and not a gap to paper over: such a deployment has been told nothing about what to fetch yet, so it fetches nothing and says it is WAITING.

Returns ​

IndexingSource<ABI> | undefined


generation ​

Get Signature ​

get generation(): GenerationId

Defined in: packages/core/src/receivingContainer.ts:1457

WHICH generation the opening fold is: the stream above, plus the fold over it.

Returns ​

GenerationId


ingestion ​

Get Signature ​

get ingestion(): StreamWriter<ABI>

Defined in: packages/core/src/receivingContainer.ts:1445

THE WRITER of the stream the opening fold folds: the live wire context a single-stream host has.

It is the DEPLOYMENT's writer of that stream and not the opening fold's engine (ADR-0087), which is why it is here at all: a fold has no receiver, and this getter used to reach for one and THROW when the opening fold turned out to be a follower -- a crash that stopped a restarted deployment from starting, in an assembly three commands share.

It exists for as long as this container holds a fold, because a writer is built with the first fold on its stream; the assertion says so rather than widening every caller's type for a case open cannot produce.

Returns ​

StreamWriter<ABI>


opening ​

Get Signature ​

get opening(): HeldFold<ABI, ProcessResultType, State>

Defined in: packages/core/src/receivingContainer.ts:1347

The fold this host OPENED with, which is the first one added.

Every singular accessor below reads through it, so a host holding one fold -- which is every host until a filter change creates a successor -- says indexer.ingestion and never indexes into a list of one.

Returns ​

HeldFold<ABI, ProcessResultType, State>


processor ​

Get Signature ​

get processor(): EventProcessor<ABI, ProcessResultType>

Defined in: packages/core/src/receivingContainer.ts:1428

The processor the opening fold runs.

Returns ​

EventProcessor<ABI, ProcessResultType>


promotion ​

Get Signature ​

get promotion(): UsedPromotionConfig

Defined in: packages/core/src/receivingContainer.ts:1723

The promotion policy this indexer runs under, resolved, so a host can see WHICH value is in force.

Returns ​

UsedPromotionConfig


state ​

Get Signature ​

get state(): State

Defined in: packages/core/src/receivingContainer.ts:1424

The state the opening fold folds into, as its factory built it.

Returns ​

State


streamConfig ​

Get Signature ​

get streamConfig(): UsedStreamConfig

Defined in: packages/core/src/receivingContainer.ts:1420

The stream config that digest was taken over, resolved.

Returns ​

UsedStreamConfig


streamDigest ​

Get Signature ​

get streamDigest(): string

Defined in: packages/core/src/receivingContainer.ts:1416

WHICH STREAM the opening fold folds, as streamDigestOf renders it.

Returns ​

string

Methods ​

add() ​

add<S>(spec): Promise<HeldFold<ABI, ProcessResultType, S>>

Defined in: packages/core/src/receivingContainer.ts:1760

Build a fold BESIDE the ones already held, and make sure its stream is being fetched.

The receiving twin of Indexer.add, and the same order for the same reason: STATE FIRST, then the fold over it (ADR-0043), then the REGISTRY -- which is written before anything writes the stream, because a stream subtree no registered generation claims is what the sweep collects.

A cap REFUSES here and no GENERATION is left behind: the record is not written and no engine is built, so nothing names or reads whatever the state factory happened to open.

What a refusal MAY leave is storage the factory itself created, and that is the host's business rather than this container's: a factory that claims its store (openForWriting, ADR-0077) migrates, and the cap is enforced one step later because the record needs the processor's identity, which needs the processor, which needs the state (ADR-0043). The CLI's SQL factory therefore leaves an empty namespace behind, reused verbatim if the bound is raised. This order cannot be swapped: a pre-check on the COUNT alone would refuse re-opening a generation this container already holds, which is the case create deliberately RESOLVES.

THERE IS NOTHING LEFT TO DETERMINE ABOUT THE FOLD (ADR-0087) ​

Every fold gets a GenerationRebuild over the stored stream and none of them gets a receiver or an appender. What the arriving fold's STREAM gets, if this is the first fold on it, is a WRITER -- the deployment's, positioned from the stream's own coverage claim. So the question that used to be decided here ("does this fold follow, or does it hold the pen") has no answer to give, which is what makes a restarted deployment holding only a successor fetch exactly as a fresh one does.

Type Parameters ​

S ​

S

Parameters ​

spec ​

ReceivedGenerationSpec<ABI, ProcessResultType, S>

Returns ​

Promise<HeldFold<ABI, ProcessResultType, S>>


canonical() ​

canonical(): Promise<GenerationRecord | undefined>

Defined in: packages/core/src/receivingContainer.ts:1472

The generation reads resolve through, which is NOT necessarily one folding here.

Returns ​

Promise<GenerationRecord | undefined>


canonicalGeneration() ​

canonicalGeneration(): Promise<GenerationId | undefined>

Defined in: packages/core/src/receivingContainer.ts:1627

WHICH GENERATION ANSWERS READS, as an identity a host can report -- or NOTHING, which is a real answer and never an empty one.

The narrow form of canonical above, and the one a serving host asks for: both halves in ONE read, so a response can never pair one generation's stream with another's fold.

Why undefined is PASSED THROUGH and not covered over ​

This used to fall back to the OPENING FOLD, on the reasoning that any registry a generation has been created in has a pointer (the first one registered takes it, which is the registry's own rule), so nothing could ever ask. That reasoning predates the answer being expressible at all: openGenerationRegistry.canonical() resolves the pointer AGAINST THE RECORDS, so it answers nothing when the pointer names a generation whose record has gone -- another process deleting it, or a half-written substrate.

In that case the opening fold is NOT the generation the pointer named, so the fallback served reads from a generation nobody asked for, silently, where a read tier over the same rows refuses (503 no-canonical-generation, ADR-0058). One database, two answers, decided by which process happened to be asking. Passing the registry's answer through makes every host agree, and the refusal an operator sees is then a fact about the DATABASE rather than about the reader.

A host holding folds still holds them and still folds: this says which generation ANSWERS, and "none of them, yet" is a state the read surface already knows how to report.

Returns ​

Promise<GenerationId | undefined>


coherenceNow() ​

coherenceNow(): string

Defined in: packages/core/src/receivingContainer.ts:1411

THE COHERENCE TOKEN IN FORCE RIGHT NOW: the one the next notification will carry, read without waiting for one.

OPAQUE exactly as it is on a notification -- COMPARED, never parsed -- and it is a READ rather than a second way of publishing: nothing rotates here, nothing is delivered, and a reader that asks twice between two blocks gets the same value.

It exists for a transport that must tell a client, AT CONNECT, whether what that client already holds may be stale. The BROWSER transports need no such thing, because a tab attaching part way through simply READS the store it shares; a REMOTE reader has neither that store nor a state query surface yet (the query layer is deferred to the-same-query-runs-against-a-worker-and-a-server), so ADR-0083 makes its convergence "be told the current position and token on connect" instead. That is what this answers, and the server's state-moved stream is its one caller today.

It is deliberately NOT a notification for a late joiner, which would have a reader invalidate for a block it may already have read: a notification is a thing that HAPPENED, and this is a fact about the producer's history.

Returns ​

string


fetchedStreams() ​

fetchedStreams(): Promise<readonly FetchedStream<ABI>[]>

Defined in: packages/core/src/receivingContainer.ts:1706

THE STREAMS THIS DEPLOYMENT FETCHES: one entry per stream a registered generation held here folds, each with the source a fetcher fetches it over and its ONE writer (FetchedStream, ADR-0087).

The FETCHING side of liveIngestions, from the same derivation, so the two cannot disagree: a host that fetches (the CLI's run, node and build) holds ONE fetcher per entry and asks this before every cycle, and so the set of fetchers follows the folds, which follow the slots:

  • a successor on a NEW stream (an upload that adds an event, a restart after a filter change) is held from its registration, so its stream appears here and is fetched BESIDE the incumbent's, which goes on being fetched and answering;
  • a PROMOTION onto it stops folding the incumbent where it read another stream (movePointer), so the old stream leaves this list and its fetcher stops, once no other fold here reads it -- on a host that says it fetches its own streams (fetchesItsOwnStreams); a push-fed receiver keeps the incumbent folding, since what fetches its streams is another process;
  • a REPLACED successor is dropped (dropReplaced), and its stream leaves with it on the same terms;
  • at open the canonical generation and the pending successor are instantiated whatever stream each is on, so a restart mid-catch-up fetches both.

Each stream has exactly ONE writer here whoever fetches it: this adds streams, never a second writer of one.

Returns ​

Promise<readonly FetchedStream<ABI>[]>


folding() ​

folding(): Promise<GenerationFolding[]>

Defined in: packages/core/src/receivingContainer.ts:1518

WHETHER EACH REGISTERED GENERATION CAN FOLD HERE: held, instantiable from its stored bundle, or frozen and why (GenerationFolding, ADR-0092).

The operator's question before a revert, answered for every generation in the order the registry lists them, so two generations that answer reads the same way are no longer two states an operator cannot tell apart. It is also where a generation whose stored code could not be built at open is REPORTED, which is the spec's story 8: that deployment starts and serves the generation frozen, and before this only a log line said why.

It BUILDS NOTHING. instantiable is decided from what is known without running the code: bytes are stored, this host can instantiate bytes, nothing in this process has failed to, and the generation's stream is the one an instantiation here would fold (the container's own, which is what a host whose seam names no source of its own folds; such a host's own source is known only by trying, and the try is remembered). Evaluating every stored bundle to answer a question would be the eager instantiation ADR-0092 rejects, with state opened for generations nobody reads.

One read of the records, then one read of the stored bytes per generation not held here, which the generation caps bound.

Returns ​

Promise<GenerationFolding[]>


foldingOf() ​

foldingOf(generation): Promise<GenerationFolding>

Defined in: packages/core/src/receivingContainer.ts:1538

THE SAME ANSWER as folding, for ONE generation: held, instantiable, or frozen and why.

For a host that needs one generation's answer and not the whole listing's, which is /status asking about the CANONICAL generation on every refresh: folding reads the stored bytes of every registered generation not held here, and a page an operator refreshes should pay for the one it reports. It is the same derivation, not a second one, so the two surfaces cannot disagree about a generation.

Parameters ​

generation ​

GenerationRecord

Returns ​

Promise<GenerationFolding>


generations() ​

generations(): Promise<GenerationRecord[]>

Defined in: packages/core/src/receivingContainer.ts:1467

Every registered generation, oldest first.

Returns ​

Promise<GenerationRecord[]>


held() ​

held(): readonly HeldFold<ABI, ProcessResultType, unknown>[]

Defined in: packages/core/src/receivingContainer.ts:1360

Every fold held, oldest first. At most one per stream; see the module JSDoc.

Returns ​

readonly HeldFold<ABI, ProcessResultType, unknown>[]


liveIngestions() ​

liveIngestions(): Promise<readonly LogIngestion[]>

Defined in: packages/core/src/receivingContainer.ts:1670

THE LIVE WIRE CONTEXTS: one STREAM WRITER per stream a registered generation held here folds.

A stream is ONE address on the wire, and what answers at that address is the deployment's writer of that stream (ADR-0087) -- not a fold, because no fold appends. So this list no longer depends on WHICH fold is present, only on whether any registered generation still folds the stream, which is the whole of what this change buys: a deployment holding nothing but a successor re-folding stored history goes on fetching and appending.

A fold is live while the generation it was registered as is still registered. That is the whole lifetime: it BEGINS when the successor is created (add, which registers before it builds anything) and ENDS when that generation is DELETED. The STREAM is no longer part of that lifetime -- it outlives every fold over it and is deleted only when asked.

What it deliberately does NOT consult is the canonical pointer. A superseded generation is RETAINED under the caps, so "the successor became canonical" is not by itself a reason to stop fetching its stream; whether it should be is a POLICY that sets what the registry holds, and this routing follows the registry either way.

What is GONE from here is reconcileWriters, which ran before every answer to move the wire to whichever generation the records now elected. There is nothing to move: the wire never belonged to a generation.

Returns ​

Promise<readonly LogIngestion[]>


onStateMoved() ​

onStateMoved(handler): StateMovedDetach

Defined in: packages/core/src/receivingContainer.ts:1386

BE TOLD THE STATE MOVED: one notification per block the CANONICAL fold applied, and one per branch it took back, with a token that says whether anything else a reader holds may now be wrong (ADR-0083). Returns the detach.

ts
const detach = container.onStateMoved((moved) => {
  if (moved.coherence !== held) {held = moved.coherence; return invalidateEverything();}
  if (moved.kind === 'applied') for (const entity of moved.entities) invalidate(entity);
});

This is the HOST's attachment point, and it is the same name and the same shape as the chain-facing container's, so a transport adapts to ONE surface: the server registers this container under a name (indexerEntryOn, @etherfold/server, which forwards this method), and the CLI's index command forwards it on the entry it writes out.

It is a SIGNAL and not a delivery of data: no rows, no mutations and no state handle, which is why it does not disturb rule 2 of the module JSDoc. Best-effort, with nothing held per subscriber: see StateMovedPublisher.

Parameters ​

handler ​

StateMovedHandler

Returns ​

StateMovedDetach


open() ​

open(): Promise<void>

Defined in: packages/core/src/receivingContainer.ts:1107

Add the fold this host was built with, and let the policy speak about it on exactly the same terms as any other (ADR-0084).

Called by openReceivingIndexer; separate from the constructor because registering a generation is a write and a CAP refuses here, at start-up, where an operator reads it.

There is no longer a gate saying "the policy is silent during open", and that is the correctness cliff ADR-0084 exists to remove: a generation registered at open could never be promoted -- not late, ever -- so restarting with a changed processor left a successor that caught up and then sat there for ever, with nothing reported and no policy value that changed it. What the gate was PROTECTING is protected by the slot instead; see applyPolicyTo.

Returns ​

Promise<void>


promote() ​

promote(id): Promise<GenerationRecord>

Defined in: packages/core/src/receivingContainer.ts:2292

MOVE THE CANONICAL POINTER: one small write, forwards or BACKWARDS.

The verb, ungated by the policy under every value: manual means "only when asked" rather than "never", and moving the pointer BACK is the SAME call at a different target -- promotion and revert are one mechanism and this is it (the operator-facing affordance over it is POST /{indexer}/admin/canonical-generation, @etherfold/server).

It does NOT require this container to hold a FOLD for the target ​

That is the whole of what makes the way back real on this runtime, and it is rule 1 of the module JSDoc applied to the WRITE side: reads here resolve the pointer to a table NAMESPACE (ADR-0053), so the canonical generation answers with no engine at all. The ORDINARY revert is exactly that case -- a host redeployed with the new processor holds only the new fold, and the generation an operator wants back is in the durable registry with its state in its own namespace -- so refusing here would mean the revert could only be performed by a process that had first been rebuilt with the old processor, which is the re-index this design exists to remove.

The refusal is therefore the REGISTRY's (UnknownGenerationError): a generation nothing registered is refused rather than reported as a silent success, and that is the one question worth asking here.

...and a target it holds no fold for is INSTANTIATED, so it FOLDS (ADR-0092) ​

Answering reads is not enough: a generation the pointer names has to advance. So where the host supplied instantiateGeneration, the target is built from the bundle stored on its row before the pointer moves, and a target that cannot be built refuses the move (GenerationInstantiationError) rather than leaving a canonical generation nothing folds. A revert also stops folding the generation it moved away from (movePointer).

A target on a stream this container cannot name (a revert across a filter change) is NOT refused: its code is not broken, its stream is simply not one this deployment fetches. The pointer moves, nothing folds it, and that is logged as an error, which is the freeze a filter change always was.

Parameters ​

id ​

GenerationId

Returns ​

Promise<GenerationRecord>


rebuildMore() ​

rebuildMore(options?): Promise<RebuildReport[]>

Defined in: packages/core/src/receivingContainer.ts:2232

ADVANCE EVERY FOLD BY ONE BOUNDED CHUNK, then settle the pointer.

The call a HOST SCHEDULES (ADR-0022), and the shape prune and compactEmissionPairs already have: bounded work per invocation, and a REPORT a scheduler acts on. It is never a side effect of a batch -- a rebuild takes arbitrarily long, and putting it on the write path would stall whichever batch happened to arrive during an upgrade, for work that batch did not cause.

A scheduler loops while a report is complete: false AND retryCanAdvance says another call can help; a serverless host re-invokes itself instead. Both halves are load-bearing: three of the six RebuildStop reasons recur identically on every call, so looping on complete === false alone polls them for ever at full rate while the follower never becomes level (ADR-0070). A report that is incomplete and cannot advance needs a human, not another call. Nothing here invents a cadence, and a call with nothing to do costs one read per fold.

It advances EVERY fold and no longer only the followers, because there are no others: under ADR-0087 no generation fetches, so every one of them catches up by re-folding the stream the deployment stored. A fold that is already level has taken each delta live as it was appended (offerToFolds), so its chunk here finds nothing new and costs one read.

The pointer is settled AFTER the chunks, once, so a successor that became level during this call is promoted in the same call rather than on the next one -- and at most ONE move happens, because promoting twice inside one advance would publish a generation nobody ever read from.

Parameters ​

options? ​
maxEmissions? ​

number

Returns ​

Promise<RebuildReport[]>


reclaim() ​

reclaim(): Promise<ReclaimReport>

Defined in: packages/core/src/receivingContainer.ts:2983

RECLAIM WHAT NOTHING NAMES: drop every registered generation no slot holds, and report what came back.

Why this exists at all ​

A cap REFUSES at its bound and never evicts, which is sound and was the ONLY instrument an operator had: when it fires they are told what they COULD delete and given nothing to delete it with, so the remedy was hand-written SQL or a deleted database. Slots make the missing verb expressible for the first time -- a generation no slot names is garbage BY DEFINITION rather than by an operator's judgement about digests and timestamps (ADR-0084) -- and the deletion itself is not new: it is the registry's deleteGeneration, which drops the row, drops the state namespace (ADR-0053 makes that a DROP) and REAPS the stream where no registered generation is left folding it.

It is a VERB an operator runs, and deliberately NOT a garbage COLLECTOR ​

Nothing calls it on a timer and nothing calls it at open. An automatic reclaim is a different decision with a different risk profile -- it deletes with nobody present -- and ADR-0084 does not make it. The one deletion that DOES happen without being asked is bounded to what a registration itself displaced (replaceTheSuccessor), which is a generation this process just replaced rather than rows it never touched.

The CAPS are untouched by it. This gives an operator an instrument; it does not raise a bound or make a refusal less likely.

What it will NEVER take, which is the property that makes it safe ​

A generation ANY slot names -- and predecessor is the one worth saying out loud, because it is not canonical right now and is exactly the way back from a bad upgrade. The rule is read as a REFCOUNT over the slot rows (unslottedGenerations) and never as "not canonical", which would delete the revert target and the pending successor both.

It DECLINES per generation rather than refusing the whole call, where the substrate would not delete one: the others still go, and the next reclaim tries again. NEWEST FIRST, which is the order the registry lists garbage in.

IT IS THE ONE PATH THAT STILL REAPS A STREAM (ADR-0087) ​

Deletion is a VERB. Every AUTOMATIC reap is gone -- a registration that displaces a successor and a promotion that drops what it superseded both leave the stream exactly where it is -- and what is left is this, which an operator ran, and deleteStream, which an operator named. So a generation reclaimed here takes its stream with it exactly when no registered generation is left folding it, which is what an operator reclaiming disk asked for and is why the report NAMES what came back.

The clause that used to RETAIN the elected writer of a stream another held fold followed is gone with the election: no generation writes a stream, so there is no duty to strand.

Returns ​

Promise<ReclaimReport>


resolveGeneration() ​

resolveGeneration(id): Promise<GenerationRecord>

Defined in: packages/core/src/receivingContainer.ts:3156

RESOLVE a generation this indexer has REGISTERED, and refuse one it has not.

It used to resolve OR CREATE, which made it a second registration route beside add -- one that carried no bundle, so it could register a generation on a Node deployment with no code stored for it (ADR-0092), which is precisely the class of frozen generation retention exists to make unexpressible. On this container a generation is registered by add and by nothing else, because add is what holds the bytes; this answers for what add (or an earlier process) already registered.

Parameters ​

id ​

GenerationId

Returns ​

Promise<GenerationRecord>

Implementation of ​

GenerationContainer.resolveGeneration


slots() ​

slots(): Promise<SlottedGenerations>

Defined in: packages/core/src/receivingContainer.ts:1489

WHAT EACH SLOT NAMES, resolved against the records, in ONE read (ADR-0084).

The operator's SEE half, and the registry's own answer forwarded rather than re-derived: which generation answers reads, which one is pending beside it, and which one a revert would move back to. Everything registered and named by NONE of them is what reclaim takes, so the two are read from the same place and cannot disagree about what a slot holds.

ONE read rather than three, for the reason the registry gives: the three answers are used together, and three reads could answer from either side of another process's write.

Returns ​

Promise<SlottedGenerations>