Implement the first version of AssetAZJournalService.
The service is the authoritative append-only event journal for AssetAZ. It must provide:
durable event persistence,
deterministic global ordering,
integrity verification,
replay,
live event delivery,
per-consumer progress tracking,
consistency barriers between services,
isolated degraded-state handling.
The journal is also the foundation from which stateful AssetAZ services can reconstruct their models after restart.
This issue intentionally defines a strict v1. Do not add recovery, snapshotting, automatic tuning, generalized object serialization, or other functionality not described here.
Architectural principles
An AssetAZ event represents a fact/state change, not a command.
Replay must rebuild state only. Replay must never re-execute historical side effects such as trades, transfers or notifications.
If multiple state changes are business-atomically inseparable, represent them as one event rather than several correlated events.
correlationId groups independently meaningful events belonging to the same logical workflow. It does not provide atomicity and does not imply an expected number of events.
Journal sequence numbers define the authoritative total ordering. Event timestamps do not.
Multiple persistent versions of an event are an implementation concern handled by codecs, journal metadata, contexts/classloaders and the surrounding Nenjim architecture.
Persistent event versioning must be abstracted away from the Java domain type.
Event codecs
Persisted event identity consists of:
eventType
eventVersion
For the first persistent representation of AssetOwnerRegistered:
Convenience conversion code for UUID, Instant, etc. may exist inside codec implementations or suitable codec-level helpers, but must not become PDE semantics.
The minimal API should support the operations required by this issue, conceptually including:
readInt() and readLong() read the applicable PDE integer representation but expose useful Java primitive ranges to callers.
Do not turn this into a general object mapper.
Do not add:
reflection-based serialization,
annotations-as-schema,
automatic Java record serialization,
Jackson-like machinery,
functionality not currently required by AssetAZ.
Three validation layers
Validation responsibilities must remain explicitly separated.
1. Journal / record integrity
Owned by the journal/record layer.
Examples:
SHA-256 checksum,
previousChecksum,
sequence continuity,
top-level record framing,
truncated records,
maximum record size.
Event codecs must not perform these checks.
2. PDE validity
Owned by the PDE decoder/framing implementation.
Examples:
valid PDE representation,
valid PDE primitive type,
valid object/value framing,
valid encoded lengths,
valid UTF-8,
typed-null handling,
conversion to the requested Java primitive range.
Event codecs must not implement PDE parsing themselves.
3. Event/schema validity
Owned by the concrete event codec.
For AssetOwnerRegistered, examples include:
exactly seven payload values,
nano-of-second is within 0..999999999,
UUID byte sequences are exactly 16 bytes,
AssetOwnerType is one of the known stable textual values,
required values are non-null,
no unexpected trailing payload values.
PDEDecoder.requireEnd() provides the mechanism for detecting remaining values, while the codec invokes it because the codec owns knowledge of where its schema ends.
A failure in any of these layers makes journal loading fail in v1.
Diagnostics should clearly identify which layer rejected the data.
Journal record format
Every journal record is exactly one top-level PDE OBJECT.
The file therefore consists conceptually of:
OBJECT record #1
OBJECT record #2
OBJECT record #3
...
The top-level object framing must allow a reader to determine the complete byte length of the next record before semantically decoding its event payload.
Journal format version 1 contains these fields in exactly this logical order:
but it must be stored as the corresponding 32 binary bytes, not as hex text.
Sequence
Journal sequence numbers are globally monotonically increasing and start at:
1
There must be no intentional gaps.
The sequence number becomes committed only when the corresponding record has been successfully and durably persisted.
If persistence fails, that candidate sequence number is not considered consumed.
Maximum record size
Use a simple hardcoded implementation safety limit:
privatestaticfinalintMAX_RECORD_SIZE_BYTES=1024;
The limit applies to the entire persisted top-level PDE record, including its object header.
No automatic resizing or adaptive calculation is required.
If a legitimate event eventually approaches the limit, the constant can be manually increased in a future change.
Do not implement configuration or automatic growth.
Writing
The fully encoded record must be rejected before any bytes are appended if:
record size > MAX_RECORD_SIZE_BYTES
Reading
After reading enough PDE framing to determine the declared top-level object length, reject it immediately if:
record size > MAX_RECORD_SIZE_BYTES
Also reject a record whose declared size extends beyond the remaining bytes in the journal file.
Reader buffer
Each journal reader should allocate one fixed buffer:
newbyte[MAX_RECORD_SIZE_BYTES]
and reuse that same buffer for every record it reads.
Do not allocate a new record-sized byte[] for every historical record.
Do not implement shared buffer pools.
Each independent reader owns its own buffer.
A decoder must always operate on a bounded byte range:
byte[]
offset
length
and never treat the entire backing array as valid data.
For example, if a 314-byte record has been read into a 1024-byte reusable buffer, bytes 314..1023 may still contain data from an older record and must be completely invisible to the decoder.
requireEnd() means:
current position == logical decoder limit
not:
current position == backing byte[].length
Child/object decoders must be bounded in the same way.
No buffer clearing such as Arrays.fill() is required.
Reader flow
Reading a record should conceptually proceed as follows:
1. Read enough PDE framing to identify the top-level OBJECT and its length.
2. Validate the declared record length against:
- MAX_RECORD_SIZE_BYTES
- remaining journal file size.
3. Read exactly that record into the reader's reusable buffer.
4. Identify the first field as the stored checksum and locate the
exact byte range containing fields 2..6.
5. Validate SHA-256 directly over those original bytes.
6. Decode the remaining journal envelope.
7. Validate previousChecksum and sequence continuity.
8. Resolve eventType + eventVersion to a codec.
9. Give only the payload decoder to that codec.
10. The codec decodes and validates its exact event schema.
Checksum validation occurs before semantic event parsing.
Minimal PDE framing necessarily happens before checksum validation because the reader must determine record boundaries and locate the checksum/body byte ranges.
Writer flow
Fields 2..6 must be encoded exactly once.
Conceptually:
encode fields 2..6
|
v
bodyBytes
|
+---- SHA-256 ----> checksum
|
v
encode checksum field
|
v
create top-level OBJECT framing
|
v
OBJECT_HEADER + checksumField + original bodyBytes
|
v
validate total size <= MAX_RECORD_SIZE_BYTES
|
v
append
|
v
FileChannel.force(true)
Do not decode/re-encode bodyBytes.
Copying already encoded bytes into the final record buffer is acceptable in v1.
Do not add generalized buffer pooling or similar allocation optimizations.
Durability and append semantics
The journal file is authoritative.
In-memory journal head/state is derived/cache state only.
Appending must be serialized.
Conceptual append ordering:
1. candidate sequence = current head + 1
2. encode complete record
3. validate record size
4. append bytes
5. FileChannel.force(true)
6. commit new journal head in memory
7. event officially exists
8. make event available to consumers
9. return sequence
Consumers must never observe an event before durable persistence has succeeded.
Uncertain write failure
If an append fails in a way that may have left a partial or uncertain journal tail, fail closed.
For the remainder of that JVM:
stop accepting new events,
do not retry automatically,
do not truncate automatically,
do not attempt automatic repair.
A subsequent startup must validate the complete authoritative journal and must fail if consistency cannot be guaranteed.
Manual repair is acceptable in v1.
Startup validation
On startup, validate the authoritative journal sufficiently to guarantee that its complete history is structurally and cryptographically consistent.
Validate at least:
record framing,
record size,
PDE validity,
all checksums,
previousChecksum chain,
exact sequence continuity,
known event type/version codecs,
exact event schemas.
If the journal cannot be proven consistent, AssetAZJournalService must not start successfully.
Journal format version changes
Journal format version and event version are separate concepts.
eventVersion versions the persistent schema of one concrete event.
Journal format version controls interpretation of the surrounding journal/common event representation.
A new journal implicitly begins in journal format version 1.
Do not require an initial version-1 header.
Future journal-format switches are represented as ordinary events in the same normal journal record envelope and checksum chain.
Conceptually:
format 1 records
JournalFormatVersionChanged(2)
format 2 records
JournalFormatVersionChanged(3)
format 3 records
JournalFormatVersionChanged(1)
format 1 records
...
Switching back to an earlier version must be representable.
Do not introduce a separate physical record type or separate unchained file header for format changes.
The format-change event itself must be integrity protected exactly like any other journal record.
The parser may react to such an event and change its parsing state for subsequent records.
The record parser may itself be an event receiver for this purpose.
Do not create an elaborate separate system-event hierarchy in v1 merely for this functionality.
Consumer API
Use one callback for both replayed and newly appended events.
There is deliberately no replay/live distinction in the event callback.
where registrationHead is captured atomically when registration is established.
Registration returns without waiting for replay to complete.
Event filters
eventTypes:
must not be null,
must not contain null,
must be defensively copied.
An empty set means:
ALL event types
A non-empty set means exact Java runtime class matches only.
Use exact event.getClass() semantics.
No inheritance matching, wildcards or other implicit behaviour in v1.
Filtering is performed by the journal service.
The consumer only receives events it subscribed to.
Duplicate consumer registration
The exact same consumer object instance may only be registered once.
Use object identity:
==
not equals().
Different objects of the same consumer class may be registered independently.
A second registration attempt for the same object must fail rather than overwrite the first registration.
A degraded registration remains registered in v1.
Independent replay
Each consumer/subscription performs its own independent historical replay.
A late-registering consumer must not restart or disturb replay/live processing for existing consumers.
It is acceptable for ten consumers to independently read the same journal ten times during catch-up.
Each replaying subscription may own:
its own journal reader/file descriptor,
its own reusable 1 KiB record buffer,
its own worker.
The simplicity and independence are intentional.
The historical reader is closed once catch-up is complete.
Consumer worker and concurrency model
Use one Java virtual thread per consumer subscription.
Do not convert unrelated existing Nenjim startup/service code to virtual threads as part of this issue.
For one consumer:
onEvent calls are always sequential
Never invoke the same consumer concurrently.
Different consumers may process concurrently.
New relevant events arriving while a consumer is replaying are queued FIFO for that consumer.
Because append order is globally serialized and the historic replay range ends at the captured registrationHead, a general sorted merge is unnecessary.
Consumer states
Add:
CATCHING_UP
ONLINE
DEGRADED
Do not add OFFLINE in this issue.
CATCHING_UP
A newly registered subscription begins in:
CATCHING_UP
The journal does not need to call:
requestState(CATCHING_UP)
during initial registration.
While catching up, the consumer may reconstruct state but must not execute actions dependent on being current.
ONLINE
After historical replay and pending events have been processed through an atomic cutover point, call:
requestState(ONLINE)
The semantic meaning is:
From and including successful return from this call, the service may begin executing actions.
requestState(ONLINE) itself should establish readiness.
It should not perform irreversible domain actions.
If an event arrives concurrently with the ONLINE cutover, the implementation must guarantee one of:
the event is processed before ONLINE, or
it becomes the first normal event processed after ONLINE.
No event may fall into an undefined gap between catch-up and online operation.
If requestState(ONLINE) throws:
mark the subscription DEGRADED,
retain the last safe processedThrough,
stop further delivery,
wake waiting barriers.
DEGRADED
If onEvent() throws:
catch the exception,
mark this subscription DEGRADED,
keep processedThrough at the last safely processed sequence,
stop further event delivery to this consumer,
wake waiting consistency barriers,
allow other consumers to continue normally.
There is no automatic recovery from DEGRADED in v1.
The current recovery mechanism is a complete JVM restart.
Do not implement unregister/re-register recovery in this issue.
Atomic consumer update requirement
A consumer must update its externally visible model atomically for each event.
The contract is:
Either the event is applied completely, or externally visible consumer state remains exactly as it was before onEvent() was called.
The consumer is responsible for its own transactional/rollback mechanism.
This matters because a DEGRADED consumer may still serve its last known-good state for consistency requests that do not require progress beyond its safe cursor.
processedThrough
Each subscription maintains:
processedThrough
Its meaning is:
Every journal record with sequence <= N has either been successfully handled if relevant to this consumer, or has been examined and determined irrelevant.
It is not the sequence number of the last event actually delivered to the consumer.
Example:
last relevant event = #1000
journal scanned through = #1100
processedThrough = #1100
This property prevents irrelevant events from blocking consistency barriers.
unrelated conversion of existing threads to virtual threads.
External checkpoints/signatures can be designed later.
The hash chain alone is intended to detect accidental modification and provide a chain that can later be anchored externally.
It is not intended by itself to prevent an attacker with full write access from changing an old record and recomputing all subsequent hashes.
OpenSpec
Create/update the necessary OpenSpec artifacts for this feature.
The specification must capture the behavioural requirements and invariants from this issue, especially:
append-only authoritative journal semantics,
durable append before publication,
global sequence ordering,
PDE top-level record framing,
record checksum/hash chain,
exact schema validation,
separation of the three validation layers,
maximum 1 KiB record size,
reusable bounded per-reader buffer,
event codec/version abstraction,
independent consumer replay,
subscription filtering,
processedThrough,
consistency barriers,
consumer state lifecycle,
atomic ONLINE cutover,
DEGRADED behaviour,
format changes represented through normal hash-chained events.
Do not describe event versions by encoding version numbers into Java domain class names.
Implementation constraints
Follow the existing repository architecture and Nenjim component/registry wiring conventions.
In particular:
codecs are normal components obtained through the registry,
AssetAZJournalService must not construct concrete event codecs,
preserve API/implementation separation consistent with the existing services,
use the repository's established @NotNull / @Nullable conventions,
avoid unrelated refactoring.
The implementation should follow the current authoritative Gitea repository and branch rather than an old checkout.
Do not stage, commit or push changes unless explicitly requested separately.
Acceptance criteria
The issue is complete when:
AssetAZJournalService can create/open an authoritative append-only PDE journal.
Each record is exactly one bounded top-level PDE object.
Records larger than 1024 bytes are rejected on both write and read.
Each reader reuses one fixed 1024-byte record buffer.
Decoders are bounded by logical offset + length, never backing-array length.
Fields 2..6 are encoded exactly once and their exact bytes are SHA-256 hashed.
checksum and previousChecksum are exactly 32 raw bytes.
The checksum chain and global sequence chain are validated during loading.
Incomplete or unverifiable journal history prevents successful startup.
Events are published only after durable FileChannel.force(true) persistence.
Uncertain append failure makes the journal fail closed for the current JVM.
Event types use stable short IDs and persistent versions independent of Java class names.
AssetOwnerRegistered exists without a version suffix in its Java type name.
AssetOwnerType is persisted using stable textual values.
AssetOwnerRegistered uses the exact seven-field positional payload schema defined above.
Exact field count is enforced; extra fields are rejected.
PDE, journal-integrity and event-schema validation responsibilities remain separated.
Consumers can independently replay and subsequently process newly appended events in sequence order.
Each consumer is processed sequentially by its own virtual-thread worker.
Event filtering and processedThrough behave according to this issue.
awaitProcessedThrough() provides the specified consistency barrier.
Consumer callback/state-transition failure produces isolated DEGRADED state without stopping unrelated consumers.
requestState(ONLINE) establishes the point from which actions are permitted.
ONLINE transition is atomic relative to newly appended events.
Format-version changes can be represented by ordinary events in the same record/hash chain.
Necessary OpenSpec artifacts are updated consistently with the implementation.
No out-of-scope recovery, snapshotting, generalized serialization or automatic tuning functionality is added.
## Summary
Implement the first version of `AssetAZJournalService`.
The service is the authoritative append-only event journal for AssetAZ. It must provide:
- durable event persistence,
- deterministic global ordering,
- integrity verification,
- replay,
- live event delivery,
- per-consumer progress tracking,
- consistency barriers between services,
- isolated degraded-state handling.
The journal is also the foundation from which stateful AssetAZ services can reconstruct their models after restart.
This issue intentionally defines a strict v1. Do not add recovery, snapshotting, automatic tuning, generalized object serialization, or other functionality not described here.
---
## Architectural principles
An AssetAZ event represents a **fact/state change**, not a command.
Replay must rebuild state only. Replay must never re-execute historical side effects such as trades, transfers or notifications.
If multiple state changes are business-atomically inseparable, represent them as one event rather than several correlated events.
`correlationId` groups independently meaningful events belonging to the same logical workflow. It does not provide atomicity and does not imply an expected number of events.
Journal sequence numbers define the authoritative total ordering. Event timestamps do not.
---
## AssetAZ events
Add the common event abstraction:
```java
public interface AssetAZEvent {
@NotNull
Instant occurredAt();
@Nullable
String note();
@Nullable
UUID correlationId();
}
```
`occurredAt` is the semantic time when the event happened or the situation was observed/registered.
It must never be used to establish journal ordering.
It is legal for a later journal sequence to contain an earlier `occurredAt`.
### First domain event: `AssetOwnerRegistered`
Add:
```java
public enum AssetOwnerType {
NATURAL_PERSON,
LEGAL_PERSON
}
```
The persistent representation of `AssetOwnerType` must use these stable textual values.
Never persist Java enum ordinals.
Add:
```java
public record AssetOwnerRegistered(
@NotNull Instant occurredAt,
@NotNull UUID assetOwnerId,
@NotNull AssetOwnerType assetOwnerType,
@NotNull String name,
@Nullable String note,
@Nullable UUID correlationId
) implements AssetAZEvent {
}
```
An asset owner represents the juridical/legal owner of assets.
Do not add addresses, national identifiers, accounting currency, accounts or other owner metadata in this issue.
### Important event-versioning rule
The Java domain class is:
```text
AssetOwnerRegistered
```
It must **not** be named:
```text
AssetOwnerRegisteredV1
AssetOwnerRegistered_v1
...
```
Multiple persistent versions of an event are an implementation concern handled by codecs, journal metadata, contexts/classloaders and the surrounding Nenjim architecture.
Persistent event versioning must be abstracted away from the Java domain type.
---
## Event codecs
Persisted event identity consists of:
```text
eventType
eventVersion
```
For the first persistent representation of `AssetOwnerRegistered`:
```text
eventType = "AssetOwnerRegistered"
eventVersion = 1
```
`eventType` must be a stable short identifier, not a Java fully-qualified class name.
Java packages/classes must be refactorable without changing already persisted journal data.
A codec is responsible for converting between one persistent event schema and the current Java domain event.
The journal service must obtain codecs through the normal Nenjim registry/component wiring mechanism.
Do not hardcode or construct concrete codecs inside `AssetAZJournalService`.
Multiple codecs for historical persistent versions may eventually coexist.
Unknown `(eventType, eventVersion)` while reading the journal is a fatal error.
Never silently skip an unknown event.
Attempting to append an event for which no appropriate codec exists must fail before writing anything.
---
## `AssetOwnerRegistered` persistent payload
The first persistent payload format is positional.
Do not persist field names.
Fields are stored in exactly this order:
```text
1. occurredAt epoch second
2. occurredAt nano-of-second
3. note
4. correlationId
5. assetOwnerId
6. assetOwnerType
7. name
```
Represent `occurredAt` using:
```text
signed 64-bit epoch seconds since 1970-01-01T00:00:00Z
nano-of-second: 0..999,999,999
```
UUIDs are persisted as exactly 16 bytes:
```text
bytes 0..7 = UUID.getMostSignificantBits()
bytes 8..15 = UUID.getLeastSignificantBits()
```
Both 64-bit values are big-endian.
Nullable UUIDs use PDE typed null.
`note` is nullable UTF-8 using PDE typed null.
The codec must reject payloads containing either fewer or more fields than the exact schema.
Unexpected trailing fields must never be silently ignored.
---
## PDE encoding layer
Implement a small reusable PDE encoder/decoder based on Jakob Jenkov's Polymorph Data Encoding.
References:
- Polymorph Data overview:
`https://www.jenkov.com/tutorials/polymorph-data/index.html`
- PDE encoding:
`https://www.jenkov.com/tutorials/polymorph-data/polymorph-data-encoding.html`
- PDL:
`https://www.jenkov.com/tutorials/polymorph-data/polymorph-data-language.html`
PDE is the authoritative binary representation.
PDL tooling is out of scope for this issue.
### PDE abstraction boundary
`PDEEncoder` and `PDEDecoder` must deal only with PDE primitives and PDE structural framing.
They must know nothing about:
```text
UUID
Instant
AssetAZEvent
AssetOwnerRegistered
AssetOwnerType
journal sequences
checksums
```
Those abstractions belong above the PDE layer.
Convenience conversion code for UUID, `Instant`, etc. may exist inside codec implementations or suitable codec-level helpers, but must not become PDE semantics.
The minimal API should support the operations required by this issue, conceptually including:
```java
public interface PDEEncoder {
void writeInteger(long value);
void writeUtf8(@NotNull String value);
void writeNullableUtf8(@Nullable String value);
void writeBytes(@NotNull byte[] value);
void writeNullableBytes(@Nullable byte[] value);
void writeObject(@NotNull Consumer<PDEEncoder> writer);
}
```
Decoder-side functionality should conceptually include:
```java
public interface PDEDecoder {
int readInt();
long readLong();
@NotNull
String readUtf8();
@Nullable
String readNullableUtf8();
@NotNull
byte[] readBytes();
@Nullable
byte[] readNullableBytes();
@NotNull
PDEDecoder readObject();
void requireEnd();
}
```
`readInt()` and `readLong()` read the applicable PDE integer representation but expose useful Java primitive ranges to callers.
Do not turn this into a general object mapper.
Do not add:
- reflection-based serialization,
- annotations-as-schema,
- automatic Java record serialization,
- Jackson-like machinery,
- functionality not currently required by AssetAZ.
---
## Three validation layers
Validation responsibilities must remain explicitly separated.
### 1. Journal / record integrity
Owned by the journal/record layer.
Examples:
- SHA-256 checksum,
- `previousChecksum`,
- sequence continuity,
- top-level record framing,
- truncated records,
- maximum record size.
Event codecs must not perform these checks.
### 2. PDE validity
Owned by the PDE decoder/framing implementation.
Examples:
- valid PDE representation,
- valid PDE primitive type,
- valid object/value framing,
- valid encoded lengths,
- valid UTF-8,
- typed-null handling,
- conversion to the requested Java primitive range.
Event codecs must not implement PDE parsing themselves.
### 3. Event/schema validity
Owned by the concrete event codec.
For `AssetOwnerRegistered`, examples include:
- exactly seven payload values,
- nano-of-second is within `0..999999999`,
- UUID byte sequences are exactly 16 bytes,
- `AssetOwnerType` is one of the known stable textual values,
- required values are non-null,
- no unexpected trailing payload values.
`PDEDecoder.requireEnd()` provides the mechanism for detecting remaining values, while the codec invokes it because the codec owns knowledge of where its schema ends.
A failure in any of these layers makes journal loading fail in v1.
Diagnostics should clearly identify which layer rejected the data.
---
## Journal record format
Every journal record is exactly one top-level PDE `OBJECT`.
The file therefore consists conceptually of:
```text
OBJECT record #1
OBJECT record #2
OBJECT record #3
...
```
The top-level object framing must allow a reader to determine the complete byte length of the next record before semantically decoding its event payload.
Journal format version 1 contains these fields in exactly this logical order:
```text
1. checksum
2. previousChecksum
3. sequence
4. eventType
5. eventVersion
6. payload
```
`payload` is itself a PDE object containing the concrete event payload.
---
## Checksum
Both:
```text
checksum
previousChecksum
```
are exactly 32 raw binary bytes.
Do not persist hashes as hexadecimal strings.
The checksum is:
```text
SHA-256 over the exact persisted PDE bytes representing fields 2..6
```
It must be calculated over the original encoded bytes.
Do not decode and re-encode values before calculating or validating the checksum.
The first record uses:
```text
SHA-256(empty byte sequence)
```
as `previousChecksum`.
That value is:
```text
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
```
but it must be stored as the corresponding 32 binary bytes, not as hex text.
---
## Sequence
Journal sequence numbers are globally monotonically increasing and start at:
```text
1
```
There must be no intentional gaps.
The sequence number becomes committed only when the corresponding record has been successfully and durably persisted.
If persistence fails, that candidate sequence number is not considered consumed.
---
## Maximum record size
Use a simple hardcoded implementation safety limit:
```java
private static final int MAX_RECORD_SIZE_BYTES = 1024;
```
The limit applies to the **entire persisted top-level PDE record**, including its object header.
No automatic resizing or adaptive calculation is required.
If a legitimate event eventually approaches the limit, the constant can be manually increased in a future change.
Do not implement configuration or automatic growth.
### Writing
The fully encoded record must be rejected before any bytes are appended if:
```text
record size > MAX_RECORD_SIZE_BYTES
```
### Reading
After reading enough PDE framing to determine the declared top-level object length, reject it immediately if:
```text
record size > MAX_RECORD_SIZE_BYTES
```
Also reject a record whose declared size extends beyond the remaining bytes in the journal file.
---
## Reader buffer
Each journal reader should allocate one fixed buffer:
```java
new byte[MAX_RECORD_SIZE_BYTES]
```
and reuse that same buffer for every record it reads.
Do not allocate a new record-sized `byte[]` for every historical record.
Do not implement shared buffer pools.
Each independent reader owns its own buffer.
A decoder must always operate on a bounded byte range:
```text
byte[]
offset
length
```
and never treat the entire backing array as valid data.
For example, if a 314-byte record has been read into a 1024-byte reusable buffer, bytes `314..1023` may still contain data from an older record and must be completely invisible to the decoder.
`requireEnd()` means:
```text
current position == logical decoder limit
```
not:
```text
current position == backing byte[].length
```
Child/object decoders must be bounded in the same way.
No buffer clearing such as `Arrays.fill()` is required.
---
## Reader flow
Reading a record should conceptually proceed as follows:
```text
1. Read enough PDE framing to identify the top-level OBJECT and its length.
2. Validate the declared record length against:
- MAX_RECORD_SIZE_BYTES
- remaining journal file size.
3. Read exactly that record into the reader's reusable buffer.
4. Identify the first field as the stored checksum and locate the
exact byte range containing fields 2..6.
5. Validate SHA-256 directly over those original bytes.
6. Decode the remaining journal envelope.
7. Validate previousChecksum and sequence continuity.
8. Resolve eventType + eventVersion to a codec.
9. Give only the payload decoder to that codec.
10. The codec decodes and validates its exact event schema.
```
Checksum validation occurs before semantic event parsing.
Minimal PDE framing necessarily happens before checksum validation because the reader must determine record boundaries and locate the checksum/body byte ranges.
---
## Writer flow
Fields 2..6 must be encoded exactly once.
Conceptually:
```text
encode fields 2..6
|
v
bodyBytes
|
+---- SHA-256 ----> checksum
|
v
encode checksum field
|
v
create top-level OBJECT framing
|
v
OBJECT_HEADER + checksumField + original bodyBytes
|
v
validate total size <= MAX_RECORD_SIZE_BYTES
|
v
append
|
v
FileChannel.force(true)
```
Do not decode/re-encode `bodyBytes`.
Copying already encoded bytes into the final record buffer is acceptable in v1.
Do not add generalized buffer pooling or similar allocation optimizations.
---
## Durability and append semantics
The journal file is authoritative.
In-memory journal head/state is derived/cache state only.
Appending must be serialized.
Conceptual append ordering:
```text
1. candidate sequence = current head + 1
2. encode complete record
3. validate record size
4. append bytes
5. FileChannel.force(true)
6. commit new journal head in memory
7. event officially exists
8. make event available to consumers
9. return sequence
```
Consumers must never observe an event before durable persistence has succeeded.
### Uncertain write failure
If an append fails in a way that may have left a partial or uncertain journal tail, fail closed.
For the remainder of that JVM:
- stop accepting new events,
- do not retry automatically,
- do not truncate automatically,
- do not attempt automatic repair.
A subsequent startup must validate the complete authoritative journal and must fail if consistency cannot be guaranteed.
Manual repair is acceptable in v1.
---
## Startup validation
On startup, validate the authoritative journal sufficiently to guarantee that its complete history is structurally and cryptographically consistent.
Validate at least:
- record framing,
- record size,
- PDE validity,
- all checksums,
- `previousChecksum` chain,
- exact sequence continuity,
- known event type/version codecs,
- exact event schemas.
If the journal cannot be proven consistent, `AssetAZJournalService` must not start successfully.
---
## Journal format version changes
Journal format version and event version are separate concepts.
`eventVersion` versions the persistent schema of one concrete event.
Journal format version controls interpretation of the surrounding journal/common event representation.
A new journal implicitly begins in journal format version 1.
Do not require an initial version-1 header.
Future journal-format switches are represented as ordinary events in the same normal journal record envelope and checksum chain.
Conceptually:
```text
format 1 records
JournalFormatVersionChanged(2)
format 2 records
JournalFormatVersionChanged(3)
format 3 records
JournalFormatVersionChanged(1)
format 1 records
...
```
Switching back to an earlier version must be representable.
Do not introduce a separate physical record type or separate unchained file header for format changes.
The format-change event itself must be integrity protected exactly like any other journal record.
The parser may react to such an event and change its parsing state for subsequent records.
The record parser may itself be an event receiver for this purpose.
Do not create an elaborate separate system-event hierarchy in v1 merely for this functionality.
---
## Consumer API
Use one callback for both replayed and newly appended events.
There is deliberately no replay/live distinction in the event callback.
Conceptually:
```java
public interface AssetAZJournalConsumer {
void onEvent(
long sequence,
@NotNull AssetAZEvent event
) throws Exception;
void requestState(
@NotNull AssetAZJournalConsumerState state
) throws Exception;
}
```
Historical events and newly arriving events are the same facts.
The difference is consumer state, not callback type.
---
## Consumer registration
Conceptually provide:
```java
@NotNull
AssetAZJournalSubscription registerConsumer(
long replayFromSequence,
@NotNull Set<Class<? extends AssetAZEvent>> eventTypes,
@NotNull AssetAZJournalConsumer consumer
);
```
### `replayFromSequence`
This means:
> The first journal sequence that must be scanned.
For v1, without snapshots, stateful services normally use:
```text
1
```
Future snapshot support may allow:
```text
snapshot includes state through #5000
replayFromSequence = 5001
```
Validate:
```text
replayFromSequence >= 1
replayFromSequence <= registrationHead + 1
```
where `registrationHead` is captured atomically when registration is established.
Registration returns without waiting for replay to complete.
### Event filters
`eventTypes`:
- must not be null,
- must not contain null,
- must be defensively copied.
An empty set means:
```text
ALL event types
```
A non-empty set means exact Java runtime class matches only.
Use exact `event.getClass()` semantics.
No inheritance matching, wildcards or other implicit behaviour in v1.
Filtering is performed by the journal service.
The consumer only receives events it subscribed to.
---
## Duplicate consumer registration
The exact same consumer object instance may only be registered once.
Use object identity:
```java
==
```
not `equals()`.
Different objects of the same consumer class may be registered independently.
A second registration attempt for the same object must fail rather than overwrite the first registration.
A degraded registration remains registered in v1.
---
## Independent replay
Each consumer/subscription performs its own independent historical replay.
A late-registering consumer must not restart or disturb replay/live processing for existing consumers.
It is acceptable for ten consumers to independently read the same journal ten times during catch-up.
Each replaying subscription may own:
- its own journal reader/file descriptor,
- its own reusable 1 KiB record buffer,
- its own worker.
The simplicity and independence are intentional.
The historical reader is closed once catch-up is complete.
---
## Consumer worker and concurrency model
Use one Java virtual thread per consumer subscription.
Do not convert unrelated existing Nenjim startup/service code to virtual threads as part of this issue.
For one consumer:
```text
onEvent calls are always sequential
```
Never invoke the same consumer concurrently.
Different consumers may process concurrently.
New relevant events arriving while a consumer is replaying are queued FIFO for that consumer.
Because append order is globally serialized and the historic replay range ends at the captured `registrationHead`, a general sorted merge is unnecessary.
---
## Consumer states
Add:
```text
CATCHING_UP
ONLINE
DEGRADED
```
Do not add `OFFLINE` in this issue.
### CATCHING_UP
A newly registered subscription begins in:
```text
CATCHING_UP
```
The journal does not need to call:
```java
requestState(CATCHING_UP)
```
during initial registration.
While catching up, the consumer may reconstruct state but must not execute actions dependent on being current.
### ONLINE
After historical replay and pending events have been processed through an atomic cutover point, call:
```java
requestState(ONLINE)
```
The semantic meaning is:
> From and including successful return from this call, the service may begin executing actions.
`requestState(ONLINE)` itself should establish readiness.
It should not perform irreversible domain actions.
If an event arrives concurrently with the ONLINE cutover, the implementation must guarantee one of:
- the event is processed before ONLINE, or
- it becomes the first normal event processed after ONLINE.
No event may fall into an undefined gap between catch-up and online operation.
If `requestState(ONLINE)` throws:
- mark the subscription `DEGRADED`,
- retain the last safe `processedThrough`,
- stop further delivery,
- wake waiting barriers.
### DEGRADED
If `onEvent()` throws:
- catch the exception,
- mark this subscription `DEGRADED`,
- keep `processedThrough` at the last safely processed sequence,
- stop further event delivery to this consumer,
- wake waiting consistency barriers,
- allow other consumers to continue normally.
There is no automatic recovery from `DEGRADED` in v1.
The current recovery mechanism is a complete JVM restart.
Do not implement unregister/re-register recovery in this issue.
---
## Atomic consumer update requirement
A consumer must update its externally visible model atomically for each event.
The contract is:
> Either the event is applied completely, or externally visible consumer state remains exactly as it was before `onEvent()` was called.
The consumer is responsible for its own transactional/rollback mechanism.
This matters because a `DEGRADED` consumer may still serve its last known-good state for consistency requests that do not require progress beyond its safe cursor.
---
## `processedThrough`
Each subscription maintains:
```text
processedThrough
```
Its meaning is:
> Every journal record with sequence <= N has either been successfully handled if relevant to this consumer, or has been examined and determined irrelevant.
It is **not** the sequence number of the last event actually delivered to the consumer.
Example:
```text
last relevant event = #1000
journal scanned through = #1100
processedThrough = #1100
```
This property prevents irrelevant events from blocking consistency barriers.
---
## Subscription API / consistency barrier
Conceptually:
```java
public interface AssetAZJournalSubscription {
long getProcessedThroughSequence();
@NotNull
AssetAZJournalConsumerState getState();
void awaitProcessedThrough(long requiredSequence)
throws DegradedException;
}
```
Do not expose `registrationHead` as part of the public API.
`requiredSequence` expresses a minimum freshness requirement.
It does **not** request a historical snapshot.
If a service has already processed through sequence 1000 and a caller asks for state valid through 800, it may return its current state at 1000.
Typical service usage:
```java
public Something getSomething(long requiredSequence)
throws DegradedException {
journalSubscription.awaitProcessedThrough(requiredSequence);
return getSomethingFromCurrentModel();
}
```
This allows callers to require causal freshness without needing to know which event types the target service consumes.
### `awaitProcessedThrough` semantics
If:
```text
processedThrough >= requiredSequence
```
return immediately, including when state is `DEGRADED`.
If:
```text
processedThrough < requiredSequence
```
and state is:
```text
CATCHING_UP
ONLINE
```
wait.
If:
```text
processedThrough < requiredSequence
```
and state is:
```text
DEGRADED
```
throw `DegradedException` immediately.
Waiters must be awakened when degradation occurs and must not hang indefinitely.
---
## Live/catch-up cutover
At registration:
```text
registrationHead = current journal head
```
is captured atomically.
Historical replay scans:
```text
replayFromSequence .. registrationHead
```
Events appended after `registrationHead` are buffered FIFO for that subscription while replay continues.
The worker must not deliver those buffered events before the historical range through `registrationHead` has been completed.
After the historical range is complete, pending live events are drained in sequence order.
The transition to ONLINE must be atomic relative to incoming event publication.
The consumer must be fully caught up at the chosen cutover point.
No event may be lost, duplicated or delivered out of order across the catch-up/live boundary.
---
## Logging and diagnostics
When a consumer fails, log both:
- the exception from the consumer,
- useful JournalService context such as consumer identity, sequence and state.
Journal parsing errors should distinguish, where possible:
```text
record integrity failure
PDE decoding/framing failure
event/schema validation failure
unknown event codec
```
---
## Out of scope
Do **not** add the following as part of this issue:
- snapshots,
- snapshot persistence,
- automatic journal repair,
- automatic tail truncation,
- automatic retry after uncertain writes,
- `DEGRADED` consumer recovery,
- unregister/re-register,
- dynamic record-size tuning,
- configurable record-size limits,
- shared buffer pools,
- generalized object serialization,
- reflection-based codec discovery,
- automatic schema migration,
- PDL tooling,
- external/signed journal checkpoints,
- causation IDs,
- broader third-party event namespaces,
- process/service restart support,
- unrelated conversion of existing threads to virtual threads.
External checkpoints/signatures can be designed later.
The hash chain alone is intended to detect accidental modification and provide a chain that can later be anchored externally.
It is not intended by itself to prevent an attacker with full write access from changing an old record and recomputing all subsequent hashes.
---
## OpenSpec
Create/update the necessary OpenSpec artifacts for this feature.
The specification must capture the behavioural requirements and invariants from this issue, especially:
- append-only authoritative journal semantics,
- durable append before publication,
- global sequence ordering,
- PDE top-level record framing,
- record checksum/hash chain,
- exact schema validation,
- separation of the three validation layers,
- maximum 1 KiB record size,
- reusable bounded per-reader buffer,
- event codec/version abstraction,
- independent consumer replay,
- subscription filtering,
- `processedThrough`,
- consistency barriers,
- consumer state lifecycle,
- atomic ONLINE cutover,
- `DEGRADED` behaviour,
- format changes represented through normal hash-chained events.
Do not describe event versions by encoding version numbers into Java domain class names.
---
## Implementation constraints
Follow the existing repository architecture and Nenjim component/registry wiring conventions.
In particular:
- codecs are normal components obtained through the registry,
- `AssetAZJournalService` must not construct concrete event codecs,
- preserve API/implementation separation consistent with the existing services,
- use the repository's established `@NotNull` / `@Nullable` conventions,
- avoid unrelated refactoring.
The implementation should follow the current authoritative Gitea repository and branch rather than an old checkout.
Do not stage, commit or push changes unless explicitly requested separately.
---
## Acceptance criteria
The issue is complete when:
1. `AssetAZJournalService` can create/open an authoritative append-only PDE journal.
2. Each record is exactly one bounded top-level PDE object.
3. Records larger than 1024 bytes are rejected on both write and read.
4. Each reader reuses one fixed 1024-byte record buffer.
5. Decoders are bounded by logical `offset + length`, never backing-array length.
6. Fields 2..6 are encoded exactly once and their exact bytes are SHA-256 hashed.
7. `checksum` and `previousChecksum` are exactly 32 raw bytes.
8. The checksum chain and global sequence chain are validated during loading.
9. Incomplete or unverifiable journal history prevents successful startup.
10. Events are published only after durable `FileChannel.force(true)` persistence.
11. Uncertain append failure makes the journal fail closed for the current JVM.
12. Event types use stable short IDs and persistent versions independent of Java class names.
13. `AssetOwnerRegistered` exists without a version suffix in its Java type name.
14. `AssetOwnerType` is persisted using stable textual values.
15. `AssetOwnerRegistered` uses the exact seven-field positional payload schema defined above.
16. Exact field count is enforced; extra fields are rejected.
17. PDE, journal-integrity and event-schema validation responsibilities remain separated.
18. Consumers can independently replay and subsequently process newly appended events in sequence order.
19. Each consumer is processed sequentially by its own virtual-thread worker.
20. Event filtering and `processedThrough` behave according to this issue.
21. `awaitProcessedThrough()` provides the specified consistency barrier.
22. Consumer callback/state-transition failure produces isolated `DEGRADED` state without stopping unrelated consumers.
23. `requestState(ONLINE)` establishes the point from which actions are permitted.
24. ONLINE transition is atomic relative to newly appended events.
25. Format-version changes can be represented by ordinary events in the same record/hash chain.
26. Necessary OpenSpec artifacts are updated consistently with the implementation.
27. No out-of-scope recovery, snapshotting, generalized serialization or automatic tuning functionality is added.
````
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Summary
Implement the first version of
AssetAZJournalService.The service is the authoritative append-only event journal for AssetAZ. It must provide:
The journal is also the foundation from which stateful AssetAZ services can reconstruct their models after restart.
This issue intentionally defines a strict v1. Do not add recovery, snapshotting, automatic tuning, generalized object serialization, or other functionality not described here.
Architectural principles
An AssetAZ event represents a fact/state change, not a command.
Replay must rebuild state only. Replay must never re-execute historical side effects such as trades, transfers or notifications.
If multiple state changes are business-atomically inseparable, represent them as one event rather than several correlated events.
correlationIdgroups independently meaningful events belonging to the same logical workflow. It does not provide atomicity and does not imply an expected number of events.Journal sequence numbers define the authoritative total ordering. Event timestamps do not.
AssetAZ events
Add the common event abstraction:
occurredAtis the semantic time when the event happened or the situation was observed/registered.It must never be used to establish journal ordering.
It is legal for a later journal sequence to contain an earlier
occurredAt.First domain event:
AssetOwnerRegisteredAdd:
The persistent representation of
AssetOwnerTypemust use these stable textual values.Never persist Java enum ordinals.
Add:
An asset owner represents the juridical/legal owner of assets.
Do not add addresses, national identifiers, accounting currency, accounts or other owner metadata in this issue.
Important event-versioning rule
The Java domain class is:
It must not be named:
Multiple persistent versions of an event are an implementation concern handled by codecs, journal metadata, contexts/classloaders and the surrounding Nenjim architecture.
Persistent event versioning must be abstracted away from the Java domain type.
Event codecs
Persisted event identity consists of:
For the first persistent representation of
AssetOwnerRegistered:eventTypemust be a stable short identifier, not a Java fully-qualified class name.Java packages/classes must be refactorable without changing already persisted journal data.
A codec is responsible for converting between one persistent event schema and the current Java domain event.
The journal service must obtain codecs through the normal Nenjim registry/component wiring mechanism.
Do not hardcode or construct concrete codecs inside
AssetAZJournalService.Multiple codecs for historical persistent versions may eventually coexist.
Unknown
(eventType, eventVersion)while reading the journal is a fatal error.Never silently skip an unknown event.
Attempting to append an event for which no appropriate codec exists must fail before writing anything.
AssetOwnerRegisteredpersistent payloadThe first persistent payload format is positional.
Do not persist field names.
Fields are stored in exactly this order:
Represent
occurredAtusing:UUIDs are persisted as exactly 16 bytes:
Both 64-bit values are big-endian.
Nullable UUIDs use PDE typed null.
noteis nullable UTF-8 using PDE typed null.The codec must reject payloads containing either fewer or more fields than the exact schema.
Unexpected trailing fields must never be silently ignored.
PDE encoding layer
Implement a small reusable PDE encoder/decoder based on Jakob Jenkov's Polymorph Data Encoding.
References:
https://www.jenkov.com/tutorials/polymorph-data/index.htmlhttps://www.jenkov.com/tutorials/polymorph-data/polymorph-data-encoding.htmlhttps://www.jenkov.com/tutorials/polymorph-data/polymorph-data-language.htmlPDE is the authoritative binary representation.
PDL tooling is out of scope for this issue.
PDE abstraction boundary
PDEEncoderandPDEDecodermust deal only with PDE primitives and PDE structural framing.They must know nothing about:
Those abstractions belong above the PDE layer.
Convenience conversion code for UUID,
Instant, etc. may exist inside codec implementations or suitable codec-level helpers, but must not become PDE semantics.The minimal API should support the operations required by this issue, conceptually including:
Decoder-side functionality should conceptually include:
readInt()andreadLong()read the applicable PDE integer representation but expose useful Java primitive ranges to callers.Do not turn this into a general object mapper.
Do not add:
Three validation layers
Validation responsibilities must remain explicitly separated.
1. Journal / record integrity
Owned by the journal/record layer.
Examples:
previousChecksum,Event codecs must not perform these checks.
2. PDE validity
Owned by the PDE decoder/framing implementation.
Examples:
Event codecs must not implement PDE parsing themselves.
3. Event/schema validity
Owned by the concrete event codec.
For
AssetOwnerRegistered, examples include:0..999999999,AssetOwnerTypeis one of the known stable textual values,PDEDecoder.requireEnd()provides the mechanism for detecting remaining values, while the codec invokes it because the codec owns knowledge of where its schema ends.A failure in any of these layers makes journal loading fail in v1.
Diagnostics should clearly identify which layer rejected the data.
Journal record format
Every journal record is exactly one top-level PDE
OBJECT.The file therefore consists conceptually of:
The top-level object framing must allow a reader to determine the complete byte length of the next record before semantically decoding its event payload.
Journal format version 1 contains these fields in exactly this logical order:
payloadis itself a PDE object containing the concrete event payload.Checksum
Both:
are exactly 32 raw binary bytes.
Do not persist hashes as hexadecimal strings.
The checksum is:
It must be calculated over the original encoded bytes.
Do not decode and re-encode values before calculating or validating the checksum.
The first record uses:
as
previousChecksum.That value is:
but it must be stored as the corresponding 32 binary bytes, not as hex text.
Sequence
Journal sequence numbers are globally monotonically increasing and start at:
There must be no intentional gaps.
The sequence number becomes committed only when the corresponding record has been successfully and durably persisted.
If persistence fails, that candidate sequence number is not considered consumed.
Maximum record size
Use a simple hardcoded implementation safety limit:
The limit applies to the entire persisted top-level PDE record, including its object header.
No automatic resizing or adaptive calculation is required.
If a legitimate event eventually approaches the limit, the constant can be manually increased in a future change.
Do not implement configuration or automatic growth.
Writing
The fully encoded record must be rejected before any bytes are appended if:
Reading
After reading enough PDE framing to determine the declared top-level object length, reject it immediately if:
Also reject a record whose declared size extends beyond the remaining bytes in the journal file.
Reader buffer
Each journal reader should allocate one fixed buffer:
and reuse that same buffer for every record it reads.
Do not allocate a new record-sized
byte[]for every historical record.Do not implement shared buffer pools.
Each independent reader owns its own buffer.
A decoder must always operate on a bounded byte range:
and never treat the entire backing array as valid data.
For example, if a 314-byte record has been read into a 1024-byte reusable buffer, bytes
314..1023may still contain data from an older record and must be completely invisible to the decoder.requireEnd()means:not:
Child/object decoders must be bounded in the same way.
No buffer clearing such as
Arrays.fill()is required.Reader flow
Reading a record should conceptually proceed as follows:
Checksum validation occurs before semantic event parsing.
Minimal PDE framing necessarily happens before checksum validation because the reader must determine record boundaries and locate the checksum/body byte ranges.
Writer flow
Fields 2..6 must be encoded exactly once.
Conceptually:
Do not decode/re-encode
bodyBytes.Copying already encoded bytes into the final record buffer is acceptable in v1.
Do not add generalized buffer pooling or similar allocation optimizations.
Durability and append semantics
The journal file is authoritative.
In-memory journal head/state is derived/cache state only.
Appending must be serialized.
Conceptual append ordering:
Consumers must never observe an event before durable persistence has succeeded.
Uncertain write failure
If an append fails in a way that may have left a partial or uncertain journal tail, fail closed.
For the remainder of that JVM:
A subsequent startup must validate the complete authoritative journal and must fail if consistency cannot be guaranteed.
Manual repair is acceptable in v1.
Startup validation
On startup, validate the authoritative journal sufficiently to guarantee that its complete history is structurally and cryptographically consistent.
Validate at least:
previousChecksumchain,If the journal cannot be proven consistent,
AssetAZJournalServicemust not start successfully.Journal format version changes
Journal format version and event version are separate concepts.
eventVersionversions the persistent schema of one concrete event.Journal format version controls interpretation of the surrounding journal/common event representation.
A new journal implicitly begins in journal format version 1.
Do not require an initial version-1 header.
Future journal-format switches are represented as ordinary events in the same normal journal record envelope and checksum chain.
Conceptually:
Switching back to an earlier version must be representable.
Do not introduce a separate physical record type or separate unchained file header for format changes.
The format-change event itself must be integrity protected exactly like any other journal record.
The parser may react to such an event and change its parsing state for subsequent records.
The record parser may itself be an event receiver for this purpose.
Do not create an elaborate separate system-event hierarchy in v1 merely for this functionality.
Consumer API
Use one callback for both replayed and newly appended events.
There is deliberately no replay/live distinction in the event callback.
Conceptually:
Historical events and newly arriving events are the same facts.
The difference is consumer state, not callback type.
Consumer registration
Conceptually provide:
replayFromSequenceThis means:
For v1, without snapshots, stateful services normally use:
Future snapshot support may allow:
Validate:
where
registrationHeadis captured atomically when registration is established.Registration returns without waiting for replay to complete.
Event filters
eventTypes:An empty set means:
A non-empty set means exact Java runtime class matches only.
Use exact
event.getClass()semantics.No inheritance matching, wildcards or other implicit behaviour in v1.
Filtering is performed by the journal service.
The consumer only receives events it subscribed to.
Duplicate consumer registration
The exact same consumer object instance may only be registered once.
Use object identity:
not
equals().Different objects of the same consumer class may be registered independently.
A second registration attempt for the same object must fail rather than overwrite the first registration.
A degraded registration remains registered in v1.
Independent replay
Each consumer/subscription performs its own independent historical replay.
A late-registering consumer must not restart or disturb replay/live processing for existing consumers.
It is acceptable for ten consumers to independently read the same journal ten times during catch-up.
Each replaying subscription may own:
The simplicity and independence are intentional.
The historical reader is closed once catch-up is complete.
Consumer worker and concurrency model
Use one Java virtual thread per consumer subscription.
Do not convert unrelated existing Nenjim startup/service code to virtual threads as part of this issue.
For one consumer:
Never invoke the same consumer concurrently.
Different consumers may process concurrently.
New relevant events arriving while a consumer is replaying are queued FIFO for that consumer.
Because append order is globally serialized and the historic replay range ends at the captured
registrationHead, a general sorted merge is unnecessary.Consumer states
Add:
Do not add
OFFLINEin this issue.CATCHING_UP
A newly registered subscription begins in:
The journal does not need to call:
during initial registration.
While catching up, the consumer may reconstruct state but must not execute actions dependent on being current.
ONLINE
After historical replay and pending events have been processed through an atomic cutover point, call:
The semantic meaning is:
requestState(ONLINE)itself should establish readiness.It should not perform irreversible domain actions.
If an event arrives concurrently with the ONLINE cutover, the implementation must guarantee one of:
No event may fall into an undefined gap between catch-up and online operation.
If
requestState(ONLINE)throws:DEGRADED,processedThrough,DEGRADED
If
onEvent()throws:DEGRADED,processedThroughat the last safely processed sequence,There is no automatic recovery from
DEGRADEDin v1.The current recovery mechanism is a complete JVM restart.
Do not implement unregister/re-register recovery in this issue.
Atomic consumer update requirement
A consumer must update its externally visible model atomically for each event.
The contract is:
The consumer is responsible for its own transactional/rollback mechanism.
This matters because a
DEGRADEDconsumer may still serve its last known-good state for consistency requests that do not require progress beyond its safe cursor.processedThroughEach subscription maintains:
Its meaning is:
It is not the sequence number of the last event actually delivered to the consumer.
Example:
This property prevents irrelevant events from blocking consistency barriers.
Subscription API / consistency barrier
Conceptually:
Do not expose
registrationHeadas part of the public API.requiredSequenceexpresses a minimum freshness requirement.It does not request a historical snapshot.
If a service has already processed through sequence 1000 and a caller asks for state valid through 800, it may return its current state at 1000.
Typical service usage:
This allows callers to require causal freshness without needing to know which event types the target service consumes.
awaitProcessedThroughsemanticsIf:
return immediately, including when state is
DEGRADED.If:
and state is:
wait.
If:
and state is:
throw
DegradedExceptionimmediately.Waiters must be awakened when degradation occurs and must not hang indefinitely.
Live/catch-up cutover
At registration:
is captured atomically.
Historical replay scans:
Events appended after
registrationHeadare buffered FIFO for that subscription while replay continues.The worker must not deliver those buffered events before the historical range through
registrationHeadhas been completed.After the historical range is complete, pending live events are drained in sequence order.
The transition to ONLINE must be atomic relative to incoming event publication.
The consumer must be fully caught up at the chosen cutover point.
No event may be lost, duplicated or delivered out of order across the catch-up/live boundary.
Logging and diagnostics
When a consumer fails, log both:
Journal parsing errors should distinguish, where possible:
Out of scope
Do not add the following as part of this issue:
DEGRADEDconsumer recovery,External checkpoints/signatures can be designed later.
The hash chain alone is intended to detect accidental modification and provide a chain that can later be anchored externally.
It is not intended by itself to prevent an attacker with full write access from changing an old record and recomputing all subsequent hashes.
OpenSpec
Create/update the necessary OpenSpec artifacts for this feature.
The specification must capture the behavioural requirements and invariants from this issue, especially:
processedThrough,DEGRADEDbehaviour,Do not describe event versions by encoding version numbers into Java domain class names.
Implementation constraints
Follow the existing repository architecture and Nenjim component/registry wiring conventions.
In particular:
AssetAZJournalServicemust not construct concrete event codecs,@NotNull/@Nullableconventions,The implementation should follow the current authoritative Gitea repository and branch rather than an old checkout.
Do not stage, commit or push changes unless explicitly requested separately.
Acceptance criteria
The issue is complete when:
AssetAZJournalServicecan create/open an authoritative append-only PDE journal.offset + length, never backing-array length.checksumandpreviousChecksumare exactly 32 raw bytes.FileChannel.force(true)persistence.AssetOwnerRegisteredexists without a version suffix in its Java type name.AssetOwnerTypeis persisted using stable textual values.AssetOwnerRegistereduses the exact seven-field positional payload schema defined above.processedThroughbehave according to this issue.awaitProcessedThrough()provides the specified consistency barrier.DEGRADEDstate without stopping unrelated consumers.requestState(ONLINE)establishes the point from which actions are permitted.