Persist Evelyn status-index history per named Evelyn instance #65

Closed
opened 2026-08-08 22:55:42 +02:00 by minimons · 0 comments
Owner

Background

Issue #64 implemented live collection of the Evelyn Price Index. Each EvelynImpl currently keeps its collected EvelynStatusIndexPoint history only in memory, so the complete history is lost whenever Evelyn is stopped or the process restarts.

Persist the history in an append-only, human-readable status file owned by each Evelyn instance. Evelyn Mission Control must remain presentation-only and continue obtaining the history through Evelyn.getStatusIndexHistory().

Use commit f463715 from Gitea as the starting point.

Named Evelyn instances

Add an instance name to every EvelynImpl constructor.

Example:

Evelyn evelynMine =
        new EvelynImpl("Mine", tickerService, eveUsdt);

The normalized instance name determines the persistent data directory:

data/evelyn/Mine/status.log

Update NenjimHubImpl to construct the existing environments as:

new EvelynImpl("Production", tickerService, eveUsdt);
new EvelynImpl("Test", tickerService, eveUsdt);

This selects:

data/evelyn/Production/status.log
data/evelyn/Test/status.log

The instance name is the permanent data identity. Renaming an instance selects a different directory; existing data must not be moved automatically.

Instance-name validation

Instance names must support Unicode, including Danish characters such as æ, ø, and å.

Before use, the name must be normalized to Unicode NFC.

Reject a name when it:

  • Is empty or contains only whitespace.
  • Begins or ends with whitespace.
  • Is . or ...
  • Ends with a period.
  • Contains control characters.
  • Contains any of the portable filename-invalid characters:
< > : " / \ | ? *
  • Is a case-insensitive Windows-reserved filename such as CON, PRN, AUX, NUL, COM1COM9, or LPT1LPT9, including names with an extension.
  • Resolves after Path.normalize() to anything other than a direct child of data/evelyn.

Preserve the normalized name’s spelling and capitalization in the directory name.

Unique instance names within the JVM

Two EvelynImpl objects in the same JVM must not be allowed to own the same persistent identity.

Maintain a static, thread-safe registry of reserved instance names. Reservation must be atomic so concurrent constructor calls cannot both reserve the same name.

Registry comparison must be:

  • Case-insensitive using Locale.ROOT.
  • Based on the NFC-normalized name.

Consequently, names such as Production and production must conflict.

Validate all constructor arguments before reserving the name so a failed constructor does not leak a reservation.

Calling stop() must not release the name. A stopped object may later be started again and must retain exclusive ownership of its persistent file.

The JVM-local registry does not need to protect against another Java process using the same directory. Cross-process file locking is outside this task.

Permanent close lifecycle

Extend the public Evelyn lifecycle with AutoCloseable or an explicit close() operation.

Lifecycle behavior must be:

  • stop() stops sampling, clears the in-memory history, retains the persistent file, and retains the instance-name reservation.
  • A later start() reloads the persistent history before sampling resumes.
  • close() permanently closes the object and releases its reserved name.
  • close() is idempotent.
  • Closing an active instance first stops its sampling thread safely.
  • A successfully closed instance cannot be started again.
  • The name must not be released unless the sampling thread has definitely terminated.
  • If safe termination fails, report the failure and retain the name reservation so another instance cannot begin writing to the same file.

Status-file format

Use an append-only UTF-8 text file:

data/evelyn/<normalized-instance-name>/status.log

Version 1 of the format is:

FORMAT_VERSION=1

# <UTC timestamp>:<type>:<type-specific fields...>
20260808190950132Z:1:-0.7

The timestamp format is:

uuuuMMddHHmmssSSS'Z'

It represents a UTC instant with millisecond precision.

Type 1 means Evelyn IOU Token Price Index. Its single type-specific field is the calculated index value.

Type identifiers must be explicit, stable numeric codes. Do not use a Java enum ordinal as the persisted identifier.

Persist BigDecimal values using toPlainString().

Blank lines and comments beginning with # must be ignored consistently with the existing Ticker history format. Text following # on a data line may also be ignored as an inline comment.

Every non-empty, non-comment entry after the version declaration must be a valid data record with the exact number of fields required by its type.

File-format version validation

The status file has its own format version, independent of every configuration file and Ticker history format.

The implementation must hardcode:

SUPPORTED_STATUS_HISTORY_FORMAT_VERSION=1

The declaration:

FORMAT_VERSION=1

must be the first actual file entry. A newly provisioned file must have it as its physical first line. As with the existing configuration convention, blank lines and full-line comments may precede the declaration when reading an existing file.

Loading must fail when the version declaration is:

  • Missing.
  • Duplicated.
  • Malformed.
  • Not the first actual entry.
  • Different from the supported version.

Version validation must happen before status records are parsed.

A breaking format change—including beginning to write a new record type that version 1 cannot understand—must increment both the file declaration and the loader’s hardcoded supported version.

Do not add a format declaration to the existing AssetAZ Ticker files as part of this task.

Existing storage and startup validation

Evelyn must not create its instance directory or status.log. Provisioning these is an explicit operator or deployment responsibility.

Before starting, Evelyn.start() must validate that:

  • data/evelyn/<instance-name> already exists.
  • The instance path is a directory.
  • The directory is readable and writable.
  • status.log already exists inside that directory.
  • status.log is a regular file.
  • status.log is readable and writable.
  • The file contains a valid FORMAT_VERSION declaration and valid persisted records.

If any validation fails:

  • start() must fail with a clear exception identifying the affected path and reason.
  • The sampling thread must not be started.
  • No in-memory history must be published.
  • Evelyn must not create, replace, repair, truncate, or otherwise modify the missing or invalid storage.

After filesystem validation succeeds:

  1. Validate the file-format version.
  2. Parse the complete file into a temporary history collection.
  3. Validate every record before publishing the loaded collection.
  4. Publish the complete validated collection as Evelyn’s in-memory history.
  5. Start the sampling thread only after loading has completed successfully.

If loading fails:

  • start() must fail.
  • The sampling thread must not be started.
  • A partially loaded history must not be exposed.
  • The exception must identify the file path, line number, and original line where applicable.
  • Evelyn must not silently repair, skip, sort, truncate, or otherwise modify invalid persisted data.

A newly provisioned empty history file must therefore be created externally with:

FORMAT_VERSION=1

A physically empty file is invalid because it has no format-version declaration.

Filesystem permission checks do not replace the actual file operations. If the directory or file passes the initial checks but opening, reading, appending, or flushing still fails, startup or sampling must fail according to the relevant error behavior.

Record parsing

For a type 1 record:

<timestamp>:1:<evelyn-price-index>

create an EvelynStatusIndexPoint containing:

  • The persisted timestamp.
  • The persisted Evelyn Price Index.
  • BigDecimal.ZERO for each of the other five not-yet-implemented indexes.

Unknown type identifiers must fail loading. They must not be ignored.

Malformed timestamps, invalid decimal values, missing fields, surplus fields, and otherwise malformed records must also fail loading.

Persisted records must be in non-decreasing timestamp order. Equal timestamps are allowed, but a record earlier than the preceding record must make the file invalid.

Do not sort the file during loading because that would hide corrupted or incorrectly written data.

Load the complete history. No size limit, retention policy, rotation, or compaction is required in this task.

A file containing only a valid format declaration represents a valid empty history.

Persist before publishing

For every successful sampling attempt:

  1. Calculate the new EvelynStatusIndexPoint as implemented by issue #64.
  2. Encode its type 1 record.
  3. Append the record to status.log.
  4. Flush it durably to the file.
  5. Only after persistence succeeds, add the point to the in-memory history visible through getStatusIndexHistory().

Use the same robust append behavior as Ticker persistence, including handling a file whose current last line does not end with a line separator.

Persistence and in-memory publication must be coordinated so callers cannot observe a point before it has been persisted.

If persistence fails:

  • Log the instance name, path, and relevant sampling context.
  • Do not add the point to memory.
  • Do not terminate Evelyn.
  • Allow the next scheduled sampling attempt to run normally.

A newly sampled timestamp earlier than the last persisted timestamp must not be appended. Treat it as a failed sampling attempt, log it, and continue with the next scheduled attempt.

Stop, restart, and EMC behavior

Stopping Evelyn must:

  • Terminate its sampling executor as before.
  • Clear its in-memory history.
  • Preserve status.log.
  • Preserve the instance-name reservation.

Restarting the same object must revalidate the storage and reload its complete history before beginning new sampling.

If the directory or file has been removed or is no longer readable and writable when start() is called again, restarting must fail in the same way as the initial start.

EMC must continue using getStatusIndexHistory() without reading files directly. When EMC opens after Evelyn starts, the restored history must therefore be available immediately.

Stopping or hiding EMC must not stop Evelyn sampling or persistence.

No change to the EvelynStatusIndexPoint public fields or EMC’s chart API is required.

OpenSpec

Create a new OpenSpec change for this issue.

Update the evelyn-status-index-history capability introduced by issue #64. It currently states that:

  • Each history begins empty.
  • Points are process-local.
  • No points are persisted or restored.
  • Stopping Evelyn permanently discards the complete history.

Replace those statements with the named-instance persistence and lifecycle behavior described by this issue.

The specification must cover at least:

  • Safe, portable Unicode instance names.
  • JVM-local uniqueness and permanent name reservation until close().
  • Per-instance data directories.
  • The requirement that directories and files are provisioned externally.
  • Startup failure when required storage is missing, invalid, unreadable, or unwritable.
  • The prohibition against Evelyn creating or repairing its storage.
  • Status format-version validation.
  • Strict record parsing and unknown-type rejection.
  • Loading before sampling starts.
  • Persist-before-publish behavior.
  • Stop, restart, and close semantics.
  • Independence from EMC’s lifecycle.

Do not change the AssetAZ Ticker or Raydium PriceSource specifications.

Verification

Do not add automated tests in this task.

Verify the implementation by:

  • Compiling main and test source sets.
  • Running strict OpenSpec validation.
  • Running git diff --check.
  • Reviewing the final diff for unrelated changes.
  • Confirming that only .tjava sources are edited where Detag-generated Java is involved.

Out of scope

  • Implementing the other five Evelyn indexes.
  • Persisting additional record types.
  • History rotation, retention, compaction, or database storage.
  • Migrating or renaming existing instance directories.
  • Automatically creating instance directories or status files.
  • Automatically repairing invalid status files.
  • Cross-process file locking.
  • Changing Ticker persistence.
  • Direct file access from Evelyn Mission Control.
  • Automated test coverage.
## Background Issue #64 implemented live collection of the Evelyn Price Index. Each `EvelynImpl` currently keeps its collected `EvelynStatusIndexPoint` history only in memory, so the complete history is lost whenever Evelyn is stopped or the process restarts. Persist the history in an append-only, human-readable status file owned by each Evelyn instance. Evelyn Mission Control must remain presentation-only and continue obtaining the history through `Evelyn.getStatusIndexHistory()`. Use commit `f463715` from Gitea as the starting point. ## Named Evelyn instances Add an instance name to every `EvelynImpl` constructor. Example: ```java Evelyn evelynMine = new EvelynImpl("Mine", tickerService, eveUsdt); ``` The normalized instance name determines the persistent data directory: ```text data/evelyn/Mine/status.log ``` Update `NenjimHubImpl` to construct the existing environments as: ```java new EvelynImpl("Production", tickerService, eveUsdt); new EvelynImpl("Test", tickerService, eveUsdt); ``` This selects: ```text data/evelyn/Production/status.log data/evelyn/Test/status.log ``` The instance name is the permanent data identity. Renaming an instance selects a different directory; existing data must not be moved automatically. ### Instance-name validation Instance names must support Unicode, including Danish characters such as `æ`, `ø`, and `å`. Before use, the name must be normalized to Unicode NFC. Reject a name when it: * Is empty or contains only whitespace. * Begins or ends with whitespace. * Is `.` or `..`. * Ends with a period. * Contains control characters. * Contains any of the portable filename-invalid characters: ```text < > : " / \ | ? * ``` * Is a case-insensitive Windows-reserved filename such as `CON`, `PRN`, `AUX`, `NUL`, `COM1`–`COM9`, or `LPT1`–`LPT9`, including names with an extension. * Resolves after `Path.normalize()` to anything other than a direct child of `data/evelyn`. Preserve the normalized name’s spelling and capitalization in the directory name. ## Unique instance names within the JVM Two `EvelynImpl` objects in the same JVM must not be allowed to own the same persistent identity. Maintain a static, thread-safe registry of reserved instance names. Reservation must be atomic so concurrent constructor calls cannot both reserve the same name. Registry comparison must be: * Case-insensitive using `Locale.ROOT`. * Based on the NFC-normalized name. Consequently, names such as `Production` and `production` must conflict. Validate all constructor arguments before reserving the name so a failed constructor does not leak a reservation. Calling `stop()` must not release the name. A stopped object may later be started again and must retain exclusive ownership of its persistent file. The JVM-local registry does not need to protect against another Java process using the same directory. Cross-process file locking is outside this task. ## Permanent close lifecycle Extend the public `Evelyn` lifecycle with `AutoCloseable` or an explicit `close()` operation. Lifecycle behavior must be: * `stop()` stops sampling, clears the in-memory history, retains the persistent file, and retains the instance-name reservation. * A later `start()` reloads the persistent history before sampling resumes. * `close()` permanently closes the object and releases its reserved name. * `close()` is idempotent. * Closing an active instance first stops its sampling thread safely. * A successfully closed instance cannot be started again. * The name must not be released unless the sampling thread has definitely terminated. * If safe termination fails, report the failure and retain the name reservation so another instance cannot begin writing to the same file. ## Status-file format Use an append-only UTF-8 text file: ```text data/evelyn/<normalized-instance-name>/status.log ``` Version 1 of the format is: ```text FORMAT_VERSION=1 # <UTC timestamp>:<type>:<type-specific fields...> 20260808190950132Z:1:-0.7 ``` The timestamp format is: ```text uuuuMMddHHmmssSSS'Z' ``` It represents a UTC instant with millisecond precision. Type `1` means **Evelyn IOU Token Price Index**. Its single type-specific field is the calculated index value. Type identifiers must be explicit, stable numeric codes. Do not use a Java enum ordinal as the persisted identifier. Persist `BigDecimal` values using `toPlainString()`. Blank lines and comments beginning with `#` must be ignored consistently with the existing Ticker history format. Text following `#` on a data line may also be ignored as an inline comment. Every non-empty, non-comment entry after the version declaration must be a valid data record with the exact number of fields required by its type. ## File-format version validation The status file has its own format version, independent of every configuration file and Ticker history format. The implementation must hardcode: ```text SUPPORTED_STATUS_HISTORY_FORMAT_VERSION=1 ``` The declaration: ```text FORMAT_VERSION=1 ``` must be the first actual file entry. A newly provisioned file must have it as its physical first line. As with the existing configuration convention, blank lines and full-line comments may precede the declaration when reading an existing file. Loading must fail when the version declaration is: * Missing. * Duplicated. * Malformed. * Not the first actual entry. * Different from the supported version. Version validation must happen before status records are parsed. A breaking format change—including beginning to write a new record type that version 1 cannot understand—must increment both the file declaration and the loader’s hardcoded supported version. Do not add a format declaration to the existing AssetAZ Ticker files as part of this task. ## Existing storage and startup validation Evelyn must not create its instance directory or `status.log`. Provisioning these is an explicit operator or deployment responsibility. Before starting, `Evelyn.start()` must validate that: * `data/evelyn/<instance-name>` already exists. * The instance path is a directory. * The directory is readable and writable. * `status.log` already exists inside that directory. * `status.log` is a regular file. * `status.log` is readable and writable. * The file contains a valid `FORMAT_VERSION` declaration and valid persisted records. If any validation fails: * `start()` must fail with a clear exception identifying the affected path and reason. * The sampling thread must not be started. * No in-memory history must be published. * Evelyn must not create, replace, repair, truncate, or otherwise modify the missing or invalid storage. After filesystem validation succeeds: 1. Validate the file-format version. 2. Parse the complete file into a temporary history collection. 3. Validate every record before publishing the loaded collection. 4. Publish the complete validated collection as Evelyn’s in-memory history. 5. Start the sampling thread only after loading has completed successfully. If loading fails: * `start()` must fail. * The sampling thread must not be started. * A partially loaded history must not be exposed. * The exception must identify the file path, line number, and original line where applicable. * Evelyn must not silently repair, skip, sort, truncate, or otherwise modify invalid persisted data. A newly provisioned empty history file must therefore be created externally with: ```text FORMAT_VERSION=1 ``` A physically empty file is invalid because it has no format-version declaration. Filesystem permission checks do not replace the actual file operations. If the directory or file passes the initial checks but opening, reading, appending, or flushing still fails, startup or sampling must fail according to the relevant error behavior. ## Record parsing For a type `1` record: ```text <timestamp>:1:<evelyn-price-index> ``` create an `EvelynStatusIndexPoint` containing: * The persisted timestamp. * The persisted Evelyn Price Index. * `BigDecimal.ZERO` for each of the other five not-yet-implemented indexes. Unknown type identifiers must fail loading. They must not be ignored. Malformed timestamps, invalid decimal values, missing fields, surplus fields, and otherwise malformed records must also fail loading. Persisted records must be in non-decreasing timestamp order. Equal timestamps are allowed, but a record earlier than the preceding record must make the file invalid. Do not sort the file during loading because that would hide corrupted or incorrectly written data. Load the complete history. No size limit, retention policy, rotation, or compaction is required in this task. A file containing only a valid format declaration represents a valid empty history. ## Persist before publishing For every successful sampling attempt: 1. Calculate the new `EvelynStatusIndexPoint` as implemented by issue #64. 2. Encode its type `1` record. 3. Append the record to `status.log`. 4. Flush it durably to the file. 5. Only after persistence succeeds, add the point to the in-memory history visible through `getStatusIndexHistory()`. Use the same robust append behavior as Ticker persistence, including handling a file whose current last line does not end with a line separator. Persistence and in-memory publication must be coordinated so callers cannot observe a point before it has been persisted. If persistence fails: * Log the instance name, path, and relevant sampling context. * Do not add the point to memory. * Do not terminate Evelyn. * Allow the next scheduled sampling attempt to run normally. A newly sampled timestamp earlier than the last persisted timestamp must not be appended. Treat it as a failed sampling attempt, log it, and continue with the next scheduled attempt. ## Stop, restart, and EMC behavior Stopping Evelyn must: * Terminate its sampling executor as before. * Clear its in-memory history. * Preserve `status.log`. * Preserve the instance-name reservation. Restarting the same object must revalidate the storage and reload its complete history before beginning new sampling. If the directory or file has been removed or is no longer readable and writable when `start()` is called again, restarting must fail in the same way as the initial start. EMC must continue using `getStatusIndexHistory()` without reading files directly. When EMC opens after Evelyn starts, the restored history must therefore be available immediately. Stopping or hiding EMC must not stop Evelyn sampling or persistence. No change to the `EvelynStatusIndexPoint` public fields or EMC’s chart API is required. ## OpenSpec Create a new OpenSpec change for this issue. Update the `evelyn-status-index-history` capability introduced by issue #64. It currently states that: * Each history begins empty. * Points are process-local. * No points are persisted or restored. * Stopping Evelyn permanently discards the complete history. Replace those statements with the named-instance persistence and lifecycle behavior described by this issue. The specification must cover at least: * Safe, portable Unicode instance names. * JVM-local uniqueness and permanent name reservation until `close()`. * Per-instance data directories. * The requirement that directories and files are provisioned externally. * Startup failure when required storage is missing, invalid, unreadable, or unwritable. * The prohibition against Evelyn creating or repairing its storage. * Status format-version validation. * Strict record parsing and unknown-type rejection. * Loading before sampling starts. * Persist-before-publish behavior. * Stop, restart, and close semantics. * Independence from EMC’s lifecycle. Do not change the AssetAZ Ticker or Raydium PriceSource specifications. ## Verification Do not add automated tests in this task. Verify the implementation by: * Compiling main and test source sets. * Running strict OpenSpec validation. * Running `git diff --check`. * Reviewing the final diff for unrelated changes. * Confirming that only `.tjava` sources are edited where Detag-generated Java is involved. ## Out of scope * Implementing the other five Evelyn indexes. * Persisting additional record types. * History rotation, retention, compaction, or database storage. * Migrating or renaming existing instance directories. * Automatically creating instance directories or status files. * Automatically repairing invalid status files. * Cross-process file locking. * Changing Ticker persistence. * Direct file access from Evelyn Mission Control. * Automated test coverage.
minimons added the enhancement label 2026-08-08 22:55:42 +02:00
minimons self-assigned this 2026-08-08 22:55:42 +02:00
minimons added this to the Evelyn project 2026-08-08 22:55:42 +02:00
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: r35157/com_r35157_nenjim-hubd-impl_ref#65