Module: utils/compatConfigCache

In-memory TTL cache for compat (compat:<id>) provider configs with Redis pub/sub invalidation for multi-process coherence.

Why

Without a cache every callProvider() / streamText() invocation for a compat slot reads api_keys from SQLite (hot path for AI generation, can be called hundreds of times per pipeline run). This module caches the decrypted { apiKey, baseUrl, model, displayName } blob with a short TTL and invalidates it on write.

Multi-process coherence

When REDIS_URL is set, every invalidate() / invalidateAll() call also publishes to sentri:compat-config:invalidate. Other server instances listening on the channel drop their cached entry for the same slot id, so a save on instance A is visible to instance B within one Redis round-trip. The published message includes an _origin field so the publisher skips its own echo (same pattern as routes/sse.js).

Single-process mode

When Redis is not available the module degrades to a purely local cache with TTL-based staleness. This is safe for single-instance deployments; the TTL (default 60 s, overridable via COMPAT_CONFIG_CACHE_TTL_MS) bounds the staleness window.

Exports

  • get — Read-through accessor (slotId, loader) => value|null.
  • invalidate — Drop a single slot's cached entry + publish.
  • invalidateAll — Drop every entry + publish (used by admin resets).
  • __test — Internals exposed for tests (stats, clear, TTL override).
Source:

Members

(inner, constant) ALL

Sentinel payload for "invalidate every cached slot".

Source:

(inner, constant) CHANNEL

Pub/sub channel name. Prefix matches the sentri: convention used by sse.js.

Source:

(inner, constant) _cache

Cache entries: slotId → { value, expiresAt }. value === null is a legitimate cached result (slot deleted) — we use expiresAt rather than presence-of-key to test staleness.

Source:

(inner, constant) _instanceId

Unique id for this process — used to skip self-echo from Redis.

Source:

(inner, constant) _stats

Stats for observability / tests.

Source:

(inner) _ttlMs

TTL in milliseconds. Exposed via __test.setTtl() so tests can shrink the TTL without waiting 60 s in real time. Read from env at module load so production deployments can tune staleness without a code change.

Source:

Methods

(static) get(slotId, loader) → {any}

Read-through accessor. Returns the cached value for slotId if fresh, otherwise calls loader(slotId) to refresh the entry. loader is supplied by the caller (rather than imported here) to avoid a circular dependency on apiKeyRepo.

Parameters:
Name Type Description
slotId string

Canonical provider id (e.g. "compat:deepseek").
loader function

() => value — called on miss / expired entry.
Source:
Returns:

The cached or freshly-loaded value.

Type
any

(static) invalidate(slotId)

Drop the cached entry for a single slot and broadcast the invalidation to other instances. Must be called by every write-path (setCompatSlot, deleteCompatSlot) so the cache never serves stale credentials.

Parameters:
Name Type Description
slotId string

Canonical provider id.
Source:

(static) invalidateAll()

Drop every cached entry and broadcast. Used by admin resets and tests.

Source: