Caching & Persistence

ConvEngine v2 is heavily optimized to decouple real-time generative latency from slow relational database I/O. The framework accomplishes this by utilizing Asynchronous Persistence Threads alongside Spring's Abstract CacheManager.

To activate this optimization on your backend, add the following decorators to your application class:

1. @EnableConvEngineCaching
2. @EnableConvEngineAsyncConversation

How the abstraction works

The actual datastore powering @EnableConvEngineCaching is entirely controlled by the Consumer application, not the ConvEngine framework. ConvEngine relies on the generic Spring CacheManager interface to save and load CeConversation instances via the ce_conversation_cache routing key.

By default, to activate ConvEngine caching, you must pull in Spring Boot's caching abstraction layer. If you import this module but do not import any external caching dependencies (like Redis or Ehcache), Spring Boot will automatically fall back to providing a ConcurrentMapCacheManager out of the box.

Dependency requirement (pom.xml):

pom.xml

XML

<dependency>
  <groupId>org.springframework.boot</groupId>
  <artifactId>spring-boot-starter-cache</artifactId>
</dependency>

Startup crashes without this dependency

If you annotate your application class with @EnableConvEngineCaching but fail to include spring-boot-starter-cache in your pom.xml, the Spring context will fatally crash on startup because it cannot find an underlying CacheManager bean to bind to.

This operates seamlessly once the module is present. It holds conversation states inside a standard Java HashMap residing in your server's active JVM memory.

• Pros: Blazing fast (0ms read/write latency), requires zero configuration.
• Cons: Cache is completely wiped if the Spring Boot application restarts or crashes; not horizontal-scale (multi-pod) friendly.

---

1. Ehcache 3 (JSR-107)

For robust, production-grade memory bounds, Ehcache is a highly recommended on-premise memory provider. It allows conversational payloads to spill over from the active heap into offheap memory to prevent JVM garbage collection thrashing during large multi-turn flows.

Dependency update (pom.xml):

pom.xml

XML

<dependency>
  <groupId>org.springframework.boot</groupId>
  <artifactId>spring-boot-starter-cache</artifactId>
</dependency>
<dependency>
  <groupId>javax.cache</groupId>
  <artifactId>cache-api</artifactId>
</dependency>
<dependency>
  <groupId>org.ehcache</groupId>
  <artifactId>ehcache</artifactId>
</dependency>

App configuration (application.yml):

application.yml

YAML

spring:
cache:
  jcache:
    config: classpath:ehcache.xml

Provider specification (src/main/resources/ehcache.xml):

ehcache.xml

XML

<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
      xmlns="http://www.ehcache.org/v3"
      xmlns:jsr107="http://www.ehcache.org/v3/jsr107"
      xsi:schemaLocation="
          http://www.ehcache.org/v3 http://www.ehcache.org/schema/ehcache-core-3.0.xsd
          http://www.ehcache.org/v3/jsr107 http://www.ehcache.org/schema/ehcache-107-ext-3.0.xsd">

  <!-- This MUST exactly match ConvEngine's requested alias -->
  <cache alias="ce_conversation_cache">
      <key-type>java.lang.String</key-type>
      <value-type>com.github.salilvnair.convengine.entity.CeConversation</value-type>
      <expiry>
          <ttl unit="minutes">60</ttl>
      </expiry>
      <resources>
          <heap unit="entries">2000</heap>
          <offheap unit="MB">100</offheap>
      </resources>
  </cache>
</config>

---

2. Caffeine Cache

If you prefer a highly optimized, high-performance local cache strictly contained inside the JVM (without off-heap spillage but statistically faster hit-rates than Ehcache), Caffeine is Spring Boot's default local provider replacement.

Dependency update (pom.xml):

pom.xml

XML

<dependency>
  <groupId>org.springframework.boot</groupId>
  <artifactId>spring-boot-starter-cache</artifactId>
</dependency>
<dependency>
  <groupId>com.github.ben-manes.caffeine</groupId>
  <artifactId>caffeine</artifactId>
</dependency>

App configuration (application.yml): Unlike Ehcache, Caffeine can be fully initialized inside application.yml without distinct XML files.

application.yml

YAML

spring:
cache:
  type: caffeine
  cache-names: ce_conversation_cache
  caffeine:
    # Max limits the cache to 5,000 conversational objects, expiring them 60 minutes after their last network write
    spec: maximumSize=5000,expireAfterWrite=60m

---

3. Redis

For horizontally scaled environments like Kubernetes (k8s) where your user traffic rotates across multiple server pods mid-conversation, you cannot use local memory (Ehcache/Caffeine). Redis binds ConvEngine's cache into an external centralized datastore.

Dependency update (pom.xml):

pom.xml

XML

<dependency>
  <groupId>org.springframework.boot</groupId>
  <artifactId>spring-boot-starter-data-redis</artifactId>
</dependency>
<dependency>
  <groupId>org.springframework.boot</groupId>
  <artifactId>spring-boot-starter-cache</artifactId>
</dependency>

App configuration (application.yml):

application.yml

YAML

spring:
data:
  redis:
    host: localhost
    port: 6379
cache:
  type: redis
  redis:
    time-to-live: 60m
    cache-null-values: false
    use-key-prefix: true
    key-prefix: "convengine:"

A note on serialization

Because Redis runs outside the active server JVM, it relies on network transmission. You must ensure that your Spring application has a generic RedisTemplate or Jackson-configured RedisCacheConfiguration bean configured to serialize CeConversation entities, otherwise Java will throw NotSerializableException when attempting to push the object to the internet.

---

4. Static Configuration Preloading

Beyond dynamic conversational sessions, ConvEngine also permanently caches all non-transactional database rules (ce_intent, ce_rule, ce_mcp_tool, ce_mcp_planner, ce_config, etc.) into a generic ce_static_configs memory tree entirely upon JVM live initiation. This ensures that the engine never triggers synchronous relational fetches during the middle of a customer's turn.

If you connect to your database and update a ce_rule row manually using SQL, the active Java application won't automatically see it until you flush this static cache.

Flushing the Cache Tree: You can do this by hitting the integrated Admin Controller endpoint mapped generically natively into your backend:

Cache Refresh Trigger

BASH

curl -X GET "http://localhost:8080/api/v1/cache/refresh"

This endpoint executes @CacheEvict(value = "ce_static_configs", allEntries = true) and immediately re-triggers the Spring Data Preloader, locking in your latest SQL tweaks live.

---

5. Cache Analyzer API (/api/v1/cache/analyze)

Use this endpoint when you want to validate cache health without guessing whether misses are caused by provider, proxying, or runtime wiring.

Analyzer endpoint contract

Method	Path	Query Param	Purpose
GET	/api/v1/cache/analyze	warmup=true|false	Returns cache/provider/proxy diagnostics and cache entry visibility.

Example calls

BASH

# Deep probe (runs warmup timing first-call vs second-call)
curl "http://localhost:8080/api/v1/cache/analyze?warmup=true"

# Passive snapshot (no extra warmup/timing calls)
curl "http://localhost:8080/api/v1/cache/analyze?warmup=false"

warmup flag behavior

warmup=true actively calls static cache loaders before reporting timing metrics. warmup=false only inspects current state and does not trigger additional warmup calls.

How to Read the Analyzer Response

Interpret each JSON block as follows:

Field-by-field meaning

Field	What it means	Healthy signal	When to investigate
warmupAndTimingEnabled	Whether analyzer executed active timing probe.	Matches your request flag.	Unexpected value indicates wrong request or proxy/caching to old endpoint.
cacheManagerClass	Actual Spring cache provider at runtime.	`ConcurrentMapCacheManager`, Caffeine, Redis, etc. as expected.	Unexpected provider means environment config drift.
registeredCacheNames	Caches currently known to provider.	Includes static `ce_*` and runtime caches.	Missing cache names means cache operations are not being registered/hit.
warmupTimingMs	First vs second invocation latency by static cache.	Second call <= first call, usually much faster.	Both large or growing means repeated misses or backend latency.
staticCaches.*.hasSimpleKeyEntry	Whether no-arg `@Cacheable` entry exists.	`true` for all static `ce_*` datasets.	`false` means static caches were not populated/hit.
staticCaches.*.listSize	Rows currently held in memory for each static table.	Non-zero sizes matching expected SQL data.	Unexpected zero/low values indicates seed/data/environment mismatch.
runtimeCaches.*.nativeEntryCount	Current conversation/history cache occupancy.	Grows when conversations run.	Always zero during active traffic indicates runtime cache path not used.

Sample response

JSON

Interpreting the sample you shared

• cacheManagerClass=ConcurrentMapCacheManager: Local in-memory cache provider is active and valid.
• Static caches all show exists=true, hasSimpleKeyEntry=true, nativeEntryCount=1: static ce_* cache population is healthy.
• Static listSize values are non-zero: static rows are loaded into cache as expected.
• firstCallMs=0 and secondCallMs=0: data is tiny and runtime is fast enough to round to 0 ms; this is normal.
• Runtime caches at nativeEntryCount=0: no conversation/historical entries cached yet at capture time, or traffic was not running when the snapshot was taken.