Data Model

ConvEngine relies entirely on configuration over code. State, relationships, and execution boundaries are encoded in the schema.

This page reflects the current v2 line and the active Semantic Query (db.semantic.interpret -> db.semantic.query -> postgres.query).

Control-plane tables

• ce_config
• ce_container_config
• ce_intent
• ce_intent_classifier
• ce_output_schema
• ce_prompt_template
• ce_response
• ce_rule
• ce_policy
• ce_mcp_tool
• ce_mcp_db_tool
• ce_mcp_planner
• ce_pending_action
• ce_verbose

Semantic Query control-plane tables

• ce_semantic_concept
• ce_semantic_synonym
• ce_semantic_mapping
• ce_semantic_query_class
• ce_semantic_ambiguity_option
• ce_semantic_concept_embedding
• ce_semantic_entity
• ce_semantic_relationship
• ce_semantic_join_hint
• ce_semantic_value_pattern
• ce_semantic_query_failures
• ce_mcp_sql_guardrail

Runtime tables

• ce_conversation
• ce_audit
• ce_conversation_history
• ce_llm_call_log

Release-line additions that change the data model

2.0.6

• cache diagnostics and refresh became first-class operational features
• static config caching became part of the expected runtime architecture

2.0.7

• ce_mcp_planner became the scoped planner prompt source
• MCP scope rules for ce_mcp_tool and ce_mcp_planner became strict instead of nullable-wildcard

2.0.8

• ce_verbose was added as a real control-plane table for runtime progress/error messaging
• step telemetry and richer MCP metadata became more important to the operational model

2.0.9

• CorrectionStep made ce_prompt_template.interaction_mode and interaction_contract part of actual routing semantics

2.0.10+

• Semantic Query introduced DB-driven semantic metadata and embeddings
• ce_semantic_query_failures.question_embedding enabled failure-memory retrieval for SQL regeneration
• ce_mcp_sql_guardrail remains the shared SQL read-only extension layer for MCP DB tools

ce_conversation_history

Introduced to store chronological multi-turn array states strictly decoupled from internal ce_audit tracking.

Key columns:

• history_id
• conversation_id
• user_input
• assistant_output
• created_at
• modified_at

ce_pending_action

Introduced for confirm-before-run logic and unambiguous multi-action resolution.

Purpose:

• declarative catalog for executable pending actions by intent/state/action key

Key columns:

• pending_action_id
• intent_code
• state_code
• action_key
• bean_name
• method_names
• priority
• enabled
• description
• created_at

Lookup index:

• (enabled, action_key, intent_code, state_code, priority)

Important:

• runtime status (OPEN/IN_PROGRESS/EXECUTED/REJECTED/EXPIRED) is not stored in this table
• runtime status is stored in context.pending_action_runtime

ce_verbose

Introduced in 2.0.8 for deterministic runtime progress/error messages emitted by the Java pipeline. In 2.0.9, the same resolver path can also be triggered from consumer code through ConvEngineVerboseAdapter.

Purpose:

• declarative message mapping by runtime context (intent/state/step/determinant/rule/tool)

Key columns:

• verbose_id
• intent_code
• state_code
• step_match (EXACT|REGEX|JSON_PATH)
• step_value
• determinant
• rule_id (optional)
• tool_code (optional)
• message
• error_message
• priority
• enabled
• created_at

Lookup index:

• (enabled, intent_code, state_code, determinant, step_match, step_value, rule_id, tool_code, priority)

ce_prompt_template

ce_prompt_template is used for both schema extraction prompts and derived response generation.

Key columns:

• template_id
• intent_code
• state_code
• response_type
• system_prompt
• user_prompt
• temperature
• interaction_mode
• interaction_contract
• enabled
• created_at

Runtime notes:

• prompts are rendered through the shared ThymeleafTemplateRenderer
• legacy {{...}} placeholders still work, but Thymeleaf expressions are the native path
• common runtime vars include user_input, resolved_user_input, standalone_query, intent, state, context, and inputParams
• see Thymeleaf and SpEL for expression examples across prompts and ce_verbose

interaction_mode is the coarse semantic contract for the turn. Supported values:

• NORMAL
• IDLE
• COLLECT
• CONFIRM
• PROCESSING
• FINAL
• ERROR
• DISAMBIGUATE
• FOLLOW_UP
• PENDING_ACTION
• REVIEW

interaction_contract is extensible JSON text for concrete capabilities and expectations. Recommended shape:

{"allows":["affirm","edit","retry","reset"],"expects":["structured_input"]}

Current conventions:

• allows: turn capabilities such as affirm, edit, retry, reset
• expects: input expectations such as structured_input

Design note:

• prefer extending interaction_contract arrays for new capabilities instead of adding new columns
• use interaction_mode for broad semantics and interaction_contract for fine-grained runtime behavior

Why this matters in current v2:

• CorrectionStep uses this metadata for confirm/edit/retry routing
• response generation still uses the same prompt table
• verbose message rendering and prompt rendering both rely on the shared Thymeleaf-backed path

Rule phase compatibility

ce_rule.phase includes phases:

• POST_DIALOGUE_ACT
• PRE_RESPONSE_RESOLUTION
• POST_AGENT_INTENT
• POST_SCHEMA_EXTRACTION
• PRE_AGENT_MCP
• POST_AGENT_MCP
• POST_TOOL_EXECUTION

Rule action compatibility

ce_rule.action includes:

• SET_INTENT
• SET_STATE
• SET_DIALOGUE_ACT
• SET_JSON
• GET_CONTEXT
• GET_SCHEMA_JSON
• GET_SESSION
• SET_TASK
• SET_INPUT_PARAM

SET_INPUT_PARAM is the lightweight runtime-flag action. Use it for values like:

• awaiting_confirmation=true
• confirmation_key=LOAN_APPLICATION_CONFIRM
• skip_schema_extraction=true
• routing_decision=PROCEED_CONFIRMED

SET_DIALOGUE_ACT is the lightweight post-classification override action. Use it when a POST_DIALOGUE_ACT rule should replace the final dialogue_act with a more appropriate normalized value (for example promoting a guarded NEW_REQUEST back to EDIT).

Tool-group compatibility

ce_mcp_tool.tool_group normalizes to:

• DB
• HTTP_API
• WORKFLOW_ACTION
• DOCUMENT_RETRIEVAL
• CALCULATOR_TRANSFORM
• NOTIFICATION
• FILES

MCP tool scope columns

ce_mcp_tool uses mandatory scope columns in the current v2 line:

• intent_code
• state_code

Behavior:

• null/blank scope rows are rejected at startup
• valid values are:
  • exact configured values
  • ANY
  • UNKNOWN
• non-matching scoped tools are skipped from MCP planning/execution
• invalid scope rows fail fast during startup validation

MCP planner scope columns

ce_mcp_planner supports scoped planner prompts with the same strict scope model:

• intent_code
• state_code
• system_prompt
• user_prompt
• enabled

Behavior:

• exact intent/state row wins
• exact intent_code + ANY row is next
• global default row (ANY/ANY) is fallback
• if not found, planner uses legacy ce_config prompts

ce_verbose

Introduced in 2.0.8 and expanded in 2.0.9 as the runtime-facing progress/error message layer.

Current design:

• statically cached
• validated at startup
• matched by:
  • intent_code
  • state_code
  • determinant
  • step_match
  • step_value
  • optional rule_id
  • optional tool_code
  • priority

Important runtime effect:

• consumers can expose readable progress to UI/QA without scraping raw audit rows
• ConvEngineVerboseAdapter can now publish through the same control-plane matching model

Semantic metadata model

Semantic Query is DB-driven through semantic metadata tables:

Semantic metadata tables

Table	Purpose	Why it exists
ce_semantic_concept	Business concepts (entity/status/intent).	Anchor semantic meaning in domain terms.
ce_semantic_synonym	Synonyms for concept matching.	Map user language to concepts reliably.
ce_semantic_query_class	Query class contracts.	Define allowed filters, defaults, and base table semantics.
ce_semantic_mapping	Business field -> physical table/column map.	Enable SQL generation from canonical intent.
ce_semantic_ambiguity_option	Clarification options.	Provide deterministic ambiguity resolution choices.
ce_semantic_concept_embedding	Concept embedding corpus.	Power vector-assisted entity/concept retrieval.
ce_semantic_entity	Entity-level override metadata.	Patch entity behavior without code changes.
ce_semantic_relationship	Relationship override metadata.	Patch relationship/join behavior without code changes.
ce_semantic_join_hint	Join hints.	Guide join planning for tricky schemas.
ce_semantic_value_pattern	Pattern metadata for value extraction.	Improve value parsing/mapping from natural language.
ce_semantic_query_failures	Failure-memory store with corrected SQL.	Enable retrieval-augmented SQL retry and correction.

Related docs

• Table Config
• MCP
• MCP Semantic Query
• Table Config