fix: Analyze with 'Claude Sonnet' then fix some bugs/issues

2026-02-25 09:29:51 +09:00
parent 9556f1dd0e
commit d11d35a796
10 changed files with 672 additions and 32 deletions
--- a/_planning/CHAT_ARCHIVE.md
+++ b/_planning/CHAT_ARCHIVE.md
@@ -301,7 +301,26 @@
    - **Verification**: Confirmed `logs/backend/backend_2026-02-24.log` is created and populated correctly via Python test.
 - **Status**: Logging is fully operational across all four platforms.

-## 38. Current Status
+## 38. Session 26: Full Codebase Gap Analysis & Critical Bug Fixes
+- **Date**: 2026-02-25
+- **Goal**: Comprehensive gap analysis of entire codebase (excluding `_archive`); fix critical bugs.
+- **Actions Taken**:
+    - **Gap Analysis**: Scanned all 20+ source directories. Confirmed Phases 1–11 fully implemented. Produced `_planning/GAP_ANALYSIS.md` with confirmed implementations, bugs, structural gaps, advancing opportunities, and a prioritized next-steps table.
+    - **Bug Fix 1 — `main.py` Import Alias Collision**: `self_evaluation_routes` and `evaluation_routes` both imported as `evaluation_router`. First was silently overwritten, making `self_evaluation_routes` completely unregistered. Fixed: renamed alias to `self_evaluation_router`, added missing `include_router`. Removed stray duplicate registration.
+    - **Bug Fix 2 — `tenants/tenant_registry.py` Duplicate Methods**: `log_usage()` and `check_quota()` each defined twice with conflicting logic. Broken int-based second definitions silently overrode the correct quota-dict-based ones. Removed duplicates; fixed redundant double-call in `is_capability_allowed`.
+- **Status**: All P0 bugs resolved. `GAP_ANALYSIS.md` saved to `_planning/`; completed items marked ✅.
+
+## 39. Session 27: Phase 12 Step 1.1 — Unified Policy Engine
+- **Date**: 2026-02-25
+- **Goal**: Consolidate six fragmented in-memory governance components into a single persistent `PolicyEngine` backed by SQLite `data/governance.db`.
+- **Components Unified**: `PolicyRegistry`, `ComplianceChecker`, `SLAMonitor`, `UsageMeter`, `BillingEngine`, `TenantPolicyStore`.
+- **Status**: In progress.
+
+## 40. Current Status
 - **Backend**: Stable and running on port 8000. Rotating file logging active in `logs/backend/`.
 - **Frontend / Mobile / Desktop**: Log forwarding implemented through `POST /api/log/remote`.
 - **Governance**: Phase 12 Step 1.1 (Unified Policy Engine) is next on the roadmap.
+
+
+
+
--- a/_planning/GAP_ANALYSIS.md
+++ b/_planning/GAP_ANALYSIS.md
@@ -0,0 +1,113 @@
+# 🔍 Codebase Gap Analysis
+**Date**: 2026-02-25 | **Scope**: All source directories (excluding `_archive`)
+
+---
+
+## ✅ Confirmed Implementations (Phases 1–11)
+
+All phases marked complete in the roadmap are verified present in code:
+
+| Area | Key Files Confirmed |
+|---|---|
+| **Governance / SLA** | `governance/policy_registry.py`, `governance/sla_monitor.py`, `governance/compliance_checker.py`, `governance/billing_engine.py`, `governance/usage_meter.py` |
+| **Tenants / RBAC** | `tenants/tenant_registry.py`, `tenants/rbac_guard.py`, `tenants/tenant_policy.py`, `tenants/tenant_admin_registry.py` |
+| **Agents (Phase 11)** | `agents/crew_manager.py`, `agents/trust_calibrator.py`, `agents/diplomacy_protocol.py`, `agents/treaty_manager.py`, `agents/conflict_resolver.py`, `agents/evolution_engine.py`, `agents/genealogy_tracker.py` |
+| **Models** | `models/registry.py`, `models/dual_engine_router.py`, `models/telemetry.py`, `models/tone_engine.py`, `models/moe_handler.py` |
+| **Memory** | `memory/memory_manager.py`, `memory/knowledge_graph.py`, `memory/episodic_store.py`, `memory/semantic_store.py`, `memory/memory_graph.py` |
+| **Ingestion** | `ingestion/document_ingestor.py`, `ingestion/multimodal_ingestor.py`, `ingestion/structural_ingestor.py`, `ingestion/media_ingestor.py` |
+| **Vector Store** | `vector_store/faiss_store.py`, `vector_store/qdrant_store.py`, `vector_store/milvus_store.py`, `vector_store/weaviate_store.py` |
+| **Voice** | `voice/voice_listener.py`, `voice/command_parser.py` |
+| **Plugins** | `plugins/plugin_registry.py`, `plugins/agent_marketplace.py`, `plugins/plugin_loader.py` |
+| **Monitoring** | `monitoring/metrics_store.py`, `monitoring/optimizer.py`, `monitoring/goal_heatmap_sql.py`, `monitoring/alert-rules.yml` |
+| **Logging** | `utils/logger.py`, `routes/logging_routes.py` |
+| **Tests** | `tests/` — 31 scripts covering Phases 6–11 |
+
+---
+
+## 🔴 Critical Bugs & Issues
+
+### ✅ 1. Import Alias Collision + Duplicate Router in `main.py` — FIXED 2026-02-25
+- Root cause: `self_evaluation_routes` and `evaluation_routes` both imported as `evaluation_router` — first was silently overwritten and never registered.
+- Fixed: renamed alias to `self_evaluation_router`, added its `include_router`, removed stray duplicate registration.
+
+### ✅ 2. Duplicate Methods in `tenants/tenant_registry.py` — FIXED 2026-02-25
+- `log_usage()` and `check_quota()` were each defined twice with conflicting logic. Broken int-based duplicates removed; canonical quota-dict-based versions restored.
+- Also fixed: redundant double-call in `is_capability_allowed`.
+
+### ✅ 3. `Wave` Dependency Error in `requirements.txt` — MITIGATED
+- `Wave` dependency caused a build failure; already commented out in `requirements.txt`.
+
+---
+
+## 🟡 Structural / Quality Issues
+
+### 4. All Governance Stores Are In-Memory Only
+- `governance/policy_registry.py` → `self.policies = {}`, `self.audit_log = []`
+- `governance/sla_monitor.py` → `self.violations = []`
+- `governance/compliance_checker.py` → `self.violations = []`
+- **Impact**: All policy/SLA/compliance data lost on every restart.
+- **Fix Target**: Phase 12 Step 1.1 — migrate to `governance.db` (SQLite).
+
+### 5. Tenant Registry is In-Memory Only
+- `tenants/tenant_registry.py` stores all tenant data in `self.tenants = {}`.
+- **Impact**: Tenant onboarding, usage logs, quotas, and version history reset on shutdown.
+
+### 6. No Hot-Reload for Policies *(Phase 12 Step 1.2 Gap)*
+- `policy_registry.py` has no mechanism to update rules without restart.
+
+### 7. No SLA-Aware Admission Control *(Phase 12 Step 1.3 Gap)*
+- `sla_monitor.py` records violations but never gates incoming requests.
+
+### 8. No Automated Decision Logging *(Phase 12 Step 2.1 Gap)*
+- `agents/reflection.py` is minimal. No structured reasoning trace captured per invocation.
+
+### 9. No Reward-Based Retraining Loop *(Phase 12 Step 2.2 Gap)*
+- `agents/retraining_agent.py` is a stub (624 bytes). No reward → evolution wiring.
+
+### 10. Plugin Discovery Is Static *(Phase 12 Step 3 Gap)*
+- `plugins/plugin_loader.py` is minimal — no folder scan or dynamic capability mapping.
+
+---
+
+## 🚀 Advancing Opportunities
+
+### A. Governance Consolidation → Phase 12 Step 1.1
+- RBAC in `tenants/rbac_guard.py`, quota in `tenants/tenant_registry.py`, compliance in `governance/compliance_checker.py` — fragmented.
+- **Opportunity**: Unify into a single `PolicyEngine` backed by persistent `governance.db`.
+
+### B. Observability Bridge
+- `models/telemetry.py` → `data/telemetry.db` (SQLite). Prometheus config and Grafana dashboards exist.
+- **Gap**: No live exporter from `telemetry.db` → Prometheus `/metrics` endpoint.
+- **Note**: `metrics_router` is registered at line 234 but the `/metrics` prefix is commented out at line 159 in `main.py`.
+
+### C. Async Migration
+- `memory/memory_manager.py` (16KB) mixes sync/async. Sync IO in async FastAPI blocks the event loop.
+- **Opportunity**: Migrate DB interactions to `aiosqlite`.
+
+---
+
+## 📋 Deprecation Cleanup
+
+| Item | Status | Action |
+|---|---|---|
+| `desktop-electron/` | ⚠️ Deprecated (favour `src-tauri/`) | Archive |
+| `desktop-mobile/` | ⚠️ Deprecated (favour `mobile-flutter/`) | Archive |
+| `FIXME.md` Wave entry | Already resolved | Remove stale entry |
+| Commented `messaging_routes` in `main.py` | Unknown | Clarify or delete |
+
+---
+
+## 🎯 Recommended Next Steps (Prioritized)
+
+| Priority | Task | Effort | Status |
+|---|---|---|---|
+| ✅ ~~P0~~ | ~~Fix duplicate `evaluation_router` / import alias collision in `main.py`~~ | 5 min | Done |
+| ✅ ~~P0~~ | ~~Reconcile duplicate `log_usage`/`check_quota` in `tenant_registry.py`~~ | 15 min | Done |
+| 🟠 **P1** | **Phase 12 Step 1.1**: Unified Policy Engine (persistent `governance.db`) | ~2 hrs | ⬅ Next |
+| 🟠 **P1** | **Phase 12 Step 1.2**: Dynamic Rule Injection | ~1 hr | Pending |
+| 🟠 **P1** | **Phase 12 Step 1.3**: SLA-aware Admission Control | ~1 hr | Pending |
+| 🟡 **P2** | Observability Bridge — enable `/metrics` → Prometheus/Grafana | ~2 hrs | Pending |
+| 🟡 **P2** | Phase 12 Step 2.1–2.2: Decision Logging + Retraining Loop | ~5 hrs | Pending |
+| 🟢 **P3** | Async Migration of `memory_manager.py` | ~3 hrs | Pending |
+| 🟢 **P3** | Plugin Dynamic Discovery (Phase 12 Step 3) | ~2 hrs | Pending |
+| 🟢 **P3** | Archive deprecated `desktop-electron/` and `desktop-mobile/` | 15 min | Pending |
--- a/governance/compliance_checker.py
+++ b/governance/compliance_checker.py
@@ -2,10 +2,11 @@
 ##INFO:

 import time
+from governance.policy_engine import policy_engine

 class ComplianceChecker:
    def __init__(self):
-        self.violations = []
+        self.violations = []  # in-memory cache; DB is source of truth

    def check(self, tenant_id: str, agent_role: str, task: str, policy: dict):
        if agent_role not in policy["allowed_roles"]:
@@ -27,12 +28,15 @@ class ComplianceChecker:
            "status": "violation"
        }
        self.violations.append(record)
+        # Persist to DB
+        policy_engine.log_violation(tenant_id, agent_role, "", task, reason, "compliance")
        return record

    def get_violations(self):
-        return self.violations
+        # Return persisted violations from DB
+        return policy_engine.get_violations(vtype="compliance")

    def get_by_tenant(self, tenant_id: str):
-        return [v for v in self.violations if v["tenant"] == tenant_id]
+        return policy_engine.get_violations(tenant_id=tenant_id, vtype="compliance")
    
 compliance_checker = ComplianceChecker()
--- a/governance/policy_engine.py
+++ b/governance/policy_engine.py
@@ -0,0 +1,305 @@
+# governance/policy_engine.py
+# Phase 12 Step 1.1 — Unified Policy Engine
+# Consolidates PolicyRegistry, ComplianceChecker, SLAMonitor, UsageMeter, BillingEngine
+# into a single, persisted SQLite-backed engine.
+
+import sqlite3
+import time
+import os
+from datetime import datetime
+from typing import Optional
+
+DB_PATH = os.path.join(os.path.dirname(__file__), "..", "data", "governance.db")
+
+
+class PolicyEngine:
+    """
+    Single entry-point for all governance decisions.
+    Persists policies, violations, and usage to data/governance.db.
+    """
+
+    def __init__(self, db_path: str = DB_PATH):
+        self.db_path = os.path.abspath(db_path)
+        os.makedirs(os.path.dirname(self.db_path), exist_ok=True)
+        self._init_db()
+
+    # ------------------------------------------------------------------
+    # DB Bootstrap
+    # ------------------------------------------------------------------
+    def _conn(self):
+        return sqlite3.connect(self.db_path)
+
+    def _init_db(self):
+        with self._conn() as con:
+            con.executescript("""
+                CREATE TABLE IF NOT EXISTS policies (
+                    id          INTEGER PRIMARY KEY AUTOINCREMENT,
+                    tenant_id   TEXT NOT NULL,
+                    allowed_roles TEXT NOT NULL,          -- JSON array
+                    restricted_tasks TEXT NOT NULL DEFAULT '[]', -- JSON array
+                    audit_level TEXT NOT NULL DEFAULT 'standard',
+                    updated_at  TEXT NOT NULL
+                );
+                CREATE UNIQUE INDEX IF NOT EXISTS idx_policies_tenant ON policies(tenant_id);
+
+                CREATE TABLE IF NOT EXISTS violations (
+                    id          INTEGER PRIMARY KEY AUTOINCREMENT,
+                    timestamp   REAL NOT NULL,
+                    tenant_id   TEXT NOT NULL,
+                    agent_role  TEXT,
+                    action      TEXT,
+                    task        TEXT,
+                    reason      TEXT,
+                    vtype       TEXT NOT NULL DEFAULT 'compliance', -- compliance | sla | quota
+                    latency_ms  REAL
+                );
+                CREATE INDEX IF NOT EXISTS idx_violations_tenant ON violations(tenant_id);
+
+                CREATE TABLE IF NOT EXISTS usage (
+                    id              INTEGER PRIMARY KEY AUTOINCREMENT,
+                    tenant_id       TEXT NOT NULL,
+                    agent_role      TEXT NOT NULL,
+                    units_used      INTEGER NOT NULL DEFAULT 0,
+                    quota_limit     INTEGER,
+                    last_updated    TEXT NOT NULL,
+                    UNIQUE(tenant_id, agent_role)
+                );
+
+                CREATE TABLE IF NOT EXISTS billing_rates (
+                    tenant_id   TEXT NOT NULL,
+                    agent_role  TEXT NOT NULL,
+                    rate        REAL NOT NULL DEFAULT 0.0,
+                    PRIMARY KEY (tenant_id, agent_role)
+                );
+
+                CREATE TABLE IF NOT EXISTS sla_thresholds (
+                    tenant_id       TEXT NOT NULL,
+                    agent_role      TEXT NOT NULL,
+                    max_latency_ms  REAL NOT NULL DEFAULT 2000.0,
+                    PRIMARY KEY (tenant_id, agent_role)
+                );
+            """)
+
+    # ------------------------------------------------------------------
+    # Policy CRUD
+    # ------------------------------------------------------------------
+    def set_policy(self, tenant_id: str, allowed_roles: list,
+                   restricted_tasks: list = None, audit_level: str = "standard"):
+        import json
+        now = datetime.utcnow().isoformat()
+        with self._conn() as con:
+            con.execute("""
+                INSERT INTO policies (tenant_id, allowed_roles, restricted_tasks, audit_level, updated_at)
+                VALUES (?, ?, ?, ?, ?)
+                ON CONFLICT(tenant_id) DO UPDATE SET
+                    allowed_roles=excluded.allowed_roles,
+                    restricted_tasks=excluded.restricted_tasks,
+                    audit_level=excluded.audit_level,
+                    updated_at=excluded.updated_at
+            """, (tenant_id, json.dumps(allowed_roles),
+                  json.dumps(restricted_tasks or []), audit_level, now))
+        return self.get_policy(tenant_id)
+
+    def get_policy(self, tenant_id: str) -> dict:
+        import json
+        with self._conn() as con:
+            row = con.execute(
+                "SELECT allowed_roles, restricted_tasks, audit_level FROM policies WHERE tenant_id=?",
+                (tenant_id,)
+            ).fetchone()
+        if row:
+            return {
+                "allowed_roles": json.loads(row[0]),
+                "restricted_tasks": json.loads(row[1]),
+                "audit_level": row[2]
+            }
+        # Default policy
+        return {
+            "allowed_roles": ["planner", "executor", "critic"],
+            "restricted_tasks": [],
+            "audit_level": "standard"
+        }
+
+    def get_all_policies(self) -> list:
+        import json
+        with self._conn() as con:
+            rows = con.execute(
+                "SELECT tenant_id, allowed_roles, restricted_tasks, audit_level, updated_at FROM policies"
+            ).fetchall()
+        return [
+            {"tenant_id": r[0], "allowed_roles": json.loads(r[1]),
+             "restricted_tasks": json.loads(r[2]), "audit_level": r[3], "updated_at": r[4]}
+            for r in rows
+        ]
+
+    # ------------------------------------------------------------------
+    # SLA Thresholds
+    # ------------------------------------------------------------------
+    def set_sla_threshold(self, tenant_id: str, agent_role: str, max_latency_ms: float):
+        with self._conn() as con:
+            con.execute("""
+                INSERT INTO sla_thresholds (tenant_id, agent_role, max_latency_ms)
+                VALUES (?, ?, ?)
+                ON CONFLICT(tenant_id, agent_role) DO UPDATE SET max_latency_ms=excluded.max_latency_ms
+            """, (tenant_id, agent_role, max_latency_ms))
+
+    def get_sla_threshold(self, tenant_id: str, agent_role: str) -> float:
+        with self._conn() as con:
+            row = con.execute(
+                "SELECT max_latency_ms FROM sla_thresholds WHERE tenant_id=? AND agent_role=?",
+                (tenant_id, agent_role)
+            ).fetchone()
+        return row[0] if row else 2000.0  # default 2s
+
+    # ------------------------------------------------------------------
+    # Quota / Usage
+    # ------------------------------------------------------------------
+    def set_quota(self, tenant_id: str, agent_role: str, limit: int):
+        now = datetime.utcnow().isoformat()
+        with self._conn() as con:
+            con.execute("""
+                INSERT INTO usage (tenant_id, agent_role, units_used, quota_limit, last_updated)
+                VALUES (?, ?, 0, ?, ?)
+                ON CONFLICT(tenant_id, agent_role) DO UPDATE SET
+                    quota_limit=excluded.quota_limit,
+                    last_updated=excluded.last_updated
+            """, (tenant_id, agent_role, limit, now))
+
+    def log_usage(self, tenant_id: str, agent_role: str, units: int = 1):
+        now = datetime.utcnow().isoformat()
+        with self._conn() as con:
+            con.execute("""
+                INSERT INTO usage (tenant_id, agent_role, units_used, last_updated)
+                VALUES (?, ?, ?, ?)
+                ON CONFLICT(tenant_id, agent_role) DO UPDATE SET
+                    units_used = usage.units_used + excluded.units_used,
+                    last_updated = excluded.last_updated
+            """, (tenant_id, agent_role, units, now))
+
+    def check_quota(self, tenant_id: str, agent_role: str) -> dict:
+        with self._conn() as con:
+            row = con.execute(
+                "SELECT units_used, quota_limit FROM usage WHERE tenant_id=? AND agent_role=?",
+                (tenant_id, agent_role)
+            ).fetchone()
+        if not row:
+            return {"status": "unlimited", "used": 0, "limit": None}
+        used, limit = row
+        if limit is None:
+            return {"status": "unlimited", "used": used, "limit": None}
+        if used >= limit:
+            return {"status": "exceeded", "used": used, "limit": limit}
+        return {"status": "ok", "used": used, "limit": limit}
+
+    # ------------------------------------------------------------------
+    # Violation Logging
+    # ------------------------------------------------------------------
+    def log_violation(self, tenant_id: str, agent_role: str, action: str,
+                      task: str, reason: str, vtype: str = "compliance",
+                      latency_ms: float = None):
+        with self._conn() as con:
+            con.execute("""
+                INSERT INTO violations (timestamp, tenant_id, agent_role, action, task, reason, vtype, latency_ms)
+                VALUES (?, ?, ?, ?, ?, ?, ?, ?)
+            """, (time.time(), tenant_id, agent_role, action, task, reason, vtype, latency_ms))
+        return {
+            "timestamp": time.time(), "tenant_id": tenant_id,
+            "agent_role": agent_role, "action": action,
+            "task": task, "reason": reason, "type": vtype
+        }
+
+    def get_violations(self, tenant_id: str = None, vtype: str = None,
+                       since_ts: float = None, limit: int = 100) -> list:
+        query = "SELECT timestamp, tenant_id, agent_role, action, task, reason, vtype, latency_ms FROM violations WHERE 1=1"
+        params = []
+        if tenant_id:
+            query += " AND tenant_id=?"
+            params.append(tenant_id)
+        if vtype:
+            query += " AND vtype=?"
+            params.append(vtype)
+        if since_ts:
+            query += " AND timestamp>=?"
+            params.append(since_ts)
+        query += " ORDER BY timestamp DESC LIMIT ?"
+        params.append(limit)
+        with self._conn() as con:
+            rows = con.execute(query, params).fetchall()
+        keys = ["timestamp", "tenant_id", "agent_role", "action", "task", "reason", "type", "latency_ms"]
+        return [dict(zip(keys, r)) for r in rows]
+
+    def clear_violations(self, tenant_id: str = None):
+        with self._conn() as con:
+            if tenant_id:
+                con.execute("DELETE FROM violations WHERE tenant_id=?", (tenant_id,))
+            else:
+                con.execute("DELETE FROM violations")
+
+    # ------------------------------------------------------------------
+    # Unified Evaluate
+    # ------------------------------------------------------------------
+    def evaluate(self, tenant_id: str, role: str, action: str,
+                 task: str = "", latency_ms: Optional[float] = None) -> dict:
+        """
+        Run all governance checks in sequence.
+        Returns: {allowed: bool, violations: [str], checks: {rbac, quota, sla}}
+        """
+        policy = self.get_policy(tenant_id)
+        violations = []
+        checks = {}
+
+        # 1. RBAC — is the role allowed?
+        if policy["allowed_roles"] and role not in policy["allowed_roles"]:
+            reason = f"Role '{role}' not in allowed_roles {policy['allowed_roles']}"
+            violations.append(reason)
+            checks["rbac"] = {"passed": False, "reason": reason}
+            self.log_violation(tenant_id, role, action, task, reason, "compliance")
+        else:
+            checks["rbac"] = {"passed": True}
+
+        # 2. Restricted tasks
+        for restricted in policy.get("restricted_tasks", []):
+            if restricted.lower() in task.lower():
+                reason = f"Task contains restricted keyword '{restricted}'"
+                violations.append(reason)
+                checks["task_restriction"] = {"passed": False, "reason": reason}
+                self.log_violation(tenant_id, role, action, task, reason, "compliance")
+                break
+        else:
+            checks["task_restriction"] = {"passed": True}
+
+        # 3. Quota check
+        quota_result = self.check_quota(tenant_id, role)
+        checks["quota"] = quota_result
+        if quota_result["status"] == "exceeded":
+            reason = f"Quota exceeded: {quota_result['used']}/{quota_result['limit']} units"
+            violations.append(reason)
+            self.log_violation(tenant_id, role, action, task, reason, "quota")
+
+        # 4. SLA latency check (only when provided)
+        if latency_ms is not None:
+            threshold = self.get_sla_threshold(tenant_id, role)
+            checks["sla"] = {"latency_ms": latency_ms, "threshold_ms": threshold}
+            if latency_ms > threshold:
+                reason = f"Latency {latency_ms}ms exceeds SLA threshold {threshold}ms"
+                violations.append(reason)
+                checks["sla"]["passed"] = False
+                self.log_violation(tenant_id, role, action, task, reason, "sla", latency_ms)
+            else:
+                checks["sla"]["passed"] = True
+
+        # Log usage for every evaluate call
+        self.log_usage(tenant_id, role)
+
+        return {
+            "allowed": len(violations) == 0,
+            "violations": violations,
+            "checks": checks,
+            "tenant_id": tenant_id,
+            "role": role,
+            "action": action
+        }
+
+
+# Singleton
+policy_engine = PolicyEngine()
--- a/governance/policy_registry.py
+++ b/governance/policy_registry.py
@@ -1,14 +1,18 @@
 # governance/policy_registry.py

 import time
+from governance.policy_engine import policy_engine

 class PolicyRegistry:
    def __init__(self):
-        self.policies = {}  # {tenant_id: {allowed_roles, restricted_tasks, audit_level}}
+        self.policies = {}  # in-memory cache
        self.global_policies = self._load_global_policies()
-        self.audit_log = []  # list of {timestamp, tenant_id, role, action, violation, reason}
+        self.audit_log = []  # local cache; DB is source of truth

    def set_policy(self, tenant_id: str, allowed_roles: list, restricted_tasks: list, audit_level: str = "standard"):
+        # Persist to DB via PolicyEngine
+        policy_engine.set_policy(tenant_id, allowed_roles, restricted_tasks, audit_level)
+        # Update in-memory cache
        self.policies[tenant_id] = {
            "allowed_roles": allowed_roles,
            "restricted_tasks": restricted_tasks,
@@ -17,11 +21,10 @@ class PolicyRegistry:
        return self.policies[tenant_id]

    def get_policy(self, tenant_id: str):
-        return self.policies.get(tenant_id, {
-            "allowed_roles": ["planner", "executor", "critic"],
-            "restricted_tasks": [],
-            "audit_level": "standard"
-        })
+        # Try in-memory cache first, then fall back to DB
+        if tenant_id in self.policies:
+            return self.policies[tenant_id]
+        return policy_engine.get_policy(tenant_id)

    def get_all(self):
        return self.policies
@@ -75,12 +78,13 @@ class PolicyRegistry:
            "reason": reason
        }
        self.audit_log.append(entry)
+        # Persist to DB
+        policy_engine.log_violation(tenant_id, role, action, "", reason, "compliance")
        return entry

    def get_audit_log(self, tenant_id: str = None):
-        if tenant_id:
-            return [r for r in self.audit_log if r["tenant_id"] == tenant_id]
-        return self.audit_log
+        # Return from DB for full history across restarts
+        return policy_engine.get_violations(tenant_id=tenant_id)

    def clear_audit_log(self, tenant_id: str = None):
        if tenant_id:
--- a/governance/sla_monitor.py
+++ b/governance/sla_monitor.py
@@ -2,6 +2,7 @@

 import time
 from datetime import datetime
+from governance.policy_engine import policy_engine

 class SLAMonitor:
    def __init__(self):
@@ -20,6 +21,9 @@ class SLAMonitor:
        }
        self.violations.append(record)
        self._log_metrics(agent_role, latency, task, output, not breach)
+        if breach:
+            reason = f"SLA breach: latency={latency}ms or success_criteria not met"
+            policy_engine.log_violation("default", agent_role, "", task, reason, "sla", latency)
        return record

    def _log_metrics(self, agent_role: str, latency: float, task: str, output: str, success: bool):
@@ -37,11 +41,12 @@ class SLAMonitor:
            "timestamp": datetime.utcnow()
        })

-    def get_violations(self):
-        return self.violations
+    def get_violations(self, since_ts: float = None):
+        return policy_engine.get_violations(vtype="sla", since_ts=since_ts)

    def get_by_agent(self, agent_role: str):
-        return [v for v in self.violations if v["agent"] == agent_role]
+        all_v = policy_engine.get_violations(vtype="sla")
+        return [v for v in all_v if v.get("agent_role") == agent_role]
    
    def get_summary(self, agent_role: str):
        metrics = self.agent_metrics.get(agent_role, {})
--- a/main.py
+++ b/main.py
@@ -76,7 +76,7 @@ from routes.role_routes import router as role_router
 from routes.collaboration_routes import router as collaboration_router
 from routes.coordination_routes import router as coordination_router
 from routes.autonomous_planner_routes import router as planner_router
-from routes.self_evaluation_routes import router as evaluation_router
+from routes.self_evaluation_routes import router as self_evaluation_router
 from routes.performance_routes import router as performance_router
 from routes.action_routes import router as action_router
 from routes.tenant_admin_routes import router as tenant_admin_router
@@ -226,6 +226,7 @@ app.include_router(role_router, prefix="/admin")
 app.include_router(collaboration_router, prefix="/admin")
 app.include_router(coordination_router, prefix="/admin")
 app.include_router(planner_router, prefix="/admin")
+app.include_router(self_evaluation_router, prefix="/admin")
 app.include_router(evaluation_router, prefix="/admin")
 app.include_router(performance_router, prefix="/admin")
 app.include_router(action_router, prefix="/admin")
@@ -243,7 +244,7 @@ app.include_router(deployment_router, prefix="/admin")
 app.include_router(plugin_router, prefix="/admin")
 app.include_router(model_router, prefix="/admin")
 app.include_router(inference_router, prefix="/admin")
-app.include_router(evaluation_router, prefix="/admin")
+# NOTE: evaluation_router was previously registered twice (duplicate removed 2026-02-25)
 app.include_router(sla_router, prefix="/admin")
 app.include_router(cluster_router, prefix="/admin")
 app.include_router(monitoring_router, prefix="/admin")
--- a/routes/governance_routes.py
+++ b/routes/governance_routes.py
@@ -2,10 +2,13 @@
 ##INFO:

 from fastapi import APIRouter
+from typing import Optional
+from pydantic import BaseModel
 from governance.usage_meter import usage_meter
 from governance.billing_engine import billing_engine
 from governance.policy_registry import policy_registry
 from governance.compliance_checker import compliance_checker
+from governance.policy_engine import policy_engine

 router = APIRouter()

@@ -49,3 +52,55 @@ def check_compliance(tenant_id: str, agent_role: str, task: str):
 def get_all_violations():
    return {"violations": compliance_checker.get_violations()}

+# -----------------------------------------------------------------------
+# Phase 12 Step 1.1 — Unified Policy Engine endpoints
+# -----------------------------------------------------------------------
+
+class EvaluateRequest(BaseModel):
+    tenant_id: str
+    role: str
+    action: str
+    task: str = ""
+    latency_ms: Optional[float] = None
+
+
+@router.post("/admin/governance/evaluate")
+def evaluate_policy(req: EvaluateRequest):
+    """Run full governance evaluation (RBAC + task restriction + quota + SLA)."""
+    return policy_engine.evaluate(
+        tenant_id=req.tenant_id,
+        role=req.role,
+        action=req.action,
+        task=req.task,
+        latency_ms=req.latency_ms
+    )
+
+
+@router.get("/admin/governance/violations/db")
+def get_db_violations(
+    tenant_id: Optional[str] = None,
+    vtype: Optional[str] = None,
+    limit: int = 100
+):
+    """Return persisted violations from governance.db (survives restarts)."""
+    return {"violations": policy_engine.get_violations(tenant_id=tenant_id, vtype=vtype, limit=limit)}
+
+
+@router.get("/admin/governance/usage/{tenant_id}/{agent_role}")
+def get_db_usage(tenant_id: str, agent_role: str):
+    """Return persisted quota/usage for a specific tenant+role from governance.db."""
+    return policy_engine.check_quota(tenant_id, agent_role)
+
+
+@router.delete("/admin/governance/violations")
+def clear_db_violations(tenant_id: Optional[str] = None):
+    """Clear violations from governance.db (admin action)."""
+    policy_engine.clear_violations(tenant_id)
+    return {"status": "cleared", "tenant_id": tenant_id or "all"}
+
+
+@router.post("/admin/governance/sla-threshold")
+def set_sla_threshold(tenant_id: str, agent_role: str, max_latency_ms: float):
+    """Set per-tenant per-role SLA latency threshold in governance.db."""
+    policy_engine.set_sla_threshold(tenant_id, agent_role, max_latency_ms)
+    return {"status": "sla threshold set", "max_latency_ms": max_latency_ms}
--- a/tenants/tenant_registry.py
+++ b/tenants/tenant_registry.py
@@ -256,22 +256,12 @@ class TenantRegistry:
        return role in allowed if allowed else True

    def is_capability_allowed(self, tenant_id: str, capability: str):
-        allowed = self.get_allowed_capabilities(tenant_id)
        allowed = self.tenants.get(tenant_id, {}).get("settings", {}).get("allowed_capabilities", [])
        return capability in allowed if allowed else True  # default allow if not set

-    def check_quota(self, tenant_id: str):
-        usage = self.tenants.setdefault(tenant_id, {}).setdefault("usage", 0)
-        limit = self.tenants[tenant_id].get("settings", {}).get("quota_limit", 1000)
-        return usage < limit
-
-    def log_usage(self, tenant_id: str, role: str, task: str):
-        self.tenants[tenant_id]["usage"] += 1
-        self.tenants[tenant_id].setdefault("usage_log", []).append({
-            "timestamp": time.time(),
-            "role": role,
-            "task": task
-        })
+    # NOTE: check_quota and log_usage canonical definitions are above (lines ~96 and ~72).
+    # Duplicate definitions removed 2026-02-25 — they were silently overwriting the correct
+    # quota-dict-based implementations with broken int-based logic.


 # Singleton
--- a/tests/verify_phase_12_step_1_1.py
+++ b/tests/verify_phase_12_step_1_1.py
@@ -0,0 +1,144 @@
+#!/usr/bin/env python3
+"""
+tests/verify_phase_12_step_1_1.py
+Phase 12 Step 1.1 — Unified Policy Engine Verification
+
+Tests:
+  1. governance.db is created with correct schema
+  2. set_policy() persists and survives re-instantiation
+  3. evaluate() → denied for disallowed role
+  4. evaluate() → denied for restricted task
+  5. evaluate() → denied when quota exceeded
+  6. evaluate() → denied when latency exceeds SLA
+  7. evaluate() → allowed for fully compliant request
+  8. get_violations() returns persisted violations after re-instantiation
+  9. Backward-compat: existing singletons still importable
+
+Run with:
+  cd /home/dev1/src/_GIT/awesome-agentic-ai
+  source venv/bin/activate
+  python tests/verify_phase_12_step_1_1.py
+"""
+
+import sys
+import os
+import tempfile
+
+sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), "..")))
+
+# Use a temp DB so tests don't pollute real governance.db
+TEST_DB = os.path.join(tempfile.mkdtemp(), "test_governance.db")
+
+from governance.policy_engine import PolicyEngine
+
+PASS = "✅ PASS"
+FAIL = "❌ FAIL"
+results = []
+
+
+def check(label, condition, detail=""):
+    status = PASS if condition else FAIL
+    results.append((label, status, detail))
+    print(f"  {status} — {label}" + (f" ({detail})" if detail else ""))
+    return condition
+
+
+print("\n=== Phase 12 Step 1.1: Unified Policy Engine ===\n")
+
+# --- Test 1: DB schema creation ---
+print("[1] governance.db schema creation")
+engine = PolicyEngine(db_path=TEST_DB)
+import sqlite3
+with sqlite3.connect(TEST_DB) as con:
+    tables = {r[0] for r in con.execute("SELECT name FROM sqlite_master WHERE type='table'").fetchall()}
+check("policies table exists", "policies" in tables)
+check("violations table exists", "violations" in tables)
+check("usage table exists", "usage" in tables)
+check("billing_rates table exists", "billing_rates" in tables)
+check("sla_thresholds table exists", "sla_thresholds" in tables)
+
+# --- Test 2: Persistence across re-instantiation ---
+print("\n[2] Policy persistence across re-instantiation")
+engine.set_policy("tenant_test", ["admin", "user"], ["delete_all"], "strict")
+engine2 = PolicyEngine(db_path=TEST_DB)
+policy = engine2.get_policy("tenant_test")
+check("allowed_roles persisted", policy["allowed_roles"] == ["admin", "user"], str(policy["allowed_roles"]))
+check("restricted_tasks persisted", policy["restricted_tasks"] == ["delete_all"])
+check("audit_level persisted", policy["audit_level"] == "strict")
+
+# --- Test 3: Denied — disallowed role ---
+print("\n[3] evaluate() → denied for disallowed role")
+result = engine.evaluate("tenant_test", role="unknown_role", action="run_agent")
+check("allowed=False", not result["allowed"])
+check("rbac check failed", not result["checks"]["rbac"]["passed"])
+
+# --- Test 4: Denied — restricted task ---
+print("\n[4] evaluate() → denied for restricted task keyword")
+result = engine.evaluate("tenant_test", role="admin", action="run_agent", task="please delete_all records")
+check("allowed=False for restricted task", not result["allowed"])
+check("task_restriction check failed", not result["checks"]["task_restriction"]["passed"])
+
+# --- Test 5: Denied — quota exceeded ---
+print("\n[5] evaluate() → denied when quota exceeded")
+engine.set_quota("tenant_q", "executor", limit=2)
+# Consume the quota manually
+engine.log_usage("tenant_q", "executor", 2)
+q = engine.check_quota("tenant_q", "executor")
+check("quota status=exceeded", q["status"] == "exceeded", str(q))
+# Set policy before evaluating
+engine.set_policy("tenant_q", ["executor"], [], "standard")
+result = engine.evaluate("tenant_q", role="executor", action="run_agent")
+check("allowed=False when quota exceeded", not result["allowed"])
+check("quota check status=exceeded", result["checks"]["quota"]["status"] == "exceeded")
+
+# --- Test 6: Denied — SLA exceeded ---
+print("\n[6] evaluate() → denied when latency exceeds SLA threshold")
+engine.set_policy("tenant_sla", ["monitor"], [], "standard")
+engine.set_sla_threshold("tenant_sla", "monitor", max_latency_ms=500.0)
+result = engine.evaluate("tenant_sla", role="monitor", action="query", latency_ms=1200.0)
+check("allowed=False for SLA breach", not result["allowed"])
+check("sla check passed=False", not result["checks"]["sla"]["passed"])
+check("latency recorded correctly", result["checks"]["sla"]["latency_ms"] == 1200.0)
+
+# --- Test 7: Allowed — fully compliant ---
+print("\n[7] evaluate() → allowed for compliant request")
+engine.set_policy("tenant_ok", ["planner", "executor"], [], "standard")
+result = engine.evaluate("tenant_ok", role="executor", action="run_task", task="analyze data", latency_ms=300.0)
+check("allowed=True", result["allowed"], str(result.get("violations", [])))
+check("rbac passed", result["checks"]["rbac"]["passed"])
+check("no sla breach", result["checks"]["sla"]["passed"])
+
+# --- Test 8: Violations persisted and readable after re-instantiation ---
+print("\n[8] Violations persist across re-instantiation")
+engine3 = PolicyEngine(db_path=TEST_DB)
+violations = engine3.get_violations()
+check("violations count > 0 after restart", len(violations) > 0, f"count={len(violations)}")
+sla_v = engine3.get_violations(vtype="sla")
+check("sla violations in DB", len(sla_v) > 0, f"count={len(sla_v)}")
+
+# --- Test 9: Backward compat — existing singletons still importable ---
+print("\n[9] Backward compatibility — existing singletons")
+try:
+    from governance.policy_registry import policy_registry
+    from governance.compliance_checker import compliance_checker
+    from governance.sla_monitor import sla_monitor
+    from governance.usage_meter import usage_meter
+    from governance.billing_engine import billing_engine
+    check("All existing singletons import OK", True)
+except Exception as e:
+    check("All existing singletons import OK", False, str(e))
+
+# --- Summary ---
+print("\n" + "="*50)
+passed = sum(1 for _, s, _ in results if s == PASS)
+failed = sum(1 for _, s, _ in results if s == FAIL)
+print(f"Results: {passed}/{passed+failed} passed")
+if failed:
+    print("\nFailed checks:")
+    for label, status, detail in results:
+        if status == FAIL:
+            print(f"  {label}: {detail}")
+    sys.exit(1)
+else:
+    print("All checks passed ✅")
+    sys.exit(0)