diff --git a/AGENTS.md b/AGENTS.md index 28308195..09f873a5 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -67,6 +67,9 @@ Use these for depth, onboarding, and review. Each expands themes from the master | [docs/blueprint-master-architecture.md](docs/blueprint-master-architecture.md) | Master blueprint index | | [docs/execution-matrix-90d-tier1.md](docs/execution-matrix-90d-tier1.md) | Phase 0–1 outcomes vs matrix | | [docs/enterprise-readiness.md](docs/enterprise-readiness.md) | B2B / enterprise preparation checklist | +| [docs/completion-program-workstreams.md](docs/completion-program-workstreams.md) | Eight workstreams: docs → enterprise runtime | +| [docs/architecture-register.md](docs/architecture-register.md) | Subsystem status (Current / Partial / Pilot / Production) | +| [docs/adr/0002-execution-matrix-canonical-source.md](docs/adr/0002-execution-matrix-canonical-source.md) | Canonical `Execution_Matrix.md` vs draft v2 | | [docs/adr/0001-tier1-execution-policy-spikes.md](docs/adr/0001-tier1-execution-policy-spikes.md) | Gated spikes: Temporal, OPA, OpenFGA | Operating overview with diagram: **[`docs/ai-operating-model.md`](docs/ai-operating-model.md)**. diff --git a/Execution_Matrix_v2.md b/Execution_Matrix_v2.md index 09dd97b3..7424f3cf 100644 --- a/Execution_Matrix_v2.md +++ b/Execution_Matrix_v2.md @@ -1,6 +1,8 @@ # Dealix Sovereign Growth OS: Master Execution Matrix v2.0 -تمثل هذه المصفوفة المحرك التشغيلي (Operational Backbone) الفعلي والنهائي لـ **Dealix Sovereign Growth OS**. توضح كل التفاصيل التنفيذية لتشغيل الـ 16 وكيلًا عبر 4 عائلات استراتيجية، مع دمج الأدوات المتقدمة (LangGraph, Claude Code, Repomix, Atomic Chat, Shannon, Goose) وتطبيق الحوكمة (Policy-as-code) وقيود الإثبات (Verification Layer). +> **DRAFT — non-canonical until merged.** The authoritative agent matrix for numbering, cross-repo links, and Phase 0–1 planning is **[`Execution_Matrix.md`](Execution_Matrix.md)**. This v2 document captures **proposed** enhancements (e.g. evidence columns, alternate tooling). Merge policy: [`docs/adr/0002-execution-matrix-canonical-source.md`](docs/adr/0002-execution-matrix-canonical-source.md). + +تمثل هذه المصفوفة مسودة تطوير للمحرك التشغيلي (Operational Backbone) لـ **Dealix Sovereign Growth OS**. توضح تفاصيل مقترحة لتشغيل الوكلاء عبر العائلات الاستراتيجية، مع دمج أفكار أدوات وتشغيل (قد لا تكون كلها مدمجة في الكود بعد). --- diff --git a/MASTER_OPERATING_PROMPT.md b/MASTER_OPERATING_PROMPT.md index e8265a41..e42149b8 100644 --- a/MASTER_OPERATING_PROMPT.md +++ b/MASTER_OPERATING_PROMPT.md @@ -27,6 +27,9 @@ Deep-dive topics live under [`docs/governance/`](docs/governance/) (keep this fi | Master architecture blueprint (index) | [`docs/blueprint-master-architecture.md`](docs/blueprint-master-architecture.md) | | 90-day Tier-1 execution matrix | [`docs/execution-matrix-90d-tier1.md`](docs/execution-matrix-90d-tier1.md) | | Enterprise readiness (B2B checklist) | [`docs/enterprise-readiness.md`](docs/enterprise-readiness.md) | +| Completion Program (8 workstreams) | [`docs/completion-program-workstreams.md`](docs/completion-program-workstreams.md) | +| Architecture register (subsystem status) | [`docs/architecture-register.md`](docs/architecture-register.md) | +| ADR: Execution matrix canonical (v1 vs v2) | [`docs/adr/0002-execution-matrix-canonical-source.md`](docs/adr/0002-execution-matrix-canonical-source.md) | | ADR: Temporal / OPA / OpenFGA spikes | [`docs/adr/0001-tier1-execution-policy-spikes.md`](docs/adr/0001-tier1-execution-policy-spikes.md) | --- diff --git a/docs/adr/0002-execution-matrix-canonical-source.md b/docs/adr/0002-execution-matrix-canonical-source.md new file mode 100644 index 00000000..46fc77a5 --- /dev/null +++ b/docs/adr/0002-execution-matrix-canonical-source.md @@ -0,0 +1,26 @@ +# ADR 0002 — Canonical source for the Master Execution Matrix + +- **Status:** Accepted +- **Date:** 2026-04-16 +- **Context:** Two documents describe the 16-agent operational backbone: [`Execution_Matrix.md`](../../Execution_Matrix.md) (root) and [`Execution_Matrix_v2.md`](../../Execution_Matrix_v2.md). They differ in columns (v2 adds evidence requirements, alternate tool names), event naming, and some SLAs/agent naming. + +## Decision + +1. **[`Execution_Matrix.md`](../../Execution_Matrix.md)** is the **canonical** matrix for **agent numbering (1–16)**, Arabic operational tables aligned with historical repo references, and cross-links from [`docs/dealix-six-tracks.md`](../dealix-six-tracks.md) and [`docs/execution-matrix-90d-tier1.md`](../execution-matrix-90d-tier1.md). + +2. **[`Execution_Matrix_v2.md`](../../Execution_Matrix_v2.md)** is a **draft enhancement** (v2.0). It must be treated as **non-authoritative** until merged into v1 via a deliberate PR that: + - reconciles event names with code and [`docs/governance/events-and-schema.md`](../governance/events-and-schema.md), + - ports the **Evidence Required** column into v1 without breaking agent IDs, + - removes or verifies speculative tool references (e.g. product names that are not integrated). + +3. Until merge completes, any new automation, router mapping, or KPI row **must** cite `Execution_Matrix.md` first; v2 may inform design only. + +## Consequences + +- Prevents split-brain between “matrix used in docs” and “matrix used in code comments.” +- Forces one PR to merge v2 deltas instead of silent drift. + +## Related + +- [`../completion-program-workstreams.md`](../completion-program-workstreams.md) WS1 +- [`0001-tier1-execution-policy-spikes.md`](0001-tier1-execution-policy-spikes.md) diff --git a/docs/architecture-register.md b/docs/architecture-register.md new file mode 100644 index 00000000..619fd832 --- /dev/null +++ b/docs/architecture-register.md @@ -0,0 +1,33 @@ +# Architecture register — subsystem status (Completion Program WS1) + +**Purpose:** Single **code-backed** snapshot of **Current / Partial / Pilot / Production** for major subsystems. Refresh per milestone or release. +**Canonical agent matrix:** [`Execution_Matrix.md`](../Execution_Matrix.md) (see [`adr/0002-execution-matrix-canonical-source.md`](adr/0002-execution-matrix-canonical-source.md) for v2 draft status). + +| Subsystem | Path / anchor | Status | Evidence / notes | +|-----------|---------------|--------|-------------------| +| FastAPI API surface | `salesflow-saas/backend/app/main.py`, `app/api/` | **Production** (dev/staging/prod per deploy) | pytest API suites | +| Agent router / executor | `salesflow-saas/backend/app/services/agents/` | **Partial** | LangGraph + routing; extend structured bundle (WS2) | +| Decision memo (Pydantic) | `salesflow-saas/backend/app/services/core_os/decision_memo.py` | **Production** | Schema used as universal memo contract | +| Decision plane bundle (A/R/S + intent) | `salesflow-saas/backend/app/services/core_os/decision_plane_contracts.py` | **Production** (initial) | WS2 — compose `memo` + `evidence_pack` + `approval_packet` + `execution_intent` | +| Tool verification ledger | `salesflow-saas/backend/app/services/core_os/verification_ledger.py` | **Partial** | File-based proofs; wire to DB/API for multi-instance (WS4) | +| Durable flows (LangGraph) | `salesflow-saas/backend/app/flows/` | **Partial** | `prospecting_durable_flow.py`, `self_improvement_flow.py` | +| Celery workers | `salesflow-saas/backend/app/workers/` | **Production** | Tasks for sequences, agents, notifications, affiliates | +| Temporal durable engine | — | **Planned** | [`adr/0001-tier1-execution-policy-spikes.md`](adr/0001-tier1-execution-policy-spikes.md) | +| Policy engine (in-app) | `salesflow-saas/backend/app/services/dealix_os/policy_engine.py` | **Partial** | OPA/FGA target in [`governance/trust-fabric.md`](governance/trust-fabric.md) | +| Strategic deals / M&A helpers | `salesflow-saas/backend/app/services/strategic_deals/` | **Partial** | Multiple modules; HITL in matrix | +| Security gate | `salesflow-saas/backend/app/services/security_gate.py` | **Partial** | Expand release gates (WS6) | +| Audit log model | `salesflow-saas/backend/app/models/audit_log.py` | **Partial** | Enterprise audit streaming TBD (WS6) | +| OpenTelemetry | — | **Planned / partial** | Correlation IDs in some paths; full OTel per radar | +| OPA / OpenFGA / Vault / Keycloak | — | **Planned** | ADR-0001 spikes only | +| Semantic metrics dictionary | `docs/semantic-metrics-dictionary.md` | **Pilot** (doc) | Code single source TBD (WS5) | +| Lineage catalog | `docs/lineage-catalog-choice.md` | **Pilot** (doc) | Default recommendation: OpenLineage until ADR | +| PDPL / NCA / AI control matrices | `docs/governance/pdpl-nca-ai-control-matrices.md` | **Pilot** (doc) | Operationalize per release (WS7) | +| Executive room UI/API | `salesflow-saas/frontend/`, APIs TBD | **Planned / partial** | [`executive-room-completion-spec.md`](executive-room-completion-spec.md) | + +## Rules + +- Promote a row to **Production** only with tests + runbook + owner sign-off. +- **Pilot** requires feature flag, scope note, and rollback. +- **Planned** rows must link to an ADR or workstream ID. + +See [`completion-program-workstreams.md`](completion-program-workstreams.md) for the eight workstreams and exit criteria. diff --git a/docs/blueprint-master-architecture.md b/docs/blueprint-master-architecture.md index 0434d910..dd87f420 100644 --- a/docs/blueprint-master-architecture.md +++ b/docs/blueprint-master-architecture.md @@ -19,7 +19,7 @@ For the classic “8 layers” service map (signal, memory, reasoning, orchestra ## Agents, events, and HITL -- **16 agents × events × KPIs × gates:** [`Execution_Matrix.md`](../Execution_Matrix.md); alternate or delta matrix: [`Execution_Matrix_v2.md`](../Execution_Matrix_v2.md) (keep a single source of truth — avoid conflicting agent IDs between files). +- **16 agents × events × KPIs × gates:** [`Execution_Matrix.md`](../Execution_Matrix.md) (**canonical** per [`adr/0002-execution-matrix-canonical-source.md`](adr/0002-execution-matrix-canonical-source.md)); draft deltas: [`Execution_Matrix_v2.md`](../Execution_Matrix_v2.md) (non-authoritative until merged). ## Execution and trust (Tier-1) @@ -38,6 +38,12 @@ For the classic “8 layers” service map (signal, memory, reasoning, orchestra - [`execution-matrix-90d-tier1.md`](execution-matrix-90d-tier1.md) +## Completion Program (docs → runtime) + +- Master workstream index: [`completion-program-workstreams.md`](completion-program-workstreams.md) +- Subsystem status register: [`architecture-register.md`](architecture-register.md) +- Execution matrix canonical decision: [`adr/0002-execution-matrix-canonical-source.md`](adr/0002-execution-matrix-canonical-source.md) + ## Enterprise readiness - B2B preparation checklist: [`enterprise-readiness.md`](enterprise-readiness.md) diff --git a/docs/completion-program-workstreams.md b/docs/completion-program-workstreams.md new file mode 100644 index 00000000..5a1cfe5f --- /dev/null +++ b/docs/completion-program-workstreams.md @@ -0,0 +1,53 @@ +# Dealix Completion Program — workstreams (execution matrix) + +**Goal:** Close the gap between Tier-1 **documentation** and **enterprise-grade runtime** (Decision + Execution + Trust + Data + Operating fabrics). +**Priority order:** Trust/Control → Execution durability → Connector facades → Semantic metrics → Saudi governance → Executive surfaces. + +**Living registers:** [`architecture-register.md`](architecture-register.md) (subsystem status), [`adr/0002-execution-matrix-canonical-source.md`](adr/0002-execution-matrix-canonical-source.md) (matrix source of truth). + +**PR #16 closure bundle (merged):** [`salesflow-saas/docs/tier1-master-closure-checklist.md`](../salesflow-saas/docs/tier1-master-closure-checklist.md) (50-item master gates) + supporting tracks under [`salesflow-saas/docs/`](../salesflow-saas/docs/) and [`salesflow-saas/docs/governance/`](../salesflow-saas/docs/governance/) — use alongside this index; prefer **one** status column between the register and the master checklist to avoid drift. + +| WS | Name | SLA (target) | Primary deliverable docs / code | +|----|------|--------------|-----------------------------------| +| WS1 | Architecture closure | 2–3 wk | This file + register + ADR 0002; banner on [`Execution_Matrix_v2.md`](../Execution_Matrix_v2.md) | +| WS2 | Decision plane hardening | 2–4 wk | [`decision_plane_contracts.py`](../salesflow-saas/backend/app/services/core_os/decision_plane_contracts.py) + [`decision_memo.py`](../salesflow-saas/backend/app/services/core_os/decision_memo.py); enforce on Class B routes incrementally | +| WS3 | Execution plane hardening | 4–8 wk | [`workflows-inventory.md`](workflows-inventory.md), [`temporal-pilot-scope.md`](temporal-pilot-scope.md), [`adr/0001-tier1-execution-policy-spikes.md`](adr/0001-tier1-execution-policy-spikes.md) | +| WS4 | Trust fabric hardening | 4–8 wk | [`trust/tool-verification-ledger-v1-completion.md`](trust/tool-verification-ledger-v1-completion.md), [`verification_ledger.py`](../salesflow-saas/backend/app/services/core_os/verification_ledger.py) | +| WS5 | Data & connector fabric | 4–8 wk | [`ws5-connector-events-metrics.md`](ws5-connector-events-metrics.md), [`semantic-metrics-dictionary.md`](semantic-metrics-dictionary.md), [`lineage-catalog-choice.md`](lineage-catalog-choice.md) | +| WS6 | Enterprise delivery fabric | 2–4 wk | [`github-enterprise-delivery-completion.md`](github-enterprise-delivery-completion.md) | +| WS7 | Saudi enterprise readiness | 2–4 wk | [`governance/pdpl-nca-ai-control-matrices.md`](governance/pdpl-nca-ai-control-matrices.md) + [`saudi-compliance-and-ai-governance.md`](governance/saudi-compliance-and-ai-governance.md) | +| WS8 | Executive & customer readiness | 3–6 wk | [`executive-room-completion-spec.md`](executive-room-completion-spec.md) | + +## Per-workstream contract (fill in owners / PRs / dates in your PM tool) + +Use columns: **Workstream → Deliverables → Owner → Evidence Gate → Exit Criteria → Dependencies → Risk → SLA**. + +Detailed field tables are preserved in the approved plan document in `.cursor/plans/` (do not duplicate full prose here); this file is the **repo execution index**. + +## Definition of Done (recap) + +Enterprise-grade when, with evidence: structured critical recommendations; durable long workflows where required; A/R/S on sensitive actions; versioned connector facades; GitHub/OIDC release discipline; OTel-style traceability on critical paths; OWASP/NIST-aligned checks on LLM surfaces; Saudi control matrices tied to releases. + +## Dependency diagram + +```mermaid +flowchart LR + ws1[WS1] + ws2[WS2] + ws3[WS3] + ws4[WS4] + ws5[WS5] + ws6[WS6] + ws7[WS7] + ws8[WS8] + ws1 --> ws2 + ws1 --> ws3 + ws1 --> ws4 + ws2 --> ws3 + ws2 --> ws4 + ws4 --> ws5 + ws4 --> ws7 + ws5 --> ws8 + ws6 --> ws3 + ws6 --> ws4 +``` diff --git a/docs/dealix-six-tracks.md b/docs/dealix-six-tracks.md index e42041fc..f132cd11 100644 --- a/docs/dealix-six-tracks.md +++ b/docs/dealix-six-tracks.md @@ -45,4 +45,7 @@ Use this to avoid claiming components that are not yet wired in production. Refr - Architecture pack (layers): [`Architecture_Pack.md`](../Architecture_Pack.md) - Tier-1 blueprint (index): [`docs/blueprint-master-architecture.md`](blueprint-master-architecture.md) - 90-day Tier-1 matrix: [`docs/execution-matrix-90d-tier1.md`](execution-matrix-90d-tier1.md) -- Enterprise readiness checklist: [`docs/enterprise-readiness.md`](enterprise-readiness.md) +- Enterprise readiness checklist: [`docs/enterprise-readiness.md`](enterprise-readiness.md) +- Completion Program (WS1–WS8): [`docs/completion-program-workstreams.md`](completion-program-workstreams.md) +- Architecture register (subsystem status): [`docs/architecture-register.md`](architecture-register.md) +- Execution matrix canonical source: [`docs/adr/0002-execution-matrix-canonical-source.md`](adr/0002-execution-matrix-canonical-source.md) diff --git a/docs/enterprise-readiness.md b/docs/enterprise-readiness.md index 36d09cdb..7d23c206 100644 --- a/docs/enterprise-readiness.md +++ b/docs/enterprise-readiness.md @@ -10,7 +10,9 @@ This checklist helps **internal teams** prepare for **B2B / enterprise** convers 4. [`governance/trust-fabric.md`](governance/trust-fabric.md) — trust substrate and tool verification. 5. [`governance/saudi-compliance-and-ai-governance.md`](governance/saudi-compliance-and-ai-governance.md) — PDPL / NCA readiness register and AI governance frames. 6. [`governance/github-and-release.md`](governance/github-and-release.md) — branch protection, environments, OIDC, audit retention. -7. [`execution-matrix-90d-tier1.md`](execution-matrix-90d-tier1.md) — Phase 0–1 measurable outcomes. +7. [`execution-matrix-90d-tier1.md`](execution-matrix-90d-tier1.md) — Phase 0–1 measurable outcomes. +8. [`completion-program-workstreams.md`](completion-program-workstreams.md) — eight workstreams from constitution to production. +9. [`architecture-register.md`](architecture-register.md) — subsystem status snapshot. ## 2. Product and legal surface diff --git a/docs/executive-room-completion-spec.md b/docs/executive-room-completion-spec.md new file mode 100644 index 00000000..d56b7e2e --- /dev/null +++ b/docs/executive-room-completion-spec.md @@ -0,0 +1,21 @@ +# Executive room & customer readiness — WS8 spec + +**Goal:** Executive-visible surfaces backed by **trusted data only** (no hallucinated KPIs). + +## Milestones + +1. **Read-only executive dashboard** — memos + evidence pack viewer fed from APIs returning [`decision_plane_contracts.assemble_decision_bundle`](../salesflow-saas/backend/app/services/core_os/decision_plane_contracts.py) payloads. +2. **Approval center** — queue of Class B items with A/R/S and approver roles. +3. **Policy violations board** — feed from audit log + tool ledger contradictions. +4. **Partner scorecards** — metrics from [`semantic-metrics-dictionary.md`](semantic-metrics-dictionary.md) keys only. +5. **Actual vs forecast** — charts bound to semantic metrics + finance exports. +6. **Risk heatmaps** — aggregated from `risk_register` on latest memos per initiative. + +## Security + +- RBAC / tenant isolation mandatory; sensitive rows require OpenFGA-style checks when available (WS4). +- No PII in client-side logs. + +## Dependencies + +WS2 (bundles), WS4 (ledger), WS5 (metrics), partial WS3 (workflow state for approvals). diff --git a/docs/github-enterprise-delivery-completion.md b/docs/github-enterprise-delivery-completion.md new file mode 100644 index 00000000..d40a54e3 --- /dev/null +++ b/docs/github-enterprise-delivery-completion.md @@ -0,0 +1,29 @@ +# Enterprise delivery fabric — WS6 checklist + +**Reference:** [`governance/github-and-release.md`](governance/github-and-release.md). + +## Repository / org controls + +- [ ] Rulesets on `main` (and release branches): no direct push, required reviews, required status checks. +- [ ] CODEOWNERS for critical paths (`backend/app/api`, auth, payments, agents). +- [ ] Merge queue (when CI stable). +- [ ] Conversation resolution required before merge (policy). + +## Environments + +- [ ] GitHub Environments: `dev`, `staging`, `canary`, `prod` with protection rules. +- [ ] Required reviewers / wait timers where **GitHub Enterprise** allows (document limits for private repos per org tier). +- [ ] “Deployments must succeed” gate where applicable. + +## Secrets and provenance + +- [ ] OIDC federation to cloud roles for deploy workflows (no long-lived cloud secrets in repo). +- [ ] Artifact attestations / provenance where supply-chain risk warrants. + +## Audit retention reality + +- Enterprise audit log retention limits; Git events short retention — **plan SIEM / warehouse streaming** for audit-grade customers (link runbooks when added). + +## Evidence + +Store screenshots or org policy links (internal) as evidence for enterprise questionnaires; do not commit secrets. diff --git a/docs/governance/README.md b/docs/governance/README.md index 8799d74d..306cfb27 100644 --- a/docs/governance/README.md +++ b/docs/governance/README.md @@ -17,7 +17,8 @@ This folder expands each major theme for navigation, review, and agent onboardin | [execution-fabric.md](execution-fabric.md) | Durable execution: current stack vs Temporal target | | [technology-radar-tier1.md](technology-radar-tier1.md) | Tier-1 technology radar (official / optional / pilot) | | [saudi-compliance-and-ai-governance.md](saudi-compliance-and-ai-governance.md) | PDPL/NCA readiness register, NIST/OWASP AI governance | +| [pdpl-nca-ai-control-matrices.md](pdpl-nca-ai-control-matrices.md) | WS7 operational control matrices (templates) | -**Tier-1 index docs (repo `docs/`):** [dealix-six-tracks.md](../dealix-six-tracks.md), [blueprint-master-architecture.md](../blueprint-master-architecture.md), [execution-matrix-90d-tier1.md](../execution-matrix-90d-tier1.md), [enterprise-readiness.md](../enterprise-readiness.md), [adr/0001-tier1-execution-policy-spikes.md](../adr/0001-tier1-execution-policy-spikes.md). +**Tier-1 index docs (repo `docs/`):** [dealix-six-tracks.md](../dealix-six-tracks.md), [blueprint-master-architecture.md](../blueprint-master-architecture.md), [completion-program-workstreams.md](../completion-program-workstreams.md), [architecture-register.md](../architecture-register.md), [execution-matrix-90d-tier1.md](../execution-matrix-90d-tier1.md), [enterprise-readiness.md](../enterprise-readiness.md), [adr/0001-tier1-execution-policy-spikes.md](../adr/0001-tier1-execution-policy-spikes.md), [adr/0002-execution-matrix-canonical-source.md](../adr/0002-execution-matrix-canonical-source.md). Repo entry points: [`../../AGENTS.md`](../../AGENTS.md), [`../../CLAUDE.md`](../../CLAUDE.md), [`../ai-operating-model.md`](../ai-operating-model.md). diff --git a/docs/governance/pdpl-nca-ai-control-matrices.md b/docs/governance/pdpl-nca-ai-control-matrices.md new file mode 100644 index 00000000..1345c2aa --- /dev/null +++ b/docs/governance/pdpl-nca-ai-control-matrices.md @@ -0,0 +1,40 @@ +# PDPL, NCA ECC, and AI control matrices (operational templates) — WS7 + +**Not legal advice.** Engineering templates to operationalize [`saudi-compliance-and-ai-governance.md`](saudi-compliance-and-ai-governance.md). + +## PDPL control matrix (template) + +| Control ID | Topic | Implementation hint | Evidence | +|------------|-------|----------------------|----------| +| PDPL-01 | Lawful basis documented | Link to consent / contract in [`salesflow-saas/docs/legal/`](../../salesflow-saas/docs/legal/) | Policy version + UI copy hash | +| PDPL-02 | Data minimization | Field-level collection review per feature | Design review sign-off | +| PDPL-03 | Subject access / export | Runbook + API capability | Test + ticket | +| PDPL-04 | Retention & deletion | TTL jobs + soft-delete | Job logs | +| PDPL-05 | Processor / subprocessor register | Table of vendors + regions | Updated quarterly | + +## NCA ECC readiness gap register (template) + +| ECC theme | Gap | Mitigation owner | Target date | Status | +|-----------|-----|------------------|-------------|--------| +| Asset management | … | … | … | Open | + +(Replace with organization-specific mapping against ECC 2-2024 controls.) + +## AI governance mapping (NIST AI RMF + OWASP LLM) + +| RMF function | Practical control | Release gate | +|--------------|-------------------|--------------| +| Govern | Model allowlist per environment | WS6 | +| Map | Data flow for prompts containing PII | WS7 | +| Measure | Offline eval + red-team subset | WS4 | +| Manage | Rollback + incident runbook | WS3/WS6 | + +| OWASP LLM risk | Mitigation | Test | +|----------------|------------|------| +| Prompt injection | Tool allowlists + input guards | Automated + manual | +| Insecure output handling | Schema validation on outputs | pytest | +| Excessive agency | Executor vs recommender separation | Policy review | + +## Region / residency flags + +Define configuration keys for **data region** and **LLM routing** per tenant; document in ADR when enforced in `policy_engine` or external PDP. diff --git a/docs/governance/saudi-compliance-and-ai-governance.md b/docs/governance/saudi-compliance-and-ai-governance.md index 482de0d4..6142fbc4 100644 --- a/docs/governance/saudi-compliance-and-ai-governance.md +++ b/docs/governance/saudi-compliance-and-ai-governance.md @@ -65,4 +65,4 @@ See [design-and-arabic.md](design-and-arabic.md). - **Quarterly:** re-read this register against shipped features and incident postmortems. - **Per major release:** update PDPL/NCA mapping appendix when product surface area changes. -See also: [technology-radar-tier1.md](technology-radar-tier1.md), [`../execution-matrix-90d-tier1.md`](../execution-matrix-90d-tier1.md). +See also: [technology-radar-tier1.md](technology-radar-tier1.md), [`../execution-matrix-90d-tier1.md`](../execution-matrix-90d-tier1.md), [pdpl-nca-ai-control-matrices.md](pdpl-nca-ai-control-matrices.md). diff --git a/docs/lineage-catalog-choice.md b/docs/lineage-catalog-choice.md new file mode 100644 index 00000000..ec94f16e --- /dev/null +++ b/docs/lineage-catalog-choice.md @@ -0,0 +1,7 @@ +# Lineage / metadata catalog — single choice (WS5) + +**Decision (draft):** Default to **OpenLineage** for lineage event emission and integration with common warehouses, **until** an ADR selects OpenMetadata (or another platform) based on ops cost and team skills. + +**Rationale:** Avoid two overlapping catalogs early ([`governance/technology-radar-tier1.md`](governance/technology-radar-tier1.md)). + +**Next step:** ADR comparing OpenLineage vs OpenMetadata for Dealix scale; pilot one pipeline (e.g. CRM sync job) emitting lineage events. diff --git a/docs/schemas/decision-plane/README.md b/docs/schemas/decision-plane/README.md new file mode 100644 index 00000000..94f0597b --- /dev/null +++ b/docs/schemas/decision-plane/README.md @@ -0,0 +1,15 @@ +# Decision plane JSON contracts + +**Source of truth (runtime):** [`salesflow-saas/backend/app/services/core_os/decision_plane_contracts.py`](../../../salesflow-saas/backend/app/services/core_os/decision_plane_contracts.py) and [`decision_memo.py`](../../../salesflow-saas/backend/app/services/core_os/decision_memo.py). + +JSON Schema files may be generated from Pydantic in a follow-up PR (`model_json_schema()`). Until then, use the Python models for validation in APIs and workers. + +Bundle keys for governed responses: + +- `memo_json` — `DecisionMemo.model_dump()` +- `evidence_pack_json` — `EvidencePack` +- `risk_register_json` — list from memo +- `approval_packet_json` — `ApprovalPacket` +- `execution_intent_json` — `ExecutionIntent` + +See [`completion-program-workstreams.md`](../../completion-program-workstreams.md) WS2. diff --git a/docs/semantic-metrics-dictionary.md b/docs/semantic-metrics-dictionary.md new file mode 100644 index 00000000..c54f7709 --- /dev/null +++ b/docs/semantic-metrics-dictionary.md @@ -0,0 +1,16 @@ +# Semantic metrics dictionary (draft v0) + +**Status:** Pilot — definitions here precede code centralization. Update when analytics modules converge. + +| Metric key | Definition | Primary source | Owner | +|------------|------------|----------------|-------| +| `revenue_sar` | Recognized revenue in SAR for period P | Billing / finance system of record | Finance | +| `pipeline_value_sar` | Weighted pipeline in SAR | CRM deals | RevOps | +| `qualified_lead_count` | Leads meeting ICP + score threshold | CRM + scoring | Sales | +| `partner_sourced_pipeline_sar` | Pipeline attributed to partner channel | CRM attribution | Partnerships | +| `synergy_realization_sar` | Post-close synergy captured vs plan | Finance + PMI tracker | CorpDev | + +## Rules + +- Do not redefine the same key in multiple services. +- Dashboards and agent memos must reference **keys** from this table (or a future `app/analytics/metrics_catalog.py`). diff --git a/docs/temporal-pilot-scope.md b/docs/temporal-pilot-scope.md new file mode 100644 index 00000000..58a5e4ba --- /dev/null +++ b/docs/temporal-pilot-scope.md @@ -0,0 +1,17 @@ +# Temporal pilot scope — WS3 + +**Status:** Planned — gated by [`adr/0001-tier1-execution-policy-spikes.md`](adr/0001-tier1-execution-policy-spikes.md). + +## Recommended first pilot (pick one) + +1. **Partner approval** — human waits, multi-day SLA, idempotent notifications. +2. **DD room state machine** — long-running, audit-heavy, compensating actions on red-flag. + +## Non-goals for pilot v0 + +- Replacing all Celery workloads. +- Running Temporal without CI integration tests and local `docker compose` recipe. + +## Exit criteria (from ADR 0001) + +Worker versioning documented; crash resume verified; secrets boundary reviewed; rollback runbook; product sign-off for second workflow migration. diff --git a/docs/trust/tool-verification-ledger-v1-completion.md b/docs/trust/tool-verification-ledger-v1-completion.md new file mode 100644 index 00000000..d9c5ff2e --- /dev/null +++ b/docs/trust/tool-verification-ledger-v1-completion.md @@ -0,0 +1,26 @@ +# Tool verification ledger v1 — WS4 completion notes + +**Implementation:** [`salesflow-saas/backend/app/services/core_os/verification_ledger.py`](../../salesflow-saas/backend/app/services/core_os/verification_ledger.py) + +## Fields (v1) + +| Field | Meaning | +|-------|---------| +| `intended_action` | What the agent meant to do | +| `claimed_action` | What the agent said it would call | +| `actual_tool_call` | Concrete tool/route name | +| `parameters_hash` | Hash of parameters for audit | +| `side_effects` | Filled on resolve | +| `evidence_paths` | Artifacts proving execution | +| `verification_status` | `verified` / `partially_verified` / `unverified` / `contradicted` | +| `contradiction_flag` | Boolean; when true, resolve forces `contradicted` | + +## Next steps (ledger v2) + +- Persist proofs in PostgreSQL for multi-instance deployments. +- Expose read API for “contradiction dashboard” (WS4 / WS8). +- Correlate with OpenTelemetry `trace_id` (WS4/WS6). + +## OPA / OpenFGA / Vault / Keycloak + +Follow ADR-0001 spikes; policies must consume the same A/R/S metadata as [`approval-policy.md`](../governance/approval-policy.md). diff --git a/docs/workflows-inventory.md b/docs/workflows-inventory.md new file mode 100644 index 00000000..39f7a87b --- /dev/null +++ b/docs/workflows-inventory.md @@ -0,0 +1,28 @@ +# Workflow inventory — Completion Program WS3 + +**Purpose:** Classify automation into **short-lived**, **medium-lived (queued)**, and **long-lived durable** to drive Temporal pilot scope per [`adr/0001-tier1-execution-policy-spikes.md`](adr/0001-tier1-execution-policy-spikes.md). + +## LangGraph flows (`salesflow-saas/backend/app/flows/`) + +| Module | Role | Durability notes | +|--------|------|------------------| +| `prospecting_durable_flow.py` | Prospecting pipeline | Checkpoint-friendly; validate persistence + idempotency keys on external steps | +| `self_improvement_flow.py` | Self-improvement loop | Async API integration; ensure no silent side effects without ledger | + +## Celery task families (`salesflow-saas/backend/app/workers/`) + +| Area | Files (examples) | Typical duration | +|------|------------------|------------------| +| Sequences | `sequence_tasks.py` | Minutes | +| Agents | `agent_tasks.py` | Minutes | +| Notifications | `notification_tasks.py` | Minutes | +| Affiliates | `affiliate_tasks.py` | Minutes–hours | +| Follow-up | `follow_up_tasks.py` | Variable | + +## Migration rule (draft) + +- **Short:** keep Celery / inline async. +- **Medium:** Celery with explicit idempotency + DLQ. +- **Long / multi-system / compensation:** candidate for **Temporal** after ADR-0001 pilot exit criteria. + +See [`temporal-pilot-scope.md`](temporal-pilot-scope.md). diff --git a/docs/ws5-connector-events-metrics.md b/docs/ws5-connector-events-metrics.md new file mode 100644 index 00000000..17fd89d7 --- /dev/null +++ b/docs/ws5-connector-events-metrics.md @@ -0,0 +1,21 @@ +# WS5 — Connector facade, events, semantic metrics (completion) + +## Connector facade standard (required for new integrations) + +Each connector MUST document: **contract**, **version**, **retry**, **timeout**, **idempotency key**, **approval policy hook**, **audit field mapping**, **observability hooks**, **rollback notes**. +Pattern reference: [`governance/connectors-and-data-plane.md`](governance/connectors-and-data-plane.md). + +**Rule:** No new agent path may call a vendor SDK directly for a production integration; add a facade module under `salesflow-saas/backend/app/services/integrations/` (or agreed package) with tests. + +## Event envelope and schema registry + +- Envelope fields: see [`governance/events-and-schema.md`](governance/events-and-schema.md). +- Registry: start with versioned JSON Schema under `docs/schemas/events/` (create per event family) before adopting a hosted registry. + +## Semantic metrics dictionary + +Authoritative definitions: [`semantic-metrics-dictionary.md`](semantic-metrics-dictionary.md). Application code must import metric keys from a single module once introduced (WS5 follow-up PR). + +## Lineage / catalog + +Single choice documented in [`lineage-catalog-choice.md`](lineage-catalog-choice.md) until an ADR changes it. diff --git a/salesflow-saas/backend/app/services/core_os/__init__.py b/salesflow-saas/backend/app/services/core_os/__init__.py index effa70bb..3a13dff1 100644 --- a/salesflow-saas/backend/app/services/core_os/__init__.py +++ b/salesflow-saas/backend/app/services/core_os/__init__.py @@ -1,3 +1,21 @@ """ Core OS Services - Dealix Sovereign Growth OS """ + +from app.services.core_os.decision_plane_contracts import ( + ApprovalPacket, + EvidencePack, + ExecutionIntent, + assemble_decision_bundle, + new_evidence_pack_id, +) +from app.services.core_os.verification_ledger import VerificationLedger + +__all__ = [ + "ApprovalPacket", + "EvidencePack", + "ExecutionIntent", + "VerificationLedger", + "assemble_decision_bundle", + "new_evidence_pack_id", +] diff --git a/salesflow-saas/backend/app/services/core_os/decision_plane_contracts.py b/salesflow-saas/backend/app/services/core_os/decision_plane_contracts.py new file mode 100644 index 00000000..354246a9 --- /dev/null +++ b/salesflow-saas/backend/app/services/core_os/decision_plane_contracts.py @@ -0,0 +1,85 @@ +""" +Decision plane bundle — structured artifacts for Completion Program WS2. + +Compose alongside DecisionMemo for Class B / governed paths: + evidence_pack, approval_packet, execution_intent (risk_register lives on DecisionMemo). +""" +from __future__ import annotations + +from datetime import datetime, timezone +from typing import Any, Dict, List, Literal, Optional + +from pydantic import BaseModel, Field + +ApprovalClass = Literal["A0", "A1", "A2", "A3"] +ReversibilityClass = Literal["R0", "R1", "R2", "R3"] +SensitivityClass = Literal["S0", "S1", "S2", "S3"] +ActorType = Literal["human", "observer_agent", "recommender_agent", "executor_system", "automated_workflow"] + + +class EvidencePack(BaseModel): + """Structured evidence for a decision or agent run.""" + + pack_id: str = Field(..., description="Stable id, e.g. ep_") + sources: List[str] = Field(default_factory=list, description="URLs, file paths, ticket ids") + assumptions: List[str] = Field(default_factory=list) + freshness_utc: str = Field( + default_factory=lambda: datetime.now(timezone.utc).isoformat(), + description="ISO-8601 UTC", + ) + compliance_notes: Optional[str] = None + artifact_refs: List[str] = Field(default_factory=list, description="Build ids, test logs, PR urls") + provenance_score: float = Field(default=0.0, ge=0.0, le=100.0) + tool_proof_ids: List[str] = Field(default_factory=list) + + +class ApprovalPacket(BaseModel): + """A/R/S + actor for routing and audit.""" + + approval_class: ApprovalClass + reversibility_class: ReversibilityClass + sensitivity_class: SensitivityClass + actor_type: ActorType + approvers_required: List[str] = Field(default_factory=list) + policy_notes: Optional[str] = None + + +class ExecutionIntent(BaseModel): + """What the execution plane should do — not narration.""" + + workflow_key: str = Field(..., description="Logical workflow name, e.g. partner_approval_v1") + idempotency_key: str + requested_side_effect_class: Literal["none", "internal_write", "external_message", "external_commitment"] + correlation_id: Optional[str] = None + payload_summary: str = Field(default="", description="Human-readable one-liner; details in workflow state") + + +def new_evidence_pack_id(prefix: str = "ep") -> str: + from uuid import uuid4 + + return f"{prefix}_{uuid4().hex[:12]}" + + +def assemble_decision_bundle( + *, + evidence_pack: EvidencePack, + approval_packet: ApprovalPacket, + execution_intent: ExecutionIntent, + memo_json: Dict[str, Any], +) -> Dict[str, Any]: + """ + Single JSON-serializable bundle for APIs and logs. + + Keys: memo_json, evidence_pack_json, risk_register_json (from memo if present), + approval_packet_json, execution_intent_json + """ + risk_register = memo_json.get("risk_register") + if risk_register is None: + risk_register = memo_json.get("risk_register_json") + return { + "memo_json": memo_json, + "evidence_pack_json": evidence_pack.model_dump(mode="json"), + "risk_register_json": risk_register if isinstance(risk_register, list) else [], + "approval_packet_json": approval_packet.model_dump(mode="json"), + "execution_intent_json": execution_intent.model_dump(mode="json"), + } diff --git a/salesflow-saas/backend/app/services/core_os/verification_ledger.py b/salesflow-saas/backend/app/services/core_os/verification_ledger.py index c8a7c21e..f0e40988 100644 --- a/salesflow-saas/backend/app/services/core_os/verification_ledger.py +++ b/salesflow-saas/backend/app/services/core_os/verification_ledger.py @@ -42,29 +42,43 @@ class VerificationLedger: "timestamps": {"started": timestamp}, "side_effects": [], "evidence_paths": [], - "verification_status": "unverified" + "verification_status": "unverified", + "contradiction_flag": False, } self._write_proof(run_id, proof) return run_id - def resolve_proof(self, run_id: str, side_effects: List[str], - evidence_paths: List[str], status: str): + def resolve_proof( + self, + run_id: str, + side_effects: List[str], + evidence_paths: List[str], + status: str, + *, + contradiction_flag: bool = False, + ): """ Updates the proof AFTER execution with actual side effects and sets status to verified. Status must be 'verified', 'partially_verified', 'unverified', or 'contradicted'. + If contradiction_flag is True, status is forced to 'contradicted' and the flag is stored. """ valid_statuses = ["verified", "partially_verified", "unverified", "contradicted"] if status not in valid_statuses: raise ValueError(f"Status must be one of {valid_statuses}") - + + final_status = "contradicted" if contradiction_flag else status + if final_status not in valid_statuses: + final_status = "contradicted" + proof = self._read_proof(run_id) if not proof: raise KeyError(f"Run ID {run_id} not found in ledger.") - + proof["side_effects"] = side_effects proof["evidence_paths"] = evidence_paths - proof["verification_status"] = status + proof["verification_status"] = final_status + proof["contradiction_flag"] = bool(contradiction_flag) proof["timestamps"]["resolved"] = datetime.utcnow().isoformat() self._write_proof(run_id, proof) diff --git a/salesflow-saas/backend/tests/test_decision_plane_contracts.py b/salesflow-saas/backend/tests/test_decision_plane_contracts.py new file mode 100644 index 00000000..41963723 --- /dev/null +++ b/salesflow-saas/backend/tests/test_decision_plane_contracts.py @@ -0,0 +1,63 @@ +"""Tests for decision plane bundle (Completion Program WS2).""" +from __future__ import annotations + +from app.services.core_os.decision_memo import DecisionMemo, AuditMetadata, FinancialImpact +from app.services.core_os.decision_plane_contracts import ( + ApprovalPacket, + EvidencePack, + ExecutionIntent, + assemble_decision_bundle, + new_evidence_pack_id, +) + + +def test_assemble_decision_bundle_keys(): + memo = DecisionMemo( + objective="Test objective", + decision_context="ctx", + inputs_used=["a"], + assumptions=["x"], + recommendation_ar="do thing", + alternatives_considered=["b"], + expected_financial_impact=FinancialImpact(), + risk_register=[], + confidence_score=80.0, + required_approvals=["CEO"], + next_best_action="approve", + rollback_plan="revert", + evidence_links=[], + audit_metadata=AuditMetadata(agent_id="test_agent", timestamp="2026-01-01T00:00:00Z"), + ) + ep = EvidencePack( + pack_id=new_evidence_pack_id(), + sources=["https://example.com/doc"], + provenance_score=70.0, + ) + ap = ApprovalPacket( + approval_class="A2", + reversibility_class="R2", + sensitivity_class="S2", + actor_type="recommender_agent", + approvers_required=["CFO"], + ) + ei = ExecutionIntent( + workflow_key="partner_approval_v1", + idempotency_key="idem-001", + requested_side_effect_class="internal_write", + correlation_id="corr-1", + ) + bundle = assemble_decision_bundle( + evidence_pack=ep, + approval_packet=ap, + execution_intent=ei, + memo_json=memo.to_json(), + ) + assert set(bundle.keys()) == { + "memo_json", + "evidence_pack_json", + "risk_register_json", + "approval_packet_json", + "execution_intent_json", + } + assert bundle["approval_packet_json"]["approval_class"] == "A2" + assert bundle["execution_intent_json"]["idempotency_key"] == "idem-001" diff --git a/salesflow-saas/backend/tests/test_verification_ledger_contradiction.py b/salesflow-saas/backend/tests/test_verification_ledger_contradiction.py new file mode 100644 index 00000000..06be54eb --- /dev/null +++ b/salesflow-saas/backend/tests/test_verification_ledger_contradiction.py @@ -0,0 +1,28 @@ +"""Verification ledger contradiction flag (Completion Program WS4).""" +from __future__ import annotations + +from pathlib import Path + +from app.services.core_os.verification_ledger import VerificationLedger + + +def test_resolve_proof_contradiction_flag_forces_status(tmp_path: Path): + ledger = VerificationLedger(ledger_path=str(tmp_path / "vl")) + run_id = ledger.create_proof( + agent_id="a1", + task_id="t1", + intended_action="send_email", + claimed_action="send_email", + current_tool_call="email.send", + parameters={"to": "x@y.com"}, + ) + ledger.resolve_proof( + run_id, + [], + [], + "verified", + contradiction_flag=True, + ) + proof = ledger._read_proof(run_id) + assert proof["verification_status"] == "contradicted" + assert proof["contradiction_flag"] is True diff --git a/salesflow-saas/docs/tier1-master-closure-checklist.md b/salesflow-saas/docs/tier1-master-closure-checklist.md index f9b0c498..d5b00f9a 100644 --- a/salesflow-saas/docs/tier1-master-closure-checklist.md +++ b/salesflow-saas/docs/tier1-master-closure-checklist.md @@ -23,7 +23,7 @@ ## Gate 3: Decision Plane | # | Item | Required Evidence | Status | |---|------|------------------|--------| -| 3.1 | 17 structured output schemas defined | `schemas/structured_outputs.py` | Done | +| 3.1 | 17 structured output schemas defined | [`backend/app/schemas/structured_outputs.py`](../backend/app/schemas/structured_outputs.py) | Done | | 3.2 | Provenance on every output (trace_id, confidence, freshness) | `Provenance` class | Done | | 3.3 | No free-text in approval/commitment paths | Schema enforcement | Pending wiring | | 3.4 | Schema adherence measured for critical outputs | Monitoring | Target | diff --git a/scripts/architecture_brief.py b/scripts/architecture_brief.py index d7ef436d..d570c78d 100644 --- a/scripts/architecture_brief.py +++ b/scripts/architecture_brief.py @@ -14,9 +14,12 @@ CONSTITUTION_PATHS = [ "docs/ai-operating-model.md", "docs/dealix-six-tracks.md", "docs/blueprint-master-architecture.md", + "docs/completion-program-workstreams.md", + "docs/architecture-register.md", "docs/execution-matrix-90d-tier1.md", "docs/enterprise-readiness.md", "docs/adr/0001-tier1-execution-policy-spikes.md", + "docs/adr/0002-execution-matrix-canonical-source.md", "docs/governance/README.md", "docs/governance/approval-policy.md", "docs/governance/planes-and-runtime.md",