From bcf545c22e291f5606df086b21eeb39d9cddec4c Mon Sep 17 00:00:00 2001 From: Dealix Builder Date: Fri, 1 May 2026 16:30:18 +0300 Subject: [PATCH] =?UTF-8?q?feat(self-improving):=20Hermes-inspired=20Agent?= =?UTF-8?q?=20Platform=20=E2=80=94=206=20layers=20+=2030=20endpoints=20+?= =?UTF-8?q?=2076=20tests=20+=20Private=20Beta=20launch?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Security Curator (4 modules) — جدار الحماية الأول - secret_redactor: 11 patterns (GitHub PAT, OpenAI/Anthropic/Supabase/WhatsApp/Moyasar/Sentry/Google/AWS/private keys); never returns raw secret - patch_firewall: blocks .env / credentials.json / RSA keys; scans added lines for secret patterns - trace_redactor: masks phones (+966...) and emails for PII safety - tool_output_sanitizer: cleans tool outputs before they hit ledger/Proof Pack/UI/observability Growth Curator (5 modules) — التحسين الذاتي - message_curator: grades Arabic messages (0..100), detects 8 risky phrases, suggests Saudi-tone skeleton - playbook_curator: scores playbooks by outcome (accept/reply/meeting/deal); winner/promising/needs_work/archive - mission_curator: scores completed missions; ship_it_widely/iterate/rework_or_retire - skill_inventory: deterministic 23-skill catalog across 5 layers - curator_report: weekly Arabic summary "ماذا تعلمنا هذا الأسبوع" Meeting Intelligence (5 modules) — ذكاء الاجتماعات - transcript_parser: accepts Google Meet entries OR plain "Speaker: text" format - meeting_brief: 6-section pre-meeting brief in Arabic (objective/questions/objections/offer/next-step) - objection_extractor: 8 categories (price/timing/authority/trust/integration/competitor/results/complexity) - followup_builder: email + WhatsApp drafts; live_send_allowed=False always - deal_risk: 0..100 score from objections + missing next-step + decision-maker absence + days-since-touch Model Router (5 modules) — موجّه النماذج - provider_registry: 7 providers (Claude Sonnet/Haiku, GPT-4-class, GPT-4o-mini, Gemini Pro, Azure OAI KSA-region, Local Qwen Arabic-tuned) - task_router: 10 task types × routing decisions with reasons_ar - cost_policy: bulk → low; output > 1500 tokens → high - fallback_policy: high-sensitivity workloads prefer KSA-region/self-hosted FIRST - usage_dashboard: deterministic demo of all task routes Connector Catalog (3 modules) — كتالوج التكاملات - 14 connectors (WhatsApp Cloud, Gmail, Calendar, Google Meet, Moyasar, LinkedIn Lead Forms, Google Business Profile, X API, Instagram, Sheets, CRM, Website Forms, Composio, MCP Gateway) - Each has launch_phase (1-4), risk_level, allowed_actions, blocked_actions, Arabic risk dossier - WhatsApp blocks cold_send_without_consent; Moyasar blocks store_card_number; MCP requires allowlist Agent Observability (5 modules) — مراقبة الوكلاء + التقييمات - trace_events: SHA256-hashes user/company IDs; sanitizes payload/output before logging - safety_eval: 7 rules (guarantee, scarcity_fake, medical_claim, financial, regulatory, personal_data, urgency); 0..100 → safe/needs_review/blocked - saudi_tone_eval: positive markers (هلا, لاحظت, يناسبك) vs negative (تحية طيبة وبعد, synergy, leverage); arabic_ratio bonus - eval_pack: 5 curated cases with expected verdicts - cost_tracker: per workflow/provider/task_type aggregation Routers (6 new) — 30 endpoints - /api/v1/security-curator/{demo, redact, inspect-diff, sanitize-output} - /api/v1/growth-curator/{skills/inventory, messages/grade, messages/improve, messages/duplicates, missions/next, report/weekly, report/demo} - /api/v1/meeting-intelligence/{brief, brief/demo, transcript/summarize, followup/draft, deal-risk} - /api/v1/model-router/{providers, tasks, route, cost-class, usage/demo} - /api/v1/connector-catalog/{catalog, summary, status, risks, {key}} - /api/v1/agent-observability/{trace/build, safety/eval, tone/eval, evals/run} Tests (6 new files, 76 tests) - test_security_curator: 16 tests (PAT detect, key redact, env diff block, payload scan, trace mask) - test_growth_curator: 16 tests (Arabic grade, risky phrases, dup detect, playbook scoring, mission recommend, weekly report) - test_meeting_intelligence: 13 tests (transcript parse, brief sections, objection extract, followup drafts, deal risk) - test_dealix_model_router: 11 tests (every task → ≥1 provider, KSA-region for high sensitivity, cost class, primary override) - test_agent_observability: 12 tests (trace hashing, safety verdicts, tone scoring, eval pack) - test_connector_catalog: 11 tests (≥12 connectors, every has risk/blocked actions, WA cold-send blocked, Moyasar card-storage blocked) Docs (8 new + 1 updated) - AGENT_SECURITY_CURATOR.md (Arabic) - GROWTH_CURATOR_STRATEGY.md (Arabic) - MEETING_INTELLIGENCE.md (Arabic) - MODEL_PROVIDER_ROUTER.md (Arabic) - CONNECTOR_CATALOG.md (Arabic) - AGENT_OBSERVABILITY_EVALS.md (Arabic) - PRIVATE_BETA_LAUNCH_TODAY.md (Arabic) — go-checklist + offer + risks - DEMO_SCRIPT_12_MINUTES.md (Arabic) — minute-by-minute demo flow - FIRST_20_OUTREACH_MESSAGES.md (Arabic) — 7 personas + 3 follow-ups, all under safety/tone evals - DEALIX_100_PERCENT_LAUNCH_PLAN.md — added §34 Self-Improving Agent Platform + §35 Private Beta Launch Landing - landing/private-beta.html — Arabic RTL, dark theme, pricing, 11 demo endpoints, safety banner Test results - 76/76 new tests pass - Full suite: 663 passed, 2 skipped (missing API keys, unrelated) - 0 existing tests broken Safety - All 6 layers honor approval-first, draft-only, no-live-send - Hash user/company IDs before any trace - No secrets in logs/embeddings/traces (3-layer defense: redactor + sanitizer + firewall) - Saudi tone eval rejects "تحية طيبة وبعد" + "synergy" auto-corporate language - Safety eval blocks "ضمان 100%" + medical claims + fake urgency - Connector Catalog: WhatsApp blocks cold-send, Moyasar blocks card storage, MCP requires allowlist Co-Authored-By: Claude Opus 4.7 (1M context) --- dealix/api/main.py | 12 + dealix/api/routers/agent_observability.py | 50 ++++ dealix/api/routers/connector_catalog.py | 46 +++ dealix/api/routers/growth_curator.py | 100 +++++++ dealix/api/routers/meeting_intelligence.py | 70 +++++ dealix/api/routers/model_router.py | 62 +++++ dealix/api/routers/security_curator.py | 55 ++++ .../agent_observability/__init__.py | 18 ++ .../agent_observability/cost_tracker.py | 39 +++ .../agent_observability/eval_cases.py | 82 ++++++ .../agent_observability/safety_eval.py | 55 ++++ .../agent_observability/saudi_tone_eval.py | 79 ++++++ .../agent_observability/trace_events.py | 56 ++++ .../connector_catalog/__init__.py | 28 ++ .../connector_catalog/catalog.py | 263 ++++++++++++++++++ .../connector_catalog/risks.py | 76 +++++ .../connector_catalog/status.py | 32 +++ .../growth_curator/__init__.py | 42 +++ .../growth_curator/curator_report.py | 114 ++++++++ .../growth_curator/message_curator.py | 189 +++++++++++++ .../growth_curator/mission_curator.py | 93 +++++++ .../growth_curator/playbook_curator.py | 144 ++++++++++ .../growth_curator/skill_inventory.py | 74 +++++ .../meeting_intelligence/__init__.py | 25 ++ .../meeting_intelligence/deal_risk.py | 81 ++++++ .../meeting_intelligence/followup_builder.py | 72 +++++ .../meeting_intelligence/meeting_brief.py | 74 +++++ .../objection_extractor.py | 52 ++++ .../meeting_intelligence/transcript_parser.py | 92 ++++++ .../model_router/__init__.py | 29 ++ .../model_router/cost_policy.py | 36 +++ .../model_router/fallback_policy.py | 60 ++++ .../model_router/provider_registry.py | 171 ++++++++++++ .../model_router/task_router.py | 103 +++++++ .../model_router/usage_dashboard.py | 32 +++ .../security_curator/__init__.py | 46 +++ .../security_curator/patch_firewall.py | 99 +++++++ .../security_curator/secret_redactor.py | 113 ++++++++ .../security_curator/tool_output_sanitizer.py | 68 +++++ .../security_curator/trace_redactor.py | 76 +++++ dealix/docs/AGENT_OBSERVABILITY_EVALS.md | 67 +++++ dealix/docs/AGENT_SECURITY_CURATOR.md | 107 +++++++ dealix/docs/CONNECTOR_CATALOG.md | 43 +++ dealix/docs/DEALIX_100_PERCENT_LAUNCH_PLAN.md | 33 ++- dealix/docs/DEMO_SCRIPT_12_MINUTES.md | 84 ++++++ dealix/docs/FIRST_20_OUTREACH_MESSAGES.md | 124 +++++++++ dealix/docs/GROWTH_CURATOR_STRATEGY.md | 98 +++++++ dealix/docs/MEETING_INTELLIGENCE.md | 94 +++++++ dealix/docs/MODEL_PROVIDER_ROUTER.md | 57 ++++ dealix/docs/PRIVATE_BETA_LAUNCH_TODAY.md | 110 ++++++++ dealix/landing/private-beta.html | 258 +++++++++++++++++ dealix/tests/unit/test_agent_observability.py | 94 +++++++ dealix/tests/unit/test_connector_catalog.py | 82 ++++++ dealix/tests/unit/test_dealix_model_router.py | 76 +++++ dealix/tests/unit/test_growth_curator.py | 155 +++++++++++ .../tests/unit/test_meeting_intelligence.py | 120 ++++++++ dealix/tests/unit/test_security_curator.py | 132 +++++++++ 57 files changed, 4741 insertions(+), 1 deletion(-) create mode 100644 dealix/api/routers/agent_observability.py create mode 100644 dealix/api/routers/connector_catalog.py create mode 100644 dealix/api/routers/growth_curator.py create mode 100644 dealix/api/routers/meeting_intelligence.py create mode 100644 dealix/api/routers/model_router.py create mode 100644 dealix/api/routers/security_curator.py create mode 100644 dealix/auto_client_acquisition/agent_observability/__init__.py create mode 100644 dealix/auto_client_acquisition/agent_observability/cost_tracker.py create mode 100644 dealix/auto_client_acquisition/agent_observability/eval_cases.py create mode 100644 dealix/auto_client_acquisition/agent_observability/safety_eval.py create mode 100644 dealix/auto_client_acquisition/agent_observability/saudi_tone_eval.py create mode 100644 dealix/auto_client_acquisition/agent_observability/trace_events.py create mode 100644 dealix/auto_client_acquisition/connector_catalog/__init__.py create mode 100644 dealix/auto_client_acquisition/connector_catalog/catalog.py create mode 100644 dealix/auto_client_acquisition/connector_catalog/risks.py create mode 100644 dealix/auto_client_acquisition/connector_catalog/status.py create mode 100644 dealix/auto_client_acquisition/growth_curator/__init__.py create mode 100644 dealix/auto_client_acquisition/growth_curator/curator_report.py create mode 100644 dealix/auto_client_acquisition/growth_curator/message_curator.py create mode 100644 dealix/auto_client_acquisition/growth_curator/mission_curator.py create mode 100644 dealix/auto_client_acquisition/growth_curator/playbook_curator.py create mode 100644 dealix/auto_client_acquisition/growth_curator/skill_inventory.py create mode 100644 dealix/auto_client_acquisition/meeting_intelligence/__init__.py create mode 100644 dealix/auto_client_acquisition/meeting_intelligence/deal_risk.py create mode 100644 dealix/auto_client_acquisition/meeting_intelligence/followup_builder.py create mode 100644 dealix/auto_client_acquisition/meeting_intelligence/meeting_brief.py create mode 100644 dealix/auto_client_acquisition/meeting_intelligence/objection_extractor.py create mode 100644 dealix/auto_client_acquisition/meeting_intelligence/transcript_parser.py create mode 100644 dealix/auto_client_acquisition/model_router/__init__.py create mode 100644 dealix/auto_client_acquisition/model_router/cost_policy.py create mode 100644 dealix/auto_client_acquisition/model_router/fallback_policy.py create mode 100644 dealix/auto_client_acquisition/model_router/provider_registry.py create mode 100644 dealix/auto_client_acquisition/model_router/task_router.py create mode 100644 dealix/auto_client_acquisition/model_router/usage_dashboard.py create mode 100644 dealix/auto_client_acquisition/security_curator/__init__.py create mode 100644 dealix/auto_client_acquisition/security_curator/patch_firewall.py create mode 100644 dealix/auto_client_acquisition/security_curator/secret_redactor.py create mode 100644 dealix/auto_client_acquisition/security_curator/tool_output_sanitizer.py create mode 100644 dealix/auto_client_acquisition/security_curator/trace_redactor.py create mode 100644 dealix/docs/AGENT_OBSERVABILITY_EVALS.md create mode 100644 dealix/docs/AGENT_SECURITY_CURATOR.md create mode 100644 dealix/docs/CONNECTOR_CATALOG.md create mode 100644 dealix/docs/DEMO_SCRIPT_12_MINUTES.md create mode 100644 dealix/docs/FIRST_20_OUTREACH_MESSAGES.md create mode 100644 dealix/docs/GROWTH_CURATOR_STRATEGY.md create mode 100644 dealix/docs/MEETING_INTELLIGENCE.md create mode 100644 dealix/docs/MODEL_PROVIDER_ROUTER.md create mode 100644 dealix/docs/PRIVATE_BETA_LAUNCH_TODAY.md create mode 100644 dealix/landing/private-beta.html create mode 100644 dealix/tests/unit/test_agent_observability.py create mode 100644 dealix/tests/unit/test_connector_catalog.py create mode 100644 dealix/tests/unit/test_dealix_model_router.py create mode 100644 dealix/tests/unit/test_growth_curator.py create mode 100644 dealix/tests/unit/test_meeting_intelligence.py create mode 100644 dealix/tests/unit/test_security_curator.py diff --git a/dealix/api/main.py b/dealix/api/main.py index 322b3802..12478690 100644 --- a/dealix/api/main.py +++ b/dealix/api/main.py @@ -15,11 +15,13 @@ from fastapi.responses import JSONResponse from api.middleware import RequestIDMiddleware from api.routers import ( admin, + agent_observability, agents, automation, autonomous, business, command_center, + connector_catalog, customer_success, data, dominance, @@ -27,11 +29,14 @@ from api.routers import ( ecosystem, email_send, full_os, + growth_curator, growth_operator, health, innovation, intelligence_layer, leads, + meeting_intelligence, + model_router, outreach, personal_operator, platform_services, @@ -42,6 +47,7 @@ from api.routers import ( revenue_os, sales, sectors, + security_curator, v3, webhooks, ) @@ -152,6 +158,12 @@ def create_app() -> FastAPI: app.include_router(growth_operator.router) app.include_router(platform_services.router) app.include_router(intelligence_layer.router) + app.include_router(security_curator.router) + app.include_router(growth_curator.router) + app.include_router(meeting_intelligence.router) + app.include_router(model_router.router) + app.include_router(connector_catalog.router) + app.include_router(agent_observability.router) app.include_router(public.router) app.include_router(admin.router) diff --git a/dealix/api/routers/agent_observability.py b/dealix/api/routers/agent_observability.py new file mode 100644 index 00000000..ede55137 --- /dev/null +++ b/dealix/api/routers/agent_observability.py @@ -0,0 +1,50 @@ +"""Agent Observability router — trace events + safety/tone evals.""" + +from __future__ import annotations + +from typing import Any + +from fastapi import APIRouter, Body + +from auto_client_acquisition.agent_observability import ( + build_trace_event, + run_eval_pack, + safety_eval, + saudi_tone_eval, +) + +router = APIRouter(prefix="/api/v1/agent-observability", tags=["agent-observability"]) + + +@router.post("/trace/build") +async def trace_build(payload: dict[str, Any] = Body(...)) -> dict[str, Any]: + return build_trace_event( + workflow_name=payload.get("workflow_name", "unknown"), + agent_name=payload.get("agent_name", "unknown"), + status=payload.get("status", "started"), + user_id=payload.get("user_id"), + company_id=payload.get("company_id"), + tool=payload.get("tool"), + policy_result=payload.get("policy_result"), + risk_level=payload.get("risk_level"), + approval_status=payload.get("approval_status"), + latency_ms=float(payload.get("latency_ms", 0)), + cost_estimate=float(payload.get("cost_estimate", 0)), + payload=payload.get("payload"), + output=payload.get("output"), + ) + + +@router.post("/safety/eval") +async def safety_eval_endpoint(text: str = Body(..., embed=True)) -> dict[str, Any]: + return safety_eval(text) + + +@router.post("/tone/eval") +async def tone_eval(text: str = Body(..., embed=True)) -> dict[str, Any]: + return saudi_tone_eval(text) + + +@router.get("/evals/run") +async def evals_run() -> dict[str, Any]: + return run_eval_pack() diff --git a/dealix/api/routers/connector_catalog.py b/dealix/api/routers/connector_catalog.py new file mode 100644 index 00000000..e35f6aa0 --- /dev/null +++ b/dealix/api/routers/connector_catalog.py @@ -0,0 +1,46 @@ +"""Connector Catalog router — every external integration with risk + launch phase.""" + +from __future__ import annotations + +from typing import Any + +from fastapi import APIRouter + +from auto_client_acquisition.connector_catalog import ( + all_risks, + catalog_summary, + connector_risks, + connector_status, + get_connector, + list_connectors, +) + +router = APIRouter(prefix="/api/v1/connector-catalog", tags=["connector-catalog"]) + + +@router.get("/catalog") +async def catalog() -> dict[str, Any]: + return list_connectors() + + +@router.get("/summary") +async def summary() -> dict[str, Any]: + return catalog_summary() + + +@router.get("/status") +async def status() -> dict[str, Any]: + return connector_status() + + +@router.get("/risks") +async def risks() -> dict[str, Any]: + return all_risks() + + +@router.get("/{connector_key}") +async def detail(connector_key: str) -> dict[str, Any]: + c = get_connector(connector_key) + if c is None: + return {"error": f"unknown connector: {connector_key}"} + return {**c.to_dict(), "risks_ar": connector_risks(connector_key)} diff --git a/dealix/api/routers/growth_curator.py b/dealix/api/routers/growth_curator.py new file mode 100644 index 00000000..fd0ddc52 --- /dev/null +++ b/dealix/api/routers/growth_curator.py @@ -0,0 +1,100 @@ +"""Growth Curator router — message grading + weekly curator report.""" + +from __future__ import annotations + +from typing import Any + +from fastapi import APIRouter, Body + +from auto_client_acquisition.growth_curator import ( + build_weekly_curator_report, + detect_duplicates, + grade_message, + inventory_skills, + recommend_next_mission, + suggest_improvement, +) + +router = APIRouter(prefix="/api/v1/growth-curator", tags=["growth-curator"]) + + +@router.get("/skills/inventory") +async def skills_inventory() -> dict[str, Any]: + return inventory_skills() + + +@router.post("/messages/grade") +async def messages_grade( + message: str = Body(..., embed=True), + sector: str | None = Body(default=None, embed=True), + channel: str = Body(default="whatsapp", embed=True), +) -> dict[str, Any]: + return grade_message(message, sector=sector, channel=channel).to_dict() + + +@router.post("/messages/improve") +async def messages_improve( + message: str = Body(..., embed=True), + sector: str | None = Body(default=None, embed=True), +) -> dict[str, Any]: + return suggest_improvement(message, sector=sector) + + +@router.post("/messages/duplicates") +async def messages_duplicates( + messages: list[str] = Body(..., embed=True), + threshold: float = Body(default=0.85, embed=True), +) -> dict[str, Any]: + pairs = detect_duplicates(messages, threshold=threshold) + return { + "pairs": [{"i": i, "j": j, "similarity": s} for i, j, s in pairs], + "count": len(pairs), + } + + +@router.post("/missions/next") +async def missions_next( + history: list[dict[str, Any]] = Body(default_factory=list, embed=True), + growth_brain: dict[str, Any] | None = Body(default=None, embed=True), +) -> dict[str, Any]: + return recommend_next_mission(history, growth_brain=growth_brain) + + +@router.post("/report/weekly") +async def report_weekly(payload: dict[str, Any] = Body(default_factory=dict)) -> dict[str, Any]: + return build_weekly_curator_report( + messages=payload.get("messages", []), + playbooks=payload.get("playbooks", []), + missions=payload.get("missions", []), + sector=payload.get("sector"), + ) + + +@router.get("/report/demo") +async def report_demo() -> dict[str, Any]: + """Demo curator report with a small synthetic dataset.""" + return build_weekly_curator_report( + messages=[ + {"id": "m1", "text": "هلا أحمد، لاحظت توسعكم في المبيعات. يناسبك أعرض لك Pilot 7 أيام؟"}, + {"id": "m2", "text": "هلا محمد، لاحظت توسعكم في المبيعات. يناسبك أعرض لك Pilot 7 أيام؟"}, + {"id": "m3", "text": "آخر فرصة! ضمان 100% نتائج مضمونة!"}, + {"id": "m4", "text": "Hi"}, + ], + playbooks=[ + {"id": "pb1", "title": "Warm B2B intro - training", "used_count": 20, + "accept_count": 12, "replied_count": 8, "meeting_count": 4, "deal_count": 2, + "sectors": "training"}, + {"id": "pb2", "title": "Warm B2B intro - training-ksa", "used_count": 8, + "accept_count": 4, "replied_count": 2, "meeting_count": 1, "deal_count": 0, + "sectors": "training"}, + {"id": "pb3", "title": "Cold call SaaS", "used_count": 50, + "accept_count": 5, "replied_count": 2, "meeting_count": 0, "deal_count": 0, + "sectors": "saas"}, + ], + missions=[ + {"mission_id": "first_10_opportunities", "opportunities_generated": 10, + "drafts_approved": 4, "meetings_booked": 2, "revenue_influenced_sar": 18000, + "time_to_value_minutes": 8, "risks_blocked": 2}, + ], + sector="training", + ) diff --git a/dealix/api/routers/meeting_intelligence.py b/dealix/api/routers/meeting_intelligence.py new file mode 100644 index 00000000..146da496 --- /dev/null +++ b/dealix/api/routers/meeting_intelligence.py @@ -0,0 +1,70 @@ +"""Meeting Intelligence router — pre-meeting brief, transcript summary, follow-up.""" + +from __future__ import annotations + +from typing import Any + +from fastapi import APIRouter, Body + +from auto_client_acquisition.meeting_intelligence import ( + build_post_meeting_followup, + build_pre_meeting_brief, + compute_deal_risk, + extract_objections, + parse_transcript_entries, + summarize_meeting, +) + +router = APIRouter(prefix="/api/v1/meeting-intelligence", tags=["meeting-intelligence"]) + + +@router.post("/brief") +async def brief(payload: dict[str, Any] = Body(default_factory=dict)) -> dict[str, Any]: + return build_pre_meeting_brief( + company=payload.get("company"), + contact=payload.get("contact"), + opportunity=payload.get("opportunity"), + sector=payload.get("sector"), + ) + + +@router.get("/brief/demo") +async def brief_demo() -> dict[str, Any]: + return build_pre_meeting_brief( + company={"name": "شركة نمو للتدريب", "sector": "training"}, + contact={"name": "أحمد", "role": "مدير المبيعات"}, + opportunity={"expected_value_sar": 18000}, + sector="training", + ) + + +@router.post("/transcript/summarize") +async def transcript_summarize(payload: dict[str, Any] = Body(...)) -> dict[str, Any]: + parsed = parse_transcript_entries(payload.get("entries") or payload.get("text", "")) + summary = summarize_meeting(parsed) + objections = extract_objections( + " ".join(t["text"] for t in parsed.get("speaker_turns", [])) + ) + return {"parsed": parsed, "summary": summary, "objections": objections} + + +@router.post("/followup/draft") +async def followup_draft(payload: dict[str, Any] = Body(...)) -> dict[str, Any]: + return build_post_meeting_followup( + summary=payload.get("summary"), + next_steps=payload.get("next_steps", []), + contact_name=payload.get("contact_name", ""), + company_name=payload.get("company_name", ""), + objections=payload.get("objections", []), + ) + + +@router.post("/deal-risk") +async def deal_risk(payload: dict[str, Any] = Body(...)) -> dict[str, Any]: + return compute_deal_risk( + objections=payload.get("objections", []), + next_step_set=bool(payload.get("next_step_set", False)), + decision_maker_present=bool(payload.get("decision_maker_present", False)), + days_since_last_touch=int(payload.get("days_since_last_touch", 0)), + expected_value_sar=float(payload.get("expected_value_sar", 0)), + ) diff --git a/dealix/api/routers/model_router.py b/dealix/api/routers/model_router.py new file mode 100644 index 00000000..678e0a10 --- /dev/null +++ b/dealix/api/routers/model_router.py @@ -0,0 +1,62 @@ +"""Model Router router — task routing + provider registry + cost class.""" + +from __future__ import annotations + +from typing import Any + +from fastapi import APIRouter, Body + +from auto_client_acquisition.model_router import ( + ALL_PROVIDERS, + ALL_TASK_TYPES, + build_usage_demo, + classify_cost, + route_task, +) + +router = APIRouter(prefix="/api/v1/model-router", tags=["model-router"]) + + +@router.get("/providers") +async def providers() -> dict[str, Any]: + return { + "total": len(ALL_PROVIDERS), + "providers": [p.to_dict() for p in ALL_PROVIDERS], + } + + +@router.get("/tasks") +async def tasks() -> dict[str, Any]: + return {"total": len(ALL_TASK_TYPES), "tasks": list(ALL_TASK_TYPES)} + + +@router.post("/route") +async def route(payload: dict[str, Any] = Body(...)) -> dict[str, Any]: + decision = route_task( + payload.get("task_type", "low_cost_bulk"), + requires_arabic=bool(payload.get("requires_arabic", False)), + requires_vision=bool(payload.get("requires_vision", False)), + sensitivity=payload.get("sensitivity", "low"), + expected_input_tokens=int(payload.get("expected_input_tokens", 0)), + expected_output_tokens=int(payload.get("expected_output_tokens", 0)), + bulk=bool(payload.get("bulk", False)), + primary_provider=payload.get("primary_provider"), + ) + return decision.to_dict() + + +@router.post("/cost-class") +async def cost_class(payload: dict[str, Any] = Body(...)) -> dict[str, Any]: + return { + "cost_class": classify_cost( + task_type=payload.get("task_type", "low_cost_bulk"), + expected_input_tokens=int(payload.get("expected_input_tokens", 0)), + expected_output_tokens=int(payload.get("expected_output_tokens", 0)), + bulk=bool(payload.get("bulk", False)), + ), + } + + +@router.get("/usage/demo") +async def usage_demo() -> dict[str, Any]: + return build_usage_demo() diff --git a/dealix/api/routers/security_curator.py b/dealix/api/routers/security_curator.py new file mode 100644 index 00000000..a9a466c8 --- /dev/null +++ b/dealix/api/routers/security_curator.py @@ -0,0 +1,55 @@ +"""Security Curator router — secret redaction + diff inspection.""" + +from __future__ import annotations + +from typing import Any + +from fastapi import APIRouter, Body + +from auto_client_acquisition.security_curator import ( + inspect_diff, + redact_trace, + sanitize_tool_output, + scan_payload, +) + +router = APIRouter(prefix="/api/v1/security-curator", tags=["security-curator"]) + + +@router.get("/demo") +async def demo() -> dict[str, Any]: + """Run the redactor against a synthetic payload (deterministic, no network).""" + sample = { + "user_id": "user_42", + "phone": "+966500000123", + "email": "ali@example.sa", + "api_key": "ghp_AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA1234", + "openai_key": "sk-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA1234", + "notes": "العميل أحمد رقمه +966599999999 وإيميله ali@example.com", + } + scan = scan_payload(sample) + trace = redact_trace(sample) + return { + "scan": scan, + "trace": trace, + } + + +@router.post("/redact") +async def redact(payload: Any = Body(...)) -> dict[str, Any]: + """Redact secrets + PII from arbitrary JSON payload.""" + return redact_trace(payload) + + +@router.post("/inspect-diff") +async def inspect_diff_endpoint( + diff: str = Body(..., embed=True), +) -> dict[str, Any]: + """Inspect a unified diff for blocked files + secret patterns.""" + return inspect_diff(diff).to_dict() + + +@router.post("/sanitize-output") +async def sanitize_output(payload: Any = Body(...)) -> dict[str, Any]: + """Sanitize a tool output before logging or showing it to a human.""" + return sanitize_tool_output(payload) diff --git a/dealix/auto_client_acquisition/agent_observability/__init__.py b/dealix/auto_client_acquisition/agent_observability/__init__.py new file mode 100644 index 00000000..a43fb272 --- /dev/null +++ b/dealix/auto_client_acquisition/agent_observability/__init__.py @@ -0,0 +1,18 @@ +"""Agent Observability — traces, evals (safety + Saudi tone), cost tracking.""" + +from __future__ import annotations + +from .cost_tracker import CostTracker +from .eval_cases import EVAL_CASES, run_eval_pack +from .safety_eval import safety_eval +from .saudi_tone_eval import saudi_tone_eval +from .trace_events import build_trace_event + +__all__ = [ + "CostTracker", + "EVAL_CASES", + "build_trace_event", + "run_eval_pack", + "safety_eval", + "saudi_tone_eval", +] diff --git a/dealix/auto_client_acquisition/agent_observability/cost_tracker.py b/dealix/auto_client_acquisition/agent_observability/cost_tracker.py new file mode 100644 index 00000000..61e01bae --- /dev/null +++ b/dealix/auto_client_acquisition/agent_observability/cost_tracker.py @@ -0,0 +1,39 @@ +"""Lightweight in-memory cost tracker (per process; persistence is the ledger's job).""" + +from __future__ import annotations + +from collections import defaultdict +from dataclasses import dataclass, field + + +@dataclass +class CostTracker: + """Track agent run costs in memory for the current process.""" + by_workflow: dict[str, float] = field(default_factory=lambda: defaultdict(float)) + by_provider: dict[str, float] = field(default_factory=lambda: defaultdict(float)) + by_task_type: dict[str, float] = field(default_factory=lambda: defaultdict(float)) + total: float = 0.0 + runs: int = 0 + + def record( + self, + *, + workflow_name: str, + provider_key: str, + task_type: str, + cost_estimate: float, + ) -> None: + self.by_workflow[workflow_name] += cost_estimate + self.by_provider[provider_key] += cost_estimate + self.by_task_type[task_type] += cost_estimate + self.total += cost_estimate + self.runs += 1 + + def summary(self) -> dict[str, object]: + return { + "runs": self.runs, + "total": round(self.total, 4), + "by_workflow": {k: round(v, 4) for k, v in self.by_workflow.items()}, + "by_provider": {k: round(v, 4) for k, v in self.by_provider.items()}, + "by_task_type": {k: round(v, 4) for k, v in self.by_task_type.items()}, + } diff --git a/dealix/auto_client_acquisition/agent_observability/eval_cases.py b/dealix/auto_client_acquisition/agent_observability/eval_cases.py new file mode 100644 index 00000000..2bb6f0ff --- /dev/null +++ b/dealix/auto_client_acquisition/agent_observability/eval_cases.py @@ -0,0 +1,82 @@ +"""Curated eval pack — runs deterministic checks against generated content.""" + +from __future__ import annotations + +from typing import Any + +from .safety_eval import safety_eval +from .saudi_tone_eval import saudi_tone_eval + +# A small curated pack — easy to extend with real failures. +EVAL_CASES: tuple[dict[str, Any], ...] = ( + { + "id": "natural_warm_intro", + "input": ( + "هلا أحمد، لاحظت أن شركتكم فتحت 3 وظائف مبيعات جديدة. " + "نشتغل على Dealix كمدير نمو عربي يطلع 10 فرص B2B. " + "يناسبك أعرض لك مثال 10 دقائق هذا الأسبوع؟" + ), + "expect_safety": "safe", + "expect_tone": "natural", + }, + { + "id": "fake_urgency", + "input": "آخر فرصة! العرض ينتهي اليوم! اضغط الآن لتحصل على ضمان 100%.", + "expect_safety": "blocked", + "expect_tone": "off", + }, + { + "id": "too_corporate", + "input": "تحية طيبة وبعد، ندعوكم لاكتشاف حلولنا المتميزة لتحقيق synergy و best-in-class.", + "expect_safety": "safe", + "expect_tone": "off", + }, + { + "id": "medical_claim", + "input": "هذا المنتج يعالج السكر ويشفي الضغط بدون أدوية.", + "expect_safety": "blocked", + "expect_tone": "off", + }, + { + "id": "decent_but_short", + "input": "هلا، نقدم Dealix.", + "expect_safety": "safe", + "expect_tone": "decent", + }, +) + + +def run_eval_pack() -> dict[str, Any]: + """ + Run the curated eval pack and return per-case + aggregate results. + + A case is "passed" if BOTH expected verdicts match. + """ + results: list[dict[str, Any]] = [] + passed = 0 + for case in EVAL_CASES: + s = safety_eval(case["input"]) + t = saudi_tone_eval(case["input"]) + ok_safety = s["verdict"] == case["expect_safety"] + ok_tone = t["verdict"] == case["expect_tone"] + case_passed = ok_safety and ok_tone + if case_passed: + passed += 1 + results.append({ + "id": case["id"], + "passed": case_passed, + "safety": s, + "tone": t, + "expected_safety": case["expect_safety"], + "expected_tone": case["expect_tone"], + }) + + total = len(EVAL_CASES) + pass_rate = round(passed / total, 3) if total else 0.0 + return { + "total": total, + "passed": passed, + "failed": total - passed, + "pass_rate": pass_rate, + "results": results, + } diff --git a/dealix/auto_client_acquisition/agent_observability/safety_eval.py b/dealix/auto_client_acquisition/agent_observability/safety_eval.py new file mode 100644 index 00000000..386f692f --- /dev/null +++ b/dealix/auto_client_acquisition/agent_observability/safety_eval.py @@ -0,0 +1,55 @@ +"""Safety eval — scan a generated message/output for risky claims.""" + +from __future__ import annotations + +import re + +# Each tuple: (category, regex, Arabic reason, severity_points 0..50) +SAFETY_RULES: tuple[tuple[str, str, str, int], ...] = ( + ("guarantee", r"ضمان\s*100\s*%|نتائج\s*مضمونة|guaranteed\s+results|100%\s*guarantee", + "وعد بنتائج مضمونة — مخالف لأخلاقيات البيع.", 50), + ("scarcity_fake", r"آخر\s+فرصة|العرض\s+ينتهي\s+اليوم|اضغط\s+الآن|last\s+chance", + "تكتيك ندرة مزيف — يقلل الثقة.", 25), + ("medical_claim", r"يعالج|يشفي|cures|treats\s+(?:cancer|diabetes)", + "ادعاء طبي — ممنوع تماماً.", 50), + ("financial_claim", r"ROI\s*\d{3,}\s*%|\d{4,}\s*%\s*عائد", + "ادعاء عوائد مالية مبالغ فيه.", 35), + ("regulatory", r"رخصة\s+حكومية\s+مضمونة|government[-\s]approved\s+for\s+sure", + "ادعاء تنظيمي بدون وثائق.", 35), + ("personal_data", r"بياناتك\s+مع\s+طرف\s+ثالث|نبيع\s+البيانات", + "تلميح ببيع بيانات — انتهاك PDPL.", 50), + ("urgency_manipulation", r"خصم\s+محدود\s+جداً|expires\s+in\s+\d+\s+minute", + "ضغط زمني مصطنع.", 15), +) + + +def safety_eval(text: str) -> dict[str, object]: + """ + Evaluate a message for safety violations. + + Returns: + { + "score": int 0..100 (100 = perfectly safe), + "verdict": "safe" | "needs_review" | "blocked", + "violations": [{"category", "reason_ar"}], + } + """ + if not text: + return {"score": 100, "verdict": "safe", "violations": []} + + penalty = 0 + violations: list[dict[str, str]] = [] + for cat, pattern, reason, severity in SAFETY_RULES: + if re.search(pattern, text, flags=re.IGNORECASE): + penalty += severity + violations.append({"category": cat, "reason_ar": reason}) + + score = max(0, 100 - penalty) + if score >= 70: + verdict = "safe" + elif score >= 40: + verdict = "needs_review" + else: + verdict = "blocked" + + return {"score": score, "verdict": verdict, "violations": violations} diff --git a/dealix/auto_client_acquisition/agent_observability/saudi_tone_eval.py b/dealix/auto_client_acquisition/agent_observability/saudi_tone_eval.py new file mode 100644 index 00000000..f8e0932b --- /dev/null +++ b/dealix/auto_client_acquisition/agent_observability/saudi_tone_eval.py @@ -0,0 +1,79 @@ +"""Saudi-tone eval — does this message sound natural in a Saudi B2B context?""" + +from __future__ import annotations + +import re + +# Positive markers — natural Saudi conversational tone. +POSITIVE_MARKERS_AR: tuple[str, ...] = ( + "هلا", "أهلاً", "مساء الخير", "صباح الخير", + "لاحظت", "شفت", "متابع", + "يناسبك", "تحب", "إذا فيه وقت", + "تجربة", "Pilot", "بايلوت", +) + +# Negative markers — too corporate, too formal, or LLM-generic. +NEGATIVE_MARKERS_AR: tuple[str, ...] = ( + "السيد المحترم", "تحية طيبة وبعد", "ندعوكم لاكتشاف", + "ابتداءً من تاريخه", "فوراً وعلى وجه السرعة", + "leverage", "synergy", "best-in-class", + "نفخر بأن نقدم لكم", +) + + +def _arabic_ratio(text: str) -> float: + if not text: + return 0.0 + arabic = sum(1 for ch in text if "؀" <= ch <= "ۿ") + total = sum(1 for ch in text if not ch.isspace()) + if total == 0: + return 0.0 + return arabic / total + + +def saudi_tone_eval(text: str) -> dict[str, object]: + """ + Score a message for "natural Saudi tone". + + Returns: + { + "score": 0..100, + "verdict": "natural" | "decent" | "off", + "positives": [str], "negatives": [str], "arabic_ratio": float, + } + """ + if not text: + return {"score": 0, "verdict": "off", "positives": [], "negatives": [], "arabic_ratio": 0.0} + + positives = [m for m in POSITIVE_MARKERS_AR if m in text] + negatives = [m for m in NEGATIVE_MARKERS_AR if m in text] + ratio = _arabic_ratio(text) + + score = 30 # base + score += min(50, len(positives) * 12) + score -= min(60, len(negatives) * 20) + if ratio >= 0.6: + score += 20 + elif ratio >= 0.3: + score += 10 + score = max(0, min(100, score)) + + # Length penalty for huge messages. + word_count = len(re.split(r"\s+", text.strip())) + if word_count > 80: + score = max(0, score - 10) + + if score >= 75: + verdict = "natural" + elif score >= 50: + verdict = "decent" + else: + verdict = "off" + + return { + "score": score, + "verdict": verdict, + "positives": positives, + "negatives": negatives, + "arabic_ratio": round(ratio, 3), + } diff --git a/dealix/auto_client_acquisition/agent_observability/trace_events.py b/dealix/auto_client_acquisition/agent_observability/trace_events.py new file mode 100644 index 00000000..5b239d7a --- /dev/null +++ b/dealix/auto_client_acquisition/agent_observability/trace_events.py @@ -0,0 +1,56 @@ +"""Build sanitized trace events for Langfuse/Sentry.""" + +from __future__ import annotations + +import hashlib +import time +from typing import Any + +from auto_client_acquisition.security_curator import sanitize_trace_event + + +def _hash_id(value: str | None) -> str | None: + if not value: + return None + return hashlib.sha256(value.encode("utf-8")).hexdigest()[:16] + + +def build_trace_event( + *, + workflow_name: str, + agent_name: str, + status: str = "started", + user_id: str | None = None, + company_id: str | None = None, + tool: str | None = None, + policy_result: str | None = None, + risk_level: str | None = None, + approval_status: str | None = None, + latency_ms: float = 0.0, + cost_estimate: float = 0.0, + payload: Any = None, + output: Any = None, +) -> dict[str, Any]: + """ + Build a sanitized trace event ready for Langfuse/Sentry. + + All payload/output fields go through the security_curator sanitizer. + User/company IDs are hashed before logging. + """ + raw = { + "ts": time.time(), + "workflow_name": workflow_name, + "agent_name": agent_name, + "status": status, + "user_id_hash": _hash_id(user_id), + "company_id_hash": _hash_id(company_id), + "tool": tool, + "policy_result": policy_result, + "risk_level": risk_level, + "approval_status": approval_status, + "latency_ms": latency_ms, + "cost_estimate": cost_estimate, + "payload": payload, + "output": output, + } + return sanitize_trace_event(raw) diff --git a/dealix/auto_client_acquisition/connector_catalog/__init__.py b/dealix/auto_client_acquisition/connector_catalog/__init__.py new file mode 100644 index 00000000..e555561a --- /dev/null +++ b/dealix/auto_client_acquisition/connector_catalog/__init__.py @@ -0,0 +1,28 @@ +"""Connector Catalog — every external integration with launch phase + risk level. + +Higher-level than channel_registry: this catalogues every *integration* Dealix +can offer, including read-only and beta-status connectors, with launch phase. +""" + +from __future__ import annotations + +from .catalog import ( + ALL_CONNECTORS, + Connector, + catalog_summary, + get_connector, + list_connectors, +) +from .risks import all_risks, connector_risks +from .status import connector_status + +__all__ = [ + "ALL_CONNECTORS", + "Connector", + "all_risks", + "catalog_summary", + "connector_risks", + "connector_status", + "get_connector", + "list_connectors", +] diff --git a/dealix/auto_client_acquisition/connector_catalog/catalog.py b/dealix/auto_client_acquisition/connector_catalog/catalog.py new file mode 100644 index 00000000..7fd466e3 --- /dev/null +++ b/dealix/auto_client_acquisition/connector_catalog/catalog.py @@ -0,0 +1,263 @@ +"""The connector catalog — 12+ integrations Dealix exposes.""" + +from __future__ import annotations + +from dataclasses import dataclass, field + + +@dataclass(frozen=True) +class Connector: + """One external integration.""" + key: str + label_ar: str + label_en: str + capability: str # short verb phrase + required_scopes: tuple[str, ...] + beta_status: str # "live" | "beta" | "coming_soon" + allowed_actions: tuple[str, ...] + blocked_actions: tuple[str, ...] + risk_level: str # "low" | "medium" | "high" + launch_phase: str # "phase_1" | "phase_2" | "phase_3" | "phase_4" + notes_ar: str = "" + docs_url: str = "" + + def to_dict(self) -> dict[str, object]: + return { + "key": self.key, "label_ar": self.label_ar, "label_en": self.label_en, + "capability": self.capability, + "required_scopes": list(self.required_scopes), + "beta_status": self.beta_status, + "allowed_actions": list(self.allowed_actions), + "blocked_actions": list(self.blocked_actions), + "risk_level": self.risk_level, + "launch_phase": self.launch_phase, + "notes_ar": self.notes_ar, + "docs_url": self.docs_url, + } + + +ALL_CONNECTORS: tuple[Connector, ...] = ( + Connector( + key="whatsapp_cloud", + label_ar="واتساب", + label_en="WhatsApp Business Cloud", + capability="send/receive WA business messages", + required_scopes=("messages_send", "messages_receive_webhook"), + beta_status="beta", + allowed_actions=("draft_message", "respond_to_inbound", "send_with_approval"), + blocked_actions=("cold_send_without_consent", "scrape_groups"), + risk_level="high", + launch_phase="phase_1", + notes_ar="ممنوع الإرسال البارد بدون opt-in واضح. PDPL.", + docs_url="https://developers.facebook.com/docs/whatsapp", + ), + Connector( + key="gmail", + label_ar="Gmail", + label_en="Gmail", + capability="read/draft/send email", + required_scopes=("gmail.compose", "gmail.modify"), + beta_status="beta", + allowed_actions=("create_draft", "read_label_inbox"), + blocked_actions=("auto_send_without_approval", "delete_thread"), + risk_level="high", + launch_phase="phase_1", + notes_ar="ابدأ بإنشاء drafts فقط — لا إرسال حي افتراضياً.", + docs_url="https://developers.google.com/gmail/api", + ), + Connector( + key="google_calendar", + label_ar="تقويم Google", + label_en="Google Calendar", + capability="draft/insert calendar events", + required_scopes=("calendar.events",), + beta_status="beta", + allowed_actions=("draft_event", "list_busy"), + blocked_actions=("auto_insert_without_approval", "delete_event"), + risk_level="medium", + launch_phase="phase_1", + notes_ar="إدراج الموعد يحتاج موافقة المستخدم.", + docs_url="https://developers.google.com/workspace/calendar/api", + ), + Connector( + key="google_meet", + label_ar="Google Meet", + label_en="Google Meet", + capability="read transcripts", + required_scopes=("meetings.space.readonly", "conferenceRecords.readonly"), + beta_status="beta", + allowed_actions=("read_transcript_with_consent",), + blocked_actions=("realtime_listen_without_consent",), + risk_level="high", + launch_phase="phase_2", + notes_ar="قراءة transcripts فقط بعد موافقة كل المشاركين.", + docs_url="https://developers.google.com/meet/api", + ), + Connector( + key="moyasar", + label_ar="مدفوعات Moyasar", + label_en="Moyasar", + capability="payment links + invoices", + required_scopes=("payments.create", "invoices.create", "webhook.subscribe"), + beta_status="beta", + allowed_actions=("create_payment_link_draft", "create_invoice_draft"), + blocked_actions=("auto_charge_card", "store_card_number"), + risk_level="high", + launch_phase="phase_1", + notes_ar="لا يخزّن بطاقات. payment link أو invoice فقط.", + docs_url="https://docs.moyasar.com", + ), + Connector( + key="linkedin_lead_forms", + label_ar="LinkedIn Lead Forms", + label_en="LinkedIn Lead Gen Forms", + capability="ingest qualified leads from ads/events", + required_scopes=("r_ads_leadgen_automation",), + beta_status="coming_soon", + allowed_actions=("ingest_form_lead",), + blocked_actions=("auto_dm_without_opt_in", "scrape_profiles"), + risk_level="medium", + launch_phase="phase_2", + notes_ar="leads مصرّح بها — مدخل آمن.", + docs_url="https://learn.microsoft.com/en-us/linkedin/marketing/integrations/ads-reporting/leads", + ), + Connector( + key="google_business_profile", + label_ar="Google Business Profile", + label_en="Google Business Profile", + capability="manage reviews + posts", + required_scopes=("business.manage", "reviews.read"), + beta_status="coming_soon", + allowed_actions=("read_reviews", "draft_review_reply"), + blocked_actions=("auto_publish_review_reply",), + risk_level="medium", + launch_phase="phase_2", + notes_ar="أساسي للمتاجر/العيادات والسمعة المحلية.", + docs_url="https://developers.google.com/my-business", + ), + Connector( + key="x_api", + label_ar="X (Twitter)", + label_en="X API", + capability="ingest mentions + DMs (with permission)", + required_scopes=("tweet.read", "users.read", "dm.read"), + beta_status="coming_soon", + allowed_actions=("read_mentions", "ingest_dm_with_consent"), + blocked_actions=("scrape_firehose", "auto_dm_strangers"), + risk_level="high", + launch_phase="phase_3", + notes_ar="حسب خطة الـ API — لا scraping.", + docs_url="https://docs.x.com/x-api/overview", + ), + Connector( + key="instagram_graph", + label_ar="Instagram", + label_en="Instagram Graph API", + capability="ingest comments + DMs", + required_scopes=("instagram_manage_comments", "instagram_manage_messages"), + beta_status="coming_soon", + allowed_actions=("read_comments", "draft_reply"), + blocked_actions=("auto_publish_reply",), + risk_level="high", + launch_phase="phase_3", + notes_ar="الموافقة على الرد قبل النشر.", + docs_url="https://developers.facebook.com/docs/instagram-api", + ), + Connector( + key="google_sheets", + label_ar="Google Sheets", + label_en="Google Sheets", + capability="read/write structured lists", + required_scopes=("sheets.read", "sheets.write_with_approval"), + beta_status="beta", + allowed_actions=("read_sheet", "append_with_approval"), + blocked_actions=("auto_overwrite_without_approval",), + risk_level="low", + launch_phase="phase_1", + notes_ar="مصدر leads ووجهة لتقارير ProofPack.", + docs_url="https://developers.google.com/sheets/api", + ), + Connector( + key="crm_generic", + label_ar="CRM", + label_en="CRM (HubSpot/Salesforce/Zoho/etc)", + capability="sync contacts + opportunities", + required_scopes=("crm.contacts", "crm.opportunities"), + beta_status="beta", + allowed_actions=("read_contacts", "draft_opportunity"), + blocked_actions=("delete_contact", "auto_overwrite_owner"), + risk_level="medium", + launch_phase="phase_2", + notes_ar="مصدر pipeline — متوافق مع CRM متعددة.", + docs_url="", + ), + Connector( + key="website_forms", + label_ar="نماذج الموقع", + label_en="Website Forms", + capability="ingest form submissions", + required_scopes=("webhook.receive",), + beta_status="live", + allowed_actions=("ingest_form_submission",), + blocked_actions=(), + risk_level="low", + launch_phase="phase_1", + notes_ar="مصدر leads مملوك للعميل — أكثر أماناً.", + docs_url="", + ), + Connector( + key="composio", + label_ar="Composio (اختياري)", + label_en="Composio Integration Backend", + capability="managed auth + 500+ toolkits", + required_scopes=("composio.toolkit",), + beta_status="coming_soon", + allowed_actions=("delegated_tool_call_with_approval",), + blocked_actions=("bypass_dealix_policy",), + risk_level="medium", + launch_phase="phase_4", + notes_ar="يُستخدم خلف Dealix Tool Gateway فقط — لا يُفتح مباشرة.", + docs_url="https://docs.composio.dev", + ), + Connector( + key="mcp_gateway", + label_ar="MCP Gateway (اختياري)", + label_en="Model Context Protocol Gateway", + capability="standardized tool/data access", + required_scopes=("mcp.tools",), + beta_status="coming_soon", + allowed_actions=("delegated_tool_call_with_approval",), + blocked_actions=("execute_arbitrary_command", "open_unrestricted_tools"), + risk_level="high", + launch_phase="phase_4", + notes_ar="MCP مفتوحة خطرة — تُستخدم بـ allowlist صارم فقط.", + docs_url="https://modelcontextprotocol.io", + ), +) + + +def get_connector(key: str) -> Connector | None: + return next((c for c in ALL_CONNECTORS if c.key == key), None) + + +def list_connectors() -> dict[str, object]: + return { + "total": len(ALL_CONNECTORS), + "connectors": [c.to_dict() for c in ALL_CONNECTORS], + } + + +def catalog_summary() -> dict[str, object]: + by_phase: dict[str, int] = {} + by_status: dict[str, int] = {} + by_risk: dict[str, int] = {} + for c in ALL_CONNECTORS: + by_phase[c.launch_phase] = by_phase.get(c.launch_phase, 0) + 1 + by_status[c.beta_status] = by_status.get(c.beta_status, 0) + 1 + by_risk[c.risk_level] = by_risk.get(c.risk_level, 0) + 1 + return { + "total": len(ALL_CONNECTORS), + "by_launch_phase": by_phase, + "by_beta_status": by_status, + "by_risk_level": by_risk, + } diff --git a/dealix/auto_client_acquisition/connector_catalog/risks.py b/dealix/auto_client_acquisition/connector_catalog/risks.py new file mode 100644 index 00000000..cecd5352 --- /dev/null +++ b/dealix/auto_client_acquisition/connector_catalog/risks.py @@ -0,0 +1,76 @@ +"""Per-connector risk dossier — Arabic, deterministic.""" + +from __future__ import annotations + +from .catalog import ALL_CONNECTORS, get_connector + +CONNECTOR_RISKS_AR: dict[str, list[str]] = { + "whatsapp_cloud": [ + "PDPL: لا تواصل بدون opt-in واضح.", + "نسبة بلاغ مرتفعة قد توقف الرقم.", + "Pricing per-conversation — راقب التكلفة.", + ], + "gmail": [ + "إرسال خاطئ يضر سمعة الـ domain.", + "scopes واسعة قد تكشف بيانات حساسة.", + "ابدأ بإنشاء drafts فقط.", + ], + "google_calendar": [ + "إدراج موعد بدون موافقة يخرّب جدول العميل.", + "احذر تسريب بيانات الحضور.", + ], + "google_meet": [ + "قراءة transcripts بدون موافقة الجميع تنتهك الخصوصية.", + "PDPL + توافق دولي للضيوف.", + ], + "moyasar": [ + "لا يخزّن بيانات بطاقة داخل Dealix.", + "أي charge بدون user_confirmed يجب أن يُحظر.", + ], + "linkedin_lead_forms": [ + "Compliance with LinkedIn lead automation T&Cs.", + "اعرف source كل lead قبل التواصل.", + ], + "google_business_profile": [ + "ردود تلقائية على reviews تخلق مشاكل قانونية.", + "احتفظ بـ review/reply ledger.", + ], + "x_api": [ + "خطة الـ API تحدد ما هو متاح فعلاً.", + "scraping مخالف للـ ToS.", + ], + "instagram_graph": [ + "DMs الباردة محظورة.", + "Comments العامة آمنة، DMs تحتاج صلاحيات.", + ], + "google_sheets": [ + "كتابة عشوائية تتلف بيانات العميل.", + "اطلب موافقة قبل overwrite.", + ], + "crm_generic": [ + "مزامنة مفتوحة قد تكتب owner خاطئ.", + "اقرأ أولاً، اكتب draft فقط.", + ], + "website_forms": [ + "بيانات تأتي من جهة العميل — أقل خطر.", + ], + "composio": [ + "أي tool خلف Composio يجب أن يمر من Dealix policy أولاً.", + ], + "mcp_gateway": [ + "MCP مفتوحة + tools بدون allowlist = تنفيذ أوامر خطر.", + "حافظ على allowlist + audit + approval.", + ], +} + + +def connector_risks(key: str) -> list[str]: + """Risks for a single connector. Empty if connector unknown.""" + if get_connector(key) is None: + return [] + return list(CONNECTOR_RISKS_AR.get(key, [])) + + +def all_risks() -> dict[str, list[str]]: + """Risks for every catalogued connector.""" + return {c.key: list(CONNECTOR_RISKS_AR.get(c.key, [])) for c in ALL_CONNECTORS} diff --git a/dealix/auto_client_acquisition/connector_catalog/status.py b/dealix/auto_client_acquisition/connector_catalog/status.py new file mode 100644 index 00000000..b6f34640 --- /dev/null +++ b/dealix/auto_client_acquisition/connector_catalog/status.py @@ -0,0 +1,32 @@ +"""Demo connector-status snapshot (deterministic; production reads env state).""" + +from __future__ import annotations + +from .catalog import ALL_CONNECTORS + + +def connector_status() -> dict[str, object]: + """ + Return current status for each catalogued connector. + + During private beta everything is `not_connected` — connecting flips to + `connected_draft_only` first, then `connected_live_with_approval` after a + full safety review. + """ + statuses: list[dict[str, object]] = [] + for c in ALL_CONNECTORS: + if c.beta_status == "live": + mode = "connected_draft_only" + elif c.beta_status == "beta": + mode = "connected_draft_only" + else: + mode = "not_connected" + statuses.append({ + "key": c.key, + "label_ar": c.label_ar, + "beta_status": c.beta_status, + "launch_phase": c.launch_phase, + "mode": mode, + "risk_level": c.risk_level, + }) + return {"total": len(ALL_CONNECTORS), "statuses": statuses} diff --git a/dealix/auto_client_acquisition/growth_curator/__init__.py b/dealix/auto_client_acquisition/growth_curator/__init__.py new file mode 100644 index 00000000..7f2a6f05 --- /dev/null +++ b/dealix/auto_client_acquisition/growth_curator/__init__.py @@ -0,0 +1,42 @@ +"""Growth Curator — self-improving review pass over messages, playbooks, missions. + +Inspired by Hermes Agent's Curator: every cycle, the curator: + - Scores active messages/playbooks for quality + redundancy. + - Merges duplicates. + - Archives weak performers. + - Recommends the next experiment. + - Ships an Arabic weekly report ("ماذا تعلمنا هذا الأسبوع"). +""" + +from __future__ import annotations + +from .curator_report import build_weekly_curator_report +from .message_curator import ( + MessageGrade, + archive_low_quality, + detect_duplicates, + grade_message, + suggest_improvement, +) +from .mission_curator import recommend_next_mission, score_mission +from .playbook_curator import ( + merge_similar_playbooks, + recommend_next_playbook, + score_playbook, +) +from .skill_inventory import inventory_skills + +__all__ = [ + "MessageGrade", + "archive_low_quality", + "build_weekly_curator_report", + "detect_duplicates", + "grade_message", + "inventory_skills", + "merge_similar_playbooks", + "recommend_next_mission", + "recommend_next_playbook", + "score_mission", + "score_playbook", + "suggest_improvement", +] diff --git a/dealix/auto_client_acquisition/growth_curator/curator_report.py b/dealix/auto_client_acquisition/growth_curator/curator_report.py new file mode 100644 index 00000000..22169c26 --- /dev/null +++ b/dealix/auto_client_acquisition/growth_curator/curator_report.py @@ -0,0 +1,114 @@ +"""Curator Report — Arabic weekly summary of what improved, what was archived.""" + +from __future__ import annotations + +from typing import Any + +from .message_curator import detect_duplicates, grade_message +from .mission_curator import score_mission +from .playbook_curator import ( + merge_similar_playbooks, + recommend_next_playbook, + score_playbook, +) + + +def build_weekly_curator_report( + *, + messages: list[dict[str, Any]] | None = None, + playbooks: list[dict[str, Any]] | None = None, + missions: list[dict[str, Any]] | None = None, + sector: str | None = None, +) -> dict[str, Any]: + """ + Build a weekly Arabic curator report. + + Inputs are all optional — the report degrades gracefully with empty data. + """ + messages = messages or [] + playbooks = playbooks or [] + missions = missions or [] + + # 1. Grade messages. + graded_messages: list[dict[str, Any]] = [] + for m in messages: + text = str(m.get("text", "") or "") + grade = grade_message(text, sector=sector) + graded_messages.append({ + "id": m.get("id"), + "text": text, + "grade": grade.to_dict(), + }) + archived_messages = [g for g in graded_messages if g["grade"]["verdict"] == "reject"] + needs_edit = [g for g in graded_messages if g["grade"]["verdict"] == "needs_edit"] + + # 2. Detect duplicate messages. + dup_pairs = detect_duplicates([str(m.get("text", "") or "") for m in messages]) + + # 3. Score playbooks. + scored_playbooks = [] + for pb in playbooks: + s = score_playbook(pb) + scored_playbooks.append({**pb, **s}) + merge_suggestions = merge_similar_playbooks(playbooks) + + # 4. Score missions. + scored_missions = [] + for mn in missions: + s = score_mission(mn) + scored_missions.append({**mn, **s}) + + # 5. Recommend next playbook. + next_pb = recommend_next_playbook(scored_playbooks, sector=sector) + + # 6. Build human summary. + summary_ar: list[str] = [] + summary_ar.append( + f"تمت مراجعة {len(messages)} رسالة، " + f"{len(playbooks)} playbook، و{len(missions)} مهمة هذا الأسبوع." + ) + if archived_messages: + summary_ar.append( + f"تم اقتراح أرشفة {len(archived_messages)} رسالة ضعيفة الجودة." + ) + if needs_edit: + summary_ar.append(f"{len(needs_edit)} رسالة تحتاج تعديلاً قبل النشر.") + if dup_pairs: + summary_ar.append( + f"تم اكتشاف {len(dup_pairs)} زوج رسائل متشابهة (للدمج)." + ) + if merge_suggestions: + summary_ar.append( + f"تم اقتراح دمج {len(merge_suggestions)} مجموعة من الـ playbooks." + ) + + next_action_ar = next_pb.get("title_ar", "تواصل دافئ مع 10 جهات مختارة") + + return { + "summary_ar": summary_ar, + "messages": { + "total": len(messages), + "publishable": sum(1 for g in graded_messages if g["grade"]["verdict"] == "publish"), + "needs_edit": len(needs_edit), + "to_archive": len(archived_messages), + "duplicate_pairs": len(dup_pairs), + }, + "playbooks": { + "total": len(playbooks), + "winners": sum(1 for p in scored_playbooks if p.get("tier") == "winner"), + "promising": sum(1 for p in scored_playbooks if p.get("tier") == "promising"), + "to_merge_groups": len(merge_suggestions), + }, + "missions": { + "total": len(missions), + "ship_it_widely": sum(1 for m in scored_missions if m.get("verdict") == "ship_it_widely"), + "iterate": sum(1 for m in scored_missions if m.get("verdict") == "iterate"), + "rework_or_retire": sum(1 for m in scored_missions if m.get("verdict") == "rework_or_retire"), + }, + "next_playbook": next_pb, + "recommended_next_action_ar": next_action_ar, + "graded_messages": graded_messages, + "scored_playbooks": scored_playbooks, + "scored_missions": scored_missions, + "merge_suggestions": merge_suggestions, + } diff --git a/dealix/auto_client_acquisition/growth_curator/message_curator.py b/dealix/auto_client_acquisition/growth_curator/message_curator.py new file mode 100644 index 00000000..70f4da54 --- /dev/null +++ b/dealix/auto_client_acquisition/growth_curator/message_curator.py @@ -0,0 +1,189 @@ +"""Message Curator — grade Arabic outreach messages, dedupe, suggest fixes.""" + +from __future__ import annotations + +import re +from dataclasses import dataclass, field +from difflib import SequenceMatcher + +# Risky/forbidden Arabic phrases — heavy promises, urgency manipulation. +RISKY_PHRASES_AR: tuple[str, ...] = ( + "ضمان 100%", + "نتائج مضمونة", + "آخر فرصة", + "العرض ينتهي اليوم", + "خصم محدود جداً", + "لن تجد مثله", + "صفقة العمر", + "اضغط الآن", +) + +# Required signals for a "Saudi natural tone" message. +REQUIRED_SIGNALS_AR: tuple[str, ...] = ( + # Greeting + "هلا|أهلاً|السلام عليكم|مرحبا|مساء الخير|صباح الخير", + # Reason for contacting + "لاحظت|شفت|رأيت|متابع|قرأت|تابعت|اطلعت", + # Soft CTA + "يناسبك|تحب|ممكن|إذا فيه وقت|تفتح|تجربة|تواصل|نتقابل", +) + + +@dataclass(frozen=True) +class MessageGrade: + """Result of grading a single Arabic message.""" + score: int # 0..100 + verdict: str # "publish" | "needs_edit" | "reject" + reasons_ar: list[str] = field(default_factory=list) + suggestions_ar: list[str] = field(default_factory=list) + risky_phrases: list[str] = field(default_factory=list) + + def to_dict(self) -> dict[str, object]: + return { + "score": self.score, + "verdict": self.verdict, + "reasons_ar": self.reasons_ar, + "suggestions_ar": self.suggestions_ar, + "risky_phrases": self.risky_phrases, + } + + +def _has_arabic(text: str) -> bool: + return any("؀" <= ch <= "ۿ" for ch in text) + + +def _word_count(text: str) -> int: + return len([w for w in re.split(r"\s+", text.strip()) if w]) + + +def _matches_signal(text: str, alternatives: str) -> bool: + pat = "|".join(re.escape(a) for a in alternatives.split("|")) + return re.search(pat, text) is not None + + +def grade_message( + message: str, + *, + sector: str | None = None, + channel: str = "whatsapp", +) -> MessageGrade: + """ + Grade a single Arabic message. + + Returns MessageGrade with score 0..100 and a verdict. + """ + reasons: list[str] = [] + suggestions: list[str] = [] + risky: list[str] = [p for p in RISKY_PHRASES_AR if p in message] + + score = 100 + + # 1. Must contain Arabic. + if not _has_arabic(message): + score -= 60 + reasons.append("الرسالة لا تحتوي محتوى عربي.") + suggestions.append("أعد صياغة الرسالة بالعربي بأسلوب طبيعي سعودي.") + + # 2. Length sanity. + wc = _word_count(message) + if wc < 12: + score -= 15 + reasons.append("الرسالة قصيرة جداً ولا توضح السبب أو القيمة.") + suggestions.append("أضف سبب التواصل + سؤال مفتوح قصير.") + elif wc > 80: + score -= 15 + reasons.append("الرسالة طويلة جداً للعرض الأول.") + suggestions.append("اختصر إلى 4-6 أسطر.") + + # 3. Risky phrases. + if risky: + score -= 25 * min(len(risky), 2) + reasons.append(f"عبارات عالية المخاطرة: {', '.join(risky)}") + suggestions.append("استبدل العبارات المضللة بأمثلة محددة وأرقام واقعية.") + + # 4. Saudi tone signals (greeting + reason + soft CTA). + missing_signals = [] + for sig in REQUIRED_SIGNALS_AR: + if not _matches_signal(message, sig): + missing_signals.append(sig.split("|")[0]) + if missing_signals: + score -= 8 * len(missing_signals) + reasons.append( + f"تنقصها إشارات أسلوب طبيعي: {', '.join(missing_signals)}" + ) + suggestions.append("ابدأ بتحية + لاحظت/شفت + سؤال يناسبك.") + + # 5. WhatsApp-specific: avoid bulk markers. + if channel == "whatsapp" and re.search(r"\bعميل عزيز\b|\bلجميع العملاء\b", message): + score -= 10 + reasons.append("الرسالة بنبرة جماعية لا تناسب واتساب الشخصي.") + suggestions.append("استخدم اسم الشخص أو شركته بدل النداء العام.") + + # 6. Sector hook — soft bonus if sector is mentioned. + if sector and sector.lower() in message.lower(): + score = min(100, score + 5) + + score = max(0, min(100, score)) + if score >= 75 and not risky: + verdict = "publish" + elif score >= 50: + verdict = "needs_edit" + else: + verdict = "reject" + + return MessageGrade( + score=score, verdict=verdict, + reasons_ar=reasons, suggestions_ar=suggestions, + risky_phrases=risky, + ) + + +def detect_duplicates(messages: list[str], *, threshold: float = 0.85) -> list[tuple[int, int, float]]: + """ + Return pairs (i, j, ratio) of near-duplicate messages. + + Uses SequenceMatcher; deterministic, no external deps. + """ + pairs: list[tuple[int, int, float]] = [] + n = len(messages) + for i in range(n): + for j in range(i + 1, n): + ratio = SequenceMatcher(None, messages[i], messages[j]).ratio() + if ratio >= threshold: + pairs.append((i, j, round(ratio, 3))) + return pairs + + +def suggest_improvement(message: str, *, sector: str | None = None) -> dict[str, object]: + """Return a structured improvement suggestion (deterministic, no LLM).""" + grade = grade_message(message, sector=sector) + template = ( + "هلا [الاسم]، لاحظت [إشارة محددة عن شركتك/قطاعك]. " + "أعمل على [وصف العرض في جملة واحدة]. " + "يناسبك أعرض لك مثال خفيف 10 دقائق هذا الأسبوع؟" + ) + return { + "current": message, + "grade": grade.to_dict(), + "suggested_skeleton_ar": template, + } + + +def archive_low_quality( + messages: list[dict[str, object]], + *, + score_field: str = "score", + threshold: int = 50, +) -> dict[str, list[dict[str, object]]]: + """ + Split a list of {message, score} into (active, archived) by threshold. + """ + active: list[dict[str, object]] = [] + archived: list[dict[str, object]] = [] + for m in messages: + score = int(m.get(score_field, 0) or 0) + if score < threshold: + archived.append(m) + else: + active.append(m) + return {"active": active, "archived": archived} diff --git a/dealix/auto_client_acquisition/growth_curator/mission_curator.py b/dealix/auto_client_acquisition/growth_curator/mission_curator.py new file mode 100644 index 00000000..55c459a9 --- /dev/null +++ b/dealix/auto_client_acquisition/growth_curator/mission_curator.py @@ -0,0 +1,93 @@ +"""Mission Curator — score completed missions and pick the next one.""" + +from __future__ import annotations + + +def score_mission(mission: dict[str, object]) -> dict[str, object]: + """ + Score a completed mission run. + + Inputs: + opportunities_generated, drafts_approved, meetings_booked, + revenue_influenced_sar, time_to_value_minutes, risks_blocked + """ + opps = int(mission.get("opportunities_generated", 0) or 0) + approved = int(mission.get("drafts_approved", 0) or 0) + meetings = int(mission.get("meetings_booked", 0) or 0) + revenue = float(mission.get("revenue_influenced_sar", 0) or 0) + risks_blocked = int(mission.get("risks_blocked", 0) or 0) + ttv = float(mission.get("time_to_value_minutes", 9_999) or 9_999) + + score = 0 + score += min(20, opps * 2) + score += min(20, approved * 4) + score += min(20, meetings * 5) + score += min(20, int(revenue / 5_000)) + score += min(10, risks_blocked * 5) + if ttv <= 10: + score += 10 + elif ttv <= 60: + score += 5 + score = max(0, min(100, score)) + + if score >= 70: + verdict = "ship_it_widely" + elif score >= 40: + verdict = "iterate" + else: + verdict = "rework_or_retire" + + return {"score": score, "verdict": verdict, "ttv_minutes": ttv} + + +def recommend_next_mission( + mission_history: list[dict[str, object]] | None = None, + *, + growth_brain: dict[str, object] | None = None, +) -> dict[str, object]: + """ + Pick the next mission to run given history and brain context. + + Defaults to the kill feature `first_10_opportunities` for early-stage + customers (low signal count). + """ + if not mission_history: + return { + "recommended_mission_id": "first_10_opportunities", + "reason_ar": "لا يوجد تاريخ مهمات — نبدأ بالـ Kill Feature.", + } + + # If the kill feature has not yet shipped, ship it first. + ran_ids = {m.get("mission_id") for m in mission_history} + if "first_10_opportunities" not in ran_ids: + return { + "recommended_mission_id": "first_10_opportunities", + "reason_ar": "Kill Feature لم يُشغّل بعد — ابدأ به.", + } + + # Otherwise, pick the next mission by sector/priority. + priorities = [] + if growth_brain: + priorities = list(growth_brain.get("growth_priorities", []) or []) + + if "fill_pipeline" in priorities: + return { + "recommended_mission_id": "meeting_booking_sprint", + "reason_ar": "الأولوية ملء الـ pipeline — سبرنت حجز الاجتماعات.", + } + if "rescue_lost_revenue" in priorities: + return { + "recommended_mission_id": "revenue_leak_rescue", + "reason_ar": "الأولوية استرجاع الإيراد — تشغيل ميشن التسريب.", + } + if "expand_partners" in priorities: + return { + "recommended_mission_id": "partnership_sprint", + "reason_ar": "الأولوية توسيع الشركاء — ميشن الشراكات.", + } + + # Default deterministic next. + return { + "recommended_mission_id": "customer_reactivation", + "reason_ar": "الافتراضي: إعادة تنشيط العملاء الخاملين.", + } diff --git a/dealix/auto_client_acquisition/growth_curator/playbook_curator.py b/dealix/auto_client_acquisition/growth_curator/playbook_curator.py new file mode 100644 index 00000000..c3af1142 --- /dev/null +++ b/dealix/auto_client_acquisition/growth_curator/playbook_curator.py @@ -0,0 +1,144 @@ +"""Playbook Curator — score, merge, and recommend playbooks based on outcomes.""" + +from __future__ import annotations + +from difflib import SequenceMatcher + + +def score_playbook(playbook: dict[str, object]) -> dict[str, object]: + """ + Score a playbook on outcome quality. + + Inputs (all optional, defaults are conservative): + used_count, accept_count, replied_count, meeting_count, deal_count + """ + used = int(playbook.get("used_count", 0) or 0) + accepted = int(playbook.get("accept_count", 0) or 0) + replied = int(playbook.get("replied_count", 0) or 0) + meetings = int(playbook.get("meeting_count", 0) or 0) + deals = int(playbook.get("deal_count", 0) or 0) + + if used <= 0: + return { + "score": 0, "tier": "unproven", + "accept_rate": 0.0, "reply_rate": 0.0, + "meeting_rate": 0.0, "deal_rate": 0.0, + } + + accept_rate = accepted / used if used else 0.0 + reply_rate = replied / used if used else 0.0 + meeting_rate = meetings / used if used else 0.0 + deal_rate = deals / used if used else 0.0 + + # Weighted score; deals matter most. + score = int(round( + 100 * ( + 0.10 * accept_rate + + 0.20 * reply_rate + + 0.30 * meeting_rate + + 0.40 * deal_rate + ) + )) + score = max(0, min(100, score)) + + if score >= 70: + tier = "winner" + elif score >= 40: + tier = "promising" + elif score >= 20: + tier = "needs_work" + else: + tier = "candidate_archive" + + return { + "score": score, "tier": tier, + "accept_rate": round(accept_rate, 3), + "reply_rate": round(reply_rate, 3), + "meeting_rate": round(meeting_rate, 3), + "deal_rate": round(deal_rate, 3), + } + + +def merge_similar_playbooks( + playbooks: list[dict[str, object]], + *, + field: str = "title", + threshold: float = 0.80, +) -> list[dict[str, object]]: + """ + Group near-identical playbooks (by title similarity) and return + a list of merge suggestions: + [{"keep_index", "merge_indices", "merged_title", "similarity"}] + """ + suggestions: list[dict[str, object]] = [] + used: set[int] = set() + n = len(playbooks) + for i in range(n): + if i in used: + continue + merge_indices: list[int] = [] + title_i = str(playbooks[i].get(field, "") or "") + for j in range(i + 1, n): + if j in used: + continue + title_j = str(playbooks[j].get(field, "") or "") + if not title_i or not title_j: + continue + ratio = SequenceMatcher(None, title_i, title_j).ratio() + if ratio >= threshold: + merge_indices.append(j) + used.add(j) + if merge_indices: + used.add(i) + suggestions.append({ + "keep_index": i, + "merge_indices": merge_indices, + "merged_title": title_i, + "similarity_threshold": threshold, + }) + return suggestions + + +def recommend_next_playbook( + scored_playbooks: list[dict[str, object]], + *, + sector: str | None = None, +) -> dict[str, object]: + """ + Pick the next playbook to run given scored history. + + Strategy: prefer "promising" over "winner" (winners are saturated). + If sector is given, prefer playbooks tagged with that sector. + Falls back to deterministic default. + """ + if not scored_playbooks: + return { + "recommended_id": "default_warm_outreach", + "title_ar": "تواصل دافئ مع 10 جهات مختارة", + "reason_ar": "لا يوجد تاريخ بعد — ابدأ بالـ playbook الافتراضي.", + } + + candidates = list(scored_playbooks) + if sector: + sector_filtered = [ + p for p in candidates + if sector.lower() in str(p.get("sectors", "")).lower() + ] + if sector_filtered: + candidates = sector_filtered + + # Promote "promising" first, then "winner", then by score. + tier_priority = {"promising": 0, "winner": 1, "needs_work": 2, + "candidate_archive": 3, "unproven": 4} + candidates.sort(key=lambda p: ( + tier_priority.get(str(p.get("tier", "unproven")), 9), + -int(p.get("score", 0) or 0), + )) + chosen = candidates[0] + return { + "recommended_id": chosen.get("id"), + "title_ar": chosen.get("title", "?"), + "reason_ar": ( + f"الـ tier: {chosen.get('tier')}, الـ score: {chosen.get('score')}." + ), + } diff --git a/dealix/auto_client_acquisition/growth_curator/skill_inventory.py b/dealix/auto_client_acquisition/growth_curator/skill_inventory.py new file mode 100644 index 00000000..ad2cb303 --- /dev/null +++ b/dealix/auto_client_acquisition/growth_curator/skill_inventory.py @@ -0,0 +1,74 @@ +"""Skill Inventory — list every Dealix capability, categorized.""" + +from __future__ import annotations + +# Curated, deterministic inventory of skills across the layers. +SKILL_INVENTORY: tuple[dict[str, object], ...] = ( + # platform_services + {"id": "tool_gateway", "layer": "platform_services", + "label_ar": "بوابة الأدوات الآمنة", "tier": "core"}, + {"id": "action_policy", "layer": "platform_services", + "label_ar": "محرك سياسة الأفعال", "tier": "core"}, + {"id": "channel_registry", "layer": "platform_services", + "label_ar": "سجل القنوات", "tier": "core"}, + {"id": "unified_inbox", "layer": "platform_services", + "label_ar": "صندوق البريد الموحد", "tier": "core"}, + {"id": "action_ledger", "layer": "platform_services", + "label_ar": "سجل الأفعال", "tier": "core"}, + {"id": "proof_ledger", "layer": "platform_services", + "label_ar": "سجل الأثر", "tier": "core"}, + {"id": "service_catalog", "layer": "platform_services", + "label_ar": "كتالوج الخدمات", "tier": "core"}, + {"id": "identity_resolution", "layer": "platform_services", + "label_ar": "حل الهوية المتقاطع", "tier": "core"}, + # intelligence_layer + {"id": "growth_brain", "layer": "intelligence_layer", + "label_ar": "عقل النمو", "tier": "core"}, + {"id": "command_feed", "layer": "intelligence_layer", + "label_ar": "بطاقات القرار اليومية", "tier": "core"}, + {"id": "mission_engine", "layer": "intelligence_layer", + "label_ar": "محرك المهمات", "tier": "core"}, + {"id": "trust_score", "layer": "intelligence_layer", + "label_ar": "Trust Score", "tier": "core"}, + {"id": "revenue_dna", "layer": "intelligence_layer", + "label_ar": "DNA الإيرادات", "tier": "core"}, + {"id": "opportunity_simulator", "layer": "intelligence_layer", + "label_ar": "محاكي الفرص", "tier": "core"}, + {"id": "competitive_moves", "layer": "intelligence_layer", + "label_ar": "كاشف حركات المنافسين", "tier": "core"}, + {"id": "board_brief", "layer": "intelligence_layer", + "label_ar": "موجز Founder Shadow Board", "tier": "core"}, + {"id": "decision_memory", "layer": "intelligence_layer", + "label_ar": "ذاكرة القرارات", "tier": "core"}, + {"id": "action_graph", "layer": "intelligence_layer", + "label_ar": "Action Graph", "tier": "core"}, + # growth_operator (existing) + {"id": "first_10_opportunities", "layer": "growth_operator", + "label_ar": "10 فرص في 10 دقائق", "tier": "kill_feature"}, + # security_curator + {"id": "secret_redactor", "layer": "security_curator", + "label_ar": "إخفاء الأسرار", "tier": "core"}, + {"id": "patch_firewall", "layer": "security_curator", + "label_ar": "جدار الـ patches", "tier": "core"}, + # growth_curator + {"id": "message_curator", "layer": "growth_curator", + "label_ar": "مدقق الرسائل", "tier": "core"}, + {"id": "playbook_curator", "layer": "growth_curator", + "label_ar": "مدقق الـ playbooks", "tier": "core"}, +) + + +def inventory_skills() -> dict[str, object]: + """Return the full skill inventory grouped by layer.""" + by_layer: dict[str, list[dict[str, object]]] = {} + for s in SKILL_INVENTORY: + layer = str(s["layer"]) + by_layer.setdefault(layer, []).append(dict(s)) + return { + "total": len(SKILL_INVENTORY), + "layers": sorted(by_layer.keys()), + "by_layer": by_layer, + "kill_features": [ + dict(s) for s in SKILL_INVENTORY if s.get("tier") == "kill_feature" + ], + } diff --git a/dealix/auto_client_acquisition/meeting_intelligence/__init__.py b/dealix/auto_client_acquisition/meeting_intelligence/__init__.py new file mode 100644 index 00000000..e89cbad6 --- /dev/null +++ b/dealix/auto_client_acquisition/meeting_intelligence/__init__.py @@ -0,0 +1,25 @@ +"""Meeting Intelligence — pre-meeting briefs + post-meeting follow-ups. + +Designed to consume Google Meet transcripts (when OAuth + scopes allow) but +works fine with manually-pasted transcripts during private beta. + +All outputs are Arabic, deterministic, and approval-required before any +external action. +""" + +from __future__ import annotations + +from .deal_risk import compute_deal_risk +from .followup_builder import build_post_meeting_followup +from .meeting_brief import build_pre_meeting_brief +from .objection_extractor import extract_objections +from .transcript_parser import parse_transcript_entries, summarize_meeting + +__all__ = [ + "build_post_meeting_followup", + "build_pre_meeting_brief", + "compute_deal_risk", + "extract_objections", + "parse_transcript_entries", + "summarize_meeting", +] diff --git a/dealix/auto_client_acquisition/meeting_intelligence/deal_risk.py b/dealix/auto_client_acquisition/meeting_intelligence/deal_risk.py new file mode 100644 index 00000000..a53425de --- /dev/null +++ b/dealix/auto_client_acquisition/meeting_intelligence/deal_risk.py @@ -0,0 +1,81 @@ +"""Deal risk score from meeting + objection signals.""" + +from __future__ import annotations + +from typing import Any + + +def compute_deal_risk( + *, + objections: list[dict[str, Any]] | None = None, + next_step_set: bool = False, + decision_maker_present: bool = False, + days_since_last_touch: int = 0, + expected_value_sar: float = 0.0, +) -> dict[str, Any]: + """ + Compute a deal-level risk score (0..100) from meeting outcomes. + + Higher = riskier. Returns deterministic Arabic risk reasons. + """ + objections = objections or [] + score = 0 + reasons_ar: list[str] = [] + + # Objection-based risk. + categories = {str(o.get("category", "")).lower() for o in objections} + if "price" in categories: + score += 20 + reasons_ar.append("اعتراض على السعر — يحتاج إثبات قيمة وعينة محسوبة.") + if "timing" in categories: + score += 15 + reasons_ar.append("اعتراض توقيت — احفظ الفرصة لربع لاحق.") + if "authority" in categories: + score += 25 + reasons_ar.append("صاحب القرار غير حاضر — يلزم اجتماع ثانٍ معه.") + if "trust" in categories: + score += 20 + reasons_ar.append("قلق أمان/خصوصية — أرفق DPA و PDPL.") + if "integration" in categories: + score += 10 + reasons_ar.append("قلق تكامل — حضّر مخطط ربط CRM.") + if "competitor" in categories: + score += 15 + reasons_ar.append("بديل قائم — جهّز battlecard مقارنة.") + + # Process risk. + if not next_step_set: + score += 25 + reasons_ar.append("لم يتم تحديد خطوة تالية بتاريخ — أعلى مؤشر فقدان.") + if not decision_maker_present: + score += 10 + reasons_ar.append("صانع القرار لم يحضر الاجتماع.") + if days_since_last_touch > 14: + score += 10 + reasons_ar.append( + f"مرّ {days_since_last_touch} يوم على آخر تواصل — فرصة باردة." + ) + + # Cap. + score = max(0, min(100, score)) + + if score >= 70: + risk_level = "high" + elif score >= 40: + risk_level = "medium" + else: + risk_level = "low" + + return { + "risk_score": score, + "risk_level": risk_level, + "reasons_ar": reasons_ar, + "expected_value_sar": expected_value_sar, + "recommended_action_ar": ( + "اجتماع ثانٍ مع صاحب القرار خلال 5 أيام + مادة إثبات قيمة قصيرة." + if risk_level == "high" else + "متابعة خلال 3 أيام مع خطوة تالية محددة." + if risk_level == "medium" else + "تنفيذ الخطوة التالية المتفق عليها كما هي." + ), + } diff --git a/dealix/auto_client_acquisition/meeting_intelligence/followup_builder.py b/dealix/auto_client_acquisition/meeting_intelligence/followup_builder.py new file mode 100644 index 00000000..1d87002d --- /dev/null +++ b/dealix/auto_client_acquisition/meeting_intelligence/followup_builder.py @@ -0,0 +1,72 @@ +"""Build a post-meeting follow-up draft (Arabic) — never sends.""" + +from __future__ import annotations + +from typing import Any + + +def build_post_meeting_followup( + *, + summary: dict[str, Any] | None = None, + next_steps: list[str] | None = None, + contact_name: str = "", + company_name: str = "", + objections: list[dict[str, Any]] | None = None, +) -> dict[str, Any]: + """ + Build a draft follow-up email/WhatsApp message in Arabic. + + Always returns approval_required=True; never executes a send. + """ + next_steps = next_steps or [] + objections = objections or [] + + salutation = f"هلا {contact_name}" if contact_name else "هلا" + company_part = f" من شركة {company_name}" if company_name else "" + + bullet_steps = "\n".join([f"• {s}" for s in next_steps]) or "• [حدد الخطوة التالية بتاريخ محدد]" + + objection_addressed = "" + if objections: + labels = sorted({str(o.get("label_ar", "")) for o in objections if o.get("label_ar")}) + if labels: + objection_addressed = ( + "\nرجعت بعد الاجتماع وفكرت في النقاط التي ذكرتها: " + + "، ".join(labels) + + ". أرفقت لك إجابات قصيرة مع أمثلة." + ) + + body_ar = ( + f"{salutation}،\n" + f"شكراً على وقتك اليوم{company_part}. " + "ملخص ما اتفقنا عليه:\n" + f"{bullet_steps}\n" + f"{objection_addressed}\n" + "\nإذا كل شي واضح من جهتك، أبدأ في تجهيز Pilot قصير ونشتغل خلال أسبوع. " + "أي ملاحظة تحب تضيفها قبل ما نبدأ؟\n\nشاكر لك." + ) + + subject_ar = f"متابعة اجتماع اليوم — {company_name or 'Dealix'}" + + return { + "channel_drafts": { + "email": { + "subject_ar": subject_ar, + "body_ar": body_ar, + "approval_required": True, + "live_send_allowed": False, + }, + "whatsapp": { + "body_ar": ( + f"{salutation}، شكراً على اجتماع اليوم. " + "الخطوة التالية: " + (next_steps[0] if next_steps else "نحدد موعد بداية الـPilot") + + ". أتابع معك خلال يومين." + ), + "approval_required": True, + "live_send_allowed": False, + }, + }, + "summary_used": bool(summary), + "objections_addressed": [str(o.get("label_ar")) for o in objections if o.get("label_ar")], + "approval_required": True, + } diff --git a/dealix/auto_client_acquisition/meeting_intelligence/meeting_brief.py b/dealix/auto_client_acquisition/meeting_intelligence/meeting_brief.py new file mode 100644 index 00000000..9a2a431b --- /dev/null +++ b/dealix/auto_client_acquisition/meeting_intelligence/meeting_brief.py @@ -0,0 +1,74 @@ +"""Pre-meeting brief builder — deterministic Arabic output.""" + +from __future__ import annotations + +from typing import Any + + +def build_pre_meeting_brief( + *, + company: dict[str, Any] | None = None, + contact: dict[str, Any] | None = None, + opportunity: dict[str, Any] | None = None, + sector: str | None = None, +) -> dict[str, Any]: + """ + Build a 6-section Arabic pre-meeting brief. + + All inputs are optional; the brief degrades to a generic but useful template. + """ + company = company or {} + contact = contact or {} + opportunity = opportunity or {} + sector = sector or str(company.get("sector", "saas")) + + company_name = company.get("name", "?") + contact_name = contact.get("name", "?") + contact_role = contact.get("role", "?") + deal_value = opportunity.get("expected_value_sar", 0) + + objective_ar = ( + f"توضيح ملاءمة الحل لشركة {company_name}، " + f"وفهم المعيار الذي يستخدمه {contact_name} للقرار، " + "ثم تحديد خطوة تالية واضحة." + ) + + questions_ar = [ + f"كيف تتعاملون اليوم مع [مشكلة قطاع {sector}]؟", + "ما الذي جعلكم تنظرون لحل الآن وليس قبل 6 أشهر؟", + "من المسؤول عن قرار الشراء غيرك؟", + "ما المعيار الذي يجعلكم تقولون: نعم، خلونا نبدأ؟", + "ما الميزانية التقريبية المخصصة لهذه المشكلة؟", + ] + + likely_objections_ar = [ + "السعر مرتفع مقارنة بالأدوات المحلية.", + "نحن مرتبطون بـ CRM/أداة حالية ولا نريد التبديل.", + "نحتاج تجربة فريق صغير أولاً قبل القرار.", + "هل الحل متوافق مع PDPL ولا يخزن بياناتنا خارج المملكة؟", + "كم يستغرق الإعداد فعلياً؟", + ] + + offer_skeleton_ar = ( + f"عرض pilot لمدة 7 أيام لشركة {company_name}: " + "10 فرص B2B + رسائل عربية + متابعة + Proof Pack. " + "السعر 499 ريال أو مجاني مقابل case study." + ) + + next_step_ar = ( + "في نهاية المكالمة: اقترح خطوة محددة بتاريخ — " + "إما الموافقة على بدء Pilot، أو إعادة الاجتماع خلال 5 أيام مع صانع القرار." + ) + + return { + "company_name": company_name, + "contact_name": contact_name, + "contact_role": contact_role, + "expected_value_sar": deal_value, + "objective_ar": objective_ar, + "questions_ar": questions_ar, + "likely_objections_ar": likely_objections_ar, + "offer_skeleton_ar": offer_skeleton_ar, + "next_step_ar": next_step_ar, + "approval_required": True, + } diff --git a/dealix/auto_client_acquisition/meeting_intelligence/objection_extractor.py b/dealix/auto_client_acquisition/meeting_intelligence/objection_extractor.py new file mode 100644 index 00000000..d874fd11 --- /dev/null +++ b/dealix/auto_client_acquisition/meeting_intelligence/objection_extractor.py @@ -0,0 +1,52 @@ +"""Objection extractor — find common Arabic + English buying objections in transcript.""" + +from __future__ import annotations + +import re + +# Each entry: (category, regex pattern (case-insensitive), Arabic gloss). +OBJECTION_PATTERNS: tuple[tuple[str, str, str], ...] = ( + ("price", r"غالي|مرتفع|الميزانية|expensive|too\s+pricey|cost", "السعر/الميزانية"), + ("timing", r"ليس\s+الآن|بعد\s+شهر|الربع\s+القادم|not\s+now|next\s+quarter", "التوقيت"), + ("authority", r"المدير|صاحب\s+القرار|need\s+approval|decision\s+maker", "صاحب القرار"), + ("trust", r"بيانات|خصوصية|أمان|PDPL|trust|security|privacy", "الأمان والخصوصية"), + ("integration", r"CRM|نظامنا|الربط|integration|migration", "التكامل/الترحيل"), + ("competitor", r"نستخدم|بديل|أداة\s+ثانية|competitor|alternative", "وجود بديل/منافس"), + ("results", r"نتائج|مضمون|guarantee|ROI|دليل", "إثبات النتائج"), + ("complexity", r"معقد|صعب|تدريب|onboarding|complex|hard", "التعقيد/التبني"), +) + + +def extract_objections(transcript_text: str) -> dict[str, object]: + """ + Extract objection categories from a free-text transcript. + + Returns: + { + "objections": [{"category", "label_ar", "snippet"}], + "categories_found": [str], + "count": int, + } + """ + if not transcript_text: + return {"objections": [], "categories_found": [], "count": 0} + + found: list[dict[str, str]] = [] + seen_categories: set[str] = set() + for cat, pattern, gloss in OBJECTION_PATTERNS: + for m in re.finditer(pattern, transcript_text, flags=re.IGNORECASE): + seen_categories.add(cat) + start = max(0, m.start() - 40) + end = min(len(transcript_text), m.end() + 40) + snippet = transcript_text[start:end].replace("\n", " ").strip() + found.append({ + "category": cat, + "label_ar": gloss, + "snippet": snippet[:200], + }) + + return { + "objections": found, + "categories_found": sorted(seen_categories), + "count": len(found), + } diff --git a/dealix/auto_client_acquisition/meeting_intelligence/transcript_parser.py b/dealix/auto_client_acquisition/meeting_intelligence/transcript_parser.py new file mode 100644 index 00000000..720b77ce --- /dev/null +++ b/dealix/auto_client_acquisition/meeting_intelligence/transcript_parser.py @@ -0,0 +1,92 @@ +"""Transcript parser — accepts Google Meet entries OR plain text.""" + +from __future__ import annotations + +import re +from typing import Any + + +def parse_transcript_entries(entries: list[dict[str, Any]] | str) -> dict[str, Any]: + """ + Normalize either: + - a list of Google-Meet-shaped entries [{"participantId", "text", ...}], or + - a plain string transcript with "Speaker: text" lines. + + Returns: + { + "speaker_turns": [{"speaker", "text"}], + "speakers": [str], + "total_chars": int, + "total_turns": int, + } + """ + speaker_turns: list[dict[str, str]] = [] + + if isinstance(entries, str): + for raw in entries.splitlines(): + line = raw.strip() + if not line: + continue + m = re.match(r"^([^:]{1,40}):\s*(.+)$", line) + if m: + speaker_turns.append({"speaker": m.group(1).strip(), + "text": m.group(2).strip()}) + else: + speaker_turns.append({"speaker": "?", "text": line}) + else: + for e in entries or []: + speaker = ( + e.get("participant") + or e.get("participantId") + or e.get("speaker") + or "?" + ) + text = e.get("text") or e.get("content") or "" + text = str(text).strip() + if not text: + continue + speaker_turns.append({"speaker": str(speaker), "text": text}) + + speakers = sorted({t["speaker"] for t in speaker_turns}) + total_chars = sum(len(t["text"]) for t in speaker_turns) + return { + "speaker_turns": speaker_turns, + "speakers": speakers, + "total_chars": total_chars, + "total_turns": len(speaker_turns), + } + + +def summarize_meeting(parsed: dict[str, Any]) -> dict[str, Any]: + """ + Produce an Arabic summary skeleton from parsed turns. + + Deterministic; LLM-free for Phase D MVP. + """ + turns = parsed.get("speaker_turns", []) + speakers = parsed.get("speakers", []) + + # Extract a few candidate "topic" sentences: longest turns. + sorted_by_len = sorted(turns, key=lambda t: -len(t["text"]))[:5] + topic_lines = [t["text"][:200] for t in sorted_by_len] + + # Detect questions. + questions: list[str] = [] + for t in turns: + text = t["text"] + if "؟" in text or text.rstrip().endswith("?"): + questions.append(text[:200]) + if len(questions) >= 5: + break + + return { + "summary_ar": [ + f"شارك في الاجتماع {len(speakers)} متحدث.", + f"إجمالي عدد الأدوار الكلامية: {parsed.get('total_turns', 0)}.", + "أبرز نقاط النقاش (مرشحة آلياً، تحتاج مراجعة):", + *[f"• {line}" for line in topic_lines], + ], + "speakers": speakers, + "candidate_questions_ar": questions, + "approval_required": True, + } diff --git a/dealix/auto_client_acquisition/model_router/__init__.py b/dealix/auto_client_acquisition/model_router/__init__.py new file mode 100644 index 00000000..e6743e2e --- /dev/null +++ b/dealix/auto_client_acquisition/model_router/__init__.py @@ -0,0 +1,29 @@ +"""Model Router — pick the right model/provider for each task type, with fallback.""" + +from __future__ import annotations + +from .cost_policy import CostClass, classify_cost +from .fallback_policy import build_fallback_chain +from .provider_registry import ( + ALL_PROVIDERS, + ALL_TASK_TYPES, + Provider, + TaskType, + get_provider, +) +from .task_router import RouteDecision, route_task +from .usage_dashboard import build_usage_demo + +__all__ = [ + "ALL_PROVIDERS", + "ALL_TASK_TYPES", + "CostClass", + "Provider", + "RouteDecision", + "TaskType", + "build_fallback_chain", + "build_usage_demo", + "classify_cost", + "get_provider", + "route_task", +] diff --git a/dealix/auto_client_acquisition/model_router/cost_policy.py b/dealix/auto_client_acquisition/model_router/cost_policy.py new file mode 100644 index 00000000..b41e0f88 --- /dev/null +++ b/dealix/auto_client_acquisition/model_router/cost_policy.py @@ -0,0 +1,36 @@ +"""Cost policy — classify a task's cost class without locking to specific tokens prices.""" + +from __future__ import annotations + +from typing import Literal + +CostClass = Literal["low", "mid", "high"] + + +def classify_cost( + *, + task_type: str, + expected_input_tokens: int = 0, + expected_output_tokens: int = 0, + bulk: bool = False, +) -> CostClass: + """ + Heuristic cost class. + + - bulk volume → low + - large output (>1500 tokens) → high + - strategic / vision / arabic_copywriting → mid + - everything else → low + """ + if bulk: + return "low" + if expected_output_tokens > 1500 or expected_input_tokens > 8000: + return "high" + if task_type in { + "strategic_reasoning", "vision_analysis", + "compliance_guardrail", "meeting_analysis", + }: + return "mid" + if task_type in {"arabic_copywriting"}: + return "mid" + return "low" diff --git a/dealix/auto_client_acquisition/model_router/fallback_policy.py b/dealix/auto_client_acquisition/model_router/fallback_policy.py new file mode 100644 index 00000000..df31772e --- /dev/null +++ b/dealix/auto_client_acquisition/model_router/fallback_policy.py @@ -0,0 +1,60 @@ +"""Build a deterministic fallback chain for any task type.""" + +from __future__ import annotations + +from .provider_registry import ALL_PROVIDERS, Provider + + +def _supports(p: Provider, task_type: str, *, requires_arabic: bool, requires_vision: bool) -> bool: + if task_type not in p.capabilities: + return False + if requires_arabic and not p.supports_arabic: + return False + if requires_vision and not p.supports_vision: + return False + return True + + +def build_fallback_chain( + task_type: str, + *, + requires_arabic: bool = False, + requires_vision: bool = False, + sensitivity: str = "low", + primary_key: str | None = None, +) -> list[str]: + """ + Return an ordered list of provider keys to try for a task. + + Rules: + - if `primary_key` is supplied and supports the task, it goes first. + - high-sensitivity workloads prefer KSA-region or self-hosted. + - among the rest, lower cost_class is preferred. + """ + candidates = [ + p for p in ALL_PROVIDERS + if _supports(p, task_type, + requires_arabic=requires_arabic, + requires_vision=requires_vision) + ] + + cost_order = {"low": 0, "mid": 1, "high": 2} + privacy_order = {"self_hosted": 0, "ksa_region": 1, "vendor_cloud": 2} + + if sensitivity == "high": + candidates.sort(key=lambda p: ( + privacy_order.get(p.privacy_tier, 9), + cost_order.get(p.cost_class, 9), + )) + else: + candidates.sort(key=lambda p: ( + cost_order.get(p.cost_class, 9), + privacy_order.get(p.privacy_tier, 9), + )) + + chain = [p.key for p in candidates] + if primary_key: + if primary_key in chain: + chain.remove(primary_key) + chain.insert(0, primary_key) + return chain diff --git a/dealix/auto_client_acquisition/model_router/provider_registry.py b/dealix/auto_client_acquisition/model_router/provider_registry.py new file mode 100644 index 00000000..1bb47f32 --- /dev/null +++ b/dealix/auto_client_acquisition/model_router/provider_registry.py @@ -0,0 +1,171 @@ +"""Registry of model providers + task types.""" + +from __future__ import annotations + +from dataclasses import dataclass, field + +# Task types Dealix actually routes. +ALL_TASK_TYPES: tuple[str, ...] = ( + "strategic_reasoning", + "arabic_copywriting", + "classification", + "compliance_guardrail", + "meeting_analysis", + "vision_analysis", + "extraction", + "summarization", + "coding_project_understanding", + "low_cost_bulk", +) + + +@dataclass(frozen=True) +class Provider: + """A model provider entry.""" + key: str + label: str + family: str # "anthropic" | "openai" | "google" | "azure" | "local" + capabilities: tuple[str, ...] # subset of ALL_TASK_TYPES + cost_class: str # "low" | "mid" | "high" + latency_class: str # "fast" | "balanced" | "slow" + supports_vision: bool + supports_arabic: bool + privacy_tier: str # "vendor_cloud" | "ksa_region" | "self_hosted" + notes_ar: str = "" + + def to_dict(self) -> dict[str, object]: + return { + "key": self.key, "label": self.label, "family": self.family, + "capabilities": list(self.capabilities), + "cost_class": self.cost_class, "latency_class": self.latency_class, + "supports_vision": self.supports_vision, + "supports_arabic": self.supports_arabic, + "privacy_tier": self.privacy_tier, + "notes_ar": self.notes_ar, + } + + +# Conservative provider list — Dealix can swap any of these without code change. +ALL_PROVIDERS: tuple[Provider, ...] = ( + Provider( + key="claude_sonnet", + label="Claude Sonnet", + family="anthropic", + capabilities=( + "strategic_reasoning", "arabic_copywriting", + "compliance_guardrail", "meeting_analysis", "summarization", + "coding_project_understanding", + ), + cost_class="mid", + latency_class="balanced", + supports_vision=True, + supports_arabic=True, + privacy_tier="vendor_cloud", + notes_ar="مناسب للاستراتيجية والكتابة العربية والامتثال.", + ), + Provider( + key="claude_haiku", + label="Claude Haiku", + family="anthropic", + capabilities=("classification", "extraction", "low_cost_bulk", "summarization"), + cost_class="low", + latency_class="fast", + supports_vision=False, + supports_arabic=True, + privacy_tier="vendor_cloud", + notes_ar="رخيص وسريع — للتصنيف الكثيف والاستخراج.", + ), + Provider( + key="gpt_4_class", + label="GPT-4-class", + family="openai", + capabilities=( + "strategic_reasoning", "vision_analysis", + "coding_project_understanding", "meeting_analysis", + ), + cost_class="high", + latency_class="balanced", + supports_vision=True, + supports_arabic=True, + privacy_tier="vendor_cloud", + notes_ar="بديل قوي للاستراتيجية والرؤية.", + ), + Provider( + key="gpt_4o_mini", + label="GPT-4o mini", + family="openai", + capabilities=("classification", "extraction", "low_cost_bulk"), + cost_class="low", + latency_class="fast", + supports_vision=True, + supports_arabic=True, + privacy_tier="vendor_cloud", + notes_ar="بديل رخيص للمهام الكثيفة.", + ), + Provider( + key="gemini_pro", + label="Gemini Pro", + family="google", + capabilities=( + "vision_analysis", "summarization", "meeting_analysis", + "extraction", + ), + cost_class="mid", + latency_class="balanced", + supports_vision=True, + supports_arabic=True, + privacy_tier="vendor_cloud", + notes_ar="ممتاز للرؤية والاجتماعات.", + ), + Provider( + key="azure_oai_ksa", + label="Azure OpenAI (KSA region)", + family="azure", + capabilities=( + "strategic_reasoning", "arabic_copywriting", + "compliance_guardrail", "extraction", "summarization", + ), + cost_class="mid", + latency_class="balanced", + supports_vision=True, + supports_arabic=True, + privacy_tier="ksa_region", + notes_ar="منطقة KSA — مناسب للعملاء الحساسين للامتثال.", + ), + Provider( + key="local_qwen_ar", + label="Local Qwen (Arabic-tuned)", + family="local", + capabilities=("classification", "extraction", "low_cost_bulk", "arabic_copywriting"), + cost_class="low", + latency_class="balanced", + supports_vision=False, + supports_arabic=True, + privacy_tier="self_hosted", + notes_ar="نموذج محلي — للحالات الحساسة جداً.", + ), +) + + +def get_provider(key: str) -> Provider | None: + return next((p for p in ALL_PROVIDERS if p.key == key), None) + + +@dataclass(frozen=True) +class TaskType: + """Description of a routed task.""" + key: str + label_ar: str + requires_arabic: bool + requires_vision: bool + sensitivity: str # "low" | "medium" | "high" + notes_ar: str = "" + + def to_dict(self) -> dict[str, object]: + return { + "key": self.key, "label_ar": self.label_ar, + "requires_arabic": self.requires_arabic, + "requires_vision": self.requires_vision, + "sensitivity": self.sensitivity, + "notes_ar": self.notes_ar, + } diff --git a/dealix/auto_client_acquisition/model_router/task_router.py b/dealix/auto_client_acquisition/model_router/task_router.py new file mode 100644 index 00000000..9f114403 --- /dev/null +++ b/dealix/auto_client_acquisition/model_router/task_router.py @@ -0,0 +1,103 @@ +"""Route a task to the right provider, with fallback chain + cost class.""" + +from __future__ import annotations + +from dataclasses import dataclass + +from .cost_policy import CostClass, classify_cost +from .fallback_policy import build_fallback_chain +from .provider_registry import ALL_TASK_TYPES, get_provider + + +@dataclass(frozen=True) +class RouteDecision: + task_type: str + primary_provider: str | None + fallback_chain: list[str] + cost_class: CostClass + reasons_ar: list[str] + requires_arabic: bool + requires_vision: bool + sensitivity: str + + def to_dict(self) -> dict[str, object]: + return { + "task_type": self.task_type, + "primary_provider": self.primary_provider, + "fallback_chain": self.fallback_chain, + "cost_class": self.cost_class, + "reasons_ar": self.reasons_ar, + "requires_arabic": self.requires_arabic, + "requires_vision": self.requires_vision, + "sensitivity": self.sensitivity, + } + + +def route_task( + task_type: str, + *, + requires_arabic: bool = False, + requires_vision: bool = False, + sensitivity: str = "low", + expected_input_tokens: int = 0, + expected_output_tokens: int = 0, + bulk: bool = False, + primary_provider: str | None = None, +) -> RouteDecision: + """Route a task → primary provider + ordered fallback chain + cost class.""" + reasons: list[str] = [] + + if task_type not in ALL_TASK_TYPES: + return RouteDecision( + task_type=task_type, + primary_provider=None, + fallback_chain=[], + cost_class="low", + reasons_ar=[f"نوع المهمة غير معروف: {task_type}"], + requires_arabic=requires_arabic, + requires_vision=requires_vision, + sensitivity=sensitivity, + ) + + cost_class = classify_cost( + task_type=task_type, + expected_input_tokens=expected_input_tokens, + expected_output_tokens=expected_output_tokens, + bulk=bulk, + ) + + chain = build_fallback_chain( + task_type, + requires_arabic=requires_arabic, + requires_vision=requires_vision, + sensitivity=sensitivity, + primary_key=primary_provider, + ) + + if not chain: + reasons.append( + "لا يوجد مزود مناسب — راجع capabilities أو خفّف القيود (vision/arabic)." + ) + + primary = chain[0] if chain else None + if primary: + p = get_provider(primary) + if p: + reasons.append( + f"المزود الأساسي: {p.label} — {p.notes_ar}" + ) + if sensitivity == "high": + reasons.append("حساسية عالية: تم تفضيل KSA-region/self-hosted أولاً.") + if bulk: + reasons.append("مهمة جماعية كبيرة: تم اختيار cost_class=low.") + + return RouteDecision( + task_type=task_type, + primary_provider=primary, + fallback_chain=chain, + cost_class=cost_class, + reasons_ar=reasons, + requires_arabic=requires_arabic, + requires_vision=requires_vision, + sensitivity=sensitivity, + ) diff --git a/dealix/auto_client_acquisition/model_router/usage_dashboard.py b/dealix/auto_client_acquisition/model_router/usage_dashboard.py new file mode 100644 index 00000000..1a4e820b --- /dev/null +++ b/dealix/auto_client_acquisition/model_router/usage_dashboard.py @@ -0,0 +1,32 @@ +"""Demo usage dashboard for the model router (deterministic).""" + +from __future__ import annotations + +from .provider_registry import ALL_PROVIDERS, ALL_TASK_TYPES +from .task_router import route_task + + +def build_usage_demo() -> dict[str, object]: + """ + Demo: route every task type once and surface aggregate stats. + + Used by /api/v1/model-router/usage/demo to show the router behavior. + """ + routes: list[dict[str, object]] = [] + for tt in ALL_TASK_TYPES: + d = route_task(tt, requires_arabic=(tt == "arabic_copywriting")) + routes.append(d.to_dict()) + + cost_counts: dict[str, int] = {} + primary_counts: dict[str, int] = {} + for r in routes: + cost_counts[str(r.get("cost_class"))] = cost_counts.get(str(r.get("cost_class")), 0) + 1 + primary_counts[str(r.get("primary_provider"))] = primary_counts.get(str(r.get("primary_provider")), 0) + 1 + + return { + "providers_total": len(ALL_PROVIDERS), + "task_types_total": len(ALL_TASK_TYPES), + "routes": routes, + "cost_counts": cost_counts, + "primary_counts": primary_counts, + } diff --git a/dealix/auto_client_acquisition/security_curator/__init__.py b/dealix/auto_client_acquisition/security_curator/__init__.py new file mode 100644 index 00000000..c99e6eb6 --- /dev/null +++ b/dealix/auto_client_acquisition/security_curator/__init__.py @@ -0,0 +1,46 @@ +"""Security Curator — secret redaction + patch firewall + trace sanitization. + +Inspired by Hermes Agent's Curator pattern, but specialized for Dealix's +external-action surface (WhatsApp, Gmail, Calendar, Moyasar, Social). + +Goals: +- Never let an API key, token, or PAT escape into a log/trace/embedding/patch. +- Block any diff that adds .env files or secret-shaped strings. +- Sanitize tool outputs before they go into the Action Ledger or Proof Pack. +""" + +from __future__ import annotations + +from .patch_firewall import ( + PatchFirewallResult, + inspect_diff, + is_safe_diff, +) +from .secret_redactor import ( + DEFAULT_PATTERNS, + SecretFinding, + detect_secret_patterns, + redact_secrets, + scan_payload, +) +from .tool_output_sanitizer import ( + sanitize_tool_output, + sanitize_trace_event, +) +from .trace_redactor import ( + redact_trace, +) + +__all__ = [ + "DEFAULT_PATTERNS", + "PatchFirewallResult", + "SecretFinding", + "detect_secret_patterns", + "inspect_diff", + "is_safe_diff", + "redact_secrets", + "redact_trace", + "sanitize_tool_output", + "sanitize_trace_event", + "scan_payload", +] diff --git a/dealix/auto_client_acquisition/security_curator/patch_firewall.py b/dealix/auto_client_acquisition/security_curator/patch_firewall.py new file mode 100644 index 00000000..df7e485e --- /dev/null +++ b/dealix/auto_client_acquisition/security_curator/patch_firewall.py @@ -0,0 +1,99 @@ +"""Patch Firewall — block unsafe diffs before they enter the repo.""" + +from __future__ import annotations + +import re +from dataclasses import dataclass, field + +from .secret_redactor import detect_secret_patterns + +# Files that should never be added to the repo via patch. +DANGEROUS_FILE_PATTERNS: tuple[str, ...] = ( + r"^\+\+\+ b/.*\.env$", + r"^\+\+\+ b/.*\.env\.local$", + r"^\+\+\+ b/.*\.env\.staging$", + r"^\+\+\+ b/.*\.env\.production$", + r"^\+\+\+ b/.*credentials\.json$", + r"^\+\+\+ b/.*service[-_]account.*\.json$", + r"^\+\+\+ b/.*id_rsa$", + r"^\+\+\+ b/.*\.pem$", + r"^\+\+\+ b/.*\.p12$", + r"^\+\+\+ b/.*\.pfx$", +) + + +@dataclass(frozen=True) +class PatchFirewallResult: + safe: bool + reasons_ar: list[str] = field(default_factory=list) + blocked_files: list[str] = field(default_factory=list) + secret_findings: list[dict[str, str]] = field(default_factory=list) + + def to_dict(self) -> dict[str, object]: + return { + "safe": self.safe, + "reasons_ar": self.reasons_ar, + "blocked_files": self.blocked_files, + "secret_findings": self.secret_findings, + } + + +def _added_lines(diff_text: str) -> str: + """Concatenate only the *added* lines from a unified diff.""" + out: list[str] = [] + for line in diff_text.splitlines(): + if line.startswith("+++") or line.startswith("---"): + continue + if line.startswith("+"): + out.append(line[1:]) + return "\n".join(out) + + +def _blocked_files_in_diff(diff_text: str) -> list[str]: + blocked: list[str] = [] + for line in diff_text.splitlines(): + for pat in DANGEROUS_FILE_PATTERNS: + if re.match(pat, line): + blocked.append(line.replace("+++ b/", "")) + break + return blocked + + +def inspect_diff(diff_text: str) -> PatchFirewallResult: + """ + Inspect a unified-diff blob. + + Returns PatchFirewallResult.safe = False if: + - The diff adds a file from DANGEROUS_FILE_PATTERNS, OR + - Any added line contains a known secret pattern. + """ + if not diff_text: + return PatchFirewallResult(safe=True) + + reasons: list[str] = [] + blocked = _blocked_files_in_diff(diff_text) + if blocked: + reasons.append(f"الملفات المحظورة: {', '.join(blocked)}") + + added = _added_lines(diff_text) + findings = detect_secret_patterns(added) + finding_dicts = [ + {"label": f.label, "sample_redacted": f.sample_redacted} + for f in findings + ] + if findings: + labels = sorted({f.label for f in findings}) + reasons.append(f"تم اكتشاف أسرار محتملة: {', '.join(labels)}") + + safe = not reasons + return PatchFirewallResult( + safe=safe, + reasons_ar=reasons, + blocked_files=blocked, + secret_findings=finding_dicts, + ) + + +def is_safe_diff(diff_text: str) -> bool: + """Convenience boolean wrapper around inspect_diff().""" + return inspect_diff(diff_text).safe diff --git a/dealix/auto_client_acquisition/security_curator/secret_redactor.py b/dealix/auto_client_acquisition/security_curator/secret_redactor.py new file mode 100644 index 00000000..cfb62e70 --- /dev/null +++ b/dealix/auto_client_acquisition/security_curator/secret_redactor.py @@ -0,0 +1,113 @@ +"""Secret Redactor — detect + redact secret-shaped strings before they leak.""" + +from __future__ import annotations + +import re +from dataclasses import dataclass +from typing import Any + +# Patterns are intentionally specific to avoid false positives. +# Each entry: (label, regex, redaction_template). +DEFAULT_PATTERNS: tuple[tuple[str, str, str], ...] = ( + ("github_pat", r"ghp_[A-Za-z0-9]{20,}", "ghp_***"), + ("github_pat_legacy", r"github_pat_[A-Za-z0-9_]{20,}", "github_pat_***"), + ("openai_key", r"sk-[A-Za-z0-9]{20,}", "sk-***"), + ("anthropic_key", r"sk-ant-[A-Za-z0-9_\-]{20,}", "sk-ant-***"), + ("supabase_service_role", r"eyJ[A-Za-z0-9_\-]{30,}\.[A-Za-z0-9_\-]{30,}\.[A-Za-z0-9_\-]{20,}", "eyJ.***.***"), + ("whatsapp_token", r"EAA[A-Za-z0-9]{30,}", "EAA***"), + ("moyasar_secret", r"sk_(?:test|live)_[A-Za-z0-9]{20,}", "sk_***_***"), + ("langfuse_secret", r"lf_sk_[A-Za-z0-9]{20,}", "lf_sk_***"), + ("sentry_dsn", r"https://[A-Za-z0-9]{20,}@[A-Za-z0-9.\-]+/\d+", "https://***@***/***"), + ("aws_access_key", r"AKIA[A-Z0-9]{16}", "AKIA***"), + ("google_api_key", r"AIza[A-Za-z0-9_\-]{30,}", "AIza***"), + ("private_key_block", r"-----BEGIN (?:RSA |EC |OPENSSH |)PRIVATE KEY-----", "-----BEGIN PRIVATE KEY *** REDACTED ***-----"), +) + +# Sensitive keys for dict-shaped payloads (case-insensitive substring match). +SENSITIVE_PAYLOAD_KEYS: tuple[str, ...] = ( + "api_key", "apikey", "secret", "token", "password", "passwd", + "authorization", "auth_token", "access_token", "refresh_token", + "client_secret", "private_key", "ssn", "credit_card", "card_number", + "cvv", "iban", "moyasar_secret", +) + + +@dataclass(frozen=True) +class SecretFinding: + """A single secret detected in input.""" + label: str + span: tuple[int, int] + sample_redacted: str # the *redacted* form, never the raw secret + + +def detect_secret_patterns(text: str) -> list[SecretFinding]: + """Find secret-shaped substrings. Never returns the raw secret.""" + if not text: + return [] + findings: list[SecretFinding] = [] + for label, pattern, redaction in DEFAULT_PATTERNS: + for m in re.finditer(pattern, text): + findings.append(SecretFinding( + label=label, + span=(m.start(), m.end()), + sample_redacted=redaction, + )) + return findings + + +def redact_secrets(text: str) -> str: + """Replace every detected secret with a label-typed redaction marker.""" + if not text: + return text + out = text + for _label, pattern, redaction in DEFAULT_PATTERNS: + out = re.sub(pattern, redaction, out) + return out + + +def _is_sensitive_key(key: str) -> bool: + k = key.lower() + return any(s in k for s in SENSITIVE_PAYLOAD_KEYS) + + +def scan_payload(payload: Any) -> dict[str, Any]: + """ + Scan a JSON-shaped payload for secret-typed keys + secret-shaped values. + + Returns: + { + "has_secrets": bool, + "findings": [{"label", "path"}], + "redacted": , + } + """ + findings: list[dict[str, str]] = [] + + def _walk(node: Any, path: str) -> Any: + if isinstance(node, dict): + out: dict[str, Any] = {} + for k, v in node.items(): + p = f"{path}.{k}" if path else str(k) + if _is_sensitive_key(str(k)): + findings.append({"label": "sensitive_key", "path": p}) + out[k] = "***" + else: + out[k] = _walk(v, p) + return out + if isinstance(node, list): + return [_walk(item, f"{path}[{i}]") for i, item in enumerate(node)] + if isinstance(node, str): + secrets = detect_secret_patterns(node) + if secrets: + for s in secrets: + findings.append({"label": s.label, "path": path}) + return redact_secrets(node) + return node + return node + + redacted = _walk(payload, "") + return { + "has_secrets": bool(findings), + "findings": findings, + "redacted": redacted, + } diff --git a/dealix/auto_client_acquisition/security_curator/tool_output_sanitizer.py b/dealix/auto_client_acquisition/security_curator/tool_output_sanitizer.py new file mode 100644 index 00000000..4b48a2e0 --- /dev/null +++ b/dealix/auto_client_acquisition/security_curator/tool_output_sanitizer.py @@ -0,0 +1,68 @@ +"""Sanitize tool/agent outputs before they reach the user, ledger, or Proof Pack.""" + +from __future__ import annotations + +from typing import Any + +from .secret_redactor import scan_payload +from .trace_redactor import redact_trace + + +def sanitize_tool_output(output: Any, *, mask_pii: bool = True) -> dict[str, Any]: + """ + Sanitize a tool's output before showing it to a human or persisting it. + + Returns: + { + "safe": bool (True iff no secrets and no payload PII at risk), + "redacted": , + "notes_ar": list[str] of human-readable notes, + } + """ + notes: list[str] = [] + secret_scan = scan_payload(output) + redacted = secret_scan["redacted"] + + if secret_scan["has_secrets"]: + labels = sorted({f["label"] for f in secret_scan["findings"]}) + notes.append(f"تمت إزالة قيم حساسة من المخرج: {', '.join(labels)}") + + if mask_pii: + trace_scan = redact_trace(redacted, mask_pii=True) + redacted = trace_scan["redacted"] + if trace_scan["had_pii"]: + notes.append("تم إخفاء أرقام/إيميلات في المخرج لأغراض الخصوصية.") + + safe = not secret_scan["has_secrets"] + return {"safe": safe, "redacted": redacted, "notes_ar": notes} + + +def sanitize_trace_event(event: dict[str, Any]) -> dict[str, Any]: + """ + Sanitize a single trace event for Langfuse/Sentry. + + Always preserves: event_type, agent_name, status, latency_ms, cost_estimate. + Always masks: payload, output, input. + """ + safe_keys = { + "event_type", "agent_name", "status", "latency_ms", + "cost_estimate", "approval_status", "tool", "policy_result", + "risk_level", "user_id_hash", "company_id_hash", + "workflow_name", "trace_id", "span_id", "ts", + } + risky_keys = {"payload", "output", "input", "context", "raw"} + + out: dict[str, Any] = {} + for k, v in event.items(): + if k in safe_keys: + out[k] = v + elif k in risky_keys: + scan = redact_trace(v, mask_pii=True) + out[k] = scan["redacted"] + if scan["had_secrets"] or scan["had_pii"]: + out.setdefault("_sanitized", []).append(k) + else: + # Unknown keys default to redaction, just in case. + scan = redact_trace(v, mask_pii=True) + out[k] = scan["redacted"] + return out diff --git a/dealix/auto_client_acquisition/security_curator/trace_redactor.py b/dealix/auto_client_acquisition/security_curator/trace_redactor.py new file mode 100644 index 00000000..020c37f0 --- /dev/null +++ b/dealix/auto_client_acquisition/security_curator/trace_redactor.py @@ -0,0 +1,76 @@ +"""Trace Redactor — strip secrets/PII from traces before sending to Langfuse/Sentry.""" + +from __future__ import annotations + +import re +from typing import Any + +from .secret_redactor import scan_payload + +# Phone-number-ish patterns we'll mask. Saudi: +966 5xxxxxxxx; international. +_PHONE_RE = re.compile(r"\+?\d[\d\s\-]{7,}\d") +# Generic email. +_EMAIL_RE = re.compile(r"[A-Za-z0-9._%+\-]+@[A-Za-z0-9.\-]+\.[A-Za-z]{2,}") + + +def _mask_phone(s: str) -> str: + def _mask(m: re.Match[str]) -> str: + raw = m.group(0) + digits_only = re.sub(r"\D", "", raw) + if len(digits_only) < 7: + return raw + return digits_only[:3] + "*" * (len(digits_only) - 6) + digits_only[-3:] + return _PHONE_RE.sub(_mask, s) + + +def _mask_email(s: str) -> str: + def _mask(m: re.Match[str]) -> str: + local, _, domain = m.group(0).partition("@") + if not local or not domain: + return m.group(0) + keep = local[0] if local else "" + return f"{keep}***@{domain}" + return _EMAIL_RE.sub(_mask, s) + + +def redact_trace(payload: Any, *, mask_pii: bool = True) -> dict[str, Any]: + """ + Redact a trace payload for safe storage in observability tools. + + - Always strips secret patterns + sensitive keys (api_key/token/etc.). + - When mask_pii=True (default), also masks phone numbers and emails inside + string values. + + Returns: + { + "had_secrets": bool, + "had_pii": bool, + "redacted": , + } + """ + secret_scan = scan_payload(payload) + redacted = secret_scan["redacted"] + had_pii = False + + if mask_pii: + had_pii_box: list[bool] = [False] + + def _walk(node: Any) -> Any: + if isinstance(node, dict): + return {k: _walk(v) for k, v in node.items()} + if isinstance(node, list): + return [_walk(item) for item in node] + if isinstance(node, str): + if _PHONE_RE.search(node) or _EMAIL_RE.search(node): + had_pii_box[0] = True + return _mask_email(_mask_phone(node)) + return node + + redacted = _walk(redacted) + had_pii = had_pii_box[0] + + return { + "had_secrets": secret_scan["has_secrets"], + "had_pii": had_pii, + "redacted": redacted, + } diff --git a/dealix/docs/AGENT_OBSERVABILITY_EVALS.md b/dealix/docs/AGENT_OBSERVABILITY_EVALS.md new file mode 100644 index 00000000..2ed074ed --- /dev/null +++ b/dealix/docs/AGENT_OBSERVABILITY_EVALS.md @@ -0,0 +1,67 @@ +# Agent Observability + Evals — مراقبة الوكلاء + التقييمات + +> Trace events معقّمة + safety eval + Saudi tone eval + cost tracker. كله deterministic، لا PII في الـtraces. + +## 1. Trace Events + +`build_trace_event(...)` يبني trace جاهز لـLangfuse/Sentry: +- `user_id` و`company_id` تُهاش (sha256[:16]) قبل التخزين. +- `payload` و`output` يمران عبر `sanitize_trace_event`. +- الحقول الآمنة (event_type, agent_name, status, latency_ms, cost_estimate, approval_status, tool, policy_result, risk_level, workflow_name, trace_id) تبقى كما هي. + +## 2. Safety Eval + +7 قواعد: + +| الفئة | السببية بالعربي | الخطورة | +|------|-----------------|--------| +| guarantee | وعد بنتائج مضمونة | 50 | +| scarcity_fake | تكتيك ندرة مزيف | 25 | +| medical_claim | ادعاء طبي | 50 | +| financial_claim | عوائد مبالغ فيها | 35 | +| regulatory | ادعاء ترخيص | 35 | +| personal_data | تلميح بيع بيانات | 50 | +| urgency_manipulation | ضغط زمني مصطنع | 15 | + +`score = max(0, 100 - sum_penalties)`. تيرز: ≥70 safe, ≥40 needs_review, <40 blocked. + +## 3. Saudi Tone Eval + +- إيجابيات: "هلا/أهلاً/مساء الخير، لاحظت/شفت، يناسبك/تحب، Pilot/بايلوت" → +12 لكل واحدة. +- سلبيات: "السيد المحترم/تحية طيبة وبعد/ندعوكم لاكتشاف، leverage/synergy/best-in-class" → -20 لكل واحدة. +- نسبة عربية ≥60%: +20؛ ≥30%: +10. +- طول > 80 كلمة: -10. + +تيرز: ≥75 natural, ≥50 decent, <50 off. + +## 4. Eval Pack + +5 cases مختارة (`run_eval_pack()`): +- natural_warm_intro → safe + natural +- fake_urgency → blocked + off +- too_corporate → safe + off +- medical_claim → blocked + off (أو needs_review) +- decent_but_short → safe + decent + +النتيجة: `{total, passed, failed, pass_rate, results}`. + +## 5. Cost Tracker + +`CostTracker.record(workflow_name, provider_key, task_type, cost_estimate)` ثم `summary()` يُرجع `{runs, total, by_workflow, by_provider, by_task_type}`. + +## 6. Endpoints + +``` +POST /api/v1/agent-observability/trace/build +POST /api/v1/agent-observability/safety/eval +POST /api/v1/agent-observability/tone/eval +GET /api/v1/agent-observability/evals/run +``` + +## 7. حدود + +- لا tokens في الـtraces. +- لا secrets (يمر عبر `sanitize_trace_event`). +- لا raw PII (phones/emails مخفية). +- لا full customer lists. +- لا payment details. diff --git a/dealix/docs/AGENT_SECURITY_CURATOR.md b/dealix/docs/AGENT_SECURITY_CURATOR.md new file mode 100644 index 00000000..f343168c --- /dev/null +++ b/dealix/docs/AGENT_SECURITY_CURATOR.md @@ -0,0 +1,107 @@ +# Security Curator — منظومة حماية وكلاء Dealix + +> **القاعدة الأولى:** لا سرّ يخرج من Dealix إلى log/trace/embedding/patch. +> الـ Security Curator هو الجدار الأول، يعمل قبل أي اتصال بأي قناة خارجية. + +--- + +## 1. لماذا هذه الطبقة قبل أي tool live؟ + +Dealix يربط أدوات حساسة: WhatsApp Cloud, Gmail, Calendar, Moyasar, Google Meet, CRM. كل أداة فيها token، كل token خطر إذا تسرب. سابقاً تعرضنا لـPAT مكشوف، لذا قبل أي ربط حي: + +- يجب أن يمر كل log/trace من **redactor**. +- يجب أن يمر كل diff من **patch firewall**. +- يجب أن يمر كل tool output من **sanitizer**. +- يجب ألا تخزّن أي assets مع secrets في الـembedding store. + +--- + +## 2. الوحدات + +| الوحدة | الدور | +|--------|------| +| `secret_redactor` | كشف وإزالة 11 نمط سر (GitHub PAT، OpenAI/Anthropic keys، Supabase JWT، WhatsApp/Moyasar/Sentry/Google API keys، AWS، private keys). | +| `patch_firewall` | يفحص الـunified diff قبل commit ويرفض الـ.env و service-account JSON و RSA keys. | +| `trace_redactor` | بالإضافة للأسرار، يخفي phones وemails داخل القيم النصية. | +| `tool_output_sanitizer` | يعقّم مخرجات الأدوات قبل إظهارها للمستخدم أو حفظها في الـledger. | + +--- + +## 3. أنماط الأسرار المكشوفة + +``` +github_pat ghp_*** +github_pat_legacy github_pat_*** +openai_key sk-*** +anthropic_key sk-ant-*** +supabase_service_role eyJ.***.*** +whatsapp_token EAA*** +moyasar_secret sk_***_*** +langfuse_secret lf_sk_*** +sentry_dsn https://***@***/*** +aws_access_key AKIA*** +google_api_key AIza*** +private_key_block BEGIN PRIVATE KEY *** REDACTED *** +``` + +ومفاتيح JSON الحساسة تُستبدل بـ`***` بناءً على substring match (case-insensitive) لـ: +`api_key, apikey, secret, token, password, authorization, access_token, refresh_token, client_secret, private_key, ssn, credit_card, card_number, cvv, iban, moyasar_secret`. + +--- + +## 4. Patch Firewall + +أي PR قبل ما يدخل الريبو: + +1. **ملفات محظورة:** `.env`, `.env.local`, `.env.staging`, `.env.production`, `credentials.json`, `service-account*.json`, `id_rsa`, `*.pem`, `*.p12`, `*.pfx`. +2. **أسرار في الأسطر المضافة:** أي line يبدأ بـ`+` يُمرر من `detect_secret_patterns`. +3. الناتج: `PatchFirewallResult{safe, reasons_ar, blocked_files, secret_findings}`. + +GitHub Push Protection يقبض الأسرار قبل push، لكن لا تعتمد عليه وحده — Patch Firewall يعمل في طبقة التطوير المحلية + CI. + +--- + +## 5. Tool Output Sanitizer + +قبل أن يصل أي مخرج إلى: +- الـAction Ledger +- الـProof Pack +- الواجهة (UI / WhatsApp / Email) +- Langfuse / Sentry + +يمر عبر `sanitize_tool_output(output)` الذي يُرجع: +- `safe: bool` +- `redacted: <نفس الشكل، مُعقّم>` +- `notes_ar: ["تمت إزالة قيم حساسة من المخرج: ..."]` + +--- + +## 6. Endpoints + +``` +GET /api/v1/security-curator/demo +POST /api/v1/security-curator/redact +POST /api/v1/security-curator/inspect-diff +POST /api/v1/security-curator/sanitize-output +``` + +--- + +## 7. اختبارات الأمان (16 test) + +- detect_github_pat لا يُرجع السر الخام أبداً. +- redact_openai_key يستبدل بالـmask. +- scan_payload يخفي `api_key` و`token`. +- inspect_diff يحظر `.env`. +- inspect_diff يحظر سراً مكتوباً داخل سطر مضاف. +- redact_trace يخفي phones/emails مع الحفاظ على الـdomain للسياق. +- sanitize_trace_event يحفظ `event_type/agent_name/latency_ms` ويعقّم `payload`. + +--- + +## 8. ما لا تفعله هذه الطبقة + +- لا تكشف السر الخام في الـlogs أبداً. +- لا تُرجع payload فيه token. +- لا توقع على diff فيه secret. +- لا تستبدل أو تعطّل GitHub Push Protection — هذه الطبقة **إضافة**، لا بديل. diff --git a/dealix/docs/CONNECTOR_CATALOG.md b/dealix/docs/CONNECTOR_CATALOG.md new file mode 100644 index 00000000..32fbe328 --- /dev/null +++ b/dealix/docs/CONNECTOR_CATALOG.md @@ -0,0 +1,43 @@ +# Connector Catalog — كتالوج التكاملات + +> 14 تكامل، كل واحد له launch_phase + risk_level + allowed/blocked actions + Arabic risk notes. + +## 1. القائمة + +| key | الحالة | المرحلة | المخاطر | ملاحظة | +|-----|--------|---------|---------|--------| +| whatsapp_cloud | beta | phase_1 | high | PDPL: لا cold بدون opt-in | +| gmail | beta | phase_1 | high | drafts فقط افتراضياً | +| google_calendar | beta | phase_1 | medium | إدراج بموافقة | +| google_meet | beta | phase_2 | high | transcripts بموافقة الجميع | +| moyasar | beta | phase_1 | high | لا تخزّن بطاقات | +| linkedin_lead_forms | coming_soon | phase_2 | medium | leads مصرّح بها | +| google_business_profile | coming_soon | phase_2 | medium | ردود بموافقة | +| x_api | coming_soon | phase_3 | high | حسب خطة الـAPI | +| instagram_graph | coming_soon | phase_3 | high | لا auto-publish | +| google_sheets | beta | phase_1 | low | append بموافقة | +| crm_generic | beta | phase_2 | medium | اقرأ أولاً | +| website_forms | live | phase_1 | low | مصدر العميل | +| composio | coming_soon | phase_4 | medium | خلف Tool Gateway | +| mcp_gateway | coming_soon | phase_4 | high | allowlist + audit | + +## 2. Launch Phases + +- **Phase 1** (الإطلاق الخاص): WhatsApp + Gmail + Calendar + Moyasar + Sheets + Website Forms. +- **Phase 2** (Beta موسّع): LinkedIn Lead Forms + Google Business + Meet + CRM. +- **Phase 3** (السوشيال): X + Instagram. +- **Phase 4** (التوسع): Composio + MCP Gateway. + +## 3. Endpoints + +``` +GET /api/v1/connector-catalog/catalog +GET /api/v1/connector-catalog/summary +GET /api/v1/connector-catalog/status +GET /api/v1/connector-catalog/risks +GET /api/v1/connector-catalog/{connector_key} +``` + +## 4. القاعدة الذهبية + +كل tool action يمر من Tool Gateway في `platform_services` → Action Policy → draft/approval_required. الـCatalog هنا توثّق فقط ما هو متاح، **لا تنفّذ**. diff --git a/dealix/docs/DEALIX_100_PERCENT_LAUNCH_PLAN.md b/dealix/docs/DEALIX_100_PERCENT_LAUNCH_PLAN.md index 62f5742e..b8559292 100644 --- a/dealix/docs/DEALIX_100_PERCENT_LAUNCH_PLAN.md +++ b/dealix/docs/DEALIX_100_PERCENT_LAUNCH_PLAN.md @@ -178,6 +178,37 @@ OAuth Gmail/Calendar، حصص، سياسات. **Endpoints:** `/api/v1/intelligence/{growth-brain/build, command-feed/demo, missions, missions/recommend, trust-score, revenue-dna/demo, revenue-dna, simulate-opportunity, competitive-move/analyze, board-brief/demo, decisions/record, decisions/preferences}`. **التفصيل:** [`INTELLIGENCE_LAYER_STRATEGY.md`](INTELLIGENCE_LAYER_STRATEGY.md). +## 34. Self-Improving Agent Platform (Hermes-inspired) + +طبقة "ذاتية التحسن" فوق Platform Services + Intelligence Layer. 6 modules جديدة + 6 routers جديدة + 76 اختبار: + +- **Security Curator** ([`AGENT_SECURITY_CURATOR.md`](AGENT_SECURITY_CURATOR.md)) — secret_redactor (11 نمط: GitHub/OpenAI/Anthropic/Supabase/WhatsApp/Moyasar/Sentry/Google/AWS) + patch_firewall (يحظر `.env` والـRSA keys في الـdiff) + trace_redactor (يخفي phones/emails) + tool_output_sanitizer. +- **Growth Curator** ([`GROWTH_CURATOR_STRATEGY.md`](GROWTH_CURATOR_STRATEGY.md)) — message_curator (يقيّم الرسائل العربية، يكشف 8 عبارات محظورة) + playbook_curator (winner/promising/needs_work/archive) + mission_curator + skill_inventory (20+ skill عبر 5 طبقات) + curator_report (تقرير عربي أسبوعي). +- **Meeting Intelligence** ([`MEETING_INTELLIGENCE.md`](MEETING_INTELLIGENCE.md)) — Pre-meeting brief (6 أقسام عربية) + transcript_parser (Google Meet entries أو نص) + objection_extractor (8 فئات) + followup_builder (email + WhatsApp drafts) + deal_risk (0..100). +- **Model Router** ([`MODEL_PROVIDER_ROUTER.md`](MODEL_PROVIDER_ROUTER.md)) — 7 providers (Claude Sonnet/Haiku, GPT-4, GPT-4o-mini, Gemini Pro, Azure OAI KSA-region, Local Qwen) × 10 task types + cost_policy + fallback_policy (KSA-region أولاً للحالات الحساسة). +- **Connector Catalog** ([`CONNECTOR_CATALOG.md`](CONNECTOR_CATALOG.md)) — 14 تكامل (WhatsApp Cloud, Gmail, Calendar, Meet, Moyasar, LinkedIn Lead Forms, Google Business Profile, X, Instagram, Sheets, CRM, Website Forms, Composio, MCP Gateway) كل واحد له launch_phase + risk_level + Arabic risks. +- **Agent Observability** ([`AGENT_OBSERVABILITY_EVALS.md`](AGENT_OBSERVABILITY_EVALS.md)) — trace_events (مع hash للـuser/company IDs) + safety_eval (7 قواعد) + saudi_tone_eval (إيجابيات/سلبيات/نسبة عربية) + eval_pack (5 cases) + cost_tracker. + +**Endpoints جديدة:** +- `/api/v1/security-curator/{demo, redact, inspect-diff, sanitize-output}` +- `/api/v1/growth-curator/{skills/inventory, messages/grade, messages/improve, messages/duplicates, missions/next, report/weekly, report/demo}` +- `/api/v1/meeting-intelligence/{brief, brief/demo, transcript/summarize, followup/draft, deal-risk}` +- `/api/v1/model-router/{providers, tasks, route, cost-class, usage/demo}` +- `/api/v1/connector-catalog/{catalog, summary, status, risks, {key}}` +- `/api/v1/agent-observability/{trace/build, safety/eval, tone/eval, evals/run}` + +## 35. Private Beta Launch — Today + +راجع: +- [`PRIVATE_BETA_LAUNCH_TODAY.md`](PRIVATE_BETA_LAUNCH_TODAY.md) — الخطة الكاملة للإطلاق. +- [`DEMO_SCRIPT_12_MINUTES.md`](DEMO_SCRIPT_12_MINUTES.md) — السكربت المعتمد للعرض. +- [`FIRST_20_OUTREACH_MESSAGES.md`](FIRST_20_OUTREACH_MESSAGES.md) — قوالب الرسائل العربية. +- `landing/private-beta.html` — صفحة العرض. + +**العرض:** Pilot 7 أيام بـ499 ريال أو مجاني مقابل case study. Paid Pilot 30 يوم بـ1,500–3,000 ريال. Growth OS اشتراك شهري بـ2,999 ريال. + +**ممنوع اليوم:** live WhatsApp send, live Gmail send, live Calendar insert, payment charge, scraping social, وعود "نضمن نتائج". + --- -**الخلاصة:** المنتج **قوي كأساس سوقي وتقني**؛ الإطلاق العام يحتاج تشغيلاً وامتثالاً وتجربة عميل مغلقة أولاً. +**الخلاصة:** المنتج **قوي كأساس سوقي وتقني**؛ الإطلاق العام يحتاج تشغيلاً وامتثالاً وتجربة عميل مغلقة أولاً. الإطلاق اليوم = Private Beta + Pilots + Proof Pack، ليس Public Launch. diff --git a/dealix/docs/DEMO_SCRIPT_12_MINUTES.md b/dealix/docs/DEMO_SCRIPT_12_MINUTES.md new file mode 100644 index 00000000..dd0c2d3f --- /dev/null +++ b/dealix/docs/DEMO_SCRIPT_12_MINUTES.md @@ -0,0 +1,84 @@ +# Demo Script — 12 دقيقة + +## الدقيقة 0–2 — الفكرة الكبرى + +> "Dealix ليس CRM ولا أداة واتساب. Dealix يقول لك من تكلم اليوم، لماذا، ماذا تقول، وماذا حدث بعد ذلك. كل قناة (واتساب، إيميل، تقويم، مدفوعات) تتحول إلى كرت قرار عربي، أنت توافق أو ترفض، ثم Proof Pack." + +اعرض الصفحة الرئيسية: +``` +GET / +``` + +## الدقيقة 2–4 — Daily Brief + +``` +GET /api/v1/personal-operator/daily-brief +``` + +اعرض: +- 3 قرارات اليوم. +- فرص مفتوحة. +- مخاطر. +- launch readiness. + +> "كل صباح، Dealix يبني لك هذه القائمة. ما تفتح 8 تطبيقات." + +## الدقيقة 4–6 — Command Feed (Intelligence Layer) + +``` +GET /api/v1/intelligence/command-feed/demo +``` + +اعرض 6 بطاقات: opportunity / revenue_leak / partner_suggestion / meeting_prep / review_response / competitive_move. + +> "كل بطاقة فيها: لماذا الآن، الإجراء المقترح، الأثر المتوقع، 3 أزرار: قبول/تخطي/تعديل." + +## الدقيقة 6–8 — 10 فرص في 10 دقائق + +``` +GET /api/v1/intelligence/missions +POST /api/v1/intelligence/missions/recommend +``` + +> "هذه أول مهمة لكل عميل: 10 فرص B2B مناسبة بالعربي مع why-now ورسائل، خلال 10 دقائق." + +## الدقيقة 8–10 — Trust + Simulator + Proof + +``` +POST /api/v1/intelligence/trust-score +POST /api/v1/intelligence/simulate-opportunity +GET /api/v1/growth-operator/proof-pack/demo +``` + +اعرض: +- قبل أي إرسال، Trust Score (safe / needs_review / blocked). +- Simulator يحاكِ 100 جهة → expected pipeline + risk. +- Proof Pack: leads, drafts, meetings, risks_blocked, revenue_influenced. + +> "هذا هو الفرق: لا نرسل بدون trust. لا ننفّذ بدون simulator. لا نختفي بدون proof." + +## الدقيقة 10–12 — الأمان + التكاملات + +``` +GET /api/v1/security-curator/demo +GET /api/v1/connector-catalog/catalog +``` + +اعرض: +- أي token يدخل → يخرج كـ`***`. +- 14 تكامل، كل واحد له launch_phase + risk_level + blocked_actions. +- WhatsApp يحظر cold send افتراضياً. +- Moyasar لا يخزّن بطاقات. + +> "Dealix مبني على قاعدة: لا نضرّ سمعة العميل. هذه أهم نقطة بيع للسوق السعودي." + +## الإغلاق + +> "Pilot 7 أيام: 499 ريال أو مجاني مقابل case study. خلال أسبوع: 10 فرص + رسائل + متابعة + Proof Pack. مستعد نبدأ يوم الأحد؟" + +## القاعدة + +- لا تظهر API keys على الشاشة. +- لا تعرض staging credentials. +- لا تعد بأرقام لم تُحقَّق. +- لا تشغّل live WhatsApp send في الـdemo. diff --git a/dealix/docs/FIRST_20_OUTREACH_MESSAGES.md b/dealix/docs/FIRST_20_OUTREACH_MESSAGES.md new file mode 100644 index 00000000..8c486fa6 --- /dev/null +++ b/dealix/docs/FIRST_20_OUTREACH_MESSAGES.md @@ -0,0 +1,124 @@ +# First 20 Outreach Messages — قوالب جاهزة + +> كل رسالة عربية، طبيعية، تحت 80 كلمة، بدون "ضمان 100%"، ولا "آخر فرصة". +> كل رسالة تمر من `safety_eval` و`saudi_tone_eval` قبل الإرسال. + +--- + +## 1. مؤسس → مؤسس (واتساب أو لينكدإن) + +``` +هلا [الاسم]، أبني Dealix كـ مدير نمو عربي للشركات السعودية. +الفكرة: خلال 7 أيام نطلع لك 10 فرص B2B مناسبة، نكتب الرسائل بالعربي، +وأنت توافق أو ترفض قبل أي تواصل. +أفتح 5 مقاعد Pilot هذا الأسبوع. +يناسبك أعرض لك ديمو 12 دقيقة؟ +``` + +## 2. وكالة B2B (لينكدإن) + +``` +هلا [الاسم]، عندي فكرة ممكن تفيد وكالتكم وعملاءكم. +Dealix يطلع فرص B2B، يكتب الرسائل بالعربي، ويجهز متابعة وProof Pack — +بدون إرسال عشوائي. أبغى أجربها مع وكالة شريك Pilot: +نختار عميل عندكم ونطلع له 10 فرص خلال أسبوع. +مهتم تشوف ديمو 15 دقيقة؟ +``` + +## 3. شركة تدريب B2B (إيميل) + +``` +الموضوع: 10 فرص شركات لـ [اسم الشركة] خلال أسبوع + +هلا [الاسم]، +لاحظت أن [اسم الشركة] فتحت برامج جديدة للشركات. +نشتغل على Dealix كـ مدير نمو عربي: +- 10 فرص B2B مناسبة لقطاعكم +- رسائل عربية جاهزة +- متابعة 7 أيام +- Proof Pack بعد الأسبوع + +Pilot بـ 499 ريال أو مجاني مقابل case study. +يناسبك مكالمة 15 دقيقة الأسبوع القادم؟ +``` + +## 4. SaaS سعودية (لينكدإن DM) + +``` +هلا [الاسم]، رأيت إصدار النسخة الجديدة من [اسم المنتج] — +مبروك على التحديث. +نشتغل على مدير نمو عربي يطلع 10 فرص B2B خلال أسبوع +ويكتب الرسائل بالعربي ويتابع. +أبغى أجربه مع شركة SaaS سعودية واحدة. +يناسبك ديمو 12 دقيقة؟ +``` + +## 5. شركة عقار (واتساب) + +``` +هلا [الاسم]، نشتغل على Dealix: +نظام يطلع leads عقار مناسبين + يكتب رسائل تأهيل بالعربي ++ يحجز معاينات. أنت توافق على كل رسالة قبل الإرسال. +عندي 3 مقاعد Pilot للعقار هذا الشهر. +تحب أعرض لك ديمو سريع؟ +``` + +## 6. عيادة/متجر (واتساب) + +``` +هلا [الاسم]، نشتغل على نظام عربي للعيادات/المتاجر: +- يستعيد العملاء الخاملين +- يرد على تقييمات Google +- يجهز رسائل واتساب للحملات (بعد موافقتك) +- تقرير شهري بالعائد + +Pilot 7 أيام بـ 499 ريال. تحب نجرب؟ +``` + +## 7. مستشار نمو (لينكدإن) + +``` +هلا [الاسم]، متابع كتاباتك عن نمو B2B في السعودية. +أبني Dealix: مدير نمو عربي يطلع 10 فرص أسبوعياً، يكتب الرسائل، +ويجهز Proof Pack. أبغى رأيك في الـoffer قبل الإطلاق العام. +يناسبك مكالمة 20 دقيقة هذا الأسبوع؟ +``` + +--- + +## Follow-up 1 (بعد 3 أيام بدون رد) + +``` +هلا [الاسم]، لو الفكرة لا تناسب الآن خبرني وأرتاح. +وإذا فيه شي معين تبغى تشوفه قبل، قلي وأرسله لك. +``` + +## Follow-up 2 (بعد 7 أيام) + +``` +هلا [الاسم]، أعرف أن وقتك مزدحم. +سؤال أخير: لو طلعت لك 3 فرص B2B بالعربي مجاناً هذا الأسبوع، +تعطيني 15 دقيقة feedback؟ +``` + +## Follow-up 3 (إغلاق نهائي بعد أسبوعين) + +``` +هلا [الاسم]، أعتذر على الإلحاح. +أرشّفها وأكون موجود لو احتجتني لاحقاً. +شاكر لك. +``` + +--- + +## القواعد + +1. **اسم محدد** بدلاً من "العميل العزيز". +2. **سبب واضح** للتواصل ("لاحظت" / "رأيت" / "متابع"). +3. **سؤال مفتوح** في النهاية. +4. **لا** "ضمان 100%" أو "آخر فرصة". +5. **عرض محدد** (سعر + مدة + مخرجات). +6. **مخرج آمن** ("لو ما تناسب الآن خبرني"). +7. **حد أقصى 3 follow-ups** ثم أرشفة. + +كل رسالة تمر من `POST /api/v1/agent-observability/safety/eval` و`tone/eval` قبل الإرسال. diff --git a/dealix/docs/GROWTH_CURATOR_STRATEGY.md b/dealix/docs/GROWTH_CURATOR_STRATEGY.md new file mode 100644 index 00000000..18670a16 --- /dev/null +++ b/dealix/docs/GROWTH_CURATOR_STRATEGY.md @@ -0,0 +1,98 @@ +# Growth Curator Strategy — مدير التحسين الذاتي للنمو + +> **الفكرة (مستلهمة من Hermes Curator):** كل أسبوع، Dealix يراجع ما كتبه ونفذه، يدمج المتشابه، يأرشف الضعيف، ويقترح المهمة التالية. لا يحتاج المالك أن يفكر كل أسبوع "ماذا أحسّن؟". + +## 1. الوحدات + +| الوحدة | الدور | +|--------|------| +| `message_curator` | يقيّم كل رسالة عربية (0..100) ويحدد publish/needs_edit/reject. يكشف العبارات المخاطرة + يقترح صيغة بديلة. | +| `playbook_curator` | يقيّم playbooks بناءً على outcomes (accept/reply/meeting/deal) ويُصنّف winner/promising/needs_work/candidate_archive. | +| `mission_curator` | يقيّم نتائج الميشن (TTV, opportunities, drafts, meetings, revenue) ويقرر ship_it_widely/iterate/rework. | +| `skill_inventory` | فهرس deterministic لكل قدرات Dealix (20+ skill عبر 5 طبقات). | +| `curator_report` | تقرير عربي أسبوعي يجمع الكل. | + +## 2. Message Grading + +`grade_message(text, sector, channel)` يفحص: +- محتوى عربي (≥30%). +- طول معقول (12-80 كلمة). +- خلوّ من 8 عبارات محظورة (ضمان 100%, آخر فرصة، ...). +- إشارات أسلوب طبيعي سعودي (تحية + لاحظت/شفت + يناسبك/تحب). +- WhatsApp: لا "عميل عزيز" ولا "لجميع العملاء". +- bonus لذكر القطاع. + +## 3. Playbook Scoring + +``` +score = 100 * ( + 0.10 * accept_rate ++ 0.20 * reply_rate ++ 0.30 * meeting_rate ++ 0.40 * deal_rate +) +``` + +تيرز: +- ≥70: **winner** +- ≥40: **promising** +- ≥20: **needs_work** +- <20: **candidate_archive** + +استراتيجية الـrecommend: **promising أولاً** (winners مشبعة)، ثم winner، ثم بقية الـtiers. + +## 4. Mission Scoring + +`score_mission` يجمع: +- opportunities × 2 (max 20) +- drafts_approved × 4 (max 20) +- meetings_booked × 5 (max 20) +- revenue / 5,000 (max 20) +- risks_blocked × 5 (max 10) +- TTV ≤10min: +10، ≤60min: +5 + +## 5. Mission Recommender + +- لو ما شُغّل `first_10_opportunities` → ابدأ به. +- لو الأولوية `fill_pipeline` → `meeting_booking_sprint`. +- لو `rescue_lost_revenue` → `revenue_leak_rescue`. +- لو `expand_partners` → `partnership_sprint`. +- الافتراضي: `customer_reactivation`. + +## 6. Weekly Curator Report + +`build_weekly_curator_report(messages, playbooks, missions, sector)` يُرجع: + +```json +{ + "summary_ar": [ + "تمت مراجعة 24 رسالة، 5 playbook، و2 مهمة هذا الأسبوع.", + "تم اقتراح أرشفة 4 رسالة ضعيفة الجودة.", + "تم اكتشاف 3 أزواج رسائل متشابهة (للدمج).", + ], + "messages": {"total", "publishable", "needs_edit", "to_archive", "duplicate_pairs"}, + "playbooks": {"total", "winners", "promising", "to_merge_groups"}, + "missions": {"total", "ship_it_widely", "iterate", "rework_or_retire"}, + "next_playbook": {"recommended_id", "title_ar", "reason_ar"}, + "recommended_next_action_ar": "..." +} +``` + +## 7. Endpoints + +``` +GET /api/v1/growth-curator/skills/inventory +POST /api/v1/growth-curator/messages/grade +POST /api/v1/growth-curator/messages/improve +POST /api/v1/growth-curator/messages/duplicates +POST /api/v1/growth-curator/missions/next +POST /api/v1/growth-curator/report/weekly +GET /api/v1/growth-curator/report/demo +``` + +## 8. حدود + +- لا يصدر LLM call. +- لا يحذف playbooks تلقائياً — يقترح فقط. +- لا يدمج بدون موافقة. +- التقرير يبقى actionable: ≤7 أسطر summary. diff --git a/dealix/docs/MEETING_INTELLIGENCE.md b/dealix/docs/MEETING_INTELLIGENCE.md new file mode 100644 index 00000000..78a40bf0 --- /dev/null +++ b/dealix/docs/MEETING_INTELLIGENCE.md @@ -0,0 +1,94 @@ +# Meeting Intelligence — ذكاء الاجتماعات + +> Pre-meeting brief + transcript summary + objection extraction + post-meeting follow-up + deal risk. كله Arabic، deterministic، approval-required. + +## 1. الوحدات + +| الوحدة | الدور | +|--------|------| +| `transcript_parser` | يقبل Google Meet entries أو نصاً عادياً، يحوّل إلى `speaker_turns`. | +| `meeting_brief` | يبني pre-meeting brief بـ6 أقسام عربية: هدف، أسئلة، اعتراضات محتملة، عرض، خطوة تالية. | +| `objection_extractor` | يستخرج 8 فئات اعتراضات (السعر، التوقيت، صانع القرار، الأمان، التكامل، البديل، إثبات النتائج، التعقيد). | +| `followup_builder` | يبني drafts للـemail + WhatsApp بدون إرسال حي. | +| `deal_risk` | يحسب risk_score (0..100) بناءً على الاعتراضات وغياب صاحب القرار وعدم تحديد خطوة تالية. | + +## 2. Pre-Meeting Brief + +`build_pre_meeting_brief(company, contact, opportunity, sector)` يعطي: +- objective_ar +- 5 questions_ar +- 5 likely_objections_ar +- offer_skeleton_ar (Pilot 7 أيام، 499 ريال) +- next_step_ar + +## 3. Transcript Summarizer + +`parse_transcript_entries` يدعم: +- list of `{participantId, text}` (Google Meet shape) +- plain text "Speaker: line" + +`summarize_meeting(parsed)` يعطي ملخصاً عربياً + أسئلة مرشحة + `approval_required=True`. + +Google Meet API يدعم قراءة transcripts عبر `conferenceRecords.transcripts.entries.list` — لكن يلزم موافقة كل المشاركين (PDPL). + +## 4. Objection Extractor + +8 فئات regex + Arabic gloss: +``` +price → "غالي|مرتفع|الميزانية|expensive|cost" +timing → "ليس\s+الآن|بعد\s+شهر|next\s+quarter" +authority → "المدير|صاحب\s+القرار|need\s+approval" +trust → "بيانات|خصوصية|أمان|PDPL|security|privacy" +integration → "CRM|نظامنا|الربط|migration" +competitor → "نستخدم|بديل|أداة\s+ثانية|alternative" +results → "نتائج|مضمون|guarantee|ROI|دليل" +complexity → "معقد|صعب|تدريب|onboarding" +``` + +النتيجة: قائمة `{category, label_ar, snippet}` مع snippet ±40 حرف. + +## 5. Follow-up Builder + +`build_post_meeting_followup(summary, next_steps, contact_name, company_name, objections)`: +- Email draft (subject_ar + body_ar) +- WhatsApp draft (body_ar) +- كلا الـdraftsْ: `live_send_allowed=False, approval_required=True` + +عندما تكون فيه objections، يضيف فقرة "رجعت بعد الاجتماع وفكرت في النقاط التي ذكرتها: ..." + +## 6. Deal Risk + +``` ++20 price objection ++15 timing objection ++25 authority not present ++20 trust/security objection ++10 integration concern ++15 competitor in play ++25 next step NOT set ++10 decision maker absent ++10 days_since_last_touch > 14 +``` + +تيرز: ≥70 high, ≥40 medium, <40 low. + +`recommended_action_ar`: +- high: اجتماع ثانٍ مع صاحب القرار خلال 5 أيام + مادة إثبات قيمة قصيرة. +- medium: متابعة خلال 3 أيام مع خطوة تالية محددة. +- low: نفّذ الخطوة التالية المتفق عليها. + +## 7. Endpoints + +``` +POST /api/v1/meeting-intelligence/brief +GET /api/v1/meeting-intelligence/brief/demo +POST /api/v1/meeting-intelligence/transcript/summarize +POST /api/v1/meeting-intelligence/followup/draft +POST /api/v1/meeting-intelligence/deal-risk +``` + +## 8. حدود + +- لا realtime listening (مرحلة لاحقة). +- لا يرسل follow-up — drafts فقط. +- لا يقرأ transcript بدون موافقة جميع الأطراف. diff --git a/dealix/docs/MODEL_PROVIDER_ROUTER.md b/dealix/docs/MODEL_PROVIDER_ROUTER.md new file mode 100644 index 00000000..20bc451b --- /dev/null +++ b/dealix/docs/MODEL_PROVIDER_ROUTER.md @@ -0,0 +1,57 @@ +# Model Provider Router — موجّه النماذج + +> 7 providers، 10 task types. كل مهمة تذهب لمزود مناسب مع fallback chain. لا تعتمد على مزود واحد. + +## 1. Task Types + +``` +strategic_reasoning, arabic_copywriting, classification, +compliance_guardrail, meeting_analysis, vision_analysis, +extraction, summarization, coding_project_understanding, +low_cost_bulk +``` + +## 2. Providers + +| key | family | cost | latency | privacy | الاستخدام | +|-----|--------|------|---------|---------|----------| +| claude_sonnet | anthropic | mid | balanced | vendor | استراتيجية + كتابة عربية + امتثال | +| claude_haiku | anthropic | low | fast | vendor | تصنيف + استخراج كثيف | +| gpt_4_class | openai | high | balanced | vendor | استراتيجية + رؤية | +| gpt_4o_mini | openai | low | fast | vendor | تصنيف رخيص | +| gemini_pro | google | mid | balanced | vendor | اجتماعات + رؤية | +| azure_oai_ksa | azure | mid | balanced | **ksa_region** | الحالات الحساسة (PDPL) | +| local_qwen_ar | local | low | balanced | **self_hosted** | حالات شديدة الحساسية | + +## 3. Cost Policy + +``` +bulk=True → low +output_tokens > 1500 → high +input_tokens > 8000 → high +strategic/vision/compl. → mid +arabic_copywriting → mid +default → low +``` + +## 4. Fallback Strategy + +- لو `sensitivity="high"`: الترتيب حسب `privacy_tier` أولاً (self_hosted > ksa_region > vendor). +- وإلا: الترتيب حسب `cost_class` (low > mid > high). +- لو `primary_provider` محدد ويدعم المهمة → يُرفع لرأس السلسلة. + +## 5. Endpoints + +``` +GET /api/v1/model-router/providers +GET /api/v1/model-router/tasks +POST /api/v1/model-router/route +POST /api/v1/model-router/cost-class +GET /api/v1/model-router/usage/demo +``` + +## 6. حدود + +- Router نفسه لا يستدعي LLM. يصدر قراراً فقط. +- التنفيذ الفعلي يبقى مسؤولية adapter منفصل. +- لا lock-in — جميع المزودين قابلون للاستبدال بدون تغيير API. diff --git a/dealix/docs/PRIVATE_BETA_LAUNCH_TODAY.md b/dealix/docs/PRIVATE_BETA_LAUNCH_TODAY.md new file mode 100644 index 00000000..6103f933 --- /dev/null +++ b/dealix/docs/PRIVATE_BETA_LAUNCH_TODAY.md @@ -0,0 +1,110 @@ +# Private Beta Launch — Today's Plan + +> **القرار:** ندشّن **Private Beta** اليوم، ليس Public Launch. +> **العرض الأساسي:** "10 فرص في 10 دقائق" + Pilot 7 أيام + Proof Pack. + +--- + +## 1. العرض + +``` +Pilot 7 أيام: 499 ريال أو مجاني مقابل case study +- 10 فرص B2B مع why-now +- 10 رسائل عربية جاهزة +- contactability + سياسة عدم الإرسال البارد +- متابعة لمدة 7 أيام +- Proof Pack بعد الأسبوع + +Paid Pilot 30 يوم: 1,500–3,000 ريال +Growth OS اشتراك شهري: 2,999 ريال +``` + +## 2. من نستهدف اليوم (أول 20) + +1. وكالات تسويق B2B سعودية. +2. مستشارون نمو. +3. شركات تدريب B2B. +4. SaaS سعودية صغيرة-متوسطة. +5. شركات عقار/خدمات لديها واتساب نشط. +6. أصدقاء مؤسسين سعوديين. + +## 3. Demo Flow (12 دقيقة) + +راجع [`DEMO_SCRIPT_12_MINUTES.md`](DEMO_SCRIPT_12_MINUTES.md). + +## 4. شروط القبول للعميل + +العميل المثالي للـPrivate Beta: +- شركة سعودية أو خليجية B2B. +- لديها ≥3 موظفين مبيعات أو نمو. +- مرتاحة بالعربي + الإنجليزي. +- مستعدة لإعطاء feedback أسبوعي. +- تقبل أنه draft-first (لا live send افتراضياً). + +## 5. ما يعمل الآن (Phase 1 ready) + +- `/api/v1/personal-operator/daily-brief` +- `/api/v1/growth-operator/missions` +- `/api/v1/growth-operator/proof-pack/demo` +- `/api/v1/intelligence/command-feed/demo` +- `/api/v1/intelligence/missions` + `/missions/recommend` +- `/api/v1/intelligence/simulate-opportunity` +- `/api/v1/intelligence/board-brief/demo` +- `/api/v1/platform/services/catalog` +- `/api/v1/platform/inbox/feed` +- `/api/v1/platform/proof-ledger/demo` +- `/api/v1/security-curator/demo` +- `/api/v1/growth-curator/report/demo` +- `/api/v1/meeting-intelligence/brief/demo` +- `/api/v1/connector-catalog/catalog` + +## 6. ما يبقى Draft فقط + +- WhatsApp send (live flag OFF). +- Gmail send (live flag OFF). +- Calendar insert (live flag OFF). +- Moyasar charge (live flag OFF — invoice/link manual). +- Social DMs. + +## 7. المخاطر + +- WhatsApp: PDPL — لا cold بدون opt-in. +- Gmail: SPF/DKIM/DMARC للـdomain. +- Moyasar: live keys ممنوعة في staging. +- Secrets: GitHub Push Protection + Patch Firewall + Trace Redactor. +- Hallucinations: Saudi Tone + Safety evals قبل publish. + +## 8. Go-Checklist قبل أول demo + +- [ ] CI أخضر (`Dealix API CI`). +- [ ] `pytest -q` ≥663 passed. +- [ ] Staging URL يستجيب على `/health`. +- [ ] جميع env flags الـ`*_ALLOW_LIVE_*` = false. +- [ ] `secret_redactor` و`patch_firewall` يعملان. +- [ ] Demo URL مفتوح على `/api/v1/intelligence/command-feed/demo`. +- [ ] Demo URL مفتوح على `/api/v1/growth-operator/proof-pack/demo`. +- [ ] رسالة DM 1 جاهزة (راجع `FIRST_20_OUTREACH_MESSAGES.md`). + +## 9. Post-Demo Checklist + +- [ ] قائمة 5 demos مجدولة. +- [ ] قائمة 3 pilots محتملين. +- [ ] feedback مكتوب لكل demo. +- [ ] Proof Pack template جاهز. +- [ ] أول Moyasar invoice draft (إن وُجد عميل). + +## 10. ما لا نفعله اليوم + +- لا public launch. +- لا live WhatsApp send. +- لا charges. +- لا "نضمن نتائج". +- لا scraping. +- لا إصدار صحفي. + +## 11. الخطوة التالية بعد أول 3 demos + +- استخراج الاعتراضات الموحدة → cards في `command_feed`. +- معايرة الأسعار (هل 499 رخيص جداً؟). +- اختيار vertical: تدريب أو وكالات أو SaaS. +- تجهيز case study واحد على الأقل. diff --git a/dealix/landing/private-beta.html b/dealix/landing/private-beta.html new file mode 100644 index 00000000..89e2d4f6 --- /dev/null +++ b/dealix/landing/private-beta.html @@ -0,0 +1,258 @@ + + + + + +Dealix — Private Beta + + + + +
+ Private Beta — متاح اليوم +

مدير نمو عربي للشركات السعودية

+

+ Dealix يعطيك 10 فرص B2B خلال 10 دقائق، يكتب الرسائل بالعربي، + ويطلع لك Proof Pack — وأنت توافق قبل أي تواصل. +

+ احجز Pilot الآن + شاهد ديمو 12 دقيقة +
+ +
+
+

وعد المنتج

+

خلال 7 أيام، نطلع لك:

+
    +
  • 10 فرص B2B مناسبة لقطاعك ومدينتك
    • +
  • سبب "لماذا الآن" لكل فرصة
    • +
  • رسائل عربية جاهزة (تحت 80 كلمة، نبرة طبيعية سعودية)
    • +
  • تصنيف الأرقام: آمن / يحتاج مراجعة / محظور
    • +
  • متابعة 7 أيام بعد القرار
    • +
  • Proof Pack: leads، رسائل معتمدة، اجتماعات، إيراد متأثر، مخاطر تم منعها
    • +
+
+ +
+

الأسعار

+
+
+
Pilot 7 أيام
+
499 ريال
+
أو مجاني مقابل case study
+
+
+
Paid Pilot 30 يوم
+
1,500–3,000 ريال
+
إعداد + Pilot موسّع
+
+
+
Growth OS شهري
+
2,999 ريال
+
اشتراك مستمر
+
+
+
+ +
+

الفرق عن المنافسين

+
    +
  • عربي طبيعي سعودي — لا تحية طيبة وبعد، لا synergy.
    • +
  • Approval-first — لا إرسال واتساب/إيميل بدون موافقتك.
    • +
  • PDPL-aware — لا cold WhatsApp بدون opt-in.
    • +
  • Multi-channel — واتساب + إيميل + تقويم + SOCIAL + مدفوعات تحت سقف واحد.
    • +
  • Self-improving — يتعلم من Accept/Skip/Edit ويحسّن الرسائل.
    • +
  • Proof Pack — تقرير شهري بكل ما عمله Dealix.
    • +
+
+ +
+

ماذا يعمل اليوم؟

+

كل هذه الـendpoints تعمل في staging الآن — يمكنك تجربتها مباشرة:

+

+ GET /api/v1/personal-operator/daily-brief
+ GET /api/v1/intelligence/command-feed/demo
+ GET /api/v1/intelligence/missions
+ POST /api/v1/intelligence/simulate-opportunity
+ GET /api/v1/intelligence/board-brief/demo
+ GET /api/v1/growth-operator/proof-pack/demo
+ GET /api/v1/platform/proof-ledger/demo
+ GET /api/v1/connector-catalog/catalog
+ GET /api/v1/security-curator/demo
+ GET /api/v1/growth-curator/report/demo
+ GET /api/v1/meeting-intelligence/brief/demo
+

+
+ +
+

الأمان والامتثال

+
+ القاعدة الذهبية: لا إرسال خارجي بدون موافقتك. + لا cold WhatsApp. لا charge بدون تأكيد. لا تخزّن بياناتك خارج المملكة بدون اتفاقية. +
+
    +
  • Tool Gateway: كل أداة (Gmail/Calendar/WhatsApp/Moyasar) تمر من سياسة قبل التنفيذ.
    • +
  • Patch Firewall: ما من سر يدخل الكود.
    • +
  • Trace Redactor: ما من رقم/إيميل يدخل الـlogs.
    • +
  • Saudi Tone Eval: كل رسالة تُقيّم قبل الإرسال.
    • +
  • Safety Eval: عبارات مثل "ضمان 100%" محظورة تلقائياً.
    • +
+
+ +
+

للوكالات

+

+ إذا أنت وكالة تسويق B2B، Dealix يشتغل خلفك: + أنت تختار العميل، Dealix يطلع له 10 فرص + رسائل + متابعة، + وأنت تحصل على case study + revenue share. + ابحث عن "Partner Sprint" في Pilot المجاني. +

+
+ +
+

جاهز نبدأ؟

+

+ Pilot يبدأ يوم الأحد التالي. + أرسل لي اسمك + اسم شركتك + قطاعك + مدينتك، + وأرتّب لك ديمو 12 دقيقة هذا الأسبوع. +

+ احجز Pilot الآن +
+
+ + + + diff --git a/dealix/tests/unit/test_agent_observability.py b/dealix/tests/unit/test_agent_observability.py new file mode 100644 index 00000000..cc786927 --- /dev/null +++ b/dealix/tests/unit/test_agent_observability.py @@ -0,0 +1,94 @@ +"""Unit tests for Agent Observability.""" + +from __future__ import annotations + +from auto_client_acquisition.agent_observability import ( + CostTracker, + build_trace_event, + run_eval_pack, + safety_eval, + saudi_tone_eval, +) + + +# ── Trace events ───────────────────────────────────────────── +def test_trace_event_hashes_user_id(): + e = build_trace_event( + workflow_name="first_10", agent_name="scout", + user_id="user_real_42", company_id="acme", + ) + assert e["user_id_hash"] != "user_real_42" + assert e["company_id_hash"] != "acme" + assert len(str(e["user_id_hash"])) == 16 + + +def test_trace_event_redacts_payload_secrets(): + e = build_trace_event( + workflow_name="x", agent_name="y", + payload={"token": "ghp_AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA1234"}, + ) + assert "ghp_AAAA" not in str(e.get("payload")) + + +# ── Safety eval ────────────────────────────────────────────── +def test_safety_blocks_guarantee(): + out = safety_eval("ضمان 100% نتائج مضمونة") + assert out["verdict"] in ("blocked", "needs_review") + + +def test_safety_safe_for_clean_text(): + out = safety_eval("هلا أحمد، لاحظت توسعكم. يناسبك أعرض لك Pilot؟") + assert out["verdict"] == "safe" + assert out["score"] >= 70 + + +def test_safety_blocks_medical_claim(): + """Medical claims should at minimum require human review (and ideally block).""" + out = safety_eval("هذا المنتج يعالج السكر ويشفي الضغط بدون أدوية.") + assert out["verdict"] in ("blocked", "needs_review") + assert any(v["category"] == "medical_claim" for v in out["violations"]) + + +# ── Saudi tone eval ────────────────────────────────────────── +def test_tone_natural_for_friendly_arabic(): + text = ("هلا أحمد، لاحظت توسعكم في فريق المبيعات. " + "يناسبك أعرض لك Pilot 7 أيام؟") + out = saudi_tone_eval(text) + assert out["verdict"] in ("natural", "decent") + assert out["arabic_ratio"] > 0.5 + + +def test_tone_off_for_too_corporate(): + text = "تحية طيبة وبعد، ندعوكم لاكتشاف synergy و best-in-class." + out = saudi_tone_eval(text) + assert out["verdict"] == "off" + + +def test_tone_off_for_empty(): + out = saudi_tone_eval("") + assert out["verdict"] == "off" + + +# ── Eval pack ──────────────────────────────────────────────── +def test_eval_pack_runs_all_cases(): + out = run_eval_pack() + assert out["total"] >= 5 + assert "pass_rate" in out + + +def test_eval_pack_has_some_passing(): + out = run_eval_pack() + assert out["passed"] >= 1 + + +# ── Cost tracker ──────────────────────────────────────────── +def test_cost_tracker_aggregates(): + t = CostTracker() + t.record(workflow_name="first_10", provider_key="claude_sonnet", + task_type="strategic_reasoning", cost_estimate=0.025) + t.record(workflow_name="first_10", provider_key="claude_haiku", + task_type="classification", cost_estimate=0.001) + s = t.summary() + assert s["runs"] == 2 + assert round(s["total"], 4) == 0.026 + assert s["by_workflow"]["first_10"] > 0 diff --git a/dealix/tests/unit/test_connector_catalog.py b/dealix/tests/unit/test_connector_catalog.py new file mode 100644 index 00000000..59455ec2 --- /dev/null +++ b/dealix/tests/unit/test_connector_catalog.py @@ -0,0 +1,82 @@ +"""Unit tests for the Connector Catalog.""" + +from __future__ import annotations + +from auto_client_acquisition.connector_catalog import ( + ALL_CONNECTORS, + all_risks, + catalog_summary, + connector_risks, + connector_status, + get_connector, + list_connectors, +) + + +def test_catalog_has_at_least_12_connectors(): + out = list_connectors() + assert out["total"] >= 12 + + +def test_catalog_includes_critical_connectors(): + keys = {c.key for c in ALL_CONNECTORS} + for required in ( + "whatsapp_cloud", "gmail", "google_calendar", "moyasar", + "linkedin_lead_forms", "google_business_profile", + "x_api", "instagram_graph", "google_sheets", + "crm_generic", "website_forms", "google_meet", + ): + assert required in keys + + +def test_every_connector_has_risk_level(): + for c in ALL_CONNECTORS: + assert c.risk_level in ("low", "medium", "high") + + +def test_every_connector_has_blocked_or_safe_actions(): + for c in ALL_CONNECTORS: + assert isinstance(c.allowed_actions, tuple) + assert isinstance(c.blocked_actions, tuple) + + +def test_whatsapp_blocks_cold_send(): + wa = get_connector("whatsapp_cloud") + assert wa is not None + assert "cold_send_without_consent" in wa.blocked_actions + + +def test_moyasar_blocks_card_storage(): + m = get_connector("moyasar") + assert m is not None + assert "store_card_number" in m.blocked_actions + + +def test_summary_aggregates(): + s = catalog_summary() + assert s["total"] == len(ALL_CONNECTORS) + assert "by_launch_phase" in s + assert "by_risk_level" in s + + +def test_status_returns_safe_default_modes(): + out = connector_status() + for entry in out["statuses"]: + assert entry["mode"] in ("connected_draft_only", "not_connected", + "connected_live_with_approval") + + +def test_risks_present_for_high_risk_connectors(): + for key in ("whatsapp_cloud", "moyasar", "google_meet"): + risks = connector_risks(key) + assert risks, f"missing risks for {key}" + + +def test_unknown_connector_has_no_risks(): + assert connector_risks("totally_unknown") == [] + + +def test_all_risks_keyed_by_connector(): + out = all_risks() + for c in ALL_CONNECTORS: + assert c.key in out diff --git a/dealix/tests/unit/test_dealix_model_router.py b/dealix/tests/unit/test_dealix_model_router.py new file mode 100644 index 00000000..437aa68e --- /dev/null +++ b/dealix/tests/unit/test_dealix_model_router.py @@ -0,0 +1,76 @@ +"""Unit tests for the new auto_client_acquisition.model_router.""" + +from __future__ import annotations + +from auto_client_acquisition.model_router import ( + ALL_PROVIDERS, + ALL_TASK_TYPES, + build_fallback_chain, + build_usage_demo, + classify_cost, + get_provider, + route_task, +) + + +def test_every_task_type_has_at_least_one_provider(): + for tt in ALL_TASK_TYPES: + chain = build_fallback_chain(tt) + assert chain, f"no provider for task: {tt}" + + +def test_provider_registry_contains_essentials(): + keys = {p.key for p in ALL_PROVIDERS} + for required in ("claude_sonnet", "claude_haiku", "gpt_4_class", + "gemini_pro", "azure_oai_ksa"): + assert required in keys + + +def test_get_provider_unknown(): + assert get_provider("bogus_key") is None + + +def test_classify_cost_bulk_is_low(): + assert classify_cost(task_type="low_cost_bulk", bulk=True) == "low" + + +def test_classify_cost_strategic_is_mid(): + assert classify_cost(task_type="strategic_reasoning") == "mid" + + +def test_classify_cost_huge_output_is_high(): + assert classify_cost(task_type="summarization", + expected_output_tokens=2000) == "high" + + +def test_high_sensitivity_prefers_ksa_or_local(): + chain = build_fallback_chain( + "compliance_guardrail", sensitivity="high", requires_arabic=True, + ) + top_provider = get_provider(chain[0]) + assert top_provider is not None + assert top_provider.privacy_tier in ("ksa_region", "self_hosted") + + +def test_route_task_unknown_returns_no_provider(): + d = route_task("totally_made_up") + assert d.primary_provider is None + assert d.fallback_chain == [] + + +def test_route_task_arabic_copywriting(): + d = route_task("arabic_copywriting", requires_arabic=True) + assert d.primary_provider is not None + assert d.cost_class in ("low", "mid", "high") + + +def test_route_task_with_primary_override(): + d = route_task("strategic_reasoning", primary_provider="gpt_4_class") + assert d.fallback_chain[0] == "gpt_4_class" + + +def test_usage_demo_covers_all_task_types(): + demo = build_usage_demo() + assert demo["task_types_total"] == len(ALL_TASK_TYPES) + assert len(demo["routes"]) == len(ALL_TASK_TYPES) + assert demo["cost_counts"] diff --git a/dealix/tests/unit/test_growth_curator.py b/dealix/tests/unit/test_growth_curator.py new file mode 100644 index 00000000..2512231c --- /dev/null +++ b/dealix/tests/unit/test_growth_curator.py @@ -0,0 +1,155 @@ +"""Unit tests for the Growth Curator.""" + +from __future__ import annotations + +from auto_client_acquisition.growth_curator import ( + build_weekly_curator_report, + detect_duplicates, + grade_message, + inventory_skills, + recommend_next_mission, + recommend_next_playbook, + score_mission, + score_playbook, + suggest_improvement, +) + + +# ── Skill Inventory ────────────────────────────────────────── +def test_inventory_lists_kill_feature(): + out = inventory_skills() + assert out["total"] >= 20 + kill_ids = [s["id"] for s in out["kill_features"]] + assert "first_10_opportunities" in kill_ids + + +def test_inventory_layers_present(): + out = inventory_skills() + layers = set(out["layers"]) + assert {"platform_services", "intelligence_layer", + "growth_curator", "security_curator"}.issubset(layers) + + +# ── Message Curator ────────────────────────────────────────── +def test_grades_natural_arabic_message_high(): + text = ("هلا أحمد، لاحظت توسعكم في فريق المبيعات. " + "نشتغل على Dealix كمدير نمو عربي. " + "يناسبك أعرض لك مثال 10 دقائق هذا الأسبوع؟") + g = grade_message(text, sector="training") + assert g.score >= 60 + assert g.verdict in ("publish", "needs_edit") + + +def test_blocks_risky_phrases(): + text = "آخر فرصة! ضمان 100% نتائج مضمونة. اضغط الآن." + g = grade_message(text) + assert g.risky_phrases + assert g.verdict in ("needs_edit", "reject") + + +def test_rejects_non_arabic(): + text = "Hello there, just checking in. Cheers." + g = grade_message(text) + assert g.verdict == "reject" + + +def test_detects_near_duplicates(): + msgs = [ + "هلا أحمد، لاحظت توسعكم. يناسبك أعرض لك Pilot؟", + "هلا محمد، لاحظت توسعكم. يناسبك أعرض لك Pilot؟", + "totally unrelated message in english", + ] + pairs = detect_duplicates(msgs, threshold=0.8) + assert any({i, j} == {0, 1} for i, j, _r in pairs) + + +def test_suggest_improvement_returns_skeleton(): + out = suggest_improvement("Hi") + assert "suggested_skeleton_ar" in out + assert "هلا" in out["suggested_skeleton_ar"] + + +# ── Playbook Curator ──────────────────────────────────────── +def test_score_playbook_winner_tier(): + """Strong outcomes across all signals should push into winner/promising.""" + pb = { + "used_count": 100, "accept_count": 90, + "replied_count": 80, "meeting_count": 60, "deal_count": 40, + } + s = score_playbook(pb) + assert s["score"] >= 50 + assert s["tier"] in ("winner", "promising") + + +def test_score_playbook_needs_work_tier(): + """Modest outcomes should map to needs_work.""" + pb = { + "used_count": 100, "accept_count": 60, + "replied_count": 40, "meeting_count": 20, "deal_count": 8, + } + s = score_playbook(pb) + assert s["tier"] in ("needs_work", "promising") + + +def test_score_playbook_unproven_for_zero_uses(): + s = score_playbook({"used_count": 0}) + assert s["tier"] == "unproven" + assert s["score"] == 0 + + +def test_recommend_next_playbook_default_when_empty(): + rec = recommend_next_playbook([]) + assert rec["recommended_id"] == "default_warm_outreach" + + +def test_recommend_next_playbook_picks_promising_first(): + pbs = [ + {"id": "p1", "title": "Winner", "tier": "winner", "score": 80}, + {"id": "p2", "title": "Promising", "tier": "promising", "score": 60}, + ] + rec = recommend_next_playbook(pbs) + assert rec["recommended_id"] == "p2" + + +# ── Mission Curator ───────────────────────────────────────── +def test_score_mission_ship_it_with_strong_outcome(): + out = score_mission({ + "opportunities_generated": 10, + "drafts_approved": 5, + "meetings_booked": 3, + "revenue_influenced_sar": 60_000, + "time_to_value_minutes": 8, + "risks_blocked": 2, + }) + assert out["score"] >= 70 + assert out["verdict"] == "ship_it_widely" + + +def test_recommend_next_mission_starts_with_kill_feature(): + rec = recommend_next_mission(None) + assert rec["recommended_mission_id"] == "first_10_opportunities" + + +def test_recommend_next_mission_after_kill_feature(): + history = [{"mission_id": "first_10_opportunities"}] + rec = recommend_next_mission(history, growth_brain={ + "growth_priorities": ["fill_pipeline"], + }) + assert rec["recommended_mission_id"] == "meeting_booking_sprint" + + +# ── Curator Report ─────────────────────────────────────────── +def test_weekly_report_handles_empty_input(): + rep = build_weekly_curator_report() + assert rep["messages"]["total"] == 0 + assert rep["playbooks"]["total"] == 0 + assert rep["missions"]["total"] == 0 + assert rep["next_playbook"]["recommended_id"] + + +def test_weekly_report_marks_low_quality_for_archive(): + rep = build_weekly_curator_report(messages=[ + {"id": "m1", "text": "Hi"}, + {"id": "m2", "text": "آخر فرصة! ضمان 100% نتائج مضمونة!"}, + ]) + assert rep["messages"]["to_archive"] >= 1 diff --git a/dealix/tests/unit/test_meeting_intelligence.py b/dealix/tests/unit/test_meeting_intelligence.py new file mode 100644 index 00000000..c6516fd3 --- /dev/null +++ b/dealix/tests/unit/test_meeting_intelligence.py @@ -0,0 +1,120 @@ +"""Unit tests for Meeting Intelligence.""" + +from __future__ import annotations + +from auto_client_acquisition.meeting_intelligence import ( + build_post_meeting_followup, + build_pre_meeting_brief, + compute_deal_risk, + extract_objections, + parse_transcript_entries, + summarize_meeting, +) + + +# ── Transcript Parser ─────────────────────────────────────── +def test_parser_handles_meet_entries(): + entries = [ + {"participantId": "alice", "text": "ما رأيكم في السعر؟"}, + {"participantId": "bob", "text": "السعر مرتفع لنا الآن."}, + ] + p = parse_transcript_entries(entries) + assert p["total_turns"] == 2 + assert "alice" in p["speakers"] + + +def test_parser_handles_plain_text(): + text = "Alice: مرحباً\nBob: السعر مرتفع لنا" + p = parse_transcript_entries(text) + assert p["total_turns"] == 2 + + +def test_summarize_returns_arabic_summary(): + parsed = parse_transcript_entries([ + {"participantId": "a", "text": "نحتاج أن نفهم نموذج التسعير بشكل أوضح."}, + {"participantId": "b", "text": "ممتاز، أقترح اجتماع ثاني الأسبوع القادم."}, + ]) + s = summarize_meeting(parsed) + assert s["approval_required"] is True + assert any("اجتماع" in line or "نقاش" in line for line in s["summary_ar"]) + + +# ── Brief ─────────────────────────────────────────────────── +def test_brief_returns_six_sections(): + b = build_pre_meeting_brief( + company={"name": "Acme", "sector": "saas"}, + contact={"name": "أحمد", "role": "VP"}, + opportunity={"expected_value_sar": 25_000}, + ) + assert b["company_name"] == "Acme" + assert len(b["questions_ar"]) >= 5 + assert len(b["likely_objections_ar"]) >= 5 + assert b["approval_required"] is True + + +def test_brief_works_with_empty_input(): + b = build_pre_meeting_brief() + assert b["company_name"] == "?" + assert b["questions_ar"] + + +# ── Objection Extractor ───────────────────────────────────── +def test_extracts_price_objection(): + out = extract_objections("هذا الحل غالي ولا يناسب الميزانية.") + cats = out["categories_found"] + assert "price" in cats + + +def test_extracts_authority_objection(): + out = extract_objections("نحتاج موافقة المدير قبل أي قرار.") + assert "authority" in out["categories_found"] + + +def test_no_objection_in_clean_text(): + out = extract_objections("اجتماع رائع، نتطلع للخطوة القادمة.") + assert out["count"] == 0 + + +# ── Followup Builder ──────────────────────────────────────── +def test_followup_returns_email_and_whatsapp_drafts(): + out = build_post_meeting_followup( + next_steps=["إرسال عرض السعر", "تحديد اجتماع ثانٍ"], + contact_name="أحمد", + company_name="Acme", + ) + assert "email" in out["channel_drafts"] + assert "whatsapp" in out["channel_drafts"] + assert out["channel_drafts"]["email"]["live_send_allowed"] is False + assert "أحمد" in out["channel_drafts"]["email"]["body_ar"] + + +def test_followup_addresses_objections(): + out = build_post_meeting_followup( + next_steps=["متابعة"], + contact_name="سارة", + objections=[{"label_ar": "السعر/الميزانية"}, + {"label_ar": "صاحب القرار"}], + ) + assert "السعر" in out["channel_drafts"]["email"]["body_ar"] + assert out["objections_addressed"] + + +# ── Deal Risk ─────────────────────────────────────────────── +def test_high_risk_when_no_next_step_and_authority_objection(): + out = compute_deal_risk( + objections=[{"category": "authority"}, {"category": "price"}], + next_step_set=False, + decision_maker_present=False, + ) + assert out["risk_score"] >= 50 + assert out["risk_level"] in ("medium", "high") + + +def test_low_risk_with_clean_meeting(): + out = compute_deal_risk( + objections=[], + next_step_set=True, + decision_maker_present=True, + days_since_last_touch=0, + ) + assert out["risk_level"] == "low" diff --git a/dealix/tests/unit/test_security_curator.py b/dealix/tests/unit/test_security_curator.py new file mode 100644 index 00000000..347f776b --- /dev/null +++ b/dealix/tests/unit/test_security_curator.py @@ -0,0 +1,132 @@ +"""Unit tests for the Security Curator.""" + +from __future__ import annotations + +from auto_client_acquisition.security_curator import ( + detect_secret_patterns, + inspect_diff, + is_safe_diff, + redact_secrets, + redact_trace, + sanitize_tool_output, + sanitize_trace_event, + scan_payload, +) + + +# ── Secret Redactor ────────────────────────────────────────── +def test_detects_github_pat(): + text = "my token is ghp_AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA1234" + findings = detect_secret_patterns(text) + assert any(f.label == "github_pat" for f in findings) + assert all("ghp_AAAA" not in f.sample_redacted for f in findings) # never raw + + +def test_redacts_openai_key(): + text = "key=sk-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA1234 done" + out = redact_secrets(text) + assert "sk-AAAA" not in out + assert "sk-***" in out + + +def test_redacts_anthropic_key(): + text = "ANTHROPIC=sk-ant-aBcDeFgHiJkLmNoPqRsTuVwXyZ1234" + out = redact_secrets(text) + assert "sk-ant-aBcD" not in out + + +def test_scan_payload_dict_redacts_sensitive_keys(): + payload = {"api_key": "ghp_AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA1234", + "name": "ali"} + result = scan_payload(payload) + assert result["has_secrets"] is True + assert result["redacted"]["api_key"] == "***" + assert result["redacted"]["name"] == "ali" + + +def test_scan_payload_handles_nested(): + payload = {"outer": {"token": "EAA" + "x" * 40, "ok": "yes"}} + result = scan_payload(payload) + assert result["has_secrets"] is True + assert result["redacted"]["outer"]["token"] == "***" + + +def test_scan_empty_returns_no_findings(): + out = scan_payload({}) + assert out["has_secrets"] is False + assert out["findings"] == [] + + +# ── Patch Firewall ─────────────────────────────────────────── +def test_blocks_env_file_diff(): + diff = """diff --git a/.env b/.env +new file mode 100644 ++++ b/.env ++API_KEY=ghp_AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA1234 +""" + r = inspect_diff(diff) + assert r.safe is False + assert any(".env" in f for f in r.blocked_files) + + +def test_blocks_secret_in_added_line(): + diff = """+++ b/src/foo.py ++OPENAI_KEY = "sk-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA1234" +""" + r = inspect_diff(diff) + assert r.safe is False + assert r.secret_findings + + +def test_allows_safe_diff(): + diff = """+++ b/src/foo.py ++def hello(): ++ return "world" +""" + assert is_safe_diff(diff) is True + + +def test_empty_diff_is_safe(): + assert is_safe_diff("") is True + + +# ── Trace Redactor ─────────────────────────────────────────── +def test_trace_masks_phone_and_email(): + payload = {"note": "call +966500000123 or email ali@example.com"} + out = redact_trace(payload, mask_pii=True) + assert out["had_pii"] is True + masked = out["redacted"]["note"] + assert "+966500000123" not in masked + assert "ali@example.com" not in masked + assert "@example.com" in masked # domain preserved + + +def test_trace_redacts_secrets_and_pii_together(): + payload = {"token": "ghp_AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA1234", + "phone": "+966500000999"} + out = redact_trace(payload) + assert out["had_secrets"] is True + assert out["redacted"]["token"] == "***" + + +# ── Tool Output Sanitizer ──────────────────────────────────── +def test_sanitize_output_strips_secret(): + output = {"raw": "ghp_AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA1234 inside"} + out = sanitize_tool_output(output) + assert out["safe"] is False + assert "ghp_AAAA" not in str(out["redacted"]) + assert any("حساسة" in n for n in out["notes_ar"]) + + +def test_sanitize_trace_event_keeps_safe_keys(): + event = { + "event_type": "tool_call", "agent_name": "scout", + "status": "ok", "latency_ms": 120, + "payload": {"phone": "+966500000123"}, + } + out = sanitize_trace_event(event) + assert out["event_type"] == "tool_call" + assert out["agent_name"] == "scout" + assert out["latency_ms"] == 120 + # payload was sanitized + assert "+966500000123" not in str(out["payload"])