From a329957a3b0b4a1f2cab1bdc59c537f86eeeaefe Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 11 Apr 2026 07:40:39 +0000 Subject: [PATCH] feat: Add AI engine, PDPL compliance, sequences, CPQ, and governance layers Phase 1-6 implementation for Dealix AI Revenue OS: - AI Arabic Engine: NLP (arabic_nlp.py), lead scoring (lead_scoring.py) - PDPL Compliance: consent manager, data rights handler, consent model - Sequence Engine: multi-channel sequences with WhatsApp/Email/SMS - CPQ System: quote engine, AI proposal generator - Security Gate: pre-release checks, PDPL message validation - Tool Verification: agent action audit trail - Project Operating Files: AGENTS.md, CLAUDE.md - Project Memory: architecture, ADRs, provider routing, PDPL checklist - Design System: IBM Plex Sans Arabic tokens, RTL-safe components - Sequence/Consent models for database https://claude.ai/code/session_01LsnvBa7HwF5hs99VZbgLGj --- salesflow-saas/AGENTS.md | 122 ++++ salesflow-saas/CLAUDE.md | 55 ++ salesflow-saas/backend/app/models/consent.py | 113 ++++ salesflow-saas/backend/app/models/sequence.py | 107 ++++ .../backend/app/services/ai/__init__.py | 27 + .../backend/app/services/ai/arabic_nlp.py | 418 +++++++++++++ .../backend/app/services/ai/lead_scoring.py | 571 ++++++++++++++++++ .../backend/app/services/cpq/__init__.py | 15 + .../app/services/cpq/proposal_generator.py | 227 +++++++ .../backend/app/services/cpq/quote_engine.py | 321 ++++++++++ .../backend/app/services/pdpl/__init__.py | 6 + .../app/services/pdpl/consent_manager.py | 301 +++++++++ .../backend/app/services/pdpl/data_rights.py | 352 +++++++++++ .../backend/app/services/security_gate.py | 208 +++++++ .../backend/app/services/sequence_engine.py | 357 +++++++++++ .../backend/app/services/tool_verification.py | 176 ++++++ salesflow-saas/frontend/src/design-tokens.ts | 202 +++++++ salesflow-saas/memory/adr/001-multi-tenant.md | 23 + .../memory/adr/002-whatsapp-first.md | 28 + .../memory/architecture/system-overview.md | 52 ++ .../memory/providers/routing-strategy.md | 36 ++ .../memory/security/pdpl-checklist.md | 46 ++ 22 files changed, 3763 insertions(+) create mode 100644 salesflow-saas/AGENTS.md create mode 100644 salesflow-saas/CLAUDE.md create mode 100644 salesflow-saas/backend/app/models/consent.py create mode 100644 salesflow-saas/backend/app/models/sequence.py create mode 100644 salesflow-saas/backend/app/services/ai/__init__.py create mode 100644 salesflow-saas/backend/app/services/ai/arabic_nlp.py create mode 100644 salesflow-saas/backend/app/services/ai/lead_scoring.py create mode 100644 salesflow-saas/backend/app/services/cpq/__init__.py create mode 100644 salesflow-saas/backend/app/services/cpq/proposal_generator.py create mode 100644 salesflow-saas/backend/app/services/cpq/quote_engine.py create mode 100644 salesflow-saas/backend/app/services/pdpl/__init__.py create mode 100644 salesflow-saas/backend/app/services/pdpl/consent_manager.py create mode 100644 salesflow-saas/backend/app/services/pdpl/data_rights.py create mode 100644 salesflow-saas/backend/app/services/security_gate.py create mode 100644 salesflow-saas/backend/app/services/sequence_engine.py create mode 100644 salesflow-saas/backend/app/services/tool_verification.py create mode 100644 salesflow-saas/frontend/src/design-tokens.ts create mode 100644 salesflow-saas/memory/adr/001-multi-tenant.md create mode 100644 salesflow-saas/memory/adr/002-whatsapp-first.md create mode 100644 salesflow-saas/memory/architecture/system-overview.md create mode 100644 salesflow-saas/memory/providers/routing-strategy.md create mode 100644 salesflow-saas/memory/security/pdpl-checklist.md diff --git a/salesflow-saas/AGENTS.md b/salesflow-saas/AGENTS.md new file mode 100644 index 00000000..c373d6fa --- /dev/null +++ b/salesflow-saas/AGENTS.md @@ -0,0 +1,122 @@ +# AGENTS.md — Dealix AI Revenue OS + +## Project Identity +- **Name**: Dealix (ديلكس) +- **Type**: AI-Powered CRM SaaS for Saudi Arabia +- **Stack**: FastAPI + Next.js 15 + PostgreSQL + Redis + Celery +- **Market**: Saudi SMBs (real estate, healthcare, retail, contracting, education) +- **Language**: Arabic-first, bilingual (AR/EN) + +## Architecture Boundaries + +### Backend (`salesflow-saas/backend/`) +- FastAPI 0.115.6 on Python 3.12 +- SQLAlchemy 2.0 async with PostgreSQL 16 +- Celery 5.x with Redis broker +- JWT authentication (python-jose) +- Multi-tenant data isolation via `tenant_id` + +### Frontend (`salesflow-saas/frontend/`) +- Next.js 15 with App Router +- TypeScript 5.7, Tailwind CSS 3.4 +- RTL-first layout (dir="rtl") +- Fonts: IBM Plex Sans Arabic (primary), Tajawal (secondary) + +### AI Layer (`backend/app/services/ai/`) +- LLM Provider: Groq (primary) → OpenAI (fallback) +- Arabic NLP with Saudi dialect support +- Model routing via `services/model_router.py` + +### Agent System (`backend/app/services/agents/`) +- Manus-style orchestrator with 8 specialized roles +- Event-to-agent routing via `router.py` +- Executor with retry logic and escalation + +## Coding Conventions +- Python: async/await, type hints, Pydantic models, 4-space indent +- TypeScript: strict mode, functional components, Tailwind classes +- Database: all queries through SQLAlchemy ORM, never raw SQL +- API: RESTful, versioned (/api/v1/), proper HTTP status codes +- Naming: snake_case (Python), camelCase (TypeScript) +- Arabic: all user-facing strings must have Arabic versions +- Currency: SAR default, Numeric type for money fields +- Timezone: Asia/Riyadh (UTC+3) + +## Forbidden Actions +- Never hardcode API keys or secrets +- Never bypass tenant isolation +- Never send messages without PDPL consent check +- Never delete data without soft-delete first +- Never push directly to main branch +- Never skip security review for auth/payment changes +- Never use synchronous DB calls in async endpoints +- Never store PII in logs + +## Policy Classes + +### Class A — Auto-allowed +- Code reading and inspection +- Test generation and execution +- Documentation updates +- Memory/knowledge base updates +- Linting and formatting +- Architecture analysis + +### Class B — Approval Required +- Database migrations +- Customer-facing message sends +- Payment/billing changes +- Permission model changes +- External API integrations +- Production deployments +- PDPL consent configuration changes + +### Class C — Forbidden +- Secret exfiltration +- Bypassing branch protections +- Silent destructive changes +- Disabling security gates +- Cross-tenant data access +- Ungoverned bulk messaging + +## How to Install +```bash +cd salesflow-saas +cp .env.example .env # Configure your environment +docker-compose up -d +make migrate +make seed +``` + +## How to Test +```bash +cd salesflow-saas/backend +pytest -v +# Or with coverage +pytest --cov=app --cov-report=html +``` + +## How to Run +```bash +docker-compose up # All services +# Or individually: +cd backend && uvicorn app.main:app --reload --port 8000 +cd frontend && npm run dev +``` + +## Provider Preferences +1. **Fast classification**: Groq (llama-3.1-70b) +2. **Arabic NLP**: Groq with Arabic context prompts +3. **Sales copy/proposals**: Claude (via model_router) +4. **Research/analysis**: Gemini (via model_router) +5. **Coding tasks**: DeepSeek (via model_router) +6. **Fallback**: OpenAI GPT-4o-mini + +## Release Process +1. Feature branch → PR → Code review +2. Run tests + security scan +3. Deploy to staging +4. Smoke test (Arabic + English) +5. Deploy to production with canary (10%) +6. Monitor 30 min → full rollout +7. Rollback plan documented per release diff --git a/salesflow-saas/CLAUDE.md b/salesflow-saas/CLAUDE.md new file mode 100644 index 00000000..37c02926 --- /dev/null +++ b/salesflow-saas/CLAUDE.md @@ -0,0 +1,55 @@ +# CLAUDE.md — Dealix Project Context for AI Agents + +## Quick Context +Dealix is an AI-powered CRM built for the Saudi market. It combines Salesforce-grade AI with WhatsApp-first communication, PDPL compliance, and Arabic-first UX. + +## Key Directories +- `backend/app/api/v1/` — API routes (FastAPI) +- `backend/app/models/` — SQLAlchemy models +- `backend/app/services/` — Business logic layer +- `backend/app/services/ai/` — AI engine (Arabic NLP, scoring, forecasting) +- `backend/app/services/pdpl/` — PDPL compliance engine +- `backend/app/services/cpq/` — Configure, Price, Quote +- `backend/app/services/agents/` — Multi-agent orchestration +- `backend/app/services/llm/` — LLM provider abstraction +- `backend/app/workers/` — Celery async tasks +- `backend/app/integrations/` — WhatsApp, Email, SMS adapters +- `frontend/src/app/` — Next.js pages +- `seeds/` — Industry templates (JSON) +- `memory/` — Project knowledge base + +## Database +- PostgreSQL 16 with async driver (asyncpg) +- Multi-tenant: every table has `tenant_id` +- Alembic for migrations +- Money fields use `Numeric` type (never Float) + +## AI Architecture +- Provider abstraction: Groq → OpenAI fallback +- Model router: task-specific model selection +- Arabic NLP: intent, sentiment, entity extraction +- Lead scoring: 0-100 composite score +- Conversation intelligence: Arabic dialogue analysis +- Sales agent: autonomous WhatsApp qualification bot + +## PDPL Compliance (Critical) +- Check consent before ANY outbound message +- Track consent purpose, channel, timestamp +- Support data subject rights (access, correct, delete) +- Audit trail for all consent changes +- Auto-expire consent after 12 months +- Penalty: up to SAR 5 million per violation + +## Testing +```bash +pytest -v # All tests +pytest tests/test_ai/ -v # AI engine tests +pytest tests/test_pdpl/ -v # PDPL compliance tests +pytest tests/test_api/ -v # API endpoint tests +``` + +## Common Tasks +- Add new API endpoint: create route in `api/v1/`, register in `main.py` +- Add new model: create in `models/`, add to `models/__init__.py`, create migration +- Add new AI feature: create in `services/ai/`, wire to relevant API/worker +- Add industry template: create JSON in `seeds/`, match existing schema diff --git a/salesflow-saas/backend/app/models/consent.py b/salesflow-saas/backend/app/models/consent.py new file mode 100644 index 00000000..0ba8909d --- /dev/null +++ b/salesflow-saas/backend/app/models/consent.py @@ -0,0 +1,113 @@ +"""PDPL consent and data request models for Dealix CRM.""" + +import enum +from sqlalchemy import Column, String, DateTime, ForeignKey, Index, Text +from sqlalchemy.orm import relationship +from datetime import datetime, timezone + +from app.models.base import TenantModel +from app.models.compat import UUID, JSONB + + +class ConsentPurpose(str, enum.Enum): + MARKETING = "marketing" + SALES = "sales" + SERVICE = "service" + ANALYTICS = "analytics" + + +class ConsentChannel(str, enum.Enum): + WHATSAPP = "whatsapp" + EMAIL = "email" + SMS = "sms" + PHONE = "phone" + + +class ConsentStatusEnum(str, enum.Enum): + GRANTED = "granted" + REVOKED = "revoked" + EXPIRED = "expired" + + +class DataRequestType(str, enum.Enum): + ACCESS = "access" + CORRECTION = "correction" + DELETION = "deletion" + RESTRICTION = "restriction" + + +class DataRequestStatus(str, enum.Enum): + PENDING = "pending" + PROCESSING = "processing" + COMPLETED = "completed" + REJECTED = "rejected" + + +class PDPLConsent(TenantModel): + """Tracks PDPL consent per contact, purpose, and channel.""" + + __tablename__ = "pdpl_consents" + __table_args__ = ( + Index("ix_pdpl_consent_contact_purpose", "contact_id", "purpose"), + Index("ix_pdpl_consent_tenant_status", "tenant_id", "status"), + Index("ix_pdpl_consent_expires", "expires_at"), + ) + + contact_id = Column(UUID(as_uuid=True), ForeignKey("leads.id"), nullable=False, index=True) + purpose = Column(String(50), nullable=False) + channel = Column(String(50), nullable=False) + status = Column(String(50), nullable=False, default=ConsentStatusEnum.GRANTED.value) + granted_at = Column(DateTime(timezone=True), default=lambda: datetime.now(timezone.utc)) + revoked_at = Column(DateTime(timezone=True), nullable=True) + expires_at = Column(DateTime(timezone=True), nullable=False) + ip_address = Column(String(45), nullable=True) + consent_text = Column(Text, nullable=True) + granted_by = Column(UUID(as_uuid=True), ForeignKey("users.id"), nullable=True) + + contact = relationship("Lead", foreign_keys=[contact_id]) + granting_user = relationship("User", foreign_keys=[granted_by]) + + +class PDPLConsentAudit(TenantModel): + """Immutable audit trail for every consent change.""" + + __tablename__ = "pdpl_consent_audit" + __table_args__ = ( + Index("ix_pdpl_audit_consent", "consent_id"), + Index("ix_pdpl_audit_contact", "contact_id"), + ) + + consent_id = Column(UUID(as_uuid=True), ForeignKey("pdpl_consents.id"), nullable=False) + contact_id = Column(UUID(as_uuid=True), ForeignKey("leads.id"), nullable=False) + action = Column(String(50), nullable=False) # granted, revoked, expired, renewed + actor_id = Column(UUID(as_uuid=True), ForeignKey("users.id"), nullable=True) + channel = Column(String(50), nullable=False) + purpose = Column(String(50), nullable=False) + details = Column(JSONB, default=dict) + ip_address = Column(String(45), nullable=True) + + consent = relationship("PDPLConsent", foreign_keys=[consent_id]) + contact = relationship("Lead", foreign_keys=[contact_id]) + actor = relationship("User", foreign_keys=[actor_id]) + + +class DataRequest(TenantModel): + """Data subject rights requests under PDPL.""" + + __tablename__ = "pdpl_data_requests" + __table_args__ = ( + Index("ix_pdpl_dr_contact", "contact_id"), + Index("ix_pdpl_dr_status", "status"), + ) + + contact_id = Column(UUID(as_uuid=True), ForeignKey("leads.id"), nullable=False, index=True) + request_type = Column(String(50), nullable=False) + status = Column(String(50), nullable=False, default=DataRequestStatus.PENDING.value) + requested_at = Column(DateTime(timezone=True), default=lambda: datetime.now(timezone.utc)) + completed_at = Column(DateTime(timezone=True), nullable=True) + response_data = Column(JSONB, default=dict) + handled_by = Column(UUID(as_uuid=True), ForeignKey("users.id"), nullable=True) + notes = Column(Text, nullable=True) + + contact = relationship("Lead", foreign_keys=[contact_id]) + handler = relationship("User", foreign_keys=[handled_by]) diff --git a/salesflow-saas/backend/app/models/sequence.py b/salesflow-saas/backend/app/models/sequence.py new file mode 100644 index 00000000..382482c6 --- /dev/null +++ b/salesflow-saas/backend/app/models/sequence.py @@ -0,0 +1,107 @@ +"""Sequence engine models for multi-channel outreach.""" + +import enum +from sqlalchemy import Column, String, Integer, Boolean, DateTime, ForeignKey, Index, Text +from sqlalchemy.orm import relationship +from datetime import datetime, timezone + +from app.models.base import TenantModel, BaseModel +from app.models.compat import UUID, JSONB + + +class SequenceStatus(str, enum.Enum): + ACTIVE = "active" + PAUSED = "paused" + COMPLETED = "completed" + STOPPED = "stopped" + + +class SequenceEventStatus(str, enum.Enum): + SENT = "sent" + DELIVERED = "delivered" + OPENED = "opened" + REPLIED = "replied" + FAILED = "failed" + + +class Sequence(TenantModel): + """A reusable multi-step outreach sequence.""" + + __tablename__ = "sequences" + __table_args__ = ( + Index("ix_seq_tenant_active", "tenant_id", "is_active"), + ) + + name = Column(String(255), nullable=False) + name_ar = Column(String(255), nullable=True) + description = Column(Text, nullable=True) + trigger_event = Column(String(100), nullable=True) # lead_created, stage_changed, etc. + is_active = Column(Boolean, default=True) + created_by = Column(UUID(as_uuid=True), ForeignKey("users.id"), nullable=False) + + steps = relationship("SequenceStep", back_populates="sequence", order_by="SequenceStep.step_order") + enrollments = relationship("SequenceEnrollment", back_populates="sequence") + creator = relationship("User", foreign_keys=[created_by]) + + +class SequenceStep(BaseModel): + """A single step within a sequence.""" + + __tablename__ = "sequence_steps" + __table_args__ = ( + Index("ix_step_sequence_order", "sequence_id", "step_order"), + ) + + sequence_id = Column(UUID(as_uuid=True), ForeignKey("sequences.id", ondelete="CASCADE"), nullable=False) + step_order = Column(Integer, nullable=False) + channel = Column(String(50), nullable=False) # whatsapp, email, sms + delay_minutes = Column(Integer, nullable=False, default=0) + template_content = Column(Text, nullable=False) + template_content_ar = Column(Text, nullable=True) + variant = Column(String(1), nullable=True) # A or B for A/B testing + conditions = Column(JSONB, default=dict) # e.g. {"only_if_no_reply": true} + + sequence = relationship("Sequence", back_populates="steps") + events = relationship("SequenceEvent", back_populates="step") + + +class SequenceEnrollment(BaseModel): + """Tracks a single lead's progress through a sequence.""" + + __tablename__ = "sequence_enrollments" + __table_args__ = ( + Index("ix_enroll_sequence_status", "sequence_id", "status"), + Index("ix_enroll_lead", "lead_id"), + ) + + sequence_id = Column(UUID(as_uuid=True), ForeignKey("sequences.id", ondelete="CASCADE"), nullable=False) + lead_id = Column(UUID(as_uuid=True), ForeignKey("leads.id"), nullable=False) + current_step = Column(Integer, default=0) + status = Column(String(50), nullable=False, default=SequenceStatus.ACTIVE.value) + enrolled_at = Column(DateTime(timezone=True), default=lambda: datetime.now(timezone.utc)) + completed_at = Column(DateTime(timezone=True), nullable=True) + next_step_at = Column(DateTime(timezone=True), nullable=True) + + sequence = relationship("Sequence", back_populates="enrollments") + lead = relationship("Lead", foreign_keys=[lead_id]) + events = relationship("SequenceEvent", back_populates="enrollment") + + +class SequenceEvent(BaseModel): + """Records every message sent or event within a sequence enrollment.""" + + __tablename__ = "sequence_events" + __table_args__ = ( + Index("ix_sevent_enrollment", "enrollment_id"), + Index("ix_sevent_step", "step_id"), + ) + + enrollment_id = Column(UUID(as_uuid=True), ForeignKey("sequence_enrollments.id", ondelete="CASCADE"), nullable=False) + step_id = Column(UUID(as_uuid=True), ForeignKey("sequence_steps.id"), nullable=False) + channel = Column(String(50), nullable=False) + status = Column(String(50), nullable=False, default=SequenceEventStatus.SENT.value) + sent_at = Column(DateTime(timezone=True), default=lambda: datetime.now(timezone.utc)) + metadata = Column(JSONB, default=dict) + + enrollment = relationship("SequenceEnrollment", back_populates="events") + step = relationship("SequenceStep", back_populates="events") diff --git a/salesflow-saas/backend/app/services/ai/__init__.py b/salesflow-saas/backend/app/services/ai/__init__.py new file mode 100644 index 00000000..280b1004 --- /dev/null +++ b/salesflow-saas/backend/app/services/ai/__init__.py @@ -0,0 +1,27 @@ +""" +Dealix AI Engine — Arabic-first AI services for Saudi CRM. +Provides NLP, lead scoring, conversation intelligence, message writing, and forecasting. +""" + +from app.services.ai.arabic_nlp import ArabicNLPService, LanguageDetection, IntentResult, EntityResult, SentimentResult +from app.services.ai.lead_scoring import LeadScoringEngine, LeadScoreResult, ScoreBreakdown +from app.services.ai.conversation_intelligence import ConversationIntelligence, ConversationInsight +from app.services.ai.message_writer import MessageWriter, MessageDraft +from app.services.ai.forecasting import SalesForecastingEngine, ForecastResult + +__all__ = [ + "ArabicNLPService", + "LanguageDetection", + "IntentResult", + "EntityResult", + "SentimentResult", + "LeadScoringEngine", + "LeadScoreResult", + "ScoreBreakdown", + "ConversationIntelligence", + "ConversationInsight", + "MessageWriter", + "MessageDraft", + "SalesForecastingEngine", + "ForecastResult", +] diff --git a/salesflow-saas/backend/app/services/ai/arabic_nlp.py b/salesflow-saas/backend/app/services/ai/arabic_nlp.py new file mode 100644 index 00000000..8c0ccf95 --- /dev/null +++ b/salesflow-saas/backend/app/services/ai/arabic_nlp.py @@ -0,0 +1,418 @@ +""" +Arabic NLP Service — Language detection, intent extraction, entity recognition, +and sentiment analysis optimized for Saudi Arabic dialect. +Uses regex for fast pattern matching and LLM for complex analysis. +""" + +import re +import json +import logging +from dataclasses import dataclass, field +from typing import Optional + +from app.services.llm.provider import get_llm + +logger = logging.getLogger(__name__) + +# --------------------------------------------------------------------------- +# Data models +# --------------------------------------------------------------------------- + +@dataclass +class LanguageDetection: + language: str # "ar", "en", "mixed" + confidence: float # 0.0 - 1.0 + is_saudi_dialect: bool + dialect_markers_found: list[str] = field(default_factory=list) + region_hint: Optional[str] = None # "najdi", "hijazi", "sharqawi" + + +@dataclass +class IntentResult: + intent: str # buying_intent, pricing_inquiry, appointment_request, complaint, general_inquiry + confidence: float + sub_intent: Optional[str] = None + raw_signals: list[str] = field(default_factory=list) + + +@dataclass +class EntityResult: + names: list[str] = field(default_factory=list) + phone_numbers: list[str] = field(default_factory=list) + dates: list[str] = field(default_factory=list) + amounts: list[dict] = field(default_factory=list) # {"value": 5000, "currency": "SAR", "raw": "..."} + locations: list[str] = field(default_factory=list) + + +@dataclass +class SentimentResult: + sentiment: str # "positive", "neutral", "negative" + confidence: float + emotional_tone: Optional[str] = None # "satisfied", "frustrated", "eager", "hesitant" + key_phrases: list[str] = field(default_factory=list) + + +# --------------------------------------------------------------------------- +# Constants +# --------------------------------------------------------------------------- + +ARABIC_RANGE = re.compile(r"[\u0600-\u06FF\u0750-\u077F\uFB50-\uFDFF\uFE70-\uFEFF]") + +SAUDI_DIALECT_MARKERS = [ + "وش", "ايش", "كيف حالك", "يعطيك العافية", "هلا والله", "كذا", + "خلاص", "يا رجال", "وش لونك", "مب", "ماعندي", "الحين", "دحين", + "تمام", "يا زين", "شلونك", "أبي", "أبغى", "يا بوي", "كيفك", + "عطني", "خلني", "ماشي", "يالله", "بس", "حلو", "ياخي", "زين", +] + +REGIONAL_MARKERS = { + "najdi": ["ايش", "كذا", "يا رجال", "وش لونك", "الحين", "أبي", "أبغى"], + "hijazi": ["كده", "ليش", "يا زين", "دحين", "عايز", "ابغى"], + "sharqawi": ["شلونك", "هاي", "بعد", "يا بوي", "اشلون"], +} + +INTENT_PATTERNS = { + "buying_intent": [ + r"أبي\s*أشتري", r"أبغى\s*أطلب", r"أبي\s*عرض\s*سعر", r"نبي\s*ن[شس]تري", + r"ودي\s*آخذ", r"interested", r"want to buy", r"أبي\s*أعرف\s*السعر", + r"جاهز\s*أشتري", r"أبي\s*أكمل\s*الطلب", r"نبي\s*نبدأ", + ], + "pricing_inquiry": [ + r"كم\s*السعر", r"كم\s*سعر", r"بكم", r"وش\s*[اأ]سعار", r"أسعاركم", + r"how much", r"price", r"عرض\s*سعر", r"كم\s*تكلفة", r"كم\s*ريال", + r"عندكم\s*عرض", r"فيه\s*خصم", r"أرخص\s*سعر", + ], + "appointment_request": [ + r"أبي\s*موعد", r"نبي\s*اجتماع", r"متى\s*فاضي", r"متى\s*تقدر", + r"نتقابل", r"ممكن\s*نتواصل", r"schedule", r"meeting", + r"أبي\s*زيارة", r"ودي\s*أجي", r"متى\s*نتقابل", + ], + "complaint": [ + r"عندي\s*مشكلة", r"ما\s*يشتغل", r"خربان", r"مو\s*راضي", + r"أبي\s*أشتكي", r"complaint", r"problem", r"زعلان", + r"ما\s*عجبني", r"سيئ", r"خدمة\s*سيئة", r"ما\s*رديتوا", + ], +} + +SAUDI_CITIES = [ + "الرياض", "جدة", "مكة", "المدينة", "الدمام", "الخبر", "الظهران", + "الطائف", "تبوك", "بريدة", "عنيزة", "حائل", "جازان", "نجران", + "أبها", "خميس مشيط", "الجبيل", "ينبع", "المجمعة", "الأحساء", + "القطيف", "حفر الباطن", "سكاكا", "الباحة", "عرعر", + "Riyadh", "Jeddah", "Makkah", "Madinah", "Dammam", "Khobar", + "Dhahran", "Tabuk", "Abha", "Jubail", "Yanbu", "NEOM", +] + +PHONE_PATTERNS = [ + re.compile(r"(?:\+?966|00966|0)[\s-]?5\d[\s-]?\d{3}[\s-]?\d{4}"), + re.compile(r"05\d{8}"), + re.compile(r"\+9665\d{8}"), +] + +AMOUNT_PATTERN = re.compile( + r"(\d[\d,]*(?:\.\d{1,2})?)\s*(?:ريال|ر\.س|SAR|sar|ر\.س\.)" +) + +ARABIC_DATE_PATTERNS = [ + re.compile(r"\d{1,2}\s*/\s*\d{1,2}\s*/\s*\d{2,4}"), + re.compile(r"\d{4}-\d{2}-\d{2}"), + re.compile( + r"(?:يوم|بتاريخ|في)\s+" + r"(?:السبت|الأحد|الاثنين|الثلاثاء|الأربعاء|الخميس|الجمعة)" + r"(?:\s+\d{1,2})?" + ), + re.compile(r"(?:بعد\s+)?(?:أسبوع|شهر|يومين|ثلاث\s+أيام|باكر|بكرة|غداً)"), +] + +POSITIVE_MARKERS = [ + "ممتاز", "حلو", "رائع", "تمام", "أحسنت", "ماشاء الله", "يعطيك العافية", + "مشكور", "شكراً", "الله يعطيك", "زين", "عجبني", "مبسوط", "ممنون", + "great", "amazing", "thank", "good", "excellent", "love", +] + +NEGATIVE_MARKERS = [ + "غالي", "سيئ", "مو كويس", "ما عجبني", "زعلان", "مشكلة", "خربان", + "ما يشتغل", "خدمة سيئة", "مو راضي", "ما رديتوا", "بطيء", + "bad", "terrible", "worst", "hate", "angry", "disappointed", +] + + +# --------------------------------------------------------------------------- +# Service +# --------------------------------------------------------------------------- + +class ArabicNLPService: + """Arabic NLP processing with Saudi dialect support.""" + + def __init__(self): + self._llm = get_llm() + + # ── Language Detection ──────────────────────── + + async def detect_language(self, text: str) -> LanguageDetection: + """Detect whether text is Arabic, English, or mixed, plus Saudi dialect markers.""" + if not text or not text.strip(): + return LanguageDetection(language="en", confidence=0.0, is_saudi_dialect=False) + + arabic_chars = len(ARABIC_RANGE.findall(text)) + total_alpha = sum(1 for c in text if c.isalpha()) + if total_alpha == 0: + return LanguageDetection(language="en", confidence=0.5, is_saudi_dialect=False) + + arabic_ratio = arabic_chars / total_alpha + + # Find Saudi dialect markers + text_lower = text.lower() + found_markers = [m for m in SAUDI_DIALECT_MARKERS if m in text_lower or m in text] + is_saudi = len(found_markers) > 0 + + # Detect region + region_hint = None + if is_saudi: + region_scores = {} + for region, markers in REGIONAL_MARKERS.items(): + region_scores[region] = sum(1 for m in markers if m in text_lower or m in text) + best_region = max(region_scores, key=region_scores.get) + if region_scores[best_region] > 0: + region_hint = best_region + + if arabic_ratio > 0.7: + lang = "ar" + confidence = min(arabic_ratio + 0.1, 1.0) + elif arabic_ratio > 0.3: + lang = "mixed" + confidence = 0.7 + else: + lang = "en" + confidence = min((1 - arabic_ratio) + 0.1, 1.0) + + return LanguageDetection( + language=lang, + confidence=round(confidence, 2), + is_saudi_dialect=is_saudi, + dialect_markers_found=found_markers, + region_hint=region_hint, + ) + + # ── Intent Detection ───────────────────────── + + async def extract_intent(self, text: str) -> IntentResult: + """Extract user intent from Arabic or English message.""" + if not text or not text.strip(): + return IntentResult(intent="general_inquiry", confidence=0.3) + + # Fast regex pass + regex_result = self._regex_intent(text) + if regex_result and regex_result.confidence >= 0.8: + return regex_result + + # LLM for ambiguous or complex text + try: + llm_result = await self._llm_intent(text) + if regex_result: + # Combine: prefer LLM intent but boost confidence if regex agrees + if llm_result.intent == regex_result.intent: + llm_result.confidence = min(llm_result.confidence + 0.15, 1.0) + llm_result.raw_signals.extend(regex_result.raw_signals) + return llm_result + except Exception as e: + logger.warning(f"LLM intent extraction failed: {e}") + return regex_result or IntentResult(intent="general_inquiry", confidence=0.3) + + def _regex_intent(self, text: str) -> Optional[IntentResult]: + """Fast regex-based intent detection.""" + best_intent = None + best_score = 0 + best_signals = [] + + for intent, patterns in INTENT_PATTERNS.items(): + signals = [] + for pattern in patterns: + matches = re.findall(pattern, text, re.IGNORECASE) + if matches: + signals.extend(matches) + if len(signals) > best_score: + best_score = len(signals) + best_intent = intent + best_signals = signals + + if best_intent and best_score > 0: + confidence = min(0.6 + (best_score * 0.1), 0.95) + return IntentResult( + intent=best_intent, + confidence=round(confidence, 2), + raw_signals=best_signals[:5], + ) + return None + + async def _llm_intent(self, text: str) -> IntentResult: + """LLM-based intent detection for complex text.""" + system_prompt = ( + "أنت محلل نوايا العملاء في نظام CRM سعودي. " + "حلل الرسالة التالية واستخرج النية الأساسية.\n" + "أجب بصيغة JSON فقط بالشكل التالي:\n" + '{"intent": "buying_intent|pricing_inquiry|appointment_request|complaint|general_inquiry", ' + '"confidence": 0.0-1.0, "sub_intent": "وصف مختصر بالعربي أو null", ' + '"signals": ["إشارة1", "إشارة2"]}' + ) + response = await self._llm.complete( + system_prompt=system_prompt, + user_message=text, + json_mode=True, + temperature=0.1, + max_tokens=256, + fast=True, + ) + parsed = response.parse_json() + if parsed: + return IntentResult( + intent=parsed.get("intent", "general_inquiry"), + confidence=float(parsed.get("confidence", 0.5)), + sub_intent=parsed.get("sub_intent"), + raw_signals=parsed.get("signals", []), + ) + return IntentResult(intent="general_inquiry", confidence=0.4) + + # ── Entity Extraction ──────────────────────── + + async def extract_entities(self, text: str) -> EntityResult: + """Extract names, phones, dates, amounts, locations from Arabic/English text.""" + result = EntityResult() + + # Phone numbers (regex) + for pattern in PHONE_PATTERNS: + phones = pattern.findall(text) + result.phone_numbers.extend( + [re.sub(r"[\s-]", "", p) for p in phones] + ) + result.phone_numbers = list(set(result.phone_numbers)) + + # Amounts in SAR (regex) + for match in AMOUNT_PATTERN.finditer(text): + raw_value = match.group(1).replace(",", "") + try: + result.amounts.append({ + "value": float(raw_value), + "currency": "SAR", + "raw": match.group(0), + }) + except ValueError: + pass + + # Dates (regex) + for pattern in ARABIC_DATE_PATTERNS: + dates = pattern.findall(text) + result.dates.extend(dates) + result.dates = list(set(result.dates)) + + # Saudi cities (string match) + for city in SAUDI_CITIES: + if city in text: + result.locations.append(city) + result.locations = list(set(result.locations)) + + # Names via LLM (hard to do with regex for Arabic) + try: + names = await self._extract_names_llm(text) + result.names = names + except Exception as e: + logger.warning(f"LLM name extraction failed: {e}") + + return result + + async def _extract_names_llm(self, text: str) -> list[str]: + """Use LLM to extract person names from Arabic text.""" + system_prompt = ( + "استخرج أسماء الأشخاص فقط من النص التالي. " + "أجب بصيغة JSON: {\"names\": [\"اسم1\", \"اسم2\"]}. " + "إذا لم يوجد أسماء أرجع قائمة فارغة." + ) + response = await self._llm.complete( + system_prompt=system_prompt, + user_message=text, + json_mode=True, + temperature=0.0, + max_tokens=128, + fast=True, + ) + parsed = response.parse_json() + if parsed and "names" in parsed: + return parsed["names"] + return [] + + # ── Sentiment Analysis ─────────────────────── + + async def analyze_sentiment(self, text: str) -> SentimentResult: + """Analyze sentiment in Arabic/English text with Saudi cultural awareness.""" + if not text or not text.strip(): + return SentimentResult(sentiment="neutral", confidence=0.5) + + # Fast regex pass + regex_result = self._regex_sentiment(text) + if regex_result.confidence >= 0.85: + return regex_result + + # LLM for nuanced analysis + try: + return await self._llm_sentiment(text, regex_result) + except Exception as e: + logger.warning(f"LLM sentiment analysis failed: {e}") + return regex_result + + def _regex_sentiment(self, text: str) -> SentimentResult: + """Fast regex sentiment scoring.""" + pos_count = sum(1 for m in POSITIVE_MARKERS if m in text) + neg_count = sum(1 for m in NEGATIVE_MARKERS if m in text) + matched_phrases = ( + [m for m in POSITIVE_MARKERS if m in text] + + [m for m in NEGATIVE_MARKERS if m in text] + ) + + if pos_count > neg_count: + sentiment = "positive" + confidence = min(0.5 + (pos_count - neg_count) * 0.1, 0.95) + elif neg_count > pos_count: + sentiment = "negative" + confidence = min(0.5 + (neg_count - pos_count) * 0.1, 0.95) + else: + sentiment = "neutral" + confidence = 0.5 + + return SentimentResult( + sentiment=sentiment, + confidence=round(confidence, 2), + key_phrases=matched_phrases[:5], + ) + + async def _llm_sentiment(self, text: str, regex_hint: SentimentResult) -> SentimentResult: + """LLM sentiment analysis with Saudi cultural context.""" + system_prompt = ( + "أنت محلل مشاعر متخصص في اللهجة السعودية والثقافة السعودية.\n" + "حلل المشاعر في النص التالي مع مراعاة:\n" + '- "يعطيك العافية" = إيجابي/شكر\n' + '- "ان شاء الله" بدون تفاصيل = قد يكون تردد\n' + '- "خلني أفكر" = حياد/تردد\n' + '- "ماشي" = موافقة خفيفة\n' + "أجب بصيغة JSON:\n" + '{"sentiment": "positive|neutral|negative", ' + '"confidence": 0.0-1.0, ' + '"emotional_tone": "satisfied|frustrated|eager|hesitant|neutral", ' + '"key_phrases": ["عبارة1", "عبارة2"]}' + ) + response = await self._llm.complete( + system_prompt=system_prompt, + user_message=text, + json_mode=True, + temperature=0.1, + max_tokens=256, + fast=True, + ) + parsed = response.parse_json() + if parsed: + return SentimentResult( + sentiment=parsed.get("sentiment", regex_hint.sentiment), + confidence=float(parsed.get("confidence", regex_hint.confidence)), + emotional_tone=parsed.get("emotional_tone"), + key_phrases=parsed.get("key_phrases", regex_hint.key_phrases), + ) + return regex_hint diff --git a/salesflow-saas/backend/app/services/ai/lead_scoring.py b/salesflow-saas/backend/app/services/ai/lead_scoring.py new file mode 100644 index 00000000..02777b49 --- /dev/null +++ b/salesflow-saas/backend/app/services/ai/lead_scoring.py @@ -0,0 +1,571 @@ +""" +AI Lead Scoring Engine — Calculates a 0-100 score for each lead +based on engagement, profile, behavior, and Arabic NLP intent analysis. +""" + +import logging +from dataclasses import dataclass, field +from datetime import datetime, timezone, timedelta +from typing import Optional + +from sqlalchemy import select, func +from sqlalchemy.ext.asyncio import AsyncSession + +from app.services.ai.arabic_nlp import ArabicNLPService + +logger = logging.getLogger(__name__) + +# --------------------------------------------------------------------------- +# Data models +# --------------------------------------------------------------------------- + +@dataclass +class ScoreBreakdown: + category: str + score: float + max_score: float + explanation_ar: str + explanation_en: str + factors: list[dict] = field(default_factory=list) + + +@dataclass +class LeadScoreResult: + lead_id: str + total_score: int # 0-100 + grade: str # A, B, C, D, F + breakdowns: list[ScoreBreakdown] = field(default_factory=list) + summary_ar: str = "" + summary_en: str = "" + recommended_action_ar: str = "" + last_scored_at: str = "" + + +# --------------------------------------------------------------------------- +# Scoring thresholds and weights +# --------------------------------------------------------------------------- + +GRADE_THRESHOLDS = {"A": 80, "B": 60, "C": 40, "D": 20, "F": 0} + +RESPONSE_SPEED_THRESHOLDS = { + "excellent": timedelta(minutes=15), + "good": timedelta(hours=1), + "average": timedelta(hours=4), + "slow": timedelta(hours=24), +} + +IMPORTANT_INDUSTRIES = [ + "real_estate", "healthcare", "retail", "ecommerce", "construction", + "education", "hospitality", "automotive", "fintech", "logistics", +] + + +# --------------------------------------------------------------------------- +# Service +# --------------------------------------------------------------------------- + +class LeadScoringEngine: + """Calculates AI-powered lead scores with Arabic NLP intent analysis.""" + + def __init__(self): + self._nlp = ArabicNLPService() + + async def score_lead( + self, lead_id: str, tenant_id: str, db: AsyncSession + ) -> LeadScoreResult: + """Calculate a comprehensive lead score (0-100).""" + now = datetime.now(timezone.utc) + + try: + lead_data = await self._fetch_lead_data(lead_id, tenant_id, db) + except Exception as e: + logger.error(f"Failed to fetch lead data for {lead_id}: {e}") + return LeadScoreResult( + lead_id=lead_id, + total_score=0, + grade="F", + summary_ar="تعذر حساب النتيجة - لم يتم العثور على بيانات العميل المحتمل", + summary_en="Score calculation failed - lead data not found", + last_scored_at=now.isoformat(), + ) + + engagement = await self._score_engagement(lead_data) + profile = self._score_profile(lead_data) + behavioral = self._score_behavioral(lead_data) + intent = await self._score_intent(lead_data) + + total = int( + engagement.score + profile.score + behavioral.score + intent.score + ) + total = max(0, min(100, total)) + grade = self._calculate_grade(total) + + summary_ar = self._build_arabic_summary(total, grade, engagement, profile, behavioral, intent) + summary_en = self._build_english_summary(total, grade) + action_ar = self._recommend_action_ar(total, grade, intent, engagement) + + result = LeadScoreResult( + lead_id=lead_id, + total_score=total, + grade=grade, + breakdowns=[engagement, profile, behavioral, intent], + summary_ar=summary_ar, + summary_en=summary_en, + recommended_action_ar=action_ar, + last_scored_at=now.isoformat(), + ) + + await self._persist_score(lead_id, tenant_id, result, db) + return result + + # ── Data Fetching ──────────────────────────── + + async def _fetch_lead_data(self, lead_id: str, tenant_id: str, db: AsyncSession) -> dict: + """Fetch all lead-related data for scoring.""" + from app.models.lead import Lead + + stmt = select(Lead).where(Lead.id == lead_id, Lead.tenant_id == tenant_id) + row = await db.execute(stmt) + lead = row.scalar_one_or_none() + if not lead: + raise ValueError(f"Lead {lead_id} not found") + + messages = await self._fetch_messages(lead_id, db) + activities = await self._fetch_activities(lead_id, db) + + return { + "lead": lead, + "full_name": getattr(lead, "full_name", "") or "", + "email": getattr(lead, "email", "") or "", + "phone": getattr(lead, "phone", "") or "", + "company_name": getattr(lead, "company_name", "") or "", + "sector": getattr(lead, "sector", "") or "", + "city": getattr(lead, "city", "") or "", + "source": getattr(lead, "source", "") or "", + "status": getattr(lead, "status", "new"), + "created_at": getattr(lead, "created_at", None), + "messages": messages, + "activities": activities, + } + + async def _fetch_messages(self, lead_id: str, db: AsyncSession) -> list[dict]: + """Fetch WhatsApp/email messages for a lead.""" + try: + from app.models.message import Message + stmt = ( + select(Message) + .where(Message.lead_id == lead_id) + .order_by(Message.created_at.desc()) + .limit(50) + ) + rows = await db.execute(stmt) + return [ + { + "content": getattr(m, "content", ""), + "channel": getattr(m, "channel", ""), + "direction": getattr(m, "direction", ""), + "created_at": getattr(m, "created_at", None), + } + for m in rows.scalars().all() + ] + except Exception: + logger.debug("Message model not available, skipping message fetch") + return [] + + async def _fetch_activities(self, lead_id: str, db: AsyncSession) -> list[dict]: + """Fetch activity log for a lead.""" + try: + from app.models.activity import Activity + stmt = ( + select(Activity) + .where(Activity.lead_id == lead_id) + .order_by(Activity.created_at.desc()) + .limit(100) + ) + rows = await db.execute(stmt) + return [ + { + "type": getattr(a, "type", ""), + "metadata": getattr(a, "metadata", {}), + "created_at": getattr(a, "created_at", None), + } + for a in rows.scalars().all() + ] + except Exception: + logger.debug("Activity model not available, skipping activity fetch") + return [] + + # ── Sub-Scores ─────────────────────────────── + + async def _score_engagement(self, data: dict) -> ScoreBreakdown: + """Engagement Score: WhatsApp messages, email opens, response speed. Max 30 pts.""" + factors = [] + score = 0.0 + + # WhatsApp message count (max 12 pts) + messages = data.get("messages", []) + wa_messages = [m for m in messages if m.get("channel") == "whatsapp"] + inbound_wa = [m for m in wa_messages if m.get("direction") == "inbound"] + wa_count = len(inbound_wa) + if wa_count >= 10: + wa_score = 12.0 + elif wa_count >= 5: + wa_score = 9.0 + elif wa_count >= 2: + wa_score = 6.0 + elif wa_count >= 1: + wa_score = 3.0 + else: + wa_score = 0.0 + score += wa_score + factors.append({ + "name": "رسائل واتساب", + "value": wa_count, + "points": wa_score, + "max": 12, + }) + + # Email engagement (max 9 pts) + email_msgs = [m for m in messages if m.get("channel") == "email"] + email_opens = len([ + a for a in data.get("activities", []) + if a.get("type") == "email_open" + ]) + email_score = min(email_opens * 1.5, 9.0) + if not email_opens and email_msgs: + email_score = 2.0 + score += email_score + factors.append({ + "name": "تفاعل البريد الإلكتروني", + "value": email_opens, + "points": email_score, + "max": 9, + }) + + # Response speed (max 9 pts) + response_score = self._calc_response_speed(messages) + score += response_score + factors.append({ + "name": "سرعة الرد", + "value": f"{response_score:.1f}/9", + "points": response_score, + "max": 9, + }) + + explanation_ar = f"مجموع التفاعل: {score:.0f} من 30 نقطة" + if wa_count == 0 and email_opens == 0: + explanation_ar += " — لم يتم رصد أي تفاعل بعد" + elif score >= 20: + explanation_ar += " — تفاعل ممتاز" + + return ScoreBreakdown( + category="engagement", + score=round(min(score, 30), 1), + max_score=30, + explanation_ar=explanation_ar, + explanation_en=f"Engagement: {score:.0f}/30", + factors=factors, + ) + + def _calc_response_speed(self, messages: list[dict]) -> float: + """Calculate response speed score from message timestamps.""" + inbound = [m for m in messages if m.get("direction") == "inbound"] + outbound = [m for m in messages if m.get("direction") == "outbound"] + if not inbound or not outbound: + return 3.0 # neutral default + + # Check last inbound -> next outbound gap + try: + last_inbound_time = inbound[0].get("created_at") + response_times = [] + for ob in outbound: + ob_time = ob.get("created_at") + if ob_time and last_inbound_time and ob_time > last_inbound_time: + continue + if ob_time and last_inbound_time: + gap = abs((last_inbound_time - ob_time).total_seconds()) + response_times.append(gap) + + if not response_times: + return 4.5 + + avg_gap_seconds = sum(response_times) / len(response_times) + if avg_gap_seconds < 900: # < 15 min + return 9.0 + elif avg_gap_seconds < 3600: # < 1 hr + return 7.0 + elif avg_gap_seconds < 14400: # < 4 hrs + return 5.0 + elif avg_gap_seconds < 86400: # < 24 hrs + return 3.0 + else: + return 1.0 + except Exception: + return 3.0 + + def _score_profile(self, data: dict) -> ScoreBreakdown: + """Profile Score: Contact completeness, company info, industry match. Max 25 pts.""" + factors = [] + score = 0.0 + + # Contact completeness (max 10 pts) + completeness = 0 + if data.get("full_name"): + completeness += 2.5 + if data.get("email"): + completeness += 2.5 + if data.get("phone"): + completeness += 2.5 + if data.get("city"): + completeness += 2.5 + score += completeness + factors.append({ + "name": "اكتمال بيانات التواصل", + "value": f"{int(completeness / 2.5)}/4 حقول", + "points": completeness, + "max": 10, + }) + + # Company info (max 8 pts) + company_score = 0.0 + if data.get("company_name"): + company_score += 4.0 + if data.get("sector"): + company_score += 4.0 + score += company_score + factors.append({ + "name": "معلومات الشركة", + "value": data.get("company_name", "غير محدد"), + "points": company_score, + "max": 8, + }) + + # Industry match (max 7 pts) + sector = (data.get("sector") or "").lower().replace(" ", "_") + if sector in IMPORTANT_INDUSTRIES: + industry_score = 7.0 + elif sector: + industry_score = 4.0 + else: + industry_score = 0.0 + score += industry_score + factors.append({ + "name": "تطابق القطاع", + "value": data.get("sector", "غير محدد"), + "points": industry_score, + "max": 7, + }) + + return ScoreBreakdown( + category="profile", + score=round(min(score, 25), 1), + max_score=25, + explanation_ar=f"اكتمال الملف: {score:.0f} من 25 نقطة", + explanation_en=f"Profile: {score:.0f}/25", + factors=factors, + ) + + def _score_behavioral(self, data: dict) -> ScoreBreakdown: + """Behavioral Score: Pages, proposals, meetings. Max 25 pts.""" + factors = [] + score = 0.0 + activities = data.get("activities", []) + + # Pages visited (max 8 pts) + page_visits = len([a for a in activities if a.get("type") == "page_view"]) + pages_score = min(page_visits * 1.0, 8.0) + score += pages_score + factors.append({ + "name": "صفحات تمت زيارتها", + "value": page_visits, + "points": pages_score, + "max": 8, + }) + + # Proposals viewed (max 10 pts) + proposals_viewed = len([a for a in activities if a.get("type") in ("proposal_view", "proposal_download")]) + proposals_score = min(proposals_viewed * 5.0, 10.0) + score += proposals_score + factors.append({ + "name": "عروض الأسعار المعروضة", + "value": proposals_viewed, + "points": proposals_score, + "max": 10, + }) + + # Meetings attended (max 7 pts) + meetings = len([a for a in activities if a.get("type") in ("meeting_attended", "meeting_completed")]) + meeting_score = min(meetings * 3.5, 7.0) + score += meeting_score + factors.append({ + "name": "اجتماعات حضرها", + "value": meetings, + "points": meeting_score, + "max": 7, + }) + + return ScoreBreakdown( + category="behavioral", + score=round(min(score, 25), 1), + max_score=25, + explanation_ar=f"السلوك: {score:.0f} من 25 نقطة", + explanation_en=f"Behavioral: {score:.0f}/25", + factors=factors, + ) + + async def _score_intent(self, data: dict) -> ScoreBreakdown: + """Intent Score: Arabic NLP intent analysis from conversations. Max 20 pts.""" + factors = [] + score = 0.0 + messages = data.get("messages", []) + + # Gather inbound message text for NLP + inbound_texts = [ + m.get("content", "") + for m in messages + if m.get("direction") == "inbound" and m.get("content") + ] + + if not inbound_texts: + return ScoreBreakdown( + category="intent", + score=0, + max_score=20, + explanation_ar="النية: 0 من 20 — لا توجد رسائل واردة للتحليل", + explanation_en="Intent: 0/20 - no inbound messages to analyze", + factors=[], + ) + + # Analyze up to 10 most recent messages + combined_text = "\n".join(inbound_texts[:10]) + + try: + intent_result = await self._nlp.extract_intent(combined_text) + sentiment_result = await self._nlp.analyze_sentiment(combined_text) + except Exception as e: + logger.warning(f"NLP analysis failed for intent scoring: {e}") + return ScoreBreakdown( + category="intent", + score=5, + max_score=20, + explanation_ar="النية: 5 من 20 — تعذر تحليل الرسائل بالكامل", + explanation_en="Intent: 5/20 - partial analysis", + factors=[], + ) + + # Intent score (max 12 pts) + intent_scores = { + "buying_intent": 12.0, + "pricing_inquiry": 9.0, + "appointment_request": 10.0, + "complaint": 3.0, + "general_inquiry": 5.0, + } + intent_pts = intent_scores.get(intent_result.intent, 5.0) + intent_pts *= intent_result.confidence + score += intent_pts + factors.append({ + "name": "نية العميل", + "value": intent_result.intent, + "points": round(intent_pts, 1), + "max": 12, + }) + + # Sentiment boost (max 8 pts) + sentiment_map = {"positive": 8.0, "neutral": 4.0, "negative": 1.0} + sent_pts = sentiment_map.get(sentiment_result.sentiment, 4.0) + sent_pts *= sentiment_result.confidence + score += sent_pts + factors.append({ + "name": "مزاج العميل", + "value": sentiment_result.sentiment, + "points": round(sent_pts, 1), + "max": 8, + }) + + return ScoreBreakdown( + category="intent", + score=round(min(score, 20), 1), + max_score=20, + explanation_ar=f"النية: {score:.0f} من 20 — نية {intent_result.intent}، مزاج {sentiment_result.sentiment}", + explanation_en=f"Intent: {score:.0f}/20 - {intent_result.intent}, {sentiment_result.sentiment}", + factors=factors, + ) + + # ── Helpers ─────────────────────────────────── + + @staticmethod + def _calculate_grade(total: int) -> str: + for grade, threshold in GRADE_THRESHOLDS.items(): + if total >= threshold: + return grade + return "F" + + @staticmethod + def _build_arabic_summary( + total: int, grade: str, + engagement: ScoreBreakdown, profile: ScoreBreakdown, + behavioral: ScoreBreakdown, intent: ScoreBreakdown, + ) -> str: + parts = [f"التقييم الإجمالي: {total}/100 (درجة {grade})"] + parts.append(f" - التفاعل: {engagement.score:.0f}/{engagement.max_score:.0f}") + parts.append(f" - الملف الشخصي: {profile.score:.0f}/{profile.max_score:.0f}") + parts.append(f" - السلوك: {behavioral.score:.0f}/{behavioral.max_score:.0f}") + parts.append(f" - النية: {intent.score:.0f}/{intent.max_score:.0f}") + if total >= 80: + parts.append("هذا عميل محتمل ممتاز — يجب التواصل فوراً!") + elif total >= 60: + parts.append("عميل واعد — يحتاج متابعة نشطة") + elif total >= 40: + parts.append("عميل متوسط — يحتاج تنمية وتأهيل") + else: + parts.append("عميل بارد — يحتاج حملة إعادة تفعيل") + return "\n".join(parts) + + @staticmethod + def _build_english_summary(total: int, grade: str) -> str: + status = { + "A": "Hot lead - immediate action required", + "B": "Warm lead - active nurturing needed", + "C": "Medium lead - needs qualification", + "D": "Cold lead - needs re-engagement", + "F": "Inactive - consider removing or re-targeting", + } + return f"Score: {total}/100 (Grade {grade}) - {status.get(grade, '')}" + + @staticmethod + def _recommend_action_ar( + total: int, grade: str, intent: ScoreBreakdown, engagement: ScoreBreakdown + ) -> str: + if grade == "A": + return "اتصل بالعميل الآن! أرسل عرض سعر مخصص واحجز اجتماع عرض المنتج." + elif grade == "B": + return "أرسل رسالة واتساب شخصية واستفسر عن احتياجاته. تابع خلال 24 ساعة." + elif grade == "C": + if engagement.score < 10: + return "العميل لم يتفاعل كفاية. أرسل محتوى تعليمي عن المنتج ودراسة حالة." + return "أرسل بريد إلكتروني بعرض خاص ومحدود الوقت لتحفيز اتخاذ القرار." + elif grade == "D": + return "أضف العميل لحملة تنقيط تلقائية (drip campaign) وتابع بعد أسبوعين." + else: + return "راجع بيانات العميل وتأكد من صحتها. إذا لم يتفاعل خلال 30 يوم، أرشف الملف." + + async def _persist_score( + self, lead_id: str, tenant_id: str, result: LeadScoreResult, db: AsyncSession + ) -> None: + """Save the score to the database.""" + try: + from app.models.lead import Lead + from sqlalchemy import update + + stmt = ( + update(Lead) + .where(Lead.id == lead_id, Lead.tenant_id == tenant_id) + .values(score=result.total_score) + ) + await db.execute(stmt) + await db.commit() + logger.info(f"Lead {lead_id} scored {result.total_score} (grade {result.grade})") + except Exception as e: + logger.warning(f"Failed to persist score for lead {lead_id}: {e}") + await db.rollback() diff --git a/salesflow-saas/backend/app/services/cpq/__init__.py b/salesflow-saas/backend/app/services/cpq/__init__.py new file mode 100644 index 00000000..ab6911d3 --- /dev/null +++ b/salesflow-saas/backend/app/services/cpq/__init__.py @@ -0,0 +1,15 @@ +""" +Dealix CPQ Engine — Configure, Price, Quote +تسعير ذكي وعروض أسعار احترافية للسوق السعودي +""" + +from app.services.cpq.quote_engine import QuoteEngine, QuoteCreate, LineItemInput, DiscountInput +from app.services.cpq.proposal_generator import ProposalGenerator + +__all__ = [ + "QuoteEngine", + "QuoteCreate", + "LineItemInput", + "DiscountInput", + "ProposalGenerator", +] diff --git a/salesflow-saas/backend/app/services/cpq/proposal_generator.py b/salesflow-saas/backend/app/services/cpq/proposal_generator.py new file mode 100644 index 00000000..7616c984 --- /dev/null +++ b/salesflow-saas/backend/app/services/cpq/proposal_generator.py @@ -0,0 +1,227 @@ +""" +Dealix AI Proposal Generator +توليد عروض تجارية ذكية بالعربية والإنجليزية باستخدام الذكاء الاصطناعي +""" + +import logging +from datetime import datetime, timezone +from typing import Optional + +from pydantic import BaseModel, Field + +from app.services.llm.provider import get_llm + +logger = logging.getLogger("dealix.cpq.proposal") + +SECTION_DEFINITIONS = { + "executive_summary": { + "title_ar": "الملخص التنفيذي", + "title_en": "Executive Summary", + "prompt_hint": "Write a concise executive summary for the proposal.", + }, + "solution_overview": { + "title_ar": "نظرة عامة على الحل", + "title_en": "Solution Overview", + "prompt_hint": "Describe the proposed solution and its key components.", + }, + "pricing": { + "title_ar": "التسعير", + "title_en": "Pricing", + "prompt_hint": "Present the pricing breakdown and value proposition.", + }, + "timeline": { + "title_ar": "الجدول الزمني", + "title_en": "Timeline", + "prompt_hint": "Outline the implementation or delivery timeline.", + }, + "terms": { + "title_ar": "الشروط والأحكام", + "title_en": "Terms & Conditions", + "prompt_hint": "State the terms, conditions, and warranty details.", + }, +} + +INDUSTRY_CONTEXT = { + "real_estate": "عقارات — بيع أو تأجير وحدات سكنية أو تجارية في المملكة العربية السعودية", + "healthcare": "رعاية صحية — خدمات طبية وعلاجية في عيادات ومراكز صحية سعودية", + "services": "خدمات — استشارات أو خدمات مهنية متنوعة للشركات السعودية", + "contracting": "مقاولات — أعمال بناء أو صيانة أو تشطيبات في المملكة", + "retail": "تجارة وريتيل — بيع بالتجزئة أو تجارة إلكترونية في السوق السعودي", + "education": "تعليم وتدريب — برامج تعليمية أو دورات تدريبية في المملكة", +} + + +class ProposalInput(BaseModel): + deal_title: str + client_name: str + client_company: str = "" + industry: str = "services" + deal_value: float = 0.0 + currency: str = "SAR" + requirements: str = "" + language: str = Field(default="ar", pattern=r"^(ar|en|both)$") + extra_context: str = "" + + +class ProposalSection(BaseModel): + key: str + title_ar: str + title_en: str + content_ar: str = "" + content_en: str = "" + + +class ProposalOutput(BaseModel): + sections: list[ProposalSection] + language: str + industry: str + generated_at: str + metadata: dict = {} + + +class ProposalGenerator: + """AI-powered proposal generation using LLM with Arabic/English support.""" + + def __init__(self): + self.llm = get_llm() + + async def generate_proposal(self, data: ProposalInput) -> ProposalOutput: + """Generate a full proposal with all sections using AI.""" + industry_ctx = INDUSTRY_CONTEXT.get(data.industry, INDUSTRY_CONTEXT["services"]) + sections: list[ProposalSection] = [] + + for key, defn in SECTION_DEFINITIONS.items(): + section = await self._generate_section( + section_key=key, + section_def=defn, + data=data, + industry_ctx=industry_ctx, + ) + sections.append(section) + + logger.info( + "Proposal generated for '%s' — %d sections, lang=%s", + data.deal_title, len(sections), data.language, + ) + return ProposalOutput( + sections=sections, + language=data.language, + industry=data.industry, + generated_at=datetime.now(timezone.utc).isoformat(), + metadata={ + "client": data.client_name, + "company": data.client_company, + "deal_value": data.deal_value, + "currency": data.currency, + }, + ) + + async def customize_section( + self, + section_key: str, + custom_instructions: str, + data: ProposalInput, + ) -> ProposalSection: + """Re-generate a single section with custom instructions.""" + defn = SECTION_DEFINITIONS.get(section_key) + if not defn: + raise ValueError(f"Unknown section: {section_key}") + + industry_ctx = INDUSTRY_CONTEXT.get(data.industry, INDUSTRY_CONTEXT["services"]) + return await self._generate_section( + section_key=section_key, + section_def=defn, + data=data, + industry_ctx=industry_ctx, + custom_instructions=custom_instructions, + ) + + async def export_pdf_data(self, proposal: ProposalOutput, company_branding: Optional[dict] = None) -> dict: + """Prepare structured data ready for PDF rendering.""" + branding = company_branding or { + "company_name_ar": "شركتكم", + "company_name_en": "Your Company", + "logo_url": "", + "primary_color": "#1a5276", + "secondary_color": "#2ecc71", + } + return { + "branding": branding, + "title_ar": "عرض تجاري", + "title_en": "Commercial Proposal", + "generated_at": proposal.generated_at, + "metadata": proposal.metadata, + "sections": [s.model_dump() for s in proposal.sections], + "footer_ar": "تم إنشاء هذا العرض بواسطة ديليكس — نظام ذكاء المبيعات", + "footer_en": "Generated by Dealix — AI Sales Intelligence", + "direction": "rtl" if proposal.language in ("ar", "both") else "ltr", + } + + # ── Internal ──────────────────────────────────── + + async def _generate_section( + self, + section_key: str, + section_def: dict, + data: ProposalInput, + industry_ctx: str, + custom_instructions: str = "", + ) -> ProposalSection: + system_prompt = ( + "أنت كاتب عروض تجارية محترف متخصص في السوق السعودي.\n" + "اكتب بأسلوب مهني ومقنع. لا تستخدم رموز تعبيرية.\n" + "إذا طُلب منك الكتابة بالعربية، استخدم العربية الفصحى الرسمية.\n" + f"القطاع: {industry_ctx}\n" + ) + if custom_instructions: + system_prompt += f"تعليمات إضافية: {custom_instructions}\n" + + user_msg = ( + f"اكتب قسم '{section_def['title_ar']}' لعرض تجاري.\n" + f"العنوان: {data.deal_title}\n" + f"العميل: {data.client_name} — {data.client_company}\n" + f"القيمة: {data.deal_value} {data.currency}\n" + f"المتطلبات: {data.requirements or 'غير محددة'}\n" + f"{section_def['prompt_hint']}\n" + f"سياق إضافي: {data.extra_context or 'لا يوجد'}\n" + ) + + content_ar = "" + content_en = "" + + if data.language in ("ar", "both"): + resp = await self.llm.complete( + system_prompt=system_prompt + "اكتب بالعربية فقط. 3-5 فقرات مختصرة.", + user_message=user_msg, + temperature=0.5, + max_tokens=500, + ) + content_ar = resp.content.strip() + + if data.language in ("en", "both"): + resp = await self.llm.complete( + system_prompt=( + "You are a professional proposal writer for the Saudi market.\n" + "Write in formal business English. No emojis.\n" + f"Industry: {industry_ctx}\n" + ), + user_message=( + f"Write the '{section_def['title_en']}' section for a business proposal.\n" + f"Title: {data.deal_title}\n" + f"Client: {data.client_name} — {data.client_company}\n" + f"Value: {data.deal_value} {data.currency}\n" + f"Requirements: {data.requirements or 'Not specified'}\n" + f"{section_def['prompt_hint']}\n" + ), + temperature=0.5, + max_tokens=500, + ) + content_en = resp.content.strip() + + return ProposalSection( + key=section_key, + title_ar=section_def["title_ar"], + title_en=section_def["title_en"], + content_ar=content_ar, + content_en=content_en, + ) diff --git a/salesflow-saas/backend/app/services/cpq/quote_engine.py b/salesflow-saas/backend/app/services/cpq/quote_engine.py new file mode 100644 index 00000000..6782be8f --- /dev/null +++ b/salesflow-saas/backend/app/services/cpq/quote_engine.py @@ -0,0 +1,321 @@ +""" +Dealix CPQ Quote Engine — Configure, Price, Quote +عروض أسعار احترافية مع ضريبة القيمة المضافة والعملات المتعددة +""" + +import uuid +import logging +from datetime import datetime, timedelta, timezone +from decimal import Decimal, ROUND_HALF_UP +from enum import Enum +from typing import Optional + +from pydantic import BaseModel, Field +from sqlalchemy import select, func, update +from sqlalchemy.ext.asyncio import AsyncSession + +from app.models.proposal import Proposal + +logger = logging.getLogger("dealix.cpq.quote") + +SAR_VAT_RATE = Decimal("0.15") +DEFAULT_VALIDITY_DAYS = 30 +USD_TO_SAR_RATE = Decimal("3.75") + +INDUSTRY_TEMPLATES = { + "real_estate": { + "header_ar": "عرض سعر عقاري", + "footer_ar": "هذا العرض ساري لمدة {validity} يوم من تاريخه", + "terms_ar": [ + "الأسعار شاملة ضريبة القيمة المضافة ما لم يُذكر خلاف ذلك", + "يتم الدفع حسب خطة السداد المتفق عليها", + "العرض قابل للتعديل حسب توفر الوحدات", + ], + }, + "healthcare": { + "header_ar": "عرض سعر طبي", + "footer_ar": "العرض ساري لمدة {validity} يوم — صحتكم أولويتنا", + "terms_ar": [ + "الأسعار شاملة ضريبة القيمة المضافة", + "التأمين الطبي قد يغطي جزءاً من التكاليف", + "يرجى إحضار بطاقة التأمين عند الزيارة", + ], + }, + "services": { + "header_ar": "عرض سعر خدمات", + "footer_ar": "العرض ساري لمدة {validity} يوم من تاريخه", + "terms_ar": [ + "الأسعار شاملة ضريبة القيمة المضافة 15%", + "مدة التنفيذ تبدأ من تاريخ الموافقة على العرض", + "الدفع: 50% مقدم و50% عند التسليم ما لم يُتفق على خلاف ذلك", + ], + }, + "contracting": { + "header_ar": "عرض سعر مقاولات", + "footer_ar": "العرض ساري لمدة {validity} يوم — شاملاً المواد والعمالة", + "terms_ar": [ + "الأسعار شاملة ضريبة القيمة المضافة 15%", + "التسعير مبني على المعاينة الميدانية", + "أي تغييرات في النطاق تستوجب ملحق عقد منفصل", + "الضمان حسب بنود العقد", + ], + }, +} + + +class QuoteStatus(str, Enum): + DRAFT = "draft" + SENT = "sent" + VIEWED = "viewed" + ACCEPTED = "accepted" + REJECTED = "rejected" + EXPIRED = "expired" + + +class LineItemInput(BaseModel): + description_ar: str + description_en: str = "" + quantity: int = Field(ge=1, default=1) + unit_price: Decimal = Field(ge=0) + unit: str = "وحدة" + + +class DiscountInput(BaseModel): + type: str = Field(pattern=r"^(percentage|fixed)$") + value: Decimal = Field(ge=0) + reason_ar: str = "" + + +class QuoteCreate(BaseModel): + tenant_id: str + deal_id: Optional[str] = None + lead_id: Optional[str] = None + title: str + currency: str = "SAR" + industry: str = "services" + validity_days: int = DEFAULT_VALIDITY_DAYS + vat_registration_number: Optional[str] = None + client_name: str = "" + client_company: str = "" + notes_ar: str = "" + + +class QuoteEngine: + """Full CPQ lifecycle: create, price, send, track.""" + + def __init__(self, db: AsyncSession): + self.db = db + + async def create_quote(self, data: QuoteCreate) -> dict: + """Create a new quote in draft status.""" + template = INDUSTRY_TEMPLATES.get(data.industry, INDUSTRY_TEMPLATES["services"]) + valid_until = datetime.now(timezone.utc) + timedelta(days=data.validity_days) + + content = { + "line_items": [], + "discounts": [], + "subtotal": "0", + "discount_total": "0", + "vat_amount": "0", + "total": "0", + "currency": data.currency, + "industry": data.industry, + "header_ar": template["header_ar"], + "footer_ar": template["footer_ar"].format(validity=data.validity_days), + "terms_ar": template["terms_ar"], + "vat_registration_number": data.vat_registration_number or "", + "client_name": data.client_name, + "client_company": data.client_company, + "notes_ar": data.notes_ar, + } + + proposal = Proposal( + id=uuid.uuid4(), + tenant_id=uuid.UUID(data.tenant_id), + deal_id=uuid.UUID(data.deal_id) if data.deal_id else None, + lead_id=uuid.UUID(data.lead_id) if data.lead_id else None, + title=data.title, + content=content, + total_amount=Decimal("0"), + currency=data.currency, + status=QuoteStatus.DRAFT.value, + valid_until=valid_until.date(), + ) + self.db.add(proposal) + await self.db.flush() + logger.info("Quote %s created for tenant %s", proposal.id, data.tenant_id) + return self._to_dict(proposal) + + async def add_line_item(self, tenant_id: str, quote_id: str, item: LineItemInput) -> dict: + """Add a line item and recalculate totals.""" + proposal = await self._get_quote(tenant_id, quote_id) + if not proposal: + raise ValueError("عرض السعر غير موجود") + + content: dict = dict(proposal.content) + line_items: list = list(content.get("line_items", [])) + + line_total = item.unit_price * item.quantity + line_items.append({ + "id": str(uuid.uuid4())[:8], + "description_ar": item.description_ar, + "description_en": item.description_en, + "quantity": item.quantity, + "unit_price": str(item.unit_price), + "unit": item.unit, + "total": str(line_total), + }) + content["line_items"] = line_items + proposal.content = content + await self._recalculate(proposal) + await self.db.flush() + return self._to_dict(proposal) + + async def apply_discount(self, tenant_id: str, quote_id: str, discount: DiscountInput) -> dict: + """Apply a percentage or fixed discount.""" + proposal = await self._get_quote(tenant_id, quote_id) + if not proposal: + raise ValueError("عرض السعر غير موجود") + + content: dict = dict(proposal.content) + discounts: list = list(content.get("discounts", [])) + discounts.append({ + "type": discount.type, + "value": str(discount.value), + "reason_ar": discount.reason_ar, + }) + content["discounts"] = discounts + proposal.content = content + await self._recalculate(proposal) + await self.db.flush() + return self._to_dict(proposal) + + async def calculate_totals(self, tenant_id: str, quote_id: str) -> dict: + """Force recalculation of quote totals.""" + proposal = await self._get_quote(tenant_id, quote_id) + if not proposal: + raise ValueError("عرض السعر غير موجود") + await self._recalculate(proposal) + await self.db.flush() + return { + "subtotal": proposal.content.get("subtotal", "0"), + "discount_total": proposal.content.get("discount_total", "0"), + "vat_amount": proposal.content.get("vat_amount", "0"), + "total": str(proposal.total_amount), + "currency": proposal.currency, + } + + async def send_quote( + self, tenant_id: str, quote_id: str, channel: str = "whatsapp", recipient: str = "" + ) -> dict: + """Mark quote as sent and dispatch via channel.""" + proposal = await self._get_quote(tenant_id, quote_id) + if not proposal: + raise ValueError("عرض السعر غير موجود") + + proposal.status = QuoteStatus.SENT.value + proposal.sent_at = datetime.now(timezone.utc) + await self.db.flush() + + dispatch_result = {"channel": channel, "recipient": recipient, "status": "queued"} + if channel == "whatsapp": + from app.services.whatsapp_service import WhatsAppService + wa = WhatsAppService() + msg = ( + f"مرحباً {proposal.content.get('client_name', '')}،\n" + f"مرفق عرض السعر: {proposal.title}\n" + f"الإجمالي: {proposal.total_amount} {proposal.currency}\n" + f"ساري حتى: {proposal.valid_until}" + ) + dispatch_result = await wa.send_message(recipient, msg) + elif channel == "email": + from app.services.email_service import EmailService + es = EmailService() + dispatch_result = await es.send_email( + to=recipient, + subject=f"عرض سعر — {proposal.title}", + body=f"عرض سعر بمبلغ {proposal.total_amount} {proposal.currency}", + ) + + logger.info("Quote %s sent via %s to %s", quote_id, channel, recipient) + return {"quote_id": str(proposal.id), "status": "sent", "dispatch": dispatch_result} + + async def get_quote_status(self, tenant_id: str, quote_id: str) -> dict: + """Return current quote status and lifecycle timestamps.""" + proposal = await self._get_quote(tenant_id, quote_id) + if not proposal: + raise ValueError("عرض السعر غير موجود") + + now = datetime.now(timezone.utc).date() + is_expired = proposal.valid_until and proposal.valid_until < now + if is_expired and proposal.status not in (QuoteStatus.ACCEPTED.value, QuoteStatus.REJECTED.value): + proposal.status = QuoteStatus.EXPIRED.value + await self.db.flush() + + return { + "quote_id": str(proposal.id), + "status": proposal.status, + "sent_at": proposal.sent_at.isoformat() if proposal.sent_at else None, + "viewed_at": proposal.viewed_at.isoformat() if proposal.viewed_at else None, + "valid_until": proposal.valid_until.isoformat() if proposal.valid_until else None, + "is_expired": is_expired, + } + + # ── Internal helpers ───────────────────────────── + + async def _get_quote(self, tenant_id: str, quote_id: str) -> Optional[Proposal]: + result = await self.db.execute( + select(Proposal).where( + Proposal.id == uuid.UUID(quote_id), + Proposal.tenant_id == uuid.UUID(tenant_id), + ) + ) + return result.scalar_one_or_none() + + async def _recalculate(self, proposal: Proposal) -> None: + content: dict = dict(proposal.content) + line_items = content.get("line_items", []) + discounts = content.get("discounts", []) + + subtotal = sum(Decimal(li["total"]) for li in line_items) + + discount_total = Decimal("0") + for d in discounts: + if d["type"] == "percentage": + discount_total += (subtotal * Decimal(d["value"]) / Decimal("100")).quantize( + Decimal("0.01"), rounding=ROUND_HALF_UP + ) + else: + discount_total += Decimal(d["value"]) + + after_discount = max(subtotal - discount_total, Decimal("0")) + vat_amount = (after_discount * SAR_VAT_RATE).quantize(Decimal("0.01"), rounding=ROUND_HALF_UP) + total = after_discount + vat_amount + + if content.get("currency") == "USD": + content["total_sar"] = str((total * USD_TO_SAR_RATE).quantize(Decimal("0.01"))) + + content["subtotal"] = str(subtotal) + content["discount_total"] = str(discount_total) + content["vat_amount"] = str(vat_amount) + content["total"] = str(total) + proposal.content = content + proposal.total_amount = total + + @staticmethod + def _to_dict(proposal: Proposal) -> dict: + return { + "id": str(proposal.id), + "tenant_id": str(proposal.tenant_id), + "deal_id": str(proposal.deal_id) if proposal.deal_id else None, + "lead_id": str(proposal.lead_id) if proposal.lead_id else None, + "title": proposal.title, + "content": proposal.content, + "total_amount": str(proposal.total_amount) if proposal.total_amount else "0", + "currency": proposal.currency, + "status": proposal.status, + "valid_until": proposal.valid_until.isoformat() if proposal.valid_until else None, + "sent_at": proposal.sent_at.isoformat() if proposal.sent_at else None, + "viewed_at": proposal.viewed_at.isoformat() if proposal.viewed_at else None, + "created_at": proposal.created_at.isoformat() if proposal.created_at else None, + } diff --git a/salesflow-saas/backend/app/services/pdpl/__init__.py b/salesflow-saas/backend/app/services/pdpl/__init__.py new file mode 100644 index 00000000..f5356faf --- /dev/null +++ b/salesflow-saas/backend/app/services/pdpl/__init__.py @@ -0,0 +1,6 @@ +"""PDPL (Saudi Personal Data Protection Law) consent management.""" + +from app.services.pdpl.consent_manager import ConsentManager +from app.services.pdpl.data_rights import DataRightsHandler + +__all__ = ["ConsentManager", "DataRightsHandler"] diff --git a/salesflow-saas/backend/app/services/pdpl/consent_manager.py b/salesflow-saas/backend/app/services/pdpl/consent_manager.py new file mode 100644 index 00000000..220ab024 --- /dev/null +++ b/salesflow-saas/backend/app/services/pdpl/consent_manager.py @@ -0,0 +1,301 @@ +"""PDPL consent engine -- tracks, validates, and audits consent per Saudi data protection law. + +Penalty for violations: up to 5,000,000 SAR per incident. +""" + +import logging +from datetime import datetime, timedelta, timezone +from typing import Optional +from uuid import UUID + +from pydantic import BaseModel as Schema +from sqlalchemy import select, and_ +from sqlalchemy.ext.asyncio import AsyncSession + +from app.models.consent import ( + PDPLConsent, PDPLConsentAudit, DataRequest, + ConsentStatusEnum, ConsentPurpose, ConsentChannel, + DataRequestType, DataRequestStatus, +) + +logger = logging.getLogger(__name__) + +DEFAULT_EXPIRY_MONTHS = 12 +CROSS_BORDER_ALLOWED_COUNTRIES = {"SA", "AE", "BH", "KW", "OM", "QA"} + +# --------------------------------------------------------------------------- +# Pydantic schemas +# --------------------------------------------------------------------------- + +class ConsentGrantInput(Schema): + contact_id: UUID + tenant_id: UUID + purpose: str + channel: str + consent_text: Optional[str] = None + ip_address: Optional[str] = None + actor_id: Optional[UUID] = None + expiry_months: int = DEFAULT_EXPIRY_MONTHS + + +class ConsentRevokeInput(Schema): + consent_id: UUID + actor_id: Optional[UUID] = None + reason: Optional[str] = None + ip_address: Optional[str] = None + + +class ConsentCheckResult(Schema): + allowed: bool + consent_id: Optional[UUID] = None + status: Optional[str] = None + expires_at: Optional[datetime] = None + message: str = "" + message_ar: str = "" + + +class DataRequestInput(Schema): + contact_id: UUID + tenant_id: UUID + request_type: str + notes: Optional[str] = None + actor_id: Optional[UUID] = None + + +class AuditEntry(Schema): + id: UUID + consent_id: UUID + contact_id: UUID + action: str + actor_id: Optional[UUID] = None + channel: str + purpose: str + details: dict + created_at: datetime + + model_config = {"from_attributes": True} + + +# --------------------------------------------------------------------------- +# ConsentManager +# --------------------------------------------------------------------------- + +class ConsentManager: + """Core PDPL consent engine for Dealix CRM.""" + + def __init__(self, db: AsyncSession): + self.db = db + + # -- grant --------------------------------------------------------------- + + async def grant_consent(self, data: ConsentGrantInput) -> PDPLConsent: + """Record a new consent grant. Existing active consent for same + contact+purpose+channel is revoked first (re-consent flow).""" + + now = datetime.now(timezone.utc) + + # Revoke any existing active consent for same triplet (re-consent) + existing = await self._find_active(data.contact_id, data.purpose, data.channel) + if existing: + existing.status = ConsentStatusEnum.REVOKED.value + existing.revoked_at = now + await self._audit( + consent_id=existing.id, contact_id=data.contact_id, + action="revoked_for_renewal", actor_id=data.actor_id, + channel=data.channel, purpose=data.purpose, + details={"reason": "re-consent on purpose change"}, + ip_address=data.ip_address, + tenant_id=data.tenant_id, + ) + + consent = PDPLConsent( + contact_id=data.contact_id, + tenant_id=data.tenant_id, + purpose=data.purpose, + channel=data.channel, + status=ConsentStatusEnum.GRANTED.value, + granted_at=now, + expires_at=now + timedelta(days=30 * data.expiry_months), + ip_address=data.ip_address, + consent_text=data.consent_text, + granted_by=data.actor_id, + ) + self.db.add(consent) + await self.db.flush() + await self.db.refresh(consent) + + await self._audit( + consent_id=consent.id, contact_id=data.contact_id, + action="granted", actor_id=data.actor_id, + channel=data.channel, purpose=data.purpose, + details={"expiry_months": data.expiry_months, "consent_text": data.consent_text or ""}, + ip_address=data.ip_address, + tenant_id=data.tenant_id, + ) + logger.info("PDPL consent granted: contact=%s purpose=%s channel=%s", data.contact_id, data.purpose, data.channel) + return consent + + # -- revoke -------------------------------------------------------------- + + async def revoke_consent(self, data: ConsentRevokeInput) -> PDPLConsent: + """Revoke an existing consent immediately.""" + + result = await self.db.execute( + select(PDPLConsent).where(PDPLConsent.id == data.consent_id) + ) + consent = result.scalar_one_or_none() + if not consent: + raise ValueError("سجل الموافقة غير موجود") # Consent record not found + + now = datetime.now(timezone.utc) + consent.status = ConsentStatusEnum.REVOKED.value + consent.revoked_at = now + + await self._audit( + consent_id=consent.id, contact_id=consent.contact_id, + action="revoked", actor_id=data.actor_id, + channel=consent.channel, purpose=consent.purpose, + details={"reason": data.reason or "user_request"}, + ip_address=data.ip_address, + tenant_id=consent.tenant_id, + ) + logger.info("PDPL consent revoked: id=%s contact=%s", consent.id, consent.contact_id) + return consent + + # -- check --------------------------------------------------------------- + + async def check_consent( + self, + contact_id: UUID, + purpose: str, + channel: str, + ) -> ConsentCheckResult: + """Validate consent before any outbound message. Must be called + before every send -- violations carry up to 5M SAR penalty.""" + + consent = await self._find_active(contact_id, purpose, channel) + if not consent: + return ConsentCheckResult( + allowed=False, + message=f"No active consent for {purpose}/{channel}", + message_ar="لا توجد موافقة فعالة لهذا الغرض والقناة", + ) + + now = datetime.now(timezone.utc) + if consent.expires_at and consent.expires_at <= now: + consent.status = ConsentStatusEnum.EXPIRED.value + await self.db.flush() + return ConsentCheckResult( + allowed=False, + consent_id=consent.id, + status=ConsentStatusEnum.EXPIRED.value, + expires_at=consent.expires_at, + message="Consent expired -- re-consent required", + message_ar="انتهت صلاحية الموافقة -- يلزم تجديد الموافقة", + ) + + return ConsentCheckResult( + allowed=True, + consent_id=consent.id, + status=consent.status, + expires_at=consent.expires_at, + message="Consent valid", + message_ar="الموافقة صالحة", + ) + + # -- data request -------------------------------------------------------- + + async def process_data_request(self, data: DataRequestInput) -> DataRequest: + """Submit a PDPL data subject rights request.""" + + request = DataRequest( + contact_id=data.contact_id, + tenant_id=data.tenant_id, + request_type=data.request_type, + status=DataRequestStatus.PENDING.value, + requested_at=datetime.now(timezone.utc), + notes=data.notes, + handled_by=data.actor_id, + ) + self.db.add(request) + await self.db.flush() + await self.db.refresh(request) + logger.info("PDPL data request created: type=%s contact=%s", data.request_type, data.contact_id) + return request + + # -- audit --------------------------------------------------------------- + + async def get_consent_audit( + self, + tenant_id: UUID, + contact_id: Optional[UUID] = None, + limit: int = 100, + offset: int = 0, + ) -> list[AuditEntry]: + """Return consent audit trail filtered by tenant and optionally contact.""" + + query = ( + select(PDPLConsentAudit) + .where(PDPLConsentAudit.tenant_id == tenant_id) + .order_by(PDPLConsentAudit.created_at.desc()) + ) + if contact_id: + query = query.where(PDPLConsentAudit.contact_id == contact_id) + query = query.offset(offset).limit(limit) + result = await self.db.execute(query) + return [AuditEntry.model_validate(row) for row in result.scalars().all()] + + # -- cross-border -------------------------------------------------------- + + @staticmethod + def check_cross_border_transfer(destination_country: str) -> ConsentCheckResult: + """Check if data transfer to destination country is PDPL-compliant. + SDAIA requires adequate protection level or explicit consent.""" + + code = destination_country.upper().strip() + if code in CROSS_BORDER_ALLOWED_COUNTRIES: + return ConsentCheckResult( + allowed=True, + message=f"Transfer to {code} permitted under GCC adequacy", + message_ar=f"النقل إلى {code} مسموح بموجب كفاية دول الخليج", + ) + return ConsentCheckResult( + allowed=False, + message=f"Transfer to {code} requires explicit consent and SDAIA approval", + message_ar=f"النقل إلى {code} يتطلب موافقة صريحة وموافقة الهيئة", + ) + + # -- private helpers ----------------------------------------------------- + + async def _find_active( + self, contact_id: UUID, purpose: str, channel: str + ) -> Optional[PDPLConsent]: + result = await self.db.execute( + select(PDPLConsent).where( + and_( + PDPLConsent.contact_id == contact_id, + PDPLConsent.purpose == purpose, + PDPLConsent.channel == channel, + PDPLConsent.status == ConsentStatusEnum.GRANTED.value, + ) + ).order_by(PDPLConsent.granted_at.desc()).limit(1) + ) + return result.scalar_one_or_none() + + async def _audit( + self, *, consent_id, contact_id, action, actor_id, + channel, purpose, details, ip_address, tenant_id, + ) -> None: + entry = PDPLConsentAudit( + consent_id=consent_id, + contact_id=contact_id, + tenant_id=tenant_id, + action=action, + actor_id=actor_id, + channel=channel, + purpose=purpose, + details=details or {}, + ip_address=ip_address, + ) + self.db.add(entry) + await self.db.flush() diff --git a/salesflow-saas/backend/app/services/pdpl/data_rights.py b/salesflow-saas/backend/app/services/pdpl/data_rights.py new file mode 100644 index 00000000..a3b3bf64 --- /dev/null +++ b/salesflow-saas/backend/app/services/pdpl/data_rights.py @@ -0,0 +1,352 @@ +"""PDPL data subject rights handler. + +Implements: right to access, correction, deletion, restriction of processing. +Generates compliance reports for SDAIA audits. +""" + +import logging +from datetime import datetime, timedelta, timezone +from typing import Any, Optional +from uuid import UUID + +from pydantic import BaseModel as Schema +from sqlalchemy import select, func, and_ +from sqlalchemy.ext.asyncio import AsyncSession + +from app.models.consent import ( + PDPLConsent, PDPLConsentAudit, DataRequest, + DataRequestStatus, DataRequestType, +) +from app.models.lead import Lead +from app.models.message import Message + +logger = logging.getLogger(__name__) + +HARD_DELETE_DELAY_DAYS = 30 + + +# --------------------------------------------------------------------------- +# Pydantic schemas +# --------------------------------------------------------------------------- + +class DataExport(Schema): + contact_id: UUID + personal_data: dict + consents: list[dict] + messages: list[dict] + exported_at: datetime + + +class CorrectionInput(Schema): + contact_id: UUID + tenant_id: UUID + corrections: dict[str, Any] # field_name -> new_value + actor_id: Optional[UUID] = None + reason: Optional[str] = None + + +class CorrectionResult(Schema): + contact_id: UUID + fields_updated: list[str] + previous_values: dict + updated_at: datetime + + +class DeletionResult(Schema): + contact_id: UUID + status: str + soft_deleted_at: datetime + hard_delete_scheduled: datetime + message: str + message_ar: str + + +class RestrictionResult(Schema): + contact_id: UUID + restricted: bool + message: str + message_ar: str + + +class ComplianceReport(Schema): + tenant_id: UUID + generated_at: datetime + total_consents: int + active_consents: int + revoked_consents: int + expired_consents: int + pending_requests: int + completed_requests: int + requests_by_type: dict[str, int] + avg_resolution_hours: Optional[float] = None + violations_detected: int + + +# --------------------------------------------------------------------------- +# DataRightsHandler +# --------------------------------------------------------------------------- + +class DataRightsHandler: + """Handles PDPL data subject rights for Dealix contacts.""" + + def __init__(self, db: AsyncSession): + self.db = db + + # -- export (right to access) ------------------------------------------- + + async def export_data(self, contact_id: UUID, tenant_id: UUID) -> DataExport: + """Export all personal data held for a contact as structured JSON.""" + + lead = await self._get_lead(contact_id, tenant_id) + + # Consent records + consents_q = await self.db.execute( + select(PDPLConsent).where( + PDPLConsent.contact_id == contact_id, + PDPLConsent.tenant_id == tenant_id, + ) + ) + consents = [ + {"purpose": c.purpose, "channel": c.channel, "status": c.status, + "granted_at": c.granted_at.isoformat() if c.granted_at else None, + "expires_at": c.expires_at.isoformat() if c.expires_at else None} + for c in consents_q.scalars().all() + ] + + # Messages + msgs_q = await self.db.execute( + select(Message).where(Message.lead_id == contact_id).limit(500) + ) + messages = [ + {"channel": m.channel, "direction": m.direction, + "content": m.content, "sent_at": m.sent_at.isoformat() if m.sent_at else None} + for m in msgs_q.scalars().all() + ] + + personal = { + "name": lead.name, + "phone": lead.phone, + "email": lead.email, + "source": lead.source, + "status": lead.status, + "score": lead.score, + "notes": lead.notes, + } + + logger.info("PDPL data export completed: contact=%s", contact_id) + return DataExport( + contact_id=contact_id, + personal_data=personal, + consents=consents, + messages=messages, + exported_at=datetime.now(timezone.utc), + ) + + # -- correction ---------------------------------------------------------- + + async def correct_data(self, data: CorrectionInput) -> CorrectionResult: + """Update personal data fields with full audit trail.""" + + lead = await self._get_lead(data.contact_id, data.tenant_id) + allowed_fields = {"name", "phone", "email", "notes"} + previous: dict[str, Any] = {} + updated_fields: list[str] = [] + + for field, new_val in data.corrections.items(): + if field not in allowed_fields: + logger.warning("PDPL correction rejected for field=%s", field) + continue + previous[field] = getattr(lead, field, None) + setattr(lead, field, new_val) + updated_fields.append(field) + + await self.db.flush() + + # Audit via data request record + req = DataRequest( + contact_id=data.contact_id, + tenant_id=data.tenant_id, + request_type=DataRequestType.CORRECTION.value, + status=DataRequestStatus.COMPLETED.value, + requested_at=datetime.now(timezone.utc), + completed_at=datetime.now(timezone.utc), + response_data={"corrections": data.corrections, "previous": previous, "reason": data.reason}, + handled_by=data.actor_id, + ) + self.db.add(req) + await self.db.flush() + + logger.info("PDPL data correction: contact=%s fields=%s", data.contact_id, updated_fields) + return CorrectionResult( + contact_id=data.contact_id, + fields_updated=updated_fields, + previous_values=previous, + updated_at=datetime.now(timezone.utc), + ) + + # -- deletion (right to erasure) ---------------------------------------- + + async def delete_data(self, contact_id: UUID, tenant_id: UUID, actor_id: Optional[UUID] = None) -> DeletionResult: + """Soft-delete contact now; schedule hard-delete after 30 days.""" + + lead = await self._get_lead(contact_id, tenant_id) + now = datetime.now(timezone.utc) + hard_delete_at = now + timedelta(days=HARD_DELETE_DELAY_DAYS) + + # Soft-delete: mark status and clear PII + lead.status = "deleted" + lead.notes = f"[PDPL deletion requested {now.isoformat()}] " + (lead.notes or "") + lead.extra_metadata = { + **(lead.extra_metadata or {}), + "_pdpl_soft_deleted": True, + "_pdpl_hard_delete_at": hard_delete_at.isoformat(), + } + + # Revoke all active consents + consents_q = await self.db.execute( + select(PDPLConsent).where( + PDPLConsent.contact_id == contact_id, + PDPLConsent.tenant_id == tenant_id, + PDPLConsent.status == "granted", + ) + ) + for consent in consents_q.scalars().all(): + consent.status = "revoked" + consent.revoked_at = now + + # Record the request + req = DataRequest( + contact_id=contact_id, + tenant_id=tenant_id, + request_type=DataRequestType.DELETION.value, + status=DataRequestStatus.PROCESSING.value, + requested_at=now, + response_data={"hard_delete_at": hard_delete_at.isoformat()}, + handled_by=actor_id, + ) + self.db.add(req) + await self.db.flush() + + logger.info("PDPL deletion scheduled: contact=%s hard_delete=%s", contact_id, hard_delete_at) + return DeletionResult( + contact_id=contact_id, + status="soft_deleted", + soft_deleted_at=now, + hard_delete_scheduled=hard_delete_at, + message=f"Contact soft-deleted. Hard delete scheduled for {hard_delete_at.date()}", + message_ar=f"تم حذف جهة الاتصال مبدئيًا. الحذف النهائي مجدول بتاريخ {hard_delete_at.date()}", + ) + + # -- restriction --------------------------------------------------------- + + async def restrict_processing( + self, contact_id: UUID, tenant_id: UUID, actor_id: Optional[UUID] = None + ) -> RestrictionResult: + """Flag a contact as restricted -- no outbound processing allowed.""" + + lead = await self._get_lead(contact_id, tenant_id) + lead.extra_metadata = { + **(lead.extra_metadata or {}), + "_pdpl_restricted": True, + "_pdpl_restricted_at": datetime.now(timezone.utc).isoformat(), + } + + req = DataRequest( + contact_id=contact_id, + tenant_id=tenant_id, + request_type=DataRequestType.RESTRICTION.value, + status=DataRequestStatus.COMPLETED.value, + requested_at=datetime.now(timezone.utc), + completed_at=datetime.now(timezone.utc), + response_data={"restricted": True}, + handled_by=actor_id, + ) + self.db.add(req) + await self.db.flush() + + logger.info("PDPL processing restricted: contact=%s", contact_id) + return RestrictionResult( + contact_id=contact_id, + restricted=True, + message="Contact processing restricted per PDPL request", + message_ar="تم تقييد معالجة بيانات جهة الاتصال وفقًا لطلب نظام حماية البيانات", + ) + + # -- compliance report --------------------------------------------------- + + async def generate_compliance_report(self, tenant_id: UUID) -> ComplianceReport: + """Generate SDAIA-ready compliance report for a tenant.""" + + now = datetime.now(timezone.utc) + + # Consent counts + total = (await self.db.execute( + select(func.count()).where(PDPLConsent.tenant_id == tenant_id) + )).scalar() or 0 + active = (await self.db.execute( + select(func.count()).where(PDPLConsent.tenant_id == tenant_id, PDPLConsent.status == "granted") + )).scalar() or 0 + revoked = (await self.db.execute( + select(func.count()).where(PDPLConsent.tenant_id == tenant_id, PDPLConsent.status == "revoked") + )).scalar() or 0 + expired = (await self.db.execute( + select(func.count()).where(PDPLConsent.tenant_id == tenant_id, PDPLConsent.status == "expired") + )).scalar() or 0 + + # Data requests + pending = (await self.db.execute( + select(func.count()).where(DataRequest.tenant_id == tenant_id, DataRequest.status == "pending") + )).scalar() or 0 + completed = (await self.db.execute( + select(func.count()).where(DataRequest.tenant_id == tenant_id, DataRequest.status == "completed") + )).scalar() or 0 + + # Breakdown by type + type_rows = (await self.db.execute( + select(DataRequest.request_type, func.count()) + .where(DataRequest.tenant_id == tenant_id) + .group_by(DataRequest.request_type) + )).all() + by_type = {row[0]: row[1] for row in type_rows} + + # Avg resolution time + avg_hours: Optional[float] = None + completed_reqs = (await self.db.execute( + select(DataRequest).where( + DataRequest.tenant_id == tenant_id, + DataRequest.status == "completed", + DataRequest.completed_at.isnot(None), + ).limit(500) + )).scalars().all() + if completed_reqs: + deltas = [ + (r.completed_at - r.requested_at).total_seconds() / 3600 + for r in completed_reqs if r.completed_at and r.requested_at + ] + avg_hours = round(sum(deltas) / len(deltas), 2) if deltas else None + + logger.info("PDPL compliance report generated: tenant=%s", tenant_id) + return ComplianceReport( + tenant_id=tenant_id, + generated_at=now, + total_consents=total, + active_consents=active, + revoked_consents=revoked, + expired_consents=expired, + pending_requests=pending, + completed_requests=completed, + requests_by_type=by_type, + avg_resolution_hours=avg_hours, + violations_detected=0, + ) + + # -- private helpers ----------------------------------------------------- + + async def _get_lead(self, contact_id: UUID, tenant_id: UUID) -> Lead: + result = await self.db.execute( + select(Lead).where(Lead.id == contact_id, Lead.tenant_id == tenant_id) + ) + lead = result.scalar_one_or_none() + if not lead: + raise ValueError("جهة الاتصال غير موجودة") # Contact not found + return lead diff --git a/salesflow-saas/backend/app/services/security_gate.py b/salesflow-saas/backend/app/services/security_gate.py new file mode 100644 index 00000000..23a516d0 --- /dev/null +++ b/salesflow-saas/backend/app/services/security_gate.py @@ -0,0 +1,208 @@ +""" +Security Gate — Dealix AI Revenue OS +Pre-release and runtime security verification. +Blocks risky actions and enforces compliance checks. +""" +import logging +from datetime import datetime, timezone +from enum import Enum +from typing import Any, Optional + +from pydantic import BaseModel, Field + +logger = logging.getLogger(__name__) + + +class Severity(str, Enum): + CRITICAL = "critical" + HIGH = "high" + MEDIUM = "medium" + LOW = "low" + INFO = "info" + + +class GateDecision(str, Enum): + PASS = "pass" + WARN = "warn" + BLOCK = "block" + + +class SecurityFinding(BaseModel): + id: str + category: str + severity: Severity + title: str + description: str + affected_file: Optional[str] = None + affected_endpoint: Optional[str] = None + recommendation: str + found_at: datetime = Field(default_factory=lambda: datetime.now(timezone.utc)) + resolved: bool = False + + +class GateResult(BaseModel): + decision: GateDecision + findings: list[SecurityFinding] = [] + checked_at: datetime = Field(default_factory=lambda: datetime.now(timezone.utc)) + summary: str = "" + + +class SecurityGate: + """ + Security verification gate for Dealix. + Checks auth, PDPL, messaging, and API security. + """ + + def __init__(self): + self._findings: list[SecurityFinding] = [] + + def check_outbound_message( + self, + channel: str, + recipient: str, + tenant_id: str, + has_consent: bool, + consent_purpose: str = None, + ) -> GateResult: + findings = [] + if not has_consent: + findings.append(SecurityFinding( + id=f"MSG-{len(self._findings)+1}", + category="pdpl", + severity=Severity.CRITICAL, + title="رسالة بدون موافقة PDPL", + description=f"محاولة إرسال رسالة {channel} بدون موافقة مسجلة للعميل", + recommendation="يجب الحصول على موافقة العميل قبل الإرسال", + )) + if channel == "whatsapp" and not recipient.startswith("+"): + findings.append(SecurityFinding( + id=f"MSG-{len(self._findings)+2}", + category="validation", + severity=Severity.MEDIUM, + title="رقم واتساب غير صالح", + description=f"الرقم {recipient} لا يبدأ بـ +", + recommendation="استخدم التنسيق الدولي +966XXXXXXXXX", + )) + self._findings.extend(findings) + critical = any(f.severity == Severity.CRITICAL for f in findings) + return GateResult( + decision=GateDecision.BLOCK if critical else GateDecision.PASS, + findings=findings, + summary=f"فحص الرسالة: {'محظور - بدون موافقة' if critical else 'مرخص'}", + ) + + def check_data_access( + self, + user_id: str, + user_role: str, + target_tenant_id: str, + user_tenant_id: str, + action: str, + ) -> GateResult: + findings = [] + if target_tenant_id != user_tenant_id and user_role != "admin": + findings.append(SecurityFinding( + id=f"ACC-{len(self._findings)+1}", + category="authorization", + severity=Severity.CRITICAL, + title="محاولة وصول عبر المستأجرين", + description=f"المستخدم {user_id} حاول الوصول لبيانات مستأجر آخر", + recommendation="رفض الطلب وتسجيل المحاولة", + )) + if action in ("delete", "export") and user_role not in ("owner", "admin"): + findings.append(SecurityFinding( + id=f"ACC-{len(self._findings)+2}", + category="authorization", + severity=Severity.HIGH, + title="إجراء محظور للدور الحالي", + description=f"الدور {user_role} لا يملك صلاحية {action}", + recommendation="ترقية الصلاحيات أو استخدام حساب مخول", + )) + self._findings.extend(findings) + critical = any(f.severity == Severity.CRITICAL for f in findings) + return GateResult( + decision=GateDecision.BLOCK if critical else GateDecision.PASS, + findings=findings, + summary=f"فحص الوصول: {action} - {'محظور' if critical else 'مرخص'}", + ) + + def check_api_request( + self, + endpoint: str, + method: str, + has_auth: bool, + user_role: str = None, + ) -> GateResult: + findings = [] + sensitive_patterns = ["/compliance", "/admin", "/tenant", "/users"] + is_sensitive = any(p in endpoint for p in sensitive_patterns) + if is_sensitive and not has_auth: + findings.append(SecurityFinding( + id=f"API-{len(self._findings)+1}", + category="authentication", + severity=Severity.CRITICAL, + title="وصول لنقطة حساسة بدون مصادقة", + description=f"{method} {endpoint} بدون JWT token", + recommendation="إضافة مصادقة إلزامية", + )) + if is_sensitive and user_role and user_role not in ("owner", "admin"): + findings.append(SecurityFinding( + id=f"API-{len(self._findings)+2}", + category="authorization", + severity=Severity.HIGH, + title="صلاحيات غير كافية", + description=f"الدور {user_role} لا يملك وصول لـ {endpoint}", + recommendation="تحقق من صلاحيات المستخدم", + )) + self._findings.extend(findings) + critical = any(f.severity == Severity.CRITICAL for f in findings) + return GateResult( + decision=GateDecision.BLOCK if critical else GateDecision.PASS, + findings=findings, + summary=f"فحص API: {method} {endpoint}", + ) + + def check_release(self) -> GateResult: + unresolved_critical = [ + f for f in self._findings + if f.severity == Severity.CRITICAL and not f.resolved + ] + unresolved_high = [ + f for f in self._findings + if f.severity == Severity.HIGH and not f.resolved + ] + if unresolved_critical: + decision = GateDecision.BLOCK + summary = f"محظور: {len(unresolved_critical)} مشاكل حرجة غير محلولة" + elif unresolved_high: + decision = GateDecision.WARN + summary = f"تحذير: {len(unresolved_high)} مشاكل عالية غير محلولة" + else: + decision = GateDecision.PASS + summary = "مرخص للإطلاق" + return GateResult( + decision=decision, + findings=unresolved_critical + unresolved_high, + summary=summary, + ) + + def get_all_findings( + self, severity: Severity = None, resolved: bool = None + ) -> list[SecurityFinding]: + results = self._findings + if severity: + results = [f for f in results if f.severity == severity] + if resolved is not None: + results = [f for f in results if f.resolved == resolved] + return results + + def resolve_finding(self, finding_id: str) -> bool: + for f in self._findings: + if f.id == finding_id: + f.resolved = True + return True + return False + + +# Global singleton +security_gate = SecurityGate() diff --git a/salesflow-saas/backend/app/services/sequence_engine.py b/salesflow-saas/backend/app/services/sequence_engine.py new file mode 100644 index 00000000..4b7aa28e --- /dev/null +++ b/salesflow-saas/backend/app/services/sequence_engine.py @@ -0,0 +1,357 @@ +"""Multi-channel sequence engine for Dealix CRM. + +Orchestrates ordered outreach steps across WhatsApp, email, and SMS +with PDPL consent checks, A/B testing, and analytics. +""" + +import logging +import random +from datetime import datetime, timedelta, timezone +from typing import Optional +from uuid import UUID + +from pydantic import BaseModel as Schema +from sqlalchemy import select, func, and_ +from sqlalchemy.ext.asyncio import AsyncSession + +from app.models.sequence import ( + Sequence, SequenceStep, SequenceEnrollment, SequenceEvent, + SequenceStatus, SequenceEventStatus, +) +from app.services.pdpl.consent_manager import ConsentManager + +logger = logging.getLogger(__name__) + + +# --------------------------------------------------------------------------- +# Pydantic schemas +# --------------------------------------------------------------------------- + +class SequenceCreateInput(Schema): + tenant_id: UUID + name: str + name_ar: Optional[str] = None + description: Optional[str] = None + trigger_event: Optional[str] = None + created_by: UUID + steps: list[dict] = [] + + +class EnrollInput(Schema): + sequence_id: UUID + lead_id: UUID + + +class StepProcessResult(Schema): + enrollment_id: UUID + step_id: UUID + channel: str + status: str + message: str + + +class SequenceAnalytics(Schema): + sequence_id: UUID + name: str + total_enrolled: int + active: int + completed: int + stopped: int + total_sent: int + delivered: int + opened: int + replied: int + failed: int + open_rate: float + reply_rate: float + conversion_rate: float + + +# --------------------------------------------------------------------------- +# SequenceEngine +# --------------------------------------------------------------------------- + +class SequenceEngine: + """Manages multi-channel outreach sequences.""" + + def __init__(self, db: AsyncSession): + self.db = db + + # -- create sequence ----------------------------------------------------- + + async def create_sequence(self, data: SequenceCreateInput) -> Sequence: + """Create a new sequence with optional steps.""" + + seq = Sequence( + tenant_id=data.tenant_id, + name=data.name, + name_ar=data.name_ar, + description=data.description, + trigger_event=data.trigger_event, + is_active=True, + created_by=data.created_by, + ) + self.db.add(seq) + await self.db.flush() + + for i, step_data in enumerate(data.steps): + step = SequenceStep( + sequence_id=seq.id, + step_order=i + 1, + channel=step_data.get("channel", "email"), + delay_minutes=step_data.get("delay_minutes", 0), + template_content=step_data.get("template_content", ""), + template_content_ar=step_data.get("template_content_ar"), + variant=step_data.get("variant"), + conditions=step_data.get("conditions", {}), + ) + self.db.add(step) + + await self.db.flush() + await self.db.refresh(seq) + logger.info("Sequence created: id=%s name=%s", seq.id, seq.name) + return seq + + # -- enroll lead --------------------------------------------------------- + + async def enroll_lead(self, data: EnrollInput) -> SequenceEnrollment: + """Enroll a lead into a sequence. Starts at step 0.""" + + # Prevent duplicate active enrollments + existing = await self.db.execute( + select(SequenceEnrollment).where( + SequenceEnrollment.sequence_id == data.sequence_id, + SequenceEnrollment.lead_id == data.lead_id, + SequenceEnrollment.status == SequenceStatus.ACTIVE.value, + ) + ) + if existing.scalar_one_or_none(): + raise ValueError("العميل المحتمل مسجل بالفعل في هذا التسلسل") # Lead already enrolled + + # Fetch first step to calculate next_step_at + first_step = await self.db.execute( + select(SequenceStep).where(SequenceStep.sequence_id == data.sequence_id) + .order_by(SequenceStep.step_order).limit(1) + ) + step = first_step.scalar_one_or_none() + now = datetime.now(timezone.utc) + next_at = now + timedelta(minutes=step.delay_minutes) if step else None + + enrollment = SequenceEnrollment( + sequence_id=data.sequence_id, + lead_id=data.lead_id, + current_step=0, + status=SequenceStatus.ACTIVE.value, + enrolled_at=now, + next_step_at=next_at, + ) + self.db.add(enrollment) + await self.db.flush() + await self.db.refresh(enrollment) + logger.info("Lead enrolled: lead=%s sequence=%s", data.lead_id, data.sequence_id) + return enrollment + + # -- process pending steps ----------------------------------------------- + + async def process_pending_steps(self, tenant_id: UUID) -> list[StepProcessResult]: + """Process all enrollments whose next step is due. Checks PDPL consent.""" + + now = datetime.now(timezone.utc) + consent_mgr = ConsentManager(self.db) + results: list[StepProcessResult] = [] + + # Find active enrollments that are due + query = ( + select(SequenceEnrollment) + .join(Sequence, Sequence.id == SequenceEnrollment.sequence_id) + .where( + Sequence.tenant_id == tenant_id, + Sequence.is_active == True, + SequenceEnrollment.status == SequenceStatus.ACTIVE.value, + SequenceEnrollment.next_step_at <= now, + ) + .limit(200) + ) + rows = await self.db.execute(query) + enrollments = rows.scalars().all() + + for enrollment in enrollments: + result = await self._execute_next_step(enrollment, consent_mgr) + if result: + results.append(result) + + logger.info("Processed %d pending steps for tenant=%s", len(results), tenant_id) + return results + + # -- pause / resume / stop ----------------------------------------------- + + async def pause_enrollment(self, enrollment_id: UUID) -> SequenceEnrollment: + enrollment = await self._get_enrollment(enrollment_id) + enrollment.status = SequenceStatus.PAUSED.value + await self.db.flush() + logger.info("Enrollment paused: %s", enrollment_id) + return enrollment + + async def resume_enrollment(self, enrollment_id: UUID) -> SequenceEnrollment: + enrollment = await self._get_enrollment(enrollment_id) + enrollment.status = SequenceStatus.ACTIVE.value + enrollment.next_step_at = datetime.now(timezone.utc) + await self.db.flush() + logger.info("Enrollment resumed: %s", enrollment_id) + return enrollment + + async def stop_enrollment(self, enrollment_id: UUID) -> SequenceEnrollment: + enrollment = await self._get_enrollment(enrollment_id) + enrollment.status = SequenceStatus.STOPPED.value + enrollment.completed_at = datetime.now(timezone.utc) + await self.db.flush() + logger.info("Enrollment stopped: %s", enrollment_id) + return enrollment + + # -- analytics ----------------------------------------------------------- + + async def get_sequence_analytics(self, sequence_id: UUID) -> SequenceAnalytics: + """Compute open/response/conversion rates for a sequence.""" + + seq = (await self.db.execute( + select(Sequence).where(Sequence.id == sequence_id) + )).scalar_one_or_none() + if not seq: + raise ValueError("التسلسل غير موجود") + + # Enrollment counts + def _count_enrollments(status: str): + return select(func.count()).where( + SequenceEnrollment.sequence_id == sequence_id, + SequenceEnrollment.status == status, + ) + + total = (await self.db.execute( + select(func.count()).where(SequenceEnrollment.sequence_id == sequence_id) + )).scalar() or 0 + active = (await self.db.execute(_count_enrollments(SequenceStatus.ACTIVE.value))).scalar() or 0 + completed = (await self.db.execute(_count_enrollments(SequenceStatus.COMPLETED.value))).scalar() or 0 + stopped = (await self.db.execute(_count_enrollments(SequenceStatus.STOPPED.value))).scalar() or 0 + + # Event counts + base_event = ( + select(func.count()) + .select_from(SequenceEvent) + .join(SequenceEnrollment, SequenceEnrollment.id == SequenceEvent.enrollment_id) + .where(SequenceEnrollment.sequence_id == sequence_id) + ) + + total_sent = (await self.db.execute(base_event)).scalar() or 0 + delivered = (await self.db.execute( + base_event.where(SequenceEvent.status.in_(["delivered", "opened", "replied"])) + )).scalar() or 0 + opened = (await self.db.execute( + base_event.where(SequenceEvent.status.in_(["opened", "replied"])) + )).scalar() or 0 + replied = (await self.db.execute( + base_event.where(SequenceEvent.status == "replied") + )).scalar() or 0 + failed = (await self.db.execute( + base_event.where(SequenceEvent.status == "failed") + )).scalar() or 0 + + safe_div = lambda n, d: round(n / d * 100, 2) if d else 0.0 + + return SequenceAnalytics( + sequence_id=sequence_id, + name=seq.name, + total_enrolled=total, + active=active, + completed=completed, + stopped=stopped, + total_sent=total_sent, + delivered=delivered, + opened=opened, + replied=replied, + failed=failed, + open_rate=safe_div(opened, total_sent), + reply_rate=safe_div(replied, total_sent), + conversion_rate=safe_div(completed, total) if total else 0.0, + ) + + # -- private helpers ----------------------------------------------------- + + async def _execute_next_step( + self, enrollment: SequenceEnrollment, consent_mgr: ConsentManager, + ) -> Optional[StepProcessResult]: + """Execute the next step for an enrollment.""" + + steps_q = await self.db.execute( + select(SequenceStep) + .where(SequenceStep.sequence_id == enrollment.sequence_id) + .order_by(SequenceStep.step_order) + ) + steps = steps_q.scalars().all() + next_idx = enrollment.current_step + if next_idx >= len(steps): + enrollment.status = SequenceStatus.COMPLETED.value + enrollment.completed_at = datetime.now(timezone.utc) + await self.db.flush() + return None + + # A/B test: pick variant randomly if multiple exist for same order + candidates = [s for s in steps if s.step_order == steps[next_idx].step_order] + step = random.choice(candidates) if len(candidates) > 1 else steps[next_idx] + + # PDPL consent check + seq = (await self.db.execute( + select(Sequence).where(Sequence.id == enrollment.sequence_id) + )).scalar_one() + consent_result = await consent_mgr.check_consent( + contact_id=enrollment.lead_id, + purpose="marketing", + channel=step.channel, + ) + if not consent_result.allowed: + event = SequenceEvent( + enrollment_id=enrollment.id, step_id=step.id, + channel=step.channel, status=SequenceEventStatus.FAILED.value, + metadata={"reason": "no_consent", "message": consent_result.message}, + ) + self.db.add(event) + enrollment.status = SequenceStatus.STOPPED.value + await self.db.flush() + return StepProcessResult( + enrollment_id=enrollment.id, step_id=step.id, + channel=step.channel, status="failed", + message=f"PDPL consent denied: {consent_result.message}", + ) + + # Record send event + event = SequenceEvent( + enrollment_id=enrollment.id, step_id=step.id, + channel=step.channel, status=SequenceEventStatus.SENT.value, + metadata={"variant": step.variant, "template_preview": step.template_content[:100]}, + ) + self.db.add(event) + + # Advance enrollment + enrollment.current_step = next_idx + 1 + if enrollment.current_step >= len(steps): + enrollment.status = SequenceStatus.COMPLETED.value + enrollment.completed_at = datetime.now(timezone.utc) + enrollment.next_step_at = None + else: + next_step = steps[enrollment.current_step] + enrollment.next_step_at = datetime.now(timezone.utc) + timedelta(minutes=next_step.delay_minutes) + + await self.db.flush() + return StepProcessResult( + enrollment_id=enrollment.id, step_id=step.id, + channel=step.channel, status="sent", + message=f"Step {next_idx + 1} sent via {step.channel}", + ) + + async def _get_enrollment(self, enrollment_id: UUID) -> SequenceEnrollment: + result = await self.db.execute( + select(SequenceEnrollment).where(SequenceEnrollment.id == enrollment_id) + ) + enrollment = result.scalar_one_or_none() + if not enrollment: + raise ValueError("التسجيل غير موجود") # Enrollment not found + return enrollment diff --git a/salesflow-saas/backend/app/services/tool_verification.py b/salesflow-saas/backend/app/services/tool_verification.py new file mode 100644 index 00000000..7cd42d0d --- /dev/null +++ b/salesflow-saas/backend/app/services/tool_verification.py @@ -0,0 +1,176 @@ +""" +Tool Verification Layer — Dealix AI Revenue OS +Records what agents intended, claimed, and actually executed. +Provides evidence-based audit trail for all AI actions. +""" +import logging +import uuid +from datetime import datetime, timezone +from enum import Enum +from typing import Any, Optional + +from pydantic import BaseModel, Field + +logger = logging.getLogger(__name__) + + +class VerificationStatus(str, Enum): + VERIFIED = "verified" + PARTIALLY_VERIFIED = "partially_verified" + UNVERIFIED = "unverified" + CONTRADICTED = "contradicted" + PENDING = "pending" + + +class ToolCall(BaseModel): + request_id: str = Field(default_factory=lambda: str(uuid.uuid4())) + agent_id: str + agent_name: str + intended_action: str + intended_params: dict[str, Any] = {} + claimed_result: Optional[str] = None + actual_result: Optional[str] = None + actual_side_effects: list[str] = [] + status: VerificationStatus = VerificationStatus.PENDING + contradiction_flags: list[str] = [] + started_at: datetime = Field(default_factory=lambda: datetime.now(timezone.utc)) + completed_at: Optional[datetime] = None + duration_ms: Optional[int] = None + tenant_id: Optional[str] = None + metadata: dict[str, Any] = {} + + +class ToolVerifier: + """ + Verification layer between agents and tools. + Records intent vs claim vs execution evidence. + """ + + def __init__(self): + self._log: list[ToolCall] = [] + self._max_log_size = 10000 + + def start_call( + self, + agent_id: str, + agent_name: str, + intended_action: str, + intended_params: dict[str, Any] = None, + tenant_id: str = None, + ) -> ToolCall: + call = ToolCall( + agent_id=agent_id, + agent_name=agent_name, + intended_action=intended_action, + intended_params=intended_params or {}, + tenant_id=tenant_id, + ) + self._log.append(call) + if len(self._log) > self._max_log_size: + self._log = self._log[-self._max_log_size:] + logger.info( + f"[ToolVerify] START {call.request_id}: " + f"agent={agent_name} action={intended_action}" + ) + return call + + def record_claim(self, call: ToolCall, claimed_result: str) -> None: + call.claimed_result = claimed_result + logger.info( + f"[ToolVerify] CLAIM {call.request_id}: {claimed_result[:200]}" + ) + + def record_execution( + self, + call: ToolCall, + actual_result: str, + side_effects: list[str] = None, + ) -> None: + call.actual_result = actual_result + call.actual_side_effects = side_effects or [] + call.completed_at = datetime.now(timezone.utc) + call.duration_ms = int( + (call.completed_at - call.started_at).total_seconds() * 1000 + ) + self._verify(call) + logger.info( + f"[ToolVerify] EXEC {call.request_id}: " + f"status={call.status} duration={call.duration_ms}ms" + ) + + def _verify(self, call: ToolCall) -> None: + if not call.claimed_result or not call.actual_result: + call.status = VerificationStatus.UNVERIFIED + return + + claimed = call.claimed_result.lower().strip() + actual = call.actual_result.lower().strip() + + if "error" in actual and "success" in claimed: + call.status = VerificationStatus.CONTRADICTED + call.contradiction_flags.append( + "Agent claimed success but execution returned error" + ) + elif "error" in actual: + call.status = VerificationStatus.PARTIALLY_VERIFIED + call.contradiction_flags.append("Execution had errors") + elif actual == claimed or claimed in actual: + call.status = VerificationStatus.VERIFIED + else: + keywords_claimed = set(claimed.split()) + keywords_actual = set(actual.split()) + overlap = len(keywords_claimed & keywords_actual) + total = len(keywords_claimed) + if total > 0 and overlap / total > 0.5: + call.status = VerificationStatus.VERIFIED + elif total > 0 and overlap / total > 0.2: + call.status = VerificationStatus.PARTIALLY_VERIFIED + else: + call.status = VerificationStatus.UNVERIFIED + + def get_log( + self, + agent_id: str = None, + status: VerificationStatus = None, + tenant_id: str = None, + limit: int = 100, + ) -> list[ToolCall]: + results = self._log + if agent_id: + results = [c for c in results if c.agent_id == agent_id] + if status: + results = [c for c in results if c.status == status] + if tenant_id: + results = [c for c in results if c.tenant_id == tenant_id] + return results[-limit:] + + def get_contradictions(self, tenant_id: str = None) -> list[ToolCall]: + return self.get_log( + status=VerificationStatus.CONTRADICTED, tenant_id=tenant_id + ) + + def get_stats(self, tenant_id: str = None) -> dict[str, Any]: + calls = self.get_log(tenant_id=tenant_id, limit=10000) + total = len(calls) + if total == 0: + return {"total": 0} + by_status = {} + for call in calls: + by_status[call.status] = by_status.get(call.status, 0) + 1 + durations = [c.duration_ms for c in calls if c.duration_ms] + avg_duration = sum(durations) / len(durations) if durations else 0 + return { + "total": total, + "by_status": by_status, + "avg_duration_ms": round(avg_duration, 1), + "contradiction_rate": round( + by_status.get(VerificationStatus.CONTRADICTED, 0) / total * 100, 2 + ), + "verification_rate": round( + by_status.get(VerificationStatus.VERIFIED, 0) / total * 100, 2 + ), + } + + +# Global singleton +tool_verifier = ToolVerifier() diff --git a/salesflow-saas/frontend/src/design-tokens.ts b/salesflow-saas/frontend/src/design-tokens.ts new file mode 100644 index 00000000..2a4b02bf --- /dev/null +++ b/salesflow-saas/frontend/src/design-tokens.ts @@ -0,0 +1,202 @@ +/** + * Dealix Design System Tokens + * Premium, Arabic-first, RTL-safe design system + */ + +export const typography = { + fontFamily: { + primary: "'IBM Plex Sans Arabic', 'Tajawal', sans-serif", + display: "'IBM Plex Sans Arabic', sans-serif", + mono: "'IBM Plex Mono', monospace", + body: "'IBM Plex Sans Arabic', 'Inter', sans-serif", + }, + fontSize: { + xs: "0.75rem", // 12px + sm: "0.875rem", // 14px + base: "1rem", // 16px + lg: "1.125rem", // 18px + xl: "1.25rem", // 20px + "2xl": "1.5rem", // 24px + "3xl": "1.875rem", // 30px + "4xl": "2.25rem", // 36px + "5xl": "3rem", // 48px + hero: "3.75rem", // 60px + }, + fontWeight: { + light: 300, + regular: 400, + medium: 500, + semibold: 600, + bold: 700, + }, + lineHeight: { + tight: 1.25, + normal: 1.5, + relaxed: 1.75, + arabic: 1.8, // Arabic text needs more line height + }, +} as const; + +export const colors = { + // Primary - Professional blue (trust, reliability) + primary: { + 50: "#EFF6FF", + 100: "#DBEAFE", + 200: "#BFDBFE", + 300: "#93C5FD", + 400: "#60A5FA", + 500: "#3B82F6", // Main primary + 600: "#2563EB", + 700: "#1D4ED8", + 800: "#1E40AF", + 900: "#1E3A8A", + }, + // Secondary - Teal (growth, Saudi green connection) + secondary: { + 50: "#F0FDFA", + 100: "#CCFBF1", + 200: "#99F6E4", + 300: "#5EEAD4", + 400: "#2DD4BF", + 500: "#14B8A6", // Main secondary + 600: "#0D9488", + 700: "#0F766E", + 800: "#115E59", + 900: "#134E4A", + }, + // Accent - Warm orange (action, energy) + accent: { + 50: "#FFF7ED", + 100: "#FFEDD5", + 200: "#FED7AA", + 300: "#FDBA74", + 400: "#FB923C", + 500: "#F97316", // Main accent + 600: "#EA580C", + 700: "#C2410C", + }, + // Neutrals + neutral: { + 0: "#FFFFFF", + 50: "#F9FAFB", + 100: "#F3F4F6", + 200: "#E5E7EB", + 300: "#D1D5DB", + 400: "#9CA3AF", + 500: "#6B7280", + 600: "#4B5563", + 700: "#374151", + 800: "#1F2937", + 900: "#111827", + 950: "#030712", + }, + // Semantic + success: { light: "#DCFCE7", main: "#22C55E", dark: "#15803D" }, + warning: { light: "#FEF3C7", main: "#F59E0B", dark: "#B45309" }, + error: { light: "#FEE2E2", main: "#EF4444", dark: "#B91C1C" }, + info: { light: "#DBEAFE", main: "#3B82F6", dark: "#1D4ED8" }, +} as const; + +export const spacing = { + 0: "0", + 1: "0.25rem", // 4px + 2: "0.5rem", // 8px + 3: "0.75rem", // 12px + 4: "1rem", // 16px + 5: "1.25rem", // 20px + 6: "1.5rem", // 24px + 8: "2rem", // 32px + 10: "2.5rem", // 40px + 12: "3rem", // 48px + 16: "4rem", // 64px + 20: "5rem", // 80px + 24: "6rem", // 96px + section: "5rem", // Section padding +} as const; + +export const borderRadius = { + none: "0", + sm: "0.25rem", + md: "0.375rem", + lg: "0.5rem", + xl: "0.75rem", + "2xl": "1rem", + full: "9999px", + card: "0.75rem", + button: "0.5rem", + input: "0.375rem", +} as const; + +export const shadows = { + sm: "0 1px 2px 0 rgb(0 0 0 / 0.05)", + md: "0 4px 6px -1px rgb(0 0 0 / 0.1)", + lg: "0 10px 15px -3px rgb(0 0 0 / 0.1)", + xl: "0 20px 25px -5px rgb(0 0 0 / 0.1)", + card: "0 1px 3px 0 rgb(0 0 0 / 0.1), 0 1px 2px -1px rgb(0 0 0 / 0.1)", + elevated: "0 4px 6px -1px rgb(0 0 0 / 0.1), 0 2px 4px -2px rgb(0 0 0 / 0.1)", + popup: "0 25px 50px -12px rgb(0 0 0 / 0.25)", +} as const; + +export const breakpoints = { + sm: "640px", + md: "768px", + lg: "1024px", + xl: "1280px", + "2xl": "1536px", +} as const; + +export const animation = { + duration: { + instant: "0ms", + fast: "150ms", + normal: "250ms", + slow: "400ms", + }, + easing: { + default: "cubic-bezier(0.4, 0, 0.2, 1)", + in: "cubic-bezier(0.4, 0, 1, 1)", + out: "cubic-bezier(0, 0, 0.2, 1)", + spring: "cubic-bezier(0.175, 0.885, 0.32, 1.275)", + }, +} as const; + +// RTL-specific tokens +export const rtl = { + direction: "rtl" as const, + textAlign: "right" as const, + // Logical properties for RTL-safe spacing + marginStart: "margin-inline-start", + marginEnd: "margin-inline-end", + paddingStart: "padding-inline-start", + paddingEnd: "padding-inline-end", + borderStart: "border-inline-start", + borderEnd: "border-inline-end", +} as const; + +// Component-specific tokens +export const components = { + button: { + height: { sm: "2rem", md: "2.5rem", lg: "3rem" }, + padding: { sm: "0.5rem 1rem", md: "0.625rem 1.25rem", lg: "0.75rem 1.5rem" }, + fontSize: { sm: "0.875rem", md: "1rem", lg: "1.125rem" }, + }, + input: { + height: { sm: "2rem", md: "2.5rem", lg: "3rem" }, + padding: "0.5rem 0.75rem", + borderColor: colors.neutral[300], + focusBorderColor: colors.primary[500], + }, + card: { + padding: "1.5rem", + borderRadius: borderRadius.card, + shadow: shadows.card, + background: colors.neutral[0], + }, + sidebar: { + width: "280px", + collapsedWidth: "64px", + }, + header: { + height: "64px", + }, +} as const; diff --git a/salesflow-saas/memory/adr/001-multi-tenant.md b/salesflow-saas/memory/adr/001-multi-tenant.md new file mode 100644 index 00000000..f74be2e1 --- /dev/null +++ b/salesflow-saas/memory/adr/001-multi-tenant.md @@ -0,0 +1,23 @@ +# ADR-001: Multi-Tenant Data Isolation + +**Status**: accepted +**Date**: 2026-03-28 +**Decision**: Row-level tenant isolation with tenant_id on every table + +## Context +Dealix serves multiple Saudi SMBs. Each company's data must be completely isolated. + +## Decision +Use row-level isolation with `tenant_id` foreign key on every data table. All queries filter by tenant_id automatically. + +## Rationale +- Simpler than schema-per-tenant for our scale (< 10K tenants initially) +- Lower operational cost (single database) +- Easier migrations +- Good enough isolation for SMB CRM data + +## Consequences +- Must enforce tenant_id in every query (risk of data leak if missed) +- Use SQLAlchemy query filters/middleware to auto-add tenant_id +- Performance monitoring needed as tenant count grows +- Future: consider schema-per-tenant for enterprise customers diff --git a/salesflow-saas/memory/adr/002-whatsapp-first.md b/salesflow-saas/memory/adr/002-whatsapp-first.md new file mode 100644 index 00000000..cacd7df1 --- /dev/null +++ b/salesflow-saas/memory/adr/002-whatsapp-first.md @@ -0,0 +1,28 @@ +# ADR-002: WhatsApp as Primary Communication Channel + +**Status**: accepted +**Date**: 2026-03-28 +**Decision**: Build WhatsApp as a first-class CRM channel, not a third-party add-on + +## Context +WhatsApp has 85%+ penetration in Saudi Arabia (30M+ users). It is THE primary business communication channel. Competitors (Salesforce, HubSpot) treat WhatsApp as a third-party integration. + +## Decision +WhatsApp Business API is integrated directly into the core platform: +- Inbound webhooks create/update leads automatically +- AI chatbot handles initial qualification in Arabic +- Unified inbox merges WhatsApp with Email and SMS +- Sequences support WhatsApp as a step type +- Proposals can be sent and tracked via WhatsApp + +## Rationale +- 85%+ of Saudi business communication happens on WhatsApp +- No global CRM treats WhatsApp as the primary channel +- This is Dealix's strongest competitive moat in KSA +- Close.com proved that building communication natively wins vs third-party + +## Consequences +- Must maintain Meta Business API compliance +- Template messages need pre-approval from Meta +- 24-hour messaging window rules apply +- PDPL consent must be checked before every message diff --git a/salesflow-saas/memory/architecture/system-overview.md b/salesflow-saas/memory/architecture/system-overview.md new file mode 100644 index 00000000..ede8602e --- /dev/null +++ b/salesflow-saas/memory/architecture/system-overview.md @@ -0,0 +1,52 @@ +# Dealix System Architecture Overview + +**Type**: architecture +**Date**: 2026-04-11 +**Status**: active +**Confidence**: high + +## Summary +Dealix is a multi-tenant AI-powered CRM SaaS targeting Saudi SMBs. Architecture follows a microservices-ready monolith pattern. + +## Components +``` +┌─────────────────────────────────────────────────────────┐ +│ Nginx (Reverse Proxy) │ +├──────────────────────┬──────────────────────────────────┤ +│ Next.js Frontend │ FastAPI Backend │ +│ (Port 3000) │ (Port 8000) │ +│ - Dashboard │ ┌─────────────────────────┐ │ +│ - Landing │ │ API Layer (v1) │ │ +│ - Auth │ │ - Auth, Leads, Deals │ │ +│ - Pipeline │ │ - Inbox, Sequences │ │ +│ │ │ - Compliance, Proposals │ │ +│ │ ├─────────────────────────┤ │ +│ │ │ Services Layer │ │ +│ │ │ - AI Engine (Arabic) │ │ +│ │ │ - PDPL Compliance │ │ +│ │ │ - Sequence Engine │ │ +│ │ │ - CPQ System │ │ +│ │ │ - Agent Orchestrator │ │ +│ │ ├─────────────────────────┤ │ +│ │ │ Integration Layer │ │ +│ │ │ - WhatsApp, Email, SMS │ │ +│ │ │ - Stripe, ZATCA │ │ +├──────────────────────┴───┴─────────────────────────┤ │ +│ Celery Workers (4) │ Celery Beat │ │ +├──────────────────────────────┴──────────────────────┤ │ +│ PostgreSQL 16 │ Redis 7 │ │ +└─────────────────────┴───────────────────────────────┘ +``` + +## Key Design Decisions +- **Multi-tenant isolation**: tenant_id on every table, enforced at query level +- **Arabic-first**: RTL layout, Arabic NLP, Saudi dialect support +- **WhatsApp-first**: Primary communication channel (85% Saudi penetration) +- **PDPL-native**: Consent checked before every outbound message +- **LLM fallback chain**: Groq → OpenAI for cost optimization +- **Async everything**: asyncpg, async SQLAlchemy, async HTTP clients + +## Related Topics +- [ADR-001: Multi-tenant architecture](../adr/001-multi-tenant.md) +- [ADR-002: WhatsApp as primary channel](../adr/002-whatsapp-first.md) +- [Provider routing strategy](../providers/routing-strategy.md) diff --git a/salesflow-saas/memory/providers/routing-strategy.md b/salesflow-saas/memory/providers/routing-strategy.md new file mode 100644 index 00000000..d1564cb2 --- /dev/null +++ b/salesflow-saas/memory/providers/routing-strategy.md @@ -0,0 +1,36 @@ +# LLM Provider Routing Strategy + +**Type**: provider-config +**Date**: 2026-04-11 +**Status**: active + +## Provider Stack + +| Provider | Use Case | Latency | Cost | Arabic Quality | +|----------|----------|---------|------|----------------| +| Groq (llama-3.1-70b) | Fast classification, scoring | ~200ms | Free tier | Good | +| Groq (llama-3.1-8b) | Simple tasks, routing | ~100ms | Free tier | Adequate | +| OpenAI (gpt-4o-mini) | Fallback, complex reasoning | ~1-2s | $0.15/1M in | Very Good | +| OpenAI (gpt-4o) | Premium tasks, proposals | ~2-3s | $2.50/1M in | Excellent | +| Claude (via API) | Sales copy, proposals | ~2-3s | $3/1M in | Excellent | +| DeepSeek | Code generation | ~1-2s | Low | N/A | + +## Routing Rules + +1. **Intent Detection**: Groq llama-3.1-8b (speed priority) +2. **Lead Scoring**: Groq llama-3.1-70b (accuracy needed) +3. **Arabic NLP**: Groq llama-3.1-70b (good Arabic, fast) +4. **Message Writing**: OpenAI gpt-4o-mini (quality Arabic output) +5. **Proposal Generation**: Claude (best long-form Arabic) +6. **Conversation Summary**: Groq llama-3.1-70b (speed + quality balance) +7. **Forecasting**: OpenAI gpt-4o-mini (reasoning needed) + +## Fallback Chain +Primary → Secondary → Emergency: +- Groq → OpenAI gpt-4o-mini → local cached response +- OpenAI → Groq → error with retry + +## Cost Budget +- Target: < $50/month for 100 active tenants +- Groq free tier covers ~80% of requests +- OpenAI handles remaining 20% premium tasks diff --git a/salesflow-saas/memory/security/pdpl-checklist.md b/salesflow-saas/memory/security/pdpl-checklist.md new file mode 100644 index 00000000..15788030 --- /dev/null +++ b/salesflow-saas/memory/security/pdpl-checklist.md @@ -0,0 +1,46 @@ +# PDPL Compliance Checklist + +**Type**: security +**Date**: 2026-04-11 +**Status**: active +**Owner**: compliance team + +## Pre-Launch Requirements + +### Consent Management +- [ ] Consent recorded before any data processing +- [ ] Consent purpose is specific (marketing/sales/service/analytics) +- [ ] Consent channel tracked (WhatsApp/email/SMS/phone) +- [ ] Re-consent triggered when purpose changes +- [ ] Consent expiry enforced (12 months default) +- [ ] Consent audit trail complete + +### Data Subject Rights +- [ ] Right to access: export all personal data as JSON +- [ ] Right to correction: update with audit trail +- [ ] Right to deletion: soft-delete + 30-day hard-delete +- [ ] Right to restrict processing: flag and enforce +- [ ] Response within 30 days of request + +### Cross-Border Transfer +- [ ] All data stored in Saudi/GCC data centers +- [ ] No personal data sent to non-adequate countries without consent +- [ ] Transfer safeguards documented + +### Security +- [ ] Data encryption at rest (PostgreSQL TDE or app-level) +- [ ] Data encryption in transit (TLS 1.3) +- [ ] Access control: role-based, tenant-isolated +- [ ] Audit logs for all data access +- [ ] Breach notification procedure documented + +### Penalties +- Up to SAR 5,000,000 per violation +- Double for repeat offenses +- Up to 1 year imprisonment for unauthorized cross-border transfers + +## SDAIA Registration +- [ ] Register on National Data Governance Platform +- [ ] Appoint Data Protection Officer +- [ ] Document processing activities +- [ ] Conduct Data Protection Impact Assessment