mirror of
https://github.com/x1xhlol/system-prompts-and-models-of-ai-tools.git
synced 2026-06-18 07:19:35 +00:00
195 lines
7.1 KiB
Markdown
195 lines
7.1 KiB
Markdown
# Architecture | معمارية النظام
|
||
|
||
## System overview
|
||
|
||
```mermaid
|
||
graph TB
|
||
subgraph Clients["Clients & Sources"]
|
||
W[Website Forms]
|
||
WA[WhatsApp Business]
|
||
E[Email]
|
||
LI[LinkedIn]
|
||
end
|
||
|
||
subgraph Gateway["FastAPI Gateway (api/main.py)"]
|
||
MW[Middleware: RequestID, CORS, structured logs]
|
||
R1[/api/v1/leads/]
|
||
R2[/api/v1/sales/]
|
||
R3[/api/v1/sectors/]
|
||
R4[/api/v1/agents/]
|
||
R5[/api/v1/webhooks/]
|
||
R6[/health/]
|
||
end
|
||
|
||
subgraph Pipeline["Phase 8 — Acquisition Pipeline"]
|
||
direction TB
|
||
P1[Intake] --> P2[Pain Extractor]
|
||
P2 --> P3[ICP Matcher]
|
||
P3 --> P4[Qualification]
|
||
P4 --> P5[CRM Sync]
|
||
P5 --> P6[Booking]
|
||
P6 --> P7[Proposal]
|
||
end
|
||
|
||
subgraph Growth["Phase 9 — Growth"]
|
||
G1[Sector Intel]
|
||
G2[Market Research]
|
||
G3[Content Creator]
|
||
G4[Distribution]
|
||
G5[Enrichment]
|
||
G6[Competitor Monitor]
|
||
end
|
||
|
||
subgraph LLM["LLM Router (core/llm/router.py)"]
|
||
CL[Anthropic Claude<br/>reasoning, writing]
|
||
GM[Gemini<br/>research, multimodal]
|
||
GQ[Groq<br/>fast classification]
|
||
DS[DeepSeek<br/>code]
|
||
GL[GLM<br/>Arabic, bulk]
|
||
end
|
||
|
||
subgraph Storage["Persistence"]
|
||
PG[(PostgreSQL<br/>Leads, Deals)]
|
||
RD[(Redis<br/>cache, queues)]
|
||
MG[(MongoDB<br/>unstructured)]
|
||
end
|
||
|
||
subgraph External["External Services"]
|
||
HS[HubSpot CRM]
|
||
CAL[Calendly / Google Cal]
|
||
EMAIL[Resend / SendGrid]
|
||
N8N[n8n workflows]
|
||
LF[Langfuse tracing]
|
||
end
|
||
|
||
Clients --> Gateway
|
||
R1 --> Pipeline
|
||
R3 --> Growth
|
||
Pipeline --> LLM
|
||
Growth --> LLM
|
||
Pipeline --> Storage
|
||
Growth --> Storage
|
||
Pipeline --> External
|
||
Growth --> External
|
||
LLM -.tracing.-> LF
|
||
```
|
||
|
||
## Key design decisions
|
||
|
||
### 1. `.env`-only configuration via `pydantic-settings`
|
||
|
||
All secrets live in `core/config/settings.py`, wrapped in `SecretStr`. No module may read env vars directly — they go through `get_settings()`. This keeps secrets centralized, typed, and out of application code.
|
||
|
||
### 2. Task → Provider routing with fallback
|
||
|
||
`core/config/models.py` declares a routing table mapping every `Task` enum value to a primary `Provider`, plus a fallback chain. The `ModelRouter` tries primary, then each fallback, recording usage metrics and fallback triggers. This means **any single LLM outage is non-fatal** for the system.
|
||
|
||
### 3. Agents share a common base class
|
||
|
||
`core/agents/base.py::BaseAgent` provides:
|
||
- A bound structlog logger with `agent`/`agent_id` context
|
||
- Access to the shared `ModelRouter`
|
||
- A robust `parse_json_response()` utility that handles fenced blocks, raw JSON, and messy LLM output
|
||
|
||
Every agent exposes a single `async def run(**kwargs)` method — easy to compose.
|
||
|
||
### 4. Pipeline as orchestrated, not monolithic
|
||
|
||
`auto_client_acquisition/pipeline.py::AcquisitionPipeline` runs steps sequentially but **wraps each step in its own try/except**. If ICP match fails, qualification still runs. The final `PipelineResult` object contains a `warnings` list listing any non-fatal failures. This mirrors real-world sales ops where you want partial data rather than total failure.
|
||
|
||
### 5. Bilingual first-class
|
||
|
||
Arabic is not an afterthought:
|
||
- Locale detection in `core/utils.py` from Unicode ranges
|
||
- Arabic-heavy tasks routed to GLM by default (`Task.ARABIC_TASKS`)
|
||
- Sales scripts and prompts ship in both AR and EN
|
||
- Content agent picks GLM or Claude based on target locale
|
||
|
||
### 6. Security defaults
|
||
|
||
- `.gitignore` blocks `.env*` except `.env.example`
|
||
- Pre-commit runs `gitleaks` + `detect-secrets` on every commit
|
||
- CI re-runs those + `trufflehog` on every PR
|
||
- LinkedIn integration disabled by default (explicit ToS choice)
|
||
- Webhook signatures verified (`WhatsAppClient.verify_signature`)
|
||
- Docker container runs as non-root `app` user
|
||
|
||
### 7. Observable
|
||
|
||
- Every request gets a `X-Request-ID` header (via `RequestIDMiddleware`)
|
||
- All logs are structured JSON in production (structlog)
|
||
- LLM router tracks calls, tokens, errors, and fallback count per provider
|
||
- Optional Langfuse integration for LLM-level tracing
|
||
|
||
## Folder map
|
||
|
||
```
|
||
ai-company-saudi/
|
||
├── core/ # foundation
|
||
│ ├── config/ # settings + routing table
|
||
│ ├── llm/ # provider clients + router
|
||
│ ├── agents/ # BaseAgent + orchestrator
|
||
│ ├── prompts/ # engineered prompts + sales scripts
|
||
│ ├── logging.py
|
||
│ ├── errors.py
|
||
│ └── utils.py
|
||
├── auto_client_acquisition/ # Phase 8
|
||
│ ├── agents/ # 9 agents
|
||
│ └── pipeline.py # orchestrator
|
||
├── autonomous_growth/ # Phase 9
|
||
│ ├── agents/ # 6 agents
|
||
│ └── orchestrator.py
|
||
├── integrations/ # external APIs
|
||
│ ├── whatsapp.py
|
||
│ ├── email.py
|
||
│ ├── calendar.py
|
||
│ ├── hubspot.py
|
||
│ ├── linkedin.py
|
||
│ ├── n8n.py
|
||
│ └── saudi_market.py
|
||
├── api/ # FastAPI app
|
||
│ ├── main.py # factory
|
||
│ ├── routers/ # 6 routers
|
||
│ ├── schemas/ # Pydantic models
|
||
│ ├── middleware.py
|
||
│ └── dependencies.py # DI
|
||
├── db/ # persistence
|
||
│ ├── models.py # SQLAlchemy 2.0
|
||
│ ├── session.py
|
||
│ └── migrations/ # Alembic
|
||
├── dashboard/
|
||
│ └── analytics.py # KPI aggregation
|
||
├── tests/
|
||
│ ├── conftest.py # fixtures + LLM mocks
|
||
│ ├── unit/ # intake, ICP, pain, router
|
||
│ └── integration/ # API, pipeline
|
||
├── scripts/ # seed + demo
|
||
├── docs/ # this folder
|
||
├── .github/ # CI/CD + templates
|
||
├── Dockerfile
|
||
├── docker-compose.yml
|
||
├── Makefile
|
||
└── pyproject.toml
|
||
```
|
||
|
||
## Request lifecycle example
|
||
|
||
A `POST /api/v1/leads` with a new lead flows as:
|
||
|
||
1. **Middleware** assigns a request ID, binds it to structlog context.
|
||
2. **Router** (`api/routers/leads.py`) validates input via Pydantic.
|
||
3. **DI** resolves a singleton `AcquisitionPipeline`.
|
||
4. **Pipeline** runs:
|
||
- `IntakeAgent` normalizes + dedup
|
||
- `PainExtractorAgent` runs keyword pass, tries LLM enrichment (Arabic→GLM)
|
||
- `ICPMatcherAgent` scores Fit across 5 dimensions
|
||
- `QualificationAgent` generates BANT questions (Claude)
|
||
- `CRMAgent` upserts to HubSpot (best-effort)
|
||
- If fit≥0.5: `BookingAgent` picks Calendly or Google Cal
|
||
- If fit≥0.7 + opt-in: `ProposalAgent` drafts proposal via Claude
|
||
5. **Pipeline** returns a `PipelineResult` with any `warnings` from failed steps.
|
||
6. **Router** serializes to `PipelineResponse`.
|
||
7. **Middleware** logs completion with duration.
|
||
|
||
Total LLM calls: 2–4 depending on flags. With caching and Groq for classification, p50 < 5s.
|