From 605f55d99fcce85a5f65adfc6141b71fdb989913 Mon Sep 17 00:00:00 2001 From: "google-labs-jules[bot]" <161369871+google-labs-jules[bot]@users.noreply.github.com> Date: Tue, 3 Jun 2025 07:52:27 +0000 Subject: [PATCH] Generate AI coding tool insights from research This commit adds three new files based on the analysis of research.learn.md: 1. best_practices_article.md: An article summarizing best practices for developing AI-assisted coding tools, covering prompting, UI/UX, tooling, system design, and more. 2. tool_list.md: A Markdown list of tools and functions used by various AI coding platforms (VS Code, v0, Manus, Lovable, etc.), derived from their system prompt documentation. 3. system_prompt_insights.md: A document highlighting key structural elements, organizational strategies, and design insights for creating effective system prompts for AI agents. --- best_practices_article.md | 167 ++++++++++++++++++++++++++++++++++++++ system_prompt_insights.md | 88 ++++++++++++++++++++ tool_list.md | 92 +++++++++++++++++++++ 3 files changed, 347 insertions(+) create mode 100644 best_practices_article.md create mode 100644 system_prompt_insights.md create mode 100644 tool_list.md diff --git a/best_practices_article.md b/best_practices_article.md new file mode 100644 index 00000000..3c44b072 --- /dev/null +++ b/best_practices_article.md @@ -0,0 +1,167 @@ +# Best Practices for AI-Assisted Coding (Vibe Coding) + +## Introduction + +AI-assisted coding tools, sometimes referred to as "vibe coding systems," are rapidly changing the landscape of software development. These systems leverage large language models (LLMs) and sophisticated tooling to understand user intent, generate code, and automate various development tasks. To maximize the effectiveness and reliability of these tools, it's crucial to adhere to a set of best practices in their design, development, and usage. This article synthesizes key best practices identified from research and observations of various vibe coding systems. + +## Prompting Techniques + +Effective communication with the AI is fundamental. The quality of the output often directly correlates with the clarity and structure of the input prompts. + +* **Use Clear and Concise Language:** Employ basic prompting phrases and keywords to guide the AI. + * **Emphasis:** Use words like `IMPORTANT:`, `NOTICE:`, `NEVER`, `Always`, `DO`, `DON'T` to highlight critical instructions. Bolding important words (`**important words**`) can also help. + * **Sparing Use of Emphasis:** Keywords like `NOTE:`, `MUST`, or `!IMPORTANT` should be used sparingly to emphasize truly critical information, as overuse can diminish their impact. +* **Structured Input:** For some systems, like those based on Claude, using XML-like structures in prompts can improve understanding and parsing of instructions. This is seen in tools like VS Code Copilot, same.dev, and Windsurf. + ```xml + + You are an AI programming assistant. + When asked for your name, you must respond with "GitHub Copilot". + ... + + + ... + + ``` +* **Keywords for Organization (Replit):** Systems like Replit utilize keywords to organize prompts into logical sections, which can help in structuring complex requests: + * `` + * `` + * `` + * `` + * `` + * `` + +## UI/UX Considerations + +The user interface and experience are critical for making AI coding tools accessible and productive. + +* **Dynamic Rendering:** Using MDX (Markdown + JSX) for responses allows for dynamic and interactive content rendering, similar to streaming content in a browser. This is a notable feature in tools like `v0`. +* **Accessibility:** Always implement accessibility best practices to ensure tools are usable by everyone (`v0`). +* **Markdown Template Engines:** Employ Markdown template engines for rendering and compositing complex views. This is a good practice observed in code generation tools. +* **Suggestion Systems:** Proactively suggest relevant follow-up actions to the user. This can help guide the user, spark new ideas, and highlight potential next steps (`v0`). + ``` + + + + + ``` +* **UI Frameworks:** Utilizing established UI frameworks and libraries like `shadcn/ui` in CLI-mode or preset-mode is considered a best practice for UI design (`same.dev`, `v0`). + +## Tooling and Capabilities + +The underlying tools and defined capabilities of the AI system dictate its effectiveness. + +* **Clearly Defined Capabilities:** A `Capabilities` section is essential to describe the boundaries of the AI's abilities. This helps manage user expectations (`v0`). + ```md + Users interact with v0 online. Here are some capabilities of the v0 UI: + + - Users can attach (or drag and drop) images and text files in the prompt form. + - Users can execute JavaScript code in the Node.js Executable code block + ... + ``` +* **Domain-Specific Knowledge:** + * Incorporate a `domain knowledge` section for specialized information. + * All domain knowledge used by the system (e.g., in `v0`) **MUST** be cited, often using a format like `[^index]` linking to a sources section. This is especially important for private knowledge bases. + * Model Context Protocol (MCP) is a good practice for managing and vending domain knowledge. +* **Handling Refusals:** Implement a clear `Refusal` section to define how the system responds to inappropriate or harmful requests. The response should be direct and avoid apologies or explanations (`v0`). + ```md + # Refusals + + REFUSAL_MESSAGE = "I'm sorry. I'm not able to assist with that." + + 1. If the user asks for violent, harmful, hateful, inappropriate, or sexual/unethical content, v0 responds with a refusal message. + 2. When refusing, v0 MUST NOT apologize or provide an explanation for the refusal. v0 simply states the REFUSAL_MESSAGE. + ``` +* **Data Serialization:** Using libraries like `zod` to serialize and deserialize JSON schema to data is a good practice, ensuring data integrity and structure (`same.dev`). Alternatively, function parameters can be used directly. +* **Tool Categorization:** Clearly define and categorize the tools and actions an agent can perform. This is seen in systems like `lovable` and `manus`, using XML-like tags for operations: + ```xml + for creating or updating files. Must include complete file contents. + for renaming files from original path to new path. + ``` + +## System Design + +The architectural design of the AI system plays a vital role in its robustness and ability to handle complex tasks. + +* **Thinking Tags for Planning:** Encourage the system to use "thinking tags" (e.g., `...`) to plan and outline steps before generating code or taking action. This is similar to writing a Product Requirements Document (PRD) and is a practice used by `v0`. +* **Event Stream Architecture (Manus):** Organizing events and actions into an event stream is a good practice. This allows for tracking task progress and system state effectively. The stream can include: + * `Message`: User inputs. + * `Action`: Tool use. + * `Observation`: Results from actions. + * `Plan`: Task planning updates. + * `Knowledge`: Task-related best practices. +* **Modular Design (Manus):** A modular architecture, as seen in `manus`, enhances maintainability and clarity. Key modules include: + * **Planner Module:** For overall task planning, represented by numbered pseudocode. + * **Knowledge Module:** Provides task-relevant knowledge and best practices. + * **Datasource Module:** Manages access to authoritative data APIs. +* **Agent Loop (Manus):** Operating in an agent loop allows for iterative task completion: + 1. Analyze Events + 2. Select Tools + 3. Wait for Execution + 4. Iterate + 5. Submit Results + 6. Enter Standby + +## Development Practices + +Adhering to sound software development principles is just as important for AI-assisted coding as it is for traditional development. + +* **File and Code Editing (VS Code Copilot):** + * **Read Before Edit:** Don't try to edit an existing file without reading it first. + * **Use Dedicated Tools:** Employ specific tools (e.g., `insert_edit_into_file`) for file modifications rather than printing code blocks representing changes. + * **Concise Edits:** When using editing tools, avoid repeating existing code. Use comments like `// ...existing code...` to represent unchanged regions. + * **Validate Changes:** After editing a file, call validation functions (e.g., `get_errors`) and fix relevant errors. +* **Environment Description:** Providing information about the project's current state (e.g., version numbers, directory structure, linter errors, terminal logs) is beneficial (`same.dev`). +* **General Coding Best Practices (`lovable`):** + * **Code Quality and Organization:** Create small, focused components (e.g., < 50 lines), use TypeScript, follow project structure, implement responsive designs, and use extensive console logs. + * **Component Creation:** Create new files for components, use UI libraries like `shadcn/ui`, follow atomic design, and ensure proper file organization. + * **State Management:** Use appropriate libraries for server state (e.g., React Query) and local state (e.g., `useState`/`useContext`), avoid prop drilling, and cache responses. + * **Error Handling:** Use toast notifications, implement error boundaries, log errors, and provide user-friendly error messages. + * **Performance:** Implement code splitting, optimize images, use React hooks correctly, and minimize re-renders. + * **Security:** Validate inputs, implement proper authentication, sanitize data, and follow security guidelines (e.g., OWASP). + * **Testing:** Write unit and integration tests, test responsive layouts, and verify error handling. + * **Documentation:** Document complex functions, keep READMEs updated, include setup instructions, and document APIs. + +## Communication and Workflow (Manus) + +Defining clear rules for how the AI interacts with the user and manages its workflow is crucial for a smooth experience. + +* **Task Management (`todo_rules`):** + * Use a `todo.md` file as a checklist based on task planning. + * Update `todo.md` immediately after completing each item. +* **Messaging (`message_rules`):** + * Communicate via message tools, replying promptly to new user messages. + * Distinguish between `notify` (non-blocking) and `ask` (blocking) messages. + * Provide results and deliverables as attachments. +* **File Operations (`file_rules`):** + * Use dedicated file tools for I/O to avoid shell command issues. + * Save intermediate results and organize reference information. +* **Information Gathering (`info_rules`):** + * Prioritize information: authoritative datasource API > web search > model's internal knowledge. + * Access original pages from search results, not just snippets. +* **Browser Interaction (`browser_rules`):** + * Use browser tools to access URLs from users or search results. + * Actively explore links for deeper information. + * Handle cases where content extraction might be incomplete by scrolling. +* **Shell Usage (`shell_rules`):** + * Avoid commands requiring confirmation (use `-y` or `-f` flags). + * Chain commands with `&&`. + * Use `bc` for simple calculations, Python for complex math. +* **Coding Execution (`coding_rules`):** + * Save code to files before execution. + * Use search tools for unfamiliar problems. +* **Deployment (`deploy_rules`):** + * Use expose port tools for temporary external access. + * Ensure services listen on `0.0.0.0`. + * Test services locally before exposing them. +* **Content Generation (`writing_rules`):** + * Write in continuous paragraphs; avoid lists unless requested. + * Cite sources when writing based on references. + * For lengthy documents, save sections separately then combine. +* **Error Handling (`error_handling`):** + * Verify tool names and arguments upon error. + * Attempt to fix issues based on error messages or try alternative methods. + * Report failures to the user if unresolved. + +## Conclusion + +The development and application of AI-assisted coding tools represent a significant advancement in software engineering. By adhering to best practices in prompting, UI/UX design, tooling, system architecture, development processes, and communication workflows, we can build more effective, reliable, and user-friendly vibe coding systems. These practices, drawn from observations of systems like v0, Manus, and VS Code Copilot, provide a strong foundation for harnessing the full potential of AI in the coding domain. Continuous learning and refinement of these practices will be essential as the field evolves. diff --git a/system_prompt_insights.md b/system_prompt_insights.md new file mode 100644 index 00000000..c941cbbe --- /dev/null +++ b/system_prompt_insights.md @@ -0,0 +1,88 @@ +# Insights into System Prompt Architecture for AI Coding Assistants + +## Introduction + +System prompts are the foundational instructions and context provided to an AI model to guide its behavior, define its capabilities, and ensure it operates within desired parameters. A well-structured system prompt is crucial for the effectiveness, reliability, and safety of AI-assisted coding tools, often referred to as "vibe coding systems." This document analyzes examples from `research.learn.md` to distill common structural components, organizational strategies, and insights for designing robust system prompt architectures. + +## Common Structural Components + +Several recurring structural elements are observed in system prompts across different AI platforms: + +* **Identity Definition:** Clearly defining the AI's persona and role. + * Example: VS Code Copilot uses `` tags: + ```xml + + You are an AI programming assistant. + When asked for your name, you must respond with "GitHub Copilot". + Follow the user's requirements carefully & to the letter. + ... + + ``` +* **Core Instructions:** General operational guidelines and rules. + * Example: VS Code Copilot uses `` tags. +* **Tool Usage Instructions:** Specifics on how the AI should use available tools. + * Example: VS Code Copilot uses `` and `` for defining how to invoke tools and structure parameters: + ```xml + + + + [param_value] + + + + ``` + * It also uses `` for detailed guidance on file manipulation. +* **Function Definitions:** Describing available functions/tools the AI can call, often in JSON format. + * Example: VS Code Copilot includes a `` block with JSON objects detailing function names, descriptions, and parameters. +* **Context Provision:** Supplying relevant information about the current state, environment, or task. + * Example: VS Code Copilot's `` block includes date, OS, workspace folders, and file structure. +* **Reminders:** Highlighting key instructions or constraints that the AI should always keep in mind. + * Example: VS Code Copilot uses `` tags for specific tool usage advice. +* **Capabilities Section:** Explicitly listing what the system can and cannot do. This manages user expectations and defines operational boundaries. + * Example: `v0` uses a Markdown section titled "Capabilities" to list UI interaction capabilities, code execution, preview functionalities, etc. +* **Refusal Messages and Policies:** Defining how the AI should respond to inappropriate, harmful, or out-of-scope requests. + * Example: `v0` has a "Refusals" section with a predefined `REFUSAL_MESSAGE` and clear rules on when and how to use it. +* **"Thinking" Blocks:** A mechanism for the AI to plan or reason before responding or acting. + * Example: `v0` uses `` tags for the AI to outline project structure, styling, etc., before generating code. This is also noted as a general good practice. + +## Organizational Strategies and Their Benefits + +Different systems employ various strategies to organize their prompts, each with its advantages: + +* **XML for Structured Data (Claude-based systems):** + * Systems like VS Code Copilot, same.dev, and Windsurf, particularly those based on Claude, appear to benefit from XML-structured prompts. + * **Benefit:** XML provides a clear, hierarchical way to delineate different sections of the prompt (e.g., ``, ``, ``), making it easier for the model to parse and understand distinct blocks of information. +* **Markdown for Readability and Sectioning:** + * Systems like `v0` use Markdown headings and bullet points to define sections like "Capabilities" and "Refusals." + * **Benefit:** Markdown is human-readable and allows for easy structuring of information into logical sections, which can be effective for conveying policies and capabilities. +* **Keyword-Based Emphasis and Organization:** + * **Basic Keywords:** Using `IMPORTANT:`, `NOTICE:`, `NEVER`, `Always`, `DO`, `DON'T`, and bolding (`**important words**`) to draw attention to critical instructions. + * **Benefit:** Simple yet effective way to highlight crucial parts of the prompt. + * **Tagged Sections (Replit):** Replit uses tags like ``, ``, `` to categorize different aspects of its operational logic. + * **Benefit:** This provides a thematic organization, allowing the AI to consult specific blocks of instructions based on the current task or phase. +* **Comprehensive Modular Sections (Manus):** + * The Manus system prompt is highly detailed and uses XML-like tags to create distinct modules for almost every aspect of its operation (e.g., ``, ``, ``, ``, ``, ``, ``, ``, and numerous `_rules` sections like ``, ``, ``, ``, ``). + * **Benefit:** This exhaustive, modular approach provides extreme clarity and fine-grained control over the AI's behavior. Each module acts as a specific policy or protocol for different situations, reducing ambiguity and promoting consistent responses. +* **JSON for Tool/Function Definitions:** + * Embedding JSON objects within prompts (e.g., in VS Code Copilot's `` section, or the `web_search` example) to define tools. + * **Benefit:** JSON is a machine-readable format that precisely defines the tool's name, description, and parameters, facilitating reliable tool invocation. + +## Designing Effective System Prompt Architectures: Insights + +The examples in `research.learn.md` offer valuable clues for designing effective system prompt architectures: + +1. **Start with Identity:** Clearly define *what* the AI is. This sets the stage for its behavior and interactions. +2. **Explicit Instructions are Key:** Don't assume the AI knows what to do. Provide clear, unambiguous instructions for tasks, tool use, and policies. +3. **Structure for Clarity:** Whether using XML, Markdown, or custom tags, a well-defined structure helps the AI differentiate between various types of information (identity, rules, context, tools). +4. **Dedicated Tool Sections:** Clearly define available tools, their parameters, and how they should be invoked. The `` and `` examples from VS Code Copilot are good models. +5. **Context is Crucial:** Provide relevant environmental and task-specific context to enable more accurate and relevant responses. +6. **Implement Safeguards:** Include explicit refusal policies and messages for handling inappropriate requests. The `v0` "Refusals" section is a good example. +7. **Define Capabilities:** Clearly outline what the AI *can* and *cannot* do to manage expectations (both for the AI and the user). +8. **Encourage Planning:** Incorporate mechanisms like `` tags to prompt the AI to plan before acting, potentially leading to more coherent and well-structured outputs. +9. **Modularity for Complexity (Manus Model):** For complex AI systems, a highly modular approach with specific rules for different operations (file handling, messaging, coding, deployment, error handling) can provide robustness and predictability. +10. **Use Reminders:** For critical instructions that must always be followed, use `` tags or similar mechanisms to reinforce them. +11. **Iterative Refinement:** System prompt design is likely an iterative process. The "Classic VibeCoding Project SystemPrompt" link suggests that these prompts evolve. + +## Conclusion + +Designing a robust system prompt is fundamental to building effective and reliable AI coding assistants. Key considerations include establishing a clear identity, providing explicit instructions, structuring the prompt logically (using XML, Markdown, or tags), clearly defining tools and capabilities, managing context, and implementing safeguards like refusal policies. The detailed, modular approach seen in the Manus example, combined with the structured clarity of VS Code Copilot's XML tags and `v0`'s explicit capability and refusal sections, offers a rich set of patterns to draw from. By carefully architecting system prompts, developers can significantly influence the AI's performance, safety, and overall utility. diff --git a/tool_list.md b/tool_list.md new file mode 100644 index 00000000..16fb1c45 --- /dev/null +++ b/tool_list.md @@ -0,0 +1,92 @@ +# AI System Tools, Functions, and Commands List + +This document lists tools, functions, and commands used by various AI-assisted coding systems as identified in `research.learn.md`. + +## VS Code / GitHub Copilot + +* **`create_new_workspace`** (`VS Code`): Get steps to help the user create any project in a VS Code workspace. Use this tool to help users set up new projects, including TypeScript-based projects, Model Context Protocol (MCP) servers, VS Code extensions, Next.js projects, Vite projects, or any other project. +* **`insert_edit_into_file`** (`VS Code`): Tool to edit files. Avoids repeating existing code by using comments like `// ...existing code...`. +* **`get_errors`** (`VS Code`): Called after editing a file to validate the change and fix errors if relevant. +* **``** (`VS Code`): Generic tool invocation format. + * `[param_value]` + +## v0 + +* **Node.js Executable code block** (`v0`): Allows users to execute JavaScript code. +* **Inline SQL code block** (`v0`): Allows users to execute SQL queries directly in chat to query and modify databases. +* **URL processing** (`v0`): Users can provide URL(s) to websites, and the system will automatically send a screenshot. +* **"add to codebase" button** (`v0`): Installs Code Projects (code written by the AI) and handles setup of required dependencies in an existing project or helps create a new project. +* **"Deploy" button** (`v0`): Deploys Code Projects to Vercel. +* **`` tags** (`v0`): Used by the AI to think through project structure, styling, images, media, formatting, frameworks, libraries, and caveats before creating a Code Project. +* **`` component** (`v0`): Used to suggest relevant follow-up actions to the user. + * **``** (`v0`): Defines a suggested action and its description. + +## same.dev + +* **`web_search`** (`same.dev` - example JSON, platform not explicitly stated but context implies general AI tooling): Search the web for real-time text and image responses. + * `fetch_content` (parameter): Whether to crawl and include the content of each search result. + * `search_term` (parameter): The search term to look up on the web. + * `type` (parameter): The type of search to perform (text or images). + +## Manus + +The Manus system describes general capabilities and rules which imply the use of several types of tools: + +* **Message tools** (`Manus`): For communication with users. Divided into: + * `notify`: Non-blocking, no reply needed from users. + * `ask`: Blocking, reply required from users. +* **Shell tools** (`Manus`): Access to a Linux sandbox environment. Includes ability to: + * Install required software packages and dependencies. + * Use flags like `-y` or `-f` for automatic confirmation. + * Chain commands with `&&`. + * Use pipe operator `|`. + * **`bc` command** (`Manus`): For simple non-interactive calculations. + * **`uptime` command** (`Manus`): For sandbox status check. +* **Text editor tools** (`Manus`): For file manipulation. +* **Browser tools** (`Manus`): To access and comprehend URLs, explore links, and extract page content. +* **Python execution** (`Manus`): To write and run Python code, especially for complex mathematical calculations, analysis, and calling Data APIs. +* **Deployment tools** (`Manus`): To deploy websites or applications. + * **Expose port tool** (`Manus`): For temporary external access to services. +* **File tools** (`Manus`): For reading, writing, appending, and editing files. + * **Text replacement tool** (`Manus`): Implied by `todo_rules` for updating markers in `todo.md`. + * **Append mode of file writing tool** (`Manus`): For merging text files. +* **Search tools** (`Manus`): Preferred over browser access to search engine result pages for information gathering. +* **Data API interaction** (`Manus`): Called through Python code to access authoritative datasources. (Note: Data APIs themselves are not "tools" in this context but are accessed via Python code). + +## Lovable + +* **File Operations:** + * **``** (`Lovable`): For creating or updating files. Must include complete file contents. + * **``** (`Lovable`): For renaming files from original path to new path. + * **``** (`Lovable`): For removing files from the project. + * **``** (`Lovable`): For installing new packages or updating existing ones. +* **Code Block Structure:** + * **``** (`Lovable`): To wrap all code changes and technical details. + * **``** (`Lovable`): To show your thought process (optional). + * **``** (`Lovable`): To display error messages when they occur. + * **``** (`Lovable`): To confirm successful operations. +* **Response Format Tags (Informational):** + * `` (`Lovable`) + * `` (`Lovable`) + * `` (`Lovable`) + * `` (`Lovable`) + * `` (`Lovable`) + * `` (`Lovable`) + * `` (`Lovable`) + * `` (`Lovable`) + * `` (`Lovable`) + * `` (`Lovable`) +* **Shell Commands (Example Setup Guide):** + * **`git clone `** (`Lovable`): Clones a repository. + * **`cd `** (`Lovable`): Navigates to a project directory. + * **`npm i`** (`Lovable`): Installs necessary dependencies. + * **`npm run dev`** (`Lovable`): Starts the development server. + +## General / Unspecified Platform + +* **` tags`** (General, mentioned for `v0` but a general concept): To outline project requirements, user stories, and acceptance criteria before implementation. +* **Text replacement tool** (General, implied by Manus `todo_rules`): For updating markers in todo.md or similar checklist files. +* **File tools for reading, writing, appending, editing** (General, implied by Manus `file_rules`): Basic file operations. +* **Search tools** (General, implied by Manus `info_rules` and `coding_rules`): For finding solutions or information. +* **Deployment tools** (General, implied by Manus `coding_rules` and `deploy_rules`): For packaging and deploying applications. +* **Expose port tool** (General, implied by Manus `deploy_rules`): For making services temporarily accessible.