Foundation models to suggest PR review prompts

Foundation models, such as large language models (LLMs), can significantly streamline and enhance the process of code review by generating contextual pull request (PR) review prompts. These models are capable of understanding both code and natural language, making them ideal for prompting reviewers to think more deeply and consistently about specific aspects of a PR.

Below are various ways foundation models can be leveraged to generate PR review prompts, categorized by the type of insights they can provide:

1. General Review Prompts

These prompts help ensure reviewers cover essential areas of the code review process.

• “Does the code follow the team’s coding standards and style guidelines?”

• “Are there any obvious bugs, logic errors, or unhandled edge cases?”

• “Can any part of this code be simplified or made more efficient?”

• “Are variable and function names clear and self-explanatory?”

• “Is the code modular and reusable where appropriate?”

2. Security-Focused Prompts

Foundation models trained on secure coding practices can suggest prompts like:

• “Are all inputs properly validated and sanitized?”

• “Could any of the changes introduce vulnerabilities like XSS, SQL injection, or buffer overflows?”

• “Does the code properly handle authentication, authorization, and sensitive data?”

3. Performance and Scalability Prompts

These help identify potential performance bottlenecks or scalability issues.

• “Are there any loops, queries, or computations that could become performance bottlenecks?”

• “Can the new code handle large data sets or high request volumes efficiently?”

• “Are there any opportunities to cache expensive operations?”

4. Testing and Coverage Prompts

Models can detect lack of test coverage and generate relevant queries.

• “Are there sufficient unit and integration tests for the new or changed logic?”

• “Are edge cases and failure paths tested?”

• “Do any tests need to be updated or refactored to reflect the changes?”

5. Documentation and Readability Prompts

Helpful for ensuring long-term maintainability.

• “Is the code sufficiently documented for future developers?”

• “Would a code comment or docstring help clarify this block?”

• “Do function and class headers explain their purpose and usage?”

6. Design and Architecture Prompts

Ideal for evaluating larger refactors or architectural changes.

• “Does this change introduce any tight coupling between components?”

• “Is the new design aligned with existing patterns and architecture?”

• “Could this functionality be extracted into a separate module or service?”

7. Dependency and Configuration Prompts

When changes involve dependencies, configs, or builds.

• “Are new dependencies necessary, secure, and minimal?”

• “Does the change affect CI/CD or deployment configuration?”

• “Do changes introduce breaking changes in downstream systems?”

How Foundation Models Generate Contextual Prompts

Foundation models can be fine-tuned or prompted to provide PR-specific insights based on:

• Code Diffs: Analyze what changed and suggest prompts tailored to that.

• Commit Messages: Understand the intent and surface relevant questions.

• File Types: Suggest prompts specific to languages, frameworks, or file roles (e.g., frontend vs backend).

• Project History: Use patterns in previous PRs to generate useful and personalized prompts.

Example Use Case

Given a PR introducing a new API endpoint, the model might auto-generate:

• “Does this endpoint follow the naming and versioning conventions?”

• “Is the API input/output schema consistent with existing endpoints?”

• “Has rate limiting, authentication, and error handling been considered?”

Implementation Ideas

To integrate foundation models for PR review prompts:

1. GitHub/GitLab Integration: Use webhooks and APIs to analyze PRs and comment with generated prompts.

2. IDE Plugins: Real-time feedback while writing or reviewing code.

3. Custom CLI Tools: Developers can run a command like review-prompts pr-123 to get suggestions.

4. CI Pipelines: Automatically post a summary of prompts as a checklist or comment on the PR.

Benefits

• Standardized Review Process: Ensures consistency across reviewers and teams.

• Time-Saving: Reduces cognitive load by surfacing key questions.

• Improved Quality: Encourages thorough, multi-dimensional reviews.

• Better Onboarding: Junior developers can learn what to look for.

Challenges

• Context Awareness: Foundation models must understand nuanced project-specific logic.

• Noise vs. Signal: Avoid generating too many generic or irrelevant prompts.

• Performance & Cost: Analyzing large diffs in real time can be resource-intensive.

• Customization: Prompts need to reflect team norms and priorities.

Future Enhancements

• Feedback Loops: Improve model suggestions based on which prompts developers interact with.

• Semantic Diff Analysis: Go beyond line changes to understand meaning and intent.

• Natural Language Summaries: Accompany prompts with explanations or suggestions for improvement.

Foundation models are uniquely positioned to assist with PR reviews by automating and enhancing the thought process behind manual review. With thoughtful integration, they can become a valuable co-pilot for every reviewer—helping teams ship better, safer code faster.