PROMPT-0001: Template
| Metadata | Value |
|---|
| Version | 1.0 |
| Status | Draft | Testing | Production | Deprecated |
| Created | YYYY-MM-DD |
| Author(s) | @username |
| Model(s) | GPT-4o, Claude 3.5 Sonnet |
| Task Type | System | Task | Few-Shot | Chain |
Purpose
What task does this prompt accomplish? When should it be used?
Prompt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
| [SYSTEM]
You are a [role] that [primary responsibility].
## Context
- You are working with [context about the codebase/domain]
- The user is a [target user persona]
## Instructions
1. [Step 1]
2. [Step 2]
3. [Step 3]
## Constraints
- [Constraint 1]
- [Constraint 2]
## Output Format
Respond with a JSON object:
```json
{
"summary": "string",
"items": ["string"],
"confidence": "high | medium | low"
}
```
[USER]
{{user_input}}
[ASSISTANT]
|
Variables
| Variable | Type | Description | Example |
|---|
{{user_input}} | string | The user’s request | “Review this PR” |
{{context}} | string | Additional context | File contents |
{{language}} | string | Programming language | “typescript” |
Examples
Example 1: Happy Path
Input:
{{user_input}}: "Summarize the changes in this PR"
{{context}}: [diff content]
Expected Output:
1
2
3
4
5
6
7
8
9
| {
"summary": "This PR adds user authentication using JWT tokens",
"items": [
"New auth middleware in src/middleware/auth.ts",
"User model updated with password hash field",
"Login and register endpoints added"
],
"confidence": "high"
}
|
Example 2: Edge Case
Input:
{{user_input}}: "Summarize the changes"
{{context}}: [empty diff]
Expected Output:
1
2
3
4
5
| {
"summary": "No changes detected in this PR",
"items": [],
"confidence": "high"
}
|
Evaluation
Test Cases
| ID | Input | Expected Output | Pass Criteria |
|---|
| TC-1 | Simple PR | Accurate summary | Summary matches human review |
| TC-2 | Large PR | Grouped changes | Max 10 items, logically grouped |
| TC-3 | Empty PR | Empty state | Handles gracefully |
| TC-4 | Binary files | Skip binary | Notes binary files exist |
Quality Metrics
| Metric | Target | Current |
|---|
| Accuracy | > 90% | - |
| Format compliance | 100% | - |
| Latency (p95) | < 5s | - |
Usage
Code Example
1
2
3
4
5
6
7
8
9
10
| import { renderPrompt } from '@/lib/prompts';
import { PROMPT_PR_SUMMARY } from '@/prompts/pr-summary';
const prompt = renderPrompt(PROMPT_PR_SUMMARY, {
user_input: "Summarize this PR",
context: diffContent,
language: "typescript"
});
const response = await llm.complete(prompt);
|
Integration Points
- PR Bot: Triggered on PR open/update
- Slack Bot: On-demand summaries
- CLI:
train summarize-pr <pr-number>
Changelog
| Version | Date | Author | Changes |
|---|
| 1.0 | YYYY-MM-DD | @author | Initial version |
| 1.1 | YYYY-MM-DD | @author | Added confidence field |
Notes
Known Limitations
- May struggle with PRs > 5000 lines
- Binary file changes not summarized
- Requires repo context for accurate naming
Model-Specific Adjustments
| Model | Adjustments |
|---|
| GPT-4o | Works as-is |
| Claude 3.5 | Add “Be concise” to constraints |
| GPT-3.5 | Simplify output format |
References