feat: add email delivery test subagent for spam filter testing and inbox placement#929
feat: add email delivery test subagent for spam filter testing and inbox placement#929marcusquinn merged 1 commit intomainfrom
Conversation
…box placement Create services/email/email-delivery-test.md covering: - Spam filter scoring methodology and content trigger words - SpamAssassin rule testing and text-to-image ratio guidance - Seed list testing workflow across Gmail, Outlook, Yahoo, iCloud - Provider-specific filtering behaviour (Gmail ML, Outlook SmartScreen, Yahoo) - IP warming schedules and sender reputation management - Feedback loop (FBL) setup for complaint processing - Pre-send checklist and full testing sequence using existing helper scripts - Troubleshooting for spam, promotions tab, and intermittent failures Complements existing email-health-check.md (DNS auth) and email-testing.md (design rendering + infrastructure) — this fills the gap for content-level deliverability analysis. Also updates subagent-index.toon to include email-delivery-test in the services/email/ entry. t214.3
Summary of ChangesHello @marcusquinn, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request introduces a new subagent dedicated to email delivery testing. It provides extensive guidance on how to ensure emails successfully reach recipients' inboxes by covering spam filter mechanisms, content analysis, inbox placement strategies, and sender reputation management. This addition enhances the existing email-related subagents by focusing specifically on content-level deliverability. Highlights
Changelog
Activity
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
WalkthroughA new email delivery testing guide documentation is introduced at Changes
Estimated code review effort🎯 2 (Simple) | ⏱️ ~12 minutes Possibly related issues
Poem
🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
🔍 Code Quality Report�[0;35m[MONITOR]�[0m Code Review Monitoring Report �[0;34m[INFO]�[0m Latest Quality Status: �[0;34m[INFO]�[0m Recent monitoring activity: 📈 Current Quality Metrics
Generated on: Tue Feb 10 14:53:05 UTC 2026 Generated by AI DevOps Framework Code Review Monitoring |
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a comprehensive new subagent guide for email delivery testing, covering spam filters, inbox placement, and reputation management. The guide is well-structured and detailed. I've added a few suggestions to improve clarity, consistency, and correctness in the provided examples and links. Specifically, I've pointed out some outdated commands, non-direct URLs, and minor inconsistencies in terminology. I also suggested making a shell command example more robust to align with the repository's scripting style guide.
|
|
||
| ## Quick Reference | ||
|
|
||
| - **Purpose**: Test emails against spam filters, verify inbox placement, and analyse content deliverability |
There was a problem hiding this comment.
There's an inconsistency in the spelling of 'analyse' (British English) versus 'analyze' (American English) throughout the document. The associated script is named email-test-suite-helper.sh analyze-headers, which uses 'analyze'. For consistency with the script name and to avoid confusion, it would be best to use 'analyze' and 'analysis' throughout this guide. This also applies to line 457 (Analyse spam copy headers).
| - **Purpose**: Test emails against spam filters, verify inbox placement, and analyse content deliverability | |
| - **Purpose**: Test emails against spam filters, verify inbox placement, and analyze content deliverability |
| ```bash | ||
| # Install SpamAssassin | ||
| brew install spamassassin # macOS | ||
| sudo apt-get install spamassassin # Linux |
There was a problem hiding this comment.
The apt-get command is somewhat dated. The modern and preferred command is apt, which offers a more user-friendly interface and combines functionality from apt-get and apt-cache. It's best practice to use the more current command in new documentation.
| sudo apt-get install spamassassin # Linux | |
| sudo apt install spamassassin # Linux |
| spamassassin -t < test-email.eml | ||
|
|
||
| # Get detailed score breakdown | ||
| spamassassin -t -D < test-email.eml 2>&1 | grep -E "^(score|hits|required)" |
There was a problem hiding this comment.
The repository's shell script style guide recommends using || true guards for commands like grep that can exit with a non-zero status if no match is found. Although this is documentation, the commands should reflect project best practices, as they might be copied into scripts using set -e and set -o pipefail. Adding || true ensures the command is more robust in such scenarios.
| spamassassin -t -D < test-email.eml 2>&1 | grep -E "^(score|hits|required)" | |
| spamassassin -t -D < test-email.eml 2>&1 | grep -E "^(score|hits|required)" || true |
References
- Use
|| trueguards for commands that may fail underset -e(grep, arithmetic) (link)
|
|
||
| | Factor | Impact | Notes | | ||
| |--------|--------|-------| | ||
| | **Authentication** | Very High | DMARC p=reject enforced for yahoo.com senders | |
There was a problem hiding this comment.
This note, 'DMARC p=reject enforced for yahoo.com senders', is ambiguous and potentially misleading. It appears to describe Yahoo's own policy on the yahoo.com domain, which prevents spoofing of @yahoo.com addresses. For a sender trying to deliver mail to Yahoo, this isn't an actionable requirement. The key requirement, correctly stated on line 327, is for the sender to have a valid DMARC policy on their own domain. To prevent confusion, I recommend removing this line or rephrasing it to clarify it's about Yahoo's outbound domain protection, not an inbound delivery requirement.
| | **Yahoo** | help.yahoo.com/kb/postmaster | | ||
| | **AOL** | help.yahoo.com/kb/postmaster (merged with Yahoo) | | ||
| | **Comcast** | postmaster.comcast.net | |
There was a problem hiding this comment.
The links for Yahoo/AOL and Comcast FBL registration point to general postmaster pages, not directly to the sign-up forms. Providing more direct links improves the guide's usability.
- For Yahoo/AOL, the current portal is
https://senders.yahooinc.com/. - For Comcast, the feedback loop page is
https://postmaster.comcast.net/feedback-loop.html.
I suggest updating these links to be more specific.
| | **Yahoo** | help.yahoo.com/kb/postmaster | | |
| | **AOL** | help.yahoo.com/kb/postmaster (merged with Yahoo) | | |
| | **Comcast** | postmaster.comcast.net | | |
| | **Yahoo** | senders.yahooinc.com/ | | |
| | **AOL** | senders.yahooinc.com/ (merged with Yahoo) | | |
| | **Comcast** | postmaster.comcast.net/feedback-loop.html | |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Fix all issues with AI agents
In @.agents/services/email/email-delivery-test.md:
- Around line 14-52: The new email-delivery-test.md duplicates inbox placement
content from email-testing.md (specifically the "Inbox placement" layer/Layer 2
and provider-specific filtering); decide and implement one of two fixes: either
narrow email-delivery-test.md to only content-level spam trigger analysis and
seed list testing (remove or collapse the "Inbox placement" layer and related
commands/quick-commands, update the Overview and Quick Reference to point
readers to email-testing.md for inbox placement and provider tools), or
explicitly partition responsibilities by adding a clear sentence in the Overview
and Quick Reference that defines "content-triggered inbox placement" as the
remit of email-delivery-test.md and "infrastructure/provider scoring" as
email-testing.md's remit, then edit the Layer 2 section to describe only the
content-triggered aspects and reference email-testing.md for provider-specific
tools (e.g., Microsoft SNDS) and scoring.
🧹 Nitpick comments (2)
.agents/services/email/email-delivery-test.md (2)
133-150: SpamAssassin debug grep pattern may not match actual output format.Line 142: The
-Dflag produces extremely verbose debug output. The grep pattern"^(score|hits|required)"targets line-start anchors, but SpamAssassin's debug output doesn't typically start lines with those bare keywords. The summary scores appear in a different format (e.g.,Content analysis details:block). A more reliable approach:♻️ Suggested improvement for score extraction
-# Get detailed score breakdown -spamassassin -t -D < test-email.eml 2>&1 | grep -E "^(score|hits|required)" +# Get score summary +spamassassin -t < test-email.eml | grep -E "^(X-Spam-Status|X-Spam-Flag|score=)" + +# Get full rule breakdown (verbose) +spamassassin -t -D < test-email.eml 2>&1 | tail -50
244-329: Provider-specific filtering section is thorough — one minor accuracy note on Yahoo thresholds.Line 329 specifies Yahoo's complaint rate threshold at 0.3%, while Lines 258/340/413 reference 0.1% for Google. These are indeed different thresholds, but it's worth noting that Yahoo's published guidance actually recommends staying well below 0.3%. Consider adding a brief note that 0.1% is the safer universal target even for Yahoo.
📝 Optional clarification
- Spam complaint rate must stay under 0.3% +- Spam complaint rate must stay under 0.3% (0.1% recommended as universal safe target)
| # Email Delivery Testing | ||
|
|
||
| <!-- AI-CONTEXT-START --> | ||
|
|
||
| ## Quick Reference | ||
|
|
||
| - **Purpose**: Test emails against spam filters, verify inbox placement, and analyse content deliverability | ||
| - **Scripts**: `email-test-suite-helper.sh check-placement [domain]`, `email-health-check-helper.sh check [domain]` | ||
| - **Focus**: Content-level spam triggers, provider-specific filtering, seed list testing, reputation signals | ||
| - **Complements**: `email-health-check.md` (DNS auth), `email-testing.md` (design rendering + infrastructure) | ||
|
|
||
| **Quick commands:** | ||
|
|
||
| ```bash | ||
| # Inbox placement score | ||
| email-test-suite-helper.sh check-placement example.com | ||
|
|
||
| # DNS authentication health | ||
| email-health-check-helper.sh check example.com | ||
|
|
||
| # SMTP delivery test | ||
| email-test-suite-helper.sh test-smtp-domain example.com | ||
|
|
||
| # Header analysis (check spam verdicts) | ||
| email-test-suite-helper.sh analyze-headers headers.txt | ||
| ``` | ||
|
|
||
| <!-- AI-CONTEXT-END --> | ||
|
|
||
| ## Overview | ||
|
|
||
| Email delivery testing validates that messages reach the inbox rather than spam or promotions folders. This guide covers three layers: | ||
|
|
||
| 1. **Content analysis** - Spam trigger detection in subject lines, body, and HTML | ||
| 2. **Inbox placement** - Provider-specific filtering behaviour and seed list testing | ||
| 3. **Reputation signals** - Sender score, domain age, engagement history | ||
|
|
||
| For DNS authentication (SPF, DKIM, DMARC) see `email-health-check.md`. | ||
| For design rendering and SMTP infrastructure see `email-testing.md`. |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Check for overlapping content between the new file and existing email subagents
echo "=== Checking for content overlap with existing email subagents ==="
echo "--- Spam filter / spam trigger references in other email agents ---"
rg -n -i "spam filter|spam trigger|content trigger|trigger word" .agents/services/email/ --glob '!email-delivery-test.md'
echo "--- Inbox placement references in other email agents ---"
rg -n -i "inbox placement|seed list|seed account" .agents/services/email/ --glob '!email-delivery-test.md'
echo "--- SpamAssassin references in other email agents ---"
rg -n -i "spamassassin" .agents/services/email/ --glob '!email-delivery-test.md'
echo "--- IP warming references in other email agents ---"
rg -n -i "ip warming|warming schedule" .agents/services/email/ --glob '!email-delivery-test.md'
echo "--- Provider-specific filtering references in other email agents ---"
rg -n -i "smartscreen|postmaster tools|snds|promotions tab" .agents/services/email/ --glob '!email-delivery-test.md'
echo "--- Reputation management references in other email agents ---"
rg -n -i "sender reputation|feedback loop|complaint rate" .agents/services/email/ --glob '!email-delivery-test.md'Repository: marcusquinn/aidevops
Length of output: 1930
🏁 Script executed:
# List existing email agents to understand scope
echo "=== Listing all email subagents ==="
ls -la .agents/services/email/Repository: marcusquinn/aidevops
Length of output: 488
🏁 Script executed:
# Check the structure and content of email-health-check.md and email-testing.md
echo "=== Checking email-health-check.md content ==="
head -100 .agents/services/email/email-health-check.md | grep -i "spam\|inbox\|spamassassin\|ip warming\|reputation\|smartscreen\|seed"
echo "=== Checking email-testing.md content ==="
head -100 .agents/services/email/email-testing.md | grep -i "spam\|inbox\|spamassassin\|ip warming\|reputation\|smartscreen\|seed"Repository: marcusquinn/aidevops
Length of output: 627
Overlapping inbox placement scope detected—clarify boundaries or consolidate with email-testing.md.
Verification found significant overlap between email-delivery-test.md and email-testing.md:
- Inbox placement testing: Both files cover inbox placement analysis.
email-testing.md(lines 68, 152–171) includes comprehensive scoring for inbox placement factors and provider-specific tools (Microsoft SNDS). The new file duplicates this as layer 2 of its three-layer structure. - Provider-specific filtering:
email-testing.mdalready covers provider-specific filtering behaviour via its Inbox Placement Scoring section.
The file redirects readers to email-testing.md for "design rendering and SMTP infrastructure," but still duplicates inbox placement scope. Decide whether email-delivery-test.md should focus exclusively on content-level spam trigger analysis and seed list testing, leaving inbox placement to email-testing.md, or explicitly partition the inbox placement scope between both files with clear distinctions (e.g., content-triggered vs. infrastructure-triggered filtering).
🤖 Prompt for AI Agents
In @.agents/services/email/email-delivery-test.md around lines 14 - 52, The new
email-delivery-test.md duplicates inbox placement content from email-testing.md
(specifically the "Inbox placement" layer/Layer 2 and provider-specific
filtering); decide and implement one of two fixes: either narrow
email-delivery-test.md to only content-level spam trigger analysis and seed list
testing (remove or collapse the "Inbox placement" layer and related
commands/quick-commands, update the Overview and Quick Reference to point
readers to email-testing.md for inbox placement and provider tools), or
explicitly partition responsibilities by adding a clear sentence in the Overview
and Quick Reference that defines "content-triggered inbox placement" as the
remit of email-delivery-test.md and "infrastructure/provider scoring" as
email-testing.md's remit, then edit the Layer 2 section to describe only the
content-triggered aspects and reference email-testing.md for provider-specific
tools (e.g., Microsoft SNDS) and scoring.
marcusquinn
dismissed
coderabbitai[bot]’s stale review
Bot review dismissed — suggestions noted
1 participant
Summary
What is Covered
How It Fits
Complements existing email subagents:
Task
t214.3
Summary by CodeRabbit
Documentation