skills/autoresearch/references/security-workflow.md

1137 lines
41 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# Security Workflow — $autoresearch security
Autonomous security auditing that uses the autoresearch loop to iteratively discover, validate, and report vulnerabilities. Combines STRIDE threat modeling, OWASP Top 10 sweeps, and red-team adversarial analysis into a single autonomous loop.
**Output:** A severity-ranked security report with threat model, findings, mitigations, and iteration log.
## Trigger
- User invokes `$autoresearch security`
- User says "security audit", "run a security sweep", "threat model this codebase", "find vulnerabilities"
- User says "red-team this app", "OWASP audit", "STRIDE analysis"
## Loop Support
Works with both unbounded and bounded modes:
```
# Unlimited — keep finding vulnerabilities until interrupted
$autoresearch security
# Bounded — run exactly N security sweep iterations
$autoresearch security
Iterations: 10
# With target scope
$autoresearch security
Scope: src/api/**/*.ts, src/middleware/**/*.ts
Focus: authentication and authorization flows
```
## PREREQUISITE: Interactive Setup (when invoked without flags)
**CRITICAL — BLOCKING PREREQUISITE:** If `$autoresearch security` is invoked without `--diff`, scope, or focus, you MUST scan the codebase first, then use direct prompting to gather user input BEFORE proceeding to ANY phase. DO NOT skip this step.
**Single batched call — all 3 questions at once:**
You MUST call direct prompting with all 3 questions in ONE call:
| # | Header | Question | Options (from codebase scan) |
|---|--------|----------|------------------------------|
| 1 | `Scope` | "What should I audit?" | "Entire codebase (comprehensive)", "API routes + middleware only", "Authentication + authorization", "External-facing code only" |
| 2 | `Depth` | "How thorough?" | "Quick scan (5 iterations)", "Standard audit (15 iterations)", "Deep audit (30+ iterations)", "Unlimited" |
| 3 | `Action` | "What should I do with confirmed vulnerabilities?" | "Report only (read-only)", "Report + auto-fix Critical/High", "Report + CI gate (fail on critical)" |
**IMPORTANT:** Always ask all questions in a single call — never one at a time.
If flags are provided inline, skip interactive setup and proceed directly.
## Architecture
```
┌─────────────────────────────────────────────────────────────┐
│ SETUP PHASE (once) │
│ │
│ 1. Scan codebase → identify tech stack, frameworks, APIs │
│ 2. Map assets → data stores, auth, external services │
│ 3. Identify trust boundaries → client/server, API/DB │
│ 4. Generate STRIDE threat model │
│ 5. Build attack surface map │
│ 6. Create security-audit-results.tsv log │
│ 7. Establish baseline (count known issues) │
│ │
├─────────────────────────────────────────────────────────────┤
│ AUTONOMOUS LOOP │
│ │
│ Each iteration: pick ONE attack vector from the threat │
│ model, attempt to find/validate the vulnerability, │
│ log the result, move to next vector. │
│ │
│ LOOP (FOREVER or N times): │
│ 1. Review: threat model + past findings + results log │
│ 2. Select: pick next untested attack vector │
│ 3. Analyze: deep-dive into target code for the vector │
│ 4. Validate: construct proof (code path, input, output) │
│ 5. Classify: severity + OWASP category + STRIDE category │
│ 6. Log: append to results log │
│ 7. Repeat │
│ │
└─────────────────────────────────────────────────────────────┘
```
## Setup Phase — Threat Model Generation
### Step 1: Codebase Reconnaissance
Scan the project to build context:
```
READ:
- package.json / requirements.txt / go.mod (dependencies)
- .env.example / config files (secrets handling)
- Dockerfile / docker-compose.yml (infrastructure)
- API route files (attack surface)
- Auth/middleware files (trust boundaries)
- Database schemas/models (data assets)
- CI/CD configs (supply chain)
```
### Step 2: Asset Identification
Catalog every asset that has security relevance:
| Asset Type | Examples | Priority |
|------------|----------|----------|
| **Data stores** | Database, Redis, file storage, cookies, localStorage | Critical |
| **Authentication** | Login, OAuth, JWT, sessions, API keys | Critical |
| **API endpoints** | REST routes, GraphQL resolvers, webhooks | High |
| **External services** | Payment APIs, email providers, CDN, analytics | High |
| **User input surfaces** | Forms, URL params, headers, file uploads | High |
| **Configuration** | Environment variables, feature flags, CORS settings | Medium |
| **Static assets** | Public files, uploaded content, generated files | Low |
### Step 3: Trust Boundary Mapping
Identify where trust levels change:
```
Trust Boundaries:
├── Browser ←→ Server (client-side vs server-side)
├── Server ←→ Database (application vs data layer)
├── Server ←→ External APIs (internal vs third-party)
├── Public routes ←→ Authenticated routes
├── User role ←→ Admin role (privilege levels)
├── CI/CD ←→ Production (deployment boundary)
└── Container ←→ Host (infrastructure boundary)
```
### Step 4: STRIDE Threat Model
For each asset + trust boundary combination, analyze threats using STRIDE:
| Threat | Question | Example Findings |
|--------|----------|------------------|
| **S**poofing | Can an attacker impersonate a user/service? | Weak auth, missing CSRF, forged JWTs |
| **T**ampering | Can data be modified in transit or at rest? | Missing input validation, SQL injection, prototype pollution |
| **R**epudiation | Can actions be denied without evidence? | Missing audit logs, unsigned transactions |
| **I**nformation Disclosure | Can sensitive data leak? | Error messages expose internals, PII in logs, debug endpoints |
| **D**enial of Service | Can the service be disrupted? | Missing rate limiting, regex DoS, resource exhaustion |
| **E**levation of Privilege | Can a user gain unauthorized access? | IDOR, broken access control, path traversal |
Output the threat model as a structured table in the security report.
### Step 5: Attack Surface Map
Generate an attack surface map showing:
```
Attack Surface:
├── Entry Points
│ ├── GET /api/users/:id → IDOR risk (user enumeration)
│ ├── POST /api/auth/login → Brute force, credential stuffing
│ ├── POST /api/upload → File upload, path traversal
│ ├── WebSocket /ws → Auth bypass, injection
│ └── Webhook /api/webhooks/* → Signature verification
├── Data Flows
│ ├── User input → DB query → Injection risk
│ ├── JWT → route handler → Token validation
│ └── File upload → storage → Malicious file execution
└── Abuse Paths
├── Rate limit bypass → account takeover
├── IDOR chain → data exfiltration
└── SSRF → internal service access
```
### Step 6: Baseline
Count existing security issues before the loop starts:
- Run any existing security linting (`npm audit`, `eslint-plugin-security`, `bandit`, etc.)
- Count issues as baseline metric
- Record in results log as iteration #0
## The Security Loop
### Iteration Protocol
Each iteration follows the autoresearch pattern but adapted for security:
#### Phase 1: Review (Select Attack Vector)
Priority order for selecting the next vector to test:
1. **Critical STRIDE threats** not yet tested
2. **OWASP Top 10 categories** not yet covered
3. **High-severity attack paths** from the surface map
4. **Dependency vulnerabilities** (supply chain)
5. **Configuration weaknesses** (headers, CORS, CSP)
6. **Business logic flaws** (race conditions, state manipulation)
7. **Information disclosure** (error handling, debug modes)
Track coverage in the results log. The goal is comprehensive coverage.
#### Phase 2: Analyze (Deep Dive)
For the selected vector:
1. Read all relevant code files
2. Trace data flow from entry point to data store
3. Identify missing validation, sanitization, or access checks
4. Look for known vulnerability patterns
#### Phase 3: Validate (Proof Construction)
For each potential finding, construct proof:
```
Finding Proof Structure:
├── Vulnerable code location (file:line)
├── Attack scenario (step-by-step)
├── Input that triggers the vulnerability
├── Expected vs actual behavior
├── Impact assessment
└── Confidence level (Confirmed / Likely / Possible)
```
**Validation Rules:**
- **Confirmed** — Code path clearly allows the attack, no guards present
- **Likely** — Guards exist but are bypassable or incomplete
- **Possible** — Theoretical risk, depends on configuration or runtime conditions
Do NOT report findings without supporting code evidence.
**Credential hygiene in finding output (mandatory):**
Findings, PoCs, attack scenarios, and reproduction commands SHOULD NOT contain real secrets even when the secret IS the vulnerability. Always mask before writing to any file:
| Pattern | Mask form |
|---|---|
| API keys, JWTs, OAuth tokens | `<REDACTED_TOKEN>` (preserve length class: short/medium/long) |
| Connection strings with embedded passwords | `protocol://user:<REDACTED_PASSWORD>@host/db` |
| Environment variable values | reference the var name only: `$DATABASE_URL`, never the value |
| Private keys, certs | first 8 chars + `<...REDACTED...>` + last 8 chars |
| Sample request bodies | replace value, keep field name: `{"api_key": "<REDACTED>"}` |
When a finding's reproduction needs real credentials, write the PoC as a *template* the user fills in at runtime — never as a copy-paste-ready command containing the live secret. Reject any draft finding that contains a value matching: a JWT (`eyJ...`), 32+ char hex, AWS key prefixes (`AKIA`, `ASIA`), or known token formats. Re-mask and re-emit.
#### Phase 4: Classify
Assign severity and categories:
**Severity (CVSS-inspired):**
| Severity | Criteria |
|----------|----------|
| **Critical** | RCE, auth bypass, SQL injection, data breach, admin takeover |
| **High** | XSS (stored), SSRF, privilege escalation, mass data exposure |
| **Medium** | CSRF, open redirect, info disclosure, missing rate limits |
| **Low** | Missing headers, verbose errors, weak session config |
| **Info** | Best practice suggestions, hardening recommendations |
**OWASP Top 10 (2021) mapping:**
| ID | Category |
|----|----------|
| A01 | Broken Access Control |
| A02 | Cryptographic Failures |
| A03 | Injection |
| A04 | Insecure Design |
| A05 | Security Misconfiguration |
| A06 | Vulnerable Components |
| A07 | Auth & Identification Failures |
| A08 | Software & Data Integrity Failures |
| A09 | Security Logging & Monitoring Failures |
| A10 | Server-Side Request Forgery |
**STRIDE mapping:** Tag each finding with the applicable STRIDE category.
#### Phase 5: Log
Append to security-audit-results.tsv:
```tsv
iteration vector severity owasp stride confidence location description
0 - - - - - - baseline — 3 npm audit warnings
1 IDOR High A01 EoP Confirmed src/api/users.ts:42 GET /api/users/:id returns any user data without ownership check
2 XSS Medium A03 Tampering Likely src/components/comment.tsx:18 User input rendered via dangerouslySetInnerHTML
3 rate-limit Medium A05 DoS Confirmed src/api/auth.ts:15 POST /login has no rate limiting — brute force possible
```
#### Phase 6: Repeat
- **Unbounded:** Keep finding vulnerabilities. Never stop. Never ask.
- **Bounded (Iterations: N):** After N iterations, generate final report and stop.
- **Coverage tracking:** Every 5 iterations, print coverage summary.
### Coverage Summary Format
```
=== Security Audit Progress (iteration 10) ===
STRIDE Coverage: S[✓] T[✓] R[✗] I[✓] D[✓] E[✓] — 5/6
OWASP Coverage: A01[✓] A02[✗] A03[✓] A04[✗] A05[✓] A06[✓] A07[✓] A08[✗] A09[✗] A10[✗] — 5/10
Findings: 4 Critical, 2 High, 3 Medium, 1 Low
Confirmed: 7 | Likely: 2 | Possible: 1
```
## Final Report Structure
Generated at loop completion (bounded) or on interrupt (unbounded):
```markdown
# Security Audit Report
## Executive Summary
- **Date:** {date}
- **Scope:** {files/directories scanned}
- **Iterations:** {N}
- **Total Findings:** {count} ({critical} Critical, {high} High, {medium} Medium, {low} Low)
## Threat Model
### Assets
{table of identified assets}
### Trust Boundaries
{diagram of trust boundaries}
### STRIDE Analysis
{threat model table}
### Attack Surface Map
{entry points, data flows, abuse paths}
## Findings (Descending Severity)
### [CRITICAL] Finding 1: {title}
- **OWASP:** {category}
- **STRIDE:** {category}
- **Location:** `{file}:{line}`
- **Confidence:** Confirmed / Likely / Possible
- **Description:** {what's wrong}
- **Attack Scenario:** {step-by-step exploitation}
- **Code Evidence:**
```{lang}
{vulnerable code snippet}
```
- **Mitigation:**
```{lang}
{fixed code snippet}
```
- **References:** {CWE, CVE if applicable}
### [HIGH] Finding 2: ...
...
## Coverage Matrix
| OWASP Category | Tested | Findings |
|----------------|--------|----------|
| A01 Broken Access Control | ✓ | 2 |
| A02 Cryptographic Failures | ✓ | 0 |
| ... | ... | ... |
| STRIDE Category | Tested | Findings |
|-----------------|--------|----------|
| Spoofing | ✓ | 1 |
| Tampering | ✓ | 2 |
| ... | ... | ... |
## Dependency Audit
{npm audit / pip audit / go vulnerabilities}
## Security Headers Check
{CSP, HSTS, X-Frame-Options, etc.}
## Recommendations (Priority Order)
1. {Critical fix 1}
2. {Critical fix 2}
...
## Iteration Log
{full TSV content}
```
## OWASP Checks Reference
Detailed checks to perform for each OWASP category:
### A01 — Broken Access Control
- [ ] IDOR on all parameterized routes (`:id`, `:slug`)
- [ ] Missing authorization middleware on protected routes
- [ ] Horizontal privilege escalation (user A accessing user B's data)
- [ ] Vertical privilege escalation (user accessing admin functions)
- [ ] Directory traversal on file operations
- [ ] CORS misconfiguration allowing unauthorized origins
- [ ] Missing function-level access control
### A02 — Cryptographic Failures
- [ ] Sensitive data in plaintext (passwords, tokens, PII)
- [ ] Weak hashing algorithms (MD5, SHA1 for passwords)
- [ ] Hardcoded secrets/API keys in source
- [ ] Missing encryption at rest / in transit
- [ ] Weak random number generation for security tokens
- [ ] Exposed .env files or config with secrets
### A03 — Injection
- [ ] SQL/NoSQL injection in database queries
- [ ] Command injection in shell executions (exec, spawn)
- [ ] XSS (stored, reflected, DOM-based)
- [ ] Template injection (SSTI)
- [ ] LDAP injection
- [ ] Path injection in file operations
- [ ] Header injection (CRLF)
### A04 — Insecure Design
- [ ] Missing rate limiting on sensitive endpoints
- [ ] No account lockout after failed login attempts
- [ ] Predictable resource identifiers
- [ ] Race conditions in critical operations
- [ ] Missing CSRF protection on state-changing operations
- [ ] Insecure direct object references in design
### A05 — Security Misconfiguration
- [ ] Debug mode enabled in production
- [ ] Default credentials / admin pages exposed
- [ ] Verbose error messages exposing internals
- [ ] Missing security headers (CSP, HSTS, X-Content-Type-Options)
- [ ] Unnecessary HTTP methods enabled
- [ ] Directory listing enabled
- [ ] Stack traces in error responses
### A06 — Vulnerable and Outdated Components
- [ ] Known CVEs in dependencies (npm audit, pip audit)
- [ ] Outdated frameworks with security patches available
- [ ] Unmaintained dependencies
- [ ] Dependencies with known prototype pollution
### A07 — Identification and Authentication Failures
- [ ] Weak password policies
- [ ] Missing multi-factor authentication for admin
- [ ] Session fixation vulnerabilities
- [ ] JWT vulnerabilities (none algorithm, weak secret, no expiry)
- [ ] Insecure password reset flows
- [ ] Missing session invalidation on logout/password change
### A08 — Software and Data Integrity Failures
- [ ] Missing integrity checks on CI/CD pipelines
- [ ] Unsigned or unverified updates/dependencies
- [ ] Insecure deserialization
- [ ] Missing CSP or SRI for external scripts
- [ ] Unsigned webhooks / API callbacks
### A09 — Security Logging and Monitoring Failures
- [ ] Missing audit logs for security events
- [ ] No logging of failed authentication attempts
- [ ] Sensitive data in logs (passwords, tokens)
- [ ] Missing alerting on suspicious activity
- [ ] Log injection vulnerabilities
### A10 — Server-Side Request Forgery (SSRF)
- [ ] Unvalidated URLs in server-side requests
- [ ] DNS rebinding vulnerabilities
- [ ] Missing allowlist for external service calls
- [ ] Proxy/redirect endpoints without validation
## Red-Team Adversarial Lenses
Adapted from the plan red-team workflow for security context:
### Security Adversary (Primary)
**Mindset:** "I'm a hacker trying to breach this system"
- Focus: auth bypass, injection, data exposure, privilege escalation
- Method: trace every input to its sink, find missing guards
- Priority: exploitable findings over theoretical risks
### Supply Chain Attacker
**Mindset:** "I'm compromising dependencies or build pipeline"
- Focus: dependency vulnerabilities, CI/CD weaknesses, unsigned artifacts
- Method: audit dependency tree, check for typosquatting, verify integrity
- Priority: dependencies with known CVEs, build pipeline access
### Insider Threat
**Mindset:** "I'm a malicious employee or compromised account"
- Focus: privilege escalation, data exfiltration, access control gaps
- Method: check what a low-privilege user can access, find horizontal movement
- Priority: admin bypass, bulk data export, missing audit trails
### Infrastructure Attacker
**Mindset:** "I'm attacking the deployment, not the code"
- Focus: container escape, exposed services, network segmentation
- Method: check Docker configs, K8s manifests, exposed ports, env vars
- Priority: secrets in environment, overly permissive configs
## Strix-Inspired Patterns
Learned from Strix (AI-powered security testing platform):
### Proof-of-Concept Validation
Never report a finding without proof. For each vulnerability:
1. Identify the exact code path
2. Construct a concrete exploit input
3. Trace execution through the vulnerability
4. Show the impact (data leaked, access gained, etc.)
### Multi-Agent Attack Collaboration
Each iteration should build on prior findings:
- Iteration 1 finds open endpoint → Iteration 2 chains with IDOR
- Iteration 3 finds missing rate limit → Iteration 4 tests brute force feasibility
- Findings compound. Each iteration reads past findings for chaining opportunities.
### Dynamic Analysis Verification
Where possible, suggest or construct verification commands:
```bash
# Test for missing rate limiting
for i in {1..100}; do curl -s -o /dev/null -w "%{http_code}" https://app/api/login; done
# Test for IDOR
curl -H "Authorization: Bearer USER_A_TOKEN" https://app/api/users/USER_B_ID
# Test for XSS
curl https://app/search?q=%3Cscript%3Ealert(1)%3C/script%3E
```
### Comprehensive Vulnerability Categories (from Strix)
- **Access Control** — IDOR, privilege escalation, auth bypass
- **Injection Attacks** — SQL, NoSQL, command injection
- **Server-Side** — SSRF, XXE, deserialization flaws
- **Client-Side** — XSS, prototype pollution, DOM vulnerabilities
- **Business Logic** — Race conditions, workflow manipulation
- **Authentication** — JWT vulnerabilities, session management
- **Infrastructure** — Misconfigurations, exposed services
## Metric for the Loop
The security audit uses a **coverage + finding count** composite metric:
```
metric = (owasp_categories_tested / 10) * 50 + (stride_categories_tested / 6) * 30 + min(finding_count, 20)
```
- **Direction:** higher is better (more coverage + more findings = more thorough)
- **Maximum theoretical:** 50 + 30 + 20 = 100
- **Baseline:** 0 (nothing tested yet)
This incentivizes the loop to cover ALL categories before going deep on any one.
## Flags & Modes
### `--diff` — Delta Mode (v1.0.3)
Only audit files changed since the last audit. Reads the most recent `security/` subfolder to establish what was already tested.
```
$autoresearch security --diff
```
**How it works:**
1. Find the latest `security/*/overview.md` by timestamp in folder name
2. Parse `findings.md` from that folder to get previously tested files
3. Run `git diff --name-only {last_audit_commit}..HEAD` to find changed files
4. Scope the current audit to ONLY those changed files
5. In the final report, mark findings as:
- **New** — found in changed files, not in previous audit
- **Fixed** — was in previous audit, no longer present in changed code
- **Recurring** — still present from previous audit (unchanged)
**Delta report additions:**
The overview.md gains a `## Delta Summary` section:
```markdown
## Delta Summary (vs {previous_audit_folder})
| Status | Count | Details |
|--------|-------|---------|
| New findings | 3 | Found in changed files |
| Fixed | 2 | No longer present |
| Recurring | 5 | Still present from last audit |
| Files changed | 12 | Since last audit |
| Files audited | 8 | (security-relevant subset) |
```
If no previous audit folder exists, `--diff` falls back to full audit with a warning.
### `--fail-on` — Severity Threshold Gate (v1.0.3)
Exit with non-zero code if findings meet or exceed a severity threshold. Designed for CI/CD blocking.
```
$autoresearch security --fail-on critical
$autoresearch security --fail-on high
$autoresearch security --fail-on medium
```
| Flag Value | Blocks on |
|------------|-----------|
| `critical` | Any Critical finding |
| `high` | Any Critical or High finding |
| `medium` | Any Critical, High, or Medium finding |
**Behavior:**
- Runs the full audit normally
- After generating the report, checks findings against threshold
- If threshold met: prints `SECURITY GATE FAILED: {N} findings at {severity} or above` and exits non-zero
- If threshold not met: prints `SECURITY GATE PASSED` and exits 0
**CI/CD usage:**
```bash
# In GitHub Actions or CI scripts
claude -p "$autoresearch security --fail-on critical --iterations 10"
# Exit code 1 if any Critical findings → blocks the pipeline
```
### `--fix` — Auto-Remediation Mode
After completing the audit, switches to standard autoresearch modify→verify loop to fix confirmed findings. Uses the security audit report as its goal.
```
$autoresearch security --fix
$autoresearch security --fix
Iterations: 10
```
**How it works:**
1. Run the full security audit (setup + loop + report)
2. Filter findings: only **Confirmed** severity **Critical** and **High**
3. Switch to `$autoresearch fix` with findings context:
- **Target:** Re-run the security checks that found each vulnerability
- **Scope:** Files referenced in findings (file:line locations)
- Pass the filtered findings list as context so fix knows WHAT to fix
- Fix picks highest-severity unfixed finding each iteration
4. For each fix iteration:
- Pick the highest-severity unfixed finding
- Apply the mitigation from `recommendations.md`
- Commit the fix
- Re-verify: does the vulnerability still exist?
- If fixed → keep commit, mark finding as "Fixed" in report
- If still vulnerable → revert, try different approach
- If new findings introduced → revert immediately
**Fix report additions:**
After fixes complete, updates the audit folder:
- `findings.md` gains a `Status` column: `Open` / `Fixed` / `Fix attempted`
- `recommendations.md` gains checkmarks on applied fixes
- New file: `fix-log.md` with iteration details
**Safety rules:**
- NEVER fix Low or Info findings automatically (too subjective)
- NEVER modify test files (fixes must not break existing tests)
- Run existing tests after each fix — revert if any test fails
- Maximum 3 fix attempts per finding, then skip
- User can combine with `--fail-on` for gated fix: fix first, then gate
### `--chain <targets>` — Downstream Chaining
Chain to downstream tool(s) after the audit completes. Comma-separated for multi-chain. Spaces after commas tolerated.
```
$autoresearch security --chain fix
$autoresearch security --chain fix,scenario,debug
```
See **Chain Conversion** section below for how security findings map to each downstream tool.
Note: `--fix` is a shortcut for `--chain fix` (auto-remediation with full fix loop).
### Combining Flags
Flags can be combined:
```
# Delta audit + auto-fix critical/high + block on remaining criticals
$autoresearch security --diff --fix --fail-on critical
Iterations: 15
# Quick delta check in CI
$autoresearch security --diff --fail-on high
Iterations: 5
```
**Execution order when combined:**
1. `--diff` narrows scope
2. Security audit runs (with narrowed scope if `--diff`)
3. `--fix` runs remediation loop on confirmed Critical/High
4. `--fail-on` checks remaining (unfixed) findings against threshold
### CI/CD GitHub Action Template
When `$autoresearch security` detects a `.github/workflows/` directory, it offers to generate a security workflow:
```
direct prompting:
question: "I see you use GitHub Actions. Want me to generate a security audit workflow?"
header: "CI/CD"
options:
- label: "Yes, generate it (Recommended)"
description: "Creates .github/workflows/security-audit.yml"
- label: "No, skip"
description: "Continue without CI/CD setup"
```
**Generated workflow:** `.github/workflows/security-audit.yml`
```yaml
name: Security Audit
on:
pull_request:
branches: [main, master]
schedule:
- cron: '0 2 * * 1' # Weekly Monday 2am UTC
permissions:
contents: read
pull-requests: write
jobs:
security-audit:
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0 # Full history for delta mode
- name: Install Claude Code
run: npm install -g @anthropic-ai/claude-code
- name: Install Autoresearch Skill
run: |
git clone https://github.com/uditgoenka/autoresearch.git /tmp/autoresearch
cp -r /tmp/autoresearch/skills/autoresearch .agents/skills/autoresearch
cp -r /tmp/autoresearch/commands/autoresearch .claude/commands/autoresearch
cp /tmp/autoresearch/commands/autoresearch.md .claude/commands/autoresearch.md
- name: Run Security Audit
env:
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
run: |
# Delta mode on PRs, full audit on schedule
if [ "${{ github.event_name }}" = "pull_request" ]; then
claude -p "$autoresearch security --diff --fail-on critical --iterations 5"
else
claude -p "$autoresearch security --fail-on high --iterations 15"
fi
- name: Upload Security Report
if: always()
uses: actions/upload-artifact@v4
with:
name: security-audit-report
path: security/
retention-days: 90
- name: Comment PR with Summary
if: github.event_name == 'pull_request' && always()
uses: actions/github-script@v7
with:
script: |
const fs = require('fs');
const glob = require('glob');
const overviews = glob.sync('security/*/overview.md');
if (overviews.length > 0) {
const latest = overviews.sort().pop();
const content = fs.readFileSync(latest, 'utf-8');
const summary = content.split('## Summary')[1]?.split('##')[0] || 'See full report in artifacts.';
await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.issue.number,
body: `## 🔒 Security Audit Results\n\n${summary}\n\n> Full report available in workflow artifacts.`
});
}
```
**The template is generated ONCE** — after initial creation, it's the user's file to customize.
### Historical Comparison
When a previous audit exists in `security/`, the current run automatically generates a comparison section.
**Detection:** At setup, scan `security/` for existing audit folders sorted by date.
**Comparison logic:**
```
For each finding in current audit:
Search previous audit findings.md for same location (file:line) or same description
If found → mark as "Recurring"
If not found → mark as "New"
For each finding in previous audit:
Search current audit findings for same location or description
If not found → mark as "Fixed"
```
**Output in overview.md:**
```markdown
## Historical Comparison
**Previous audit:** security/260310-1430-stride-owasp-full-audit/ (5 days ago)
### Trend
| Metric | Previous | Current | Change |
|--------|----------|---------|--------|
| Critical | 3 | 1 | ↓ -2 (improved) |
| High | 4 | 5 | ↑ +1 (regressed) |
| Medium | 2 | 3 | ↑ +1 |
| Total | 9 | 9 | → 0 |
| OWASP coverage | 6/10 | 8/10 | ↑ +2 |
| STRIDE coverage | 4/6 | 5/6 | ↑ +1 |
### Finding Status
| Status | Count | Details |
|--------|-------|---------|
| Fixed since last audit | 4 | JWT algo, CORS, 2 XSS |
| New findings | 4 | SSRF, rate limit, 2 IDOR |
| Recurring (unfixed) | 5 | See findings.md |
### Regression Alert
⚠️ 4 new findings detected since last audit. Review [findings.md](./findings.md) for details.
```
**findings.md additions:**
Each finding gets a `History` tag:
- `🆕 New` — first time detected
- `🔄 Recurring` — present in previous audit too
- `✅ Fixed` (only in previous audit's context) — no longer present
## Error Recovery
| Error | Recovery |
|-------|----------|
| Can't determine tech stack | Ask user for framework/language |
| No API routes found | Scan for all exported functions with HTTP-like patterns |
| Dependency audit fails | Skip, note in report, continue with code analysis |
| Code too large for context | Focus on files matching attack surface (API, auth, DB) |
| False positive suspected | Mark as "Possible" confidence, include caveats |
### Chain Conversion
#### `--chain fix`
Each confirmed vulnerability becomes a fix target sorted by STRIDE severity. Only Confirmed Critical and High findings are passed unless `--chain fix` is used explicitly (vs `--fix` which also limits to Confirmed).
```
$autoresearch fix
Scope: {files from findings.md — file:line locations}
Target: {top Critical vulnerability title}
From-Security: true
```
#### `--chain debug`
Investigate findings deeper with empirical testing — validate that code paths are actually reachable and exploitable under real conditions.
```
$autoresearch debug
Scope: {files from confirmed findings}
Symptom: security audit found {N} vulnerabilities — empirical validation needed
Hypotheses:
H-01 [CRITICAL] {vulnerability title} — {attack vector}
H-02 [HIGH] {vulnerability title} — {attack vector}
```
#### `--chain scenario`
Each confirmed threat becomes a scenario seed for attack simulation and blast radius exploration.
```
$autoresearch scenario
Scenario: {vulnerability title} — {attack description}
Domain: security
Depth: standard
```
#### `--chain predict`
Security findings become the goal for multi-persona swarm impact prediction — "what else might be compromised given these vulnerabilities."
```
$autoresearch predict
Scope: {files from findings.md}
Goal: predict cascading impact of confirmed vulnerabilities: {comma-separated titles}
```
#### `--chain plan`
Remediation planning for confirmed vulnerabilities — organize fixes into a structured implementation plan.
```
$autoresearch plan
Goal: remediate confirmed security vulnerabilities
Source: security/{slug}/recommendations.md
```
#### `--chain learn`
Security patterns and STRIDE/OWASP findings documented for codebase security awareness.
```
$autoresearch learn
Topic: security vulnerabilities, STRIDE patterns, and OWASP findings
Source: security/{slug}/findings.md
```
#### `--chain reason`
Complex mitigations with tradeoffs go through adversarial design refinement before implementation.
```
$autoresearch reason
Task: determine best mitigation strategy for complex security findings
Evidence: security/{slug}/recommendations.md
```
#### `--chain ship`
Vulnerabilities become ship gate blockers — CRITICAL/HIGH block shipping, MEDIUM warn.
```
$autoresearch ship
Gate: {FAIL if any Critical/High confirmed findings, WARN if Medium findings exist}
Blockers: {count of Critical/High confirmed findings}
```
#### `--chain probe`
Security gaps reveal missing or ambiguous requirements — interrogate what the system was supposed to prevent.
```
$autoresearch probe
Topic: security requirement gaps revealed by: {comma-separated vulnerability titles}
Source: security/{slug}/findings.md
```
### Multi-Chain Execution
`--chain fix,scenario,ship` executes sequentially:
1. Write `handoff.json` after security audit completes
2. Launch `fix` with chain conversion above
3. After `fix` completes, convert fix results + `handoff.json``scenario` context
4. After `scenario` completes, convert scenario findings → `ship` gate
5. Each stage's output feeds the next via updated `handoff.json`
**Empirical evidence rule:** Downstream loop results ALWAYS override upstream security audit findings. If debug or fix disproves a security finding, the empirical result wins — mark finding as `DISPROVEN by {tool} loop` in the security report.
## Anti-Patterns
- **Do NOT report theoretical risks without code evidence** — every finding needs a file:line reference
- **Do NOT skip categories** — the loop should aim for 100% OWASP + STRIDE coverage
- **Do NOT auto-fix vulnerabilities** — report only, user decides what to fix
- **Do NOT test against live production** — analyze code statically, suggest dynamic tests
- **Do NOT report the same finding twice** — check results log for duplicates before logging
- **Do NOT prioritize quantity over quality** — 5 confirmed critical > 50 theoretical lows
## Report Output — Structured Folder
Every `$autoresearch security` run creates a dedicated folder inside a `security/` directory at the project root (similar to how `/plan --hard` creates plan directories).
### Folder Structure
```
{project_root}/
└── security/
├── 260315-0945-stride-owasp-full-audit/
│ ├── overview.md ← Executive summary + links to all reports
│ ├── threat-model.md ← STRIDE threat model (assets, boundaries, threats)
│ ├── attack-surface-map.md ← Entry points, data flows, abuse paths
│ ├── findings.md ← All findings ranked by severity (Critical → Low)
│ ├── owasp-coverage.md ← OWASP Top 10 coverage matrix + per-category results
│ ├── dependency-audit.md ← npm audit / pip audit / go vuln results
│ ├── recommendations.md ← Prioritized mitigations with code snippets
│ └── security-audit-results.tsv ← Iteration log (every vector tested)
├── 260320-1430-auth-api-focused-audit/
│ ├── overview.md
│ ├── threat-model.md
│ ├── ...
│ └── security-audit-results.tsv
└── ... ← One subfolder per audit run
```
### Folder Naming Convention
```
security/{YYMMDD}-{HHMM}-{audit-type-slug}/
```
| Component | Source | Example |
|-----------|--------|---------|
| `YYMMDD` | Current date | `260315` |
| `HHMM` | Current time (24h) | `0945` |
| `audit-type-slug` | Inferred from scope/focus | `stride-owasp-full-audit` |
**Slug generation rules:**
- If no scope/focus specified → `stride-owasp-full-audit`
- If scope is auth-related → `auth-authorization-audit`
- If scope is API-related → `api-security-audit`
- If scope is infra-related → `infrastructure-security-audit`
- If user provides a focus string → kebab-case it (e.g., "payment flow" → `payment-flow-audit`)
### File Descriptions
#### overview.md
```markdown
# Security Audit — {audit-type}
**Date:** {YYYY-MM-DD HH:MM}
**Scope:** {files/directories}
**Focus:** {user-provided focus or "comprehensive"}
**Iterations:** {N completed} ({bounded or unlimited})
**Duration:** {approximate time}
## Summary
- **Total Findings:** {count}
- Critical: {n} | High: {n} | Medium: {n} | Low: {n} | Info: {n}
- **STRIDE Coverage:** {n}/6 categories tested
- **OWASP Coverage:** {n}/10 categories tested
- **Confirmed:** {n} | Likely: {n} | Possible: {n}
## Top 3 Critical Findings
1. [{title}]({findings.md#finding-1}) — {one-line description}
2. [{title}]({findings.md#finding-2}) — {one-line description}
3. [{title}]({findings.md#finding-3}) — {one-line description}
## Files in This Report
- [Threat Model](./threat-model.md) — STRIDE analysis, assets, trust boundaries
- [Attack Surface Map](./attack-surface-map.md) — entry points, data flows, abuse paths
- [Findings](./findings.md) — all findings ranked by severity
- [OWASP Coverage](./owasp-coverage.md) — per-category test results
- [Dependency Audit](./dependency-audit.md) — known CVEs in dependencies
- [Recommendations](./recommendations.md) — prioritized mitigations
- [Iteration Log](./security-audit-results.tsv) — raw data from every iteration
```
#### threat-model.md
Contains the full STRIDE analysis generated in the Setup Phase:
- Asset inventory table
- Trust boundary diagram
- STRIDE threat matrix (per asset × boundary)
- Risk ratings per threat
#### attack-surface-map.md
Contains the attack surface generated in the Setup Phase:
- Entry points (all API routes, webhooks, WebSocket endpoints)
- Data flows (input → processing → storage)
- Abuse paths (chained attack scenarios)
#### findings.md
All findings from the loop, in descending severity:
- Each finding uses the full proof structure (OWASP, STRIDE, location, evidence, mitigation)
- Findings are numbered and linkable via anchors (`#finding-1`, `#finding-2`)
#### owasp-coverage.md
Coverage matrix showing which OWASP categories were tested and results:
```markdown
| ID | Category | Tested | Findings | Status |
|----|----------|--------|----------|--------|
| A01 | Broken Access Control | ✓ | 2 | ⚠️ Issues found |
| A02 | Cryptographic Failures | ✓ | 0 | ✅ Clean |
| A03 | Injection | ✓ | 1 | ⚠️ Issues found |
| ... | ... | ... | ... | ... |
```
Also includes per-category detail: which specific checks were run and their results.
#### dependency-audit.md
Output of dependency security tools:
- `npm audit` / `yarn audit` (Node.js)
- `pip audit` / `safety check` (Python)
- `go vuln` (Go)
- `cargo audit` (Rust)
- Known CVEs, severity, affected versions, fix versions
#### recommendations.md
Prioritized action items with code fix snippets:
```markdown
## Priority 1 — Critical (Fix Immediately)
### 1. Restrict JWT Algorithm
**Finding:** [JWT Algorithm Confusion](./findings.md#finding-2)
**Effort:** 5 minutes
**Fix:**
\```typescript
// Before (vulnerable)
jwt.verify(token, secret);
// After (secure)
jwt.verify(token, secret, { algorithms: ['HS256'] });
\```
### 2. Add IDOR Protection
...
## Priority 2 — High (Fix This Sprint)
...
## Priority 3 — Medium (Plan for Next Sprint)
...
```
### Creation Protocol
1. At the **start** of `$autoresearch security`, create the folder:
```
mkdir -p security/{YYMMDD}-{HHMM}-{slug}
```
2. During the **Setup Phase**, write:
- `threat-model.md` (after STRIDE analysis)
- `attack-surface-map.md` (after surface mapping)
- `security-audit-results.tsv` (header row + baseline iteration)
3. During the **Loop**, append to:
- `security-audit-results.tsv` (after each iteration)
4. At **completion** (bounded loop end or interrupt), write:
- `findings.md` (all findings consolidated)
- `owasp-coverage.md` (coverage matrix)
- `dependency-audit.md` (tool output)
- `recommendations.md` (prioritized mitigations)
- `overview.md` (executive summary — written LAST, links to all other files)
5. Print the folder path to the user:
```
Security audit complete. Report saved to:
security/260315-0945-stride-owasp-full-audit/overview.md
```
### Gitignore
Add to `.gitignore` (if not already present):
```
security-audit-results.tsv
```
The `.tsv` iteration log is a working file. The `.md` reports are meant to be committed and shared.