Commit graph

19 commits

Author SHA1 Message Date
Test User
d3310d1f3f feat(session): add session lifecycle management with auto-reset triggers
- Add session management functions: get_session_id(), reset_session(),
  log_session_transition(), init_session_tracking()
- Session auto-reset on: circuit breaker open, manual interrupt,
  project completion, manual circuit reset
- Add --reset-session CLI flag for manual session reset
- Add session history tracking (.ralph_session_history, last 50 entries)
- New config: RALPH_SESSION_FILE, RALPH_SESSION_HISTORY_FILE
- Add 26 comprehensive tests for session continuity (TDD)
- All 265 tests pass (up from 239)
- Update CLAUDE.md with v0.9.7 release notes
2026-01-10 10:27:42 -07:00
Test User
a2e7e9385c fix(analyzer): address code review feedback
- Fix BSD date parsing to handle milliseconds in ISO timestamps
  (e.g., 2026-01-09T10:30:00.123+00:00)
- Document error_count mapping behavior when only has_errors=true is present
- Remove unused has_session_id_field variable
- Add debug logging for session persistence (controlled by VERBOSE_PROGRESS)
- Standardize session filename to .claude_session_id across all files

All 239 tests passing.
2026-01-09 17:59:08 -07:00
Test User
fdff095c18 feat(analyzer): add Claude CLI JSON format support and session management
- Extend parse_json_response() to support both flat and Claude CLI formats
  - Extract result, sessionId, and metadata fields
  - Support metadata.files_changed, metadata.has_errors, completion_status
  - Parse progress_indicators array for confidence boosting
- Add session management functions for continuity tracking:
  - store_session_id(): Persist session with ISO timestamp
  - get_last_session_id(): Retrieve stored session ID
  - should_resume_session(): Check session validity (24-hour expiration)
- Add get_epoch_seconds() to date_utils.sh for cross-platform epoch time
- Auto-persist sessionId to .session_id file during response analysis
- Add 16 new TDD tests for Claude CLI format and session management
- Update documentation for v0.9.6 (239 tests total)

Test count: 239 (up from 223)
2026-01-09 17:46:02 -07:00
Test User
b06d979b0b test(import): add 22 comprehensive tests for ralph_import.sh
Add integration tests for PRD to Ralph format conversion:
- File format support tests (.md, .txt, .json)
- Output file creation tests (PROMPT.md, @fix_plan.md, specs/requirements.md)
- Project naming tests (custom names, auto-detection from filename)
- Error handling tests (missing files, dependencies, conversion failures)
- Help and usage tests
- Full workflow integration tests
- Edge case handling (hyphens, uppercase, subdirectory paths)

Test infrastructure:
- Mock ralph-setup command using PATH manipulation
- Mock Claude Code CLI for isolated conversion testing
- Added create_sample_prd_txt() fixture helper

Test count: 223 (up from 201)
2026-01-09 16:13:23 -07:00
Test User
a6587aa189 fix(test): improve test reliability and remove tautologies
- Remove unused 'load mocks' (mocks.bash not needed)
- Use GIT_AUTHOR_*/GIT_COMMITTER_* env vars instead of git config --global
- Prefix git commands with 'command' to bypass shell function overrides
- Fix tautological assertions in edge case tests:
  - Rename test to "succeeds when run in existing directory (idempotent)"
  - Assert success ($status -eq 0) instead of always-true condition
2026-01-09 15:47:41 -07:00
Test User
c68e484f7d test(setup): add 36 comprehensive tests for setup.sh
Add integration tests validating project initialization:
- Directory creation (project dir, subdirectories)
- Template copying (PROMPT.md, @fix_plan.md, @AGENT.md, specs)
- Git initialization (repo, commit, message)
- README creation and content
- Custom/default project names
- Working directory behavior
- Error handling (missing templates)
- Output message validation

Test count: 201 (up from 165)
2026-01-09 15:01:53 -07:00
frankbria
3503b9b27b fix: address code review feedback
README.md:
- Update version badge to v0.9.3
- Update test count to 165 in all locations
- Update test coverage breakdown (111 unit + 54 integration)

test_installation.bats:
- Add missing mock setup.sh in setup() function
- Fix dependency test to mock all three deps (jq, git, node/npx)
- Remove unused source_install_functions helper function
2026-01-09 14:20:58 -07:00
frankbria
6688c27681 test(install): add 14 comprehensive installation tests
- Add test_installation.bats with full coverage of install.sh
- Tests cover directory creation, command installation, permissions
- Template and lib file copying verification
- Dependency detection with mocked failures (jq, git, node)
- PATH detection and warning system tests
- Uninstallation cleanup verification
- Idempotency testing (run twice without errors)
- End-to-end installation workflow validation
- All tests use isolated temp directories for safety
- Update CLAUDE.md with new test count (165 total)
- Fix npm test script to run tests recursively
- Version bump to v0.9.3
2026-01-09 14:05:46 -07:00
frankbria
30cdf78331 fix(loop): replace non-existent --prompt-file with -p flag
The build_claude_command() function was incorrectly using --prompt-file
which doesn't exist in Claude Code CLI. This fix:

- Replaces --prompt-file with -p flag plus prompt content
- Reads prompt content via $(cat "$prompt_file") before execution
- Adds error handling for missing prompt files
- Maintains shell injection safety through array-based command building
- Updates comments to reflect the correct approach

Adds 6 TDD tests verifying the fix:
- Uses -p flag instead of --prompt-file
- Reads prompt file content correctly
- Handles missing prompt file
- Includes all modern CLI flags
- Handles multiline prompt content
- Prevents shell injection

Test count: 145 -> 151 (all passing)
2026-01-09 13:44:50 -07:00
frankbria
7407f0f45b test(cli): add --allowed-tools test and code review report
Address code review finding by adding dedicated test for
--allowed-tools flag validation.

Add code review report documenting:
- 0 critical issues
- 0 major issues
- 1 minor issue (addressed in this commit)
- 6 positive findings

Test count: 27 CLI parsing tests (105 total unit tests)

Refs #10
2026-01-08 22:42:50 -07:00
frankbria
fffcc26427 test(cli): add comprehensive CLI argument parsing tests
Add 26 new BATS tests validating all CLI flags in ralph_loop.sh:

- Help flag tests (2): --help, -h short flag
- Flag value tests (6): --calls, --prompt, --monitor, --verbose, --timeout
- Status flag tests (2): --status with/without status.json
- Circuit breaker tests (2): --reset-circuit, --circuit-status
- Invalid input tests (3): unknown flag, invalid timeout, invalid format
- Multiple flags tests (3): combinations, all flags, early exit
- Flag order tests (2): verify order independence
- Short flag tests (6): -c, -p, -s, -m, -v, -t equivalence

Test strategy uses --help as early-exit escape to validate parsing
without triggering main loop execution.

Closes #10
2026-01-08 22:38:23 -07:00
frankbria
da5640ef8e feat(cli): add modern CLI commands with JSON output support (Phase 1.1)
Implements Issue #28 - modernize CLI commands for better Claude integration.

Key changes:
- Add JSON output format support with --output-format flag (default: json)
- Add session continuity with --continue flag and .claude_session_id file
- Add tool permissions via --allowed-tools flag
- Add build_loop_context() for loop-aware context injection
- Add detect_output_format() and parse_json_response() for JSON parsing
- Maintain backward compatibility with text output fallback
- Add version checking with check_claude_version()

New CLI options:
- --output-format json|text: Control Claude output format
- --allowed-tools "Write,Read,Bash(git *)": Restrict tool permissions
- --no-continue: Disable session continuity

Test coverage:
- 20 new JSON parsing tests (test_json_parsing.bats)
- 23 new CLI modern tests (test_cli_modern.bats)
- All 98 tests passing (100% pass rate)
2026-01-08 20:39:18 -07:00
frankbria
63590b3d77 fix(circuit-breaker): fix multi-line error matching in detect_stuck_loop
Addresses CodeRabbit outside diff range comment on lines 268-283:

CRITICAL BUG:
The detect_stuck_loop function had a multi-line string handling bug where
only the first error line was checked against historical outputs when
multiple distinct errors were present.

Problem:
  grep -q "$current_errors" file.log
  # When $current_errors has multiple lines, grep only matches first line

Impact:
Stuck loops with multiple recurring errors would not be detected correctly,
potentially allowing Ralph to continue running despite being genuinely stuck.

Fix:
Changed from simple grep to nested loop checking:
- For each historical output file
  - For each error line in current output
    - ALL error lines must appear in that file
- Only returns "stuck" if ALL files contain ALL current errors

Used grep -qF for literal fixed-string matching (not regex) to avoid
edge cases with special characters in error messages.

Test Coverage:
Added 2 new test scenarios (7 → 9 total tests):

Test 8: Multiple distinct errors where ALL repeat across history
  Current: Error: Build failed + Fatal: DB lost + Exception: NPE
  History: All 3 files contain all 3 errors
  Expected: Stuck detected 

Test 9: Multiple errors where only some repeat
  Current: Error: Build failed + Fatal: DB lost
  History: Only first error appears consistently
  Expected: Not stuck 

All existing tests continue to pass, validating backward compatibility.

Test results:
✓ Error detection tests: 13/13 passing
✓ Stuck loop tests: 9/9 passing (was 7/7)
✓ Total: 22/22 tests passing

This ensures detect_stuck_loop correctly handles the real-world scenario
where Ralph gets stuck on multiple simultaneous recurring errors.

Addresses CodeRabbit review comment:
- Outside diff range (lines 268-283): Multi-line error matching bug
2025-12-31 14:03:56 -07:00
frankbria
890720a4f6 fix(circuit-breaker): fix detect_stuck_loop function and add tests
Addresses additional CodeRabbit findings in lib/response_analyzer.sh:

CRITICAL (line 267):
- Fixed detect_stuck_loop() to use two-stage filtering
- Was using naive grep -i "error\|failed" pattern
- Now filters JSON fields before extracting errors
- Pattern aligned with analyze_response() for consistency

DEAD CODE (line 18):
- Removed unused STUCK_INDICATORS array
- Array was defined but never referenced in code
- Reduces maintenance burden per coding guidelines

TEST COVERAGE:
- Added comprehensive test suite for detect_stuck_loop()
- New file: tests/test_stuck_loop_detection.sh
- 7 test scenarios validating:
  * JSON fields don't trigger false stuck detection
  * Actual repeated errors are correctly detected
  * Type annotations are properly excluded
  * Function returns appropriate exit codes

Test results:
✓ Error detection tests: 13/13 passing
✓ Stuck loop tests: 7/7 passing
✓ Total: 20/20 tests passing

This ensures both error detection functions (analyze_response and
detect_stuck_loop) use identical filtering logic, preventing circuit
breaker false positives across all code paths.

Addresses CodeRabbit review comments:
- Outside diff range comment: line 267 (Critical)
- Outside diff range comment: line 18 (Dead code)
- Outside diff range comment: lines 254-286 (Test coverage)
2025-12-31 13:45:02 -07:00
frankbria
90fb5587a1 fix(circuit-breaker): address CodeRabbit review feedback
Aligns error detection patterns across all implementations and improves
test coverage based on CodeRabbit's critical and major findings.

Changes:
1. CRITICAL: Align lib/response_analyzer.sh pattern with ralph_loop.sh
   - Changed Stage 1 filter from '"[^"]*\(error\|failed\)"[^"]*":' to '"[^"]*error[^"]*":'
   - Removed bare 'cannot' and 'unable to' from Stage 2 (prevent false positives in prose)
   - Both files now use identical patterns for consistency

2. MAJOR: Improved test coverage
   - Renamed test 10 from "Cannot/unable in error context" to "Error prefix with descriptive message"
   - Added test 10a to validate bare "cannot/unable" DON'T trigger false positives
   - Now testing 13 scenarios (was 12)

3. MINOR: Added comprehensive test strategy documentation
   - Header comments explain two-stage filtering approach
   - Documents pattern consistency requirement
   - Lists all 13 test scenarios and their purpose

Test results:
✓ All 13 tests passing
✓ Pattern consistency validated across ralph_loop.sh and lib/response_analyzer.sh
✓ False positive scenarios properly excluded

Addresses CodeRabbit review comments:
- r2655862688 (Critical pattern inconsistency)
- r2655862689 (Major test coverage gap)
- r2655862690 (Minor misleading test name)
2025-12-31 13:34:21 -07:00
frankbria
8fc53755bf fix(circuit-breaker): eliminate JSON field false positives in error detection
Fixes circuit breaker opening prematurely due to naive error pattern matching
that treated JSON field names like "is_error": false as actual errors.

Changes:
- ralph_loop.sh: Implement two-stage error detection with JSON filtering
- lib/response_analyzer.sh: Apply same filtering to error counting
- tests/test_error_detection.sh: Add comprehensive test suite (12 scenarios)

Error detection now:
- Filters out JSON field patterns before searching for errors
- Uses context-specific patterns (^Error:, ]: error, Exception, Fatal)
- Avoids type annotations (error: Error) and code identifiers
- Includes debug logging when VERBOSE_PROGRESS=true

Test coverage validates:
✓ JSON fields don't trigger false positives
✓ Real error messages are correctly detected
✓ Mixed content handled properly
✓ Code diffs and documentation excluded

This prevents the consecutive_same_error counter from incrementing on
false positives, eliminating unnecessary circuit breaker trips.
2025-12-31 13:25:09 -07:00
frankbria
3ae67f66ac Phase 2: Requirements, testing, and documentation enhancements
Implements all Phase 2 high-priority recommendations from expert panel:

**1. Requirements Improvement** (Karl Wiegers, Gojko Adzic)
- Enhanced templates/PROMPT.md with 6 concrete Given/When/Then scenarios
- Specification by Example format for all exit conditions
- Clear expectations for each scenario type:
  * Successful completion
  * Test-only loops
  * Stuck on errors
  * No work remaining
  * Making progress
  * Blocked on dependencies

**2. Use Case Documentation** (Alistair Cockburn)
- Created comprehensive USE_CASES.md (600+ lines)
- Defined 6 primary use cases with full Cockburn format:
  * UC-1: Execute Development Loop
  * UC-2: Detect Project Completion
  * UC-3: Prevent Resource Waste
  * UC-4: Handle API Rate Limits
  * UC-5: Provide Loop Monitoring
  * UC-6: Reset Circuit Breaker
- Includes actors, goals, success scenarios, extensions, edge cases
- Clear goal hierarchy and success metrics

**3. Enhanced Test Coverage** (Lisa Crispin, Janet Gregory)
- Added tests/integration/test_edge_cases.bats (20 new tests)
- Edge cases: empty files, large files, corrupted JSON, unicode
- Boundary conditions: exact thresholds, overflow scenarios
- Error conditions: missing git, malformed data, rapid transitions
- All 40 integration tests passing (100% success rate)

**4. Circuit Breaker Robustness**
- Enhanced init_circuit_breaker() with corruption detection
- Auto-recovery from corrupted state/history files
- Validates JSON before use, recreates if invalid

**5. Specification Workshop Guide**
- Created SPECIFICATION_WORKSHOP.md
- Three Amigos methodology with templates
- Includes complete example workshop
- Best practices and red flags
- Quick 15-minute template for small features

**Test Results**: 40/40 integration tests passing
**Documentation Added**: 1,200+ lines (USE_CASES.md, SPECIFICATION_WORKSHOP.md)
**Coverage Improvement**: Edge cases and error conditions fully tested

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-01 21:34:54 -07:00
frankbria
2cf06b0de2 Implement Phase 1 critical fixes: Response analyzer & circuit breaker
Implements all Phase 1 recommendations from expert panel review:

1. **Response Analysis Pipeline** (Martin Fowler recommendation)
   - New lib/response_analyzer.sh component
   - Parses Claude Code output for completion signals
   - Detects test-only loops and stagnation
   - Updates .exit_signals file with structured data
   - Tracks confidence scores and progress indicators

2. **Circuit Breaker Pattern** (Michael Nygard recommendation)
   - New lib/circuit_breaker.sh component
   - Three-state pattern: CLOSED → HALF_OPEN → OPEN
   - Prevents runaway token consumption
   - Detects: no progress (3 loops), same errors (5 loops)
   - Automatic halt with clear user guidance
   - Manual reset capability

3. **Structured Output Contract** (Sam Newman recommendation)
   - Updated PROMPT.md template with RALPH_STATUS format
   - Defines clear JSON-parseable exit signals
   - SMART criteria for completion detection
   - Concrete examples for all scenarios

4. **Integration & Testing**
   - ralph_loop.sh integration of both components
   - 20 comprehensive BATS integration tests (all passing)
   - Tests cover: signal detection, circuit states, full loop flows
   - Validates Phase 1 implementation correctness

**Impact**: Solves infinite loop problem, enables reliable exit detection,
prevents token waste through systematic stagnation detection.

**Test Results**: 20/20 integration tests passing

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-01 21:11:18 -07:00
frankbria
8ad49e6f27 Add comprehensive test infrastructure and core unit tests
Implemented Phase 1 of the test implementation plan:

Test Infrastructure:
- BATS testing framework with helper utilities
- Mock system for external dependencies
- Fixture library for test data
- GitHub Actions CI/CD pipeline
- npm test scripts configured

Core Unit Tests (35 tests, 100% pass rate):
- Rate limiting tests (15 tests)
  * can_make_call() function - 7 tests
  * increment_call_counter() function - 6 tests
  * Edge cases - 2 tests

- Exit detection tests (20 tests)
  * Test saturation detection - 4 tests
  * Done signals detection - 4 tests
  * Completion indicators - 3 tests
  * @fix_plan.md validation - 5 tests
  * Error handling - 4 tests

Documentation:
- IMPLEMENTATION_PLAN.md - Complete 6-week roadmap
- TEST_IMPLEMENTATION_SUMMARY.md - Detailed achievement report
- STATUS.md - Quick status overview

Test Coverage:
- ~87% coverage of core ralph_loop.sh logic
- All tests passing with 100% success rate
- Average execution time: <1 second per test

Files Added:
- tests/unit/test_rate_limiting.bats
- tests/unit/test_exit_detection.bats
- tests/helpers/test_helper.bash
- tests/helpers/mocks.bash
- tests/helpers/fixtures.bash
- .github/workflows/test.yml
- package.json with test scripts

Next Steps: Continue with Weeks 2-6 per IMPLEMENTATION_PLAN.md

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-09-30 22:56:02 -07:00