Commit graph

17 commits

Author SHA1 Message Date
Frank Bria
f6fde6780b
fix(exit-detection): use explicit EXIT_SIGNAL instead of confidence threshold (#132)
Replace confidence-based heuristic in update_exit_signals() with explicit
EXIT_SIGNAL checking. JSON mode always has confidence >= 70 due to
deterministic scoring, causing completion_indicators to fill after 5 loops
and triggering premature exits even when Claude sets EXIT_SIGNAL: false.

Changes:
- lib/response_analyzer.sh: Check exit_signal == "true" instead of
  confidence >= 60 when updating completion_indicators array
- ralph_loop.sh: Update safety circuit breaker comment to reflect that
  completion_indicators now only accumulates on EXIT_SIGNAL=true
- tests/unit/test_exit_detection.bats: Add 4 TDD tests (32-35) validating
  the fix for update_exit_signals() behavior
- CLAUDE.md: Document fix as v0.11.1, update test counts (420 → 424)

Test count: 424 passing (100% pass rate)

Co-authored-by: Test User <test@example.com>
2026-01-26 16:41:05 -07:00
Test User
761db2f67d Merge main and apply code review fixes
Merged origin/main into PR branch and applied review feedback:

Review fixes:
- Guard against empty result_obj if jq fails (Macroscope)
- Prioritize result object's session_id over init message (CodeRabbit)
- Add regression test for arrays with session_id only in result element

Resolved conflicts:
- CLAUDE.md: Updated test count to 319

All 319 tests pass.
2026-01-21 21:42:29 -07:00
Test User
da1aed34c6 fix(analyzer,monitor): EXIT_SIGNAL detection and monitor paths (v0.10.1)
Fixes #113 - EXIT_SIGNAL not detected in JSON output format
Fixes #117 - ralph_monitor.sh uses wrong paths after v0.10.0 migration

Bug fixes:
- Parse EXIT_SIGNAL from .result field when Claude CLI returns JSON format
- Add safety circuit breaker: force exit after 5 consecutive completion indicators
- Fix checkbox parsing for indented markdown with POSIX [[:space:]]* pattern
- Update ralph_monitor.sh paths: status.json, logs/ralph.log, progress.json

Files changed:
- lib/response_analyzer.sh: Extract RALPH_STATUS from embedded .result text
- ralph_loop.sh: Safety circuit breaker + indented checkbox patterns
- ralph_monitor.sh: All paths updated for .ralph/ subfolder
- create_files.sh: Indented checkbox pattern consistency
- README.md: v0.10.1 changelog, version bump, test count update
- CLAUDE.md: Version bump to v0.10.1

All 310 tests pass.
2026-01-21 21:31:50 -07:00
zerone0x
5ce9bc81d7 fix(analyzer): handle Claude CLI JSON array output format
Claude Code CLI outputs a JSON array instead of a single object:
[{type: "system", ...}, {type: "assistant", ...}, {type: "result", ...}]

This caused parse_json_response to fail with "jq: invalid JSON text"
because it assumed the top-level JSON was an object.

Changes:
- Detect if JSON is an array before parsing
- Extract the "result" type message from the array
- Preserve session_id from init message for continuity
- Normalize to object format for existing parsing logic
- Clean up temporary file after processing

Fixes #112

Co-Authored-By: Claude <noreply@anthropic.com>
2026-01-21 22:56:53 +08:00
Frank Bria
9b19d70e35
feat(structure): migrate Ralph files to .ralph/ subfolder (#109)
* feat(structure): migrate Ralph files to .ralph/ subfolder

BREAKING CHANGE: Ralph configuration files now live in .ralph/ subfolder

This refactoring moves all Ralph-specific files into a hidden .ralph/
directory while keeping src/ at the project root. This improves
compatibility with existing tooling and keeps the project root clean.

Changes:
- Move PROMPT.md, @fix_plan.md, @AGENT.md to .ralph/
- Move specs/, logs/, docs/generated/, examples/ to .ralph/
- Move state files (.response_analysis, .circuit_breaker_state, etc.) to .ralph/
- Keep src/ at project root (unchanged)
- Add RALPH_DIR=".ralph" configuration variable
- Add ralph-migrate command for existing projects
- Create migrate_to_ralph_folder.sh migration script
- Update all path references in scripts and tests
- Update documentation (README.md, CLAUDE.md)

New project structure:
  project/
  ├── .ralph/           # Ralph configuration
  │   ├── PROMPT.md
  │   ├── @fix_plan.md
  │   ├── @AGENT.md
  │   ├── specs/
  │   ├── logs/
  │   └── docs/generated/
  └── src/              # Source code (unchanged)

Migration: Run `ralph-migrate` in existing projects to upgrade.

All 310 tests pass (100% pass rate).

* chore: add .claude/settings.local.json to .gitignore

* fix: address code review feedback for .ralph/ subfolder structure

Fixes multiple path-related issues identified in code review:

Test fixes:
- Fix create_sample_prompt to use $RALPH_DIR/PROMPT.md in test_session_continuity.bats
- Fix result_file path to use $RALPH_DIR/.json_parse_result in test_json_parsing.bats
- Fix @fix_plan.md and .response_analysis paths in test_cli_modern.bats
- Update templates directory missing test to account for global fallback

Template fix:
- Fix @fix_plan.md reference in templates/PROMPT.md to use .ralph/ prefix

Script fixes:
- Fix PROMPT_FILE comparison in ralph_loop.sh to use $RALPH_DIR/PROMPT.md
- Fix examples migration logic in migrate_to_ralph_folder.sh (remove premature mkdir)
- Move templates directory check AFTER cd in setup.sh (was checking wrong location)
- Add template directory validation with fallback to global templates

All 310 tests pass.

* Update migrate_to_ralph_folder.sh

Co-authored-by: macroscopeapp[bot] <170038800+macroscopeapp[bot]@users.noreply.github.com>

* fix: address code review feedback for .ralph/ subfolder structure

Code Review Fixes:
- Fix test_json_parsing.bats: all result_file and session file paths now use $RALPH_DIR prefix
- Fix ralph_loop.sh help text: paths now show .ralph/.ralph_session, .ralph/.call_count, etc.
- Fix migrate_to_ralph_folder.sh:
  - Proper error handling for date command (separate local declaration)
  - Use cp -a source/. dest/ pattern to preserve dotfiles and attributes
  - Remove 2>/dev/null suppression to surface copy errors
- Update create_files.sh to use .ralph/ structure for embedded scripts
- Update .gitignore with all .ralph/ state file paths
- Add old structure detection in ralph_loop.sh with helpful migration message

Version Update:
- Bump to v0.10.0 (breaking change: structural reorganization)
- Update README.md and CLAUDE.md with new version and release notes
- Add ralph-migrate documentation to Key Commands section

All 310 tests pass.

---------

Co-authored-by: macroscopeapp[bot] <170038800+macroscopeapp[bot]@users.noreply.github.com>
2026-01-20 23:22:30 -07:00
Frank Bria
aca3670cc7
fix(loop): respect Claude's EXIT_SIGNAL when checking completion indicators (#90)
* fix(loop): respect Claude's EXIT_SIGNAL when checking completion indicators

The should_exit_gracefully() function was exiting prematurely based solely
on completion_indicators heuristics, ignoring Claude's explicit EXIT_SIGNAL
in the RALPH_STATUS block. This caused premature exits during productive
iterations when Claude reported work in progress.

Changes:
- ralph_loop.sh: Added dual-condition check requiring BOTH completion
  indicators >= 2 AND exit_signal == true before exiting
- response_analyzer.sh: Added explicit_exit_signal_found flag to prevent
  natural language heuristics from overriding Claude's explicit intent
- Added 14 new tests (10 unit + 4 integration) covering EXIT_SIGNAL behavior

Decision matrix:
| indicators >= 2 | EXIT_SIGNAL | Result |
|-----------------|-------------|--------|
| true            | true        | Exit   |
| true            | false       | Continue |
| true            | missing     | Continue (defaults to false) |
| false           | true        | Continue (threshold not met) |

Fixes premature exit bug during productive development iterations.

* Update lib/response_analyzer.sh

Co-authored-by: macroscopeapp[bot] <170038800+macroscopeapp[bot]@users.noreply.github.com>

* test(exit): add STATUS=COMPLETE vs EXIT_SIGNAL=false conflict test

docs(exit): update CLAUDE.md with EXIT_SIGNAL gate documentation

- Added test for STATUS=COMPLETE with EXIT_SIGNAL=false conflict
  (EXIT_SIGNAL takes precedence, allowing phase completion without loop exit)
- Updated "Intelligent Exit Detection" section with dual-condition explanation
- Added "Completion Indicators with EXIT_SIGNAL Gate" section with decision table
- Documented conflict resolution behavior and implementation details

---------

Co-authored-by: Test User <test@example.com>
Co-authored-by: macroscopeapp[bot] <170038800+macroscopeapp[bot]@users.noreply.github.com>
2026-01-12 22:07:24 -07:00
Test User
a2e7e9385c fix(analyzer): address code review feedback
- Fix BSD date parsing to handle milliseconds in ISO timestamps
  (e.g., 2026-01-09T10:30:00.123+00:00)
- Document error_count mapping behavior when only has_errors=true is present
- Remove unused has_session_id_field variable
- Add debug logging for session persistence (controlled by VERBOSE_PROGRESS)
- Standardize session filename to .claude_session_id across all files

All 239 tests passing.
2026-01-09 17:59:08 -07:00
Test User
fdff095c18 feat(analyzer): add Claude CLI JSON format support and session management
- Extend parse_json_response() to support both flat and Claude CLI formats
  - Extract result, sessionId, and metadata fields
  - Support metadata.files_changed, metadata.has_errors, completion_status
  - Parse progress_indicators array for confidence boosting
- Add session management functions for continuity tracking:
  - store_session_id(): Persist session with ISO timestamp
  - get_last_session_id(): Retrieve stored session ID
  - should_resume_session(): Check session validity (24-hour expiration)
- Add get_epoch_seconds() to date_utils.sh for cross-platform epoch time
- Auto-persist sessionId to .session_id file during response analysis
- Add 16 new TDD tests for Claude CLI format and session management
- Update documentation for v0.9.6 (239 tests total)

Test count: 239 (up from 223)
2026-01-09 17:46:02 -07:00
frankbria
c7c6c9389d fix(security): use jq for JSON construction in analyze_response()
Replace two heredocs that write analysis results with jq construction
to prevent JSON injection via work_summary or other string fields.

Fixed locations:
- Line 205-235: JSON parsing path analysis result
- Line 351-381: Text parsing path analysis result

Both now use jq with --arg for strings and --argjson for numeric/boolean
fields, ensuring proper escaping of special characters.
2026-01-08 21:10:33 -07:00
frankbria
50f6ef7a96 fix(security): address code review security findings
Fixes three security issues identified in Phase 1.1 code review:

1. JSON injection in parse_json_response() (response_analyzer.sh:113-132)
   - Replace heredoc with jq construction using --arg for strings
   - Use --argjson for numeric/boolean fields
   - Ensures proper escaping of quotes, newlines, backslashes

2. Input validation for --allowed-tools flag (ralph_loop.sh:903-905)
   - Add VALID_TOOL_PATTERNS whitelist
   - Add validate_allowed_tools() function
   - Validate against whitelist in argument parsing
   - Allow Bash(...) patterns with any content

3. Shell injection in build_claude_command() (ralph_loop.sh:439-444)
   - Convert from string concatenation to command array
   - Use global CLAUDE_CMD_ARGS array
   - Execute with "${CLAUDE_CMD_ARGS[@]}" instead of bash -c
   - No manual escaping needed - array handles metacharacters

All 98 tests passing.

Closes #48, #50
2026-01-08 21:02:46 -07:00
frankbria
da5640ef8e feat(cli): add modern CLI commands with JSON output support (Phase 1.1)
Implements Issue #28 - modernize CLI commands for better Claude integration.

Key changes:
- Add JSON output format support with --output-format flag (default: json)
- Add session continuity with --continue flag and .claude_session_id file
- Add tool permissions via --allowed-tools flag
- Add build_loop_context() for loop-aware context injection
- Add detect_output_format() and parse_json_response() for JSON parsing
- Maintain backward compatibility with text output fallback
- Add version checking with check_claude_version()

New CLI options:
- --output-format json|text: Control Claude output format
- --allowed-tools "Write,Read,Bash(git *)": Restrict tool permissions
- --no-continue: Disable session continuity

Test coverage:
- 20 new JSON parsing tests (test_json_parsing.bats)
- 23 new CLI modern tests (test_cli_modern.bats)
- All 98 tests passing (100% pass rate)
2026-01-08 20:39:18 -07:00
frankbria
3d7db2c3ae feat(date): add cross-platform date compatibility for macOS and Linux
Add cross-platform date utility library to handle differences between
GNU date (Linux) and BSD date (macOS). Fixes issues with:
- ISO 8601 timestamp formatting (-Iseconds flag)
- Date arithmetic operations (-d vs -v flags)

Changes:
- Created lib/date_utils.sh with get_iso_timestamp() and get_next_hour_time()
- Updated ralph_loop.sh to use date utilities (2 instances)
- Updated lib/circuit_breaker.sh to use date utilities (4 instances)
- Updated lib/response_analyzer.sh to use date utilities (1 instance)

All date operations now work consistently across both platforms without
modification. The utility automatically detects the OS and uses the
appropriate date command syntax.

Tested on Linux with GNU date - all syntax checks and integration tests pass.
2025-12-31 16:04:49 -07:00
frankbria
63590b3d77 fix(circuit-breaker): fix multi-line error matching in detect_stuck_loop
Addresses CodeRabbit outside diff range comment on lines 268-283:

CRITICAL BUG:
The detect_stuck_loop function had a multi-line string handling bug where
only the first error line was checked against historical outputs when
multiple distinct errors were present.

Problem:
  grep -q "$current_errors" file.log
  # When $current_errors has multiple lines, grep only matches first line

Impact:
Stuck loops with multiple recurring errors would not be detected correctly,
potentially allowing Ralph to continue running despite being genuinely stuck.

Fix:
Changed from simple grep to nested loop checking:
- For each historical output file
  - For each error line in current output
    - ALL error lines must appear in that file
- Only returns "stuck" if ALL files contain ALL current errors

Used grep -qF for literal fixed-string matching (not regex) to avoid
edge cases with special characters in error messages.

Test Coverage:
Added 2 new test scenarios (7 → 9 total tests):

Test 8: Multiple distinct errors where ALL repeat across history
  Current: Error: Build failed + Fatal: DB lost + Exception: NPE
  History: All 3 files contain all 3 errors
  Expected: Stuck detected 

Test 9: Multiple errors where only some repeat
  Current: Error: Build failed + Fatal: DB lost
  History: Only first error appears consistently
  Expected: Not stuck 

All existing tests continue to pass, validating backward compatibility.

Test results:
✓ Error detection tests: 13/13 passing
✓ Stuck loop tests: 9/9 passing (was 7/7)
✓ Total: 22/22 tests passing

This ensures detect_stuck_loop correctly handles the real-world scenario
where Ralph gets stuck on multiple simultaneous recurring errors.

Addresses CodeRabbit review comment:
- Outside diff range (lines 268-283): Multi-line error matching bug
2025-12-31 14:03:56 -07:00
frankbria
890720a4f6 fix(circuit-breaker): fix detect_stuck_loop function and add tests
Addresses additional CodeRabbit findings in lib/response_analyzer.sh:

CRITICAL (line 267):
- Fixed detect_stuck_loop() to use two-stage filtering
- Was using naive grep -i "error\|failed" pattern
- Now filters JSON fields before extracting errors
- Pattern aligned with analyze_response() for consistency

DEAD CODE (line 18):
- Removed unused STUCK_INDICATORS array
- Array was defined but never referenced in code
- Reduces maintenance burden per coding guidelines

TEST COVERAGE:
- Added comprehensive test suite for detect_stuck_loop()
- New file: tests/test_stuck_loop_detection.sh
- 7 test scenarios validating:
  * JSON fields don't trigger false stuck detection
  * Actual repeated errors are correctly detected
  * Type annotations are properly excluded
  * Function returns appropriate exit codes

Test results:
✓ Error detection tests: 13/13 passing
✓ Stuck loop tests: 7/7 passing
✓ Total: 20/20 tests passing

This ensures both error detection functions (analyze_response and
detect_stuck_loop) use identical filtering logic, preventing circuit
breaker false positives across all code paths.

Addresses CodeRabbit review comments:
- Outside diff range comment: line 267 (Critical)
- Outside diff range comment: line 18 (Dead code)
- Outside diff range comment: lines 254-286 (Test coverage)
2025-12-31 13:45:02 -07:00
frankbria
90fb5587a1 fix(circuit-breaker): address CodeRabbit review feedback
Aligns error detection patterns across all implementations and improves
test coverage based on CodeRabbit's critical and major findings.

Changes:
1. CRITICAL: Align lib/response_analyzer.sh pattern with ralph_loop.sh
   - Changed Stage 1 filter from '"[^"]*\(error\|failed\)"[^"]*":' to '"[^"]*error[^"]*":'
   - Removed bare 'cannot' and 'unable to' from Stage 2 (prevent false positives in prose)
   - Both files now use identical patterns for consistency

2. MAJOR: Improved test coverage
   - Renamed test 10 from "Cannot/unable in error context" to "Error prefix with descriptive message"
   - Added test 10a to validate bare "cannot/unable" DON'T trigger false positives
   - Now testing 13 scenarios (was 12)

3. MINOR: Added comprehensive test strategy documentation
   - Header comments explain two-stage filtering approach
   - Documents pattern consistency requirement
   - Lists all 13 test scenarios and their purpose

Test results:
✓ All 13 tests passing
✓ Pattern consistency validated across ralph_loop.sh and lib/response_analyzer.sh
✓ False positive scenarios properly excluded

Addresses CodeRabbit review comments:
- r2655862688 (Critical pattern inconsistency)
- r2655862689 (Major test coverage gap)
- r2655862690 (Minor misleading test name)
2025-12-31 13:34:21 -07:00
frankbria
8fc53755bf fix(circuit-breaker): eliminate JSON field false positives in error detection
Fixes circuit breaker opening prematurely due to naive error pattern matching
that treated JSON field names like "is_error": false as actual errors.

Changes:
- ralph_loop.sh: Implement two-stage error detection with JSON filtering
- lib/response_analyzer.sh: Apply same filtering to error counting
- tests/test_error_detection.sh: Add comprehensive test suite (12 scenarios)

Error detection now:
- Filters out JSON field patterns before searching for errors
- Uses context-specific patterns (^Error:, ]: error, Exception, Fatal)
- Avoids type annotations (error: Error) and code identifiers
- Includes debug logging when VERBOSE_PROGRESS=true

Test coverage validates:
✓ JSON fields don't trigger false positives
✓ Real error messages are correctly detected
✓ Mixed content handled properly
✓ Code diffs and documentation excluded

This prevents the consecutive_same_error counter from incrementing on
false positives, eliminating unnecessary circuit breaker trips.
2025-12-31 13:25:09 -07:00
frankbria
2cf06b0de2 Implement Phase 1 critical fixes: Response analyzer & circuit breaker
Implements all Phase 1 recommendations from expert panel review:

1. **Response Analysis Pipeline** (Martin Fowler recommendation)
   - New lib/response_analyzer.sh component
   - Parses Claude Code output for completion signals
   - Detects test-only loops and stagnation
   - Updates .exit_signals file with structured data
   - Tracks confidence scores and progress indicators

2. **Circuit Breaker Pattern** (Michael Nygard recommendation)
   - New lib/circuit_breaker.sh component
   - Three-state pattern: CLOSED → HALF_OPEN → OPEN
   - Prevents runaway token consumption
   - Detects: no progress (3 loops), same errors (5 loops)
   - Automatic halt with clear user guidance
   - Manual reset capability

3. **Structured Output Contract** (Sam Newman recommendation)
   - Updated PROMPT.md template with RALPH_STATUS format
   - Defines clear JSON-parseable exit signals
   - SMART criteria for completion detection
   - Concrete examples for all scenarios

4. **Integration & Testing**
   - ralph_loop.sh integration of both components
   - 20 comprehensive BATS integration tests (all passing)
   - Tests cover: signal detection, circuit states, full loop flows
   - Validates Phase 1 implementation correctness

**Impact**: Solves infinite loop problem, enables reliable exit detection,
prevents token waste through systematic stagnation detection.

**Test Results**: 20/20 integration tests passing

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-01 21:11:18 -07:00