When Claude Code is denied permission to execute commands (e.g., npm install), Ralph now detects this from the permission_denials array in the JSON output and halts the loop immediately with clear guidance for the user. Changes: - Add permission denial detection to parse_json_response() in response_analyzer.sh - Extract permission_denials array from Claude Code JSON output - Track has_permission_denials, permission_denial_count, denied_commands - Add analyze_response() support for permission denial fields - Add permission denial exit condition to should_exit_gracefully() in ralph_loop.sh - Permission denial takes highest priority among exit conditions - Display helpful guidance for updating ALLOWED_TOOLS in .ralphrc - Update circuit breaker with CB_PERMISSION_DENIAL_THRESHOLD=2 - Track consecutive_permission_denials in state file - Open circuit after 2 consecutive loops with permission denials - Add 11 new TDD tests (6 in test_json_parsing.bats, 5 in test_exit_detection.bats) - Update documentation in CLAUDE.md and README.md Test count: 452 (up from 452 - added 11 new tests) Fixes #101 Co-authored-by: Test User <test@example.com>
This commit is contained in:
parent
5cad271eac
commit
328294847d
7 changed files with 519 additions and 6 deletions
22
CLAUDE.md
22
CLAUDE.md
|
|
@ -378,6 +378,28 @@ fi
|
|||
- `CB_NO_PROGRESS_THRESHOLD=3` - Open circuit after 3 loops with no file changes
|
||||
- `CB_SAME_ERROR_THRESHOLD=5` - Open circuit after 5 loops with repeated errors
|
||||
- `CB_OUTPUT_DECLINE_THRESHOLD=70%` - Open circuit if output declines by >70%
|
||||
- `CB_PERMISSION_DENIAL_THRESHOLD=2` - Open circuit after 2 loops with permission denials (Issue #101)
|
||||
|
||||
### Permission Denial Detection (Issue #101)
|
||||
|
||||
When Claude Code is denied permission to execute commands (e.g., `npm install`), Ralph detects this from the `permission_denials` array in the JSON output and halts the loop immediately:
|
||||
|
||||
1. **Detection**: The `parse_json_response()` function extracts `permission_denials` from Claude Code output
|
||||
2. **Fields tracked**:
|
||||
- `has_permission_denials` (boolean)
|
||||
- `permission_denial_count` (integer)
|
||||
- `denied_commands` (array of command strings)
|
||||
3. **Exit behavior**: When `has_permission_denials=true`, Ralph exits with reason "permission_denied"
|
||||
4. **User guidance**: Ralph displays instructions to update `ALLOWED_TOOLS` in `.ralphrc`
|
||||
|
||||
**Example `.ralphrc` tool patterns:**
|
||||
```bash
|
||||
# Broad patterns (recommended for development)
|
||||
ALLOWED_TOOLS="Write,Read,Edit,Bash(git *),Bash(npm *),Bash(pytest)"
|
||||
|
||||
# Specific patterns (more restrictive)
|
||||
ALLOWED_TOOLS="Write,Read,Edit,Bash(git commit),Bash(npm install)"
|
||||
```
|
||||
|
||||
### Error Detection
|
||||
|
||||
|
|
|
|||
|
|
@ -681,6 +681,11 @@ tail -f .ralph/logs/ralph.log
|
|||
- **tmux Session Lost** - Use `tmux list-sessions` and `tmux attach` to reconnect
|
||||
- **Session Expired** - Sessions expire after 24 hours by default; use `--reset-session` to start fresh
|
||||
- **timeout: command not found (macOS)** - Install GNU coreutils: `brew install coreutils`
|
||||
- **Permission Denied** - Ralph halts when Claude Code is denied permission for commands:
|
||||
1. Edit `.ralphrc` and update `ALLOWED_TOOLS` to include required tools
|
||||
2. Common patterns: `Bash(npm *)`, `Bash(git *)`, `Bash(pytest)`
|
||||
3. Run `ralph --reset-session` after updating `.ralphrc`
|
||||
4. Restart with `ralph --monitor`
|
||||
|
||||
## Contributing
|
||||
|
||||
|
|
|
|||
|
|
@ -19,6 +19,7 @@ CB_HISTORY_FILE="$RALPH_DIR/.circuit_breaker_history"
|
|||
CB_NO_PROGRESS_THRESHOLD=3 # Open circuit after N loops with no progress
|
||||
CB_SAME_ERROR_THRESHOLD=5 # Open circuit after N loops with same error
|
||||
CB_OUTPUT_DECLINE_THRESHOLD=70 # Open circuit if output declines by >70%
|
||||
CB_PERMISSION_DENIAL_THRESHOLD=2 # Open circuit after N loops with permission denials (Issue #101)
|
||||
|
||||
# Colors
|
||||
RED='\033[0;31m'
|
||||
|
|
@ -44,6 +45,7 @@ init_circuit_breaker() {
|
|||
"last_change": "$(get_iso_timestamp)",
|
||||
"consecutive_no_progress": 0,
|
||||
"consecutive_same_error": 0,
|
||||
"consecutive_permission_denials": 0,
|
||||
"last_progress_loop": 0,
|
||||
"total_opens": 0,
|
||||
"reason": ""
|
||||
|
|
@ -98,11 +100,13 @@ record_loop_result() {
|
|||
local current_state=$(echo "$state_data" | jq -r '.state')
|
||||
local consecutive_no_progress=$(echo "$state_data" | jq -r '.consecutive_no_progress' | tr -d '[:space:]')
|
||||
local consecutive_same_error=$(echo "$state_data" | jq -r '.consecutive_same_error' | tr -d '[:space:]')
|
||||
local consecutive_permission_denials=$(echo "$state_data" | jq -r '.consecutive_permission_denials // 0' | tr -d '[:space:]')
|
||||
local last_progress_loop=$(echo "$state_data" | jq -r '.last_progress_loop' | tr -d '[:space:]')
|
||||
|
||||
# Ensure integers
|
||||
consecutive_no_progress=$((consecutive_no_progress + 0))
|
||||
consecutive_same_error=$((consecutive_same_error + 0))
|
||||
consecutive_permission_denials=$((consecutive_permission_denials + 0))
|
||||
last_progress_loop=$((last_progress_loop + 0))
|
||||
|
||||
# Detect progress from multiple sources:
|
||||
|
|
@ -131,6 +135,18 @@ record_loop_result() {
|
|||
ralph_files_modified=$((ralph_files_modified + 0))
|
||||
fi
|
||||
|
||||
# Track permission denials (Issue #101)
|
||||
local has_permission_denials="false"
|
||||
if [[ -f "$response_analysis_file" ]]; then
|
||||
has_permission_denials=$(jq -r '.analysis.has_permission_denials // false' "$response_analysis_file" 2>/dev/null || echo "false")
|
||||
fi
|
||||
|
||||
if [[ "$has_permission_denials" == "true" ]]; then
|
||||
consecutive_permission_denials=$((consecutive_permission_denials + 1))
|
||||
else
|
||||
consecutive_permission_denials=0
|
||||
fi
|
||||
|
||||
# Determine if progress was made
|
||||
if [[ $files_changed -gt 0 ]]; then
|
||||
# Git shows uncommitted changes - clear progress
|
||||
|
|
@ -167,7 +183,11 @@ record_loop_result() {
|
|||
case $current_state in
|
||||
"$CB_STATE_CLOSED")
|
||||
# Normal operation - check for failure conditions
|
||||
if [[ $consecutive_no_progress -ge $CB_NO_PROGRESS_THRESHOLD ]]; then
|
||||
# Permission denials take highest priority (Issue #101)
|
||||
if [[ $consecutive_permission_denials -ge $CB_PERMISSION_DENIAL_THRESHOLD ]]; then
|
||||
new_state="$CB_STATE_OPEN"
|
||||
reason="Permission denied in $consecutive_permission_denials consecutive loops - update ALLOWED_TOOLS in .ralphrc"
|
||||
elif [[ $consecutive_no_progress -ge $CB_NO_PROGRESS_THRESHOLD ]]; then
|
||||
new_state="$CB_STATE_OPEN"
|
||||
reason="No progress detected in $consecutive_no_progress consecutive loops"
|
||||
elif [[ $consecutive_same_error -ge $CB_SAME_ERROR_THRESHOLD ]]; then
|
||||
|
|
@ -181,7 +201,11 @@ record_loop_result() {
|
|||
|
||||
"$CB_STATE_HALF_OPEN")
|
||||
# Monitoring mode - either recover or fail
|
||||
if [[ "$has_progress" == "true" ]]; then
|
||||
# Permission denials take highest priority (Issue #101)
|
||||
if [[ $consecutive_permission_denials -ge $CB_PERMISSION_DENIAL_THRESHOLD ]]; then
|
||||
new_state="$CB_STATE_OPEN"
|
||||
reason="Permission denied in $consecutive_permission_denials consecutive loops - update ALLOWED_TOOLS in .ralphrc"
|
||||
elif [[ "$has_progress" == "true" ]]; then
|
||||
new_state="$CB_STATE_CLOSED"
|
||||
reason="Progress detected, circuit recovered"
|
||||
elif [[ $consecutive_no_progress -ge $CB_NO_PROGRESS_THRESHOLD ]]; then
|
||||
|
|
@ -209,6 +233,7 @@ record_loop_result() {
|
|||
"last_change": "$(get_iso_timestamp)",
|
||||
"consecutive_no_progress": $consecutive_no_progress,
|
||||
"consecutive_same_error": $consecutive_same_error,
|
||||
"consecutive_permission_denials": $consecutive_permission_denials,
|
||||
"last_progress_loop": $last_progress_loop,
|
||||
"total_opens": $total_opens,
|
||||
"reason": "$reason",
|
||||
|
|
@ -317,6 +342,7 @@ reset_circuit_breaker() {
|
|||
"last_change": "$(get_iso_timestamp)",
|
||||
"consecutive_no_progress": 0,
|
||||
"consecutive_same_error": 0,
|
||||
"consecutive_permission_denials": 0,
|
||||
"last_progress_loop": 0,
|
||||
"total_opens": 0,
|
||||
"reason": "$reason"
|
||||
|
|
|
|||
|
|
@ -178,6 +178,22 @@ parse_json_response() {
|
|||
# Progress indicators: from Claude CLI metadata (optional)
|
||||
local progress_count=$(jq -r '.metadata.progress_indicators | if . then length else 0 end' "$output_file" 2>/dev/null)
|
||||
|
||||
# Permission denials: from Claude Code output (Issue #101)
|
||||
# When Claude Code is denied permission to run commands, it outputs a permission_denials array
|
||||
local permission_denial_count=$(jq -r '.permission_denials | if . then length else 0 end' "$output_file" 2>/dev/null)
|
||||
permission_denial_count=$((permission_denial_count + 0)) # Ensure integer
|
||||
|
||||
local has_permission_denials="false"
|
||||
if [[ $permission_denial_count -gt 0 ]]; then
|
||||
has_permission_denials="true"
|
||||
fi
|
||||
|
||||
# Extract denied commands for logging/display
|
||||
local denied_commands_json="[]"
|
||||
if [[ $permission_denial_count -gt 0 ]]; then
|
||||
denied_commands_json=$(jq -r '[.permission_denials[].command // empty]' "$output_file" 2>/dev/null || echo "[]")
|
||||
fi
|
||||
|
||||
# Normalize values
|
||||
# Convert exit_signal to boolean string
|
||||
if [[ "$exit_signal" == "true" || "$status" == "COMPLETE" || "$completion_status" == "complete" || "$completion_status" == "COMPLETE" ]]; then
|
||||
|
|
@ -233,6 +249,9 @@ parse_json_response() {
|
|||
--argjson loop_number "$loop_number" \
|
||||
--arg session_id "$session_id" \
|
||||
--argjson confidence "$confidence" \
|
||||
--argjson has_permission_denials "$has_permission_denials" \
|
||||
--argjson permission_denial_count "$permission_denial_count" \
|
||||
--argjson denied_commands "$denied_commands_json" \
|
||||
'{
|
||||
status: $status,
|
||||
exit_signal: $exit_signal,
|
||||
|
|
@ -245,6 +264,9 @@ parse_json_response() {
|
|||
loop_number: $loop_number,
|
||||
session_id: $session_id,
|
||||
confidence: $confidence,
|
||||
has_permission_denials: $has_permission_denials,
|
||||
permission_denial_count: $permission_denial_count,
|
||||
denied_commands: $denied_commands,
|
||||
metadata: {
|
||||
loop_number: $loop_number,
|
||||
session_id: $session_id
|
||||
|
|
@ -300,6 +322,11 @@ analyze_response() {
|
|||
local json_confidence=$(jq -r '.confidence' $RALPH_DIR/.json_parse_result 2>/dev/null || echo "0")
|
||||
local session_id=$(jq -r '.session_id' $RALPH_DIR/.json_parse_result 2>/dev/null || echo "")
|
||||
|
||||
# Extract permission denial fields (Issue #101)
|
||||
local has_permission_denials=$(jq -r '.has_permission_denials' $RALPH_DIR/.json_parse_result 2>/dev/null || echo "false")
|
||||
local permission_denial_count=$(jq -r '.permission_denial_count' $RALPH_DIR/.json_parse_result 2>/dev/null || echo "0")
|
||||
local denied_commands_json=$(jq -r '.denied_commands' $RALPH_DIR/.json_parse_result 2>/dev/null || echo "[]")
|
||||
|
||||
# Persist session ID if present (for session continuity across loop iterations)
|
||||
if [[ -n "$session_id" && "$session_id" != "null" ]]; then
|
||||
store_session_id "$session_id"
|
||||
|
|
@ -337,6 +364,9 @@ analyze_response() {
|
|||
--argjson exit_signal "$exit_signal" \
|
||||
--arg work_summary "$work_summary" \
|
||||
--argjson output_length "$output_length" \
|
||||
--argjson has_permission_denials "$has_permission_denials" \
|
||||
--argjson permission_denial_count "$permission_denial_count" \
|
||||
--argjson denied_commands "$denied_commands_json" \
|
||||
'{
|
||||
loop_number: $loop_number,
|
||||
timestamp: $timestamp,
|
||||
|
|
@ -351,7 +381,10 @@ analyze_response() {
|
|||
confidence_score: $confidence_score,
|
||||
exit_signal: $exit_signal,
|
||||
work_summary: $work_summary,
|
||||
output_length: $output_length
|
||||
output_length: $output_length,
|
||||
has_permission_denials: $has_permission_denials,
|
||||
permission_denial_count: $permission_denial_count,
|
||||
denied_commands: $denied_commands
|
||||
}
|
||||
}' > "$analysis_result_file"
|
||||
rm -f "$RALPH_DIR/.json_parse_result"
|
||||
|
|
@ -489,6 +522,7 @@ analyze_response() {
|
|||
fi
|
||||
|
||||
# Write analysis results to file (text parsing path) using jq for safe construction
|
||||
# Note: Permission denial fields default to false/0 since text output doesn't include this data
|
||||
jq -n \
|
||||
--argjson loop_number "$loop_number" \
|
||||
--arg timestamp "$(get_iso_timestamp)" \
|
||||
|
|
@ -517,7 +551,10 @@ analyze_response() {
|
|||
confidence_score: $confidence_score,
|
||||
exit_signal: $exit_signal,
|
||||
work_summary: $work_summary,
|
||||
output_length: $output_length
|
||||
output_length: $output_length,
|
||||
has_permission_denials: false,
|
||||
permission_denial_count: 0,
|
||||
denied_commands: []
|
||||
}
|
||||
}' > "$analysis_result_file"
|
||||
|
||||
|
|
|
|||
|
|
@ -394,9 +394,24 @@ should_exit_gracefully() {
|
|||
recent_completion_indicators=$(echo "$signals" | jq '.completion_indicators | length' 2>/dev/null || echo "0")
|
||||
|
||||
log_status "INFO" "DEBUG: Exit counts - test_loops:$recent_test_loops, done_signals:$recent_done_signals, completion:$recent_completion_indicators" >&2
|
||||
|
||||
|
||||
# Check for exit conditions
|
||||
|
||||
|
||||
# 0. Permission denials (highest priority - Issue #101)
|
||||
# When Claude Code is denied permission to run commands, halt immediately
|
||||
# to allow user to update .ralphrc ALLOWED_TOOLS configuration
|
||||
if [[ -f "$RESPONSE_ANALYSIS_FILE" ]]; then
|
||||
local has_permission_denials=$(jq -r '.analysis.has_permission_denials // false' "$RESPONSE_ANALYSIS_FILE" 2>/dev/null || echo "false")
|
||||
if [[ "$has_permission_denials" == "true" ]]; then
|
||||
local denied_count=$(jq -r '.analysis.permission_denial_count // 0' "$RESPONSE_ANALYSIS_FILE" 2>/dev/null || echo "0")
|
||||
local denied_cmds=$(jq -r '.analysis.denied_commands | join(", ")' "$RESPONSE_ANALYSIS_FILE" 2>/dev/null || echo "unknown")
|
||||
log_status "WARN" "🚫 Permission denied for $denied_count command(s): $denied_cmds"
|
||||
log_status "WARN" "Update ALLOWED_TOOLS in .ralphrc to include the required tools"
|
||||
echo "permission_denied"
|
||||
return 0
|
||||
fi
|
||||
fi
|
||||
|
||||
# 1. Too many consecutive test-only loops
|
||||
if [[ $recent_test_loops -ge $MAX_CONSECUTIVE_TEST_LOOPS ]]; then
|
||||
log_status "WARN" "Exit condition: Too many test-focused loops ($recent_test_loops >= $MAX_CONSECUTIVE_TEST_LOOPS)"
|
||||
|
|
@ -1238,6 +1253,45 @@ main() {
|
|||
# Check for graceful exit conditions
|
||||
local exit_reason=$(should_exit_gracefully)
|
||||
if [[ "$exit_reason" != "" ]]; then
|
||||
# Handle permission_denied specially (Issue #101)
|
||||
if [[ "$exit_reason" == "permission_denied" ]]; then
|
||||
log_status "ERROR" "🚫 Permission denied - halting loop"
|
||||
reset_session "permission_denied"
|
||||
update_status "$loop_count" "$(cat "$CALL_COUNT_FILE")" "permission_denied" "halted" "permission_denied"
|
||||
|
||||
# Display helpful guidance for resolving permission issues
|
||||
echo ""
|
||||
echo -e "${RED}╔════════════════════════════════════════════════════════════╗${NC}"
|
||||
echo -e "${RED}║ PERMISSION DENIED - Loop Halted ║${NC}"
|
||||
echo -e "${RED}╚════════════════════════════════════════════════════════════╝${NC}"
|
||||
echo ""
|
||||
echo -e "${YELLOW}Claude Code was denied permission to execute commands.${NC}"
|
||||
echo ""
|
||||
echo -e "${YELLOW}To fix this:${NC}"
|
||||
echo " 1. Edit .ralphrc and update ALLOWED_TOOLS to include the required tools"
|
||||
echo " 2. Common patterns:"
|
||||
echo " - Bash(npm *) - All npm commands"
|
||||
echo " - Bash(npm install) - Only npm install"
|
||||
echo " - Bash(pnpm *) - All pnpm commands"
|
||||
echo " - Bash(yarn *) - All yarn commands"
|
||||
echo ""
|
||||
echo -e "${YELLOW}After updating .ralphrc:${NC}"
|
||||
echo " ralph --reset-session # Clear stale session state"
|
||||
echo " ralph --monitor # Restart the loop"
|
||||
echo ""
|
||||
|
||||
# Show current ALLOWED_TOOLS if .ralphrc exists
|
||||
if [[ -f ".ralphrc" ]]; then
|
||||
local current_tools=$(grep "^ALLOWED_TOOLS=" ".ralphrc" 2>/dev/null | cut -d= -f2- | tr -d '"')
|
||||
if [[ -n "$current_tools" ]]; then
|
||||
echo -e "${BLUE}Current ALLOWED_TOOLS:${NC} $current_tools"
|
||||
echo ""
|
||||
fi
|
||||
fi
|
||||
|
||||
break
|
||||
fi
|
||||
|
||||
log_status "SUCCESS" "🏁 Graceful exit triggered: $exit_reason"
|
||||
reset_session "project_complete"
|
||||
update_status "$loop_count" "$(cat "$CALL_COUNT_FILE")" "graceful_exit" "completed" "$exit_reason"
|
||||
|
|
|
|||
|
|
@ -699,3 +699,182 @@ EOF
|
|||
local indicator_count=$(jq '.completion_indicators | length' "$EXIT_SIGNALS_FILE")
|
||||
assert_equal "$indicator_count" "0"
|
||||
}
|
||||
|
||||
# =============================================================================
|
||||
# PERMISSION DENIAL EXIT TESTS (Issue #101)
|
||||
# =============================================================================
|
||||
# When Claude Code is denied permission to run commands, Ralph should detect
|
||||
# this from the permission_denials field and halt the loop to allow user intervention.
|
||||
|
||||
# Helper function with permission denial support
|
||||
should_exit_gracefully_with_denials() {
|
||||
if [[ ! -f "$EXIT_SIGNALS_FILE" ]]; then
|
||||
echo ""
|
||||
return 1
|
||||
fi
|
||||
|
||||
local signals=$(cat "$EXIT_SIGNALS_FILE")
|
||||
|
||||
local recent_test_loops
|
||||
local recent_done_signals
|
||||
local recent_completion_indicators
|
||||
|
||||
recent_test_loops=$(echo "$signals" | jq '.test_only_loops | length' 2>/dev/null || echo "0")
|
||||
recent_done_signals=$(echo "$signals" | jq '.done_signals | length' 2>/dev/null || echo "0")
|
||||
recent_completion_indicators=$(echo "$signals" | jq '.completion_indicators | length' 2>/dev/null || echo "0")
|
||||
|
||||
# Check for permission denials first (highest priority - Issue #101)
|
||||
if [[ -f "$RESPONSE_ANALYSIS_FILE" ]]; then
|
||||
local has_permission_denials=$(jq -r '.analysis.has_permission_denials // false' "$RESPONSE_ANALYSIS_FILE" 2>/dev/null || echo "false")
|
||||
if [[ "$has_permission_denials" == "true" ]]; then
|
||||
echo "permission_denied"
|
||||
return 0
|
||||
fi
|
||||
fi
|
||||
|
||||
# 1. Too many consecutive test-only loops
|
||||
if [[ $recent_test_loops -ge $MAX_CONSECUTIVE_TEST_LOOPS ]]; then
|
||||
echo "test_saturation"
|
||||
return 0
|
||||
fi
|
||||
|
||||
# 2. Multiple "done" signals
|
||||
if [[ $recent_done_signals -ge $MAX_CONSECUTIVE_DONE_SIGNALS ]]; then
|
||||
echo "completion_signals"
|
||||
return 0
|
||||
fi
|
||||
|
||||
# 3. Strong completion indicators (only if Claude's EXIT_SIGNAL is true)
|
||||
local claude_exit_signal="false"
|
||||
if [[ -f "$RESPONSE_ANALYSIS_FILE" ]]; then
|
||||
claude_exit_signal=$(jq -r '.analysis.exit_signal // false' "$RESPONSE_ANALYSIS_FILE" 2>/dev/null || echo "false")
|
||||
fi
|
||||
|
||||
if [[ $recent_completion_indicators -ge 2 ]] && [[ "$claude_exit_signal" == "true" ]]; then
|
||||
echo "project_complete"
|
||||
return 0
|
||||
fi
|
||||
|
||||
echo ""
|
||||
return 1
|
||||
}
|
||||
|
||||
# Test 36: Exit on permission denial detected
|
||||
@test "should_exit_gracefully exits on permission_denied" {
|
||||
echo '{"test_only_loops": [], "done_signals": [], "completion_indicators": []}' > "$EXIT_SIGNALS_FILE"
|
||||
|
||||
# Create response analysis with permission denials
|
||||
cat > "$RESPONSE_ANALYSIS_FILE" << 'EOF'
|
||||
{
|
||||
"loop_number": 1,
|
||||
"output_format": "json",
|
||||
"analysis": {
|
||||
"has_completion_signal": false,
|
||||
"is_test_only": false,
|
||||
"is_stuck": false,
|
||||
"has_progress": false,
|
||||
"files_modified": 0,
|
||||
"confidence_score": 70,
|
||||
"exit_signal": false,
|
||||
"work_summary": "Tried to run npm install but permission denied",
|
||||
"has_permission_denials": true,
|
||||
"permission_denial_count": 1,
|
||||
"denied_commands": ["npm install"]
|
||||
}
|
||||
}
|
||||
EOF
|
||||
|
||||
result=$(should_exit_gracefully_with_denials)
|
||||
assert_equal "$result" "permission_denied"
|
||||
}
|
||||
|
||||
# Test 37: No exit when no permission denials
|
||||
@test "should_exit_gracefully continues when no permission denials" {
|
||||
echo '{"test_only_loops": [], "done_signals": [], "completion_indicators": []}' > "$EXIT_SIGNALS_FILE"
|
||||
|
||||
# Create response analysis without permission denials
|
||||
cat > "$RESPONSE_ANALYSIS_FILE" << 'EOF'
|
||||
{
|
||||
"loop_number": 1,
|
||||
"output_format": "json",
|
||||
"analysis": {
|
||||
"has_completion_signal": false,
|
||||
"is_test_only": false,
|
||||
"is_stuck": false,
|
||||
"has_progress": true,
|
||||
"files_modified": 3,
|
||||
"confidence_score": 70,
|
||||
"exit_signal": false,
|
||||
"work_summary": "Implementing feature",
|
||||
"has_permission_denials": false,
|
||||
"permission_denial_count": 0,
|
||||
"denied_commands": []
|
||||
}
|
||||
}
|
||||
EOF
|
||||
|
||||
result=$(should_exit_gracefully_with_denials || true)
|
||||
assert_equal "$result" ""
|
||||
}
|
||||
|
||||
# Test 38: Permission denial takes priority over other signals
|
||||
@test "permission_denied takes priority over test_saturation" {
|
||||
# Set up test saturation condition
|
||||
echo '{"test_only_loops": [1,2,3], "done_signals": [], "completion_indicators": []}' > "$EXIT_SIGNALS_FILE"
|
||||
|
||||
# Create response analysis with permission denials
|
||||
cat > "$RESPONSE_ANALYSIS_FILE" << 'EOF'
|
||||
{
|
||||
"loop_number": 3,
|
||||
"analysis": {
|
||||
"is_test_only": true,
|
||||
"has_permission_denials": true,
|
||||
"permission_denial_count": 1,
|
||||
"denied_commands": ["npm install"]
|
||||
}
|
||||
}
|
||||
EOF
|
||||
|
||||
# Permission denied should take priority
|
||||
result=$(should_exit_gracefully_with_denials)
|
||||
assert_equal "$result" "permission_denied"
|
||||
}
|
||||
|
||||
# Test 39: Multiple permission denials detected
|
||||
@test "should_exit_gracefully detects multiple permission denials" {
|
||||
echo '{"test_only_loops": [], "done_signals": [], "completion_indicators": []}' > "$EXIT_SIGNALS_FILE"
|
||||
|
||||
cat > "$RESPONSE_ANALYSIS_FILE" << 'EOF'
|
||||
{
|
||||
"loop_number": 1,
|
||||
"analysis": {
|
||||
"has_permission_denials": true,
|
||||
"permission_denial_count": 3,
|
||||
"denied_commands": ["npm install", "pnpm install", "yarn add lodash"]
|
||||
}
|
||||
}
|
||||
EOF
|
||||
|
||||
result=$(should_exit_gracefully_with_denials)
|
||||
assert_equal "$result" "permission_denied"
|
||||
}
|
||||
|
||||
# Test 40: Missing has_permission_denials field defaults to false (backward compat)
|
||||
@test "should_exit_gracefully handles missing permission denial fields" {
|
||||
echo '{"test_only_loops": [], "done_signals": [], "completion_indicators": []}' > "$EXIT_SIGNALS_FILE"
|
||||
|
||||
# Old format response analysis without permission denial fields
|
||||
cat > "$RESPONSE_ANALYSIS_FILE" << 'EOF'
|
||||
{
|
||||
"loop_number": 1,
|
||||
"analysis": {
|
||||
"has_completion_signal": false,
|
||||
"is_test_only": false,
|
||||
"exit_signal": false
|
||||
}
|
||||
}
|
||||
EOF
|
||||
|
||||
result=$(should_exit_gracefully_with_denials || true)
|
||||
assert_equal "$result" ""
|
||||
}
|
||||
|
|
|
|||
|
|
@ -919,3 +919,193 @@ EOF
|
|||
local session_id=$(jq -r '.session_id' "$result_file")
|
||||
assert_equal "$session_id" "session-in-result-only"
|
||||
}
|
||||
|
||||
# =============================================================================
|
||||
# PERMISSION DENIAL DETECTION TESTS (Issue #101)
|
||||
# =============================================================================
|
||||
# Tests for detecting permission_denials from Claude Code JSON output.
|
||||
# When Claude Code is denied permission to execute commands (e.g., npm install),
|
||||
# the JSON output contains a permission_denials array that Ralph should detect.
|
||||
|
||||
@test "parse_json_response detects permission_denials array" {
|
||||
local output_file="$LOG_DIR/test_output.log"
|
||||
|
||||
# Create JSON output with permission denials (as Claude Code outputs)
|
||||
cat > "$output_file" << 'EOF'
|
||||
{
|
||||
"result": "I tried to run npm install but was denied permission.",
|
||||
"sessionId": "session-denied-123",
|
||||
"is_error": false,
|
||||
"permission_denials": [
|
||||
{"tool": "Bash", "command": "npm install", "reason": "Tool not in allowed list"}
|
||||
]
|
||||
}
|
||||
EOF
|
||||
|
||||
run parse_json_response "$output_file"
|
||||
assert_equal "$status" "0"
|
||||
|
||||
local result_file="$RALPH_DIR/.json_parse_result"
|
||||
[[ -f "$result_file" ]]
|
||||
|
||||
# Should extract has_permission_denials flag
|
||||
local has_denials=$(jq -r '.has_permission_denials' "$result_file")
|
||||
assert_equal "$has_denials" "true"
|
||||
}
|
||||
|
||||
@test "parse_json_response extracts permission_denial_count" {
|
||||
local output_file="$LOG_DIR/test_output.log"
|
||||
|
||||
cat > "$output_file" << 'EOF'
|
||||
{
|
||||
"result": "Multiple commands were denied.",
|
||||
"sessionId": "session-multi-deny",
|
||||
"permission_denials": [
|
||||
{"tool": "Bash", "command": "npm install", "reason": "Not allowed"},
|
||||
{"tool": "Bash", "command": "pnpm install", "reason": "Not allowed"},
|
||||
{"tool": "Bash", "command": "yarn add lodash", "reason": "Not allowed"}
|
||||
]
|
||||
}
|
||||
EOF
|
||||
|
||||
run parse_json_response "$output_file"
|
||||
assert_equal "$status" "0"
|
||||
|
||||
local result_file="$RALPH_DIR/.json_parse_result"
|
||||
[[ -f "$result_file" ]]
|
||||
|
||||
# Should count denials correctly
|
||||
local denial_count=$(jq -r '.permission_denial_count' "$result_file")
|
||||
assert_equal "$denial_count" "3"
|
||||
}
|
||||
|
||||
@test "parse_json_response extracts denied_commands list" {
|
||||
local output_file="$LOG_DIR/test_output.log"
|
||||
|
||||
cat > "$output_file" << 'EOF'
|
||||
{
|
||||
"result": "Permission denied for npm install",
|
||||
"sessionId": "session-extract-cmds",
|
||||
"permission_denials": [
|
||||
{"tool": "Bash", "command": "npm install express", "reason": "Not allowed"}
|
||||
]
|
||||
}
|
||||
EOF
|
||||
|
||||
run parse_json_response "$output_file"
|
||||
assert_equal "$status" "0"
|
||||
|
||||
local result_file="$RALPH_DIR/.json_parse_result"
|
||||
[[ -f "$result_file" ]]
|
||||
|
||||
# Should extract the denied commands
|
||||
local denied_cmds=$(jq -r '.denied_commands[0]' "$result_file")
|
||||
[[ "$denied_cmds" == *"npm install"* ]]
|
||||
}
|
||||
|
||||
@test "parse_json_response handles empty permission_denials array" {
|
||||
local output_file="$LOG_DIR/test_output.log"
|
||||
|
||||
cat > "$output_file" << 'EOF'
|
||||
{
|
||||
"result": "All commands executed successfully.",
|
||||
"sessionId": "session-no-denials",
|
||||
"permission_denials": []
|
||||
}
|
||||
EOF
|
||||
|
||||
run parse_json_response "$output_file"
|
||||
assert_equal "$status" "0"
|
||||
|
||||
local result_file="$RALPH_DIR/.json_parse_result"
|
||||
[[ -f "$result_file" ]]
|
||||
|
||||
# Should set has_permission_denials to false
|
||||
local has_denials=$(jq -r '.has_permission_denials' "$result_file")
|
||||
assert_equal "$has_denials" "false"
|
||||
|
||||
local denial_count=$(jq -r '.permission_denial_count' "$result_file")
|
||||
assert_equal "$denial_count" "0"
|
||||
}
|
||||
|
||||
@test "parse_json_response handles missing permission_denials field (backward compat)" {
|
||||
local output_file="$LOG_DIR/test_output.log"
|
||||
|
||||
# Old format without permission_denials field
|
||||
cat > "$output_file" << 'EOF'
|
||||
{
|
||||
"status": "COMPLETE",
|
||||
"exit_signal": true,
|
||||
"work_type": "IMPLEMENTATION",
|
||||
"files_modified": 5
|
||||
}
|
||||
EOF
|
||||
|
||||
run parse_json_response "$output_file"
|
||||
assert_equal "$status" "0"
|
||||
|
||||
local result_file="$RALPH_DIR/.json_parse_result"
|
||||
[[ -f "$result_file" ]]
|
||||
|
||||
# Should default to no denials
|
||||
local has_denials=$(jq -r '.has_permission_denials' "$result_file")
|
||||
assert_equal "$has_denials" "false"
|
||||
|
||||
local denial_count=$(jq -r '.permission_denial_count' "$result_file")
|
||||
assert_equal "$denial_count" "0"
|
||||
}
|
||||
|
||||
@test "analyze_response includes permission denial info in analysis result" {
|
||||
local output_file="$LOG_DIR/test_output.log"
|
||||
|
||||
cat > "$output_file" << 'EOF'
|
||||
{
|
||||
"result": "Tried npm install but permission was denied.",
|
||||
"sessionId": "session-analyze-denial",
|
||||
"permission_denials": [
|
||||
{"tool": "Bash", "command": "npm install", "reason": "Tool not allowed"}
|
||||
]
|
||||
}
|
||||
EOF
|
||||
|
||||
analyze_response "$output_file" 1
|
||||
|
||||
assert_file_exists "$RALPH_DIR/.response_analysis"
|
||||
|
||||
# Should include permission denial in analysis
|
||||
local has_denials=$(jq -r '.analysis.has_permission_denials' "$RALPH_DIR/.response_analysis")
|
||||
assert_equal "$has_denials" "true"
|
||||
|
||||
local denial_count=$(jq -r '.analysis.permission_denial_count' "$RALPH_DIR/.response_analysis")
|
||||
assert_equal "$denial_count" "1"
|
||||
}
|
||||
|
||||
@test "parse_json_response handles Claude CLI array format with permission denials" {
|
||||
local output_file="$LOG_DIR/test_output.log"
|
||||
|
||||
# Claude CLI array format with permission denials in result
|
||||
cat > "$output_file" << 'EOF'
|
||||
[
|
||||
{"type": "system", "subtype": "init", "session_id": "session-array-deny"},
|
||||
{"type": "assistant", "message": {"content": [{"type": "text", "text": "Trying to install..."}]}},
|
||||
{
|
||||
"type": "result",
|
||||
"subtype": "success",
|
||||
"result": "Could not run npm install - permission denied",
|
||||
"session_id": "session-array-deny",
|
||||
"permission_denials": [
|
||||
{"tool": "Bash", "command": "npm install", "reason": "Not in allowed tools"}
|
||||
]
|
||||
}
|
||||
]
|
||||
EOF
|
||||
|
||||
run parse_json_response "$output_file"
|
||||
assert_equal "$status" "0"
|
||||
|
||||
local result_file="$RALPH_DIR/.json_parse_result"
|
||||
[[ -f "$result_file" ]]
|
||||
|
||||
local has_denials=$(jq -r '.has_permission_denials' "$result_file")
|
||||
assert_equal "$has_denials" "true"
|
||||
}
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue