Merge pull request #59 from frankbria/feature/installation-tests

test(install): add comprehensive installation tests
This commit is contained in:
Frank Bria 2026-01-09 14:31:45 -07:00 committed by GitHub
commit 4a83b879c2
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
4 changed files with 571 additions and 19 deletions

View file

@ -6,7 +6,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
This is the Ralph for Claude Code repository - an autonomous AI development loop system that enables continuous development cycles with intelligent exit detection and rate limiting.
**Version**: v0.9.2 | **Tests**: 151 passing (100% pass rate) | **CI/CD**: GitHub Actions
**Version**: v0.9.3 | **Tests**: 165 passing (100% pass rate) | **CI/CD**: GitHub Actions
## Core Architecture
@ -95,7 +95,7 @@ tmux attach -t <session-name>
### Running Tests
```bash
# Run all tests (151 tests)
# Run all tests (165 tests)
npm test
# Run specific test suites
@ -272,7 +272,7 @@ Ralph uses advanced error detection with two-stage filtering to eliminate false
## Test Suite
### Test Files (151 tests total)
### Test Files (165 tests total)
| File | Tests | Description |
|------|-------|-------------|
@ -283,6 +283,7 @@ Ralph uses advanced error detection with two-stage filtering to eliminate false
| `test_rate_limiting.bats` | 15 | Rate limiting behavior |
| `test_loop_execution.bats` | 20 | Integration tests |
| `test_edge_cases.bats` | 20 | Edge case handling |
| `test_installation.bats` | 14 | Global installation/uninstall workflows |
### Running Tests
```bash
@ -298,6 +299,17 @@ bats tests/unit/test_cli_parsing.bats
## Recent Improvements
### Installation Tests (v0.9.3)
- Added 14 comprehensive tests for `install.sh` global installation script
- Tests cover: directory creation, command installation, template copying, lib copying
- Dependency detection tests (jq, git, node) with mocked failures
- PATH detection and warning system tests
- Uninstallation cleanup verification
- Idempotency testing (run twice without errors)
- End-to-end installation workflow validation
- All tests use isolated temp directories (no system modifications)
- Test count: 165 (up from 151)
### Prompt File Fix (v0.9.2)
- Fixed critical bug: replaced non-existent `--prompt-file` CLI flag with `-p` flag
- Modern CLI mode now correctly passes prompt content via `CLAUDE_CMD_ARGS+=("-p" "$prompt_content")`

View file

@ -1,10 +1,11 @@
# Ralph for Claude Code
![Version](https://img.shields.io/badge/version-0.9.1-blue)
![Version](https://img.shields.io/badge/version-0.9.3-blue)
![Status](https://img.shields.io/badge/status-active%20development-yellow)
![Tests](https://img.shields.io/badge/tests-145%20passing-green)
![Tests](https://img.shields.io/badge/tests-165%20passing-green)
![Coverage](https://img.shields.io/badge/coverage-informational-lightgrey)
[![Mentioned in Awesome Claude Code](https://awesome.re/mentioned-badge.svg)](https://github.com/hesreallyhim/awesome-claude-code)
[![Follow on X](https://img.shields.io/twitter/follow/FrankBria18044?style=social)](https://x.com/FrankBria18044)
> **Autonomous AI development loop with intelligent exit detection and rate limiting**
@ -14,9 +15,9 @@ Ralph is an implementation of the Geoffrey Huntley's technique for Claude Code t
## Project Status
**Version**: v0.9.1 - Active Development
**Version**: v0.9.3 - Active Development
**Core Features**: Working and tested
**Test Coverage**: 145 tests, 100% pass rate
**Test Coverage**: 165 tests, 100% pass rate
### What's Working Now
- Autonomous development loops with intelligent exit detection
@ -31,7 +32,7 @@ Ralph is an implementation of the Geoffrey Huntley's technique for Claude Code t
- tmux integration for live monitoring
- PRD import functionality
- **CI/CD pipeline with GitHub Actions**
- 151 passing tests across 7 test files
- 165 passing tests across 8 test files
### Recent Improvements
@ -360,8 +361,8 @@ If you want to run the test suite:
# Install BATS testing framework
npm install -g bats bats-support bats-assert
# Run all tests (145 tests)
bats tests/
# Run all tests (165 tests)
npm test
# Run specific test suites
bats tests/unit/test_rate_limiting.bats
@ -377,10 +378,10 @@ bats tests/integration/test_loop_execution.bats
```
Current test status:
- **145 tests** across 7 test files
- **100% pass rate** (145/145 passing)
- **165 tests** across 8 test files
- **100% pass rate** (165/165 passing)
- Comprehensive unit and integration tests
- Specialized tests for JSON parsing, CLI flags, and circuit breaker functionality
- Specialized tests for JSON parsing, CLI flags, circuit breaker, and installation workflows
> **Note on Coverage**: Bash code coverage measurement with kcov has fundamental limitations when tracing subprocess executions. Test pass rate (100%) is the quality gate. See [bats-core#15](https://github.com/bats-core/bats-core/issues/15) for details.
@ -597,23 +598,24 @@ tmux attach -t <name> # Reattach to detached session
Ralph is under active development with a clear path to v1.0.0. See [IMPLEMENTATION_PLAN.md](IMPLEMENTATION_PLAN.md) for the complete roadmap.
### Current Status: v0.9.1
### Current Status: v0.9.3
**What's Delivered:**
- Core loop functionality with intelligent exit detection
- Rate limiting (100 calls/hour) and circuit breaker pattern
- Response analyzer with semantic understanding
- 145 comprehensive tests (100% pass rate)
- 165 comprehensive tests (100% pass rate)
- tmux integration and live monitoring
- PRD import functionality
- Installation system and project templates
- Modern CLI commands with JSON output support
- CI/CD pipeline with GitHub Actions
- Comprehensive installation test suite
**Test Coverage Breakdown:**
- Unit Tests: 105 (CLI parsing, JSON, exit detection, rate limiting)
- Integration Tests: 40 (loop execution, edge cases)
- Test Files: 7
- Unit Tests: 111 (CLI parsing, JSON, exit detection, rate limiting)
- Integration Tests: 54 (loop execution, edge cases, installation)
- Test Files: 8
### Path to v1.0.0 (~4 weeks)

View file

@ -9,7 +9,7 @@
"test": "tests"
},
"scripts": {
"test": "bats tests/",
"test": "bats tests/unit/ tests/integration/",
"test:unit": "bats tests/unit/",
"test:integration": "bats tests/integration/",
"test:e2e": "bats tests/e2e/"

View file

@ -0,0 +1,538 @@
#!/usr/bin/env bats
# Integration tests for Ralph install.sh - Global Installation Script
load '../helpers/test_helper'
load '../helpers/mocks'
load '../helpers/fixtures'
# Store original values
ORIGINAL_HOME="$HOME"
ORIGINAL_PATH="$PATH"
PROJECT_ROOT=""
setup() {
# Save project root for sourcing install.sh
PROJECT_ROOT="${BATS_TEST_DIRNAME}/../.."
# Create unique temp directories for isolated testing
export TEST_HOME="$(mktemp -d)"
export TEST_INSTALL_DIR="$TEST_HOME/.local/bin"
export TEST_RALPH_HOME="$TEST_HOME/.ralph"
# Override HOME to isolate tests
export HOME="$TEST_HOME"
# Create mock source directories with required files
export MOCK_SOURCE_DIR="$(mktemp -d)"
mkdir -p "$MOCK_SOURCE_DIR/templates/specs"
mkdir -p "$MOCK_SOURCE_DIR/lib"
# Create mock template files
echo "# Mock PROMPT.md" > "$MOCK_SOURCE_DIR/templates/PROMPT.md"
echo "# Mock fix_plan.md" > "$MOCK_SOURCE_DIR/templates/fix_plan.md"
echo "# Mock AGENT.md" > "$MOCK_SOURCE_DIR/templates/AGENT.md"
# Create mock lib files
cat > "$MOCK_SOURCE_DIR/lib/circuit_breaker.sh" << 'EOF'
#!/bin/bash
# Mock circuit_breaker.sh
init_circuit_breaker() { :; }
EOF
cat > "$MOCK_SOURCE_DIR/lib/response_analyzer.sh" << 'EOF'
#!/bin/bash
# Mock response_analyzer.sh
analyze_response() { :; }
EOF
cat > "$MOCK_SOURCE_DIR/lib/date_utils.sh" << 'EOF'
#!/bin/bash
# Mock date_utils.sh
get_iso_timestamp() { date -Iseconds; }
EOF
# Create mock main scripts
cat > "$MOCK_SOURCE_DIR/ralph_loop.sh" << 'EOF'
#!/bin/bash
# Mock ralph_loop.sh
echo "Ralph loop running"
EOF
cat > "$MOCK_SOURCE_DIR/ralph_monitor.sh" << 'EOF'
#!/bin/bash
# Mock ralph_monitor.sh
echo "Ralph monitor running"
EOF
cat > "$MOCK_SOURCE_DIR/ralph_import.sh" << 'EOF'
#!/bin/bash
# Mock ralph_import.sh
echo "Ralph import running"
EOF
cat > "$MOCK_SOURCE_DIR/setup.sh" << 'EOF'
#!/bin/bash
# Mock setup.sh
echo "Setup running"
EOF
chmod +x "$MOCK_SOURCE_DIR"/*.sh
chmod +x "$MOCK_SOURCE_DIR/lib"/*.sh
}
teardown() {
# Restore original environment
export HOME="$ORIGINAL_HOME"
export PATH="$ORIGINAL_PATH"
# Clean up test directories
if [[ -n "$TEST_HOME" && -d "$TEST_HOME" ]]; then
rm -rf "$TEST_HOME"
fi
if [[ -n "$MOCK_SOURCE_DIR" && -d "$MOCK_SOURCE_DIR" ]]; then
rm -rf "$MOCK_SOURCE_DIR"
fi
}
# Helper: Run install.sh in isolated environment
run_install() {
local action="${1:-install}"
# Set up environment for isolated install
export SCRIPT_DIR="$MOCK_SOURCE_DIR"
# Create a modified install.sh that uses our mock paths
local temp_install="$(mktemp)"
sed -e "s|INSTALL_DIR=\"\$HOME/.local/bin\"|INSTALL_DIR=\"$TEST_INSTALL_DIR\"|g" \
-e "s|RALPH_HOME=\"\$HOME/.ralph\"|RALPH_HOME=\"$TEST_RALPH_HOME\"|g" \
-e "s|SCRIPT_DIR=\"\$(cd \"\$(dirname \"\${BASH_SOURCE\[0\]}\")\" && pwd)\"|SCRIPT_DIR=\"$MOCK_SOURCE_DIR\"|g" \
"$PROJECT_ROOT/install.sh" > "$temp_install"
chmod +x "$temp_install"
# Run with specified action
if [[ "$action" == "install" ]]; then
bash "$temp_install" install 2>&1
else
bash "$temp_install" "$action" 2>&1
fi
local exit_code=$?
rm -f "$temp_install"
return $exit_code
}
# =============================================================================
# Test 1-2: Directory Creation Tests
# =============================================================================
@test "install.sh creates ~/.ralph directory" {
run run_install
# Check main ralph directory was created
assert_dir_exists "$TEST_RALPH_HOME"
# Check subdirectories
assert_dir_exists "$TEST_RALPH_HOME/templates"
assert_dir_exists "$TEST_RALPH_HOME/lib"
}
@test "install.sh creates ~/.local/bin directory" {
run run_install
# Check bin directory was created
assert_dir_exists "$TEST_INSTALL_DIR"
# Verify directory has correct permissions (should be accessible)
[[ -x "$TEST_INSTALL_DIR" ]]
}
# =============================================================================
# Test 3-4: Command Installation Tests
# =============================================================================
@test "install.sh creates ~/.local/bin commands" {
run run_install
# Check all four wrapper commands exist
assert_file_exists "$TEST_INSTALL_DIR/ralph"
assert_file_exists "$TEST_INSTALL_DIR/ralph-monitor"
assert_file_exists "$TEST_INSTALL_DIR/ralph-setup"
assert_file_exists "$TEST_INSTALL_DIR/ralph-import"
# Verify each command contains proper shebang
grep -q "#!/bin/bash" "$TEST_INSTALL_DIR/ralph"
grep -q "#!/bin/bash" "$TEST_INSTALL_DIR/ralph-monitor"
grep -q "#!/bin/bash" "$TEST_INSTALL_DIR/ralph-setup"
grep -q "#!/bin/bash" "$TEST_INSTALL_DIR/ralph-import"
}
@test "install.sh sets executable permissions" {
run run_install
# Verify executable bit on all commands
[[ -x "$TEST_INSTALL_DIR/ralph" ]]
[[ -x "$TEST_INSTALL_DIR/ralph-monitor" ]]
[[ -x "$TEST_INSTALL_DIR/ralph-setup" ]]
[[ -x "$TEST_INSTALL_DIR/ralph-import" ]]
# Verify executable bit on main scripts
[[ -x "$TEST_RALPH_HOME/ralph_loop.sh" ]]
[[ -x "$TEST_RALPH_HOME/ralph_monitor.sh" ]]
[[ -x "$TEST_RALPH_HOME/setup.sh" ]]
[[ -x "$TEST_RALPH_HOME/ralph_import.sh" ]]
# Verify lib scripts are executable
[[ -x "$TEST_RALPH_HOME/lib/circuit_breaker.sh" ]]
[[ -x "$TEST_RALPH_HOME/lib/response_analyzer.sh" ]]
[[ -x "$TEST_RALPH_HOME/lib/date_utils.sh" ]]
}
# =============================================================================
# Test 5-6: Template and Library Copying Tests
# =============================================================================
@test "install.sh copies templates correctly" {
run run_install
# Check template files were copied
assert_file_exists "$TEST_RALPH_HOME/templates/PROMPT.md"
assert_file_exists "$TEST_RALPH_HOME/templates/fix_plan.md"
assert_file_exists "$TEST_RALPH_HOME/templates/AGENT.md"
# Verify content matches source
diff -q "$MOCK_SOURCE_DIR/templates/PROMPT.md" "$TEST_RALPH_HOME/templates/PROMPT.md"
diff -q "$MOCK_SOURCE_DIR/templates/fix_plan.md" "$TEST_RALPH_HOME/templates/fix_plan.md"
diff -q "$MOCK_SOURCE_DIR/templates/AGENT.md" "$TEST_RALPH_HOME/templates/AGENT.md"
}
@test "install.sh copies lib/ directory" {
run run_install
# Check lib files were copied
assert_file_exists "$TEST_RALPH_HOME/lib/circuit_breaker.sh"
assert_file_exists "$TEST_RALPH_HOME/lib/response_analyzer.sh"
assert_file_exists "$TEST_RALPH_HOME/lib/date_utils.sh"
# Verify files are executable
[[ -x "$TEST_RALPH_HOME/lib/circuit_breaker.sh" ]]
[[ -x "$TEST_RALPH_HOME/lib/response_analyzer.sh" ]]
[[ -x "$TEST_RALPH_HOME/lib/date_utils.sh" ]]
}
# =============================================================================
# Test 7-8: Dependency Detection Tests
# =============================================================================
@test "install.sh detects missing dependencies (jq, git, node)" {
# Create a modified install.sh with mocked command -v
local temp_script="$(mktemp)"
cat > "$temp_script" << 'EOF'
#!/bin/bash
set -e
# Override command to simulate missing jq, git, and node/npx
command() {
if [[ "$1" == "-v" ]]; then
case "$2" in
jq|git|node|npx)
return 1
;;
esac
fi
builtin command "$@"
}
# Mock check_dependencies from install.sh
check_dependencies() {
local missing_deps=()
if ! command -v node &> /dev/null && ! command -v npx &> /dev/null; then
missing_deps+=("Node.js/npm")
fi
if ! command -v jq &> /dev/null; then
missing_deps+=("jq")
fi
if ! command -v git &> /dev/null; then
missing_deps+=("git")
fi
if [ ${#missing_deps[@]} -ne 0 ]; then
echo "ERROR: Missing required dependencies: ${missing_deps[*]}"
exit 1
fi
echo "SUCCESS: Dependencies check completed"
}
check_dependencies
EOF
chmod +x "$temp_script"
# Run and expect failure
run bash "$temp_script"
# Should fail
[[ "$status" -ne 0 ]]
# Should mention missing dependencies (all three)
[[ "$output" =~ "Missing required dependencies" ]]
[[ "$output" =~ "jq" ]]
[[ "$output" =~ "git" ]]
[[ "$output" =~ "Node.js" ]]
rm -f "$temp_script"
}
@test "install.sh detects all dependencies present" {
# Skip if actual dependencies are missing
if ! command -v node &> /dev/null && ! command -v npx &> /dev/null; then
skip "Node.js not available"
fi
if ! command -v jq &> /dev/null; then
skip "jq not available"
fi
if ! command -v git &> /dev/null; then
skip "git not available"
fi
# Create a test script that checks dependencies
local temp_script="$(mktemp)"
cat > "$temp_script" << 'EOF'
#!/bin/bash
check_dependencies() {
local missing_deps=()
if ! command -v node &> /dev/null && ! command -v npx &> /dev/null; then
missing_deps+=("Node.js/npm")
fi
if ! command -v jq &> /dev/null; then
missing_deps+=("jq")
fi
if ! command -v git &> /dev/null; then
missing_deps+=("git")
fi
if [ ${#missing_deps[@]} -ne 0 ]; then
echo "ERROR: Missing required dependencies: ${missing_deps[*]}"
exit 1
fi
echo "Dependencies OK"
exit 0
}
check_dependencies
EOF
chmod +x "$temp_script"
run bash "$temp_script"
# Should succeed
assert_success
rm -f "$temp_script"
}
# =============================================================================
# Test 9-10: PATH Detection Tests
# =============================================================================
@test "install.sh PATH detection warns when not in PATH" {
# Set PATH to exclude install directory
export PATH="/usr/bin:/bin"
# Extract and test check_path function
local temp_script="$(mktemp)"
cat > "$temp_script" << EOF
#!/bin/bash
INSTALL_DIR="$TEST_INSTALL_DIR"
check_path() {
if [[ ":\$PATH:" != *":\$INSTALL_DIR:"* ]]; then
echo "WARN: \$INSTALL_DIR is not in your PATH"
echo "Add this to your ~/.bashrc:"
echo " export PATH=\"\\\$HOME/.local/bin:\\\$PATH\""
return 0
else
echo "SUCCESS: \$INSTALL_DIR is already in PATH"
return 0
fi
}
check_path
EOF
chmod +x "$temp_script"
run bash "$temp_script"
# Should warn about PATH
[[ "$output" =~ "not in your PATH" ]] || [[ "$output" =~ "WARN" ]]
rm -f "$temp_script"
}
@test "install.sh PATH detection succeeds when already in PATH" {
# Set PATH to include install directory
export PATH="$TEST_INSTALL_DIR:/usr/bin:/bin"
# Extract and test check_path function
local temp_script="$(mktemp)"
cat > "$temp_script" << EOF
#!/bin/bash
INSTALL_DIR="$TEST_INSTALL_DIR"
check_path() {
if [[ ":\$PATH:" != *":\$INSTALL_DIR:"* ]]; then
echo "WARN: \$INSTALL_DIR is not in your PATH"
return 0
else
echo "SUCCESS: \$INSTALL_DIR is already in PATH"
return 0
fi
}
check_path
EOF
chmod +x "$temp_script"
run bash "$temp_script"
# Should succeed
[[ "$output" =~ "SUCCESS" ]] || [[ "$output" =~ "already in PATH" ]]
rm -f "$temp_script"
}
# =============================================================================
# Test 11-12: Uninstallation Tests
# =============================================================================
@test "install.sh uninstall removes all files" {
# First run installation
run run_install install
assert_success
# Verify files exist
assert_file_exists "$TEST_INSTALL_DIR/ralph"
assert_file_exists "$TEST_INSTALL_DIR/ralph-monitor"
assert_file_exists "$TEST_INSTALL_DIR/ralph-setup"
assert_file_exists "$TEST_INSTALL_DIR/ralph-import"
# Run uninstall
run run_install uninstall
assert_success
# Verify command files are removed
assert_file_not_exists "$TEST_INSTALL_DIR/ralph"
assert_file_not_exists "$TEST_INSTALL_DIR/ralph-monitor"
assert_file_not_exists "$TEST_INSTALL_DIR/ralph-setup"
assert_file_not_exists "$TEST_INSTALL_DIR/ralph-import"
}
@test "install.sh uninstall cleans up directories" {
# First run installation
run run_install install
assert_success
# Verify ralph home exists
assert_dir_exists "$TEST_RALPH_HOME"
# Run uninstall
run run_install uninstall
assert_success
# Verify ralph home is removed
[[ ! -d "$TEST_RALPH_HOME" ]]
}
# =============================================================================
# Test 13-14: Idempotency and Integration Tests
# =============================================================================
@test "installation idempotency (run twice without errors)" {
# First installation
run run_install install
assert_success
# Capture file counts after first install
local ralph_count_1=$(ls "$TEST_INSTALL_DIR" | wc -l)
local template_count_1=$(ls "$TEST_RALPH_HOME/templates" | wc -l)
# Second installation (should overwrite cleanly)
run run_install install
assert_success
# Capture file counts after second install
local ralph_count_2=$(ls "$TEST_INSTALL_DIR" | wc -l)
local template_count_2=$(ls "$TEST_RALPH_HOME/templates" | wc -l)
# Counts should be the same (no duplicates or missing files)
assert_equal "$ralph_count_1" "$ralph_count_2"
assert_equal "$template_count_1" "$template_count_2"
# All files should still exist and be valid
assert_file_exists "$TEST_INSTALL_DIR/ralph"
assert_file_exists "$TEST_RALPH_HOME/templates/PROMPT.md"
assert_file_exists "$TEST_RALPH_HOME/lib/circuit_breaker.sh"
}
@test "complete installation workflow end-to-end" {
# Skip if dependencies missing
if ! command -v jq &> /dev/null; then
skip "jq not available"
fi
if ! command -v git &> /dev/null; then
skip "git not available"
fi
# Run full installation
run run_install install
assert_success
# Verify all directories created
assert_dir_exists "$TEST_INSTALL_DIR"
assert_dir_exists "$TEST_RALPH_HOME"
assert_dir_exists "$TEST_RALPH_HOME/templates"
assert_dir_exists "$TEST_RALPH_HOME/lib"
# Verify all commands installed
assert_file_exists "$TEST_INSTALL_DIR/ralph"
assert_file_exists "$TEST_INSTALL_DIR/ralph-monitor"
assert_file_exists "$TEST_INSTALL_DIR/ralph-setup"
assert_file_exists "$TEST_INSTALL_DIR/ralph-import"
# Verify all templates copied
assert_file_exists "$TEST_RALPH_HOME/templates/PROMPT.md"
assert_file_exists "$TEST_RALPH_HOME/templates/fix_plan.md"
assert_file_exists "$TEST_RALPH_HOME/templates/AGENT.md"
# Verify all lib files copied
assert_file_exists "$TEST_RALPH_HOME/lib/circuit_breaker.sh"
assert_file_exists "$TEST_RALPH_HOME/lib/response_analyzer.sh"
assert_file_exists "$TEST_RALPH_HOME/lib/date_utils.sh"
# Verify all scripts in ralph home
assert_file_exists "$TEST_RALPH_HOME/ralph_loop.sh"
assert_file_exists "$TEST_RALPH_HOME/ralph_monitor.sh"
assert_file_exists "$TEST_RALPH_HOME/setup.sh"
assert_file_exists "$TEST_RALPH_HOME/ralph_import.sh"
# Verify all permissions correct
[[ -x "$TEST_INSTALL_DIR/ralph" ]]
[[ -x "$TEST_RALPH_HOME/ralph_loop.sh" ]]
[[ -x "$TEST_RALPH_HOME/lib/circuit_breaker.sh" ]]
# Verify output contains success message
[[ "$output" =~ "installed" ]] || [[ "$output" =~ "SUCCESS" ]] || [[ "$output" =~ "success" ]]
}