Capstone Project: Hacktiv8 x IBM SkillsBuild

AutoTest-CLI

An IBM Bob CLI AI Agent that generates comprehensive, idiomatic Go table-driven unit tests with edge-case coverage directly from your source code.

Go 1.22+ IBM Bob CLI + ACP Zed Editor Ready 100% Test Coverage

View Code Prompt Architecture

zsh ~ autotest-cli-capstone

~/autotest-cli-capstone $ bob acp --trust

# IBM Bob CLI started in ACP mode

Connected to Agent Client Protocol endpoint

# Bob reads AGENTS.md, analyzes validator.go...

Analyzing exported functions: ValidateDispatch

Mapping 3 logic branches, 4 error paths

Generating table-driven tests (20 cases)...

validator_test.go written successfully

~/autotest-cli-capstone $ go test -v -cover ./...

=== RUN TestValidateDispatch

--- PASS: happy path: valid inputs (0.00s)

--- PASS: boundary: prompt at exact maxLen (0.00s)

--- PASS: negative: empty tenant ID (0.00s)

--- PASS: negative: nil AllowedRooms (0.00s)

... 16 more cases ...

--- PASS: TestValidateDispatch (0.00s)

coverage: 100.0% of statements

ok github.com/indraprhmbd/autotest-cli-capstone 0.669s

~/autotest-cli-capstone $

Problem Statement

In fast-moving software development, unit tests are frequently skipped or written incompletely. Most manually-written tests cover only the happy path, leaving boundary conditions, negative paths, and anomalous inputs untested.

Regression bugs slip into production when new code changes miss undocumented edge-case assumptions.

Potential cross-tenant access leaks in multi-room AI routing services go undetected without thorough isolation tests.

Writing boilerplate test scaffolding repeatedly wastes engineering time that should be spent on building features.

Target Users

Software / Backend Developer

Needs a terminal-native assistant to auto-generate high-quality unit tests immediately after writing new business logic in Go.

QA Automation Engineer

Requires a complete edge-case and negative-test reference based on actual code logic to maximize code coverage benchmarks.

Solo Developer / Systems Architect

Manages microservice infrastructure without a dedicated QA team and needs to maintain codebase stability efficiently.

4-Layer Prompt Architecture

Defined declaratively in AGENTS.md and executed through IBM Bob CLI to enforce deterministic, production-ready test output.

Layer 1

Role & Persona

Establishes the agent as a Senior Go QA Engineer specialized in deterministic, table-driven testing for AI agent services.

// Senior Go QA
// Reliability Eng.
// Table-driven tests

Layer 2

Technical Constraints

Restricts output to Go standard library only. No external assertions. Enforces t.Run() table-driven patterns. Source files are never modified.

import "testing"
// no testify
t.Run("...", ...)

Layer 3

Analytical Matrix

Maps all branch paths before generating tests. Covers happy path, boundary conditions, and negative/security scenarios for every exported function.

+ happy path

~ boundary

- negative

Layer 4

Verification Loop

Instructs Bob CLI to write the test file and immediately run go test -v -cover ./..., self-correcting if any assertion fails.

PASS coverage: 100.0%
// self-verified
// production-ready

Prompt Iteration Timeline

Three refinement stages that progressively hardened the agent from a naive baseline to a fully self-verifying test generator.

Iteration 1: Baseline Prompt

Failed

"Buatkan unit test untuk file validator.go ini menggunakan bahasa Go."

Output wrote separate TestXxx functions; repetitive, non-table-driven structure.

Imported stretchr/testify: compile error, not in go.mod.

Only covered happy path scenarios. No boundary or negative cases.

Iteration 2: Go Standard Constraints

Partial

Added: stdlib-only, t.Run table-driven format, negative case coverage.

Compiled cleanly with no external dependencies. Table-driven structure enforced.

Added invalid input scenarios: empty tenant, unauthorized room, whitespace prompt.

Agent did not verify error assertions against actual runtime behavior. No execution step.

Iteration 3: Self-Verification Loop

Production-Ready

Added: Layer 4 in AGENTS.md: run go test -v -cover ./... and self-correct on failure.

All 20 test cases: PASS on first execution. Zero manual correction required.

Statement coverage locked at 100.0%: all branches exercised including fallback paths.

Output is directly deployable. No additional developer intervention needed.

Code Showcase

Source file, agent spec, and generated test suite.

package router

import (
	"errors"
	"strings"
)

var (
	ErrEmptyTenantID    = errors.New("tenant ID cannot be empty")
	ErrUnauthorizedRoom = errors.New("room ID is not in the allowlist")
	ErrPayloadTooLarge  = errors.New("message payload exceeds maximum token limit")
	ErrInvalidPrompt    = errors.New("prompt contains kosong or whitespace only")
)

type DispatchConfig struct {
	AllowedRooms  []string
	MaxPayloadLen int
}

// ValidateDispatch validates a message request before routing to the AI Agent container.
func ValidateDispatch(tenantID, roomID, prompt string, cfg DispatchConfig) (string, error) {
	cleanTenant := strings.TrimSpace(tenantID)
	if cleanTenant == "" {
		return "", ErrEmptyTenantID
	}

	if len(cfg.AllowedRooms) == 0 {
		return "", ErrUnauthorizedRoom
	}

	allowed := false
	for _, r := range cfg.AllowedRooms {
		if r == roomID {
			allowed = true
			break
		}
	}
	if !allowed {
		return "", ErrUnauthorizedRoom
	}

	cleanPrompt := strings.TrimSpace(prompt)
	if cleanPrompt == "" {
		return "", ErrInvalidPrompt
	}

	maxLen := cfg.MaxPayloadLen
	if maxLen <= 0 {
		maxLen = 4096 // default fallback
	}

	if len(cleanPrompt) > maxLen {
		return "", ErrPayloadTooLarge
	}

	return cleanTenant + ":" + roomID, nil
}

# AutoTest-CLI Agent Specification

## Layer 1: Role & Persona
You are a Senior Go QA & Reliability Engineer specializing in
deterministic unit testing, table-driven tests, and edge-case
discovery for lightweight AI agent services.

## Layer 2: Technical Constraints & Standards
- Use ONLY Go's standard `testing` package (`testing`, `errors`, `strings`).
- Do NOT install or import external assertion libraries (like `testify`).
- Enforce idiomatic Go Table-Driven Tests (`[]struct{ name string; ... }`)
  executed with `t.Run()`.
- Never modify the target source file (`validator.go`).

## Layer 3: Analytical & Edge-Case Matrix
For every exported function, cover:
1. Happy Path (valid tenant, allowed room, normal prompt,
   custom & default MaxPayloadLen).
2. Boundary Conditions (whitespace-only strings,
   `MaxPayloadLen <= 0` fallback, exact max length vs `maxLen + 1`).
3. Negative & Security Cases (empty tenant ID, empty
   `AllowedRooms` slice, unauthorized room ID, empty/whitespace prompt).

## Layer 4: Execution & Verification
1. Generate `validator_test.go` in the same directory.
2. Run `go test -v -cover ./...` to verify all test cases pass
   and coverage exceeds 95%.

package router

import (
	"errors"
	"strings"
	"testing"
)

func TestValidateDispatch(t *testing.T) {
	defaultCfg := DispatchConfig{
		AllowedRooms:  []string{"room-1", "room-2"},
		MaxPayloadLen: 100,
	}

	tests := []struct {
		name      string
		tenantID  string
		roomID    string
		prompt    string
		cfg       DispatchConfig
		wantRoute string
		wantErr   error
	}{
		// Happy Path
		{
			name:      "happy path: valid inputs with custom MaxPayloadLen",
			tenantID:  "tenant-abc",
			roomID:    "room-1",
			prompt:    "Hello, agent!",
			cfg:       defaultCfg,
			wantRoute: "tenant-abc:room-1",
			wantErr:   nil,
		},
		{
			name:     "happy path: MaxPayloadLen <= 0 falls back to 4096",
			tenantID: "tenant-fallback",
			roomID:   "room-1",
			prompt:   "A short prompt.",
			cfg: DispatchConfig{
				AllowedRooms:  []string{"room-1"},
				MaxPayloadLen: 0,
			},
			wantRoute: "tenant-fallback:room-1",
			wantErr:   nil,
		},
		// Boundary Conditions
		{
			name:      "boundary: prompt at exact MaxPayloadLen",
			tenantID:  "tenant-boundary",
			roomID:    "room-1",
			prompt:    strings.Repeat("x", 100),
			cfg:       defaultCfg,
			wantRoute: "tenant-boundary:room-1",
			wantErr:   nil,
		},
		{
			name:      "boundary: prompt at maxLen + 1 returns ErrPayloadTooLarge",
			tenantID:  "tenant-abc",
			roomID:    "room-1",
			prompt:    strings.Repeat("y", 101),
			cfg:       defaultCfg,
			wantRoute: "",
			wantErr:   ErrPayloadTooLarge,
		},
		// Negative & Security Cases
		{
			name:      "negative: empty tenant ID",
			tenantID:  "",
			roomID:    "room-1",
			prompt:    "Valid prompt.",
			cfg:       defaultCfg,
			wantRoute: "",
			wantErr:   ErrEmptyTenantID,
		},
		{
			name:      "negative: whitespace-only tenant ID",
			tenantID:  "   ",
			roomID:    "room-1",
			prompt:    "Valid prompt.",
			cfg:       defaultCfg,
			wantRoute: "",
			wantErr:   ErrEmptyTenantID,
		},
		{
			name:      "negative: nil AllowedRooms slice",
			tenantID:  "tenant-abc",
			roomID:    "room-1",
			prompt:    "Valid prompt.",
			cfg: DispatchConfig{
				AllowedRooms:  nil,
				MaxPayloadLen: 100,
			},
			wantRoute: "",
			wantErr:   ErrUnauthorizedRoom,
		},
		{
			name:      "negative: whitespace-only prompt",
			tenantID:  "tenant-abc",
			roomID:    "room-1",
			prompt:    "   ",
			cfg:       defaultCfg,
			wantRoute: "",
			wantErr:   ErrInvalidPrompt,
		},
		// ... 12 more cases covering all logic branches
	}

	for _, tc := range tests {
		t.Run(tc.name, func(t *testing.T) {
			gotRoute, gotErr := ValidateDispatch(
				tc.tenantID, tc.roomID, tc.prompt, tc.cfg,
			)
			if gotRoute != tc.wantRoute {
				t.Errorf("route: got %q, want %q", gotRoute, tc.wantRoute)
			}
			if !errors.Is(gotErr, tc.wantErr) {
				t.Errorf("error: got %v, want %v", gotErr, tc.wantErr)
			}
		})
	}
}

20 test cases 3 categories coverage: 100.0%

All tests PASS