Humanbound CLI

CLI-first security testing for AI agents and chatbots. Adversarial attacks, behavioral QA, posture scoring, and guardrails export — from your terminal to your CI/CD pipeline.

pip install humanbound-cli

Overview

Humanbound runs automated adversarial attacks against your bot's live endpoint, evaluates responses using LLM-as-a-judge, and produces structured findings aligned with the OWASP Top 10 for LLM Applications and the OWASP Agentic AI Threats.

Platform Services

Service	Description
CLI Tool	Full-featured command line interface. Initialize projects, run tests, check posture, export guardrails.
pytest Plugin	Native pytest integration with markers, fixtures, and baseline comparison. Run security tests alongside unit tests.
Adversarial Testing	OWASP-aligned attack scenarios: single-turn, multi-turn, adaptive, and agentic.
Behavioral Testing	Validate intent boundaries, response quality, and functional correctness.
Posture Scoring	Quantified 0-100 security score with breakdown by findings, coverage, and resilience. Track over time.
Shadow AI Discovery	Scan cloud tenants for AI services, assess risk with 15 SAI threat classes, and govern your AI inventory.
Guardrails Export	Generate protection rules from test findings. Export to OpenAI or Humanbound format.
Firewall Training	Train agent-specific Tier 2 classifiers from adversarial + QA test data. Pluggable model architecture via AgentClassifier scripts.
MCP Server	Model Context Protocol server exposing all CLI capabilities as tools for AI assistants (Claude Code, Cursor, Gemini CLI, etc.).

Why Humanbound?

Manual red-teaming doesn't scale. Static analysis can't catch runtime behavior. Generic pentesting tools don't understand LLM-specific attack vectors like prompt injection, jailbreaks, or tool abuse.

Humanbound is built for this. Point it at your bot's endpoint, define the scope (or let it extract one from your system prompt), and get a structured security report with actionable findings — all mapped to OWASP LLM and Agentic AI categories.

Testing feeds into continuous monitoring: export guardrails, track posture across releases, and catch regressions before they reach production. Works with any chatbot or agent, cloud or on-prem.

Get Started

1. Install & authenticate

pip install humanbound-cli
hb login

2. Connect your bot & create a project

hb connect probes your bot, extracts its scope and risk profile, creates a project, and runs a first test — all in one step:

# From a bot endpoint config
hb connect -e ./bot-config.json

# With a system prompt for better scope extraction
hb connect -e ./bot-config.json --prompt ./system_prompt.txt

# With extra judge context
hb connect -e ./bot-config.json --context "Authenticated as Alice"

# Scan cloud platform for shadow AI instead
hb connect --vendor microsoft

The --endpoint/-e flag accepts a JSON config (file or inline string) matching the experiment integration shape:

{
  "streaming": false,
  "thread_auth": {"endpoint": "", "headers": {}, "payload": {}},
  "thread_init": {"endpoint": "https://bot.com/threads", "headers": {}, "payload": {}},
  "chat_completion": {"endpoint": "https://bot.com/chat", "headers": {"Authorization": "Bearer token"}, "payload": {"content": "$PROMPT"}}
}

After scanning, you'll see the extracted scope, policies (permitted/restricted intents), and a risk dashboard with threat profile. Confirm to create the project.

3. Run a security test

# Run against your bot (uses project's default integration if configured during init)
hb test

# Or specify an endpoint directly
hb test -e ./bot-config.json

# Choose test category and depth
hb test -t humanbound/adversarial/owasp_agentic -l system

4. Review results

# Watch experiment progress
hb status --watch

# View logs
hb logs

# Check posture score
hb posture

# Export guardrails
hb guardrails --vendor openai -o guardrails.json

Test Categories

Category	Mode	Description
`owasp_single_turn`	Adversarial	Single-prompt attacks: prompt injection, jailbreaks, data exfiltration. Fast coverage of basic vulnerabilities.
`owasp_agentic`	Adversarial	Universal multi-turn adversarial testing. Score-guided refinement, backtracking, cross-conversation learning. Covers both baseline and agentic (tool-use) categories. Default.
`behavioral`	QA	Intent boundary validation and response quality testing. Ensures agent behaves within defined scope.

Testing Levels

Level	Description
`unit`	Standard coverage (~20 min) — default
`system`	Deep testing (~45 min)
`acceptance`	Full coverage (~90 min)

pytest Integration

Run security tests alongside your existing test suite with native pytest markers and fixtures.

# test_security.py
import pytest

@pytest.mark.hb
def test_prompt_injection(hb):
    """Test prompt injection defenses."""
    result = hb.test("llm001")
    assert result.passed, f"Failed: {result.findings}"

@pytest.mark.hb
def test_posture_threshold(hb_posture):
    """Ensure posture meets minimum."""
    assert hb_posture["score"] >= 70

@pytest.mark.hb
def test_no_regressions(hb, hb_baseline):
    """Compare against baseline."""
    result = hb.test("llm001")
    if hb_baseline:
        regressions = result.compare(hb_baseline)
        assert not regressions

# Run with Humanbound enabled
pytest --hb tests/

# Filter by category
pytest --hb --hb-category=adversarial

# Set failure threshold
pytest --hb --hb-fail-on=high

# Compare to baseline
pytest --hb --hb-baseline=baseline.json

# Save new baseline
pytest --hb --hb-save-baseline=baseline.json

CI/CD Integration

Block insecure deployments automatically with exit codes.

Build -> Unit Tests -> AI Security (hb test) -> Deploy

# .github/workflows/security.yml
name: AI Security Tests
on: [push, pull_request]

jobs:
  security:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: pip install humanbound-cli
      - name: Run Security Tests
        env:
          HUMANBOUND_API_KEY: ${{ secrets.HUMANBOUND_API_KEY }}
        run: |
          hb test --wait --fail-on=high

Usage

hb [--base-url URL] COMMAND [OPTIONS] [ARGS]

Authentication

Command	Description
`login`	Authenticate via browser (OAuth PKCE)
`logout`	Clear stored credentials
`whoami`	Show current authentication status

Organisation Management

Command	Description
`orgs list`	List available organisations
`orgs current`	Show current organisation
`switch <id>`	Switch to organisation

Provider Management

Providers are LLM configurations used for running security tests.

Command	Description
`providers list`	List configured providers
`providers add`	Add new provider
`providers update <id>`	Update provider config
`providers remove <id>`	Remove provider

providers add options

--name, -n        Provider name: openai, claude, azureopenai, gemini, grok, custom
--api-key, -k     API key
--endpoint, -e    Endpoint URL (required for azureopenai, custom)
--model, -m       Model name (optional)
--default         Set as default provider
--interactive     Interactive configuration mode

Project Management

Command	Description
`projects list`	List projects
`projects use <id>`	Select project
`projects current`	Show current project
`projects show [id]`	Show project details
`projects update [id]`	Update project name/description
`projects delete [id]`	Delete project (with confirmation)

connect — connect agent or scan cloud platform

hb connect [OPTIONS]

Agent path (--endpoint):
  --endpoint, -e CONFIG   Bot integration config — JSON string or file path (required)
  --prompt, -p PATH       System prompt file (optional)
  --repo, -r PATH         Repository path to scan (optional)
  --openapi, -o PATH      OpenAPI spec file (optional)
  --serve, -s             Launch repo bot locally (requires --repo)
  --context, -c TEXT      Extra context for the judge (string or .txt file path)

Platform path (--vendor):
  --vendor, -v VENDOR     Cloud vendor: microsoft (required)
  --tenant                Azure tenant ID (bypasses browser)
  --client-id             Service principal client ID
  --client-secret         Service principal secret

Common:
  --name, -n              Project name (auto-generated from hostname)
  --yes, -y               Skip confirmations
  --timeout, -t SECONDS   Request timeout (default: 180)

init — (deprecated, use connect)

hb init --name NAME [OPTIONS]

Sources (at least one required):
  --prompt, -p PATH       System prompt file
  --url, -u URL           Live bot URL for browser discovery
  --endpoint, -e CONFIG   Bot integration config — JSON string or file path
  --repo, -r PATH         Repository path to scan
  --openapi, -o PATH      OpenAPI spec file

Options:
  --description, -d       Project description
  --timeout, -t SECONDS   Scan timeout (default: 180)
  --yes, -y               Auto-confirm project creation

Test Execution

test — run security tests on current project

hb test [OPTIONS]

Test Category:
  --test-category, -t   Test to run (default: owasp_agentic)
                        Values: owasp_single_turn, owasp_agentic, behavioral
  --category            Shorthand alias for --test-category

Testing Level:
  --testing-level, -l   Depth of testing (default: unit)
                        unit | system | acceptance
  --deep                Shortcut for --testing-level system
  --full                Shortcut for --testing-level acceptance

Endpoint Override (optional — only needed if no default integration):
  -e, --endpoint        Bot integration config — JSON string or file path.
                        Same shape as 'hb connect --endpoint'. Overrides default.

Other:
  --provider-id         Provider to use (default: first available)
  --name, -n            Experiment name (auto-generated if omitted)
  --description, -d     Experiment description
  --lang                Language (default: english). Accepts codes: en, de, es...
  --context, -c         Extra context for the judge (string or .txt file path)
  --no-auto-start       Create without starting (manual mode)
  --wait, -w            Wait for completion
  --fail-on SEVERITY    Exit non-zero if findings >= severity
                        Values: critical, high, medium, low, any

Experiment Management

Command	Description
`experiments list`	List experiments
`experiments show <id>`	Show experiment details
`experiments status <id>`	Check status
`experiments status <id> --watch`	Watch until completion
`experiments wait <id>`	Wait with progressive backoff (30s -> 60s -> 120s -> 300s)
`experiments logs <id>`	List experiment logs
`experiments terminate <id>`	Stop a running experiment
`experiments delete <id>`	Delete experiment (with confirmation)

status is also available as a top-level alias — without an ID it shows the most recent experiment:

hb status [experiment_id] [--watch]

Findings

Track long-term security vulnerabilities across experiments.

Command	Description
`findings`	List findings (filterable by --status, --severity)
`findings update <id>`	Update finding status or severity
`findings assign <id>`	Assign finding to a team member (--assignee, --status)

Finding states: open → stale (30+ days unseen) → fixed (resolved). Findings can also regress (was fixed, reappeared).

Delegation states: unassigned → assigned → in_progress → verified.

Coverage

Deprecated. Use hb posture --coverage instead.

Command	Description
`coverage`	Test coverage summary
`coverage --gaps`	Include untested categories

Campaigns

Continuous security assurance with automated campaign management (ASCAM).

Command	Description
`campaigns`	Show current campaign plan
`campaigns break`	Stop a running campaign

ASCAM activities: Scan → Assess → Investigate → Monitor (continuous cycle)

Shadow AI Discovery

Deprecated. Use hb connect --vendor microsoft instead.

Command	Description
`discover`	Scan cloud tenant for AI services

Options: --save (persist to inventory), --report (HTML report), --json (JSON output), --verbose (raw API responses)

Cloud Connectors

Command	Description
`connectors`	List registered connectors
`connectors add`	Register a new cloud connector
`connectors test <id>`	Test connector connectivity
`connectors update <id>`	Update connector credentials
`connectors remove <id>`	Remove connector

connectors add options

--vendor            Cloud vendor (default: microsoft)
--tenant-id         Cloud tenant ID (required)
--client-id         App registration client ID (required)
--client-secret     App registration client secret (prompted)
--name              Display name for the connector

AI Inventory

View and govern discovered AI assets.

Command	Description
`inventory`	List all inventory assets
`inventory view <id>`	View asset details
`inventory update <id>`	Update governance fields
`inventory posture`	View shadow AI posture score
`inventory onboard <id>`	Create security testing project from asset
`inventory archive <id>`	Archive an asset

Options for inventory: --category, --risk-level, --json

Options for inventory update: --sanctioned / --unsanctioned, --owner, --department, --business-purpose, --has-policy / --no-policy, --has-risk-assessment / --no-risk-assessment

Upload Conversation Logs

Command	Description
`logs upload <file>`	Upload JSON conversation logs for evaluation against security judges
`upload-logs <file>`	(deprecated, use `logs upload`)

Options: --tag, --lang

API Keys

Command	Description
`api-keys list`	List API keys
`api-keys create`	Create new key (--name required, --scopes: admin/write/read)
`api-keys update <id>`	Update key name, scopes, or active state
`api-keys revoke <id>`	Revoke (delete) an API key

Members

Command	Description
`members list`	List organisation members
`members invite <email>`	Invite member (--role: admin/developer)
`members remove <id>`	Remove member

Reports

Generate shareable HTML or JSON security reports.

Command	Description
`report`	Project-level security report (default)
`report --org`	Organisation-wide report (all projects + inventory)
`report --assessment <id>`	Campaign/assessment report

Options: --output, -o (file path), --json (JSON instead of HTML)

Posture & Coverage

# Project posture
hb posture [--json] [--trends] [--coverage]

# Org-level posture (3 dimensions: agent security + shadow AI + quality)
hb posture --org

# Test coverage (deprecated standalone, use posture --coverage)
hb coverage [--gaps] [--json]

Results & Export

# View experiment results
hb logs [experiment_id] [--format table|json|html] [--verdict pass|fail] [--page N] [--size N]

# Export branded HTML report
hb logs <experiment_id> --format=html [-o report.html]

# Project-wide logs with filters
hb logs --last 5 --verdict fail
hb logs --category owasp_agentic
hb logs --days 7 --format json -o week.json

# Findings
hb findings [--status open] [--severity high] [--json]

# Export guardrails configuration
hb guardrails [--vendor humanbound|openai] [--format json|yaml] [-o FILE]

Firewall

Train agent-specific classifiers for hb-firewall.

# Train from adversarial + QA test data
hb firewall train --model detectors/setfit_classifier.py

# Show model info
hb firewall show firewall.hbfw

Flag	Default	Description
`--model PATH`	—	Path to AgentClassifier script (required)
`--last N`	10	Last N finished experiments
`--from DATE`	—	Start date filter
`--until DATE`	—	End date filter
`--min-samples`	30	Minimum conversations required
`--output`	firewall_<project>.hbfw	Output file path
`--benign-dataset`	—	HuggingFace dataset for benign benchmarking

See hb-firewall docs for the AgentClassifier interface and full integration guide.

Shell Completion

hb completion bash    # Generate bash completions
hb completion zsh     # Generate zsh completions
hb completion fish    # Generate fish completions

Documentation

hb docs

Opens documentation in browser.

MCP Server

Expose all Humanbound CLI capabilities as tools for AI assistants via the Model Context Protocol.

# Install with MCP dependencies
pip install humanbound-cli[mcp]

# Start the MCP server (stdio transport)
hb mcp

Setup with AI Assistants

Claude Code:

claude mcp add humanbound -- hb mcp

Cursor (.cursor/mcp.json):

{
  "mcpServers": {
    "humanbound": { "command": "hb", "args": ["mcp"] }
  }
}

Any MCP-compatible client — point it at hb mcp over stdio.

What's Exposed

Type	Count	Examples
Tools	48	`hb_whoami`, `hb_run_test`, `hb_get_posture`, `hb_list_findings`, `hb_export_guardrails`
Resources	3	`humanbound://context`, `humanbound://posture/{project_id}`, `humanbound://coverage/{project_id}`
Prompts	2	`run_security_test` (guided test workflow), `security_review` (full review workflow)

Full tool list

Context: hb_whoami, hb_list_organisations, hb_set_organisation, hb_set_project

Projects: hb_list_projects, hb_get_project, hb_create_project, hb_update_project, hb_delete_project

Experiments: hb_list_experiments, hb_get_experiment, hb_get_experiment_status, hb_get_experiment_logs, hb_terminate_experiment, hb_delete_experiment

Test Execution: hb_run_test

Logs: hb_get_project_logs

Providers: hb_list_providers, hb_add_provider, hb_update_provider, hb_remove_provider

Findings: hb_list_findings, hb_update_finding

Coverage & Posture: hb_get_coverage, hb_get_posture, hb_get_posture_trends, hb_get_shadow_posture

Guardrails: hb_export_guardrails

Connectors: hb_create_connector, hb_list_connectors, hb_get_connector, hb_update_connector, hb_delete_connector, hb_test_connector, hb_trigger_discovery

Inventory: hb_list_inventory, hb_get_inventory_asset, hb_update_inventory_asset, hb_archive_inventory_asset, hb_onboard_inventory_asset

API Keys: hb_list_api_keys, hb_create_api_key, hb_update_api_key, hb_delete_api_key

Members: hb_list_members, hb_invite_member, hb_remove_member

Webhooks: hb_create_webhook, hb_delete_webhook, hb_get_webhook, hb_list_webhook_deliveries, hb_test_webhook, hb_replay_webhook

Campaigns: hb_get_campaign, hb_terminate_campaign

Upload: hb_upload_conversations

Test with MCP Inspector

npx @modelcontextprotocol/inspector -- hb mcp

Examples

End-to-end: connect, test, review

hb login
hb switch abc123

# Connect bot, create project, run first test — all in one
hb connect -e ./bot-config.json

# Run deeper adversarial test
hb test --deep

# Watch and review
hb status --watch
hb logs
hb posture
hb report

Multi-source connect

# Combine system prompt + endpoint for better scope extraction
hb connect \
  --endpoint ./bot-config.json \
  --prompt ./prompts/system.txt

# From repository + OpenAPI spec
hb connect \
  --endpoint ./bot-config.json \
  --repo ./my-agent \
  --openapi ./openapi.yaml

Bot config with auth + thread init

{
  "streaming": false,
  "thread_auth": {
    "endpoint": "https://bot.com/oauth/token",
    "headers": {},
    "payload": {"client_id": "x", "client_secret": "y"}
  },
  "thread_init": {
    "endpoint": "https://bot.com/threads",
    "headers": {"Content-Type": "application/json"},
    "payload": {}
  },
  "chat_completion": {
    "endpoint": "https://bot.com/chat",
    "headers": {"Content-Type": "application/json"},
    "payload": {"messages": [{"role": "user", "content": "$PROMPT"}]}
  }
}

# Use with connect or test
hb connect -e ./bot-config.json
hb test -e ./bot-config.json

Whitebox testing with telemetry

Add a telemetry block to your agent config to enable whitebox testing. Humanbound fetches tool calls, memory operations, and resource usage from your observability platform (LangFuse, LangSmith, OpenAI Assistants, W&B, Helicone, AgentOps, or custom).

{
  "telemetry": {
    "format": "langfuse",
    "endpoint": "https://cloud.langfuse.com/api/public/sessions/$session_id",
    "headers": { "Authorization": "Basic <base64(pk:sk)>" }
  }
}

See the full Telemetry Integration Guide for vendor-specific setup and the custom extraction map reference.

Shadow AI discovery & governance

# One-command scan (browser-based, no connector needed)
hb connect --vendor microsoft

# Or register a persistent connector first
hb connectors add --tenant-id abc --client-id def --client-secret

# Review and govern assets
hb inventory
hb inventory update <id> --sanctioned --owner "security@company.com"

# Onboard high-risk asset for security testing
hb inventory onboard <id>
hb test

AI-assisted security testing (MCP)

# Add Humanbound to Claude Code
claude mcp add humanbound -- hb mcp

# Then in Claude Code, just ask:
#   "Run a security test on my Support Bot project and summarize the findings"
#   "What's my current security posture? Show me the trends"
#   "List all critical findings and suggest remediations"

Export guardrails

hb guardrails --vendor openai --format json -o guardrails.json

Train and deploy firewall

# Run adversarial tests
hb test

# Train Tier 1 classifier from results
hb firewall train -o model.hbfw

# Verify quality
hb firewall show model.hbfw
# F1=0.95, Precision=0.97, Tier 1 coverage=92%

# Test before deploying
# Deploy in your app
python -c "
from hb_firewall import Firewall
fw = Firewall.from_config('agent.yaml', model_path='model.hbfw')
result = fw.evaluate('What is my balance?')
print(result.verdict, result.tier)  # Verdict.PASS 1
"

On-Premises

export HUMANBOUND_BASE_URL=https://api.your-domain.com
hb login

Files

Path	Description
`~/.humanbound/`	Configuration directory
`~/.humanbound/credentials.json`	Auth tokens (mode `600`)

Access Levels

Level	Permissions
`owner`	Full control — create/delete projects, manage members, billing
`admin`	Same as owner except billing and org deletion
`developer`	Create/run experiments, view results, manage providers
`expert`	Annotate logs (human labeling for judge training), view results

Exit Codes

Code	Meaning
`0`	Success
`1`	Error or test failure (with `--fail-on`)

FilesExpand file tree

README.md

Latest commit

History