Skip to content

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 

History

3 Commits
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

TestFang

AI-powered autonomous testing agent that reads your codebase, generates exhaustive test suites, executes them in isolated sandboxes, and delivers results directly to your repository.


The Problem

Software testing is broken.

Developers write tests that confirm what they already know. They test the happy path, the obvious edge cases, and the things they expect to work. The bugs that make it to production are the ones nobody thought to testβ€”race conditions, memory leaks, concurrency issues, security holes, and performance cliffs hiding in the corners of the codebase.

Writing good tests is tedious. Writing great tests requires thinking like an attacker. Most developers are not attackers. They are builders.

TestFang is the attacker.


What It Does

TestFang ingests your entire repository, understands its structure, dependencies, and critical paths, and generates a test suite designed to break your code. It does not test for coverage. It tests for destruction.

The test suite runs in an isolated microVM. If the tests pass, TestFang opens a pull request with the generated test files. If the tests fail, it opens an issue with detailed failure logs, stack traces, and an analysis of what went wrong.

You do not write tests. You review them.


How It Works

1. Trigger

You initiate a test run via the CLI or dashboard. TestFang clones your repository and prepares the environment.

2. Sandbox

Firecracker boots a lightweight microVM. Startup takes approximately 125 milliseconds and consumes roughly 5 megabytes of memory. The VM is isolated from the host system and from other test runs.

3. Runtime Installation

The microVM installs the DeepSeek TUI agent via npm. This provides the AI with the capability to read files, write files, execute shell commands, and interact with the repository.

4. Secret Decryption

Secrets required for the run (API keys, tokens) are stored encrypted using .anv. They are decrypted at runtime and injected into the environment. No secrets are written to disk.

5. Repository Analysis

The AI reads your entire repository. It analyzes file structure, dependencies, function signatures, error handling patterns, concurrency primitives, and security boundaries. Nothing is ignored.

6. Test Generation

The AI writes a comprehensive test suite. It targets:

  • Boundary conditions
  • Null and undefined handling
  • Concurrency and race conditions
  • Error recovery paths
  • Security vulnerabilities
  • Performance regressions
  • Type mismatches
  • Invalid input handling

7. Test Execution

The AI executes the tests inside the sandbox. It captures stdout, stderr, exit codes, and any generated artifacts.

8. Result Handling

If all tests pass:

  • The AI stages the test files
  • Creates a new branch
  • Commits and pushes
  • Opens a pull request with a detailed description of what was tested and why

If any tests fail:

  • The AI gathers failure logs
  • Analyzes the root cause where possible
  • Opens an issue with the logs and recommended fixes

9. Cleanup

The microVM terminates. All state is destroyed. No logs remain on disk. No secrets persist.


Architecture


β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚                     User Action                            β”‚
β”‚                CLI or Dashboard Trigger                    β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
β”‚
β–Ό
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚                  Firecracker MicroVM                       β”‚
β”‚                                                           β”‚
β”‚  β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”   β”‚
β”‚  β”‚  Runtime Installation                              β”‚   β”‚
β”‚  β”‚  npm install -g deepseek-tui                     β”‚   β”‚
β”‚  β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜   β”‚
β”‚                          β”‚                                  β”‚
β”‚                          β–Ό                                  β”‚
β”‚  β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”   β”‚
β”‚  β”‚  Secret Decryption                                 β”‚   β”‚
β”‚  β”‚  .anv decrypt β†’ environment variables              β”‚   β”‚
β”‚  β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜   β”‚
β”‚                          β”‚                                  β”‚
β”‚                          β–Ό                                  β”‚
β”‚  β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”   β”‚
β”‚  β”‚  AI Agent                                          β”‚   β”‚
β”‚  β”‚  - Reads repository                                β”‚   β”‚
β”‚  β”‚  - Writes test files                               β”‚   β”‚
β”‚  β”‚  - Executes test commands                          β”‚   β”‚
β”‚  β”‚  - Interprets results                              β”‚   β”‚
β”‚  β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜   β”‚
β”‚                          β”‚                                  β”‚
β”‚                          β–Ό                                  β”‚
β”‚  β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”   β”‚
β”‚  β”‚  Outcome                                           β”‚   β”‚
β”‚  β”‚  β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”    β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”            β”‚   β”‚
β”‚  β”‚  β”‚  Tests Pass   β”‚    β”‚  Tests Fail   β”‚            β”‚   β”‚
β”‚  β”‚  β”‚  Open PR      β”‚    β”‚  Open Issue   β”‚            β”‚   β”‚
β”‚  β”‚  β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜    β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜            β”‚   β”‚
β”‚  β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜   β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
β”‚
β–Ό
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚                     Cleanup                                β”‚
β”‚                VM terminates. State destroyed.              β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜


Two-Model Validation

TestFang employs two AI models in adversarial validation.

The primary model generates the test suite. It is instructed to be ruthless, exhaustive, and destructive.

The validator model reviews the generated test suite. It identifies gaps, challenges assumptions, and proposes additional tests. The validator is instructed to be even more critical than the primary model.

This adversarial process ensures that the final test suite is comprehensive. The first model writes the tests. The second model tries to break them. If the validator finds a flaw, the primary model revises the test suite. The process repeats until the validator is satisfied.

No single model is trusted. The system trusts the conflict between them.


Use Cases

Continuous Integration

Integrate TestFang into your CI pipeline. Run tests on every pull request before merge. Catch issues before they reach production.

Release Validation

Run TestFang before major releases. Ensure that critical paths have been tested exhaustively.

Legacy Codebases

For projects with minimal test coverage, TestFang can generate an initial test suite that covers the entire codebase. This provides a safety net for refactoring.

Security Audits

Use TestFang to identify security vulnerabilities. The AI is instructed to think like an attacker and test for injection, privilege escalation, and data exposure.

Performance Testing

TestFang can generate performance tests that stress your system under load. Identify bottlenecks before they impact users.


Security

Isolation

Each test run executes in its own Firecracker microVM. The VM has its own kernel and memory space. No test run can interfere with another.

Secrets

All secrets are stored encrypted using .anv. They are decrypted at runtime and injected into the environment. No secrets are written to disk. The VM terminates and the secrets are gone.

Permissions

The AI runs as a non-privileged user. It cannot modify system files, access other processes, or escalate privileges.

Network

Network access is restricted. The AI can communicate with GitHub for PR/issue creation. It cannot reach arbitrary external services.

Data Retention

No test artifacts persist after the VM terminates. Logs, test files, and generated data are destroyed. The only persistent output is the PR or issue opened in your repository.


Pricing

Plan Price Features
Self-Hosted $0 Full source code. You configure and run your own infrastructure.
Pay-Per-Run $3 One full test suite run. PR or issue generated. No commitments.
Enterprise Custom Volume pricing. Dedicated infrastructure. Audit logs. SLA.

No subscriptions. No recurring charges. You pay only when you run tests.


Installation

Self-Hosted

git clone https://github.com/slick-lab/testfang
cd testfang
zig build
./zig-out/bin/testfang --help

SaaS

Visit testfang.com. Sign up. Add your repository. Run tests from the dashboard.


Example Run

$ testfang run https://github.com/your-org/your-repo

Cloning repository...
Repository cloned. Analyzing structure...
AI analyzing 147 files...
AI generating test suite...
- Edge cases: 24 tests
- Concurrency: 12 tests
- Security: 8 tests
- Performance: 6 tests
Running tests...
Passed: 47 / 50
Failed: 3 tests
Opening issue with failure logs...
Issue opened: https://github.com/your-org/your-repo/issues/104

Philosophy

Happy path tests are for beginners.

Ruthless tests are for production.

TestFang does not test for coverage. It tests for destruction. The goal is not to confirm that your code works. The goal is to find all the ways it can break.

If your code survives TestFang, it is ready for production. If it does not survive, you have a clear understanding of what needs to be fixed.


Contributing

Contributions are welcome.

  • Open an issue to report a bug or request a feature
  • Submit a pull request with improvements
  • Join the community discussions

See CONTRIBUTING.md for detailed guidelines.


License

MIT


Maintainers


Support

  • GitHub Issues: bug reports and feature requests
  • Community Forum: discussions and questions

FAQ

What languages does TestFang support?

Any language. The AI adapts to the repository structure and generates appropriate test commands.

Can I run TestFang on private repositories?

Yes. The CLI and SaaS versions support both public and private repositories.

How long does a test run take?

Typically 2-5 minutes, depending on repository size and test complexity.

What happens to my code?

TestFang clones your repository into a temporary directory within the microVM. It writes test files, executes them, and then destroys the VM. Your code is not stored or shared.

Can I review the tests before they are merged?

Yes. TestFang opens a pull request. You review the generated tests before merging.

What if the AI generates bad tests?

The validator model reviews all generated tests. If the validator identifies issues, the primary model revises the tests. The process repeats until the validator is satisfied.

Is there a free tier?

The self-hosted version is free. You provide your own infrastructure and API keys. The SaaS version is pay-per-run.

How do I get API keys?

You provide your own DeepSeek API key for self-hosted deployments. The SaaS version includes API access in the per-run price.

About

High quality test creator for your codebase

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages