This is the documentation index for system-tests. Start from the section that matches what you need, and follow the links to go deeper.
For a quick start guide, see the main README.
Slack: #apm-shared-testing
Learn what system-tests is, how it works, and what the key concepts are.
- Understanding overview -- entry point for concepts
- Architecture overview -- components, containers, data flow, and what system-tests is good (and bad) at
- Scenarios -- the different types of tests: end-to-end, parametric, SSI, Kubernetes
- Weblogs -- the test applications that get instrumented by tracer libraries
- Test flow -- the full test execution flow (setup, wait, test)
- Glossary -- definitions of terms like pass, fail, xpass, xfail, enabled, disabled
Everything you need to build images and execute test scenarios.
- Execute overview -- entry point for running tests
- Build -- build options, weblog variants, image names
- Run -- run options, test selection, scenarios, sleep mode, timeouts
- Test outcomes -- understanding test result symbols and states
- Logs -- the log folder structure and what each file contains
- Replay mode -- re-run test methods without rebuilding the environment
- Custom tracer versions -- testing with local or unmerged tracer builds
- Troubleshooting -- common issues, debugging tips, tracer debug output
- SSI execution -- running SSI-specific scenarios
- End-to-end execution -- running end-to-end scenarios
How to add, modify, enable, and disable tests.
- Edit overview -- entry point for editing tests
- Add a new test -- step-by-step guide for creating a test
- Add a new scenario -- creating new test scenarios
- Enable a test -- activating a test for a library version
- Skip / disable tests -- decorators for conditional test skipping
- Manifests -- the YAML files that control test activation per library
- Versions -- version specification guidelines
- Features -- linking tests to the feature parity dashboard
- Formatting -- linter and code style (
format.sh) - IAST validations -- marking tests with vulnerabilities
- Remote config -- writing remote config tests
- Update Docker images -- modifying test app Docker images
- Runbook -- operational runbook
- Flushing -- how data flushing works and implementing
/flush - Library interface validation -- API reference for validating tracer-to-agent traces
- Agent interface validation -- API reference for validating agent-to-backend data
- Backend interface validation -- API reference for validating backend API responses
Deep dives into each scenario type.
- End-to-end: covered in scenarios overview and architecture
- Parametric: overview and contributing guide
- AWS SSI / Onboarding: full guide and provision structure
- Docker SSI: guide
- Kubernetes lib injection: overview, details, and injector dev
- Other scenarios: AWS Lambda, integration frameworks (IPv6 and Go proxies are covered in the scenarios overview)
Adding system-tests to your continuous integration pipeline.
- CI overview -- general steps to integrate
- GitHub Actions -- GitHub Actions workflow
- System-tests CI -- how the system-tests own CI pipeline works
The test applications that tracers instrument.
- Weblog overview -- types of weblogs
- End-to-end weblog spec -- all endpoints and their expected behavior
- GraphQL weblog -- GraphQL-specific weblog
Deep dives for maintainers and contributors to the system-tests framework itself.
- Internals overview -- index of internal documentation
- End-to-end lifecycle -- how e2e scenarios execute step by step
- Parametric lifecycle -- how parametric scenarios execute
- MITM certificate -- recreating the proxy certificate
- Core dump generation -- generating core dumps for debugging
- Async model revamp -- asynchronous model documentation
- PR reviews -- pull request review guidelines
- Manifest RFC -- the original manifest design proposal
- AI integration guide -- built-in rules for AI-assisted development
- Cursor guide -- comprehensive Cursor AI guide
- Cursor examples -- practical usage examples
- Prompt validation -- testing AI prompts with promptfoo
- GitHub MCP server -- GitHub MCP server integration
- Java endpoint prompt -- Java endpoint creation prompt