ccc’s map is static: it links a test to the code the test calls. A test that reaches code another way is invisible to it, for example over HTTP to another service, through a framework, or through a table of handlers. A change in one service then leaves you running every suite that might touch it.
Runtime instrumentation fills that gap. Record one run of your suites with ccc’s probes in, and ccc knows which tests actually executed which functions, across every service and language the tests passed through. For any change, it picks the tests that ran the changed code.
examples/ holds two copies of the same blob-store SaaS (disclosure: all examples are generated by Agents): five Go services (gateway, auth, metadata, storage and billing) behind one public gateway, and 50 tests.
| Suite | Tests | How it reaches the services |
|---|---|---|
| Go unit | 20 | In process |
| Go integration | 6 | Over HTTP, through test servers |
| Python integration: customer flows | 12 | Over HTTP |
| Python contract: response shapes | 8 | Over HTTP |
| Python load: throughput and latency | 4 | Over HTTP |
The same three changes went into each project. The numbers are how many of the 50 tests each project has to run.
| Scenario | Change | Changed functions | Without instrumentation (Project-A) | With: safe (Project-BBB) | With: minimal (Project-BBB) |
|---|---|---|---|---|---|
| 1 | metadata counts list and delete calls for a metrics endpoint | 2 | 50 | 11 | 2 |
| 2 | storage hashes chunks through a streaming hasher | 1 | 50 | 22 | 1 |
| 3 | a hardening release across all five services | 6 | 50 | 42 | 5 |
1. A simple change that still needs more than one test. Index.Delete and Index.List sit on different request paths.
minimal needs two unit tests, one per path:go test -run '^(TestDeleteObject|TestListObjectsSortedWithPrefix)$' ./services/metadata
2. A one-function change deep in the stack. Checksum runs on every upload and download.
minimal needs one test:go test -run '^(TestPutGetChunk)$' ./services/storage
3. A complex change across every service. It changes six functions in five services.
minimal needs five, one per service:go test -run '^(TestVerifyKnownKey|TestRecordAccumulatesUsage|TestJoinChunksReversesSplit|TestCreateBucketRejectsBadNames|TestPutGetChunk)$' \
./services/auth ./services/billing ./services/gateway ./services/metadata ./services/storage
minimal promises that the changed code runs, not that every regression is caught. It prefers cheap tests: cost is the recorded duration weighted by kind, from unit 1 up to load 6. When a change matters enough, run safe. In these scenarios minimal never needs a load test, while safe keeps every load test that ran the change.
The probes also count how many times each test ran each function. ccc prints the count beside every test it selects, as in Checksum ×2, and the VS Code hover shows it too. The counts don’t change which tests are selected. They show how hard each test works the changed code:
| Scenario | minimal |
The busiest test in safe |
|---|---|---|
| 1 | TestDeleteObject: Delete ×1TestListObjectsSortedWithPrefix: List ×2 |
test_list_under_load: List ×101 |
| 2 | TestPutGetChunk: Checksum ×2 |
test_upload_throughput: Checksum ×600 |
| 3 | Five unit tests, each running its function 1 to 6 times | test_download_latency_p95: Verify ×202, PriceCents ×201, joinChunks ×200, Get ×200, validBucketName ×1 |
A minimal test runs the change a handful of times. A regression that only appears under repetition, such as a slowdown, a leak or a race, is far more likely to show in a test that runs the code hundreds of times.
To see it yourself, run bash examples/run-scenarios.sh. It needs only ccc, git and Python 3.
unit, contract, interop, integration, smoke or load), its file and its language.baggage header, ccc.fixture=…,ccc.file=…,ccc.lang=…,ccc.kind=…, and each service passes it on to the services it calls.ccc-observed/1 files. Each line covers one fixture in one process: the fixture, how long it took, and the functions it ran, by file, name and line, with how many times each ran. CLI.md has the format.observed in .ccc/map.json, from --observed, or from .ccc/observed/. It ties each recorded function to its own map by file, name and owner, so a recording still matches after the code has moved.ccc insights and the test_triggers MCP tool, test_triggers.observed lists the safe and minimal sets, each with its run commands. ccc changes and the VS Code hints count a function a fixture ran as tested, with evidence observed. Each of these shows how many times the fixture ran the function.ccc probe writes an instrumented copy of the project. The source in the repository never changes.
ccc probe go,python . --out /tmp/probed
cd /tmp/probed
export CCC_OBSERVED_DIR=$PWD/observed
go test ./... && pytest
CLI.md has every option. In the Go copy, for example:
-package storage
+package storage; import cccprobe "ccc.invalid/probe"
-func (s *Store) Put(id string, data []byte, checksum string) error {
+func (s *Store) Put(id string, data []byte, checksum string) error { cccprobe.Hit("services/storage/storage.go", "Store.Put", 39);
-func TestPutGetChunk(t *testing.T) {
+func TestPutGetChunk(t *testing.T) { cccprobe.Test(t, "services/storage/storage_test.go", "unit");
- srv := &http.Server{Handler: h, ReadHeaderTimeout: 5 * time.Second}
+ srv := &http.Server{Handler: cccprobe.Handler(h), ReadHeaderTimeout: 5 * time.Second}
The probes are inserted as text, on the line they belong to, so every comment stays where it was and every line keeps its number. Each function is named the way ccc’s own map names it, so the records match the map exactly.
What goes into the copy, by language:
| Language | Counts its runs | Names itself a fixture | Takes the fixture in | Sends the fixture out | Hands it to other threads |
|---|---|---|---|---|---|
| Go | every function and method except init |
TestXxx(t *testing.T), with its subtests |
net/http servers (http.Server, ListenAndServe, Serve, httptest), gRPC servers |
http.DefaultTransport, http.Client transports, gRPC dials |
goroutines, through the goroutine that started them (Go 1.21+) |
| Rust | every function except const fn |
#[test], #[tokio::test], #[rstest], #[test_case] and the like |
axum 0.7+ and tonic 0.12+ through a tower layer, actix-web through wrap_fn |
reqwest .send(), tonic::Request::new |
tokio::spawn, spawn_blocking, std::thread::spawn |
| C++ | every function except constexpr |
gtest TEST, TEST_F, TEST_P, Catch2 and doctest TEST_CASE, SCENARIO, Boost.Test |
cpp-httplib routes, grpc++ servers | cpp-httplib clients, grpc::CreateChannel |
std::thread, std::jthread, std::async |
| TypeScript | every function, method and arrow bound to a name | it, test and specify callbacks, under jest, vitest, mocha and node --test |
Node http and https servers (express, fastify, koa and the rest), grpc-js servers |
http and https requests (axios, got, node-fetch), fetch, grpc-js clients |
every async continuation, through AsyncLocalStorage |
| Python | every function and method | pytest tests, unittest methods | WSGI (Flask, Django, wsgiref), ASGI (Starlette, FastAPI, Django), http.server, grpcio |
http.client (urllib, requests, urllib3), httpx, aiohttp, grpcio |
threads and ThreadPoolExecutor |
How each runtime reaches the build:
ccc.invalid/probe, in .ccc-probe/go. Each probed go.mod requires it and replaces it with that directory, and so do go.work and vendor/ where there are any. A module that requires gRPC also gets internal/cccprobegrpc.ccc_probe, in .ccc-probe/rust, added to each probed package as a path dependency, with its tower feature on where axum or tonic is used. It needs Rust 1.70. A --locked build needs its lock file updated first..ccc-probe/cpp, included by relative path with a #line after it. No build file changes. It needs C++17.node_modules/ccc-probe, imported by every probed file. The copy’s node_modules links each installed package back to the original..ccc-probe/python that every probed file finds by itself. It registers with pytest as the test files load, so nothing needs installing.A test’s kind comes from its file name, then its directories, innermost first: contract, interop, integration (or e2e, it, acceptance, functional), smoke, and load (or perf, stress, bench). A Rust crate’s tests/ directory is integration. Everything else is unit. --kind 'qa/**=smoke' overrides it.
Every probe stays off unless CCC_OBSERVED_DIR names a directory, so the copy runs unchanged otherwise. CCC_SERVICE names the process in its records, CCC_RUN names the run (local by default), and CCC_COMMIT overrides the commit the copy was made from.
A test’s record is written when the test ends. A service writes what its fixtures ran every two seconds, and again when it exits.
Limits:
--test-threads=1, to attribute everything..c files, .h files stay unprobed, because a C source might include them.On the default branch, nightly or on every merge:
ccc probe <languages> . --out $WORK makes the instrumented copy.$WORK with CCC_OBSERVED_DIR set. Start each service the suites call from the copy too, with CCC_SERVICE naming it.cat $CCC_OBSERVED_DIR/*.jsonl > observed.jsonl, and publish the file as a build artifact.Then point ccc at it:
"observed": [{ "from": "https://ci.example.com/artifacts/observed.jsonl", "auth": "env:CI_TOKEN" }]
The example’s record.sh predates ccc probe and uses the hand-written probes in examples/project-bbb/ccc. ccc probe go,python examples/project-a produces the same 50 records from the plain project, function for function and call for call.
ccc fetches a URL at most once every ten minutes. Records are matched by function name, so they stay useful as the code moves. A function written since the recording has no record yet. It’s listed under unobserved, and the static call-graph list covers it until the next recording.
Available Now
ccc-observed/1 format and the reader.ccc changes, test_triggers and the VS Code hints.ccc probe, for Go, Rust, C++, TypeScript and Python, carrying the fixture over HTTP and gRPC.Should be available in 2.0.0
safe.