Standalone Data Generation

[kommundsen]

Motivation

Conjecture's strategy engine is a powerful data generation system, but it's currently only accessible inside [Property] test methods. Many teams need realistic, structured test data outside of tests: seeding development databases, populating staging environments, generating API fixtures, creating sample datasets for demos, and feeding integration test harnesses. Exposing the strategy engine as a standalone tool unlocks this entire use case.

.NET Advantage

The existing Conjecture.Mcp dotnet tool already demonstrates that Conjecture's engine can operate outside a test runner. .NET's dotnet tool infrastructure provides global/local tool installation, and System.Text.Json serialization (with .NET 10's strict mode and PipeWriter support) enables high-throughput structured output. The [Arbitrary] source generator means user-defined types can be generated without reflection.

Key Ideas

CLI Tool

# Generate 1000 Person records as JSON
dotnet conjecture generate --type MyApp.Person --count 1000 --output persons.json

# Generate with seed for reproducibility
dotnet conjecture generate --type MyApp.Order --count 500 --seed 42 --format csv

# Generate with constraints
dotnet conjecture generate --type MyApp.Person --count 100 --settings '{"MaxExamples": 100}'

# Pipe to another tool
dotnet conjecture generate --type MyApp.Event --count 10000 | dotnet ef database seed --stdin

Programmatic API

// Outside of test context
var people = Generate.Integers<int>(1, 100)
    .Sample(count: 50, seed: 42);

var dataset = Generate.From<PersonArbitrary>()
    .SampleMany(count: 1000)
    .ToList();

// Stream large datasets
await foreach (var record in Generate.From<OrderArbitrary>().Stream(count: 100_000, seed: 42))
{
    await writer.WriteAsync(record);
}

Output Formats

• JSON (default, via System.Text.Json)
• CSV (flat types)
• JSON Lines (streaming)
• Custom IOutputFormatter extensibility

Integration Points

• EF Core database seeding
• HTTP API test fixtures
• Message queue population (Azure Service Bus, RabbitMQ)
• File-based test data (*.json, *.csv in test projects)

Design Decisions to Make

1. Extend existing Conjecture.Mcp tool or create a new Conjecture.Tool / conjecture-cli?
2. How to discover user-defined types and their strategies from a compiled assembly? (Reflection vs requiring a generator project reference)
3. Should Sample() / SampleMany() be on Strategy<T> directly or a separate DataGen class?
4. Shrinking doesn't apply to data generation — should the API clearly separate generation from testing?
5. Streaming large datasets: how to handle memory for 100K+ records?
6. Deterministic output: given the same seed and type, output must be identical across runs.

Scope Estimate

Medium. CLI scaffolding + Sample()/Stream() APIs are ~2 cycles. Output formatters and EF Core integration add ~1-2 more.

Dependencies

• Conjecture.Core strategy engine
• System.Text.Json for serialization
• System.CommandLine or existing MCP infrastructure for CLI
• [Arbitrary] source generator for user-defined types

Open Questions

• Should the CLI tool load user assemblies dynamically (like dotnet ef does) or require a project reference?
• How to handle types that require complex strategy composition (not just [Arbitrary])?
• Should we support generating data that satisfies relational constraints (e.g., foreign keys across tables)?
• Is there demand for non-.NET output targets (e.g., generate data for a Python service's test suite)?

[kommundsen]

Design Decisions to Make

1. Extend Conjecture.Mcp or create a new Conjecture.Tool / conjecture-cli?
→ New Conjecture.Tool project. Keeps AI-assistant tooling and data generation CLI separate.

2. How to discover user-defined types from a compiled assembly?
→ Dynamic assembly loading via AssemblyLoadContext (like dotnet ef). The [Arbitrary] source generator already emits IStrategyProvider<T> into the user's compiled assembly; the CLI scans for that interface. No arbitrary reflection needed — users just build their project and pass --assembly path/to/MyApp.dll.

3. Should Sample() / SampleMany() be on Strategy<T> or a separate DataGen class?
→ Separate DataGen static class. Keeps Strategy<T> lean and makes the separation from test-context generation explicit. DataGen lives in Conjecture.Core to retain access to internal generation APIs, and because it's a user-facing API, not just a CLI helper.

4. Should the API clearly separate generation from testing (shrinking doesn't apply)?
→ Yes, implicitly. DataGen.Sample() / DataGen.Stream() operate outside any test runner and never trigger shrinking. The separation is structural.

5. Streaming large datasets — how to handle memory for 100K+ records?
→ IAsyncEnumerable<T>. Supporting PipeWriter in the generation API would couple generation to serialization; byte-level output is owned by IOutputFormatter. Formatters can use PipeWriter internally if needed.

6. Deterministic output — same seed + type must produce identical output across runs.
→ Guaranteed by design. SplittableRandom is a deterministic PRNG; given the same seed the generation sequence is fixed. DataGen accepts an optional ulong? seed parameter; omitting it draws a random seed.

Output formats (not in original list, decided during planning)
→ JSON + JSON Lines built-in, plus an IOutputFormatter extensibility interface with Stream as the output sink. Custom formatters (CSV, SQL, etc.) are left to users.

---

Open Questions

Should the CLI load user assemblies dynamically or require a project reference?
→ Dynamic assembly loading (answered above under Decision 2).

How to handle types that require complex strategy composition (not just [Arbitrary])?
→ The CLI scans all IStrategyProvider<T> implementations, not just [Arbitrary]-generated ones. Hand-written providers implement the same interface and are discovered automatically.

Should we support generating data that satisfies relational constraints (e.g., foreign keys)?
→ Yes, in scope. A JSON plan file declares ordered generation steps; later steps bind properties to values from earlier steps via $ref expressions (e.g., "CustomerId": { "$ref": "customers[*].Id" }). The CLI gains a --plan flag alongside --type.

Is there demand for non-.NET output targets (e.g., a Python service's test suite)?
→ No extra work needed. JSON/JSON Lines output is already language-agnostic. Additional formats can be added via IOutputFormatter.

[kommundsen]

In cycle 76.1 (PR incoming), Stream was changed to return IEnumerable instead of IAsyncEnumerable. All generation is synchronous — wrapping it in an async envelope added complexity with no benefit. If async generation is ever needed (e.g. external data sources), a dedicated StreamAsync branch returning IAsyncEnumerable<T> is the right place for it.