aarondb/distributed_harness

distributed_harness — deterministic adversarial safety oracle

This module deliberately models reproducible fault schedules and invariant checks only. The process/network adapter owns delivery; it records the resulting observations here so a failure can be replayed without timing or scheduler luck.

Types

Artifact

pub type Artifact {
  Artifact(
    seed: Int,
    schedule: List(Fault),
    observations: List(Observation),
    violations: List(Violation),
  )
}

Constructors

Artifact(
  seed: Int,
  schedule: List(Fault),
  observations: List(Observation),
  violations: List(Violation),
)

Fault

pub type Fault {
  Partition(String, String)
  Heal(String, String)
  DuplicateRpc(String)
  ReorderRpc(String)
  Crash(String)
  Restart(String)
  DiskFault(String)
  SlowFollower(String)
  MembershipChurn(String)
  ClockSkew(String, Int)
}

Constructors

```
Partition(String, String)
```
```
Heal(String, String)
```
```
DuplicateRpc(String)
```
```
ReorderRpc(String)
```
```
Crash(String)
```
```
Restart(String)
```
```
DiskFault(String)
```
```
SlowFollower(String)
```
```
MembershipChurn(String)
```
```
ClockSkew(String, Int)
```

Observation

pub type Observation {
  Leaders(term: Int, nodes: List(String))
  Applied(index: Int, copies: Int)
  Fence(resource: String, expected: Int, received: Int)
  RecoveryAlarm(present: Bool)
}

Constructors

```
Leaders(term: Int, nodes: List(String))
```
```
Applied(index: Int, copies: Int)
```

Fence(resource: String, expected: Int, received: Int)

```
RecoveryAlarm(present: Bool)
```

Run

pub type Run {
  Run(
    seed: Int,
    schedule: List(Fault),
    observations: List(Observation),
  )
}

Constructors

Run(
  seed: Int,
  schedule: List(Fault),
  observations: List(Observation),
)

Violation

pub type Violation {
  SplitBrain(term: Int, leaders: List(String))
  DuplicateApply(index: Int, copies: Int)
  StaleFenceAccepted(
    resource: String,
    expected: Int,
    received: Int,
  )
  UnsafeRecoveryUnalarmed
}

Constructors

SplitBrain(term: Int, leaders: List(String))

```
DuplicateApply(index: Int, copies: Int)
```

StaleFenceAccepted(
  resource: String,
  expected: Int,
  received: Int,
)

```
UnsafeRecoveryUnalarmed
```

Values

artifact

pub fn artifact(run: Run) -> Artifact

Persist this value exactly when an adapter run fails. It contains every deterministic input needed to reproduce the invariant failure locally or in CI; raw process logs may be attached alongside it but are not required to reconstruct the schedule.

inspect

pub fn inspect(run: Run) -> List(Violation)

passed

pub fn passed(run: Run) -> Bool

replay

pub fn replay(seed: Int, observations: List(Observation)) -> Run

schedule

pub fn schedule(seed: Int) -> List(Fault)

The stable seeds are a public replay corpus. Keep them fixed: changing a seed changes a reproducibility contract and belongs in a release note.