Ben Smyth

I'm thinking about AI hacking swarms; software's future is compliance:

Cryptographers don't trust themselves, their colleagues neither, they trust proofs, cold and unforgiving, because human confidence fails far more frequently than maths.

Society operates on critical infrastructure — pipelines, roads, rails, span the ground beneath us, cargo fleets moving economies through vast oceans, aircraft and satellites up in the sky — "good" isn't good enough. These systems must not fail. AI is our rude awakening!

My agents and I systematically built a battleground. Us against critical infrastructure. Seeking paths to systemic collapse. Every chain reaction shrouded in complexity. Six days, six zero days.

Oops…I…did it again.

Current count: Twelve zero days. This is not a drill. Every system we rely on, at the brink of collapse. Software is entering a new era, traditional engineering processes are obsolete.

Complex systems are non-deterministic. We cannot a priori predict what exactly they'll do. Yet we need to control them. Rather than blindly trust, we insist systems are demonstratable compliant — they must be proven to do what they're supposed to do.

The twelve zero days, they exist precisely because I was able to force non-compliance. Derail systems in catastrophic fashion. Had those systems been formally proven, they'd not have fallen.

software's future is compliance