Darkwood · PHP · C · ARM64 · SQLite

I Built a C Compiler
in PHP
That Can Compile SQLite

PHP → C frontend → ARM64 → SQLite

C

→

PHP

→

.s

→

native

The question

How much of a C compiler
can I implement in PHP
before it can compile SQLite?

The result

darkwood

SQLite 3.46.0

↓

PHP compiler

↓

ARM64

↓

native executable

↓

46/46 regression fixtures

What “compiled in PHP” means

PHP

Lexer
Preprocessor
Parser
AST
Sema
ARM64 Codegen

macOS

as
linker
native executable

Clang does not compile sqlite3.c.

Deliberately small

source

↓

tokens

↓

AST

↓

typed AST

↓

ARM64 .s

No LLVM

No SSA

No CIR

No custom assembler

src/Compiler/Driver/Compiler.php

The driver

$tokens = $preprocessor->preprocess($fileId);

$parser = new Parser($tokens, $this->diagnostics);
$decls = $parser->parse();

$sema = new Sema($this->diagnostics);
$tast = $sema->analyze($decls);

$codegen = new Codegen($enumConstants);
$assembly = $codegen->generate($tast);

SOURCE → TOKENS → AST → TYPES → ASM

SourceLoc.php

Source locations

final readonly class SourceLoc
{
    public function __construct(
        public int $fileId,
        public int $offset,
    ) {}
}

fileId + offset

↓

SourceManager

↓

line : column

Token.php

A token is more than text

public function __construct(
    public TokenKind $kind,
    public string $spelling,
    public SourceLoc $loc,
    public array $hideSet = [],
) {}

Why does a token need a hideSet?

Macro expansion needs memory

#define A B
#define B A

A

↓ expand

B  [hide A]

↓ expand

A  [hide A,B]

↓

STOP

MacroTable.php · MacroExpander

Inside the preprocessor

if ($token->hideSet !== []
    && in_array($token->spelling, $token->hideSet, true)) {
    $output[] = $token;
    continue;
}

$newHideSet = $this->unionHideSet($token->hideSet, [$macro->name]);
$expanded = $this->expandObjectLike(...);
for ($i = count($expanded) - 1; $i >= 0; --$i) {
    $pendingReversed[] = $expanded[$i];
}

The preprocessor is a compiler subsystem

#define

function macros

arguments

#if / #elif / #else

defined()

# / ##

__VA_ARGS__

rescanning

hide sets

SQLite never reaches the parser unless this works.

Parsing C by hand

No parser generator

recursive descent

+

Pratt expressions

FuncDecl

BinaryExpr

CallExpr

CompoundLiteralExpr

C declarators fight back

≠

int *p[4];

array of pointers

int (*p)[4];

pointer to array

Parser::applyArraySuffix()

if ($type instanceof PointerCType) {
    $to = $type->to;
    $array = new ArrayCType($to, $count);
    return new PointerCType($array);
}

int *p[4]

Array
 └── Pointer
      └── int

int (*p)[4]

Pointer
 └── Array[4]
      └── int

Ast.php

The AST is just PHP

final class BinaryExpr extends Expr
{
    public function __construct(
        public BinaryOp $op,
        public Expr $left,
        public Expr $right,
        SourceLoc $loc,
        public ?CType $resolvedType = null,
    ) {}
}

Parser builds structure. Sema gives it meaning.

CType.php

Representing C types in PHP

CType
├── IntCType / CharCType / …
├── PointerCType
├── ArrayCType
├── FunctionCType
├── StructCType / UnionCType
└── EnumCType

$this instanceof EnumCType => true,  // isInteger()

Array ≠ pointer — except when it is

int a[3];
int *p = a + 1;

ArrayCType remains

↓ expression context

pointer-like behavior

↓

+ 1 means + sizeof(int)

Semantic analysis

Scopes

Symbols

Typedefs

Enums

Types

Conversions

$this->enumConstants[$case->name] = $case->value;

enum { VALUE = 42 };

→

.long 42

Global initialization is codegen too

C

static unsigned char table[4] = {
    84, 92, 134, 0
};

ARM64

_table:
.byte 84
.byte 92
.byte 134
.byte 0

This once became .quad.

A PHP namespace bug corrupted C memory

instanceof ArrayCType

Expected

App\Compiler\Common\ArrayCType

PHP resolved

App\Compiler\CodeGen\ArrayCType

wrong class

↓

wrong branch

↓

.quad

↓

runtime fail

It assembled. It linked. It was wrong.

From AST to ARM64

int add(int a, int b)
{
    return a + b;
}

_add:
stp x29, x30, [sp, #-16]!
mov x29, sp
sub sp, sp, #16
str x0, [x29, #-8]
str x1, [x29, #-16]
… ldrsw … add …
mov x0, x9
ldp x29, x30, [sp], #16
ret

x0 / x1 args

x29 frame · x30 LR

x0 return

x9–x15 scratch

Stack frame

high
──────────────
saved x29 / x30
──────────────
params · locals
temps · compounds
──────────────
SP
low

sub sp, sp, #0  ; FRAME_SIZE_PLACEHOLDER

Patched after emission · 16-byte aligned

Pointer arithmetic means scaling

int a[3];
a + 1

Wrong

add x?, x?, #1

Correct

address + 1 × sizeof(int)

lsl …, #2

035-array-ptr-arith

Direct vs indirect calls

Direct

bl _function

Function pointer

blr xN

int call(int (*fn)(int), int value)
{
    return fn(value);
}

The ABI is where “valid assembly” stops

syntax correct
≠
ABI correct

arguments

register lifetime

stack alignment

variadics

structs

aggregate returns

Struct returns use registers too

≤16 bytes

→

x0 first word

x1 second word

struct S b(void) { return a(); }

Bug

preserve only part of x0/x1

Fix

str x0, [...]
str x1, [...]

054-struct-return-chain · Loop 48

SQLite is an integration oracle

SQLite

Preprocessor

Declarators

Globals

Enums

Structs / Unions

Function pointers

Variadic calls

ABI

Static data

Failure → fixture → fix

flowchart TB
  A[SQLite / clang mismatch]
  B[reduce]
  C[tiny C fixture]
  D[find subsystem]
  E[fix one behavior]
  F[all fixtures]
  G[SQLite]
  A --> B --> C --> D --> E --> F --> G
  G -.-> A

Bugs became permanent knowledge

010char array global

016variadic spill

034compound literal argument

035array pointer arithmetic

044nested designated initializer

051pointer-to-array

054struct return chain

Every fixture is a bug the compiler should only have once.

Two different pipelines

COMPILER

tokens
 ↓ AST
 ↓ Sema
 ↓ ARM64

OPERATION

compile
 ↓ assemble
 ↓ link
 ↓ run
 ↓ validate

Where should Flow live?

Where Darkwood Flow fits

Compiler internals

plain PHP

Operational pipeline

darkwood/flow

yield $this->timed('compile_sqlite', $this->compileSqlite);
yield $this->timed('assemble_sqlite', $this->assembleSqlite);
yield $this->timed('compile_harness', $this->compileHarness);
yield $this->timed('assemble_harness', $this->assembleHarness);
yield $this->timed('link', $this->linkSmokeExecutable);
yield $this->timed('run', $this->runSmokeTest);

One command, many artifacts

php bin/console app:compiler-sqlite

flowchart TB
  C[sqlite3.c] --> S[sqlite3.s]
  S --> O[sqlite3.o]
  H[harness.c] --> HS[harness.s]
  HS --> HO[harness.o]
  O --> L[link]
  HO --> L
  L --> E[executable]
  E --> D[darkwood]

The engineering loop

probe

↓

execute

↓

compare

↓

reduce

↓

fix

↓

validate

↓

KEEP

AI becomes useful when it cannot escape the executable feedback loop.

The five-minute wake loop

WAKE TICKresume + probe

ENGINEERING LOOPinvestigated change

KEEPsurvives validation

Loop 48 = last retained engineering change

Eventually

No mismatch.

46/46
SQLite → darkwood

The correct decision was to stop changing the compiler.

What this proves

PHPcan host a non-trivial compiler

Direct pathfrontend → ARM64 can go surprisingly far

SQLiteforces subsystems to become honest

Feedbackexecutable evidence beats plausible code

What it does NOT prove

Not full C compliance

Not a Clang replacement

Not a production compiler

Not “AI removes validation”

The Darkwood takeaway

Compiler algorithms → plain PHP

Operational pipelines → Darkwood Flow

Correctness → executable evidence

darkwood

SQLite

↓

PHP

↓

ARM64

↓

It assembled. It linked. It queried SQLite.