Hello Xi, why not

On being tired of assembling a service out of a few hundred thousand lines of other people’s packages

[Almost Ad] This one is about Xi, which I make. xi.samalstudios.com ↗

I started a small service last year. Before I had written a line of logic I had a lock file with hundreds of entries in it, a framework I had to learn the opinions of, three config formats, and a container image large enough to make the build take longer than the work.

None of that was the problem I sat down to solve.

That feeling is where Xi came from. Not a belief that the world needs another programming language. Just a stubborn question: what would a language look like if the things I reach for in every service were part of it, rather than assembled each time from other people’s packages?

The things I always end up needing

Every backend service I have written in the last ten years needed roughly the same furniture.

• Dependency injection, so the pieces can be swapped and tested.
• Configuration that is typed, and fails at startup rather than at midnight.
• Logging that is structured.
• A way to say what a function is for, not only what types it takes.
• Validation on values, so an email address is not simply a string.
• Something to express business rules without a cliff of nested conditions.

In most languages each of those arrives as a dependency, and each dependency brings its own conventions, its own annotations, its own startup cost and its own upgrade treadmill. The result works. It is also a tower of other people’s decisions, and when it breaks you debug the tower rather than your program.

Xi puts them in the language.

What that looks like

Dependency injection is syntax, not a framework. You declare what a function needs and the compiler wires it, at compile time, with no reflection and no runtime container to boot.

Functions carry their intent. A mapper, a predicate, a producer, a consumer, a reducer, a decision. The kind is part of the signature, so the compiler knows what sort of thing you are writing and so does the next person to read it.

Types can be refined. A value is not just a string, it is a string that has passed the rule attached to its type, and the compiler keeps track of where that has been checked.

Decision tables are a language construct, because business rules are tables in every meeting I have ever been in, and turning them into nested conditionals is how they drift from what was agreed.

It compiles ahead of time to a native binary. No virtual machine to install, no interpreter on the host, no base image with a runtime in it. The build output is a file you can copy to a server and run.

The compiler compiles itself

The lexer, the parser, the code generator and the driver are all written in Xi. The compiler that builds Xi programs is an Xi program.

That property has a name: self-hosting. A compiler is self-hosting when it can compile its own source code, which sounds like a parlour trick and is actually the first serious test a language gets.

Getting there is a chicken and egg problem, and the way out is a bootstrap. A seed binary, built from an earlier generation of the compiler, compiles the current source. The result compiles the source again. From then on the language builds itself, and the seed is only history.

Two things make it worth the effort.

The first is that it forces honesty. Writing a compiler is a demanding program: text handling, data structures, recursion, error reporting, file handling, performance that matters. If the language is awkward, the compiler is where you feel it first, and you cannot blame the user. I rewrote several parts of Xi because writing the compiler in it was annoying, which is the correct reason to change a language.

The second is independence. A language whose compiler is written in someone else’s language inherits that toolchain forever. Once it compiles itself, the only thing it needs to keep existing is a copy of its own binary, and anyone can reproduce it from source.

The honest state of it

Xi is at version 0.1.13. That number is doing real work in that sentence.

The compiler is written in Xi. The standard library covers logging, conversion, time, collections, HTTP handlers, events, scheduled jobs, channels and async functions backed by worker threads, and there is a playground that runs in a browser so you can try it without installing anything. You can install it with a single brew command if you want it locally.

What it is not: battle-tested, widely used, or finished. There is no ecosystem. If you need a client library for some service, you will probably be writing it. The error messages are better than they were and worse than they should be.

I am not going to pitch it as ready for your production system. It is ready for me to build things with, which is the bar I set before telling anyone it existed.

Why bother at all

Because the alternative was to keep accepting that starting a service means inheriting a few hundred thousand lines of other people’s work before writing my own.

There is a reasonable counter-argument, and I hold it myself on alternate days: all that machinery exists because people solved real problems with it, and reinventing it in a language nobody uses is the most expensive way to learn why those solutions look the way they do.

Both things are true. I decided I would rather learn it from the inside.

Xi is open source. If the idea of a language where injection, intent, refinement and decisions are built in rather than bolted on sounds interesting, the compiler and the examples are there to read, and the playground is one click away.

If it sounds like a terrible idea, that is also a reasonable position. It has been a very good year of work regardless.

#programminglanguages#compilers#xi#backend#opensource#systems