pythonopen-sourcelibrariesfree-threading October 13, 2026

Reading back what a person typed: compactref 1.1.0

I wanted to ship a new version of my library, and my first ideas were adding Pyrefly or raising the Python version. Neither changed anything for the people who use it. What did: accepting a reference typed in lower case without breaking the 1.0 promise, and proving it is safe without the GIL instead of assuming it.

compactref is a Python library that generates short, readable references like ORD-20260710-HCP3CSX from a ULID or a UUID. The long identifier stays the primary key. The short reference is the one a person reads over the phone, types into a search box, or dictates to support.

It came out in July, reached 1.0 a few days later, and three applications use it in production today. 1.0 came with a promise: the API does not change incompatibly without a 2.0. I had not touched it since.

This week I wanted to ship a new version. Here is what went in, what did not, and why.

My first idea helped nobody

My two initial ideas were adding Pyrefly, Meta’s type checker that reached 1.0 in May, or raising the minimum Python version.

Both are maintenance, not product. Pyrefly runs in CI and changes not one line of what ships to PyPI. Raising the minimum from 3.10 to 3.11 takes something away from whoever is still on 3.10 and gives nothing to anyone: a library with no dependencies and no modern syntax at stake gains almost nothing from it.

Both had their place, but neither justified a release. What did was already written down in my own roadmap since 1.0.

The support agent who types in lower case

References can use Crockford’s base32 alphabet: the digits and the letters, without I, L, O or U. The first three were dropped because they get confused with 1 and 0, and U so the code does not spell obscenities by accident.

Crockford designed that alphabet to survive a human, and his spec says how to read back what a person writes: lower case as upper, I and L as 1, O as 0. compactref took the alphabet but not that tolerance. If a support agent typed hcp3csx, the library told them the reference was invalid.

That is exactly the confusion the check character exists to prevent. 1.1.0 fixes it:

from compactref import CROCKFORD_BASE32, CompactRef

orders = CompactRef(
    prefix="ORD",
    separator="-",
    alphabet=CROCKFORD_BASE32,
    check=True,
    tolerant=True,
)

orders.verify("ORD-20260710-hcp3csx")     # True
orders.verify("ORD-2O26O7lO-HCP3CSX")     # True: O for 0, l for 1
orders.normalize("ORD-20260710-hcp3csx")  # 'ORD-20260710-HCP3CSX'

Why it is not on by default

My roadmap said this was safe to add in any 1.x: accepting more input changes nothing that is generated and invalidates no stored reference. True, but incomplete.

The verify() docstring promised something else: True means the string is exactly what this scheme would have produced. Some people use verify() as a gate before saving:

if orders.verify(user_input):
    save(user_input)

If tolerance were on by default, that code would start saving references in lower case, and the next exact lookup in the database would not find them. Nothing fails, nothing warns, and the problem shows up weeks later.

That is why tolerant is off by default and is the last field of the class, so no positional argument moved. It is also why normalize() exists: if True no longer means “this is the stored spelling”, something has to give you the stored spelling back.

Tolerance belongs to the alphabet, not the verifier

The second decision was where the rule lives. Upper-casing would be a mistake in an alphabet that holds a and A as different values, and reading I as 1 means nothing in "abcdef". So tolerant=True with any alphabet other than Crockford’s raises ValueError when the scheme is built.

It does not touch the whole string either. Only the fields the alphabet spells are read back: the suffix, the check character, and the date of a checked scheme. The prefix is a label chosen by whoever configured the scheme, and the separators are punctuation, so both are still matched exactly. ord-20260710-hcp3csx is refused.

That left an open question I wrote down in the roadmap: if someone who types in lower case types all of it in lower case, tolerance stops one field short. I would rather decide that when someone brings me a real case.

Without the GIL: test it instead of assuming it

Python 3.15 came out on October 9, and free-threaded (no GIL) builds have existed since 3.13. The compactref README recommends building a scheme once at import time and sharing it. Without a GIL, “sharing” means several threads really use it at the same time.

I read through the code: CompactRef is a frozen dataclass and the module holds no mutable state. It should be safe. But “should” is a claim, not a test. So I wrote the test:

barrier = threading.Barrier(THREADS)

def run(_: int) -> list[tuple[str, bool, str]]:
    barrier.wait()  # everyone starts together
    return all_of_them()

with ThreadPoolExecutor(max_workers=THREADS) as pool:
    outcomes = list(pool.map(run, range(THREADS)))

assert all(outcome == expected for outcome in outcomes)

Eight threads use the same scheme at once, and each must get exactly what a single thread gets. A second test starts a clean interpreter and checks that importing compactref does not turn the GIL back on, which Python does on its own, with little more than a warning, when it imports any extension that has not declared itself safe without it.

Both passed on the first run, and that made me suspicious. A test that has never failed has not shown that it can fail. So I added a bug on purpose: a shared global buffer inside the encoder, the kind of “optimization” someone might add a year from now. The test failed without the GIL, and with it too. I removed it, and now I know the test has teeth.

With that, pyproject.toml declares Free Threading :: 3 - Stable, and CI runs the suite on 3.14t on Linux and on 3.15t on Linux, macOS and Windows. I did not use 4 - Resilient: one concurrency test is not enough to claim that.

Pyrefly, as a warning

Pyrefly did go in, but without the power to block. mypy is still the type checker that decides. Pyrefly ships every month and adds new rules, and a new rule that flags code nobody touched should be a note on the PR, not a red build.

I made it a separate job rather than a step with continue-on-error inside the lint job. A step that fails quietly leaves its job green, and nobody opens a warning hidden in a green job. As its own job it shows up as its own check.

On its first run it found nine implicit Anys that mypy --strict lets through: an untyped empty list and eight unannotated lambdas in a test. None was a bug, but I fixed them so it starts at zero. That way, anything it reports from now on is new.

What broke without anyone touching it

The first CI run on the PR failed in lint, on README lines the PR had not touched.

Ruff 0.17 started formatting the code blocks inside Markdown. CI installs the latest ruff, so the same commit that passed yesterday failed today. And the formatting it wanted would have broken the comments I had aligned into columns on purpose, so the examples read as a table of results.

I excluded Markdown from the formatter, but the cause is still there: the development tools have no upper version bound. It went into the roadmap: ranges for ruff and mypy, and raising the ceiling deliberately, in its own commit, after reading what each release brings.

Test what you publish, not what you build

PyPI versions cannot be replaced, so before publishing I did a rehearsal on TestPyPI. I installed what was uploaded into clean environments with 3.10, 3.15 and 3.15t, and ran the full suite against the installed package, not the source tree. I also compared the contents of the TestPyPI wheel with a local build of main, and then the PyPI wheel with the TestPyPI one: identical, py.typed included.

Is there competition?

I searched PyPI. There are neighbors, but none does the same thing:

  • shortuuid rewrites a UUID in base57. Truncate it and it stops being unique, and it does not help you know by how much.
  • sqids and hashids turn integers into short IDs for URLs, and can be decoded. They start from numbers, not from a ULID or a UUID.
  • nanoid generates random IDs: not deterministic, and not derived from an existing ID.

compactref derives a deterministic reference from an ID you already have, with a date, a namespace and a check character, and gives you the math to choose its length for your volume. It is a small niche, but it is the problem I had.

What I take away

  • What your users need is often already in your roadmap, not in the new tools you are itching to try.
  • “It is additive” is not the same as “it is safe to turn on by default.” Accepting more input does not break stored references, but it can change what a True means.
  • A test that has never failed has not proven anything yet. Break it on purpose once.
  • Anything without an upper version bound changes even when you do nothing.

1.1.0 is on PyPI, and the full detail is in the CHANGELOG.

Comments