Three LLM calls and not one more

When you build something with AI, the temptation is to ask the LLM for everything: read, compare, decide and assign the grade. It works in the demo, and it turns expensive, slow and hard to audit as soon as there is real traffic.

On this project I started from a simple rule: the LLM is expensive, embeddings are cheap. Everything else came from taking that rule seriously. The result is an API where the LLM shows up in exactly three places, and in none of them does it decide the number that matters.

The project

djobmatch is a small REST API built with Django and Django Ninja, with Postgres and pgvector behind it. It has three endpoints:

• POST /api/jobs/parse turns the text of a job posting into JSON: hard and soft skills, real seniority, salary range and red flags.
• POST /api/resumes/parse does the same for a resume.
• POST /api/match takes an already-parsed resume and job and returns a score from 0 to 100 plus an explanation.

The domain doesn’t matter much. What’s interesting is where the LLM is, and where it isn’t.

Decision 1: the LLM in three places, not one more

The LLM only does what nothing cheaper can do well: understand messy text and write prose. Specifically:

1. Extract the JSON of a job posting.
2. Extract the JSON of a resume.
3. Write the explanation of a match.

Everything else (comparing, scoring, remembering) is done by embeddings, Postgres and plain code. The rule is written in the repo’s README and CLAUDE.md, so any agent that touches the code follows it too.

The blue boxes are the only ones that cost an LLM call, and neither runs if the data is already cached.

Decision 2: the LLM doesn’t set the score

The 0–100 number comes from the cosine distance between two embeddings, computed inside Postgres with pgvector. The LLM never touches it.

distance = (
    Resume.objects.filter(pk=resume.pk)
    .annotate(d=CosineDistance("embedding", job.embedding))
    .values_list("d", flat=True)
    .first()
)
similarity = 1.0 - float(distance)
score = round((similarity + 1) / 2 * 100)

Why it matters:

• Deterministic. The same pair always gets the same score. An LLM at temperature 0 doesn’t guarantee that either.
• Cheap. An embedding costs a fraction of a chat call, and the database computes the cosine.
• Testable. similarity_to_score is a pure function; it’s tested with no network and no mocks.

One detail makes a difference: I don’t embed the raw text. I embed a canonical text built from the parsed JSON (seniority + hard skills + soft skills). That puts the job and the resume in the same space, so a recruiter’s writing style isn’t competing against a candidate’s.

Decision 3: cache in the database, and tests that guard it

Every job and resume is stored with a content_hash: the SHA-256 of the text with whitespace normalized. If the same text arrives again, even with extra spaces, the service returns the existing row and calls no provider.

Matches follow the same idea: there’s a UNIQUE (resume, job) constraint, so each pair is computed once. The first thing every service does is check the cache, and all of them return (object, cached) so the client knows what happened.

What makes this hold up over time is the tests. They don’t just check the response: they count the provider calls.

first = client.post("/match", json={"resume_id": resume.pk, "job_id": job.pk})
second = client.post("/match", json={"resume_id": resume.pk, "job_id": job.pk})

assert second.json()["cached"] is True
assert len(_patch_llm.calls) == 1  # not called on the cache hit

If someone adds an LLM call on the cache path, the cost goes up silently. With this test, CI fails before the bill arrives.

Decision 4: the provider behind an interface

Neither the endpoints nor the services import the OpenAI SDK. They depend on two small interfaces:

class LLMProvider(abc.ABC):
    def complete_json(self, system_prompt: str, user_content: str) -> dict: ...

class EmbeddingProvider(abc.ABC):
    def embed(self, text: str) -> list[float]: ...

A get_llm() function picks the implementation based on the LLM_PROVIDER variable. Switching to Anthropic or Voyage means writing one class and adding a branch to the factory.

The benefit I use most isn’t switching providers, it’s the tests. Every service takes optional llm= and embedder= arguments, so the suite injects fakes that record their calls. No test touches a real API.

What it doesn’t solve yet

This architecture isn’t perfect, and I’d rather say so before someone else does:

• The score isn’t calibrated. I map cosine similarity from [-1, 1] to [0, 100] with a linear rule. But text embeddings almost never produce negative similarities and tend to sit in a narrow range, so I expect most scores to land in the upper half of the scale. I haven’t measured it with real data yet. Cheap and deterministic doesn’t automatically mean correct.
• The explanation inherits the score. The LLM gets the number along with both JSONs, so it tends to justify it rather than question it. It’s an explanation, not a second opinion.
• The embedding depends on the extraction. If the LLM misses a skill while parsing, that skill doesn’t exist for the score. An error in the expensive stage carries over into the cheap one.

None of these problems come from the “the LLM is expensive” rule. They’re the next step: measure and calibrate.

Lessons learned

The first is that the rule has to be written down before the code. Once I put “the LLM in three places only” in the README and in CLAUDE.md, it stopped being my intention and became something anyone, an agent included, could read and follow.

The second: a number you’re going to show someone shouldn’t depend on a generative model. If the same resume against the same job can get a different score each time, nobody will trust that score.

The third: caching by content is simpler than it sounds. A SHA-256 of the normalized text and a UNIQUE constraint in the database were enough, with no Redis and no queues.

And the one that helped me most: a cost rule is worth little if nobody guards it. Having the tests count provider calls is what keeps this from degrading over time.

The project is small on purpose: no Celery, no Docker, no extra layers. Keeping the idea in a handful of files was a decision too. The code is on GitHub.