A test can tell me that a function returns 200. A linter can tell me that a variable is never used. Neither one always tells me whether the change actually respects what the code is supposed to mean.
I have run into this more than once: the diff looked reasonable, the tests passed, and only a later review showed that an important decision had moved to the wrong place. Coding agents make the problem more visible. They can produce a convincing patch while quietly changing a responsibility, weakening an assumption, or explaining a feature without making the important part any clearer.
That is why I built Jeff, a Go CLI for semantic checking through rules. Jeff is for developers who want to check code or Markdown content against meaning-based rules before a review, a merge, or another agent gets involved.
Jeff uses Jev, TypeSafe’s model for returning structured decisions that software can use directly. It does not rewrite files and it does not replace ordinary tests. It adds a check for the part of a change that is difficult to express as a deterministic assertion.
Jev, explained simply#
What first interested me about Jev was the separation between the content being evaluated and the question being asked about it. Conceptually, an exchange looks like this:
state: the text of an article
question: does the introduction tell its intended reader who it is for?
answer: 0.82The program sends Jev a state, the content to evaluate, together with a typed question. Jev returns a structured decision rather than a paragraph that the caller has to parse. TypeSafe describes Jev as its first System One model: a model designed to make decisions that software can use, rather than to keep a conversation going.
That difference matters. Asking a language model to answer “yes” or “no” still leaves the application responsible for interpreting whatever text comes back. With Jev, the question has a defined answer type, and the result can be compared with a threshold in code.
Choice, Score, and Noul#
TypeSafe exposes three main primitives:
- Choice, when the answer should be one option from a list;
- Score, when the answer belongs on a scale;
- Noul, when the question is whether a statement is true.
Jeff currently supports only Noul rules. A Noul answer is a number between 0 and 1. A value close to 0 means that the answer tends towards “no”; a value close to 1 means that it tends towards “yes”. A value in the middle is not automatically a failure. It is a useful signal that the case is ambiguous and should go to a reviewer or another step in the workflow.
If I ask whether an article makes its search intent clear, Jev is not giving that article a general quality score. It is estimating how likely the answer to that one question is yes. The official documentation on reading a Noul answer goes into more detail about this distinction and about choosing thresholds.
What is Jeff?#
Jeff is a CLI. It reads files, evaluates the rules that apply to them, and prints the result. It does not edit the files it checks or apply automatic rewrites. Their contents remain unchanged, although checks may create a local cache.
I built it as a middle step between deterministic tooling and a manual review. A compiler still handles syntax. A test still checks a behaviour that has already been specified. Jeff is useful when the question is closer to this one:
Does what I see in this file match the responsibility its name and context suggest?
Rules are YAML fragments and the project configuration lives in jeff.toml. That makes it possible to choose which directories to inspect, which paths to exclude, and which external rule files to load.
The basic local check is:
jeff check .For a CI job or a coding agent, JSON is usually more useful:
jeff check --output-format json .Authentication can use TYPESAFE_API_KEY or the operating system keyring through jeff auth login. Installation, configuration, and release updates are documented in the Jeff README on GitHub. Prebuilt binaries are available on the releases page.
A check has three useful outcomes: pass, violation, and inconclusive. The rule defines two thresholds. Below the first threshold the check passes; at or above the second there is a violation. The range between them is left for review. Treating every uncertain answer as a failure would make the CLI noisy very quickly.
The same workflow also works for prose. Jeff does not need a special mode for a Markdown article; the rule defines which files it can inspect and what question it should ask. That makes content review a useful example, so the next rule checks a basic SEO concern instead of a code property.
A custom SEO rule#
Semantic rules do not have to inspect functions and classes. A Markdown document is a valid state too. Jeff normally avoids sending documents and configuration files as source input, so a rule that intentionally reads Markdown has to opt in with allow-non-source: true.
A useful SEO check is not “does this page contain enough keywords?”. That is too vague and tends to reward the wrong behaviour. A better first question is whether the article makes its search intent clear in the opening sections: can the intended reader tell what problem the page is going to solve?
The following rules/seo.yml fragment defines that check:
family: SEO
rules:
- code: SEO001
name: search-intent-clarity
message: >-
The article may not make its search intent explicit; ask the writing
agent to clarify the problem, reader, and expected outcome.
scope: file
files:
include: ["**/*.md"]
allow-non-source: true
question:
type: noul
instructions: >-
Evaluate whether the document leaves the problem it solves implicit in
its opening sections, relying only on generic phrasing or repetition of
the main keyword.
criteria:
"true": >-
The intent remains vague: the opening does not identify the reader's
problem or present a concrete outcome or answer.
"false": >-
The intent is clear: the text identifies the reader's problem and
presents a concrete outcome or verifiable answer.
decision:
pass_below: 0.30
fail_at_or_above: 0.75The minimum project configuration needed to load the fragment is:
rule-files = ["rules/*.yml"]I deliberately kept this rule narrow. Title, search intent, structure, and information quality are related, but they are not the same check. If one question tries to measure “SEO” as a whole, it becomes hard to understand what the writer should change when it fails.
What can Jeff do?#
The SEO rule above is just one example. Jeff is useful whenever a review question can be stated clearly and the rule can describe what counts as a violation.
For source code, a rule can look for a failure path that leaves state inconsistent, a relevant side effect hidden behind an innocent-looking interface, or a function whose name contradicts its observable behaviour. These are the kinds of checks included in Jeff’s current built-in catalogue.
For documentation and other Markdown content, a rule can check whether the introduction answers the reader’s question, whether important limits are missing, or whether the same claim is repeated without adding evidence. The point is not to assign a general quality score to a document. It is to isolate one review question and make it repeatable.
Jeff also works as a handoff point for an agent. A JSON result can contain the rule code, the message, and the affected path. An agent can use that information to propose a rewrite while preserving facts, caveats, and the author’s voice. The rewrite stays a separate decision.
In CI, the same output can support different paths: pass a clear result, stop on a violation, and send an inconclusive result to a human reviewer. The thresholds belong to the project, because the cost of a false positive is different for a code change, a published page, and an automated action.
That describes what Jeff can flag, not what it can prove. The boundary is important.
What Jeff does not do#
Jeff is not a compiler, an end-to-end test suite, or a complete SEO audit. It does not inspect Search Console by itself, guarantee a ranking, or turn a probability into a fact.
Its job is smaller and more practical: make a review question explicit and repeatable. The question still has to be written well. The threshold should reflect the cost of a false positive. An uncertain result should be allowed to stop for human review instead of silently becoming an automatic action.
This is also how TypeSafe presents Jev: combine its decisions with deterministic logic in the surrounding program. The useful starting points are the Jev introduction, the official quick start, the guide to question primitives, and the API documentation.
Try your own rules#
If you want to inspect the implementation, install Jeff, or propose a rule, start with Jeff on GitHub. The project is intentionally small: a CLI, a rule catalogue, and an output format that can be read by a person or consumed by another agent.
For me, the interesting part is not adding another automatic score to a workflow. It is taking a vague question, “is something wrong here?”, and turning it into a concrete, repeatable rule that is also honest about when it is unsure.