AI Anonymous / Practical field guide / November 10, 2026

Same answer? Look closer.

An exact-match check rejects an answer that looks right. Before you rewrite the prompt, inspect what your scorer compared.

In this field note · 6 sections
  1. Reproduce the mismatch
  2. Compare both answers in NFC
  3. Write down what counts as equal
  4. Keep the original answers
  5. Read the Python and Unicode references
  6. Run the lab or share the field note

Imagine a test that asks an assistant to copy a storefront name from a short note. Your reference says “Café.” The answer looks like “Café,” but your scorer returns false. The reference ends in one code point for é. The answer ends in e followed by a combining acute accent. Those sequences are canonically equivalent in Unicode.

What we ran. Five fictional answer pairs and four automated checks on Windows with CPython 3.13.15 and Unicode data 15.1.0. The lab runs offline using the Python standard library.
AI Anonymous poster reading “Same answer? Look closer.” Two rough-edged paper slips each read “Café” in black letterpress type. A brass loupe, metal type blocks and a worn wooden tray sit on a warm cream work surface. The footer reads JABSYSTEMS.IO.
Original AI-generated illustration of two letterpress proofs that look alike.

01 / Reproduce

Reproduce the mismatch

Extract the practice ZIP and open PowerShell in the extracted folder. Use Python 3.10 or later. The first command prints a row per fictional answer pair. The second verifies the expected results and checks how the policy handles case, trailing whitespace, and compatibility characters.

python .\answer_check.py
python .\test_answer_check.py

Compare the JSON with observed-results.json. Raw comparison matched one of five pairs. NFC comparison matched three. The two additional matches came from different code-point sequences for “Café” and “résumé.” Missing the accent in “Café” still failed. So did substituting the plain digit 1 for ①.

02 / Inspect

Compare both answers in NFC

Run this in a Python shell. The raw comparison returns False. The endings are U+00E9 and U+0065 U+0301. NFC (Normalization Form C) brings both strings to the same canonical representation, so the second comparison returns True.

from unicodedata import normalize

expected = "Caf\u00e9"
answer = "Cafe\u0301"

for value in (expected, answer):
    print(" ".join(f"U+{ord(char):04X}" for char in value))

print(expected == answer)
print(normalize("NFC", expected) == normalize("NFC", answer))

Normalize both operands at the comparison boundary. The compare() function keeps the raw verdict alongside the NFC verdict so you can see where the policy changed the result. NFC preserves the distinction between ① and 1. NFKC applies compatibility mappings as well, including that conversion. Choose the form that matches your rubric.

03 / Make it your own

Write down what counts as equal

  1. Copy cases.json to my-cases.json. Append a case with ID extra-space, expected "North", and answer "North " with a trailing space.
  2. Run the command below. The new pair should return false in both columns. The base five cases stay available for rerunning the tests.
  3. Pick a text field your rubric scores, such as a storefront name. Label one pair that should match and one that must remain distinct before changing the comparison. Decide separately how to handle whitespace, case, punctuation, and compatibility characters.
python .\answer_check.py .\my-cases.json

The practice README includes an optional assistant prompt for proposing counterexamples to your rubric.

Boundaries

Keep the original answers

Store the raw answer, reference, and comparison policy with the evaluation so someone can reproduce the verdict. If the task requires exact code points or bytes, as in a literal transcription test, use a comparison that enforces that requirement. NFC may change string length, so keep offsets tied to the original if you need to highlight a span. The examples are authored fixtures. The checks cover the scorer only. Model accuracy, the optional prompt, production eval integrations, and performance remain untested.

Primary sources / Checked November 10, 2026

Read the Python and Unicode references

Downloads / Practice and social files

Run the lab or share the field note

Explore more AI Anonymous field notes →

Share this post.

Found this useful? Pass it along. Download the artwork, add your own take, and tag @jabsystems.io so I can see your post.