Why first-hit Wikidata search breaks AI agents

A lot of AI agents identify things the same way: search Wikidata for the name, take the first result, move on. It is one line of code and it usually works. We measured how often it does not, and what happens when it fails.

The one-line identity step

Here is the pattern, in some form, in many agent tools and RAG pipelines:

hits = requests.get("https://www.wikidata.org/w/api.php", params={
    "action": "wbsearchentities", "search": name,
    "language": "en", "type": "item", "format": "json"}).json()["search"]
entity_id = hits[0]["id"] if hits else None

wbsearchentities is a good search box. It matches labels and aliases and ranks the results. It never sees the sentence the name came from, so the first hit is the most likely meaning of the name in general, not in your text.

Try it on this sentence:

Orange reported higher mobile revenue this quarter.

The first hit for “Orange” is Q187796, a commune in Vaucluse, France. The company is Q1431486. An agent that takes the first hit now believes a town in Provence publishes quarterly results.

How often it goes wrong

We ran the six gold sets from the resolve() repository: 126 short sentences in English, Italian, French, German, Spanish and Portuguese, with 351 marked names and the correct Wikidata ID for each. We gave the same names to the first-hit lookup and to resolve(), our entity resolution API, which reads the sentence before it decides. These are our own development sets, not held-out test data, so they favour resolve(); the first-hit errors below do not depend on that.

The first hit is right 82% of the time. That is why the pattern survives: most names in most sentences have one obvious meaning. The trouble is the other 18%, and where it lands.

What the 61 errors look like

Almost every error is a famous meaning beating the right one:

French was the hardest language for the first hit (70.3 F1): rivers came back as departments and communes, the Louvre as a metro station.

Why a wrong ID is worse than no ID

Nothing crashes when an agent picks the wrong Liverpool. The wrong ID is stored. It ships in the JSON-LD on a page, lands in a knowledge graph, and gets recalled from the agent’s memory next week. Every system downstream repeats it, and nobody notices, because a plausible wrong answer looks exactly like a right one.

That is why the useful property is not only being right more often. It is being able to say “I can’t tell”. A search box always returns a first hit. Across 351 names, the first-hit lookup declined twice, both times because no label matched the exact name (“Fighting Temeraire”, “Zürcher Flughafen”), not because it doubted the answer.

What reading the sentence fixes, and what it does not

resolve() got 39 of the first hit’s 61 wrong answers right. Nearly all are cases where the sentence clearly points away from the famous meaning: a football score, a river with countries around it, planets in orbit.

It is not free of errors. On 8 names the first hit was right and resolve() was not: 6 wrong answers, most of them a place one level off, and 2 refusals:

• “Liverpool is a city in the north west of England” came back with England as the United Kingdom.
• “La Juventus ha battuto il Milan allo Stadium di Torino” came back with Torino as the football club, not the city.
• “Oscar Niemeyer projetou a Catedral de Brasília no Distrito Federal” came back as Mexico City.

22 names beat both, among them “Jordan won six championships with the Chicago Bulls” and “Mozart” in a German sentence about Salzburg. And resolve() declined 7 times, which is either a missed answer or a saved mistake depending on the name.

These are our own gold sets and they are small, 37 to 100 names per language, so treat differences of a few points within one language as noise. The sets, the scoring rule and the harness are open. Run them yourself, and send us corrections: at least one of our gold labels is debatable.

What to do instead of taking the first hit

In order of effort:

1. Pass the type. If you know the mention is a person or a football club, filter candidates by type before choosing. It should remove the given-name and family-name errors above (Shakira here; Turner and Aida in the full results).
2. Read the sentence. Score the candidates against the context, not just the label.
3. Allow “no”. Give your agent a way to receive “unresolved” and do something sensible: ask, flag, or leave the name unlinked. Never let it create a record because the lookup came back empty or unsure.

resolve() does all three in one call. It returns a stable Wikidata ID or your own ID, or unresolved with a reason, and it can check your own knowledge graph before Wikidata.

r = requests.post(f"https://api.wordlift.io/accounts/{ACCOUNT}/resolve",
    headers={"Authorization": f"Key {WL_KEY}"},
    json={"text": "Orange reported higher mobile revenue this quarter.", "language": "en"})

Try the three agent demos on your own text, no sign-up: wordlift.io/resolve/demos. How resolve() compares with other entity linking APIs on a public benchmark: Entity linking APIs compared.