{"id":31643,"date":"2026-09-18T16:28:31","date_gmt":"2026-09-18T14:28:31","guid":{"rendered":"https:\/\/wordlift.io\/blog\/en\/?p=31643"},"modified":"2026-09-18T16:28:38","modified_gmt":"2026-09-18T14:28:38","slug":"content-analysis","status":"publish","type":"post","link":"https:\/\/wordlift.io\/blog\/en\/content-analysis\/","title":{"rendered":"Content Analysis v3: From the Semantic Web to the Entity Layer of AI"},"content":{"rendered":"\n

Fifteen years after IKS<\/a> and Apache Stanbol, entity resolution is becoming core infrastructure for AI.<\/em><\/p>\n\n\n\n

Generative AI has made producing text remarkably easy. Understanding what that text is actually about<\/em> remains much harder.<\/p>\n\n\n\n

Take a simple sentence:<\/p>\n\n\n\n

\n

Apple opened a new store in Paris.<\/p>\n<\/blockquote>\n\n\n\n

A language model reads it effortlessly. A system that wants to build reliable knowledge from it has to answer more precise questions. Is Apple<\/em> the company, the fruit or the record label? Is Paris<\/em> the French capital or the town in Texas? What types of things are they, and which identifiers represent them? And, most importantly: when should the system admit that it does not have enough evidence to decide?<\/strong><\/p>\n\n\n\n

These questions sound very current in the age of AI agents. For us they are not. We have been working on them for more than fifteen years. Content Analysis v3 is the latest chapter of a journey that began before WordLift existed as a company, with the European IKS project, Apache Stanbol, DBpedia<\/a> and a group of brilliant people who taught us to see content not as strings of words but as a network of identifiable things.<\/p>\n\n\n\n

What has changed is the scale of the problem. Entity resolution is no longer only about enriching a CMS or adding structured data to a page. It increasingly determines how AI systems reconstruct the world. Which brings us back to the oldest question in our field: what are we talking about?<\/strong><\/p>\n\n\n\n

It started with IKS<\/h2>\n\n\n\n
\"\"
Rupert Westenthaler and myself at the new Redlink office in Salzburg a few weeks ago \ud83d\udc4f<\/figcaption><\/figure>\n\n\n\n

In 2009 the European Commission launched the Interactive Knowledge Stack (IKS), a four-year FP7 research project coordinated by Salzburg Research. Its goal sounds remarkably contemporary: give content management systems a layer of semantic intelligence, so that software could operate on knowledge and not only on documents.<\/p>\n\n\n\n

From that work came Apache Stanbol, an open-source framework for semantic content management. Stanbol brought together what we would now call an AI enrichment pipeline: language detection, named entity recognition<\/a>, entity linking, controlled vocabularies, semantic indexing, reasoning.<\/p>\n\n\n\n

In 2012 Olivier Grisel and Rupert Westenthaler presented Automated Linking Data with Apache Stanbol<\/em> at WWW2012. Rupert would later become my partner in Redlink; at the time, he and Olivier were among the people teaching us how to think rigorously about semantic content analysis. One phrase from that work has stayed with me ever since:<\/p>\n\n\n\n

\n

Get to know your content.<\/strong><\/p>\n<\/blockquote>\n\n\n\n

It sounds simple. It is not. The point was never to detect the word Paris<\/em>. It was to determine that the text referred to Paris, France, attach a stable identifier to it, understand what kind of thing it was, and connect it to a wider graph of knowledge. That shift, from strings to things, changed the way we thought about content.<\/p>\n\n\n\n

<\/p>\n\n\n\n

From Stanbol to WordLift<\/h2>\n\n\n\n

We were not watching from the outside. By 2012 some of the early technology that would become WordLift was already being contributed back to Apache Stanbol: language processing with FreeLing, entity recognition against Freebase, enrichment with DBpedia, and a refactoring layer that translated extracted knowledge toward Schema.org.<\/p>\n\n\n\n

The technologies have changed almost completely since then. The architectural idea has not: content analysis should produce reusable knowledge, not annotations trapped inside an NLP service.<\/strong> An entity needs an identity. That identity must be resolvable and reconcilable with other knowledge, and publishers and organizations should be able to build their own knowledge on top of it.<\/p>\n\n\n\n

This is why we still support the DBpedia project today. Open knowledge infrastructure and shared identifiers remain fundamental to an interoperable web. This is not nostalgia for the Semantic Web. It has become more relevant in the age of AI.<\/p>\n\n\n\n

The world has come back to entities<\/h2>\n\n\n\n

For several years the dominant AI narrative moved in the opposite direction. Embeddings could encode meaning, vector databases could retrieve similar passages, LLMs could extract facts directly from text. Perhaps explicit entities and identifiers were no longer necessary.<\/p>\n\n\n\n

Then we started looking inside modern AI systems, and the entities were there.<\/p>\n\n\n\n

In our research on ChatGPT’s web interface, we observed OpenAI carrying structured entity objects<\/a> alongside conversational responses: people, organizations, events, products. More recently, studying local search inside ChatGPT, we found something even more concrete. ChatGPT resolves local businesses into structured objects with an internal entity_lookup_id<\/code> (of the form lbe1_...<\/code>), carrying name, coordinates, address, category, website, ratings and reviews. We were able to connect these objects with externally verifiable Google place identities. And we saw the resolved entity being carried forward into the experience:<\/p>\n\n\n\n

query \u2192 entity resolution \u2192 entity panel \u2192 follow-up retrieval<\/strong><\/p>\n\n\n\n

These are observations of the current ChatGPT web experience, not a public OpenAI API contract. But the architectural direction is clear. The question for an AI system is no longer only which document should I retrieve?<\/em> It is increasingly which real-world entity does this document refer to?<\/em> Once that identity is established, it becomes part of the interface and part of the context for whatever happens next.<\/p>\n\n\n\n

What Rupert and Olivier were solving inside a semantic CMS in 2012 is now happening inside one of the world’s most advanced AI interfaces. The scale is different and the models are vastly better, but the problem is familiar: language has to be reconciled with identity.<\/strong><\/p>\n\n\n\n

Enter Content Analysis v3<\/h2>\n\n\n\n

Content Analysis v3 is our newest implementation of that idea, and one of the foundations of WordLift’s Context Engine: a semantic-resolution layer that turns unstructured language into entities with stable identities, types and evidence, ready to enter a Knowledge Graph or an AI workflow.<\/p>\n\n\n\n

The production system works across eight languages (English, Italian, French, German, Spanish, Portuguese, Japanese and Chinese) against a Knowledge Base of approximately 13.67 million Wikidata entities, 34 million surface forms and 13.67 million semantic vectors, plus almost 11 million DBpedia abstracts covering more than 5 million entities.<\/p>\n\n\n\n

It still starts from a fundamental distinction: finding an entity is not the same as knowing what it is. Named Entity Recognition tells us that Apple<\/em> is an Organization and Paris<\/em> is a Location. Useful, but insufficient for a graph. A graph needs identity, and that is the job of Named Entity Disambiguation: Apple<\/em> is Wikidata Q312<\/code>, Paris<\/em> is Q90<\/code>. A label tells us that something looks like an organization. An identifier tells us which<\/strong> organization.<\/p>\n\n\n\n

That stable identity is what lets knowledge discovered on one page connect with data found elsewhere: reconciled across languages, joined with proprietary data, linked to DBpedia, mapped into Schema.org, used by retrieval, or preserved as context for an agent.<\/p>\n\n\n\n

So Content Analysis v3 treats recognition and disambiguation as a single system:<\/p>\n\n\n\n

content \u2192 language detection \u2192 entity recognition \u2192 candidate retrieval \u2192 reranking \u2192 abstention \u2192 entity identity<\/strong><\/p>\n\n\n\n

The output is not a list of keywords. It is grounded context.<\/p>\n\n\n\n

How it resolves identity<\/h2>\n\n\n\n

Recognition starts with GLiNER on a multilingual DeBERTa architecture. The harder stage comes afterward: for each mention, the system has to find possible identities among millions of entities and decide which candidate, if any, the author actually meant.<\/p>\n\n\n\n

Candidate retrieval combines two kinds of evidence. Lexical: a search across millions of labels and aliases for exact and approximate matches. Semantic: BGE-M3 embeddings provide vector retrieval over the entity collection, so we can find candidates even when the text and the canonical label do not match.<\/p>\n\n\n\n

The candidates are then evaluated by a multilingual BGE reranker cross-encoder. Unlike a vector similarity score, the cross-encoder reads the mention in its surrounding context jointly with the candidate’s label, description and encyclopedic information. In production we deliberately keep a large candidate pool, because at this scale eliminating the right candidate too early is often impossible to repair later.<\/p>\n\n\n\n

Finally, the system combines that semantic evidence with lexical priors, entity typing and other signals before deciding what to return. And one possible answer is, deliberately, nothing.<\/p>\n\n\n\n

A false edge is worse than no edge<\/h2>\n\n\n\n

This is probably the most important principle in Content Analysis v3: the system is allowed to abstain.<\/p>\n\n\n\n

A missing relationship can be enriched tomorrow. A wrong one quietly contaminates everything built on top of it: retrieval, recommendations, analytics, generation, agent reasoning.<\/p>\n\n\n\n

This led us to separate evidence from confidence. A model can produce a high numerical score and still be wrong. Worse, apparent confidence can shift simply because the candidate set changed. So Content Analysis asks two distinct questions: which candidate should rank first?<\/em> and do we have enough evidence to accept it?<\/em> Signals that are useful for ranking do not automatically count as evidence for accepting an identity.<\/p>\n\n\n\n

That may look conservative for an NLP application. For a Context Engine it is essential.<\/p>\n\n\n\n

The Knowledge Base is part of the model<\/h2>\n\n\n\n\n