We structure content on websites to let people find what they want. There is always beauty in understanding how things are organized on an e-commerce website. In SEO, when we are involved with user experience, our ultimate goal is to find the truth (the essence of any webpage) and to render the intent. Peter Morville will say that, when organizing content, we create environments for understanding<\/em>. <\/p>\n\n\n\n Here is the outline for this article. If you prefer to jump right into code here is the Colab<\/a>.<\/p>\n\n\n\n We will create recommended links using a small set of commands in Python and a minimal amount of deep learning. Before anything else, let\u2019s review two essential aspects:<\/p>\n\n\n\n In layman’s terms, we need a function, a simple system that, when we input an X (let’s say the title of a category), will give as a Y (the set of the top 4 or 5 related categories).<\/p>\n\n\n\n E-Commerce category pages tend to have breadcrumbs and hierarchical links (with the entire list of categories and subcategories). Instead, we want to add a navigational element that can traverse the hierarchical tree in a meaningful way.<\/p>\n\n\n\n Links on category pages are usually limited to breadcrumbs trail and taxonomy-based filters (the characteristics of the set of products). Recommending links brings the following SEO benefits:<\/p>\n\n\n\n Moreover, on the business side, having the ability to recommend categories helps the shop owner improve the business relevancy of search by:<\/p>\n\n\n\n There are various examples of internal links on e-commerce (and non e-commerce) websites, let\u2019s review a few of them:<\/p>\n\n\n\n Amazon uses a block of 6 elements that, as we can see, tend to broaden (keyboard, gaming pc), narrow (gaming keyboard 60 percent), or horizontally expand (gaming monitors, gaming mouse) the initial search.<\/p>\n\n\n\n In Alibaba, the textual relatedness is weaker. The semantic jump between men’s coats and dog coats is extreme. Besides the questionable association between men and dogs, the focal points remain clothing for men. The design is essential.<\/p>\n\n\n\n Kijiji labels it as “popular,” and its generation process cannot detect that \u200b\u200blawn mower and \u200b\u200blawnmower are synonyms. At least in this example, it tends to narrow the search intent. The terms being used are keywords and not proper category names.<\/p>\n\n\n\n Artsper introduces the concept of search refinement by characterizing the block with “Refine your search”. The navigation elements help us move in multiple directions without clear sorting criteria. This is per s\u00e9 not a bad thing, quite the opposite, we perceive a sense of freedom, and we can quickly skim through the terms. Visually the terms are presented as refinement chips.<\/p>\n\n\n\n Here is how things work on Google Search, Google Images and Google Arts and Culture. This is a random exploration of various types of widgets that should give us some ideas on how things can be implemented.<\/p>\n\n\n\n Being primarily an image-centric medium, Google Images helps us with the use of images, in this specific occurrence, to broaden (cake) or to expand the search (meat, pecan).<\/p>\n\n\n\n As seen, with this limited selection of the different implementations, we can highlight the following:<\/p>\n\n\n\n Generative AI<\/a> can help to create internal links. The workflow is simple and provides a base for understanding how things work behind the scenes. You can find the Google Colab here<\/a>.<\/p>\n\n\n\n I used as a reference website fila.com a sportswear manufacturer originally from Biella in the north part of Italy. They are not clients of ours, and here is what we will do:<\/p>\n\n\n\n The UX of the website is clean and the site lacks a related search widget.<\/p>\n\n\n\n We will parse the sitemap and extract the list of category pages by removing any page that ends with “.html” (as this characterizes product pages) and a series of other pages that don’t correspond with the product listing (i.e., “news”, “about-” and so on).<\/p>\n\n\n\n We will then extract from each page a minimum set of information, including the short intro text below the page’s title, the breadcrumbs, the meta description and the page’s title. In the snippet below, we can see that we are running a custom extraction using xpath<\/a>. If the intro text is missing, we can rely on the other textual elements of the page. We will need to be very careful in removing oddities or other terms that might compromise the search.<\/p>\n\n\n\n Advertools will store the captured data on fl_category_crawl.jl. We might keep this file so that information will be re-used for the next crawl. Here we can see the result of the extraction.<\/p>\n\n\n\n After extracting the title of the page and the short intro text, we will analyze the breadcrumbs and create a data frame. This helps us gain an understanding of the site structure. We might reuse this data frame while composing the final list of suggestions. We might, for example, decide to exclude a link already in the Breadcrumbs for that page. Repeating the same link can be annoying, especially if the related search widget is displayed close to the Breadcrumbs.<\/p>\n\n\n\n To clean up the captured text, I have used spaCy and a list of site-specific stopwords. We will also remove special characters, numbers, and other oddities. I decided to lemmarize terms; this means bringing back the base or dictionary form of a word. We will create embeddings<\/a> afterward and I want consistency from the beginning. This will help as we have a limited amount of text available.<\/p>\n\n\n\n Optionally I have also prepared the code to capture data from Google Search Console. You can take advantage of the list of queries behind each page and the number of clicks. Queries can be extremely valuable as you might decide to use them instead of the title of the pages.<\/p>\n\n\n\n Let me give you an example. We might have a long title like \u201cMen’s Casual Sneakers + Athletic Shoes | FILA\u201d; in this case it would be better to display something more compact like \u201cMen’s Sneakers\u201d. You will need to authenticate on GSC to extract the data. The information will be merged with the crawl dataset by running a loop with all the crawled urls. <\/p>\n\n\n\n Here comes the AI bit of this workflow. We are going to use the SentenceTransformers<\/a> (SBERT) library. This open-source library allows us to replace the underlying model and choose the best models that fit our needs<\/a>. Models are available in the HuggingFace Model Hub. We can also eventually train our model<\/a> to improve the performance further.<\/p>\n\n\n\n We will index the text extracted from each page and use the title as a query. We will use the native semantic search functionality of SBERT.<\/p>\n\n\n\n The idea behind is as simple as encoding the text in the \u201cclean text\u201d column and comparing it, within the same vector space, with the embedding of the title.<\/p>\n\n\n\n Once we run the same query on the complete list of category pages, we will get a new data frame that, for each page, will provide a list of recommended links.<\/p>\n\n\n\n Now, depending on the CMS, you can change the output format and get ready to publish it. In our case, we will write the data back into the Knowledge Graph<\/a> and send it to the CMS using a REST interface (i.e., https:\/\/api.wordlift.io\/data\/https\/www.example.com\/en-us\/category\/my-category-page). In the Colab, the data is stored in a JSON file, and you can explore it directly from the notebook.<\/p>\n\n\n\n We will import the data into the knowledge graph and present it to search engines using structured data markup. A related search widget is a navigation site element; we can use the schema markup<\/a> for SiteNavigationElement, a subclass of the WebPageElement.<\/p>\n\n\n\n This markup will help search engines understand how things are connected.<\/p>\n\n\n\n One of the biggest challenges when adopting AI into SEO workflows is the design of a lifecycle that will scale across sites of different sizes and with other characteristics.<\/p>\n\n\n\n Working on Colab helps me envision how things should work; I can easily experiment with new ideas, but at some point, I will need to run the inference on sites with potentially thousands of category pages. Also, I need to have the flexibility to replace the model fine-tuning it. Even more importantly, on e-commerce sites, I want to be able to work with multiple modalities (text + images). On large properties like fila.com, the textual content is very well optimized, and I can easily rely on it but on smaller sites I will need to combine features from text with features extracted from images.<\/p>\n\n\n\n To do that, we partnered with Jina AI<\/a>. As a quick introduction here, I have added the code to replicate the same neural search provided by SBERT using Jina NOW<\/a> text-to-text search functionality. As you will see in the code, we will connect to an end-point on the Jina Cloud infrastructure and run queries there. This means having a dedicated pool of machines working on the generation of the embeddings and on running the neural search. The load will be distributed, and we will be able to autoscale resources as we increase the size of the dataset.<\/p>\n\n\n\n On the SEO front, there are other essential analyses to be done. For proper link sculpting, we will need to prevent any form of cannibalization and also evaluate how to distribute links equally. Moreover, based on the website, I want to take into account the most representative products for a category and add the support for the analysis of product images and product descriptions.<\/p>\n\n\n\n On the tech side, Jina NOW has just recently launched and we are still working with the team at Jina AI to improve how things work behind the scenes. We want to be able to control the re-ranking directly inside Jina\u2019s flow.<\/p>\n\n\n\n Happy SEO-automation!<\/p>\n\n\n\n PDP stands for Product Detail Page and represents the webpage that describes a single product. PLP stands for Product Listing Page and refers to a page that lists a category of products.<\/p>\n\n\n\n Jina AI is a neural search framework to build scalable deep learning search applications. In this blog post we use Jina NOW, the simplest way to use semantic search in a distributed environment.<\/p>\n\n\n\n To optimize your e-commerce SEO<\/a> strategy and get more conversions you can:<\/p>\n\n\n\n <\/p>\n\n\n\n\n","protected":false},"excerpt":{"rendered":" Learn how to automate internal link building for your e-commerce category pages by creating related search widgets. <\/p>\n","protected":false},"author":6,"featured_media":22005,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"wl_entities_gutenberg":"","_wlpage_enable":"","footnotes":""},"categories":[28,4202],"tags":[],"wl_entity_type":[30,3303],"coauthors":[],"class_list":["post-21979","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-world-summit-ai","category-e-commerce","wl_entity_type-article","wl_entity_type-faq-page"],"acf":[],"yoast_head":"\nSEO Advantages Of Related Search Links<\/h2>\n\n\n\n
How A Related Search Widget Should Look Like<\/h2>\n\n\n\n
![\"\"](\"https:\/\/wordlift.io\/blog\/en\/wp-content\/uploads\/sites\/3\/2022\/06\/related-search-amazon-gaming-keyboad-1024x217.png\")
![\"\"](\"https:\/\/wordlift.io\/blog\/en\/wp-content\/uploads\/sites\/3\/2022\/06\/related-search-alibaba-men-coats-1024x88.png\")
![\"\"](\"https:\/\/wordlift.io\/blog\/en\/wp-content\/uploads\/sites\/3\/2022\/06\/popular-kijiji-outdoor-garden-1024x53.png\")
![\"\"](\"https:\/\/wordlift.io\/blog\/en\/wp-content\/uploads\/sites\/3\/2022\/06\/refine-search-artsper-pop-art-paintings--1024x149.png\")
Related Search On Google Properties<\/h3>\n\n\n\n
![\"\"](\"https:\/\/wordlift.io\/blog\/en\/wp-content\/uploads\/sites\/3\/2022\/06\/google-images-apple-pie.jpg\")
![\"\"](\"https:\/\/wordlift.io\/blog\/en\/wp-content\/uploads\/sites\/3\/2022\/06\/google-search-sunglasses.png\")
![\"\"](\"https:\/\/wordlift.io\/blog\/en\/wp-content\/uploads\/sites\/3\/2022\/06\/google-culture-and-art-giorgio-de-chirico.jpg\")
The UX Of A Related Search Widget<\/h3>\n\n\n\n
Creating Internal Links On Scale For An E-Commerce Website<\/h2>\n\n\n\n
![\"\"](\"https:\/\/wordlift.io\/blog\/en\/wp-content\/uploads\/sites\/3\/2022\/06\/search-fila.png\")
<\/figure>\n\n\n\n1. Accessing the sitemap using Advertools<\/h2>\n\n\n\n
![\"\"](\"https:\/\/wordlift.io\/blog\/en\/wp-content\/uploads\/sites\/3\/2022\/06\/list-of-category-pages-advertools.png\")
<\/figure>\n\n\n\n2. Extract textual elements from each page<\/h2>\n\n\n\n
![\"\"](\"https:\/\/wordlift.io\/blog\/en\/wp-content\/uploads\/sites\/3\/2022\/06\/texal-elements.png\")
<\/figure>\n\n\n\n![\"\"](\"https:\/\/wordlift.io\/blog\/en\/wp-content\/uploads\/sites\/3\/2022\/06\/crawl-advertools.png\")
<\/figure>\n\n\n\nParsing the navigational breadcrumbs<\/h3>\n\n\n\n
![\"\"](\"https:\/\/wordlift.io\/blog\/en\/wp-content\/uploads\/sites\/3\/2022\/06\/breadcrubs_1.png\")
<\/figure>\n\n\n\n![\"\"](\"https:\/\/wordlift.io\/blog\/en\/wp-content\/uploads\/sites\/3\/2022\/06\/breadcrubs_2.png\")
<\/figure>\n\n\n\n3. Extract queries from GSC (optional)<\/h2>\n\n\n\n
![\"\"](\"https:\/\/wordlift.io\/blog\/en\/wp-content\/uploads\/sites\/3\/2022\/06\/data-from-GSC.png\")
<\/figure>\n\n\n\n4. Computing semantic similarity<\/h2>\n\n\n\n
![\"\"](\"https:\/\/wordlift.io\/blog\/en\/wp-content\/uploads\/sites\/3\/2022\/06\/semantica-search-using-SBERT.png\")
![\"\"](\"https:\/\/wordlift.io\/blog\/en\/wp-content\/uploads\/sites\/3\/2022\/06\/semantic-search_2.png\")
<\/figure>\n\n\n\n5. Preparing the output file<\/h2>\n\n\n\n
![\"\"](\"https:\/\/wordlift.io\/blog\/en\/wp-content\/uploads\/sites\/3\/2022\/06\/linking-category-page.gif\")
The site navigation schema markup<\/h3>\n\n\n\n
![\"\"](\"https:\/\/wordlift.io\/blog\/en\/wp-content\/uploads\/sites\/3\/2022\/06\/schema-markup-example.png\")
6. Scaling the workflow – a better AI lifecycle using NOW<\/h2>\n\n\n\n
![\"\"](\"https:\/\/wordlift.io\/blog\/en\/wp-content\/uploads\/sites\/3\/2022\/06\/Jina-AI-NOW.png\")
Conclusions And Future Work<\/h2>\n\n\n\n
Additional Questions<\/h3>\n\n\n\n
What are PDPs and PLPs in e-commerce websites?<\/h4>\n\n\n\n
What is Jina AI?<\/h4>\n\n\n\n
How to optimize e-commerce website SEO?<\/h4>\n\n\n\n