Automated Related Posts - How do I add related posts for my blog?

The Ahrefs tool has told me that my blog posts only have one internal follow link, which is a problem. Only having one link to a page makes it harder for users and search bots to discover your pages. I like to automate processes where possible, and related posts are no different. After much reading and not wanting to manually add links to each post with content links in the body of the post, I decided to add a section at the bottom for related posts. Using the Jaccard algorithm as a base, all my posts will have related posts by the end of my weekend.

In doing so, I learned about identifying text similarity using a Jaccard implementation with unigrams, bigrams, and trigrams, creating intersections and unions. Prepoccesed text to remove the stop and joining words to improve match quality and remove suffixes to increase accuracy. Now, when I save or create a blog post, it checks for similarities in the title, meta title, and meta description.

Highlighting No Choesion - The Brutal Honesty of Numbers

Blogs should have a theme, speak with any marketing executive and they will talk about finding your niche, my post's similarity scores were revealing. The scores were really low, below 0.02. The first iteration only counted the blog titles which didn't help as it was too limited for finding unions. This highlighted that my posts were not similar, adding the additional text fields the numbers were telling me a story. I haven't been focusing on a niche and have taken a relaxed editorial approach. While the themes are e-commerce, web development, SEO and business opinion they needed to become one.

This post was written in part to tell the tool's development story. It also brings the subject matters together nicely which means it should link and be linked to many posts, which it has.

If I'd manually added content links, I would have crowbarred links into the body of text which isn't helpful and makes for awkward sentences. One of this blog's goals is to flex my writing skills, which would be hampered by doing so. Who likes those context links anyway?

Developing the Blog Tool - Jaccard

Aside from making the same mistake of not capitalising a file name in one bit of code which I picked up in the push to staging rather than breaking the main site this time, the development of the addition went well.

Another goal of this blog is to make a Laravel package for a blog that is good and potentially usable for others. This enhancement is simple and IMHO brings it in line with the basic offerings on other platforms, additions for popular posts are on my radar too!

Implementation of Jaccard, which can be used for finding the similarity in text, using single words, two & three-word phrases, these are described as ngrams, and looking for the overlap produces a score between 0-1. Jaccard is relatively simple and relies on exact matches in the overlaps these could be for any length of phrase, and this implementation allows me to assign different weights, a three-word overlap is worth more than a single word.

Stop words and joining words create noise, and the first run-through didn't handle them at all, this created some strange similarities, especially for the single word overlaps, the eventual implementation removes them, it also has a rule for simple suffix removals with the thinking that it was more likely to pick up similarities. With the scores generated, they were placed in the database and triggered automatically by model observers each time a post is created or updated. For both of these, a configuration file contains an array for stop, joining and modifying suffixes.

Jaccard does have limitations that have to be acknowledged. Spelling mistakes and subtle variations will cause it to miss, and it doesn't account for frequency. In smaller texts, this is less of a problem, but as the text gets longer, it could become an issue.

Limitations of methods should be acknowledged and the implementation is sympathetic to that, it does not attempt to read entire posts. There are better methods of establishing longer text similarity in the posts. The next step could be using Cosine similarity while it doesn't understand meaning, it evaluates frequency which could be more valuable than adding the additional text into the Jaccard function.

Other improvements will include adding a feedback loop to auto-update the weightings used to decide n-grams. A definition of a success metric will be required for that to happen. The success metric that is easy to measure is new pages being indexed, and a baseline click-through rate could be established for the visible links rewarding higher rates as an indication of user engagement.

Do posts Have more than one Link Now?

The purpose of this was not to create a perfect related posts tool in a single weekend, it was to remove some warning lights from the Ahrefs site audit, which is the goal of every SEO person right? Did it? Yes, I'm down to three pages with only one no-follow link, and one of those is the forgotten password page, which I'm not worried about.

What a way to spend the weekend, improving the blog tool!