Log similarity

Log similarity is part of the Some numbers-module of the Finds tab in the Profile Stats. It is an indication of how much the user's logs differ, a high score mean their logs have a lot of repeated content, a low score mean they are more diverse.

How does it work?

It is the content of the logs, not the actual logs that are similar. If the number is 57% that does not mean that 57% of the users logs are identical, but that the content of the logs coming after the former one has an average of 57% similarity.

The code is not aware of the language or words, it is based on characters only. So, if the user writes "ab" in one log and "cd" in the next log, those logs will have 0% similarity. So logging the three logs "ab", "cd" and "cd" in a row would give 50% log similarity (0% similarity for "ab" and "cd", 100% similarity for "cd" and "cd". (0+100)/2=50 ), posting another log of "ab" after that would result in 33% ((0+100+0)/3=33). Since the code is not based on words it does not matter if some words are very common in the language of the user since all languages will have characters that are more common, as an example the words "Rabbit" and "Tea" have a 22% similarity.

What is considered low log similarity?

If the number is lower than most others, then the user is doing a better job of writing varying logs than the majority. That being said, everything below 50% is fairly low. It is definitely possible to achieve a high word count and a low log similarity at the same time. In theory one could reach 0% but that would definitely requiring aiming for it, it's not likely to happen by chance.

Math and programming

The exact math can be learned by studying documentation and source code. The algorithm rely on PHP's open source function similar_text.

In an ideal world the Levenshtein algorithm would be used, and at first it actually was. But it is much slower and it scales worse so with long logs it is just too slow. The similar_text-function is more linear in its scaling.