Log similarity

Log similarity is part of the some numbers-module of the Finds tab in the Profile Stats. It is an indication of how much the user's logs differ, a high score mean their logs have a lot of repeated content, a low score mean they are more diverse.

How does it work?

It is the content of the logs, not the actual logs that are similar. If the number is 57% that does not mean that 57% of the users logs are identical, but that the content of the logs coming after the former one has a average of 57% similarity.

The code is not aware of the language or words, it is based on characters only. So, if the user write "ab" in one log and "cd" in the next log, those logs will have 0% similarity. So logging the three logs "ab", "cd" and "cd" in a row would give 50% log similarity (0% similarity for "ab" and "cd", 100% similarity for "cd" and "cd". (0+100)/2=50 ), posting an other log of "ab" after that would get 33% ((0+100+0)/3=33). Since the code is not based on words it does not matter if some words are very common in the language of the user since all languages will have characters that are more common, as an example the words "Rabbit" and "Tea" have 22% similarity.

What is considered low log similarity?

If the number is lower than most others, then the user is doing a better job of writing varying logs than the majority. That beeing said, everything below 50% is fairly low. It is definitely possible to achive a high word count and a low log similarity at the same thime. In theory one could reach 0% but that would definitely requiring aming for it and it will not happen by chance.

Math and programming

The exact math can be learned by studying documentation and source code. We rely on an open source function that is called similar_text.

In an ideal world we would rather use levenshtein and at first we did! But it is much slower and it scales worse so with long logs it is just too slow. The similar_text-function is more linear in its scaling.