Do chess engines make mistakes? It is not that simple
Chess engines that are lower-tiered will regularly make mistakes or even a blunder. Strong chess engines are susceptible to making inaccuracies if not to their fullest depth, but never seem to make a mistake. The strongest chess computers never commit a mistake or a blunder.
Chess engines have demonstrated their ability to find moves so well that people started to treat them as the gods of chess, that they are entities that could never seem to make a mistake.
That is what I am going to explore in this article. Whether these reputable chess computers can have flaws that we humans cannot fully accept, which is that we could make a mistake. Are engines human enough to have the same characteristic?
When I was just starting to play chess I was under the same impression, that chess engines are just perfect entities that we humans have created.
I am knowledgeable enough to truly answer this question now, which I will use this opportunity to do so in order to clarify some things on this topic. Without wasting too much time, let’s get going.
Weak chess engines regularly make mistakes
To fully answer this question, we need to understand that chess engines are not the same and they have a different level of strength. Some will be better than others because they have the right technology.
Depending on the level of the chess engine it could also make a mistake, a weak chess computer will occasionally (or regularly) play bad moves that even humans can evaluate.
This is the thing with chess engines, it is hard to pinpoint whether they have actually made a mistake since they are so reputable.
We could be more confident with our own evaluations if we are stronger than the chess computer in question. If the level of the engine that we are talking about is on the weaker side (usually an unknown chess engine) then they definitely make mistakes.
I can verify this since I have actually played with a weak chess engine in my own cell phone (if you can’t find one just play against Stockfish 1-4), and I can crush such programs easily.
This doesn’t mean that I just play perfectly, in fact, I play terribly on most occasions against these computers. It is just that they make significantly more mistakes than I do that my own mistakes do not count at all.
If it is a weak chess computer then it is not perfect, it can definitely make a mistake.
Chess engines on moderate strength can make mistakes
Chess engines that are only moderately strong can make mistakes as well, only the established competitive chess computers are considered the ones who cannot seem to make a mistake.
It is not only about the weak chess engines, even computers that are considered above them can occasionally make mistakes as well (and even blunders) but not as frequently.
Competitive chess engines are usually at the center of this topic since they are so strong that even grandmasters take their recommendations seriously.
But if it is only a moderately strong chess engine then it is not perfect and you can definitely beat it if you are strong enough.
Strong chess engines don’t make mistakes
If we are talking about strong chess engines then the answer would be different, at this level the question becomes more serious. The answer would become different as well.
Strong chess engines do miss critical moves on very rare occasions but they never seem to make an actual mistake, all recommendations of a strong chess computer are considered good moves.
There are three levels of making a bad move in chess, it is either a blunder, a mistake, or an inaccuracy. These levels warrant different evaluations since their intensity is different.
If we are talking about mistakes specifically, a strong chess engine seems to never make a mistake. If we are talking about a blunder (which is even worse), strong chess computers do not make them as well.
Inaccuracies though are another question, even strong chess engines do make them against popular opinion. But if we are talking about mistakes, I can definitively tell that a working strong chess computer would not make one in its games.
Strong chess engines are almost perfect
If we are talking about the strongest chess engines in the world, it is safe to say that their algorithms are strong enough to make them play at least one of the three actual best moves in the position all the time.
The quality of the moves will vary depending on the depth of the engine in question as well as its programming (maybe even its neural network). But they all play good moves for the most part.
Even at the top level of the chess engine rankings, they do play different best moves here and there, but all of their moves are also being considered by other strong chess computers (all of them play the best 1-3 moves in the position).
I do not say that strong chess computers are perfect in any way, but they are almost perfect since they play so effectively.
Depth determines bad moves in chess engines
Depth is basically how far the chess engine can look in its search tree (basically the algorithm that it uses to calculate) and its strength will vary depending on its current depth.
Strong chess computers have different levels of strengths depending on their current depth, if the depth is too low then it might miss the best move in the position (though it is not necessarily called a mistake).
This is where inaccuracy comes in, if a strong chess engine does not set its depth in its maximum power then it might not make the actual best move in the position. Blunders and mistakes are a different story though.
Even if the strong chess computer does not set its depth to its maximum potential, it would still never make an actual mistake or blunder.
It can only make an inaccuracy, which may not be the best move but is definitely not on the same level as a mistake or a blunder.
Strong chess engines lose because of inaccuracies
You might have witnessed strong grandmasters beating strong chess engines here and there, maybe on youtube. There is something that I need to make clear in these cases.
Strong chess engines lose against human players because of its inaccuracy, not because it has made a mistake or a blunder.
Strong chess engines usually lose games not because they have made a mistake in their match, but rather they have just been outplayed by their human opponent who made better moves.
So no, there is not enough evidence to suggest that strong chess computers make mistakes because they lose to human players, it is just that they make enough inaccuracies (depending on its current depth) that enable humans to outplay them.
Supposed mistakes by chess engines are debatable
It is very hard to evaluate whether a strong chess computer has made a mistake, after all, some of the questionable recommendations from these entities are still in the “unclear zone” and are debatable.
If somebody claims that a chess engine has made an actual mistake, it is hard to judge whether the mistake is actually true since we can’t use another computer to evaluate.
I mean we can, but there has not been any case where a strong chess computer has completely invalidated the findings of another strong chess computer.
Even if two strong chess computers came to different findings, how are we going to determine which one is correct? Even if an actual mistake has been found it is likely debatable.
It is likely something that is subtle and we can’t really tell if it is a mistake. This makes answering this question quite troublesome.
Anti-engine moves are not mistakes, they are inaccuracies
There have been few instances where the chess engine missed the best move in the position (some call these moves anti-engine moves) failing to win a relatively won position.
Usually these are positions where the recommendation of the chess engine only leads to a draw, and to where human players can win using their own expertise.
But the thing is, these anti-engine moves are usually a form of inaccuracy and not necessarily a mistake. A mistake is something that would crumble the position and would probably lose it.
This usually happens because the depth of the engine in question has not been to its maximum potential, although it is rare nowadays since chess computers have become so advanced.
Disagreement between chess engines are inaccuracies
Chess engines do disagree with each other on several occasions, however, the moves that they suggest are usually not considered a mistake. They are all good moves and within the realms of effective play.
When two chess engines disagree, it is not like one is definitively right and the other is wrong (although chess computers have different strengths as well) if the two chess engines are strong they are likely to be both right.
What I mean by this is it’s not like one of the suggestions is a mistake or a blunder, the other one is likely making an inaccuracy that is not necessarily on the same level of a bad move.
Two chess engines disagreeing does not mean that one of them absolutely made a mistake or a blunder.
Strong chess engines can make a mistake if there is a bug
Chess engines (even strong ones) could make a mistake if there is a bug in their system, in this case, the malfunction might make it recommend moves that can actually be considered a mistake.
But I think these cases are special, it doesn’t necessarily mean that the engine has made a mistake when it is functioning properly. I don’t think that this counts but I just have to mention it in this article.
A software bug is basically an error or a flaw in the system that makes it produce unlikely results than if it is working normally.
It is rare for a strong, properly-maintained chess computer to have a bug but it can definitely happen. If the system is faulty then the recommendation can definitely be a mistake since it is the result of an error.
I just have to mention this, but this doesn’t really answer the question directly since it is an unfortunate technicality. If the strong chess engine is running properly though then it would not make a mistake.
It seems that chess engines are truly almost perfect, if we are talking about a mistake they seem to never make one (at least the strong ones). They are not perfect though since they also make inaccuracies.
I have discussed several technicalities within this article as to why chess engines might make a mistake or even a blunder (strength, level of depth, a bug), but these are all exceptional cases.
At their fullest potential, chess engines will never make a mistake since they are just that better than humans. Even if they have made an actual mistake we are not confident enough to definitively identify the mistake.
That is just one of the limitations of being the creator, that the creations may surpass us one day. At least it did in this case, sleep well and play chess.