All Categories

Correlation vs Causation

Correlation vs Causation

Search
How to join our moderation team?
Sep 1, 2017, 22:3609/01/17
64

Correlation vs Causation

There is a lot of discussion about how the game mechanics work, and most of the expert advice is based upon testing by players. (As opposed to actually looking at the source code.)

Because of the evidence based nature of these expert opinions, it is warranted to discuss the difference between correlation and causation.

For example:

When you attack another player, and you lose 20K troops, and you see that they lost 20k troops, you might conclude that they lost 20K because you lost 20k. Or you might conclude that you lost 20K because they lost 20K. Either way, the loss on 1 side is attributed to the loss on the other, but this is not correct.

What is actually happening is that the formula for losses on 1 side also causes similar losses on the other side. Since the losses for most battles are reciprocals for the offense and defense, they will yield the same results when the % is multiplied. This is correlation due to a common cause.

Incorrectly attributing the cause of a result will often lead to false conclusions, which in turn leads to bad choices or surprising results.
Views
2k
Comments
3
Comments
Sep 4, 2017, 10:4309/04/17
02/29/16
2645

Hi!

I understand your point, but our developers have their own vision of the game, and it should come as no surprise that the game development process is mostly based on their vision and their plans. Of course, multiple factors are taken into consideration: analytics, stats, player feedback and suggestions, marketing research, and many other things. However, neither of the factors can be prevalent. We understand that some players would like their own battle or reward calculation. However, I'm afraid we'll not change this algorithm.


Sep 4, 2017, 15:4909/04/17
64

My comment was not about how the game works, but rather how people "break the code" on how things work.

To take a non-game related example:

Scientists will use correlation as proof of causation and label things as "carcinogens". It is possible that (for example) lacking specific nutrients caused mice in a group to be more cancer prone, and that the foods tested were causing malnutrition. This would cause a correlation between eating huge amounts of the food and getting cancer, even if the food itself was not cancer causing.

Bringing this back to the game:

Attacking troops usually causes a loss of troops. If I track the losses and troops killed, they tend to be very close numbers. If I don't have access to the code, I could attribute the reward system to be either driven by my losses, or by the troops I've killed off, and the data might support either argument in 99% of my battles.

People need to be careful about concluding data proves a particular position. Often the data will support many different possible conclusions.

The best approach, barring asking the devs and getting inside information, is to gather data on as many factors as possible, then see which possibilities are ruled out. When someone insists that X causes Y, and the data doesn't support that conclusion, you can go back and check for causes of error in the data. Until you can resolve such a conflict though, you are compelled to conclude that the data doesn't support their position.

With respect to battling locations, there are numerous factors you can consider:

Information provided in the field manual.

Information provided in the forums by those with inside understanding. (Usually in the form of a negative. eg - This doesn't affect that.)

Cost of troops used. 

Cost of troops lost on both sides of the battle.

Ratio of losses in each battle.

Food upkeep of forces in the battle / lost / ratio

Power of the forces

Active boosts and their sources

etc

Until you can rule something out as being irrelevant: track it. Track it well enough, and then test it.

My experience has been that it is exceedingly difficult to isolate a single factor, but with enough battles: the data should disclose the secrets. 

You just need to be able to tell the difference between correlation and causation, and sometimes you can sift that out with testing.

Sep 5, 2017, 13:4109/05/17
02/29/16
2645

Badlag247 said:


My comment was not about how the game works, but rather how people "break the code" on how things work.

To take a non-game related example:

Scientists will use correlation as proof of causation and label things as "carcinogens". It is possible that (for example) lacking specific nutrients caused mice in a group to be more cancer prone, and that the foods tested were causing malnutrition. This would cause a correlation between eating huge amounts of the food and getting cancer, even if the food itself was not cancer causing.

Bringing this back to the game:

Attacking troops usually causes a loss of troops. If I track the losses and troops killed, they tend to be very close numbers. If I don't have access to the code, I could attribute the reward system to be either driven by my losses, or by the troops I've killed off, and the data might support either argument in 99% of my battles.

People need to be careful about concluding data proves a particular position. Often the data will support many different possible conclusions.

The best approach, barring asking the devs and getting inside information, is to gather data on as many factors as possible, then see which possibilities are ruled out. When someone insists that X causes Y, and the data doesn't support that conclusion, you can go back and check for causes of error in the data. Until you can resolve such a conflict though, you are compelled to conclude that the data doesn't support their position.

With respect to battling locations, there are numerous factors you can consider:

Information provided in the field manual.

Information provided in the forums by those with inside understanding. (Usually in the form of a negative. eg - This doesn't affect that.)

Cost of troops used. 

Cost of troops lost on both sides of the battle.

Ratio of losses in each battle.

Food upkeep of forces in the battle / lost / ratio

Power of the forces

Active boosts and their sources

etc

Until you can rule something out as being irrelevant: track it. Track it well enough, and then test it.

My experience has been that it is exceedingly difficult to isolate a single factor, but with enough battles: the data should disclose the secrets. 

You just need to be able to tell the difference between correlation and causation, and sometimes you can sift that out with testing.

Oh, I've understood you. Thanks. 

Well, yeah, sometimes it's difficult to distinguish if some result was caused by one or another factor. And that's why your investigations should be based on other players' investigations and guides. As for me, it's something like a matter of trial and error until the right way or reason is found.