In our recent research published in Scientific Reports (Springer Nature) we analyzed data from more than 140 countries to explore why neutrals are usually more attracted by the anti-vaccine side. Our analysis included cutting-edge methods from Social Influence, Agent-Based Modeling and Attitude Networks!
We know that the more similar two people are, the more they will influence each other. The “pro-vaccine” are too far from the neutrals. However, people in between can be close to both, connecting the two in a chain of influence.
The chain of influence
We analyzed this idea of the chain by using attitudes networks. By exploring the network we found that countries with a weaker chain performed worse in terms of vaccination coverage and trust.
Agent-Based Model of social influence helped us in connecting the data with the theory. Using them we confirmed that social influence models would predict as well worse results when the chain is weak. Furthermore, they highlighted the dynamic nature of the process!
Indeed, countries with a weaker chain will not immediately perform worse, but only in the following year. This is due to the fact that trust needs time to spread across the system.
So if we are planning policies to improve trust we should be careful in doing so without breaking the chain. If this happens, we may observe an immediate increase in trust, but this may be quickly eroded as we left the neutrals behind.
In our paper we also simulate that it is possible to make policies which will boost trust while even enhancing the link with the neutrals!
As you may already know, I am a researcher in a field which goes under many different names: sociophysics, social simulations, computational social science, etc. These are actually just different ways to say that “we want to study social systems in a similar fashion to physics.“
However, several people react to these studies as they were something silly. After all, “how could you study people when people are so complex and unpredictable? People are not as simple as inert matter!“
In this post, we will see why this approach it actually makes sense to work for a “physics of people”. Specifically, we will show things such as:
If you think the physical systems are simple and predictable, you clearly don’t know physics.
Definitely we won’t be able to predict everything (also physics is not able to do that) but this doesn’t mean we won’t be able to make some big progress.
We already have at least a couple of “physics of people,” so I it is totally possible to make it.
Ready?
(1) You cannot predict everything
During a conference, I have seen a person showing some pretty interesting results on simulating and predicting how people walk in a train station. These types of studies can be used for many things. For example, to design paths in such a way that a running person will not bump into other people stopping the flow. Also, similar studies have been used to design more efficient emergency exits, thus avoiding people clogging the path and getting stuck (and so, losing their lives…)
However, at the end of the presentation, a person asked “can your model predict what will happen if a person in the crowd has a heart attack? How will the rest of the crowd react?” Since the answer was a “no, I cannot do that” the person who asked the question started speaking about how these models were limited and so, of no practical use.
Let’s apply this to physics!
To understand if this argumentation is correct, let’s try to apply it to the world of physics. Let’s suppose we are in the early years of 1700 and that a young philosopher has just discovered that she can predict the trajectory of a stone after it has been thrown (given its initial position and speed).
Now, let’s ask the young philosopher: “can you still make your predictions if a bird catches the stone mid-air? Or what happens if someone else throws another rock which hits our rock during the trajectory? Is your equation able to take all of this into account?”
Of course, the young philosopher cannot give a precise answer, therefore, we can conclude that this “mechanics” that she is blabbing about is going to be pretty much useless…
Back to the argument
What we just saw is that even physics cannot predict everything. Indeed, every kind of science always requires that the system will not have too many external disturbances.
Does this make these sciences useless? Well, we still managed to discover medicines to cure diseases, send a man to the moon, create amazing devices such as smartphones, etc. So I would say that even if a scientific field is limited, we can still use it to do some pretty amazing things.
(2) But people are so random…
… for example, if you ask someone how they feel, their answer may depend on their sentimental life, on the weather at that moment, or even on the food they eat at lunch that now is giving them a stomach ache!
This argument is true, but also misleading. For example, let me see if you can make some predictions even with very limited information. Where am I while writing this blog?
a) At my place, sitting in front of my PC
b) In the jungle, while walking to avoid mosquitos
c) In a volcano, because ninja tacos!
Option (c) may sound weird (or maybe I should say “random…”) but it is already showing us that maybe people are less random than what we may think.
Option (b) makes more sense, but we can still tell that it is a very uncommon option. Yes, it may happen, but the majority of people writing a blog would be in a situation similar to (a). Maybe just instead of being at their place, they may be in another place where they can sit quietly for some time (e.g. a train or a bar). However, even with no additional information, we can tell which situations are likely to happen and which, instead are not. If people were really random, we could not distinguish between option a, b or c.
Another funny thing is that I have seen this comment mostly coming from people working in psychology and sociology. But, if people were really random, then how could we have fields like these two? If people had no regularities to study, then what would be the purpose of these fields? Just saying that people are random? I hope not!
This tells us that yes, you cannot exactly predict or exactly guess people’s behaviour. But still, people have quite a lot of regularities that we can study and eventually use for modelling.
(3) Still, people are not that simple!
This argument comes often when someone takes a look at the models we are currently developing in fields such as agent-based modelling. In these models, people are massively simplified to follow some simple behavioural rules.
Is this wrong? Or maybe it is a good scientific approach?
Also physics simplifies a lot
Let’s get back to the example of predicting the trajectory of a stone. Do you know how people usually perform this calculation in physics? They approximate the stone as a point and the surrounding space as vacuum*. (we also have a good example of this approach from the Big Bang Theory!)
Probably some centuries ago people were complaining of this approximation saying: “how can you simplify stones that way? Each stone has a unique shape and it is made of different materials. This one, for example, contains tiny quartz fragments which make it so shiny and beautiful! Can you really simplify it as a point?”
Since now we have hundreds of years of experience with physics, we know that “being shiny” is an optical property, which (usually) has practically no effect on the trajectory. Similarly, now we know that subjective properties (such as the beauty of an object) are completely separate from physical properties.
However, it took humanity thousands of years to figure this out. For example, alchemy books included praying in the steps for obtaining the right chemical reaction. Indeed, they often connected the chemical processes to the processes of someone’s soul.
Before getting the conclusions to this point, let me introduce you to the next one:
(4) Humans systems are so entangled!
This means, for example, that something that I have seen when I was 5 years old, may have an impact on my current decisions now. Furthermore, my decisions will impact the mood of the person next to me, which will then impact their lifestyle, etc. However, in physics, everything is independent… right? Or maybe not…
Actually, I am sorry to inform you, that also in physics everything is quite untangled (and I am not even referring to quantum entanglement). For example, the stone we have just thrown is entangled to the Moon! Yes. Indeed, the Moon has a gravitational field, pulling everything towards itself.
In most cases, this force is so weak that we can neglect it. But, even if we neglect it, the rock is still entangled with the Moon (and the rest of the universe). We just know that this effect is so small, that including this in our model will increase the complexity, without adding much to the precision. However, this is not always the case.
To show this a little better, let’s suppose we are playing a game where we have to throw our stone beyond a certain distance (if you want to spice things up, suppose this is part of the Squid Game and your life depends on this throw). Unfortunately, our throw was a little too weak, and our stone is going to land 0.2 mm before the distance; thus making us lose. However, if the Moon was in the right position, it will pull our stone enough to land beyond the threshold distance.
So…
All of this is to say that also in physics things are also complex and entangled. However, we studied physical systems so much, that we can distinguish between effects to take into account for each problem.
For example, if we are studying colors, we know we will have to rely mostly on optical properties. If we are studying water pipes, we will look for fluid dynamics. Finally, if we are sending a man to the moon, we will have to take into account quite a lot of effects!
Unfortunately, our knowledge of social system is still very primitive. For example, we really don’t know if publicly insulting anti-vaxxers will, in the long run, increase or decrease the number of people getting a vaccine shot.
We do not even know what would be the main factors in such a process. For example, would be the aggressivity of the insult play a major role? Or any type of insult will produce very similar effects? Does it matter if the insults come from experts, institutions or common people? We really don’t know!
I think the major problem we are facing now is not about the entanglement of the system (which happens also in every other discipline). I think the major problem is that we still did not accumulate enough knowledge and methods in the social sciences to be able to address similar problems.
(5) We already have two “physics of people”
Yes, it may sound weird, but we have not one, but even 2 different “physics of people.”
The first one is a bit unpractical, but it is based on the fact that people are actually made of physical matter. Therefore, our body and brain can be studied and predicted using just physics.
At the moment this sounds like an absurdly complicated task (simulating human behaviour via simulating our entire body), however, we already have some projects moving in this direction, such as the Blue Brain Project.
The second physics of people is actually even more famous and it goes under the name of “economics.“ Yes, actually we already have a field that, exactly like physics, use equations for modelling people’s behaviour (specifically on the exchange goods).
So, probably, saying that we will never have a physics of people is actually quite wrong.
In conclusion
We have shown that our “physics of people” will not allow us to predict everything. Indeed, this goal is not achievable even in the hard sciences. Instead, what we want to achieve, is to find conditions in which people have fairly regular behaviour and see what happens when this is perturbed.
Studying how people behave under stress may help us design better emergency exits or buildings that will protect more people in case of a terrorist attack. Knowing how people will react to new technologies will help us design better products and communication campaigns, without producing distrust in people. Figuring out how people communicate and argue, may help us in designign social media that will help us grow as a society, instead of considering us as two opposite groups in an ideological war.
To do this, we will not need only knowledge, but also the development of new tools. For example, advancements in physics have made possible both by physical tools (e.g. telescope, atomic-force microscope…) and by theoretical tools (integrals, differential equations, Fourier transform…).
Definitely, all of this would be extremely hard and complex. Also, it will require decades or even centuries to develop to maturity. However, I don’t think this is a good excuse to just sit back and stop moving towards a better future.
Want to know more about Agent-Based modelling?
Check out this video!
Notes
*As mentioned later, the amount of features taken into account depends on the situation and required precision. For example, air friction usually may be neglected when throwing a stone, but not when throwing a feather. Furthermore, if we really need high precision, we cannot neglect friction even in the case of the stone.
You analyze a dataset and figure out that political polarization has been steadily rising in UK for the last 10 years. You tell this to your colleague who looks surprised: apparently, in her analysis, polarization has been decreasing for the last 10 years! And she used exactly the same dataset as you. What is going on?
What we know about measurements
i.e. recap of the last post on measurements
In our last post we discussed that, in the social sciences, quantities are defined in a general way. This is different in fields like physics where they employ “operative definitions” which tell you exactly how to measure a specific quantity.
Of course, we have many more interesting details in that post, but the short story is: different people will end up measuring different things (and these measurements are usually non-comparable).
We used the example of counting potato chips and noticed that everyone will count a “full chip” as one. However, everyone will have a different approach when counting broken chips. For example, one may count each fragment as a separate chip, while someone else may not consider them as “chips”.
End of the recap!
What about the dynamic behavior?
The big question now is: we know these measurements are non-comparable, but should they all produce the same dynamic behaviour?
For example, if I add a chip to the bag, all the measurements will increase by one. And if I remove 10 chips from the bag, all measurements will decrease by 10, right?
Yes, indeed, in some cases all the measurements will behave nicely and produce the same behaviour. However, what happens if we start crushing some chips in the bag?
Now, the measurement which counts only the full chips will tell you the number of chips is going down (we are losing a lot of full chips) but the measurement counting each fragment separately will register an increase! We are observing two opposite dynamic behaviours even if we are still measuring the same “macro-concept” (i.e. the number of chips)
Why we have this, again?
As discussed last time, we should be careful with the difference between “macro-concept” and the measurement itself (i.e. “operative definition”). Indeed, while the macro-concept may sound clear and intuitive (“we are just counting how many chips we have in the bag”) it is actually hiding the fact that we can perform this operation in many different ways. And, unfortunately, every way consists of measuring something different.
But maybe you are still not seeing exactly what is causing this problem, nor how this can impact the social sciences. Well, in the next paragraphs we will address this using political polarization and a few elephants.
What is an elephant?
There is a nice story about some blind men trying to figure out what an elephant actually is. One grabs its trunk (i.e. the nose) and thinks that elephants are like snakes; while another one grabs one leg and thinks that elephants are like trees, etc. Maybe this story may teach you something important about life, but we are not interested in life, but in science, measurements and definitions!
So, let us replace the blind men with 3 researchers wanting to write an article on Nature Elephants. They also ask independently their assistants to put a sensor on the elephant to track its movements. One assistant puts the sensor on the tip of the nose, one on the leg and the last one on the chest. However, the researchers are unaware of these differences; they only know that their sensors are on the elephant (and nothing more).
When the elephant is moving, all three measure roughly the same behaviour (i.e. horizontal motion). But when it stops walking, they start observing completely different dynamics.
One, measuring the chest, start observing a rhythmic movement. The one on the leg measure almost complete stillness, while the sensor on the nose shows big irregular movements in different directions.
The three eventually start arguing and debating. Should they use more precise sensors? Or maybe run an experiment with 1,000 elephants? Should they use some statistical technique for removing big or small movements?
This may sound very silly, but unfortunately, it is not too different from the way we discuss when we face more abstract concepts, such as political polarization…
Is polarization increasing?
Some times ago, I gave a talk in which I showed how, with a method we are currently developing, we could explore different aspects of political polarization. At the end of my talk, a person asked me: “so, practically, is polarization increasing or not in the US?“
If are not familiar with the term, polarization represents how “split in two” a certain society is. As you may guess, this is a quite general concept, which incorporates almost endless measurements. Let us see a couple of examples. Polarization can be thought as:
How many people are at the “extremes” (e.g. most people hold neutral attitudes VS most peple hold more extreme attitudes)
How extremists are the ideas (e.g. in society 1 extremists think we should not accept immigrants; in society 2 extremists think violence against immigrants is acceptable).
How “flat” the society is (e.g. a society in which if you are a left-winger you also have specific food, music, and other habits which you share with he other left-wingers)
Take also into account that you can also combine the previous definitions. For example, you can produce a measurement that is mostly sensitive to effect (1) and only a little to effect (2). This means that we have pretty much infinite possible measurements!
Ok, but this was not the initial question. The question was: “is polarization increasing?” And it is clear that the person who asked wants a pretty simple answer, such as “yes” or “no.”
But, before answering it, let us warm up testing it on the elephant.
The elephant game
Let us play the following game: I will give you a situation and a question. You can answer only “yes” or “no.” Ready?
SITUATION: The elephant is running. QUESTION: Is the elephant moving? Wow, you are pretty good at this game!
SITUATION: The elephant is standing while eating. QUESTION: Is the elephant moving? Ok…
SITUATION: The elephant is sleeping. QUESTION: Is the elephant moving? Interesting…
We could go on, but you probably got the point: a simple “yes” or “no” does not fully represent what is going on. Actually, it does not represent it at all.
And this is the reason why I totally disappoint the person asking me the question initial question, by answering her: “Polarization is increasing or decreasing depending on what you mean by polarization.”
What about the models?
As you know, I am very interested in the dynamic modelling of social phenomena. Specifically, in agent-based models. So, how does this knowledge related to modelling?
1. Making the model
The first important thing to consider is how you will build the model. Some people build a model directly from data and experiments, other from more general theories and frameworks. Whatever you do, you will also select a specific definition of the phenomenon you want to model (even if you are not aware of this choice).
So, make sure you know which choice you are making, and that you keep being coherent with that choice. For example, you may mix two theories, or experiments, but make sure they refer to the same specific phenomenon.
2. Using the data
It is probable that one day you will use data with your model. Maybe it will be for calibrating it, or for validation, or whatever. The thing that you have to keep in mind is that there will be many types of data all under the same name.
To stick with the elephant example, you will find data on the nose’s movement, on the ear’s movement, etc. But, probably, they will all be labelled as “elephant’s movement.”
So, when using data with your model, you will have to make sure that both model and data refer to the exact same phenomenon.
3. What can go wrong?
The short answer is: everything. Here, just as an example, enjoy the following:
The case in which your model makes totally different predictions depending on the measurement used for collecting the data [link].
The case in which you can actually transform one model into another just by changing the measurement scale [link].
This does not fully explain it!
As you may see, one solution is the simple “just stick to one measurement.” However, this is a little limiting. Indeed, whenever we stick to one single concept of “polarization” we are also neglecting all the others. Similarly, if we track only the elephant’s nose, we lose all the information about its ears, eyes, tail, etc. What to do?
A common thing that people say is something along the lines of “this problem is just too complex because humans are infinitely complex. We should not even attempt to solve this problem. Enjoy life while it lasts.”
While I agree (especially with enjoying life), I think there are a series of things we can do to make things better while doing our research:
Being as clear as possible about definitions, models and measuring process
Study the phenomenon using different definitions and measurement (but being clear about these differences!)
Trying to summarize the results and look for similarities and differences between different measurements and models
Will this solve all the problems? Definitely not, but I am pretty sure it will allow us to make some pretty amazing discoveries.
What’s next?
In these posts we analyzed this problem on a very general level (i.e. broad definitions vs operative definitions).
But it is time to start facing a little more in detail how measurements are made in the social world.
Coming up next, ordinal scales and measurement distortions!
Can I compare my measurement of happiness with the one in another study? Is it just like comparing meters to miles? Why my measurement shows that polarization is increasing but another study shows the opposite?
Today we will clarify one crucial aspect of the social sciences: measurements. Especially, we will see that (1) these measurements are quite different from measurements in physics, (2) what we can do with them and (3) when we have to be careful.
What is this all about?
Let me start by telling you that this is not about the difference between interval and ordinal scales. They are important too, and they can mess everything up as well; that is why we will have an entire series just dedicated to them. But not today.
Today we speak about something even more basic: how quantities are defined. Let us start by looking at the world of physics.
Concepts in physics
I am pretty sure you are familiar with the concepts of length and distance. And you can discuss these with anyone without worrying that the other person may interpret distance in a completely different way. Besides relying on common knowledge, we can also check how the units of distance are defined, reading for example that:
The metre is currently defined as the length of the path travelled by light in vacuum in 1/299 792 458 of a second.
(from wikipedia)
Something you may be wondering now is: why does this sounds so ugly and boring at the same time? Why do we have a damn fraction in the definition?
The short answer is: because physics heavily relies on operative definitions. These definitions are not aimed at explaining a general concept, but more at telling you how to practically measure something.
Have you ever heard of the fact that science is “reproducible?”
Well, these definitions are aimed exactly at that. They make sure that everyone would measure exactly the same things. They make sure that we all know exactly what 100 meters are, with no room for interpretation.
Concepts in the social sciences
As you may expect, the social sciences do not rely much on operative definitions. This is not because social sciences are bad, or worse, or anything else along these lines; but mostly because they focus on general concepts.
an emotion of joy, gladness, satisfaction, and well-being.
As you can tell, this does not contain any information about the measuring process. But is this a problem?
As we will see in the next lines, this will definitely be a problem if we do not understand this process.
Let’s explore it better
Ok, let’s suppose we want to measure how many potato chips are in a bag of chips. Sounds like an easy task, right? Well, actually it is quite a complex one. Indeed, while we have no problem with “full chips,” we do not really know what to do with a broken potato chip.
Take as an example the image on the left. Should we count this as 1 chip? It someway makes sense as you could recompose it to be a “full” chip. But it also makes sense to consider it 0 as, it is clearly not a full chip.
Someone else may also claim that we count each fragment separately, as every piece in the mouth is indistinguishable from a small chip. Therefore we should count this as 6.
Notice that this debate could go on forever getting progressively more and more complex, with questions such as:
How big should be a fragent to be still considered in the count?
When a “full chip” becomes a fragment? (consider a chip with a very small missing piece)
etc.
The main problem is that we do not have an operative definition of potato chips. Nor do we have a unit of measurement for chips.
This means that every person will measure a different number of chips.
Can we convert them?
Let us suppose that the measurement that takes into account both full chips and fragments tell us that we have 100 chips. How many full chips do we have? That is: how do we convert this number into another measurement?
As you may expect you cannot precisely do this, as the first measurement simply merged everything together (i.e. full and fragments of chips). So the only thing you know is that the number you are looking for is between 0 and 100; which is not very precise…
Similarly, if you know that in another bag you have 50 full chips, you still have no idea of how many fragments+full chips you may have. Maybe it is 50, maybe it is 10,000; who knows?
And this is quite a big range of uncertainty!
This is not a statistical problem
Many people here may feel like this is the same old problem of sampling: you may get a bag with 20 chips or a bag with 30 chips; so what’s new here?
The fact is that this is not a sampling problem but a measurement one. Indeed, the bag is always the same. We did not resample or replaced it with anything else. What we changed is how we are measuring, but the object is still the same.
Is this an artifact?
An argument that I hear often is that “this is an artefact.” This can also be rephrased as “one of these measurements is the correct one and the other is simply wrong“. And, someway, this argument is correct; but it is also quite wrong. Let’s see why.
Let us suppose we want to predict the number of times a certain child (le’ts call her “child X”) will put her hand in the bag of chips for eating. We know that this child picks fragments and full chips one by one, as long as they are above a certain size S.
In this case, we want to count as 1 each peace above size S and ignore smaller pieces. Every other measurement would generate artefacts… in this context.
Suppose, instead, we are dealing with child Y. This child eats full chips one by one, while she does not eat fragments. So, in this case, the correct measurement would be counting as 1 full chips and every fragment should be counted as 0.
This means that the right measurement is determined by what we want to measure. And all the other measurements will introduce some artifacts.
Therefore, we cannot have a measurement which is good in every situation.
Just use differnt names
Another interesting argument that I hear sometimes is that we need better classification. For example, instead of using the general concept of “chips” we may distinguish them into “full chips” and “fragments.”
While this approach is helpful, as it limits the possibilities we have, it still does not completely solve the problem. Indeed, as we discussed before, when does a full chip become a fragment?
You can observe something similar in this article where they notice that the concept of “polarization” is too vague and the authors come up with 4 main sub-types of polarization. However, the same article then highlights how the same sub-type can be still measured in different ways.
Indeed, at the end of the day, what specifies exactly how to measure something is the measurement process itself (i.e. the operative definition). This is why better (non-operative) definitions ay help but not solve the problem.
Can correlation save us?
An important ally of every scientist in the social sciences is our friend correlation. Indeed, as we will see, it can strongly help us in solving some of these problems. Even if we should not blindly trust it as we may still end up with some bad surprise.
When to trust it
Consider a chips brand whose bags contain usually 90% full chips and 10% fragments. In this case, you can easily convert one measurement into the other. For example, if you measure 100 in the measurement which counts also fragments, you should have a number very close to 90 in the full-chips measurement.
If this relationship (i.e. 90-10) is not given to you as initial data, you can still explore it using tools such as linear modelling or simple correlation. You just need the process to be reliable. In this case, you will be able to know:
How to transform one measurement in the other
How precise your estimateof the second measurement will be (i.e. how uncertain your prediction is going to be)
If it is so simple, why even bother with the first part of this post? The problem is that things are not always so simple…
When you should not trust it
You figured out that for brand X, the percent full/fragment is 90/10. So now you can use both the full-chips measurement and the full+fragment since you can convert one into the other; very well!
What happens now if you apply this relationship (90/10) to another brand? Or if the same brand changes something in their production chain altering this ratio?
The problem here is that you can convert the two measurements as long as they have a stable relationship. But this relationship may change in time or not be universal at all (i.e. it works only for a specific brand).
For example, two measurements of polarization may be perfectly equivalent in France but not in Germany. If you know this phenomenon, you will not be surprised to see the two methods diverging. However, many scientists are unaware of this and they may get totally puzzled by these results.
Summing up…
Some people may reach this point and ask: if we are always measuring the same thing, why do we end up having different results?
And the answer is: because we are actually measuring different things!
Yes, we started from the same macro-definition (chips, polarization, happiness, …). But then, we ended up using different operative definitions. This means that practically we measured different things (e.g. full chips vs fragments). This generates the following situations/problems:
We cannot directly compare results.
We can estimate one measurement from the other by using correlation/linear modelling and making sure that we are not changing anything important between the two measurements (finger crossed🤞).
The measurement which is the best for us may actually be bad for other people/studies.
Different measurements may actually produce different dynamic behaviors (e.g. one measurement shows increasing polarization and the other shows decreasing)
While we explored points 1 to 3, we did not really discuss point number 4. This is because it deserves a lot of attention and we will have a post just on that (coming up in 1 or 2 weeks).
If you are interested in measurements and how this may affect modelling (especially I am interested in agent-based modelling), check out this blog or my social media, as I will keep exploring this topic.
In the last couple of days there were some troubles with the website. You may still experience some problems as I am setting everything back to normal, but it will not take long.
The way we measure effects in the social sciences may be way more important than what you think…
This post is for a broad academic readership
The mystery of the top earners
Just yesterday I came across this post from the NeuroNeurotic blog. The idea is very interesting as it discusses how some “psychological effects” may actually not be psychological at all. Instead, the effect may appear just from some data manipulation (aka an artefact).
The blog’s post takes a look at this other article from the Guardian. Here a study shows how the top earners in Germany believe their earnings are almost the average ones. This claim is someway supported by this pretty cool visualization:
On the left, people are divided in deciles. For example, the maximum decile (i.e. 10) would be the top 10% earners. On the right, we have some kind of perceived income. (More details later!)
The problem we now face is: are we sure this picture is telling the truth? Which can be reformulated in: “do we really need some psychological effect to obtain this graph? Or can it be obtained just from data manipulation?”
NeuroNeurotic’s solution
According to the NeuroNeurotic blog, the previous image does not really support the claim. Indeed, it may just be an artefact due to binning.
For those who do not know yet this binning guy, he is just the cousin of rounding. Indeed, when we round, we take a lot of numbers and collapse them into fewer groups. For example, all the numbers from 1.5 to 2.499 will be grouped into the number 2.
Similarly, we may take person 1 to 1,000 and put all of them into the same bin/group. Thus, deciles are a way to group people into 10 bins.
Representation of how we may bring an entire population into 2 bins.
The main idea behind the blog’s argument is that binning is putting in the same group people that, maybe, should not be together. For example, the top decile will contain people which may have gigantic differences in earnings. Thus, averaging these values together will bring them closer to the mean value.
For a more detailed explanation, you may look at the original post. However, what I found extremely interesting is how the author was able to reproduce a similar image in simulations even without any psychological effect!
Indeed, he assumed that the distribution of earnings followed a normal (aka Gaussian) distribution. Then, he assumed that every person is just answering their real earning and collected the average value per decile. The striking result is the image below.
My question this time is: is the simulation really reproducing the results from the article? Which can also be restated as: “is it just a matter of binning?”
The surprising effect of binning
Let us try to simulate something slightly different now. Earnings are still normally distributed like before, and people are still divided into deciles (i.e. binned). However, this time we ask people: “in which decile of the population do you think you are?“
This means that in the previous simulation everyone was answering her own earnings. Now, everyone will answer her own decile. Similarly to the previous simulation, also here everyone is answering correctly (i.e. no errors or effects).
The interesting fact is that if we run this simulation, we obtain the following image. Why? In this case we still have binning but the result disappeared!
The short answer is that everyone is just answering her own decile. So all the people in the 10th decile are answering 10 and the mean value would still be 10.
The longer answer is that we are actually facing a problem of measurement…
A problem of measurement
What was not really clear here is that we are currently dealing with two different scales of income. The first scale is just the earnings and it is measured in dollars. Meaning that if I earn 1,000 $ and you earn 5,000 $, the difference between us would be 4,000 $.
However, there is also a second hidden scale: ranking. In this scale, each person receives a score (aka number) according to how they place. For example, the poorest person would be number 1, the second-poorest would be number 2, etc.
To understand why this difference is important let us take the two poorest people in the simulation. Let us say one has 1 cent and the other has 5 cents. Thus, their difference in dollars would be 4 cents. However, their difference on the ranking scale would be 1.
This difference of 1 would also be the difference between the two richest. However, their difference in dollars may be of some millions or even billions.
This tells us that the relationship between the two scales is someway weird. This “weirdness” is called “non-linearity” in mathematical terms, but let us stay away from obscure mathematical concepts.
Instead, let us plot the relationship between the ranking and the dollar scale. Does it look someway similar to something else? Notice how most of the lines are again tilted towards the center!
What we just observed is the fact that when we change scale we produce some distortions on the graph. This may result in compression (e.g. all the lines going towards the centre) or expansion depending on the two scales.
Furthermore, if we bring back our old friend Mr. binning, we will be back to our initial effect. As you see, for example, the top line is not horizontal anymore as it has been averaged with the other top 10% lines.
So what?
Our analysis shows us some little interesting facts:
Binning alone is not sufficient to produce the effect in the article. Indeed, it would result in straight horizontal lines.
Scale transformation is a beautiful way to create a mess. Indeed, the relationship between the two scales would look like a mishmash of tilted lines.
Scale transformation + binning is the ultimate key for a disaster. One creates a mess while the other averages it out partially. This creates a cool relationship between the two scales which may be confused for an effect.
Then, is the study wrong?
The short answer is: “we don’t know.” Actually, everything depends on the question that was asked to participants.
If the authors asked “what decile do you think you belong to?” then everything is fine. Indeed, the two scales would be decile VS perceived decile. Here we have no scale change and binning alone cannot do anything to explain this.
For example, the study showed that the top decile answered an average of 6.5. This means that, roughly, people in the top 10% think they are only in the top 40% and that there are still 30% of people richer than them. This bias is definitely an interesting psychological effect!
However, what if they asked something that made people think in terms of earnings instead of ranking? In that case, the plot would be affected by scale transformation. Indeed, the first column would be a ranking while the second would be an earning scale! Thus, we would have all the ugly effects we discussed before.
For example, we may ask “on a scale 1 to 100 how does your earning compare to the richest person? With 100 being the same earning as the top one, 50 being half of it and 1 being 1/100 of it.“
Let us suppose the richest person earns 1 million and the second richest earns 0.7 millions. Even without any psychological effects, person one will answer 100 and the second will answer 70. Thus, the line of the top earners would not be horizontal but tilted towards 50!
In conclusion
Always be careful of how you measure things, especially in the social sciences. Indeed, changes of measurement have the potential of messing things pretty badly.
Next time we will discuss about another effect that may be present in this study!
If you are reading these lines probably you fall into one of the following cases:
You were wondering who this Dino Carpentras is (probably you found me on Linkedin) and looked at my website to know more.
You were looking for something related to Social complexity, Agent-based modelling, Rationality or Critical thinking.
I always feel uncomfortable if a list does not have 3 points.
In any case, hurray! you are in the right place. Indeed, here you will find a lot of information about me, my research and many posts on related topics.
When looking at a post, check for the following symbols to know the difficulty level:
I have been thinking for a long time about what to do with this website, as well as with my youtube channel. The main problem was that I wanted to distinguish between my work and my communication activities.
However, as many academics, I pretty much have no time. This means that it is already hard for me to have one website. Two of them would be simply impossible.
Because of that, I decided to merge them, creating this new identity for the website. From now on we will have both information about me and what I am working on, as well as more general information.
I also came to this conclusion as many topics I wanted to discuss were someway both work-related and also for a broader audience. This was finally resulting in duplicate articles and a lot of useless work.
I hope I will be able to distinguish between technical content and what, instead, should be aimed at beginners. I will update quite soon with a method for distinguishing between them.
In any case, welcome back to Dino Carpentras + Social Complexity!
Together with the other members of ESSA@work we organized a track at the Social Simulation Fest.
For people who do not know about it, ESSA is the European Social Simulation Association. Several events are organized by ESSA always focused on the topic of social simulations.
In most of these events ESSA@work produces a track based on work in progress. Indeed, while all other tracks focus on completed projects, we focus on the stage before. In this way we create a friendly environment in which people can discuss about their project and receive feedback from experts.
A new preprint came out today from a work I developed together with Alejandro Dinkelberg and Prof. Mike Quayle at UL, Limerick.
Opinion dynamics is a field focused on understanding the evolution of people’s opinions. Within this field, opinions are usually represented as numbers without any other requirement. So we started wondering: can this have some implications on the models?
For example, as we convert meters to feet, we can also convert different scales for measuring people’s opinions. How do these models change when we change their “units?”
From our study, it appears that while physics equations are unaffected by changes of the variable (you just need to rescale) this is not true for opinion dynamics models.
Indeed, we found that scale conversion (even in the case of perfect measurement) can totally change the model’s dynamics. This result in a change on the final outcome up to 100%. Furthermore, by changing scale, we were able to convert one model into another one.
If you are interested in this research, you can find it here:
This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No 891347.
These posts are for everyone
These are for experts and researchers