The Formation of Unreflected Beliefs

On the fine balance between changing one's mind and getting mind-hacked.

Thinking is not free. It takes time, attention, and energy.

But there's a hidden cost that people ignore. Thinking about anything changes who we are. The more we think about [a thing], the more we condition our minds to automatically think about it.

Mental Loops and Conditioning

Earworms are small sections of music that get stuck in our mind and keep repeating. The more we repeat them, the worse they get. (By the way, ADHD people seem to have it a lot, and much less so on meds.)

Rumination is a psychological phenomenon where a depressed or anxious person keeps… ruminating, thinking endlessly, about a depressing or anxiety-inducing thought. They make no progress; they just keep repeating it.

These mental loops result from conditioning our mind. If a thought is self-reinforcing, then there's a vicious circle where the more we think about it, the more we change our mind to naturally think about it.

Mental loops differ in how strong, serious, or noticeable they are.

For instance, in a previous essay, I described The Ideological Spiral, a psychosocial pattern where someone constantly reinforces their ideological attitudes over time. This is more serious (yet harder to notice) than a musical earworm.

Learning as Mind Conditioning

Mental loops are extreme cases, but mind conditioning is a pervasive phenomenon.

For one, to a great extent, Learning is Mind Conditioning.

When I learn about a mathematical concept, I am trying to condition my mind to replace my naïve thoughts and intuitions with the ones from the book. I am trying to condition my mind to come up with the correct tools to face various formal problems. I am trying to condition my mind to come up with 63 when I think 9×7.

But conditioning is deeper than mere association. By training my mind to apply a specific concept to many different situations, I am trying to condition my mind to mould itself in the shape of the concept. I am conditioning it to react in a specific way. Such ways are more varied than merely conjuring a thought: I might condition my mind to view all the ways chess pieces might move, to see and feel all the different voicings of a chord on a piano, or to hype myself up in situations where I would have naturally felt anxious.

In general, I have found that thinking of learning in terms of mind conditioning makes me much better at coming up with learning and adapting procedures for both myself and others.

Unreflected Beliefs

Mind Conditioning is also how we form Unreflected Beliefs.

Some of our beliefs are formed through learning or reflection. We think about a question for a long time, and prepare an answer to it. Because we thought about it many times, we developed a single meaningful answer to this question.

But most of our beliefs are not like this. They are unreflected, implicit, unchecked. It is simply not possible nor practical to reflect and make explicit every single one of our beliefs.

We have implicit definitions for most of the words and concepts that we use. Most of our practical knowledge is derived from imitation rather than books or explicit learning. Et cetera.

This is good and lets us quickly learn and deal with new situations.

The Vicious Circle of Obliviousness

However, as explained in my essay on The Experience of Obliviousness, this is also how we can stay wrong for a long time.

When we interact with a topic, we develop a lot of unreflected beliefs around it for a long time.

As our brains do not come equipped with a built-in way to immediately feel the difference between a reflected and an unreflected belief, we become confident in said beliefs.

Such confidence is enough to banish any curious thought and sense of wonder. A naive child may be utterly confused about how smartphones work, yet it will all be normal to the average adult, even if they may not be able to explain much beyond randomly uttering the words "electricity", "waves", "circuits" and "5G".

This is a vicious circle: unreflected beliefs lead to many other unreflected beliefs.

If we develop the unreflected belief that it is impossible to change our institutions, any bad news about our institutions will reinforce said beliefs. Watching said news, we will form more logically consequent beliefs: that we must tear everything down, that we must start a revolution, that we must bypass our institutions and create parallel societies, or that we must forcefully act against people who believe in institutions.

Conversely, if we develop the belief that it is in fact possible to change our institutions, our reaction will be different. Each bit of bad news about our institutions will reinforce it. But the beliefs we form as a result will be more about the urgency of institutional change, or the issues that we should address first.

Adversarial Mind Conditioning

Above, I described a vicious circle that happens naturally. But Mind Conditioning is often used adversarially.

Sadly, this is the bread and butter of most influencers, whether they are journalists or your local Twitter celebrity.

Their stock-in-trade is to take one political stance, like "Leftists/Rightists are stupid" and only publish examples of that one stance. This makes any long-time follower of them develop many unreflected beliefs, through sheer exposition.

Such unreflected beliefs precede our rational thoughts. Regardless of what I rationally know about leftists or rightists around me, if the last 100 examples of a leftist/rightist I have seen are of them being stupid, this is what I am going to have in mind.

A way to notice that someone is cognitively FUBAR is if they react that way when presented with a contradicting piece of information:

At that point, that they unreflectively believe in a claim is enough to convince them of the claim.

There's not much use in thinking, reflecting or debating!

They are completely at the mercy of their Mind Conditioning. And of course, unscrupulous actors abuse of it.

One-Shot Conditioning

Repeating and unceasing news cycles, historically from traditional media, and now accelerated through social media, condition our minds through repetition.

But it is even worse than that!

Effective writing can mould our intuition in one book, one movie, one article, one scene, or one slogan; without the need for repetition. Through potent emotional appeals or identification with a character, one can condition our minds in one shot.

This is the other side of the phenomenon that makes it easier for us to learn things that we care about. When things are more salient, valent, and important to us, we need fewer repetitions to remember them.

This is how people can get one-shotted by communism, libertarianism, AI companions, their first love, a book, and more.

This is why people keep doing propaganda, ads, and things like that. They work.

Our Minds as Children

Our minds get conditioned by anything and everything. They do not wait for a careful evaluation before changing. They are completely disconnected from what is true.

Our minds are just like a stupid animal. They don't understand much. They recognise what is salient to us.

Their perceptions may come from fiction, from someone we hate, from propagandesque repetition, or from something traumatising that we'd better forget. Nevertheless, our minds will get conditioned.

There is no way around that.

Being smart, being more autistic, being more rational, etc. None of that changes anything.

To make it more concrete, I like to imagine that I am stuck in a prison, and the only way I can interact with the rest of the world is through a 5-year-old messenger. That messenger is stupid. Whatever it tells me about the world will only be the words of a 5-year-old.

This forces me to find some balance.

If I completely distrusted the child or my mind, then I would not be able to interact with the world any more.

However, if I completely trust its words, then I would become as dumb as it is. I must instead maintain some critical distance over what it believes.

Conclusion

In practice, I am very defensive of what I pay attention to. I put a strong barrier to considering ideas: I am very critical of where they come from and how they were generated. I do not want to train my mind to mould itself in the shape of stupid generators.

I interpret the metaphor framing our mind as children quite seriously. I consider it very important to maintain a healthy relationship with my mind. Although I will not become paranoid towards myself solely because my mind may be conditioned to believe stupid shit, I will remember that it may.

I also treat others as I treat myself, and I am honest about it, which they usually dislike quite a bit. If they keep paying attention to stupid shit, I know that will start unreflectively believing stupid shit: there's just not enough time to rationally list and examine all of our beliefs.

Yet, when I tell them, the first reaction is usually taking some offence at it, as if I dared question their intelligence and rationality. Nay, their entire worth as a human being.

But nah. That's just how we all work.

On this, cheers!