News Human thought runs at just 10 bits per second, say Caltech scientists — that's why we are mostly single-taskers

[Co BIY]

What we see with our eyes isn't actually what we see, and our brain fills a huge portion of the missing bits. (This is why we fall for visual tricks)

We also disregard an incredible amount of data produced by our senses in order to focus on that which we consider important. Some of the mental illnesses appear to be a result of not being able to properly suppress or ignore that input.

So we are simultaneously shedding "useless data" that the senses are creating and creating approximations or filler data in other areas.

 

[watzupken]

I think the finding is applicable to these so called scientists that came up with this.

 

[Co BIY]

As for radio listening and conversations, these are just using idle cycles in your brain. When not much is happening and your autonomic nervous system can pretty much handle it, you can focus on them. However, when you reach a busy intersection, with a lot of cars or people doing different things, you'll find that you stop talking/listening and focus your attention on the task of driving.

Based on the traffic I see I think this is pretty much the opposite of what happens but the basic mechanism is correctly described. The brain cannot effectively divide attention on two complex tasks.

 

[Co BIY]

I’d trust nature on this one otherwise we’d probably need to eat candy by the tanker load?

This explains my eating patterns, trying to push the brain evolution into gear.

 

[bit_user]

Bit, 0 or 1. Not much information there. 2, 4, 8, 16, 32, 64, 128, 256, 512, 1024...

English is made up of about 170,000 total words, and most people know 20,000 to 30,000 of those words. Effectively, at 10 bits per second it would take several seconds to simply parse out a single word you want.

You're treating each word as having the same information content? Clearly, that's not so. A word like "the" contains almost no information, which you can clearly tell by the fact that you can entirely remove them from most cases where they're used and meaning is neither changed nor lost. By contrast, a word like antidisestablishmentarianism is a complex and conceptually-rich term that loaded with meaning and context.

To look at it another way, which is more in line with the article, Claude Shannon defined information as surprise. So, when you can predict the next word, with high confidence, it's unsurprising and thus has low informational content. This was best exemplified in a demonstration Shannon did with some students, way back in 1951. The goal was to estimate the entropy of the English language by attempting to guess the next letter.

https://mathweb.ucsd.edu/~crypto/java/ENTROPY/

The relevance to the finding in this paper is that a human speaker isn't making a decision, word by word, of what to say next. The speaker is making decisions about words, phrases, and even sentences. For instance, if I choose to conclude a short anecdote with the phrase "all is well that ends well", that was essentially one decision I made. Yes, I plucked that phrase from a large space of possible options, but it's not as large as it may seem when you consider I might have only some hundreds of such phrases that I use in my everyday speech.

Now, if they would argue that we are processing at 10 64-bit bytes per second, I could possible believe something like that.

But, it's not. So, the place to start disputing the claim is by understand their definitions and methodology. It's a shame the paper is pay-walled, but I'm sure more can be gleaned about these details, given I've heard this same finding repeated from other news outlets.

 

[bit_user]

Tokens per second is not a measure of activity, just ask OpenAI, they'll tell you how many megawatt/seconds they used to produce a few tokens, a few dozen characters, not even counting amortizing the immense training time.

Tokens per second is very much analogous to what the authors of this study are talking about. Power consumption is something completely different.

 

[JohnyFin]

10 bits is enough. Rest of computation do brain which has still unbitable speed in universe.

 

[newtechldtech]

The brain can be trained. it is not as simple as what such "fake" researchers are "assuming" .

Lets look at a Karate fighter , or a Boxer , ETC ...

at the beginning of his training he cant use his brain , just a slow punch will hit him... the eyes and ears ETC catches the DATA but the brain is slow in "thoughts" how to counter.

After a year in training , the fighter will block/dodge 20 fighters around him ALL throwing attacks on him ... the brain will think about ALL the attacks , calculate fast and THEN decide 20+ counter attacks/dodge/block 360 degrees ... and think for the next step for EACH attacker , AND probabilities of ESCAPE and Movement or advance and movement , WITH balance thoughts and whatsnot.

so keep your 10 bits per second to your self. I believe that the brain has no limits ... the more you train it the more it will process thoughts . there is NO LIMIT.

These researchers are studing the human brain is "Sleep mode" "idle mode" , if compared to PC and NOT "full load all cores running"

 

[ottonis]

The human brain is notoriously bad / slow at conscious serial number crunching - something that classical computers do endlessly better than humans. That being said, the human brain allows for a seamless intgration of multimodal perception (tactile, visual, olfactory, auditory), abstraction at various levels of complexity, and communication.
And with regards to conscious number crunching or serial calculations: this is rather limited by firmware than by hardware. The so called "Savant-syndrome" has shown that some people, after experiencing damage to certain parts of thir brain, literally "unlock" serious capabilities, e.g. doing math at another level, or being able to memorize virtually endless strings of numbers or places etc. Some even gain entirely new artistic abilities and start playing the piano or painting, which they had never done before. However, this often comes at a serious cost, as these Savants are more often than not uncapable of doing some "easy" everyday tasks.

 

[evdjj3j]

This is not Science. this is complete utter rubbish . no one "knows" how thoughts happens in the brain. Full Stop .

Agreed, how do people get funding for this nonsense.

 

[palladin9479]

We also disregard an incredible amount of data produced by our senses in order to focus on that which we consider important. Some of the mental illnesses appear to be a result of not being able to properly suppress or ignore that input.

So we are simultaneously shedding "useless data" that the senses are creating and creating approximations or filler data in other areas.

We're not shedding "useless data", our subconscious is processing it all.

It's very important to realize that the human brain is effectively several co-processors working in tandem each processing a information at different levels. Broadly they can be divided into conscious, subconscious and unconscious parts. Unconscious is what's handling your autonomic systems (ANS), breathing, walking, enzyme control, heart rate, all the things that happen without thought even while you sleep. Subconscious it the primary processor for sensory data, it's taking in massive amounts of information, cross referencing it with both long and short term memory, coming up with conclusions about that information then communicating those conclusions to your conscious mind in the form of emotions. Yes your subconscious is where all your emotions come from, it's your subconsciousness's way of initiating action. The conscious mind only process's a very select amount of information, stuff the unconscious and subconscious hasn't yet filtered out and acted upon or stuff they have decided it needs to communicate to you.

Example

Your in your home, and your unconscious mind detects a low amount of blood sugar or water and communicates that need to your subconscious. The subconscious determines that means you need to eat and references the history of stuff that has previously fulfilled that need and generates a set of feelings to the conscious mind "your hungry" and "these places made you feel nice before". The conscious mind then decides "I'm gonna eat at a sandwich shop down the street", your unconscious mind then handles the movement of your legs and you start walking down the street. While walking you hit your foot into a pole, the unconscious mind receives the damage signals from the nerve endings and communicates this to the subconscious which determines this is a "bad" thing and sends the feelings of "pain, avoid, anger" to your conscious. Your conscious mind then process's it as "OUCH, I need to be more careful about those steps".

The subconscious mind can even take over for the conscious mind if you've done an action frequently enough for there to be a behavioral pattern. How many times have you driven to work and can't remember the drive? Or that you did a set of actions but can't remember the details? Both of those were your subconscious mind handling the basic decisions.

This whole system is like this because abstract higher thought is stupidly expensive resource wise. In extreme resource exhaustion scenarios the conscious mind will start to slow down and move almost all decisions to the subconscious to conserve resources. This is why you need rest and food to think clearly and make rational decisions.

 

[Kamen Rider Blade]

The brain can be trained. it is not as simple as what such "fake" researchers are "assuming" .

Lets look at a Karate fighter , or a Boxer , ETC ...

at the beginning of his training he cant use his brain , just a slow punch will hit him... the eyes and ears ETC catches the DATA but the brain is slow in "thoughts" how to counter.

After a year in training , the fighter will block/dodge 20 fighters around him ALL throwing attacks on him ... the brain will think about ALL the attacks , calculate fast and THEN decide 20+ counter attacks/dodge/block 360 degrees ... and think for the next step for EACH attacker , AND probabilities of ESCAPE and Movement or advance and movement , WITH balance thoughts and whatsnot.

so keep your 10 bits per second to your self. I believe that the brain has no limits ... the more you train it the more it will process thoughts . there is NO LIMIT.

These researchers are studing the human brain is "Sleep mode" "idle mode" , if compared to PC and NOT "full load all cores running"

I think you're missing the forest from the trees.
All that training develops muscle memory & habits.
Repetition & Training that programs your body to act at a sub concious level.

This way the inner concious mind can act swiftly and trigger it's pre learned routines easily & quickly. Almost like a fighting game.

The 10 bits is for the average persons upper limit of their mental engine.
Most of the time you won't need it or use it's full capacity unless you're doing something intense that requires that mental load & real time processing capabilities.

Some smarter folks with larger mental processing capabilities would realistically be able to have the equivalent of more bits to process info simultaneously. So I wouldn't be surprised if some people can process with > 10-bits.

Age & physical health might affect the # of bits processing capacity as well.

 

[newtechldtech]

I think you're missing the forest from the trees.
All that training develops muscle memory & habits.
Repetition & Training that programs your body to act at a sub concious level.

This way the inner concious mind can act swiftly and trigger it's pre learned routines easily & quickly. Almost like a fighting game.

The 10 bits is for the average persons upper limit of their mental engine.
Most of the time you won't need it or use it's full capacity unless you're doing something intense that requires that mental load & real time processing capabilities.

Some smarter folks with larger mental processing capabilities would realistically be able to have the equivalent of more bits to process info simultaneously. So I wouldn't be surprised if some people can process with > 10-bits.

Age & physical health might affect the # of bits processing capacity as well.

You are talking as if the 10bit per second is a fact ? no it is not.

You are missing the whole point of what I said , The Brain controls the body , and gives it orders , so you need the body to be ready for those "order" . thats not the "brain training" this is Body training.

Brain training s the ability to process the situation AND give the right orders to the "moving parts" ...

Also , the brain must process tons of data in my example , and then choose the right Action .. this alone is tons of information to be "thought out" THEN
choose from the "trained" counter movements for each case in milliseconds.

Not only that , there is 3D calculations and position of each person fighting , and each part of their body position and movement and distance and how fast it is , AND AND AND .. and at the same time takng the data from the ears to Listen to sounds from behind and blind spot , foot steps , Air pressure from thrown objects , calculate ALL this and then Decides how to counter ?

and if one of your body parts is injured there is a calculation which part to use and adapt to this ...

it is not as simple as you put it , there is Billions of "thoughts" being processed in the battle ... and it is not FIXED as well .

The training not only trains the muscles , it trains the brain ... makes it get out of IDLE state and start processing the battle and find a solution .. and this is ALOT of DATA and Thoughts ... the more you train it the more the brain process it around ..

do you think in a fight , the brain calculating a thrown object trajectory is an easy "thought task" ? and it happens in millisecond ? and in the middle of some 20 persons around you at the same time ?

The more you train your mind , the more it uses its "idle" potential .

and then they work in parallel ALL together ... if you dont train it , it stays IDLE but it is THERE needs to be unlocked.

The brain is a Miracle .. no one knows how it really works.

 

[Kamen Rider Blade]

You are talking as if the 10bit per second is a fact ? no it is not.

You are missing the whole point of what I said , The Brain controls the body , and gives it orders , so you need the body to be ready for those "order" . thats not the "brain training" this is Body training.

Brain training s the ability to process the situation AND give the right orders to the "moving parts" ...

Also , the brain must process tons of data in my example , and then choose the right Action .. this alone is tons of information to be "thought out" THEN
choose from the "trained" counter movements for each case in milliseconds.

Not only that , there is 3D calculations and position of each person fighting , and each part of their body position and movement and distance and how fast it is , AND AND AND .. and at the same time takng the data from the ears to Listen to sounds from behind and blind spot , foot steps , Air pressure from thrown objects , calculate ALL this and then Decides how to counter ?

and if one of your body parts is injured there is a calculation which part to use and adapt to this ...

it is not as simple as you put it , there is Billions of "thoughts" being processed in the battle ... and it is not FIXED as well .

The training not only trains the muscles , it trains the brain ... makes it get out of IDLE state and start processing the battle and find a solution .. and this is ALOT of DATA and Thoughts ... the more you train it the more the brain process it around ..

do you think in a fight , the brain calculating a thrown object trajectory is an easy "thought task" ? and it happens in millisecond ? and in the middle of some 20 persons around you at the same time ?

The more you train your mind , the more it uses its "idle" potential .

and then they work in parallel ALL together ... if you dont train it , it stays IDLE but it is THERE needs to be unlocked.

The brain is a Miracle .. no one knows how it really works.

All that is part of training and muscle memory. To get better you need to train out all those things, then your body is able to process them effectively in combat. That's why training in the dojo or on the range or in the field is critical. It gets your body accustomed and ready to know what to do.

I think we have an issue of what the definitions are. What we call One thing, you're calling a different thing. But the general jist is the same.

There are tons of people who study the brain and how it works. It's a ongoing learning cycle.

This is one part of it.

 

[mothdusty]

There is a lot of assumptions by commenters about this, and a lot of negative reception surprisingly. 10 bits is a bit misleading imo, as we don't truly understand the compression techniques our brain might use. If we were to imagine a rather unoptimized machine using 10 bits to speak, eat, etc. it would be a rather slow process. But that's because we've never tried to make efficient programs for such tasks.

To put it into perspective, a SNES can generate an entire simulated 2d world with basic physics at 30fps. This is an extremely inefficient system compared to our brains, and is only 8 bit. With vastly increased efficiency and nigh infinite supporting VRAM (the rest of our brain) I truly wonder how much could be accomplished with a 10 bit system. This isn't about processing power, but speed, and if compression in our brains is advanced enough the bitrate wouldn't truly matter. How that bitrate makes use of such highly compressed data is much more interesting.

And obviously everyone's brain is different. But the negative responses here, as well as the resuscitation of the multitasking argument, just reminds me of main character syndrome:
"My brain doesn't run at 10 bits because I'm so amazing at doing so many things so fast" is a truly contrived response to this and signifies a narcissistic attitude that doesn't actually engage and digest the information they're given.

 

[greymaterial]

I miss the point where they quantify information processing in the brain in digital. As far as I can imagine, neuron action potentials and probablities of depolarization events should be understood in analogue sense, and these in the multiples of how many neurons are active at any given moment will be large enough that it is not some "10 bit/s" unless the A to D process of the quantification can justify the uninformative nature of almost all of the neuronal activities.

 

[bit_user]

I think you're missing the forest from the trees.
All that training develops muscle memory & habits.
Repetition & Training that programs your body to act at a sub concious level.

This way the inner concious mind can act swiftly and trigger it's pre learned routines easily & quickly. Almost like a fighting game.

The 10 bits is for the average persons upper limit of their mental engine.

Right. It's conscious thought they clearly seem to be talking about. If you can build reflexes, then it offloads processing from your conscious decision-making machinery.

One pleasant surprise I had from a couple years of daily foosball games with some lads at the office was an uncanny ability to catch falling objects without even thinking about it. Before my conscious mind even finished deliberating on the fact that I had knocked over some item and what action should be taken, my hand was already reaching out to intercept it and I'd catch it like 9 times out of 10. Not having played any serious sports as a kid, I don't consider myself particularly well-coordinated, so this new ability came as quite a surprise.

Most of the time you won't need it or use it's full capacity unless you're doing something intense that requires that mental load & real time processing capabilities.

Right, I think they're talking about the maximum limit of conscious decision-making.

However, it's 10 bits per second, not like continuously processing data in 10-bit chunks. The way I imagine it (and I might be taking liberties here, since I haven't read more than the paper's abstract), is that you're moving through a decision-space at 10 bits/s. So, if you're making a sequence of simple binary decisions, it could perhaps amount to 10 decisions per second, while decisions involving a larger range of options would occur at a slower rate that still amount to a max of 10 bits of externally-visible information being generated at 10 bits/s.

An example of a larger decision-space would be speaking. There, you're deciding on a message and how to structure it, then finally which words and phrases to use. Each of these decisions is typically more than binary, but still choosing among a somewhat limited set of options. By the time you're picking words, each word is not independently selected out of thousands, but from among mere dozens (or fewer) that would make sense in that context. Since we speak at only a couple words per second, this usually gives our brains enough time to make the various decisions involved in speech. Not always, as you can tell by how common pauses are, in conversations.

Age & physical health might affect the # of bits processing capacity as well.

I'm sure. I think they were looking at the limits of human capacity, so it'd be an upper limit.

 

[bit_user]

To put it into perspective, a SNES can generate an entire simulated 2d world with basic physics at 30fps. This is an extremely inefficient system compared to our brains, and is only 8 bit.

A very simple 2d world, yes. The CPU is based on the W65C816S, which has a 16-bit word length (i.e. the size of internal registers and what the instructions natively process). The external data bus is only 8-bits, however. This is quite similar to the Intel 8088, which was the mainstay of early 80's IBM PC home computers and is classically regarded as a 16-bit CPU.

With vastly increased efficiency and nigh infinite supporting VRAM (the rest of our brain) I truly wonder how much could be accomplished with a 10 bit system.

Not sure exactly what you mean by VRAM, but human working memory is incredibly small. I'm not familiar with the underlying neuroscience, but you often hear people talk about a limit of 7 words you can hold in your working memory, and that's usually if you're concentrating and not doing anything else.

This isn't about processing power, but speed, and if compression in our brains is advanced enough the bitrate wouldn't truly matter. How that bitrate makes use of such highly compressed data is much more interesting.

I think a good example of this is the memory palace technique for storing longer lists of items. By finding associations and building a narrative structure connecting the different items, the dimensional space of storing each item seems to be effectively reduced.

And obviously everyone's brain is different. But the negative responses here, as well as the resuscitation of the multitasking argument, just reminds me of main character syndrome:
"My brain doesn't run at 10 bits because I'm so amazing at doing so many things so fast" is a truly contrived response to this and signifies a narcissistic attitude that doesn't actually engage and digest the information they're given.

Agreed. This sort of response seems to arise quite frequently in any discussions about AI. I think it generally fits into the class of status threats.

 

[bit_user]

I miss the point where they quantify information processing in the brain in digital.

From the paper's intro:

"The general approach is to assess the range of possible actions that a person may execute in a given time. Along the way, one needs a clear criterion to distinguish the action from its noisy variations. This distinction of “signal” and “noise” is quantified by Shannon’s entropy and ultimately leads to an information rate, expressed in bits/s.

This information-theoretic approach allows us to compare the speed of processing across different mental tasks and processes, between different neural structures in the same brain, across different species, and between brains and machines. This is just one framework for characterizing human experience, but it offers valuable insights through comparative analysis."
​

Wikipedia describes Shannon Entropy like this:

Shannon's theory defines a data communication system composed of three elements: a source of data, a communication channel, and a receiver. The "fundamental problem of communication" – as expressed by Shannon – is for the receiver to be able to identify what data was generated by the source, based on the signal it receives through the channel. Shannon considered various ways to encode, compress, and transmit messages from a data source, and proved in his source coding theorem that the entropy represents an absolute mathematical limit on how well data from the source can be losslessly compressed onto a perfectly noiseless channel.
​

 

[Kamen Rider Blade]

Right, I think they're talking about the maximum limit of conscious decision-making.

However, it's 10 bits per second, not like continuously processing data in 10-bit chunks. The way I imagine it (and I might be taking liberties here, since I haven't read more than the paper's abstract), is that you're moving through a decision-space at 10 bits/s. So, if you're making a sequence of simple binary decisions, it could perhaps amount to 10 decisions per second, while decisions involving a larger range of options would occur at a slower rate that still amount to a max of 10 bits of externally-visible information being generated at 10 bits/s.

An example of a larger decision-space would be speaking. There, you're deciding on a message and how to structure it, then finally which words and phrases to use. Each of these decisions is typically more than binary, but still choosing among a somewhat limited set of options. By the time you're picking words, each word is not independently selected out of thousands, but from among mere dozens (or fewer) that would make sense in that context. Since we speak at only a couple words per second, this usually gives our brains enough time to make the various decisions involved in speech. Not always, as you can tell by how common pauses are, in conversations.

I think of it as a average person having the ability to process the equivalent of a 10-bit register on the upper end. How many bits of individual person might have, will very drastically. The average will be 10 bits. Processing of information will scale up and down accordingly, along the bit based binary tree.

 

[greymaterial]

From the paper's intro:

​

"The general approach is to assess the range of possible actions that a person may execute in a given time. Along the way, one needs a clear criterion to distinguish the action from its noisy variations. This distinction of “signal” and “noise” is quantified by Shannon’s entropy and ultimately leads to an information rate, expressed in bits/s.​

​

This information-theoretic approach allows us to compare the speed of processing across different mental tasks and processes, between different neural structures in the same brain, across different species, and between brains and machines. This is just one framework for characterizing human experience, but it offers valuable insights through comparative analysis."​

​

Wikipedia describes Shannon Entropy like this:

​

Shannon's theory defines a data communication system composed of three elements: a source of data, a communication channel, and a receiver. The "fundamental problem of communication" – as expressed by Shannon – is for the receiver to be able to identify what data was generated by the source, based on the signal it receives through the channel. Shannon considered various ways to encode, compress, and transmit messages from a data source, and proved in his source coding theorem that the entropy represents an absolute mathematical limit on how well data from the source can be losslessly compressed onto a perfectly noiseless channel.​

​

so it is a naive research process that limits to only selected actions are seen as information processing, then a high bit rate random walker can be smarter than human for that sense. not quite insightful imo.

 

[bit_user]

so it is a naive research process that limits to only selected actions are seen as information processing, then a high bit rate random walker can be smarter than human for that sense.

By definition, a random walk is random and thus contains all entropy and no information.

not quite insightful imo.

See the paper link, for their rationale about the significance of their conclusion.

 

[greymaterial]

By definition, a random walk is random and thus contains all entropy and no information.


See the paper link, for their rationale about the significance of their conclusion.

Entropy definition including noise and error rate is not applied to their method of assessment, but only considering "making an action" as a form of quantification to bit rate to the conscious. Of course, I get their point about how there is a discrepancy between sensory processing speed and consciousness, but that seems to me an unfair treatment between two fundamentally different natures of neural activities: grey matter and white matter. High cognitive information "processing" is generally considered to happen in the grey matter superficial cortex, while sensory and more basic information routing, are both of different functions of neural activities.

 

[JRStern]

Tokens per second is very much analogous to what the authors of this study are talking about. Power consumption is something completely different.

No, no, no, Let me try again.
A computer program thinks for five minutes and makes a move.
This guy would call that one bit processing in five minutes.

 

[bit_user]

A computer program thinks for five minutes and makes a move.
This guy would call that one bit processing in five minutes.

Not a single bit, because the decision space is a lot larger than that. However, I think you have the right basic idea about their measurement methodology.

Clearly, not all thought-intensive activities will generate their peak output rate. That doesn't invalidate their point about what the limit appears to be.

To use a crude analogy, you could look at the speed of a car going up a steep hill. Since the car has to use a lower gear, it will be unable to reach its top speed in that setting. That doesn't make it meaningless to measure what the car's top speed is on a flat surface. It just means you won't see that same rate in all settings where it's engine-limited.