Why We're Terrible at Predicting the Future (And How to Fix It)

You’ve seen hundreds of new inventions come into the world during your lifetime. 

In just the last 20 years there’s been the rise (and fall for some) of:

• iPods
• Email
• Social media platforms
• Online video games
• AWS
• Blu-ray
• Smartphones
• Electric cars
• Health and fitness devices
• Bitcoin

I could go on but we’re not here to talk about new technology. I’m here to point out our failure to estimate the impact of it. The error of minimal expectations (and how you can avoid it).

Even experts fail to estimate the impact of new technologies. Being well-informed does not mean you understand the consequences or impact something entirely new, on the frontier of technological innovation.

A British parliamentary committee believed Thomas Edison’s lightbulb was “unworthy of the attention of practical or scientific men.” Edison himself only thought his invention of the phonograph would be used for business dictations.

The Wright brothers demonstrated flight after “heavier-than-air” travel was thought impossible. Despite the proof, astronomer, William H. Pickering said “it is clear that with our present devices there is no hope of competing for racing speed with either our locomotives or our automobiles.” In a single statement the world accepted commercial air travel wouldn’t work.

There is a frequent misunderstanding. When it comes to new technology our expectations are set too low. But the ones that escape this error in reasoning leapfrog the competition around them.

The slow burn of innovation

Roy Amara began making the point that we suck at forecasting innovation. Later known as Amara’s Law.

Mercedes knew more about the automobile industry than anyone else. Yet they failed to forecast the future of their cars. In the beginning the founders, Karl Benz, Gottlieb Daimler, Wilhelm Maybach and Emil Jellinek, didn’t believe there would be more than a million cars in use worldwide (for some perspective they sold 2 million of their cars in 2022).

Amara’s Law states: people tend to overestimate the impact of new technology in the short run, but under-estimate it in the long run.

A simple explanation for why you often miss new technologies that later upend old ones.

Innovation is a gradual process. Unrealistic expectations create the Amara hype cycle, where all new innovations disappoint in the beginning. The trough of progress leads to the common error in expecting less.

In How Innovation Works, Matt Ridley says “I reckon these days it is fifteen years down the line. We expect too much of an innovation in the first ten years and too little in the first twenty, but get it about right looking fifteen years ahead.”

The effect is slow, like social media replacing newspapers, but obvious in hindsight.

The cost of playing it safe

According to Arthur C. Clarke, a science fiction writer, there are two reasons why you fail to anticipate the future: failure of nerves and failure of imagination.

The most common is failure of nerves. You can easily imagine a future that looks like a Jetson’s episode. You can imagine the possibilities, but lack the nerve to risk being wrong. 

You would rather fail to see the future than make a prediction that’s overblown or embarrassing in retrospect.

In 2006, Facebook became available to the public. The next year the iPhone launched. Followed by the creation of Airbnb and Tesla Roadster. All of these are giant companies today. During this same period, intelligent and talented people were wasting their potential away. Distracted from the disruption. 

More than half of Harvard graduates in 2007 went to work as an investment banker or management consultant.

Instead of putting creative potential to use, they were diverted towards a role where they move money or provide recommendations on how others should use their resources. Both lucrative roles. But an extremely unproductive way to shake something up in the world.

I know what you’re probably thinking. What’s wrong with investment banking? It’s a difficult and prestigious job. But take a second to imagine how much innovation is lost every year by the potential that’s wasted because of their failure of nerves to pursue a new future.

Spotting the next big thing

Escaping the error of minimal expectation that plagues 80% of people is the easiest way to become a leveraged expert. Your nerve to take a risk combined with your imagination to see the potential future that’s possible will springboard your career.

There are two things that are strong signals for a new technology is in the middle of the Amara hype cycle:

1. Where attention goes, money flows

The rise of arcades around Boston caused a series of reactions from local parents and government. Parents complained that video games “will rot your brain”. The city of Boston refused to license video games in residential areas because kids spent too much time playing Pac-Man.

A lot of attention was being spent on arcade games. Even after trying to shun them, the video game industry is a market worth more than $300B today.

When people are spending a significant amount of time on something that’s a strong signal. If communities are forming around the technology that’s a stronger signal. When the technology has build a cult like following (i.e., the internet) that’s the strongest signal there is.

If one particular innovation is getting a lot of attention it’s probably worth doing your own research. Collect the facts to see if there’s an inescapable conclusion you come to.

2. Minimum idea alignment

An idea doesn’t have to be adopted by the majority of people to be feasible. When it has a weak opposition against it in the beginning.

In his 1970 patent filing, Bernard Sadow wrote “the luggage actually glides. Further, substantially any person, regardless of size, strength or age, can easily pull the luggage along without effort or strain.”

Sadow was describing the wheeled suitcase. A man landed on the moon before we put wheels on luggage. Five other attempts were made in the past to add wheels to bags. But all of them failed. They lacked minimum idea alignment.

Prior to the expansion of air travel, airports were small, men worried about looking weak, and bag boys were everywhere. 

Wheeled luggage didn’t get idea alignment until the Rollaboard created by Robert Plath. A pilot who put wheels on a bag and sold it to other pilots he worked with. Once passengers saw the pilots using them they wanted the same luggage for themselves. Pilots offered the right place for the idea to take hold.

All new innovations (even low-tech) take a significant amount of effort to be adopted by the masses. When there’s opposition or pushback the idea must be stronger than the consensus against it.

Your job is to size up the forces battling against an idea. Minimum idea alignment is required for an idea to blow away expectations, but mass acceptance is not in the beginning.