Book notes: Where Good Ideas Come From

by Steven Johnson

Seven Patterns


Ideas are works of bricolage.

First-order combinations build on each other. New combinations users new combinations into the adjacent possible.

When ideas are “ahead of their time”, the first-order ingredients are not completely in place. E.g., Babbage’s Analytical Engine.

The trick is to get more spare parts on the table to use as first-order ingredients.


Networks to be effective: 1) need to have sheer size 2) must be very plastic and adaptable.

Carbon-life is so likely because abundance of molecules and the number of configurations that molecules can connect to it.

Must be at the edge of chaos: the fertile zone between too much order and too much anarchy

Information spillover: when sharing a culture in a high-density environment ideas tend to flow from mind to mind.

Talking shop: Most important ideas emerge during regular lab meetings, where researchers huddle around a conference table and informally present and discuss their latest work.

Work environments need to straddle balance between chaos and order. Modular office space is ideal.


Secret to hunch cultivation is to WRITE EVERYTHING DOWN.

The commonplace book is a fine line between order and chaos. You don’t want too much organization. Categories build barriers between disparate ideas.

In the past, reading and writing were inseparable activities. Reading was cross-referenced with other books.


The more disorganized your brain is, the smarter you are.

The electric noise of the chaos mode allows the brain to experiment with new links between neurons that ould otherwise fail to connect in more orderly settings. The phase-lock mode is where the brain executes an established plan or habit. The chaos mode is where the brain assimilates new information, explores strategies for responding to a changed situation. Chaos mode is a kind of background dreaming: a wash of noise that makes new connections possible.

Serendipity needs unlikely collision and discoveries, but it also needs something to anchor those discoveries.


  • Go for a walk, take long showers, or soak in tubs
  • Batch read from varied collections in a condensed amount of time to assimilate new ideas
  • Avoid the Filter Bubble
  • Make a Commonplace Book


Penicillin from mold due to science experiment mistake.

The errors of the great mind exceed in number those of the less vigorous one.

Sheer quantity ultimately leads to quality.

Error often creates a path that leads you out of your comfortable assumptions. Being right keeps you in place. Being wrong forces you to explore.

Good ideas are more likely to emerge in environments that contain a certain amount of noise and error.

DNA mutation introduces noise and opens up the adjacent possible. There’s a small but stable error rate in DNA transcoding that is an optimal balance between too much mutation and too much stability.

Mutation rate in human germ cells is roughly one in thirty million base pairs. This means genetic inheritance from parents to child comes with roughly 150 mutations.

Stress causes more mutation to occur. More mutations when life is a struggle may give just the advantage needed to survive. But is tempered by sex.


Gutenberg press was made possible by the wine press—borrowing a mature technology from an entirely different field, and putting to work to solve an unrelated problem.

Examptation: an organism develops a trait optimized for a specific use, but then the trait gets hijacked for a completely different function.


  • symmetrical feathers provide insulation. Assymetrical feathers act as an airfoil. A feather adapted for warmth is now exapted for flight
  • vacuum tube adapted to make signals louder, exapted to turn those signals into information
  • hyperlinks adapted for navigation, exapted for assessing quality via PageRank
  • narrative writing devices like stream of consciousness adapted for specific use, exapted as a new genre
  • skeuomorphism: desktop with pieces of paper exapted as graphical interface in computers

All decisive events in the history of scientific thought can be described in terms of mental cross-fertilization between different disciplines.

Concepts from one domain migrate to another as a kind of structuring metaphor, thereby unlocking some secret door that had long been hidden from view.

Cities are environments that are ripe for exaptation, because they cultivate specialized skills and interests, and they create a liquid network where information can leak out of those subcultures, and influence their neighbors in surprising ways.

Third place, a connective environment distinct from the more insular world of home or office. Coffeehouses. Breeding ground for cross-fertilization.

Diverse fields of expertise, horizontal social networks are three times more innovative than uniform, vertical networks.

Weak Ties Across Disparate Fields

A new technology developed in one idea-space can migrate over to another idea-space through these long-distance connections; in that new environment, the technology may turn out to have unanticipated properties, or may trigger a connection that leads to a new breakthrough.

Apple’s concurrent or parallel production: all the groups—design, manufacturing, engineering, sales—meet continuously through the product-development cycle, brainstorming, trading ideas and solutions, strategizing over the most pressing issues, and generally keeping the conversation open to a diverse group of perspectives.

Have lots of hobbies

Ben Franklin was a dilettante in many fields.

One project takes center stage for a series of hours or days, yet the other projects linger in the margins of consciousness throughout. That cognitive overlap is what makes this mode so innovative. The current project can exapt ideas from the projects at the margins. Allow the mind to move through multiple boxes. Moving from box to box forces the mind to approach intellectual roadblocks from new angles, or to borrow tools from one discipline to solve problems in another.


The platform builders and ecosystem engineers do not just open a door in the adjacent possible. They build an entire new floor.

Example: advent of GPS (by tracking Sputnik) has allowed Web 2.0 mashups; leading to APIs.

The Web is a layers upon layers of platforms.

Genres supply a set of implicit rules that have enough coherence that traditionalists can safely play inside them, and more adventurous artists can confound our expectations by playing with them. Genres are the platforms and paradigms of the creative world.

THE FOURTH QUADRANT (Non-Market/Network)

It is in the nature of good ideas to stand on the shoulders of the giants who came before them. This means is fundamentally a network affair.

Matrix based on Market/Non-Market and Individual/Network

The more the government thinks of itself as an open platform instead of a centralized bureaucracy, the better it will be for all of us, citizens and activists and entrepreneurs alike.

E.g., initiative.

All the patterns of innovation—liquid networks, slow hunches, serendipity, noise, exaptation, emergent platforms—do best in open environments where ideas flow in unregulated channels.

When you introduce financial rewards into a system, barricades and secrecy emerge, making it harder for the open patterns of innovation to work their magic.

Universities have ivory-tower isolation from the real world, but most of the paradigmatic ideas in science and technology have roots in academic research.

Fourth-quadrant innovation creates a new open platform that commercial entities can then build upon, either by repackaging and refining the original breakthrough, or by developing emergent innovations on top of the underlying platform.

He who receives an idea from me, receives instruction himself without lessening mine; as he who lights his taper at mine, receives light without darkening me. —Thomas Jefferson

The natural state of ideas is flow and spillover and connection.

In summary: Go for a walk; cultivate hunches; write everything down, but keep your folders messy; embrace serendipity; make generative mistakes; take on multiple hobbies; frequent coffeehouses and other liquid networks; follow the links; let others build on your ideas; borrow, recycle, reinvent.

Book notes: Where Good Ideas Come From

Book notes: Algorithms to Live By

by Brian Christian and Tom Griffiths

Optimal Stopping

37% Rule / The Secretary Problem / Marriage Problem
The optimal stopping rule prescribes always rejecting the first 37% of applicants after the interview and then stopping at the first applicant who is better than every applicant interviewed so far (or continuing to the last applicant if this never occurs). You will get the best candidate 37% of the time following this algorithm.

Merge Sort and Binary Search


Least Recently Used (LRU) Caching adaptations in the real world

  • Noguchi Filing System
  • Valet stand for most often used clothing
  • Library front book display area

Computational Kindness / Load (don’t make me think!)

  • Car parking starting from closest to store – first available space is the best (instead of using 37% rule)
  • Pick a few restaurants to choose from
  • Choose a few date/times for meetings
  • Bus stop with next bus arrival time sign
  • Block vs Spin: Restaurant wait-list vs open seating policy

pp. 259-261


  • Washing clothes: shortest washing times at the start, and shortest drying times at the end

Bayes’s Rule

Laplace’s law – estimate probability of future event based on previous results. The expectation is the number of previous wins plus one, divided by the number of attempts plus two:

(w)ins + 1

(n)umber of attempts + 2

Example: you flip heads 4 times out of 7 attempts. So 4 + 1 divided by 7 + 2 equals 5/9 or 56% chance. The more previous attempts, the more accurate the prediction.

This can be used to calculate the chance that the bus is late, baseball team will win, or things with much more previous results like chance baby is boy or girl or the sun will rise.

Copernican Principle or Mediocrity Principle (Bayes’ Rule without priors) – unless we know better (from priors) we can expect to have shown up precisely halfway into the the duration of any given phenomenon.

E.g., USA will most likely last another 250 years or so; Google until 2032; 7 days since last accident on the job, 7 days until next accident; bear activity in campground 10 days ago, bear again in 10 days. Looking at serial number on tram, double the number to make best estimate of number of trams

  • Normal distribution curve – lifespan of humans
  • Use Average Rule

  • Power-law distribution curve – average mean income in US
  • Use Multiplicative Rule

  • Erlang distribution curve – amount of time politicians stay in office
  • Use Additive Rule


Occam’s Razor – all things being equal, the simplest possible hypothesis is probably the correct one

Regularization solves the problem when things aren’t completely equal.

Regularization – introduce an additional term to your calculations that penalizes more complex solutions

Early Stopping is a form or regularization that forces you to stop thinking so much by limiting the number of variables that over-complicate things

E.g., Use a thick pen when designing something on paper at first so you don’t put in unnecessary details.

Darwin’s pro / con list got too detailed with too many pros and cons. He could have settled for just a few of the most important ones and made a good decision there.

How to stop a pro/con list? Perhaps by the of the page.


The perfect is the enemy of the good. —Voltaire

Constraint Relaxation – remove some of the problem’s constraints and set about solving the problem you wish you had. Then, after making a certain amount of headway, try to add the constraints back in.

For example, in the traveling salesman problem, allow salesman visit same town more than once or let him retrace his steps.

Relax constraints to get a good enough solution for solving life’s most vexing questions:

  • What would you do if you weren’t afraid?
  • What would you do if you won the lottery?
  • What would you do if all jobs paid the same?
  • What would you do ifyou could not fail?

Continuous Relaxation – most are discrete optimization problmes. No smooth continuum among its solutions. No shades of gray. Possible solution is to convert discrete to a continuous optimization. Instead of sending 100 invitations, send 50 to those with most connections who will everyone else. You turn whole numbers into fractions and do lots of rounding.

Lagrangian Relaxation – an optimization problems has two parts: Rules and Scorekeeping. Take some of the problem’s constraints and bake them into the scoring system instead. That is, we take the impossible and downgrade it to costly. Allows you to color outside the lines at some cost.

E.g., sports scheduling, overfilling wedding tables,


Three Tradeoffs: time, space, and CERTAINTY

Bloom filters – comes up with an answer which saves time and space but trades off error probablility

E.g., check to see if URL already one of 70 trillion cached URLs before caching. Bloom filter will give fast result with a 2% to 3% chance that it’s wrong


Exponential Backoff – increasing the average delay after every successive failure. maximum delay length forms an exponential progression

Game Theory

Recursion – part of bluffing; you know that they know that you know; but make sure to play only one level above your opponent

Book notes: Algorithms to Live By

Book notes: Smarter Faster Better

by Charles Duhigg


Snap out of situations where you get cognitive tunnel vision. Assess the problem and put in a mental model that makes sense for the situation.

Example: Qantas Flight 32 – so many engines and flight controls were failing that it was overwhelming. Instead of continuing with practiced scenarios and fixes, he scrapped everything and changed his mental model to a small prop Cessna. He just needed to get the plane to stall just as it touched the ground.


Intermediate Disturbance Hypothesis

Reefs, forests, etc are teeming with life due to fluctuating disturbances (i.e., lightning strike, spring floods (riparian zones), medium-sized waves, etc).

Somewhat related:

The most vibrant things are at the edge. Where two disparate things meet. The meadow and the forest.

Formula to the Creative Process:

  1. Use previous experiences – how did you feel during these experiences
  2. Creative desperation – time crunches make us flexible enough to seize something new
  3. Maintain some distance – force ourselves to critique what we’ve already done. Look at it from a completely different perspective. Disturbances are essential, and we retain clear eyes by embracing destruction and upheaval as long it’s the right size.

Absorbing Data

Mental Scaffolding / Filing

Example of choosing wine by year, color, varietal, price.

Creating Disfluency

Force yourself to manipulate the raw data and come up with meaningful trends. Don’t rely on fancy programs that does all the heavy lifting for you. Use spreadsheets.

Write things down by hand. The harder it is the better. Come up with your own sentences INSTEAD of copying verbatim. Draw things out.

This is mentioned elsewhere as “desirable difficulties.” Write things by hand. Don’t copy and paste. Make flashcards.

Book notes: Smarter Faster Better