Innovation as an evolutionary process

Differentiate invention from innovation

  • Invention is the original creative act of coming up with something new, usually based on a new understanding of the world (new knowledge) through the scientific method
  • Innovation implies developing an invention to the point where it is used at scale because it is sufficiently practical, reliable, and affordable to be worth using

Innovation is the workhorse of the progress we’ve witnessed since the Enlightenment, but it is a poorly understood process - why and how does it happen?

Innovation as an evolutionary process

  • Unit of selection: a technology is an idea manifested as a product or service
  • Variation
    • The usual process of variation in idea evolution: make a conjecture (see David Deutsch), manifested as building a product or offering a service
    • Concretely: copy and tweak the leading technology, combine multiple different technologies, incorporate customer feedback
  • Selection
    • The usual process of selection in idea evolution: experiments and criticism (see David Deutsch), manifested as feedback from customer use
    • Concretely: customer feedback on the product or service to select features, customer choice between different products or services to select products/services
    • This feedback loop is a fundamental feature of innovation that grounds technology into what is useful to people

Examples of innovations that followed an evolutionary process (with detailed accounts in the book)

  • Energy and transportation: the steam engine, fracking, the locomotive and railway, the internal combustion engine and car, the plane
  • Health: vaccination (inoculation), water chlorination, antibiotics (penicillin), mosquito bed nets
  • Food: synthetic nitrogen fertilizer, genetically modified crops
  • Computing: the computer, the personal computer, the smartphone, ever shrinking transistors (Moor’s law), the Internet, search engines, social media, machine learning

In contrast

  • Nuclear fission power failed to follow an evolutionary process as variation under increasing regulation was too costly
  • Genetically modified crops, although extremely safe, failed in Europe under activism ungrounded in facts and regulatory pressure

Features of innovation as an evolutionary process

  • Innovation is a gradual and messy process by trial and error that involves many different innovators, both scientists and entrepreneurs and their teams
    • We can rarely single out a single breakthrough moment of invention or a single inventor in the history of a technology
    • Because it is a process of trial and error, it requires a culture of experimentation and failure - the Silicon Valley startup culture or Amazon’s culture of experimentation with a history of many failed products
  • Technology is recombinant: innovations combine existing building blocks
    • This makes it a compounding process: the more (and more complex) building blocks there are to evolve from, the faster the rate of improvement
    • It explains why innovation historically happened in trading hubs where ideas could most effectively be shared
  • Technology adoption and rate of improvement follows a wave pattern (an S-curve)
    • Slow take-off followed by increasingly fast pace through a virtuous cycle of customer use leading to more practical, reliable, and affordable iterations on the technology, in turn appealing to more customers, until market saturation
    • We often overlook technologies as toys because they undershoot user needs at the beginning of the S-curve
    • When we’re convinced a technology is not a toy, we often overestimate its pace of improvement in the short term and underestimate it in the long term because we fit a line to the S-curve
    • A technological wave often builds on meta-waves (general purpose technologies)
  • The most widely used technology improves fastest because it attracts the most R&D attention, which brings further customer usage
    • This rich get richer effect explains winner-takes-all dynamics: beyond network effects, simply being perceived as the market leader attracts risk-averse users and third-party development effort (integrations, etc.)
    • It is also present in academia: the most cited paper on a topic gets read and cited because it is the most cited paper
  • Technology evolution is accelerating due to
    • More technologies to build from and recombine
    • More people innovating (in the recent past, China in particular)
    • Better tools to accelerate processes of variation and selection: the personal computer, the Internet, and AI
    • A refined understanding and operationalization of the innovation process through the VC industry and the modern startup ecosystem: iterative rounds of funding to de-risk ventures, the lean methodology with emphasis on customer feedback, strong localized networks of builders, investors, early adopters, and acquirers
    • Feedback loops in the evolutionary process (e.g., AI helping to design chips)
  • Technology development and use often precedes and demands scientific understanding
    • It is a two-way relationship, as opposed to the one-directional path from science to technology many have in mind
    • Steam engines preceded thermodynamics, powered flight preceded almost all aerodynamics, animal and plant breeding preceded genetics
    • The practice of inoculation/vaccination and antibiotics long preceded the understanding of why they work
    • Neural networks have revolutionized AI, but we still have a very limited understanding of why they work
  • Innovation is a bottom-up process
    • The government is usually a secondary actor in the evolution process, most of the evolution happens in the private sector through market competition
    • China today is a workhorse of innovation because of the laissez-faire attitude of the government towards ambitious entrepreneurs; innovation will halt when this changes
    • Europe today is slow to innovate because of the regulatory burden on entrepreneurs and the lack of a culture of entrepreneurial experimentation and failure; the latter is changing for the better but the former is not
  • Innovation increases interdependence: we are more specialized in what we produce but more general in what we consume