The Art of Doing Science and Engineering – Learning to Learn

Richard W. Hamming. 1997.
The prices for the paperback from Amazon resellers are ridiculous. You can find a high quality PDF version on the web.

The unexamined life is not worth living.
Socrates (469-399 BC)
Although the book is now 20 years old and written by a scientist who lived in the last millennium, the importance and value of its lessons cannot be overstated. It is difficult to list only the most important points and naturally they will be different for every reader.
Nevertheless here are some practical hints from the book:

(Professional) Life:
  • Have a vision! The drunken sailor finds himself after n random steps only about √n steps from his origin, but with a goal you can go a distance proportional to n.
  • Assume responsibility for what you believe! (Rethink and question everything you are told.)
  • Note that experts almost always reject paradigm shifts in their own field when they are introduced.
  • Innovations: It is first necessary to prove beyond any doubt the new thing, device, method, or whatever it is, can cope with heroic tasks before it can get into the system to do the more routine, and in the long run, more useful tasks.
Product developement and manufacturing:
  • When automatizing or digitizing work, don’t replace it exactly as it is. Always think about the essential function of the job the needs to be done.
  • Plan field maintenance right from the start for any product or service. In the long run it will often dominate all other costs.
Software:
  • He predicted that by 2020 he would expect that experts in a field will usually do the programming for solving problems and not need software engineers to do it for them.
  • Programming languages need to evolve and are still far from “natural” or efficient for human use. From 1960’s to 2000 roughly a factor 90 of coding efficiency improvement has been achieved by higher level languages.
  • Software developement models: The most important one is think before you code (acceptance tests, field maintenance, are you solving the right problem,…)
  • Programming is closer to novel-writing than it is to classical engineering:-)
Information Theory:
[math] I(p)=-log_2(p) [/math]
Where I(p) is the measure of information that you gain when a event of probability p happens.
  1. This is not a definition of information, but a formula to measure the amount.
  2. The measure depends on the surprise and it does not represent the normal human attitude towards information.
  3. It is a relative measure, it depends on the state of your knowledge. If you are looking at a stream of “random numbers” from a random source then you think each number comes as a surprise, but if you know the formula for computing the “random numbers” then the next number contains no surprise at all, hence contains no information
  4. It should have been called “Communication Theory” and not “Information Theory”
Many, many more important topics are covered: Coding Theory, Simulation, Artificial Intelligence, Education, Experts, Creativity, Systems Engineering,… to name a few.

Manna: Two Visions of Humanity’s Future

Marshall Brian. BYG Publishing, Inc.; Auflage: 1 (5. März 2012). Kindle edition. (Original story from 2003)

Taking into account that the story is originally from 2003, it is impressive how current the discussion of AI and it’s implication for humanity is. The novel describes two very different societies that could arise from extrapolating current technological developments. One is very dystopian and describes a world where most are unemployed and only a few super-rich remain. They control the technology and through that the whole society, all the other humans are kept alive in houses similar to industrial live-stock farming…
M.Brain argues that if we don’t act this will be the likely future we are heading towards.

The second society is technology-utopian, where all positive technological and scientific possibilities are realized. It is an egalitarian society which is only limited by the amount of resources each person is allowed to use. Nobody has to work and people are completely free to do what they want (as long as they don’t breach a set of predefined rules). Most humans spend their time with creative work in arts, design or science, while some move completely to virtual reality.

I like the concrete and specific description of the two worlds, but I think it is a bit too black and white. Also I missed a serious discussion of the risks and possible failures of the utopian society. A few critical issues are mentioned, but in my opinion not convincingly resolved. E.g in a society which is that technologically advanced (full brain-computer interface, robots that can produce everything for free, an AI which controls that everybody behaves according to the rules…) how comes that only humans are creative and not computers? How did they avoid super-intelligence of AI?

Life 3.0: Being Human in the Age of Artificial Intelligence

Max Tegmark. Knopf (29. August 2017). Kindle Edition.

If we don’t keep improving our technology, the question isn’t whether humanity will go extinct, but how.
Having read already many books on this topic (Superintelligence, How To Create a Mind, Our Final Invention,…) there was not that much I haven’t heard before. The focus of this book is on the impact on humans, society, ethics and a discussion what we can/should/want to do about it. It’s written in a language that is easy to understand for non-experts. The enthusiasm that Tegmark radiates for the topic is contagious.

Some interesting notes:
Limits of computation: approx 1050 operations/sec for a 1kg, 1L computer. Way to go Moore! Even if the ultimate limit cannot be reached by 10 orders of magnitudes, the exponential speed-up of computation could still continue for more than 100 years!
Combined with the speed-limit of light one can conclude for artificial consciousness:
  • A brain-sized artificial consciousness (AC) could have millions of times more conscious experiences per second than us
  • An earth-sized AC could have about 10 global conscious experiences /sec – similar to us humans
  • And a galaxy-sized AC would only have one all-including thought every 100’000 years
See Seth Lloyd: Ultimate physical limits to computation and Computational Capacity of the Universe

Natural laws and life: A key aspect of a living system is that it maintains or reduces its entropy by increasing the entropy around it (by a larger factor than the decrease).
See Schrödinger “What is Life?“.
And Jeremy England‘s theory of “dissipation-driven adaption” that explains how the existence of life could be driven by the laws of physics.

Why Information Grows: The Evolution of Order, from Atoms to Economies

Cesar Hidalgo. Basic Books (2. Juni 2015). Kindle Edition.

In this book C. Hidalgo, a professor at MIT, explains the growth of economies based on information theory and physics. In short, economies grow because the information contained grows. But what is information? How is it related to entropy? And how to states of physical order? Why are there pockets of high order (e.g. life, intelligence) in an universe where entropy only increases?
Information is defined as the physical order(arrangement) of things. Though most information is not contained in perfectly ordered structures, but in fractal-like, aperiodic structures that have correlations at many different scales.
According to Nobel Prize winner Ilya Prigogine: Information emerges naturally in the steady states of physical systems that are (far) out of equilibrium (for example a whirlpool). He showed in 1947 that the steady state of out-of-equilibrium systems minimizes the production of entropy.

For information to be able to grow, additional conditions are necessary:
Existence of solid states – to increase stability of information and to allow for accumulation of information.
And the ability of matter to compute – the selective accumulation of information.
Matter can compute on many different scales, from molecules to bacteria up to human brains and a companies. Hidalgo’s ideas on computation are related to the idea of intelligence as emergent properties of networks by F. Vertosick.

Hidalgo then translates these information concepts into a theory of economics. Although an extremely interesting idea, I think the step from basic physics to humans and economies is a bit too large (and quite a jump in the text).
He first introduces a quantization of knowhow and knowledge:
Personbyte -the maximum knowledge and knowhow carrying capacity of a human (not a precise limit). A limit that requires a network of persons (e.g. a company) to allow for an increase in knowhow and knowledge.
Firmbyte– quantization limit analogous to the personbyte, but instead of requiring the distribution of knowledge and knowhow among people, it requires them to be distributed among a network of firms.

Hildago then shows how his theories can explain why only diversified economies are capable of complex economic activities and that over the long run a region’s level of income approaches the complexity of its economy (compared to similar economies). He defines a measure of economic complexity based on the diversity and ubiquity of products that a region exports. He showed that this measure is highly predictive for the long term economic growth. Data for many countries and products are published here: http://atlas.media.mit.edu/en/

Axiomatic

Greg Egan. Gollancz (30. Dezember 2010).Kindle Edition.

A collection of science fiction short stories by Greg Egan.
Most of the stories were highly entertaining and contained fantastic novel ideas. What would a world look like where everybody could send very bandwidth-limited information back in time? Why are unstable wormholes appearing at random locations on the planet? Could we create life based on artificial nucleobases instead of the standard bases and what would that mean?
I loved many ideas and how they are based on real and current scientific discoveries from physics, chemistry and biology.
Though not all stories were original, quite a few reminded me of Stanislaw Lem, maybe clothed in a bit more modern style.
I’m definitely going to read more by Greg Egan.

Competing Against Luck: The Story of Innovation and Customer Choice

Clayton M. Christensen. HarperBusiness; Auflage: 1 (4. Oktober 2016). Kindle Edition

Innovation is less about producing something new and more about enabling something new and important for customers.
Christensen’s latest book shows a path to more predictable innovation. He introduces the “Theory of Jobs to Be Done” as a tool to discover market opportunities. The leading question is:
What job is the customer hiring the product or service for? (Why are they buying it).
A “job” is defined as the progress that a person is trying to make in a particular circumstance.
The important words are “progress” and “circumstance” indicating that it goes far beyond classical customer segmentation. Context is king! A customer is often only a customer in very specific circumstances (e.g. when hungry or looking to buy a house,…).
A “job” has (at least) three important dimensions: Functional, Social and Emotional. Many companies make the mistake to focus on the functional aspect only.
A well-defined job will be a guideline for innovation and is more specific than “needs”or “personal values”. A helpful method is to imagine one would be filming a minidocumentary of a person(customer) struggling to make progress in a specific circumstance.
  1. What progress is that person trying to make?
  2. What are the circumstances of the struggle?
  3. What obstacles are getting in the way of the person making that progress?
  4. Are consumers making do with imperfect solutions through some kind of compensating behavior?
  5. How would they define what “quality” means for a better solution, and what trade-offs are they willing to make?
A well-defined job also helps to see who the true competition is (e.g. smoking a cigarettes could be a competition to Facebook!) or it can help identifying groups of “nonconsumers” which currently do not hire any solution.
Where to look for jobs:
  1. Own life
  2. Nonconsumers
  3. Compensating Behavior
  4. How customer use products
  5. What people don’t want to do
“Big-hire”: The moment a customer buys a product.
A question that is often neglected is: What has to get fired for my product to get hired? Consumers are used to do it a certain way or are used to live with the problem. Consumers have “anxiety of choosing something new.”
“Little-hire”: The moment a consumer actually uses (consumes) the product. Most companies track the big-hire, though often the little-hire is much more insightful.
When you interview customers a good idea is to draw storyboards of their struggles and then check if patterns occur among customers. Can you tell a complete story about how your customers go from a circumstance of struggle, to firing their current solution, and ultimately hiring yours (both the Big and the Little Hires)? Where are there gaps in your storyboard and how can you fill them in?
Successful companies don’t sell products but experiences! After uncovering a job you need to think about the desired experience for your customers. Do not only think about who should hire your product but also about who should not hire your product.
Organize your company around the job to be done (and not according to functional areas, geography,…). Use the job spec to define the processes in your company. Use the job spec to define and track the “right” metrics.
Three fallacies of innovation data:
  1. Active versus passive data
  2. Passive data (e.g. customer context)drives innovation, once you start selling risk is that active data replaces passive data and focus changes on active data.
  3. Surface Growth (sell more to existing customer and i.e. lose focus on job to be done)
  4. Conforming data – Confirmation bias
Often leaders spend significant time analyzing the data presented, instead they should spend more effort to determine what data should be created and what dimensions of the phenomena should be collected and what should be ignored.
Excellent business book, highly recommended. It will be interesting to combine the ideas of this book with lean startup.

This Will Make You Smarter: New Scientific Concepts to Improve Your Thinking (Edge Question Series)

John Brockman. Harper Perennial; Auflage: Original (14. Februar 2012). Kindle Edition.

The ideas presented on Edge are speculative; they represent the frontiers in such areas as evolutionary biology, genetics, computer science, neurophysiology, psychology, and physics. Emerging out of these contributions is a new natural philosophy, new ways of understanding physical systems, new ways of thinking that call into question many of our basic assumptions.
The Edge Question 2011:
What Scientific Concept Would Improve Everybody’s Cognitive Toolkit?
This book is a collection of 150 Scientific Concepts (short essays) answering the 2011 Edge question.
Here are some interesting ideas that are discussed in various contexts:
  • We are unique but we are not the center of the universe…
  • The scientific principle(s) should be applied to all areas of life
  • Be aware of bias(es)
  • Embrace uncertainty and learn to love probability
  • Positive-Sum Game
  • Path dependence – Which things have to be the way they are today and which were just due to randomness? (This is a tricky one!)
The ideas presented in this book are inspiring and often thought-provoking.