From PageRank to word2vec to Attention
One old saying, three algorithms Physics is easy, life is hard At Fermi, my colleague Madhu taught a physics lesson on why physics is easy and life is hard. He talked about stable equilibrium. He said simple harmonic motion is the foundation of physics. It sounds like an exaggeration until you see why. Pull almost anything a little away from a stable equilibrium. For small moves, the force pulling it back grows in proportion to how far it moved. That is simple harmonic motion. A pendulum, a spring, a sound wave and an atom vibrating in a crystal follow the same equation, with different knobs. The physicist Sidney Coleman is often quoted joking that the career of a young theoretical physicist is treating the harmonic oscillator in ever-increasing levels of abstraction. ...