When you get cold, your nervous system signals tiny muscles at the base of your hairs. The muscles contract and pull the hairs straight up, raising small bumps.
In furry animals, the raised hairs trap a warm layer of air. Our ancestors were much hairier, but our sparse hair traps little, so some scientists call goosebumps a leftover reflex.
The name comes from plucked geese, whose skin shows similar bumps where feathers were.