Under the Hood

Your CPU predicts the future. Every time your code hits an if, the processor guesses which way it goes and starts executing that path before the condition is evaluated. When the guess is right, you pay nothing. When it is wrong, the pipeline flushes and restarts. Fifteen to twenty wasted cycles. For most branches, the predictor gets it right over 95% of the time. But some branches resist prediction. Those are the ones worth eliminating — and eliminating them opens a door to something bigger. ...

You write a struct with three fields. A char, a double, another char. Ten bytes of data. sizeof returns 24. Fourteen bytes appeared out of nowhere. To understand where they came from, you need to see the grid your data lives on — and the two opposing forces that grid creates. Cache lines and the hardware grid Your memory is not a flat sea of bytes. The hardware divides it into 64-byte blocks called cache lines. ...

You have a test suite. One feature breaks. Forty tests go red. Not because forty things are wrong — because forty tests each validated the same contract they do not own. This happens more often than most teams admit. A session management change breaks connection pool tests. A database schema migration fails authentication tests. Every test that ever touched a shared precondition lights up, and you spend the first twenty minutes of debugging just figuring out what actually broke. ...

Alex Stepanov — the creator of the STL — had a simple litmus test for any programming language: Look at how it implements swap. Three lines. The simplest function imaginable. But what C++ lets those three lines become is a beautiful piece of engineering. The test: a smooth gradient of implementations Stepanov’s point was not about swap itself. It was about generic programming. You define what swap means — exchange the values of two objects. Then you ask the language: can you give me a smooth gradient of implementations, from the most generic to the most specialized, without losing efficiency at any level? ...

You have called push_back() a thousand times. But do you know how many times your vector actually reallocated memory to make that happen? Not a thousand. Not even close. About ten. Behind that simple interface, four mechanisms are working together — each one invisible during normal use, each one shaping your performance in ways that push_back() will never tell you about. Exponential growth and amortized O(1) When a vector runs out of capacity, it does not grow by one element. It allocates a new block that is some multiple of the current capacity, copies (or moves) everything over, and frees the old block. ...