Why is positive premature optimization still wrong?

When ju­nior pro­gram­mers do pre­ma­ture op­ti­miza­tion, it is easy to see how it could harm the pro­ject. They guess that a giv­en change would make their code faster, and since they don't mea­sure the ac­tu­al per­for­mance, most of­ten than not, they end up with code that is not just hard­er to main­tain, but also ap­pears to be slow­er.

Now, some de­vel­op­ers with a bit more ex­pe­ri­ence in a giv­en lan­guage are of­ten tempt­ed writ­ing code in a way that makes it op­ti­mized. It hap­pens that, as they have an ex­ten­sive knowl­edge of the in­ner work­ings of a lan­guage or its frame­work, they also learn many tricks re­lat­ed to per­for­mance—lit­tle tech­niques that are a real ice break­er when they met their fel­lows dur­ing manda­to­ry cor­po­rate team build­ing ses­sions.

Those lit­tle op­ti­miza­tions, of­ten, do work, that is, they do make the code faster.

That's a good thing, right?

Re­cent­ly, I had an op­por­tu­ni­ty to work with such de­vel­op­ers.

Did you know that, in C#, Count > 0 is faster than Any()? They do.

Or did you know that it's faster to use x = Array.Empty(); than x = [];? They do.

What's the ac­tu­al im­pact, in terms of mem­o­ry, when you re­place a con­di­tion­al in a loop by a Where() at its source? I can't tell you that, but they can.

Let's do a few bench­marks.

As I said, one of the op­ti­miza­tions is to rely on Array.Empty<T>() when it comes to cre­at­ing an emp­ty col­lec­tion. Let's com­pare it to [].

using System.Collections.Generic;
using System.Linq;
using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Jobs;
using BenchmarkDotNet.Running;

BenchmarkRunner.Run<ArrayEmpty>();

[SimpleJob(RuntimeMoniker.Net10_0)]
[PlainExporter]
public class ArrayEmpty
{
    [Benchmark] public ICollection<int> ArrayDotEmpty() => Array.Empty<int>();
    [Benchmark] public ICollection<int> SquareSquare() => [];
}

For some rea­son, I al­ways thought that the com­pil­er will op­ti­mize [] by re­plac­ing it by some­thing that would be sim­i­lar to Array.Empty<T>(). It ap­pears that not only it doesn't do that, but the [] ap­proach ac­tu­al­ly cre­ates an in­stance of a col­lec­tion—an op­er­a­tion that costs 4.4 nanosec­onds on my ma­chine.

Let's try an­oth­er one. You know about CA1860? The one that says that when­ev­er you have a col­lec­tion, you should nev­er ever use Any(), un­less “per­for­mance isn't a con­cern.”

public class CountOrAny
{
    private ICollection<int> collection;
    [Params(0, 1, 10)] public int length;
    [GlobalSetup] public void Setup() => collection = [..Enumerable.Range(0, length)];
    [Benchmark] public bool Any() => collection.Any();
    [Benchmark] public bool Count() => collection.Count > 0;
}

On a col­lec­tion that has el­e­ments, us­ing .Count > 0 in­stead of .Any() saves up to 0.03 nanosec­onds. On an emp­ty col­lec­tion, it saves 0.15 nanosec­onds.

What about us­ing LINQ? Those guys re­al­ly de­spise LINQ, as, ac­cord­ing to them, it has a neg­a­tive im­pact on mem­o­ry. Dur­ing a re­cent re­view, a de­vel­op­er had a method that, giv­en a col­lec­tion, would fil­ter its el­e­ments based on a cri­te­ria, and re­turn the sum of the el­e­ments that passed the fil­ter. He used LINQ's Where() and Sum(), and was in­vit­ed by a re­view­er to rewrite the code, us­ing a sim­ple loop and a con­di­tion­al state­ment.

var values = Enumerable.Range(0, 100);

Console.WriteLine($"Loop: {MeasureMemory(() => Loop(values))} bytes.");
Console.WriteLine($"Partial: {MeasureMemory(() => Partial(values))} bytes.");
Console.WriteLine($"Linq: {MeasureMemory(() => Linq(values))} bytes.");

static long MeasureMemory(Action action)
{
    action();  // Warm up JIT.
    var before = GC.GetAllocatedBytesForCurrentThread();
    action();
    var after = GC.GetAllocatedBytesForCurrentThread();
    return after - before;
}

static int Loop(IEnumerable<int> collection)
{
    var sum = 0;

    foreach (var x in collection)
    {
        if (x % 2 == 0 || x % 3 == 0)
        {
            sum += x;
        }
    }

    return sum;
}

static int Partial(IEnumerable<int> collection)
{
    var sum = 0;

    foreach (var x in collection.Where(x => x % 2 == 0 || x % 3 == 0))
    {
        sum += x;
    }

    return sum;
}

static int Linq(IEnumerable<int> collection) =>
    collection.Where(x => x % 2 == 0 || x % 3 == 0).Sum();

Es­sen­tial­ly, every one of their sug­ges­tions was tech­ni­cal­ly true, that is, the so­lu­tion they were sug­gest­ing was ei­ther faster or used less mem­o­ry.

And still, they are com­plete­ly wrong.

It's like those peo­ple who would un­plug their USB charg­er when it's not in use, and feel well, be­cause they did some­thing great for the plan­et and their elec­tric­i­ty bill—miss­ing that they have that old 2 kW oven where you need to wait for an ex­tra ten min­utes to heat any­thing.

Do you re­mem­ber the last time you were spend­ing a week try­ing to fig­ure out why a nasty task takes hours in pro­duc­tion on beefy servers, but would hap­pi­ly run in a mat­ter of min­utes on your decade old lap­top? Com­pared to the painful hours spent at a tricky prob­lem, a straight­for­ward rule such as CA1860 looks like a bar­gain. But it is by no means zero-cost.

And I'm not even talk­ing about the time it takes typ­ing Array.Empty<T>(). I'm pret­ty sure there is a way to make an IDE make such change for you.

The ac­tu­al cost comes from how puz­zling the code is.

A sim­ple x = []; is great. It is not clear—if it was, I wouldn't be puz­zled by the ac­tu­al types be­ing used by the com­pil­er un­der the hood. But it is at least a very in­tu­itive ab­strac­tion, in­tu­itive even for be­gin­ners—in a sort of a mag­ic way where every time you use it, it will do what you in­tend­ed to do.

On oth­er oth­er hand, Array.Empty<T>() is not great. It's not an ab­strac­tion. It's low lev­el stuff creep­ing into code. Not only that, but it also pre­sents a risk of prop­a­gat­ing ar­rays into code, which is a bad thing.

How many times do you use AVX2 in your C# code? I'm pret­ty sure there are lots of places where you can save much more than 0.15 nanosec­onds by us­ing SIMD. And there are op­por­tu­ni­ties where in­tro­duc­ing SIMD would be as “easy and straight­for­ward” as adding Array.Empty<T>(). But we don't do that, be­cause it's low lev­el. Well, we do, but in ex­treme­ly rare cas­es where the pro­fil­er have shown that mov­ing to SIMD would make a spe­cif­ic com­pu­ta­tion much faster.

And this is it. The key is how much is be­ing saved, and is it worth it. Aside fun­ny case stud­ies, low lev­el op­ti­miza­tion has its place only when the pro­fil­er have shown that in­tro­duc­ing, and then keep­ing low-lev­el code has a gain that makes it worth the ef­fort. Out­side, a pro­fes­sion­al de­vel­op­er is ex­pect­ed to write code that is clear and clean, code that com­mu­ni­cates in­tent, and that re­lies on the ab­strac­tions, as soon as they are in­tu­itive and nat­ur­al. And not prone to er­rors; that's im­por­tant too. Con­sid­er the CA1860 ex­am­ple:

collection.Any()

is as sim­ple and ab­stract as it could get. There is zero chance to make a mis­take here. Not so much for:

collection.Count > 0

as a > mis­spelled as >= could break hav­oc. “What hav­oc?—would you ask—The er­ror would be caught im­me­di­ate­ly by unit tests!” Well, first, not im­me­di­ate­ly, but when and if some­one runs them. And then, fail­ing the test to go back to the code to search for a mis­take could eas­i­ly waste min­utes of time.

Let's say a test did fail and the pro­gram­mer wast­ed two min­utes notic­ing the prob­lem, search­ing for the cul­prit, fix­ing it and re­run­ning the failed test. Let's also imag­ine that at run­time, half of the time, the col­lec­tion is emp­ty. In av­er­age, the re­liance on Count > 0 would save us, then, 0.09 nanosec­onds—one can round it to 0.1 ns. Let's also pre­tend that the com­pa­ny is pay­ing 100$/hour to the pro­gram­mer, and the same amount for the serv­er farm com­posed of ten servers. If each serv­er is call­ing this method in a loop a huge one mil­lion times per hour, it would take about 120,000 hours, that is, about 13.7 years, to “com­pen­sate” for the orig­i­nal loss of two min­utes. That's mon­ey well spent!

Nano-op­ti­miza­tion has an ad­di­tion­al neg­a­tive im­pact: it makes it look like the job is be­ing done, and so steals time from the ac­tu­al op­ti­miza­tion ef­forts.

As an il­lus­tra­tion, the soft­ware prod­uct de­vel­oped by the afore­men­tioned team is just ter­ri­ble. Per­fect­ly un­us­able. On a 32 GB lap­top, it fre­quent­ly fills all the mem­o­ry. Pret­ty much any ac­tion is slow as hell. Just start­ing it re­quires to wait for sev­er­al min­utes, the time it eats a few gi­ga­bytes of mem­o­ry, just for the sake of show­ing an emp­ty screen.

And nano-op­ti­miza­tion is one of the rea­sons for such a mess.

From the per­spec­tive of man­age­ment, it looks like op­ti­miza­tion is be­ing done. Af­ter all, one guy had to rewrite his code to stop us­ing LINQ, an­oth­er one was chas­ing collection.Any()—that's ef­fort, right? It is. And it gives an im­pres­sion that the team cares about per­for­mance. But by not fo­cus­ing on what mat­ters, they end up with an un­us­able prod­uct. In­stead, they should have been pro­fil­ing, gath­er­ing data about the time their users are wait­ing for the soft­ware, and lis­ten­ing to the ac­tu­al users.