The Future Of Programming

There is a presentation by Bob Martin about the history (and future) of programming I quite like. The main take-away is that present day programming lacks quality control, and that is not an acceptable state of affairs. Unless programmers from around the world manage to come together and implement some sort of binding rules and practices, governments will step in and do it for us.

The processes and systems and languages programmers will be required/allowed to develop professional code will be decided by elected officials, heavily lobbied by MSFT and AAPL, rather than you and me. So we’d better get our collective asses into gear and tackle this.

But you know just as well as I that that’s not going to happen. Programmers don’t take well to being told what to do, like the petulant know-it-all children that we mostly still are. Martin makes the observation that the number of programmers in the world roughly doubles every five years, meaning that at any one moment in time, half of all programmers have less than five years experience. Unless you’re programming 6 hours a day, every day, five years is not enough to get to the magical ten-thousand-hours point at which you start to become good at something. Educating all programmers to be more conscientious is thus not an option. There simply aren’t enough good programmers to teach the new incoming programmers.

Until this growth slows down, the only sensible solution thus lies in better programming tools. Tools that make it easy to do the right thing, and hard to do the wrong thing. And then once those tools exist (they already do for the most part) you have to make people want to use them.

Which leads me to my own musings about the language I primarily work in, and why I think it’s a terrible language, despite it being the best language¹ I’ve used in my 20-odd years of programming. The main problem with C# is that it doesn’t help you write clean, maintainable code. It’s possible to write clean, maintainable code, but you’re on your own. It’s possible to write reliable code, but you have to jump through hoops. It’s possible to write scalable code, but it takes elbow grease. Which is why people generally don’t write clean, maintainable, reliable, scalable code.

New features are being added to C# with every release, and some of them are good, but none of them fix existing issues. They can’t. Not without breaking backwards compatibility, which Microsoft is understandably keen to avoid. Nevertheless, a boy can dream, so here’s a list of things I wish C# did instead of (not in addition to!) what it’s doing now.

1. Immutable by Default

Instead of the readonly keyword which marks fields and types as immutable, everything should be immutable by default. If you really want to be able to modify a value after construction, you’ll have to use the mutable keyword. Ideally mutable/immutable states are not purely local, but act more like the const keyword in C++. That is, the following are all different types, and function signatures would have to pick the appropriate one:

var a = new List<MyType>();
var b = new List<mutable MyType>();
var c = new mutable List<MyType>();
var d = new mutable List<mutable MyType>

Values can not be added to, removed from or assigned to the a and b lists. So what if we want to populate a list over time? Well, you do it like this:

mutable var a = new List<MyType>();
foreach (var b in someEnumerationOfB)
  a = a.Add(new MyType(b));

And before you protest saying that this is ridiculously inefficient, consider that patterns like this can be detected by the compiler and converted into functionally equivalent but more efficient runtime code. Immutability is only there to protect programmers from themselves, the compiler is free to mutate all the memory it wants. The only difference is that instead of every single programmer everywhere having to think about race conditions and shared resources all the time, the thinking is instead delegated to those few very smart cookies writing the C#/.NET framework code. They have maths degrees and everything. I trust them more than I trust me. Let alone you.

There are very few downsides to immutability, especially in a garbage collected environment. The benefits meanwhile are almost impossible to overstate. It makes it drastically easier to build a mental model of code since side-effects are all but eliminated. It makes it trivial to parallelise code as race conditions are no longer an issue. It removes the problem of cache invalidation, as caches will never expire.

2. Integrated Unit-Tests

Unit tests are one of the most important tools programmers have to make their code reliable. If code is fully covered, unintended consequences of code changes are almost certainly going to be caught before they end up in production. There are several unit testing frameworks the conscientious C# developer may pick from, which of course includes the “standard” framework provided by Microsoft and the three alternative “good” frameworks provided by people who know better but aren’t official. This is already a serious problem; which one do you pick? But even worse is that these testing frameworks require you to create a separate project just for the purposes of testing.

No.

Why?!

Testing needs to be an integral part of a programming language. I want to be able to add my tests right next to the methods they’re supposed to validate. So let’s add the test and fail keywords to the language spec. Any method which returns a test type is not allowed to have access modifiers, will not be included in the compiled code, and cannot be called from any regular, non-test method. Any method which returns a test type and which has no arguments, will automatically be executed as part of the test suite, while test methods with arguments can be invoked only by other test methods. A test method either runs to completion, in which case it passes, or it hits one of its fail statements.

For example, here’s how I would like to test Json (de)serialisation in my code:

public sealed class MyType
{
  // The actual implementation of this type goes here.

  test TestJsonDuplication()
  {
    var instanceA = new MyType("Hello", "World", 2025);
    var json = instanceA.ToJson();
    if (string.IsNullOrWhitespace(json))
      fail "Json serialisation yielded an empty string.";

    var instanceB = MyType.FromJson(json);
    if (instanceB is null)
      fail "Json deserialisation failed to construct an instance.";

    if (instanceA != instanceB)
      fail "Json deserialisation was incorrect.";
  }
}

This is particularly important because it allows me to write tests for private methods, which are otherwise only testable by invoking other methods first, which add to the complexity of a test. Not having to mess around with different files, let alone different projects would also be a huge benefit. Of course if you want to create a separate test project, this mechanism still allows you to do so.

3. Integrated Cancellation

The CancellationTokenSource/CancellationToken mechanism introduced in 2010 is a reasonably solid approach to cancellation. I’d certainly be proud of it if I wrote it. But it’s not integrated into the language, and the main drawback of it is that tokens need to passed around to all functions which support cancellation, which not only pollutes their signatures but also adds overhead due to struct copies.

I’d like to turn any function into a cancellable function just by adding abort; statements to it.

public static double Average(IEnumerable<double> values)
{
  var count = 0;
  var mean = 0.0;
  foreach (var value in values)
  {
    abort; // Instead of token.ThrowIfCancellationRequested();
    mean += (value - mean) / ++count;
  }
  if (count == 0)
    return double.NaN;
  return mean;
}

We can signal cancellation requests using a cancellation context, which is something the runtime needs to keep track of, either on a per thread or call stack basis, or perhaps something more complex since it will also have to play nice with the TPL. It might look something like this:

using CancellationContext(token)
{
  var numbers = GetABunchOfNumbersFromSomewhere();
  var average = Average(numbers);
  Console.WriteLine($"The average value of your {numbers.Length} numbers equals {average:0.0###}.");
}

I think it makes sense to repurpose the already existing using keyword like this. Now any function can define cancellation for any of the many functions which end up getting called within some scope. If the compiler determines that none of the functions which could possibly get called support cancellation, then the context can be omitted entirely during compilation. The compiler is also free to create two versions of each cancellable method—one with and one without cancellation checking—and call the appropriate one, reducing checking overhead.

4. No Exceptions

I’d be thrilled if exceptions weren’t a thing. Obviously sometimes a program exceeds some limited resource at which point it must be terminated lest it continues running in an undefined state, but those are typically exceptions (OutOfMemory, StackOverflow) you can’t gracefully handle anyway.

I don’t understand why a function which could fail has to throw an exception, apart from maybe the ability to communicate a failure way, way up the call stack to a function which might care and has error handling logic, thus separating the happy paths from the bad paths. But it’s a pretty weak argument in my opinion.

In my mind there are two distinct types of errors that can occur within a program; internal and external. Internal exceptions are caused when the code is buggy; indexing an array at -1, referencing a null or disposed instance, dividing by zero. By the time you’re compiling production code none of these issues should still exist, and really the compiler should always prevent you from ever writing buggy code like that. External exceptions on the other hand are imposed upon the code by the environment it runs in; insufficient privileges to access a file, USB-stick yanked out halfway during a file save, internet connection interrupted mid license validation. Those can be foreseen, but not avoided.

For the latter, having functions return failure/success information seems like a far more explicit way to signal that something can go wrong and that the caller must take care. Consider for example the string File.ReadAllText(string path) method which, given a file uri, returns the contents of that file as a string. Clearly, for reasons that are outside the responsibility of the programmer, this may fail in a wide variety of ways. As a result, this method throws one of nine different exception types, although that set is not guaranteed to stay constant over time. If you were inclined to handle them specifically, as is the recommendation, you’d have to consult the documentation each time you’re writing a new series of catch statements.

Instead I’d much rather this method had one of the following signatures:

• bool File.ReadAllText(string path, out string contents)
• FileAccessFailure File.ReadAllText(string path, out string contents)
• string? File.ReadAllText(string path, out FileAccessFailure failure)

The FileAccessFailure could be just an enum, or it could be a complicated type containing a lot of information. Maybe it derives from a Failure base class, just like all exceptions derive from an Exception base class. Or maybe instead of throwing exceptions we just return them, why throw out the baby with the bathwater?

Unhandled exceptions are one of the foremost reasons for applications ending up in a corrupt state, so anything which prevents them from going unhandled can only be a good thing. There is however a problem with my suggestion, which is that exceptions are pretty instrumental in cancellation. All code could potentially run in a cancellation context, but it would be ridiculous to expect all code to have failure return types. It is not clear to me how cancellation can be safely implemented without exceptions, but maybe it’s enough to hide exceptions from the programmer and allow the runtime to still throw and catch them itself.

5. General Contexts

Continuing on from point 3, there is no reason why cancellation should be the only kind of context. A mechanism for creating custom contexts would be most welcome, as it would clean up a lot of function signatures by moving arguments into contexts. String comparison and Culture locales are also very common states that pollute the signatures of dozens if not hundreds of methods, and within a product like Rhinoceros, angular and absolute tolerances must commonly be passed down the call stack to low level functions which require them.

What might C# code which sets and gets contextuals look like? Let’s begin by defining our own contextual type by deriving from the Context<T> base type.

public sealed class AbsoluteToleranceContext : Context<double>
{
  // The constructor must specify a default value which acts as a fallback.
  // Ideally this would be a constant value, but C# OOP doesn't know how to 
  // demand const or static overrides, or how to put non-primitive types into
  // Attributes, so a better solution I suspect requires more language changes.
  public AbsoluteToleranceContext() : base(1e-3) { }
  
  // Optional validation. Not sure this is required, although it would be
  // nice to be able to rely on contextual values always being valid.
  protected override bool ValidateValue(double value)
  {
    return value >= 1e-32;
  }
}

Now to begin a new contextual block, temporarily overwriting whatever the absolute tolerance context was:

// A 'context' keyword may be required here. I'm bad at coming up with all
// possible ramifications, inconsistencies and ambiguities of language features.
using new AbsoluteToleranceContext(1e-6)
{
  // All code executing in this block will use an absolute tolerance
  // of one-millionth, unless it begins a new context scope itself.
}
// From here onwards the absolute tolerance reverts to what it was before the block.

And two suggestions for getting the currently relevant contextual value:

// Accessing a static method on the Context base class. Very wordy.
var tolerance = Context.Current<AbsoluteToleranceContext>();

// Or maybe using a pattern similar to typeof()
var tolerance = context(AbsoluteToleranceContext);

6. Type Simplification

I really don’t feel qualified to always correctly pick between struct and class, and now we even have record to consider. Why can’t there just be a single kind of object called type which can act as either one? The compiler may create both value and reference types out of a single type definition and use them interchangeably, depending on what makes the most sense in any given context.

How does the compiler know what makes the most sense? Well, as mentioned previously we’ll be providing it with examples of how the code is expected to be used via integrated unit tests. Various implementations of a type can be automatically created and profiled by running all the tests. This is not just good for the compiler, but it also rewards us for writing proper tests which mimic real world usage.

public type Vector3
{
  public double X;
  public double Y;
  public double Z;

  public num Length
  {
    get { return Sqrt(X * X + Y * Y + Z * Z); }
  }

  public Vector3 Unitise()
  {
    return new Vector(X / Length, Y / Length, Z / Length);
  }
}

The argument that reference types can be null while value types cannot, and that we therefore need to be able to choose which of those two flavours we want is nonsense. If an argument can be null, or if a function can return null, it should be clearly marked as such using a nullable operator, and always checked prior to being referenced anyway.

Another argument that value types are copied while reference types are not, and that as such they behave differently is equally nonsense. Not only is that a confusing property that we’d be better off without, when types are immutable the difference stops mattering, at least from a behavioural point of view. From a performance point of view the difference may be stark, but again the compiler is in a better position to optimise that than we are.

When types are simple enough, such as the Vector3 example above, a lot of boilerplate code could be done away with by the compiler automatically adding constructors and setter methods. More recent versions of C# have some allowance for this via records, but I find the notation still quite cumbersome, involving as it does having to type curly brackets as well as the name of the property. If for every public value or property X a WithX() method were emitted which returns a modified copy of the type, it allows for a much more succinct notation. Furthermore, I imagine chained WithX() invokes can easily be detected and thus optimised.

var vectorA = new Vector3(4.0, 5.0, 0.0);     // Automatic constructor.
var vectorB = vectorA.Unitise();
var vectorC = vectorB.WithX(0.0).WithZ(-1.0); // Chained modifiers can be merged.

8. Stupid Ideas

Finally, here’s some probably not-so-good suggestions.

We don’t need > and >= operators. Typing the wrong comparison operator is not the most common bug I have to deal with, but it’s up there. If we only had access to smaller-than and smaller-than-or-equal operators, we could train ourselves to always think about comparisons in the same way. Not to mention overriding these operators on custom types would be less work.

Can we please just use a single integer type? The compiler can create variants of methods and types with fast 32/64 bit integers and slow but unlimited BigIntegers. We clearly need the performance of fixed integer types, but also the correctness of variable-width integer types.

Can loops whose iterations do not rely on previous iterations be automatically multi-threaded? Parallel.ForEach is fine, but the notation is awkward compared to normal loops.

Do we really need break; statements inside case blocks? Seems redundant.

The [Flags] attribute for enums is stupid. It should have been a completely separate base type. Enum values can be exactly one of the predefined constants, Flag values can be any combination of the constants. I should not be left in charge of picking exact powers-of-two for my flag constants lest my bit-masks go awry.

The is not pattern introduced in C# 9 is weird. Clearly it should have been aint.

I’d be interested in constraints on function arguments. It might help turn functions which could fail into functions which never fail, although it is hard to see how it would stretch to non-primitive types. This would allow for overloads where the constraints do not overlap, and it could push argument validation to the caller, thus reducing the possibility of failure:

public static bool IsPrime(int n where n <= 1)
{
  return false;
}
public static bool IsPrime(int n where n > 1)
{
  // ...
}

And lastly I will leave you with some amazing actual work already done to really up the game for programming tools. I will never understand why Microsoft or Apple aren’t pouring their billions into research like this. Why are they not trying to create the best development environment imaginable?

1. Please don’t write to me telling me how Rust, Clojure, F#, Go, or Kotlin are better than C#. That’s not the point of this post. ↩︎