> Did you ever attempt to get to the bottom of an intermittent error that was driving you insane, only to discover much later that a private variable wasn’t synchronized properly? Have you ever spent hours stepping through the lifecycle of an object, going through many different components spread out across the codebase and wished that you had spent all that time solving new problems instead?
I see 3 issues here.
1) Most people haven't dealt with those issues enough. There are complex programs but there are also many simple program as well. And unless one deals with hard to debug concurrency / pointer / mutable state errors they will not appreciate immutability. They might pay lip service to it, because it is cool, but they will not appreciate it enough to start rewriting their codebase in it. That is a bit like fault tolerance, unless they have been woken up at 3am because their main larger server process segfaulted and had to debug, they will not think about isolated units of failure, about supervision, durable peristent snapshots/checkpoints, monitoring etc. They of course will say that "fault tollerance is good" but it is good in an abstract, general way.
2) Those that dealt with these issue might accept the state (pun indended) of affairs and never realize there is another way of doing things. I mean they just accept that you have to have locks and mutable state and mutexes and dangling pointers and then spend months debugging hard to track concurrency errors at 3am. That is just how software development works and it is as good as it gets.
3) People realize there is something better, they know about, read about, played with it. But they don't have enough energy, political power, or time do fix it.
I see another issue. Have I ever gotten to the bottom of an intermittent error only to determine it is a synchronization error? Yes.
The problem is I have also gotten down to the bottom of a problem and found it was a stale data problem. In a system that was performing gymnastics to keep things immutable. To the point that fixing it was not a simple matter of just updating the variable.
Worst were the Multiversion concurrency control (MCC) systems I have seen rolled to "help keep all facts of the system immutable." Suddenly, we had to have database implementation experts just to run basic software...
Trying to keep data immutable introduces scenarios involving stale data, and a new problem to solve. I have seen interesting problems with cache coherence that were quite subtle in their manifestations.
Understanding the system-wide implications of immutable-only data is a non-trivial design task.
the issue I have with this statement is that stale data exists when going full-imperative as well.
The solutions might be harder to implement in an immutable environment if your setup isn't right, but not an order of magnitude harder. And you get all the advantages of immutability.
People complain about stale data issues in immutable programming styles not because they're more prevalent, but because they can rule out so many other classes of bugs immediately.
> People complain about stale data issues in immutable programming styles not because they're more prevalent, but because they can rule out so many other classes of bugs immediately.
This generalisation is too broad (as with almost any generalisation).
In any case, I want to mention one interesting experience -- a very large financial portfolio management system that I worked on, over a decade-and-a-half ago.
The system had multiple in-coming tickers, feeding huge amounts of data. The central data structures of the computational pipeline system were immutable. Initially, there was a very haphazard system for determining when a particular piece of data was to be deemed stale. In time, formal definitions were established. However, the software was not amenable to those definitions, since shared data was not `seen' updating instantaneously.
The core was too big to be changed in any non-trivial manner, without risking the entire business.
We ended up introducing an explicit notion of time into every pipeline processor to ameliorate (not entirely solve) the situation.
Every major design decision has its own trade-offs that are specific to the problem on hand. But, those built into the foundations of the system should be made pragmatically, not by adhering to a philosophy!
I agree that things need to be pragmatic. But there are classes of bugs ruled out absolutely by things like immutable structures and referential transparency.
Now, the thing is that there's always a way to model state, but at least I can rule out things like the term "x" to mean different things depending on how many times f was called beforehand for the most part.
The problem I have with this line, is it ignores the dealing with mutable state you already do on a daily basis.
Are you confused that the listing on hackernews changes depending on time of day? Why would you be confused that a call to "incrementCounter" changes the value of the counter? Or that a call to "printf" adds to the output?
I agree that having a large function where the value of something changes at the top and at the bottom can be confusing. But so can a function where you have two variables of type String because you had to sanitize the input and assign it to a new variable.
Are there ways to avoid that particular error. Certainly. Yet I have encountered that about as many times lately as I have encountered "mutable" based errors.
For 1) I meant that there are lot of utilities, micrservices, small web backends, scripts, and hopefully, applications that have already been split into isolated components that can fail and restart separately (in other words someone did the work already). So if one starts dealing with those in their careers they just might never have a compeling reason to look at Haskell, F#, Erlang, OCaml and such.
Also note that quite often one way to deal with large mutable state is to simply shove it in database. Usually good databases just make it explicit what happens to shared state.
The most compelling reason is being able to write way less code. When I write in C# I'm constantly annoyed at how pointlessly verbose it is. I feel hampered, like I'm trying to talk to a 5 year old that doesn't really understand English.
I've done comparisons where I've written the exact same program in C# and F#. The reduction in unneeded type annotations alone is massive. F# required only 1/20th the number of type annotations.
Type inference can make the code easier to write. But, is the code easier to read, and overall less effort for the business? Code tends to be read significantly more than it's written. This is a simplification but seems to be true enough; modification by subsequent maintainers involves a lot of reading.
I worry about features like overuse of type inference when it means the types in question aren't explicit from the code that you're reading (agreed, not all type inference involves overuse). For example, if I write:
var x = f();
What's the type of x? What's the return value of f? I can answer those questions, certainly, but I have to go look for the answer. I don't immediately have any mental concept or name to hang the idea off of. Whereas in a language like Java, I'd end up writing:
Foo x = f();
Now I know the return value is a "Foo". This may not seem like much, but suddenly if there are many different Foos in the code, I can see the patterns I didn't see before, and I can draw correlations:
Foo x = f();
Foo y = g();
h(x,y);
Everything can always be understood with a degree of thought, but type annotations seem to, in my experience, make programs a lot easier to read and modify. The original authors typically have the whole thing in their head, so it doesn't make a difference to them, but it can be a huge difference for those who come after.
There's a certain simplicity that comes from types that are so concrete, clearly named, and linkable as Java libraries are. You can figure out something about this code without reading the declaration or definition:
You have a pointer for what to search for and an immediate name for the concept. I suppose it's like an index, in a way. An index to make code more readable and referenceable is included throughout. It's hard to explain it, but I find this valuable even when I have support from a sophisticated IDE. (Plus I'm often working outside an IDE.)
Including these types is more time consuming up front than languages that support type inference, or are dynamic, but for long-lived applications it seems like the overall tradeoff is usually worth it. To be fair, I haven't maintained any F# applications. If the type inference is purely saving annotations that are completely obvious even to a beginner to the codebase, then I could see that being worth it. (It seems to me that we are all beginners to the codebase in meaningfully large platforms.) The actual act of typing the annotations is also often pretty easy and handled automatically by the IDE, but I suppose you have that experience with C#.
True, but I think part of this is about shaping behavior, and about self-discipline. And perhaps also the cognitive bias that comes from knowing one's own program well. A program I wrote always seems more obvious to me than it does to others. It seems to me that the mind, while aware of gaps in knowledge between ourselves and others, cannot fully compensate for this. It's like Hofstadter's law ("It always takes longer than you expect, even when you take into account Hofstadter's Law.")
So I don't know if I fully trust myself to decide which type annotations are obvious up front. Though maybe I'm just used to this way of working. I'm not aware of many solid studies of the differences that move far beyond personal preference. Especially studies that measure the performance of teams over time (vs virtuoso performances). Also, how happy would you be as the maintainer of a codebase if a new guy came along and submitted a pull request which was purely the addition of an already-inferred type?
Type declarations tend to be pretty low effort for me, these days. The way that I write code in Eclipse works like this:
// I type out this part
f(x)
Then I invoke auto-complete and ask Eclipse to automatically assign the expression to a new local variable or field. If f() doesn't exist, I might ask it to create that method. It will infer part of the signature from the type of x. Once I write the method, I can auto-complete its return type. Or if the method exists, taking the type of the variable "x", it can suggest that I probably want "x" as the parameter. I just type "f(" and invoke auto-complete. Anyway, once I ask it to assign to a variable, I end up with something like:
Foo foo = f(x);
I'm given a choice of common variable names based on the type in question which are easy to choose between with good defaults. So most of the time I'm sort of minimally describing or hinting at what I want, and the IDE guesses extremely well what I mean -- better than a compiler ever could, because it's allowed to make multiple guesses and be wrong -- and then I capture or snapshot that by saving it as text.
The type inference is actually just as present as in the other languages, it's just happening up front at editing time, rather than at compile time. And these benefits don't only apply during original composition. If I edit the method f() and change its return type, I can just as easily auto-complete (or more broadly, use automated refactoring for) the change to the type declaration for the "foo" variable.
Maybe if you're a programmer of dynamic languages or heavily type-inferred languages, this will seem like "much ado about nothing". "Isn't it the same in the end?" The difference to me is that it's all there in the text, and on the screen, by default; and it's all comprehensible even without an IDE and without an understanding of all of the types involved. I still drop out of the IDE surprisingly often to read code at the command line or in a web browser.
I can also see where the alternative view is coming from though that, if I don't have to define the type of variable "foo", then I don't have to change the text when "f()" changes. I would support a "var" keyword in Java, and agree the problem is really about when type inference goes too far. But I see the effort of changing "foo" as much less of a problem than easily and frictionlessly understanding what the code means, immediately when reading it, with minimal reliance on context.
I would be interested to take a look at a large F# program and see how easily I can figure it out.
I certainly agree that code is usually more readable close to when it's written, in both personage and time. What I'm skeptical of is that redundant type annotations always (or usually) make code more readable. I do believe they often do, but I think they should ideally be reserved for those cases. I won't always get it right, but that's something that can be improved with experience and especially in code review.
"Also, how happy would you be as the maintainer of a codebase if a new guy came along and submitted a pull request which was purely the addition of an already-inferred type?"
Slightly happier than someone submitting a pull request which was purely the addition of a comment. In both cases, my response is to try and understand why they felt the code less readable without it and see if that motivates other changes, but then probably to happily merge it.
Indeed, it's frustrating when you're reading flat files of code. When reading foreign F#, I pull the code into Visual Studio so I can hover over the variables. It'd be quite convenient for GitHub to provide the type information on hover...
This is the big one for me. I deal with assorted systems that I can't get migrated off ancient shitty middleware platforms because any comprehension of them vanishes into the ether somewhere around the upper management types.
"Functional programming" isn't even anywhere on the radar when I currently have no chance of even completely getting rid of PHP.
I see 3 issues here.
1) Most people haven't dealt with those issues enough. There are complex programs but there are also many simple program as well. And unless one deals with hard to debug concurrency / pointer / mutable state errors they will not appreciate immutability. They might pay lip service to it, because it is cool, but they will not appreciate it enough to start rewriting their codebase in it. That is a bit like fault tolerance, unless they have been woken up at 3am because their main larger server process segfaulted and had to debug, they will not think about isolated units of failure, about supervision, durable peristent snapshots/checkpoints, monitoring etc. They of course will say that "fault tollerance is good" but it is good in an abstract, general way.
2) Those that dealt with these issue might accept the state (pun indended) of affairs and never realize there is another way of doing things. I mean they just accept that you have to have locks and mutable state and mutexes and dangling pointers and then spend months debugging hard to track concurrency errors at 3am. That is just how software development works and it is as good as it gets.
3) People realize there is something better, they know about, read about, played with it. But they don't have enough energy, political power, or time do fix it.