A bit of Internet dispute about cleverness. When is code too clever to live? I have an example in mind.

There was a bit of a dispute over the weekend about a slide from some random talk, saying:

It’s programming if clever is a compliment,
It’s software engineering if clever is an accusation.

Some of my dear colleagues felt that the slide was divisive, some felt it was rather elitist. Many of us thought that the idea could be better put. I, myself, I regret to say, fell among those who thought the idea could be put more effectively, but who did not see the harm in it.

I can only add that I have claimed for years to be a programmer, and have never claimed to be a “software engineer”, to the best of my recollection. I haven’t devoted great thought to it, but I am not sure that I believe in “software engineering” as an engineering discipline, so I guess I’d fall into the class of those who might have been offended. But I am famous for being oblivious, so possibly it went over my head.

But “cleverness”. Those in the conversation with me all agreed that cleverness in code is not a compliment, and that cleverness in code is generally not a good thing. What that mostly tells you is that those in the conversation were probably of greater than average age for programmers, and had been around the block more than a few times.

I think it was Chet Hendrickson who said “Cleverness is what gets you called in to work at 3 AM”. Whoever said it had a point.

Drilling Down

I wonder, if we were to drill down on what is “good” and “bad” about clever code, what dimensions we might dig out.

To begin with, what is it about clever code that gets you called at 3 AM?

One obvious case is clever code that doesn’t work. Maybe somehow it combines 13 of 16 possible cases to make some distinction that the program needs, via some combination of ands and ors and xors, all in one if statement. And maybe there’s one rare combination that breaks the combination and gets the wrong side if the branch–at 3 AM.

Another case might be clever code that is hard to maintain. Maybe the 13 of 16 code is actually correct, but the rules change. Now some programmer is faced with a dilemma that is difficult either way. Either they work out all the combinations and manage a new sequence of boolean operations that “should work”. or they unravel the original booleans into a sequence of if statements that work, and then manage to edit that sequence so that it “should work”.

Or–trilemma?–maybe they realize that they can build a sixteen-way branch lookup and dispatch directly, which is less clever, requires a 16-address table, and is easy to get right, easy to test, and won’t break at 3 AM.

So maybe code that is too clever is code that obscures what it does and how it does it. Maybe code that is too clever is hard to change correctly.

Certainly I would not argue in favor of obscure code or code that is hard to change, and I hope you wouldn’t either. But, of course, there is no bright line between clear and obscure, no bright line between easy to change and hard to change.

It requires judgment … and more.

Intelligence in the Light of Experience

Nero Wolfe had some saying like that when advising his mobile sidekick, Archie Goodwin: “You are to be guided by intelligence in the light of experience”. I like to say that we use judgment in the light of experience, and I find that I’m not the only person who ever wrote those words. Either way, our experiences differ–as, I suppose, do our intelligence and judgment–so that the code we write may be clear to some and obscure to others.

Object-oriented code that is good according to my training and judgment, with short methods, many method calls, and methods unwinding down and down until some method returns 3 and it all unwinds … that kind of code seems very obscure to other programmers who have only experienced what I would call “procedural” code.

And I’m not even all that good at it. I know people whose OO code is more that way than mine. It’s better for it, and I have no trouble understanding it. I’m just not as good at creating it.

But an OO programmer with more “procedural” experience likes to see the code telling its story in a longer, less modular fashion. They often feel that they have to jump around between objects and methods to understand how anything works. They’re not comfortable thinking “OK, it asks the Tile for its legal neighbors”, and moving on with what is done with those neighbors. First they need to drill down and see how the Tile does it, and to follow that thread down and around, winding among the objects until finally the answer is there and then they wind back around and up until they’re back where they started. They grab a breath and move on to find out what is done with the neighbors.

I can’t even do that. I can’t keep multiple levels in my head at the same time. I can’t remember who calls whom after a few steps. So while the person who favors the procedural style may find my OO code hard to understand, I also find theirs hard to understand.

I’d probably not describe the procedural style as “clever”, but I could imagine a more procedural individual describing my tiny methods calling methods “clever”.

So, I guess, clever is dependent on what we’re used to, and clever code apparently deviates from what we’re used to, in a way that we understand but are perhaps not adept with.

I have an example in mind.

Lua Objects

Most modern languages support classes and objects directly. We all understand how they work, but the details of the implementation aren’t something we need to think about much.

Lua is different. Lua has numbers, strings, functions, and tables, and not much else. It doesn’t have classes and objects natively. They are implemented in terms of tables. There are a few ways of doing it, and her’s how it is done in Codea Lua. We’ll explore some of the details below:

function class(base)
    local c = {}    -- a new class instance
    if type(base) == 'table' then
        -- our new class is a shallow copy of the base class!
        for i,v in pairs(base) do
            c[i] = v
        end
        c._base = base
    end
    
    -- the class will be the metatable for all its objects,
    -- and they will look up their methods in it.
    c.__index = c
    
    -- expose a constructor which can be called by <classname>(<args>)
    local mt = {}
    mt.__call = function(class_tbl, ...)
        local obj = {}
        setmetatable(obj,c)
        if class_tbl.init then
            class_tbl.init(obj,...)
        else
            -- make sure that any stuff from the base class is initialized!
            if base and base.init then
                base.init(obj, ...)
            end
        end
        
        return obj
    end
    
    c.is_a = function(self, klass)
        local m = getmetatable(self)
        while m do
            if m == klass then return true end
            m = m._base
        end
        return false
    end
    
    setmetatable(c, mt)
    return c
end

Let me see if I can explain this. Frankly, I think it is “clever”.

First, a word about metatables. Any table in Lua can have a “metatable” This is a second table associated with the original table in some magic fashion called setmetatable. When you try to do something with the original table such as accessing a missing key, instead of doing the standard thing (return nil in this case), if there is a metatable present, it can change things.

As a trivial example, you could give a table a metatable so that instead of returning nil for missing keys, the table would return 42. The metatable isn’t visible inside your table, other than with the special methods to get and set it.

With that in mind, let’s see what happens when we write

MyClass = class()

We can see that the class function creates a local table c, at the beginning, and returns it at the end. So a Codea class is a table.

We can ignore the stuff about base for now. It’s about providing a superclass for the class.

The code says “the class will be the metatable for all its objects, and they will look up their methods in it.” Then it says

c.__index = c

This just means, if I’m not mistaken, that when you look for something in a table whose metatable is c, and do not find it, look for it in c. We’ll come back to that in just a moment.

Recall that we define methods in Lua like this:

function MyClass:myMethod(...)
-- code for myMethod
end

So all the instance methods for MyClass are functions in this table c. They are defined in the class table c.

Now we have the code for creating an instance. That’s this code:

    -- expose a constructor which can be called by <classname>(<args>)
    local mt = {}
    mt.__call = function(class_tbl, ...)
        local obj = {}
        setmetatable(obj,c)
        if class_tbl.init then
            class_tbl.init(obj,...)
        else
            -- make sure that any stuff from the base class is initialized!
            if base and base.init then
                base.init(obj, ...)
            end
        end
        
        return obj
    end

We’re still in the function class, mind you. We create a table mt, and we give it an element __call, which is a function. This is another of Codea’s bult-in metamethods and if a table foo has this metamethod pointing to a function, then you can call the table.

If we say

foo = {}
foo()

Lua will say “attempt to call a table value” and barf. You can’t generally call a table: it’s data, not code. But if the table has a metatable defining a _call function, then when you try to call the table, that function is executed. So it looks like you called the table.

Recall that we are still executing inside the original statement:

MyClass = class()

We create another local table mt, and we give it a function __call`. What does that function do if it is ever called?

It creates a new object obj, and sets that table’s metatable to c, the class table, the one that will be returned when class finishes and that will therefore be stored into MyClass. So now, the table obj has our class as its metatable and because that table has __index-c, any functions called on obj, like obj:foo(), will be looked up in c, that is, looked up in the class.

Then there is code in the creation function to call init if it is provided in MyClass, and to deal with the superclass if there is one. (My understanding of this code is that if a class has an init, the inits of any superclasses are not called. I could be wrong. We could test that.)

Then the function saved in __call returns obj, the initialized instance.

Now that function is defined, and the original class function moves on to define an is_a function that treks up the __base links to see whether the metatable of the proposed instance matches any of them, thus answering the question.

Trekking up the list allows all of these questions to return true:

Princess:is_a(Player)
Princess:is_a(Entity)
Princess:is_a(DungeonObject)

Finally, having defined the is_a function, we set the metatable of c to mt, and return the table c as our class.

If I can figure out a decent picture or two for this, I’ll include them in this article. I don’t mind telling you that while I “understand” this well enough to have written what I just did, I don’t understand it well enough to produce it myself, and I would not even claim to understand it well enough to explain it to a class without notes or the code in front of us.

diagram of MyClass, c, and mt and their relationship

another equally poor diagram

As you can see, I’ve not found a picture that seems ideal. I’ll keep trying for a while afte publishing this and will improve it if I find one.

Is this code clever? I would argue that it is. I would also argue that most everyday Lua programmers probably couldn’t invent it, assuming that most everyday Lua programmers write classes and methods and rarely if ever use metatables.

But without this clever code, Lua wouldn’t have classes and objects.

Is This Good Cleverness, or Bad?

How about that? It depends who you are. If you’re a Lua implementor, it’s probably good cleverness. You’re probably adept enough with metatables, having been there when they were invented, to understand this thing, and to write it or things like it.

And if you’re a regular everyday Lua programmer implementing Dungeons and space games, this code may be clever enough to put you in danger if you write things like it. I think this makes it a team decision whether some code is too clever to exist or not and I think this means that the answer could change over time.

At some point in a Lua team’s work, they may well come across something that is best dealt with using Lua’s metamethods. I’ve done it in some of my articles. There are metamethods in the Dungeon program.

I use the __tostring metamethod a lot, to allow a clearer display of an object when things break:

function Monster:__tostring()
    return string.format("Monster (%d,%d)", self:getTile():pos().x,self:getTile():pos().y)
end

But I’ve done more than that:

function InventoryItem:__eq(item)
    return self.name == item.name
end

This arranges things so that given two inventory items thing1 and thing2, you can say

if thing1==thing2 then ...

So that’s handy and not overly clever (in my view). It does have an odd property, which is that any two inventory items of the same name are equal, but that’s what we wanted.

Here’s another use:

function MapPoint:__add(aMapDirection)
    if aMapDirection == nil then error("nil aMapPoint") end
    if aMapDirection:is_a(MapDirection) then
        -- subverts the true check.
        return MapPoint(self:x()+aMapDirection.x, self:y()+aMapDirection.y, self:z()+aMapDirection.z, self._isHex, true)
    else
        error("Attempt to add a MapPoint plus a non-MapDirection")
    end
end

This code arranges things so that given a MapPoint, you can add a MapDirection to it, to get another MapPoint. That’s how points and vectors work. You can add a vector to a point and get a new point. So this allows for simpler code, and clearer code … at least after you get over the first “huh” when you see something like this:

function Maps:surroundingPoints(aPoint)
    local dirs = Maps:allNeighborDirections()
    local result = {}
    for _i,dir in ipairs(dirs) do
        table.insert(result, aPoint+dir)
    end
    return result
end

And you wonder how we managed to add a dir to a point. Or, optionally you might say “OK, somehow that works” and move on to whatever you were doing. It’s that other matter of style, where you either drill down or don’t, depending on experience and preference.

One more subtopic, then let’s sum up.

Upcoming Clever Idea?

This topic caught my attention over the weekend, because I have in mind a change that might be too clever. It’s too late this morning to do it, but I’ve written about it a few times recently.

There are objects way down in the system that know about objects high up.

Sometimes they direct connections. We’ve been working to limit and narrow those connections, with our various TileFinder and other little objects that cover and protect or replace larger objects like Dungeon.

There are still objects that know the GameRunner, which is the top level object in the whole system. This is really not great design. It is commonly solved in one of a few ways:

  • Global: Make the GameRunner a global and then objects can reference it without having a pointer to it in the object.
  • Singleton: Use SIngleton pattern to allow GameRunner:instance() to return the instance. This is nothing but a baby global.
  • Pass it as a parameter. This is often not too awful, but isn’t always applicable. Our objects act somewhat autonomously, and so they are not typically called in ways that make it convenient to pass in the runner … and perhaps I have avoided some places where it wouldn’t be too bad. I’d have to review code to have a good opinion.

Generally speaking, I’ve been taught that passing a thing as a parameter is generally better than using a global or singleton. Generally. Since my code has this tendency to use a global Runner or DungeonContents, I have been taught that my code could use some improvement. And it happens that I agree with that assessment, because I’m not just quoting back what I’ve been taught, it’s also something I’ve learned on my own (time and time again) from writing code that does things like this and that gets messy.

So I have code that uses globals or mini-globals, and I’d like to improve it. I’d also not like passing the former global thing around as a parameter, even though the actual changes might not be too difficult. And I have this clever idea.

Suppose there is a low-level object that wants some information from the top level, like “where is the princess”. Today it might refer to a global saying something like:

local playerTile = Runner:getPlayerTile()

But we do have the Bus, and its publish-subscribe method. So if the GameRunner were to subscribe to “whereIsThePlayer”, we might be able to say something like this:

local result = {}
Bus:publish("whereIsThePlayer", self, result)
local playerTile = result[1]

The Bus needs to be global, so the latter code eliminates the Runner global without adding a new one. And the Runner would just add the PlayerTile to the result table. And when the publish returns, that is already done, so we can pull the answer out.

That would work right now. I’d just have to write the correct method in GameRunner to handle a subscription to “whereIsThePlayer”.

At some point, perhaps tomorrow, I am going to implement this “solution”, if I can find a place where a better approach doesn’t present itself.

Will it be too clever? Should I take a poll? Or is it down to the cat and me?

Cleverness

So the bottom line … there isn’t one. The current subtotal line is … when code seems clever, I want to look at it carefully, and decide whether it’s too close to the limit of what I can quickly understand. I’d suggest that I should be quick to avoid and slow to accept, clever code. But I do learn a language as I use it, and the things that seemed clever a month ago often seem normal today.

Judgment in the light of experience. There you go.