I’ve been trying for days to get a decent TCP/IP socket connection working on the iPad, apparently for my sins. So far, I’ve failed miserably. Today, Saturday, another try.

I’ve started this article so many times and abandoned it. Each time, I think I’m going to sort out the socket connection, and each time, I’ve bogged down in confusion, with things that I swear should be working but thar are not, and with no idea what’s wrong or what to do next.

Today, I’m going to build—perhaps the phrase I’m looking for is “try to build”—an object that will handle the World side of a connection to a robot. I’m going to build it in a separate Codea project with the probably vain hope that in the simple environment I can focus on just this one problem.

I’m going to try to TDD it, and that’s going to be difficult because my “design”, that is to say my “rough idea” is that I’ll build a WorldConnection object and then it will wire itself into Codea’s tween timing loop to execute. Because I’ll be running tests against it, I think it will need to be created in the CodeaUnit before and torn down in after, and I know for sure that if I don’t do that just right, bad things will happen.

I’ve got various snippets of code that I think “work” with sockets, and I’ll use those to build the thing up.

Here goes nothing:

WorldConnection (But Wait)

Here’s my first layout of the test:

-- TestWorldConnection
-- RJ 20220709

local tweenSave

function test_WorldConnection()
    
    _:describe("WorldConnection", function()
        
        _:before(function()
            tweenSave = tween.update
        end)
        
        _:after(function()
            tween.update = tweenSave
        end)
        
        _:test("WorldConnection", function()
            local wc = WorldConnection()
            _:expect(2).is(2)
        end)
        
    end)
end

WorldConnection = class()

This runs green. Note that I’m saving and restoring tween.update, which is the function that is called on every frame to run tweens. In due time, I plan to install my connection code on that call, to run it repeatedly. No true threads here in Lua.

Let’s teach WC (unfortunate name shorthand) to get our IP address.

        _:test("WorldConnection", function()
            local wc = WorldConnection()
            _:expect(wc:localIP()).is("1.2.3.4")
        end)

This will fail for want of localIP:

1: WorldConnection -- TestWorldConnection:20: attempt to call a nil value (method 'localIP')

I have that code somewhere …

function WorldConnection:localIP()
    local socket = require("socket")
    local server=socket.udp()
    server:setpeername("1.1.1.1",80)
    local myIP,myPort=server:getsockname()
    return myIP
end

Once I learn the difference between myIp and myIP, I get this error:

1: WorldConnection  -- 
Actual: 192.168.197.211, 
Expected: 1.2.3.4

Which tells me the value to plug into the test, which is now green. This would be a good time to set up WorkingCopy and commit: Initial Commit. localIP() works.

I move the socket variable up to file-local and initialize it in before, as we’ll need it often:

local tweenSave
local socket

function test_WorldConnection()
    
    _:describe("WorldConnection", function()
        
        _:before(function()
            tweenSave = tween.update
            socket = require("socket")
        end)
        
        _:after(function()
            tween.update = tweenSave
        end)
        
        _:test("WorldConnection", function()
            local wc = WorldConnection()
            _:expect(wc:localIP()).is("192.168.197.211")
        end)
        
    end)
end

WorldConnection = class()

function WorldConnection:localIP()
    local server=socket.udp()
    server:setpeername("1.1.1.1",80)
    local myIP,myPort=server:getsockname()
    return myIP
end

Should be green. Green, commit: provide persistent socket access.

I think the next step will be to create a server socket.

        _:test("Create server", function()
            local wc = WorldConnection()
            local server = wc:createServer()
            server:close()
        end)

This is where it all starts getting tricky, because I have to be careful to close anything that I open.

I have code for creating a server somewhere:

function WorldConnection:createServer()
    local ip = self:localIP()
    local server = socket.tcp()
    ok,err = server:bind(ip,8888)
    assert(ok,err)
    server:listen()
    return server
end

This runs but of course doesn’t have any expectations. What could we possibly ask, I wonder?

In my research, I find that listen returns with 1, or nil with an error message, so:

function WorldConnection:createServer()
    local ok,err
    local ip = self:localIP()
    local server = socket.tcp()
    ok,err = server:bind(ip,8888)
    assert(ok,err)
    ok,err = server:listen()
    assert(ok,err)
    return server
end

Test again. Still green, carry on to get an expect in there …

        _:test("Create server", function()
            local wc = WorldConnection()
            local server = wc:createServer()
            local ip,port = server:getsockname()
            _:expect(ip).is("192.168.197.211")
            _:expect(port).is("8888")
            server:close()
        end)

This is green. I think I may have a server object. Commit: createServer passes test.

I wonder whether I can manage to get all this to work inside tests, without setting all of it up in before and after. Let’s try. First, let’s change the thing to have IP and server, not just return them.

        _:test("Server setup", function()
            local wc = WorldConnection()
            wc:setup()
            _:expect(wc._ip).is("192.168.197.211")
            local server = wc._server
            local ip,port = server:getsockname()
            _:expect(ip).is("192.168.197.211")
            _:expect(port).is("8888")
            wc:teardown()
        end)

And:

function WorldConnection:setup()
    self._ip = self:localIP()
    self._server = self:createServer()
end

function WorldConnection:teardown()
    if self._server then self._server:close() end
end

I expect this to pass. And it does. Commit: setup and teardown tested.

This is just about all the framework I can think of. Let’s see what we want in terms of operation.

We’re assuming that WorldConnection will install itself on a timer and on each tick, do two things:

  1. Check to see if there are any incoming connections, and if so, create a client for that connection, and put it in a table of clients.
  2. For each client in the table, check to see if there is a message, and if there is a message, call the standard callback (to be provided) passing the client and the message.

I probably should have mentioned those two things sooner, but while they were in my mind, they were not near the top.

Here’s where testing gets tricky. In order to test the discovery of incoming connections, I’ll need to create a client socket connected to our ip and port. I may need to send something to it, because I have some reason to believe that you don’t see the connection until after you send. This could just be me being confused.

Similarly for testing receiving messages, I’ll need a client socket.

One more thing: the current implementation of WorldConnection is poor, because it doesn’t get correctly initialized unless you call things in the right order, whereas it could just init itself properly right off the bat. But the tests don’t reflect that. We could improve things, but I think we’ll let that slide for now. I may regret that. We’ll see.

We could test-drive a bit more of this. Let’s do that. It’s a bit awkward and feels backward, but the result should be that I’ll have confidence in more of the plain vanilla code, so that when things inevitably hit the fan, I can focus on the difficult bits.

Let’s test-drive adding a client to a table.

This is a bit of a big test but it tells the story:

        _:test("add client", function()
            local wc = WorldConnection()
            local closeCount = 0
            local fake = {
                close=function() closeCount = closeCount+1 end
            }
            wc:setup()
            _:expect(wc:clientCount()).is(0)
            wc:addClient(fake)
            _:expect(wc:clientCount()).is(1)
            wc:addClient(fake)
            _:expect(wc:clientCount()).is(2)
            wc:teardown()
            _:expect(closeCount).is(2)
            _:expect(wc:clientCount()).is(0)
        end)

The fake object just implements a close function that ticks our local counter, so that we can be sure the object closed all the clients on teardown. And clientCount, of course, will just return the size of the client table.

That goes smoothly, resulting in:

function WorldConnection:addClient(client)
    table.insert(self._clients, client)
end

function WorldConnection:clientCount()
    return #self._clients
end

function WorldConnection:closeClients()
    for i,c in ipairs(self._clients) do
        c:close()
    end
    self._clients = {}
end

function WorldConnection:setup()
    self._ip = self:localIP()
    self._server = self:createServer()
    self._clients = {}
end

function WorldConnection:teardown()
    if self._server then self._server:close() end
    self:closeClients()
end

Note that a client, so far, just has to be able to deal with close, which is enough to let us test with that simple table fake.

I think if we turn that fake into a real object, we might be able to test further. Let’s do that.

        _:test("add client", function()
            local wc = WorldConnection()
            local closeCount = 0
            wc:setup()
            _:expect(wc:clientCount()).is(0)
            wc:addClient(Fake("hello"))
            _:expect(wc:clientCount()).is(1)
            wc:addClient(Fake("hiya"))
            _:expect(wc:clientCount()).is(2)
            wc:teardown()
            _:expect(closeCount).is(2)
            _:expect(wc:clientCount()).is(0)
        end)

And, with a bit more function than is tested yet, but this is a fake, cut me a break …

Fake= class()

function Fake:init(message)
    self._msg = message
end

function Fake:close()
end

function Fake:receive()
    return self._msg
end

Break

I just took a tiny break, and came to a realization.

If we treat the current object as a connection factory, we could have another object, the actual connection, that we could test almost entirely with fake objects. That would ensure that, if the sockets work as I believe they do, the object should perform as I expect it to.

We could even drive the connection with multiple (fake) clients, possibly even (at some future time) deal with error conditions and other dark subjects.

This is a good idea, well worth the walk down the hall. Let’s recast the tests to rename this thing.

First I have to make things run with my Fake, which takes some tweaking that I’ll spare you. Commit: Fake object serves as client.

Now let’s rename WorldConnection to ServerFactory. Tests are green. Well done, Codea. Commit: rename WorldConnection to ServerFactory.

Now let’s TDD the “real” WorldConnection.

WorldConnection For Real

I have a good feeling about this. It should let me test almost everything that needs testing. One minor issue is that I’ll be testing with fake objects, but even that may pay off: if the real objects don’t behave like my Fakes, I can figure out what’s wrong and fix up the fakes and the code to be right.

I’m going to make a new tab for this.

Everything hooked up and green. Commit: adding tab for WorldConnection.

OK. Now the WorldConnection needs to be given a server, on which it’ll do its accept call, and which will give him clients. Which reminds me, some of that last stuff in the old tab belongs here in WorldConnection now:

Grr. I probably should have disassembled it differently. No matter, I’ll TDD forward:

        _:test("WorldConnection", function()
            local server = FakeServer()
            local wc = WorldConnection(server)
            wc:addClient(Fake())
            _:expect(wc:clientCount()).is(1)
            wc:close()
            _:expect(wc:clientCount()).is(0)
        end)

I’ve decided to create my own fake here, and made the one in the old tab local, as I’ll do here as well.

It took a bit of code, but the test is green with the following:

local Fake = class()

function Fake:close()
end

WorldConnection = class()

function WorldConnection:init(server)
    self._server = server
    self._clients = {}
end

function WorldConnection:addClient(client)
    table.insert(self._clients, client)
end

function WorldConnection:clientCount()
    return #self._clients
end

function WorldConnection:close()
    self._server:close()
    for i,c in ipairs(self._clients) do
        c:close()
    end
    self._clients = {}
end

FakeServer = class()

function FakeServer:close()
end

Most of that was just the same as the old WorldConnection. Let’s commit: New WorldConnection passing client adding test.

I’m moving too fast. Let’s cool down by removing excess code and tests from the former WorldConnection, now ServerFactory.

I decide to keep these tests:

        _:test("ServerFactory", function()
            local wc = ServerFactory()
            _:expect(wc:localIP()).is("192.168.197.211")
        end)
        
        _:test("Create server", function()
            local wc = ServerFactory()
            local server = wc:createServer()
            local ip,port = server:getsockname()
            _:expect(ip).is("192.168.197.211")
            _:expect(port).is("8888")
            server:close()
        end)

And remove these:

        _:test("Server setup", function()
            local wc = ServerFactory()
            wc:setup()
            _:expect(wc._ip).is("192.168.197.211")
            local server = wc._server
            local ip,port = server:getsockname()
            _:expect(ip).is("192.168.197.211")
            _:expect(port).is("8888")
            wc:teardown()
        end)
        
        _:test("add client", function()
            local wc = ServerFactory()
            closeCount = 0
            wc:setup()
            _:expect(wc:clientCount()).is(0)
            wc:addClient(Fake("hello"))
            _:expect(wc:clientCount()).is(1)
            wc:addClient(Fake("hiya"))
            _:expect(wc:clientCount()).is(2)
            wc:teardown()
            _:expect(closeCount).is(2)
            _:expect(wc:clientCount()).is(0)
        end)

And the corresponding code. OK, nice.

Now back to the WorldConnection. I think that in operation it will have two main functions, acceptConnections and receiveMessages. The former will check to see if the server responds to accept with a client, and if it does, it’ll add the client. The latter message will go through each existing client and see whether receive returns a message, and if it does … I want it to call a provided function on a provided object, passing the client and the message.

Let’s work on that part first. Here’s a test that I would like to have work:

        _:test("client receive", function()
            local server = FakeServer()
            local wc = WorldConnection(server)
            local a = Fake("aaa")
            local b = Fake("bbb")
            wc:addClient(a)
            wc:addClient(b)
            _:expect(wc:clientCount()).is(2)
            b:ready()
            wc:processClients()
            _:expect(result).is("bbb")
        end)

We create a server that has two clients a and b. We tell b that it is “ready” and process clients. After that, the variable result should contain “bbb”, the value in the b client fake.

This will not run for want of ready:

2: client receive -- TestWorldConnection:97: attempt to call a nil value (method 'ready')

OK …

function Fake:init(msg)
    self._msg = msg
    self._ready = false
end

function Fake:ready()
    self._ready = true
end

Now we’ll fail wanting processClients:

2: client receive -- TestWorldConnection:103: attempt to call a nil value (method 'processClients')

Implement that:

function WorldConnection:processClients()
    for i,c in ipairs(self._clients) do
        data,err = c:receive()
        if data then
            result = data
        end
    end
end

Test fails for want of receive on the Fake:

2: client receive -- TestWorldConnection:44: attempt to call a nil value (method 'receive')

If the client has a message, it returns it. If it has not, it returns a nil followed by an error. Therefore:

function Fake:receive()
    if self._ready then
        return self._msg
    else
        return nil,"timeout"
    end
end

I expect this to work.

It does. I realize that if I make the test harder, it’s going to break. So I do:

        _:test("client receive", function()
            local server = FakeServer()
            local wc = WorldConnection(server)
            local a = Fake("aaa")
            local b = Fake("bbb")
            wc:addClient(a)
            wc:addClient(b)
            _:expect(wc:clientCount()).is(2)
            b:ready()
            wc:processClients()
            _:expect(result).is("bbb")
            a:ready()
            wc:processClients()
            _:expect(result).is("aaa")
        end)

This is going to fail with bbb, because I don’t unready the clients after they return their message:

2: client receive  -- 
Actual: bbb, 
Expected: aaa

Good. Fix:

function Fake:receive()
    if self._ready then
        self._ready = false
        return self._msg
    else
        return nil,"timeout"
    end
end

OK. Commit: WorldConnection has partially-functioning processClients, needs to provide target object/method.

Arrgh, I forgot to commit the trimmed down ServerFactory. No harm done, it’s committed now.

I’m tired and want a break. I’ll finish this article, push it, and do more later today or tomorrow or Monday …

Summary

I think this is going better than the past few days. Two things are helping me.

First, as we’ve perhaps discussed in the past, good judgment comes from experience, and experience comes from bad judgment. I’ve had a few days now of doing wrong things with sockets, and in the course of the days I’ve gained some experience that was useful today. I even had a bit of socket code that seems to work.

Second, with a clearer picture in mind of what’s needed, and a stronger than usual dedication to testing first, I’ve begun to create a reliable WorldConnection object that will properly deal with objects that can accept and receive. This is requiring the use of some simple fake objects, but has the great advantage that I can be as confident as I wish that the right functions are being called at the right times.

So far, we have the basic client processing loop in place, in simple form. We need to improve it so that the server is given an object and a method to call on that object, passing in the message received. It’s that method that will call the World and send back the response.

I think we have a solid base here, and I think I’ll go watch a bit of the Tour.

See you next time!