Moving toward standard request-response. Seems about time. But Never would be better. ALSO: A very brief introduction to industry standards of completeness.

Since I’ve designed World and Robot to respond to methods with arguments, which seems to me to be the way, I need to layer in the creation and interpretation of requests and responses, which our spec has defined as dictionaries containing various named values and arrays. These get converted to and from JSON for transmission between the client and server.

This is a truly irritating requirement. We’ve already spiked sending and receiving messages across the socket interface. We’ve shown that we can encode and decode well-formed objects to and from JSON. We’ve shown that we can pack arguments into dictionary structures and get them back out.

There is very little learning left here, with one exception. We have not shown that we can manage a number of simultaneous connections to our server, with connections, disconnections, and all that jazz. And, here on my iPad, we’re not going to do that, or at least that’s my current plan. The iPad doesn’t support multi-tasking well, I’m not going to bore a hole in my firewall, and I’m not going to spin up Java or Kotlin or something on my Mac just to message a server that I plan never to use.

Unless, of course, I decide to do it just to flex at the universe.

What Would You LIKE To Do?

Thanks for asking. What I’d like to do would be to begin to create a game around the Robot Worlds idea, but not the game of Robot v Robot that is contemplated in the spec. Perhaps a game of exploration and discovery. And, of course, it’s my house, my rules, so I can surely do that.

But perhaps I owe it to the students who may one day read this, faced by the Robot Worlds spec, in hopes of learning something, even if it’s how to avoid my bad examples, to get closer to completion than we are now. It’ll be kind of a pain. It seems to me that it will have no real value beyond learning …

But no! Wait! One thing that is possible is that if there ever were to be a server that plays the Robot Worlds game, my Robot code could be adapted to connect to it from an iPad, and to play the game. That would be rather nice.

OK, I’ve generated a small amount of motivation to continue with the protocol. I hate it when I talk myself into doing something that seems right but that I don’t really want to do.

How Should We Proceed?

There’s a fundamental notion in programming, “Separation of Concerns”. Closely related to the notion of Cohesion, the SoC notion says that if there are two different concerns to be dealt with, things go best if the code is also separated. In our case, we have at least these concerns:

  • Being a good robot;
  • Being a good world;
  • Communicating between robot and world;
  • Communicating across sockets;

Those items break down further, of course. Details that come to mind include:

  • Convenient calling sequences;
  • Convenient return information;
  • Standardized calls and returns;
  • Standard network-compatible messages;

And so on and so on, until finally we have machine instructions doing all the things.

To make this concrete, I’d like to separate, as completely as possible, the Robot’s desired calls and returns from the requests, responses, and JSON. To the extent possible, the Robot should be able to organize its calls, returns and saved information to its own liking.

Now the situation with World is a bit different. I’d still like separation of concerns, of course, but the World is a general server. Each robot could be on a different computer, programmed in a different language, but there is a single World server that handles them. And the form of the requests and responses is part of the World’s definition, part of its responsibilities. So we may find that the World should be a bit more cognizant of the form of its inputs and outputs, a bit more shaped to deal with them.

In other words, the World doesn’t have to flex in order to deal with something our particular Robot wants to do.

Does that matter? Honestly I don’t know. I’m just trying to find my way here.

Have You Even Got A Clue??

Well, yes. Kind of. I want the WorldProxy object, and whatever other objects it might need, to “completely” isolate my Robot from the realities of the Request and Response. For high values of “completely”, perhaps in the range of 0.75 to 0.9 Comps1.

We’re already on this course, though the code doesn’t fully support it. So WorldProxy’s job is to pretend to be the kind of World we want, consistent with the spec, but couched in terms of the Robot’s convenience. Inside its mechanisms, the WorldProxy will create requests and receive responses. WorldProxy, in its own turn, may talk directly to the World, handing it a request and receiving back its response, or it might hand the request to a JSON thing that hands it to a socket that passes it to another JSON thing that reinflates the request and hands it to the World …

From the Robot’s viewpoint, I don’t want to care. I will probably limit the Robot’s requests to the WorldProxy to individual operations that the World can actually do. We could imagine a WorldProxy smart enough to make multiple requests to the World, combining responses and finally returning some result of the group of requests, but we’ll set that idea firmly aside, probably forever.

Steps?

We’ll move toward calls from Robot having individual methods in WorldProxy (as they do), and having those methods return what the Robot wants to know. The WorldProxy’s job is to serve the Robot.

On the World side, we’ll move more toward receiving requests in the spec’s form and returning responses as the spec says. So our WorldProxy will need to deal with that.

Where Are We Now?

We’re kind of part way there. If I recall way back to yesterday, we have a partially-formatted response dictionary coming back to each of those. Let’s review the related code.

function Robot:init(name,aWorld)
    assert(aWorld:is_a(WorldProxy), "expected WorldProxy")
    self._world = aWorld
    self._state = State(aWorld:launchRobot(name, self))
    self._x, self._y = self._state:position()
    self._name = name
    self.knowledge = Knowledge()    
end

function Robot:forward(steps)
    self._state = self._world:forward(self._name, steps)
    self.knowledge = self.knowledge:newLensAt(self:x(),self:y())
end

These two things are not alike. The first sets state to a State object. The second does not. Presumably the proxy code differs:

function WorldProxy:launchRobot(...)
    local response = self.world:launchRobot(...)
    return response.state
end

function WorldProxy:forward(name,steps)
    local result
    local rq = {
        robot=name,
        command="forward",
        arguments={steps}
    }
    local jsonRq = json.encode(rq)
    local decoded = json.decode(jsonRq)
    local dName = decoded.robot
    local dCommand = decoded.command
    if dCommand == "forward" then
        local dSteps = decoded.arguments[1]
        local responseDict = self.world:forward(dName,dSteps)
        return State(responseDict.state)
    end
    assert(false,"impossible command "..dCommand)
end

Oh, we also have the scan function to consider.

function Robot:scan()
    local packets = self._world:scan(self._name)
    for i,packet in ipairs(packets) do
        self:addLook(packet)
    end
end

function WorldProxy:scan(...)
    local jsonString = self.world:scan(...)
    local outcome = json.decode(jsonString)
    local packets = outcome.data.objects
    local result = {}
    for i,p in ipairs(packets) do
        local lp = LookPacket:fromObject(p)
        table.insert(result, lp)
    end
    return result
end

Looking at these three possibilities, we see that, often, the primary concern of the robot is to deal with the information returned, such as the results of scanning. It doesn’t care about its state … but it should. It might have been shot at by another robot. It might even be dead.

That makes me think that the robot needs to deal, one way or another, with the two main components that come back in a response, the (result) data and the (robot) state.

Be that as it may, I think there are two things that need to be done soonest:

  1. Always send a request to the World, rather than calling it directly;
  2. Always return a response to the request, rather than some subset object.

There is another matter, seemingly separate from those. The WorldProxy forward method above is a sketch of one possible flow of control. It’s pretending, down there in the if statement, that any number of commands might flow through there, and that they’d want different kinds of returns.

As I look at that now, I think it’s an interesting sketch, and I’m glad it’s there to look at, but I think it’s premature. I do think there will be common setup and common return code for all the requests … but I think it’s too soon to do anything as fancy as that.

For a while, we’ll let the request methods be separate, and we’ll look for duplication.

And one more thing: we’ll work toward the proxy passing a request dictionary directly to World, and getting a response dictionary back, finessing all the json and socket stuff. We can deal with those with additional adapters if and when we want to.

Oh, and yet another thing: On the World side, we’ll probably need support objects as well, but we’ll start with a single method, “request”, to which all the request thingies are sent.

We have to start somewhere. Let’s start somewhere easy, with launch.

function WorldProxy:launchRobot(...)
    local response = self.world:launchRobot(...)
    return response.state
end

We’re going to need to know and understand the arguments to these methods now, because they pertain to the contract between the proxy and the robot, not to the contract between robot and world. That contract has to do with the request-response, and the robot does not know that: the proxy does.

In this case the parameters are name and the robot itself.

function WorldProxy:launchRobot(name,robot)
    local response = self.world:launchRobot(...)
    return response.state
end

Let’s look up what the request is supposed to be. I’ll put it in the code for now:

function WorldProxy:launchRobot(name,robot)
    local kind = "RonRobot1"
    local shields = 99
    local shots = 99
    local request = {
        robot=name,
        command="launch",
        arguments={kind,shields,shots}
    }
    local response = self.world:launchRobot(name,robot)
    return response.state
end

The request asks for all those bits of info that we don’t have yet, so I’m just putting them in the method.

I rather suspect that the proxy needs to know the robot, but so far I’m not saving it. We certainly have it at this point.

All this should just work and run green: I didn’t change the call to the World. Yes, all good. I could commit. OK, sure, let’s get in the habit. Commit: WorldProxy:launchRobot forms a request, does not use it.

Now let’s see about TDDing a request method for World, that understands the launch command.

        _:test("request method", function()
            local world = World(25,25)
            local rq = {
                robot="xyzzy",
                command="launch",
                arguments={"plugh", 99,99}
            }
            world:request(rq)
            local robotState = world:getRobot("xyzzy")
            _:expect(robotState._name).is("xyzzy")
        end)

I’m positing a method getRobot that doesn’t exist, as well as a method request which also doesn’t exist. That’ll be the first fail.

15: request method -- TestWorld:345: attempt to call a nil value (method 'request')

Implement:

function World:request(rq)
    local c = rq.command
    local args = rq.arguments
    if c == "launch" then
        local result = self:launchRobot(rq.robot, args[1], args[2], args[3])
        return result
    else
        error("no such command "..tostring(c))
    end
end

This is nasty but most of it is because the spec is the way it is. Maybe we can make it better: we’ll see. This fails, as expected, on getRobot.

function World:getRobot(name)
    return self._robots[name]
end

I think this test is going to run. It does. Commit: Initial World:request function supports launch.

Now we can use the function in the proxy:

function WorldProxy:launchRobot(name,robot)
    local kind = "RonRobot1"
    local shields = 99
    local shots = 99
    local request = {
        robot=name,
        command="launch",
        arguments={kind,shields,shots}
    }
    local response = self.world:request(request)
    return response.state
end

I expect this to work. It does. Commit: Robot launch is done in WorldProxy as a request-response.

I daresay a cry of Woot! would go well at this point. Let’s reflect.

Reflection

We have a round-trip going, call to request, request to world, response back to proxy, results back to robot. This is a good thing, and it has already show up some issues.

The biggest one, to me, is seen here:

function World:request(rq)
    local c = rq.command
    local args = rq.arguments
    if c == "launch" then
        local result = self:launchRobot(rq.robot, args[1], args[2], args[3])
        return result
    else
        error("no such command "..tostring(c))
    end
end

We are passing in three arguments in addition to the robot name, and they are just passed as three elements of an array of arguments. It seems to me that the writers of the spec got tired and just didn’t want to go to the trouble of specifying named arguments for all the many calls to the World, so they just dumped things into arrays.

Similar things happen in the response.

Honestly, I understand doing it that way. I’m sure I’ve done it that way myself, more than once. But it’s very error-prone and generally unsafe. And any errors could be deferred for a long time. Suppose I were to set shields to a string. Or, here’s an interesting cheat … what if I set it to 1.5. With any luck, the World code would decrement it and check for zero. It’d never reach zero, and I’d never die.

Anyway, this interface is error-prone and if we were to really validate it, it would take forever and be a major irritation. It would, unfortunately, need to be done, because we can’t allow the server to crash no matter what ridiculous garbage the robots throw at it. As for here in my code … we may address how to do that but I really doubt that I’m going to do it. I’m not here to write about tedium and you aren’t here to read about it.

I did go beyond my test a bit, as I often do. Let’s write a test for that case …

        _:test("request throws on bad request", function()
            local world = World(25,25)
            local rq = {}
            local doRq = function() world:request(rq) end
            _:expect(doRq).throws("no such command nil")
        end)

Test passes. I’m almost surprised. Let’s glance at our code:

function World:request(rq)
    local c = rq.command
    local args = rq.arguments
    if c == "launch" then
        local result = self:launchRobot(rq.robot, args[1], args[2], args[3])
        return result
    else
        error("no such command "..tostring(c))
    end
end

We could return the response directly instead of caching it.

function World:request(rq)
    local c = rq.command
    local args = rq.arguments
    if c == "launch" then
        return self:launchRobot(rq.robot, args[1], args[2], args[3])
    else
        error("no such command "..tostring(c))
    end
end

We could do something better than error, returning an error result. I’ll look at the spec and see if they specify one.

If I were a fanatic, I’d protect against an error in the whole process. In Lua, there is a function pcall (protected call) to manage that. Just for fun, let’s try it.

There is the possibility to return an error response. Let’s first do that:

        _:test("request returns error response on bad request", function()
            local world = World(25,25)
            local rq = {}
            local resp = world:request(rq)
            _:expect(resp.result).is("ERROR")
            _:expect(resp.data.message).is("No such command nil")
        end)

This will fail by throwing.

15: request returns error response on bad request -- TestWorld:197: no such command nil

We implement:

function World:request(rq)
    local c = rq.command
    local args = rq.arguments
    if c == "launch" then
        return self:launchRobot(rq.robot, args[1], args[2], args[3])
    else
        return self:error("No such command "..tostring(c))
    end
end

function World:error(message)
    return {
        result="ERROR",
        data={message=message}
    }
end

Test should pass. Does. Commit: unknown command returns ERROR response.

Now let’s do the protected call. How can I test that? Ah, I know. Passing in a nil robot name should do it.

        _:test("return error response if server throws", function()
            local world = World(25,25)
            local rq = { command="launch" }
            local resp = world:request(rq)
            _:expect(resp.result).is("ERROR")
            _:expect(resp.data.message).is("SERVER ERROR")
        end)

This will fail with an exception I think … Perfect!

17: return error response if server throws -- 
TestWorld:195: attempt to index a nil value (local 'args')

And now …

function World:request(rq)
    ok, result = pcall(function() return self:requestProtected(rq) end)
    if ok then
        return result
    else
        return self:error("SERVER ERROR")
    end
end

function World:requestProtected(rq)
    local c = rq.command
    local args = rq.arguments
    if c == "launch" then
        return self:launchRobot(rq.robot, args[1], args[2], args[3])
    else
        return self:error("No such command "..tostring(c))
    end
end

I’ve never done a pcall before, but I expect this to work. All is good. We are now rather certain that a call to World:request will return a response in any case. We could, however, break this certainty if any branch in the requestProtected failed to return a result.

Maybe having them return directly out of the if isn’t a good idea.

Let’s write a test that will deal with that.

        _:test("request always returns a response", function()
            local world = World(25,25)
            local rq = { command="noResponse" }
            local resp = world:request(rq)
            _:expect(resp.result).is("ERROR")
            _:expect(resp.data.message).is("SERVER RETURNED NO RESPONSE TO noResponse")
        end)

First we implement our buggy command:

function World:requestProtected(rq)
    local c = rq.command
    local args = rq.arguments
    if c == "noResponse" then
        return {a=1}
    elseif c == "launch" then
        return self:launchRobot(rq.robot, args[1], args[2], args[3])
    else
        return self:error("No such command "..tostring(c))
    end
end

Test will fail.

18: request always returns a response  -- 
Actual: nil, 
Expected: ERROR
18: request always returns a response -- TestWorld:403: attempt to index a nil value (field 'data')

Now to make things better:

function World:requestProtected(rq)
    local c = rq.command
    local args = rq.arguments
    local response
    if c == "noResponse" then
        -- nothing
    elseif c == "launch" then
        response = self:launchRobot(rq.robot, args[1], args[2], args[3])
    else
        response = self:error("No such command "..tostring(c))
    end
    if self:isValidResponse(response) then
        return response
    else
        return self:error("SERVER RETURNED NO RESPONSE TO "..tostring(c))
    end
end

function World:isValidResponse(r)
    if not r then return false end
    if r.result ~= "OK" and r.result ~="ERROR" then return false end
    if not r.data then return false end
    return true
end

I probably should have done that in smaller steps but we’re good. The new rule is that the commands should return a response. The dispatcher will store whatever comes back in local response and validate it before returning it. The validation isn’t terribly comprehensive but covers the case of no return or something that’s seriously unlike a valid response.

Nearly 1320. I started late this morning, but we’re still about three hours in, so let’s sum up.

Summary

We have a round trip command, launchRobot, that creates a request on the robot side, in the proxy, sends it to World:request, and gets a proper response back, returning to the robot what it currently wants, just the state.

That probably needs to be revised, so that the Robot deals with both response details and state. But it’s what we wanted so far.

The request method in World will become the only one we call directly (or via socket json stuff). That method protects against exceptions in the server, and checks for valid responses—albeit weakly—ensuring that a response is always returned to the caller.

Not bad, and a decent framework for building up the rest of the request-response connection.

A good morning’s work. That’s my story, and I’m sticking to it.

See you next time!


  1. The Comp, as you probably know, is the international standard unit of completeness, which has a maximum value of 1.0 and a minimum value of 0.0. ↩