Building a Go Web API

I’m learning Go by building a small API-backed web application, and wanted to share the process in case it helps someone else. By the end of this post, we’ll have a functioning HTTP API with basic CRUD features and an SQL backend. The product we’ll be building is a webapp to manage GitHub “stars”.

If this was Python or JavaScript, we’d probably start by choosing a web application framework like Django or Express.js. While Go does have many competing web frameworks, the more popular way to build web applications seems to be choosing individual libraries for different components: routers, ORMs, middleware, etc. There are a lot of options, and it can be hard to understand what any of them do if you don’t already have some Go experience.

I spent some time going through and comparing different Go libraries, and figuring out what I wanted to use. I’ll introduce each external library as it comes up, and explain why I chose it; feel free to swap them out for alternatives if you want something different.

What You’ll Need

Before we get started, you’ll need a few things:

• Go installed on your computer.
• A database client that supports SQLite (e.g. the sqlite3 command).
• A web browser.
• A text editor.

You may also want to skim through the tour of Go or another introduction to the Go language if you haven’t already, though you shouldn’t need more than a basic understanding.

If you run into anything unclear in this post, feel free to open an issue on GitHub and let me know!

HTTP Server

Let’s start with a minimal functioning HTTP server, which I pulled from “Golang for Node.js Developers”:

package main

import (
  "net/http"
)

func handler(w http.ResponseWriter, r *http.Request) {
  w.WriteHeader(200)
  w.Write([]byte("hello world"))
}

func main() {
  http.HandleFunc("/", handler)
  if err := http.ListenAndServe(":8080", nil); err != nil {
    panic(err)
  }
}

Save this as main.go, and run it with go run main.go. We should be able to browse to http://localhost:8080 and see the “hello world” text.

The http.HandleFunc call tells Go to send any requests to URLs beginning with / to our handler function.

Inside the handler function, we write an HTTP “200 OK” response with our “hello world” text.

You can read more about http.HandleFunc, the URL pattern matching syntax, and http.ResponseWriter in the Go docs.

Now that we have a basic HTTP server, let’s hook it up to a database so it can do something other than serve hardcoded text.

Database

We need to define some way of representing objects in the database so that Go understands it. We could define the schema ourselves and use raw SQL queries to access records, but that’s difficult to maintain. Instead, I looked for a Go library that could act like an ORM (for simpler code), handle migrations (for easier maintenance), and supported multiple databases (such as MySQL and SQLite3, for flexibility). I eventually landed on GORM, which does all of the above.

Install GORM and the SQLite driver:

go get github.com/jinzhu/gorm
go get github.com/mattn/go-sqlite3

And add them to the import list at the top of main.go:

import (
  "net/http"
  "github.com/jinzhu/gorm"
  _ "github.com/jinzhu/gorm/dialects/sqlite"
)

(The import _ "package" syntax allows importing just for the side effects, in this case setting up SQLite as the SQL driver for GORM.)

Since the goal of this app is to manage stars, add a Star struct in main.go, just above the handler function:

type Star struct {
  ID uint
  Name string `gorm:"unique"`
  Description string
  URL string
}

If you’re familiar with the MVC patten used by many web frameworks, this Star struct is our “model”. Instead of defining tables in our SQL database, we define models, and GORM manages the database for us. If we update the Star struct with new fields, or create additional models, GORM automatically figures out the differences and migrates the database.

You’ll notice the Name field has a special struct tag that tells GORM that values in the name column must be unique: there can’t be two stars with the same name. GORM’s default behavior is a little more complex than we need right now, so we’re doing this instead. You can read more about GORM models in their documentation.

HTTP Handler

Because we want to read from the database, we’ll need some way to reference it inside the handler. There isn’t a way to pass values in to the handler function when it gets called, but we can refactor it a little to be a struct method, which gives it access to the fields inside the struct.

We’ll start by creating an App struct to hold a database pointer:

type App struct {
  DB *gorm.DB
}

Write a function (we’ll call it Initialize here, but you could call it anything; there are no constructors in Go) to initialize App objects and open a connection to the database:

func (a *App) Initialize(dbDriver string, dbURI string) {
  db, err := gorm.Open(dbDriver, dbURI)
  if err != nil {
    panic("failed to connect database")
  }
  a.DB = db

  // Migrate the schema.
  a.DB.AutoMigrate(&Star{})
}

And call it in our main function, passing in sqlite3 as the database driver and test.db as the database URI:

func main() {
  a := &App{}
  a.Initialize("sqlite3", "test.db")

  http.HandleFunc("/", handler)
  if err := http.ListenAndServe(":8080", nil); err != nil {
    panic(err)
  }

  defer a.DB.Close()
}

Note the defer call at the end of main, so that the database connection is always closed when main exits.

Now, we can refactor the handler function to be a “pointer receiver” method for App pointers, and use app.DB to access our database:

func (a *App) handler(w http.ResponseWriter, r *http.Request) {
  // Create a test Star.
  a.DB.Create(&Star{Name: "test"})

  // Read from DB.
  var star Star
  a.DB.First(&star, "name = ?", "test")

  // Write to HTTP response.
  w.WriteHeader(200)
  w.Write([]byte(star.Name))

  // Delete.
  a.DB.Delete(&star)
}

For now, our handler function just contains some test code to verify we can write, read, and delete from the database.

Finally, update the http.HandleFunc call in main to call the new App.handler function on our App record:

  http.HandleFunc("/", a.handler)

Let’s run go run main.go again and take a look in our browser: the page should now say “test” instead of “hello world”. We’re successfully reading and writing to the database!

Router

For our API, we’ll be following standard REST conventions, which means we’ll be using several HTTP verbs: GET, POST, PUT, and DELETE, and paths containing object IDs that need to be parsed out. It’s possible to do this with the Go standard library, but the resulting code isn’t especially maintainable. To make it a little easier, we’re going to use mux, a “router and dispatcher” that adds some useful features on top of Go’s http.ServeMux.

Install mux:

go get github.com/gorilla/mux

Add it to the import list:

import (
  "net/http"
  "github.com/jinzhu/gorm"
  _ "github.com/jinzhu/gorm/dialects/sqlite"
  "github.com/gorilla/mux"
)

And update main to use it:

func main() {
  a := &App{}
  a.Initialize("sqlite3", "test.db")

  r := mux.NewRouter()

  r.HandleFunc("/", a.handler)

  http.Handle("/", r)
  if err := http.ListenAndServe(":8080", nil); err != nil {
    panic(err)
  }

  defer a.DB.Close()
}

For now, this should work exactly the same as it did before we tied in mux. Let’s set up our new API routes.

We want to have five handlers for our API:

List Handler

Let’s start with the List handler:

func (a *App) ListHandler(w http.ResponseWriter, r *http.Request) {
  var stars []Star

  // Select all stars and convert to JSON.
  a.DB.Find(&stars)
  starsJSON, _ := json.Marshal(stars)

  // Write to HTTP response.
  w.WriteHeader(200)
  w.Write([]byte(starsJSON))
}

We start by declaring a slice of Star records. This acts as a dynamically allocated array, which Find fills with every star in our database, resizing the slice as needed. We then convert, or marshal, the slice to a JSON string, and write that JSON out as our API response.

Since we’re using Go’s built-in JSON support, we’ll need to add that to the imports at the top of the file:

import (
  "net/http"
  "encoding/json"
  "github.com/jinzhu/gorm"
  _ "github.com/jinzhu/gorm/dialects/sqlite"
  "github.com/gorilla/mux"
)

And we can hook up our new ListHandler to our router in main by replacing the old a.handler route:

  r.HandleFunc("/", a.handler)

With the new a.ListHandler route:

  r.HandleFunc("/stars", a.ListHandler).Methods("GET")

(You can delete the old handler function too.)

Note that we changed the path from / to /stars, and restricted the route to only handle GET requests. If we run the app again and open our browser to http://localhost:8080/stars, we should see a (presumably empty) list of all the stars in our database:

[]

With our SQL client, we open test.db and insert a star into our database with a query like this:

insert into stars values (1, "My Star", "This is a description", "http://example.com");

Now, when we refresh the API response in our browser, it should show the new star:

[{"ID":1,"Name":"My Star","Description":"This is a description","URL":"http://example.com"}]

We notice the JSON keys (Name, Description, URL) are capitalized; by convention, these are usually lowercase, so let’s update that before continuing:

type Star struct {
  ID uint `json:"id"`
  Name string `gorm:"unique" json:"name"`
  Description string `json:"description"`
  URL string `json:"url"`
}

Adding these special JSON struct tags to our Star struct tells the JSON encoder to use the lowercased key names we provided:

[{"id":1,"name":"My Star","description":"This is a description","url":"http://example.com"}]

View Handler

Next up, we need a handler to show the details of a single star, specified by its name.

func (a *App) ViewHandler(w http.ResponseWriter, r *http.Request) {
  var star Star
  vars := mux.Vars(r)

  // Select the star with the given name, and convert to JSON.
  a.DB.First(&star, "name = ?", vars["name"])
  starJSON, _ := json.Marshal(star)

  // Write to HTTP response.
  w.WriteHeader(200)
  w.Write([]byte(starJSON))
}

We check mux.Vars for the name variable (defined in the route below), and grab the first matching star from the database. (Since name is marked as unique, there will only ever be one.) We then convert the star record to JSON just like we did in the List handler, and write it out in the API response, again with a ‘200 OK’ HTTP status code.

Set up the View route directly below our List route in main:

  r.HandleFunc("/stars/{name:.+}", a.ViewHandler).Methods("GET")

The {variable:pattern} syntax in the path lets us match and extract named variables from the URL. By default, variables will match anything until the next forward slash /, but since GitHub repositories have slashes in the name (owner/repository), we have to expand the regex pattern to match everything: .*. Put this all together and /stars/{name:.*} will match http://localhost:8080/stars/owner/repository and pass in owner/repository as the name when we check mux.Vars.

With the route and handler in place, we should be able to open http://localhost:8080/stars/My Star and see the star details:

{"id":1,"name":"My Star","description":"This is a description","url":"http://example.com"}

Create Handler

The next three handlers use HTTP methods other than GET, so you’ll need something like curl if you want to test the API endpoints. (Or check out the next post, where we’ll write automated tests for this API.)

Following REST API conventions, the Create handler will allow us to send POST requests to create new star records.

func (a *App) CreateHandler(w http.ResponseWriter, r *http.Request) {
  // Parse the POST body to populate r.PostForm.
  if err := r.ParseForm(); err != nil {
    panic("failed in ParseForm() call")
  }

  // Create a new star from the request body.
  star := &Star{
    Name: r.PostFormValue("name"),
    Description: r.PostFormValue("description"),
    URL: r.PostFormValue("url"),
  }
  a.DB.Create(star)

  // Form the URL of the newly created star.
  u, err := url.Parse(fmt.Sprintf("/stars/%s", star.Name))
  if err != nil {
    panic("failed to form new Star URL")
  }
  base, err := url.Parse(r.URL.String())
  if err != nil {
    panic("failed to parse request URL")
  }

  // Write to HTTP response.
  w.Header().Set("Location", base.ResolveReference(u).String())
  w.WriteHeader(201)
}

First, we call http.Request.ParseForm to parse out the variables (name, description, url) set in the POST request, which we can then access with http.Request.PostFormValue. Pass in the newly created star record to GORM’s Create function, and the star is saved to our database.

Then comes a tricky bit. The response to POST requests is supposed to return a Location header with the URL of the newly created resource - in this case the star we just saved to the database. In order to form that URL, we need two pieces of information: the star name and the base URL of our API, as well as a way to combine them without doing a bunch of messy manual URL parsing. Luckily, Go has a built-in URL parser with a url.URL.ResolveReference function that does exactly this! We get the API base URL with the URL.String method on the URL field of the http.Request struct, and combine it with /star/{name} to form the full URL. We then add that URL to the response header as the Location. Finally, we set the HTTP status code to “201 Created” to signify we’ve successfully created the star.

Hook up the Create handler with a new route in main:

  r.HandleFunc("/stars", a.CreateHandler).Methods("POST")

Add the fmt import, since we call fmt.Sprintf:

import (
  "fmt"
  "net/http"
  "net/url"

And it’s ready to go!

Update Handler

The Update handler accepts PUT requests to a given star name, and updates the database with any new variables passed in the request.

func (a *App) UpdateHandler(w http.ResponseWriter, r *http.Request) {
  vars := mux.Vars(r)

  // Parse the request body to populate r.PostForm.
  if err := r.ParseForm(); err != nil {
    panic("failed in ParseForm() call")
  }

  // Set new star values from the request body.
  star := &Star{
    Name: r.PostFormValue("name"),
    Description: r.PostFormValue("description"),
    URL: r.PostFormValue("url"),
  }

  // Update the star with the given name.
  a.DB.Model(&star).Where("name = ?", vars["name"]).Updates(&star)

  // Write to HTTP response.
  w.WriteHeader(204)
}

We parse the request and read the form values just as we did in the Create handler, and use GORM to Update the database.

We don’t really need to include anything in the response, so we set the response code to 204 No Content and return.

Add the Update route in main below the others:

  r.HandleFunc("/stars/{name:.+}", a.UpdateHandler).Methods("PUT")

Delete Handler

Last but not least, the Delete handler accepts DELETE requests for a given star, deletes it from the database, and again returns an empty body with HTTP status 204 No Content.

func (a *App) DeleteHandler(w http.ResponseWriter, r *http.Request) {
  vars := mux.Vars(r)

  // Delete the star with the given name.
  a.DB.Where("name = ?", vars["name"]).Delete(Star{})

  // Write to HTTP response.
  w.WriteHeader(204)
}

Hook it up below the other routes in main:

  r.HandleFunc("/stars/{name:.+}", a.DeleteHandler).Methods("DELETE")

And our API is complete!

You can check out the complete code for this post on GitHub.

Conclusion

A quick recap:

1. We started with a minimal HTTP server.
2. Added an SQL library to map our model struct to a database and store all our information.
3. Added a router library to make defining API routes easier and more readable.
4. Created five functions to handle standard Create/Read/Update/Delete (CRUD) functionality.

In the process, we covered several Go features and concepts:

• Structs and struct tags.
• Methods on struct types.
• Deferred execution.
• JSON marshalling.
• Error handling with panic.
• Creating HTTP servers.
• Third-party libraries (GORM and mux).
• And more!

In future posts, I’ll revisit this API and walk through writing automated tests, improving error handling, adding middleware like logging and authentication, and creating a frontend for the star app.

Credits

Thanks to Patrick Gabbett (@NightinGem) for proofreading and editing this post!

Additional References

• Golang RESTful API using GORM and Gorilla Mux
• Passing in parameters to a http.HandlerFunc
• Go’s http package by example
• Go by Example
• Writing Web Applications

Ryan Shipp

Software Engineer