GitHub Issuer

Welcome back! Though if you haven’t read the first part, you may want to. We’re expanding on the code that we write last time. Adding in the ability to actually create new issues in our TODO repository and add them to the kanban board. Yes the most over-engineered TODO “system” is going to get an upgrade. With that out of the way, let's get right into it.

Lets Go

Our imports have expanded as we’re pulling in a bunch of bits from the standard library and a few external packages. The go-github package is going to do quite a bit of heavy lifting for us. oauth2 is coming along for the ride so we can use a GitHub personal access token to authorize our requests.

package main

import (
  "bytes"
  "context"
  "encoding/json"
  "fmt"
  "io/ioutil"
  "log"
  "net/http"
  "os"

  "github.com/google/go-github/v25/github"
  "github.com/shindakun/envy"
  "golang.org/x/oauth2"
)

Currently, we’re setting a few constants. We may bring these up out of the code and make the environment variables in the “production” version. For local testing though it’s probably fine. The token, however, is already set as an environment variable, which should keep me from accidentally committing it to GitHub. It’s good practice to keep tokens out of the code whenever possible.

const (

  // RepoOwner is the owner of the repo we want to open an issue in
  RepoOwner = "shindakun"

  // IssueRepo is the repo we want to open this new issue in.
  IssueRepo = "to"

  // ProjectColumn is the TODO column number of the project we want to add the issue to
  ProjectColumn = 5647145
)

// Token is the GitHub Personal Access Token
var Token string

// Secret is used to validate webhook payloads
var Secret string

Our Payload is pretty much set, we don’t need anything else from the responses for now. Our status handler will remain the same as well.

// Payload of GitHub webhook
type Payload struct {
  Action string `json:"action"`
  Issue struct {
    URL string `json:"url"`
    RepositoryURL string `json:"repository_url"`
    Number int `json:"number"`
    Title string `json:"title"`
    Body string `json:"body"`
  } `json:"issue"`
  Repository struct {
    Name string `json:"name"`
  } `json:"repository"`
}

func status(res http.ResponseWriter, req *http.Request) {
  fmt.Fprintf(res, "Hello!")
}

The webhook handler starts off the same. But quickly deviates.

func handleWebhook(res http.ResponseWriter, req *http.Request) {
  var Payload Payload
  defer req.Body.Close()

We take our incoming request and pass it and our Secret into github.ValidatePayload(). The X-Hub-Signature on the incoming request comes with a signature compare against our calculated signature. If it matches we’re good to go.

The HMAC hex digest of the response body. This header will be sent if the webhook is configured with a secret. The HMAC hex digest is generated using the sha1 hash function and the secret as the HMAC key.

This protects us from someone accidentally finding our endpoint and submitting requests. Sure the chances are low but why take chances. If the request doesn’t pass validation we simply return and carry on.

  p, err := github.ValidatePayload(req, []byte(Secret))
  if err != nil {
    http.Error(res, "bad request: "+err.Error(), 400)
    log.Printf("bad request: %v", err.Error())
    return
  }

github.ValidatePayload() returns a []byte of the payload which we need to wrap in a “ReadCloser” which we can then pass to jsonNewDecoder() so we can parse the JSON object as our final Payload. Again, if anything goes wrong we’ll log the error and return. If all goes well, we pass our Payload to createNewIssue().

Update As @kunde21 points out in the comments, this really should have been re-written to use json.Unmarshall(). This does work but is a bit unsightly and likely not as performant.

  decoder := json.NewDecoder(ioutil.NopCloser(bytes.NewBuffer(p)))
  err = decoder.Decode(&Payload)
  if err != nil {
    http.Error(res, "bad request: "+err.Error(), 400)
    log.Printf("bad request: %v", err.Error())
    return
  }

  err = createNewIssue(&Payload)
  if err != nil {
    log.Printf("bad request: %v", err.Error())
    return
  }
}

createNewIssue() first starts by logging out the details of our payload. This is just for testing purposes and will be removed I think.

func createNewIssue(p *Payload) error {
  log.Printf("Creating New Issue.\n")
  log.Printf(" Name: %#v\n", p.Repository.Name)
  log.Printf(" Title: %#v\n", p.Issue.Title)
  log.Printf(" Body: %#v\n", p.Issue.Body)
  log.Printf(" URL: %#v\n", p.Issue.URL)

First things first, we’ll get our oauth2 and GitHub client ready to go. This is as recommended by the go-github repo.

  ctx := context.Background()
  ts := oauth2.StaticTokenSource(
    &oauth2.Token{AccessToken: Token},
  )
  tc := oauth2.NewClient(ctx, ts)

  client := github.NewClient(tc)

Now it’s time to build our new issue. I wanted the title to reflect which repo it was coming from.

[From repo] Remember to write a post

The body of the repo holds whatever was originally entered and a link back to the source repo. We then pack the title and body into github.IssueRequest and create the new issue!

  title := fmt.Sprintf("[%s] %s", p.Repository.Name, p.Issue.Title)
  body := fmt.Sprintf("%s\n%s/%s#%d", p.Issue.Body, RepoOwner, p.Repository.Name, p.Issue.Number)

  issue := &github.IssueRequest{
    Title: &title,
    Body: &body,
  }

  ish, _, err := client.Issues.Create(ctx, RepoOwner, IssueRepo, issue)
  if err != nil {
    log.Printf("error: %v", err)
    return err
  }

We are not quite done though. I want to make sure the new issue is added to the TODO kanban board. So we take the details from the new issue, extract the issue ID number and set up a new “card” with github.ProjectCardOptions.

  id := *ish.ID
  card := &github.ProjectCardOptions{
    ContentID: id,
    ContentType: "Issue",
  }

We aren’t too concerned with the details return from this call so we just check for an error and return if need be.

  _, _, err = client.Projects.CreateProjectCard(ctx, ProjectColumn, card)
  if err != nil {
    log.Printf("error: %v", err)
    return err
  }

  return nil
}

And that brings us to our updated main(). We’ve added a bit of code to grab our environment variables and if not set we’ll bail out with an error.

func main() {
  log.Println("Issuer")
  var err error

  Token, err = envy.Get("GITHUBTOKEN")
  if err != nil || Token == "" {
    log.Printf("error: %v", err)
    os.Exit(1)
  }

  Secret, err = envy.Get("SECRET")
  if err != nil || Secret == "" {
    log.Printf("error: %v", err)
    os.Exit(1)
  }

  http.HandleFunc("/", status)
  http.HandleFunc("/webhook", handleWebhook)
  http.ListenAndServe(":3000", nil)
}

Running

Alright, let's run it and make a new issue in our test “from” repo.

SECRET=TESTSECRET GITHUBTOKEN=1234567890 go run main.go
2019/06/15 11:23:32 Issuer
2019/06/15 11:24:42 Creating New Issue.
2019/06/15 11:24:42 Name: "from"
2019/06/15 11:24:42 Title: "asdfasdf"
2019/06/15 11:24:42 Body: "asdfasdfasdfasdfasdf"
2019/06/15 11:24:42 URL: "https://api.github.com/repos/shindakun/from/issues/13"

Perfect! Now, all we need to do is throw it on a box and point our GitHub repos webhook settings at the proper URL.

Next time

That went pretty smooth! Next time I think we’ll convert this into something we can deploy on Google Cloud Functions! Which will make it much easier to deploy.

Questions and comments are welcome!

You can find the code for this and most of the other Attempting to Learn Go posts in the repo on GitHub.