Using Gemini models in Go with LangChainGo

In a previous post I've discussed how to access Google's multimodal Gemini models from Go (with a nice free tier!)

Recently, Google's SDKs were added as providers for LangChainGo; this makes it possible to use the capabilities of the LangChain framework with Google's Gemini models as LLM providers.

This post shows some samples of using these new providers and how simple it is to switch providers from Google AI (which uses API keys) to Vertex (which requires a GCP project).

LangChainGo examples with GoogleAI

Let's start with the GoogleAI provider. We'll need the latest release of langchaingo. Here's a complete example of asking the model a basic textual question:

package main

import (
  "context"
  "fmt"
  "log"
  "os"

  "github.com/tmc/langchaingo/llms"
  "github.com/tmc/langchaingo/llms/googleai"
)

func main() {
  ctx := context.Background()
  apiKey := os.Getenv("API_KEY")
  llm, err := googleai.New(ctx, googleai.WithAPIKey(apiKey))
  if err != nil {
    log.Fatal(err)
  }

  prompt := "What is the L2 Lagrange point?"
  answer, err := llms.GenerateFromSinglePrompt(ctx, llm, prompt)
  if err != nil {
    log.Fatal(err)
  }

  fmt.Println(answer)
}

llms.GenerateFromSinglePrompt is a convenience function in LangChainGo for cases where we have a single string input and want a single string output. The more general API is the Model.GenerateContent method, which supports multiple messages and different message kinds (text, images, etc.); Here's an example using it to reproduce our question about the difference between turtle images from the previous post:

func main() {
  ctx := context.Background()
  apiKey := os.Getenv("API_KEY")
  llm, err := googleai.New(ctx, googleai.WithAPIKey(apiKey))
  if err != nil {
    log.Fatal(err)
  }

  imgData1, err := os.ReadFile(filepath.Join(imagesPath, "turtle1.png"))
  if err != nil {
    log.Fatal(err)
  }

  imgData2, err := os.ReadFile(filepath.Join(imagesPath, "turtle2.png"))
  if err != nil {
    log.Fatal(err)
  }

  parts := []llms.ContentPart{
    llms.BinaryPart("image/png", imgData1),
    llms.BinaryPart("image/png", imgData2),
    llms.TextPart("Describe the difference between these two pictures, with scientific detail"),
  }

  content := []llms.MessageContent{
    {
      Role:  schema.ChatMessageTypeHuman,
      Parts: parts,
    },
  }

  resp, err := llm.GenerateContent(ctx, content, llms.WithModel("gemini-pro-vision"))
  if err != nil {
    log.Fatal(err)
  }

  bs, _ := json.MarshalIndent(resp, "", "    ")
  fmt.Println(string(bs))
}

Note that we pass the specific model we want to utilize here using the WithModel option to llm.GenerateContent.

Finally, an example of using an embedding model to calculate embeddings for text:

func main() {
  ctx := context.Background()
  apiKey := os.Getenv("API_KEY")
  llm, err := googleai.New(ctx, googleai.WithAPIKey(apiKey))
  if err != nil {
    log.Fatal(err)
  }

  texts := []string{"lion", "parrot"}
  emb, err := llm.CreateEmbedding(ctx, texts)
  if err != nil {
    log.Fatal(err)
  }

  fmt.Println("Num of embedding vectors:", len(emb))
  for i, e := range emb {
    fmt.Printf("%d: %v...\n", i, e[:10])
  }
}

Switching to Vertex

Switching to the Vertex provider is very easy, since it implements exactly the same LangChainGo interfaces. We have to change the import line from:

import "github.com/tmc/langchaingo/llms/googleai"

To:

import "github.com/tmc/langchaingo/llms/googleai/vertex"

And then replace our llm value creation with:

ctx := context.Background()
project := os.Getenv("VERTEX_PROJECT")
location := os.Getenv("VERTEX_LOCATION")
llm, err := vertex.New(ctx, vertex.WithCloudProject(project), vertex.WithCloudLocation(location))
if err != nil {
  log.Fatal(err)
}

The rest of the code remains the same!

Code

The full, runnable code for these samples (both for Google AI and Vertex) is available on GitHub.