Calling LangChain from Go (Part 1)

Motivation

Following my “holiday” tests (previous posts…) on using Golang and LLMs, I was looking for an easy way to implement LangChain calling in Go, and preferably using watsonx.ai.

Luckily I found the following Github repository: https://github.com/tmc/langchaingo (curtsy to Travis Cline https://github.com/tmc).

In his repository, there is this specific folder: https://github.com/tmc/langchaingo/blob/main/examples/watsonx-llm-example/watsonx_example.go which caught my attention!

So as usual I built a project and tried to implement it and also put my own ideas (à ma sauce 😄).

Implementation

As usual as there is a need on environment variables, I set up an .env file which is later used in the app.

export WATSONX_API_KEY="your-watsonx-api-key"
export WATSONX_PROJECT_ID="your-watsonx-projectid"
# I used the US-SOUTH, could be any other region of IBM Cloud
export SERVICE_URL="https://us-south.ml.cloud.ibm.com" 

In a previous post I mentioned trying to count the number of tokens sent to and received from a LLM. That work is still WIP, so I used directly the “tiktoken-go” library inside my app with an idea of making some changes to it (in a near future?). Anyways, in the case of my current state of progress it does not really work, but it is there.

For the app by itself, I used Travis’ code from his repository alomost as is, and added and wrapped it with the following features;

• using a dialog box for the prompt input (🙄 I love dialog-boxes 😂)
• “attempt” to count the number of “tokens” sent to and received back from the LLM.

The code by itself is the following;

package main

import (
 "context"
 "fmt"
 "log"
 "os"
 "os/exec"
 "runtime"

 "fyne.io/fyne/v2"
 "fyne.io/fyne/v2/app"
 "fyne.io/fyne/v2/container"
 "fyne.io/fyne/v2/dialog"
 "fyne.io/fyne/v2/widget"

 "github.com/joho/godotenv"
 "github.com/pkoukk/tiktoken-go"
 "github.com/tmc/langchaingo/llms"
 "github.com/tmc/langchaingo/llms/watsonx"
)

const (
 _tokenApproximation = 4
)

const (
 _gpt35TurboContextSize   = 4096
 _gpt432KContextSize      = 32768
 _gpt4ContextSize         = 8192
 _textDavinci3ContextSize = 4097
 _textBabbage1ContextSize = 2048
 _textAda1ContextSize     = 2048
 _textCurie1ContextSize   = 2048
 _codeDavinci2ContextSize = 8000
 _codeCushman1ContextSize = 2048
 _textBisonContextSize    = 2048
 _chatBisonContextSize    = 2048
 _defaultContextSize      = 2048
)

// nolint:gochecknoglobals
var modelToContextSize = map[string]int{
 "gpt-3.5-turbo":    _gpt35TurboContextSize,
 "gpt-4-32k":        _gpt432KContextSize,
 "gpt-4":            _gpt4ContextSize,
 "text-davinci-003": _textDavinci3ContextSize,
 "text-curie-001":   _textCurie1ContextSize,
 "text-babbage-001": _textBabbage1ContextSize,
 "text-ada-001":     _textAda1ContextSize,
 "code-davinci-002": _codeDavinci2ContextSize,
 "code-cushman-001": _codeCushman1ContextSize,
}

var tokens int

func runCmd(name string, arg ...string) {
 cmd := exec.Command(name, arg...)
 cmd.Stdout = os.Stdout
 cmd.Run()
}

func ClearTerminal() {
 switch runtime.GOOS {
 case "darwin":
  runCmd("clear")
 case "linux":
  runCmd("clear")
 case "windows":
  runCmd("cmd", "/c", "cls")
 default:
  runCmd("clear")
 }
}

func promptEntryDialog() string {

 var promptEntry string

 // Create a new Fyne application
 myApp := app.New()
 myWindow := myApp.NewWindow("Prompt Entry Dialog")

 // Variable to store user input
 var userInput string

 // Button to show the dialog
 button := widget.NewButton("Click to Enter your prompt's text", func() {
  entry := widget.NewEntry()
  dialog.ShowCustomConfirm("Input Dialog", "OK", "Cancel", entry, func(confirm bool) {
   if confirm {
    userInput = entry.Text
    promptEntry = userInput
    fmt.Println("User Input:", userInput) // Print to the console
    myWindow.Close()
   }
  }, myWindow)
 })

 // Add the button to the window
 myWindow.SetContent(container.NewVBox(
  widget.NewLabel("Click the button below to enter text:"),
  button,
 ))

 // Set the window size and run the application
 myWindow.Resize(fyne.NewSize(400, 200))
 myWindow.ShowAndRun()
 return promptEntry
}

func CountTokens(model, text string, inorout string) int {
 var txtLen int
 e, err := tiktoken.EncodingForModel(model)
 if err != nil {
  e, err = tiktoken.GetEncoding("gpt2")
  if err != nil {
   log.Printf("[WARN] Failed to calculate number of tokens for model, falling back to approximate count")
   txtLen = len([]rune(text))

   fmt.Println("Guessed tokens for the "+inorout+" text:", txtLen/_tokenApproximation)

   return txtLen
  }
 }
 return len(e.Encode(text, nil, nil))
}

func GetModelContextSize(model string) int {
 contextSize, ok := modelToContextSize[model]
 if !ok {
  return _defaultContextSize
 }
 return contextSize
}

func CalculateMaxTokens(model, text string) int {
 return GetModelContextSize(model) - CountTokens(model, text, text)
}

func main() {
 var prompt, model string

 // read the '.env' file
 err := godotenv.Load()
 if err != nil {
  log.Fatal("Error loading .env file")
 }

 ApiKey := os.Getenv("WATSONX_API_KEY")
 if ApiKey == "" {
  log.Fatal("WATSONX_API_KEY environment variable is not set")
 }
 ServiceURL := os.Getenv("SERVICE_URL")
 if ServiceURL == "" {
  log.Fatal("SERVICE_URL environment variable is not set")
 }
 ProjectID := os.Getenv("WATSONX_PROJECT_ID")
 if ProjectID == "" {
  log.Fatal("WATSONX_PROJECT_ID environment variable is not set")
 }

 // LLM from watsonx.ai
 model = "ibm/granite-13b-instruct-v2"
 // model = "meta-llama/llama-3-70b-instruct"

 llm, err := watsonx.New(
  model,
  //// Optional parameters: to be implemented if needed - Not used at this stage but all ready
  // wx.WithWatsonxAPIKey(ApiKey),
  // wx.WithWatsonxProjectID("YOUR WATSONX PROJECT ID"),
 )

 if err != nil {
  log.Fatal(err)
 }
 ctx := context.Background()

 prompt = promptEntryDialog()

 // for the output visibility on the consol - getting rid of system messages
 ClearTerminal()

 // Use the entry variable here
 fmt.Println("Calling the llm with the user's prompt:", prompt)

 tokens = CountTokens(model, prompt, "input")

 completion, err := llms.GenerateFromSinglePrompt(
  ctx,
  llm,
  prompt,
  llms.WithTopK(10),
  llms.WithTopP(0.95),
  llms.WithSeed(25),
 )
 // Check for errors
 if err != nil {
  log.Fatal(err)
 }
 fmt.Println(completion)

 tokens = CountTokens(model, completion, "output")

}

Which works fine as the output is shown below.

Calling the llm with the user's prompt: What is the distance in Kilmometers from Earth to Moon?
2024/12/31 11:08:04 [WARN] Failed to calculate number of tokens for model, falling back to approximate count
Guessed tokens for the input text: 13
The distance from Earth to the Moon is about 384,400 kilometers.
2024/12/31 11:08:04 [WARN] Failed to calculate number of tokens for model, falling back to approximate count
Guessed tokens for the output text: 16

#####


Calling the llm with the user's prompt: What is the name of the capital city of France?
2024/12/31 11:39:28 [WARN] Failed to calculate number of tokens for model, falling back to approximate count
Guessed tokens for the input text: 11
Paris
2024/12/31 11:39:28 [WARN] Failed to calculate number of tokens for model, falling back to approximate count
Guessed tokens for the output text: 1

Voilà!

Next steps

I would implement the following features for the version 0.2;

• Proposing the model the user wants to use,
• A more accurate way to determine the # of tokens,
• Some real LangChain implementation.

Conclusion

This is a very simple reflection of my work around calling LangChain from a Go application.

Stay tuned for more to come 💡