<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0"><channel><title><![CDATA[Pradip Parmar]]></title><description><![CDATA[Pradip Parmar]]></description><link>https://pradipgolang.hashnode.dev</link><generator>RSS for Node</generator><lastBuildDate>Sat, 31 Oct 2026 09:05:54 GMT</lastBuildDate><atom:link href="https://pradipgolang.hashnode.dev/rss.xml" rel="self" type="application/rss+xml"/><language><![CDATA[en]]></language><ttl>60</ttl><item><title><![CDATA[Writing an MCP Server in Go: Exposing Tools to LLM Agents]]></title><description><![CDATA[Build a Model Context Protocol server in Go with the official SDK: typed tools, validation, read vs write tools, stdio and HTTP, Cursor setup.]]></description><link>https://pradipgolang.hashnode.dev/writing-an-mcp-server-in-go-exposing-tools-to-llm-agents</link><guid isPermaLink="true">https://pradipgolang.hashnode.dev/writing-an-mcp-server-in-go-exposing-tools-to-llm-agents</guid><category><![CDATA[Go Language]]></category><category><![CDATA[mcp]]></category><category><![CDATA[AI]]></category><category><![CDATA[llm]]></category><category><![CDATA[ai agents]]></category><dc:creator><![CDATA[Pradip Parmar]]></dc:creator><pubDate>Thu, 29 Oct 2026 07:43:36 GMT</pubDate><content:encoded><![CDATA[<p>The Model Context Protocol (MCP) gives AI clients such as Cursor a standard way to discover and call tools that live in your systems. Instead of writing a custom plugin for every assistant, you write one MCP server, and any MCP-capable client can use it.</p>
<p>I built an MCP server for <strong>sprint-management</strong>, a Jira-style project management product I built with a Go API, React and PostgreSQL. The server lets Cursor read and update work items directly from the editor. This post covers how to build that kind of server in Go, and the design choices that matter more than the protocol itself. The code is a simplified example, not the product's source.</p>
<h2>What an MCP server actually is</h2>
<p>At its core, an MCP server is a process that:</p>
<ol>
<li>Tells the client which <strong>tools</strong> it offers (name, description, JSON Schema for input and output).</li>
<li>Receives <strong>tool calls</strong> with JSON arguments.</li>
<li>Returns results the model can read.</li>
</ol>
<p>It talks JSON-RPC over a transport: usually <strong>stdio</strong> (the client launches your binary) or <strong>streamable HTTP</strong> (your server runs as a service). Go is a good fit: a single static binary is easy to point a client at, and struct tags give you schemas almost for free.</p>
<h2>Setup</h2>
<p>Use the official Go SDK:</p>
<pre><code class="language-bash">go mod init example.com/sprint-mcp
go get github.com/modelcontextprotocol/go-sdk/mcp
</code></pre>
<h2>A typed read tool</h2>
<p>With the official SDK, a tool is a normal Go function with typed input and output. The SDK infers the JSON Schema from your structs, and the <code>jsonschema</code> tag becomes the field description the model sees.</p>
<pre><code class="language-go">type ListIssuesInput struct {
    ProjectKey string `json:"project_key" jsonschema:"project key, for example CORE"`
    Status     string `json:"status,omitempty" jsonschema:"optional filter: todo, in_progress or done"`
    Limit      int    `json:"limit,omitempty" jsonschema:"max issues to return, 1-50, default 20"`
}

type Issue struct {
    Key      string `json:"key"`
    Title    string `json:"title"`
    Status   string `json:"status"`
    Assignee string `json:"assignee,omitempty"`
}

type ListIssuesOutput struct {
    Issues []Issue `json:"issues"`
}

func (s *Server) listIssues(ctx context.Context, req *mcp.CallToolRequest, in ListIssuesInput) (
    *mcp.CallToolResult, ListIssuesOutput, error,
) {
    if in.Limit &lt;= 0 || in.Limit &gt; 50 {
        in.Limit = 20
    }
    if in.Status != "" &amp;&amp; !validStatus[in.Status] {
        return nil, ListIssuesOutput{}, fmt.Errorf("unknown status %q; use todo, in_progress or done", in.Status)
    }
    issues, err := s.store.ListIssues(ctx, s.user, in.ProjectKey, in.Status, in.Limit)
    if err != nil {
        return nil, ListIssuesOutput{}, err
    }
    return nil, ListIssuesOutput{Issues: issues}, nil
}
</code></pre>
<p>Notice the error message. It's written for the model, not for a log file. When the model passes a bad status, a clear message telling it the valid values lets it correct itself on the next call. With the official SDK, an error returned from a tool handler is sent back to the client as a tool result with <code>IsError</code> set, so the model sees it and can retry. The SDK also validates arguments against the generated schema before your handler runs, but schema checks can't know your business rules, so validate those yourself.</p>
<h2>A write tool, with guardrails</h2>
<p>Read tools are low risk. Write tools are where an agent can do real damage, so they get stricter handling:</p>
<pre><code class="language-go">type MoveIssueInput struct {
    IssueKey string `json:"issue_key" jsonschema:"issue key, for example CORE-123"`
    ToStatus string `json:"to_status" jsonschema:"target status: todo, in_progress or done"`
}

type MoveIssueOutput struct {
    IssueKey   string `json:"issue_key"`
    FromStatus string `json:"from_status"`
    ToStatus   string `json:"to_status"`
}

func (s *Server) moveIssue(ctx context.Context, req *mcp.CallToolRequest, in MoveIssueInput) (
    *mcp.CallToolResult, MoveIssueOutput, error,
) {
    if !issueKeyRE.MatchString(in.IssueKey) {
        return nil, MoveIssueOutput{}, fmt.Errorf("issue_key %q is not in the form ABC-123", in.IssueKey)
    }
    if !validStatus[in.ToStatus] {
        return nil, MoveIssueOutput{}, fmt.Errorf("unknown status %q", in.ToStatus)
    }
    // Same permission check the REST API uses. The MCP server is just another client.
    if !s.authz.Can(s.user, "issue:update", in.IssueKey) {
        return nil, MoveIssueOutput{}, fmt.Errorf("user %s may not update %s", s.user.Email, in.IssueKey)
    }
    from, err := s.store.MoveIssue(ctx, in.IssueKey, in.ToStatus)
    if err != nil {
        return nil, MoveIssueOutput{}, err
    }
    s.audit.Log(ctx, s.user, "mcp.move_issue", in.IssueKey, from, in.ToStatus)
    return nil, MoveIssueOutput{IssueKey: in.IssueKey, FromStatus: from, ToStatus: in.ToStatus}, nil
}
</code></pre>
<p>Four rules I apply to every write tool:</p>
<ul>
<li><strong>Narrow, typed operations.</strong> <code>move_issue</code> with an enum, not <code>run_sql</code> or <code>update_issue(fields map[string]any)</code>. The smaller the surface, the fewer ways to be wrong.</li>
<li><strong>Reuse your real authorization.</strong> The MCP server should call the same domain service and RBAC checks as your REST API. Never give it a privileged database connection that bypasses them.</li>
<li><strong>Return what changed.</strong> Including <code>from_status</code> lets the model (and the human) confirm the effect.</li>
<li><strong>Audit everything.</strong> Agent-initiated writes should be traceable separately from human ones.</li>
</ul>
<h2>Wiring up the server</h2>
<pre><code class="language-go">func main() {
    logger := slog.New(slog.NewTextHandler(os.Stderr, nil)) // never stdout with stdio transport

    srv, err := NewServer(os.Getenv("SPRINT_API_TOKEN"), logger)
    if err != nil {
        logger.Error("startup", "err", err)
        os.Exit(1)
    }

    server := mcp.NewServer(&amp;mcp.Implementation{Name: "sprint-management", Version: "v0.1.0"}, nil)

    mcp.AddTool(server, &amp;mcp.Tool{
        Name:        "list_issues",
        Description: "List issues in a project, optionally filtered by status. Read-only.",
    }, srv.listIssues)

    mcp.AddTool(server, &amp;mcp.Tool{
        Name:        "move_issue",
        Description: "Move one issue to a new status. Changes data. Use list_issues first to find the key.",
    }, srv.moveIssue)

    if err := server.Run(context.Background(), &amp;mcp.StdioTransport{}); err != nil {
        logger.Error("server stopped", "err", err)
        os.Exit(1)
    }
}
</code></pre>
<p><strong>The stdio gotcha:</strong> with the stdio transport, stdout <em>is</em> the protocol channel. A stray <code>fmt.Println</code> or a logger writing to stdout corrupts the JSON-RPC stream, and the client fails in confusing ways. Send all logs to stderr.</p>
<p>Descriptions matter more than they look. The model chooses tools from their names and descriptions alone. Say what the tool does, whether it changes data, and which tool to call first.</p>
<h2>Connecting it to Cursor</h2>
<p>Build the binary and register it in your project's <code>.cursor/mcp.json</code>:</p>
<pre><code class="language-json">{
  "mcpServers": {
    "sprint-management": {
      "command": "/usr/local/bin/sprint-mcp",
      "env": { "SPRINT_API_TOKEN": "your-token" }
    }
  }
}
</code></pre>
<p>Cursor launches the process, lists its tools, and the agent can now call <code>list_issues</code> and <code>move_issue</code> while you work. The token identifies the user, so the agent acts with that user's permissions, nothing more.</p>
<h2>When to use HTTP instead of stdio</h2>
<p>Stdio is the simplest choice for a developer tool on one machine. If several users or remote agents need the same server, run it as a service with the SDK's streamable HTTP handler:</p>
<pre><code class="language-go">handler := mcp.NewStreamableHTTPHandler(func(r *http.Request) *mcp.Server {
    return server
}, nil)
http.Handle("/mcp", authMiddleware(handler))
log.Fatal(http.ListenAndServe(":8080", nil))
</code></pre>
<p>Over HTTP, authentication is your job. Put it in middleware, derive the user from the request, and apply the same per-user permission checks inside each tool.</p>
<h2>Testing</h2>
<p>Tool handlers are plain Go functions, so most tests don't need the protocol at all: call <code>srv.moveIssue</code> directly with a fake store and assert on validation, authorization and audit behaviour. For one or two end-to-end tests, the SDK includes a client (<code>mcp.NewClient</code>) and an in-memory transport, so you can connect a real client to your server in-process and call tools by name.</p>
<h2>Design checklist</h2>
<ul>
<li>One tool per clear intent; avoid "do anything" tools.</li>
<li>Typed inputs with descriptions and enums; validate anyway.</li>
<li>Error messages that tell the model how to fix the call.</li>
<li>Mark write tools clearly and keep them narrow.</li>
<li>Same auth, RBAC and audit as your REST API.</li>
<li>Logs to stderr on stdio.</li>
</ul>
<p>MCP itself is a thin layer. The value of a good MCP server comes from the same things that make a good API: small typed operations, strong permissions and honest errors.</p>
<hr />
<p><em>Pradip Parmar is a Go backend and AI agent engineer (writing Go since 2018) who builds MCP servers, RAG systems and LLM agents for businesses, including an MCP server that connects a Jira-style product to Cursor. If you're looking for an MCP server developer in Go, contact him on <a href="https://www.linkedin.com/in/pradip-golang/">LinkedIn</a>, <a href="https://www.upwork.com/freelancers/~014c0e14f455a1a95d">Upwork</a> or <a href="https://contra.com/pradip_parmar_6oq55fq2">Contra</a>. Code: <a href="https://github.com/princeparmar">GitHub</a>.</em></p>
]]></content:encoded></item><item><title><![CDATA[LLM Tool Calling in Go: A ReAct-Style Agent Loop with Guardrails and Cost Control]]></title><description><![CDATA[Build a provider-agnostic ReAct agent loop in Go: typed tool registry, step and token budgets, per-tool timeouts, loop detection, write approvals.]]></description><link>https://pradipgolang.hashnode.dev/llm-tool-calling-in-go-a-react-style-agent-loop-with-guardrails-and-cost-control</link><guid isPermaLink="true">https://pradipgolang.hashnode.dev/llm-tool-calling-in-go-a-react-style-agent-loop-with-guardrails-and-cost-control</guid><category><![CDATA[Go Language]]></category><category><![CDATA[AI]]></category><category><![CDATA[llm]]></category><category><![CDATA[ai agents]]></category><category><![CDATA[tool calling]]></category><dc:creator><![CDATA[Pradip Parmar]]></dc:creator><pubDate>Thu, 29 Oct 2026 06:32:15 GMT</pubDate><content:encoded><![CDATA[<p>Most of my agent projects in Go share one piece of code: the loop. The model looks at the conversation, decides to call a tool or answer, the program runs the tool, feeds the result back, and repeats. That's the ReAct pattern (reason, act, observe) in its practical form.</p>
<p>I've built this loop several times: in <strong>agent-helper</strong>, a shared Go library that runs the same chat loop on Gemini or local Ollama models with an installable tool registry; in a Gemini tool-calling assistant for a Jira-style product; in a Clean Architecture ReAct agent that drafts social posts; and in a local voice assistant that uses Ollama tool calling. The happy path is about 40 lines. Most of the engineering is in the guardrails. This post builds a simplified version of that loop with the guardrails included.</p>
<h2>Step 1: a provider-agnostic interface</h2>
<p>Don't let the agent loop import a specific SDK. Define the minimum it needs:</p>
<pre><code class="language-go">type Role string

const (
    RoleUser      Role = "user"
    RoleAssistant Role = "assistant"
    RoleTool      Role = "tool"
)

type ToolCall struct {
    ID   string
    Name string
    Args json.RawMessage
}

type Message struct {
    Role       Role
    Content    string
    ToolCalls  []ToolCall // set when the model asks for tools
    ToolCallID string     // set on tool results
}

type Usage struct{ InputTokens, OutputTokens int }

type ChatModel interface {
    Chat(ctx context.Context, msgs []Message, tools []ToolSpec) (Message, Usage, error)
}
</code></pre>
<p>Each provider (Gemini, OpenAI-compatible APIs, Ollama) gets a small adapter that converts to and from these types. Switching between a hosted model and a local one then becomes a config change, which is exactly how agent-helper moves the same agent between Gemini and Ollama.</p>
<h2>Step 2: a typed tool registry</h2>
<p>Tools are plug-ins. The loop shouldn't change when you add one.</p>
<pre><code class="language-go">type ToolSpec struct {
    Name        string
    Description string
    Schema      json.RawMessage // JSON Schema for the arguments
}

type Tool struct {
    Spec    ToolSpec
    Mutates bool          // changes external state?
    Timeout time.Duration // per-call limit
    Run     func(ctx context.Context, args json.RawMessage) (string, error)
}

type Registry struct{ tools map[string]Tool }

func (r *Registry) Register(t Tool) {
    if r.tools == nil {
        r.tools = map[string]Tool{}
    }
    r.tools[t.Spec.Name] = t
}

func (r *Registry) Specs() []ToolSpec {
    specs := make([]ToolSpec, 0, len(r.tools))
    for _, t := range r.tools {
        specs = append(specs, t.Spec)
    }
    sort.Slice(specs, func(i, j int) bool { return specs[i].Name &lt; specs[j].Name })
    return specs
}
</code></pre>
<p>Sorting the specs keeps the prompt stable between calls, which helps reproducibility and, with providers that cache prompt prefixes, cost.</p>
<p>Inside <code>Run</code>, decode arguments strictly into a struct and validate them. The model's JSON is untrusted input:</p>
<pre><code class="language-go">func decodeArgs[T any](raw json.RawMessage) (T, error) {
    var v T
    dec := json.NewDecoder(bytes.NewReader(raw))
    dec.DisallowUnknownFields()
    if err := dec.Decode(&amp;v); err != nil {
        return v, fmt.Errorf("invalid arguments: %w", err)
    }
    return v, nil
}
</code></pre>
<h2>Step 3: the loop, with budgets</h2>
<p>Here's the core. Every guardrail is explicit:</p>
<pre><code class="language-go">type Limits struct {
    MaxSteps       int           // model round-trips
    MaxTotalTokens int           // input + output across the run
    MaxRepeats     int           // identical tool calls allowed
    RunTimeout     time.Duration // wall clock for the whole run
}

type Approver func(ctx context.Context, call ToolCall) (bool, error)

type Agent struct {
    Model   ChatModel
    Tools   *Registry
    Limits  Limits
    Approve Approver // nil means write tools are refused
}

var ErrBudget = errors.New("agent budget exhausted")

func (a *Agent) Run(ctx context.Context, msgs []Message) (string, error) {
    ctx, cancel := context.WithTimeout(ctx, a.Limits.RunTimeout)
    defer cancel()

    used := 0
    seen := map[string]int{}

    for step := 0; step &lt; a.Limits.MaxSteps; step++ {
        reply, usage, err := a.Model.Chat(ctx, msgs, a.Tools.Specs())
        if err != nil {
            return "", fmt.Errorf("step %d: %w", step, err)
        }
        used += usage.InputTokens + usage.OutputTokens
        msgs = append(msgs, reply)

        if len(reply.ToolCalls) == 0 {
            return reply.Content, nil // final answer
        }
        if used &gt; a.Limits.MaxTotalTokens {
            return "", fmt.Errorf("%w: %d tokens", ErrBudget, used)
        }

        for _, call := range reply.ToolCalls {
            key := call.Name + string(call.Args)
            seen[key]++
            var result string
            if seen[key] &gt; a.Limits.MaxRepeats {
                result = "error: you already made this exact call; use the earlier result or try a different approach"
            } else {
                result = a.execute(ctx, call)
            }
            msgs = append(msgs, Message{Role: RoleTool, ToolCallID: call.ID, Content: result})
        }
    }
    return "", fmt.Errorf("%w: max steps (%d)", ErrBudget, a.Limits.MaxSteps)
}
</code></pre>
<p>What each guardrail prevents:</p>
<ul>
<li><strong>MaxSteps</strong> stops a model that never decides it's done.</li>
<li><strong>MaxTotalTokens</strong> caps cost per run. Every step resends the whole conversation, so input tokens grow with each round-trip. A ten-step run costs much more than ten times a single call.</li>
<li><strong>MaxRepeats</strong> catches the common failure where a model calls the same tool with the same arguments again and again.</li>
<li><strong>RunTimeout</strong> bounds wall-clock time, and because it's a <code>context</code>, it cancels in-flight HTTP calls and tools too.</li>
</ul>
<h2>Step 4: executing tools safely</h2>
<pre><code class="language-go">func (a *Agent) execute(ctx context.Context, call ToolCall) string {
    tool, ok := a.Tools.tools[call.Name]
    if !ok {
        return fmt.Sprintf("error: unknown tool %q", call.Name)
    }
    if tool.Mutates {
        if a.Approve == nil {
            return "error: this tool changes data and is not allowed in this session"
        }
        ok, err := a.Approve(ctx, call)
        if err != nil || !ok {
            return "error: the user did not approve this action"
        }
    }

    tctx, cancel := context.WithTimeout(ctx, tool.Timeout)
    defer cancel()

    out, err := tool.Run(tctx, call.Args)
    if err != nil {
        return "error: " + err.Error()
    }
    const maxResult = 8_000
    if len(out) &gt; maxResult {
        out = out[:maxResult] + "\n[truncated]"
    }
    return out
}
</code></pre>
<p>Three details matter here:</p>
<ol>
<li><strong>Errors go back to the model as text</strong>, not up the stack. A model that sees "invalid arguments: unknown field <code>prio</code>" usually fixes the call on the next step. Only infrastructure failures (the model API itself failing) should end the run.</li>
<li><strong>Write tools need approval.</strong> The <code>Approver</code> can be a CLI prompt, a Telegram button or a web confirmation. Read tools run freely; anything that sends, deletes or updates waits for a human or a policy check.</li>
<li><strong>Truncate tool output.</strong> One tool returning a 2 MB web page will blow your token budget and dilute the model's attention. Cap results, or summarise them before they go back.</li>
</ol>
<h2>Cost control beyond the loop</h2>
<p>The loop's budgets are the safety net. These keep costs low in normal operation:</p>
<ul>
<li><strong>Expose fewer tools.</strong> Every tool schema is sent on every step. Register only the tools the current task needs.</li>
<li><strong>Keep tool results compact.</strong> Return the fields the model needs, not whole database rows.</li>
<li><strong>Use smaller or local models where possible.</strong> With a provider-agnostic interface, routine steps can run on a local Ollama model and harder ones on a hosted model.</li>
<li><strong>Log usage per run.</strong> Record steps, tokens and tool calls for each run. You can't control a cost you don't measure.</li>
</ul>
<h2>Testing the loop</h2>
<p>Because <code>ChatModel</code> is an interface, a scripted fake makes the loop fully testable without network calls:</p>
<pre><code class="language-go">type scriptedModel struct{ replies []Message; i int }

func (m *scriptedModel) Chat(ctx context.Context, _ []Message, _ []ToolSpec) (Message, Usage, error) {
    r := m.replies[m.i]
    m.i++
    return r, Usage{InputTokens: 100, OutputTokens: 20}, nil
}
</code></pre>
<p>Write tests for the boring cases: unknown tool, invalid arguments, repeated call, budget exceeded, write tool without approval. Those are the paths that protect you in production.</p>
<h2>Takeaways</h2>
<p>The model decides what to try. Your Go code decides what is allowed, how long it may take and how much it may cost. Keep the loop small, keep the tools typed, and make every limit explicit.</p>
<hr />
<p><em>Pradip Parmar is a Go backend and AI agent engineer who has worked in Go since 2018. He builds LLM agents with tool calling, MCP servers and RAG systems for businesses, from hosted Gemini and OpenAI-compatible models to fully local Ollama setups. To hire a Go AI agent developer, find him on <a href="https://www.linkedin.com/in/pradip-golang/">LinkedIn</a>, <a href="https://www.upwork.com/freelancers/~014c0e14f455a1a95d">Upwork</a> or <a href="https://contra.com/pradip_parmar_6oq55fq2">Contra</a>, and on <a href="https://github.com/princeparmar">GitHub</a>.</em></p>
]]></content:encoded></item><item><title><![CDATA[Building a Production RAG API in Go with pgvector and Gemini]]></title><description><![CDATA[Build a production RAG API in Go with PostgreSQL, pgvector and Gemini: hard safety filters before the LLM, deterministic ranking, vector fallback.]]></description><link>https://pradipgolang.hashnode.dev/building-a-production-rag-api-in-go-with-pgvector-and-gemini</link><guid isPermaLink="true">https://pradipgolang.hashnode.dev/building-a-production-rag-api-in-go-with-pgvector-and-gemini</guid><category><![CDATA[Go Language]]></category><category><![CDATA[RAG ]]></category><category><![CDATA[AI]]></category><category><![CDATA[PostgreSQL]]></category><category><![CDATA[pgvector]]></category><dc:creator><![CDATA[Pradip Parmar]]></dc:creator><pubDate>Thu, 29 Oct 2026 05:54:24 GMT</pubDate><content:encoded><![CDATA[<p><em>Lessons from building the Restaurant Suggestion System, a Go/Gin backend with a conversational recommendation API.</em></p>
<p>A RAG demo takes an afternoon: embed some documents, run a similarity search, paste the results into a prompt. A RAG API you can put in front of real users takes longer, because most of the work is not the LLM call. It's everything around it.</p>
<p>I built the Restaurant Suggestion System (RSS) as a Go/Gin backend where users chat to get dish and restaurant suggestions. The answers come from RAG over real menu data in PostgreSQL with pgvector and Gemini embeddings. Because the recommendations have to respect allergies, "the model usually gets it right" was never good enough.</p>
<p>This post walks through the shape of that pipeline and the Go patterns that made it dependable. The snippets are simplified and focused on the ideas, not copied from the private repo.</p>
<h2>The pipeline</h2>
<p>Every chat request goes through the same steps:</p>
<ol>
<li><strong>Embed</strong> the user's message (plus stored preferences) with Gemini.</li>
<li><strong>Retrieve</strong> candidates from Postgres with pgvector, with <strong>allergen and health filters applied in SQL</strong>.</li>
<li><strong>Score</strong> candidates deterministically in Go.</li>
<li><strong>Generate</strong> a conversational answer with the LLM, using only the top candidates as context.</li>
<li><strong>Validate</strong> the answer against the candidates we sent.</li>
<li><strong>Fall back</strong> to a vector-only response if the LLM is slow or unavailable.</li>
</ol>
<p>The rule behind all of it: the LLM explains and converses. It does not decide what is safe or what exists.</p>
<h2>Schema: keep filters as real columns</h2>
<p>It's tempting to put everything into the embedded text and let similarity sort it out. Don't do that for anything safety-related. Allergens belong in a column you can filter on exactly.</p>
<pre><code class="language-sql">CREATE EXTENSION IF NOT EXISTS vector;

CREATE TABLE menu_items (
    id            BIGSERIAL PRIMARY KEY,
    restaurant_id BIGINT NOT NULL REFERENCES restaurants(id),
    name          TEXT NOT NULL,
    description   TEXT NOT NULL,
    allergens     TEXT[] NOT NULL DEFAULT '{}',
    price_cents   INT NOT NULL,
    is_available  BOOLEAN NOT NULL DEFAULT TRUE,
    embedding     vector(768) NOT NULL
);

CREATE INDEX ON menu_items USING hnsw (embedding vector_cosine_ops);
CREATE INDEX ON menu_items USING gin (allergens);
</code></pre>
<p>The dimension (768 here) must match the embedding model's output configuration. Pick one, store it in config, and fail fast at startup if they disagree. A silent dimension mismatch is a miserable bug to find in production.</p>
<h2>Embeddings with Gemini</h2>
<p>RSS uses Gemini embeddings in two places: when indexing menu items, and during onboarding to capture user preferences as a vector. Wrap the provider behind a small interface so the rest of the code doesn't care which SDK you use:</p>
<pre><code class="language-go">type Embedder interface {
    Embed(ctx context.Context, text string) ([]float32, error)
}

type GeminiEmbedder struct {
    client *genai.Client
    model  string
    dims   int32
}

func (g *GeminiEmbedder) Embed(ctx context.Context, text string) ([]float32, error) {
    ctx, cancel := context.WithTimeout(ctx, 5*time.Second)
    defer cancel()

    res, err := g.client.Models.EmbedContent(ctx, g.model,
        genai.Text(text),
        &amp;genai.EmbedContentConfig{OutputDimensionality: &amp;g.dims},
    )
    if err != nil {
        return nil, fmt.Errorf("embed: %w", err)
    }
    if len(res.Embeddings) == 0 || len(res.Embeddings[0].Values) != int(g.dims) {
        return nil, errors.New("embed: unexpected embedding shape")
    }
    return res.Embeddings[0].Values, nil
}
</code></pre>
<p>Two habits worth copying: a hard timeout on every external call, and a shape check on the response. The interface also makes tests easy, since a fake embedder returns fixed vectors.</p>
<h2>Retrieval: filter in SQL, before the LLM</h2>
<p>This is the most important design decision in the system. Allergen and health filters run <strong>in the retrieval query</strong>, so excluded items never reach the model. A prompt like "don't suggest peanuts" is a request. A <code>WHERE</code> clause is a guarantee.</p>
<pre><code class="language-go">type Candidate struct {
    ID         int64
    Name       string
    Allergens  []string
    PriceCents int
    Distance   float64
}

const retrieveSQL = `
SELECT id, name, allergens, price_cents, embedding &lt;=&gt; $1 AS distance
FROM menu_items
WHERE is_available
  AND NOT (allergens &amp;&amp; $2::text[])
ORDER BY embedding &lt;=&gt; $1
LIMIT $3`

func (r *Repo) Retrieve(ctx context.Context, q []float32, exclude []string, k int) ([]Candidate, error) {
    rows, err := r.db.Query(ctx, retrieveSQL, pgvector.NewVector(q), exclude, k)
    if err != nil {
        return nil, err
    }
    defer rows.Close()

    var out []Candidate
    for rows.Next() {
        var c Candidate
        if err := rows.Scan(&amp;c.ID, &amp;c.Name, &amp;c.Allergens, &amp;c.PriceCents, &amp;c.Distance); err != nil {
            return nil, err
        }
        out = append(out, c)
    }
    return out, rows.Err()
}
</code></pre>
<p>One pgvector gotcha: with an approximate index such as HNSW, filtering happens <em>after</em> the index returns its nearest neighbours. With a strict filter you can get fewer than <code>k</code> rows back. pgvector 0.8+ supports iterative index scans (<code>SET hnsw.iterative_scan = relaxed_order;</code>), which keep scanning until enough rows pass the filter. On older versions, raise <code>hnsw.ef_search</code> or over-fetch and trim. Either way, write a test where most rows are filtered out and check you still get results.</p>
<h2>Deterministic scoring</h2>
<p>Vector distance is one signal, not the ranking. RSS ranks filtered candidates with a deterministic scoring engine, so the same input gives the same order and every recommendation can be explained.</p>
<pre><code class="language-go">type Weights struct {
    Similarity, Preference, Budget float64
}

func Score(c Candidate, prefSim float64, budgetCents int, w Weights) float64 {
    s := w.Similarity*(1-c.Distance) + w.Preference*prefSim
    if budgetCents &gt; 0 &amp;&amp; c.PriceCents &lt;= budgetCents {
        s += w.Budget
    }
    return s
}

func Rank(cands []Candidate, scores map[int64]float64) {
    sort.SliceStable(cands, func(i, j int) bool {
        si, sj := scores[cands[i].ID], scores[cands[j].ID]
        if si != sj {
            return si &gt; sj
        }
        return cands[i].ID &lt; cands[j].ID // stable tie-break
    })
}
</code></pre>
<p>Note the tie-break on ID. Without it, equal scores can shuffle between runs, and your tests become flaky. The signals here are illustrative; the point is that ranking lives in plain Go you can unit-test, not in a prompt.</p>
<h2>Generation: ground it, then check it</h2>
<p>Only the top candidates go into the prompt, each with its ID. A pattern I recommend on top of that: ask the model for structured output that references those IDs, and validate the response before returning it:</p>
<pre><code class="language-go">type LLMAnswer struct {
    Message string  `json:"message"`
    ItemIDs []int64 `json:"item_ids"`
}

func validateAnswer(a LLMAnswer, sent []Candidate) error {
    allowed := make(map[int64]bool, len(sent))
    for _, c := range sent {
        allowed[c.ID] = true
    }
    for _, id := range a.ItemIDs {
        if !allowed[id] {
            return fmt.Errorf("model referenced unknown item %d", id)
        }
    }
    return nil
}
</code></pre>
<p>If the model mentions something you didn't give it, that's a hallucination by definition, and you can detect it in a few lines.</p>
<h2>The fallback: vector-only results</h2>
<p>LLM APIs time out, hit rate limits and have outages. RSS handles that by degrading gracefully: if the LLM call fails, the API still returns the filtered, scored candidates from vector search. In the sketch below, an answer that fails validation takes the same path:</p>
<pre><code class="language-go">func (s *Service) Recommend(ctx context.Context, req ChatRequest) (ChatResponse, error) {
    cands, err := s.retrieveAndRank(ctx, req)
    if err != nil {
        return ChatResponse{}, err // no data, no answer
    }

    genCtx, cancel := context.WithTimeout(ctx, 8*time.Second)
    defer cancel()

    ans, err := s.llm.Answer(genCtx, req.Message, cands)
    if err == nil {
        err = validateAnswer(ans, cands)
    }
    if err != nil {
        s.log.Warn("llm unavailable, using vector-only fallback", "err", err)
        return fallbackResponse(cands), nil
    }
    return ChatResponse{Message: ans.Message, ItemIDs: ans.ItemIDs, Source: "llm"}, nil
}
</code></pre>
<p>Because the safety filters ran before the LLM, the fallback is just as safe as the full path. It's less chatty, not less correct. Returning a <code>Source</code> field also helps the frontend and your dashboards see how often you're degrading.</p>
<h2>The rest of the production checklist</h2>
<p>The RAG pipeline is maybe a third of the system. The rest is ordinary backend work that still matters:</p>
<ul>
<li><strong>Auth:</strong> passwordless email OTP, JWT, and role-based access control so admin endpoints stay admin-only.</li>
<li><strong>Data ingest:</strong> CRUD plus CSV/JSON bulk upload and AWS file upload. Whatever your ingest path, make sure every write also refreshes the item's embedding, or search goes stale.</li>
<li><strong>Privacy:</strong> admin analytics built without exposing PII.</li>
<li><strong>Ops:</strong> Postgres with pgvector in Docker, so local, CI and production run the same database.</li>
<li><strong>Tests:</strong> a Newman (Postman) end-to-end suite that hits the real API. Make sure the fallback path has its own tests too.</li>
</ul>
<h2>Takeaways</h2>
<ul>
<li>Put safety rules in SQL and Go, never only in the prompt.</li>
<li>Rank deterministically, with stable tie-breaks.</li>
<li>Validate model output against the context you sent.</li>
<li>Design the fallback first. If the LLM disappears, users should still get a correct answer.</li>
</ul>
<p>Go suits this well: strict types for model output, <code>context</code> deadlines on every call, and a single binary that's easy to run next to Postgres.</p>
<hr />
<p><em>Pradip Parmar is a Go backend and AI agent engineer who has been writing Go since 2018. He builds RAG systems, MCP servers and LLM agents for businesses, and previously led the Python-to-Go migration of the Flights backend at MakeMyTrip. If you need a Go AI agent developer for a RAG or agent project, you can reach him on <a href="https://www.linkedin.com/in/pradip-golang/">LinkedIn</a>, <a href="https://www.upwork.com/freelancers/~014c0e14f455a1a95d">Upwork</a> or <a href="https://contra.com/pradip_parmar_6oq55fq2">Contra</a>, or see his code on <a href="https://github.com/princeparmar">GitHub</a>.</em></p>
]]></content:encoded></item></channel></rss>