Step 23 of 24

Deploying Go services

Load config from the environment, shut down gracefully, expose health checks and ship a small, secure container image.

17 min 7 sections

A service that works on your laptop still has to survive production: configuration that differs per environment, deploys that restart it several times a day, orchestrators that decide whether it's healthy, and images that must be small and secure. Go makes this easier than most languages, because it compiles to a single static binary. This lesson covers the four things every Go service needs before it ships: configuration, graceful shutdown, health checks, and a container image.

Configuration from the environment

Configuration that changes between environments (database URLs, ports, log levels, secrets) belongs in environment variables, not in code or in the image. The same image then runs in staging and production, and secrets never end up in your Git history or container registry.

Load everything into one typed struct at startup, apply defaults, and fail fast with every problem listed at once:

go
type Config struct {
	Addr            string
	DatabaseURL     string
	LogLevel        slog.Level
	DrainDelay      time.Duration
	ShutdownTimeout time.Duration
}

func LoadConfig() (Config, error) {
	cfg := Config{
		Addr:        envOr("ADDR", ":8080"),
		DatabaseURL: os.Getenv("DATABASE_URL"),
	}
	var errs []error
	if cfg.DatabaseURL == "" {
		errs = append(errs, errors.New("DATABASE_URL is required"))
	}
	if err := cfg.LogLevel.UnmarshalText([]byte(envOr("LOG_LEVEL", "info"))); err != nil {
		errs = append(errs, fmt.Errorf("LOG_LEVEL: %w", err))
	}
	var err error
	if cfg.DrainDelay, err = time.ParseDuration(envOr("DRAIN_DELAY", "5s")); err != nil {
		errs = append(errs, fmt.Errorf("DRAIN_DELAY: %w", err))
	}
	if cfg.ShutdownTimeout, err = time.ParseDuration(envOr("SHUTDOWN_TIMEOUT", "20s")); err != nil {
		errs = append(errs, fmt.Errorf("SHUTDOWN_TIMEOUT: %w", err))
	}
	return cfg, errors.Join(errs...)
}

func envOr(key, def string) string {
	if v, ok := os.LookupEnv(key); ok {
		return v
	}
	return def
}
text
$ LOG_LEVEL=loud ./server
2026/10/11 08:26:08 ERROR fatal err="config: DATABASE_URL is required\nLOG_LEVEL: slog: level string \"loud\": unknown name"

A service that refuses to start is caught by the deploy and rolled back. A service that starts with a typo'd setting and fails on the first request is an incident. Read configuration only here, never with os.Getenv scattered through the code, and never log the struct whole: it contains secrets.

Graceful shutdown

Every deploy, scale-down or node drain ends the same way: the orchestrator sends your process SIGTERM, waits a grace period (30 seconds by default in Kubernetes), then sends SIGKILL. By default a Go program dies instantly on SIGTERM, cutting off every in-flight request. Users see errors on every deploy.

http.Server.Shutdown fixes this. It stops accepting new connections, closes idle ones, and waits for active requests to finish, or for its context to expire. Combine it with signal.NotifyContext, which returns a context that is cancelled when a signal arrives:

go
func main() {
	if err := run(context.Background()); err != nil {
		slog.Error("fatal", "err", err)
		os.Exit(1)
	}
}

func run(ctx context.Context) error {
	ctx, stop := signal.NotifyContext(ctx, os.Interrupt, syscall.SIGTERM)
	defer stop()

	cfg, err := LoadConfig()
	if err != nil {
		return fmt.Errorf("config: %w", err)
	}
	slog.SetDefault(slog.New(slog.NewJSONHandler(os.Stdout,
		&slog.HandlerOptions{Level: cfg.LogLevel})))

	pool, err := pgxpool.New(ctx, cfg.DatabaseURL)
	if err != nil {
		return fmt.Errorf("database: %w", err)
	}
	defer pool.Close() // runs after the server has shut down

	var draining atomic.Bool
	srv := &http.Server{
		Addr:              cfg.Addr,
		Handler:           routes(pool, &draining),
		ReadHeaderTimeout: 5 * time.Second,
		ReadTimeout:       10 * time.Second,
		WriteTimeout:      30 * time.Second,
		IdleTimeout:       120 * time.Second,
	}

	serveErr := make(chan error, 1)
	go func() {
		slog.Info("listening", "addr", cfg.Addr, "version", version)
		serveErr <- srv.ListenAndServe()
	}()

	select {
	case err := <-serveErr:
		return err // could not start, e.g. the port is in use
	case <-ctx.Done():
	}
	stop() // restore default signal handling: a second Ctrl+C exits immediately

	slog.Info("shutdown started; draining")
	draining.Store(true)       // readiness now fails...
	time.Sleep(cfg.DrainDelay) // ...give load balancers time to notice

	shutdownCtx, cancel := context.WithTimeout(context.Background(), cfg.ShutdownTimeout)
	defer cancel()
	if err := srv.Shutdown(shutdownCtx); err != nil {
		return fmt.Errorf("shutdown: %w", err)
	}
	slog.Info("shutdown complete")
	return nil
}

Some details matter here:

  • run returns an error instead of calling os.Exit deep inside. os.Exit skips deferred calls, so the pool would never close and buffered logs or traces would never flush. Only main exits.
  • The shutdown context is created from context.Background(), not from ctx, because ctx is already cancelled at that point. Deriving from it would make Shutdown give up immediately.
  • ListenAndServe always returns an error. After Shutdown it's http.ErrServerClosed, which nobody reads here because run has moved on; a real startup failure arrives through serveErr.
  • The drain delay. In Kubernetes, removing a terminating pod from load balancers happens in parallel with sending SIGTERM. For a few seconds, new requests can still arrive. Failing readiness and waiting briefly before Shutdown avoids refusing them.
  • Budget the time. Drain delay plus shutdown timeout (5 + 20 seconds) must fit inside the orchestrator's grace period (30), or SIGKILL arrives mid-drain.

Health checks

Orchestrators and load balancers poll two kinds of endpoint, and confusing them causes outages.

  • Liveness (/healthz): "is this process stuck?" If it fails, the orchestrator restarts the container. It should check only that the process can serve HTTP, never its dependencies. If liveness pinged the database, a short database blip would restart every instance at once.
  • Readiness (/readyz): "should this instance receive traffic right now?" If it fails, the instance is taken out of the load balancer but left running. Check critical dependencies here, with a short timeout, and fail it while draining.
go
func routes(pool *pgxpool.Pool, draining *atomic.Bool) http.Handler {
	mux := http.NewServeMux()

	// Liveness: the process is up and serving HTTP. No dependency checks.
	mux.HandleFunc("GET /healthz", func(w http.ResponseWriter, r *http.Request) {
		w.Write([]byte("ok\n"))
	})

	// Readiness: this instance should receive traffic right now.
	mux.HandleFunc("GET /readyz", func(w http.ResponseWriter, r *http.Request) {
		if draining.Load() {
			http.Error(w, "shutting down", http.StatusServiceUnavailable)
			return
		}
		ctx, cancel := context.WithTimeout(r.Context(), time.Second)
		defer cancel()
		if err := pool.Ping(ctx); err != nil {
			http.Error(w, "database unavailable", http.StatusServiceUnavailable)
			return
		}
		w.Write([]byte("ok\n"))
	})

	// ... your API routes ...
	return mux
}

In Kubernetes, these are wired up as probes:

yaml
spec:
  terminationGracePeriodSeconds: 30
  containers:
    - name: orders-api
      image: registry.example.com/orders-api:v1.4.2
      env:
        - name: DATABASE_URL
          valueFrom:
            secretKeyRef: { name: orders-db, key: url }
      livenessProbe:
        httpGet: { path: /healthz, port: 8080 }
        periodSeconds: 10
      readinessProbe:
        httpGet: { path: /readyz, port: 8080 }
        periodSeconds: 5
      resources:
        limits: { cpu: "1", memory: 512Mi }

Building a production binary

With cgo disabled, go build produces a statically linked binary with no runtime dependencies at all:

bash
CGO_ENABLED=0 go build -trimpath \
  -ldflags="-s -w -X main.version=$(git describe --tags --always)" \
  -o bin/server ./cmd/server
  • CGO_ENABLED=0 avoids linking against the system C library, so the binary runs on images that have none. Pure-Go packages such as pgx don't need cgo.
  • -trimpath removes your local file paths from the binary, for reproducible builds.
  • -s -w strips the symbol table and DWARF debug info, typically shrinking the binary by around 30%. Panics still print full stack traces.
  • -X main.version=... sets the package-level string var version = "dev" at link time, so the binary can report which build it is in logs and on a /version endpoint.

Cross-compiling is just environment variables: GOOS=linux GOARCH=arm64 go build ... produces an ARM Linux binary from any machine.

A multi-stage Dockerfile

A container image needs the binary, not the Go toolchain. A multi-stage build compiles in a full golang image and copies only the result into a minimal runtime image:

dockerfile
# syntax=docker/dockerfile:1

# ---- build stage: full Go toolchain ----
FROM golang:1.27 AS build
WORKDIR /src

# Download modules first so this layer is cached until go.mod/go.sum change.
COPY go.mod go.sum ./
RUN go mod download

COPY . .
ARG VERSION=dev
RUN --mount=type=cache,target=/root/.cache/go-build \
    CGO_ENABLED=0 go build -trimpath \
      -ldflags="-s -w -X main.version=${VERSION}" \
      -o /out/server ./cmd/server

# ---- runtime stage: just the binary ----
FROM gcr.io/distroless/static-debian12:nonroot
COPY --from=build /out/server /server
EXPOSE 8080
ENTRYPOINT ["/server"]

Layer order is what makes rebuilds fast: changing a .go file invalidates only the layers from COPY . . onwards, so modules aren't downloaded again, and the BuildKit cache mount keeps compiled packages between builds. Add a .dockerignore (with .git, .env and local build output) so secrets and junk never enter the build context. The builder's Go version must be at least the go line in your go.mod.

The runtime base image is a real choice:

BaseSizeContainsNotes
distroless/static:nonrootAbout 2 MBCA certificates, time zone data, a non-root userNo shell or package manager; a good default
scratch0 MBNothingYou must copy in CA certs (for outbound TLS) and set a user yourself
alpineAbout 8 MBShell, package managerHandy for debugging, but more to patch

The final image is the binary plus a couple of megabytes, typically under 20 MB in total. It has no shell for an attacker to use, and the :nonroot tag runs the process as UID 65532 instead of root.

Two runtime settings matter in containers. Since Go 1.25 the runtime sets GOMAXPROCS from the container's CPU limit automatically; on older versions it saw every CPU on the host and over-scheduled. And set GOMEMLIMIT to roughly 90% of the memory limit (for example GOMEMLIMIT=460MiB for 512 MiB), so the garbage collector works harder as you approach the limit instead of the container being killed for running out of memory.

Practice

  1. Put LoadConfig, run and routes in cmd/server/main.go, with var version = "dev", and add a GET /slow route that sleeps 5 seconds before writing finished.
  2. Build the image with docker build --build-arg VERSION=v1.4.2 -t orders-api:v1.4.2 . and check its size with docker images.
  3. Run it with DATABASE_URL set, start curl localhost:8080/slow, and run docker stop while the request is in flight. The curl must still receive finished, and the logs must show a clean shutdown.
  4. Run the image without DATABASE_URL and confirm it exits with code 1 and a clear message.
Show solution

The extra route, inside routes:

go
mux.HandleFunc("GET /slow", func(w http.ResponseWriter, r *http.Request) {
	time.Sleep(5 * time.Second)
	w.Write([]byte("finished\n"))
})
bash
docker build --build-arg VERSION=v1.4.2 -t orders-api:v1.4.2 .
docker run -d --name orders -p 8080:8080 \
  --add-host=host.docker.internal:host-gateway \
  -e DATABASE_URL="postgres://app:[email protected]:5432/app" orders-api:v1.4.2

curl -s localhost:8080/slow &   # in flight...
sleep 1 && docker stop orders  # sends SIGTERM, waits up to 10 s, then SIGKILL
wait                           # curl prints: finished
docker logs orders
text
{"level":"INFO","msg":"listening","addr":":8080","version":"v1.4.2"}
{"level":"INFO","msg":"shutdown started; draining"}
{"level":"INFO","msg":"shutdown complete"}

docker stop returns after about five seconds: the drain delay overlaps the slow request, then Shutdown waits for it to finish. Docker's default stop timeout is 10 seconds, shorter than this service's 25-second budget, so pass docker stop -t 30 (or set Kubernetes' grace period) for real workloads.

Without configuration:

text
$ docker run --rm orders-api:v1.4.2; echo "exit=$?"
2026/10/11 08:28:32 ERROR fatal err="config: DATABASE_URL is required"
exit=1

That last line has the default text format because the failure happens before the JSON logger is configured.

Key takeaways

  • Read configuration from environment variables into one typed struct at startup, and refuse to start if anything is missing or invalid.
  • Use signal.NotifyContext and srv.Shutdown with a fresh timeout context so deploys never drop in-flight requests; keep os.Exit in main only.
  • Liveness checks only the process; readiness checks dependencies and fails while draining.
  • Build static binaries with CGO_ENABLED=0 -trimpath -ldflags="-s -w -X main.version=...".
  • Use a multi-stage Dockerfile with a distroless nonroot runtime image and an exec-form ENTRYPOINT so signals reach your process.
  • Fit drain delay plus shutdown timeout inside the grace period, and set GOMEMLIMIT below the container's memory limit.

Check yourself

Quiz

0/4 correct · 4 left

  1. 1.Why does run create the shutdown context with context.WithTimeout(context.Background(), ...) rather than deriving it from ctx?

  2. 2.Your liveness probe pings PostgreSQL. The database has a 40-second failover. What happens?

  3. 3.A Dockerfile ends with `ENTRYPOINT /server` (shell form) on an alpine base. What goes wrong on `docker stop`?

  4. 4.What does `-ldflags="-X main.version=v1.4.2"` do?

Concepts in this lesson

One-page notes on each idea, for review or a quick lookup later.

Finished reading?

Mark it complete to fill in your roadmap.