Concurrency

A backend spends most of its time waiting: for the network, for the database, for other services. Go's answer is cheap concurrency built into the language. A Go HTTP server already runs every request in its own goroutine, so even if you never write the go keyword yourself, your code runs concurrently. This lesson covers goroutines, channels, select, protecting shared memory with sync, cancelling work with context, and the patterns and pitfalls you will meet in real services.

#Goroutines and WaitGroup

A goroutine is a function running concurrently with the rest of your program. Start one by putting go before a function call. Goroutines are managed by the Go runtime, not the operating system, and start with a few kilobytes of stack, so running tens of thousands is normal.

The catch: when main returns, the program exits, taking any running goroutines with it. To wait for goroutines to finish, use a sync.WaitGroup:

package main

import (
	"fmt"
	"sync"
	"time"
)

func main() {
	var wg sync.WaitGroup
	for i := range 3 {
		wg.Add(1) // before starting the goroutine, never inside it
		go func() {
			defer wg.Done()
			time.Sleep(time.Duration(3-i) * 10 * time.Millisecond)
			fmt.Println("worker", i, "done")
		}()
	}
	wg.Wait() // blocks until the counter drops to zero
	fmt.Println("all done")
}

worker 2 done
worker 1 done
worker 0 done
all done

Each goroutine captures its own i because loop variables are per-iteration since Go 1.22. The output order comes from the different sleeps; without them, the order would be unpredictable. Never assume goroutines run in any particular order. (Go 1.25 adds wg.Go(func() { ... }), which does the Add and Done for you.)

#Channels: communicating between goroutines

A channel is a typed pipe that goroutines use to pass values and to synchronise. Create one with make, send with ch <- v, receive with v := <-ch:

func squares(n int, out chan<- int) { // chan<- means send-only
	for i := range n {
		out <- i * i
	}
	close(out) // no more values coming
}

func main() {
	ch := make(chan int)
	go squares(4, ch)
	for v := range ch { // receives until ch is closed
		fmt.Println(v) // 0, 1, 4, 9
	}
}

The key behaviours:

• An unbuffered channel (make(chan int)) has no storage. A send blocks until another goroutine receives, so a send and receive are a hand-off that synchronises both sides.
• A buffered channel (make(chan int, 10)) accepts sends without a receiver until the buffer is full, then blocks.
• close(ch) signals "no more values". Receivers drain the remaining values, then receive the zero value immediately. Use v, ok := <-ch to tell: ok is false once the channel is closed and empty.
• Only the sender closes, and only when receivers need to know it is finished. Sending on a closed channel panics. Channels do not need to be closed for garbage collection.
• Direction types (chan<- int send-only, <-chan int receive-only) in function signatures document intent and let the compiler catch misuse.

Try these rules on a live channel below. Start unbuffered (capacity 0): press Send next and watch the producer block until you press Receive, and notice the value never sits in the channel. Then raise the capacity, fill the buffer, close it, and keep receiving until you get 0 false.

If every goroutine is blocked, the runtime detects it and crashes with a clear message:

func main() {
	ch := make(chan int)
	ch <- 1 // nobody will ever receive
}

fatal error: all goroutines are asleep - deadlock!

#select: waiting on several things

select waits on several channel operations and runs whichever is ready first. Its most common job in backend code is a timeout:

func slowLookup() <-chan string {
	ch := make(chan string, 1) // buffered: see the warning below
	go func() {
		time.Sleep(2 * time.Second)
		ch <- "result"
	}()
	return ch
}

func main() {
	select {
	case res := <-slowLookup():
		fmt.Println(res)
	case <-time.After(500 * time.Millisecond):
		fmt.Println("timed out")
	}
}

If several cases are ready, select picks one at random. A default case runs immediately when nothing else is ready, which turns a blocking operation into a non-blocking attempt.

#Sharing memory safely: sync.Mutex and the race detector

Channels are ideal for passing ownership of data. When several goroutines need to read and update the same data, such as an in-memory cache shared by all request handlers, use a lock instead. Here is the bug you are protecting against:

func main() {
	total := 0
	var wg sync.WaitGroup
	for range 100 {
		wg.Add(1)
		go func() {
			defer wg.Done()
			for range 1000 {
				total++ // read, add, write: three steps that can interleave
			}
		}()
	}
	wg.Wait()
	fmt.Println(total) // should be 100000; prints e.g. 84788, different each run
}

This is a data race: two goroutines access the same memory at the same time and at least one writes. The result is undefined, and with maps it is worse: concurrent map writes crash the program with fatal error: concurrent map writes. Go ships a race detector that finds these bugs while your code runs. Use it in tests and during development:

go run -race .
go test -race ./...

==================
WARNING: DATA RACE
Read at 0x00c000012158 by goroutine 11:
  main.main.func1()
      /home/you/race/main.go:16 +0x99
...

The fix is a sync.Mutex, which only one goroutine can hold at a time. Put the mutex next to the data it protects and keep the locked section short:

type Counter struct {
	mu     sync.Mutex
	counts map[string]int
}

func NewCounter() *Counter { return &Counter{counts: make(map[string]int)} }

func (c *Counter) Inc(key string) {
	c.mu.Lock()
	defer c.mu.Unlock()
	c.counts[key]++
}

func (c *Counter) Get(key string) int {
	c.mu.Lock()
	defer c.mu.Unlock()
	return c.counts[key]
}

Methods use pointer receivers because copying a struct copies its mutex, and a copied lock protects nothing (go vet warns about this). Related tools: sync.RWMutex allows many concurrent readers or one writer, sync/atomic types such as atomic.Int64 handle single counters without a lock, and sync.Once runs initialisation exactly once.

#context: cancellation and deadlines

Suppose a client disconnects while your handler is waiting on a slow database query. Ideally the query stops too. The context package carries that cancellation signal, plus deadlines, through every function involved in a piece of work.

func slowQuery(ctx context.Context) (string, error) {
	select {
	case <-time.After(2 * time.Second): // pretend work
		return "rows", nil
	case <-ctx.Done(): // closed when ctx is cancelled or times out
		return "", ctx.Err()
	}
}

func main() {
	ctx, cancel := context.WithTimeout(context.Background(), 500*time.Millisecond)
	defer cancel()

	_, err := slowQuery(ctx)
	fmt.Println(err)                                      // context deadline exceeded
	fmt.Println(errors.Is(err, context.DeadlineExceeded)) // true
}

How it fits together:

• context.Background() is the empty root. WithTimeout, WithDeadline and WithCancel derive a child that is cancelled when its parent is, when the deadline passes, or when you call cancel.
• ctx.Done() returns a channel that is closed on cancellation, so it slots straight into select. ctx.Err() then reports context.Canceled or context.DeadlineExceeded.
• Always call cancel, usually with defer, even if the work finishes early. It releases the timer and the context's resources.
• Pass the context as the first parameter, named ctx. Do not store it in a struct.

Standard library functions that may block accept a context: http.NewRequestWithContext, db.QueryContext, and so on. In an HTTP handler you get one from r.Context(); the server cancels it when the client goes away.

#Worker pools: bounding concurrency

Starting one goroutine per item is fine for ten items and dangerous for a million: you can exhaust memory, database connections, or a partner API's rate limit. A worker pool runs a fixed number of goroutines that pull jobs from a channel:

func worker(id int, jobs <-chan int, results chan<- string, wg *sync.WaitGroup) {
	defer wg.Done()
	for j := range jobs {
		results <- fmt.Sprintf("worker %d processed job %d", id, j)
	}
}

func main() {
	jobs := make(chan int)
	results := make(chan string)

	var wg sync.WaitGroup
	for id := range 3 {
		wg.Add(1)
		go worker(id, jobs, results, &wg)
	}

	go func() { // feed jobs, then signal there are no more
		for j := range 9 {
			jobs <- j
		}
		close(jobs)
	}()

	go func() { // close results once every worker has exited
		wg.Wait()
		close(results)
	}()

	for r := range results {
		fmt.Println(r)
	}
}

Follow the closes: the feeder closes jobs, which ends each worker's range; when all workers are done, a helper closes results, which ends the main loop. Every goroutine has a clear exit, so nothing leaks. The golang.org/x/sync/errgroup package packages this pattern with error propagation and a concurrency limit, and is worth adopting once you are comfortable with the pieces.

#Practice

Write checkAll(ctx context.Context, urls []string, limit int) map[string]error that "checks" every URL concurrently, running at most limit checks at once and stopping when ctx is cancelled. Use this fake check, whose delay depends on the URL's length:

func check(ctx context.Context, url string) error {
	select {
	case <-time.After(time.Duration(len(url)*20) * time.Millisecond):
		if strings.Contains(url, "bad") {
			return errors.New("status 500")
		}
		return nil
	case <-ctx.Done():
		return ctx.Err()
	}
}

Call it with a 300 ms timeout, a limit of 2, and the URLs a.example, bad.example and slow-service.example, then print the results in sorted order. Run it with -race to confirm there are no data races.

Hint: a buffered channel of capacity limit works as a semaphore: send to acquire a slot, receive to release it.

package main

import (
	"context"
	"errors"
	"fmt"
	"maps"
	"slices"
	"strings"
	"sync"
	"time"
)

func check(ctx context.Context, url string) error {
	select {
	case <-time.After(time.Duration(len(url)*20) * time.Millisecond):
		if strings.Contains(url, "bad") {
			return errors.New("status 500")
		}
		return nil
	case <-ctx.Done():
		return ctx.Err()
	}
}

func checkAll(ctx context.Context, urls []string, limit int) map[string]error {
	var (
		mu      sync.Mutex
		wg      sync.WaitGroup
		results = make(map[string]error, len(urls))
		sem     = make(chan struct{}, limit)
	)
	for _, u := range urls {
		sem <- struct{}{} // acquire a slot; blocks while `limit` checks are running
		wg.Add(1)
		go func() {
			defer wg.Done()
			defer func() { <-sem }() // release the slot
			err := check(ctx, u)
			mu.Lock()
			results[u] = err
			mu.Unlock()
		}()
	}
	wg.Wait()
	return results
}

func main() {
	ctx, cancel := context.WithTimeout(context.Background(), 300*time.Millisecond)
	defer cancel()

	urls := []string{"a.example", "bad.example", "slow-service.example"}
	res := checkAll(ctx, urls, 2)
	for _, u := range slices.Sorted(maps.Keys(res)) {
		fmt.Printf("%-22s %v\n", u, res[u])
	}
}

a.example              <nil>
bad.example            status 500
slow-service.example   context deadline exceeded

#Key takeaways

• go f() starts a goroutine; use a sync.WaitGroup (calling Add before go) to wait for goroutines to finish.
• Channels pass values and synchronise goroutines; only the sender closes, and range reads until close.
• select waits on several channel operations, which is how you implement timeouts and react to cancellation.
• Protect shared mutable state with a sync.Mutex, and run tests with -race to catch data races.
• Pass context.Context as the first parameter, check ctx.Done() in blocking work, and always defer cancel().
• Bound concurrency with worker pools or a semaphore, and make sure every goroutine has a way to exit.