9.3 Profiling and Performance

Understanding and optimizing performance is crucial for building efficient Go applications. Go provides excellent built-in profiling tools to help identify bottlenecks and optimize your code.

Profiling Tools

Go includes several profiling tools in the standard library and toolchain.

Profiling is important because it helps identify bottlenecks (find where your code spends time) and detect memory issues (memory leaks and excessive allocations).

Available profiling tools

  1. CPU profiling: Identifies where CPU time is spent
  2. Memory profiling: Shows memory allocation patterns
  3. Block profiling: Identifies blocking operations
  4. Mutex profiling: Shows mutex contention
  5. Goroutine profiling: Shows goroutine states

Profiling is enabled via HTTP endpoints or programmatically:

package main

import (
    "log"
    "net/http"
    _ "net/http/pprof"  // Registers profiling handlers
)

func main() {
    // Start HTTP server for profiling
    go func() {
        log.Println(http.ListenAndServe("localhost:6060", nil))
    }()

    // Your application code
}

Then access profiles at:

CPU Profiling

CPU profiling shows where your program spends CPU time.

CPU profiling matters because it helps find hot spots (functions that consume most CPU), identifies optimization targets (know what to optimize), and reveals inefficiencies (find expensive operations).

Collect CPU profiles using:

Method 1: HTTP endpoint

import _ "net/http/pprof"

// Access http://localhost:6060/debug/pprof/profile?seconds=30
// Downloads a 30-second CPU profile

Method 2: Programmatic

package main

import (
    "os"
    "runtime/pprof"
)

func main() {
    f, _ := os.Create("cpu.prof")
    defer f.Close()

    pprof.StartCPUProfile(f)
    defer pprof.StopCPUProfile()

    // Your code to profile
    doWork()
}

Analyze CPU profiles using:

go tool pprof cpu.prof
# Or
go tool pprof http://localhost:6060/debug/pprof/profile

Common pprof commands

(pprof) top          # Show top functions by CPU time
(pprof) top10        # Show top 10 functions
(pprof) list FuncName  # Show source code with CPU time
(pprof) web          # Generate SVG visualization
(pprof) png          # Generate PNG visualization

Example CPU profile analysis

(pprof) top
Showing nodes accounting for 2.50s, 100% of 2.50s total
      flat  flat%   sum%        cum   cum%
     1.50s  60.00%  60.00%      1.50s  60.00%  main.expensiveFunction
     1.00s  40.00% 100.00%      1.00s  40.00%  main.anotherFunction

Understanding the output

Memory Profiling

Memory profiling shows memory allocation patterns and helps identify memory leaks.

Memory profiling matters because it helps find memory leaks (allocations that aren't freed), identifies excessive allocations (unnecessary allocations), and helps understand memory consumption patterns.

Collect memory profiles using:

package main

import (
    "os"
    "runtime"
    "runtime/pprof"
)

func main() {
    f, _ := os.Create("mem.prof")
    defer f.Close()

    // Your code
    doWork()

    // Write memory profile
    runtime.GC()  // Force GC before profiling
    pprof.WriteHeapProfile(f)
}

Or via HTTP

import _ "net/http/pprof"

// Access http://localhost:6060/debug/pprof/heap

To analyze a memory profile, use:

go tool pprof mem.prof

Memory profile commands

(pprof) top          # Top memory allocators
(pprof) top -cum     # Top by cumulative allocations
(pprof) list FuncName  # Show allocations in function
(pprof) alloc_space  # Show allocations by space
(pprof) alloc_objects  # Show allocations by count

Example memory profile

(pprof) top
Showing nodes accounting for 512.50MB, 100% of 512.50MB total
      flat  flat%   sum%        cum   cum%
  256.25MB  50.00%  50.00%   256.25MB  50.00%  main.allocateMemory
  256.25MB  50.00% 100.00%   256.25MB  50.00%  main.anotherAllocation

Detecting memory leaks

Compare profiles at different times:

# Take profile at time T1
go tool pprof http://localhost:6060/debug/pprof/heap

# After some time, take another profile
go tool pprof http://localhost:6060/debug/pprof/heap

# Compare
go tool pprof -base=heap1.prof heap2.prof

Benchmarking

Benchmarks measure code performance and help validate optimizations.

Benchmarking is important because it helps catch performance degradation (performance regression), verify that optimizations actually help (optimization validation), and compare different implementations (comparison).

Basic benchmark

func BenchmarkFunction(b *testing.B) {
    for i := 0; i < b.N; i++ {
        Function()
    }
}

Running benchmarks

go test -bench=.              # Run all benchmarks
go test -bench=BenchmarkFunction  # Run specific benchmark
go test -bench=. -benchmem    # Include memory stats
go test -bench=. -cpuprofile=cpu.prof  # CPU profile during benchmark

Benchmark output

BenchmarkFunction-8    10000000    120 ns/op    16 B/op    1 allocs/op

Comparing benchmarks

func BenchmarkOldWay(b *testing.B) {
    for i := 0; i < b.N; i++ {
        oldImplementation()
    }
}

func BenchmarkNewWay(b *testing.B) {
    for i := 0; i < b.N; i++ {
        newImplementation()
    }
}

Benchmark with setup

func BenchmarkWithSetup(b *testing.B) {
    // Setup (not timed)
    data := prepareData()

    b.ResetTimer()  // Reset timer after setup

    for i := 0; i < b.N; i++ {
        process(data)
    }
}

Setup code shouldn't be included in benchmark timing. ResetTimer() excludes setup from measurements, ensuring only the code being benchmarked is measured.

Benchmark different sizes

func BenchmarkAppend(b *testing.B) {
    sizes := []int{10, 100, 1000, 10000}

    for _, size := range sizes {
        b.Run(fmt.Sprintf("size-%d", size), func(b *testing.B) {
            slice := make([]int, 0, size)
            for i := 0; i < b.N; i++ {
                slice = append(slice, i)
                if len(slice) >= size {
                    slice = slice[:0]
                }
            }
        })
    }
}

Optimization Techniques

1. Reduce allocations

// ❌ BAD: Many allocations
func process(items []Item) []Result {
    results := []Result{}  // Allocates
    for _, item := range items {
        results = append(results, processItem(item))  // May reallocate
    }
    return results
}

// ✅ GOOD: Pre-allocate
func process(items []Item) []Result {
    results := make([]Result, 0, len(items))  // Pre-allocate capacity
    for _, item := range items {
        results = append(results, processItem(item))
    }
    return results
}

2. Use sync.Pool for temporary objects

var bufferPool = sync.Pool{
    New: func() interface{} {
        return new(bytes.Buffer)
    },
}

func process() {
    buf := bufferPool.Get().(*bytes.Buffer)
    defer bufferPool.Put(buf)
    buf.Reset()

    // Use buf
}

3. Avoid unnecessary conversions

// ❌ BAD: Unnecessary conversion
func process(s string) {
    b := []byte(s)  // Allocation
    // Use b
}

// ✅ GOOD: Work with string directly
func process(s string) {
    // Use s directly, or use unsafe if really needed
}

4. Use strings.Builder for string concatenation

// ❌ BAD: Many allocations
func buildString(parts []string) string {
    result := ""
    for _, part := range parts {
        result += part  // Allocates new string each time
    }
    return result
}

// ✅ GOOD: Use Builder
func buildString(parts []string) string {
    var builder strings.Builder
    builder.Grow(len(parts) * 10)  // Pre-allocate if you know size
    for _, part := range parts {
        builder.WriteString(part)
    }
    return builder.String()
}

5. Prefer value receivers for small structs

// ✅ GOOD: Value receiver for small struct
type Point struct {
    X, Y int
}

func (p Point) Distance() float64 {
    return math.Sqrt(float64(p.X*p.X + p.Y*p.Y))
}

// ⚠️ CONSIDER: Pointer receiver for large structs
type LargeStruct struct {
    // Many fields
}

func (s *LargeStruct) Process() {
    // ...
}

6. Use appropriate data structures

// ✅ GOOD: Use map for lookups
func find(items []Item, id int) *Item {
    index := make(map[int]*Item, len(items))
    for i := range items {
        index[items[i].ID] = &items[i]
    }
    return index[id]
}

// ❌ BAD: Linear search
func find(items []Item, id int) *Item {
    for i := range items {
        if items[i].ID == id {
            return &items[i]
        }
    }
    return nil
}

7. Profile before optimizing

// Always measure first
func BenchmarkCurrent(b *testing.B) {
    // Current implementation
}

func BenchmarkOptimized(b *testing.B) {
    // Optimized implementation
}

// Compare results before deciding

8. Use compiler optimizations

go build -gcflags="-m"  # See escape analysis
go build -ldflags="-s -w"  # Strip debug info (smaller binary)

9. Consider escape analysis

// Understanding where values are allocated
func process() {
    x := 42  // May escape to heap
    // Use x
}

10. Use appropriate concurrency

// ✅ GOOD: Worker pool for controlled concurrency
func process(items []Item) {
    const numWorkers = 10
    jobs := make(chan Item, len(items))
    var wg sync.WaitGroup

    for w := 0; w < numWorkers; w++ {
        wg.Add(1)
        go func() {
            defer wg.Done()
            for item := range jobs {
                processItem(item)
            }
        }()
    }

    for _, item := range items {
        jobs <- item
    }
    close(jobs)
    wg.Wait()
}

Performance optimization workflow

  1. Measure: Use profiling to find bottlenecks
  2. Identify: Determine what's actually slow
  3. Optimize: Apply optimization techniques
  4. Validate: Benchmark to verify improvement
  5. Repeat: Iterate until performance is acceptable

Best practices

Profiling and optimization are iterative processes. Use the tools Go provides to identify bottlenecks, apply optimizations, and validate improvements. Remember: premature optimization is the root of all evil—measure first, then optimize.

9.3 Profiling and Performance