Memory overhead of maps in Go

Question

map[byte]byte{0:10} should be using least 2 bytes, one for value and one per key. But as each hashmap implmentation, there is also a hidden cost per item. What is the memory

Answer 1

Here is an experiment to measure the overhead of maps. Works under Linux only.

package main

import (
    "fmt"
    "io/ioutil"
    "log"
    "os"
    "runtime"
    "strconv"
    "strings"
)

func ReadRss() int {
    data, err := ioutil.ReadFile("/proc/self/statm")
    if err != nil {
        log.Fatal(err)
    }
    rss, err := strconv.Atoi(strings.Fields(string(data))[1])
    if err != nil {
        log.Fatal(err)
    }
    return rss * os.Getpagesize()
}

func main() {
    hs := []*map[byte]byte{}
    before := ReadRss()
    n := 10000
    for i := 0; i < n; i++ {
        h := map[byte]byte{}
        hs = append(hs, &h)
    }
    after := ReadRss()
    empty_per_map := float64(after-before)/float64(n)
    fmt.Printf("Bytes used for %d empty maps: %d, bytes/map %.1f\n", n, after-before, empty_per_map)
    hs = nil
    runtime.GC()

    before = ReadRss()
    for i := 0; i < n; i++ {
        h := map[byte]byte{}
        for j := byte(0); j < 100; j++ {
            h[j] = j
        }
        hs = append(hs, &h)
    }
    after = ReadRss()
    full_per_map := float64(after-before)/float64(n)
    fmt.Printf("Bytes used for %d maps with 100 entries: %d, bytes/map %.1f\n", n, after-before, full_per_map)
    fmt.Printf("Bytes per entry %.1f\n", (full_per_map - empty_per_map)/100)

}

It prints this on my 64 bit Linux machine using go 1.0.3

Bytes used for 10000 empty maps: 1695744, bytes/map 169.6
Bytes used for 10000 maps with 100 entries: 43876352, bytes/map 4387.6
Bytes per entry 42.2

Or using go 1.0

Bytes used for 10000 empty maps: 1388544, bytes/map 138.9
Bytes used for 10000 maps with 100 entries: 199323648, bytes/map 19932.4
Bytes per entry 197.9

The memory measurements are done using the Linux OS rather than Go's internal memory stats. The memory measurements are coarse as they are returned in 4k pages, hence the large number of maps created.

So in round numbers each map costs about 170 bytes and each entry costs 42 bytes using go 1.0.3 (much more for 1.0)