golang unmarshal complex json

Question

I have the following JSON blob, and I'm trying to decode it into Go.

["contig", "32", {"a":[33,41,35], "b":[44,34,42]}]

I believe that I have to model the data structure of the JSON. I tried using a struct called Line:

package main

import (
"encoding/json"
"fmt"
)

type Line struct {
    Contig string
    Base   string
    PopMap map[string][]int
}

func main() {
    j := []byte(`["contig", "32", {"a":[33,41,35], "b":[44,34,42]}]`)
    var dat Line
    err := json.Unmarshal(j, &dat)
    fmt.Println(dat)
    fmt.Println(err)
}

I got the following error:

{  map[]}
json: cannot unmarshal array into Go value of type main.Line

What am I doing wrong?

Link to sandbox for trying out code

Answer 1

The JSON input you specified is an array of different types, so as it is, you can't unmarshal it into a struct, but only into a slice of different types: []interface{}.

in := `["contig", "32", {"a":[33,41,35], "b":[44,34,42]}]`

var arr []interface{}
if err := json.Unmarshal([]byte(in), &arr); err != nil {
    panic(err)
}
fmt.Println(arr)

Output:

[contig 32 map[a:[33 41 35] b:[44 34 42]]]

Filling the struct

Good, you now have the values, just not in the struct you want them to be. You can use type assertion to obtain the types you want:

l := Line{PopMap: map[string][]int{}}
l.Contig = arr[0].(string)
l.Base = arr[1].(string)

m := arr[2].(map[string]interface{})
for k, v := range m {
    nums := v.([]interface{})
    pops := make([]int, len(nums))
    for i, val := range nums {
        pops[i] = int(val.(float64))
    }
    l.PopMap[k] = pops
}

fmt.Printf("%+v", l)

Output (try it on the Go Playground):

{Contig:contig Base:32 PopMap:map[a:[33 41 35] b:[44 34 42]]}

Some notes:

The "internal" arrays of the values of "a" and "b" are unmarshaled into values of type []interface{} which you cannot simply convert to []int or []float64 hence the for loops to iterate over them and use type assertion on each of their elements. Also note that the json package unmarshals the numbers into values of type float64 and not int (because not just integers can be in the JSON text so float64 is used which can accommodate both).

Also note that the success of type assertions are not checked in the above example. If the unmarshaled array has less than 3 elements, or any of the type assertion fails, a runtime panic occurs.

Using recover()

You can add a defer function which calls recover() to catch this panic (try it on the Go Playground):

defer func() {
    if r := recover(); r != nil {
        fmt.Println("Failed to unmarshal")
    }
}()

l := Line{PopMap: map[string][]int{}}
// ...and here comes the code that uses type assertions
// and stores values into...

Code with checks

Or you can add checks for the type assertions. The type assertion has a special form v, ok = x.(T) which when used never panics, but rather if the type assertion doesn't hold, ok will be false (and will be true if type assertion holds).

Try it on the Go Playground:

if len(arr) < 3 {
    return
}
var ok bool
l := Line{PopMap: map[string][]int{}}
if l.Contig, ok = arr[0].(string); !ok {
    return
}
if l.Base, ok = arr[1].(string); !ok {
    return
}
if m, ok := arr[2].(map[string]interface{}); !ok {
    return
} else {
    for k, v := range m {
        var nums []interface{}
        if nums, ok = v.([]interface{}); !ok {
            return
        }
        pops := make([]int, len(nums))
        for i, val := range nums {
            if f, ok := val.(float64); !ok {
                return
            } else {
                pops[i] = int(f)
            }
        }
        l.PopMap[k] = pops
    }
}

fmt.Printf("%+v", l)

Answer 2

Your JSON contains an array literal and you're trying to deserialize it as a struct. You need to change the JSON to an object literal where the keys are the property names of your struct.

j := []byte(`{
    "Contig": "contig",
    "Base": "32",
    "PopMap": {"a":[33,41,35], "b":[44,34,42]}
}`)

If the JSON isn't something you have the ability to change, then you will need to deserialize it into an untyped array and perform your own transformation into the struct type.

Answer 3

Because you have an array literal rather than an object, the best way to parse is by unmarshaling first to a slice of json.RawMessages, and then go through the fields in the resulting slice:

package main

import (
     "encoding/json"
     "fmt"
)


func main() {
    j := []byte(`["contig", "32", {"a":[33,41,35], "b":[44,34,42]}]`)

    var entries []json.RawMessage
    err := json.Unmarshal(j, &entries)
    if err != nil {
        fmt.Println(err)
    }

    var contig string
    var num string
    var obj struct {
        A []int `json:"a"`
        B []int `json:"b"`
    }

    err = json.Unmarshal(entries[0], &contig)
    err = json.Unmarshal(entries[1], &num)
    err = json.Unmarshal(entries[2], &obj)

    fmt.Println(contig)
    fmt.Println(num)
    fmt.Println(obj)
}

This gives the correct result:

contig 32 {[33 41 35] [44 34 42]}

Playground: https://play.golang.org/p/jcYvINkTTn

If you have control over the source of the JSON though, changing it to an object literal would be the most straightforward way to go about it.