fixing versions of tools used by go

Question

I am looking to create reproducible builds with go. For individual projects we are using glide.

So for example I use:

glide get github.com/stretchr/test         


        

Answer 1

The solution for go1.11 using go modules is to create a fake tools package. You create a tools.go file like the following:

// +build tools

package tools

import (
        _ "github.com/tebeka/go2xunit"
)

+build tools is a magic comment which prevents the package being built.

>go mod init tools

Will create a go.mod file for the fake tools package

>go install github.com/tebeka/go2xunit

Will install go2xunit and update go.mod as follows.

module tools

require github.com/tebeka/go2xunit v1.4.8 // indirect

Now if you run go install github.com/tebeka/go2xunit in the future (for a clean build say) its version will be fixed to v1.4 by the go.mod

---

For versions of go before 1.11 the tool to use is retool. It works like this:

bootstrap:

go get github.com/twitchtv/retool

add tool:

retool add github.com/jteeuwen/go-bindata/go-bindata origin/master

use tool:

retool do go-bindata -pkg testdata -o ./testdata/testdata.go ./testdata/data.json

Adding support for this may be on the roadmap to target go 1.12 (https://github.com/golang/go/issues/27653)

Answer 2

I did this very similarly, but just different enough that I think it's worth sharing again:

If you get an error

I was not seeing the dependency that I wanted added to the go.mod and I was getting this error:

tools/tools.go:6:5: import "github.com/UnnoTed/fileb0x" is a program, not an importable package

(fileb0x is the thing I'm trying to add)

I'm not 100% clear on the sequence of events that fixed it, but I did all of these things:

Using a "tools" package

I made a tools directory:

mkdir -p tools

I put the tools package inside of it (as mentioned above):

// +build tools

package tools

import (
    _ "github.com/UnnoTed/fileb0x"
)

Note that the tag is mostly not important. You could use foo:

// +build foo

However, you cannot use ignore. That's a special predefined tag.

// +build ignore

// NO NO NO NO NO
// `ignore` is a special keyword which (surprise) will cause
// the file to be ignore, even for dependencies

Updating go.mod

The best way is probably to run go mod tidy:

go mod tidy

However, before I did that I ran a number of commands trying to figure out which one would cause it to go into go.mod:

go install github.com/UnnoTed/fileb0x # didn't seem to do the trick
go get
go generate ./...
go build ./...
go install ./...
go mod vendor

Later I did a git reset and rm -rf ~/go/pkg/mod; mkdir ~/go/pkg/mod and found that go mod tidy did well enough on its own.

vendoring

In order to actually take advantage of the modules cache in a project you need to copy-in the source code

go mod vendor

That will grab all dependencies from go.mod

You also need to change nearly all of your go commands to use -mod=vendor in any Makefiles, Dockerfiles or other scripts.

go fmt -mod=vendor ./... # has a bug which should be fixed in go1.15
go generate -mod=vendor ./...
go build -mod=vendor ./...

That includes go build, go get, go install, and any go run called by go generate (and even the go generate itself)

//go:generate go run -mod=vendor github.com/UnnoTed/fileb0x b0x.toml
package main

// ...