[BUG] Not usable as a Go module

[aidansteele]

Describe the bug
I am trying to build a project that uses soci-snapshotter. I ran go get github.com/awslabs/soci-snapshotter@latest, used some of the exported functions and tried go build. It failed with the following error:

/usr/local/go/pkg/tool/linux_arm64/link: running gcc failed: exit status 1
/usr/bin/ld: cannot find -lindexer
collect2: error: ld returned 1 exit status

I think this is because right now the code expects the indexer C library to be built using make.

Is the intention for this package to be usable as a Go module? I would find it quite useful, but maybe it's not a goal for this project?

[sbuckfelder]

Thanks for highlighting this. We definitely want to add better support for other projects to incorporate soci-snapshotter and the Go module path is something we are discussing. Just curious - would you be willing to share your use case?

[aidansteele]

Sure! I started work on a web interface for browsing Docker images. The idea was that you would e.g. navigate to example.com/public.ecr.aws/bitnami/golang:1.19.1 and it would show a filesystem tree on the left and the contents of files on the right. Showing the file contents on the right would have required me to use estargz to transform and store duplicate image layers in my own S3 bucket - a cost I didn't want to incur for a free service as a hobby. soci-snapshotter would mean that I just needed to store the span metadata, a much more manageable volume of data.

Not the most important use case in the world, more just a fun self-educational project.

[Kern--]

This is done as of #120. I will caution you that neither this projects exported functions nor the format of a zTOC is stable and may change on any commit, but it works if you want to try it out. Happy hacking!

Here's a trivial example:

main.go

package main

import (
    "github.com/awslabs/soci-snapshotter/soci"

    "fmt"
)

func main() {
	z, err := soci.BuildZtoc("main.go.tar.gz", 1 << 8, "My custom tool!")
	if err != nil {
		fmt.Println(err)
		return
	}
	fmt.Println(z.TOC)
}

$ tar -czf main.go.tar.gz main.go
$ go run main.go
{[{main.go reg 512 236  420 1000 1000 admin admin 2022-12-09 23:13:56 +0000 UTC 0 0 map[]}]}