Lifecycle hook doesn't execute when callilng service.Up in the API

[chris-gputrader]

I've been pounding my head against this and am not sure if I'm not doing something incorrect or if its the codebase but lifecycle hooks don't seem to get executed when calling the API directly. I have the following example code to test this.

package main

import (
	"context"
	"flag"
	"log"
	"os"
	"time"

	"github.com/compose-spec/compose-go/v2/loader"
	"github.com/compose-spec/compose-go/v2/types"
	"github.com/docker/cli/cli/command"
	"github.com/docker/cli/cli/flags"
	"github.com/docker/compose/v2/pkg/api"
	"github.com/docker/compose/v2/pkg/compose"
)

func main() {
	// Add command line flag for config file
	configFile := flag.String("config", "compose.yaml", "Path to compose config file (yaml or json)")
	flag.Parse()

	// Read the configuration file
	bytes, err := os.ReadFile(*configFile)
	if err != nil {
		log.Fatalf("Failed to read config file: %v", err)
	}

	// Setup Docker client config
	dockerCli, err := command.NewDockerCli()
	if err != nil {
		log.Fatalf("Failed to create docker cli: %v", err)
	}

	opts := flags.NewClientOptions()
	if err := dockerCli.Initialize(opts); err != nil {
		log.Fatalf("Failed to initialize docker cli: %v", err)
	}

	// Create compose service
	service := compose.NewComposeService(dockerCli)

	// Load the project from config
	project, err := loader.Load(types.ConfigDetails{
		ConfigFiles: []types.ConfigFile{
			{
				Content: bytes,
			},
		},
		Environment: types.NewMapping(os.Environ()),
	})
	if err != nil {
		log.Fatalf("Failed to load project: %v", err)
	}

	// Setup options
	options := api.UpOptions{
		Create: api.CreateOptions{
			RemoveOrphans: true,
		},
		Start: api.StartOptions{
			Wait: true,
		},
	}

	// Create context with timeout
	ctx, cancel := context.WithTimeout(context.Background(), 10*time.Minute)
	defer cancel()

	// Pull the images
	if err := service.Pull(ctx, project, api.PullOptions{}); err != nil {
		log.Fatalf("Failed to pull images: %v", err)
	}

	//log the project object
	log.Printf("Project: %v\n", project)

	// Start the project
	if err := service.Up(ctx, project, options); err != nil {
		log.Fatalf("Failed to start project: %v", err)
	}

	log.Println("Services started successfully!")
}

Here is a simple compose.yml to test with:

services:
  alpine:
    image: alpine
    command: sleep 1000000
    post_start:
      - command: /bin/sh -c "echo somecontent > /root/file.txt"      
        privileged: true

The container gets created but the post_start hook never runs.

Any ideas or am I just calling this wrong?

[maggie44]

#11210 (comment)

[maggie44]

It would be worth checking this to make sure it creates the services properly. docker inspect will help indicate if there are any differences between this approach and using docker compose up directly. It you do explore it further, do come back and post any updates as they will help me too.

https://gist.github.com/maggie44/98d870da0961b61c32f6e55407697f76

[chris-gputrader]

It would be worth checking this to make sure it creates the services properly. docker inspect will help indicate if there are any differences between this approach and using docker compose up directly. It you do explore it further, do come back and post any updates as they will help me too.

https://gist.github.com/maggie44/98d870da0961b61c32f6e55407697f76

Thanks, I updated that gist and included a lifecycle hook 'post_start' and verified that the container did not run the lifecycle hook, so something is not right here.

[chris-gputrader]

Not sure if this has something to do with it but just looking through the code from the change to include lifecycle hooks: #12166

It appears as though no changes were made to up.go which might be why the hooks are being ignored, changes were made to start.go and others.

[maggie44]

I assume it is working when using the docker compose binary on the same host (i.e. using docker compose up instead of this custom code)?

[chris-gputrader]

Yes here is the code that I used to test:

package main

import (
	"context"
	"fmt"
	"log/slog"
	"os"

	"github.com/compose-spec/compose-go/v2/loader"
	"github.com/compose-spec/compose-go/v2/types"
	"github.com/compose-spec/compose-go/v2/utils"
	"github.com/docker/cli/cli/command"
	"github.com/docker/cli/cli/flags"
	"github.com/docker/compose/v2/pkg/api"
	"github.com/docker/compose/v2/pkg/compose"
)

const exampleCompose = `
services:
  example:
    image: nginx:latest # Use the latest Nginx image
    container_name: example
    environment:
      - FOO=${SET_ME} # Use the environment variable SET_ME
      - BAR=${AND_ME} # Use the environment variable AND_ME
    ports:
      - "8080:80" # Map port 80 in the container to port 8080 on the host
    post_start:
      - command: /bin/sh -c "echo somecontent > /root/file.txt"
    networks:
      - webnet # Connect the web service to the webnet network
     
networks:
  webnet: # Define a custom network for communication between services
`

func main() {
	err := run()
	if err != nil {
		slog.Error(err.Error())
		os.Exit(1)
	}
}

func run() error {
	// Create a Docker client
	cli, err := command.NewDockerCli()
	if err != nil {
		return err
	}

	// Initialize the Docker client with the default options
	err = cli.Initialize(flags.NewClientOptions())
	if err != nil {
		return err
	}

	// Create the compose API service instance with the Docker cli
	composeService := compose.NewComposeService(cli)

	// Set the environment variables
	envVars := []string{
		"SET_ME=ok",
		"AND_ME=thanks",
	}

	// Convert []string of env vars to map
	envVarMap := utils.GetAsEqualsMap(envVars)

	configDetails := types.ConfigDetails{
		ConfigFiles: []types.ConfigFile{
			{
				Content: []byte(exampleCompose),
			},
		},
		Environment: envVarMap,
	}

	// Create a new project from the compose config details
	project, err := loader.LoadWithContext(context.TODO(), configDetails,
		func(opts *loader.Options) {
			// We are passing in our own environment
			opts.SkipResolveEnvironment = true
			// Set the compose project name
			opts.SetProjectName("example", true)
		})
	if err != nil {
		fmt.Println("Error loading project: ", err)
		return err
	}

	// Optional: Drops networks/volumes/secrets/configs that are not referenced by active services
	project = project.WithoutUnnecessaryResources()

	// Show what is about to be started
	compiledYAML, err := project.MarshalYAML()
	if err != nil {
		return err
	}

	fmt.Println("Compose file to be started:")
	fmt.Println(string(compiledYAML))

	// Optional: configurations
	opts := api.UpOptions{
		Create: api.CreateOptions{
			RemoveOrphans: true,
		},
		Start: api.StartOptions{},
	}

	// Sets the labels required for Docker to recognise that these are Compose services
	for i, s := range project.Services {
		s.CustomLabels = map[string]string{
			api.ProjectLabel: project.Name,
			api.ServiceLabel: s.Name,
			api.VersionLabel: api.ComposeVersion,
			api.OneoffLabel:  "False",
		}

		project.Services[i] = s
	}

	// Run the up command
	err = composeService.Up(context.TODO(), project, opts)
	if err != nil {
		return err
	}

	return nil
}

The container will run fine but "/root/file.txt" does not exist. If I run docker compose up with that yml the hook runs and I see "/root/file.txt" as expected.

[maggie44]

Try adding project to the api.StartOptions:

Start: api.StartOptions{
	Project: project,
},

[chris-gputrader]

Try adding project to the api.StartOptions:

Start: api.StartOptions{
	Project: project,
},

Thanks so much, that solved the issue!

[chris-gputrader]

Sorry to reopen, curious if api.DownOptions would also need to have the Project passed in to execute pre_stop hooks?

[maggie44]

Sorry to reopen, curious if api.DownOptions would also need to have the Project passed in to execute pre_stop hooks?

Did you try it? I would suspect that if projectName is set then it isn't necessary.

...
composeService := compose.NewComposeService(cli)
err = composeService.Down(ctx, projectName, api.DownOptions{RemoveOrphans: true, Volumes: false})

[ndeloof]

Just a quick note: docker/compose is not a library, and despite we have an api package for legacy reasons, using it programmatically is "at your own risks"

[ndeloof]

DownOptions would also need to have the Project passed in to execute pre_stop hooks?

it does. hooks are defined by Project they're not known by the runtime resources - hooks are not a docker engine feature. if you don't pass a Project, no hook will be executed.

[maggie44]

It's a shame it is not available as a library as there is a real need for it. This particular issue has come up as Portainer now uses this approach. It's an understandable change as Portainer used to call a compose binary with exec and parse the stdout response which is of course all kinds of problematic.

I will add the caveat to the top of the gist but hoping to pool our resources and testers to establish a working approach.

[maggie44]

I also added a down example for any input.

[ndeloof]

It's a shame it is not available as a library as there is a real need for it

It never has been designed to be one. Docker compose is a command line. exec and parse stdout is the recommended way - we offer json formatted output for this purpose.

[maggie44]

It never has been designed to be one. Docker compose is a command line. exec and parse stdout is the recommended way - we offer json formatted output for this purpose.

Absolutely, trying to get it to work as a library is clunky and a lot of guess work, as well as not being reliable as changes are made to Compose there is no guarantee it won't break. The decision wasn't taken lightly, for my use case I have been back and forth for a year on how best to implement this.

Neither approach is perfect, importing the code as a library is unsupported and I don't want to have to support it, so been using exec approach.

But the exec approach has it's own problems, it doesn't scale well leaving lots of zombie processes to clean up, it is more resource intensive as it is duplicating lots of the same processes, it involves storing a binary compose inside the container which adds size and is harder to version control and write tests for, the list goes on. Ironically, it also means I can't follow the Docker principle of one process, one container, because I have to keep a compose binary in the container (a compose API would be nice, that would allow the micro service architecture at least) (technically calling it from my Go binary via exec does mean it is pid 1, but you get the idea, it's clunky).

I also looked at alternative libraries, Podman doesn't have a Go Compose implementation though, nerdctl was considered but it doesn't share images without hacking it together and also provides maintenance issues, and recreating the Docker Compose commands via the Docker SDK (which is intended as a library) is prone to even more issues.

In short, I don't want to do it this way, it sucks. But after weighing up all the different options over a long time, right now this is what we seem to be left with.

[maggie44]

• for sake of a comprehensive list, also considered using the Kubernetes yaml format that works through an api for use with kubectl to Docker. But seems Docker Desktop oriented, not really designed for it, and needs enabling as a feature that doesn't seem to be available via the docker SDK so doesn't make for a great user experience.

If anyone comes up with a better alternative it would be great to hear, I'm still feeling like that might be a better way.