[BUG] When using the compose api.Service to execute Up, it prompts 'err: no container found for project "hello": not found.'

[feeeei]

Description

I intend to use compose api.Service to create a container, but I receive an error message: "err: no container found for project 'hello': not found."

The expectation is to create successfully, and I anticipate that the RecreateForce parameter will take effect upon repeated execution.

Steps To Reproduce

This is my minimal reproduction code.

package main

import (
	"context"
	"fmt"
	composeTypes "github.com/compose-spec/compose-go/types" // v1.20.1
	"github.com/docker/cli/cli/command" // v24.0.6+incompatible
	"github.com/docker/cli/cli/flags"
	"github.com/docker/compose/v2/pkg/api" // v2.23.1
	"github.com/docker/compose/v2/pkg/compose"
)

func main() {
	cli, err := command.NewDockerCli()
	if err != nil {
		panic(err)
	}
	if err := cli.Initialize(flags.NewClientOptions()); err != nil {
		panic(err)
	}
	compose := compose.NewComposeService(cli)

	project := &composeTypes.Project{
		Name: "hello",
		Services: []composeTypes.ServiceConfig{
			{
				Name:          "hello",
				ContainerName: "hello",
				Image:         "hello-world:latest",
			},
		},
	}

	err = compose.Up(context.Background(), project, api.UpOptions{Create: api.CreateOptions{Recreate: api.RecreateForce}})
	fmt.Println("err:", err)
}

Compose Version

docker-compose version 1.29.2, build 5becea4c
docker-py version: 5.0.0
CPython version: 3.9.0
OpenSSL version: OpenSSL 1.1.1h  22 Sep 2020

Docker Environment

Client:
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc., v0.8.2)
  compose: Docker Compose (Docker Inc., v2.6.1)
  extension: Manages Docker extensions (Docker Inc., v0.2.7)
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc., 0.6.0)
  scan: Docker Scan (Docker Inc., v0.17.0)

Server:
 Containers: 39
  Running: 0
  Paused: 0
  Stopped: 39
 Images: 25
 Server Version: 20.10.17
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 10c12954828e7c7c9b6e0ea9b0c02b01407d3ae1
 runc version: v1.1.2-0-ga916309
 init version: de40ad0
 Security Options:
  seccomp
   Profile: default
  cgroupns
 Kernel Version: 5.10.104-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: x86_64
 CPUs: 6
 Total Memory: 7.773GiB
 Name: docker-desktop
 ID: 3AJ2:3AO4:KVU4:O3GO:IBBQ:DEXR:6G6H:ISQF:CDXO:NQEP:JY6A:TNKX
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  hubproxy.docker.internal:5000
  127.0.0.0/8
 Registry Mirrors:
  https://docker.mirrors.ustc.edu.cn/
 Live Restore Enabled: false

Anything else?

[glours]

Hello @feeeei
If you debug step by step your code, you'll see that Compose isn't able to retrieve your container here and this because the projectFilter(project.Name) is looking for containers having a specific Compose label com.docker.compose.project.
Compose relies on labels being set on containers to track resources, and those labels are stored in the Project structure, you can see an exemple here

Compose isn't build to be use as an API, we don't recommend to programmatically building types.Project, the best option is to use ToProject function which will create this types.Project for you.

[maggie44]

Something like this might be a helpful building block. Will still need to work out how to configure it properly, it seems the naming and services have some role in identifying existing deploys? WIP.

package main

import (
	"context"
	"log/slog"
	"os"

	"github.com/docker/cli/cli/command"
	"github.com/docker/cli/cli/flags"
	cmdCompose "github.com/docker/compose/v2/cmd/compose"
	"github.com/docker/compose/v2/pkg/api"
	"github.com/docker/compose/v2/pkg/compose"
)

func composeFunc() {
	// Create a Docker client
	cli, err := command.NewDockerCli()
	if err != nil {
		slog.Error(err.Error())
		os.Exit(1)
	}

	// Initialize the Docker client with the default options
	err = cli.Initialize(flags.NewClientOptions())
	if err != nil {
		slog.Error(err.Error())
		os.Exit(1)
	}

	// Create the compose API service instance with the Docker cli
	apiService := compose.NewComposeService(cli)

	// Create a default project options struct
	projectOptions := cmdCompose.ProjectOptions{}

	// Turn projectOptions into a project with default values
	projectType, err := projectOptions.ToProject(cli, []string{})
	if err != nil {
		slog.Error(err.Error())
		os.Exit(1)
	}

	// Set the compose file to use
	projectType.ComposeFiles = []string{"./docker-compose-1.yml"}

	// Create a default options struct for the up command
	upOptions := api.UpOptions{}

	// Run the up command
	err = apiService.Up(context.Background(), projectType, upOptions)
	if err != nil {
		slog.Error(err.Error())
		os.Exit(1)
	}
}

[maggie44]

Adding here for reference, an example implementation. Feedback and caveats noticed would be most welcome: https://gist.github.com/maggie44/98d870da0961b61c32f6e55407697f76