Immutability rules issue with character "/" #20102

gaglimax · 2024-03-07T12:38:47Z

Expected behavior and actual behavior:
When I define an immutability rule excluding several repos containing the charactere "/", the rule should work (this is the only rule enabled) :

However, the rule is not working, no tag is immutable.

Steps to reproduce the problem:
In a project, push next images (just the name is important) :

argocd:latest
cloudnative-pg/postgresql:latest
crossplane/crossplane:latest
redis:latest

Then, create the rule above and see that you can remove all artifacts whereas redis:latest should be immutable.

Versions:

harbor version: 2.10.0
docker engine version: N/A
docker-compose version: N/A

MinerYang · 2024-03-19T06:52:03Z

Hi @gaglimax ,
Could you provide the specific details of your immutable rule?

docker exec -it <harbor-database-container> /bin/bash
psql -d registry
select * from  immutable_tag_rule where project_id=<replace_with_real_project_id>;

gaglimax · 2024-03-25T10:02:10Z

Hi,
Here is the result (the first rule is disabled) :

id   | project_id |                                                                                                                                                                        tag_filter                                                                                                                                                                         | disabled |       creation_time        
-------+------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------+----------------------------
 45625 |       3196 | {"id":0,"project_id":3196,"disabled":false,"priority":0,"action":"immutable","template":"immutable_template","tag_selectors":[{"kind":"doublestar","decoration":"matches","pattern":"**"}],"scope_selectors":{"repository":[{"kind":"doublestar","decoration":"repoMatches","pattern":"**"}]}}                                                            | t        | 2024-01-15 15:19:13.791467
 49873 |       3196 | {"id":49873,"project_id":3196,"disabled":false,"priority":0,"action":"immutable","template":"immutable_template","tag_selectors":[{"kind":"doublestar","decoration":"matches","pattern":"**"}],"scope_selectors":{"repository":[{"kind":"doublestar","decoration":"repoExcludes","pattern":"{argocd,crossplane/crossplane,cloudnative-pg/postgresql}"}]}} | f        | 2024-03-07 10:40:06.002116
(2 rows)

MinerYang · 2024-03-27T06:08:53Z

Emmm.. it should work as expected.

Could you also get the result of tags table

psql -d registry

\d tag

select * from tag where artifact_id=(select id from artifact where repository_name='repo/redis');

And the result of api call

curl -k -u 'admin:<pwd>' https://<hostname>/api/v2.0/projects/<project-name>/repositories/redis/artifacts/latest?with_tag=true&with_immutable_status=true

MinerYang · 2024-03-27T07:03:47Z

Did you create the rule via API? Could you try to create one through UI?

gaglimax · 2024-03-27T08:09:27Z

Here it is :

breghr1=> \d tag
                                          Table "public.tag"
    Column     |            Type             | Collation | Nullable |             Default             
---------------+-----------------------------+-----------+----------+---------------------------------
 id            | integer                     |           | not null | nextval('tag_id_seq'::regclass)
 repository_id | integer                     |           | not null | 
 artifact_id   | integer                     |           | not null | 
 name          | character varying(255)      |           | not null | 
 push_time     | timestamp without time zone |           |          | now()
 pull_time     | timestamp without time zone |           |          | 
Indexes:
    "tag_pkey" PRIMARY KEY, btree (id)
    "idx_tag_artifact_id" btree (artifact_id)
    "idx_tag_push_time" btree (push_time)
    "unique_tag" UNIQUE CONSTRAINT, btree (repository_id, name)
Foreign-key constraints:
    "tag_artifact_id_fkey" FOREIGN KEY (artifact_id) REFERENCES artifact(id)

breghr1=> select * from tag where artifact_id=(select id from artifact where repository_name='repo/redis');
   id   | repository_id | artifact_id |     name     |         push_time          | pull_time 
--------+---------------+-------------+--------------+----------------------------+-----------
 257881 |         13330 |      650845 | 7.2.4-alpine | 2024-03-06 10:53:54.485468 | 
(1 row)

$ curl -k -u 'admin' https://<hostname>/api/v2.0/projects/repo/repositories/redis/artifacts/7.2.4-alpine?with_tag=true&with_immutable_status=true
{
    "accessories": null,
    "addition_links": {
        "build_history": {
            "absolute": false,
            "href": "/api/v2.0/projects/repo/repositories/redis/artifacts/sha256:edbb71d676ba3572ce1258255a8ad2971dff7a400fadf0c635cb8d2ebfaa643a/additions/build_history"
        },
        "vulnerabilities": {
            "absolute": false,
            "href": "/api/v2.0/projects/repo/repositories/redis/artifacts/sha256:edbb71d676ba3572ce1258255a8ad2971dff7a400fadf0c635cb8d2ebfaa643a/additions/vulnerabilities"
        }
    },
    "annotations": {
        "org.opencontainers.image.created": "2024-01-09T16:09:57Z",
        "org.opencontainers.image.revision": "b77450d34ae54354f41970fc44bf840353f59ef4",
        "org.opencontainers.image.source": "https://github.com/docker-library/redis.git#b77450d34ae54354f41970fc44bf840353f59ef4:7.2/alpine",
        "org.opencontainers.image.url": "https://hub.docker.com/_/redis",
        "org.opencontainers.image.version": "7.2.4-alpine"
    },
    "digest": "sha256:edbb71d676ba3572ce1258255a8ad2971dff7a400fadf0c635cb8d2ebfaa643a",
    "extra_attrs": {
        "architecture": "amd64",
        "author": "",
        "config": {
            "ArgsEscaped": true,
            "Cmd": [
                "redis-server"
            ],
            "Entrypoint": [
                "docker-entrypoint.sh"
            ],
            "Env": [
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
                "GOSU_VERSION=1.17",
                "REDIS_VERSION=7.2.4",
                "REDIS_DOWNLOAD_URL=http://download.redis.io/releases/redis-7.2.4.tar.gz",
                "REDIS_DOWNLOAD_SHA=8d104c26a154b29fd67d6568b4f375212212ad41e0c2caa3d66480e78dbd3b59"
            ],
            "ExposedPorts": {
                "6379/tcp": {}
            },
            "Volumes": {
                "/data": {}
            },
            "WorkingDir": "/data"
        },
        "created": "2024-01-09T16:09:57Z",
        "os": "linux"
    },
    "icon": "sha256:0048162a053eef4d4ce3fe7518615bef084403614f8bca43b40ae2e762e11e06",
    "id": 650845,
    "labels": null,
    "manifest_media_type": "application/vnd.oci.image.manifest.v1+json",
    "media_type": "application/vnd.oci.image.config.v1+json",
    "project_id": 3196,
    "pull_time": "0001-01-01T00:00:00.000Z",
    "push_time": "2024-03-06T10:53:54.308Z",
    "references": null,
    "repository_id": 13330,
    "size": 17272379,
    "tags": [
        {
            "artifact_id": 650845,
            "id": 257881,
            "immutable": false,
            "name": "7.2.4-alpine",
            "pull_time": "0001-01-01T00:00:00.000Z",
            "push_time": "2024-03-06T10:53:54.485Z",
            "repository_id": 13330
        }
    ],
    "type": "IMAGE"
}

The rule was indeed created by an API call. I just tried to delete it and recreate it though UI, and the result is the same :

gaglimax · 2024-04-30T08:26:56Z

Hi @MinerYang,

Any news about this issue ?

MinerYang · 2024-05-06T05:20:03Z

Hi @gaglimax ,

I can not reproduce the issue accordingly. We may need more contexts to triage.

Could you try to dump the logs while you creating the rules as well as applying the rules
Could you try to delete the deactive rule OR try similar rule on other project/repository.

gaglimax · 2024-05-06T09:35:51Z

Hi,

I tried to reproduce the issue on a staging Harbor instance and I have the same behaviour.
However, I just noticed the issue occurs when there are multiple repos with "/" :

If the rule is : For the repositories excluding foo,bar, tags matching **, it works as expected .
If the rule is : For the repositories excluding editor/foo, tags matching **, it works as expected.
If the rule is : For the repositories excluding editor/foo,editor/bar, tags matching **, it doesn't work as expected.

I tried to tailing logs of all pods while creating/activating the rule, but nothing is logged.

MinerYang · 2024-05-17T09:55:32Z

Any errors message in the harbor-core while you apply this rule? like syntax error in pattern

gaglimax · 2024-05-23T12:30:59Z

No nothing regarding this issue in harbor-core...

github-actions · 2024-07-23T09:03:52Z

This issue is being marked stale due to a period of inactivity. If this issue is still relevant, please comment or remove the stale label. Otherwise, this issue will close in 30 days.

absynth76 · 2024-08-21T12:22:54Z

We do have the same issue, following the error in the core logs:

2024-08-21T12:16:44Z [ERROR] [/lib/http/error.go:56]: {"errors":[{"code":"UNKNOWN","message":"launcher: syntax error in pattern"}]}
/harbor/src/pkg/retention/launcher.go:263, github.com/goharbor/harbor/src/pkg/retention.launcherError
/harbor/src/pkg/retention/launcher.go:167, github.com/goharbor/harbor/src/pkg/retention.(*launcher).Launch
/harbor/src/controller/retention/controller.go:235, github.com/goharbor/harbor/src/controller/retention.(*defaultController).TriggerRetentionExec
/harbor/src/server/v2.0/handler/retention.go:262, github.com/goharbor/harbor/src/server/v2.0/handler.(*retentionAPI).TriggerRetentionExecution
/harbor/src/server/v2.0/restapi/configure_harbor.go:3467, github.com/goharbor/harbor/src/server/v2.0/restapi.HandlerAPI.func187
/harbor/src/server/v2.0/restapi/operations/retention/trigger_retention_execution.go:21, github.com/goharbor/harbor/src/server/v2.0/restapi/operations/retention.TriggerRetentionExecutionHandlerFunc.Handle
/harbor/src/server/v2.0/restapi/operations/retention/trigger_retention_execution.go:71, github.com/goharbor/harbor/src/server/v2.0/restapi/operations/retention.(*TriggerRetentionExecution).ServeHTTP
/harbor/src/server/v2.0/restapi/operations/harbor_api.go:1954, github.com/goharbor/harbor/src/server/v2.0/restapi/operations.(*HarborAPI).handler.func1
/usr/local/go/src/net/http/server.go:2122, net/http.HandlerFunc.ServeHTTP
/usr/local/go/src/net/http/server.go:2122, net/http.HandlerFunc.ServeHTTP
/usr/local/go/src/net/http/server.go:2122, net/http.HandlerFunc.ServeHTTP
/usr/local/go/src/net/http/server.go:2122, net/http.HandlerFunc.ServeHTTP
/usr/local/go/src/net/http/server.go:2122, net/http.HandlerFunc.ServeHTTP
/harbor/src/server/middleware/apiversion/api_version.go:29, github.com/goharbor/harbor/src/server/middleware/apiversion.Middleware.func1.1
/usr/local/go/src/net/http/server.go:2122, net/http.HandlerFunc.ServeHTTP
/harbor/src/server/router/router.go:92, github.com/goharbor/harbor/src/server/router.(*Route).Handler.func1
/harbor/src/vendor/github.com/beego/beego/v2/server/web/router.go:1149, github.com/beego/beego/v2/server/web.(*ControllerRegister).serveHttp
/harbor/src/vendor/github.com/beego/beego/v2/server/web/filter.go:83, github.com/beego/beego/v2/server/web.(*FilterRouter).filter
/harbor/src/vendor/github.com/beego/beego/v2/server/web/router.go:1002, github.com/beego/beego/v2/server/web.(*ControllerRegister).ServeHTTP
/harbor/src/server/middleware/readonly/readonly.go:77, github.com/goharbor/harbor/src/server/middleware/readonly.MiddlewareWithConfig.func1
/harbor/src/server/middleware/middleware.go:57, github.com/goharbor/harbor/src/server/middleware.New.func1.1
/usr/local/go/src/net/http/server.go:2122, net/http.HandlerFunc.ServeHTTP
/harbor/src/server/middleware/security/security.go:75, github.com/goharbor/harbor/src/server/middleware/security.UnauthorizedMiddleware.func1
/harbor/src/server/middleware/middleware.go:57, github.com/goharbor/harbor/src/server/middleware.New.func1.1
/usr/local/go/src/net/http/server.go:2122, net/http.HandlerFunc.ServeHTTP
/harbor/src/server/middleware/security/security.go:62, github.com/goharbor/harbor/src/server/middleware/security.Middleware.func1
/harbor/src/server/middleware/middleware.go:57, github.com/goharbor/harbor/src/server/middleware.New.func1.1
/usr/local/go/src/net/http/server.go:2122, net/http.HandlerFunc.ServeHTTP
/harbor/src/server/middleware/artifactinfo/artifact_info.go:62, github.com/goharbor/harbor/src/server/middleware/artifactinfo.Middleware.func1.1
/usr/local/go/src/net/http/server.go:2122, net/http.HandlerFunc.ServeHTTP
/harbor/src/server/middleware/transaction/transaction.go:60, github.com/goharbor/harbor/src/server/middleware/transaction.Middleware.func1.1
/harbor/src/lib/orm/orm.go:157, github.com/goharbor/harbor/src/lib/orm.WithTransaction.func1
/harbor/src/server/middleware/transaction/transaction.go:69, github.com/goharbor/harbor/src/server/middleware/transaction.Middleware.func1
/harbor/src/server/middleware/middleware.go:57, github.com/goharbor/harbor/src/server/middleware.New.func1.1
/usr/local/go/src/net/http/server.go:2122, net/http.HandlerFunc.ServeHTTP
/harbor/src/server/middleware/notification/notification.go:31, github.com/goharbor/harbor/src/server/middleware/notification.Middleware.func1
/harbor/src/server/middleware/middleware.go:57, github.com/goharbor/harbor/src/server/middleware.New.func1.1
/usr/local/go/src/net/http/server.go:2122, net/http.HandlerFunc.ServeHTTP
/harbor/src/server/middleware/orm/orm.go:54, github.com/goharbor/harbor/src/server/middleware/orm.MiddlewareWithConfig.func1
/harbor/src/server/middleware/middleware.go:57, github.com/goharbor/harbor/src/server/middleware.New.func1.1
/usr/local/go/src/net/http/server.go:2122, net/http.HandlerFunc.ServeHTTP
/harbor/src/server/middleware/csrf/csrf.go:48, github.com/goharbor/harbor/src/server/middleware/csrf.attach.func1
/usr/local/go/src/net/http/server.go:2122, net/http.HandlerFunc.ServeHTTP
/harbor/src/vendor/github.com/gorilla/csrf/csrf.go:297, github.com/gorilla/csrf.(*csrf).ServeHTTP
/harbor/src/server/middleware/csrf/csrf.go:68, github.com/goharbor/harbor/src/server/middleware/csrf.Middleware.func2
/harbor/src/server/middleware/middleware.go:57, github.com/goharbor/harbor/src/server/middleware.New.func1.1

I tried by escaping the "/" using "\" w/o success (no error reported but the corresponding repository is not excluded as expected)

absynth76 · 2024-08-21T12:29:46Z

looks similar issue like mentioned in #14259 but we are on 2.8.1 and the ticket stipulates fix is on 2.2

github-actions · 2024-10-21T09:04:44Z

This issue is being marked stale due to a period of inactivity. If this issue is still relevant, please comment or remove the stale label. Otherwise, this issue will close in 30 days.

github-actions · 2024-11-20T09:06:23Z

This issue was closed because it has been stalled for 30 days with no activity. If this issue is still relevant, please re-open a new issue.

MinerYang self-assigned this Mar 11, 2024

MinerYang added the needs/follow-up label Mar 19, 2024

github-actions bot added the Stale label Jul 23, 2024

github-actions bot removed the Stale label Aug 22, 2024

github-actions bot added the Stale label Oct 21, 2024

github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale Nov 20, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Immutability rules issue with character "/" #20102

Immutability rules issue with character "/" #20102

gaglimax commented Mar 7, 2024

MinerYang commented Mar 19, 2024

gaglimax commented Mar 25, 2024

MinerYang commented Mar 27, 2024

MinerYang commented Mar 27, 2024

gaglimax commented Mar 27, 2024

gaglimax commented Apr 30, 2024

MinerYang commented May 6, 2024

gaglimax commented May 6, 2024

MinerYang commented May 17, 2024

gaglimax commented May 23, 2024

github-actions bot commented Jul 23, 2024

absynth76 commented Aug 21, 2024 •

edited

Loading

absynth76 commented Aug 21, 2024

github-actions bot commented Oct 21, 2024

github-actions bot commented Nov 20, 2024

Immutability rules issue with character "/" #20102

Immutability rules issue with character "/" #20102

Comments

gaglimax commented Mar 7, 2024

MinerYang commented Mar 19, 2024

gaglimax commented Mar 25, 2024

MinerYang commented Mar 27, 2024

MinerYang commented Mar 27, 2024

gaglimax commented Mar 27, 2024

gaglimax commented Apr 30, 2024

MinerYang commented May 6, 2024

gaglimax commented May 6, 2024

MinerYang commented May 17, 2024

gaglimax commented May 23, 2024

github-actions bot commented Jul 23, 2024

absynth76 commented Aug 21, 2024 • edited Loading

absynth76 commented Aug 21, 2024

github-actions bot commented Oct 21, 2024

github-actions bot commented Nov 20, 2024

absynth76 commented Aug 21, 2024 •

edited

Loading