PermissionError: [Errno 13] Permission denied: 'Deployment.md'

[ademariag]

Describe the bug/feature
We are experiencing a problem with Cloud Build and the latest version of Kapitan (latest)

To Reproduce
Steps to reproduce the behavior:

1. Run a step in cloud build with Kapitan compile

Expected behavior
Build fails with [Errno 13] Permission denied: 'Deployment.md'

Screenshots

FETCHSOURCE
...
Operation completed over 1 objects/671.5 KiB.
BUILD
Pulling image: deepmind/kapitan:latest
latest: Pulling from deepmind/kapitan
8f91359f1fff: Pulling fs layer
...
1d02a5aeed01: Pull complete
94c20afc3f23: Pull complete
038c275b129f: Pull complete
Digest: sha256:ecfbefac28fecc3afe087fb099d442e3d6fa04135e40b47c4ea170817f6d680b
Status: Downloaded newer image for deepmind/kapitan:latest
docker.io/deepmind/kapitan:latest
Compiled production-simulations (1.44s)
...
Compiled dev-jobs (4.96s)
Compiled demo-federation (9.61s)
Unknown (Non-Kapitan) Error occured
Traceback (most recent call last):
File "/opt/venv/lib/python3.7/site-packages/kapitan/targets.py", line 106, in compile_targets
shutil.rmtree(compile_path)
File "/usr/local/lib/python3.7/shutil.py", line 494, in rmtree
_rmtree_safe_fd(fd, path, onerror)
File "/usr/local/lib/python3.7/shutil.py", line 432, in _rmtree_safe_fd
_rmtree_safe_fd(dirfd, fullname, onerror)
File "/usr/local/lib/python3.7/shutil.py", line 452, in _rmtree_safe_fd
onerror(os.unlink, fullname, sys.exc_info())
File "/usr/local/lib/python3.7/shutil.py", line 450, in _rmtree_safe_fd
os.unlink(entry.name, dir_fd=topfd)
PermissionError: [Errno 13] Permission denied: 'Deployment.md'


[Errno 13] Permission denied: 'Deployment.md'
ERROR
ERROR: build step 0 "deepmind/kapitan:latest" failed: exit status 1

** If it's a bug (please complete the following information):**

  1 options:
  2   machineType: 'N1_HIGHCPU_8'
  3 substitutions:
  5   _KAPITAN_VERSION: deepmind/kapitan:latest
  6 steps:
  7   - name: '${_KAPITAN_VERSION}'
  8     args: ['compile']
  9     dir: kapitan-templates
 10     id: kapitan-compile

Additional context
Add any other context about the problem here.

[ademariag]

BTW, using the image deepmind/kapitan@sha256:a5a3f85e997b6f96afe1208ddfc05b587c45048da9a5dfdf6b456a3ba5d13f6f of kapitan works fine.

[uberspot]

It seems to crash when trying to delete the ./compiled folder shutil.rmtree(compile_path) (?).
Might be because the user used inside the image is "kapitan" with a userid of 1000.
cc @taharah and @yoshi-1224 as it's highly relevant to the work you've done. :)

[uberspot]

As a quick (and very dirty) solution we could just use root inside the docker images instead of the kapitan user. Otherwise anyone running the Image and mounting a kapitan project directory will have to make sure user "1000" has permissions on it which seems implausible. :/

[uberspot]

I made #375 to add yq to the docker images.
Also @ademariag when I run a locally built test image of kapitan with:
docker run -it --rm -v $(pwd):/src:delegated -u $(id -u) kapitan_test compile -c -p 12 it works fine (?).
Do you test it with -u ? Does the user you run it as have a different uid than 1000 ?

[ademariag]

locally it works fine, confirmed. the issue was only on cloud build

[ramaro]

How about we turn this into a `kapitan_docker.sh` shell script which we'll test and maintain?

…

On Tue, Sep 17, 2019 at 5:09 PM ademariag ***@***.***> wrote: locally it works fine, confirmed. the issue was only on cloud build — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#374>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAEFAAGKT34CWULOW76KKTLQKD6M3ANCNFSM4IXMXFIA> .

[ademariag]

I will try this again with the -u flag.

[ademariag]

I am still having issues with this. Am I the only one using kapitan with cloud build?

cloudbuild.yml

options:
  machineType: 'N1_HIGHCPU_8'
substitutions:
  _KAPITAN_IMAGE: deepmind/kapitan:0.25.0
steps:
  - name: '${_KAPITAN_IMAGE}'
    args: ['compile']
    dir: kapitan-templates
    id: kapitan-compile

error

Unknown (Non-Kapitan) Error occured
Traceback (most recent call last):
File "/opt/venv/lib/python3.7/site-packages/kapitan/targets.py", line 111, in compile_targets
shutil.rmtree(compile_path)
File "/usr/local/lib/python3.7/shutil.py", line 494, in rmtree
_rmtree_safe_fd(fd, path, onerror)
File "/usr/local/lib/python3.7/shutil.py", line 432, in _rmtree_safe_fd
_rmtree_safe_fd(dirfd, fullname, onerror)
File "/usr/local/lib/python3.7/shutil.py", line 432, in _rmtree_safe_fd
_rmtree_safe_fd(dirfd, fullname, onerror)
File "/usr/local/lib/python3.7/shutil.py", line 452, in _rmtree_safe_fd
onerror(os.unlink, fullname, sys.exc_info())
File "/usr/local/lib/python3.7/shutil.py", line 450, in _rmtree_safe_fd
os.unlink(entry.name, dir_fd=topfd)
PermissionError: [Errno 13] Permission denied: 'global-secret.yml'


[Errno 13] Permission denied: 'global-secret.yml'
ERROR
ERROR: build step 0 "deepmind/kapitan:0.25.0" failed: exit status 1

[ademariag]

The issue goes away if I use the deepmind/kapitan:0.25.0-ci image.

[ademariag]

Won't fix I guess. On cloud build the image to use is the CI one.