Podman cannot mount /etc/localtime in some containers on ubuntu-20.04

[jennydaman]

Description

Some new warnings appear in output since Podman v3.0.0

time="2021-02-19T10:11:31Z" level=warning msg="Failed to add podman to systemd sandbox cgroup: exec: \"dbus-launch\": executable file not found in $PATH"

Which may or may not be related to an error when trying to mount /etc/localtime into a container

$ podman run -v /etc/localtime:/etc/localtime:ro debian ls /tmp
time="2021-02-19T10:11:31Z" level=warning msg="Failed to add podman to systemd sandbox cgroup: exec: \"dbus-launch\": executable file not found in $PATH"
Error: open `..`: No such file or directory: OCI not found

Not 100% sure if these are bugs in podman itself. If they are, I am not 100% sure how to replicate Github Action's configuration for podman to be able to independently reproduce the errors.

Area for Triage:

Containers

Question, Bug, or Feature?:

Bug

Virtual environments affected

• Ubuntu 16.04
• Ubuntu 18.04
• Ubuntu 20.04
• macOS 10.15
• macOS 11.0
• Windows Server 2016 R2
• Windows Server 2019

Image version

Virtual Environment
  Environment: ubuntu-20.04
  Version: 20210216.1

Expected behavior

Can mount /etc/localtime into containers using podman

Actual behavior

Fails

Error: open `..`: No such file or directory: OCI not found

Repro steps

Example .github/workflows/text.yml

name: test

on:
  push:
    branches: [ master ]
  pull_request:
    branches: [ master ]

jobs:
  test-podman:
    runs-on: ubuntu-20.04
    strategy:
      matrix:
        image: [ 'alpine', 'debian' ]
        engine: [ 'podman', 'docker' ]
    steps:
      - name: Pull
        run: ${{ matrix.engine }} pull ${{ matrix.image }}
      - run: stat /etc/localtime
      - run: ${{ matrix.engine }} run ${{ matrix.image }} ls /tmp
      - run: ${{ matrix.engine }} run -v /etc/localtime:/etc/localtime:ro ${{ matrix.image }} ls /tmp

Result: https://github.com/FNNDSC/chrispile/runs/1934431316

• Works for everything except for (debian, podman).
• Works for (alpine, podman)
• Works for docker

I am unable to narrow down the minimum example any further. The example below works:

podman run -v /etc/localtime:/wow:ro debian ls /tmp

In other tests the facts that /etc/localtime is a symlink owned by root to a file owned by root do not seem to matter.

[srd424]

Looks like containers/crun#593 may have fixed this - it's in crun 0.18, which hasn't made it into the kubic PPA yet.

[srd424]

(Alternatively, runc doesn't seem to have this problem, but that will presumably be more resource intensive.)

[al-cheb]

warning msg is fixed in containers/podman#9353

[al-cheb]

@lsm5, Do you have any ETA to update crun to 0.18 in OPS repo?

[aojea]

in the meantime using runc solves the issue, but this is a temporal fix
kubernetes-sigs/kind#2086
we'll need the new crun 0.18 version #2759 (comment) cc: @rhatdan

[lsm5]

@lsm5, Do you have any ETA to update crun to 0.18 in OPS repo?

Working on it now ...

[lsm5]

Building on OBS atm https://build.opensuse.org/package/show/devel:kubic:libcontainers:stable/crun

Should land on the repos in an hour or less.

[lsm5]

I screwed up the build. My bad, building a new one now.

[lsm5]

building 0.18-2 now https://build.opensuse.org/package/show/devel:kubic:libcontainers:stable/crun

[al-cheb]

The new image with building 0.18-2 has been deployed.