docker.service Failed with result 'start-limit-hit'. #33931

rommik · 2017-07-03T18:00:17Z

Provisioning a new node with docker-machine on Digital Ocean.

New node created OK
Docker installed OK
Starting Docker FAILED.

Docker-machine error output:

Error creating machine: Error running provisioning: ssh command error:
command : sudo systemctl -f start docker
err     : exit status 1
output  : Job for docker.service failed because the control process exited with error code.
See "systemctl status docker.service" and "journalctl -xe" for details.

Docker-machine v 0.10.0, build 76ed2a6
Docker installed on the node version Docker version 17.06.0-ce, build 02c1d87

systemctl status docker.service log

docker.service - Docker Application Container Engine
   Loaded: loaded (/lib/systemd/system/docker.service; enabled; vendor preset: enabled)
  Drop-In: /etc/systemd/system/docker.service.d
           └─10-machine.conf
   Active: failed (Result: start-limit-hit) since Mon 2017-07-03 17:48:03 UTC; 3min 38s ago
     Docs: https://docs.docker.com
  Process: 5803 ExecStart=/usr/bin/docker daemon -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --storage-driver aufs --tlsverify --tlscacert /etc/dock
 Main PID: 5803 (code=exited, status=1/FAILURE)

Jul 03 17:48:03 swarm-node-demo-16 systemd[1]: Failed to start Docker Application Container Engine.
Jul 03 17:48:03 swarm-node-demo-16 systemd[1]: docker.service: Unit entered failed state.
Jul 03 17:48:03 swarm-node-demo-16 systemd[1]: docker.service: Failed with result 'exit-code'.
Jul 03 17:48:03 swarm-node-demo-16 systemd[1]: docker.service: Service hold-off time over, scheduling restart.
Jul 03 17:48:03 swarm-node-demo-16 systemd[1]: Stopped Docker Application Container Engine.
Jul 03 17:48:03 swarm-node-demo-16 systemd[1]: docker.service: Start request repeated too quickly.
Jul 03 17:48:03 swarm-node-demo-16 systemd[1]: Failed to start Docker Application Container Engine.
Jul 03 17:48:03 swarm-node-demo-16 systemd[1]: docker.service: Unit entered failed state.
Jul 03 17:48:03 swarm-node-demo-16 systemd[1]: docker.service: Failed with result 'start-limit-hit'.

journalctl -xe log


Jul 03 17:48:03 swarm-node-demo-16 systemd[1]: docker.service: Service hold-off time over, scheduling restart.
Jul 03 17:48:03 swarm-node-demo-16 systemd[1]: Stopped Docker Application Container Engine.
-- Subject: Unit docker.service has finished shutting down
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
--
-- Unit docker.service has finished shutting down.
Jul 03 17:48:03 swarm-node-demo-16 systemd[1]: Closed Docker Socket for the API.
-- Subject: Unit docker.socket has finished shutting down
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
--
-- Unit docker.socket has finished shutting down.
Jul 03 17:48:03 swarm-node-demo-16 systemd[1]: Stopping Docker Socket for the API.
-- Subject: Unit docker.socket has begun shutting down
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
--
-- Unit docker.socket has begun shutting down.
Jul 03 17:48:03 swarm-node-demo-16 systemd[1]: Starting Docker Socket for the API.
-- Subject: Unit docker.socket has begun start-up
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
--
-- Unit docker.socket has begun starting up.
Jul 03 17:48:03 swarm-node-demo-16 systemd[1]: Listening on Docker Socket for the API.
-- Subject: Unit docker.socket has finished start-up
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
--
-- Unit docker.socket has finished starting up.
--
-- The start-up result is done.
Jul 03 17:48:03 swarm-node-demo-16 systemd[1]: docker.service: Start request repeated too quickly.
Jul 03 17:48:03 swarm-node-demo-16 systemd[1]: Failed to start Docker Application Container Engine.
-- Subject: Unit docker.service has failed
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
--
-- Unit docker.service has failed.
--
-- The result is failed.
Jul 03 17:48:03 swarm-node-demo-16 systemd[1]: docker.socket: Unit entered failed state.
Jul 03 17:48:03 swarm-node-demo-16 systemd[1]: docker.service: Unit entered failed state.
Jul 03 17:48:03 swarm-node-demo-16 systemd[1]: docker.service: Failed with result 'start-limit-hit'.

I have previously installed
v17.03.0-ce
v17.03.1-ce
v17.04.0-ce
v17.05.0-ce
using the same method. This is the first time I install 17.06.0-ce, and first time encounter this issue.

Installation script

docker-machine create --driver digitalocean \
--digitalocean-image  ubuntu-16-10-x64 \
--digitalocean-region nyc1 \
--digitalocean-size 4gb \
--digitalocean-access-token $DOTOKEN $NODE_NAME

UPDATE: July 3, 2017 Downgrading to 17.05.0~~ce-0~~ubuntu-yakkety allows me to restart docker.

The text was updated successfully, but these errors were encountered:

cpuguy83 · 2017-07-03T19:22:01Z

Please upgrade to machine 0.12.1 to work with 17.06.
Unfortunately docker-machine was using a deprecated command in the systemd unit file that causes docker to not start now that it has been removed in 17.06.

You can verify this by tailing the docker logs journaltctl -fu docker.

thaJeztah · 2017-07-04T08:16:23Z

Closing, because this is an issue with docker machine, not docker itself, and should be resolved. Feel free to continue the conversation after I closed

AmplifyDamage · 2017-07-05T02:30:49Z

docker-machine is not installed on my computer, I'm using ubuntu 17.04 and have the same problem after upgrade from docker-ce:amd64 17.05.0~~ce-0~~ubuntu-zesty to docker-ce:amd64 17.06.0~~ce-0~~ubuntu. How can I fix this without downgrade docker-ce?

thaJeztah · 2017-07-05T03:13:56Z

@AmplifyDamage check your systemd unit-file; it should have ExecStart=/usr/bin/dockerd ...... as command, not ExecStart=/usr/bin/docker daemon ......

AmplifyDamage · 2017-07-05T09:06:30Z

@thaJeztah
about this? /lib/systemd/system/docker.service

[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
After=network-online.target docker.socket firewalld.service
Wants=network-online.target
Requires=docker.socket

[Service]
Type=notify
# the default is not to use systemd for cgroups because the delegate issues still
# exists and systemd currently does not support the cgroup feature set required
# for containers run by docker
ExecStart=/usr/bin/dockerd -H fd://
ExecReload=/bin/kill -s HUP $MAINPID
LimitNOFILE=1048576
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNPROC=infinity
LimitCORE=infinity
# Uncomment TasksMax if your systemd version supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process
# restart the docker process if it exits prematurely
Restart=on-failure
StartLimitBurst=3
StartLimitInterval=60s

[Install]
WantedBy=multi-user.target

In a row ExecStart=/usr/bin/dockerd -H fd:// already used 'dockerd'

thaJeztah · 2017-07-05T09:49:51Z

@AmplifyDamage what error message do you get? If you get the same error message, be sure to check you don't have a drop-in / override file somewhere

AmplifyDamage · 2017-07-05T22:30:26Z

@thaJeztah thx a lot. Success. Drop-in file (/etc/systemd/system/docker.service.d/docker.conf) should have ExecStart=/usr/bin/dockerd ...... as command, not ExecStart=/usr/bin/docker daemon ......

rommik · 2017-07-07T14:34:30Z

@thaJeztah I updated my docker-machine as recommended, and I still have this issue. My server is new without prior usage.

lsb_release -a

No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 16.10
Release:        16.10
Codename:       yakkety

Is it possible docker installation is wrong for Ubuntu?

Update confirmed
/etc/systemd/system/docker.service.d/docker.conf contains ExecStart=/usr/bin/docker daemon and not
ExecStart=/usr/bin/dockerd . Why is it the case for a fresh installation?

rommik · 2017-07-07T14:58:04Z

@thaJeztah & @cpuguy83 I think we should re-open this. I don't think it is related to docker-machine, but to Docker (17.06.00-ce) installation of Ubuntu.

thaJeztah · 2017-07-07T15:05:56Z

@rommik did you manually install or through docker machine? these are the only ExecStarts in the code; https://github.com/docker/docker-ce/search?utf8=✓&q=ExecStart&type=

rommik · 2017-07-07T15:11:29Z

@thaJeztah I did it through docker machine and DigitalOcean Provider. I used the latest version of DM as per @cpuguy83 comment.

rommik · 2017-07-07T15:17:28Z

Interesting that on Ubuntu 17.04 the error in the logs is a bit different.

root@swarm-node-demo-17:/etc/systemd/system/docker.service.d# systemctl status docker.service
? docker.service - Docker Application Container Engine
   Loaded: loaded (/lib/systemd/system/docker.service; enabled; vendor preset: enabled)
  Drop-In: /etc/systemd/system/docker.service.d
           └─10-machine.conf
   Active: failed (Result: exit-code) since Fri 2017-07-07 15:14:10 UTC; 1min 59s ago
     Docs: https://docs.docker.com
  Process: 5883 ExecStart=/usr/bin/dockerd daemon -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --storage-driver aufs --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker
 Main PID: 5883 (code=exited, status=1/FAILURE)
      CPU: 32ms

Jul 07 15:14:09 swarm-node-demo-17 dockerd[5883]:                                                 and quit
Jul 07 15:14:09 swarm-node-demo-17 dockerd[5883]: Run 'dockerd COMMAND --help' for more information on a command.
Jul 07 15:14:09 swarm-node-demo-17 systemd[1]: docker.service: Unit entered failed state.
Jul 07 15:14:09 swarm-node-demo-17 systemd[1]: docker.service: Failed with result 'exit-code'.
Jul 07 15:14:10 swarm-node-demo-17 systemd[1]: docker.service: Service hold-off time over, scheduling restart.
Jul 07 15:14:10 swarm-node-demo-17 systemd[1]: Stopped Docker Application Container Engine.
Jul 07 15:14:10 swarm-node-demo-17 systemd[1]: docker.service: Start request repeated too quickly.
Jul 07 15:14:10 swarm-node-demo-17 systemd[1]: Failed to start Docker Application Container Engine.
Jul 07 15:14:10 swarm-node-demo-17 systemd[1]: docker.service: Unit entered failed state.
Jul 07 15:14:10 swarm-node-demo-17 systemd[1]: docker.service: Failed with result 'exit-code'.

no more start-limit-hit error

rommik · 2017-07-07T15:56:05Z

I apologize, it was my fault. I made a typo
You're correct .
The issue is with docker-machine, it uploads 10-machine.conf to

/etc/systemd/system/docker.service.d

and inside it the line is

ExecStart=/usr/bin/docker daemon -H tcp://0.0.0.0:2376 ...

and it should be

ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 ...

@thaJeztah should this issue be referenced on docker-machine forums? What's the best way to communicate here?

thaJeztah · 2017-07-07T23:34:23Z

@rommik yes, please open an issue in the docker-machine issue tracker (https://github.com/docker/machine/issues) if there's no issue yet. It's not something that can be fixed in this repository, so should be tracked there

rogaha · 2018-04-10T03:24:39Z

I'm experiencing the same issue on ARM:

pi@sevenmob:~ $ sudo apt-get install docker-ce
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following extra packages will be installed:
  aufs-tools cgroupfs-mount pigz
The following NEW packages will be installed:
  aufs-tools cgroupfs-mount docker-ce pigz
0 upgraded, 4 newly installed, 0 to remove and 0 not upgraded.
Need to get 23.7 MB/23.8 MB of archives.
After this operation, 117 MB of additional disk space will be used.
Do you want to continue? [Y/n] Y
Get:1 https://download.docker.com/linux/debian/ jessie/stable docker-ce armhf 18.03.0~ce-0~debian [23.7 MB]
Get:2 http://mirrordirector.raspbian.org/raspbian/ jessie/main pigz armhf 2.3.1-2 [48.2 kB]
Fetched 23.7 MB in 22s (1,047 kB/s)
Selecting previously unselected package pigz.
(Reading database ... 43495 files and directories currently installed.)
Preparing to unpack .../pigz_2.3.1-2_armhf.deb ...
Unpacking pigz (2.3.1-2) ...
Selecting previously unselected package aufs-tools.
Preparing to unpack .../aufs-tools_1%3a3.2+20130722-1.1_armhf.deb ...
Unpacking aufs-tools (1:3.2+20130722-1.1) ...
Selecting previously unselected package cgroupfs-mount.
Preparing to unpack .../cgroupfs-mount_1.1_all.deb ...
Unpacking cgroupfs-mount (1.1) ...
Selecting previously unselected package docker-ce.
Preparing to unpack .../docker-ce_18.03.0~ce-0~debian_armhf.deb ...
Unpacking docker-ce (18.03.0~ce-0~debian) ...
Processing triggers for man-db (2.7.5-1~bpo8+1) ...
Processing triggers for systemd (215-17+deb8u7) ...
Setting up pigz (2.3.1-2) ...
Setting up aufs-tools (1:3.2+20130722-1.1) ...
Setting up cgroupfs-mount (1.1) ...
Setting up docker-ce (18.03.0~ce-0~debian) ...
Installing new version of config file /etc/bash_completion.d/docker ...
Installing new version of config file /etc/init.d/docker ...
Job for docker.service failed. See 'systemctl status docker.service' and 'journalctl -xn' for details.
invoke-rc.d: initscript docker, action "start" failed.
dpkg: error processing package docker-ce (--configure):
 subprocess installed post-installation script returned error exit status 1
Processing triggers for libc-bin (2.19-18+deb8u10) ...
Processing triggers for systemd (215-17+deb8u7) ...
Errors were encountered while processing:
 docker-ce
E: Sub-process /usr/bin/dpkg returned an error code (1)
pi@sevenmob:~ $ systemctl status docker.service
● docker.service - Docker Application Container Engine
   Loaded: loaded (/lib/systemd/system/docker.service; enabled)
   Active: failed (Result: start-limit) since Tue 2018-04-10 03:20:53 UTC; 26s ago
     Docs: https://docs.docker.com
 Main PID: 3900 (code=dumped, signal=ILL)
pi@sevenmob:~ $ uname -a
Linux sevenmob 4.9.35+ #1014 Fri Jun 30 14:34:49 BST 2017 armv6l GNU/Linux

/cc @thaJeztah

thaJeztah · 2018-04-10T03:51:18Z

@rogaha anything in the logs?

rogaha · 2018-04-17T03:23:39Z

@thaJeztah there is nothing there:

root@sevenmob:/home/pi# service docker start
Job for docker.service failed. See 'systemctl status docker.service' and 'journalctl -xn' for details.
root@sevenmob:/home/pi# systemctl status docker.service
● docker.service - Docker Application Container Engine
   Loaded: loaded (/lib/systemd/system/docker.service; enabled)
   Active: failed (Result: start-limit) since Tue 2018-04-17 03:22:17 UTC; 35s ago
     Docs: https://docs.docker.com
  Process: 21208 ExecStart=/usr/bin/dockerd -H fd:// (code=dumped, signal=ILL)
 Main PID: 21208 (code=dumped, signal=ILL)

Apr 17 03:22:17 sevenmob systemd[1]: Unit docker.service entered failed state.
Apr 17 03:22:17 sevenmob systemd[1]: docker.service holdoff time over, scheduling restart.
Apr 17 03:22:17 sevenmob systemd[1]: Stopping Docker Application Container Engine...
Apr 17 03:22:17 sevenmob systemd[1]: Starting Docker Application Container Engine...
Apr 17 03:22:17 sevenmob systemd[1]: docker.service start request repeated too quickly, refusing to start.
Apr 17 03:22:17 sevenmob systemd[1]: Failed to start Docker Application Container Engine.
Apr 17 03:22:17 sevenmob systemd[1]: Unit docker.service entered failed state.
Apr 17 03:22:46 sevenmob systemd[1]: Starting Docker Application Container Engine...
Apr 17 03:22:46 sevenmob systemd[1]: docker.service start request repeated too quickly, refusing to start.
Apr 17 03:22:46 sevenmob systemd[1]: Failed to start Docker Application Container Engine.
root@sevenmob:/home/pi#

rogaha · 2018-04-17T03:24:55Z

is docker officially supporting ARM? /cc @andrewhsu

imriss · 2018-06-29T17:17:46Z

In case it could be helpful, I faced a similar behavior (docker.service start request repeated too quickly) when by mistake used incorrect -max-concurrent-uploads instead of its correct form --max-concurrent-uploads in the ExecStart=/usr/bin/dockerd line. The error disappeared after the typo was fixed.

SeWieland · 2018-07-12T13:47:32Z

I had the same error message once because of an empty /etc/docker/daemon.json file.
Delete it if you don't use it!

eloff · 2018-08-31T19:57:06Z

I got this error with a /etc/docker/daemon.json file that seemed to have a hard-coded local network IP. When the laptop was connected to a different network, docker failed to start. Removing this file solved the problem.

jam49 · 2018-11-08T23:13:55Z

ditto ^

N00bSkywalker · 2018-11-19T15:33:55Z

FYI, after upgrading packages on one of our Ubuntu16.04 nodes, we also experienced this behavior ... our ExecStart command within /lib/systemd/system/docker.service looked like this:

ExecStart=/usr/bin/dockerd -H fd:// -H tcp://0.0.0.0:2375 -H unix:///var/run/docker.sock --iptables=true --dns <removed> --dns 8.8.8.8 --dns 8.8.4.4

but Docker refused to start, throwing this error:

Nov 19 09:45:13 ts-build1 systemd[1]: Failed to start Docker Application Container Engine.
Nov 19 09:45:13 ts-build1 systemd[1]: docker.service: Unit entered failed state.
Nov 19 09:45:13 ts-build1 systemd[1]: docker.service: Failed with result 'exit-code'.
Nov 19 09:45:15 ts-build1 systemd[1]: docker.service: Service hold-off time over, scheduling restart.
Nov 19 09:45:15 ts-build1 systemd[1]: Stopped Docker Application Container Engine.
Nov 19 09:45:15 ts-build1 systemd[1]: docker.service: Start request repeated too quickly.
Nov 19 09:45:15 ts-build1 systemd[1]: Failed to start Docker Application Container Engine.
Nov 19 09:45:15 ts-build1 systemd[1]: docker.service: Unit entered failed state.
Nov 19 09:45:15 ts-build1 systemd[1]: docker.service: Failed with result 'start-limit-hit'.

Found removing the -H fd:// parameter allowed Docker to start ...

ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2375 -H unix:///var/run/docker.sock --iptables=true --dns <removed> --dns 8.8.8.8 --dns 8.8.4.4

chuongnh3atgmaildotcom · 2018-11-29T14:39:52Z

Same issue as #33931 (comment) and same fix (removing -H fd:// ).
I'm on Ubuntu 16.04 and edit /etc/systemd/system/docker.service.d/docker.service.conf
from
ExecStart=/usr/bin/dockerd -H fd:// $DOCKER_OPTS
to
ExecStart=/usr/bin/dockerd $DOCKER_OPTS

debragail · 2018-11-30T18:12:20Z

What was the fix for this? Experiencing this on a recent build on Xenial.

cpuguy83 · 2018-11-30T18:13:42Z

@debragail Likely fixing your systemd service config, but check the logs to see whats up... e.g. journalctl -fu docker

danielecr · 2019-10-11T17:01:28Z

for me it does not work, it started to give problem with last update, to docker 19.03
Should I open a newer issue?

sudo systemctl edit docker.service

writing this:

[Service]
ExecStart=
ExecStart=/usr/bin/dockerd -H unix:// -H tcp://127.0.0.1:2375

sudo systemctl daemon-reload
sudo systemctl restart docker.service

Job for docker.service failed because the control process exited with error code.
See "systemctl status docker.service" and "journalctl -xe" for details.

$ journalctl -fu docker
-- Logs begin at Mon 2019-08-26 08:10:52 CEST. --
ott 11 19:03:41 daniele-SATELLITE-L50-B dockerd[31284]: failed to start daemon: error initializing graphdriver: /var/lib/docker contains several valid graphdrivers: devicemapper, overlay2; Please cleanup or explicitly choose storage driver (-s <DRIVER>)
ott 11 19:03:41 daniele-SATELLITE-L50-B systemd[1]: docker.service: Main process exited, code=exited, status=1/FAILURE
ott 11 19:03:41 daniele-SATELLITE-L50-B systemd[1]: docker.service: Failed with result 'exit-code'.
ott 11 19:03:41 daniele-SATELLITE-L50-B systemd[1]: Failed to start Docker Application Container Engine.
ott 11 19:03:43 daniele-SATELLITE-L50-B systemd[1]: docker.service: Service RestartSec=2s expired, scheduling restart.
ott 11 19:03:43 daniele-SATELLITE-L50-B systemd[1]: docker.service: Scheduled restart job, restart counter is at 3.
ott 11 19:03:43 daniele-SATELLITE-L50-B systemd[1]: Stopped Docker Application Container Engine.
ott 11 19:03:43 daniele-SATELLITE-L50-B systemd[1]: docker.service: Start request repeated too quickly.
ott 11 19:03:43 daniele-SATELLITE-L50-B systemd[1]: docker.service: Failed with result 'exit-code'.
ott 11 19:03:43 daniele-SATELLITE-L50-B systemd[1]: Failed to start Docker Application Container Engine.

... I am seeing this

thaJeztah · 2019-10-13T01:20:55Z

@danielecr

error initializing graphdriver: /var/lib/docker contains several valid graphdrivers: devicemapper, overlay2; Please cleanup or explicitly choose storage driver (-s <DRIVER>)

Looks like you have used both devicemapper and overlay2 in the past, and the daemon therefore won't know which storage-driver if should select; that's not a bug, but to prevent it picks the wrong one

danielecr · 2019-10-13T18:02:21Z

@thaJeztah sorry, I was editing docker.json and not daemon.json, I am really embarassed about it.

benaich · 2019-12-31T12:05:24Z

I had the same error because of conflict between /etc/docker/daemon.json and /etc/systemd/system/docker.service.d/options.conf

sudo journalctl --no-hostname --no-pager -b -u docker.service

Dec 31 11:45:11 dockerd[1918]: unable to configure the Docker daemon with file /etc/docker/daemon.json: the following directives are specified both as a flag and in the configuration file: iptables: (from flag: false, from file: false)
Dec 31 11:45:11 systemd[1]: docker.service: Main process exited, code=exited, status=1/FAILURE
Dec 31 11:45:11 systemd[1]: docker.service: Failed with result 'exit-code'.
Dec 31 11:45:11 systemd[1]: Failed to start Docker Application Container Engine.

removing the option in one of the files fixed the problem

xiaozhoz · 2020-01-08T09:45:53Z

Solved. This problem came out after I modified daemon.json. I just removed daemon.json and reboot machine, then docker auto started.

zhxqgithub · 2020-01-09T09:56:30Z

have the same issue after upgrade systemd from 219-30 to systemd-219-67.el7_7.1.x86_64(reboot machine). the docker start error log:

systemd: Starting Docker Application Container Engine...
systemd: Failed at step LIMITS spawning /usr/bin/dockerd: Operation not permitted
systemd: docker.service: main process exited, code=exited, status=205/LIMITS
systemd: Failed to start Docker Application Container Engine.
systemd: Unit docker.service entered failed state.
systemd: docker.service failed.
systemd: docker.service holdoff time over, scheduling restart.
systemd: Stopped Docker Application Container Engine.
systemd: Starting Docker Application Container Engine...
systemd: Failed at step LIMITS spawning /usr/bin/dockerd: Operation not permitted
systemd: docker.service: main process exited, code=exited, status=205/LIMITS
systemd: Failed to start Docker Application Container Engine.
systemd: Unit docker.service entered failed state.
systemd: docker.service failed.

systemctl status docker

 systemctl status docker
● docker.service - Docker Application Container Engine
   Loaded: loaded (/usr/lib/systemd/system/docker.service; enabled; vendor preset: disabled)
   Active: failed (Result: start-limit) since Thu 2020-01-09 16:43:36 CST; 8min ago
     Docs: https://docs.docker.com
  Process: 160921 ExecStart=/usr/bin/dockerd --bip=${FLANNEL_SUBNET} --mtu=${FLANNEL_MTU} (code=exited, status=205/LIMITS)
 Main PID: 160921 (code=exited, status=205/LIMITS)

docker version: 18.06.0-ce

after comment the value in /usr/lib/systemd/system/docker.service ,the docker can start success. why?
#LimitNOFILE=infinity
#LimitNPROC=infinity
#LimitCORE=infinity

chuanchang · 2020-02-12T10:54:06Z

I met the same issue to moby-engine-18.09.8-2.ce.git0dd43dd.fc31.x86_64
and systemd-243.6-1.fc31.x86_64 on Fedora 31.

ashishkarnad · 2020-05-18T10:21:56Z

@AmplifyDamage check your systemd unit-file; it should have ExecStart=/usr/bin/dockerd ...... as command, not ExecStart=/usr/bin/docker daemon ......

Thanks this helped . i was struggling since 2 days. actually i was trying test connection in jenkins but was giving error

Connection refused: /IP:2375
java.net.ConnectException: Connection refused
Caused: io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: /IP:2375
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:714)
at io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:352)

i changed from ExecStart=/usr/bin/docker daemon -H tcp://0.0.0.0:2375 -H unix:///var/run/docker.sock

to
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2375 -H unix:///var/run/docker.sock
then
sudo systemctl daemon-reload
sudo systemctl restart docker

Incase anybody faces same issues . this might help

thaJeztah · 2020-05-18T12:10:21Z

i changed from ExecStart=/usr/bin/docker daemon -H tcp://0.0.0.0:2375 -H unix:///var/run/docker.sock

Be really really careful when using -H tcp://0.0.0.0:2375. That configuration exposes the docker remote API on the public interface of your host, and without access control / not secured with TLS. If the machine you're running on is accessible from the internet, that port gives unrestricted root access on your host. See Docker daemon attack surface in the documentation.

rkdrepo · 2020-09-14T07:05:44Z

I had the same error message once because of an empty /etc/docker/daemon.json file.
Delete it if you don't use it!

I had extra comma at the end of json, causing problem. Its working now.

omidekz · 2020-10-31T10:23:55Z

@AmplifyDamage check your systemd unit-file; it should have ExecStart=/usr/bin/dockerd ...... as command, not ExecStart=/usr/bin/docker daemon ......

i have but get thus error

sereysethy · 2020-12-09T10:46:04Z

I just updated docker to the new version 20.10.0 on Raspbian, and the problem reappears, I have never encountered any problems before. Everything worked fine for the old version 19.03.13.

In the docker.service file, the socket file is pointed to --containerd=/run/containerd/containerd.sock.

Setting up docker-ce (5:20.10.0~3-0~raspbian-buster) ...
Job for docker.service failed because the control process exited with error code.
See "systemctl status docker.service" and "journalctl -xe" for details.
invoke-rc.d: initscript docker, action "restart" failed.
● docker.service - Docker Application Container Engine
   Loaded: loaded (/lib/systemd/system/docker.service; enabled; vendor preset: enabled)
   Active: activating (auto-restart) (Result: exit-code) since Wed 2020-12-09 11:30:20 CET; 34ms ago
     Docs: https://docs.docker.com
  Process: 1155 ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock (code=exited, status=2)
 Main PID: 1155 (code=exited, status=2)
dpkg: error processing package docker-ce (--configure):
 installed docker-ce package post-installation script subprocess returned error exit status 1
Errors were encountered while processing:
 docker-ce
E: Sub-process /usr/bin/dpkg returned an error code (1)

Another output of journald command after the (re)-start of docker service:

journalctl -xe
Dec 09 11:42:34 redacted systemd[1]: docker.service: Scheduled restart job, restart counter is at 3.
-- Subject: Automatic restarting of a unit has been scheduled
-- Defined-By: systemd
-- Support: https://www.debian.org/support
-- 
-- Automatic restarting of the unit docker.service has been scheduled, as the result for
-- the configured Restart= setting for the unit.
Dec 09 11:42:34 redacted systemd[1]: Stopped Docker Application Container Engine.
-- Subject: A stop job for unit docker.service has finished
-- Defined-By: systemd
-- Support: https://www.debian.org/support
-- 
-- A stop job for unit docker.service has finished.
-- 
-- The job identifier is 1141 and the job result is done.
Dec 09 11:42:34 redacted systemd[1]: docker.service: Start request repeated too quickly.
Dec 09 11:42:34 redacted systemd[1]: docker.service: Failed with result 'exit-code'.
-- Subject: Unit failed
-- Defined-By: systemd
-- Support: https://www.debian.org/support
-- 
-- The unit docker.service has entered the 'failed' state with result 'exit-code'.
Dec 09 11:42:34 redacted systemd[1]: Failed to start Docker Application Container Engine.
-- Subject: A start job for unit docker.service has failed
-- Defined-By: systemd
-- Support: https://www.debian.org/support
-- 
-- A start job for unit docker.service has finished with a failure.
-- 
-- The job identifier is 1141 and the job result is failed.
Dec 09 11:42:34 redacted systemd[1]: docker.socket: Failed with result 'service-start-limit-hit'.
-- Subject: Unit failed
-- Defined-By: systemd
-- Support: https://www.debian.org/support
-- 
-- The unit docker.socket has entered the 'failed' state with result 'service-start-limit-hit'.

sereysethy · 2020-12-09T12:26:51Z

Apparently there was some conflicts of old containers that were running and were stopped/removed... my solution was to do a clean install by removing /var/lib/docker folder. And everything seems to be working fine.

thaJeztah · 2020-12-09T12:27:09Z

@sereysethy unfortunately the command that systemd suggest to get details (journalctl -xe) is not giving any useful information most of the time (other than "it failed!"); can you try

journalctl -n 500 -xu docker.service

and/or

journalctl -n 500 -xu containerd.service

(-n 500 gives the last 500 log entries, so adjust accordingly)

thaJeztah · 2020-12-09T12:27:39Z

Ah, sorry, typical race condition; my comment crossed your comment

sereysethy · 2020-12-09T12:28:58Z

Thank you @thaJeztah for your comment, I am not so sure what was the exact reason, but maybe as you said, a race condition. But how it happened? A race condition of what?

thaJeztah · 2020-12-09T19:24:01Z

Oh, ha! Sorry for the confusion; with "race condition" I meant that we posted a comment at the same time 😁

Unfortunately, I can't tell what the reason was for the daemon not starting (without the logs from the time when it failed initially)

umallik133 · 2021-02-23T08:14:08Z

docker.service - Docker Application Container Engine
Loaded: loaded (/lib/systemd/system/docker.service; enabled; vendor preset: enabled)
Active: failed (Result: start-limit-hit) since Tue 2021-02-23 08:10:48 UTC; 23s ago
Docs: https://docs.docker.com
Process: 8807 ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock (code=exited, status=2
Main PID: 8807 (code=exited, status=2)

Feb 23 08:10:45 vacker360 systemd[1]: docker.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
Feb 23 08:10:45 vacker360 systemd[1]: Failed to start Docker Application Container Engine.
Feb 23 08:10:45 vacker360 systemd[1]: docker.service: Unit entered failed state.
Feb 23 08:10:45 vacker360 systemd[1]: docker.service: Failed with result 'exit-code'.
Feb 23 08:10:48 vacker360 systemd[1]: docker.service: Service hold-off time over, scheduling restart.
Feb 23 08:10:48 vacker360 systemd[1]: Stopped Docker Application Container Engine.
Feb 23 08:10:48 vacker360 systemd[1]: docker.service: Start request repeated too quickly.
Feb 23 08:10:48 vacker360 systemd[1]: Failed to start Docker Application Container Engine.
Feb 23 08:10:48 vacker360 systemd[1]: docker.service: Unit entered failed state.
Feb 23 08:10:48 vacker360 systemd[1]: docker.service: Failed with result 'start-limit-hit'.

umallik133 · 2021-02-23T08:14:40Z

I am getting this error and I tried the above mentioned solutions but those did not work

thaJeztah · 2021-02-23T09:14:47Z

@umallik133 Did you check if the logs provide anything useful? See #33931 (comment)

umallik133 · 2021-02-23T09:39:41Z

journalctl -n 500 -xu containerd.service shows this:

Feb 23 09:16:52 vacker360 containerd[13557]: time="2021-02-23T09:16:52.907938992Z" level=info msg="starting containerd" revision=894b81a4b802e4eb2a91d1ce216b8817763c29fb version=1.2.6
Feb 23 09:16:52 vacker360 containerd[13557]: time="2021-02-23T09:16:52.908689244Z" level=info msg="loading plugin "io.containerd.content.v1.content"..." type=io.containerd.content.v1
Feb 23 09:16:52 vacker360 containerd[13557]: time="2021-02-23T09:16:52.908977909Z" level=info msg="loading plugin "io.containerd.snapshotter.v1.btrfs"..." type=io.containerd.snapshotter.v1
Feb 23 09:16:52 vacker360 containerd[13557]: time="2021-02-23T09:16:52.909526554Z" level=warning msg="failed to load plugin io.containerd.snapshotter.v1.btrfs" error="path /var/lib/container
Feb 23 09:16:52 vacker360 containerd[13557]: time="2021-02-23T09:16:52.909788965Z" level=info msg="loading plugin "io.containerd.snapshotter.v1.aufs"..." type=io.containerd.snapshotter.v1
Feb 23 09:16:52 vacker360 containerd[13557]: time="2021-02-23T09:16:52.912585988Z" level=info msg="loading plugin "io.containerd.snapshotter.v1.native"..." type=io.containerd.snapshotter.v1
Feb 23 09:16:52 vacker360 containerd[13557]: time="2021-02-23T09:16:52.912887242Z" level=info msg="loading plugin "io.containerd.snapshotter.v1.overlayfs"..." type=io.containerd.snapshotter.
Feb 23 09:16:52 vacker360 containerd[13557]: time="2021-02-23T09:16:52.913203444Z" level=info msg="loading plugin "io.containerd.snapshotter.v1.zfs"..." type=io.containerd.snapshotter.v1
Feb 23 09:16:52 vacker360 containerd[13557]: time="2021-02-23T09:16:52.913747375Z" level=warning msg="failed to load plugin io.containerd.snapshotter.v1.zfs" error="path /var/lib/containerd/
Feb 23 09:16:52 vacker360 containerd[13557]: time="2021-02-23T09:16:52.913982977Z" level=info msg="loading plugin "io.containerd.metadata.v1.bolt"..." type=io.containerd.metadata.v1

umallik133 · 2021-02-23T09:41:09Z

and journalctl -n 500 -xu docker.service gives this:
Feb 23 09:21:33 vacker360 dockerd[16198]: time="2021-02-23T09:21:33.823569851Z" level=warning msg="Failed getting network for ep 159ab38e4ef704b17a97368fabb9ef69036d9afe03427cd63fe87b507c237
Feb 23 09:21:33 vacker360 dockerd[16198]: time="2021-02-23T09:21:33.823630881Z" level=warning msg="Failed getting network for ep ce40cd7397d908ad5f9d34e1048cd3692e347131e4ff5c67f46a868c96e24
Feb 23 09:21:33 vacker360 dockerd[16198]: time="2021-02-23T09:21:33.823665877Z" level=error msg="Failed to delete sandbox 50ee24f1aeb825d6e83ad0d4413fd5b54e5905cee5120956a213ea1aa167d4c8 whi
Feb 23 09:21:34 vacker360 dockerd[16198]: time="2021-02-23T09:21:34.003461418Z" level=error msg="getNetworkFromStore for nid ba4e1dcd1226e464ba33cf3b7d0475103a7c30115ccc1d3c0d1e5da511122359
Feb 23 09:21:34 vacker360 dockerd[16198]: time="2021-02-23T09:21:34.003933764Z" level=error msg="getNetworkFromStore for nid 91f1001d08984fdab3bc739a0fe235b2577ab5490cf53adff1970c80577afeb9
Feb 23 09:21:34 vacker360 dockerd[16198]: time="2021-02-23T09:21:34.004221751Z" level=error msg="getNetworkFromStore for nid ba4e1dcd1226e464ba33cf3b7d0475103a7c30115ccc1d3c0d1e5da511122359
Feb 23 09:21:34 vacker360 dockerd[16198]: time="2021-02-23T09:21:34.004489746Z" level=error msg="getNetworkFromStore for nid ba4e1dcd1226e464ba33cf3b7d0475103a7c30115ccc1d3c0d1e5da511122359
Feb 23 09:21:34 vacker360 dockerd[16198]: time="2021-02-23T09:21:34.004757222Z" level=error msg="getNetworkFromStore for nid ba4e1dcd1226e464ba33cf3b7d0475103a7c30115ccc1d3c0d1e5da511122359

sereysethy · 2021-02-23T15:28:23Z

Hi, my advise is to try to remove old containers which were corrupted when docker was updated. That was what I did when I encountered that problem. Hope this will help you.

umallik133 · 2021-02-23T15:30:46Z

updated
but by doing so I would lose all my data too

sereysethy · 2021-02-23T15:37:02Z

you saved data in containers? you should use volume or saved it in a mounted folder.

umallik133 · 2021-02-23T15:44:26Z

you saved data in containers? you should use volume or saved it in a mounted folder.

my bad, it is in stored in volume. Just to make it clear, I can simply remove the old containers and use the new one instead right? Thank you for the reply @sereysethy

sereysethy · 2021-02-23T19:06:19Z

Yes you can safely remove stopped containers. In my case I have to actually remove folders of containers in order to get ride of them.

AnrDaemon · 2021-03-26T13:00:50Z

For BTRFS host flesystem, you might need a kernel update.

Jean-Baptiste-Lasselle · 2021-11-07T07:25:58Z

FYI, after upgrading packages on one of our Ubuntu16.04 nodes, we also experienced this behavior ... our ExecStart command within /lib/systemd/system/docker.service looked like this:

ExecStart=/usr/bin/dockerd -H fd:// -H tcp://0.0.0.0:2375 -H unix:///var/run/docker.sock --iptables=true --dns <removed> --dns 8.8.8.8 --dns 8.8.4.4

but Docker refused to start, throwing this error:

Nov 19 09:45:13 ts-build1 systemd[1]: Failed to start Docker Application Container Engine.
Nov 19 09:45:13 ts-build1 systemd[1]: docker.service: Unit entered failed state.
Nov 19 09:45:13 ts-build1 systemd[1]: docker.service: Failed with result 'exit-code'.
Nov 19 09:45:15 ts-build1 systemd[1]: docker.service: Service hold-off time over, scheduling restart.
Nov 19 09:45:15 ts-build1 systemd[1]: Stopped Docker Application Container Engine.
Nov 19 09:45:15 ts-build1 systemd[1]: docker.service: Start request repeated too quickly.
Nov 19 09:45:15 ts-build1 systemd[1]: Failed to start Docker Application Container Engine.
Nov 19 09:45:15 ts-build1 systemd[1]: docker.service: Unit entered failed state.
Nov 19 09:45:15 ts-build1 systemd[1]: docker.service: Failed with result 'start-limit-hit'.

Found removing the -H fd:// parameter allowed Docker to start ...

ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2375 -H unix:///var/run/docker.sock --iptables=true --dns <removed> --dns 8.8.8.8 --dns 8.8.4.4

omg so thk u, it worked for me, first time in years of docker that i met that issue...

thaJeztah · 2021-11-08T10:30:07Z

If you're adding -H tcp://0.0.0.0:2375 to the ExecStart, make sure that's really what you want; see my earlier comment above; as that option publicly exposes the Docker API without any protection, giving anyone who can access your host full root access to your host.

i changed from ExecStart=/usr/bin/docker daemon -H tcp://0.0.0.0:2375 -H unix:///var/run/docker.sock

Be really really careful when using -H tcp://0.0.0.0:2375. That configuration exposes the docker remote API on the public interface of your host, and without access control / not secured with TLS. If the machine you're running on is accessible from the internet, that port gives unrestricted root access on your host. See Docker daemon attack surface in the documentation.

cpuguy83 added the status/more-info-needed label Jul 3, 2017

thaJeztah closed this as completed Jul 4, 2017

milindchawre mentioned this issue Jul 17, 2017

Docker does not start #34104

Closed

SpComb mentioned this issue Jul 17, 2017

Ubuntu kontena-agent package is not compatible with Docker 17.06 kontena/kontena#2588

Closed

docker.service Failed with result 'start-limit-hit'. #33931

docker.service Failed with result 'start-limit-hit'. #33931

Comments

rommik commented Jul 3, 2017 • edited Loading

cpuguy83 commented Jul 3, 2017

thaJeztah commented Jul 4, 2017

AmplifyDamage commented Jul 5, 2017 • edited Loading

thaJeztah commented Jul 5, 2017

AmplifyDamage commented Jul 5, 2017 • edited Loading

thaJeztah commented Jul 5, 2017

AmplifyDamage commented Jul 5, 2017 • edited Loading

rommik commented Jul 7, 2017 • edited Loading

rommik commented Jul 7, 2017

thaJeztah commented Jul 7, 2017

rommik commented Jul 7, 2017

rommik commented Jul 7, 2017

rommik commented Jul 7, 2017 • edited Loading

thaJeztah commented Jul 7, 2017

rogaha commented Apr 10, 2018

thaJeztah commented Apr 10, 2018

rogaha commented Apr 17, 2018

rogaha commented Apr 17, 2018

imriss commented Jun 29, 2018

SeWieland commented Jul 12, 2018

eloff commented Aug 31, 2018

jam49 commented Nov 8, 2018

N00bSkywalker commented Nov 19, 2018

chuongnh3atgmaildotcom commented Nov 29, 2018

debragail commented Nov 30, 2018

cpuguy83 commented Nov 30, 2018

danielecr commented Oct 11, 2019 • edited Loading

thaJeztah commented Oct 13, 2019

danielecr commented Oct 13, 2019

benaich commented Dec 31, 2019 • edited Loading

xiaozhoz commented Jan 8, 2020

zhxqgithub commented Jan 9, 2020

chuanchang commented Feb 12, 2020

ashishkarnad commented May 18, 2020

thaJeztah commented May 18, 2020

rkdrepo commented Sep 14, 2020 • edited Loading

omidekz commented Oct 31, 2020

sereysethy commented Dec 9, 2020

sereysethy commented Dec 9, 2020

thaJeztah commented Dec 9, 2020

thaJeztah commented Dec 9, 2020

sereysethy commented Dec 9, 2020

thaJeztah commented Dec 9, 2020

umallik133 commented Feb 23, 2021

umallik133 commented Feb 23, 2021

thaJeztah commented Feb 23, 2021

umallik133 commented Feb 23, 2021

umallik133 commented Feb 23, 2021 • edited Loading

sereysethy commented Feb 23, 2021

umallik133 commented Feb 23, 2021

sereysethy commented Feb 23, 2021

umallik133 commented Feb 23, 2021 • edited Loading

sereysethy commented Feb 23, 2021

AnrDaemon commented Mar 26, 2021

Jean-Baptiste-Lasselle commented Nov 7, 2021

thaJeztah commented Nov 8, 2021

rommik commented Jul 3, 2017 •

edited

Loading

AmplifyDamage commented Jul 5, 2017 •

edited

Loading

AmplifyDamage commented Jul 5, 2017 •

edited

Loading

AmplifyDamage commented Jul 5, 2017 •

edited

Loading

rommik commented Jul 7, 2017 •

edited

Loading

rommik commented Jul 7, 2017 •

edited

Loading

danielecr commented Oct 11, 2019 •

edited

Loading

benaich commented Dec 31, 2019 •

edited

Loading

rkdrepo commented Sep 14, 2020 •

edited

Loading

umallik133 commented Feb 23, 2021 •

edited

Loading

umallik133 commented Feb 23, 2021 •

edited

Loading