Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Folder extensions #2010

Merged
merged 3 commits into from
Sep 7, 2023
Merged

Conversation

RaulTrombin
Copy link
Member

@RaulTrombin RaulTrombin commented Aug 21, 2023

This PR able blueos-core to access /usr/blueos/extensions folder.

@@ -25,8 +25,8 @@ DELTA_JSON = {
'bind': '/etc/dhcpcd.conf',
'mode': 'rw'
},
"/usr/blueos/userdata": {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suspect this could cause some setups to end up with both /usr/blueos/userdata and /usr/blueos bindings. can you check?

@RaulTrombin
Copy link
Member Author

from the factory SD card with beta 28, made upgrade to this PR.
Funny the shorcut display like this in a first use.
image
But if you create the extension folder or just run some extension that uses it, works fine.
image

@RaulTrombin
Copy link
Member Author

@Williangalvani you are right, it created a second bind for userdata.
Docker inspect:

        },
        "Mounts": [
            {
                "Type": "bind",
                "Source": "/dev",
                "Destination": "/dev",
                "Mode": "rw",
                "RW": true,
                "Propagation": "rprivate"
            },
            {
                "Type": "bind",
                "Source": "/etc/dhcpcd.conf",
                "Destination": "/etc/dhcpcd.conf",
                "Mode": "rw",
                "RW": true,
                "Propagation": "rprivate"
            },
            {
                "Type": "bind",
                "Source": "/etc/resolv.conf.host",
                "Destination": "/etc/resolv.conf.host",
                "Mode": "ro",
                "RW": false,
                "Propagation": "rprivate"
            },
            {
                "Type": "bind",
                "Source": "/usr/blueos/userdata",
                "Destination": "/usr/blueos/userdata",
                "Mode": "rw",
                "RW": true,
                "Propagation": "rprivate"
            },
            {
                "Type": "bind",
                "Source": "/var/run/wpa_supplicant/wlan0",
                "Destination": "/var/run/wpa_supplicant/wlan0",
                "Mode": "rw",
                "RW": true,
                "Propagation": "rprivate"
            },
            {
                "Type": "bind",
                "Source": "/home/pi/.ssh",
                "Destination": "/home/pi/.ssh",
                "Mode": "rw",
                "RW": true,
                "Propagation": "rprivate"
            },
            {
                "Type": "bind",
                "Source": "/run/udev",
                "Destination": "/run/udev",
                "Mode": "ro",
                "RW": false,
                "Propagation": "rprivate"
            },
            {
                "Type": "bind",
                "Source": "/sys",
                "Destination": "/sys",
                "Mode": "ro",
                "RW": false,
                "Propagation": "rprivate"
            },
            {
                "Type": "bind",
                "Source": "/tmp/wpa_playground",
                "Destination": "/tmp/wpa_playground",
                "Mode": "rw",
                "RW": true,
                "Propagation": "rprivate"
            },
            {
                "Type": "bind",
                "Source": "/etc/machine-id",
                "Destination": "/etc/machine-id",
                "Mode": "ro",
                "RW": false,
                "Propagation": "rprivate"
            },
            {
                "Type": "bind",
                "Source": "/var/logs/blueos",
                "Destination": "/var/logs/blueos",
                "Mode": "rw",
                "RW": true,
                "Propagation": "rprivate"
            },
            {
                "Type": "bind",
                "Source": "/var/run/docker.sock",
                "Destination": "/var/run/docker.sock",
                "Mode": "rw",
                "RW": true,
                "Propagation": "rprivate"
            },
            {
                "Type": "bind",
                "Source": "/usr/blueos",
                "Destination": "/usr/blueos",
                "Mode": "rw",
                "RW": true,
                "Propagation": "rprivate"
            },
            {
                "Type": "bind",
                "Source": "/root/.config/blueos",
                "Destination": "/root/.config",
                "Mode": "rw",
                "RW": true,
                "Propagation": "rprivate"
            }
        ],

@CLAassistant
Copy link

CLAassistant commented Aug 23, 2023

CLA assistant check
All committers have signed the CLA.

@RaulTrombin
Copy link
Member Author

@Williangalvani @patrickelectric
EDIT: The idea to bind the entire blueos folder resulted into duoubled binds as shown on the PR.

For now, It was uptaded to just create/use a new folder extensions and related settings.

Just need to validade, waiting for the blueos image.

@ES-Alexander
Copy link
Collaborator

@RaulTrombin is there anything holding this up? It's probably not getting reviews because it's still marked as draft.

I'd be keen to get it merged so we can start documenting the recommended location for extensions to store persistent data :-)

@RaulTrombin
Copy link
Member Author

@ES-Alexander it seems almost fine, just need to re-test it. As it also changes the docker bootstrap image/script.

Did a test this morning, and it forced a reboot aparently,
will re-run it on a clean stable os image now.

@RaulTrombin
Copy link
Member Author

Here I used a fresh stable image, then updated to this PR image.
It rebooted right after the update, need to check if this is expected.

Other than that, it's working.

image

pi@blueos:~ $ docker ps -a
CONTAINER ID   IMAGE                                 
[system_logs (10).zip](https://github.com/bluerobotics/BlueOS/files/12537439/system_logs.10.zip)
[system_logs (10).zip](https://github.com/bluerobotics/BlueOS/files/12537452/system_logs.10.zip)
       COMMAND                  CREATED          STATUS              PORTS     NAMES
6f6de7803899   raulelektron/blueos-core:folder-extensions   "/bin/sh -c '/bin/ba…"   58 seconds ago   Up 57 seconds                 blueos-core
56490fee8161   bluerobotics/blueos-bootstrap:1.1.0          "/bin/sh -c /main.py"    7 days ago       Up About a minute             blueos-bootstrap
pi@blueos:~ $ docker inspect 6f6
[
    {
        "Id": "6f6de780389912c29d1974846ada393b86c2d5f49326434d3fb613069db91805",
        "Created": "2023-09-06T12:42:30.529526225Z",
        "Path": "/bin/sh",
        "Args": [
            "-c",
            "/bin/bash -i /usr/bin/start-blueos-core && sleep infinity"
        ],
        "State": {
            "Status": "running",
            "Running": true,
            "Paused": false,
            "Restarting": false,
            "OOMKilled": false,
            "Dead": false,
            "Pid": 1084,
            "ExitCode": 0,
            "Error": "",
            "StartedAt": "2023-09-06T12:42:31.437352521Z",
            "FinishedAt": "0001-01-01T00:00:00Z"
        },
        "Image": "sha256:9d6f80dddf81ea3fd522a34302dd815f363b2798e73003ccfb300929ddf2cb96",
        "ResolvConfPath": "/var/lib/docker/containers/6f6de780389912c29d1974846ada393b86c2d5f49326434d3fb613069db91805/resolv.conf",
        "HostnamePath": "/var/lib/docker/containers/6f6de780389912c29d1974846ada393b86c2d5f49326434d3fb613069db91805/hostname",
        "HostsPath": "/var/lib/docker/containers/6f6de780389912c29d1974846ada393b86c2d5f49326434d3fb613069db91805/hosts",
        "LogPath": "/var/lib/docker/containers/6f6de780389912c29d1974846ada393b86c2d5f49326434d3fb613069db91805/6f6de780389912c29d1974846ada393b86c2d5f49326434d3fb613069db91805-json.log",
        "Name": "/blueos-core",
        "RestartCount": 0,
        "Driver": "overlay2",
        "Platform": "linux",
        "MountLabel": "",
        "ProcessLabel": "",
        "AppArmorProfile": "",
        "ExecIDs": null,
        "HostConfig": {
            "Binds": [
                "/dev/:/dev/:rw",
                "/etc/dhcpcd.conf:/etc/dhcpcd.conf:rw",
                "/etc/machine-id:/etc/machine-id:ro",
                "/etc/resolv.conf.host:/etc/resolv.conf.host:ro",
                "/home/pi/.ssh:/home/pi/.ssh:rw",
                "/run/udev:/run/udev:ro",
                "/sys/:/sys/:ro",
                "/tmp/wpa_playground:/tmp/wpa_playground:rw",
                "/usr/blueos/extensions:/usr/blueos/extensions:rw",
                "/usr/blueos/userdata:/usr/blueos/userdata:rw",
                "/var/logs/blueos:/var/logs/blueos:rw",
                "/var/run/docker.sock:/var/run/docker.sock:rw",
                "/var/run/wpa_supplicant/wlan0:/var/run/wpa_supplicant/wlan0:rw",
                "/root/.config/blueos:/root/.config:rw"
            ],
            "ContainerIDFile": "",
            "LogConfig": {
                "Type": "json-file",
                "Config": {}
            },
            "NetworkMode": "host",
            "PortBindings": null,
            "RestartPolicy": {
                "Name": "",
                "MaximumRetryCount": 0
            },
            "AutoRemove": false,
            "VolumeDriver": "",
            "VolumesFrom": null,
            "ConsoleSize": [
                0,
                0
            ],
            "CapAdd": null,
            "CapDrop": null,
            "CgroupnsMode": "private",
            "Dns": null,
            "DnsOptions": null,
            "DnsSearch": null,
            "ExtraHosts": null,
            "GroupAdd": null,
            "IpcMode": "private",
            "Cgroup": "",
            "Links": null,
            "OomScoreAdj": 0,
            "PidMode": "",
            "Privileged": true,
            "PublishAllPorts": false,
            "ReadonlyRootfs": false,
            "SecurityOpt": [
                "label=disable"
            ],
            "UTSMode": "",
            "UsernsMode": "",
            "ShmSize": 67108864,
            "Runtime": "runc",
            "Isolation": "",
            "CpuShares": 0,
            "Memory": 0,
            "NanoCpus": 0,
            "CgroupParent": "",
            "BlkioWeight": 0,
            "BlkioWeightDevice": null,
            "BlkioDeviceReadBps": null,
            "BlkioDeviceWriteBps": null,
            "BlkioDeviceReadIOps": null,
            "BlkioDeviceWriteIOps": null,
            "CpuPeriod": 0,
            "CpuQuota": 0,
            "CpuRealtimePeriod": 0,
            "CpuRealtimeRuntime": 0,
            "CpusetCpus": "",
            "CpusetMems": "",
            "Devices": null,
            "DeviceCgroupRules": null,
            "DeviceRequests": null,
            "MemoryReservation": 0,
            "MemorySwap": 0,
            "MemorySwappiness": null,
            "OomKillDisable": null,
            "PidsLimit": null,
            "Ulimits": null,
            "CpuCount": 0,
            "CpuPercent": 0,
            "IOMaximumIOps": 0,
            "IOMaximumBandwidth": 0,
            "MaskedPaths": null,
            "ReadonlyPaths": null
        },
        "GraphDriver": {
            "Data": {
                "LowerDir": "/var/lib/docker/overlay2/e29f86b763e68a344bb5719e4569145f728a05595acbee7e7e9b7d478ad09fbc-init/diff:/var/lib/docker/overlay2/2e751d428af3324cedc850d7f88094681238bb1c795466aa7ebbe4e95bbf8795/diff:/var/lib/docker/overlay2/2a7779114d3f51e98d17e69aea30f4b5457a66349ffb1b459bfc68255529c5a8/diff:/var/lib/docker/overlay2/a35ae751cf311faaad4890a0f98bcac2e54129325b65170d84159026c1a7d25d/diff:/var/lib/docker/overlay2/50058bd49a56df0433ddbdeb7eebd81a6ddf7de421348a9248285e7c11e7c04d/diff:/var/lib/docker/overlay2/61332fce6a1e330ce644f395e54f19241aa8eb85760429b0ea1e9fb3c65b1e64/diff:/var/lib/docker/overlay2/c42c963bdf778bda6866458a33c9306d5398f130f2d630fbadc5097c5886280a/diff:/var/lib/docker/overlay2/7ffd9eaa28ba1eae289fad416fa58b8dda1276e9ceb076d3e45928b291de225e/diff:/var/lib/docker/overlay2/77670c8c40869e26677af3392a5e537fd10532628147c79f21b089ceddc51879/diff:/var/lib/docker/overlay2/4a2cc12a8a802c153cf05afbdf66cd6f6d954401d6f500bc90a5ee540bb2430c/diff:/var/lib/docker/overlay2/b8f8a0132ecd8adbfddafd3f200b2cd408c6216d254d4541a3b1ec48e5a7cefe/diff:/var/lib/docker/overlay2/04252df75d3177b85b72bfd2de062fac78e34213f0f9734f9102db15a22c1c2f/diff:/var/lib/docker/overlay2/a9fdaca07b6a1c2f93f8923360a1eb40b744590bbca3bbdb2e7f218ad6beaa1e/diff:/var/lib/docker/overlay2/942595784eb8fa76d8bb9cdaf8b38c000b94bf8e485f99651f6c210e57972166/diff:/var/lib/docker/overlay2/9a95d578d9b56acea7dfb6a41c73242646be8017d46e5837e19b222e34d58562/diff:/var/lib/docker/overlay2/a6d214c587d2ed59ee2cd205958f7a9aa10d0fa571658e33b2bc37badd9874d1/diff:/var/lib/docker/overlay2/166af464ea1890db988bb603618baa9499fed34e5e01b0fa91e8c0b4f84ac882/diff:/var/lib/docker/overlay2/91104151f491b4c9be3fb0c7467d15619afcdf75e061300782bace9a95e7f59e/diff:/var/lib/docker/overlay2/4abaaae0601efec9a3e7b1e1a5348681d256e582312358430dcc1c51e2bb885f/diff:/var/lib/docker/overlay2/b6c66f794a48b5885fa41a65a7599b0462d4dea15af8cf4c3f3319ecc0e0fb3f/diff:/var/lib/docker/overlay2/683a9f7d99460e089ea8c91c7d126007502cccc2b5597cf633e51e1d13e6fe00/diff:/var/lib/docker/overlay2/fe846fa550e1b4cfffddab30ded35db578078f31d3ebc32613e0efa7e165e4d3/diff:/var/lib/docker/overlay2/03de4058f6b52d7a2da6db628de9bb52e4d3d7f0eaa2c4825492cdb949b5a301/diff:/var/lib/docker/overlay2/fb6c79786895a6c8d07f3079057ca59e6f767831d471d130f5060734f7123112/diff:/var/lib/docker/overlay2/eb30a4b000a335b241b31052ebe63cba1bb026a3839a3114db731e097d861cff/diff:/var/lib/docker/overlay2/b484ae0234ec54a3086a57267df13587235d45029112d737e0f8765f6bb358ad/diff:/var/lib/docker/overlay2/f390a3c173b74a98bdead466fb697c6755a7ef16616e2e8b8a8ea982adb837df/diff:/var/lib/docker/overlay2/6d3dd6d4539826ad5ccfbd43cd1d69cf4206d1dccf9bb5715cf85d45edaddc4c/diff:/var/lib/docker/overlay2/f7588953ece7fff044e2defcc7efc3aea3ce2eeae14e8277159a06a3a2c13dfc/diff:/var/lib/docker/overlay2/e8c75f77b2b7a925097e16ee8f7d3557cb1a688d646009214b4eeb8ddab1de59/diff:/var/lib/docker/overlay2/8f248dfb955e3b6e6b1804b4931dfd2f26f1d995b92ebf5534a1e892054782e7/diff:/var/lib/docker/overlay2/02d5217f5cafcb155fb8b45bb70e0866ded1359df69a1ce3041e6ac3331b4095/diff:/var/lib/docker/overlay2/00a3e07846a99d856a2c6be34c608d6364558f619330e030f6ef27d91efb556d/diff:/var/lib/docker/overlay2/514c3afb64e2f3bb923527d30c28ae8d7be63c42626fb96ac6be2eb58bce4ba3/diff:/var/lib/docker/overlay2/ccd8794e053a934d857c322e350f0b30c0b1f6dfc8b922fa2da08adff56db656/diff:/var/lib/docker/overlay2/01d07a4e42b84801c7049277c7d9a4aab15ccc3859f683b73156a6812e65eccc/diff:/var/lib/docker/overlay2/fc66f843547543fe6c812d321466e1141e4366c895b5fd1a47cd3bc36fe49402/diff:/var/lib/docker/overlay2/5b128a6fd523b15679577a7997ed509937c9dfe333d6dd6a26256ab938e6a107/diff",
                "MergedDir": "/var/lib/docker/overlay2/e29f86b763e68a344bb5719e4569145f728a05595acbee7e7e9b7d478ad09fbc/merged",
                "UpperDir": "/var/lib/docker/overlay2/e29f86b763e68a344bb5719e4569145f728a05595acbee7e7e9b7d478ad09fbc/diff",
                "WorkDir": "/var/lib/docker/overlay2/e29f86b763e68a344bb5719e4569145f728a05595acbee7e7e9b7d478ad09fbc/work"
            },
            "Name": "overlay2"
        },
        "Mounts": [
            {
                "Type": "bind",
                "Source": "/dev",
                "Destination": "/dev",
                "Mode": "rw",
                "RW": true,
                "Propagation": "rprivate"
            },
            {
                "Type": "bind",
                "Source": "/etc/dhcpcd.conf",
                "Destination": "/etc/dhcpcd.conf",
                "Mode": "rw",
                "RW": true,
                "Propagation": "rprivate"
            },
            {
                "Type": "bind",
                "Source": "/etc/machine-id",
                "Destination": "/etc/machine-id",
                "Mode": "ro",
                "RW": false,
                "Propagation": "rprivate"
            },
            {
                "Type": "bind",
                "Source": "/etc/resolv.conf.host",
                "Destination": "/etc/resolv.conf.host",
                "Mode": "ro",
                "RW": false,
                "Propagation": "rprivate"
            },
            {
                "Type": "bind",
                "Source": "/var/run/docker.sock",
                "Destination": "/var/run/docker.sock",
                "Mode": "rw",
                "RW": true,
                "Propagation": "rprivate"
            },
            {
                "Type": "bind",
                "Source": "/usr/blueos/extensions",
                "Destination": "/usr/blueos/extensions",
                "Mode": "rw",
                "RW": true,
                "Propagation": "rprivate"
            },
            {
                "Type": "bind",
                "Source": "/usr/blueos/userdata",
                "Destination": "/usr/blueos/userdata",
                "Mode": "rw",
                "RW": true,
                "Propagation": "rprivate"
            },
            {
                "Type": "bind",
                "Source": "/var/run/wpa_supplicant/wlan0",
                "Destination": "/var/run/wpa_supplicant/wlan0",
                "Mode": "rw",
                "RW": true,
                "Propagation": "rprivate"
            },
            {
                "Type": "bind",
                "Source": "/home/pi/.ssh",
                "Destination": "/home/pi/.ssh",
                "Mode": "rw",
                "RW": true,
                "Propagation": "rprivate"
            },
            {
                "Type": "bind",
                "Source": "/run/udev",
                "Destination": "/run/udev",
                "Mode": "ro",
                "RW": false,
                "Propagation": "rprivate"
            },
            {
                "Type": "bind",
                "Source": "/sys",
                "Destination": "/sys",
                "Mode": "ro",
                "RW": false,
                "Propagation": "rprivate"
            },
            {
                "Type": "bind",
                "Source": "/tmp/wpa_playground",
                "Destination": "/tmp/wpa_playground",
                "Mode": "rw",
                "RW": true,
                "Propagation": "rprivate"
            },
            {
                "Type": "bind",
                "Source": "/var/logs/blueos",
                "Destination": "/var/logs/blueos",
                "Mode": "rw",
                "RW": true,
                "Propagation": "rprivate"
            },
            {
                "Type": "bind",
                "Source": "/root/.config/blueos",
                "Destination": "/root/.config",
                "Mode": "rw",
                "RW": true,
                "Propagation": "rprivate"
            }
        ],
        "Config": {
            "Hostname": "blueos",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
                "PATH=/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
                "LANG=C.UTF-8",
                "GPG_KEY=E3FF2839C048B25C084DEBE9B26995E310250568",
                "PYTHON_VERSION=3.9.17",
                "PYTHON_PIP_VERSION=23.0.1",
                "PYTHON_SETUPTOOLS_VERSION=58.1.0",
                "PYTHON_GET_PIP_URL=https://github.com/pypa/get-pip/raw/9af82b715db434abb94a0a6f3569f43e72157346/public/get-pip.py",
                "PYTHON_GET_PIP_SHA256=45a2bb8bf2bb5eff16fdd00faef6f29731831c7c59bd9fc2bf1f3bed511ff1fe"
            ],
            "Cmd": [
                "/bin/sh",
                "-c",
                "/bin/bash -i /usr/bin/start-blueos-core && sleep infinity"
            ],
            "Image": "raulelektron/blueos-core:folder-extensions",
            "Volumes": {
                "/dev/": {},
                "/etc/dhcpcd.conf": {},
                "/etc/machine-id": {},
                "/etc/resolv.conf.host": {},
                "/home/pi/.ssh": {},
                "/root/.config": {},
                "/run/udev": {},
                "/sys/": {},
                "/tmp/wpa_playground": {},
                "/usr/blueos/extensions": {},
                "/usr/blueos/userdata": {},
                "/var/logs/blueos": {},
                "/var/run/docker.sock": {},
                "/var/run/wpa_supplicant/wlan0": {}
            },
            "WorkingDir": "",
            "Entrypoint": null,
            "OnBuild": null,
            "Labels": {}
        },
        "NetworkSettings": {
            "Bridge": "",
            "SandboxID": "671aa34cd9529462dbf980ddd942011ae4092f47a4f608da25b6f07fba66a921",
            "HairpinMode": false,
            "LinkLocalIPv6Address": "",
            "LinkLocalIPv6PrefixLen": 0,
            "Ports": {},
            "SandboxKey": "/var/run/docker/netns/default",
            "SecondaryIPAddresses": null,
            "SecondaryIPv6Addresses": null,
            "EndpointID": "",
            "Gateway": "",
            "GlobalIPv6Address": "",
            "GlobalIPv6PrefixLen": 0,
            "IPAddress": "",
            "IPPrefixLen": 0,
            "IPv6Gateway": "",
            "MacAddress": "",
            "Networks": {
                "host": {
                    "IPAMConfig": null,
                    "Links": null,
                    "Aliases": null,
                    "NetworkID": "9b36d3629e43ac85fde7469e77ed17d29f2cfc76999c5fd94afc4b89db918305",
                    "EndpointID": "85174ca72f9783de5b985aadc060d7bee70b5ac600257ebb3be7397a62e742c2",
                    "Gateway": "",
                    "IPAddress": "",
                    "IPPrefixLen": 0,
                    "IPv6Gateway": "",
                    "GlobalIPv6Address": "",
                    "GlobalIPv6PrefixLen": 0,
                    "MacAddress": "",
                    "DriverOpts": null
                }
            }
        }
    }
]

@RaulTrombin RaulTrombin marked this pull request as ready for review September 6, 2023 13:06
@joaoantoniocardoso
Copy link
Member

@RaulTrombin

It rebooted right after the update, need to check if this is expected.

This is expected: any modification to the DELTA_JSON will trigger a reboot when the upgrade takes place because the running bootstrap/startup.json would require a patch to match the new configuration. If correctly configured, this shouldn't happen in a fresh image. 👍

@Williangalvani
Copy link
Member

We should probably make it restart only the container, not the host computer, though..
out of scope for this PR, anyway

@patrickelectric patrickelectric merged commit f5d4180 into bluerobotics:master Sep 7, 2023
@ES-Alexander ES-Alexander added the docs-needed Change needs to be documented label Sep 7, 2023
@ES-Alexander ES-Alexander added docs-complete Change documentation has been completed and removed docs-needed Change needs to be documented labels Apr 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
docs-complete Change documentation has been completed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

bug: [File-Manager] : User can't see new directory made from an extension
6 participants