solidcommunity.net servers

Some history solid.community (deprecated) --> solidcommunity.net

update 17 Feb 2025

We migrated solidcommunity.net to pivot. The server is now working as follows:

• it's using 1.1.1.1 and 8.8.8.8 as its nameservers for outgoing traffic. this works for host and for node but for some reason not for ping. We don't know why but the important thing is that node can resolve DNS, which it can, so we'll leave this as it is.
• we don't know why but after a reboot it needs a networking-restart before the networking comes up. Something is apparently broken in the way the boot order is set up, and it's giving up on DHCP before DHCP is successful. For that, we put a script in the root crontab that runs every 2 minutes.
• so far, the memory increase seems to have had the desired effect, and the server no longer reboots every 3 minutes. we think it should probably be possible to run solidcommunity.net on a 4 Gb machine but for now we'll be running it on an 8 Gb machine.

We decided these 3 things are acceptable for the next 6 months or so, and we'll re-evaluate after that. Maybe we should rebuild the droplet from scratch, but for now we'll leave it running as it is.

Howto

• error 500 on https://<pod>/ : this may mean that https://<pod>/.meta is invalid.
  • Goto that link
  • login and delete the content
  • try url https://<pod>/. If it is OK. You MUST add the following content in url https://<pod>/.meta. Replace with your podName. Check twice.

# Root Meta resource for the user account
# Used to discover the account's WebID URI, given the account URI
<https://<your pod>/profile/card#me>
  <http://www.w3.org/ns/solid/terms#account>
  </>.

• fully remove a pod : reclaim a subdomain
• restore a pod folder (or folder in a pod) see backup
• blacklist pod names issue#6
• migrate old contacts group from NSS < v5.6.7 issue#50

Servers Management

Server Host

solidcommunity.net is hosted on a paid plan on DigitalOcean There are 2 volumes :

• /mnt/volume_lon1_01. It’s 100G, and 75% utilized in May 2024 (50% in June 2021)
• /mnt/volume_lon1_03 is used for backup.

solidcommunity.net accounts statistics :

date	accounts nb.
20200117	55670
20210607	57341
20211229	58306
20220603	59030
20230910	59426
20230430	61023
20240226	61870
20240524	62209
20241019	62742

server host lock-out

If it appears that servers are not at all accessible.

• ping for a solidcommunity.net droplet server restart by solid DigitalOcean administrators
• there is no clear identification of the issue that appeared with ubuntu 20 Clearing the buffer/cache regularly seems to avoid the pb

# echo 3 | tee /proc/sys/vm/drop_caches

various

• Open a new port :
  • Open new port in linux Ubuntu https://www.digitalocean.com/community/tutorials/opening-a-port-on-linux with ufw
  • The port must also be open on Digital Ocean console
• Add a newuser with ssh key : [email protected]

  • set the /home/newuser directory. Do not create the password.

    https://www.digitalocean.com/community/questions/setting-up-a-new-user-using-ssh.

  • copy/paste the newuser ssh public key in the /home/newuser/.ssh/authorized_keys

Servers administration

let's encrypt certificates issue#18

• solidcommunity.net is delivered by gandi
• certificate for solidcommunity.net are created with the gandi plugin.
  • /etc/letsencrypt/renewal/solidcommunity.net.conf
  • /etc/letsencrypt/gandi.ini
• certificate update

MANUAL
   # certbot renew --dry-run # and after that run : certbot renew
   # pm2 restart all         # both servers (solidcommunity.net and solidcommunity.net:8443)

AUTO
 - systemd certs renewal every 2 months
   /lib/systemd/system/certbot.timer
 
 - cron
   # restart on monday every 3 weeks at 1am to eventually update the certs that are renewed with systemd every 2 months
   0 1 * * 6 [[ $(("( $(date +%s) - $(date +%s --date=20220606) ) / 86400 % 21")) -eq  0 ]] && pm2 restart all

nginx

• redirect http to https issue#8
• nginx defaults parameter are limiting the unit file size

email recovery

uses :

• [email protected]
• smtp.sengrid.net API key

pm2 : solid-server instances are managed using pm2

• nvm : nvm is used to set the node version

• upgrade node version needs a pm2 reinstall and update

  All pm2 instances are concerned and need to be reinstalled (improvement needed)

  # npm install pm2 -g && pm2 update

• node --experimental-modules:
  • needs to be called by launching pm2 --node-args='--experimental-require-module'
  • or npm -- run

  # pm2 start --name test-solid npm -- run solid start ## `npm -- run solid` calls the use of node --experimental-modules

solidcommunity.net:443 backup

• a root cron job every sunday at 1.00 am the server being CET - 2 hours it should be not heavily used worldwide.

  0 1 * * 0 rsync -a /mnt/volume_lon1_01/solid.community/ /mnt/volume_lon1_03/solidcommunity.net/data --stats > /var/log/solid/$(date +\%Y\%m\%d)-data.log
  0 2 * * 0 rsync -a /mnt/volume_lon1_01/.db/oidc/users /mnt/volume_lon1_03/solidcommunity.net/.db/oidc/ --stats > /var/log/solid/$(date +\%Y\%m\%d)-.db-oidc-users.log

• Summary logs available at /var/log/solid
• rsync backup do not delete files for the time being. It shall be implemented someday with a backup rotation.

solidcommunity.net:443 NSS instance

• solid-config file

  root@solidcommunity:/solid-config  
  root@solidcommunity:~/solid-config# cat config.json
  {
    "root": "/mnt/volume_lon1_01/solid.community",
    "port": "443",
    "serverUri": "https://solidcommunity.net",
    "webid": true,
    "mount": "/",
    "configPath": "./config",
    "configFile": "./config.json",
    "dbPath": "/mnt/volume_lon1_01/.db",
    "sslKey": "/etc/letsencrypt/live/solidcommunity.net/privkey.pem",
    "sslCert": "/etc/letsencrypt/live/solidcommunity.net/fullchain.pem",
    "multiuser": true,
    "corsProxy": "/proxy",
    "redirectHTTPFrom": 80,
    "disablePasswordChecks": false,
    "server": {
      "name": "solidcommunity.net",
      "description": "An experimental community solid server",
      "logo": ""
    },
    "email": {
      "host": "smtp.sendgrid.net",
      "port": "465",
      "sender": "[email protected]",
      "secure": true,
      "auth": {
        "user": "apikey",
        "pass": "XXXXXXXXXX"
      }
    }
  }
  

• config folder : /usr/local/bin/config
• install solid app : /usr/local/bin --> ../lib/node_modules/solid-server/bin/solid

  # cd /usr/local/lib
  # npm install solid-server ## this is a local installation

• pm2 new instance

  # cd /usr/local/bin
  # pm2 --node-args='--experimental-require-module' start solid -- start --config-file ~/solid-config/config.json
  # pm2 save

• pm2 update

  # pm2 restart solid

• storage : due to a harsh migration from solid.community in automn 2020. The implementation is clumsy
  • data location : /mnt/volume_lon1_01/solid.community
  • .db location : /mnt/volume_lon1_01/.db

solidcommunity.net:8443 NSS test-server instance

• config file : `/home/solid/test-solid/config.json` must be created once.

  root@solidcommunity:/home/solid/test-solid# cat config.json
  {
    "root": "/mnt/volume_lon1_01/test-solidcommunity.net/data",
    "port": "8443",
    "serverUri": "https://solidcommunity.net:8443",
    "webid": true,
    "mount": "/",
    "configPath": "./config",
    "configFile": "./config.json",
    "dbPath": "/mnt/volume_lon1_01/test-solidcommunity.net/.db",
    "sslKey": "/etc/letsencrypt/live/solidcommunity.net/privkey.pem",
    "sslCert": "/etc/letsencrypt/live/solidcommunity.net/fullchain.pem",
    "multiuser": true,
    "corsProxy": "/proxy",
    "redirectHTTPFrom": 80,
    "disablePasswordChecks": false,
    "server": {
      "name": "test solidcommunity.net",
      "description": "An experimental community solid server",
      "logo": ""
    },
    "email": {
      "host": "smtp.sendgrid.net",
      "port": "465",
      "sender": "[email protected]",
      "secure": true,
      "auth": {
        "user": "apikey",
        "pass": "XXXXXXXXXX"
      }
    }
  }
  
  

• config folder : /home/solid/test-solid/config always uses default as test
• app

  # cd /home/solid/test-solid
  # git clone https://github.com/solid/node-solid-server
  # npm ci

• new instance

  # pm2 start --name test-solid /home/solid/test-solid/bin/solid -- start
this does not work with node --experimental-modules, you need to 
add --node-args='--experimental-require-module' parameter to pm2
  # pm2 --node-args='--experimental-require-module' start --name test-solid /home/solid/test-solid/bin/solid -- start
or with npm -- run
  # pm2 start --name test-solid npm -- run solid start
  # pm2 save

• update

  # pm2 restart test-solid

• storage : data and .db

root@solidcommunity:/mnt/volume_lon1_01/test-solidcommunity.net# ls -a
.  ..  .db  data

CSS test server

CSS server test on solidcommunity.net

Each new major CSS version imply to update the configuration file and the pm2 bash command file.

For minor versions of CSS or upgrade of mashlib, use minor update.

• CSS source

  We are using https://github.com/SolidOS/css-mashlib as the git source for CSS mashlib recipe and the templates resources

  • cd /home/solid/git clone https://github.com/SolidOS/css-mashlib test-css
  • cd ../test-css

minor updates

- cd /home/solid/test-css
  update `package.json` dependencies
- npm ci
- pm2 ls # to display existing running servers
- pm2 restart css-subdomain
- pm2 restart css-suffix

major update or new installation

• git clone https://github.com/SolidOS/css-mashlib /home/solid/test-css
• mkdir /home/solid/css-config

• cd /home/solid/css-config

  root@solidcommunity:/home/solid/css-config# ls -al
  total 64
  drwxr-xr-x 2 root root 4096 Dec  6 17:33 .
  drwxr-xr-x 5 root root 4096 Feb  9  2023 ..
  -rw-r--r-- 1 root root 2862 Feb  9  2023 config-mashlib.json
  -rw-r--r-- 1 root root 3649 Feb  9  2023 https-mashlib-subdomain-file.json
  -rw-r--r-- 1 root root 3646 Feb  9  2023 https-mashlib-suffix-file.json
  -rw-r--r-- 1 root root  419 Feb  9  2023 override-pod-templates-v5.json
  -rw-r--r-- 1 root root  295 Dec  6 15:59 subdomain-3000-v7.sh
  -rw-r--r-- 1 root root  348 Dec  6 17:25 subdomain-3000-v7test.sh
  -rw-r--r-- 1 root root 3725 Feb  9  2023 subdomain-mashlib-v5.json
  -rw-r--r-- 1 root root 4116 Nov 23 18:41 subdomain-mashlib-v7.json
  -rw-r--r-- 1 root root  289 Nov 24 11:06 suffix-3001-v7.sh
  -rw-r--r-- 1 root root  194 Mar  7  2023 suffix-3001.sh
  -rw-r--r-- 1 root root 3722 Feb  9  2023 suffix-mashlib-v5.json
  -rw-r--r-- 1 root root 4117 Nov 24 11:11 suffix-mashlib-v7.json
  

1. CSS subdomain test server : https://solidcommunity.net:3000

• configuration file : subdomain-mashlib-v7.json

  root@solidcommunity:/home/solid/css-config# cat subdomain-mashlib-v7.json
  {
    "@context": [
      "https://linkedsoftwaredependencies.org/bundles/npm/@solid/community-server/^7.0.0/components/context.jsonld"
    ],
    "import": [
      "css:config/app/init/default.json",
      "css:config/app/main/default.json",
      "css:config/app/variables/default.json",
      "css:config/http/handler/default.json",
      "css:config/http/middleware/default.json",
      "css:config/http/notifications/all.json",
      "css:config/http/server-factory/https.json",
      "css:config/http/static/default.json",
      "css:config/identity/access/public.json",
      "css:config/identity/email/example.json",
      "css:config/identity/handler/default.json",
      "css:config/identity/oidc/default.json",
      "css:config/identity/ownership/token.json",
      "css:config/identity/pod/static.json",
      "css:config/ldp/authentication/dpop-bearer.json",
      "css:config/ldp/authorization/webacl.json",
      "css:config/ldp/handler/default.json",
      "css:config/ldp/metadata-parser/default.json",
      "css:config/ldp/metadata-writer/default.json",
      "css:config/ldp/modes/default.json",
      "css:config/storage/backend/file.json",
      "css:config/storage/key-value/resource-store.json",
      "css:config/storage/location/pod.json",
      "css:config/storage/middleware/default.json",
      "css:config/util/auxiliary/acl.json",
      "css:config/util/identifiers/subdomain.json",
      "css:config/util/logging/winston.json",
      "css:config/util/representation-conversion/default.json",
      "css:config/util/resource-locker/file.json",
      "css:config/util/variables/default.json"
    ],
    "@graph": [
      {
        "comment": "The settings of your email server.",
        "@type": "Override",
        "overrideInstance": {
          "@id": "urn:solid-server:default:EmailSender"
        },
        "overrideParameters": {
          "@type": "BaseEmailSender",
          "senderName": "[email protected]",
          "emailConfig_host": "smtp.sendgrid.net",
          "emailConfig_port": 465,
          "emailConfig_auth_user": "apikey",
          "emailConfig_auth_pass": "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"
        }
      },
      {
        "comment": "The location of the new pod templates folder.",
        "@type": "Override",
        "overrideInstance": {
          "@id": "urn:solid-server:default:PodResourcesGenerator"
        },
        "overrideParameters": {
          "@type": "StaticFolderGenerator",
          "templateFolder": "/home/solid/test-css/templates/pod"
        }
      },
      {
        "comment": "Where the WebID is located in the generated pod, relative to the root.",
        "@type": "Override",
        "overrideInstance": {
          "@id": "urn:solid-server:default:PodCreator"
        },
        "overrideParameters": {
          "@type": "BasePodCreator",
          "relativeWebIdPath": "profile/card#me"
        }
      },
      {
        "comment": [
          "A filesystem-based server with Databrowser as UI.",
          "Derived from config/file-no-setup.json"
        ]
      },
      {
        "comment": "Serve Databrowser as default representation",
        "@id": "urn:solid-server:default:DefaultUiConverter",
        "@type": "ConstantConverter",
        "contentType": "text/html",
        "filePath": "./node_modules/mashlib/dist/databrowser.html",
        "options_container": true,
        "options_document": true,
        "options_minQuality": 1,
        "options_disabledMediaRanges": [
          "image/*",
          "application/pdf"
        ]
      },
      {
        "comment": "Serve Mashlib static files.",
        "@id": "urn:solid-server:default:StaticAssetHandler",
        "@type": "StaticAssetHandler",
        "assets": [
          {
            "@type": "StaticAssetEntry",
            "relativeUrl": "/mash.css",
            "filePath": "./node_modules/mashlib/dist/mash.css"
          },
          {
            "@type": "StaticAssetEntry",
            "relativeUrl": "/mashlib.min.js",
            "filePath": "./node_modules/mashlib/dist/mashlib.min.js"
          },
          {
            "@type": "StaticAssetEntry",
            "relativeUrl": "/mashlib.min.js.map",
            "filePath": "./node_modules/mashlib/dist/mashlib.min.js.map"
          }
        ]
      }
    ]
  }
  

• bash file for pm2 start

root@solidcommunity:/home/solid/css-config# cat subdomain-3000-v7.sh
npx @solid/community-server -c /home/solid/css-config/subdomain-mashlib-v7.json -f /mnt/volume_lon1_01/test-css-subdomain -p 3000 -b https://solidcommunity.net:3000 --httpsKey /etc/letsencrypt/live/solidcommunity.net/privkey.pem --httpsCert /etc/letsencrypt/live/solidcommunity.net/fullchain.pem

• pm2 new instance

- cd /home/solid/test-css
- npm ci
- pm2 del css-subdomain
- pm2 start /home/solid/css-config/subdomain-3000-v7.sh --name css-subdomain

2. CSS suffix test server https://solidcommunity.net:3001

• configuration file : suffix-mashlib-v7.json

  root@solidcommunity:/home/solid/css-config# cat suffix-mashlib-v7.json
  {
    "@context": [
      "https://linkedsoftwaredependencies.org/bundles/npm/@solid/community-server/^7.0.0/components/context.jsonld"
    ],
    "import": [
      "css:config/app/init/static-root.json",
      "css:config/app/main/default.json",
      "css:config/app/variables/default.json",
      "css:config/http/handler/default.json",
      "css:config/http/middleware/default.json",
      "css:config/http/notifications/all.json",
      "css:config/http/server-factory/https.json",
      "css:config/http/static/default.json",
      "css:config/identity/access/public.json",
      "css:config/identity/email/example.json",
      "css:config/identity/handler/default.json",
      "css:config/identity/oidc/default.json",
      "css:config/identity/ownership/token.json",
      "css:config/identity/pod/static.json",
      "css:config/ldp/authentication/dpop-bearer.json",
      "css:config/ldp/authorization/webacl.json",
      "css:config/ldp/handler/default.json",
      "css:config/ldp/metadata-parser/default.json",
      "css:config/ldp/metadata-writer/default.json",
      "css:config/ldp/modes/default.json",
      "css:config/storage/backend/file.json",
      "css:config/storage/key-value/resource-store.json",
      "css:config/storage/location/pod.json",
      "css:config/storage/middleware/default.json",
      "css:config/util/auxiliary/acl.json",
      "css:config/util/identifiers/suffix.json",
      "css:config/util/logging/winston.json",
      "css:config/util/representation-conversion/default.json",
      "css:config/util/resource-locker/file.json",
      "css:config/util/variables/default.json"
    ],
    "@graph": [
      {
        "comment": "The settings of your email server.",
        "@type": "Override",
        "overrideInstance": {
          "@id": "urn:solid-server:default:EmailSender"
        },
        "overrideParameters": {
          "@type": "BaseEmailSender",
          "senderName": "[email protected]",
          "emailConfig_host": "smtp.sendgrid.net",
          "emailConfig_port": 465,
          "emailConfig_auth_user": "apikey",
          "emailConfig_auth_pass": "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"
        }
      },
      {
        "comment": "The location of the new pod templates folder.",
        "@type": "Override",
        "overrideInstance": {
          "@id": "urn:solid-server:default:PodResourcesGenerator"
        },
        "overrideParameters": {
          "@type": "StaticFolderGenerator",
          "templateFolder": "/home/solid/test-css/templates/pod"
        }
      },
      {
        "comment": "Where the WebID is located in the generated pod, relative to the root.",
        "@type": "Override",
        "overrideInstance": {
          "@id": "urn:solid-server:default:PodCreator"
        },
        "overrideParameters": {
          "@type": "BasePodCreator",
          "relativeWebIdPath": "profile/card#me"
        }
      },
      {
        "comment": [
          "A filesystem-based server with Databrowser as UI.",
          "Derived from config/file-no-setup.json"
        ]
      },
      {
        "comment": "Serve Databrowser as default representation",
        "@id": "urn:solid-server:default:DefaultUiConverter",
        "@type": "ConstantConverter",
        "contentType": "text/html",
        "filePath": "./node_modules/mashlib/dist/databrowser.html",
        "options_container": true,
        "options_document": true,
        "options_minQuality": 1,
        "options_disabledMediaRanges": [
          "image/*",
          "application/pdf"
        ]
      },
      {
        "comment": "Serve Mashlib static files.",
        "@id": "urn:solid-server:default:StaticAssetHandler",
        "@type": "StaticAssetHandler",
        "assets": [
          {
            "@type": "StaticAssetEntry",
            "relativeUrl": "/mash.css",
            "filePath": "./node_modules/mashlib/dist/mash.css"
          },
          {
            "@type": "StaticAssetEntry",
            "relativeUrl": "/mashlib.min.js",
            "filePath": "./node_modules/mashlib/dist/mashlib.min.js"
          },
          {
            "@type": "StaticAssetEntry",
            "relativeUrl": "/mashlib.min.js.map",
            "filePath": "./node_modules/mashlib/dist/mashlib.min.js.map"
          }
        ]
      }
    ]
  }
  

• bash file for pm2 start command

root@solidcommunity:/home/solid/css-config# cat suffix-3001-v7.sh
npx @solid/community-server -c /home/solid/css-config/suffix-mashlib-v7.json -f /mnt/volume_lon1_01/test-css-suffix -p 3001 -b https://solidcommunity.net:3001 --httpsKey /etc/letsencrypt/live/solidcommunity.net/privkey.pem --httpsCert /etc/letsencrypt/live/solidcommunity.net/fullchain.pem

• pm2 new instance

- cd /home/solid/test-css
- npm ci
- pm2 del css-suffix
- pm2 start /home/solid/css-config/suffix-3001-v7.sh --name css-suffix