logger_LoRaWan

LoRaWAN automatic datalogger

This tutorial will describe the main procedure to develop an automatic connected datalogger to record and send data using the low-power LoRaWan network. The design was tested to record data from an ephemeral weather station on the surface of a glacier in the Swiss Alps. Data collected were air temperature, humidity and pressure as well as precipitation. In this first case, to 4G network was available so that a local LoRa network was developped to receive and store the data to a base station about 5 km below the glacier. A second set-up was installed near the city of Lausanne (Switzerland) were 4G network was available so that data were then transmited online to a remote server.

The architecture of the system is composed of different blocks :

1. The actual arduino-based datalogger with LoRa antenna. It reads data and send them via LoRa RF;
2. A LoRaWAN gateway to receive any LoRa RF signal. It translate the data and send it via 4G or ethernet to a remote server (hosted on the web or your own connected Raspberry Pi). Alternatively, if no internet is available, a local WIFI network (without internet) can be set-up and the data can be pushed to a local server (Raspberry Pi) ;
3. A server to communicate with the LoRaWAN gateway to store the data and provide queries for vizualisation (hosted on the web or your own connected Raspberry Pi).

I also want to acknowledge the following tutorial which helped me get started !

Arduino-based datalogger

We tested two different dataloggers. The first one is a CubeCell Lora Dev-Board. The second is a SODAQ Mbili equipped with a LoRa bee-module.

Hardware required

Links below are examples of hardware providers for Switzerland in 2022. Cost : ~50 to 75 CHF + sensors cost.

• IP67 Outdoor Plastic Enclosure
• Cable glands (M12 or M16)
• 3.7V 2500mAh LiPo battery (with 2 mm JST connector)
• 0.5W Solar Panel 55x70mm (with 2 mm JST connector)
• The sensor you want to plug in. In our case, a typical Tipping Bucket Rain Gauge (see post here) and an air temperature/humidity/pressure protected by a simple radiation shield (see post here).

For the SODAQ Mbili:

• SODAQ Mbili
• LoRa bee-module
• A cheap 4 GB SD card (or even smaller, you'll only need a few MB...)
• Any USB-mini (B) cable for communication

For the CubeCell Lora Dev-Board:

• CubeCell board
• Any USB-Micro (B) cable for communication

Configuration

Firstly you will need to install the arduino IDE and configure it to work with your board. For the Mbili you can check this page. For the CubeCell board, follow this tutorial.

Then download the arduino code. You may need to download a few arduino libraries in the library manager (in Arduino IDE : Sketch -> Include Library -> Manage Library or Add .ZIP library), depending on the sensors you are connecting.

Ephemeral weather station equipped with a CubeCell board enclosed in a simple watertight lunch box. Data are directly sent to a base station (LoRaWAN gateway), located ~5 km below the glacier. It is connected to a rain gauge and an air temperature/humiditiy/pressure protected by a simple radiation shield. The solar pannel can be left inside the box if the plastic is transparent. The station was installed on the Otemma glacier.

LoRaWan gateway

The LoRaWAN gateway is the base station which will receive and decode any Lora RF signal (from all your devices) and transmit it to a remote server. As the LoRa network is becoming the main solution for the internet of Things (IoT), many gateways are beeing installed in urban areas. The Things Network (TTN) is a global collaborative IoT ecosystem which allows to create an open network of LoRaWAN gateways on which you can freely rely. If your project is located inside the range of an existing gateway connected to TTN, you will be able to retrieve the data without the need to install your own gateway. In remote locations tough, you will need your own gateway. If you don't need to set-up your own gateway, you can skip the next part and go to "Configuration of The Things Network"

Installing your own LoRaWAN gateway

Hardware required

Links below are examples of hardware providers for Switzerland in 2022. Cost : ~600 CHF

• Dragino gateway
• 12V Battery ~20 Ah
• Solar panel ~45W
• Solar charge controler
• Connection cables
• Cable glands (M16)
• A waterproof box (we use a pelican case in the field)

Configuration of the gateway

We used an outdoor 4G Dragino gateway. The configuration is detailed in the manual, but here are the main steps. Once the antenna is powered, it should create a WIFI access point to which you can connect in your wifi settings and access the settings using the IP : "10.130.1.1". User Name: root, Password: dragino. You should then configure the LoRa configuration and LoRaWAN configuration (see image below). That's all ! Check in the home page that everything seems to work.

This is the configuration if your antenna is connected to internet. In case, you work in a remote area without 4G access, you can create a local LoRa network which does not connect to The Things Network, but rather directly send the data to a local server (raspberry pi) using MQTT. In this case, you need to configure the gateway to work with the built-in ABP decoder to MQTT mode (in LoRaWAN configuration). Please write to me in case you need help (I will try to do a tutorial later).

The LoRaWAN Gateway (Dragino DLOS8) equipped with a solar system and a raspberry pi to create a local LoRa network in the glacier forefield on the Otemma glacier (Switzerland).

Configuration of The Things Network

In this part, we will see how to add your end-device (the Cubecell or Sodaq logger) to The Thing Network (TTN) to retrieve your data. LoRaWAN uses two different activation mode (the way the device connects to the gateway), called ABP or OTAA. In both cases, the LoRa signal sent by the end-device is encrypted and you will need to enter some device security keys in TTN and in your end-device configuration in order to recognize and read the data, this is called "activation". ABP is simpler but a bit less secure, the end-device will only send data ("blindly") to the gateway but does not receive information in return. OTAA is more secure and after each activation or data sent from the device, the gateway will send back some confirmation message to the device, this is obviously the best procedure, but my experience showed that it is sometimes difficult to receive the return confirmation if you're a bit far from the gateway. More detailed info here. Let's use OTAA activation in the next part.

1. First go to the europe server of TTN and create an account : https://eu1.cloud.thethings.network/console/.
2. Go to application -> +add application. In this application, you will be able to add mutliple end-devices.
3. Go to "Add end-device", and find your device (in our case, we will just do the configuration manually, as in the picture below. You can generate the keys automatically, and then add them to your Arduino code (here for example). Once done, your device should be able to connect to TTN if a gateway is in the range. You should see some activity in the main "Overview" tab of your end-device.
   
   
   
4. Finally, we will need to decode the data sent in bytes to actual numbers. This is done in Application -> payload formatter -> Uplink. Here you will need to check how you coded those information in your arduino code. Using this example, we will decode the temperature, humditiy, pressure sensors as well as rain and battery state. We will use a "Custom javascript formatter" such as this one. Now in the end-dice Live Data tab you should see your decoded payload !
5. Finally, now that we can read the data, we need to store them on a remote server. There are mutliple ways to achieve this. If you want to store your data online, one easy way is to use "The Thing Speak", following this tutorial. Another possibility is to have your own web server. This last part is for more advanced users and is described in the next part. The main idea is to communicate between TTN and your server (in our case a Raspberry pi) using MQTT. In TTN, go to "Application -> Integrations -> MQTT" and write down the "Connection information".

Server

In this last part, I propose to developp a server to store the data and provide an interface for vizualisation or queries using a Raspberry Pi. This server can be used locally without internet access. In this case, the server is connected to the local Access Point of the LoRa Gateway and data are sent using the built-in MQTT mode from the gateway. Alternatively, the server can be connected to internet and will then receive the data from TTN using the MQTT protocol. The main configuration is similar in both cases.

Hardware required

• Raspberry pi 3 model B (other model should work too)
• 16 GB SDHC card

Configuration of the Rapsberry pi

I will not detail the whole process here, but will highlight the steps or tutorial which helped me get there. You will need to following functions :

1. Install Rapbian OS. I usually do this without screen directly is SSH mode (see this tutorial).
2. You may then want to configure the WIFI connection (to access internet or the local access point). Read here.
3. Update the Raspberry Pi. Read here.

Then comes the main part, we will need a series of software which work together: Mosquitto, an MQTT broker which receives the data from TTN, influxDB to store the data, telegraf to transfer the data from Mosquitoo to the database and Grafana an application to vizualise the data and create queries. A few tutorials summarize the Influx, Telegraf, Grafana installation such as this one.

The main steps are :

1. First, let install the MQTT broker Mosquitto, read here.
2. Next, we will install InfluxDB (read here). This is an open-source database which is specifically designed to work fast for time series. Once installed, go on and create a database. Then add a user to allow editing of the database. In SSH command line type :

influx -execute "CREATE USER "telegraf" WITH PASSWORD '<your_password>' WITH ALL PRIVILEGES;"

3. Then install Telegraf, read here. Then, let's configure telegraf:

sudo nano /etc/telegraf/telegraf.conf

and edit the following section :

[[inputs.mqtt_consumer]]
  servers = ["tcp://localhost:1883"]
  qos = 0
  connection_timeout = "30s"
  topics = [ "+/+/data" ]
  client_id = ""
  username = ""
  password = ""
  data_format = "json"

and :

[[inputs.mqtt_consumer]]
  servers = ["tcp://eu1.cloud.thethings.network:1883"]
  qos = 0
  connection_timeout = "30s"
  topics = [ "v3/+/devices/+/up" ]
  client_id = ""
  username = "<check_in_TTN>"
  password = "<check_in_TTN>"
  data_format = "json"

Then save and exit. Run the lines below to check that it is working.

  sudo systemctl reload telegraf.service
  sudo systemctl status telegraf.service

4. Finally, we can install Grafana, read here. Just in case run :

  sudo systemctl enable influxdb grafana-server telegraf
  sudo systemctl start influxdb grafana-server telegraf

Now we can finally access Grafana. In your browser, type : http://your_raspberry_IP:3000, and configure it as in the picture below. The name of your database was defined in InfluxDB, and the user is the name to which you gave ALL PRIVILEGES. Note that you will need to be connected to the same WiFi as your server. If you want to access the data remotely, you will need to use your public IP and create a web server.

Finally, in Grafana, go to Dashboard, create a new dashboard (an empty pannel) and edit the query. For example, for my temperature data I use the following query ("cubecellapp@ttn" is the username in TTN MQTT and "cubecell1" is the end-device name in TTN) :

SELECT distinct("uplink_message_decoded_payload_temperature") FROM "mqtt_consumer" WHERE ("topic" = 'v3/cubecellapp@ttn/devices/cubecell1/up') AND $timeFilter GROUP BY time($__interval) fill(null)

Final results

You can find on this page the final result from a simple weather station using the cubecell board !

Good luck with this long tutorial !