Skip to content

A containerized real-time IoT data visualization environment using multiple visualization tools

Notifications You must be signed in to change notification settings

Data-Viz-Labs/dataviz-iot-realtime

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

28 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

IoT Real-Time Data Visualization Practice

A practical environment for visualizing simulated IoT sensor data in real-time using industry-standard tools. This project stores time-series data in TimescaleDB and makes it available through multiple visualization platforms.

Components

  • Data Generator: Python script that simulates IoT sensor data
  • TimescaleDB: Specialized time-series database for efficient storage
  • Grafana: Real-time monitoring and visualization dashboards
  • Metabase: Business intelligence and analytics platform

Quick Start

Requirements

  • Docker/Podman
  • Docker Compose/Podman Compose
  • Make

Starting the Environment

make start

Access Points

Tool URL Default Credentials
Grafana http://localhost:3000 admin / admin
Metabase http://localhost:3001 Set on first access

Database Connection Details

  • Host: timescaledb
  • Port: 5432
  • Database: iotdata
  • Username: iotuser
  • Password: iotpass

Data Model

The simulated IoT data includes:

  • Timestamps
  • Geolocation (within Iberian Peninsula)
  • Environmental readings (temperature, humidity, pressure)
  • Device status and categories
  • Event classifications

Project Structure

.
├── docker-compose.yml    # Service definitions
├── Makefile              # Convenience commands
├── data_generator/       # Python data simulation
└── timescaledb/          # Database configuration

Available Commands

Command Description
make start Start all services
make stop Stop all services
make logs View service logs
make shell-db Access database shell
make clean Remove containers and images
make clean-all Remove all containers, images and volumes

Practice Tasks

  1. Create a real-time monitoring dashboard in Grafana
  2. Build an analytical dashboard in Metabase for historical analysis
  3. Analyze sensor data patterns and anomalies

Troubleshooting

Common Issues and Solutions

Database Connection Problems

If the data generator can't connect to the database:

  1. Check if the TimescaleDB container is running:

    make ps
  2. Use the debug script to test database connectivity:

    docker-compose run data_generator python debug.py
  3. Access the database shell to verify the schema:

    make shell-db
    # Then inside the container:
    psql -U iotuser -d iotdata -c "\dt"

Data Not Appearing in Visualizations

  1. Verify data is being generated:

    make logs data_generator
  2. Check if data exists in the database:

    make shell-db
    # Then inside the container:
    psql -U iotuser -d iotdata -c "SELECT COUNT(*) FROM sensor_data;"
  3. For Grafana issues, verify the data source configuration:

    • Access Grafana at http://localhost:3000
    • Go to Configuration > Data Sources
    • Check the TimescaleDB connection settings

Utility Scripts

The repository includes several utility scripts to help with debugging and setup:

  • bin/debug-db.sh: Provides commands to access and verify the database schema and data
  • data_generator/debug.py: Tests database connectivity and inserts a test device

Logs and Monitoring

To view logs from all services:

make logs

To view logs from a specific service:

docker-compose logs timescaledb

For more detailed database diagnostics:

make shell-db
# Then inside the container:
psql -U iotuser -d iotdata -c "SELECT pg_size_pretty(pg_database_size('iotdata'));"

About

A containerized real-time IoT data visualization environment using multiple visualization tools

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published