Building a Real-Time IoT Dashboard: A Freelance Journey

Introduction

From July 2023 to October 2023, I embarked on a freelance project to develop a real-time IoT dashboard for a production house. This dashboard was designed to manage over 500 IoT devices per user, significantly enhancing operational efficiency and asset management. This article delves into the technical aspects of the project, the tools and technologies used, and my role in bringing this project to life.

Project Overview

Key Achievements

• Enhanced Operational Efficiency: By creating a real-time IoT dashboard, I improved asset management by 35% and reduced asset tracking time by 40%. This was achieved through detailed data visualization and real-time updates, allowing users to monitor and manage their assets more effectively.
• User Engagement: I established a secure admin login and user management system with referral link generation. This not only enhanced system security but also increased user engagement by 20%, as it streamlined the process of adding new users and managing existing ones.
• Database Optimization: I architected and managed a SQL database schema, which improved data retrieval efficiency by 12.3%. This optimization was crucial for handling the large volume of data generated by the IoT devices and ensuring quick access to this data.
• Real-Time Updates: By integrating React and Recoil with backend APIs, I ensured seamless data flow and real-time updates, reducing latency by 15%. This integration was key to providing users with up-to-date information on their assets.
• Standardized Development Environments: I directed a team in setting up development environments using Docker. This standardization reduced setup times by 35% and ensured consistency across local and development testing stages.

Tech Stack

Frontend

• React: Used for building the user interface, React allowed for the creation of dynamic and responsive components that provided a smooth user experience.
• Recoil: Employed for state management, Recoil facilitated the handling of complex state logic and ensured that the UI remained consistent with the underlying data.
• Figma: Utilized for designing the user interface, Figma enabled the creation of visually appealing and user-friendly designs that were easy to implement in React.

Backend

• Java: Chosen for backend services, Java provided a robust and scalable platform for handling the business logic and data processing required by the IoT dashboard.
• Keycloak: Implemented for authentication and authorization, Keycloak ensured secure access to the application and managed user roles and permissions effectively.
• SQL Database: Used for data storage and retrieval, the SQL database was optimized to handle the large volume of data generated by the IoT devices and provide quick access to this data.

DevOps

• Docker: Employed for containerization, Docker facilitated the creation of consistent development environments and simplified the deployment process.
• Terraform: Used for cloud deployment, Terraform allowed for the automated provisioning of infrastructure and ensured that the deployment process was repeatable and reliable.
• Prometheus, Grafana, OpenSearch, EFK, DataDog: These tools were used for infrastructure monitoring, providing insights into the performance and health of the application and enabling proactive maintenance.
• GitHub Actions: Implemented for the CI/CD pipeline, GitHub Actions automated the code deployment process and ensured smooth deployments across development, staging, and production environments.

My Role and Responsibilities

Development Setup

I initiated the project by setting up the development environment. This involved:

• Constructing Client Side UI: I designed and implemented the client-side user interface using React, which allowed for the creation of dynamic and responsive components that provided a smooth user experience.
• Configuring Client Side Authentication: I set up client side authentication and secured communication and verification of users and admins.
• Preparing Docker containers and Docker Compose for local development and testing: I created Docker containers for the various components of the application and used Docker Compose to orchestrate these containers, simplifying the development and testing process.

Database Design

I designed the SQL database schema to handle various user and device operations. This included:

• User management: I created tables and relationships to store user information and manage user roles and permissions.
• Admin profile settings: I designed the schema to store admin-specific settings and preferences.
• Web admin operations: I ensured that the database could handle the various operations performed by web admins, such as adding and removing users and managing assets.

Authentication and Authorization

I implemented secure admin login and user management features:

• Admin Login: I created a special page for admin login, allowing admins to generate referral links. This involved integrating the frontend with Keycloak to handle the authentication process and generate JWT tokens.
• Customer Onboarding: I designed pages for admin operations and integrated them with backend APIs. This allowed admins to send referral links to new users and manage their profiles.
• User Signup: I developed a user management page for admins to add, remove, or change user roles. This page was integrated with the backend APIs to handle the various user management operations.

Real-Time IoT Data Dashboard

I built a comprehensive IoT data dashboard to display:

• Asset list and customer details: I created components to display a list of assets and their associated customer details.
• Asset location on Google Maps: I integrated Google Maps to show the location of each asset, providing a visual representation of their distribution.
• Charts for asset power, energy, revenue, and emissions: I used charting libraries to create visualizations of various asset metrics, such as power consumption, energy usage, revenue generation, and emissions.

Home Page and Alarms

I developed the home page to show:

• Plant details: I created components to display detailed information about the plant, such as its location, capacity, and status.
• Diagrams (bar/line/notifications): I used charting libraries to create bar and line charts to visualize various metrics, as well as notification components to display alerts and updates.

For alarms, I created features to display:

• Filters: I implemented filtering options to allow users to narrow down the list of alarms based on various criteria.
• Details of active alarms: I created components to display detailed information about active alarms, such as their severity, status, and timestamp.
• Today's alarms: I implemented a feature to show a list of alarms that occurred today, providing users with a quick overview of recent issues.

Analysis and Payment Gateway Integration

I implemented features for data analysis, including:

• Line charts: I used charting libraries to create line charts that visualized trends and patterns in the data.
• Data in tabular format: I created tables to display data in a structured format, making it easy for users to analyze and interpret the information.

Although the payment gateway integration was yet to be decided, I provided estimates for its implementation. This involved researching various payment gateway options and outlining the steps required to integrate them into the application.

Cloud Deployment and Monitoring

I set up cloud deployment using Terraform, which included:

• RDS, EC2 (K8s), EKS, Secret Manager, Certificate Manager, S3: I used Terraform to provision these AWS services, ensuring that the infrastructure was scalable, secure, and reliable.
• Infrastructure monitoring tools like Prometheus, Grafana, OpenSearch, EFK, DataDog, and integration with Slack: I set up these monitoring tools to track the performance and health of the application, and integrated them with Slack to provide real-time alerts and notifications.

CI/CD Pipeline

I automated the code deployment process using GitHub Actions, ensuring smooth deployments across development, staging, and production environments. This involved:

• Creating workflows for building, testing, and deploying the code: I set up GitHub Actions workflows to automate the various stages of the deployment process, from building the code to running tests and deploying to different environments.
• Integrating with Docker and Terraform: I configured the workflows to use Docker for containerization and Terraform for infrastructure provisioning, ensuring that the deployments were consistent and reliable.

Basic Flow of the Application

1. User Authentication: Admins and users log in using credentials. Keycloak handles authentication and generates JWT tokens.
2. Dashboard Access: Based on their roles, users access different parts of the dashboard.
3. Real-Time Data: The dashboard displays real-time data from IoT devices, including asset details, locations, and various metrics.
4. User Management: Admins manage users, send invites, and assign roles.
5. Data Analysis: Users can analyze data through charts and tables.
6. Cloud Deployment: The application is deployed on the cloud, with continuous monitoring and automated CI/CD pipelines.

Conclusion

This freelance project was a rewarding experience, allowing me to leverage my skills in frontend development, database management, and DevOps. The real-time IoT dashboard is now actively used in a production house, significantly improving their operational efficiency and asset management. The project not only showcased my technical abilities but also demonstrated the power of modern web technologies in solving real-world problems.