Home & Property Monitoring Overview
Updated: 28Aug2024 21:40:11 UTC 2024-08-28T21:40:11Z
Rating: (0 reviewsThis article has not been rated yet)
This overview of home and property monitoring covers hardware components (sensors, microcontrollers, transceivers, computers, and power backup), different ways of communicating between devices, and computer software that reads the data transmission, records, monitors, and sends alert notifications.
Hardware
The basic hardware components for a monitoring system includes a sensor, microcontroller, wireless transmitter (or transceiver), and a computer.
- The sensor makes measurements that are read by a microcontroller.
- The microcontroller is connected to a WiFi or RFRadio Frequency transceiver for wireless communication to a gateway receiver.
- The gateway receiver is connected to a server that captures the data and provides access to the data with a phone, tablet, or computer. The server can be on a local computer, WiFi microcontroller, or offsite on a hosted service.
Sensors
Sensors can be used to detect events like doors or windows opening and the presence of visitors or intruders, and They can also be used to take quantitative measurements, such as the temperature inside your house, how much electricity is consumed, and the amount water flowing through a hose or pipe.
Motion Sensors
The purpose of a motion sensor is detect the presence or absence of people, animals, or vehicles on your property. Their are different kinds of motion sensors, such as switch sensors, PIR sensors, microwave sensors, and IR beams.
A simple type of sensor to detect the open/closed state of a door or window is a magnetic contact switch, which consists of a reed switch and magnet in separate enclosures (ABSAcrylonitrile Butadiene Styrene thermoplastic polymer plastic or metal). The Magnetic Contact Switches article covers different configurations of this switch and its usage.
The presence of visitors or intruders can be sensed passively with PIRPassive Infrared motion sensors. The PIR sensor detects positive differences in infrared (IRInfrared) light radiating from warm objects, such as people or animals, that move into the sensor's field of view. They have an average detection range of approximately 10 meters, but anywhere from 5 meters to 12 meters is the actual detection range of the sensor. The PIR Sensors article covers the most common modules, how they operate, and their capabilities.
Learn more about motion detection monitoring system components in the article below.
Created:
04Nov2022 18:48:41 UTC
2022-11-04T18:48:41Z
Updated:
29Aug2024 05:40:28 UTC
2024-08-29T05:40:28Z
Rating: (0 reviewsThis article has not been rated yet)
Components of a motion detection monitoring system using a motion sensor connected to a microcontroller and transceiver that transmits wireless data to a computer.
Environment Sensors
Environment sensors can measure ambient temperature, humidity, pressure, and/or air quality. This is useful for maintaining a stable and comfortable environment indoors or planning activities outdoors.
Temperature and humidity can be measured with a DHT22 sensor module, which contains both a thermistor temperature sensor and a capacitive humidity sensor that outputs a digital signal of both measurements.
There are many other Temperature and Humidity sensors available below with different sensing ranges, interfaces, and supply voltages.
The Temperature Sensors article covers different kinds of temperature sensors, such as Thermistors, Semiconductor ICs, Resistance Temperature Detectors (RTDs), and Non-Contact Infrared (IR) sensors.
Learn more about motion environment monitoring system components in the article below.
Created:
07Dec2022 01:51:36 UTC
2022-12-07T01:51:36Z
Updated:
29Aug2024 05:47:10 UTC
2024-08-29T05:47:10Z
Rating: (0 reviewsThis article has not been rated yet)
Components of an environment monitoring system using a temperature, humidity, pressure, and air quality sensors connected to a microcontroller that transmits wireless data to a computer.
Power Sensors
Power sensors can measure the voltage, current, and power consumption of an electrical circuit. For example, to measure power consumption from your mains electricity, a non-invasive split-core transformer ACAlternating Current Current Sensor can be clamped onto your main line (live or neutral wire) without having to cut wires. This sensor measures current by acting as an inductor that responds to a changing magnetic field around a current carrying wire.
Learn more about electrical sensors in the articles voltage sensors, current sensors, and power sensors. You can find a variety of different electrical sensor products shown below.
Learn more about the components of an electrical power and energy monitoring system in the article below.
Created:
07Dec2022 02:06:53 UTC
2022-12-07T02:06:53Z
Updated:
29Aug2024 19:42:12 UTC
2024-08-29T19:42:12Z
Rating: (0 reviewsThis article has not been rated yet)
Components of an electrical power and energy monitoring system for appliances, electronic devices, and mains electricity using voltage, current, and power sensors connected to a microcontroller that transmits wireless data to a computer.
Water Sensors
Water sensors can detect the presence of water (rain or leak) or can measure soil moisture, water level, water consumption, and water quality.
A water flow sensor can be used to find out how much water is used for water conservation. This can also be used to automatically limit the amount water given to a plant, pet, or livestock when combined with an electronic shut-off valve.
Learn more about the omponents of a water monitoring system in the article below.
Created:
07Dec2022 02:09:08 UTC
2022-12-07T02:09:08Z
Updated:
29Aug2024 23:27:00 UTC
2024-08-29T23:27:00Z
Rating: (0 reviewsThis article has not been rated yet)
Components of a water monitoring system with water detection, level, flow, and quality sensors connected to a microcontroller that transmits wireless data to a computer.
These are just a few examples of sensors that can be used for home and property monitoring. Many more sensor products are available below.
Microcontrollers
Microcontrollers are designed for monitoring and/or control tasks. They are a self contained micro-processing system that includes memory (RAMRandom Access Memory, Flash), timers, interrupts, ADCAnalog-to-Digital Converter (ADC, A/D, or A-to-D), and GPIOGeneral Purpose Input Output ports. This integrated system allows programming to control interfacing with external electronic devices.
Microcontrollers come packaged in stand-alone single ICIntegrated Circuit chips (e.g., Atmega328P, RP2040) where the number of pins, size, structure, and architecture vary depending on the model.
Microcontrollers are also available in development boards (e.g., Arduino, Raspberry Pi Pico) with the MCUMicrocontroller Unit built onto a single PCBPrinted Circuit Board along with the power conditioning circuit, reset, clock circuit, USB chip and connector, and I/OInput/Output connectors. Development boards utilize the MCUMicrocontroller Unit in a way that is easy to setup and interface to: plugging in power, USB connection, loading software, and connecting I/O.
For most home and property monitoring applications, you don't need a high end microcontroller. Usually, a low speed microcontroller with a few I/O ports and small size will do the job. Below are some popular compact microcontroller board products that can be used inside an application.
Transmitters & Receivers
A transmitter and receiver (or both combined in a transceiver) is needed for wireless communication. There are many microcontroller boards that have a WiFi transceiver integrated into the board (e.g., RPi Pico W, ESP8266, ESP32-WROOM-32 NodeMCU, Arduino Nano 33 IoT or RP2040 Connect), which makes the setup simpler.
Some WiFi microcontrollers, such as the ESP and the RPi Pico W, can also have a web server on them to create a web page displaying sensor results to your devices (phone, computer, tablet) over your network. Storing data on the microcontroller is limited, but you could store more data on an external SDSecure Digital removable miniaturized flash memory cards card with a module or a USB flash drive with a USB host board.
Alternatively, you could connect an external WiFi module (e.g., ESP8266, ESP32, or Adafruit ATWINC1500) to the serial port of the microcontroller to establish a WiFi connection.
Another option is to use an RFRadio Frequency transceiver module to transmit data to the computer. You would need two transceiver modules, one at sensor-microcontroller that transmits the data and the other a the computer or microcontroller gateway that receives the data. In order to interface the receiver with the computer, another microcontroller or USB Serial Adapter needs to be implemented. Below are some options for purchasing a RF modules.
So which is better for sensor networks, WiFi modules or RF modules? There are many factors that play a role in deciding that depend on your application, such as the infrastructure you already have in place and how much time you want to spend setting it up. Below are some of these factors to consider.
WiFi/RF Hardware
If you already have a router and computer with WiFi that you can use for your sensors, then RF modules would require more hardware with an extra receiving module and adapter for the computer.
However, there are some things to consider about using your router and WiFi network.
- How much usage and traffic there is over the network
- How reliable the router is
- How often there are changes in the credentials or configuration
When the router goes down or is rebooting, all your sensors cannot send data. Every time you change your router credentials, you have to update all the sensor transmitters accordingly. If your network already has a lot of usage and traffic, then the separate channels used by RF modules may be a better option.
WiFi/RF Range
RF modules generally have greater range than WiFi. RF modules, such as the HC-12, NRF24L01+, and RFM69, typically get up to a few hundred feet indoors to several thousand meters in open air (this is dependent on the data rate setting, transmission power, and antenna used). WiFi typically gets about 150ft indoors and up to around 300ft in open air.
WiFi/RF Power Consumption
RF modules have a lot less power consumption than WiFi. If you're setting up a battery operated application, then RF modules would be the better option. The lower power consumption of RF modules comes at the cost of having less bandwidth and lower data transfer rates than WiFi.
WiFi/RF Bandwidth
Wifi modules have much more bandwidth and higher data transfer rates than RF modules. RF module are suitable for transmitting small amounts of sensor data at a low rate, but not for audio, images, or video that WiFi could handle well.
WiFi/RF Error Handling
Another aspect to consider is error detection and correction of the transmission itself. Sending messages over a WiFi or RF module could have errors from noise in the electronics or interference from the transmission medium environment.
It is important that any errors in transmission are detected to prevent miscommunication. If an error is detected on the receiver end, then it can request that the message be resent or perform some kind of auto-recover to correct some of the errors in corrupted data.
WiFi has built-in Forward Error Correction (FECForward Error Correction), the process of adding redundant data such as Error Correction Code (ECCError Correction Code) to a message so that it can be recovered by a receiver when errors are introduced. This redundancy allows the receiver to detect a limited number of errors that may occur anywhere in the message, and often to correct these errors without retransmission.
RF modules vary in their built-in capability of any error detection and correction. For example, the HC-12 433MHz RF module only has the built-in capability of appending a parity bit to the message for error detection, but you have to implement the algorithm to compute the parity bit in the software that sends the message, and there is no built-in error correction. However, the NRF24L01 2.4GHz RF module has built-in hardware CRCCyclic Redundancy Check error detection, error correction, and resending.
If an RF module does not have any built-in error detection, you can compute your own checksum of the message, like a CRC, and add it to the end of the message. You can also add your own error correction algorithm and auto resend in software.
WiFi/RF Setup Time
In general, RF modules are more time consuming to setup and configure initially. Once this is done, however, it becomes it's own self-contained network that doesn't rely on anything else.
Computer
The computer hardware constantly monitors for any transmissions from the sensor(s) and records the data. This means that the computer needs to be on 24/7. A desktop computer will work if left on all the time, but are often used heavily on a daily basis which may be an issue sharing resources and updating the operating system constantly would interrupt your monitoring system.
Single Board Computers (SBCSingle Board Computer) are relatively inexpensive (compared to desktop computers), have a small form factor, can be left on all the time, don't require as many updates, and are typically not used for daily activities as much as a desktop computer. SBCs also consume a lot less power than a desktop computer and can last much longer on battery backup in the event of a power outage. Below are some options for purchasing SBCs.
Power Management
The sensors and microcontroller/SBC devices need to be powered in some way, either by plugging them into a nearby outlet with a power adapter or by battery. When using power from an outlet, you should consider how the system will respond in the event of a power outage. To avoid having your monitoring system down during a power outage, you can implement an Uninterruptible Power Supply (UPSUninterruptible Power Supply) battery backup for the microcontroller/SBC and sensor.
It would be convenient if the UPS had the following features:
- Uses a common battery as the backup power source that is small in size and easily replacable, such as a 3.7V/4.2V Li-ionLithium-ion or LiPoLithium Polymer.
- Battery protection for short circuit, over discharge, and over charging.
- Load sharing circuitry that allows the input power to supply the load while charging the battery with uninterrupted battery supply power to the load when the input power is cut-off.
- (Optional) Boost converter with 5V output needed for a 5V microcontroller or SBC
Commercial UPS products used for computers are widely available, but are costly, large in size, and may be overkill for a low power microcontroller or SBC. The replacement battery is also expensive and it is sometimes difficult to find the right one.
Many USB power banks do not provide output power while charging the battery at the same time and even the ones that do support this feature still have the issue of momentarily tripping the output power when the supply power is cut off, causing a microcontroller or SBC to crash/reboot.
Most battery charger modules (e.g., TP4056) must have the load disconnected when charging the battery in order to determine when the battery is full and avoid overcharging.
There are very few UPS modules available that meet the criteria above. One option is the Adafruit PowerBoost 1000 Charger (PID 2465) module shown below. It has a built-in load sharing battery charging circuit with an input power Micro-B USB jack and 5.2V DC output power. This module also comes with a low battery output pin to alert when the battery is low (below 3.2V) so you can automatically shut down the device.
Another option is to use a microcontroller with a built-in battery charge management circuit with load sharing, such as the Pimoroni Pico LiPo, SparkFun Thing Plus-RP2040, and Waveshare RP2040-Plus microcontroller boards shown below (these products can be found in the Battery RP2040 Boards). When the power drops out on the USB, battery power will kick in automatically. When power is restored to the USB, it will switch over to USB power and begin recharging the battery.
Data Communication
The physical means of communicating between devices can be wired or wireless. Although wired communication is by far the most reliable, installing wires throughout a home or property can be difficult, time consuming, or even impractical when sensors are far away from the gateway device that collects the data. Under these circumstances, wireless transmission either using WiFi or RF is a much better option.
WiFi Communication
One common way of sending data between two devices on a WiFi network is to establish a socket connection. A socket is an endpoint with a general-purpose communication interface that establishes a two-way connection between two nodes to exchange data over a network. Sockets provide a low-level interface and are not tied to any specific protocol. When a socket is setup, the user provides an IP address, port number, and which protocol to use (e.g., TCPTransmission Control Protocol/UDPUser Datagram Protocol).
Another common way of sending data between devices on a WiFi network is to use a standard protocol in a client-server configuration. There are many different network protocols, but the most common ones used for IoTInternet Of Things are HTTPHypertext Transfer Protocol, WebSocketWebSocket is a communication protocol, providing simultaneous two-way communication channels over a single Transmission Control Protocol (TCP) connection. WebSockets are typically run from a web browser connected to an application server that allows messages to be passed back and forth while keeping the connection open., and MQTTMQ Telemetry Transport (MQTT) is a lightweight publish-subscribe messaging protocol for small sensors and mobile devices, optimized for high-latency or unreliable networks. Historically, the "MQ" in "MQTT" came from the IBM MQ (then 'MQSeries') product line, where it stands for "Message Queue". However, the protocol provides publish-and-subscribe messaging (no queues, in spite of the name)..
HTTP
Hypertext Transfer Protocol (HTTPHypertext Transfer Protocol) is an application layer request/response client-server protocol for transmitting hypermedia documents, such as HTMLHyperText Markup Language, where requests are sent by the client and the server provides a response.
HTTP has three main versions: HTTP/1, HTTP/2, and HTTP/3. All three protocols are backwards compatible with the previous protocol.
HTTP/1 was published in 1996 and is text-based with an ordered and blocking format.
HTTP/2 was published in 2015 and provides more efficient semantics to speed up browser-side and server-side transactions. HTTP/2 is binary rather than textual, has less resource usage, utilizes multiplexing to handle TCP requests instead of the ordered and blocking format used by HTTP/1, and implements server push and bi-directional (full-duplex) communication.
HTTP/3 was published in 2022 and has even lower latency for web pages and loads faster than HTTP/2 and HTTP/1.
HTTP/2 or HTTP/3 work well for monitoring systems with its lower overhead that relieves the strain on hardware in terms of memory and power consumption.
WebSocket
WebSockets is a protocol that provides bi-directional full-duplex communication in a persistent connection between client and server.
It was developed from the limitation of HTTP/1 as a strictly unidirectional protocol where any data sent from server to the client must first be requested by the client.
Websockets operate over HTTP through a single TCP/IPTCP/IP is a suite of internet communication protocols. TCP (Transmission Control Protocol) is responsible for establishing reliable, ordered data delivery, and handling congestion control, while IP (Internet Protocol) focuses on routing packets of data to their destination. socket connection, using HTTP as the initial transport mechanism for the initial handshake, but then keeps the TCP/IP connection alive after the HTTP response is received in order for sending messages back and forth between client and server using a basic framed message protocol.
The bidirectional capability of websockets is ideal for monitoring systems that require constantly fast updates.
MQTT
MQ Telemetry Transport (MQTTMQ Telemetry Transport (MQTT) is a lightweight publish-subscribe messaging protocol for small sensors and mobile devices, optimized for high-latency or unreliable networks. Historically, the "MQ" in "MQTT" came from the IBM MQ (then 'MQSeries') product line, where it stands for "Message Queue". However, the protocol provides publish-and-subscribe messaging (no queues, in spite of the name).) is a lightweight protocol to facilitate machine-to-machine communication.
This messaging protocol is based on the publish/subscribe model, which can provide real-time and reliable messaging service for IoTInternet Of Things devices. It has a small footprint and minimal bandwidth consumption.
MQTT messaging is between a broker and clients, where there is only one broker (server) and multiple clients (e.g., sensors) in a MQTT system. MQTT has some advantages that make it more convenient to use than HTTP and WebSockets.
- The broker in MQTT will deliver a message from a publisher to any (all) subscribers where interested parties can subscribe to messages that may interest them.
- MQTT handles all connection, disconnection, and reconnection events.
- Support for Quality of Service (QoS) and guaranteed delivery
- The MQTT broker also has a "Last Will and Testament" (LWT) defined in the protocol that will generate an automatic message when a client fails or abnormally disconnects.
RF Communication
Duplex
Most RFRadio Frequency modules, such as the HC-12, NRF24L01, RFM69, and LoRaLoRa (short for Long Range) is a spread spectrum modulation technique derived from Chirp Spread Spectrum (CSS) technology. modules perform half-duplex communication: they can either transmit or receive data at a given time on one frequency channel, but cannot receive and send data over the air both at the same time. Full-duplex on an RF transceiver is more complex and expensive to implement, requiring two separate frequencies to be very far apart and requires filters on the receiver to block the signal from the adjacent transmitter.
Data Transfer
Data is usually exchanged between a microcontroller and RF module by serial UARTUniversal Asynchronous Receiver-Transmitter (e.g., HC-12, Reyax LoRa Modules) or SPISerial Peripheral Interface (e.g., NRF24L01, RFM69, LoRa RFM95). This data is transmitted wireless by sending packets of a certain size and content structure (header, payload, parity bit or CRCCyclic Redundancy Check, etc.) that depend on the specific device.
Error Handling
When sending a data packet, you want to make sure the receiving end got the packet and the correct data. Typically you would have the receiver check the data with a parity bit or CRC and send an acknowledgment (ACKAcknowledgment) to the transmitter letting it know it got the data and everything was ok. The transmitter would have some preset time interval, sometimes called an Auto Retransmit Delay (ARDAuto Retransmit Delay), of how long it should wait for the ACK before accepting failure and retransmitting the message.
What if the ACK is lost? Each message sent usually has a unique message ID in the header of the packet, so when the receiver sees the retransmitting message with the same message ID, it just discards it and sends the ACK again. This process repeats itself until transmitter obtains an ACK. Some modules handle this process automatically without the involvement of the microcontroller (e.g., NRF24L01, RFM69, LoRa), while with others (e.g., Generic 433Mhz transmitters and receivers, HC-12) you have to implement this process yourself in the microcontroller software.
Identification
When multiple transceivers are setup in a network, you need to establish a way to determine what device the transmission came from. For identifying a transmitter, something simple can be setup where each transmitter can include a unique number (address) in its message.
Message Collisions
A multi-transceiver network must also avoid collisions between messages that are received at the same time. The transmitters could be setup to transmit at different time intervals, but that is not controllable with event/detection induced transmissions. To mitigate collisions, the retransmit time intervals should be different between transmitters, which could be based on something unique like the transmitter address. Sensor networks can become complex and require careful planning.
Transmission Methods
There are three main communication techniques to consider when designing a monitoring system.
- Send the data only when requested by the gateway device (e.g., computer)
- Send the data only when an event has been detected
- Send the data at regular intervals continuously
The first method allows you to control when the data is sent to the gateway, but you miss out on being alerted in real-time when an event detection occurs, such as a door opening, presence of an intruder, lights being turned on, etc.
The second method has the advantage of alerting in real-time, but there is no way to tell if the detector or microcontroller is down (i.e., disconnected, malfunctioning, or the microcontroller program is not running).
The advantage of the third method of sending the state at regular intervals is if the signal stops being sent, then you know something is wrong. Sending periodic signals at smaller time intervals gets you closer to real-time alerts, but constantly sending signals has the most power consumption, especially over WiFi, which is not good for battery operated applications.
You can also combine different methods. For instance, sending the data only when a detection has occurred and also send a monitor health status periodically over longer time intervals.
Software
A gateway device receives data from all the different sensors and channels it to a server that can record and provides a means of accessing and displaying the data through a phone, tablet, or computer.
The server can be on a local computer within your local network, over the internet through a hosted service, or even on some microcontrollers with WiFi.
The following subsections describes computer software techniques for collecting, recording, monitoring, and alerting using the Python programming language.
Software Interface
For RF modules the receiver is usually connected serially over USB to the computer. To collect the data from the USB port on the computer you can use the Python PySerial software interface. The functionality of this interface is described in more detail in the article Python PySerial I/O.
For WiFi communication, the software interface on the computer depends on the type of network connection between devices and communication protocol used. Common software interfaces for IoTInternet Of Things are establishing a socket connection or a client-server configuration with the use of a standard protocol, such as HTTPHypertext Transfer Protocol, WebSocketWebSocket is a communication protocol, providing simultaneous two-way communication channels over a single Transmission Control Protocol (TCP) connection. WebSockets are typically run from a web browser connected to an application server that allows messages to be passed back and forth while keeping the connection open., or MQTTMQ Telemetry Transport (MQTT) is a lightweight publish-subscribe messaging protocol for small sensors and mobile devices, optimized for high-latency or unreliable networks. Historically, the "MQ" in "MQTT" came from the IBM MQ (then 'MQSeries') product line, where it stands for "Message Queue". However, the protocol provides publish-and-subscribe messaging (no queues, in spite of the name)..
Sockets
Python has a standard built-in socket module to establish TCPTransmission Control Protocol or UDPUser Datagram Protocol Socket connections and the socketserver module that has classes that make using the low-level socket functions easier. A detailed article on Python sockets and its usage can be found at realpython.com.
HTTP
Python has a standard built-in webserver library called http.server for handling HTTP requests. This module creates a webserver to handle basic HTTP communication with GET and POST methods (note that SimpleHTTPServer has been migrated to Python's http.server module in Python 3). Although this is not a fully featured web server, it can parse text and static HTML files, give response code, and you can set your own URLUniform Resource Locator address and port number of the server.
Websockets
Python has a webserver library called websockets that can be installed. The websocket is a connection between devices that can be used to either send or receive information. The websocket can be configured as a server or client, where the server listens for requests and returns a result based on the input or the server can also function as a client by sending information without a specific request.
MQTT
A comparison of MQTT implementations can be found on Wikipedia. The open source message broker Eclipse Mosquitto can be locally installed on Windows and Linux or you can use a free online broker. Python has the Paho MQTT client library that you can install to setup Broker-Client messaging. The meaning of "Paho" is "to broadcast, make widely known, announce, disseminate, or transmit."
Recording Data
A proper monitoring system wouldn't be complete without the capability of recording and keeping records of the data. Once the data is collected onto the computer by software, you have a few choices on how to store that data in order to retrieve it at a later time.
Text Files
If the data consists of simple measurements, like a single value with a date and time, then the easiest way to store it is in a delimited text file. A CSVComma-Separated Values file is a simple and common plain text format for tabular data and spreasheets that usually uses a comma as a delimiter to separate data, but other delimiters can also be used (semi-colon, tab, etc.).
CSV files are popular because they are portable (just send a file to anyone and they can open and see the data in text), are already supported by data import/export tools, and simple enough that a programmer can write their own tools in a short amount of time. The csv module in Python implements classes to easily read and write data in CSV format.
However, CSV files are not so good in terms of read/write speed, efficiency of CPU and memory use, storing different data types, data search and ordering, any kind of support for referential integrity, and multiple concurrent sources writing to and sharing data from the file at the same time. This is where databases are a better option.
Databases
Databases offer functionality for searching, sorting, and filtering data from storage. It also supports multiple concurrent users or other data sources reading and writing to the database at the same time.
Compared to text files, databases make a lot more efficient use of CPU and memory from the way it stores data and the built-in functionality of accessing the data. Databases can store different data types (numerical or text) in binary that uses less memory and you don't have to keep converting between text and numerical data when reading or writing to the database like you would with a text file.
Databases have built-in support for different data structures with referential integrity, unlike text files. The storage format and data structures facilitate much faster data access without the need to write the functionality yourself.
Some popular examples of databases for storing sensor data are SQLite, PostgreSQL (TimescaleDB is a postgres extension customized for timeseries datasets), MongoDB, InfluxDB, and many others. SQLite has a small footprint and stores the data in a file, whereas the other databases require a separate server process.
The main drawback to databases is they are not as simple to setup and use compared to text files. If you are just starting out with databases, there is a learning curve which will require more of your time.
Data Monitoring
Software on the computer can be used for controlling and monitoring data with a desktop application GUIGraphical User Interface or web server developed by the language of your choice (Python, C++, Java, etc.).
In Python, there are many desktop GUI frameworks you can use like TKinter, wxPython, PyQT, PySide 2, Kivy, PySimpleGUI, and PyGUI. All of these frameworks are cross-platform that work in Windows, macOS, and Linux. TKinter is a built-in module in Python and the easiest to get up and running and learn, has a small footprint, and is a good choice for small applications.
Another option for data monitoring is using a web page as the GUI from a server that can be ran locally or by a hosting service over the internet. The advantage of using a web page for the front end of a monitoring system is that it can be accessed by other devices (phone, tablet, laptop, etc.) either locally over your Local Area Network (LANLocal Area Network) or remotely by hosting it yourself using port forwarding on your local network or from a hosting service.
Python has web frameworks such as Flask, FastAPI, or Django. Flask is the easiest to get setup and running (you can get set up with just a few lines of code). These web frameworks can also be converted to a desktop application GUI executable.
The Flaskwebgui (PyPI, GitHub) library can create desktop applications with Flask/FastAPI/Django by just using threading underneath to start a flask server and the chrome browser in app mode. Other libraries are available for converting a Flask App to desktop App, like flask-desktop powered by PySide2 and PyFladesk that uses the QtWebEngine. You can create a standalone executable using PyInstaller and the front-end of the application can still be developed like a web page using HTMLHyperText Markup Language, JavaScript, and CSSCascading Style Sheets.
A graphical interface, such as the dashboard illustrated in the figure below, can be made to show live plots and tables to assist in visualizing and monitoring the data measured by the sensors.
Alert Notifications
You may not always be at your computer to check the monitoring system, so it would be a good idea to include the capability of automatically sending alert notifications by email and/or text messages that you can see on your phone or other devices. This additional feature can be setup to allow you to turn it on/off in the computer software.
Most computer email applications, such as Gmail, Outlook, Apple Mail, Yahoo Mail, Netscape, Mozilla Thunderbird, and many others use Simple Mail Transfer Protocol (SMTPSimple Mail Transfer Protocol) for sending email. SMTP dictates how the email messages should be formatted, encrypted, and relayed between mail servers.
SMTP is a push protocol used to send the mail whereas Post Office Protocol (POPPost Office Protocol) and Internet Message Access Protocol (IMAPInternet Message Access Protocol) are used to retrieve those emails at the receiver's side.
When you send an email, the application opens a connection to an SMTP server at a specific address or domain name and port number to send the email data over TCPTransmission Control Protocol. The address identifies the machine on the IPInternet Protocol network, while ports identify particular applications or services on a system.
Each email address, such as @gmail.com, has an underlying SMTP server that runs on the domain (e.g., smtp.gmail.com) and listens for incoming emails. The domain name for the SMTP server will usually be the name of your provider's email domain name.
For the port number, SMTP typically uses 25, 465, 587, or 2525. Ports 465 and 587 are the more common ports used to send emails to SMTP servers. Port 2525 is used as an alternative in circumstances where other ports (465 and 587) are blocked. Port 25 is used these days for relaying between mail servers and is often blocked by providers to curb the amount of spam.
The table below lists some common email providers with their SMTP server domain name and port.
| Provider | SMTP Server Domain Name | Port |
|---|---|---|
| Gmail | smtp.gmail.com | 587 |
| Outlook / Hotmail | smtp-mail.outlook.com | 587 |
| Yahoo Mail | smtp.mail.yahoo.com | 587 |
| AT&T | smpt.mail.att.net | 465 |
| Comcast | smtp.comcast.net | 587 |
| Verizon | smtp.verizon.net | 465 |
| Apple iCloud | smtp.mail.me.com | 587 |
When an email is sent by SMTP, it can be encrypted with protocols such as Secure Sockets Layer (SSLSecure Sockets Layer) or Transport Layer Security (TLSTransport Layer Security), where TLS is the more secure successor of SSL. The email application needs to be setup with the encryption standard the SMTP server uses in order to connect and communicate with it.
Some email providers have additional security settings that may need to be configured. For instance, Gmail has an additional security feature for Google accounts called application-specific passwords that needs to be setup before automating emails with your application.
You can create a program in Python to automatically send emails for you using the built-in smtplib module and the ssl module can provide TLS encryption. If you are using a Python web framework (Flask, FastAPI, or Django), there are extensions for these frameworks that can send the email for you. For example, Flask has the extension Flask-Mail to automatically send emails.
SMS Text Messages
Short Message Service (SMSShort Message Service) is a text messaging service built into telephone, internet, or mobile devices. It uses standardized communication protocols to send short text messages containing all the necessary information (including sender phone number, recipient phone number, delivery report flag, message text, etc).
Text messages can be sent using services, such as Twilio or Textbelt, or they can also be sent by SMSShort Message Service Gateways using the same routines used for sending emails. Many mobile carriers have their own gateway as shown for a few examples in the table below where [number] is the phone number the text is to be sent to.
| Carrier | SMS Gateway |
|---|---|
| AT&T | [number]@txt.att.net |
| Sprint | [number]@messaging.sprintpcs.com or [number]@pm.sprint.com |
| T-Mobile | [number]@tmomail.net |
| Verizon | [number]@vtext.com |
| Cricket | [number]@sms.mycricket.com |
To send a text message, your application would send an email to the SMS gateway through your email server (i.e., the SMS Gateway would be in the To: field of the email).
Conclusion
This was a basic introduction on main components of monitoring systems: hardware, communication, and software. The purpose of this overview was to provide different options that are available and a general idea of what they involve.
More in-depth coverage will be in found in other articles on monitoring systems for specific topics such as motion detection, environment monitoring (temperature and humidity), power consumption, and water usage.
(0) Comments
Sign in to leave a comment
Sign In