« Back to home

Wireless Communications - 433MHz Modules

David Lyon recently sent me some 433MHz transmitter and receiver modules that he is designing some Clixx tabs around. These are often used in remote control devices like garage door openers and remote power control systems which use a simple command code to activate or deactivate the remote device.

There is already a lot of software around that lets you use them to control your garage door and other devices from an Arduino or similar micro - I was more interested in using them for simple one way data communication (for remote sensing applications for example). This post describes how I went about testing how well they worked for that purpose.

Sending Data

Most computers and microcontrollers already have (or can easily have added to them) a device that will serialise data and provide some basic validation of the data - a UART (or serial port). I decided this was the best way to send and receive blocks of data using these devices - simply attach the transmitter data line to TX on the sender and the receiver data line to RX on the device that will accept the data. This also turns out to be easy to test with a pair of laptops.

For testing I simply used these USB serial cables from Adafruit but you could use any FTDI cable that provides a 5V supply pin as well as the TX and RX pins. The pin out for the cables I'm using and the connections to the radio modules are shown in the following table:

WireUsageTransmitterReceiver
Red+5VVCCVCC
BlackGroundGNDGND
GreenTXData-
WhiteRX-Data

The next step was to determine how fast data can be reliably transferred between two devices. To send data I'm using a Python script and the PySerial library to communicate with the port. It simply sends out a small packet of data periodically - the data is just the alphabet in both upper and lower case followed by a linefeed character (a total of 53 bytes per packet). Choosing ASCII values makes it easy to visually identify if the packet made it to the receiver uncorrupted.

To test data reception I used CoolTerm on OS/X and Putty on Windows. It was simply a matter of connecting to the appropriate serial device and setting the baud rate options to match that of the sender. For byte format I'm using 8N1 (8 bits per character, no parity and 1 stop bit).

The script I used is as follows ...

#!/usr/bin/env python
#----------------------------------------------------------------------------
# Simple test program to repeatedly send a block of test data over a serial
# port connection.
#----------------------------------------------------------------------------
from time import sleep


# Make sure we have pyserial
try:  
  import serial
except:  
  print "This script requires the PySerial module. Please try ..."
  print "  pip install pyserial"
  exit(1)

#----------------------------------------------------------------------------
# Main program
#----------------------------------------------------------------------------

# Defaults
BAUD_RATE  = 150  
PORT_NAME  = "/dev/ttyS0"  
SEND_DELAY = 1.5

if __name__ == "__main__":  
  port = serial.Serial(PORT_NAME, BAUD_RATE, timeout = 1)
  while True:
    port.write("abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ\n")
    sleep(SEND_DELAY)

I wanted to keep the baud rate fairly low (19200 or less) so it could be done in software on a controller that doesn't have a hardware UART available but still transmit a useful amount of data in a relatively short time. Minimising the transmission time also reduces the chance of the transmission being interrupted by random interference (someone using their garage door opener for example).

It seems as though I was aiming far to high - to get more than 50% of the strings being sent showing up uncorrupted at the receiver end I had to drop the baud rate to 150 (as you can see in the script above). This was much, much lower than I had hoped for. On the positive side - at that low a speed you will have no problem generating or interpreting the signal with a software UART.

Receiving Data

The next step was to determine if these could reliably transfer arbitrary blocks of data that could be picked up at the receiver end. To do this I needed to define a simple communications protocol that would allow me to detect and verify a valid packet out of the noise on the line.

Unlike a straight serial connection you will not get a nice flat '0' level when nothing is transmitting. The receiver picks up noise, much like static on an (analog) TV signal, and this can be interpreted as valid incoming characters. The incoming bit stream will be a series of valid data packets surrounded by random data - we need to identify and extract the meaningful data while simply ignoring the rest of the signal. To help with this I use a packet start marker - a small sequence of known values that indicate that what follows should be a valid data packet. The reception code simply throws away any data it sees until it identifies the start sequence, then it knows it has a reasonable chance of finding some valid data immediately following it.

Clixx TwinTab Boards

I decided to make the size of the packet variable to get as much flexibility out of the system as possible. The larger the packet the more chance it has of being corrupted by interference so you need to be able to use the smallest packet you can get away with - even using different sized packets depending on what data has changed and needs to be transmitted. This means we need to include an indication of the packet length inside the packet itself. For simplicity I use a single byte of data (the first byte in the packet) for this purpose allowing for packet sizes from 0 to 255 bytes. There is certainly no reason to go larger than this (at 150 baud, and taking into account the stop bits and byte spacing it will take slightly over 20 seconds to send 255 bytes). Having a zero length packet seems pointless at first but it does have a role - you can use it as a heartbeat or I have nothing to say but I am still here type of packet that lets you verify that your remote device is still functioning.

The final requirement for the packet format is some way of verifying that the content we receive is what was sent and none of the individual bytes in the body of the packet were corrupted in transit. A common way of doing this is by calculating a CRC or Cyclic Redundancy Check value for the packet content and appending this check value to the packet when it is sent. This allows the receiver to duplicate the CRC calculation on the other end and compare the value it calculated with the one provided in the packet to determine if the packet content is valid or not. In this case I'm using the CCITT 16 bit CRC formula (you can see a straight forward implementation for microcontrollers at this site). This solution adds an extra 2 bytes to the overhead for the packet but will detect almost all packet transmission errors.

The final format for the packet is shown in the image below. This format adds an additional 5 bytes (start sequence, length and CRC) to the actual payload but is easy to transmit and receive with a good mechanism for detecting any transmission errors.

Transmission Packet Format

I chose to use two bytes with the value 0xAA as the start marker. As well as being easy to detect it provides a regular sequence of bits that could be used by the receiver software to synchronise to the transmission speed, 0xAA followed by the stop bit translates into the binary sequence 101010101 - a nice repeating sequence that can be used to determine bit width. This isn't really required for a hardware UART but would help with a software only solution.

I've written a Python module that will send and receive packets over a serial port, here is the source:

#!/usr/bin/env python
#----------------------------------------------------------------------------
# Simple set of classes to read and write packets over RF through a serial
# port.
#----------------------------------------------------------------------------
from sys import argv  
from time import sleep  
from struct import pack, unpack  
from random import randint

# Make sure we have pyserial
try:  
  import serial
except:  
  print "This script requires the PySerial module. Please try ..."
  print "  pip install pyserial"   exit(1)

# Defaults
MARKER     = 0xAA  
BAUD_RATE  = 150  
PORT_NAME  = "/dev/ttyS0"  
WAIT_DELAY = 12.0 / BAUD_RATE

#----------------------------------------------------------------------------
# Helper functions
#----------------------------------------------------------------------------


# High bytes lookup table
CRC16_LOOKUP_HIGH = (  
  0x00, 0x10, 0x20, 0x30, 0x40, 0x50, 0x60, 0x70,
  0x81, 0x91, 0xA1, 0xB1, 0xC1, 0xD1, 0xE1, 0xF1
  )

# Low bytes lookup table
CRC16_LOOKUP_LOW = (  
  0x00, 0x21, 0x42, 0x63, 0x84, 0xA5, 0xC6, 0xE7,
  0x08, 0x29, 0x4A, 0x6B, 0x8C, 0xAD, 0xCE, 0xEF
  )

def generateCRC(data, offset, size):  
  """ Generate a 16 bit CCITT CRC value for a block of data
      See http://www.digitalnemesis.com/info/codesamples/embeddedcrc16/
  """
  crcLow = 0xFF
  crcHigh = 0xFF
  # Helper to update the CRC with another 4 bits
  def updateCRC(value, crcLow, crcHigh):
    # Extract the most significant 4 bits
    temp = (crcHigh >> 4) & 0x0F
    # XOR in the message data
    temp = temp ^ (value & 0x0F)
    # Shift the CRC left 4 bits
    crcHigh = ((crcHigh << 4) & 0xF0) | ((crcLow >> 4) & 0x0F)
    crcLow = (crcLow << 4) & 0xF0
    # XOR the table lookup results into the CRC
    crcHigh = crcHigh ^ CRC16_LOOKUP_HIGH[temp]
    crcLow = crcLow ^ CRC16_LOOKUP_LOW[temp]
    # Return the new low, high values
    return crcLow, crcHigh
  # Now process the data
  for index in range(size):
    crcLow, crcHigh = updateCRC((data[offset + index] >> 4) & 0x0F, crcLow, crcHigh)
    crcLow, crcHigh = updateCRC(data[offset + index] & 0x0F, crcLow, crcHigh)
  # All done
  return crcLow, crcHigh

#----------------------------------------------------------------------------
# Classes for communication
#----------------------------------------------------------------------------

class RF433:  
  """ Class for RF433 communications   """

  def __init__(self, port = PORT_NAME, baud = BAUD_RATE, timeout = WAIT_DELAY):
    self.serial = serial.Serial(port, baud, timeout = timeout)
    self.packet = list()

   def send(self, data, offset = 0, size = -1):
     """ Send a message     """
    # Check the size
    if size < 0:
      size = len(data) - offset
    if size > 255:
      raise Exception("Data exceeds maximum packet length")
    # Build up the data packet     
    packet = [ MARKER, MARKER, size, ]
    for index in range(size):
      packet.append(ord(data[index]) & 0xFF)
    # Generate the CRC for the packet
    crcLow, crcHigh = generateCRC(packet, 2, len(packet) - 2)
    # And append it
    packet.append(crcLow)
    packet.append(crcHigh)
    # Now send the packet
    packet = "".join([ chr(x) for x in packet ])
    self.serial.write(packet)
    return packet[2:]

  def read(self):
    """ Read a packet from the serial port.

     Returns a block of data representing a valid packet or None if no
     packet is available. The data returned is the entire packet including
     the one byte length header and the trailing 16 bit CRC.

     All the bytes of a packet must be sent within 1 byte length of each
     other (baud / 10) seconds. Any longer delay is considered an error.
    """
    # Start reading data
    while True:
      data = self.serial.read(1)
      if len(data) < 1:
        return None # Read timeout
      self.packet.append(ord(data[0]))
      # Look for the two markers to signify the start of a packet
      while len(self.packet) >= 3:
        # Skip forward to the first marker
        if self.packet[0] <> MARKER:
          self.packet = self.packet[1:]
          continue
        # Check for the second one
        if self.packet[1] <> MARKER:
          self.packet = self.packet[2:]
          continue
        # We potentially have a packet, check the length
        size = self.packet[2]
        if len(self.packet) >= (size + 5): # Allow for markers, length and CRC
          # Check the CRC for the packet
          crcLow, crcHigh = generateCRC(self.packet, 2, size + 1)
          if (crcLow <> self.packet[size + 3]) or (crcHigh <> self.packet[size + 4]):
            # Invalid, skip start byte and loop around
            print "Bad CRC - got %02x%02x, wanted %02x%02x" % (crcHigh, crcLow,  self.packet[size + 4], self.packet[size + 3])
            self.packet = self.packet[1:]
            continue
          # Found a packet, return it as a string (for unpacking)
          result = "".join([ chr(x) for x in self.packet[2:size + 5]])
          self.packet = self.packet[size + 5:]
          return result
        else:
          break # Not a valid packet, too short

#----------------------------------------------------------------------------
# Main program
#----------------------------------------------------------------------------

if __name__ == "__main__":  
  # Dump everything we know about a packet
  def dumpPacket(packet):
    size, sequence = unpack(">BH", packet[:3])
    crcLow, crcHigh = unpack("BB", packet[-2:])
    return "%d,%d,%02x%02x,%s" % (size, sequence, crcHigh, crcLow, "".join([ "%02x" % ord(x) for x in packet ]))
  # Create a packet
  def createPacket(sequence):
    size = randint(0, 253) # Allow two bytes for sequence number
    packet = pack(">H", sequence)     
    while len(packet) < size:
      packet = packet + chr(randint(0, 255))
    return packet
  # Main program
  if len(argv) <> 3:
    print "Usage:\n"
    print "       rf433.py port tx|rx"
    exit(1)
  # Set up communications
  rf433 = RF433(argv[1])
  if argv[2] == "tx":
    # Open the log file
    logfile = open("rf433_tx.log", "w+")
    for sequence in range(1000):
      packet = rf433.send(createPacket(sequence))
      sequence = sequence + 1
      print dumpPacket(packet)
      logfile.write("%s\n" % dumpPacket(packet))
      # Wait for the packet to be transmitted + up to 5 seconds
      delay = 12.0 / BAUD_RATE * len(packet) + 1.0 * randint(0, 5)
      print "Waiting for %4.2f seconds" % delay
      sleep(delay)
  elif argv[2] == "rx":
    # Open the log file
    logfile = open("rf433_rx.log", "w+")
    while True:
      packet = rf433.read()
      if packet is not None:
        print dumpPacket(packet)
        logfile.write("%s\n" % dumpPacket(packet))
  else:
    print "Unknown option '%s' - use 'tx' or 'rx'" % argv[2]     
    exit(1)

Now that we have the pieces in place we can now grab some real world data and determine the reliability of the communications channel. This will let us know what scenarios it would be good for and conversely, what it should be used for.

Reliability

The script can be used to test transmission and reception when invoked as a program rather than being imported as a module. In that situation it expects two command line arguments - the name of the port followed by either rx or tx to tell it what mode to run in (receiver or transmitter respectively).

In receiver mode it will listen on the specified serial port for incoming packets and log all valid packets to the file rf433_rx.log. The receiver will keep running until you Ctrl-C the program to break out of it. In transmitter mode the program will send out 1000 randomly sized packets filled with random data and with a random delay (from 0 to 5 seconds) between each packet to simulate a remote device. Every packet sent is logged to the file rf433_tx.log.

I've added a sequence number to each packet (a two byte value at the start of the packet payload) so it is easy to match the received packets to the sent packets when comparing the log files. The companion script (shown below) will take both log files and match the received packets against those sent and give some summary statistics.

#!/usr/bin/env python
#----------------------------------------------------------------------------
# Analyse the log files generated by RF433.py.
#----------------------------------------------------------------------------
LOG_TRANSMIT = "rf433_tx.log"  
LOG_RECEIVE  = "rf433_rx.log"


# Range of sizes we consider for results
SIZE_BLOCK = 4

if __name__ == "__main__":  
  # Load the transmission log to see what we should have received
  transmitted = dict()
  logfile = open(LOG_TRANSMIT, "r")
  for line in logfile:
    parts = line.strip().split(",")
    transmitted[parts[1]] = ( parts[0], parts[2], parts[3] )
  logfile.close()
  # Now compare what we received
  results = dict()
  logfile = open(LOG_RECEIVE, "r")
  for line in logfile:
    parts = line.strip().split(",")
    # Make sure we have a slot for packets of this size
    size = int(int(parts[0]) / SIZE_BLOCK)
    if not results.has_key(size):
      results[size] = [ 0, 0, 0 ] # Received, dropped, false
    # Match with the sending packet 
    if transmitted.has_key(parts[1]):
      # Verify that the packet matches what was sent
      sent = transmitted[parts[1]]
      if (sent[0] <> parts[0]) or (sent[1] <> parts[2]) or (sent[2] <> parts[3]):
        # Data and CRC do not match, false positive
        results[size][2] = results[size][2] + 1
      else:
        results[size][0] = results[size][0] + 1
        del transmitted[parts[1]]
    else:
      # Sequence number not present, false positive
      results[size][2] = results[size][2] + 1
  logfile.close()
  # Now add all the ones we missed
  for sequence in transmitted.keys():
    sent = transmitted[sequence]
    size = int(int(sent[0]) / SIZE_BLOCK)
    if not results.has_key(size):
      results[size] = [ 0, 0, 0 ] # Received, dropped, false
    results[size][1] = results[size][1] + 1
  # Display the data   
  print "Size (Min),Size (Max),Received,Dropped,False"
  for size in sorted(results.keys()):
    print "%d,%d,%d,%d,%d" % (size * SIZE_BLOCK, (size + 1) * SIZE_BLOCK - 1, results[size][0], results[size][1], results[size][2])

I ran the full test (it takes a little over 5 hours to send all 1000 packets) - the results are shown in the graph below.

Packet Reception vs Size

For the test above I was using a Windows laptop as the receiver using the command python rf433.py COM5 rx and an Ubuntu desktop as the transmitter using the command python rf433.py /dev/ttyUSB1 tx - the receiver needs to be started first to ensure all packets have a chance of being captured. The transmitter and receiver boards were simply attached to FTDI USB cables and placed about 2 meters apart. Neither board had an external antennae but because they were so physically close to each other I would expect these results to be about as good as you could get.

Unfortunately the results are far from good in terms of usefulness, no packet larger than 111 bytes was successfully received on the other end and the best result (for packets <16 bytes long) was only about 4% of transmitted packets being detected. To verify the results I ran the test again, this time limiting the maximum packet size to 32 bytes. The results are as follows:

Packet Reception vs Size

The results are a little better (10% successful transmission overall, 14% for packets less than 16 bytes in length) but certainly nothing you can count on. Essentially for every 10 packets you send only 1 will be received at the other end.

Potential Uses

I had originally hoped that these devices would be a simple way to add remote data transfer for non-critical sensing devices - something like a garden monitor that sent temperature, lighting and moisture information to a base station on a semi-regular basis. Theoretically they could be used for that but the measured transmission reliability makes them a less than ideal choice. If you sent a data packet every 6 minutes then on average you should get hourly updates which is not terrible but not ideal either.

These devices are available as a pair (both transmitter and receiver) for less than $AU 1.50 so they are appealing based on price alone. However, for around the same price you can get a NRF24L01 based transceiver that operates in the 2.4GHz frequency range. These have an SPI interface, are bidirectional, support up to 2Mbps transmission speeds and can be configured in a star topology with up to 7 nodes (1 master and 6 slaves). For a pure data transfer solution it is probably worth looking at those instead.

The 433MHz devices are really only applicable if you are using simple on/off command codes and need to integrate with existing systems. There is no feedback to indicate if the command was received and acted upon either so even it that situation I would be a little dubious about using them. They were fun to experiment with, even though I didn't get the results I'd hoped for.

I have some of the 2.4GHz modules now (they arrived in the latest parts shipment) so I'll do a similar analysis using them when I get some time. It will be interesting to see the difference.