WiFi and MQTT

Why you need this

A sensor that only shows its reading on its own screen is useful in one room. Put it on WiFi and the same reading can reach a phone, a dashboard, or a log anywhere. This is stage 5 of the pipeline, "Go wireless": the board joins a network, fetches what it needs over HTTPS, and publishes what it measures to an MQTT broker.

The idea

Three jobs, done in order, each of which can fail on its own.

1. Join the network. The ESP32 runs in station mode: it acts like a laptop joining a router, as opposed to access point mode, where it is the router. You give it the network name (the SSID) and password, then wait. Joining takes time: the radio scans, authenticates, and asks the router for an IP address (DHCP). A good program waits with a timeout, a fixed limit after which it gives up and does something sensible instead of hanging forever.

2. Talk to a server. Once the board has an IP address it can open a TCP connection. HTTP is a request and a response: "GET this file", "here it is". HTTPS is HTTP inside TLS, the encryption layer that also lets the client check it is talking to the right server. TLS is the expensive part on a microcontroller: the handshake needs working memory for buffers and cryptography, a real share of the heap on a chip whose whole RAM is measured in hundreds of kilobytes.

3. Publish readings. MQTT is a publish-subscribe protocol. A central server called the broker receives messages, each tagged with a topic (a slash-separated name like devices/my-thing/telemetry). Any client that subscribed to that topic gets a copy. The sensor never needs to know who is listening. A board can also subscribe to a topic of its own, which gives you a path for commands going the other way.

The precise version, in MicroPython terms: network.WLAN(network.STA_IF) is the station interface; active(True) powers the radio; connect(ssid, password) starts a join; isconnected() is True once the board is joined and has an IP address. status() returns one of the constants STAT_IDLE, STAT_CONNECTING, STAT_WRONG_PASSWORD, STAT_NO_AP_FOUND, STAT_CONNECT_FAIL, or STAT_GOT_IP (per the MicroPython network.WLAN documentation), which tells you why a join failed, not just that it did.

Joining with a timeout

This join function is from the author's ESP_32_SAI project, a CYD sensor dashboard that has run on real hardware. It lives in boot.py, which MicroPython runs before main.py.

def connect_wifi():
    wlan = network.WLAN(network.STA_IF)
    wlan.active(True)
    if wlan.isconnected():
        print("Wi-Fi already connected:", wlan.ifconfig()[0])
        return wlan
    print("Connecting to", WIFI_SSID, "...")
    wlan.connect(WIFI_SSID, WIFI_PASS)
    for i in range(20):
        if wlan.isconnected():
            break
        time.sleep(1)
        print(".", end="")
    ...
    return wlan

Twenty one-second checks make a 20 s timeout. The ESP32 Inspector's update stub does the same job with 250 ms checks and returns None after 20 s, so its caller can open a setup access point instead.

Keeping credentials out of the code

The SSID and password are not in the source. ESP_32_SAI reads them from a small key-value file on the board that is never committed to version control. This reader is from that project (hardware-proven); MicroPython has no os.environ, so it parses the file itself.

def load_env(path=".env"):
    env = {}
    try:
        with open(path) as f:
            for line in f:
                line = line.strip()
                if not line or line.startswith("#"):
                    continue
                if "=" in line:
                    key, value = line.split("=", 1)
                    env[key.strip()] = value.strip()
    except OSError:
        print("WARNING: {} not found".format(path))
    return env

config.py then does WIFI_SSID = _env.get("WIFI_SSID", ""). The file on the board holds lines like WIFI_SSID=your-ssid. Anyone with the board in hand can still read the file, so this keeps secrets out of a repository, not out of a thief's hands.

An HTTPS request by hand

MicroPython has no full HTTP library on every build, so the ESP32 Inspector's update stub opens the socket itself. This excerpt is from its inspector_update.py, which has fetched its manifest over TLS on a real board.

addr = socket.getaddrinfo(host, port)[0][-1]
s = socket.socket()
s.connect(addr)
try:
    if proto == "https":
        import ssl
        if hasattr(ssl, "SSLContext"):
            ctx = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT)
            ctx.check_hostname = False
            ctx.verify_mode = ssl.CERT_NONE
            s = ctx.wrap_socket(s, server_hostname=host)
    ...
finally:
    s.close()
    gc.collect()  # release TLS buffers promptly: heap headroom for the next handshake

Two details teach a lot. server_hostname=host sends the name of the site inside the handshake (called SNI); servers behind a content delivery network refuse the connection without it. verify_mode = ssl.CERT_NONE means the board encrypts but does not check the server's certificate, because the stub carries no store of trusted certificates. That is a deliberate trade, discussed on Over-the-air updates. The gc.collect() after closing frees the TLS buffers right away, because a second handshake started before the garbage collector runs can fail for lack of memory.

Publishing over MQTT

ESP_32_SAI publishes to a cloud MQTT broker that requires each device to present its own certificate (mutual TLS). This constructor is from its mqtt_client.py (hardware-proven), built on MicroPython's umqtt.simple library; the endpoint, device name, and certificate files come from configuration.

self.topic_telemetry = "devices/{}/telemetry".format(thing_name)
self.topic_commands = "devices/{}/commands".format(thing_name)
...
self.client = MQTTClient(
    client_id=thing_name,
    server=endpoint,
    port=8883,
    ssl=True,
    ssl_params={"key": key, "cert": cert, "server_hostname": endpoint},
)
self.client.set_callback(self._msg_callback)

Port 8883 is the standard port for MQTT over TLS; plain MQTT uses 1883. After connect() the client subscribes to its commands topic. Its publish() turns a Python dictionary into JSON and sends it; any exception marks the client disconnected.

The main loop from the same project's main.py reads, draws, publishes, and retries the connection when it is down:

while True:
    try:
        temp, hum = sensor.read()
        ...
        dash.update(temp, hum)
        payload = {"temp_c": temp, "humidity_pct": hum,
                   "timestamp": time.time(), "device_id": DEVICE_ID}
        if iot.connected:
            iot.publish(payload)
            iot.check_messages()
        else:
            iot.connect()
    except Exception as e:
        print("Error:", e)
    time.sleep(READ_INTERVAL)

If the broker is unreachable at startup the program prints "continuing with local display only" and keeps the screen working. check_messages() calls umqtt's check_msg(), which handles any command waiting from the broker without blocking.

The same in C with ESP-IDF

In C with ESP-IDF, Espressif's own framework, WiFi is the esp_wifi component (set station mode, give it a config, start, connect, then wait for the "got IP" event) and MQTT is the ESP-MQTT component. This sketch was written for this page from the names in Espressif's ESP-MQTT documentation and is not hardware-tested.

esp_mqtt_client_config_t cfg = {
    .broker.address.uri = "mqtts://broker.example.com",
};
esp_mqtt_client_handle_t client = esp_mqtt_client_init(&cfg);
esp_mqtt_client_start(client);
/* later, after MQTT_EVENT_CONNECTED arrives: */
esp_mqtt_client_publish(client, "devices/my-thing/telemetry",
                        "{\"temp_c\": 22.4}", 0, 0, 0);

ESP-MQTT accepts mqtt:// (default port 1883), mqtts:// (8883), ws:// (80), and wss:// (443), and it reconnects by itself after a drop unless you turn that off. In MicroPython with umqtt.simple you write the reconnect yourself, as the author's loop does.

Worked example

One reading per minute, in the shape ESP_32_SAI sends. (The project itself reads every 5 s; the arithmetic below does both.) The topic is devices/my-thing/telemetry, 26 bytes. The JSON payload is:

{"temp_c": 22.4, "humidity_pct": 41.3, "timestamp": 812345678, "device_id": "dev-01"}

That is 85 bytes. An MQTT PUBLISH packet at QoS 0 (fire and forget, no acknowledgment), laid out per the MQTT 3.1.1 specification:

Part	Bytes	Value
Packet type and flags	1	0x30 (PUBLISH, QoS 0, not retained)
Remaining length	1	113 = 0x71 (fits in one byte because it is under 128)
Topic length	2	26 = 0x001A
Topic	26	devices/my-thing/telemetry
Payload	85	the JSON above
Total	115

The remaining length is $2 + 26 + 85 = 113$, and the total is $1 + 1 + 113 = 115$ bytes.

Rate	Messages per day	MQTT bytes per day
One per minute	$86400 / 60 = 1440$	$1440 \times 115 = 165{,}600$ (about 161.7 KB, with 1 KB = 1024 bytes)
One per 5 s	$86400 / 5 = 17280$	$17280 \times 115 = 1{,}987{,}200$ (about 1940.6 KB)

These count MQTT bytes only. TCP, IP, and TLS add their own headers to every packet, so the bytes on the air are higher. Even so, the radio traffic is tiny; the real cost is keeping WiFi powered.

Timing has a trap. The loop does its work and then sleeps READ_INTERVAL. If the sensor read, the screen redraw, and the publish take $w$ seconds together, the real period is $60 + w$ seconds, not 60. Over a day the readings drift later. If you need readings on the minute, compute the next deadline with time.ticks_ms() and sleep only the remainder.

In an ESP32 project

WiFi sits at stage 5 and leans on two earlier stages. Stage 4 produced the reading: ESP_32_SAI gets temperature and humidity from an AHT10 sensor over I2C (see I2C and SPI peripherals) and draws it on the CYD screen. Stage 3 matters because boot.py runs the WiFi join before main.py starts, as described on MicroPython on the ESP32. And the same WiFi join and HTTPS fetch are the foundation of Over-the-air updates. When there is no router at all, ESP-NOW and Bluetooth LE are the alternatives.

Common mistakes

• No timeout on the join. A while not wlan.isconnected(): pass loop hangs forever with a wrong password. Symptom: the board boots, prints "Connecting", and never draws its screen. Bound the wait and check status().
• Reconnecting MQTT while WiFi is down. If WiFi drops after boot, connect() to the broker fails every loop and the cause is hidden. Symptom: "MQTT connect failed" forever. Check isconnected() and rejoin WiFi first.
• Out of memory in the TLS handshake. A second HTTPS request or a big screen buffer leaves too little heap. Symptom: OSError or a memory error during wrap_socket. Free buffers and call gc.collect() before connecting.
• Missing SNI. Leaving out server_hostname makes some servers reject the handshake. Symptom: an error from wrap_socket even though the same address works in a browser.
• Duplicate client ids. Two boards with the same MQTT client_id kick each other off the broker. Symptom: both connect, then each disconnects every few seconds.
• Credentials in the repository. Symptom: none until someone else reads them. Keep them in an uncommitted file, as ESP_32_SAI does.

Cost

Keeping the WiFi radio on is the main power cost; a board that never turns it off cannot run long on a small battery, and the usual fix is to sleep between readings and rejoin, paying the join time (seconds) each wake. TLS costs the most memory: its buffers take a real share of the heap during the handshake, which is why the Inspector stub frees them immediately after each request. MicroPython itself, its network stack, and umqtt.simple fit comfortably in an ESP32's flash; the cost a maker feels is time, because each edit means copying files to the board, rebooting, and waiting for the join. ESP-MQTT in C is faster and reconnects for you, at the price of a longer build and flash cycle.

Going further

• The MicroPython network.WLAN documentation, for status() codes and power-saving modes.
• The MQTT 3.1.1 specification, sections on QoS 1 and 2, retained messages, and the last-will message.
• Espressif's ESP-MQTT and Wi-Fi driver documentation for the C side.
• Over-the-air updates, which builds a whole update system on the HTTPS fetch shown here.
• Deep sleep between readings, for a battery-powered version of the sensor.