C with ESP-IDF

Why you need this

C with ESP-IDF is the only option here that reaches the whole chip: WiFi and USB host together, every driver, every configuration switch, at full compiled speed. It is also what sits underneath the others: MicroPython and CircuitPython for the ESP32 are themselves built with ESP-IDF, and so are the ESP32 Inspector's echo payloads. Knowing its project shape lets you read, build, and change any of them.

The idea

ESP-IDF (Espressif IoT Development Framework) is three things in one download: a C compiler for each core (Xtensa and RISC-V), a library of drivers and services, and a build system. Inside every program it runs FreeRTOS, a small real-time operating system that lets several tasks (independent loops, each with its own stack) share the cores by switching between them. The author's projects use ESP-IDF 5.4.1. No C++ here: everything on this page is plain C.

A project's layout

This is the layout of the author's ESP_32_Keyboard USB host project for the YD-ESP32-S3, which is hardware-proven (trimmed to the files that teach):

idf_hid/
  CMakeLists.txt         the project: its name, and ESP-IDF's build rules
  sdkconfig.defaults     the settings this project chose
  sdkconfig              every setting, generated (CONFIG_IDF_TARGET="esp32s3")
  dependencies.lock      exact versions of downloaded components
  managed_components/    components fetched by the component manager
  main/
    CMakeLists.txt       this component's sources and what it needs
    idf_component.yml    components to download
    hid_host_example.c   app_main lives here
    keymap.c  peers.c  web_server.c  index.html  ...
  build/                 everything the build produces

A component is a folder of code with its own CMakeLists.txt. Your code is the main component; ESP-IDF's drivers are components too. The author's main/CMakeLists.txt registers the sources, names the ESP-IDF components it uses, and even embeds the web page into the program:

idf_component_register(SRCS "hid_host_example.c" "keymap.c" "web_server.c" "peers.c"
                       INCLUDE_DIRS "."
                       PRIV_REQUIRES usb esp_driver_gpio esp_wifi nvs_flash esp_netif esp_event esp_http_server json
                       EMBED_FILES "index.html"
                       )

Managed components come from Espressif's component registry instead of ESP-IDF itself. The same project's main/idf_component.yml asks for the USB HID host driver, and the build downloads it into managed_components/:

dependencies:
  idf: ">=4.4"
  espressif/usb_host_hid: ">=1.0.0"

Configuration: menuconfig and sdkconfig

ESP-IDF has hundreds of settings: which console to use, the flash size, the CPU speed, whether the task watchdog runs. idf.py menuconfig shows them as a text menu and saves your choices to sdkconfig. A shorter sdkconfig.defaults holds only the settings you changed, and is the file worth committing. These two lines are from the ESP32 Inspector's C6 echo payload (hardware-proven): they send the console to the native USB port and turn off the task watchdog, a timer that resets the chip if a task never yields.

CONFIG_ESP_CONSOLE_USB_SERIAL_JTAG=y
CONFIG_ESP_TASK_WDT_INIT=n

Build, flash, monitor

From Espressif's ESP-IDF Get Started guide, the four commands you run in a project folder:

idf.py set-target esp32c6
idf.py build
idf.py -p PORT flash
idf.py -p PORT monitor

set-target picks the chip once per project, build compiles everything, flash writes it through esptool, and monitor opens the serial console. PORT is your board's serial port, such as COM6.

app_main and FreeRTOS

There is no main(). ESP-IDF's startup code sets up the chip, starts FreeRTOS, and calls your app_main() from a task called the main task. When app_main returns, that task ends but other tasks keep running.

The author's keyboard host shows the usual shape. Its app_main, excerpted from the ESP_32_Keyboard USB host project (hardware-proven), starts a second task for USB events, creates a queue (a thread-safe mailbox that tasks pass fixed-size messages through), and then sleeps on the queue:

void app_main(void)
{
    ...
    ESP_LOGI(TAG, "HID Host example");
    espnow_init();
    ...
    task_created = xTaskCreatePinnedToCore(usb_lib_task,
                                           "usb_events",
                                           4096,
                                           xTaskGetCurrentTaskHandle(),
                                           2, NULL, 0);
    ...
    app_event_queue = xQueueCreate(10, sizeof(app_event_queue_t));

    while (1) {
        // Wait queue
        if (xQueueReceive(app_event_queue, &evt_queue, portMAX_DELAY)) {
            ...
            if (APP_EVENT_HID_HOST ==  evt_queue.event_group) {
                hid_host_device_event(evt_queue.hid_host_device.handle,
                                      evt_queue.hid_host_device.event,
                                      evt_queue.hid_host_device.arg);
            }
        }
    }
}

The arguments to xTaskCreatePinnedToCore are the function, a name, a stack of 4096 bytes, an argument, a priority of 2, no handle, and core 0. The USB driver calls back from its own task, and the callback only drops a message in the queue:

    if (app_event_queue) {
        xQueueSend(app_event_queue, &evt_queue, 0);
    }

That split is the main FreeRTOS lesson: callbacks hand work off quickly, and one task does the slow part.

ESP_LOGI(TAG, ...) is ESP-IDF's logging. A line prints as I (198) example: HID HOST example: the level letter (E, W, I, D, V for error to verbose), milliseconds since boot, and the tag.

What a build produces

The author's C6 echo payload build lists its output in build/flash_args:

File	Size	Flash offset on the C6
bootloader.bin	21,872 bytes	0x0
partition-table.bin	3,072 bytes	0x8000
echo_rv32_usbjtag.bin (the app)	123,216 bytes	0x10000

The second-stage bootloader starts first, reads the partition table, and loads the app. The bootloader's offset is 0x0 on the S3 and C6, 0x1000 on the classic ESP32, and 0x2000 on the P4; see Flashing and the ROM bootloader.

Worked example

This is the C uppercase echo from Choosing a language, written for this page and not hardware-tested. It is the whole of main/echo.c in a project whose main/CMakeLists.txt lists SRCS "echo.c".

#include <stdio.h>
#include "freertos/FreeRTOS.h"
#include "freertos/task.h"

void app_main(void)
{
    while (1) {
        int c = getchar();                  // EOF: no byte waiting yet
        if (c == EOF) {
            clearerr(stdin);
            vTaskDelay(pdMS_TO_TICKS(10));  // sleep 10 ms so other tasks run
            continue;
        }
        if (c >= 'a' && c <= 'z') c -= 0x20;   // 'a' (0x61) becomes 'A' (0x41)
        putchar(c);
        fflush(stdout);                     // stdout is line-buffered: send now
    }
}

Line	What it does
getchar()	Asks ESP-IDF's console for one byte. Espressif's Standard I/O guide says console reads are non-blocking by default, so it returns EOF (minus 1) when nothing is waiting.
clearerr(stdin)	Resets the stream's "nothing came" flag so the next getchar really asks again.
vTaskDelay(pdMS_TO_TICKS(10))	Sleeps about 10 ms. pdMS_TO_TICKS turns milliseconds into FreeRTOS ticks; with the default 100 ticks per second, 10 ms is 1 tick. Sleeping lets the idle task run, so the task watchdog stays happy.
c -= 0x20	q is 0x71; 0x71 minus 0x20 is 0x51, Q.
fflush(stdout)	Standard output is line-buffered, so a single letter would otherwise wait for a newline.

The console it talks to is set by the project's configuration, so the same file builds for any config: UART0 on lx6-uart0 and lx7-uart0, USB-Serial-JTAG on the others. One detail from ESP-IDF 5.4.1's settings: by default the console turns an incoming carriage return (Enter, 0x0D) into a newline (0x0A) on input, and a newline back into carriage return plus newline on output, so Enter behaves the way a terminal expects.

The Inspector's echo payloads use the same project structure with assembly in place of C. Their main/CMakeLists.txt (hardware-proven) is one line:

idf_component_register(SRCS "echo.S"
                        INCLUDE_DIRS ".")

echo.S defines a global symbol named app_main, and the linker cannot tell which language it came from. ESP-IDF starts, calls app_main, and the assembly loop runs forever. Because that loop never yields, those payloads turn off the task watchdog in sdkconfig.defaults; the C echo above sleeps instead.

In an ESP32 project

This is stage 2 of the pipeline on the hub, Write the program, and it reaches into stage 3: idf.py flash is the flashing step, and the bootloader it writes is the one The boot sequence walks through. Stage 4 and 5 work in C, from GPIO to WiFi, uses components named in PRIV_REQUIRES, as the keyboard host does.

Common mistakes

These come mostly from the author's ESP-IDF build notes.

• Activating ESP-IDF from a Git Bash-derived shell on Windows. If the MSYSTEM variable is set, the setup script fails its Python check. Symptom: ERROR: MSys/Mingw is not supported and idf.py never appears.
• The wrong P4 silicon revision. ESP-IDF treats rev 1.x and rev 3.x P4 chips as separate targets; the author's rev 1.3 unit needs CONFIG_ESP32P4_REV_MIN_100=y. Symptom: an image that will not boot. Delete a stale sdkconfig before switching.
• Comments in .S files that look like preprocessor lines. ESP-IDF runs .S files through the C preprocessor, so a comment line starting # if or # else is read as a directive. Symptom: a build error pointing at a comment.
• A loop that never yields. Symptom: a task watchdog message and backtrace every few seconds. Sleep, block on a queue, or turn the watchdog off on purpose.
• Expecting the Super Mini ESP32-S3 to run after flashing. Its USB reset leaves it in download mode. Symptom: the log ends at DOWNLOAD(USB/UART0). Press RST or replug.
• Console on the wrong port. A project configured for UART0 prints nothing on a native USB port. Symptom: a silent monitor while the program runs.

Cost

The app is large for what it does: the C6 echo's 123,216 bytes (about 120.3 KB, with 1 KB = 1024 bytes) are almost all ESP-IDF's startup, FreeRTOS, and console support around one small loop, and adding WiFi adds far more. RAM goes to every task's stack, which you size by hand (4096 bytes for the keyboard host's USB task). Speed is the reward: compiled C runs at full core speed. Edit-to-run is the price: every change means a rebuild and a reflash, and the first build of a project compiles the whole framework. The author's notes warn that a combined build and flash can run longer than ten minutes, so they run it as a background job.

Going further

• Flashing and the ROM bootloader, for what idf.py flash writes and where
• The boot sequence, for what runs before app_main
• RISC-V assembly, for the payloads that put assembly in main/
• Espressif's ESP-IDF Programming Guide sections on the build system, FreeRTOS, and Standard I/O
• The example projects that ship inside ESP-IDF, which the author's keyboard host started from