AU.← Engineering answers

Edge AI · Practical answer

Why does an ESP32-CAM model return wrong or constant predictions?

The model is often fine. The usual failure is a mismatch between the camera frame and the tensor the model was trained to receive.

By Anhaj Uwaisulkarni6 min read
Direct answer

Check four things first: the camera buffer must contain decoded pixels rather than JPEG bytes; its width, height and channel order must match the input tensor; preprocessing must match training; and quantized inputs must use the model’s scale and zero point. Validate the same TFLite model on a computer before flashing it.

1. Confirm what the camera buffer contains

The ESP32 camera frame includes its pixel format. If the camera is configured with PIXFORMAT_JPEG, fb->buf contains compressed JPEG data, not a flat RGB image. Copying those bytes directly into a model tensor produces meaningless input.

Use a raw format supported by your pipeline, such as RGB565 or grayscale, or decode the JPEG before resizing and converting channels. Always inspect fb->width, fb->height, fb->format and fb->len before inference.

2. Match the tensor exactly

A model expecting [1, 96, 96, 3] needs 96×96 RGB pixels in the same channel order used during training. TensorFlow Lite stores the tensor as a flat buffer, but flattening does not remove the requirement to preserve row, column and channel order.

Print the input tensor’s dimensions and type at startup. If the camera image has a different size, resize it deliberately; do not rely on copying a shorter or longer buffer.

3. Reproduce training preprocessing

If training used values from 0 to 1, a float model normally needs each pixel divided by 255. If training used values from -1 to 1, apply that exact transform. RGB/BGR swaps, grayscale conversion and crop position can also destroy predictions even when the model invokes successfully.

4. Handle quantized models correctly

For an integer input tensor, use the quantization parameters stored in the model. The conversion is conceptually q = round(real / scale) + zero_point, clamped to the tensor’s integer range. Do not assume that every uint8 model accepts untouched 0–255 camera bytes.

Use a representative dataset that has the same preprocessing and visual distribution as deployment images. A calibration mismatch can make an otherwise accurate model appear stuck.

5. Validate before deploying

  1. Save one real camera frame and run it through the desktop TFLite interpreter.
  2. Print the model’s input shape, dtype, scale and zero point.
  3. Compare the first 20 preprocessed values on the computer and ESP32.
  4. Test an obvious positive image and an obvious negative image.
  5. Only then investigate tensor-arena size, unsupported operations or memory corruption.

Return every acquired frame with esp_camera_fb_return(fb). On memory-constrained boards, reduce input resolution, use an int8 model, keep frame buffers in PSRAM where available and register only the operations the model actually uses.

Primary references