Why Your Robot Needs Two Senses, Not One: Sensor Fusion at the Edge

Macro shot of robotic arm joint with IMU sensor chip, overlaid graph showing sensor noise being smoothed by fusion filter

The first time I mounted an MPU6050 on a robot arm and expected clean, trustworthy angle data, I got garbage. Not "slightly off" garbage — drift-into-oblivion garbage, where the reported orientation would wander several degrees off true within a couple of minutes even sitting still on a table. My first instinct was to blame the sensor. It wasn't the sensor's fault. It's just what accelerometers and gyroscopes do on their own, and almost nobody tells you that up front — most tutorials show you the happy path and skip the part where your robot slowly forgets which way is up.

This post is the fix I wish someone had handed me: sensor fusion. Specifically, how to combine two noisy, individually-flawed sensors into one signal good enough to actually control a robot with — and how to do it on hardware too small to run a "real" Kalman filter library off the shelf.

Why one sensor is never enough

Every sensor lies to you in a specific, predictable way:

  • Accelerometers give you an absolute reference (gravity), but they're hypersensitive to vibration and linear motion. Every time the robot moves, the accelerometer briefly "thinks" the vertical axis has tilted, because it can't distinguish gravity from acceleration.
  • Gyroscopes are smooth and immune to vibration, but they only measure rate of rotation. Integrate that rate to get an angle, and you're also integrating every tiny bias error in the sensor. Over time that bias accumulates into drift — the wander I saw on my bench.
  • Cameras (from my earlier post on edge vision) give you rich absolute position data, but at 15-45ms per frame it's far too slow and irregular to drive a control loop directly.

None of these are "bad" sensors. They're answering different questions at different speeds. Fusion is the art of asking each sensor only the question it's good at answering, then blending the answers.

The complementary filter: the 90% solution

Before reaching for a full Extended Kalman Filter (EKF), it's worth understanding the complementary filter — it solves most of the accelerometer/gyro problem with about four lines of code and no matrix math.

The idea: trust the gyroscope for fast changes, and trust the accelerometer to slowly correct the gyro's long-term drift.

// Complementary filter — runs at 1kHz alongside your control loop
float angle = 0.0f;
const float ALPHA = 0.98f; // trust gyro 98%, accel 2%
const float dt = 0.001f;   // 1kHz loop

void updateAngle(float gyroRate, float accelAngle) {
  angle = ALPHA * (angle + gyroRate * dt) + (1.0f - ALPHA) * accelAngle;
}

That's it. The gyro term handles moment-to-moment motion smoothly; the accelerometer term acts like a slow leash, gently pulling the estimate back toward "true" whenever it wanders. Tune ALPHA closer to 1.0 if your robot vibrates a lot (trust the gyro more); closer to 0.9 if drift is your bigger problem.

On my RRP arm's end-effector orientation, this alone cut drift from several degrees per minute to something imperceptible over a typical task duration. It's not textbook-optimal, but it's the filter that actually ships in a huge number of consumer drones and gimbals, because it's cheap, stable, and good enough.

When complementary isn't enough: fusing in vision

The complementary filter fixes gyro drift using the accelerometer. But if your robot also has a camera doing object tracking, you have a second, richer correction source — just a slow, irregular one.

This is where you graduate to a simple Kalman structure: treat vision as a low-rate "truth update" between high-rate IMU predictions.

// Predict-correct structure, running on Core 0 (1kHz)
// Vision updates arrive asynchronously from Core 1 via FreeRTOS queue

float state_estimate = 0.0f;
float estimate_uncertainty = 1.0f;

void predict(float gyroRate, float dt, float process_noise) {
  state_estimate += gyroRate * dt;
  estimate_uncertainty += process_noise;
}

void correctWithVision(float visionAngle, float measurement_noise) {
  float kalman_gain = estimate_uncertainty / (estimate_uncertainty + measurement_noise);
  state_estimate += kalman_gain * (visionAngle - state_estimate);
  estimate_uncertainty *= (1.0f - kalman_gain);
}

This should look familiar if you read my post on decoupling the 1kHz PID loop from 10-60Hz inference across FreeRTOS cores — it's the same dual-rate architecture. The IMU predicts continuously; whenever a vision frame lands in the queue, it corrects. The Kalman gain automatically decides how much to trust the new measurement versus the running estimate, based on how noisy each source has been.

A hardware note that will save you a debugging afternoon

If you're doing this on an ESP32 or similar, don't read the IMU over I2C inside your 1kHz control ISR. I made that mistake early on — I2C transactions can stall long enough to blow your loop timing, and it shows up as mysterious jitter that looks like a tuning problem but is actually a bus problem. Read the IMU in a separate task, buffer the latest reading, and let the control loop consume whatever's freshest.

FAQ

Do I need a Kalman filter, or is a complementary filter enough?
If you're only fusing an accelerometer and gyro, a complementary filter is usually enough and is far easier to tune. Reach for a Kalman filter once you're fusing a third, slower source like vision or GPS.

What's the minimum hardware to try this?
An ESP32 or similar Cortex-M part, an MPU6050 (or any 6-axis IMU) over I2C, and that's it for the complementary filter. No camera required to get started.

Why not just use a library?
Libraries are great once you understand what's happening underneath. Writing it yourself first means you can actually debug it when it misbehaves in the field, instead of guessing at a black box.

Where this goes next

Sensor fusion is the quiet infrastructure underneath almost everything else in this series — the vibration anomaly detector trusts its accelerometer readings because of exactly this kind of filtering discipline, and the actuation control loop is only as good as the state estimate feeding it. If you're building anything that moves and senses its environment at the same time, this is the layer to get right before you touch the fancier stuff.

Working on something similar? I'm always up for comparing notes on embedded ML, robotics control, or edge AI projects — whether that's a code review, a second pair of eyes on a weird bug, or teaming up on something bigger. Drop me a line at haque.uet@gmail.com.

Comments

Post a Comment

Popular posts from this blog

Vision on the Edge: Real-Time Object Detection on Low-Power Microcontrollers

Image
Vision on the Edge: Real-Time Object Detection on Low-Power Microcontrollers Running neural networks on accelerometer or temperature data is one thing; processing real-time video streams on a chip that costs less than a cup of coffee is an entirely different engineering challenge. Computer vision traditionally demands high-end GPUs, massive memory bandwidth, and active cooling. Yet today, Edge Vision allows microcontrollers consuming under 1 watt to detect objects, track faces, and classify defects in real time. This article breaks down how micro-vision pipelines work, how to bypass memory bottlenecks, and how to implement real-time inference on resource-constrained silicon. The Memory Bottleneck: Why Vision Breaks MCUs A single uncompressed RGB image at standard 1080p resolution takes up over 6.2 MB of RAM: 1920 × 1080 × 3 bytes (RGB) ≈ 6,220,800 bytes (6.22 MB) When an embedded microcontroller (like an ESP32-S3 or Cortex-M7) only has 512 KB to 1 MB of internal SRAM...
Read more

From Math to Microcontroller: A Practical Guide to Deploying Neural Networks on Embedded Silicon

Image
From Math to Microcontroller: A Practical Guide to Deploying Neural Networks on Embedded Silicon In our previous deep dive on AI in hardware, we explored why machine learning is moving from cloud data centers directly into microcontrollers and edge chips . But moving from theory to execution introduces a fundamental challenge: how do you compress a floating-point neural network into a chip with only a few hundred kilobytes of RAM without destroying its accuracy? This guide breaks down the end-to-end engineering pipeline required to train, quantize, and flash a machine learning model onto embedded hardware like an ESP32 or ARM Cortex-M . The Embedded Constraint Problem A standard deep learning framework like PyTorch or TensorFlow assumes infinite hardware resources: Format: 32-bit single-precision floating point ( float32 ). Memory: Multi-gigabyte VRAM / RAM footprints. Instruction Set: Heavy use of dynamic memory allocation ( malloc ) and continuous GPU tens...
Read more

Beyond the Cloud: The Rise of AI in Embedded Hardware

Image
  Beyond the Cloud: How AI is Moving into Silicon and Embedded Hardware When most people think about Artificial Intelligence, they picture multi-billion-dollar cloud server farms running massive Large Language Models. But a quieter, equally transformative revolution is happening at the milliwatt scale: Edge AI and TinyML . Instead of piping raw sensor data across the internet to be processed in the cloud, engineers are now embedding neural networks directly onto microcontrollers, dedicated Neural Processing Units (NPUs), and Field Programmable Gate Arrays (FPGAs). Here is why hardware-level AI matters, how the architecture works, and where the industry is heading. Why Cloud AI Fails in the Physical World Sending sensor feeds to an API endpoint works fine for chatbots, but breaks down when controlling physical systems: Deterministic Latency: A robotic arm or autonomous drone cannot wait 300 ms for a round-trip cloud ping to avoid an obstacle. Inference must occur in single-digit ...
Read more