Sensors and compute for autonomy

Two different problems

Autonomy divides cleanly into knowing where you are and knowing what is around you. They need different sensors, different maths and different amounts of compute, and conflating them is why beginner projects stall.

State estimation answers: what is my attitude, position and velocity. It runs at hundreds of hertz, it is unglamorous, and nothing works until it does.

Perception answers: what is out there. It runs at tens of hertz, it needs real compute, and it is what people think of as autonomy.

Sensors for state

• IMU: accelerometer and gyroscope. Attitude and angular rate. Every vehicle has one, and its noise floor sets your control bandwidth.
• Magnetometer: heading. Trivially disturbed by current carrying wires and by anything ferrous, which is most of a vehicle.
• Barometer: altitude by air pressure. Drifts with weather and with propeller wash, so it is fused rather than trusted.
• GNSS: absolute position, outdoors only, one to three metres. RTK brings that to centimetres with a base station.
• Optical flow and rangefinder: velocity and height above ground indoors, where GNSS is not available.

None of these is sufficient alone. They are fused, usually by an extended Kalman filter, and the filter is the part that matters.

Sensors for perception

• Depth cameras give a dense point cloud at short range. Stereo works outdoors, structured light does not.
• LiDAR gives accurate range at longer distances and in poor light, at a cost in money and mass.
• Monocular cameras give the most information per gram and the most compute per useful answer.

The choice follows the algorithm you intend to run, not the other way round. Deciding to do visual SLAM and then buying a LiDAR is a common and expensive mistake.

Interfaces matter more than you expect

How a sensor connects constrains how fast you can read it and how far it can sit from the computer.

• I2C: two wires, many devices, slow and short. Fine for a barometer, wrong for a camera.
• SPI: fast, short, one chip select per device. The usual choice for an IMU that has to be read at 8 kHz.
• UART: point to point, simple, the standard for GNSS.
• CAN: differential, noise tolerant, long runs, and the right answer once a vehicle gets big enough that wiring is a problem.
• USB and MIPI CSI: cameras, and effectively nothing else.

Compute, and the number that decides it

For anything running a neural network, the figure that matters on a vehicle is not raw throughput but TOPS per watt. Every watt is heat you have to remove and energy you are not spending on flight.

A rough ladder:

• Microcontroller, under a watt: control loops, sensor fusion, no perception.
• Single-board computer, 3 to 8 W: classical vision, SLAM, mission logic.
• Accelerated module, 10 to 60 W: real-time deep learning. A Jetson Orin class board is a meaningful fraction of a small aircraft's power budget and a large fraction of its thermal problem.

Run the control loop on a microcontroller and perception on the big board. Putting a flight-critical loop on a machine that runs a general purpose operating system is a decision you only make once.