From Robot Tank to Autonomous AI: The Saga So Far

What began as a tracked chassis with a camera and basic remote control has become a test platform for local AI, computer vision, autonomous navigation, embedded safety, sensor fusion, and custom mechanical design. It can be driven from a browser, make navigation decisions using a locally hosted vision model, fall back to onboard obstacle avoidance, and report what it sees and why it chose its next move.

It has also tried to run away, browned out its own camera controller, crashed under too many network connections, driven into corners, and forced me to redesign the turret six times.

In other words, it has been a proper robotics project.

This post is the story so far: what I built, what failed, what those failures taught me, and where I am taking the robot next.

The Starting Point#

The early project notes referred to the platform as an ELEGOO Conqueror because that was the control ecosystem I began working from. The physical build has since evolved beyond that starting point. The robot now uses a red aluminum tracked suspension chassis as the mechanical base.

The initial goal sounded straightforward:

Put a camera on a tank, send the video to an AI model, and let the model drive.

The first architecture reflected that simple idea:

1. An Arduino controlled the motors and basic sensors.
2. An ESP32 camera provided Wi-Fi, video, and a control API.
3. A server in my homelab received camera images.
4. A local vision-language model examined those images.
5. The server translated the model’s response into movement commands.

Each part worked on its own fairly quickly. Making all of them work together safely and reliably turned out to be the real project.

Keeping the AI Local#

I wanted the robot’s vision and reasoning to run in my own homelab instead of depending on a cloud service.

The control system now spans three layers:

Robot
  Arduino R4 / UNO-compatible controller
    Motors, servos, ultrasonic sensor, IR remote, safety watchdog

  ESP32-S3-CAM
    Camera, Wi-Fi, HTTP control, telemetry, serial bridge

Homelab
  Proxmox robot container
    Flask API, web dashboard, AI navigation loop, simulation

  GPU Ollama container
    NVIDIA GTX 1070, local vision-language model

The ESP32 sends camera frames to the robot service running in a Proxmox container. That service asks a local Ollama vision model to choose a navigation action and return a small, predictable JSON response. Before any command reaches the motors, a hard-coded safety layer checks it.

That separation is important. The AI can suggest a direction, but it does not get unrestricted control over motor duration or speed. Its output is treated as untrusted input.

A typical control loop looks like this:

Camera frame
    ↓
Local vision model
    ↓
Structured navigation decision
    ↓
Safety and motion limits
    ↓
ESP32 control endpoint
    ↓
Arduino motor command

The web dashboard shows the camera, telemetry, controls, and the AI’s current reasoning. I can switch between manual and AI control, stop the robot immediately, and see what the model believed it was looking at.

That last part is more useful than I expected. Watching an AI make a bad decision is one thing. Watching the image and reasoning that produced the decision makes debugging much more productive.

Teaching It to Drive#

The first autonomous behaviors were deliberately small.

I added ultrasonic obstacle detection, gyro-assisted turns using an MPU-6050, battery telemetry, and camera pan and tilt. The AI navigator learned basic priorities such as avoiding nearby obstacles and treating a detected person as an interesting target.

The robot also gained an onboard Explore mode. This mode does not require the vision model or even a working network connection. The Arduino uses the ultrasonic sensor and timed movements to move through a space, back away from obstacles, alternate turn directions, and escape when it detects that it is stuck in a corner.

Explore mode can be started from:

• The web dashboard
• A physical button
• The IR remote

This became one of the project’s most important design decisions. A remote AI brain is useful, but the robot still needs a small, dependable local brain when Wi-Fi, video, or the server is unavailable.

The robot also gained:

• A Wi-Fi signal meter
• A live AI enable/disable control
• Visible AI reasoning in the dashboard
• IR remote driving
• Gyro-calibrated turns
• Battery voltage reporting
• Camera pan and tilt
• Manual control that preempts autonomous movement
• A simulation mode for testing without the physical robot

The simulator includes a virtual arena and rendered camera view. It lets me run the Flask backend, dashboard, and navigation loop without putting the tank on the floor every time I change the software.

That shortened the feedback loop tremendously. It also reduced the number of software mistakes that could become hardware accidents.

The Runaway Incident#

The most important moment in the project was not a successful autonomous drive. It was the day the tank failed to stop.

I already had a watchdog. If the robot stopped receiving control messages, it would stop the motors. On paper, that sounded safe.

The problem was that a stuck browser key continued sending valid movement messages. From the robot’s point of view, communication had not failed. It was still receiving fresh commands, so the watchdog did exactly what it had been designed to do and kept allowing motion.

The tank kept moving.

That incident changed the way I thought about the entire system. A watchdog protects against missing commands. It does not protect against commands that are current, valid, and wrong.

I rebuilt the safety system in layers:

The ESP32 watchdog is set around 500 milliseconds, the original Arduino failsafe around one second, and the later hardware watchdog is tighter still. The dashboard also has its own dead-man behavior.

The exact numbers matter less than the lesson: no single safety mechanism is enough.

If a robot can move, safety has to cover multiple failure classes:

• The network disappears.
• The controller crashes.
• The browser gets stuck.
• The AI returns nonsense.
• A command is valid but lasts too long.
• The operator closes the page.
• One processor keeps running while another has failed.

The runaway incident was uncomfortable, but it moved the project from “remote-controlled experiment” toward an actual robotics system.

The 180-Degree Servo Lesson#

The pan-and-tilt camera produced another memorable failure.

The servos were allowed to drive into their mechanical stops. They stalled, pulled down the 5-volt rail, and caused the ESP32 to brown out. The servos became hot enough to make the lesson impossible to ignore.

The fix was not just changing an angle value. I changed how servo motion was handled:

• Mechanical limits were reduced to a safe operating range.
• Movement was slewed in small steps instead of jumping instantly.
• Only one servo was energized at a time.
• Servos detached after movement instead of holding continuously.
• The camera range was tested against the real mechanism.

The practical limits settled around 30 to 150 degrees, with roughly 3-degree steps every 15 milliseconds and automatic detach after about 800 milliseconds.

This solved both a mechanical problem and an electrical one. It also reinforced a recurring rule in the project:

Software limits are part of the hardware design.

Current limits, timeouts, travel limits, cable protection, and service access cannot be added as an afterthought. They have to be designed together with the physical mechanism.

When More Connections Made It Less Reliable#

As the dashboard became more capable, the ESP32 became less stable.

The camera stream, control requests, and status polling all competed for connections. Opening the MJPEG stream while repeatedly asking for telemetry and sending drive commands could push the camera controller into a reboot loop.

At first this looked like a random hardware problem. It was really a concurrency problem.

I reduced connection churn by:

• Serializing access to the ESP32
• Polling status in the background and caching the result
• Reducing unnecessary dashboard requests
• Broadcasting a single backend video stream to multiple viewers
• Avoiding silent fallback into simulation when the real robot disappeared

The system became much more usable, but this remains one of the places where the project taught me to respect resource limits. An embedded web server is not a desktop server. Every extra stream and status request has a cost.

Building a Better Camera Controller#

The original camera hardware eventually became another bottleneck, so I moved toward a GOOUUU ESP32-S3-CAM board with:

• 16 MB flash
• 8 MB PSRAM
• An OV3660 camera
• Dual USB-C connectors
• Exposed GPIO for additional sensors and peripherals

The updated firmware has been compile-verified and supports a much higher camera resolution than the earlier QVGA setup. The real-world balance between image quality, frame rate, AI response time, and Wi-Fi reliability still needs to be measured on the completed assembly.

That tradeoff matters. A larger image may help the vision model, but a robot that receives a perfect image too late can still hit the wall.

The Turret Became Its Own Project#

Once the electronics and software started to mature, the exposed camera mast no longer made sense. I wanted the camera, controller, lights, radar, and traverse mechanism enclosed in something that looked intentional.

That began the turret project.

The first version was an original compact design inspired by the T71 DA silhouette and mounted on a 99.7 mm lazy Susan bearing. It established the basic shell, roof, bearing adapter, mantlet, and short camera barrel.

From there, each revision solved a real packaging or service problem.

V3: High-Angle Camera Movement#

V3 replaced the square mantlet with a round camera drum driven by an SG90 servo. The modeled camera travel is 45 degrees down to 75 degrees up.

The camera sits inside a removable mantlet and looks through the short barrel. Six 5 mm headlights are integrated into the turret cheeks. Fit coupons were designed for the servo, servo horn, and bearing because low-cost components often vary enough to ruin a large print.

V4: Powered Turret Traverse#

V4 added powered rotation using a 5-volt 28BYJ-48 stepper motor and a printed 112-tooth ring gear driven by a 16-tooth pinion. That adds a 7:1 reduction beyond the motor’s internal gearbox.

A Hall sensor provides a repeatable center reference, while physical stops at approximately plus or minus 165 degrees protect the wiring from being twisted indefinitely.

The first complete FreeCAD installation exposed a collision that the isolated part models had missed: the motor occupied the same space as the front camera area.

The solution was to move the turret axis 25 mm behind the tank center and relocate the motor behind the ring, hanging downward with the pinion facing up. That preserved the camera area and kept the turret’s useful sweep.

This was a good reminder that individual parts can look perfect while the assembly is impossible.

V5: Radar#

V5 added a roof pod for an HLK-LD2450 2.4 GHz mmWave radar.

The radar is intended to detect and track people even when lighting is poor. Its measured enclosure aims the sensor 15 degrees upward and keeps metal, LED wiring, and conductive material away from the radar face.

The first ESP32 tray in this version was based on vendor dimensions and was intentionally provisional.

V6: Designed for Service#

When the actual GOOUUU board arrived, I measured it at 67.6 × 28.2 × 12.7 mm and redesigned the electronics packaging around the real board.

V6 places the ESP32 in a removable crosswise cassette directly behind the mantlet. A 77 × 42 mm armored service hatch allows the board to lift out while the turret remains installed. The USB-C connectors and microSD card can be reached without disassembling the entire robot.

The camera moved into a removable cartridge at the muzzle. Its ribbon cable has a modeled service loop that supports the full elevation range. Screw-on retainers replaced fragile printed snap features.

This version is less about adding another feature and more about making the robot maintainable.

That is an evolution I did not anticipate at the beginning. The first turret asked, “Can all of this fit?” The sixth asks, “Can I repair it after it fits?”

V7: A Complete Redesign in Autodesk Fusion#

Version 7 is not another adjustment to the existing turret. It is a complete redesign, and it marks my transition to Autodesk Fusion as the primary design environment for the project.

Versions 1 through 6 were developed through a combination of OpenSCAD and FreeCAD. OpenSCAD’s parametric approach was useful for producing repeatable dimensions, fit coupons, and printable revisions. FreeCAD made it possible to inspect the turret, bearing, motor, camera, and tank deck as a complete installed assembly. That assembly view exposed collisions that were easy to miss when the parts were modeled separately.

Both are good tools, just as KiCad is a strong option for schematics and PCB design. The challenge is that the robot’s mechanical and electrical systems are now tightly connected. Board dimensions affect the shell. Connector locations determine service access. Sensor placement affects the roof, wiring routes, and fields of view. A mechanical change can immediately create an electrical or packaging problem.

Autodesk Fusion gives me one environment for both 3D modeling and electrical design. For V7, I can treat the turret structure, electronics, mounting points, clearances, and service access as parts of one system instead of moving between separate mechanical and electrical workflows.

The switch does not mean I have abandoned KiCad. It will continue to have a place in my projects, particularly for dedicated schematic and PCB work, and I have not given up on learning it. Fusion simply makes more sense as the primary environment for this complete turret redesign.

Sensors, Lights, and the Pin-Budget Problem#

The next hardware layer extends the robot’s awareness beyond the front ultrasonic sensor and camera.

Work completed or designed includes:

• Armored front bump whiskers for the left and right track corners
• Headlight control through a MOSFET
• A turret-mounted mmWave radar pod
• Space and wiring plans for additional sensors

The bump sensors use normally closed switches so a broken wire looks like a collision instead of silently disabling protection. The printed mechanism spans the track corners that fall outside the ultrasonic sensor’s narrow cone.

A rear ultrasonic sensor is planned to address the robot’s blind spot while reversing. Today, reverse movement is deliberately capped because the robot cannot yet confirm what is behind it.

Adding sensors exposed another ordinary but important engineering constraint: pins.

The Arduino is nearly out of available I/O. The current plan is to keep time-critical collision behavior close to the motor controller while moving the rear ultrasonic and radar interfaces to the ESP32 or, later, a ride-along Raspberry Pi.

This is where robotics stops being a sequence of exciting features and becomes systems engineering. Every sensor affects wiring, power, processing, mounting, telemetry, and failure behavior.

What Is Working Now#

The project currently has a functioning foundation:

• Browser-based manual driving
• ESP32 camera streaming and HTTP control
• Arduino motor and sensor control
• Local AI navigation through Ollama
• A Flask control API and dashboard
• Live telemetry and AI reasoning
• Simulation without the physical robot
• Onboard Explore mode
• IR remote control
• Gyro turns and battery monitoring
• Layered motor safety and manual override
• Home Wi-Fi operation with access-point fallback

The newer ESP32-S3 firmware, lighting control, sensor mounts, and turret system are at various stages of compile verification, CAD validation, printing, wiring, and real-world test fitting. I am intentionally not calling the robot “finished” just because the models render correctly.

The current turret is a prototype until the fit coupons and the complete physical assembly have been tested.

What Comes Next#

The next stage is less about making the robot move and more about helping it understand where it is.

The roadmap includes:

1. Print and test the turret fit coupons before committing to the full assembly.
2. Install and validate the GOOUUU ESP32-S3-CAM.
3. Wire the headlights without loading the fragile controller rail.
4. Add the front bump sensors and their local stop behavior.
5. Integrate the mmWave radar for person tracking.
6. Add rear obstacle sensing.
7. Improve Wi-Fi roaming and embedded connection handling.
8. Evaluate 2D LiDAR for mapping, localization, and SLAM.
9. Revisit a ride-along Raspberry Pi when onboard perception justifies it.

I have looked at long-range radio, additional ultrasonics, LiDAR, and a Coral TPU. Some of those ideas will make it onto the robot and some will not. LoRa, for example, is useful for low-bandwidth control but cannot carry the camera stream, so it no longer has a place in the current turret payload.

That kind of pruning is part of the process. A robot does not improve by carrying every interesting board I own.

What This Project Has Taught Me#

The biggest lesson is that autonomy is not the AI model.

The model is one component in a chain that includes sensing, networking, structured output, safety checks, embedded control, mechanical limits, power distribution, and human override.

A few principles now guide the project:

• The AI proposes; deterministic code disposes. Model output always passes through strict limits.
• Stopping is more important than moving. Every control layer needs a safe failure mode.
• A local fallback matters. The robot should still avoid obstacles if the server or Wi-Fi disappears.
• Simulation is a safety tool. It catches software mistakes before the tracks touch the floor.
• Power problems often look like software problems. Servo stalls and voltage sag can masquerade as random controller crashes.
• Mechanical serviceability matters. A sealed design is not finished if normal maintenance requires destroying it.
• The complete assembly is the truth. Part-level CAD cannot reveal every collision or cable problem.
• More sensors are not automatically more intelligence. Each sensor needs a purpose, a mounting plan, a power budget, and a defined role in decision-making.

The Saga Continues#

The robot is not yet the fully autonomous machine I pictured at the start, but it is far more capable – and far better understood – than the original prototype.

It can see, report, accept manual control, explore locally, and make limited navigation decisions using an AI model running in my homelab. More importantly, it now has the beginnings of the safety architecture and mechanical design needed to make those abilities useful outside a demo.

The next milestone is to bring the new turret, ESP32-S3 camera, radar, bump sensors, and improved power system together on the physical chassis. After that, LiDAR and localization can move the project from reactive obstacle avoidance toward deliberate mapping and navigation.

I started with the question, “Can AI drive this tank?”

The better question turned out to be:

What does it take to build a robot that can use AI, survive its own failures, and still be trusted to move?

I am still answering that one.