collapse

Author Topic: Autonomous RC car steers with CNN (deep learning)  (Read 314 times)

maelh

  • Member
  • ****
  • m
  • Posts: 29
Autonomous RC car steers with CNN (deep learning)
« on: September 04, 2020, 01:43:24 PM »
So, I worked on an autonomous RC car project, which uses deep learning to train a neural network that steers the car. I mentioned it already in chats with some members from here briefly.


The first part was enhancing a RC car, by modifying the remote control so I could attach it to an Arduino and steer the car over my laptop.




A Raspberry Pi with a camera streams to the laptop as well, which uses the current frame to predict using a CNN (convolutional neural net) the best steering command for the car.

It essentially categorizes each frame to decide which of the steering commands to pick: left, right, straight ahead.

Pictures of the build and more details here:
https://blog.mh-nexus.de/2020/03/self-driving-car-based-on-deep-learning/

Since it was a bit too slow and the prediction not reliable enough, I tried the same in simulation, so I could slow it down to wait for the neural net to finish predicting, but also adjusting parameters like camera's FOV, steering angle sharpness, or speed of the car.



It worked well from the very little training data (just one perfect left turn at 45°, perfect right turn at 45° and one portion driving straight ahead), and generalized to unknown tracks. Some driving looks imprecise, but this is due to the limited training data, and also due to the steering angles being fixed (the car only steer left and right to about -45° and 45°, respectively).
So the car approximates between the known categories (left, right, straight ahead), and sort of iteratively approximates the curves like that.
More training could make it a lot smother, if really desired.

The blog post here has videos of the car in action, and explains how the car drives:
https://blog.mh-nexus.de/2020/03/self-driving-car-based-on-deep-learning/

Next step:
Recently I found a kit for building up a car, that has Ackermann steering, so the same way as a real car, and it can host a Jetson Nano. With this increased power the prediction frame rate increased a lot so that the car responds to changes speedily (now the mechanics are the limiting factor).

I collected some training data by driving the car manually and taking pictures at every change in manual steering, then used Google Collab to train the CNN. It's been interesting to experiment with various lighting conditions (day, night, sun shinining directly inside the room), and moving the room's furniture to see how robust the predictions remain.

Either way, it successfully drove along a "road" delimited with paper sheets, on its own.

I am still waiting for some parts to come back from warranty to make a video, in case there's interest.
« Last Edit: September 04, 2020, 01:54:40 PM by maelh »

 

* Search


* Recent Topics

Autonomous RC car steers with CNN (deep learning) by maelh
[September 04, 2020, 01:43:24 PM]


The unnamed (yet) quatruped spider project by tinhead
[August 17, 2020, 02:05:48 PM]


"1984 Nixie Time" by 1 what
[May 08, 2020, 01:04:18 AM]


2D Side Scroller Cyberpunk themed by Killer Angel
[February 06, 2020, 06:39:40 AM]


A new wing design for model aircraft / drones by OddBot
[February 06, 2020, 04:42:06 AM]


SDR (Software Defined Radio) by Gareth
[February 02, 2020, 06:15:42 AM]


Circuit Math by ZeroMax
[January 31, 2020, 01:50:18 PM]


NanOMeter by Protowrxs
[January 01, 2020, 12:59:44 PM]


Investigating the VL53L0X Laser Rangefinder by erco
[December 30, 2019, 10:45:44 PM]


PS4 Single Handed Controller Deployed (part 7 of 7) by Gareth
[December 30, 2019, 09:52:29 AM]


"D" -Pad Workio just like Magic (Will Merlin stay or Go) (part 6 of 7) by Gareth
[December 30, 2019, 09:51:27 AM]


PS4 Joystick Digitals 4,5,6,7,10 - Analog's Lx,Ly,Rx,Ry Workio (part 5 of 7) by Gareth
[December 30, 2019, 09:50:37 AM]


Menu Workio ! (part 4 of 7) by Gareth
[December 30, 2019, 09:49:49 AM]


L1 trigger design Workio (Hori controller) (part 3 of 7) by Gareth
[December 30, 2019, 09:48:50 AM]


Hori aka PS4 Joystick Mappings (part 2 of 7) by Gareth
[December 30, 2019, 09:47:17 AM]

* Recent Posts

Autonomous RC car steers with CNN (deep learning) by maelh
[September 04, 2020, 01:43:24 PM]


Re: The unnamed (yet) quatruped spider project by tinhead
[August 17, 2020, 02:05:48 PM]


Re: The unnamed (yet) quatruped spider project by maelh
[August 17, 2020, 02:10:06 AM]


Re: The unnamed (yet) quatruped spider project by tinhead
[July 01, 2020, 04:22:11 PM]


Re: The unnamed (yet) quatruped spider project by jinx
[July 01, 2020, 04:06:19 PM]


Re: "1984 Nixie Time" by 1 what
[May 08, 2020, 01:04:18 AM]


Re: "1984 Nixie Time" by tomasp
[April 13, 2020, 06:03:28 PM]


Re: 2D Side Scroller Cyberpunk themed by Killer Angel
[February 06, 2020, 06:39:40 AM]


A new wing design for model aircraft / drones by OddBot
[February 06, 2020, 04:42:06 AM]


Re: "1984 Nixie Time" by Gareth
[February 02, 2020, 06:23:01 AM]


Re: SDR (Software Defined Radio) by Gareth
[February 02, 2020, 06:15:42 AM]


Re: SDR (Software Defined Radio) by ZeroMax
[January 31, 2020, 01:54:21 PM]


Re: "1984 Nixie Time" by ZeroMax
[January 31, 2020, 01:52:29 PM]


Circuit Math by ZeroMax
[January 31, 2020, 01:50:18 PM]


Re: 2D Side Scroller Cyberpunk themed by ZeroMax
[January 31, 2020, 01:45:33 PM]