I updated this post with some of the great feedback from the comments. Also I just ported the algo used by the last DIYRobocar race winner, CompoundEye. Here's that post: https://wroscoe.github.io/compound-eye-autopilot.html#compou...

Thanks!

Well shit. Thanks for pointing that out. I'll revise.

The input values are image arrays 120x160 pixels with 3 channels for red,green,blue. The values range from 0-255 and are not normalized before they are fed into the convolution layer. I found this did not make a difference.

The output of the model is a single real number between -90(left) and 90(right). I believe a better approach would be to bin the outputs and use a classifier. This way you'd know when the model was getting confused (ie, approaching a perpendicular line.

Tensorflow anounced last week that it will incorporate Keras as its default higher level abstraction. It's a pleasure to use.

Traditional CV approaches are much easier to debug as well. I chose the ML approach with the assumption that it would be easier to build a robust autopilot that would work in many lighting conditions. Actually my short term goal is to get the car to drive around my block on the sidewalk (no lines). From my experience CV approaches have many parameters that need to be tuned specifically for each environment. While ML approaches also have parameters that need tuning they stay constant between environments.

I see, that makes sense. It'd be indeed worth it if we can apply a model trained on controlled environment to a more challenging one with little to no modification. Good luck with the project and keep us updated!

Yep, this was an exercise to compete in the DIYRobocars race in West Oakland last weekend. There were 9ish cars with 7 running end to end Tensorflow autopilots and the others using OpenCV/line detection. Open CV one the race.

I'm working on adding the throttle. This is difficult because you need to drive the correct speed and stopping or running off course can mess up the training data. This project was inspired by Otavio's carputer which does predict throttle, steering angle, and odometer.

The method to avoid overfitting was to use the model with the lowest validation loss, not training loss. I was able to change the track around my house with reasonable success. I think would need many more example turns in the training data to become robust.

I copied this method of loading datasets from Keras. https://github.com/fchollet/keras/blob/master/keras/datasets.... What's a better alternative.

Another serialization format which doesn't create objects, like JSON, XML, CSV,...

Donkey runs a client on the Pi and a remote server that runs Keras/Tensorflow.

A ha! Very cool- apologies for not seeing how it worked at first; I assumed you used the server to control/collect data manually, and then loaded the model onto the device. Thanks for the demo!