The end to end approach of regressing steering wheel angle already exists, check...

		deepnotderp on Jan 23, 2017 \| parent \| context \| favorite \| on: Lane Following Autopilot with Keras and Tensorflow The end to end approach of regressing steering wheel angle already exists, check nvidia's paper.