Regardless of what glue language you use to write ML stuff, if you follow it down to its core, most of it is going to use a BLAS library to do matrix multiplication. That's where the heavy lifting is. And BLAS libraries are in C/C++ or Fortran. For example, OpenBLAS or Intel MKL.
I thought Fortran's aliasing-free rule was just an assumption of the compiler, not something that it actually enforced.. so you could very easily write code that contained aliasing bugs.