GPU is a graphics processor optimized for parallel computing, initially created for processing video game graphics. It has proven to be very effective in AI and Deep Learning computations.