Changing programming paradigms in the age of AI

Machine learning is changing the way we program. What earlier used to be a transfer of logic to the machines is becoming more of an exercise of transfer of data to machines. The challenge and thrill of coding complex systems with intricately written logical structures would soon become a thing of the past. Machine learning for most part has reduced all the programs into large matrix multiplications. Given the user provides enough data with matching inputs and outputs, machine just has to solve a huge matrix multiplication like:

ς(ς(ς(Input  x Mat1) x Mat2) x Mat3) … = Output

Mat1, Mat2, Mat3 … are the only things that are to be determined through already well developed methods and this can solve almost any programming task. If you have enough data finding multiplier matrices is pretty trivial. And these matrices are your program!

To give an idea of how radically different it is, one may consider the MNIST handwritten letter recognition problem. If one writes a code to identify these digits through conventional coding paradigms, this might take thousands of lines of code in any high level language, while a similar machine learning code from scratch can do the job in less than 50 lines of code. In a conventional coding paradigm, you will be thinking about shapes of letters, relative brightness of pixels, curve fitting etc. but one can write a machine learning code by almost being blind to the data. One concentrates rather on getting the matrix multiplications right. Once that is done, machine finds the most optimal matrix multiplier to map the input to the output; and in a way, writes its own code!

I sometimes ask this question in class that, if you write a highly optimized ten thousand line code to recognize cars in conventional method vs google leverages its millions of car images to just train a two layer neural network which code will win? Of course google will win. Disturbingly, this also means that data is the new oil. One who has more of it, will have more power.

Leave a Reply

Your email address will not be published.