A Reflection on Summer of ’19

This summer has been very productive for our lab. Apart from my students, we had three interns and lab was buzzing with activity for all two months. We moved our base into the main campus of IIT Ropar and are the first group of CBME to do so. After moving, productivity of all my students have seen a sea change. This goes on to show, how strong an effect does the environment and behavior of other people play in the energy that we feel at the workspace.

We have been able to produce seven conference papers out of our lab in this summer spanning across areas of imaging, EEG, transfusion, ultrasound and machine learning. All three interns — Setika, Shivendra and Ankur, performed very well.  Most of these papers shall be submitted to upcoming conferences of IEEE.  We made a great leap in machine learning. I conducted more than 20 sessions for my students covering almost all important fundamentals of machine learning and deep learning. Now our lab is machine learning compliant :). I shall be offering a institute course on applied machine learning with emphasis on medical image and signal processing in the spring semester. All students helped me in multiple grant writing assignments and I am sure it is preparing them well for the future career. We acquired few hi-tech gadgets. HP Z4-G8 workstation, one of the fastest GPU workstations and HP Mixed reality headset are some of the notable additions.

The new semester is about to dawn upon us. I have two teaching assignments this semester – my flagship course on Biomedical Electronics and Communication (BM605) and part of Human Physiology (BM601). Last year, I really enjoyed designing BM605 and student feedback was excellent. This year, although it is pretty busy, I plan to add few more components to enrich this course further. Exams shall continue to be open book!

I understand that my students have been grilled a lot this summer with all the relocation, learning (may be forced learning 😉 ), presentations and writing. I would like to thank all of them for their support.

Machine Learning from Brain

Current Intel-i7 processors have about 1.75 billion transistors with a density of about 17 million transistors per sq mm. Human brain in comparison has 100 billion neurons with a density of less than a hundred thousand neurons per cubic mm. Speed of a transistor today is approaching pico-second switching times in fast processors. In comparison, a neuron can only switch at a rate of 1 ms. 109 times slower!

Although brain appears to have more number of neurons, the modern processors beats it both in terms of speed and density of transistors.  Thus, it seems computers should beat humans in all tasks. But they seldom do especially in intelligent and creative tasks. But with the advent of machine learning this is changing. Machine learning is achieving feats which were considered earlier very difficult or impossible for computers. Machines today are driving cars autonomously, making creative arts and music and doing cancer diagnostics among hundreds of other applications hitherto considered out of their domains. Neural networks inspired from brain neuronal connections are behind this revolutionary new development. Machine learning codes try to create a network of virtual neurons on the transistor chips which are then trained to map input and output relationships to discover a general mathematical representation of the input to output transformation through an extremely complex non-linear model which theoretically is pretty close to the way in which neurons represent knowledge in brain. But still humans can perform better and faster on most learning tasks with much less data. Why? This appears illogical given the vastly denser and faster transistor networks that are available in computer chips today.

Here, I think we still have to make one more critical leap in coming closer to the brain which is – Parallelism. In brain, each neuron fires independently and doesn’t wait for all previous calculations to complete; which is the case with sequential digital circuits. Synchronous digital circuits driven by a clock were an essentiality for writing conventional sequential programs. But in neural networks, the paradigm of sequential programming is no more required and hence we can very well explore chips with completely asynchronous digital circuits. Such chips shall have perceptron as the unit of computation and all perceptrons shall work asynchronously. The evolution of this new type of chip and re-emergence of asynchronous digital electronics might just be the next quantum leap of machine learning which shall make computers come closer to the brain.

Transfer Learning

I was trying to teach transfer learning in my lab and couldn’t find a simple no-frills code for it over the internet. So, I wrote my own implementation using Tensorflow and Python over a modified cats and dogs dataset obtained from kaggle.

This was trained using only 1040 images placed in test directory and it could get more than 90% accuracy on a test set of  23936 images in just four epochs. This uses VVG-16 network pre-trained on imagenet to obtain the bottleneck features. These features are then used to train a simple two layer neural network to classify dogs and cats.

Complete implementation of it along with the datasets can be downloaded from here.

You need to have Jupyter, tensorflow and opencv-python installed on your machine to run this.

Changing programming paradigms in the age of AI

Machine learning is changing the way we program. What earlier used to be a transfer of logic to the machines is becoming more of an exercise of transfer of data to machines. The challenge and thrill of coding complex systems with intricately written logical structures would soon become a thing of the past. Machine learning for most part has reduced all the programs into large matrix multiplications. Given the user provides enough data with matching inputs and outputs, machine just has to solve a huge matrix multiplication like:

ς(ς(ς(Input  x Mat1) x Mat2) x Mat3) … = Output

Mat1, Mat2, Mat3 … are the only things that are to be determined through already well developed methods and this can solve almost any programming task. If you have enough data finding multiplier matrices is pretty trivial. And these matrices are your program!

To give an idea of how radically different it is, one may consider the MNIST handwritten letter recognition problem. If one writes a code to identify these digits through conventional coding paradigms, this might take thousands of lines of code in any high level language, while a similar machine learning code from scratch can do the job in less than 50 lines of code. In a conventional coding paradigm, you will be thinking about shapes of letters, relative brightness of pixels, curve fitting etc. but one can write a machine learning code by almost being blind to the data. One concentrates rather on getting the matrix multiplications right. Once that is done, machine finds the most optimal matrix multiplier to map the input to the output; and in a way, writes its own code!

I sometimes ask this question in class that, if you write a highly optimized ten thousand line code to recognize cars in conventional method vs google leverages its millions of car images to just train a two layer neural network which code will win? Of course google will win. Disturbingly, this also means that data is the new oil. One who has more of it, will have more power.

Challenges of Collaborative MedTech Research in India

(Written for Publication of CII)

One of the things which we can be proud of, is that, after independence, we have been able to build some of the international standard medical and engineering schools in this country. If we look at the figure below, annual budget of our top engineering school and top medical school is not very far behind their top international counterparts once we balance these numbers with purchasing power parity.

I myself am a PhD from IIT Madras and have done postdoc at Harvard medical school. Comparing both these systems, I can say pretty confidently that in terms of infrastructure, funding and manpower, our elite institutions are second to none in the world. In a way, the key infrastructure and funding required for successful healthcare innovations are already there in place. But, this has not led to many significant number of translational research in medical technology space.  India contributes less than 1% of global trade of medical devices[1]. While we have many companies in other technology domains which have already arrived at the international space same has not happened in health technology and medical device space

One of the major obstacles in this translation, I believe, is the big engineer doctor divide in this country. If you look at the faculty profiles of most high ranking international medical technology research centres, there is a healthy mix of both engineering, sciences and medicine PhDs. Most of the bigger universities in the developed world, have both medical and engineering disciplines in the same campus. This is almost never the case in India. The system by its very design divides both disciplines into watertight compartments. We have not created formal channels for interaction among our institutions of eminence in medicine and research; our engineers and doctors almost never interact. I am afraid, unless we work towards bridging this gap, we shall keep trailing behind in successful medtech innovation and translation. IIT labs doing great research in health technology need support from clinical community for clinical inputs, trials and translation; and clinician community needs access to the technical capabilities and know-how of the engineering research community to translate their clinical observations and ideas into possible products.

There is a need of developing medical innovation ecosystems in this country where we may have rich interaction of all stake-holders which can include doctors, engineers, industry and policy makers. And I firmly believe that this change can happen within the existing frameworks by building formal connecting links through relatively simple policy shifts by government. Interdisciplinary collaborative regional research centers by coupling one major teaching hospital with one major engineering research institute within a region can be a great start. This kind of center can take formal assignment of associate faculty from both the institutes and may focus on facilitating interaction.

Over the last couple of years, data science and machine learning has opened up new avenues of data driven research in medical sciences. India being one of the most populous countries, our big medical centers are sitting over mines of clinical data. One of the aims of building bridges between our institutions, should be to facilitate access to these databases through proper formal channels.

Lack of formal ethical clearance structures in engineering institutions and painfully slow ethics committee clearances in major hospitals is another major roadblock in promoting medical technology research. Government and policy makers should keenly focus in removing these major roadblocks, without which doing international standard medtech research is going to be a challenge.

Industry academia collaborative research, I believe has great potential but most of the time we do not have formal channels of initiating such collaboration. There is a lot of excellent research expertise and equipment available in major research institutes which can be leveraged for research and development by small and medium enterprises, and startups. But most of it remain unexplored due to information gap. The information available on the websites of most top research institutes of the country are paltry and lack clear guidelines on starting a collaborative project with industry. As a result, industry doesn’t know whom to connect with or how to approach the academic institutes to carry out research or initiate consultancy projects. This is also true the other way round. If a faculty wants to initiate a project with industry, it is often impossible to find the right contact person in a company through an organic internet search. Only thing that works currently is to painfully sift through tertiary personal and business contacts to find the right resource person. In my opinion, Medical technology businesses, which stand to gain a lot by way of collaborative research with research institutes and hospitals should maintain a research liaison officer who can be the first point of contact for any researcher to contact.

Academic capability being conventionally measured on number of papers one can publish in a year, the focus of research tend to be on volume of publications. Faculty are pushed towards doing non-relevant research just for the sake of publications. More cited research is the considered most impactful. As international research journals are dominated by western research, Indian researchers are also driven into solving high level problems related to the developed world while many of our own problems remain untouched.  They are working for years on problems that may have no relevance to indian context. We have too many academics working either on incremental solutions for old problems or on completely theoretical and non-translatable problems. The only fate of such projects is the production of a thesis which go into the library and would be read only by four people in its lifetime. On the contrary, industry has totally different priorities. Publications do not count for much.  Industry requires fast time bound solutions to its projects which many times do not match the slow paced but meticulous approach of academia. This mismatch in expectations and timelines has to be understood and adjusted for by both parties, and funding, publications, patent and consultancy model should be clarified at the very beginning of a project to ensure successful collaboration.

I sincerely hope that the government, administrators of our elite research and medical institutes, and industry heads will push towards above policy changes which I believe would be key drivers in pushing medtech innovations in India to the global stage.

[1] https://atlas.media.mit.edu/en/profile/hs92/9018/

99 percent accurate. Really!

Today I threw an interesting challenge to the students to sensitize them on misleading nature of accuracies reported in some medical literature.

Question: A type of cancer can be contracted by 1 in 10000 people. A patient was tested positive for it by a test which gives 99% accurate results. What is the probability that the patient actually has the cancer?

Answer:

As you can see the probability of having the cancer turns out to be less than 1% even with a 99% accurate test.

But it doesn’t mean that the test is useless. Let’s see what happens when we repeat the test just one more time.

Question: After testing positive for cancer if the patient repeats the tests and is found to be positive for cancer again. Now what would be the probability?

Answer:

We can see that just by having one iteration of the test we can take the confidence to 50%. So sometimes a second opinion is necessary!

Action Potential from Cockroach Leg

Today we did a very interesting experiment on Cockroach leg samples to understand action potential. Cockroach leg was secured on a foam block using two electrodes. These two electrodes were placed across the Coxa such that there is significant amount of tissue gap between them.

Related image

Figure source: http://www.entomologa.ru/figures/22.htm

 

After this we stimulated the leg with a square wave generated from signal generator. Peak to peak voltage greater than 300 mV. We varied the frequency of stimulation first from 0.5 Hz to upto 10 Hz and back down to about 1 Hz.  We could see the matching leg flip activity with each stimulated pulse.

 

We also tried to see the reverse effect where we acquired the action potential induced due to tapping at the leg. For this we removed the stimulation line and only acquired the signals from oscilloscope. So whenever there was a small tap on the end of the leg we could see a distinct induced potential on the scope.