01/01/2025 ML from Beginning

New year's day is always the quietest day. The Christmas craze has finished and finally everyone has some time to read the books they got as their Christmas Gift.

I however have suddenly some time at my hand, since no one can find a way to wrap ebooks and put them under the Christmas tree. It is time for learning.

Back when I was doing my masters, I had a lab assistant job to investigate a deep learning approach to compressive sensing (paper https://pubmed.ncbi.nlm.nih.gov/30854198/ ). The basic idea is as follow:

• Say we have an 24*24 image, we need to make 512 measurements to get the full information of the image

• However, if we project the 24*24 image into feature spaces (imagine a fourier transform), some feature space contains more information of the image than others. 

• We can therefore only take the measurements with the most information-rich feature space, typically only about 20% of measurement (100/512) is needed 

• Typically we use some sort of iterative approach to minimise L1 or L0 norm, exploiting the fact that natural images are normally sparse in transformed domain

• In the deep learning approach, the image is re-constructed using trained CNN model

I had a pretty success run in replicating this paper's finding for my intern, but never had the time to learn about the more specifics of how this works. 

I can start on the most basic form of image super-resolution, which consist of a single Convolutional Layer and ReLU, see notebook here https://github.com/Chymaster/ML_Learning 

The image does not exactly look better after the super-resoluton. This is somewhat expected.

Note taking from this:

Single Convolutional layer can only learn basic features like edge and corner. More complex features like human faces and phone screens needs deeper network to be recognised and successfully reconstructed.