Lossless learning

I am a slow learner. It took me time to realise that. And I always thought of it as a weakness. But lately I am realising that it might be a good thing.

The reason I am a slow learner is that I don’t want to settle for surface knowledge. I am not happy if I learn about a new topic. I get happy if I feel I can implement it, or explain it to someone, or write about it without referencing the original material, or derive the solution based on first-principles thinking. Until I do these, I am never really sure if I understood a topic correctly or not.

This only happens when I study a topic multiple times from multiple sources. It took me a lot of time to understand transformers. I read a few research papers, saw some videos of people explaining those research papers, read in-depth blogs, drew diagrams myself to understand them, and finally implemented it in PyTorch to close my understanding. Once I have understood the fundamentals of a topic, additional knowledge requires smaller time investments.

This comes at a disadvantage, also. Whenever I want to deep dive into a new topic, I sometimes wait. I wait for it to emerge as an important enough topic for me to spend days or weeks understanding it completely. For example, as of now, I am not very confident in diffusion-based models. Because I have not spent weeks learning and implementing them.

But I do believe that this way of learning is useful. It is worth repeating the core material, it is worth covering the exact same topic from varied resources, and it is worth doing multiple iterations. That initial time investment, struggle, and feeling of lifting a heavy rock needs to happen. That journey is unavoidable. I can not and should not learn a fundamental topic from a single source (given that there is enough material around it). This is the only lossless way of learning.