Hmm… that really was the first thing I did when I started the Applied Data Science program. But since I’m now stuck and tired of my ugliest ever clustering algorithm, I can’t write about this monstrosity before it starts to somehow work. So, back to the beginnings..

All AI is based on those magical matrices. I did a little bit of them at university, but I never got “why am I doing this?” Not even thirteen years as a software programmer ever exposed me to any matrices… or at least I never needed to understand how they work. Now they seem to be fundamental to everything in AI, but I still couldn’t grasp the gist of them, so I asked myself a question: “Is it even possible to intuitively understand matrices?”

And my thinking was: ok, that’s a tough question, I can’t get a straight answer, but maybe, just maybe, there will be some proxy. Neural networks are somehow similar to neurons in my very own brain. So if such a network can easily answer a question about matrices, then maybe I should study them more, and just calculating them manually – which is what we do at universities – makes some sense.

I was talking a lot with ChatGPT. He is the most patient – sometimes annoying and not very smart – but still extremely patient and knowledgeable teacher I have, always there when I need him. Funny thing, how I used the word “he” instead of “it,” that’s incorrect on so many levels, but it still feels natural. Anyway, I was asking him a lot of questions about what a matrix is, what it’s for, who came up with this utterly destroying me idea, and so on. In one of such talks, there came the term “determinant” – I already forgot what it is, but it is important 🙂

I really don’t remember what a determinant is, but if it is equal to zero, it means that the matrix is bad, broken, ugly, cursed. It has no solution. And a matrix is a set of linear equations. A linear equation with two unknowns describes a line, with three describes a plane. It’s the same for matrices; the number of columns describes the dimensionality of equations. Three columns is the real world, and four describes something I’ll never be able to understand, but it obviously has some mathematical name: turboplane, hyperplane, batmanplane – something like that. While the number of rows describes the number of smaller shapes. E.g., three rows by two columns describe a triangle on a plane, five can describe a pentagon – and so, thanks to matrices, we can… no, computers can calculate if you are inside the evil zone.

I’m getting off-topic all the time. It’s late, I’m tired of the dumb clustering algorithm, and the project isn’t so important, so who cares. So the determinant, the intuition about matrices, and willingness to get my hands dirty on neural networks. It’s not my style to repeat step by step what other people do; it doesn’t teach me so much. So instead of repeating the classical “recognize the handwritten number” exercise, I thought: ok, can a neural network learn to determine if a matrix’s determinant is equal to zero.

Here is this little project. Here is this little project. Results are no. Matrices are not intuitive, don’t touch them. Leave that stuff to the silicon brains.

Not really – I learned quite a lot about them later, but one lesson is clear. I did that before any lectures about Neural Networks, and I was lost. Now, after a week of listening about them, I don’t know if I’d do that quite differently or the same way, but I’d know what I was doing.

At the moment when I was playing with it, I got stuck at stubborn 90% accuracy and couldn’t get any single percent point more! In retrospect, my idea may be insolvable, as with so small numbers, rounding error in how numbers are stored on computers can kill the results, but still, it wasn’t time lost. I got my hands dirty finally, I tried some stuff and was wandering lost in the world of AI – just like a teacher on a video who was trying to make some model working, and I learned what an epoch is, what hyperparameters are, and probably a few more things that I already forgot.

But the main lesson is – play with stuff. Just like Edison, who was, to paraphrase one famous English gentleman, always coming to the right idea – just after exhausting every other possibility. But reading the map before getting into the fog can make you feel better.