Math for AI and Beyond

I've been interested in AI since undergrad.  Hofstatder's GED was one of my favorites.  In the 90s though, AI felt like it was still purely academic.  Around 2015 I was lucky enough to work at a start up whose value proposition was focused on displacing an industry standard with AI.

I was impressed by how far things had come and decided to try to get back up to speed.  I took it upon myself to learn the math necessary for understanding modern Machine Learning and finished several math courses at a local college.  The canonical coursework calls for Calculus (through multivariable), Linear Algebra, and Statistics/Probability Theory.  I went a bit further and got into some math ended up being surprisingly helpful to everyday Software Engineering.

My goal here is to share some of them in case they help anyone else.

Information Theory There are so many immediately, practically useful ideas here for a computer scientist.  The fundamental concern is how to represent ideas as numbers.  It walks you through from Morse code to JPEG compression and beyond.

Graph Theory Software is concerned primarily with discrete values.  Almost everything we touch can be expressed as tree.  Graph Theory gets you thinking more deeply along these lines and guides you through some of the most common forms.  This helped me with graph algorithms in particular.

Linear Algebra At first this feels like just another high school/college math class.  Over the course of a couple years post completion, I started to see it everywhere in my work.  For me, the valuable takeaway was hardly, if at all about the mechanics of Linear Algebra and more about the abstractions.

Fundamentally, in software we're essentially concerned with a) representation and b) transformation.  Most of what we do outside of these activities is just moving data around.  There exists an optimal representation of a given problem space.  Achieving it involves eliminating redundancy.  Using it vastly reduces the complexity of any function within its domain.

Numerical Analysis / Computational Linear Algebra

Mostly informative for the frameworks.  Discrete vs Continuous.  Analytical vs Numerical Approximation.  Variable Space & Subject Space (variables & observations).  Concerns therein like limits of representation on a computer and rounding.  Helps add perspective on the types of problems we're concerned with as software engineers. Informative regarding various tools for solving different classifications of problems.  These naturally progress into Data Science and Machine Learning topics.

Signal Processing

Instructive in learning about translating between continuous and discrete representations.  At the frameworks level, this includes an introduction to one of the most important algorithms of our age, the Fast Fourier Transform.  The idea behind it can be generalized and applied to less specific problems.  There are times when it's easier to transform a representation to a domain where it's easier to process, process it, then transform it back.