Mathematics for AI: All the essential math topics you need

The relationship between AI and mathematics can be summed up as:

A person working in the field of AI who doesn’t know math is like a politician who doesn’t know how to persuade. Both have an inescapable area to work upon!

I won’t spend any more time on importance of learning mathematics for AI and will directly go to the main objective of this article.

A popular recommendation for learning mathematics for AI goes something like this:

• Learn linear algebra, probability, multivariate calculus, optimization and few other topics
• And then there is a list of courses and lectures that can be followed to accomplish the same

Although the above approach is perfectly fine, I personally feel there is another approach that is better especially for the people 1) who don’t have a solid quantitative background and 2) cannot afford the time to do all the prerequisite math courses. That is:

Instead of going by the subjects, go by the topics.

For example, while studying multivariate calculus you will come across the famous Stokes’ Theorem but it turns out that there is a high chance that it won’t be of any immediate use to you in practice and even in reading research papers. So, going by subject(courses) may be time-consuming and you might get lost in the vast sea of mathematics.

I recommend that you:

• go topic by topic, learn the essential concepts first, consolidate them
• And only then go for the other concepts as you encounter them during practical implementation and reading literature

Here is a list of essential topics in each subject:

Linear Algebra

• Vectors
  definition, scalars, addition, scalar multiplication, inner product(dot product), vector projection, cosine similarity, orthogonal vectors, normal and orthonormal vectors, vector norm, vector space, linear combination, linear span, linear independence, basis vectors
• Matrices
  definition, addition, transpose, scalar multiplication, matrix multiplication, matrix multiplication properties, hadamard product, functions, linear transformation, determinant, identity matrix, invertible matrix and inverse, rank, trace, popular type of matrices- symmetric, diagonal, orthogonal, orthonormal, positive definite matrix
• Eigenvalues & eigenvectors
  concept, intuition, significance, how to find
• Principle component analysis
  concept, properties, applications
• Singular value decomposition
  concept, properties, applications

Calculus

• Functions
• Scalar derivative
  definition, intuition, common rules of differentiation, chain rule, partial derivatives
• Gradient
  concept, intuition, properties, directional derivative
• Vector and matrix calculus
  how to find derivative of {scalar-valued, vector-valued} function wrt a {scalar, vector} -> four combinations- Jacobian
• Gradient algorithms
  local/global maxima and minima, saddle point, convex functions, gradient descent algorithms- batch, mini-batch, stochastic, their performance comparison

Probability

• Basic rules and axioms
  events, sample space, frequentist approach, dependent and independent events, conditional probability
• Random variables- continuous and discrete, expectation, variance, distributions- joint and conditional
• Bayes’ Theorem, MAP, MLE
• Popular distributions- binomial, bernoulli, poisson, exponential, gaussian
• Conjugate priors

Miscellaneous

• Information theory- entropy, cross-entropy, KL divergence, mutual information
• Markov Chain- definition, transition matrix, stationarity

What sources to follow?
Any source which suits you, be it a YouTube video or a classical textbook.
If you are unsure, do a simple google search for each topic [<topic name> + “machine learning”] and read from top links to develop a broad understanding.

The list may seem lengthy but it can save you a lot of time. Reading the above topics will give you the confidence to dive into the deep world of AI and explore more on your own.

In case you liked the article, do follow me Abhishek Parbhakar for more articles related to AI, philosophy, and economics.