This book provides the mathematical fundamentals of linear algebra to practicers in computer vision, machine learning, robotics, applied mathematics, and electrical engineering. By only assuming a knowledge of calculus, the authors develop, in a rigorous yet down to earth manner, the mathematical theory behind concepts such as: vectors spaces, bases, linear maps, duality, Hermitian spaces, the spectral theorems, SVD, and the primary decomposition theorem. At all times, pertinent real-world applications are provided. This book includes the mathematical explanations for the tools used which we believe that is adequate for computer scientists, engineers and mathematicians who really want to do serious research and make significant contributions in their respective fields.

For more than thirty years the senior author has been trying to learn algebraic geometry. In the process he discovered that many of the classic textbooks in algebraic geometry require substantial knowledge of cohomology, homological algebra, and sheaf theory. In an attempt to demystify these abstract concepts and facilitate understanding for a new generation of mathematicians, he along with co-author wrote this book for an audience who is familiar with basic concepts of linear and abstract algebra, but who never has had any exposure to the algebraic geometry or homological algebra. As such this book consists of two parts. The first part gives a crash-course on the homological and cohomological aspects of algebraic topology, with a bias in favor of cohomology. The second part is devoted to presheaves, sheaves, Cech cohomology, derived functors, sheaf cohomology, and spectral sequences. All important concepts are intuitively motivated and the associated proofs of the quintessential theorems are presented in detail rarely found in the standard texts.

This textbook introduces linear algebra and optimization in the context of machine learning. Examples and exercises are provided throughout the book. A solution manual for the exercises at the end of each chapter is available to teaching instructors. This textbook targets graduate level students and professors in computer science, mathematics and data science. Advanced undergraduate students can also use this textbook. The chapters for this textbook are organized as follows: 1. Linear algebra and its applications: The chapters focus on the basics of linear algebra together with their common applications to singular value decomposition, matrix factorization, similarity matrices (kernel methods), and graph analysis. Numerous machine learning applications have been used as examples, such as spectral clustering, kernel-based classification, and outlier detection. The tight integration of linear algebra methods with examples from machine learning differentiates this book from generic volumes on linear algebra. The focus is clearly on the most relevant aspects of linear algebra for machine learning and to teach readers how to apply these concepts. 2. Optimization and its applications: Much of machine learning is posed as an optimization problem in which we try to maximize the accuracy of regression and classification models. The “parent problem” of optimization-centric machine learning is least-squares regression. Interestingly, this problem arises in both linear algebra and optimization, and is one of the key connecting problems of the two fields. Least-squares regression is also the starting point for support vector machines, logistic regression, and recommender systems. Furthermore, the methods for dimensionality reduction and matrix factorization also require the development of optimization methods. A general view of optimization in computational graphs is discussed together with its applications to back propagation in neural networks. A frequent challenge faced by beginners in machine learning is the extensive background required in linear algebra and optimization. One problem is that the existing linear algebra and optimization courses are not specific to machine learning; therefore, one would typically have to complete more course material than is necessary to pick up machine learning. Furthermore, certain types of ideas and tricks from optimization and linear algebra recur more frequently in machine learning than other application-centric settings. Therefore, there is significant value in developing a view of linear algebra and optimization that is better suited to the specific perspective of machine learning.

A groundbreaking introduction to vectors, matrices, and least squares for engineering applications, offering a wealth of practical examples.

An up-to-date account of the interplay between optimization and machine learning, accessible to students and researchers in both communities. The interplay between optimization and machine learning is one of the most important developments in modern computational science. Optimization formulations and methods are proving to be vital in designing algorithms to extract essential knowledge from huge volumes of data. Machine learning, however, is not simply a consumer of optimization technology but a rapidly evolving field that is itself generating new optimization ideas. This book captures the state of the art of the interaction between optimization and machine learning in a way that is accessible to researchers in both fields. Optimization approaches have enjoyed prominence in machine learning because of their wide applicability and attractive theoretical properties. The increasing complexity, size, and variety of today's machine learning models call for the reassessment of existing assumptions. This book starts the process of reassessment. It describes the resurgence in novel contexts of established frameworks such as first-order methods, stochastic approximations, convex relaxations, interior-point methods, and proximal methods. It also devotes attention to newer themes such as regularized optimization, robust optimization, gradient and subgradient methods, splitting techniques, and second-order methods. Many of these techniques draw inspiration from other fields, including operations research, theoretical computer science, and subfields of optimization. The book will enrich the ongoing cross-fertilization between the machine learning community and these other fields, and within the broader optimization community.

The fundamental mathematical tools needed to understand machine learning include linear algebra, analytic geometry, matrix decompositions, vector calculus, optimization, probability and statistics. These topics are traditionally taught in disparate courses, making it hard for data science or computer science students, or professionals, to efficiently learn the mathematics. This self-contained textbook bridges the gap between mathematical and machine learning texts, introducing the mathematical concepts with a minimum of prerequisites. It uses these concepts to derive four central machine learning methods: linear regression, principal component analysis, Gaussian mixture models and support vector machines. For students and others with a mathematical background, these derivations provide a starting point to machine learning texts. For those learning the mathematics for the first time, the methods help build intuition and practical experience with applying mathematical concepts. Every chapter includes worked examples and exercises to test understanding. Programming tutorials are offered on the book's web site.

The goal of the Volume I Geometric Algebra for Computer Vision, Graphics and Neural Computing is to present a unified mathematical treatment of diverse problems in the general domain of artificial intelligence and associated fields using Clifford, or geometric, algebra. Geometric algebra provides a rich and general mathematical framework for Geometric Cybernetics in order to develop solutions, concepts and computer algorithms without losing geometric insight of the problem in question. Current mathematical subjects can be treated in an unified manner without abandoning the mathematical system of geometric algebra for instance: multilinear algebra, projective and affine geometry, calculus on manifolds, Riemann geometry, the representation of Lie algebras and Lie groups using bivector algebras and conformal geometry. By treating a wide spectrum of problems in a common language, this Volume I offers both new insights and new solutions that should be useful to scientists, and engineers working in different areas related with the development and building of intelligent machines. Each chapter is written in accessible terms accompanied by numerous examples, figures and a complementary appendix on Clifford algebras, all to clarify the theory and the crucial aspects of the application of geometric algebra to problems in graphics engineering, image processing, pattern recognition, computer vision, machine learning, neural computing and cognitive systems.