Numerical Machine Learning

Machine learning, a rapidly growing subfield of computer science, has had a significant impact on many industries and our lives. This chapter discusses the brief history of machine learning, its widespread adoption as a de facto feature, and fundamental concepts such as supervised and unsupervised learning, regression and classification, and underfitting and overfitting. We also emphasize the importance of understanding machine learning through numerical examples, which can bridge the gap between abstract mathematical theories and practical applications of machine learning algorithms. By developing a strong foundation in machine learning, readers/students can harness its potential to address challenges and opportunities across diverse sectors. 

In this chapter, we delve into linear regression, a fundamental machine learning algorithm for predicting numerical values. While maintaining a concise overview of the mathematical theories, we prioritize an accessible approach by focusing on a concrete numerical example with a small dataset for predicting house sale prices. Through a step-bystep walkthrough, we illustrate the inner workings of linear regression and demonstrate its practical implementation. Additionally, we offer sample codes and a comparison with the linear regression model from scikit-learn to reinforce understanding. Upon completing this chapter, readers will gain a comprehensive understanding of linear regression's inner workings and its relationship to algorithm implementation and performance, and be better prepared to apply it to real-world projects. 

This chapter delves into L1 and L2 regularization techniques within the context of linear regression, focusing on minimizing overfitting risks while maintaining a concise presentation of mathematical theories. We explore these techniques through a concrete numerical example with a small dataset for predicting house sale prices, providing a step-by-step walkthrough of the process. To further enhance comprehension, we supply sample codes and draw comparisons with the Lasso and Ridge models implemented in the scikit-learn library. By the end of this chapter, readers will acquire a well-rounded understanding of L1 and L2 regularization in the context of linear regression, their implications on model implementation and performance, and be equipped with the knowledge to apply these methods in practical use. 

This chapter delves into logistic regression, a widely used machine learning algorithm for classification tasks, with a focus on maintaining accessibility by minimizing abstract mathematical concepts. We present a concrete numerical example employing a small dataset to predict the ease of selling houses in the property market, guiding readers through each step of the process. Additionally, we supply sample codes and draw comparisons with the logistic regression model available in the scikit-learn library. Upon completion of this chapter, readers will have gained a comprehensive understanding of the inner workings of logistic regression, its relationship to algorithm implementation and performance, and the knowledge necessary to apply it to practical applications. 

In this chapter, we explore the concept of decision trees, prioritizing accessibility by minimizing abstract mathematical theories. We examine a concrete numerical example using a small dataset to predict the suitability of playing tennis based on weather conditions, guiding readers through the process step-by-step. Moreover, we provide sample codes and compare them with the decision tree classification model found in the scikit-learn library. Upon completing this chapter, readers will have gained a comprehensive understanding of the inner workings of decision tree machine learning, the relationship between the underlying principles, and the implementation and performance of the algorithm, preparing them to apply their knowledge to practical scenarios. 

In this chapter, we explore gradient boosting, a powerful ensemble machine learning method, for both regression and classification tasks. With a focus on accessibility, we minimize abstract mathematical theories and instead emphasize two concrete numerical examples with small datasets related to predicting house sale prices and ease of selling houses in the property market. By providing a step-by-step walkthrough, we illuminate the inner workings of gradient boosting and offer sample codes and comparisons to the gradient boosting models available in the scikit-learn library. Upon completing this chapter, readers will possess a comprehensive understanding of gradient boosting's mechanics, its connection to the implementation and performance of the algorithm, and be well-prepared to apply it in real-world projects. 

In this chapter, we investigate Support Vector Machines (SVM) for both linearly separable and linearly non-separable cases, emphasizing accessibility by minimizing abstract mathematical theories. We present concrete numerical examples with small datasets and provide a step-by-step walkthrough, illustrating the inner workings of SVM. Additionally, we offer sample codes and comparisons with the SVM model available in the scikit-learn library. Upon completing this chapter, readers will gain a comprehensive understanding of SVM's mechanics, and its connection to the implementation and performance of the algorithm, and be well-prepared to apply it in their practical applications.

In this chapter, we explore the K-means clustering algorithm, emphasizing an accessible approach by minimizing abstract mathematical theories. We present a concrete numerical example with a small dataset to illustrate how clusters can be formed using the Kmeans clustering algorithm. Additionally, we provide sample codes and comparisons with the K-means model available in the scikit-learn library. Upon completing this chapter, readers will gain a comprehensive understanding of the mechanics behind K-means clustering, and its connection to the implementation and performance of the algorithm, and be well-prepared to apply it in practical use.