Interactively Exploring High-Dimensional Data and Models in REditie 1

Most data arrive with more than two numeric variables which means that plotting it on a computer screen or printed page presents a challenge: how do you visually explore for associations between more than two variables? Visualising data provides the opportunity to discover what we never expected, because it requires fewer assumptions to be made. Visualising elements of a model fit is a primary way to diagnose whether the fit matches this data. Two of more numeric variables is considered to be multivariate data, and when there are substantially more we would consider it to be high-dimensional data. This book provides you with the tools to visually explore high dimensions, to uncover associations, clustering and anomalies that may be missed when only using common methods for plotting one or two variables. It also illustrates how to use visualisation to understand how your model is operating on the data, to be able to explain how it is arriving at decisions. To make effective use of this material the reader should have a basic working knowledge of R and some understanding of multivariate statistical methods or machine learning methods. The book could form an independent course on visualization or be used as part of courses on multivariate statistical methods or machine learning.

High-dimensional data visualisation is valuable for understanding dimension reduction methods, unsupervised and supervised classification. This book is organised into these three topics, following overview and introductory chapters. The dimension reduction chapters cover principal component analysis and nonlinear dimension reduction. The chapters on cluster analysis cover hierarchical and k-means algorithms, model-based and self-organising maps, and finish with ways to communicate results and how to compare different results. The chapters on classification cover linear discriminant analysis, tree and forest algorithms, support vector machines and neural networks. We explain how to break down a neural network to examine the components, how to visualize predictive probabilities, and how to incorporate explainable AI metrics to develop a deeper understanding about how the model operates.