Uday Kamath

Reinforcement learning (RL) has become the engine behind some of the most significant advances in modern artificial intelligence, from defeating world champions in Go to aligning large language models with human preferences. Yet despite its central role, RL remains poorly understood by many practitioners who work with these systems daily. Reinforcement Learning in Action: From Foundations to Frontiers bridges the gap between classical RL theory and the cutting-edge techniques driving today’s AI breakthroughs. The book traces a complete path from Markov Decision Processes and Bellman equations through deep RL methods (DQN, REINFORCE, Actor-Critic, PPO) to the modern landscape of LLM alignment (RLHF, DPO, SimPO, KTO), reasoning optimization (GRPO, VinePPO, MCTS), and agentic systems with tool use, memory, and multi-turn planning. A distinguishing feature is the book’s consistent five-layer pedagogical structure: each algorithm is presented with its key characteristics, a full mathematical derivation, an honest assessment of its advantages and limitations, a complete from-scratch Python/PyTorch implementation in which variable names match the equations, and a hands-on case study with reproducible experiments. Case studies progress from Grid World navigation and CartPole control to fine-tuning language models with DPO on the HuggingFace ecosystem, training reasoning models with GRPO on mathematical benchmarks, and building a full agentic customer support system. Written for ML engineers, researchers, and advanced students, this book provides both the conceptual depth and implementation fluency needed to understand, build, and extend the RL systems shaping the future of AI.

Reinforcement learning (RL) has become the engine behind some of the most significant advances in modern artificial intelligence, from defeating world champions in Go to aligning large language models with human preferences. Yet despite its central role, RL remains poorly understood by many practitioners who work with these systems daily. Reinforcement Learning in Action: From Foundations to Frontiers bridges the gap between classical RL theory and the cutting-edge techniques driving today’s AI breakthroughs. The book traces a complete path from Markov Decision Processes and Bellman equations through deep RL methods (DQN, REINFORCE, Actor-Critic, PPO) to the modern landscape of LLM alignment (RLHF, DPO, SimPO, KTO), reasoning optimization (GRPO, VinePPO, MCTS), and agentic systems with tool use, memory, and multi-turn planning. A distinguishing feature is the book’s consistent five-layer pedagogical structure: each algorithm is presented with its key characteristics, a full mathematical derivation, an honest assessment of its advantages and limitations, a complete from-scratch Python/PyTorch implementation in which variable names match the equations, and a hands-on case study with reproducible experiments. Case studies progress from Grid World navigation and CartPole control to fine-tuning language models with DPO on the HuggingFace ecosystem, training reasoning models with GRPO on mathematical benchmarks, and building a full agentic customer support system. Written for ML engineers, researchers, and advanced students, this book provides both the conceptual depth and implementation fluency needed to understand, build, and extend the RL systems shaping the future of AI.

Large Language Models (LLMs) have emerged as a cornerstone technology, transforming how we interact with information and redefining the boundaries of artificial intelligence. LLMs offer an unprecedented ability to understand, generate, and interact with human language in an intuitive and insightful manner, leading to transformative applications across domains like content creation, chatbots, search engines, and research tools. While fascinating, the complex workings of LLMs—their intricate architecture, underlying algorithms, and ethical considerations—require thorough exploration, creating a need for a comprehensive book on this subject. This book provides an authoritative exploration of the design, training, evolution, and application of LLMs. It begins with an overview of pre-trained language models and Transformer architectures, laying the groundwork for understanding prompt-based learning techniques. Next, it dives into methods for fine-tuning LLMs, integrating reinforcement learning for value alignment, and the convergence of LLMs with computer vision, robotics, and speech processing. The book strongly emphasizes practical applications, detailing real-world use cases such as conversational chatbots, retrieval-augmented generation (RAG), and code generation. These examples are carefully chosen to illustrate the diverse and impactful ways LLMs are being applied in various industries and scenarios. Readers will gain insights into operationalizing and deploying LLMs, from implementing modern tools and libraries to addressing challenges like bias and ethical implications. The book also introduces the cutting-edge realm of multimodal LLMs that can process audio, images, video, and robotic inputs. With hands-on tutorials for applying LLMs to natural language tasks, this thorough guide equips readers with both theoretical knowledge and practical skills for leveraging the full potential of large language models. This comprehensive resource is appropriate for a wide audience: students, researchers and academics in AI or NLP, practicing data scientists, and anyone looking to grasp the essence and intricacies of LLMs. Key Features: Over 100 techniques and state-of-the-art methods, including pre-training, prompt-based tuning, instruction tuning, parameter-efficient and compute-efficient fine-tuning, end-user prompt engineering, and building and optimizing Retrieval-Augmented Generation systems, along with strategies for aligning LLMs with human values using reinforcement learningOver 200 datasets compiled in one place, covering everything from pre- training to multimodal tuning, providing a robust foundation for diverse LLM applicationsOver 50 strategies to address key ethical issues such as hallucination, toxicity, bias, fairness, and privacy. Gain comprehensive methods for measuring, evaluating, and mitigating these challenges to ensure responsible LLM deploymentOver 200 benchmarks covering LLM performance across various tasks, ethical considerations, multimodal applications, and more than 50 evaluation metrics for the LLM lifecycleNine detailed tutorials that guide readers through pre-training, fine- tuning, alignment tuning, bias mitigation, multimodal training, and deploying large language models using tools and libraries compatible with Google Colab, ensuring practical application of theoretical conceptsOver 100 practical tips for data scientists and practitioners, offering implementation details, tricks, and tools to successfully navigate the LLM life- cycle and accomplish tasks efficiently

This book is written both for readers entering the field, and for practitioners with a background in AI and an interest in developing real-world applications. The book is a great resource for practitioners and researchers in both industry and academia, and the discussed case studies and associated material can serve as inspiration for a variety of projects and hands-on assignments in a classroom setting. I will certainly keep this book as a personal resource for the courses I teach, and strongly recommend it to my students. --Dr. Carlotta Domeniconi, Associate Professor, Computer Science Department, GMUThis book offers a curriculum for introducing interpretability to machine learning at every stage. The authors provide compelling examples that a core teaching practice like leading interpretive discussions can be taught and learned by teachers and sustained effort. And what better way to strengthen the quality of AI and Machine learning outcomes. I hope that this book will become a primer for teachers, data Science educators, and ML developers, and together we practice the art of interpretive machine learning.--Anusha Dandapani, Chief Data and Analytics Officer, UNICC and Adjunct Faculty, NYUThis is a wonderful book! I’m pleased that the next generation of scientists will finally be able to learn this important topic. This is the first book I’ve seen that has up-to-date and well-rounded coverage. Thank you to the authors!--Dr. Cynthia Rudin, Professor of Computer Science, Electrical and Computer Engineering, Statistical Science, and Biostatistics & Bioinformatics Literature on Explainable AI has up until now been relatively scarce and featured mainly mainstream algorithms like SHAP and LIME. This book has closed this gap by providing an extremely broad review of various algorithms proposed in the scientific circles over the previous 5-10 years. This book is a great guide to anyone who is new to the field of XAI or is already familiar with the field and is willing to expand their knowledge. A comprehensive review of the state-of-the-art Explainable AI methods starting from visualization, interpretable methods, local and global explanations, time series methods, and finishing with deep learning provides an unparalleled source of information currently unavailable anywhere else. Additionally, notebooks with vivid examples are a great supplement that makes the book even more attractive for practitioners of any level.Overall, the authors provide readers with an enormous breadth of coverage without losing sight of practical aspects, which makes this book truly unique and a great addition to the library of any data scientist.Dr. Andrey Sharapov, Product Data Scientist, Explainable AI Expert and Speaker, Founder ofExplainable AI-XAI Group

Transformers are becoming a core part of many neural network architectures, employed in a wide range of applications such as NLP, Speech Recognition, Time Series, and Computer Vision. Transformers have gone through many adaptations and alterations, resulting in newer techniques and methods. Transformers for Machine Learning: A Deep Dive is the first comprehensive book on transformers.Key Features:A comprehensive reference book for detailed explanations for every algorithm and techniques related to the transformers.60+ transformer architectures covered in a comprehensive manner.A book for understanding how to apply the transformer techniques in speech, text, time series, and computer vision.Practical tips and tricks for each architecture and how to use it in the real world.Hands-on case studies and code snippets for theory and practical real-world analysis using the tools and libraries, all ready to run in Google Colab.The theoretical explanations of the state-of-the-art transformer architectures will appeal to postgraduate students and researchers (academic and industry) as it will provide a single entry point with deep discussions of a quickly moving field. The practical hands-on case studies and code will appeal to undergraduate students, practitioners, and professionals as it allows for quick experimentation and lowers the barrier to entry into the field.

Transformers are becoming a core part of many neural network architectures, employed in a wide range of applications such as NLP, Speech Recognition, Time Series, and Computer Vision. Transformers have gone through many adaptations and alterations, resulting in newer techniques and methods. Transformers for Machine Learning: A Deep Dive is the first comprehensive book on transformers.Key Features:A comprehensive reference book for detailed explanations for every algorithm and techniques related to the transformers.60+ transformer architectures covered in a comprehensive manner.A book for understanding how to apply the transformer techniques in speech, text, time series, and computer vision.Practical tips and tricks for each architecture and how to use it in the real world.Hands-on case studies and code snippets for theory and practical real-world analysis using the tools and libraries, all ready to run in Google Colab.The theoretical explanations of the state-of-the-art transformer architectures will appeal to postgraduate students and researchers (academic and industry) as it will provide a single entry point with deep discussions of a quickly moving field. The practical hands-on case studies and code will appeal to undergraduate students, practitioners, and professionals as it allows for quick experimentation and lowers the barrier to entry into the field.

This book is written both for readers entering the field, and for practitioners with a background in AI and an interest in developing real-world applications. The book is a great resource for practitioners and researchers in both industry and academia, and the discussed case studies and associated material can serve as inspiration for a variety of projects and hands-on assignments in a classroom setting. I will certainly keep this book as a personal resource for the courses I teach, and strongly recommend it to my students. --Dr. Carlotta Domeniconi, Associate Professor, Computer Science Department, GMUThis book offers a curriculum for introducing interpretability to machine learning at every stage. The authors provide compelling examples that a core teaching practice like leading interpretive discussions can be taught and learned by teachers and sustained effort. And what better way to strengthen the quality of AI and Machine learning outcomes. I hope that this book will become a primer for teachers, data Science educators, and ML developers, and together we practice the art of interpretive machine learning.--Anusha Dandapani, Chief Data and Analytics Officer, UNICC and Adjunct Faculty, NYUThis is a wonderful book! I’m pleased that the next generation of scientists will finally be able to learn this important topic. This is the first book I’ve seen that has up-to-date and well-rounded coverage. Thank you to the authors!--Dr. Cynthia Rudin, Professor of Computer Science, Electrical and Computer Engineering, Statistical Science, and Biostatistics & Bioinformatics Literature on Explainable AI has up until now been relatively scarce and featured mainly mainstream algorithms like SHAP and LIME. This book has closed this gap by providing an extremely broad review of various algorithms proposed in the scientific circles over the previous 5-10 years. This book is a great guide to anyone who is new to the field of XAI or is already familiar with the field and is willing to expand their knowledge. A comprehensive review of the state-of-the-art Explainable AI methods starting from visualization, interpretable methods, local and global explanations, time series methods, and finishing with deep learning provides an unparalleled source of information currently unavailable anywhere else. Additionally, notebooks with vivid examples are a great supplement that makes the book even more attractive for practitioners of any level.Overall, the authors provide readers with an enormous breadth of coverage without losing sight of practical aspects, which makes this book truly unique and a great addition to the library of any data scientist.Dr. Andrey Sharapov, Product Data Scientist, Explainable AI Expert and Speaker, Founder ofExplainable AI-XAI Group

This textbook explains Deep Learning Architecture, with applications to various NLP Tasks, including Document Classification, Machine Translation, Language Modeling, and Speech Recognition. With the widespread adoption of deep learning, natural language processing (NLP),and speech applications in many areas (including Finance, Healthcare, and Government) there is a growing need for one comprehensive resource that maps deep learning techniques to NLP and speech and provides insights into using the tools and libraries for real-world applications. Deep Learning for NLP and Speech Recognition explains recent deep learning methods applicable to NLP and speech, provides state-of-the-art approaches, and offers real-world case studies with code to provide hands-on experience. Many books focus on deep learning theory or deep learning for NLP-specific tasks while others are cookbooks for tools and libraries, but the constant flux of new algorithms, tools, frameworks, and libraries in a rapidly evolving landscape means that there are few available texts that offer the material in this book. The book is organized into three parts, aligning to different groups of readers and their expertise. The three parts are: Machine Learning, NLP, and Speech Introduction The first part has three chapters that introduce readers to the fields of NLP, speech recognition, deep learning and machine learning with basic theory and hands-on case studies using Python-based tools and libraries. Deep Learning Basics The five chapters in the second part introduce deep learning and various topics that are crucial for speech and text processing, including word embeddings, convolutional neural networks, recurrent neural networks and speech recognition basics. Theory, practical tips, state-of-the-art methods, experimentations and analysis in using the methods discussed in theory on real-world tasks. Advanced Deep Learning Techniques for Text and Speech The third part has five chapters that discuss the latest and cutting-edge research in the areas of deep learning that intersect with NLP and speech. Topics including attention mechanisms, memory augmented networks, transfer learning, multi-task learning, domain adaptation, reinforcement learning, and end-to-end deep learning for speech recognition are covered using case studies.

This textbook explains Deep Learning Architecture, with applications to various NLP Tasks, including Document Classification, Machine Translation, Language Modeling, and Speech Recognition. With the widespread adoption of deep learning, natural language processing (NLP),and speech applications in many areas (including Finance, Healthcare, and Government) there is a growing need for one comprehensive resource that maps deep learning techniques to NLP and speech and provides insights into using the tools and libraries for real-world applications. Deep Learning for NLP and Speech Recognition explains recent deep learning methods applicable to NLP and speech, provides state-of-the-art approaches, and offers real-world case studies with code to provide hands-on experience. Many books focus on deep learning theory or deep learning for NLP-specific tasks while others are cookbooks for tools and libraries, but the constant flux of new algorithms, tools, frameworks, and libraries in a rapidly evolving landscape means that there are few available texts that offer the material in this book. The book is organized into three parts, aligning to different groups of readers and their expertise. The three parts are: Machine Learning, NLP, and Speech Introduction The first part has three chapters that introduce readers to the fields of NLP, speech recognition, deep learning and machine learning with basic theory and hands-on case studies using Python-based tools and libraries. Deep Learning Basics The five chapters in the second part introduce deep learning and various topics that are crucial for speech and text processing, including word embeddings, convolutional neural networks, recurrent neural networks and speech recognition basics. Theory, practical tips, state-of-the-art methods, experimentations and analysis in using the methods discussed in theory on real-world tasks. Advanced Deep Learning Techniques for Text and Speech The third part has five chapters that discuss the latest and cutting-edge research in the areas of deep learning that intersect with NLP and speech. Topics including attention mechanisms, memory augmented networks, transfer learning, multi-task learning, domain adaptation, reinforcement learning, and end-to-end deep learning for speech recognition are covered using case studies.

Become an advanced practitioner with this progressive set of master classes on application-oriented machine learning About This Book • Comprehensive coverage of key topics in machine learning with an emphasis on both the theoretical and practical aspects • More than 15 open source Java tools in a wide range of techniques, with code and practical usage. • More than 10 real-world case studies in machine learning highlighting techniques ranging from data ingestion up to analyzing the results of experiments, all preparing the user for the practical, real-world use of tools and data analysis. Who This Book Is For This book will appeal to anyone with a serious interest in topics in Data Science or those already working in related areas: ideally, intermediate-level data analysts and data scientists with experience in Java. Preferably, you will have experience with the fundamentals of machine learning and now have a desire to explore the area further, are up to grappling with the mathematical complexities of its algorithms, and you wish to learn the complete ins and outs of practical machine learning. What You Will Learn • Master key Java machine learning libraries, and what kind of problem each can solve, with theory and practical guidance. • Explore powerful techniques in each major category of machine learning such as classification, clustering, anomaly detection, graph modeling, and text mining. • Apply machine learning to real-world data with methodologies, processes, applications, and analysis. • Techniques and experiments developed around the latest specializations in machine learning, such as deep learning, stream data mining, and active and semi-supervised learning. • Build high-performing, real-time, adaptive predictive models for batch- and stream-based big data learning using the latest tools and methodologies. • Get a deeper understanding of technologies leading towards a more powerful AI applicable in various domains such as Security, Financial Crime, Internet of Things, social networking, and so on. In Detail Java is one of the main languages used by practicing data scientists; much of the Hadoop ecosystem is Java-based, and it is certainly the language that most production systems in Data Science are written in. If you know Java, Mastering Machine Learning with Java is your next step on the path to becoming an advanced practitioner in Data Science. This book aims to introduce you to an array of advanced techniques in machine learning, including classification, clustering, anomaly detection, stream learning, active learning, semi-supervised learning, probabilistic graph modeling, text mining, deep learning, and big data batch and stream machine learning. Accompanying each chapter are illustrative examples and real-world case studies that show how to apply the newly learned techniques using sound methodologies and the best Java-based tools available today. On completing this book, you will have an understanding of the tools and techniques for building powerful machine learning models to solve data science problems in just about any domain. Style and approach A practical guide to help you explore machine learning—and an array of Java-based tools and frameworks—with the help of practical examples and real-world use cases.