Get to grips with autogenerating code, deploying ML algorithms, and leveraging various ML lifecycle features on the Databricks Platform, guided by best practices and reusable code for you to try, alter, and build on Key Features Build machine learning solutions faster than peers only using documentation Enhance or refine your expertise with tribal knowledge and concise explanations Follow along with code projects provided in GitHub to accelerate your projects Purchase of the print or Kindle book includes a free PDF eBook Book DescriptionDiscover what makes the Databricks Data Intelligence Platform the go-to choice for top-tier machine learning solutions. Written by a team of industry experts at Databricks with decades of combined experience in big data, machine learning, and data science, Databricks ML in Action presents cloud-agnostic, end-to-end examples with hands-on illustrations of executing data science, machine learning, and generative AI projects on the Databricks Platform. You’ll develop expertise in Databricks' managed MLflow, Vector Search, AutoML, Unity Catalog, and Model Serving as you learn to apply them practically in everyday workflows. This Databricks book not only offers detailed code explanations but also facilitates seamless code importation for practical use. You’ll discover how to leverage the open-source Databricks platform to enhance learning, boost skills, and elevate productivity with supplemental resources. By the end of this book, you'll have mastered the use of Databricks for data science, machine learning, and generative AI, enabling you to deliver outstanding data products.What you will learn Set up a workspace for a data team planning to perform data science Monitor data quality and detect drift Use autogenerated code for ML modeling and data exploration Operationalize ML with feature engineering client, AutoML, VectorSearch, Delta Live Tables, AutoLoader, and Workflows Integrate open-source and third-party applications, such as OpenAI's ChatGPT, into your AI projects Communicate insights through Databricks SQL dashboards and Delta Sharing Explore data and models through the Databricks marketplace Who this book is for This book is for machine learning engineers, data scientists, and technical managers seeking hands-on expertise in implementing and leveraging the Databricks Data Intelligence Platform and its Lakehouse architecture to create data products.

Field-tested tips, tricks, and design patterns for building machine learning projects that are deployable, maintainable, and secure from concept to production. In Machine Learning Engineering in Action, you will learn: Evaluating data science problems to find the most effective solution Scoping a machine learning project for usage expectations and budget Process techniques that minimize wasted effort and speed up production Assessing a project using standardized prototyping work and statistical validation Choosing the right technologies and tools for your project Making your codebase more understandable, maintainable, and testable Automating your troubleshooting and logging practices Ferrying a machine learning project from your data science team to your end users is no easy task. Machine Learning Engineering in Action will help you make it simple. Inside, you'll find fantastic advice from veteran industry expert Ben Wilson, Principal Resident Solutions Architect at Databricks. Ben introduces his personal toolbox of techniques for building deployable and maintainable production machine learning systems. You'll learn the importance of Agile methodologies for fast prototyping and conferring with stakeholders, while developing a new appreciation for the importance of planning. Adopting well-established software development standards will help you deliver better code management, and make it easier to test, scale, and even reuse your machine learning code. Every method is explained in a friendly, peer-to-peer style and illustrated with production-ready source code. About the technology Deliver maximum performance from your models and data. This collection of reproducible techniques will help you build stable data pipelines, efficient application workflows, and maintainable models every time. Based on decades of good software engineering practice, machine learning engineering ensures your ML systems are resilient, adaptable, and perform in production. About the book Machine Learning Engineering in Action teaches you core principles and practices for designing, building, and delivering successful machine learning projects. You'll discover software engineering techniques like conducting experiments on your prototypes and implementing modular design that result in resilient architectures and consistent cross-team communication. Based on the author's extensive experience, every method in this book has been used to solve real-world projects. What's inside Scoping a machine learning project for usage expectations and budget Choosing the right technologies for your design Making your codebase more understandable, maintainable, and testable Automating your troubleshooting and logging practices About the reader For data scientists who know machine learning and the basics of object-oriented programming. About the author Ben Wilson is Principal Resident Solutions Architect at Databricks, where he developed the Databricks Labs AutoML project, and is an MLflow committer.

Data is bigger, arrives faster, and comes in a variety of formats—and it all needs to be processed at scale for analytics or machine learning. But how can you process such varied workloads efficiently? Enter Apache Spark. Updated to include Spark 3.0, this second edition shows data engineers and data scientists why structure and unification in Spark matters. Specifically, this book explains how to perform simple and complex data analytics and employ machine learning algorithms. Through step-by-step walk-throughs, code snippets, and notebooks, you’ll be able to: Learn Python, SQL, Scala, or Java high-level Structured APIs Understand Spark operations and SQL Engine Inspect, tune, and debug Spark operations with Spark configurations and Spark UI Connect to data sources: JSON, Parquet, CSV, Avro, ORC, Hive, S3, or Kafka Perform analytics on batch and streaming data using Structured Streaming Build reliable data pipelines with open source Delta Lake and Spark Develop machine learning pipelines with MLlib and productionize models using MLflow

Take your organization's data maturity to the next level by operationalizing data governance Key Features Develop the mindset and skills essential for successful data stewardship Apply practical advice and industry best practices, spanning data governance, quality management, and compliance, to enhance data stewardship Follow a step-by-step program to develop a data operating model and implement data stewardship effectively Purchase of the print or Kindle book includes a free PDF eBook Book DescriptionIn the competitive data-centric world, mastering data stewardship is not just a requirement—it's the key to organizational success. Unlock strategic excellence with Data Stewardship in Action, your guide to exploring the intricacies of data stewardship and its implementation for maximum efficiency. From business strategy to data strategy, and then to data stewardship, this book shows you how to strategically deploy your workforce, processes, and technology for efficient data processing. You’ll gain mastery over the fundamentals of data stewardship, from understanding the different roles and responsibilities to implementing best practices for data governance. You’ll elevate your data management skills by exploring the technologies and tools for effective data handling. As you progress through the chapters, you’ll realize that this book not only helps you develop the foundational skills to become a successful data steward but also introduces innovative approaches, including leveraging AI and GPT, for enhanced data stewardship. By the end of this book, you’ll be able to build a robust data governance framework by developing policies and procedures, establishing a dedicated data governance team, and creating a data governance roadmap that ensures your organization thrives in the dynamic landscape of data management.What you will learn Enhance your job prospects by understanding the data stewardship field, roles, and responsibilities Discover how to develop a data strategy and translate it into a functional data operating model Develop an effective and efficient data stewardship program Gain practical experience of establishing a data stewardship initiative Implement purposeful governance with measurable ROI Prioritize data use cases with the value and effort matrix Who this book is for This book is for professionals working in the field of data management, including business analysts, data scientists, and data engineers looking to gain a deeper understanding of the data steward role. Senior executives who want to (re)establish the data governance body in their organizations will find this resource invaluable. While accessible to both beginners and professionals, basic knowledge of data management concepts, such as data modeling, data warehousing, and data quality, is a must to get started.

The data lakehouse is the next generation of the data warehouse and data lake, designed to meet today's complex and ever-changing analytics, machine learning, and data science requirements. Learn about the features and architecture of the data lakehouse, along with its powerful analytical infrastructure. Appreciate how the universal common connector blends structured, textual, analog, and IoT data. Maintain the lakehouse for future generations through Data Lakehouse Housekeeping and Data Future-proofing. Know how to incorporate the lakehouse into an existing data governance strategy. Incorporate data catalogs, data lineage tools, and open source software into your architecture to ensure your data scientists, analysts, and end users live happily ever after.

Learn how to use, deploy, and maintain Apache Spark with this comprehensive guide, written by the creators of the open-source cluster-computing framework. With an emphasis on improvements and new features in Spark 2.0, authors Bill Chambers and Matei Zaharia break down Spark topics into distinct sections, each with unique goals. Youâ??ll explore the basic operations and common functions of Sparkâ??s structured APIs, as well as Structured Streaming, a new high-level API for building end-to-end streaming applications. Developers and system administrators will learn the fundamentals of monitoring, tuning, and debugging Spark, and explore machine learning techniques and scenarios for employing MLlib, Sparkâ??s scalable machine-learning library. Get a gentle overview of big data and Spark Learn about DataFrames, SQL, and Datasetsâ??Sparkâ??s core APIsâ??through worked examples Dive into Sparkâ??s low-level APIs, RDDs, and execution of SQL and DataFrames Understand how Spark runs on a cluster Debug, monitor, and tune Spark clusters and applications Learn the power of Structured Streaming, Sparkâ??s stream-processing engine Learn how you can apply MLlib to a variety of problems, including classification or recommendation

Analyze vast amounts of data in record time using Apache Spark with Databricks in the Cloud. Learn the fundamentals, and more, of running analytics on large clusters in Azure and AWS, using Apache Spark with Databricks on top. Discover how to squeeze the most value out of your data at a mere fraction of what classical analytics solutions cost, while at the same time getting the results you need, incrementally faster. This book explains how the confluence of these pivotal technologies gives you enormous power, and cheaply, when it comes to huge datasets. You will begin by learning how cloud infrastructure makes it possible to scale your code to large amounts of processing units, without having to pay for the machinery in advance. From there you will learn how Apache Spark, an open source framework, can enable all those CPUs for data analytics use. Finally, you will see how services such as Databricks provide the power of Apache Spark, without you having to know anything about configuring hardware or software. By removing the need for expensive experts and hardware, your resources can instead be allocated to actually finding business value in the data. This book guides you through some advanced topics such as analytics in the cloud, data lakes, data ingestion, architecture, machine learning, and tools, including Apache Spark, Apache Hadoop, Apache Hive, Python, and SQL. Valuable exercises help reinforce what you have learned. What You Will Learn Discover the value of big data analytics that leverage the power of the cloudGet started with Databricks using SQL and Python in either Microsoft Azure or AWSUnderstand the underlying technology, and how the cloud and Apache Spark fit into the bigger picture See how these tools are used in the real world Run basic analytics, including machine learning, on billions of rows at a fraction of a cost or free Who This Book Is For Data engineers, data scientists, and cloud architects who want or need to run advanced analytics in the cloud. It is assumed that the reader has data experience, but perhaps minimal exposure to Apache Spark and Azure Databricks. The book is also recommended for people who want to get started in the analytics field, as it provides a strong foundation.