The term big data refers to extremely large sets of data that are analyzed to reveal insights, such as patterns, trends, and associations. The algorithms that analyze this data to provide these insights must extract value from a wide range of data sources, including business data and live, streaming, social media data. However, the real value of these insights comes from their timeliness. Rapid delivery of insights enables anyone (not only data scientists) to make effective decisions, applying deep intelligence to every enterprise application. Apache Spark is an integrated analytics framework and runtime to accelerate and simplify algorithm development, depoyment, and realization of business insight from analytics. Apache Spark on IBM® z/OS® puts the open source engine, augmented with unique differentiated features, built specifically for data science, where big data resides. This IBM Redbooks® publication describes the installation and configuration of IBM z/OS Platform for Apache Spark for field teams and clients. Additionally, it includes examples of business analytics scenarios.

Artificial intelligence (AI) enables computers and machines to mimic the perception, learning, problem-solving, and decision-making capabilities of the human mind. AI development is made possible by the availability of large amounts of data and the corresponding development and wide availability of computer systems that can process all that data faster and more accurately than humans can. What happens if you infuse AI with a world-class database management system, such as IBM Db2®? IBM® has done just that with Db2 AI for z/OS (Db2ZAI). Db2ZAI is built to infuse AI and data science to assist businesses in the use of AI to develop applications more easily. With Db2ZAI, the following benefits are realized: Data science functionality Better built applications Improved database performance (and DBA's time and efforts are saved) through simplification and automation of error reporting and routine tasks Machine learning (ML) optimizer to improve query access paths and reduce the need for manual tuning and query optimization Integrated data access that makes data available from various vendors including private cloud providers. This IBM Redpaper® publication helps to simplify your installation by tailoring and configuration of Db2 AI for z/OS®. It was written for system programmers, system administrators, and database administrators.

Analytics is increasingly an integral part of day-to-day operations at today's leading businesses, and transformation is also occurring through huge growth in mobile and digital channels. Enterprise organizations are attempting to leverage analytics in new ways and transition existing analytics capabilities to respond with more flexibility while making the most efficient use of highly valuable data science skills. The recent growth and adoption of Apache Spark as an analytics framework and platform is very timely and helps meet these challenging demands. The Apache Spark environment on IBM z/OS® and Linux on IBM z SystemsTM platforms allows this analytics framework to run on the same enterprise platform as the originating sources of data and transactions that feed it. If most of the data that will be used for Apache Spark analytics, or the most sensitive or quickly changing data is originating on z/OS, then an Apache Spark z/OS based environment will be the optimal choice for performance, security, and governance. This IBM® RedpaperTM publication explores the enterprise analytics market, use of Apache Spark on IBM z SystemsTM platforms, integration between Apache Spark and other enterprise data sources, and case studies and examples of what can be achieved with Apache Spark in enterprise environments. It is of interest to data scientists, data engineers, enterprise architects, or anybody looking to better understand how to combine an analytics framework and platform on enterprise systems.

Open-source software (OSS) is widely available and serves as an essential component for enterprises in the artificial intelligence (AI) and machine learning (ML) industry. Specifically, the open-source programming language Python is one of the most versatile and popular programming languages that are used in the world at the time of writing. This situation is especially true in the data science community, where Python provides many libraries and tools that enable essential AI and ML functions, and where it is supported by a large community of developers that actively contribute to its development. Understanding and managing vulnerabilities within OSS can be complex because of the many components, dependencies, and contributors that are involved. Although the nature of OSS helps balance access to programming and technology, it also results in fast-paced changes to software, which emphasizes the importance of software currency to minimize security concerns. Enterprises understand the critical need to have access to and leverage reputable open-source projects with proper maintenance, updates, transparency, reliable support, and a sense of control to form a secure foundation for implementing AI solutions. Python AI Toolkit for IBM® z/OS® is a powerful set of tools and libraries that is used to establish a secure foundation for AI development and deployment on z/OS so that enterprises can leverage their existing infrastructure for these mission-critical applications. The OSS that is provided within Python AI Toolkit for IBM z/OS is scanned and vetted for security vulnerabilities so that users can make informed decisions when leveraging these Python packages. Packages can be installed and managed by using the Package Installer for Python (pip), which is a common Python package manager, enabling a familiar, flexible, and agile delivery experience while empowering developers to build AI solutions.

This IBM® Redbooks® publication describes the features and functions the latest member of the IBM Z® platform, the IBM z15TM Model T02 (machine type 8562). It includes information about the IBM z15 processor design, I/O innovations, security features, and supported operating systems. The z15 is a state-of-the-art data and transaction system that delivers advanced capabilities, which are vital to any digital transformation. The z15 is designed for enhanced modularity, which is in an industry standard footprint. This system excels at the following tasks: Making use of multicloud integration services Securing data with pervasive encryption Accelerating digital transformation with agile service delivery Transforming a transactional platform into a data powerhouse Getting more out of the platform with IT Operational Analytics Accelerating digital transformation with agile service delivery Revolutionizing business processes Blending open source and Z technologies This book explains how this system uses new innovations and traditional Z strengths to satisfy growing demand for cloud, analytics, and open source technologies. With the z15 as the base, applications can run in a trusted, reliable, and secure environment that improves operations and lessens business risk.

This IBM® Redbooks® publication describes the features and functions the latest member of the IBM Z® platform, the IBM z15TM (machine type 8561). It includes information about the IBM z15 processor design, I/O innovations, security features, and supported operating systems. The z15 is a state-of-the-art data and transaction system that delivers advanced capabilities, which are vital to any digital transformation. The z15 is designed for enhanced modularity, which is in an industry standard footprint. This system excels at the following tasks: Making use of multicloud integration services Securing data with pervasive encryption Accelerating digital transformation with agile service delivery Transforming a transactional platform into a data powerhouse Getting more out of the platform with IT Operational Analytics Accelerating digital transformation with agile service delivery Revolutionizing business processes Blending open source and Z technologies This book explains how this system uses new innovations and traditional Z strengths to satisfy growing demand for cloud, analytics, and open source technologies. With the z15 as the base, applications can run in a trusted, reliable, and secure environment that improves operations and lessens business risk.

There has been a considerable focus on performance improvements as one of the main themes in recent IBM DB2® releases, and DB2 12 for IBM z/OS® is certainly no exception. With the high-value data retained on DB2 for z/OS and the z Systems platform, customers are increasingly attempting to extract value from that data for competitive advantage. Although customers have historically moved data off platform to gain insight, the landscape has changed significantly and allowed z Systems to again converge operational systems with analytics for real-time insight. Business-critical analytics is now requiring the same levels of service as expected for operational systems, and real-time or near real-time currency of data is expected. Hence the resurgence of z Systems. As a precursor to this shift, IDAA brought the data warehouse back to DB2 for z/OS and, with its tight integration with DB2, significantly reduces data latency as compared to the ETL processing that is involved with moving data to a stand-alone data warehouse environment. That change has opened up new opportunities for operational systems to extend the breadth of analytics processing without affecting the mission-critical system and integrating near real-time analytics within that system, all while maintaining the same z Systems qualities of service. Apache Spark on z/OS and Linux for System z also allow analytics in-place, in real-time or near real-time. Enabling Spark natively on z Systems reduces the security risk of multiple copies of the Enterprise data, while providing an application developer-friendly platform for faster insight in a simplified and more secure analytics framework. How is all of this relevant to DB2 for z/OS? Given that z Systems is proving again to be the core Enterprise Hybrid Transactional/Analytical Processing (HTAP) system, it is critical that DB2 for z/OS can handle its traditional transactional applications and address the requirements for analytics processing that might not be candidates for these rapidly evolving targeted analytics systems. And not only are there opportunities for DB2 for z/OS to play an increasing role in analytics, the complexity of the transactional systems is increasing. Analytics is being integrated within the scope of those transactions. DB2 12 for z/OS has targeted performance to increase the success of new application deployments and integration of analytics to ensure that we keep pace with the rapid evolution of IDAA and Spark as equal partners in HTAP systems. This paper describes the enhancements delivered specifically by the query processing engine of DB2. This engine is generally called the optimizer or the Relational Data Services (RDS) components, which encompasses the query transformation, access path selection, run time, and parallelism. DB2 12 for z/OS also delivers improvements targeted at OLTP applications, which are the realm of the Data Manager, Index Manager, and Buffer Manager components (to name a few), and are not identified here. Although the performance measurement focus is based on reducing CPU, improvement in elapsed time is likely to be similarly achieved as CPU is reduced and performance constraints alleviated. However, elapsed time improvements can be achieved with parallelism, and DB2 12 does increase the percentage offload for parallel child tasks, which can further reduce chargeable CPU for analytics workloads.