Build efficient data flow and machine learning programs with this flexible, multi-functional open-source cluster-computing framework Key FeaturesMaster the art of real-time big data processing and machine learning Explore a wide range of use-cases to analyze large data Discover ways to optimize your work by using many features of Spark 2.x and ScalaBook Description Apache Spark is an in-memory, cluster-based data processing system that provides a wide range of functionalities such as big data processing, analytics, machine learning, and more. With this Learning Path, you can take your knowledge of Apache Spark to the next level by learning how to expand Spark's functionality and building your own data flow and machine learning programs on this platform. You will work with the different modules in Apache Spark, such as interactive querying with Spark SQL, using DataFrames and datasets, implementing streaming analytics with Spark Streaming, and applying machine learning and deep learning techniques on Spark using MLlib and various external tools. By the end of this elaborately designed Learning Path, you will have all the knowledge you need to master Apache Spark, and build your own big data processing and analytics pipeline quickly and without any hassle. This Learning Path includes content from the following Packt products: Mastering Apache Spark 2.x by Romeo KienzlerScala and Spark for Big Data Analytics by Md. Rezaul Karim, Sridhar AllaApache Spark 2.x Machine Learning Cookbook by Siamak Amirghodsi, Meenakshi Rajendran, Broderick Hall, Shuen MeiCookbookWhat you will learnGet to grips with all the features of Apache Spark 2.xPerform highly optimized real-time big data processing Use ML and DL techniques with Spark MLlib and third-party toolsAnalyze structured and unstructured data using SparkSQL and GraphXUnderstand tuning, debugging, and monitoring of big data applications Build scalable and fault-tolerant streaming applications Develop scalable recommendation enginesWho this book is for If you are an intermediate-level Spark developer looking to master the advanced capabilities and use-cases of Apache Spark 2.x, this Learning Path is ideal for you. Big data professionals who want to learn how to integrate and use the features of Apache Spark and build a strong big data pipeline will also find this Learning Path useful. To grasp the concepts explained in this Learning Path, you must know the fundamentals of Apache Spark and Scala.

Boost your Azure career by mastering essential data concepts and cloud services with this pragmatic guide Purchase of this book unlocks access to web-based exam prep resources such as mock exams, flashcards, exam tips, and the eBook PDF Key Features Gain Azure certification insights from industry veteran and Microsoft MVP, Steve Miles Dive into expertly crafted content aligned with the latest DP-900 exam requirements Test your skills with mock exams that mirror the actual certification exam Book DescriptionMicrosoft's Azure Data Fundamentals (DP-900) certification exam validates your expertise in core data concepts and Azure’s powerful data services capabilities. This comprehensive guide written by Steve Miles—a Microsoft Azure MVP and certified trainer with over 25 years of experience in cloud data services and 30+ certifications across major platforms—serves as your gateway to a future shaped by data and AI, regardless of your technical background. With the help of examples, you'll learn fundamental data concepts, including data representation, data storage options, and common workloads and gain clarity on the roles and responsibilities of key data professionals such as data administrators, engineers, and analysts. This guide covers all crucial exam domains, from data services capabilities of the Azure cloud platform to considerations for relational, non-relational, and analytics workloads, encompassing both Microsoft and open-source technologies. To supplement your exam prep, this book gives you access to a suite of online resources designed to boost your confidence, including mock tests, interactive flashcards, and invaluable exam tips By the end of this book, you’ll be fully prepared not only to pass the DP-900 exam but also to confidently tackle data solutions in Azure, setting a strong foundation for your data-driven careerWhat you will learn Analyze features of structured, semi-structured, and unstructured data Utilize Azure SQL and open-source database services confidently Identify and evaluate Azure storage options Understand the versatility of Azure Cosmos DB through use cases and APIs Apply cutting-edge strategies for large-scale analytics in Azure Master core data concepts crucial for Azure environments Explore Microsoft's cloud services for real-time analytics Demonstrate proficiency in data visualization using Power BI Who this book is for This exam guide is designed for anyone who wants to work with Azure data services and prepare for the Azure DP-900 exam. Whether you're an administrator, engineer, architect, developer, analyst, aspiring data scientist, or a non-technical enthusiast interested in learning data concepts, this book is for you. It also lays the groundwork for those planning to pursue more advanced data or AI certifications. A foundational understanding of cloud concepts and client-server applications is assumed.

This book serves as a comprehensive guide for readers who wish to understand how artificial intelligence works, how it is used, and which fields it serves with concrete examples, covering a total of 156 fundamental AI tools across 12 main categories and 49 subcategories. These tools, starting with major categories such as natural language processing, image processing, data analytics, and robotic systems, offer groundbreaking solutions in the world of information technologies with their functionality and versatility. The tools presented in this book aim to enhance the readers' academic knowledge and practical application skills by offering innovative and effective solutions in various fields. Each tool is introduced according to the fundamental principles of its respective area, with technical explanations and usage scenarios on how it works. The content of the book is designed to be beneficial to a wide audience, ranging from researchers to students, software developers to industry professionals. Each chapter of the book is detailed to ensure an in-depth understanding of artificial intelligence. Examples demonstrating the application areas, benefits, and limitations of each tool allow the reader to assimilate the information with a practical approach. We hope that this book will serve as a reference source for all readers who wish to explore innovative solutions in AI and gain deep knowledge in this field.

In order to establish and maintain a successful company in the digital age, managers are digitally transforming their organizations to include such tools as disruptive technologies and digital data to improve performance and efficiencies. As these companies continue to adopt digital technologies to improve their businesses and create new revenues and value-producing opportunities, they must also be aware of the challenges digitalization can present. Business Transformations in the Era of Digitalization is a collection of innovative research on the latest trends, business opportunities, and challenges in the digitalization of businesses. Highlighting a range of topics including business-IT alignment, cloud computing, Internet of Things (IoT), business sustainability, small and medium-sized enterprises, and digital entrepreneurship, this book is ideally designed for managers, professionals, consultants, entrepreneurs, and researchers.

Over 70 recipes to help you use Apache Spark as your single big data computing platform and master its libraries About This Book This book contains recipes on how to use Apache Spark as a unified compute engine Cover how to connect various source systems to Apache Spark Covers various parts of machine learning including supervised/unsupervised learning & recommendation engines Who This Book Is For This book is for data engineers, data scientists, and those who want to implement Spark for real-time data processing. Anyone who is using Spark (or is planning to) will benefit from this book. The book assumes you have a basic knowledge of Scala as a programming language. What You Will Learn Install and configure Apache Spark with various cluster managers & on AWS Set up a development environment for Apache Spark including Databricks Cloud notebook Find out how to operate on data in Spark with schemas Get to grips with real-time streaming analytics using Spark Streaming & Structured Streaming Master supervised learning and unsupervised learning using MLlib Build a recommendation engine using MLlib Graph processing using GraphX and GraphFrames libraries Develop a set of common applications or project types, and solutions that solve complex big data problems In Detail While Apache Spark 1.x gained a lot of traction and adoption in the early years, Spark 2.x delivers notable improvements in the areas of API, schema awareness, Performance, Structured Streaming, and simplifying building blocks to build better, faster, smarter, and more accessible big data applications. This book uncovers all these features in the form of structured recipes to analyze and mature large and complex sets of data. Starting with installing and configuring Apache Spark with various cluster managers, you will learn to set up development environments. Further on, you will be introduced to working with RDDs, DataFrames and Datasets to operate on schema aware data, and real-time streaming with various sources such as Twitter Stream and Apache Kafka. You will also work through recipes on machine learning, including supervised learning, unsupervised learning & recommendation engines in Spark. Last but not least, the final few chapters delve deeper into the concepts of graph processing using GraphX, securing your implementations, cluster optimization, and troubleshooting. Style and approach This book is packed with intuitive recipes supported with line-by-line explanations to help you understand Spark 2.x's real-time processing capabilities and deploy scalable big data solutions. This is a valuable resource for data scientists and those working on large-scale data projects.

Unlock the Potential of Scalable and Serverless Data with "Mastering Amazon DynamoDB Database" In today's data-centric world, the ability to efficiently manage and scale databases is a cornerstone of success. "Mastering Amazon DynamoDB Database" is your comprehensive guide to mastering one of the most robust and versatile NoSQL databases available – Amazon DynamoDB. Whether you're a seasoned data professional or a newcomer to NoSQL technology, this book equips you with the knowledge and skills needed to harness the full capabilities of Amazon DynamoDB. About the Book: "Mastering Amazon DynamoDB Database" takes you on a transformative journey through the intricacies of this dynamic NoSQL database. From fundamental concepts to advanced techniques, you'll explore DynamoDB's architecture, data model, and powerful features. Each chapter is meticulously crafted to provide both a deep understanding of the concepts and practical applications in real-world scenarios. Key Features: · DynamoDB Fundamentals: Lay a solid foundation by delving into DynamoDB's architecture, data model, and the principles that make it a leader in distributed databases. · Data Modeling: Learn how to design efficient schema structures that optimize storage, access patterns, and query performance in DynamoDB. · Serverless Scalability: Explore DynamoDB's seamless scalability, taking advantage of its serverless nature to accommodate growing workloads without manual intervention. · Advanced Querying: Master DynamoDB's powerful query capabilities, including filtering, indexing, and advanced querying techniques that enable complex data retrieval. · Best Practices: Dive into best practices for data modeling, indexing strategies, partition key selection, and managing read and write capacity to ensure optimal performance. · Real-World Applications: Gain insights from real-world use cases across industries, from e-commerce and gaming to IoT and beyond, showcasing DynamoDB's adaptability. · Integration and Ecosystem: Explore DynamoDB's integration with other AWS services, APIs, and developer tools, empowering you to build end-to-end solutions. · Advanced Topics: Uncover advanced concepts such as transactions, backups, global tables, security mechanisms, and best practices for disaster recovery. Who This Book Is For: "Mastering Amazon DynamoDB Database" caters to developers, data engineers, solution architects, and anyone interested in leveraging the power of NoSQL databases. Whether you're seeking to enhance your skills or dive into the world of serverless databases, this book provides the insights and tools to navigate DynamoDB's intricacies. Why You Should Read This Book: In an era where scalability and performance are paramount, Amazon DynamoDB shines as a cornerstone of data management. "Mastering Amazon DynamoDB Database" empowers you to fully harness its capabilities, enabling you to build highly available applications, deliver seamless user experiences, and scale effortlessly. © 2023 Cybellium Ltd. All rights reserved. www.cybellium.com

Master AWS data engineering services and techniques for orchestrating pipelines, building layers, and managing migrations Key Features Get up to speed with the different AWS technologies for data engineering Learn the different aspects and considerations of building data lakes, such as security, storage, and operations Get hands on with key AWS services such as Glue, EMR, Redshift, QuickSight, and Athena for practical learning Purchase of the print or Kindle book includes a free PDF eBook Book DescriptionPerforming data engineering with Amazon Web Services (AWS) combines AWS's scalable infrastructure with robust data processing tools, enabling efficient data pipelines and analytics workflows. This comprehensive guide to AWS data engineering will teach you all you need to know about data lake management, pipeline orchestration, and serving layer construction. Through clear explanations and hands-on exercises, you’ll master essential AWS services such as Glue, EMR, Redshift, QuickSight, and Athena. Additionally, you’ll explore various data platform topics such as data governance, data quality, DevOps, CI/CD, planning and performing data migration, and creating Infrastructure as Code. As you progress, you will gain insights into how to enrich your platform and use various AWS cloud services such as AWS EventBridge, AWS DataZone, and AWS SCT and DMS to solve data platform challenges. Each recipe in this book is tailored to a daily challenge that a data engineer team faces while building a cloud platform. By the end of this book, you will be well-versed in AWS data engineering and have gained proficiency in key AWS services and data processing techniques. You will develop the necessary skills to tackle large-scale data challenges with confidence.What you will learn Define your centralized data lake solution, and secure and operate it at scale Identify the most suitable AWS solution for your specific needs Build data pipelines using multiple ETL technologies Discover how to handle data orchestration and governance Explore how to build a high-performing data serving layer Delve into DevOps and data quality best practices Migrate your data from on-premises to AWS Who this book is for If you're involved in designing, building, or overseeing data solutions on AWS, this book provides proven strategies for addressing challenges in large-scale data environments. Data engineers as well as big data professionals looking to enhance their understanding of AWS features for optimizing their workflow, even if they're new to the platform, will find value. Basic familiarity with AWS security (users and roles) and command shell is recommended.

This book primarily aims to provide an in-depth understanding of recent advances in big data computing technologies, methodologies, and applications along with introductory details of big data computing models such as Apache Hadoop, MapReduce, Hive, Pig, Mahout in-memory storage systems, NoSQL databases, and big data streaming services such as Apache Spark, Kafka, and so forth. It also covers developments in big data computing applications such as machine learning, deep learning, graph processing, and many others. Features: Provides comprehensive analysis of advanced aspects of big data challenges and enabling technologies. Explains computing models using real-world examples and dataset-based experiments. Includes case studies, quality diagrams, and demonstrations in each chapter. Describes modifications and optimization of existing technologies along with the novel big data computing models. Explores references to machine learning, deep learning, and graph processing. This book is aimed at graduate students and researchers in high-performance computing, data mining, knowledge discovery, and distributed computing.

Unleash the Power of Big Data Processing In the realm of big data, the MapReduce framework stands as a cornerstone, enabling the processing of massive datasets with unparalleled efficiency. "Mastering the MapReduce Framework" is your comprehensive guide to understanding and harnessing the capabilities of this transformative technology, equipping you with the skills needed to navigate the landscape of large-scale data processing. About the Book: As the volume of data continues to grow exponentially, traditional data processing methods fall short. The MapReduce framework emerges as a powerful solution, allowing organizations to process and analyze vast datasets in parallel, thereby unlocking insights and accelerating decision-making. "Mastering the MapReduce Framework" provides a deep dive into this technology, catering to both beginners and experienced professionals seeking to maximize their proficiency in big data processing. Key Features: Foundation Building: Begin by comprehending the fundamental concepts underlying MapReduce. Understand how the framework breaks down complex tasks into smaller, manageable components that can be processed concurrently. Parallel Processing: Dive into the intricacies of parallel processing, a cornerstone of MapReduce. Learn how data is partitioned and distributed across a cluster of machines, enabling lightning-fast computation. Map and Reduce Functions: Grasp the significance of map and reduce functions in the MapReduce paradigm. Learn how to structure these functions to transform and aggregate data efficiently. Hadoop Ecosystem: Explore the Hadoop ecosystem, which houses the MapReduce framework. Understand how Hadoop integrates with other tools to create a comprehensive big data processing environment. Optimizing Performance: Discover techniques for optimizing MapReduce performance. Learn about data locality, combiners, and partitioners that enhance efficiency and reduce resource consumption. Real-World Use Cases: Gain insights into real-world applications of MapReduce across industries. From web log analysis to recommendation systems, explore how the framework powers data-driven solutions. Challenges and Solutions: Explore the challenges of working with MapReduce, such as debugging and handling skewed data. Master strategies to address these challenges and ensure smooth execution. Why This Book Matters: In a data-driven world, the ability to process and extract insights from massive datasets is a competitive advantage. "Mastering the MapReduce Framework" empowers data engineers, analysts, and technology enthusiasts to tap into the potential of big data processing, enabling them to drive innovation and make data-driven decisions with confidence. Who Should Read This Book: Data Engineers: Enhance your big data processing skills with a deep understanding of MapReduce. Data Analysts: Grasp the principles that power large-scale data analysis and gain insights from big data. Technology Enthusiasts: Dive into the world of big data processing and stay ahead of emerging trends. Harness the Power of Big Data Processing: The era of big data requires sophisticated processing tools, and the MapReduce framework stands as a pioneer in this realm. "Mastering the MapReduce Framework" equips you with the knowledge needed to harness the power of MapReduce, unleashing the potential of big data processing and enabling you to navigate the complexities of large-scale data analysis with ease. Your journey to mastering the art of big data processing begins here. © 2023 Cybellium Ltd. All rights reserved. www.cybellium.com

SmartSquare - an interdisciplinary guided tour of an urban testbed SmartSquare is an urban testbed in the emerging domain of “Smart Culture in Smart Cities”. The square is the so-called Domplatz (Cathedral Square), the location of the founding fortification Hammaburg, in the inner city of Hamburg, Germany. SmartSquare is a “Smart Service City” project with multiple-stakeholder perspectives on the activation of this culturally significant inner-city square by means of digital cultural storytelling, data analytics, simulation and service innovation. SmartSquare is a joint project of HafenCity University and the Archaeological Museum Hamburg in cooperation with the digital cluster Hamburg@Work. Funded by the German Federal Ministry for Education and Research (BMBF).