Introduction to the Azure Data Lake and U-SQL
The Big Data revolution has exposed the limitations of traditional data processing models like cubes and ETL. Learn a different way of doing things with the Azure Data Lake, using the U-SQL language to query raw data files and create databases.
What you'll learn
Building good reporting structures can be difficult, especially when those pesky users keep asking for new reports. Throw Big Data into the mix and things become a lot more complicated. What if you didn’t need to build any data models at all, or you could build models that could be quickly put up and torn down? In this course, Introduction to the Azure Data Lake and U-SQL, you'll be introduced to Azure Data Lake and the U-SQL language, and learn how to abandon ETL. First, you'll delve into querying by using the powerful U-SQL language, built straight into the Azure Data Lake. Next, you'll discover how to throw your files into the Data Lake and query them directly without needing to load them into a database. Finally, you'll learn about how Azure Data Lakes offers the best of both worlds, with support for unstructured files and structured databases. By the end of this course, you’ll not only know what a Data Lake is, you’ll know how to populate it, query it, and develop for it using Visual Studio. Software required: Visual Studio 2017 Community Edition and Azure subscription (optional).
Table of contents
- Introduction 1m
- Scenario 5m
- Setting up Files for Local Development 3m
- Setting up Files for Remote Development 2m
- Designing the Script 3m
- Writing the U-SQL Script 4m
- File Sets And Virtual Columns 4m
- Completing the U-SQL Script 3m
- Build Issues 6m
- Executing Against Azure 1m
- Bad File Names in Azure Scripts 1m
- The SSIS Solution 4m
- Summary 2m
- Introduction 1m
- Scenario 3m
- Report 1 Postcode Area Totals 6m
- Limiting Result Sets 2m
- Report 2 No Demographic Information 8m
- Report 3 Average Person Counts 3m
- Casting Data 3m
- Rounding Numbers 2m
- Report 4 Occupied Household Ranges 2m
- Ranges with the Conditional Operator 2m
- Ranges with the Case Statement 4m
- Summary 2m
About the author
Mike is a polyglot architect/developer who loves to mess around with data. He runs a consultancy company in the UK, helping organizations to build better systems and databases. Mike is married and has one daughter and two dogs, Dolly and Bertie. They keep his feet warm when he's working on courses and articles. Mike has written a book on SQL Server and writes regular articles for sqlservercentral.com. He also likes collecting old computers like the Commodore 64 and Sinclair Spectrum. No, he doe... more