Description:

This five-day intensive course provides hands-on training in building scalable data pipelines and analytics solutions on Google Cloud Platform. Participants will master BigQuery for data warehousing, Dataflow for ETL/ELT processing, Cloud Storage for data lakes, and Looker for business intelligence. The course covers cloud-native architecture patterns, data governance, and real-world case studies to enable participants to design and implement enterprise-grade data solutions on GCP.

Duration: 

5 Days

Course Code: BDT34

Learning Objectives:

After this course, you will be able to:

1. Design and deploy data lakes using Cloud Storage and implement structured data warehouses with BigQuery
2. Build scalable, serverless ETL/ELT pipelines using Dataflow (Apache Beam) and Pub/Sub for real-time streaming
3. Develop analytics solutions with BigQuery, apply data security best practices, and optimize query performance
4. Create interactive dashboards and reports using Looker, and implement data governance frameworks on GCP

1. Data Engineers and ETL developers transitioning to cloud platforms
2. Data Analysts and Business Intelligence professionals seeking cloud analytics expertise
3. Solutions Architects and IT leaders evaluating GCP for data and analytics workloads

1. Familiarity with SQL and basic database concepts
2. Understanding of ETL/ELT processes and data pipeline fundamentals
3. Basic Linux/cloud command-line knowledge (beneficial)

Course Outline:

Day 1: GCP Cloud Data Platform Fundamentals

Module 1: GCP Overview & Cloud Data Architecture

1. GCP services and data ecosystem (Compute, Storage, Big Data, Analytics)
2. Cloud-native data architecture patterns and design principles
3. Identity & Access Management (IAM) and foundational security

Module 2: Cloud Storage & Data Lakes

• Cloud Storage: Buckets, storage classes, and lifecycle policies
• Data lake design patterns and metadata management
• Hands-on: Creating and managing data lake structures

Day 2: BigQuery Data Warehouse Essentials

Module 3: BigQuery Architecture & Data Loading

1. BigQuery architecture: Columnar storage, Dremel, and query execution
2. Data loading: Batch import, streaming inserts, and data validation
3. Schema design, partitioning, and clustering for performance

Module 4: BigQuery SQL & Analytics

1. Advanced SQL: Window functions, CTEs, and complex joins
2. Query optimization and cost control strategies
3. Hands-on: Real-world analytics queries and exploratory analysis

Day 3: ETL/ELT Pipelines with Dataflow

Module 5: Apache Beam & Dataflow Fundamentals

1. Apache Beam programming model: ParDo, GroupByKey, Combine
2. Dataflow runners: Batch and streaming execution
3. Data transformation patterns and windowing strategies

Module 6: Building Production Pipelines

1. Dataflow templates and reusable pipeline components
2. Error handling, monitoring, and pipeline debugging
3. Hands-on: Building end-to-end ETL pipeline to BigQuery

Day 4: Real-Time Analytics & Pub/Sub

Module 7: Cloud Pub/Sub & Real-Time Streaming

1. Pub/Sub architecture: Topics, subscriptions, and message delivery
2. Stream processing with Dataflow and BigQuery streaming inserts
3. Real-time analytics patterns and use cases

Module 8: Data Governance & Security

1. Data classification, encryption, and access control
2. Data lineage and metadata management with Data Catalog
3. Compliance and audit logging best practices

Day 5: Analytics, BI & Capstone Project

Module 9: Looker & Business Intelligence

1. Looker fundamentals: Explores, dashboards, and data visualization
2. Looker Model Layer (LookML) for semantic definitions
3. Building interactive reports and embedding analytics

Module 10: Case Studies & Capstone

1. Real-world GCP data engineering case studies and architectures
2. Capstone project: Design and implement integrated data solution
3. Best practices, optimization strategies, and next steps

All modules include hands-on labs using GCP Console, Cloud Shell, and Python. Participants receive course materials, lab scripts, and access to GCP sandbox environment.