In the Advanced Apache Spark training course, you'll learn:
Review of Core Apache Spark Concepts
- How Spark works
- RDD Fundamentals
- SparkSQL and DataFrames
- Spark Streaming concepts
- Machine Learning basics
Understanding Spark Internals for Performance
- Schedules, jobs, and tasks
- Data structures, sets and lakes
- Shuffle and performance
- Understanding data sources and partitions
- Read, writes and performance
New Features of Spark 2
- API Stability
- Core and Spark SQL changes
- Changes to packaging and operations
Working with Spark
- Debugging/troubleshooting Spark apps
- Developing data workflows
- Automated Spark builds using Maven
Clustering with Spark
- Running a Spark cluster
- Understanding cluster resource requirements
- Managing Memory on ExecutorsWorker
- Managing memory/cores across a spark cluster
- Performance tuning
- Clarifying best practices
Spark Integration
- Implementing Spark on DataStax, Hortonworks etc.
- Integrating with Cassandra
- Integrating with Kafka
- Integrating with Elassticsearch
- Integrating with other compatible NoSQL implementations (as desired)
Machine Learning with Spark
- Common algorithms
- Commonly used algorithms with Scala
- Machine learning libraries: MLLib, H20
- Custom algorithms creation
Advanced Spark SQL and Spark Streaming
- Leveraging Spark 2 API (Spark Session etc)
- Developing with Spark Dataframes
- Writing sollid spark jobs
- Understanding when to use spark and when to not use spark
High Performance Spark applications
- Performance tuning process
- Performance tuning metrics
- SQL performance tuning
- High performant caching strategies
- Cluster resource requirements
- Creating fault-tolerance
