(Disclaimer : This is not an official post from Databricks)
Spark Summit 2015 in San Francisco was well attended. Kudos to the Databricks team for organizing this fantastic conference.
All of the conference talks are available online (both slide decks & video). The talks were excellent! Here are some highlights:
Spark: Cool Features and What’s New
Spark is very actively developed and version 1.4 was released just before the conference. Here are some talks to get the latest:
- Spark Update by Databricks founders (video) gives a nice overview of past / present / future developments of Spark
- Dataframes (Slides / Video)
- Machine Learning pipelines (Slides / Video)
- Tungsten : next-gen Spark engine (Slides / Video) & (Slides / Video)
- SparkR : R & Spark (Slides / Video) & (Slides / Video)
- IBM is getting behind Spark (slides / video)
Spark Use Cases
What are people doing with Spark?
- Some Spark use cases summary (Slides / Video)
- Spark @ NASA / JPL (slides / video)
Spark usage in wrangling data in multiple formats and SciSpark project - Digital attribution (online advertising) @ Adobe (Slide / Video)
- Spark Streaming @ Netflix (Slides / Video)
- Spark + Cassandra + Solr @ USPTO (Slides / Video)
- Predictive analytics in Operating Room @ Beth Israel Medical Center (Slides / Video)
- Price modeling @ AirBnB (Slides / Video)
- Spark @ Edmunds.com (Slides / Video)
- Online advertising use case @ Pubmatic (Slides / Video)
- Customer 360 Insights @ Toyota (Slides / Video)
- Spark & Tachyon (Terabytes data cached in memory!) @ Baidu (Slides / Video)
- Machine Learning on Online advertising data @ InMobi (Slides / Video)
- Recommendations @ OpenTable (Slides / Video)
- Machine Learning @ Autodesk (Slides / Video)
- Real time traffic analysis @ Autotrader (Slides / Video)
- Food recommendations @ MyFitnessPal (Slides / Video)
- Stream processing logs @ Salesforce (Slides / Video)
- GraphX @ 247 (Slides / Video)
Spark Development Tips
- Data formats to use and their implications : (slides / video)
Can’t decide CSV / JSON / Parquet formats? This will help - Spark performance (Slides / Video)
deep dive into Spark performance
- Spark performance 2 (Slides / Video)
Using Intel’s HiBench and HiMeter suites to analyze Spark’s performance - Spark Streaming — exactly-once processing with Kafka (Slides / Video)
- Spark and Cassandra (Slides / Video)
- Spark & Solr (Slides / Video)
- Benchmarking streaming systems (Slides / Video)
Spark Operations
- Autoscaling Spark on YARN (Slides / Video)
- Spark on Mesos (Slides / Video)
- Insight into Spark Execution with Unravel (Slides / Video)
- Tips of running Spark streaming in production (Slides / Video)
- Java8 garbage collector tips (Slides / Video)