Flight Lab / Systems and cyber-physical / 02
Lesson 02 of 05 · about 4 min
Classic MapReduce writes the output of a stage to disk before the next stage starts. Spark can keep a chain of narrow transforms in memory and only materialize when an action or a wide dependency asks it to.
Two pipelines do the same three maps. Say where each one keeps the intermediate data.