Configuring Hadoop On EC2
I am about to go live with my first production Hadoop job for a client as a proof of concept.
I found that a lot of the documentation out there is quite text dense, unnecessarily detailed or out of date, which is frustrating when you are just trying to get your first cluster up and your first MapReduce job submitted.
For that reason, I decided to write up the guide I wish I had while first getting up and running with Hadoop: a simple step by step guide to Hadoop on my preferred host, Amazon EC2.
Download the walkthrough as a PowerPoint presentation. I tried to use SlideShare but it was painfully slow at the time. I hope the deck is helpful to someone.
If you are interested in Hadoop and the wider data world, you may also like my DevOps Friday newsletter and the note on metrics driven development.
Reference shelf
Sources and further reading for this note.
- Apache Hadoop documentationThe official guide to the platform.Opens in a new tab
- Apache HadoopThe project home and releases.Opens in a new tab
- Amazon EC2The host used in the walkthrough.Opens in a new tab
- Hadoop on WikipediaBackground on MapReduce and HDFS.Opens in a new tab
- Hadoop questionsPractical answers from working teams.Opens in a new tab
- InfoQArticles and talks on data platforms.Opens in a new tab