printing out configurations of Hadoop, HBase, Accumulo clusters

Hadoop has hundreds of configurable parameters. Â And Hadoop admins and developers spend a lot of time tweaking these settings.

I wanted a quick way to dump all the configuration settings of a Hadoop cluster. So I adopted an example from Tom White’s Hadoop book. Once I got the Hadoop Configuration working,  I added a utility to dump HBase cluster configuration. Why stop there; Also added a utility for dumping Accumulo configuration 🙂 The project is OPEN SOURCE and hosted in GitHub : https://github.com/sujee/print-hadoop-confi

To get started: 1) clone the project git clone https://github.com/sujee/print-hadoop-config.git

2) compile using maven mvn package

3) edit  env.sh to reflect your settings

4) to see hadoop conf, use ./hadoop-conf.sh that is it… Just a little trivia. Testing this with Hadoop v1.2.1 results in 257 parameters! ============

Feedback from HBase mailing list post:

Dump Hadoop config by $ hadoop org.apache.hadoop.conf.Configuration

And dump HBase config by $ hbase org.apache.hadoop.hbase.HBaseConfiguration

(how ever the output isn’t very readable 🙂 )

Sujee Maniyam
Written by:

Sujee Maniyam

Sujee Maniyam is the co-founder of Elephantscale. He teaches and works on Big Data, AI and Cloud technologies. Sujee is a published author and a frequent speaker

Leave a Reply

Your email address will not be published. Required fields are marked *