1.snappy編譯安裝
下載snappy,編譯。
前置條件:gcc c++, autoconf, automake, libtool, Java 6, JAVA_HOME set, Maven 3
1)運行:autogen.sh
2)運行:
./configure
make
make check
make install
3)驗證:查看/usr/local/lib下面是否有libsnappy.*
4)拷貝到其他集群:scp -r ./google-snappy-2d99bd1/ bigdata@xxx:/ssd/bigdata/install/
2. Hadoop Snappy 編譯安裝
1、下載
2、The built tarball is at target/hadoop-snappy-0.0.1-SNAPSHOT.tar.gz. The tarball includes snappy native library。
解壓拷貝:
cp -r * $HADOOP_HOME/lib
scp -r * bigdata@xxx:/ssd/bigdata/software/hadoop/lib
3、配置環境變量:
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HADOOP_HOME/lib/native/Linux-amd64-64/:/usr/local/lib/
export
HBASE_LIBRARY_PATH=$HBASE_LIBRARY_PATH:$HBASE_HOME/lib/native/Linux-amd64-64/:/usr/local/lib/
2°、修改$HADOOP_HOME/etc/hadoop/core-site.xml:
<property>
<name>io.compression.codecs</name>
<value>
org.apache.hadoop.io.compress.GzipCodec,
org.apache.hadoop.io.compress.DefaultCodec,
org.apache.hadoop.io.compress.BZip2Codec,
org.apache.hadoop.io.compress.SnappyCodec
</value>
</property>
3°、修改$HADOOP_HOME/etc/hadoop/mapred-site.xml中有關壓縮屬性,測試snappy:
<property>
<name>mapreduce.map.output.compress</name>
<value>true</value>
</property>
<property>
<name>mapreduce.map.output.compress.codec</name>
<value>org.apache.hadoop.io.compress.SnappyCodec</value>
</property>
4°、驗證
hadoop checknative -a
5、最終編譯了hadoop2.7.3的源碼:
https://www.58jb.com/html/113.html 來支持了snappy。
編譯進行一半,只要把snappy編譯出來即可。
參考:
http://www.cnblogs.com/zlzhoulei/p/5735698.html
http://blog.csdn.net/bbaiggey/article/details/53583923
http://www.micmiu.com/bigdata/hadoop/hadoop-snappy-install-config/