hdfs调优参数

1、调整默认block大小

dfs.blocksize30m

2、内存调优

vim hadoop-env.shexport HADOOP_NAMENODE_OPTS="-Xmx512m -Xms256m -Dhadoop.security.logger=${HADOOP_SECURITY_LOGGER:-INFO,RFAS} -Dhdfs.audit.logger=${HDFS_AUDIT_LOGGER:-INFO,NullAppender} $HADOOP_NAMENODE_OPTS"export HADOOP_DATANODE_OPTS="-Xmx512m -Xms256m -Dhadoop.security.logger=ERROR,RFAS $HADOOP_DATANODE_OPTS"export HADOOP_SECONDARYNAMENODE_OPTS="-Xmx512m -Xms256m -Dhadoop.security.logger=${HADOOP_SECURITY_LOGGER:-INFO,RFAS} -Dhdfs.audit.logger=${HDFS_AUDIT_LOGGER:-INFO,NullAppender} $HADOOP_SECONDARYNAMENODE_OPTS"export HADOOP_NFS3_OPTS="$HADOOP_NFS3_OPTS"export HADOOP_PORTMAP_OPTS="-Xmx512m $HADOOP_PORTMAP_OPTS"export HADOOP_CLIENT_OPTS="-Xmx512m $HADOOP_CLIENT_OPTS"

修改hdfs启动内存

cd /opt/hadoop/hadoop-2.6.5/libexecvim hadoop-config.shJAVA_HEAP_MAX=-Xmx512m

修改yarn启动内存

cd /opt/hadoop/hadoop-2.6.5/etc/hadoopvim yarn-env.shJAVA_HEAP_MAX=-Xmx512mYARN_HEAPSIZE=512

3、扩充hdfs磁盘

（1）dfs.data.dir

dfs.data.dir /data/hadoop/hdfs,/test/hdfs

（2）在修改配置后重启hadoop集群即生效