win7搭建为伪分布式hadoop的步骤

这篇文章主要介绍“win7搭建为伪分布式hadoop的步骤”，在日常操作中，相信很多人在win7搭建为伪分布式hadoop的步骤问题上存在疑惑，小编查阅了各式资料，整理出简单好用的操作方法，希望对大家解答”win7搭建为伪分布式hadoop的步骤”的疑惑有所帮助！接下来，请跟着小编一起来学习吧！

创新互联公司是一家专注于成都网站设计、网站建设与策划设计,温县网站建设哪家好?创新互联公司做网站,专注于网站建设十载,网设计领域的专业建站公司;建站业务涵盖:温县等地区。温县做网站价格咨询:18982081108

window7下安装hadoop [32位]
1.下载hadoop
hadoop-2.2.0.tar.gz
解压
2.配置环境变量hadoop
3.修改%HADOOP_HOME%\etc\hadoop的hadoop-env.sh
export JAVA_HOME=D:\Java\jdk1.6.0_10#不能有空格
末尾添加[会出错就不添加]
set HADOOP_PREFIX=E:\hadoop\hadoop-2.2.0\
set HADOOP_CONF_DIR=%HADOOP_PREFIX%\etc\hadoop
set YARN_CONF_DIR=%HADOOP_CONF_DIR%
set PATH=%PATH%;%HADOOP_PREFIX%\bin

下载hadoop-common-2.2.0-bin-master

把里面的文件拷贝到hadoop/bin下

4.配置core-site.xml
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

<configuration>
      <property>
        <name>fs.default.name</name>
        <value>hdfs://0.0.0.0:19000</value>
      </property>
</configuration>

5.配置hdfs-site.xml

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>


    <configuration>
      <property>
        <name>dfs.replication</name>
        <value>1</value>
      </property>
    </configuration>


6.配置mapred-site.xml

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>



    <configuration>

       <property>
         <name>mapreduce.job.user.name</name>
         <value>%USERNAME%</value>
       </property>

       <property>
         <name>mapreduce.framework.name</name>
         <value>yarn</value>
       </property>

      <property>
        <name>yarn.apps.stagingDir</name>
        <value>/user/%USERNAME%/staging</value>
      </property>

      <property>
        <name>mapreduce.jobtracker.address</name>
        <value>local</value>
      </property>

    </configuration>

7.创建yarn-site.xml文件
<?xml version="1.0"?>


    <configuration>
      <property>
        <name>yarn.server.resourcemanager.address</name>
        <value>0.0.0.0:8020</value>
      </property>

      <property>
        <name>yarn.server.resourcemanager.application.expiry.interval</name>
        <value>60000</value>
      </property>

      <property>
        <name>yarn.server.nodemanager.address</name>
        <value>0.0.0.0:45454</value>
      </property>

      <property>
        <name>yarn.nodemanager.aux-services</name>
        <value>mapreduce_shuffle</value>
      </property>

      <property>
        <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
        <value>org.apache.hadoop.mapred.ShuffleHandler</value>
      </property>

      <property>
        <name>yarn.server.nodemanager.remote-app-log-dir</name>
        <value>/app-logs</value>
      </property>

      <property>
        <name>yarn.nodemanager.log-dirs</name>
        <value>/dep/logs/userlogs</value>
      </property>

      <property>
        <name>yarn.server.mapreduce-appmanager.attempt-listener.bindAddress</name>
        <value>0.0.0.0</value>
      </property>

      <property>
        <name>yarn.server.mapreduce-appmanager.client-service.bindAddress</name>
        <value>0.0.0.0</value>
      </property>

      <property>
        <name>yarn.log-aggregation-enable</name>
        <value>true</value>
      </property>

      <property>
        <name>yarn.log-aggregation.retain-seconds</name>
        <value>-1</value>
      </property>

      <property>
        <name>yarn.application.classpath</name>
        <value>%HADOOP_CONF_DIR%,%HADOOP_COMMON_HOME%/share/hadoop/common/*,%HADOOP_COMMON_HOME%/share/hadoop/common/lib/*,%HADOOP_HDFS_HOME%/share/hadoop/hdfs/*,%HADOOP_HDFS_HOME%/share/hadoop/hdfs/lib/*,%HADOOP_MAPRED_HOME%/share/hadoop/mapreduce/*,%HADOOP_MAPRED_HOME%/share/hadoop/mapreduce/lib/*,%HADOOP_YARN_HOME%/share/hadoop/yarn/*,%HADOOP_YARN_HOME%/share/hadoop/yarn/lib/*</value>
      </property>
    </configuration>

    切换到E:\hadoop\hadoop-2.2.0\etc\hadoop运行hadoop-env.cmd脚本，设置当前命令窗口执行环境变量
   执行 hdfs namenode -format
   16/02/27 11:36:23 INFO util.GSet: VM type       = 32-bit
16/02/27 11:36:23 INFO util.GSet: 0.029999999329447746% max memory = 992.3 MB
16/02/27 11:36:23 INFO util.GSet: capacity      = 2^16 = 65536 entries
16/02/27 11:36:24 INFO common.Storage: Storage directory \tmp\hadoop-goudcheng\d
fs\name has been successfully formatted.
16/02/27 11:36:24 INFO namenode.FSImage: Saving image file \tmp\hadoop-goudcheng
\dfs\name\current\fsimage.ckpt_0000000000000000000 using no compression
16/02/27 11:36:24 INFO namenode.FSImage: Image file \tmp\hadoop-goudcheng\dfs\na
me\current\fsimage.ckpt_0000000000000000000 of size 201 bytes saved in 0 seconds
.
16/02/27 11:36:24 INFO namenode.NNStorageRetentionManager: Going to retain 1 ima
ges with txid >= 0
16/02/27 11:36:24 INFO util.ExitUtil: Exiting with status 0
16/02/27 11:36:24 INFO namenode.NameNode: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at CDCH20100020-5/172.31.168.244

到此，关于“win7搭建为伪分布式hadoop的步骤”的学习就结束了，希望能够解决大家的疑惑。理论与实践的搭配能更好的帮助大家学习，快去试试吧！若想继续学习更多相关知识，请继续关注创新互联网站，小编会继续努力为大家带来更多实用的文章！

文章名称：win7搭建为伪分布式hadoop的步骤
转载注明：https://www.cdcxhl.com/article4/iiohoe.html

成都网站建设公司_创新互联，为您提供建站公司、用户体验、网站设计、App开发、微信公众号、网站排名

声明：本网站发布的内容（图片、视频和文字）以用户投稿、用户转载内容为主，如果涉及侵权请尽快告知，我们将会在第一时间删除。文章观点不代表本网站立场，如需处理请联系客服。电话：028-86922220；邮箱：631063699@qq.com。内容未经允许不得转载，或转载时需注明来源：创新互联

猜你还喜欢下面的内容