Hadoop 2.6.0的Web UI無法看到應用執行的歷史信息

環境

  • Hadoop 2.6.0
  • Java 1.8.0_151
  • Ubuntu 16.04

解決方案

開始在配置Hadoop時,由于是按照官網配置的非常簡潔版的偽分布式版本,缺少了某些選項,才會造成這個問題。

開始時,我的mapred-site.xml的配置如下:

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
  Licensed under the Apache License, Version 2.0 (the "License");
  you may not use this file except in compliance with the License.
  You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License. See accompanying LICENSE file.
-->

<!-- Put site-specific property overrides in this file. -->

<configuration>
        <property>
                <name>mapreduce.framework.name</name>
                <value>yarn</value>
        </property>
</configuration>

這個就是跟著官網上的tutorial一步步走到最后的結果。

由于我們想要查看歷史信息,那么肯定就需要有一個地方來存儲這些歷史信息啊,在之前的Hadoop版本中,使用的是JobHistoryServer,但是如果使用的mapreduce調度框架是YARN的話,就不會使用這一個了,而是ApplicationHistoryServer

所以,我們還需要額外配置一下。

將上面的配置文件修改成如這樣:

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
  Licensed under the Apache License, Version 2.0 (the "License");
  you may not use this file except in compliance with the License.
  You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License. See accompanying LICENSE file.
-->

<!-- Put site-specific property overrides in this file. -->

<configuration>
        <property>
                <name>mapreduce.framework.name</name>
                <value>yarn</value>
        </property>
        <property>
                <name>mapreduce.jobhistory.address</name>
                <value>localhost:10020</value>
        </property>
        <property>
                <name>mapreduce.jobhistory.webapp.address</name>
                <value>localhost:19888</value>
        </property>
</configuration>

然后,還需要啟動它,通過下面的命令:
sbin/yarn-daemon.sh start historyserver

好啦,這樣就大功告成啦。

?著作權歸作者所有,轉載或內容合作請聯系作者
平臺聲明:文章內容(如有圖片或視頻亦包括在內)由作者上傳并發布,文章內容僅代表作者本人觀點,簡書系信息發布平臺,僅提供信息存儲服務。

推薦閱讀更多精彩內容