Cloudera Manager 5 和 CDH5离线安装参考
0. ?主机规划 Cloudera Manager需要一台单独的主机 debugo01 Cloudera Manager, yum repo debugo02 NameNode, DataNode, Yarn RM, Spark Master, Spark Worker debugo03 DataNode, Spark Worker, Secondery NameNode 1. 准备工作(需要在所有节点操作) a). 网
0. ?主机规划
Cloudera Manager需要一台单独的主机
debugo01 Cloudera Manager, yum repo
debugo02 NameNode, DataNode, Yarn RM, Spark Master, Spark Worker …
debugo03 DataNode, Spark Worker, Secondery NameNode …
1. 准备工作(需要在所有节点操作)
a). 网络配置、 /etc/hosts或DNS 、hostname配置(略)
b). 时区和ntp
cp /usr/share/zoneinfo/Asia/Shanghai /etc/localtime chkconfig ntpd on
c). 关闭selinux和iptables
chkconfig iptables off chkconfig ip6tables off sed -i 's/^SELINUX.*/SELINUX=disabled/' /etc/selinux/config
d). 配置ISO源
/etc/yum.repos.d/CentOS-ISO.repo [base] name=CentOS-$releasever - ISO baseurl=http://debugo01/CentOS_6.5_Final enabled=1 gpgcheck=1 gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6 # 在debugo01上挂载镜像文件到http目录下 yum install -y httpd mount -o loop CentOS*.iso /var/www/html
e). 安装包
所有主机都需要安装mysql-connector、jdk和scala(需要去Oracle和Scala官方下载1.7最新版)
yum install -y mysql-connector-java rpm -qa | grep java #(需要卸载openjdk-1.6) rpm -ivh jdk-1.7.0_67-fcs.x86_64.rpm rpm -ivh scala-2.10.4-400.noarch.rpm
f). 重启主机
2. CM Server配置
离线安装有两种方式,官方推荐使用parcel来进行离线安装。但是一直尝试失败,下面使用yum repo进行离线安装。
a). 安装软件包
yum install mysql-server mysql createrepo chkconfig mysqld on
b). 下载Cloudera Manager
wget http://archive-primary.cloudera.com/cm5/cm/5/cloudera-manager-el6-cm5.1.1_x86_64.tar.gz tar xvzf cloudera-manager-el6-cm5.1.1_x86_64.tar.gz -C /opt
c). 创建MySQL用户
mysql -uroot -e "CREATE DATABASE cm DEFAULT CHARSET utf8 COLLATE utf8_general_ci" mysql -uroot -e "CREATE DATABASE cdh DEFAULT CHARSET utf8 COLLATE utf8_general_ci" mysql -uroot -e 'GRANT all on cm.* TO cm@"%" IDENTIFIED BY "cm"' mysql -uroot -e 'GRANT all on cdh.* TO cdh@"%" IDENTIFIED BY "cdh"' mysqladmin flush-privileges
d).同步yum repo
最新的repo请在http://archive.cloudera.com/ 中下载
vim /etc/yum.repo.d/cloudera-cdh5.repo [cloudera-cdh5] # Packages for Cloudera's Distribution for Hadoop, Version 5, on RedHat or CentO S 6 x86_64 name=Cloudera's Distribution for Hadoop, Version 5 baseurl=http://archive.cloudera.com/cdh5/redhat/6/x86_64/cdh/5/ gpgkey = http://archive.cloudera.com/cdh5/redhat/6/x86_64/cdh/RPM-GPG-KEY-cloude ra gpgcheck = 1 [cloudera-manager] name = Cloudera Manager, Version 5.1.1 baseurl = http://archive.cloudera.com/cm5/redhat/6/x86_64/cm/5.1.1/ gpgkey = http://archive.cloudera.com/redhat/cdh/RPM-GPG-KEY-cloudera gpgcheck = 1 enabled=1 cd /var/www/html mkdir cdh5 && cd cdh5 reposync -r cloudera-cdh5 createrepo . ln -s /var/www/html/cdh5/cloudera-cdh5/RPMS /var/www/html/cdh5/RPMS # 漫长得等待... mkdir ../cm && cd cm reposync -r cloudera-manager createrepo .
在debugo02,debugo03上测试http访问,并配置下面的repo信息。
[cloudera-cdh5]
# Packages for Cloudera’s Distribution for Hadoop, Version 5, on RedHat or CentOS 6 x86_64
name=Cloudera’s Distribution for Hadoop, Version 5
baseurl=http://debugo01/cdh5
gpgcheck = 0
enabled = 1
[cloudera-manager]
name = Cloudera Manager, Version 5.1.1
baseurl = http://debugo01/cm
gpgcheck = 0
enabled=1
e). 配置cm
sed -i 's/^server_host.*/server_host=debugo01/' /opt/cm-5.1.1/etc/cloudera-scm-agent/config.ini /opt/cm-5.1.1/share/cmf/schema/scm_prepare_database.sh mysql cm -hlocalhost -uroot -p --scm-host localhost scm scm scm Enter database password: Verifying that we can write to /opt/cm-5.1.1/etc/cloudera-scm-server Creating SCM configuration file in /opt/cm-5.1.1/etc/cloudera-scm-server Executing: /usr/java/latest/bin/java -cp /usr/share/java/mysql-connector-java.jar:/usr/share/java/oracle-connector-java.jar:/opt/cm-5.1.1/share/cmf/schema/../lib/* com.cloudera.enterprise.dbutil.DbCommandExecutor /opt/cm-5.1.1/etc/cloudera-scm-server/db.properties com.cloudera.cmf.db. [main] DbCommandExecutor INFO Successfully connected to database. All done, your SCM database is configured correctly!
f). 启动cm
/opt/cm-5.1.1/etc/init.d/cloudera-scm-agent start /opt/cm-5.1.1/etc/init.d/cloudera-scm-server start
3. 安装CDH
a). 通过浏览器访问debugo01,使用admin/admin登录
b). 选择Cloudera Express来创建cluster。express是cloudera的免费版,相比企业版只是少了升级和一些高级特性(http://www.cloudera.com/content/cloudera-content/cloudera-docs/CM5/latest/Cloudera-Manager-Installation-Guide/cm5ig_feature_differences.html?scroll=cmfeig_topic_5_1)
c). 接下来Specify hosts for your CDH cluster installation, 添加新主机debugo02,并将Currently Managed Hosts中的cm-server主机debugo01也勾选上。
d). Select Repository – Use Parcel, CDH – CDH-5.1.0-1.cdh5.1.0.p0.53
e). Provide SSH login credentials. 使用root并输入root密码
f). Cluster Installation
这里会遇到一个DNS反向解析错误,不能正确解析Cloudera Manager Server主机名的问题,解决方法将host工具改名,安装成功后再修改回来。
mv /usr/bin/host /usr/bin/host.bak
为了加快安装,可以在安装开始后手工删除debugo02、debugo03中/etc/yum.repos.d/新生成的repo文件,让cluster的直接使用本地源。
g). 安装后进入集群配置界面,输入前面创建的cdh数据库。根据需求配置角色。完成查看host和集群状态正常,安装完成。很多细节的配置警告可以根据提示来调整。
^^
原文地址:Cloudera Manager 5 和 CDH5离线安装参考, 感谢原作者分享。

Hot AI Tools

Undresser.AI Undress
AI-powered app for creating realistic nude photos

AI Clothes Remover
Online AI tool for removing clothes from photos.

Undress AI Tool
Undress images for free

Clothoff.io
AI clothes remover

Video Face Swap
Swap faces in any video effortlessly with our completely free AI face swap tool!

Hot Article

Hot Tools

Notepad++7.3.1
Easy-to-use and free code editor

SublimeText3 Chinese version
Chinese version, very easy to use

Zend Studio 13.0.1
Powerful PHP integrated development environment

Dreamweaver CS6
Visual web development tools

SublimeText3 Mac version
God-level code editing software (SublimeText3)

Hot Topics

Solution to the problem that Win11 system cannot install Chinese language pack With the launch of Windows 11 system, many users began to upgrade their operating system to experience new functions and interfaces. However, some users found that they were unable to install the Chinese language pack after upgrading, which troubled their experience. In this article, we will discuss the reasons why Win11 system cannot install the Chinese language pack and provide some solutions to help users solve this problem. Cause Analysis First, let us analyze the inability of Win11 system to

You may not be able to install guest additions to a virtual machine in OracleVirtualBox. When we click on Devices>InstallGuestAdditionsCDImage, it just throws an error as shown below: VirtualBox - Error: Unable to insert virtual disc C: Programming FilesOracleVirtualBoxVBoxGuestAdditions.iso into ubuntu machine In this post we will understand what happens when you What to do when you can't install guest additions in VirtualBox. Unable to install guest additions in VirtualBox If you can't install it in Virtua

If you have successfully downloaded the installation file of Baidu Netdisk, but cannot install it normally, it may be that there is an error in the integrity of the software file or there is a problem with the residual files and registry entries. Let this site take care of it for users. Let’s introduce the analysis of the problem that Baidu Netdisk is successfully downloaded but cannot be installed. Analysis of the problem that Baidu Netdisk downloaded successfully but could not be installed 1. Check the integrity of the installation file: Make sure that the downloaded installation file is complete and not damaged. You can download it again, or try to download the installation file from another trusted source. 2. Turn off anti-virus software and firewall: Some anti-virus software or firewall programs may prevent the installation program from running properly. Try disabling or exiting the anti-virus software and firewall, then re-run the installation

Installing Android applications on Linux has always been a concern for many users. Especially for Linux users who like to use Android applications, it is very important to master how to install Android applications on Linux systems. Although running Android applications directly on Linux is not as simple as on the Android platform, by using emulators or third-party tools, we can still happily enjoy Android applications on Linux. The following will introduce how to install Android applications on Linux systems.

If you have used Docker, you must understand daemons, containers, and their functions. A daemon is a service that runs in the background when a container is already in use in any system. Podman is a free management tool for managing and creating containers without relying on any daemon such as Docker. Therefore, it has advantages in managing containers without the need for long-term backend services. Additionally, Podman does not require root-level permissions to be used. This guide discusses in detail how to install Podman on Ubuntu24. To update the system, we first need to update the system and open the Terminal shell of Ubuntu24. During both installation and upgrade processes, we need to use the command line. a simple

Many novice friends still don’t know how to install creo, so the editor below brings relevant tutorials on creo installation. Friends in need should take a look at it. I hope it can help you. 1. Open the downloaded installation package and find the License folder, as shown in the figure below: 2. Then copy it to the directory on the C drive, as shown in the figure below: 3. Double-click to enter and see if there is a license file, as shown below As shown in the picture: 4. Then copy the license file to this file, as shown in the following picture: 5. In the PROGRAMFILES file of the C drive, create a new PLC folder, as shown in the following picture: 6. Copy the license file as well Click in, as shown in the figure below: 7. Double-click the installation file of the main program. To install, check the box to install new software.

While studying in high school, some students take very clear and accurate notes, taking more notes than others in the same class. For some, note-taking is a hobby, while for others, it is a necessity when they easily forget small information about anything important. Microsoft's NTFS application is particularly useful for students who wish to save important notes beyond regular lectures. In this article, we will describe the installation of Ubuntu applications on Ubuntu24. Updating the Ubuntu System Before installing the Ubuntu installer, on Ubuntu24 we need to ensure that the newly configured system has been updated. We can use the most famous "a" in Ubuntu system

Detailed steps to install Go language on Win7 computer Go (also known as Golang) is an open source programming language developed by Google. It is simple, efficient and has excellent concurrency performance. It is suitable for the development of cloud services, network applications and back-end systems. . Installing the Go language on a Win7 computer allows you to quickly get started with the language and start writing Go programs. The following will introduce in detail the steps to install the Go language on a Win7 computer, and attach specific code examples. Step 1: Download the Go language installation package and visit the Go official website
