Table of Contents
0. ?主机规划
1. 准备工作(需要在所有节点操作)
2. CM Server配置
3. 安装CDH
Home Database Mysql Tutorial Cloudera Manager 5 和 CDH5离线安装参考

Cloudera Manager 5 和 CDH5离线安装参考

Jun 07, 2016 pm 04:38 PM
cloudera manager Install Offline

0. ?主机规划 Cloudera Manager需要一台单独的主机 debugo01 Cloudera Manager, yum repo debugo02 NameNode, DataNode, Yarn RM, Spark Master, Spark Worker debugo03 DataNode, Spark Worker, Secondery NameNode 1. 准备工作(需要在所有节点操作) a). 网

0. ?主机规划

Cloudera Manager需要一台单独的主机
debugo01 Cloudera Manager, yum repo
debugo02 NameNode, DataNode, Yarn RM, Spark Master, Spark Worker …
debugo03 DataNode, Spark Worker, Secondery NameNode …

1. 准备工作(需要在所有节点操作)

a). 网络配置、 /etc/hosts或DNS 、hostname配置(略)
b). 时区和ntp

cp /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
chkconfig ntpd on
Copy after login

c). 关闭selinux和iptables

chkconfig iptables off
chkconfig ip6tables off
sed -i 's/^SELINUX.*/SELINUX=disabled/' /etc/selinux/config
Copy after login

d). 配置ISO源

/etc/yum.repos.d/CentOS-ISO.repo
[base]
name=CentOS-$releasever - ISO
baseurl=http://debugo01/CentOS_6.5_Final
enabled=1
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6
# 在debugo01上挂载镜像文件到http目录下
yum install -y httpd
mount -o loop CentOS*.iso /var/www/html
Copy after login

e). 安装包
所有主机都需要安装mysql-connector、jdk和scala(需要去Oracle和Scala官方下载1.7最新版)

yum install -y mysql-connector-java
rpm -qa | grep java    #(需要卸载openjdk-1.6)
rpm -ivh jdk-1.7.0_67-fcs.x86_64.rpm
rpm -ivh scala-2.10.4-400.noarch.rpm
Copy after login

f). 重启主机

2. CM Server配置

离线安装有两种方式,官方推荐使用parcel来进行离线安装。但是一直尝试失败,下面使用yum repo进行离线安装。
a). 安装软件包

yum install mysql-server mysql createrepo
chkconfig mysqld on
Copy after login

b). 下载Cloudera Manager

wget http://archive-primary.cloudera.com/cm5/cm/5/cloudera-manager-el6-cm5.1.1_x86_64.tar.gz
tar xvzf cloudera-manager-el6-cm5.1.1_x86_64.tar.gz -C /opt
Copy after login

c). 创建MySQL用户

mysql -uroot -e "CREATE DATABASE cm DEFAULT CHARSET utf8 COLLATE utf8_general_ci"
mysql -uroot -e "CREATE DATABASE cdh DEFAULT CHARSET utf8 COLLATE utf8_general_ci"
mysql -uroot -e 'GRANT all on cm.* TO cm@"%" IDENTIFIED BY "cm"'
mysql -uroot -e 'GRANT all on cdh.* TO cdh@"%" IDENTIFIED BY "cdh"'
mysqladmin flush-privileges
Copy after login

d).同步yum repo
最新的repo请在http://archive.cloudera.com/ 中下载

vim /etc/yum.repo.d/cloudera-cdh5.repo 
[cloudera-cdh5]
# Packages for Cloudera's Distribution for Hadoop, Version 5, on RedHat	or CentO
S 6 x86_64
name=Cloudera's Distribution for Hadoop, Version 5
baseurl=http://archive.cloudera.com/cdh5/redhat/6/x86_64/cdh/5/
gpgkey = http://archive.cloudera.com/cdh5/redhat/6/x86_64/cdh/RPM-GPG-KEY-cloude
ra    
gpgcheck = 1
[cloudera-manager]
name = Cloudera Manager, Version 5.1.1
baseurl = http://archive.cloudera.com/cm5/redhat/6/x86_64/cm/5.1.1/
gpgkey = http://archive.cloudera.com/redhat/cdh/RPM-GPG-KEY-cloudera
gpgcheck = 1
enabled=1
cd /var/www/html
mkdir cdh5 && cd cdh5
reposync -r cloudera-cdh5
createrepo .
ln -s /var/www/html/cdh5/cloudera-cdh5/RPMS /var/www/html/cdh5/RPMS
# 漫长得等待...
mkdir ../cm && cd cm
reposync -r cloudera-manager
createrepo .
Copy after login

在debugo02,debugo03上测试http访问,并配置下面的repo信息。
[cloudera-cdh5]
# Packages for Cloudera’s Distribution for Hadoop, Version 5, on RedHat or CentOS 6 x86_64
name=Cloudera’s Distribution for Hadoop, Version 5
baseurl=http://debugo01/cdh5
gpgcheck = 0
enabled = 1

[cloudera-manager]
name = Cloudera Manager, Version 5.1.1
baseurl = http://debugo01/cm
gpgcheck = 0
enabled=1
e). 配置cm

sed -i 's/^server_host.*/server_host=debugo01/' /opt/cm-5.1.1/etc/cloudera-scm-agent/config.ini
/opt/cm-5.1.1/share/cmf/schema/scm_prepare_database.sh mysql cm -hlocalhost -uroot -p --scm-host localhost scm scm scm
Enter database password: 
Verifying that we can write to /opt/cm-5.1.1/etc/cloudera-scm-server
Creating SCM configuration file in /opt/cm-5.1.1/etc/cloudera-scm-server
Executing:  /usr/java/latest/bin/java -cp /usr/share/java/mysql-connector-java.jar:/usr/share/java/oracle-connector-java.jar:/opt/cm-5.1.1/share/cmf/schema/../lib/* com.cloudera.enterprise.dbutil.DbCommandExecutor /opt/cm-5.1.1/etc/cloudera-scm-server/db.properties com.cloudera.cmf.db.
[main] DbCommandExecutor              INFO  Successfully connected to database.
All done, your SCM database is configured correctly!
Copy after login

f). 启动cm

/opt/cm-5.1.1/etc/init.d/cloudera-scm-agent start
/opt/cm-5.1.1/etc/init.d/cloudera-scm-server start
Copy after login

3. 安装CDH

a). 通过浏览器访问debugo01,使用admin/admin登录
b). 选择Cloudera Express来创建cluster。express是cloudera的免费版,相比企业版只是少了升级和一些高级特性(http://www.cloudera.com/content/cloudera-content/cloudera-docs/CM5/latest/Cloudera-Manager-Installation-Guide/cm5ig_feature_differences.html?scroll=cmfeig_topic_5_1)
c). 接下来Specify hosts for your CDH cluster installation, 添加新主机debugo02,并将Currently Managed Hosts中的cm-server主机debugo01也勾选上。
d). Select Repository – Use Parcel, CDH – CDH-5.1.0-1.cdh5.1.0.p0.53
e). Provide SSH login credentials. 使用root并输入root密码
f). Cluster Installation
这里会遇到一个DNS反向解析错误,不能正确解析Cloudera Manager Server主机名的问题,解决方法将host工具改名,安装成功后再修改回来。
mv /usr/bin/host /usr/bin/host.bak
为了加快安装,可以在安装开始后手工删除debugo02、debugo03中/etc/yum.repos.d/新生成的repo文件,让cluster的直接使用本地源。
g). 安装后进入集群配置界面,输入前面创建的cdh数据库。根据需求配置角色。完成查看host和集群状态正常,安装完成。很多细节的配置警告可以根据提示来调整。
1

^^

Statement of this Website
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn

Hot AI Tools

Undresser.AI Undress

Undresser.AI Undress

AI-powered app for creating realistic nude photos

AI Clothes Remover

AI Clothes Remover

Online AI tool for removing clothes from photos.

Undress AI Tool

Undress AI Tool

Undress images for free

Clothoff.io

Clothoff.io

AI clothes remover

Video Face Swap

Video Face Swap

Swap faces in any video effortlessly with our completely free AI face swap tool!

Hot Tools

Notepad++7.3.1

Notepad++7.3.1

Easy-to-use and free code editor

SublimeText3 Chinese version

SublimeText3 Chinese version

Chinese version, very easy to use

Zend Studio 13.0.1

Zend Studio 13.0.1

Powerful PHP integrated development environment

Dreamweaver CS6

Dreamweaver CS6

Visual web development tools

SublimeText3 Mac version

SublimeText3 Mac version

God-level code editing software (SublimeText3)

Solution to the problem that Win11 system cannot install Chinese language pack Solution to the problem that Win11 system cannot install Chinese language pack Mar 09, 2024 am 09:48 AM

Solution to the problem that Win11 system cannot install Chinese language pack With the launch of Windows 11 system, many users began to upgrade their operating system to experience new functions and interfaces. However, some users found that they were unable to install the Chinese language pack after upgrading, which troubled their experience. In this article, we will discuss the reasons why Win11 system cannot install the Chinese language pack and provide some solutions to help users solve this problem. Cause Analysis First, let us analyze the inability of Win11 system to

Unable to install guest additions in VirtualBox Unable to install guest additions in VirtualBox Mar 10, 2024 am 09:34 AM

You may not be able to install guest additions to a virtual machine in OracleVirtualBox. When we click on Devices>InstallGuestAdditionsCDImage, it just throws an error as shown below: VirtualBox - Error: Unable to insert virtual disc C: Programming FilesOracleVirtualBoxVBoxGuestAdditions.iso into ubuntu machine In this post we will understand what happens when you What to do when you can't install guest additions in VirtualBox. Unable to install guest additions in VirtualBox If you can't install it in Virtua

What should I do if Baidu Netdisk is downloaded successfully but cannot be installed? What should I do if Baidu Netdisk is downloaded successfully but cannot be installed? Mar 13, 2024 pm 10:22 PM

If you have successfully downloaded the installation file of Baidu Netdisk, but cannot install it normally, it may be that there is an error in the integrity of the software file or there is a problem with the residual files and registry entries. Let this site take care of it for users. Let’s introduce the analysis of the problem that Baidu Netdisk is successfully downloaded but cannot be installed. Analysis of the problem that Baidu Netdisk downloaded successfully but could not be installed 1. Check the integrity of the installation file: Make sure that the downloaded installation file is complete and not damaged. You can download it again, or try to download the installation file from another trusted source. 2. Turn off anti-virus software and firewall: Some anti-virus software or firewall programs may prevent the installation program from running properly. Try disabling or exiting the anti-virus software and firewall, then re-run the installation

How to install Android apps on Linux? How to install Android apps on Linux? Mar 19, 2024 am 11:15 AM

Installing Android applications on Linux has always been a concern for many users. Especially for Linux users who like to use Android applications, it is very important to master how to install Android applications on Linux systems. Although running Android applications directly on Linux is not as simple as on the Android platform, by using emulators or third-party tools, we can still happily enjoy Android applications on Linux. The following will introduce how to install Android applications on Linux systems.

How to install Podman on Ubuntu 24.04 How to install Podman on Ubuntu 24.04 Mar 22, 2024 am 11:26 AM

If you have used Docker, you must understand daemons, containers, and their functions. A daemon is a service that runs in the background when a container is already in use in any system. Podman is a free management tool for managing and creating containers without relying on any daemon such as Docker. Therefore, it has advantages in managing containers without the need for long-term backend services. Additionally, Podman does not require root-level permissions to be used. This guide discusses in detail how to install Podman on Ubuntu24. To update the system, we first need to update the system and open the Terminal shell of Ubuntu24. During both installation and upgrade processes, we need to use the command line. a simple

How to install creo-creo installation tutorial How to install creo-creo installation tutorial Mar 04, 2024 pm 10:30 PM

Many novice friends still don’t know how to install creo, so the editor below brings relevant tutorials on creo installation. Friends in need should take a look at it. I hope it can help you. 1. Open the downloaded installation package and find the License folder, as shown in the figure below: 2. Then copy it to the directory on the C drive, as shown in the figure below: 3. Double-click to enter and see if there is a license file, as shown below As shown in the picture: 4. Then copy the license file to this file, as shown in the following picture: 5. In the PROGRAMFILES file of the C drive, create a new PLC folder, as shown in the following picture: 6. Copy the license file as well Click in, as shown in the figure below: 7. Double-click the installation file of the main program. To install, check the box to install new software.

How to Install and Run the Ubuntu Notes App on Ubuntu 24.04 How to Install and Run the Ubuntu Notes App on Ubuntu 24.04 Mar 22, 2024 pm 04:40 PM

While studying in high school, some students take very clear and accurate notes, taking more notes than others in the same class. For some, note-taking is a hobby, while for others, it is a necessity when they easily forget small information about anything important. Microsoft's NTFS application is particularly useful for students who wish to save important notes beyond regular lectures. In this article, we will describe the installation of Ubuntu applications on Ubuntu24. Updating the Ubuntu System Before installing the Ubuntu installer, on Ubuntu24 we need to ensure that the newly configured system has been updated. We can use the most famous "a" in Ubuntu system

Detailed steps to install Go language on Win7 computer Detailed steps to install Go language on Win7 computer Mar 27, 2024 pm 02:00 PM

Detailed steps to install Go language on Win7 computer Go (also known as Golang) is an open source programming language developed by Google. It is simple, efficient and has excellent concurrency performance. It is suitable for the development of cloud services, network applications and back-end systems. . Installing the Go language on a Win7 computer allows you to quickly get started with the language and start writing Go programs. The following will introduce in detail the steps to install the Go language on a Win7 computer, and attach specific code examples. Step 1: Download the Go language installation package and visit the Go official website

See all articles