1、Oozie的安装
(1)解压oozie安装包
$ cd /opt/software/cdh/
cdh]$ tar -zxf oozie-4.0.0-cdh5.3.6.tar.gz -C /opt/cdh-5.3.6/
(2)在/opt/cdh-5.3.6/hadoop-2.5.0-cdh5.3.6/etc/hadoop/core-site.xml配置文件中添加如下两个属性:
<property>
<name>hadoop.proxyuser.beifeng.hosts</name>
<value>hadoop-senior.ibeifeng.com</value>
</property>
<property>
<name>hadoop.proxyuser.beifeng.groups</name>
<value>*</value>
</property>
(3)解压hadooplibs目录到oozie根目录
oozie-4.0.0-cdh5.3.6]$ tar -zxf oozie-hadooplibs-4.0.0-cdh5.3.6.tar.gz
(4)在oozie根目录中创建文件夹libext/
oozie-4.0.0-cdh5.3.6]$ mkdir libext
(5)拷贝hadooplibs下yarn版本文件夹中的所有jar包到新创建的文件夹libext中
oozie-4.0.0-cdh5.3.6]$ cp -r oozie-4.0.0-cdh5.3.6/hadooplibs/hadooplib-2.5.0-cdh5.3.6.oozie-4.0.0-cdh5.3.6/* libext/
(6)拷贝ExtJS压缩包到libext/目录下
oozie-4.0.0-cdh5.3.6]$ cp /opt/software/cdh/ext-2.2.zip libext/
(7)修改配置文件/opt/cdh-5.3.6/oozie-4.0.0-cdh5.3.6/conf/oozie-site.xml以下属性:
<property>
<name>oozie.service.HadoopAccessorService.hadoop.configurations</name>
<value>*=/opt/cdh-5.3.6/hadoop-2.5.0-cdh5.3.6/etc/hadoop</value>
</property>
(8)将libext/文件夹下的jar包打包成war
oozie-4.0.0-cdh5.3.6]$ bin/oozie-setup.sh prepare-war
(9)启动namenode、datanode,查看是否已退出安全模式:
hadoop-2.5.0-cdh5.3.6]$ bin/hdfs dfsadmin -safemode get
Safe mode is OFF
(10)向hdfs上传oozie所需的jar包,以时间创建文件夹存放jar包
oozie-4.0.0-cdh5.3.6]$ bin/oozie-setup.sh sharelib create -fs hdfs://hadoop-senior.ibeifeng.com:8020 -locallib oozie-sharelib-4.0.0-cdh5.3.6-yarn.tar.gz
(11)创建数据库
oozie-4.0.0-cdh5.3.6]$ bin/ooziedb.sh create -sqlfile oozie.sql -run DB Connection
(12)启动oozie
oozie-4.0.0-cdh5.3.6]$ bin/oozied.sh start
(13)查看oozie日志文件
cd /opt/cdh-5.3.6/oozie-4.0.0-cdh5.3.6/logs
more oozie.log
(14)停止oozie
oozie-4.0.0-cdh5.3.6]$ bin/oozied.sh stop
2、Oozie案例运行MapReduce Workflow
(1)解压examples压缩包
oozie-4.0.0-cdh5.3.6]$ tar -zxf oozie-examples.tar.gz
(2)将本地的examples目录上传到hdfs上主目录下的examples目录中
oozie-4.0.0-cdh5.3.6]$ /opt/cdh-5.3.6/hadoop-2.5.0-cdh5.3.6/bin/hdfs dfs -put examples/ examples/
(3)编辑配置文件/opt/cdh-5.3.6/oozie-4.0.0-cdh5.3.6/examples/apps/map-reduce/job.properties
nameNode=hdfs://hadoop-senior.ibeifeng.com:8020
jobTracker=hadoop-senior.ibeifeng.com:8032
(4)运行examples中的示例
oozie-4.0.0-cdh5.3.6]$ bin/oozie job -oozie http://localhost:11000/oozie -config examples/apps/map-reduce/job.properties -run
可以用export OOZIE_URL=http://localhost:11000/oozie
代替-oozie http://localhost:11000/oozie,
于是,运行任务写作如下:
oozie-4.0.0-cdh5.3.6]$ bin/oozie job -config examples/apps/map-reduce/job.properties -run
登录 | 立即注册