原创 HDPCD-Java-複習筆記(4)
Map Aggregation Aggregation The term refers to a Mapper combining its <key, value> pairs, with the goal of reducing th
原创 HDPCD-Java-複習筆記(7)- lab
Java Lab Booklet Lab: Distributed Grep You will search the U.S. Constitution for string patterns matching any string y
原创 使用Mybatis-Generator自動生成Dao、Model、Mapping相關文件
轉自:http://www.cnblogs.com/lichenwei/p/4145696.html Mybatis屬於半自動ORM,在使用這個框架中,工作量最大的就是書寫Mapping的映射文件,由於手動書寫很容易出錯,我們可以
原创 Nifi Docker集羣-兩個節點(2)
構建基礎鏡像前,請確保: 1.物理機或者虛擬機對外網絡通暢,ping baidu.com有收到迴應信息 2.確保Dockerfile文件所在目錄中存在以下文件及文件夾: Dockerfile, init.sh, id_rsa.pu
原创 HDPCD-Java-複習筆記(1)
1.Understand Hadoop HDFS Pig -- A scripting language that simplifies the creation of MapReduce jobs and excels at exp
原创 JAVA - Timezones
public static final int TIMEZONE_WEST = -12;public static final int TIMEZONE_EAST = 14; public static void main(String
原创 HDPCD-Java-複習筆記(23)- lab
Java lab booklet Lab: Importing Data into HBase The stock prices of the stocks that start with “A” input into a table
原创 HDPCD-Java-複習筆記(15)
Unit Testing MRUnit is an Apache project that provides Java classes for unit testing MapReduce jobs. There are basical
原创 CentOs7 關閉防火牆 - 大數據集羣內部環境需要
CentOS7版本後防火牆默認使用firewalld,因此在CentOS7中關閉防火牆使用以下命令,//臨時關閉systemctl stop firewalld//禁止開機啓動systemctl disab
原创 HDPCD-Java-複習筆記(18) - lab
Java lab booklet Understanding Pig root@ubuntu:~/java/labs/demos# pig grunt> copyFromLocal/root/java/labs/demos/pigdem
原创 SPARK APP壓測--清理檢查相關SHELL(1)
壓力測試前需要對大數據環境、日誌和存儲數據庫進行清理、檢查環境是否可用、保存測試前的環境狀態。壓力測試過程中需要定期檢查PARQUET文件生成情況、報表生成情況、HBASE數據生成情況、是否有SPARK APP FAILED、收集錯誤日誌
原创 PostgreSQL 刪除check約束
刪除check約束:1.\d table_name 查找約束名稱\d dm_alarms_fct_20170613 "dm_alarms_fct_20170613_dm_transct_date_key_check"2.alter tab
原创 Docker踩坑實錄
1.docker啓動mysql容器 docker run --name mysql -p 3306:3306 -v /root/mysql/config/mysql.cnf:/etc/mysql/mysql.cnf -v /root/my
原创 HDPCD-Java-複習筆記(16)
PIG Pig uses a high-level, SQL-like programming language namedPig Latin. Pig was created at Yahoo to make it easier to
原创 HDPCD-Java-複習筆記(19)
Hive Apache Hive maintains metadata information in a metastore to generate tables. A Hive table consists of: ·