http://blog.csdn.net/lovehuangjiaju/article/details/48577281
1.查看 hdfs 裏文件
hadoop fs -ls -R /
2.在 hdfs中新建目錄
hadoop fs -mkdir -p /data/wordcount
3.上傳文件到hdfs
hadoop fs -put /usr/local/cluster/hadoop/etc/hadoop/slaves /data/wordcount/將slaves文件傳到hdfs下的wordcount下。
4.spark 讀取hdfs文件
val readli1=sc.textFile("hdfs://hadoop-master:8020/data/wordcount/read.txt")
5. intellij idea spark 開發（參見）
http://blog.csdn.net/u012587561/article/details/52167306
6.進入hdfs： 10.6.3.200:50070
7.hadoop fs -ls hdfs：//10.6.3.200:8020/user/hadoop/output #查看指定目錄下的文件和文件夾。/user/hadoop/output是HDFS上的目錄，不是本地目錄
hadoop fs -cat /data/wordcount/read2/part-00002
#查看文件內容
8./root/spark-1.0.0-bin-2.2.0/bin/spark-submit –master spark://10.6.3.200:7077 –class main.scala.sp –executor-memory 1g spw.jar hdfs://hadoop-master:8020/data/wordcount/1.txt hdfs://hadoop-master:8020/data/wordcount/read2
提交