FastDFS-分佈式文件系統安裝(單節點)

1.首先安裝依賴包

yum install -y gcc-c++ wget unzip perl pcre-devel zlib-devel

yum -y install libevent

2.下載所需要的代碼包和工具包

2.1下載FastDFS-v5.05.tar.gz

wget http://sourceforge.net/projects/fastdfs/files/FastDFS%20Server%20Source%20Code/FastDFS%20Server%20with%20PHP%20Extension%20Source%20Code%20V5.05/FastDFS_v5.05.tar.gz

2.2下載libfastcommonV1.0.7

wget https://codeload.github.com/happyfish100/libfastcommon/tar.gz/V1.0.7 -O libfastcommon-1.0.7.tar.gz

2.3下載fastdfs-nginx-module-v1.16.tar.gz

wget http://nchc.dl.sourceforge.net/project/fastdfs/FastDFS%20Nginx%20Module%20Source%20Code/fastdfs-nginx-module_v1.16.tar.gz

2.4下載nginx

wget http://nginx.org/download/nginx-1.8.1.tar.gz

3.查看官方安裝步驟

3.1 解壓FastDFS_v5.05.tar.gz

tar -zxvf FastDFS_v5.05.tar.gz

3.2 查看解壓後文件夾FastDFS中的 ==INSTALL== 安裝步驟

cat INSTALL


#第一步：安裝 libfastcommon
#step 1. download libfastcommon source package from github and install it,
   the github address:
   https://github.com/happyfish100/libfastcommon.git

#第二步：
#step 2. download FastDFS source package and unpack it, 
tar xzf FastDFS_v5.x.tar.gz
#for example:
tar xzf FastDFS_v5.08.tar.gz

#step 3. enter the FastDFS dir
cd FastDFS

#step 4. execute:
./make.sh

#step 5. make install
./make.sh install

#step 6. edit/modify the config file of tracker and storage

#step 7. run server programs
#start the tracker server:
/usr/bin/fdfs_trackerd /etc/fdfs/tracker.conf restart
#in Linux, you can start fdfs_trackerd as a service:
/sbin/service fdfs_trackerd start

#start the storage server:
/usr/bin/fdfs_storaged /etc/fdfs/storage.conf restart
#in Linux, you can start fdfs_storaged as a service:
/sbin/service fdfs_storaged start

#step 8. run test program
#run the client test program:
/usr/bin/fdfs_test <client_conf_filename> <operation>
/usr/bin/fdfs_test1 <client_conf_filename> <operation>
#for example, upload a file:
/usr/bin/fdfs_test conf/client.conf upload /usr/include/stdlib.h

#step 9. run monitor program
#run the monitor program:
/usr/bin/fdfs_monitor <client_conf_filename>


tracker server config file sample please see conf/tracker.conf

storage server config file sample please see conf/storage.conf

client config file sample please see conf/client.conf


Item detail
1. server common items
---------------------------------------------------
|  item name            |  type  | default | Must |
---------------------------------------------------
| base_path             | string |         |  Y   |
---------------------------------------------------
| disabled              | boolean| false   |  N   |
---------------------------------------------------
| bind_addr             | string |         |  N   |
---------------------------------------------------
| network_timeout       | int    | 30(s)   |  N   |
---------------------------------------------------
| max_connections       | int    | 256     |  N   |
---------------------------------------------------
| log_level             | string | info    |  N   |
---------------------------------------------------
| run_by_group          | string |         |  N   |
---------------------------------------------------
| run_by_user           | string |         |  N   |
---------------------------------------------------
| allow_hosts           | string |   *     |  N   |
---------------------------------------------------
| sync_log_buff_interval| int    |  10(s)  |  N   |
---------------------------------------------------
| thread_stack_size     | string |  1M     |  N   |
---------------------------------------------------
memo:
   * base_path is the base path of sub dirs: 
     data and logs. base_path must exist and it's sub dirs will 
     be automatically created if not exist.
       $base_path/data: store data files
       $base_path/logs: store log files
   * log_level is the standard log level as syslog, case insensitive
     # emerg: for emergency
     # alert
     # crit: for critical
     # error
     # warn: for warning
     # notice
     # info
     # debug
   * allow_hosts can ocur more than once, host can be hostname or ip address,
     "*" means match all ip addresses, can use range like this: 10.0.1.[1-15,20]
      or host[01-08,20-25].domain.com, for example:
        allow_hosts=10.0.1.[1-15,20]
        allow_hosts=host[01-08,20-25].domain.com

2. tracker server items
---------------------------------------------------
|  item name            |  type  | default | Must |
---------------------------------------------------
| port                  | int    | 22000   |  N   |
---------------------------------------------------
| store_lookup          | int    |  0      |  N   |
---------------------------------------------------
| store_group           | string |         |  N   |
---------------------------------------------------
| store_server          | int    |  0      |  N   |
---------------------------------------------------
| store_path            | int    |  0      |  N   |
---------------------------------------------------
| download_server       | int    |  0      |  N   |
---------------------------------------------------
| reserved_storage_space| string |  1GB    |  N   |
---------------------------------------------------

memo: 
  * the value of store_lookup is:
    0: round robin (default)
    1: specify group
    2: load balance (supported since V1.1)
  * store_group is the name of group to store files.
    when store_lookup set to 1(specify group), 
    store_group must be set to a specified group name.
  * reserved_storage_space is the reserved storage space for system 
    or other applications. if the free(available) space of any stoarge
    server in a group <= reserved_storage_space, no file can be uploaded
    to this group (since V1.1)
    bytes unit can be one of follows:
      # G or g for gigabyte(GB)
      # M or m for megabyte(MB)
      # K or k for kilobyte(KB)
      # no unit for byte(B)

3. storage server items
-------------------------------------------------
|  item name          |  type  | default | Must |
-------------------------------------------------
| group_name          | string |         |  Y   |
-------------------------------------------------
| tracker_server      | string |         |  Y   |
-------------------------------------------------
| port                | int    | 23000   |  N   |
-------------------------------------------------
| heart_beat_interval | int    |  30(s)  |  N   |
-------------------------------------------------
| stat_report_interval| int    | 300(s)  |  N   |
-------------------------------------------------
| sync_wait_msec      | int    | 100(ms) |  N   |
-------------------------------------------------
| sync_interval       | int    |   0(ms) |  N   |
-------------------------------------------------
| sync_start_time     | string |  00:00  |  N   |
-------------------------------------------------
| sync_end_time       | string |  23:59  |  N   |
-------------------------------------------------
| store_path_count    | int    |   1     |  N   |
-------------------------------------------------
| store_path0         | string |base_path|  N   |
-------------------------------------------------
| store_path#         | string |         |  N   |
-------------------------------------------------
|subdir_count_per_path| int    |   256   |  N   |
-------------------------------------------------
|check_file_duplicate | boolean|    0    |  N   |
-------------------------------------------------
| key_namespace       | string |         |  N   |
-------------------------------------------------
| keep_alive          | boolean|    0    |  N   |
-------------------------------------------------
| sync_binlog_buff_interval| int |   60s |  N   |
-------------------------------------------------

memo:
  * tracker_server can ocur more than once, and tracker_server format is
    "host:port", host can be hostname or ip address.
  * store_path#, # for digital, based 0
  * check_file_duplicate: when set to true, must work with FastDHT server, 
    more detail please see INSTALL of FastDHT. FastDHT download page: 
    http://code.google.com/p/fastdht/downloads/list
  * key_namespace: FastDHT key namespace, can't be empty when 
    check_file_duplicate is true. the key namespace should short as possible

4.安裝步驟

4.1 libfastcommon-1.0.7.tar.gz

1.  tar -zxvf libfastcommon-1.0.7.tar.gz 
2.  ./make.sh 
3.  ./make.sh install
#   把/usr/lib64/libfastcommon.so文件向/usr/lib/下複製一份
#   libfastcommon安裝好後會自動將庫文件拷貝至/usr/lib64下，由於FastDFS程序引用usr/lib目錄所以需要將/usr/lib64下的庫文件拷貝至/usr/lib下
4.  cp /usr/lib64/libfastcommon.so  /usr/lib/

4.2 安裝FastDFS

1.  tar -zxvf FastDFS_v5.05.tar.gz  
2.  cd FastDFS
3.  ./make.sh 
4.  ./make.sh install
#  /usr/bin/目錄下有以fdfs開頭的文件都是編譯出來的。
#  配置文件都放到/etc/fdfs文件夾
#  FastDFS/conf目錄下的所有的配置文件都複製到/etc/fdfs下
5. cp -f conf/* /etc/fdfs/

4.3配置Tracker服務

1. cd /etc/fdfs/     #進入配置文件所在的目錄
2. mkdir -p /usr/data/fastdfs #創建自定義存放路徑
3. vim tracker.conf  #tracker.conf中的base_path 爲自己路徑
4. base_path=/usr/data/fastdfs #設置tracker日誌存放路徑
5. /usr/bin/fdfs_trackerd /etc/fdfs/tracker.conf #啓動
6. /usr/bin/fdfs_trackerd /etc/fdfs/tracker.conf restart #重啓
7. /usr/bin/fdfs_trackerd /etc/fdfs/tracker.conf stop #停止

4.4配置Storage服務 (如果是在不同的服務器安裝，4.3步的1~4需要重新執行)

1. cd /etc/fdfs/    #進入配置文件所在的目
2. mkdir -p /usr/data/fastdfs #創建自定義存放路徑(以創建不需要再創建)
3. vim storage.conf #storage.conf中的base_path 爲自己路徑
4. base_path=/usr/data/fastdfs  #設置storage日誌存放路徑
5. store_path0=/usr/data/fastdfs #設置圖片存放路徑
6. tracker_server=192.168.31.102:22122 #指定tracker服務器的地址和端口

7. /usr/bin/fdfs_storaged /etc/fdfs/storage.conf #啓動storage
8. /usr/bin/fdfs_storaged /etc/fdfs/storage.conf stop #停止storage
9. /usr/bin/fdfs_storaged /etc/fdfs/storage.conf restart #重啓storage

4.5配置client.conf


1. vim client.conf
2. base_path=/usr/data/fastdfs # 設置 客戶端日誌存放路徑
3. tracker_server=192.168.31.102:22122 #設置 tracker 服務器的地址和端口

4.6測試(huluwa.jpg是我自己上傳的圖片，系統有默認圖片也可以測試：/etc/fdfs 下的 anti-steal.jpg )

[root@localhost fdfs]# /usr/bin/fdfs_test /etc/fdfs/client.conf upload huluwa.jpg

測試成功返回的數據：



This is FastDFS client test program v5.05

Copyright (C) 2008, Happy Fish / YuQing

FastDFS may be copied only under the terms of the GNU General
Public License V3, which may be found in the FastDFS source kit.
Please visit the FastDFS Home Page http://www.csource.org/ 
for more detail.

[2017-02-23 17:55:35] DEBUG - base_path=/usr/data/fastdfs, connect_timeout=30, network_timeout=60, tracker_server_count=1, anti_steal_token=0, anti_steal_secret_key length=0, use_connection_pool=0, g_connection_pool_max_idle_time=3600s, use_storage_id=0, storage server id count: 0

tracker_query_storage_store_list_without_group: 
        server 1. group_name=, ip_addr=192.168.31.102, port=23000

group_name=group1, ip_addr=192.168.31.102, port=23000
storage_upload_by_filename
group_name=group1, remote_filename=M00/00/00/wKgfZlivkpeAbomSAADgedhzSso810.jpg
source ip address: 192.168.31.102
file timestamp=2017-02-23 17:55:35
file size=57465
file crc32=3631434442
example file url: http://192.168.31.102/group1/M00/00/00/wKgfZlivkpeAbomSAADgedhzSso810.jpg
storage_upload_slave_by_filename
group_name=group1, remote_filename=M00/00/00/wKgfZlivkpeAbomSAADgedhzSso810_big.jpg
source ip address: 192.168.31.102
file timestamp=2017-02-23 17:55:35
file size=57465
file crc32=3631434442
example file url: http://192.168.31.102/group1/M00/00/00/wKgfZlivkpeAbomSAADgedhzSso810_big.jpg

4.7搭建nginx提供http服務

4.7.1安裝fastdfs-nginx-module_v1.16.tar.gz

1.  tar -zxvf fastdfs-nginx-module_v1.16.tar.gz #解壓
2.  cp -r fastdfs-nginx-module /usr/local/  #將解壓後的文件複製到 /usr/local/ 下

3.vim config #去掉/local

修改之後

CORE_INCS="$CORE_INCS /usr/include/fastdfs /usr/include/fastcommon/" #默認安裝fastdfs後，如果這裏有local要去掉
CORE_LIBS="$CORE_LIBS -L/usr/lib -lfastcommon -lfdfsclient"

4.7.2 安裝ngin

1. tar -zxvf nginx-1.8.0.tar.gz 

2. [root@localhost local]# mkdir -p nginx # usr/local/下創建nginx文件目錄
   [root@localhost local]# cd nginx/
   [root@localhost nginx]# pwd
      /usr/local/nginx

3. [root@localhost var]# pwd
/var
[root@localhost var]#  mkdir -p temp/nginx #var下創建nginx臨時變量存儲目錄

4. 進入[root@localhost nginx-1.8.0] 修改參數：
./configure \
--prefix=/usr/local/nginx \
--pid-path=/var/run/nginx/nginx.pid \
--lock-path=/var/lock/nginx.lock \
--error-log-path=/var/log/nginx/error.log \
--http-log-path=/var/log/nginx/access.log \
--with-http_gzip_static_module \
--http-client-body-temp-path=/var/temp/nginx/client \
--http-proxy-temp-path=/var/temp/nginx/proxy \
--http-fastcgi-temp-path=/var/temp/nginx/fastcgi \
--http-uwsgi-temp-path=/var/temp/nginx/uwsgi \
--http-scgi-temp-path=/var/temp/nginx/scgi \
--add-module=/usr/local/fastdfs-nginx-module/src #fastdfs-nginx-module src 所在的目錄

5. make
6. make install

4.7.3 編輯mod_fastdfs.conf

1. cp -p /usr/local/fastdfs-nginx-module/src/mod_fastdfs.conf  /etc/fdfs/ #複製mod-fastdfs.conf到/etc/fdfs/ 下
2. base_path=/tmp #日誌存放路徑
3. tracker_server=192.168.31.102:22122 #tracker服務器地址和端口號
3. store_path0=/usr/data/fastdfs #圖片存放路徑
4. url_have_group_name = true

4.7.4 編輯nginx.conf,添加一個server

 server {
        listen       80;
        server_name  192.168.31.102;
        location /group1/M00/ {
            root /usr/data/fastdfs/data;
            ngx_fastdfs_module;
        }
    }

將libfdfsclient.so拷貝至/usr/lib下

cp -r /usr/lib64/libfdfsclient.so  /usr/lib/

啓動nginx
啓動trancer 和 storage

[root@localhost /]# cd /usr/local/nginx/sbin/
[root@localhost sbin]# ./nginx 

[root@localhost /]# /usr/bin/fdfs_trackerd /etc/fdfs/tracker.conf 
[root@localhost /]# /usr/bin/fdfs_storaged /etc/fdfs/storage.conf

瀏覽器訪問：http://192.168.31.102/group1/M00/00/00/wKgfZlivkpeAbomSAADgedhzSso810_big.jpg

拒絕訪問，開放端口：

1.  vim /etc/sysconfig/iptables
2.  -A INPUT -m state --state NEW -m tcp -p tcp --dport 22122 -j ACCEPT #插入此行 開放22122端口
3.  service iptables restart

4.7.5設置自動啓動

1. vim /etc/rc.d/rc.local 
2.  加入：/usr/local/nginx/sbin/nginx

5.一些錯誤

java 代碼中出錯：java.net.SocketTimeoutException: connect timed out

插入此行開放23000端口

  -A INPUT -m state --state NEW -m tcp -p tcp --dport 23000 -j ACCEPT

FastDFS-分佈式文件系統安裝(單節點)

1.首先安裝依賴包

2.下載所需要的代碼包和工具包

2.1下載FastDFS-v5.05.tar.gz

2.2下載libfastcommonV1.0.7

2.3下載fastdfs-nginx-module-v1.16.tar.gz

2.4下載nginx

3.查看官方安裝步驟

3.1 解壓FastDFS_v5.05.tar.gz

3.2 查看解壓後文件夾FastDFS中的 ==INSTALL== 安裝步驟

4.安裝步驟

4.1 libfastcommon-1.0.7.tar.gz

4.2 安裝FastDFS

4.3配置Tracker服務

4.4配置Storage服務 (如果是在不同的服務器安裝，4.3步的1~4需要重新執行)

4.5配置client.conf

4.6測試(huluwa.jpg是我自己上傳的圖片，系統有默認圖片也可以測試：/etc/fdfs 下的 anti-steal.jpg )

4.7搭建nginx提供http服務

4.7.1安裝fastdfs-nginx-module_v1.16.tar.gz

4.7.2 安裝ngin

4.7.3 編輯mod_fastdfs.conf

4.7.4 編輯nginx.conf,添加一個server

4.7.5設置自動啓動

5.一些錯誤

工作中用到的腳本合集

微服務實踐Aspire項目發佈到遠程k8s集羣

通過f-string編寫簡潔高效的Python格式化輸出代碼

[轉帖]20個常用的Linux工具命令

[轉帖]PostgreSQL從小白到高手教程 - 第46講：poc-tpch測試

24-5-18 X

mysql數據庫中批量更新

zookeeper僞分佈式集羣搭建

Hadoop學習(一)--分佈式環境搭建

Redis學習-持久化

CentOS 6.5 中安裝 RabbitMQ -單機版

https://yachay.unat.edu.pe/blog/index.php?comment_area=format_blog&comment_component=blog&comment_co

linux以太網驅動總結

FastDFS-分佈式文件系統安裝(單節點)

1.首先安裝依賴包

2.下載所需要的代碼包和工具包

2.1下載FastDFS-v5.05.tar.gz

2.2下載libfastcommonV1.0.7

2.3下載fastdfs-nginx-module-v1.16.tar.gz

2.4下載nginx

3.查看官方安裝步驟

3.1 解壓FastDFS_v5.05.tar.gz

3.2 查看 解壓後 文件夾FastDFS中的 ==INSTALL== 安裝步驟

4.安裝步驟

4.1 libfastcommon-1.0.7.tar.gz

4.2 安裝FastDFS

4.3配置Tracker服務

4.4配置Storage服務 (如果是在不同的服務器安裝，4.3步的1~4需要重新執行)

4.5配置client.conf

4.6測試(huluwa.jpg是我自己上傳的圖片，系統有默認圖片也可以測試：/etc/fdfs 下的 anti-steal.jpg )

4.7搭建nginx提供http服務

4.7.1安裝fastdfs-nginx-module_v1.16.tar.gz

4.7.2 安裝ngin

4.7.3 編輯mod_fastdfs.conf

4.7.4 編輯nginx.conf,添加一個server

4.7.5設置自動啓動

5.一些錯誤

3.2 查看解壓後文件夾FastDFS中的 ==INSTALL== 安裝步驟