FastDFS高可用文件系統集羣搭建詳解

幫助文檔：
https://blog.csdn.net/ZhanBiaoChina/article/details/103908976
https://www.cnblogs.com/yufeng218/p/8111961.html
https://blog.csdn.net/wc1695040842/article/details/89766064

服務器地址	規劃用途
192.168.169.3	Tracker(group1)
192.168.169.4	Tracker(group2)
192.168.169.5	Storage-group1-01
192.168.169.6	Storage-group1-02
192.168.169.7	Storage-group2-01
192.168.169.8	Storage-group2-02

一,基本環境: (所有fastdfs主機操作)
需要軟件包：
libfastcommon-master.zip
FastDFS_v5.05.tar.gz
5e5f3566bbfa57418b5506aaefbe107a42c9fcb1.zip
nginx-1.16.1.tar.gz
ngx_cache_purge-2.3.tar.gz
名稱解析
1.編輯/etc/hosts做名稱解析，scp到其他節點上去

for i in `seq 4 8`;do scp /etc/hosts [email protected].$i:/etc/; done

2.安裝依賴包

yum install gcc gcc-c++ make automake autoconf libtool pcre pcre-devel zlib zlib-devel openssl openssl-devel cmake screen lrzsz wget lsof epel-release vim  gcc libevent libevent-doc libevent-devel libevent-headers cmake -y

3.安裝FastDFS依賴包libfastcommon

cd /usr/local/src
unzip libfastcommon-master.zip
cd libfastcommon-master/
./make.sh
./make.sh install
ln -s /usr/lib64/libfastcommon.so /usr/local/lib/libfastcommon.so

4.安裝FastDFS

cd /usr/local/src/
tar zxf FastDFS_v5.05.tar.gz
cd fastdfs-5.05/
./make.sh
./make.sh install

因爲FastDFS服務腳本默認設置的bin目錄爲/usr/local/bin下，但實際安裝在了/usr/bin下，所以需要FastDFS配置文件中的路徑，修改兩個配置文件。

vim /etc/init.d/fdfs_storaged
#使用查找替換命令進統一修改
:%s+/usr/local/bin+/usr/bin
vim /etc/init.d/fdfs_trackerd
#使用查找替換命令進統一修改
:%s+/usr/local/bin+/usr/bin

注意：以上操作無論是配置 tracker 還是配置 storage 都是必須的，而 tracker 和 storage 的區別主要是在安裝完 fastdfs 之後的配置過程中。

二，修改FastDFS配置文件
配置tracker服務器(192.168.169.3,192.168.169.4)

screen -S tracker
cp /etc/fdfs/tracker.conf.sample /etc/fdfs/tracker.conf
vim /etc/fdfs/tracker.conf
修改如下內容：
# 修改的內容如下：
disabled=false              # 啓用配置文件
bind_addr=192.168.169.3		#綁定IP地址，更具自身修改
port=22122                  # tracker服務器端口（默認22122）
base_path=/fastdfs/tracker  # 存儲日誌和數據的根目錄
store_lookup=0              # 輪詢方式上傳

其它參數保留默認配置，具體配置解釋可參考官方文檔說明：
http://bbs.chinaunix.net/thread-1941456-1-1.html

創建基礎數據目錄

mkdir -p /fastdfs/tracker

啓動tracker服務器

/etc/init.d/fdfs_trackerd start
netstat -anput | grep tracker
tcp        0      0 0.0.0.0:22122           0.0.0.0:*               LISTEN      19571/fdfs_trackerd
#關閉命令
/etc/init.d/fdfs_trackerd stop

配置storage服務(192.168.169.5,192.168.169.6,192.168.169.7,192.168.169.8)
注意：
192.168.169.5,192.168.169.6爲同組group1
192.168.169.7,192.168.169.8爲同組group2

複製storage樣例配置文件，並重命名

cp /etc/fdfs/storage.conf.sample /etc/fdfs/storage.conf
screen -S storage
vim /etc/fdfs/storage.conf
disabled=false
bind_addr=192.168.169.5
#不同分組配置不同group_name
group_name=group1
# storage的端口號,同一個組的 storage 端口號必須相同
port=23000
#超時時間
connect_timeout=10
# 存儲日誌和數據的根目錄
base_path=/fastdfs/storage
# 第一個存儲目錄
store_path0=/fastdfs/storage
#存儲路徑個數，需要和store_path個數匹配
store_path_count=1
#tracker服務器的IP地址和端口，多個tracker直接添加多條配置
tracker_server=192.168.169.3:22122
tracker_server=192.168.169.4:22122
#設置http端口號
http.server_port=8888

其它參數保留默認配置，具體配置解釋可參考官方文檔說明：
http://bbs.chinaunix.net/thread-1941456-1-1.html

創建基礎數據目錄

mkdir -p /fastdfs/storage

啓動storage

fdfs_storaged /etc/fdfs/storage.conf start

查看tracker日誌：

tail -f -n 100 /fastdfs/tracker/logs/trackerd.log
INFO - file: tracker_relationship.c, line: 383, selecting leader..
INFO - file: tracker_relationship.c, line: 422, the tracker leader 192.168.169.4:22122

查看storage日誌：

INFO - file: storage_func.c, line: 254, tracker_client_ip: 192.168.169.7, my_server_id_str: 192.168.169.7, g_server_id_in_filename: 128559296
INFO - file: tracker_client_thread.c, line: 1235, tracker server 192.168.169.4:22122, set tracker leader: 192.168.169.4:22122
INFO - file: storage_sync.c, line: 2698, successfully connect to storage server 192.168.169.8:23000

查看集羣信息：

[root@server3 00]# /usr/bin/fdfs_monitor /etc/fdfs/storage.conf                                                                                 
[2020-02-12 11:49:06] DEBUG - base_path=/fastdfs/storage, connect_timeout=10, network_timeout=60, tracker_server_count=2, anti_steal_token=0, anti_steal_secret_key length=0, use_connection_pool=0, g_connection_pool_max_idle_time=3600s, use_storage_id=0, storage server id count: 0

server_count=2, server_index=1

tracker server is 192.168.169.4:22122

group count: 2

Group 1:
group name = group1
disk total space = 17394 MB
disk free space = 15706 MB
trunk free space = 0 MB
storage server count = 2
active server count = 2
storage server port = 23000
storage HTTP port = 8888
store path count = 1
subdir count per path = 256
current write server index = 0
current trunk file id = 0

	Storage 1:
		id = 192.168.169.5
		ip_addr = 192.168.169.5 (anantes-651-1-49-net.w2-0.abo.wanadoo.fr)  ACTIVE
		http domain = 
		version = 5.05
		join time = 2020-02-10 21:18:13
		up time = 2020-02-12 11:03:02
		total storage = 17394 MB
		free storage = 15706 MB
		upload priority = 10
		store_path_count = 1
		subdir_count_per_path = 256
		storage_port = 23000
		storage_http_port = 8888
		current_write_path = 0
		source storage id = 
		if_trunk_server = 0
		connection.alloc_count = 256
		connection.current_count = 1
		connection.max_count = 2
		total_upload_count = 11
		success_upload_count = 11
		total_append_count = 0
		success_append_count = 0
		total_modify_count = 0
		success_modify_count = 0
		total_truncate_count = 0
		success_truncate_count = 0
		total_set_meta_count = 0
		success_set_meta_count = 0
		total_delete_count = 0
		success_delete_count = 0
		total_download_count = 0
		success_download_count = 0
		total_get_meta_count = 0
		success_get_meta_count = 0
		total_create_link_count = 0
		success_create_link_count = 0
		total_delete_link_count = 0
		success_delete_link_count = 0
		total_upload_bytes = 30280393
		success_upload_bytes = 30280393
		total_append_bytes = 0
		success_append_bytes = 0
		total_modify_bytes = 0
		success_modify_bytes = 0
		stotal_download_bytes = 0
		success_download_bytes = 0
		total_sync_in_bytes = 22022104
		success_sync_in_bytes = 22022104
		total_sync_out_bytes = 0
		success_sync_out_bytes = 0
		total_file_open_count = 19
		success_file_open_count = 19
		total_file_read_count = 0
		success_file_read_count = 0
		total_file_write_count = 209
		success_file_write_count = 209
		last_heart_beat_time = 2020-02-12 11:48:40
		last_source_update = 2020-02-12 11:17:00
		last_sync_update = 2020-02-10 22:36:04
		last_synced_timestamp = 2020-02-10 22:36:01 (-1s delay)
	Storage 2:
		id = 192.168.169.6
		ip_addr = 192.168.169.6 (anantes-651-1-49-net.w2-0.abo.wanadoo.fr)  ACTIVE
		http domain = 
		version = 5.05
		join time = 2020-02-10 21:18:18
		up time = 2020-02-12 11:03:02
		total storage = 17394 MB
		free storage = 15755 MB
		upload priority = 10
		store_path_count = 1
		subdir_count_per_path = 256
		storage_port = 23000
		storage_http_port = 8888
		current_write_path = 0
		source storage id = 192.168.169.5
		if_trunk_server = 0
		connection.alloc_count = 256
		connection.current_count = 1
		connection.max_count = 1
		total_upload_count = 8
		success_upload_count = 8
		total_append_count = 0
		success_append_count = 0
		total_modify_count = 0
		success_modify_count = 0
		total_truncate_count = 0
		success_truncate_count = 0
		total_set_meta_count = 0
		success_set_meta_count = 0
		total_delete_count = 0
		success_delete_count = 0
		total_download_count = 0
		success_download_count = 0
		total_get_meta_count = 0
		success_get_meta_count = 0
		total_create_link_count = 0
		success_create_link_count = 0
		total_delete_link_count = 0
		success_delete_link_count = 0
		total_upload_bytes = 22022104
		success_upload_bytes = 22022104
		total_append_bytes = 0
		success_append_bytes = 0
		total_modify_bytes = 0
		success_modify_bytes = 0
		stotal_download_bytes = 0
		success_download_bytes = 0
		total_sync_in_bytes = 30280393
		success_sync_in_bytes = 30280393
		total_sync_out_bytes = 0
		success_sync_out_bytes = 0
		total_file_open_count = 19
		success_file_open_count = 19
		total_file_read_count = 0
		success_file_read_count = 0
		total_file_write_count = 209
		success_file_write_count = 209
		last_heart_beat_time = 2020-02-12 11:48:38
		last_source_update = 2020-02-10 22:36:00
		last_sync_update = 2020-02-12 11:17:06
		last_synced_timestamp = 2020-02-12 11:17:00 (0s delay)

Group 2:
group name = group2
disk total space = 40059 MB
disk free space = 31172 MB
trunk free space = 0 MB
storage server count = 2
active server count = 2
storage server port = 23000
storage HTTP port = 8888
store path count = 1
subdir count per path = 256
current write server index = 0
current trunk file id = 0

	Storage 1:
		id = 192.168.169.7
		ip_addr = 192.168.169.7 (anantes-651-1-49-net.w2-0.abo.wanadoo.fr)  ACTIVE
		http domain = 
		version = 5.05
		join time = 2020-02-10 21:18:21
		up time = 2020-02-12 11:03:02
		total storage = 39196 MB
		free storage = 37558 MB
		upload priority = 10
		store_path_count = 1
		subdir_count_per_path = 256
		storage_port = 23000
		storage_http_port = 8888
		current_write_path = 0
		source storage id = 
		if_trunk_server = 0
		connection.alloc_count = 256
		connection.current_count = 1
		connection.max_count = 2
		total_upload_count = 1
		success_upload_count = 1
		total_append_count = 0
		success_append_count = 0
		total_modify_count = 0
		success_modify_count = 0
		total_truncate_count = 0
		success_truncate_count = 0
		total_set_meta_count = 0
		success_set_meta_count = 0
		total_delete_count = 0
		success_delete_count = 0
		total_download_count = 0
		success_download_count = 0
		total_get_meta_count = 0
		success_get_meta_count = 0
		total_create_link_count = 0
		success_create_link_count = 0
		total_delete_link_count = 0
		success_delete_link_count = 0
		total_upload_bytes = 2752763
		success_upload_bytes = 2752763
		total_append_bytes = 0
		success_append_bytes = 0
		total_modify_bytes = 0
		success_modify_bytes = 0
		stotal_download_bytes = 0
		success_download_bytes = 0
		total_sync_in_bytes = 0
		success_sync_in_bytes = 0
		total_sync_out_bytes = 0
		success_sync_out_bytes = 0
		total_file_open_count = 1
		success_file_open_count = 1
		total_file_read_count = 0
		success_file_read_count = 0
		total_file_write_count = 11
		success_file_write_count = 11
		last_heart_beat_time = 2020-02-12 11:48:36
		last_source_update = 2020-02-12 11:17:01
		last_sync_update = 1970-01-01 08:00:00
		last_synced_timestamp = 1970-01-01 08:00:00 
	Storage 2:
		id = 192.168.169.8
		ip_addr = 192.168.169.8 (anantes-651-1-49-net.w2-0.abo.wanadoo.fr)  ACTIVE
		http domain = 
		version = 5.05
		join time = 2020-02-10 21:18:24
		up time = 2020-02-12 11:03:24
		total storage = 40059 MB
		free storage = 31172 MB
		upload priority = 10
		store_path_count = 1
		subdir_count_per_path = 256
		storage_port = 23000
		storage_http_port = 8888
		current_write_path = 0
		source storage id = 192.168.169.7
		if_trunk_server = 0
		connection.alloc_count = 256
		connection.current_count = 1
		connection.max_count = 1
		total_upload_count = 0
		success_upload_count = 0
		total_append_count = 0
		success_append_count = 0
		total_modify_count = 0
		success_modify_count = 0
		total_truncate_count = 0
		success_truncate_count = 0
		total_set_meta_count = 0
		success_set_meta_count = 0
		total_delete_count = 0
		success_delete_count = 0
		total_download_count = 0
		success_download_count = 0
		total_get_meta_count = 0
		success_get_meta_count = 0
		total_create_link_count = 0
		success_create_link_count = 0
		total_delete_link_count = 0
		success_delete_link_count = 0
		total_upload_bytes = 0
		success_upload_bytes = 0
		total_append_bytes = 0
		success_append_bytes = 0
		total_modify_bytes = 0
		success_modify_bytes = 0
		stotal_download_bytes = 0
		success_download_bytes = 0
		total_sync_in_bytes = 2752763
		success_sync_in_bytes = 2752763
		total_sync_out_bytes = 0
		success_sync_out_bytes = 0
		total_file_open_count = 1
		success_file_open_count = 1
		total_file_read_count = 0
		success_file_read_count = 0
		total_file_write_count = 11
		success_file_write_count = 11
		last_heart_beat_time = 2020-02-12 11:48:56
		last_source_update = 1970-01-01 08:00:00
		last_sync_update = 2020-02-12 11:17:03
		last_synced_timestamp = 2020-02-12 11:17:02 (-1s delay)

文件上傳測試(192.168.169.3)
修改Tracker服務器客戶端配置文件

cp /etc/fdfs/client.conf.sample /etc/fdfs/client.conf
vim /etc/fdfs/client.conf
# 修改以下配置，其它保持默認
connect_timeout=10
base_path=/fastdfs/tracker
tracker_server=192.168.169.3:22122 # tracker服務器IP和端口
tracker_server=192.168.169.4:22122  #tracker服務器IP2和端口

進入storage目錄下查看是否有文件

cd /fastdfs/storage/data/00/00
[root@server3 00]# ll
總用量 0

執行文件上傳命令

[root@server1 ~]# /usr/bin/fdfs_upload_file /etc/fdfs/client.conf /root/test.jpg 
group1/M00/00/00/wKipBV5DbiyAHQ44ACoA-25HStA010.jpg
[root@server1 ~]# /usr/bin/fdfs_upload_file /etc/fdfs/client.conf /root/test.jpg 
group2/M00/00/00/wKipB15Dbi6APS3gACoA-25HStA882.jpg

在所有storage節(192.168.169.5,192.168.169.6,192.168.169.7,192.168.169.8)安裝Nginx
1，fastdfs-nginx-module 作用說明
FastDFS 通過 Tracker 服務器,將文件放在 Storage 服務器存儲，但是同組存儲服務器之間需要進入文件複製，有同步延遲的問題。假設 Tracker 服務器將文件上傳到了 192.168.x.x，上傳成功後文件 ID已經返回給客戶端。此時 FastDFS 存儲集羣機制會將這個文件同步到同組存儲 192.168.x.x，在文件還沒有複製完成的情況下，客戶端如果用這個文件 ID 在 192.168.x.x 上取文件,就會出現文件無法訪問的錯誤。而 fastdfs-nginx-module 可以重定向文件連接到源服務器取文件,避免客戶端由於複製延遲導致的文件無法訪問錯誤。（解壓後的 fastdfs-nginx-module 在 nginx 安裝時使用）

2，解壓5e5f3566bbfa57418b5506aaefbe107a42c9fcb1.zip

cd /usr/local/src/
unzip 5e5f3566bbfa57418b5506aaefbe107a42c9fcb1.zip

3、修改 fastdfs-nginx-module的config配置文件

cd fastdfs-nginx-module-5e5f3566bbfa57418b5506aaefbe107a42c9fcb1/src/
vim config
將/usr/local/路徑改爲/usr/，或者直接執行命令:%s+/usr/local/+/usr/+g批量替換(注：已修改)

4、安裝nginx

groupadd nginx && useradd -g nginx nginx
cd /usr/local/src/
tar -zxvf nginx-1.16.1.tar.gz
cd nginx-1.16.1
./configure --prefix=/usr/local/nginx --with-http_ssl_module --with-http_stub_status_module --with-pcre --with-http_realip_module --add-module=/usr/local/src/fastdfs-nginx-module-5e5f3566bbfa57418b5506aaefbe107a42c9fcb1/src/ --group=nginx --user=nginx
make && make install

5，複製 fastdfs-nginx-module-…/src/ 源碼中的配置文件到 /etc/fdfs 目錄，並修改

cp /usr/local/src/fastdfs-nginx-module-5e5f3566bbfa57418b5506aaefbe107a42c9fcb1/src/mod_fastdfs.conf /etc/fdfs/
vim /etc/fdfs/mod_fastdfs.conf
#修改內容：比如連接超時時間，跟蹤路徑配置，url的group配置等。
#注意：第一組（1.5，1.6）第二組（1.7，1.8）節點修改內容，不同內容只有一個組名！
connect_timeout=10
base_path=/tmp
tracker_server=192.168.169.3:22122  # tracker服務器IP和端口
tracker_server=192.168.169.4:22122  # tracker服務器IP2和端口
group_name=group1                   # 當前服務器的group名，第一組爲group1，第二組爲group2.
url_have_group_name=true            # url中包含group名稱
store_path0=/fastdfs/storage   		# 存儲路徑
group_count=2                       # 設置組的個數

#在最後添加 
[group1]
group_name=group1
storage_server_port=23000
store_path_count=1
store_path0=/fastdfs/storage

[group2]
group_name=group2
storage_server_port=23000
store_path_count=1
store_path0=/fastdfs/storage

6，scp分發到其他storage主機上，並修改內容。

for i in `seq 6 8`;do scp /etc/fdfs/mod_fastdfs.conf [email protected].$i:/etc/fdfs/;done

7，複製 FastDFS 的部分配置文件到 /etc/fdfs 目錄

cd /usr/local/src/fastdfs-5.05/conf/
cp http.conf mime.types /etc/fdfs/
ln -s /fastdfs/storage/data/ /fastdfs/storage/data/M00

8，修改nginx配置文件

#user nobody;
worker_processes 1;
events {
    worker_connections 1024;
}
http {
    include mime.types;
    default_type application/octet-stream;
    sendfile on;
    keepalive_timeout 65;
    server {
        listen 8888;
        server_name localhost;
        location ~/group[1-2]/M00 {
            ngx_fastdfs_module;
        }
        error_page 500 502 503 504 /50x.html;
        location = /50x.html {
            root html;
        }
    }

}
#8888 端口值是要與/etc/fdfs/storage.conf中的 http.server_port=8888 相對應，因爲 http.server_port 默認爲 8888，如果想改成 80，則要對應修改過來。

9，scp分發到其他storage主機上，並啓動

for i in `seq 6 8`;do scp /usr/local/nginx/conf/nginx.conf [email protected].$i:/usr/local/nginx/conf/;done
[root@server3 conf]# /usr/local/nginx/sbin/nginx
ngx_http_fastdfs_set pid=36501

10，通過瀏覽器訪問測試時上傳的文件
http://192.168.169.5:8888/group1/M00/00/00/wKipBl5DdqmAQEW8ACoA-25HStA834.jpg
http://192.168.169.7:8888/group2/M00/00/00/wKipBl5DdqmAQEW8ACoA-25HStA834.jpg

三，在tracker節點（192.168.169.3、192.168.0.169.4）安裝Nginx
在 tracker 上安裝的 nginx 主要爲了提供 http 訪問的反向代理、負載均衡以及緩存服務
1，安裝nginx，加入ngx_cache_purge（加入緩存模塊）

cd /usr/local/src/
tar zxf nginx-1.16.1.tar.gz
tar zxf ngx_cache_purge-2.3.tar.gz
cd nginx-1.16.1/
./configure --prefix=/usr/local/nginx --with-http_ssl_module --with-http_stub_status_module --with-pcre --with-http_realip_module --add-module=/usr/local/src/ngx_cache_purge-2.3 && make && make install

2，配置nginx負載均衡和緩存

vim /usr/local/nginx/conf/nginx.conf
#添加如下內容：
user root;
worker_processes 1;
#error_log logs/error.log;
#error_log logs/error.log notice;
#error_log logs/error.log info;
#pid logs/nginx.pid;
events {
	worker_connections 1024;
	use epoll;
}
http {
	include mime.types;
	default_type application/octet-stream;
	#log_format main '$remote_addr - $remote_user [$time_local] "$request" '
	# '$status $body_bytes_sent "$http_referer" '
	# '"$http_user_agent" "$http_x_forwarded_for"';
	#access_log logs/access.log main;
	sendfile on;
	tcp_nopush on;
	#keepalive_timeout 0;
	keepalive_timeout 65;
	#gzip on;
	#設置緩存
	server_names_hash_bucket_size 128;
	client_header_buffer_size 32k;
	large_client_header_buffers 4 32k;
	client_max_body_size 300m;
	proxy_redirect off;
	proxy_set_header Host $http_host;
	proxy_set_header X-Real-IP $remote_addr;
	proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
	proxy_connect_timeout 90;
	proxy_send_timeout 90;
	proxy_read_timeout 90;
	proxy_buffer_size 16k;
	proxy_buffers 4 64k;
	proxy_busy_buffers_size 128k;
	proxy_temp_file_write_size 128k;
	#設置緩存存儲路徑、存儲方式、分配內存大小、磁盤最大空間、緩存期限
	proxy_cache_path /data/fastdfs/cache/nginx/proxy_cache levels=1:2 keys_zone=http-cache:200m max_size=1g inactive=30d;
	proxy_temp_path /data/fastdfs/cache/nginx/proxy_cache/tmp;
	#設置 group1 的服務器
	upstream fdfs_group1 {
		server 192.168.169.5:8888 weight=1 max_fails=2 fail_timeout=30s;
		server 192.168.169.6:8888 weight=1 max_fails=2 fail_timeout=30s;
	}
	#設置 group2 的服務器
	upstream fdfs_group2 {
		server 192.168.169.7:8888 weight=1 max_fails=2 fail_timeout=30s;
		server 192.168.169.8:8888 weight=1 max_fails=2 fail_timeout=30s;
	}
	server {
		listen 8000;
		server_name localhost;
		#charset koi8-r;
		#access_log logs/host.access.log main;
		#設置 group 的負載均衡參數
		location /group1/M00 {
			proxy_next_upstream http_502 http_504 error timeout invalid_header;
			proxy_cache http-cache;
			proxy_cache_valid 200 304 12h;
			proxy_cache_key $uri$is_args$args;
			proxy_pass http://fdfs_group1;
			expires 30d;
		}
		location /group2/M00 {
			proxy_next_upstream http_502 http_504 error timeout invalid_header;
			proxy_cache http-cache;
			proxy_cache_valid 200 304 12h;
			proxy_cache_key $uri$is_args$args;
			proxy_pass http://fdfs_group2;
			expires 30d;
		} 
		#設置清除緩存的訪問權限
		location ~/purge(/.*) {
			allow 127.0.0.1;
			allow 192.168.0.0/24;
			deny all;
			proxy_cache_purge http-cache $1$is_args$args;
		}
		#error_page 404 /404.html;
		# redirect server error pages to the static page /50x.html
		#
		error_page 500 502 503 504 /50x.html;
			location = /50x.html {
			root html;
		}
	}
}
[root@server1 conf]# /usr/local/nginx/sbin/nginx -t
nginx: the configuration file /usr/local/nginx/conf/nginx.conf syntax is ok
nginx: configuration file /usr/local/nginx/conf/nginx.conf test is successful

按以上 nginx 配置文件的要求，創建對應的緩存目錄：

mkdir -p /fastdfs/cache/nginx/proxy_cache
mkdir -p /fastdfs/cache/nginx/proxy_cache/tmp

啓動Nginx

/usr/local/nginx/sbin/nginx

文件訪問測試
前面直接通過訪問Storage節點中的Nginx來訪問文件
http://192.168.169.5:8888/group1/M00/00/00/wKipBl5DdqmAQEW8ACoA-25HStA834.jpg
http://192.168.169.7:8888/group2/M00/00/00/wKipBl5DdqmAQEW8ACoA-25HStA834.jpg
現在可以通過Tracker中的Nginx來進行訪問
1）通過Tracker1中的Nginx來訪問
http://192.168.169.3:8000/group1/M00/00/00/wKipBl5DdqmAQEW8ACoA-25HStA834.jpg
http://192.168.169.3:8000/group2/M00/00/00/wKipBl5DdqmAQEW8ACoA-25HStA834.jpg
2）通過Tracker2中的Nginx來訪問
http://192.168.169.4:8000/group2/M00/00/00/wKipBl5DdqmAQEW8ACoA-25HStA834.jpg
http://192.168.169.4:8000/group1/M00/00/00/wKipBl5DdqmAQEW8ACoA-25HStA834.jpg

四，Nginx高可用

1、如果是在阿里雲的ECS上搭建FastDFS集羣（或者其他雲平臺）
如果是在阿里雲上，那非常簡單，直接用SLB（或者類似負載均衡）代理到兩臺Tracker服務器上就行。

2、內網環境搭建FastDFS集羣
我們需要在內網環境中再搭建兩套Nginx+Keepalived服務，用一個VIP實現高可用。

五，java操作fastdfs集羣：

connect_timeout = 2
network_timeout = 30
charset = UTF-8
http.tracker_http_port = 8080
http.anti_steal_token = no
http.secret_key = FastDFS1234567890
tracker_server = 192.168.169.3:21222
tracker_server = 192.168.169.4:21222

ngx_http_fastdfs_module.c //nginx-module接口實現文件，用於接入fastdfs-module核心模塊邏輯
common.c //fastdfs-module核心模塊，實現了初始化、文件下載的主要邏輯
common.h //對應於common.c的頭文件
config //編譯模塊所用的配置，裏面定義了一些重要的常量，如擴展配置文件路徑、文件下載chunk大小
mod_fastdfs.conf //擴展配置文件的demo

#nginx啓動報錯
root@server6:/usr/local/nginx# /usr/local/nginx/sbin/nginx
/usr/local/nginx/sbin/nginx: error while loading shared libraries: libfdfsclient.so: cannot open shared object file: No such file or directory
root@server6:/usr/local/nginx# ldd /usr/local/nginx/sbin/nginx 
	linux-vdso.so.1 (0x00007ffe06fe4000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f6248073000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f6247e54000)
	libcrypt.so.1 => /lib/x86_64-linux-gnu/libcrypt.so.1 (0x00007f6247c1c000)
	libfastcommon.so => /usr/local/lib/libfastcommon.so (0x00007f62479d8000)
	libfdfsclient.so => not found
	libpcre.so.3 => /lib/x86_64-linux-gnu/libpcre.so.3 (0x00007f6247766000)
	libssl.so.1.1 => /usr/lib/x86_64-linux-gnu/libssl.so.1.1 (0x00007f62474d9000)
	libcrypto.so.1.1 => /usr/lib/x86_64-linux-gnu/libcrypto.so.1.1 (0x00007f624700e000)
	libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f6246df1000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f6246a00000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f6248578000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f6246662000)
root@server6:/usr/local/nginx# find / -name libfdfsclient.so
/usr/lib64/libfdfsclient.so
/usr/local/src/fastdfs-5.05/client/libfdfsclient.so
root@server6:/usr/local/nginx# vim /etc/ld.so.conf
#添加
/usr/lib64/
root@server6:/usr/local/nginx# ldconfig
root@server6:/usr/local/nginx# /usr/local/nginx/sbin/nginx
ngx_http_fastdfs_set pid=13133

FastDFS高可用文件系統集羣搭建詳解

Nginx R31 doc 官方文檔-01-nginx 如何安裝

Qt/C++音視頻開發74-合併標籤圖形/生成yolo運算結果圖形/文字和圖形合併成一個/水印濾鏡

挑戰程序設計競賽 2.2章習題 POJ - 3617 Best Cow Line 貪心

字節面試：MySQL什麼時候鎖表？如何防止鎖表？

.NET8連接SQL SERVER 2008 R2 報：證書鏈是由不受信任的頒發機構頒發的

golang開發環境搭建(win10)

python計算機視覺學習筆記——PIL庫的用法

Golang初學：獲取程序內存使用情況，std runtime

Prometheus學習筆記四–Alertmanager配置概述及路由概述

Prometheus學習筆記二–安裝部署簡單監控及使用PromQL查詢監控數據

Prometheus學習筆記三–郵件報警規則配置

Prometheus學習筆記-認識Prometheus

如何在物理機上安裝EXSI平臺經測試可用生產

https://yachay.unat.edu.pe/blog/index.php?comment_area=format_blog&comment_component=blog&comment_co

linux以太網驅動總結