Varnish---原理及應用

原理如下圖:

用戶請求到達Varnish服務器,經由網卡將請求接入進來到達tcp/ip協議棧解封裝後由varnish將報文中請求資源的uri進行hash計算,而後根據計算的得到的鍵,到進程維持的hash表對比,若鍵相同,則根據相應的鍵去值指針所標識的,內存地址空間或是硬盤地址空間將結果取出,而後構建響應報文,將結果響應給客戶端,若在hash表中沒有響應的鍵,則進入過程2將請求的報文重新封裝,發送給後端的backend server去處理,得到結果保存在緩存中,而後重新構建響應報文,響應給客戶端.

Varnish的軟件架構

Varnish的安裝

官方網站下載相應發行版的軟件包共享庫和幫助文檔repo.varnish-cache.org

然後執行yum安裝

[root@www ~]# yum install \

varnish-3.0.5-1.el6.x86_64.rpm \

varnish-docs-3.0.0-1.el6.x86_64.\

varnish-libs-3.0.5-1.el6.x86_64.rpm

需要配置epel源

配置文件分爲兩部分

1,master的配置文件:/etc/sysyconfig/varnish

2,child 的配置文件:/etc/varnish/default.vcl------>>>由master調用vcl編譯器轉換爲C語言形式而後c編譯器編譯後交由child使用這樣避免語法錯誤child可直接使用

簡單應用篇:

實驗拓撲

配置varnish

[root@www ~]# vim /etc/sysconfig/varnish

NFILES=131072

MEMLOCK=82000

NPROCS="unlimited"

RELOAD_VCL=1

VARNISH_VCL_CONF=/etc/varnish/default.vcl

VARNISH_LISTEN_PORT=80

VARNISH_ADMIN_LISTEN_ADDRESS=127.0.0.1

VARNISH_ADMIN_LISTEN_PORT=6082

VARNISH_SECRET_FILE=/etc/varnish/secret

VARNISH_MIN_THREADS=50

VARNISH_MAX_THREADS=1000

VARNISH_THREAD_TIMEOUT=120

VARNISH_STORAGE_SIZE=64M

VARNISH_STORAGE="malloc,${VARNISH_STORAGE_SIZE}"

VARNISH_TTL=120

DAEMON_OPTS="-a ${VARNISH_LISTEN_ADDRESS}:${VARNISH_LISTEN_PORT} \

-f ${VARNISH_VCL_CONF} \

-T ${VARNISH_ADMIN_LISTEN_ADDRESS}:${VARNISH_ADMIN_LISTEN_PORT} \

-t ${VARNISH_TTL} \

-p thread_pool_min=${VARNISH_MIN_THREADS} \

-p thread_pool_max=${VARNISH_MAX_THREADS} \

-p thread_pool_timeout=${VARNISH_THREAD_TIMEOUT} \

-u varnish -g varnish \

-S ${VARNISH_SECRET_FILE} \

-s ${VARNISH_STORAGE}"

[root@www ~]# vim /etc/varnish/default.vcl

vcl 4.0;

backend default {

.host = "192.168.1.2";

.port = "80";

}

配置http編輯網頁文件

Vim /var/www/html/index.html

啓動各節點服務

Service vanish start

Service httpd start

客戶端測試

進階應用

需要使用VCL[varnish configure language]

請求數據在varnish中的流向

每個vcl_*函數都要有終止語句return(X)來說明下一個步要交給哪個函數處理

VCL基本語法[詳情及變量man vcl]

1,//,#,/*comment*/ 表示註釋

2,sub name,定義一個函數

3,不支持循環,有衆多內置變量

4,使用終止語句,沒有返回值

5,域專用

6,操作符 =賦值 ==等值比較 ~模式匹配 !取反 &&邏輯與 || 邏輯或

內置函數:

regsub (str,regex,sub) 搜索str中能被指定模式匹配的第一個字符串替換爲sub

resuball (str,regex,sub) 搜索str中能被指定模式匹配的全部字符串替換爲sub

Purge:從緩從中挑選出某對象及相關變種一併刪除

Reyurn():當vcl域運行結束時控制權交由指定的下一個域處理

Return():重新運行整個vcl每次重啓都會增加req.restarts變量中的值 max_restarts限定最大次數

Vcl_recv示例:

sub vcl_recv {

if (req.http.User-Agent ~ "iPad" ||

req.http.User-Agent ~ "iPhone" ||

req.http.User-Agent ~ "Android") {

set req.http.X-Device = "mobile";

} else {

set req.http.X-Device = "desktop";

}

此示例的含義是

如果請求的個護短代理類型是ipad,iphone,或是android那麼就在求情的首部中添加一個新的首部X-device爲mobile否則就設置爲desktop

下面實驗說明;在後端service的httpd訪問日誌中記錄下X-Device的值

以下實驗緊接上一個實驗

首先配置varnish

sub vcl_recv {

# Happens before we check if we have this in cache already.

# Typically you clean up the request here, removing cookies you don't need,

# rewriting the request, etc.

if (req.http.User-Agent ~ "ipad"||

req.http.User-Agent ~ "iphone"||

req.http.User-Agent ~ "iphone"){

set req.http.X-Device = "mobile";

}else {

set req.http.X-Device = "desktop";

}

配置http日誌格式加入X-Device首部的顯示

重啓httpd varnish使用客戶端訪問站點

驗證結果

查看訪問日誌

上圖所示的客戶端地址爲前端的varnish地址若向顯示真實的地址方法很簡單只需要在上述配置中增加一條vcl語句

sub vcl_recv {

if (req.http.User-Agent ~ "ipad"||

req.http.User-Agent ~ "iphone"||

req.http.User-Agent ~ "iphone"){

set req.http.X-Device = "mobile";

}else {

set req.http.X-Device = "desktop";

}

set req.http.X-Forwarded-For = client.ip;

}

編輯http訪問日誌格式

查看日誌,客戶端的真實地址就顯示出來了

判斷緩存是否命中

sub vcl_deliver {

accounting or modifying the final object here.

if (obj.hits > 0 ) {

set resp.http.X-Cache = "hit cache" + " " + server.hostname;

}else {

set resp.http.X-Cache = "miss cache" + " " + server.hostname;

}

測試結果

如何繞過緩存

sub vcl_recv {

if (req.http.User-Agent ~ "ipad"||

req.http.User-Agent ~ "iphone"||

req.http.User-Agent ~ "iphone"){

set req.http.X-Device = "mobile";

}else {

set req.http.X-Device = "desktop";

}

set req.http.X-Forwarded-For = client.ip;

if (req.url ~ "^/") {

return(pass);

}

測試結果,無論刷新幾次都不會命中緩存

如何刪除指定的緩存

acl purgers {

"127.0.0.1";

"172.16.0.0"/16;

}

sub vcl_recv {

if (req.request == "PURGE") {

if (!client.ip ~ purgers) {

error 405 "Method not allowed";

}

return (lookup);

}

sub vcl_hit {

if (req.request == "PURGE") {

purge;

error 200 "Purged";

}

sub vcl_miss {

if (req.request == "PURGE") {

purge;

error 404 "Not in cache";

}

sub vcl_pass {

if (req.request == "PURGE") {

error 502 "PURGE on a passed object";

}

客戶端測試

這裏爲了方便測試使用linux工具curl

[root@localhost html]# curl http://192.168.1.2

查看頭部信息

[root@localhost html]# curl -I http://192.168.1.2

HTTP/1.1 200 OK

Server: nginx/1.0.15

Content-Type: text/html

Last-Modified: Mon, 29 Sep 2014 06:15:54 GMT

Content-Length: 37

Accept-Ranges: bytes

Date: Mon, 29 Sep 2014 06:23:24 GMT

X-Varnish: 1450346254 1450346253

Age: 48

Via: 1.1 varnish

Connection: keep-alive

X-Cache: HIT via localhost.localdomain

此時緩存時命中的下面指定請求的方法看是否能夠將緩存清除

[root@localhost html]# curl -X PURGE http://172.16.34.1

<?xml version="1.0" encoding="utf-8"?>

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"

"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">

<html>

<head>

<title>200 Purged</title>

</head>

<body>

<h1>Error 200 Purged</h1>

<p>Purged</p>

<h3>Guru Meditation:</h3>

<hr>

<p>Varnish cache server</p>

</body>

</html>

看到返回的信息緩存已被清除

下面再次請求主頁面驗證是否清除如果緩存沒有hit 證明清除成功

[root@localhost html]# curl -I http://172.16.34.1

HTTP/1.1 200 OK

Server: nginx/1.0.15

Content-Type: text/html

Last-Modified: Mon, 29 Sep 2014 06:15:54 GMT

Content-Length: 37

Accept-Ranges: bytes

Date: Mon, 29 Sep 2014 06:27:00 GMT

X-Varnish: 1450346262

Age: 0

Via: 1.1 varnish

Connection: keep-alive

X-Cache: MISS via localhost.localdomain

由此可見緩存MISS證明清除成功

Varnish實現負載均衡

編輯配置文件

backend web1 {

.host = "192.168.1.2";

.port = "8080";

.probe = {

.url="/.health.html";

.interval = 2s;

.window = 8;

.threshold = 3;

.initial = 2 ;

}

backend web2 {

.host = "192.168.1.3";

.port = "8080";

.probe = {

.url="/.health.html"; ------------->檢測的頁面

.interval = 2s; ------------->健康監測的時間間隔

.window = 8; ------------->檢測的次數

.threshold = 3; -------------->3次失敗才認爲後端主機不可用

.initial = 2 ; ------------->2次成功就說明後端主機正常

}

director http_server round-robin { ----->算法爲rr

{.backend=web1;}

{.backend=web2;}

}

acl purgers {

"127.0.0.1";

"172.16.0.0"/16;

}

sub vcl_recv {

if (req.url ~ "^/"){

set req.backend = http_server;

return (pass);

}

if (req.request == "PURGE") {

if (!client.ip ~ purgers) {

error 405 "Method not allowed";

}

return (lookup);

}

sub vcl_hit {

if (req.request == "PURGE") {

purge;

error 200 "Purged";

}

sub vcl_miss {

if (req.request == "PURGE") {

purge;

error 404 "Not in cache";

}

sub vcl_pass {

if (req.request == "PURGE") {

error 502 "PURGE on a passed object";

}

sub vcl_deliver {

if (obj.hits > 0) {

set resp.http.X-Cache = "HIT via" + " " + server.hostname;

} else {

set resp.http.X-Cache = "MISS via" + " " + server.hostname;

}

配置後端服務器頁面建立健康監測頁面兩個節點分別建立

Server1 :Vim /var/www/html/{index.html,.health.html}

Index.html內容 -----><h1> Bacekend-server1 192.16.1.2 </h1>

health.html內容 -----> OK

Server2 :Vim /var/www/html/{index.html,.health.html}

Index.html內容 -----><h1> Bacekend-server2 192.16.1.3 </h1>

health.html內容 -----> OK

客戶端測試

[root@localhost ~]# curl http://172.16.101.200

<h1> Backend-server1 192.168.1.2 </h1>

[root@localhost ~]# curl http://172.16.101.200

<h1> Back-server2 192.168.1.3 </h1>

[root@localhost ~]# curl http://172.16.101.200

<h1> Backend-server1 192.168.1.2 </h1>

[root@localhost ~]# curl http://172.16.101.200

<h1> Back-server2 192.168.1.3 </h1>

查看後端狀態

[root@localhost ~]# varnishadm

varnish> backend.list

200

Backend name Refs Admin Probe

web1(192.168.1.2,,8080) 1 probe Healthy 8/8

web2(192.168.1.3,,8080) 1 probe Healthy 8/8

停止其中一個backend-server

varnish> backend.list

200

Backend name Refs Admin Probe

web1(192.168.1.2,,8080) 1 probe Sick 0/8

web2(192.168.1.3,,8080) 1 probe Healthy 8/8

這是隻能訪問web2

Varnish---原理及應用

杭州的 IT 崩盤了麼？

開源高性能結構化日誌模塊NanoLog

Python 潮流週刊#55：分享 9 個高質量的技術類信息源！

Azure Virtual Network (22) 多訂閱使用Azure DNS解析問題 Windows Azure Platform 系列文章目錄

【簡寫Mybatis-02】註冊機的實現以及SqlSession處理

手繪二維碼

.NET藉助虛擬網卡實現一個簡單異地組網工具

Linux軟件管理之RPM

rsync+inotify實時同步

Keepalived

我的友情鏈接

Nginx----基礎篇

Mac下配置sublime實現LaTeX

https://yachay.unat.edu.pe/blog/index.php?comment_area=format_blog&comment_component=blog&comment_co

linux以太網驅動總結