當數據總數太大而不能從頭開始時經常出現頻繁的情況,但是新記錄的數量相當小。示例:擁有1,000,000個存檔帖子的論壇,但每天只有1,000個新帖。
在這種情況下,可以使用所謂的“主+ delta”方案實現“實時”(幾乎實時)索引更新。
簡單的實現 “主索引+增量索引”方法
在數據庫中增加一個計數表,記錄每次重新構建主索引時,被索引表的最後一個數據id,這樣在增量索引時只需要索引這個id以後的數據即可,每次重新構建主索引時都更新這個表。
1.先在mysql中插入一個計數表和兩個索引表
CREATE TABLE f12_counter( id INTEGER PRIMARY KEY NOT NULL, productid INTEGER NOT NULL);
2. sphinx 配置
source search
{
type = mysql
sql_host = localhost
sql_user = root
sql_pass =
sql_db = db_hs12fr
sql_port = 3306
sql_query_pre = SET NAMES utf8
sql_query_pre = SET SESSION query_cache_type=OFF
sql_query_pre = REPLACE INTO f12_counter SELECT 1, MAX(productid) FROM f12_product_search #數據源查詢前的操作
sql_query = SELECT productid,categoryid,sortvalue,attribute,shop_price FROM f12_product_search where productid<=(SELECT productid FROM f12_counter WHERE id=1)
sql_attr_uint = sortvalue
sql_attr_uint = categoryid
sql_attr_float = shop_price
}
source delta_search:search
{
type = mysql
sql_host = localhost
sql_user = root
sql_pass =
sql_db = db_hs12fr
sql_port = 3306
sql_query_pre = SET NAMES utf8
sql_query_pre = SET SESSION query_cache_type=OFF
sql_query = SELECT productid,categoryid,sortvalue,attribute,shop_price FROM f12_product_search WHERE productid>( SELECT productid FROM f12_counter WHERE id=1 )
sql_attr_uint = sortvalue
sql_attr_uint = categoryid
sql_attr_float = shop_price
}
index search
{ #主索引
source = search
path = /usr/local/sphinx2.2/var/search
docinfo = extern
mlock = 0
morphology = none
min_word_len = 1
charset_type = sbcs
html_strip = 0
}
index delta_search:search
{ #增量索引
source = delta_search
path = /usr/local/sphinx2.2/var/delta_search
docinfo = extern
mlock = 0
morphology = none
min_word_len = 1
charset_type = sbcs
html_strip = 0
}
啓動 sphinx
/usr/local/sphinx/bin/indexer --all --config /usr/local/sphinx/etc/sphinx.conf
/usr/local/sphinx/bin/searchd --config /usr/local/sphinx/etc/sphinx.conf
索引自動更新
可做定時腳本執行 一主索引更新腳本 一增量索引更新腳本
增量索引更新
/usr/local/sphinx/bin/indexer --config /usr/local/sphinx/etc/sphinx.conf delta_search --rotate
/usr/local/sphinx/bin/searchd --config /usr/local/sphinx/etc/sphinx.conf --stop
/usr/local/sphinx/bin/indexer --config /usr/local/sphinx/etc/sphinx.conf --merge search delta_search (索引合併)
主索引更新
/usr/local/sphinx/bin/searchd --config /usr/local/sphinx/etc/sphinx.conf --stop
/usr/local/sphinx/bin/indexer --config /usr/local/sphinx/etc/sphinx.conf search
這裏上將一些新添加數據寫進增量索引中,通過增量索引將數據與主索引數據合併,這樣我們就不必去頻繁更新有關大量數據的主索引。在 PHP API 中的 Query(keyword,index) ,這第二索引名稱
是設置多個索引名,也就沒有必要一定將兩個索引合併