本文代碼對應的github地址：https://github.com/nieandsun/redis-study
1 簡單介紹一下什麼是pipeline — 以jedis爲例
以批量根據key刪除數據爲例，如若單純的使用del命令進行刪除，那java代碼可能會這樣寫：
public static void delNoStus(String... keys) {
Jedis jedis = new Jedis(RedisTools.ip, RedisTools.port);
//在循環中進行批量刪除
for (String key : keys) {
jedis.del(key);
}
jedis.close();
}
上面的刪除命令每次都會建立一個新的網絡連接，其模型如下，這將非常耗時，尤其是跨機房部署的情況下。
而pipeline可以將多個指令進行打包，一次丟給redis服務器，並獲取返回結果，其運行模型如下：
那java操作jedis的代碼就可以寫成下面的樣子：
public static void delNoPipe(String... keys) {
Jedis jedis = new Jedis(RedisTools.ip, RedisTools.port);
Pipeline pipelined = jedis.pipelined();
for (String key : keys) {
pipelined.del(key);//封裝未提交
}
pipelined.sync();//一次性提交
jedis.close();
}
有興趣的可以clone下來我的代碼進行測試一下，我測試的結果爲對10000個key進行刪除，不使用pipelie的情況下總用時爲416ms，而使用pipeline的情況下總用時爲34ms，時間縮短了10倍還多。
2 RedisTemplate使用pipeline批量set需要注意的坑
當然使用RedisTemplate也可以使用pipeline，但是注意，有一個比較大的坑。
我看網上給出的批量set值的方法都如下：
public void setPipe(Map<String, String> map) {
List list = redisTemplate.executePipelined((RedisCallback<String>) connection -> {
Iterator<Map.Entry<String, String>> iterator = map.entrySet().iterator();
while (iterator.hasNext()) {
Map.Entry<String, String> next = iterator.next();
connection.set(next.getKey().getBytes(), next.getValue().getBytes());
}
return null;
});
System.out.println("setPipe" + list);
}
但是經過我測試發現，這種方式可以把值存入到redis服務器，但是讀取數據時，無論使不用使用pipeline都會報如下錯誤：
org.springframework.data.redis.serializer.SerializationException: Could not read JSON: Unrecognized token 'value': was expecting (JSON String, Number, Array, Object or token 'null', 'true' or 'false')
at [Source: (byte[])"value:0"; line: 1, column: 7]; nested exception is com.fasterxml.jackson.core.JsonParseException: Unrecognized token 'value': was expecting (JSON String, Number, Array, Object or token 'null', 'true' or 'false')
at [Source: (byte[])"value:0"; line: 1, column: 7]
at org.springframework.data.redis.serializer.Jackson2JsonRedisSerializer.deserialize(Jackson2JsonRedisSerializer.java:75)
at org.springframework.data.redis.core.RedisTemplate.deserializeMixedResults(RedisTemplate.java:617)
at org.springframework.data.redis.core.RedisTemplate.lambda$executePipelined$1(RedisTemplate.java:335)
at org.springframework.data.redis.core.RedisTemplate.execute(RedisTemplate.java:228)
at org.springframework.data.redis.core.RedisTemplate.execute(RedisTemplate.java:188)
at org.springframework.data.redis.core.RedisTemplate.execute(RedisTemplate.java:175)
at org.springframework.data.redis.core.RedisTemplate.executePipelined(RedisTemplate.java:324)
at org.springframework.data.redis.core.RedisTemplate.executePipelined(RedisTemplate.java:314)
at com.nrsc.redis.learning.pipeline.RedisTemplateTest1.getPipe(RedisTemplateTest1.java:86)
at com.nrsc.redis.learning.pipeline.RedisTemplateTest1.test(RedisTemplateTest1.java:36)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.springframework.test.context.junit4.statements.RunBeforeTestExecutionCallbacks.evaluate(RunBeforeTestExecutionCallbacks.java:74)
at org.springframework.test.context.junit4.statements.RunAfterTestExecutionCallbacks.evaluate(RunAfterTestExecutionCallbacks.java:84)
at org.springframework.test.context.junit4.statements.RunBeforeTestMethodCallbacks.evaluate(RunBeforeTestMethodCallbacks.java:75)
at org.springframework.test.context.junit4.statements.RunAfterTestMethodCallbacks.evaluate(RunAfterTestMethodCallbacks.java:86)
at org.springframework.test.context.junit4.statements.SpringRepeat.evaluate(SpringRepeat.java:84)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:251)
at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:97)
at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
at org.springframework.test.context.junit4.statements.RunBeforeTestClassCallbacks.evaluate(RunBeforeTestClassCallbacks.java:61)
at org.springframework.test.context.junit4.statements.RunAfterTestClassCallbacks.evaluate(RunAfterTestClassCallbacks.java:70)
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.run(SpringJUnit4ClassRunner.java:190)
at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:33)
at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:230)
at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:58)
Caused by: com.fasterxml.jackson.core.JsonParseException: Unrecognized token 'value': was expecting (JSON String, Number, Array, Object or token 'null', 'true' or 'false')
at [Source: (byte[])"value:0"; line: 1, column: 7]
at com.fasterxml.jackson.core.JsonParser._constructError(JsonParser.java:1851)
at com.fasterxml.jackson.core.base.ParserMinimalBase._reportError(ParserMinimalBase.java:717)
at com.fasterxml.jackson.core.json.UTF8StreamJsonParser._reportInvalidToken(UTF8StreamJsonParser.java:3585)
at com.fasterxml.jackson.core.json.UTF8StreamJsonParser._handleUnexpectedValue(UTF8StreamJsonParser.java:2680)
at com.fasterxml.jackson.core.json.UTF8StreamJsonParser._nextTokenNotInObject(UTF8StreamJsonParser.java:865)
at com.fasterxml.jackson.core.json.UTF8StreamJsonParser.nextToken(UTF8StreamJsonParser.java:757)
at com.fasterxml.jackson.databind.ObjectMapper._initForReading(ObjectMapper.java:4620)
at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4469)
at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3538)
at org.springframework.data.redis.serializer.Jackson2JsonRedisSerializer.deserialize(Jackson2JsonRedisSerializer.java:73)
... 40 more
2020-05-31 11:54:11.942 INFO 4912 --- [extShutdownHook] o.s.s.concurrent.ThreadPoolTaskExecutor : Shutting down ExecutorService 'applicationTaskExecutor'
2020-05-31 11:54:11.984 WARN 4912 --- [extShutdownHook] d.r.c.l.LettucePoolingConnectionProvider : LettucePoolingConnectionProvider contains unreleased connections
Process finished with exit code -1
其實錯誤提示的很明顯，就是序列化問題，百度了一下，找到了如下文章，測試發現可行。
《Redis使用Pipeline時對象序列化失敗org.springframework.data.redis.serializer.SerializationException》
這裏我貼一個完整的 RedisTemplate使用pipeline的demo，有興趣的可以運行一下：
@SpringBootTest
@RunWith(SpringRunner.class)
@Slf4j
public class RedisTemplateTest2 {
@Autowired
private RedisTemplate<String, Object> redisTemplate;
private int arrayLength = 10000;
private String[] keys = new String[arrayLength];
private List<String> keys2 = Lists.newArrayList();
@Test
public void test() {
Map<String, String> map = initData();
long t = System.currentTimeMillis();
setPipe(map);
getPipe(keys);
System.out.println(System.currentTimeMillis() - t);
}
/***
* 初始化數據
* @return
*/
public Map<String, String> initData() {
Map<String, String> map = Maps.newHashMap();
for (int i = 0; i < arrayLength; i++) {
String key = "key:" + i;
String value = "value:" + i;
map.put(key, value);
keys[i] = key;
keys2.add(key);
}
return map;
}
/****
* 通過pipeline進行批量設置
* 報了一個錯誤，參考下面的文章得到了問題的解決方案：
* https://blog.csdn.net/myfwjy/article/details/100776426
* 《Redis使用Pipeline時對象序列化失敗org.springframework.data.redis.serializer.SerializationException》
* @param map
*/
public void setPipe(Map<String, String> map) {
RedisSerializer keySerializer = redisTemplate.getKeySerializer();
RedisSerializer valueSerializer = redisTemplate.getValueSerializer();
List list = redisTemplate.executePipelined((RedisCallback<String>) connection -> {
Iterator<Map.Entry<String, String>> iterator = map.entrySet().iterator();
while (iterator.hasNext()) {
Map.Entry<String, String> next = iterator.next();
connection.set(keySerializer.serialize(next.getKey()), valueSerializer.serialize(next.getValue()));
}
return null;
//加不加下面的這一行代碼應該都可以
}, redisTemplate.getValueSerializer());
System.out.println("setPipe" + list);
}
public void getPipe(String... keys) {
List<Object> list = redisTemplate.executePipelined((RedisCallback<?>) connection -> {
for (String s : keys) {
connection.get(s.getBytes());
}
return null;
});
System.out.println("pipeline獲取結果" + list);
List<Object> list1 = redisTemplate.opsForValue().multiGet(keys2);
System.out.println("multiGet獲取結果" + list1);
}
}
簡單提一下運行結果
end！！！