記錄一次編寫一個udf的代碼優化記錄;
代碼邏輯:通過將離線訂單數統計量(每天5點更新累計到前一天24:00的量,這裏存在兩種情況,0-5時是累計到前天24時的量,5-24時是累計到昨天24點的量),實時提交訂單統計量(只包含當天和昨天),實時取消訂單統計量(只包含當天和昨天)這三個量解析出來,進行實時加減運算,計算出當前時刻某用戶實時訂單量。
原代碼如下:
@UdfFunction(udfName = "FLASH_HISTORY_ORDER_CNT")
public long computeFlashOrderCount(String offlineData, String orderCount, String orderCancel){
long offline = 0;
long count = 0;
SimpleDateFormat format = new SimpleDateFormat("yyyyMMdd");
try {
String today = format.format(new Date(System.currentTimeMillis()));
String yesterday = format.format(new Date(System.currentTimeMillis()-86400000));
Map<String, Long> orderCnt = new HashMap<>();
if(!StringUtils.isBlank(orderCount)){
orderCnt = renderFeatureMap(today, yesterday, orderCount);
}
Map<String, Long> orderCnl = new HashMap<>();
if(!StringUtils.isBlank(orderCancel)){
orderCnl = renderFeatureMap(today, yesterday, orderCancel);
}
if (!StringUtils.isBlank(offlineData)) {//離線數據存在
String offline_date = offlineData.split(":")[0];
offline = Long.valueOf(getValue(offlineData.split(":")[1]));
if (offline_date.equals(yesterday)){//判斷是否是昨天的數據
count = offline + orderCnt.get(today) - orderCnl.get(today);
}else {//離線數據不是昨天的數據
count = orderCnt.get(today) + orderCnt.get(yesterday) - orderCnl.get(today) - orderCnl.get(yesterday);
count += offline;
}
}else {//離線數據不存在
for(Long s:orderCnt.values()){
count += s;
}
for(Long s:orderCnl.values()){
count -= s;
}
}
if(count < 0){
count = 0;
}
}catch (Exception e){
log.error("FeatureUdf.fxwGivenDateValue.jsonValue.ValueError", e);
return 0L;
}
return count;
}
private Map<String, Long> renderFeatureMap(String today,String yesterday, String jsonStr){
Map<String, Long> map = new HashMap<>();
if(!StringUtils.isBlank(jsonStr)){
Map<String, String> kvMap = (Map<String, String>) JSON.parseObject(jsonStr, Map.class);
for(String k: kvMap.keySet()){
if(k.contains(today) || k.contains(yesterday))
map.put(k.substring(0,8),Long.valueOf(getValue(kvMap.get(k))));
}
map.computeIfAbsent(today,K -> 0L);
map.computeIfAbsent(yesterday,K -> 0L);
}else {
map.put(today, 0L);
return map;
}
return map;
}
private String getValue(String v) {
int index;
if (!StringUtils.isEmpty(v)) {
if ((index = v.indexOf("^")) > -1) {
return v.substring(0, index);
} else {
return v;
}
}else{
return "";
}
原代碼中,我使用了好多map對象,基本上每個map都是初始化之後開始用的。這樣會消耗大量的內存,如果QPS不高的情況下還好,QPS高的情況下會導致頻繁GC。所以需要對代碼進行優化。
優化後代碼:
@UdfFunction(udfName = "FLASH_HISTORY_ORDER_CNT")
public long computeFlashOrderCount(String offlineData, String orderCount, String orderCancel){
SimpleDateFormat format = new SimpleDateFormat("yyyyMMdd");
long offline = 0;
long count = 0;
try {
Date date = new Date();
date.setTime(System.currentTimeMillis());
String today = format.format(date) + "0000";
date.setTime(System.currentTimeMillis() - 86400000);
String yesterday = format.format(date) + "0000";
long[] orderCnt = renderFeatureMap(today, yesterday, orderCount);
long[] orderCnl = renderFeatureMap(today, yesterday, orderCancel);
if (StringUtils.isNotBlank(offlineData)) {//離線數據存在
String offline_date = "";
int index;
String offlineData_value = getValue(offlineData);
if((index = offlineData_value.indexOf(":")) > -1){
offline_date = offlineData_value.substring(0,index)+"0000";
offline = Long.valueOf(offlineData_value.split(":")[1]);
}
if (offline_date.equals(yesterday)){//判斷是否是昨天的數據
count = offline + orderCnt[0] - orderCnl[0];
}else {//離線數據不是昨天的數據
count = orderCnt[0] + orderCnt[1] - orderCnl[0] - orderCnl[1];
count += offline;
}
}else {//離線數據不存在
count = orderCnt[0] + orderCnt[1] - orderCnl[0] - orderCnl[1];
}
if(count < 0){
count = 0;
}
}catch (Exception e){
log.error("FeatureUdf.computeFlashOrderCount.jsonValue.ValueError offlineData:{}, orderCount:{}, orderCancel:{}", offlineData, orderCount, orderCancel);
return 0L;
}
return count;
}
private long[] renderFeatureMap(String today,String yesterday, String jsonStr){
long[] featureArr = new long[2];
if(StringUtils.isNotBlank(jsonStr)){
JSONObject jsonObject = JSON.parseObject(jsonStr);
if(StringUtils.isNotBlank(jsonObject.getString(today))){
featureArr[0] = Long.valueOf(getValue(jsonObject.getString(today)));
}
if(StringUtils.isNotBlank(jsonObject.getString(yesterday))){
featureArr[1] = Long.valueOf(getValue(jsonObject.getString(yesterday)));
}
}
return featureArr;
}
private String getValue(String v) {
int index;
if (!StringUtils.isEmpty(v)) {
if ((index = v.indexOf("^")) > -1) {
return v.substring(0, index);
} else {
return v;
}
}else{
return "";
}
}
優化原則:
能用數組代替的map就用數組代替,不能用數組代替的map則按照預計大小初始化容量大小,或者不進行初始化操作。
其中SimpleDateFormat當初在優化的時候曾將其當成靜態變量放置在方法體外部,但是在測試的時候發現SimpleDateFormat是線程不安全的。所以就擱置在方法體內部。