定義：
緩存，在我們日常開發中是必不可少的一種解決性能問題的方法。簡單的說，cache 就是為了提升系統性能而開辟的一塊內存空間。
作用：
緩存的主要作用是暫時在內存中保存業務系統的數據處理結果，并且等待下次訪問使用。在日常開發的很多場合，由于受限于硬盤IO的性能或者我們自身業務系統的數據處理和獲取可能非常費時，當我們發現我們的系統這個數據請求量很大的時候，頻繁的IO和頻繁的邏輯處理會導致硬盤和CPU資源的瓶頸出現。緩存的作用就是將這些來自不易的數據保存在內存中，當有其他線程或者客戶端需要查詢相同的數據資源時，直接從緩存的內存塊中返回數據，這樣不但可以提高系統的響應時間，同時也可以節省對這些數據的處理流程的資源消耗，整體上來說，系統性能會有大大的提升。
常用使用場景:
- 在高并發的數據庫訪問時，為了抗住數據庫并發連接壓力，將數據緩存起來，當有請求過來，直接返回數據；
- 當應用中的數據，更新周期較長，而且每次都查數據庫的情況下，可以采用周期更新數據，從而有效減少數據庫無效的訪問，保證效率；
衡量指標：
對于使用緩存來加速數據讀取的情況，一個很關鍵的指標是緩存命中率，因為如果緩存命中率比較低的話，就意味著還有不少的讀請求要回到數據庫中。
2. 介紹一種緩存機制
本節以JAVA重構時使用的緩存機制為例，講解我們日常工作中常用的一種比較高效的緩存機制。具體如下圖所示：
技術架構圖.png
以上緩存機制使用了一級緩存(Memcache) + 二級本地緩存(Guava)的二級緩存機制。
- 其中，Memcache是一個高性能的分布式的內存對象緩存系統，通過在內存里維護一個統一的巨大的hash表，它能夠用來存儲各種格式的數據，包括圖像、視頻、文件以及數據庫檢索的結果等。簡單的說就是將數據調用到內存中，然后從內存中讀取，從而大大提高讀取速度。
- Guava是一個全內存的本地緩存實現，它提供了線程安全的實現機制。能夠保證當多個請求獲取同一個key值時，只允許一個線程訪問服務器，其他線程阻塞等待(系統處理：直接返回，不用阻塞)，這樣就能很好的減輕服務端壓力；另外，比起服務端向Memcache獲取緩存，本地緩存減少了網絡IO開銷，提升數據返回的速度，減少了客戶端響應的時間。
當然，使用Guava作為本地緩存，需要滿足以下的適用場景：
- 愿意消耗一部分應用服務器的內存來提升速度；
- 預料某些值會被多次調用；
- 緩存數據不會超過內存總量；
那么，我們是否有疑問，反正，我這邊至少有2點疑問：
- 如何保證一級緩存和二級緩存的數據一致性；
- 如何保證一級緩存更新后，不同的服務器下的二級緩存都會進行同步更新；
基于上面的問題，又因為我們面對的是一個黑盒的緩存框架，現有的日志無法直接驗證這2點，那么直接上代碼：
//Memcache緩存更新，然后刪掉本地緩存
public void putCache(String key, Object value) {
long startTime = System.currentTimeMillis();
LOG.info("Start put cache, key = {}, value = {}", key, JSON.toJSONString(value));
ReentrantLock lock = this.getLock(key);
if(null != lock && lock.tryLock()) {
try {
CacheObject cacheObject = CacheObject.generateCacheObject(value);
this.memcachedClient.set(key, cacheObject.getUnixExpireTimeInSeconds(), cacheObject);
this.punishCacheRefreshEvent(JSON.toJSONString(Lists.newArrayList(new String[]{key})));
LOG.info("Put data to Memcached cost {}ms", Long.valueOf(System.currentTimeMillis() - startTime));
} finally {
lock.unlock();
}
}
}
//調zk刪本地緩存的方法
private void punishCacheRefreshEvent(String cacheKeyListStr) {
long startTime = System.currentTimeMillis();
this.curatorService.writeDataToNode("/localCache/event/delete", cacheKeyListStr);
LOG.info("Punish cache refresh event to ZK cost {}ms, keys = {}", Long.valueOf(System.currentTimeMillis() - startTime), cacheKeyListStr);
}
通過上面的代碼，可知更新緩存操作時，均有日志記錄。進入####-wrapper.log可進行驗證，上日志：
2017-06-15 10:54:03.120 [ItemListCacheTask-6-exe0] INFO MemcachedService - Punish cache refresh event to ZK cost 4ms, keys = ["master####_TAE_ITEM_LIST_BRAND_AREA_ID_2_174756"]
2017-06-15 10:54:03.120 [ItemListCacheTask-6-exe0] INFO MemcachedService - Put data to Memcached cost 5ms
2017-06-15 10:54:03.121 [pool-2-thread-1 ] INFO MemcachedService - Will delete caches : ["master####_TAE_ITEM_LIST_BRAND_AREA_ID_2_174756"]
2017-06-15 10:54:03.214 [ItemListCacheTask-6-exe0] INFO MemcachedService - Start put cache, key = master####_TAE_ITEM_LIST_BRAND_AREA_ID_2_174826, value = {"index_config":{"brandAreaButton":0,"brandAreaRedirectUrl":"","brandSpecialData":{"indexTopTitle":"測試我","style":"2"},"categorySwitch":1,"checkMaxVersion":"","checkMinVersion":"","couponSloganPicture":"/taobao/web_shopGuide591948637e93c_960_1280.jpg","curTab":1,"currentAndroidVersion":"","currentIosVersion":"","defaultChannelId":"","displayActivityName":1,"displayTab2Dot":0,"footerPicture":"/taobao/web_shopGuide5912d39b6305e_248_88.gif","globalPromotionData":{"image":"/taobao/web_shopGuide59194870cb96c_960_1280.jpg"},"historyDescript":"今天的新品都完了喲~\r\n看看之前錯過了什么吧！","indexTips":"每天百款新品，10點更新,10點","isIndex":1,"itemPromotionStyleData":{"autoIncrement":"1","data":[{"id":"1","image":"/taobao/web_shopGuide591007923bbfe_138_138.png","name":"測試"}]},"leftIcon":1,"listStyle":1,"loadingImage":"/taobao/web_shopGuide591409020acac_224_226.jpg","noneIndex":1,"pictureTag":{"eveningMall":"/taobao/web_shopGuide591007cb5fe2b_68_68.png","superBurst":"/taobao/web_shopGuide591007da338bb_68_68.png","timedSpike":"/taobao/web_shopGuide591007d2c8544_68_68.png","today":"/taobao/web_shopGuide59194893e88b4_68_68.png"},"playTimes":12345667,"redirectCart":0,"refreshText":"每天10點，有新鮮貨出沒\r\n每天20點，有新鮮貨出沒\r\n每天30點，有新鮮貨出沒\r\n全場低折，你要的都在這\r\n達人推薦，買什么都知道\r\n柚幣福利，驚喜不停\r\n柚姨幫你砍價啦~","shareSwitch":0,"sloganPicture":"/taobao/web_shopGuide5923f38252a22_750_350.jpg","tab1Title":"","tab2Title":"","useTbkSdk":1},"next_brand_area_data":{"1":{"tae_item":{"activityId":178382,"areaStyle":1,"brandAreaId":174560,"customTag":"","description":"","endAt":1513735200000,"isCoin":0,"isShowHot":1,"keplerSaleItemCount":1,"lowDiscount":0.0,"maxVersion":"","minVersion":"","name":"7號最后瘋搶","oneItemPic":"","orderCount":0,"ordinal":23,"picture":"","platform":"","promotionText":"","saleItemCount":49,"sortType":0,"tagIds":""},"common_item":{"$ref":"$.next_brand_area_data.1.tae_item"}},"2":{"tae_item":{},"common_item":{"activityId":170517,"areaStyle":1,"brandAreaId":166847,"customTag":"","description":"","endAt":1524103200000,"isCoin":0,"isShowHot":1,"keplerSaleItemCount":2,"lowDiscount":0.0,"maxVersion":"","minVersion":"","name":"京東用品尿褲櫥窗","oneItemPic":"","orderCount":0,"ordinal":0,"picture":"","platform":"","promotionText":"","saleItemCount":2,"sortType":0,"tagIds":""}}},"item_list":[],"brand_area_info":{"appId":2,"areaTimerType":1,"bannerPictureArray":[{"brandAreaId":174826,"createdAt":1496217409000,"height":"476","id":13816,"linkType":10017,"linkValue":"","type":1,"url":"/taobao/web_brand_area592e76550b369_361_476.jpg","width":"361"},{"brandAreaId":174826,"createdAt":1496217409000,"height":"440","id":13817,"linkType":0,"linkValue":"","type":1,"url":"/taobao/web_brand_area592e76640658a_1280_440.jpg","width":"1280"}],"bpLinkType":10017,"bpLinkValue":"","brandAreaType":1,"brandPicture":"/taobao/web_brand_area592e76550b369_361_476.jpg","description":"專場簡介：123","endAt":1504836000000,"id":174826,"isActive":1,"isCoin":0,"isShowHot":1,"isTop":0,"isZijian":0,"listStyle":2,"name":"淑錚專場2","oridinal":0,"payErrorMessage":"","promotionText":"促銷信息：促銷會顯示成什么樣子呢？","saleItemCount":0,"sortType":1,"startAt":1496217380000,"timerType":2,"topItems":"null","topTag":"頂部標簽1"}}, expireTimeInSeconds = 2592000
日志路徑：
[jumpserver@master-app-01 ####-####]$ pwd
/data/log/####-####
3. 問題定位
3.1 定位問題的步驟
工作中，經常會碰到運營后臺操作后或清除緩存后，接口數據未更新的情況。此時，我們需要掌握定位此類問題的技能，還是以JAVA端重構接口的緩存機制為例。
下圖展示了一種刪key類型的清緩存方式，一種_cache類型的更新緩存機制：
后臺-緩存.png
-
第一步：關注后臺發送的topic消息
操作后臺進行數據更新后，如上圖所示，針對2種類型的更新緩存模式，會發送MQ消息，即JAVA端訂閱的topic消息。日志路徑：
[jumpserver@master-app-01 ####-####]$ cd /data/web/log/
具體到以下2種類型，具體分析：
- 刪key類型的接口：日志為log下的refresh_cache文件夾下的info.log，如下：
[jumpserver@master-app-01 refresh_cache]$ pwd
/data/web/log/refresh_cache
[jumpserver@master-app-01 refresh_cache]$ ls
exception info.20170606.log info.20170612.log my_item_info refresh_by_methods tae_item_list_cache
info.20170601.log info.20170607.log info.20170613.log own_item_list_cache tae_active_item_list_cache
info.20170602.log info.20170608.log info.20170614.log phone_recharge tae_category_client_item_list_cache
info.20170605.log info.20170609.log info.20170615.log recommend_item_pool_cache tae_category_client_list_cache
- _cache類型的接口：針對具體的接口，日志放在不同的文件夾下，如tae_brand_list相關的首頁、專場等在refresh_brand_area文件夾下，專場分類列表在refresh_category_client_item_list文件夾下。以首頁專場為例，如下：
[jumpserver@master-app-01 refresh_brand_area]$ pwd
/data/web/log/refresh_brand_area
[jumpserver@master-app-01 refresh_brand_area]$ ls
info.20170511.log info.20170516.log info.20170521.log info.20170526.log info.20170531.log info.20170605.log info.20170610.log info.20170615.log
info.20170512.log info.20170517.log info.20170522.log info.20170527.log info.20170601.log info.20170606.log info.20170611.log
info.20170513.log info.20170518.log info.20170523.log info.20170528.log info.20170602.log info.20170607.log info.20170612.log
info.20170514.log info.20170519.log info.20170524.log info.20170529.log info.20170603.log info.20170608.log info.20170613.log
info.20170515.log info.20170520.log info.20170525.log info.20170530.log info.20170604.log info.20170609.log info.20170614.log
打開日志文件，后臺操作刪key后，打印日志如下：
info | 16687 | 1497496242.3234 | 2017:06:15 11:10:42 | 0.002588987350 | 成功發送消息: {"data":[{"v":1,"app_id":1,"app_key":"001","method":"new_user_special_item","clear_cache":1,"delete_key":true,"append_id":"174844"},{"v":1,"app_id":7,"app_key":"007","method":"new_user_special_item","clear_cache":1,"delete_key":true,"append_id":"174844"}],"time":"2017-06-15 11:10:42","cachebuildUrl":"http:\/\/test.cachebuild.master.youzibuy.com\/router\/rest","uuid":"REFRESH_CACHE__5941fab24ef424.48576463","cache_prefix":"master"}
-
第二步：關注JAVA端監聽到消息后，是否進行了處理
JAVA端的日志路徑：
/data/log/####-####
我們關注日志文件####-mq-listener.log即可。在接收到后臺發送的topic消息后，JAVA端會向Memcache系統發送相關的刪key or 更新key的請求。
[jumpserver@master-app-01 ####-####]$ tail -f ####-mq-listener.log
2017-06-15 11:10:42.322 [cacheDeleteContainer-1] WARN CacheDeleteCommandListener - Current instance is not leader, will not handle cache refresh request from mq.
2017-06-15 11:10:42.337 [cacheDeleteContainer-1] WARN AbstractListener - receieve magic message key ID:master-app-01-47896-1496894034686-3:10103:-1:1:397 content : {"data":[{"v":1,"app_id":1,"app_key":"001","method":"new_user_special_item","clear_cache":1,"delete_key":true,"append_id":"174842"},{"v":1,"app_id":7,"app_key":"007","method":"new_user_special_item","clear_cache":1,"delete_key":true,"append_id":"174842"}],"time":"2017-06-15 11:10:42","cachebuildUrl":"http:\/\/####.master.####.com\/router\/rest","uuid":"REFRESH_CACHE__5941fab24c4025.73699813","cache_prefix":"master"}
2017-06-15 11:10:42.338 [cacheDeleteContainer-1] WARN CacheDeleteCommandListener - #### del cache from mq
2017-06-15 11:10:42.338 [cacheDeleteContainer-1] WARN CacheDeleteCommandListener - Current instance is not leader, will not handle cache refresh request from mq.
-
第三步：緩存更新是否成功
該部分監控，我們可以借助MQ平臺：
http://...:**/admin/queues.jsp;jsessionid=gvurz20v706dev474xwsgd95
image.png
我們在確認上面第二部JAVA端進行了消息處理后，可關注Memcache系統有沒有更新緩存成功。具體的緩存類型，關注具體的緩存處理，其中：
- 刪key ——> refresh_cache_new_java
- _cache ——> refresh_cache_java
關注點：
- Number of Consumers，即消費消息的數量是否>0，如果等于0，那么JAVA端的緩存構建系統出現故障。
- Messages Enqueued和Messages Dequeued的數量分別+1，則表示緩存更新成功，若Messages Enqueued數量+1，而Messages Dequeued數量不變，則表示緩存更新失敗。
image.png
3.2 配置相關
從后臺發送topic消息，到JAVA端接收，二者必須保證發送的和接收的一致性。這樣才能夠最終刷緩存成功。
-
PHP端后臺配置
我們一般關注后臺發送的topic消息前綴，如操作主干后臺，通過日志可知前綴為master。
info | 26698 | 1497521442.5619 | 2017:06:15 18:10:42 | 0.008535861969 | 成功發送消息:
{"data":
[
{"v":1,"app_id":1,"app_key":"001","method":"new_user_special_item","clear_cache":1,"delete_key":true,"append_id":"174847"},{"v":1,"app_id":7,"app_key":"007","method":"new_user_special_item","clear_cache":1,"delete_key":true,"append_id":"174847"}
],
"time":"2017-06-1518:10:42",
"cachebuildUrl":"http:\/\/test.cachebuild.master.youzibuy.com\/router\/rest","uuid":"REFRESH_CACHE__59425d22892ac2.15914753",
"cache_prefix":"master"}
-
JAVA端配置
我們一般關注JAVA端接收消息的前綴，這個需要和后臺發送消息的前綴保持一致。JAVA端的配置平臺：http://dev.#####.####.com/modifyFile.html?configId=183
關注disconf-managed-config.properties下的緩存前綴，如下：
memcache.key.prifix=master
