1. 需求
- 實現文件上傳進度條展示
- 實現耗時異步任務完成消息通知
- 其他消息通知
2. 方案
文件上傳進度消息：
- 后臺使用commons-fileupload提供的功能，替代Spirng的文件解析器，注冊自定義監聽器，通過文件上傳監聽獲取當前Spring框架已經讀取的文件進度
- 服務模塊通過Feign接口向消息模塊發送文件上傳進度消息
- 消息模塊收到文件上傳進度消息，并通過WebSocket發送給文件上傳的用戶
- 客戶端收到進度，渲染上傳進度條
異步耗時任務完成消息：
- 創建自定義注解@SendMessage
- 在需要發送消息的方法上注解@SendMessage
- 創建消息通知切面類MessageAspect，對@SendMessage進行環繞切面
- 在方法前后通過Feign接口向消息模塊發送任務開始、結束消息
- 消息模塊收到開始、結束消息，通過WebSocket向瀏覽器發送消息
3. 方案對比
常見方案：
AJAX異步輪詢
優點：簡單好用
缺點：輪詢任務很多時效率較低，無法實現服務端通知WebSocket集群
WebSocket屬于全雙工通訊，與服務端建立會話后無法實現多個服務器間的會話共享，需要應用其他方案處理WebSocket集群問題。水平受限，暫未尋找到合適的集群方案，在此不做討論。
優點：支持大量用戶同時維持WebSocket通訊，服務可拓展集群實現高并發高可用單WebSocket消息模塊部署
這個是本案例中采用的方案，僅部署一個消息服務，該消息服務維護著所有與瀏覽器建立的WebSocket連接，其他模塊可以多服務部署，通過Feign接口向消息服務發送消息，消息服務將消息轉發給指定用戶，消息服務充當中間人角色。
優點：部署方便，可以實現服務端通知
缺點：單服務處理能力受限，不支持大量用戶，不適用于在線用戶多的互聯網應用
4. 文件上傳進度消息實現
4.1 引入依賴
<dependency>
<groupId>commons-io</groupId>
<artifactId>commons-io</artifactId>
<version>2.4</version>
</dependency>
<dependency>
<groupId>commons-fileupload</groupId>
<artifactId>commons-fileupload</artifactId>
<version>1.3.1</version>
</dependency>
- 編寫自定義文件上傳監聽器
- update方法為框架自行調用，因此避免性能問題應限制發送消息的次數
- update方法參數中pBytesRead pContentLength 均是當前Item，一次上傳多個文件時注意需要計算整個文件數量的百分比，但該百分比并不能反映真實進度，因為文件的大小不一致，僅能反映模擬的一個上傳進度。
- MessageDto為自定義消息實體，這個可以根據實際發送消息的格式進行自定義
package com.tba.sc.common.listener;
import com.tba.sc.common.dto.message.MessageDto;
import com.tba.sc.common.enums.EnumMessageType;
import com.tba.sc.common.feign.message.FeignMessageService;
import lombok.Data;
import lombok.extern.slf4j.Slf4j;
import org.apache.commons.fileupload.ProgressListener;
/**
* @author wangqichang
* @since 2020/4/9
*/
@Data
@Slf4j
public class RedisFileUploadProgressListener implements ProgressListener {
/**
* 上傳UUID
*/
private String uploadUUID;
private String taskName;
private int itemNum = 1;
private FeignMessageService messageService;
/**
* 已讀字節數
*/
private long megaBytes = -1;
public RedisFileUploadProgressListener(String uploadUUID, String taskName, Integer itemNum, FeignMessageService messageService) {
this.uploadUUID = uploadUUID;
this.taskName = taskName;
this.itemNum = itemNum;
this.messageService = messageService;
}
@Override
public void update(long pBytesRead, long pContentLength, int pItems) {
//避免性能問題，每讀取1M更新狀態
long mBytes = pBytesRead / 1000000;
if (megaBytes == mBytes) {
return;
}
megaBytes = mBytes;
Double doubleLength = new Double(pContentLength);
if (pContentLength > 0 && pItems > 0) {
Double ps = pBytesRead / doubleLength * 100 * pItems / itemNum;
log.info("文件上傳監聽：上傳UUID：{} 當前ITEM：{} 百分比：{}", uploadUUID, pItems, ps);
try {
messageService.send(MessageDto.builder().type(EnumMessageType.FILE_UPLOAD_PROCESS.getType()).msgId(uploadUUID).percentage(ps).message(taskName).build());
} catch (Exception e) {
log.error("調用Message模塊失敗，未能發送上傳百分比消息");
}
}
}
}
4. 2編寫自定義文件上傳解析器，封裝參數，注冊監聽
- 該解析器執行時，springmvc尚未封裝參數，因此如果監聽器必要參數需要獲取時，本例是由前端拼接URL參數，此處從URL中獲取必要參數
- cleanupMultipart方法在整個上傳方法結束后調用做清理工作，上傳文件后進行業務邏輯處理完畢后才會調用，并不是Controller獲取到文件后清理。
package com.tba.sc.common.config;
import com.tba.sc.common.feign.message.FeignMessageService;
import com.tba.sc.common.listener.RedisFileUploadProgressListener;
import org.apache.commons.fileupload.FileItem;
import org.apache.commons.fileupload.servlet.ServletFileUpload;
import org.springframework.web.multipart.MultipartHttpServletRequest;
import org.springframework.web.multipart.commons.CommonsMultipartResolver;
import javax.servlet.http.HttpServletRequest;
/**
* @author wangqichang
* @since 2020/4/10
*/
@Slf4j
public class MyCommonsMultipartResolver extends CommonsMultipartResolver {
RedisTemplate redisTemplate;
FeignMessageService feignMessageService;
public MyCommonsMultipartResolver(RedisTemplate redisTemplate, FeignMessageService feignMessageService) {
this.redisTemplate = redisTemplate;
this.feignMessageService = feignMessageService;
}
/**
* 注冊上傳監聽
*
* @param request
* @return
* @throws MultipartException
*/
@Override
protected MultipartParsingResult parseRequest(HttpServletRequest request) throws MultipartException {
//向request設置上傳文件ID
String uuid = IdUtil.fastUUID();
request.setAttribute(SystemConstants.MSG_ID_PARAM, uuid);
String encoding = determineEncoding(request);
FileUpload fileUpload = prepareFileUpload(encoding);
String queryString = request.getQueryString();
try {
RedisFileUploadProgressListener redisFileUploadProgressListener = null;
if (StrUtil.isNotBlank(queryString)) {
String[] split = queryString.split("&");
if (ArrayUtil.isNotEmpty(split) && split.length > 1) {
String[] param = split[0].split("=");
String[] itemParam = split[1].split("=");
//設置監聽
if (ArrayUtil.isNotEmpty(param) && param.length > 1 && SystemConstants.UPLOAD_TASK_NAME.equals(param[0])) {
String taskName = URLDecoder.decode(param[1], "UTF-8");
request.setAttribute(SystemConstants.UPLOAD_TASK_NAME, taskName);
Integer item = 1;
if (SystemConstants.UPLOAD_ITEM_NUM.equals(itemParam[0])) {
item = Integer.valueOf(itemParam[1]);
}
redisFileUploadProgressListener = new RedisFileUploadProgressListener(uuid, taskName, item, feignMessageService);
fileUpload.setProgressListener(redisFileUploadProgressListener);
}
}
}
List<FileItem> fileItems = ((ServletFileUpload) fileUpload).parseRequest(request);
return parseFileItems(fileItems, encoding);
} catch (FileUploadBase.SizeLimitExceededException ex) {
throw new MaxUploadSizeExceededException(fileUpload.getSizeMax(), ex);
} catch (FileUploadBase.FileSizeLimitExceededException ex) {
throw new MaxUploadSizeExceededException(fileUpload.getFileSizeMax(), ex);
} catch (FileUploadException ex) {
throw new MultipartException("Failed to parse multipart servlet request", ex);
} catch (Exception e) {
e.printStackTrace();
throw new RuntimeException(e.getMessage());
}
}
/**
* 上傳文件結束
* @param request
*/
@Override
public void cleanupMultipart(MultipartHttpServletRequest request) {
super.cleanupMultipart(request);
String uploadId = (String) request.getAttribute(SystemConstants.MSG_ID_PARAM);
String taskName = (String) request.getAttribute(SystemConstants.UPLOAD_TASK_NAME);
if (StrUtil.isNotBlank(taskName)) {
feignMessageService.send(MessageDto.builder().type(EnumMessageType.FILE_UPLOAD_PROCESS.getType()).msgId(uploadId).message(taskName + "任務文件上傳完成").percentage(100D).finalNotice(Boolean.TRUE).build());
}
}
}
4. 3 向spring容器中注入解析器
根據解析器構造，傳入必要參數。該解析器將替代默認實現
@Bean
MyCommonsMultipartResolver commonsMultipartResolver(RedisTemplate redisTemplate, FeignMessageService feignMessageService) {
return new MyCommonsMultipartResolver(redisTemplate,feignMessageService);
}
5 搭建消息服務模塊
5.1 核心依賴
spring為WebSocket提供了很好的支持，參照官方文檔即可完成服務搭建
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-websocket</artifactId>
</dependency>
5.2創建WebSocket配置類
繼承WebSocketMessageBrokerConfigurer類，重寫registerStompEndpoints() configureMessageBroker() configureClientInboundChannel()方法。
- registerStompEndpoints方法為注冊Stomp端點，暴露用于建立WebSocket的端點接口。其中DefaultHandshakeHandler為端口握手處理，重寫determineUser方法，name為當前WebSocket的唯一標識，本例中為用戶名（注意，需保證同一時間一個用戶只能在一個客戶端建立WebSocket連接）
- configureMessageBroker為配置消息代理，設置前綴及配置消息訂閱主題
- configureClientInboundChannel配置websocket權限，本例中使用stomp攜帶token標頭，實際上僅在建立連接時做判斷也是可以的
package com.tba.message.config;
import org.springframework.messaging.Message;
import org.springframework.messaging.simp.config.MessageBrokerRegistry;
import org.springframework.messaging.simp.stomp.StompCommand;
import org.springframework.messaging.support.ChannelInterceptor;
import org.springframework.web.socket.WebSocketHandler;
import org.springframework.web.socket.config.annotation.EnableWebSocketMessageBroker;
import org.springframework.web.socket.config.annotation.StompEndpointRegistry;
import org.springframework.web.socket.config.annotation.WebSocketMessageBrokerConfigurer;
import org.springframework.web.socket.server.support.DefaultHandshakeHandler;
import java.security.Principal;
/**
* STOMP over WebSocket support is available in the spring-messaging and spring-websocket modules. Once you have those dependencies, you can expose a STOMP endpoints, over WebSocket with SockJS Fallback, as the following example shows:
*
* @author wangqichang
* @since 2020/3/13
*/
@Slf4j
@Configuration
@EnableWebSocketMessageBroker
public class WebSocketConfig implements WebSocketMessageBrokerConfigurer {
@Autowired
RedisTemplate redisTemplate;
@Override
public void registerStompEndpoints(StompEndpointRegistry registry) {
/**
* is the HTTP URL for the endpoint to which a WebSocket (or SockJS) client needs to connect for the WebSocket handshake.
*/
registry
.addEndpoint("/ws")
.setHandshakeHandler(new DefaultHandshakeHandler() {
@Override
protected Principal determineUser(ServerHttpRequest request, WebSocketHandler wsHandler, Map<String, Object> attributes) {
return new UserPrincipal() {
@Override
public String getName() {
//使用了spring security框架時，框架將自動封裝Principal
// Principal principal = request.getPrincipal();
//根據自行權限框架，根據token自行封裝Principal
List<String> authToken = request.getHeaders().get(SystemConstants.TOKEN_HEADER);
if (CollUtil.isNotEmpty(authToken)) {
String token = authToken.get(0);
String redisTokenKey = RedisKeyConstants.TOKEN_PREFIX + token;
CurrentUser user = (CurrentUser) redisTemplate.opsForValue().get(redisTokenKey);
if (ObjectUtil.isNotNull(user)) {
return user.getUsername();
}
}
throw new ServiceException("無法注冊當前連接的用戶，請檢查是否攜帶用戶憑證");
}
};
}
})
.setAllowedOrigins("*")
.withSockJS();
}
@Override
public void configureMessageBroker(MessageBrokerRegistry config) {
/**
* STOMP messages whose destination header begins with /app are routed to @MessageMapping methods in @Controller classes.
* Use the built-in message broker for subscriptions and broadcasting and route messages whose destination header begins with /topic `or `/queue to the broker.
*/
config.setApplicationDestinationPrefixes("/app");
//topic 廣播主題消息 queue 一對一消息
config.enableSimpleBroker("/topic", "/queue");
}
/**
* 從stomp中獲取token標頭
*
* @param registration
*/
@Override
public void configureClientInboundChannel(ChannelRegistration registration) {
registration.interceptors(new ChannelInterceptor() {
@Override
public Message<?> preSend(Message<?> message, MessageChannel channel) {
StompHeaderAccessor accessor =
MessageHeaderAccessor.getAccessor(message, StompHeaderAccessor.class);
if (StompCommand.CONNECT.equals(accessor.getCommand())) {
List<String> nativeHeader = accessor.getNativeHeader(SystemConstants.TOKEN_HEADER);
String token = nativeHeader.get(0);
Assert.notNull(token, "未攜帶用戶憑證的請求");
//根據token從redis中獲取當前用戶
CurrentUser user = (CurrentUser) redisTemplate.opsForValue().get(RedisKeyConstants.TOKEN_PREFIX + token);
if (ObjectUtil.isNotNull(user)) {
String username = user.getUsername();
accessor.setUser(new com.tba.message.security.UserPrincipal(username));
return message;
}
throw new ServiceException("用戶憑證已過期");
}
return message;
}
});
}
}
5.3 編寫Controller，暴露發送消息的Restful接口
- 此接口暴露給其他服務調用，通過Message服務，向客戶端發送消息。Message服務相當于中間代理，因為客戶端僅與Message服務維持WebSocket連接
- 這個方法從線程變量中取出當前用戶username（線程變量中用戶信息為攔截器攔截token，查詢用戶并設置），向該用戶發送消息，歷史未結束消息放在redis緩存中，每次從redis中查詢該用戶歷史數據，通過msgId更新消息或者新增消息。最后一次提示消息發送成功則從list刪除，不進行歷史未結束消息的緩存。
package com.tba.message.controller;
import com.tba.sc.common.dto.message.MessageDto;
import com.tba.sc.common.user.CurrentUser;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.messaging.simp.SimpMessagingTemplate;
/**
* @author wangqichang
* @since 2020/4/10
*/
@Slf4j
@Controller
@RequestMapping("/msg")
@RestController
public class MsgController {
@Autowired
RedisTemplate redisTemplate;
@Autowired
private SimpMessagingTemplate simpMessagingTemplate;
@PostMapping(value = "/send")
public InvokeResult send(@RequestBody MessageDto message) {
//根據當前http請求中獲取用戶信息
CurrentUser current = UserContext.current();
Assert.notNull(current);
//從redis中獲取當前用戶的消息列表
List<MessageDto> list = (List<MessageDto>) redisTemplate.opsForValue().get(RedisKeyConstants.TOKEN_MSG + current.getToken());
if (ObjectUtil.isNull(list)) {
list = new ArrayList<>();
}
if (CollUtil.isNotEmpty(list) && StrUtil.isNotBlank(message.getMsgId())) {
for (int i = 0; i < list.size(); i++) {
//更新消息
if (message.getMsgId().equals(list.get(i).getMsgId())) {
list.set(i, message);
message.setCreateDate(list.get(i).getCreateDate());
}
}
} else {
//新增消息
list.add(message);
}
try {
this.simpMessagingTemplate.convertAndSendToUser(current.getUsername(), "/queue", list);
log.info("用戶：{} 消息數量：{} 發送新消息：{}", current.getRealname(), list.size(), message.toString());
//發送成功，刪除消息
if (message.isFinalNotice()) {
list.remove(message);
}
return InvokeResult.success();
} catch (Exception e) {
e.printStackTrace();
log.error(e.getMessage());
return InvokeResult.failure("消息發送失敗");
} finally {
//發送失敗，進緩存
redisTemplate.opsForValue().set(RedisKeyConstants.TOKEN_MSG + current.getToken(), list, 7, TimeUnit.DAYS);
}
}
}
5.4 暴露消息Feign接口
@FeignClient(name = "消息服務實例名稱", path = "/msg")
public interface FeignMessageService {
@PostMapping(value = "/send")
InvokeResult send(@RequestBody MessageDto message);
}
6 耗時任務消息發送
此處通過注解切面，在需要執行的方法前后想Message服務發送消息
6.1 自定義@SendMessage注解
@Target({ElementType.METHOD})
@Retention(RetentionPolicy.RUNTIME)
@Documented
public @interface SendMessage {
}
6.2 定義MessageAspect切面類
該切面將以@SendMessage注解為切入點，利用反射獲取形參名及參數值，封裝MessageDto，調用Feign接口向消息模塊發送消息
package com.tba.sc.common.advice;
import com.tba.sc.common.dto.message.MessageDto;
import com.tba.sc.common.enums.EnumMessageType;
import com.tba.sc.common.feign.message.FeignMessageService;
import org.aspectj.lang.ProceedingJoinPoint;
import org.aspectj.lang.annotation.Around;
import org.aspectj.lang.annotation.Aspect;
import org.aspectj.lang.annotation.Pointcut;
import org.aspectj.lang.reflect.MethodSignature;
import java.lang.reflect.Method;
import java.lang.reflect.Parameter;
/**
* @author wangqichang
* @since 2020/4/15
*/
@Slf4j
@Aspect
@Component
public class MessageAspect {
@Autowired
FeignMessageService feignMessageService;
@Around("@annotation(com.tba.sc.common.annotation.SendMessage)")
public Object BeforeMethod(ProceedingJoinPoint jp) throws Throwable {
MethodSignature methodSignature = (MethodSignature) jp.getSignature();
Method method = methodSignature.getMethod();
Object[] args = jp.getArgs();
//注意：該方法需指定編譯插件-parameters參數，否則無法獲取到形參名稱。配置在pom中maven-compiler-plugin
Parameter[] parameters = method.getParameters();
String taskName = null;
String taskId = null;
String url = null;
methodSignature.getParameterNames();
for (Parameter parameter : parameters) {
Integer index = (Integer) ReflectUtil.getFieldValue(parameter, "index");
if ("taskName".equals(parameter.getName()) && ArrayUtil.isNotEmpty(args)) {
taskName = (String) args[index];
} else if ("id".equals(parameter.getName())) {
taskId = (String) args[index];
} else if ("url".equals(parameter.getName())) {
url = (String) args[index];
}
}
log.info("taskName:{} id:{} url:{}", taskName, taskId, url);
if (StrUtil.isNotBlank(taskName)) {
String msgId = IdUtil.fastUUID();
MessageDto msg = MessageDto.builder()
.msgId(msgId)
.type(EnumMessageType.BUSINESS_NOTICE.getType())
.finalNotice(Boolean.FALSE)
.createDate(new Date())
.message(taskName + "任務開始")
.taskId(taskId)
.url(url)
.build();
try {
log.info("發送消息：{}", msg.toString());
feignMessageService.send(msg);
Object proceed = jp.proceed();
msg.setFinalNotice(Boolean.TRUE);
msg.setMessage(taskName + "任務完成");
msg.setSuccess(Boolean.TRUE);
log.info("發送消息：{}", msg.toString());
feignMessageService.send(msg);
return proceed;
} catch (Throwable throwable) {
msg.setFinalNotice(Boolean.TRUE);
msg.setMessage(taskName + "任務異常結束");
msg.setSuccess(Boolean.FALSE);
log.info("發送消息：{}", msg.toString());
feignMessageService.send(msg);
throw throwable;
}
}
log.info("未能獲取到任務名稱參數，未發送消息");
return jp.proceed();
}
}
6.2在所需接口上注解@SendMessage，并聲明形參
- 此處部分參數并未傳遞給Service，目的是為了切面類可以拿到形參及實參封裝消息實體
@PostMapping("/xxx")
@SendMessage
public InvokeResult xxx(String id, String taskName,String url) {
xxxService.xxx(id);
return InvokeResult.success();
}
7. 效果展示
文件上傳監聽日志，成功監聽上傳進度
2020-05-18 18:01:28.706 INFO 16744 --- [io-6001-exec-30] .t.s.c.l.RedisFileUploadProgressListener : 文件上傳監聽：上傳UUID：fdbdf76f-3421-436a-8614-837aa8fe7972 當前ITEM：1 百分比：93.90534762273536
2020-05-18 18:01:28.743 INFO 16744 --- [io-6001-exec-30] .t.s.c.l.RedisFileUploadProgressListener : 文件上傳監聽：上傳UUID：fdbdf76f-3421-436a-8614-837aa8fe7972 當前ITEM：1 百分比：96.51222534735146
2020-05-18 18:01:28.787 INFO 16744 --- [io-6001-exec-30] .t.s.c.l.RedisFileUploadProgressListener : 文件上傳監聽：上傳UUID：fdbdf76f-3421-436a-8614-837aa8fe7972 當前ITEM：1 百分比：99.11910307196756
文件上傳進度消息發送日志
2020-05-15 14:37:23.033 INFO 2924 --- [nio-9015-exec-9] c.tba.message.controller.MsgController : 用戶：超管 消息數量：1 發送新消息：MessageDto{msgId='3cce247c-5e67-46e1-9d18-3e4bc25cc1e4', type=2, taskId='null', message='11', percentage=99.57479978077085, finalNotice=false, success=false, createDate=null, url='null'}
2020-05-15 14:37:24.125 INFO 2924 --- [io-9015-exec-13] c.tba.message.controller.MsgController : 用戶：超管 消息數量：1 發送新消息：MessageDto{msgId='3cce247c-5e67-46e1-9d18-3e4bc25cc1e4', type=2, taskId='null', message='11', percentage=99.79995204151234, finalNotice=false, success=false, createDate=null, url='null'}
耗時任務消息模塊發送日志
2020-05-15 10:50:40.501 INFO 2924 --- [MessageBroker-5] o.s.w.s.c.WebSocketMessageBrokerStats : WebSocketSession[3 current WS(2)-HttpStream(0)-HttpPoll(1), 13 total, 0 closed abnormally (0 connect failure, 0 send limit, 0 transport error)], stompSubProtocol[processed CONNECT(11)-CONNECTED(10)-DISCONNECT(0)], stompBrokerRelay[null], inboundChannel[pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 120], outboundChannel[pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 43], sockJsScheduler[pool size = 8, active threads = 1, queued tasks = 4, completed tasks = 651]
2020-05-15 10:50:57.728 INFO 2924 --- [io-9015-exec-10] c.tba.message.controller.MsgController : 用戶：超管 消息數量：1 發送新消息：MessageDto{msgId='debbafbf-63a3-432e-8107-15cf03becebe', type=3, taskId='8afad4bd7202f283017212458b9d0111', message='測試002N3-N4任務開始', percentage=null, finalNotice=false, success=false, createDate=Fri May 15 18:50:57 CST 2020, url='/operation?type=2&id=8afad4bd7202f28301721245d3cf0112&taskName=%E6%B5%8B%E8%AF%95002&taskType=1'}
2020-05-15 10:51:06.304 INFO 2924 --- [io-9015-exec-11] c.tba.message.controller.MsgController : 用戶：超管 消息數量：1 發送新消息：MessageDto{msgId='debbafbf-63a3-432e-8107-15cf03becebe', type=3, taskId='8afad4bd7202f283017212458b9d0111', message='測試002N3-N4任務完成', percentage=null, finalNotice=true, success=true, createDate=Fri May 15 18:50:57 CST 2020, url='/operation?type=2&id=8afad4bd7202f28301721245d3cf0112&taskName=%E6%B5%8B%E8%AF%95002&taskType=1'}
前端消息渲染效果
image.png
大功告成！
尚有諸多缺點，但保證了基礎功能夠用，諸位大佬可以做個小參考。
