1、RocketMQ ACL使用
? ???ACL全稱access control list,俗稱訪問控制列表。主要包括如下角色
- 用戶(用戶密碼)
- 資源(topic、消費)
- 權限(是否可以發送或者消費消息)
- 角色(是否為管理員,并可以配置是否可以進行更新或刪除主題和訂閱組)
1.1 Broker端開啟ACL驗證
? ???首先Broker.conf文件配置 aclEnable=true ,然后需要將 plain_acl.yml 放在 ${ROCKETMQ_HOME}/conf目錄, plain_acl.yml
globalWhiteRemoteAddresses: // 設置IP白名單
- 10.10.103.*
- 192.168.0.*
accounts: // 配置用戶信息
- accessKey: RocketMQ
secretKey: 12345678
whiteRemoteAddress: // 用戶級別的IP地址白名單
admin: false // 當為 true 可以執行更新、刪除主題或者訂閱組
defaultTopicPerm: DENY // DENY拒絕、SUB 訂閱權限、PUB 發送權限
defaultGroupPerm: SUB
topicPerms:
- topicA=DENY
- topicB=PUB|SUB
- topicC=SUB
groupPerms:
- groupA=DENY
- groupB=PUB|SUB
- groupC=SUB
1.2 服務端驗證
? ??? 服務端當配置好plain_acl.yml后并在 broker.conf中開啟 aclEnable=true ,服務端則會進行下面邏輯驗證
- 客戶端請求Ip和全局白名單匹配
- 請求是否包含用戶名并判斷用戶是否匹配
- 用戶級別的白名單
- 簽名驗證
- 該用戶是否具有admin權限
- 判斷admin配置了需要驗證的權限并進行驗證
2、源碼實現
2、1客戶端層面
? ???在構造函數添加 RPCHook ,進行創建ACL對象實例。
AclClientRPCHook aclClientRPCHook = new AclClientRPCHook(new SessionCredentials("rocketmq", "123456"))
DefaultMQProducer producer = new DefaultMQProducer("GID_test_class", aclClientRPCHook);
? ???發送消息前置執行鉤子函數并驗證ACL權限,若拋異常后則無法發送消息。
DefaultMQProducerImpl#executeSendMessageHookBefore
public void executeSendMessageHookBefore(final SendMessageContext context) {
if (!this.sendMessageHookList.isEmpty()) {
for (SendMessageHook hook : this.sendMessageHookList) {
try {
hook.sendMessageBefore(context);
} catch (Throwable e) {
log.warn("failed to executeSendMessageHookBefore", e);
}
}
}
}
2、2服務端層面
Broker端初始化ALC配置, 加載 AccessValidator配置
1. 核心是基于SPI機制,讀取META-INF/service/org.apache.rocketmq.acl.AccessValidator 訪問驗證器
2. 遍歷訪問驗證器,向Broker注冊鉤子函數。RPCHook在接受請求前進行處理請求
3. 調用AccessValidator#validate,驗證acl信息,如果擁有該執行權限則通過,否則報AclException
private void initialAcl() {
if (!this.brokerConfig.isAclEnable()) {
return;
}
List<AccessValidator> accessValidators = ServiceProvider.load(ServiceProvider.ACL_VALIDATOR_ID, AccessValidator.class){
for (AccessValidator accessValidator: accessValidators) {
final AccessValidator validator = accessValidator;
accessValidatorMap.put(validator.getClass(),validator);
this.registerServerRPCHook(new RPCHook() {
@Override
public void doBeforeRequest(String remoteAddr, RemotingCommand request) {
validator.validate(validator.parse(request, remoteAddr));
}
});
}
}
AccessValidator 是訪問驗證器接口,PlainAccessValidator是該接口的具體實現。
AccessResource parse(RemotingCommand request, String remoteAddr);
從遠端請求中解析本次請求對應的訪問資源
void validate(AccessResource accessResource);
根據本次需要訪問的權限,與請求用戶擁有的權限進行對比驗證,判斷是否擁有,如果沒有則ACLException
當遠端請求過來后,觸發鉤子函數RPCHook,調用 PlainAccessValidator#parse ,并根據 client 端創建 PlainAccessResource實例對象
PlainAccessResource
private String accessKey; // 訪問Key,用戶名
private String secretKey; // 用戶密碼
private String whiteRemoteAddress; // 遠程IP地址白名單
private boolean admin; // 是否是管理員角色
private byte defaultTopicPerm = 1; //默認topic訪問權限,如果沒有配置topic的權限,則Topic默認的訪問權限為1,表示為DENY
private byte defaultGroupPerm = 1; // 默認的消費組訪問權限,默認為DENY
private Map<String, Byte> resourcePermMap; // 資源需要的訪問權限映射表
private RemoteAddressStrategy remoteAddressStrategy; //遠程IP地址驗證
private int requestCode; //請求類型code
private byte[] content; // 請求內容
private String signature; // 簽名字符串,client端進行將請求參數排序,使用secretKey生成簽名字符串。服務端則驗證簽名
private String secretToken;
private String recognition;
- vPlainAccessValidator#parse,解析遠端請求過程 進行驗證并轉化為PlainAccessResource實例。
- 封裝PlainAccessResource對象實例,包括遠程訪問IP地址、requestCode、accessKey(請求用戶名)、簽名字符串(signature)、secretToken
根據請求命令,設置本次請求需要擁有的權限。 - 驗證簽名,根據擴展字段進行排序,便于生成簽名字符串,然后將擴展字段與請求體(body)寫入content字段。完成從請求頭中解析出本次請求需要驗證的權限。
PlainAccessValidator#parse
public AccessResource parse(RemotingCommand request, String remoteAddr) {
PlainAccessResource accessResource = new PlainAccessResource();
accessResource.setXXX(request.getExtFields().get(SessionCredentials.SECURITY_TOKEN));
try {
switch (request.getCode()) {
case RequestCode.SEND_MESSAGE:
accessResource.addResourceAndPerm(request.getExtFields().get("topic"), Permission.PUB);
break;
case RequestCode.SEND_MESSAGE_V2:
accessResource.addResourceAndPerm(request.getExtFields().get("b"), Permission.PUB);
break;
case RequestCode.CONSUMER_SEND_MSG_BACK:
accessResource.addResourceAndPerm(request.getExtFields().get("originTopic"), Permission.PUB);
accessResource.addResourceAndPerm(getRetryTopic(request.getExtFields().get("group")), Permission.SUB);
break;
case RequestCode.PULL_MESSAGE:
accessResource.addResourceAndPerm(request.getExtFields().get("topic"), Permission.SUB);
accessResource.addResourceAndPerm(getRetryTopic(request.getExtFields().get("consumerGroup")), Permission.SUB);
break;
case RequestCode.QUERY_MESSAGE:
accessResource.addResourceAndPerm(request.getExtFields().get("topic"), Permission.SUB);
break;
case RequestCode.HEART_BEAT:
HeartbeatData heartbeatData = HeartbeatData.decode(request.getBody(), HeartbeatData.class);
case RequestCode.UNREGISTER_CLIENT:
final UnregisterClientRequestHeader unregisterClientRequestHeader =
(UnregisterClientRequestHeader) request
.decodeCommandCustomHeader(UnregisterClientRequestHeader.class);
accessResource.addResourceAndPerm(getRetryTopic(unregisterClientRequestHeader.getConsumerGroup()), Permission.SUB);
break;
case RequestCode.GET_CONSUMER_LIST_BY_GROUP:
final GetConsumerListByGroupRequestHeader getConsumerListByGroupRequestHeader =
(GetConsumerListByGroupRequestHeader) request
.decodeCommandCustomHeader(GetConsumerListByGroupRequestHeader.class);
accessResource.addResourceAndPerm(getRetryTopic(getConsumerListByGroupRequestHeader.getConsumerGroup()), Permission.SUB);
break;
case RequestCode.UPDATE_CONSUMER_OFFSET:
final UpdateConsumerOffsetRequestHeader updateConsumerOffsetRequestHeader =
(UpdateConsumerOffsetRequestHeader) request
.decodeCommandCustomHeader(UpdateConsumerOffsetRequestHeader.class);
accessResource.addResourceAndPerm(getRetryTopic(updateConsumerOffsetRequestHeader.getConsumerGroup()), Permission.SUB);
accessResource.addResourceAndPerm(updateConsumerOffsetRequestHeader.getTopic(), Permission.SUB);
break;
default:
break;
}
}
SortedMap<String, String> map = new TreeMap<String, String>();
for (Map.Entry<String, String> entry : request.getExtFields().entrySet()) {
if (!SessionCredentials.SIGNATURE.equals(entry.getKey()) && !MixAll.UNIQUE_MSG_QUERY_FLAG.equals(entry.getKey())) {
map.put(entry.getKey(), entry.getValue());
}
}
accessResource.setContent(AclUtils.combineRequestContent(request, map));
return accessResource;
}
- 加載配置acl配置文件,可以運行時動態修改,最后加載到內存中
- PlainAccessValidator#validate -> PlainPermissionManager#validate
根據訪問的權限與Broker端配置的權限(plain_acl.yml)進行對比驗證,并驗證.
public PlainPermissionManager() {
load();
watch();
}
// 解析用戶配置的訪問資源,全局白名單,并加載到內存中
public void load() {
JSONObject plainAclConfData = AclUtils.getYamlDataObject(fileHome + File.separator + fileName, JSONObject.class);
if (globalWhiteRemoteAddressesList != null && !globalWhiteRemoteAddressesList.isEmpty()) {
for (int i = 0; i < globalWhiteRemoteAddressesList.size(); i++) {
globalWhiteRemoteAddressStrategy.add(remoteAddressStrategyFactory.
getRemoteAddressStrategy(globalWhiteRemoteAddressesList.getString(i)));
}
}
JSONArray accounts = plainAclConfData.getJSONArray(AclConstants.CONFIG_ACCOUNTS);
if (accounts != null && !accounts.isEmpty()) {
List<PlainAccessConfig> plainAccessConfigList = accounts.toJavaList(PlainAccessConfig.class);
for (PlainAccessConfig plainAccessConfig : plainAccessConfigList) {
PlainAccessResource plainAccessResource = buildPlainAccessResource(plainAccessConfig);
plainAccessResourceMap.put(plainAccessResource.getAccessKey(),plainAccessResource);
}
}
}
// 監聽器,默認以500ms的頻率判斷文件的內容是否變化(根據文件md5簽名進行對比),并重新加載配置文件。該方法啟動一個守護線程處理
private void watch() {
try {
String watchFilePath = fileHome + fileName;
FileWatchService fileWatchService = new FileWatchService(new String[] {watchFilePath}, new FileWatchService.Listener() {
@Override
public void onChanged(String path) {
load();
}
});
fileWatchService.start();
this.isWatchStart = true;
}
}
1、如果當前的請求命令屬于必須是Admin用戶才能訪問的權限,并且當前用戶并不是管理員角色,則拋出異常
2、遍歷需要權限與擁有的權限進行對比,如果配置對應的權限,則判斷是否匹配;如果未配置權限,則判斷默認權限時是否允許
public void validate(PlainAccessResource plainAccessResource) {
for (RemoteAddressStrategy remoteAddressStrategy : globalWhiteRemoteAddressStrategy) {
if (remoteAddressStrategy.match(plainAccessResource)) {
return;
}
}
PlainAccessResource ownedAccess = plainAccessResourceMap.get(plainAccessResource.getAccessKey());
if (ownedAccess.getRemoteAddressStrategy().match(plainAccessResource)) {
return;
}
String signature = AclUtils.calSignature(plainAccessResource.getContent(), ownedAccess.getSecretKey());
checkPerm(plainAccessResource, ownedAccess);
}
void checkPerm(PlainAccessResource needCheckedAccess, PlainAccessResource ownedAccess) {
if (Permission.needAdminPerm(needCheckedAccess.getRequestCode()) && !ownedAccess.isAdmin()) {
throw new AclException(String.format("Need admin permission for request code=%d, but accessKey=%s is not", needCheckedAccess.getRequestCode(), ownedAccess.getAccessKey()));
}
Map<String, Byte> needCheckedPermMap = needCheckedAccess.getResourcePermMap();
Map<String, Byte> ownedPermMap = ownedAccess.getResourcePermMap();
if (needCheckedPermMap == null) {
return;
}
if (ownedPermMap == null && ownedAccess.isAdmin()) {
return;
}
for (Map.Entry<String, Byte> needCheckedEntry : needCheckedPermMap.entrySet()) {
String resource = needCheckedEntry.getKey();
Byte neededPerm = needCheckedEntry.getValue();
boolean isGroup = PlainAccessResource.isRetryTopic(resource);
if (ownedPermMap == null || !ownedPermMap.containsKey(resource)) {
// Check the default perm
byte ownedPerm = isGroup ? ownedAccess.getDefaultGroupPerm() :
ownedAccess.getDefaultTopicPerm();
if (!Permission.checkPermission(neededPerm, ownedPerm)) {
throw new AclException(String.format("No default permission for %s", PlainAccessResource.printStr(resource, isGroup)));
}
continue;
}
if (!Permission.checkPermission(neededPerm, ownedPermMap.get(resource))) {
throw new AclException(String.format("No default permission for %s", PlainAccessResource.printStr(resource, isGroup)));
}
}
}