• 先看官網(wǎng)一張圖

  
  
  image.png
  
  

  這就是dubbo的集群設(shè)計(jì)了，本章主要解析的就是圖中的主要幾個(gè)藍(lán)色點(diǎn)，簡(jiǎn)單堆土做個(gè)說(shuō)明：

• Cluster對(duì)于dubbo集群整個(gè)管控，會(huì)有各種方案，比如快速失敗、安全失敗等

• Directory在consumer章節(jié)中就已經(jīng)接觸過，主要維護(hù)的invoker的動(dòng)態(tài)管理

• Router一看名詞就是路由相關(guān)了

• LoadBalance講的就是負(fù)載均衡

  整體結(jié)合起來(lái)就是：consumer去遠(yuǎn)程調(diào)用一個(gè)具體的provider時(shí)，會(huì)通過集群中路由、負(fù)載均衡等策略選取最終一個(gè)具體的服務(wù)完成具體調(diào)用。

具體解析

一.Cluster

• 看下官網(wǎng)描述：

Cluster 將 Directory 中的多個(gè) Invoker 偽裝成一個(gè) Invoker，對(duì)上層透明，偽裝過程包含了容錯(cuò)邏輯，調(diào)用失敗后，重試另一個(gè)

• 具體接口

  /**
   * Cluster. (SPI, Singleton, ThreadSafe)
   * <p>
   * <a >Cluster</a>
   * <a >Fault-Tolerant</a>
   *
   * Cluster 將 Directory 中的多個(gè) Invoker 偽裝成一個(gè) Invoker，對(duì)上層透明，偽裝過程包含了容錯(cuò)邏輯，調(diào)用失敗后，重試另一個(gè)
   * 應(yīng)對(duì)出錯(cuò)情況采取的策略-9種實(shí)現(xiàn)
   */
  @SPI(FailoverCluster.NAME)
  public interface Cluster {
  
      /**
       * Merge the directory invokers to a virtual invoker.
       *
       * @param <T>
       * @param directory
       * @return cluster invoker
       * @throws RpcException
       */
      @Adaptive
      <T> Invoker<T> join(Directory<T> directory) throws RpcException;
  
  }
  

  很明顯默認(rèn)擴(kuò)展是FailoverCluster，里面就一個(gè)很熟悉的方法，join，在consumer中已經(jīng)出現(xiàn)過，那么跟蹤一下；

  
  
  image.png
  
  

  MockClusterInvoker里面構(gòu)造的是FailoverClusterInvoker，因此最終的invoker不斷調(diào)用下傳，繼續(xù):

  
  
  image.png

image.png

image.png

image.png

image.png

image.png

這個(gè)代碼無(wú)非就是將相關(guān)的consumer調(diào)用信息進(jìn)行構(gòu)造封裝，返回，但真正發(fā)揮作用的地方就是那個(gè)返回的Invoker: MockClusterInvoker-->FailoverClusterInvoker,為什么？因?yàn)檫@一步直接決定最終發(fā)起遠(yuǎn)程調(diào)用時(shí)所使用的ClusterInvoker，也就是如下的doInvoker方法：

• 先看MockClusterInvoker

  /**
   * 降級(jí)處理方案
   * 原理就是改變注冊(cè)在zookeeper上的節(jié)點(diǎn)信息.從而zookeeper通知重新生成invoker
   */
  @Override
  public Result invoke(Invocation invocation) throws RpcException {
      Result result = null;
  
      String value = directory.getUrl().getMethodParameter(invocation.getMethodName(), Constants.MOCK_KEY, Boolean.FALSE.toString()).trim();
      if (value.length() == 0 || value.equalsIgnoreCase("false")) {
          /**
           * 無(wú)降級(jí): no mock
           * 這里的invoker是FailoverClusterInvoker
           */
          result = this.invoker.invoke(invocation);
      } else if (value.startsWith("force")) {
          if (logger.isWarnEnabled()) {
              logger.info("force-mock: " + invocation.getMethodName() + " force-mock enabled , url : " + directory.getUrl());
          }
          /**
           * 屏蔽: force:direct mock
           * mock=force:return+null
           * 表示消費(fèi)方對(duì)方法的調(diào)用都直接返回null，不發(fā)起遠(yuǎn)程調(diào)用
           * 可用于屏蔽不重要服務(wù)不可用的時(shí)候，對(duì)調(diào)用方的影響
           */
          //
          result = doMockInvoke(invocation, null);
      } else {
          /**
           * 容錯(cuò): fail-mock
           * mock=fail:return+null
           * 表示消費(fèi)方對(duì)該服務(wù)的方法調(diào)用失敗后，再返回null，不拋異常
           * 可用于對(duì)不重要服務(wù)不穩(wěn)定的時(shí)候，忽略對(duì)調(diào)用方的影響 */
          try {
              result = this.invoker.invoke(invocation);
          } catch (RpcException e) {
              if (e.isBiz()) {
                  throw e;
              } else {
                  if (logger.isWarnEnabled()) {
                      logger.warn("fail-mock: " + invocation.getMethodName() + " fail-mock enabled , url : " + directory.getUrl(), e);
                  }
                  result = doMockInvoke(invocation, e);
              }
          }
      }
      return result;
  }
  

  這里出現(xiàn)了consumer配置項(xiàng)中的一個(gè)重要配置:mock;代碼邏輯很清楚，講的是容器的容錯(cuò)與降級(jí)方案。

• 繼續(xù)跟著看FailoverClusterInvoker

  @Override
  @SuppressWarnings({"unchecked", "rawtypes"})
  public Result doInvoke(Invocation invocation, final List<Invoker<T>> invokers, LoadBalance loadbalance) throws RpcException {
  
      // Invoker列表
      List<Invoker<T>> copyinvokers = invokers;
  
      //確認(rèn)下Invoker列表不為空
      checkInvokers(copyinvokers, invocation);
  
      //重試次數(shù)
      int len = getUrl().getMethodParameter(invocation.getMethodName(), Constants.RETRIES_KEY, Constants.DEFAULT_RETRIES) + 1;
      if (len <= 0) {
          len = 1;
      }
      // retry loop.
      RpcException le = null; // last exception.
      List<Invoker<T>> invoked = new ArrayList<Invoker<T>>(copyinvokers.size()); // invoked invokers.
      Set<String> providers = new HashSet<String>(len);
      for (int i = 0; i < len; i++) {
          //Reselect before retry to avoid a change of candidate `invokers`.
          //NOTE: if `invokers` changed, then `invoked` also lose accuracy.
          /**
           * 重試時(shí)，進(jìn)行重新選擇，避免重試時(shí)invoker列表已發(fā)生變化.
           * 注意：如果列表發(fā)生了變化，那么invoked判斷會(huì)失效，因?yàn)閕nvoker示例已經(jīng)改變
           */
          if (i > 0) {
              checkWhetherDestroyed();
              copyinvokers = list(invocation);
              // check again
              //重新檢查一下
              checkInvokers(copyinvokers, invocation);
          }
  
          /** 使用loadBalance選擇一個(gè)Invoker返回 */
          Invoker<T> invoker = select(loadbalance, invocation, copyinvokers, invoked);
          invoked.add(invoker);
          RpcContext.getContext().setInvokers((List) invoked);
          try {
  
              /** 使用選擇的結(jié)果Invoker進(jìn)行調(diào)用，返回結(jié)果 */
              Result result = invoker.invoke(invocation);
              if (le != null && logger.isWarnEnabled()) {
                  logger.warn("Although retry the method " + invocation.getMethodName()
                          + " in the service " + getInterface().getName()
                          + " was successful by the provider " + invoker.getUrl().getAddress()
                          + ", but there have been failed providers " + providers
                          + " (" + providers.size() + "/" + copyinvokers.size()
                          + ") from the registry " + directory.getUrl().getAddress()
                          + " on the consumer " + NetUtils.getLocalHost()
                          + " using the dubbo version " + Version.getVersion() + ". Last error is: "
                          + le.getMessage(), le);
              }
              return result;
          } catch (RpcException e) {
              if (e.isBiz()) { // biz exception.
                  throw e;
              }
              le = e;
          } catch (Throwable e) {
              le = new RpcException(e.getMessage(), e);
          } finally {
              providers.add(invoker.getUrl().getAddress());
          }
      }
      throw new RpcException(le != null ? le.getCode() : 0, "Failed to invoke the method "
              + invocation.getMethodName() + " in the service " + getInterface().getName()
              + ". Tried " + len + " times of the providers " + providers
              + " (" + providers.size() + "/" + copyinvokers.size()
              + ") from the registry " + directory.getUrl().getAddress()
              + " on the consumer " + NetUtils.getLocalHost() + " using the dubbo version "
              + Version.getVersion() + ". Last error is: "
              + (le != null ? le.getMessage() : ""), le != null && le.getCause() != null ? le.getCause() : le);
  }
  

  看看發(fā)起遠(yuǎn)程調(diào)用的debug情況:

  
  
  image.png
  
  

  恩 確實(shí)進(jìn)來(lái)了，既然FailoverCluster的策略是：失敗自動(dòng)切換，當(dāng)出現(xiàn)失敗，重試其它服務(wù)器，那么這個(gè)策略的體現(xiàn)邏輯就在這個(gè)doInvoker的for循環(huán)重試?yán)?/p>
  

  image.png
  
  len的取值就是配置項(xiàng)retries，即重試次數(shù)，默認(rèn)是3次；注意：重試時(shí)，進(jìn)行重新選擇，避免重試時(shí)invoker列表已發(fā)生變化.
  至于當(dāng)前invoker節(jié)點(diǎn)失敗后重試的機(jī)制如何，就是select如何再次選擇的問題了

  Invoker<T> invoker = select(loadbalance, invocation, copyinvokers, invoked);
  

  次實(shí)現(xiàn)在父類AbstractClusterInvoker中：

   /**
   *
   * 使用loadbalance選擇invoker.</br>
   * a)先lb選擇，如果在selected列表中 或者 不可用且做檢驗(yàn)時(shí)，進(jìn)入下一步(重選),否則直接返回</br>
   * b)重選驗(yàn)證規(guī)則：selected > available .保證重選出的結(jié)果盡量不在select中，并且是可用的
   *
   * @param selected 已選過的invoker.注意：輸入保證不重復(fù)
   *
   * Select a invoker using loadbalance policy.</br>
   * a)Firstly, select an invoker using loadbalance. If this invoker is in previously selected list, or, 
   * if this invoker is unavailable, then continue step b (reselect), otherwise return the first selected invoker</br>
   * b)Reslection, the validation rule for reselection: selected > available. This rule guarantees that
   * the selected invoker has the minimum chance to be one in the previously selected list, and also 
   * guarantees this invoker is available.
   *
   * @param loadbalance load balance policy
   * @param invocation
   * @param invokers invoker candidates
   * @param selected  exclude selected invokers or not
   * @return
   * @throws RpcException
   */
  protected Invoker<T> select(LoadBalance loadbalance, Invocation invocation, List<Invoker<T>> invokers, List<Invoker<T>> selected) throws RpcException {
      if (invokers == null || invokers.isEmpty()) return null;
  
      String methodName = invocation == null ? "" : invocation.getMethodName();
  
      // sticky，滯連接用于有狀態(tài)服務(wù)，盡可能讓客戶端總是向同一提供者發(fā)起調(diào)用，除非該提供者掛了，再連另一臺(tái)。
      boolean sticky = invokers.get(0).getUrl().getMethodParameter(methodName, Constants.CLUSTER_STICKY_KEY, Constants.DEFAULT_CLUSTER_STICKY);
      {
          //ignore overloaded method
          if (stickyInvoker != null && !invokers.contains(stickyInvoker)) {
              stickyInvoker = null;
          }
          //ignore concurrency problem
          if (sticky && stickyInvoker != null && (selected == null || !selected.contains(stickyInvoker))) {
              if (availablecheck && stickyInvoker.isAvailable()) {
                  return stickyInvoker;
              }
          }
      }
      Invoker<T> invoker = doSelect(loadbalance, invocation, invokers, selected);
  
      if (sticky) {
          stickyInvoker = invoker;
      }
      return invoker;
  }
  

  繼續(xù)看核心方法:doSelect

  private Invoker<T> doSelect(LoadBalance loadbalance, Invocation invocation, List<Invoker<T>> invokers, List<Invoker<T>> selected) throws RpcException {
  
      if (invokers == null || invokers.isEmpty()) return null;
  
      // 只有一個(gè)invoker，直接返回，不需要處理
      if (invokers.size() == 1)  return invokers.get(0);
  
      if (loadbalance == null) {
          loadbalance = ExtensionLoader.getExtensionLoader(LoadBalance.class).getExtension(Constants.DEFAULT_LOADBALANCE);
      }
  
      /** 通過具體的負(fù)載均衡的算法得到一個(gè)invoker,最后調(diào)用 */
      Invoker<T> invoker = loadbalance.select(invokers, getUrl(), invocation);
  
      //If the `invoker` is in the  `selected` or invoker is unavailable && availablecheck is true, reselect.
      /** 如果 selected中包含（優(yōu)先判斷） 或者 不可用&&availablecheck=true 則重試. */
      if ((selected != null && selected.contains(invoker)) || (!invoker.isAvailable() && getUrl() != null && availablecheck)) {
          try {
  
              /**
               * 重新選擇
               * 先從非selected的列表中選擇，沒有在從selected列表中選擇
               */
              Invoker<T> rinvoker = reselect(loadbalance, invocation, invokers, selected, availablecheck);
              if (rinvoker != null) {
                  invoker = rinvoker;
              } else {
                  //Check the index of current selected invoker, if it's not the last one, choose the one at index+1.
                  /** 看下第一次選的位置，如果不是最后，選+1位置. */
                  int index = invokers.indexOf(invoker);
                  try {
                      //Avoid collision
                      //最后在避免碰撞
                      invoker = index < invokers.size() - 1 ? invokers.get(index + 1) : invokers.get(0);
                  } catch (Exception e) {
                      logger.warn(e.getMessage() + " may because invokers list dynamic change, ignore.", e);
                  }
              }
          } catch (Throwable t) {
              logger.error("cluster reselect fail reason is :" + t.getMessage() + " if can not solve, you can set cluster.availablecheck=false in url", t);
          }
      }
      return invoker;
  }
  

  大致如下:

  • 通過具體的負(fù)載均衡的算法得到一個(gè)invoker(后面詳細(xì)說(shuō)負(fù)債均衡)
  • 如果 selected中包含（優(yōu)先判斷） 或者 不可用&&availablecheck=true 則重試

  這里有個(gè)重要的細(xì)節(jié)：sticky配置

  
  
  image.png
  
  

  看代碼就知道其作用：

  滯連接用于有狀態(tài)服務(wù)，盡可能讓客戶端總是向同一提供者發(fā)起調(diào)用，除非該提供者掛了，再連另一臺(tái)。

  FailoverCluster失敗重試策略就差不多講完了，大概回顧下:

• 選擇Cluster

• 決定ClusterInvoker

• 執(zhí)行doInvoker時(shí)實(shí)現(xiàn)具體策略

  這里不是很難，下面各種策略幾乎類似方式處理，就簡(jiǎn)單根據(jù)官網(wǎng)介紹下其實(shí)現(xiàn)的效果:

• FailfastCluster

  快速失敗，只發(fā)起一次調(diào)用，失敗立即報(bào)錯(cuò)。通常用于非冪等性的寫操作，比如新增記錄。

   public class FailfastClusterInvoker<T> extends AbstractClusterInvoker<T> {

   public FailfastClusterInvoker(Directory<T> directory) {
       super(directory);
   }

   @Override
   public Result doInvoke(Invocation invocation, List<Invoker<T>> invokers, LoadBalance loadbalance) throws RpcException {
       checkInvokers(invokers, invocation);
       Invoker<T> invoker = select(loadbalance, invocation, invokers, null);
       try {
           return invoker.invoke(invocation);
       } catch (Throwable e) {
           if (e instanceof RpcException && ((RpcException) e).isBiz()) { // biz exception.
               throw (RpcException) e;
           }
           throw new RpcException(e instanceof RpcException ? ((RpcException) e).getCode() : 0, "Failfast invoke providers " + invoker.getUrl() + " " + loadbalance.getClass().getSimpleName() + " select from all providers " + invokers + " for service " + getInterface().getName() + " method " + invocation.getMethodName() + " on consumer " + NetUtils.getLocalHost() + " use dubbo version " + Version.getVersion() + ", but no luck to perform the invocation. Last error is: " + e.getMessage(), e.getCause() != null ? e.getCause() : e);
       }
     }
   }

代碼邏輯一目倆然

• FailsafeCluster

  失敗安全，出現(xiàn)異常時(shí)，直接忽略。通常用于寫入審計(jì)日志等操作。

public class FailsafeClusterInvoker<T> extends AbstractClusterInvoker<T> {
       private static final Logger logger = LoggerFactory.getLogger(FailsafeClusterInvoker.class);
   
       public FailsafeClusterInvoker(Directory<T> directory) {
           super(directory);
       }
   
       @Override
       public Result doInvoke(Invocation invocation, List<Invoker<T>> invokers, LoadBalance loadbalance) throws RpcException {
           try {
               checkInvokers(invokers, invocation);
               Invoker<T> invoker = select(loadbalance, invocation, invokers, null);
               return invoker.invoke(invocation);
           } catch (Throwable e) {
               logger.error("Failsafe ignore exception: " + e.getMessage(), e);
               return new RpcResult(); // ignore
           }
       }
}   

• FailbackCluster

  失敗自動(dòng)恢復(fù)，后臺(tái)記錄失敗請(qǐng)求，定時(shí)重發(fā)。通常用于消息通知操作。

   public class FailbackClusterInvoker<T> extends AbstractClusterInvoker<T> {

       private static final Logger logger = LoggerFactory.getLogger(FailbackClusterInvoker.class);
   
       private static final long RETRY_FAILED_PERIOD = 5 * 1000;
   
       /**
        * Use {@link NamedInternalThreadFactory} to produce {@link com.alibaba.dubbo.common.threadlocal.InternalThread}
        * which with the use of {@link com.alibaba.dubbo.common.threadlocal.InternalThreadLocal} in {@link RpcContext}.
        */
       private final ScheduledExecutorService scheduledExecutorService = Executors.newScheduledThreadPool(2,
               new NamedInternalThreadFactory("failback-cluster-timer", true));
   
       private final ConcurrentMap<Invocation, AbstractClusterInvoker<?>> failed = new ConcurrentHashMap<Invocation, AbstractClusterInvoker<?>>();
       private volatile ScheduledFuture<?> retryFuture;
   
       public FailbackClusterInvoker(Directory<T> directory) {
           super(directory);
       }
   
       private void addFailed(Invocation invocation, AbstractClusterInvoker<?> router) {
           if (retryFuture == null) {
               synchronized (this) {
                   if (retryFuture == null) {
                       retryFuture = scheduledExecutorService.scheduleWithFixedDelay(new Runnable() {
   
                           @Override
                           public void run() {
                               // collect retry statistics
                               try {
                                   retryFailed();
                               } catch (Throwable t) { // Defensive fault tolerance
                                   logger.error("Unexpected error occur at collect statistic", t);
                               }
                           }
                       }, RETRY_FAILED_PERIOD, RETRY_FAILED_PERIOD, TimeUnit.MILLISECONDS);
                   }
               }
           }
           failed.put(invocation, router);
       }
   
       void retryFailed() {
           if (failed.size() == 0) {
               return;
           }
           for (Map.Entry<Invocation, AbstractClusterInvoker<?>> entry : new HashMap<Invocation, AbstractClusterInvoker<?>>(failed).entrySet()) {
               Invocation invocation = entry.getKey();
               Invoker<?> invoker = entry.getValue();
               try {
                   invoker.invoke(invocation);
                   failed.remove(invocation);
               } catch (Throwable e) {
                   logger.error("Failed retry to invoke method " + invocation.getMethodName() + ", waiting again.", e);
               }
           }
       }
   
       @Override
       protected Result doInvoke(Invocation invocation, List<Invoker<T>> invokers, LoadBalance loadbalance) throws RpcException {
           try {
               checkInvokers(invokers, invocation);
               Invoker<T> invoker = select(loadbalance, invocation, invokers, null);
               return invoker.invoke(invocation);
           } catch (Throwable e) {
               logger.error("Failback to invoke method " + invocation.getMethodName() + ", wait for retry in background. Ignored exception: " + e.getMessage() + ", ", e);
               addFailed(invocation, this);
               return new RpcResult(); // ignore
           }
       }
}

• ForkingCluster

  并行調(diào)用多個(gè)服務(wù)器，只要一個(gè)成功即返回。通常用于實(shí)時(shí)性要求較高的讀操作，但需要浪費(fèi)更多服務(wù)資源?？赏ㄟ^ forks="2" 來(lái)設(shè)置最大并行數(shù)。

public class ForkingClusterInvoker<T> extends AbstractClusterInvoker<T> {
       /**
        * Use {@link NamedInternalThreadFactory} to produce {@link com.alibaba.dubbo.common.threadlocal.InternalThread}
        * which with the use of {@link com.alibaba.dubbo.common.threadlocal.InternalThreadLocal} in {@link RpcContext}.
        */
       private final ExecutorService executor = Executors.newCachedThreadPool(new NamedInternalThreadFactory("forking-cluster-timer", true));
   
       public ForkingClusterInvoker(Directory<T> directory) {
           super(directory);
       }
   
       @Override
       @SuppressWarnings({"unchecked", "rawtypes"})
       public Result doInvoke(final Invocation invocation, List<Invoker<T>> invokers, LoadBalance loadbalance) throws RpcException {
           checkInvokers(invokers, invocation);
           final List<Invoker<T>> selected;
           final int forks = getUrl().getParameter(Constants.FORKS_KEY, Constants.DEFAULT_FORKS);
           final int timeout = getUrl().getParameter(Constants.TIMEOUT_KEY, Constants.DEFAULT_TIMEOUT);
           if (forks <= 0 || forks >= invokers.size()) {
               selected = invokers;
           } else {
               selected = new ArrayList<Invoker<T>>();
               for (int i = 0; i < forks; i++) {
                   // TODO. Add some comment here, refer chinese version for more details.
                   Invoker<T> invoker = select(loadbalance, invocation, invokers, selected);
                   if (!selected.contains(invoker)) {//Avoid add the same invoker several times.
                       selected.add(invoker);
                   }
               }
           }
           RpcContext.getContext().setInvokers((List) selected);
           final AtomicInteger count = new AtomicInteger();
           final BlockingQueue<Object> ref = new LinkedBlockingQueue<Object>();
           for (final Invoker<T> invoker : selected) {
               executor.execute(new Runnable() {
                   @Override
                   public void run() {
                       try {
                           Result result = invoker.invoke(invocation);
                           ref.offer(result);
                       } catch (Throwable e) {
                           int value = count.incrementAndGet();
                           if (value >= selected.size()) {
                               ref.offer(e);
                           }
                       }
                   }
               });
           }
           try {
               Object ret = ref.poll(timeout, TimeUnit.MILLISECONDS);
               if (ret instanceof Throwable) {
                   Throwable e = (Throwable) ret;
                   throw new RpcException(e instanceof RpcException ? ((RpcException) e).getCode() : 0, "Failed to forking invoke provider " + selected + ", but no luck to perform the invocation. Last error is: " + e.getMessage(), e.getCause() != null ? e.getCause() : e);
               }
               return (Result) ret;
           } catch (InterruptedException e) {
               throw new RpcException("Failed to forking invoke provider " + selected + ", but no luck to perform the invocation. Last error is: " + e.getMessage(), e);
           }
       }
}

• BroadcastCluster

  廣播調(diào)用所有提供者，逐個(gè)調(diào)用，任意一臺(tái)報(bào)錯(cuò)則報(bào)錯(cuò)。通常用于通知所有提供者更新緩存或日志等本地資源信息。

   public class BroadcastClusterInvoker<T> extends AbstractClusterInvoker<T> {

       private static final Logger logger = LoggerFactory.getLogger(BroadcastClusterInvoker.class);
   
       public BroadcastClusterInvoker(Directory<T> directory) {
           super(directory);
       }
   
       @Override
       @SuppressWarnings({"unchecked", "rawtypes"})
       public Result doInvoke(final Invocation invocation, List<Invoker<T>> invokers, LoadBalance loadbalance) throws RpcException {
           checkInvokers(invokers, invocation);
           RpcContext.getContext().setInvokers((List) invokers);
           RpcException exception = null;
           Result result = null;
           for (Invoker<T> invoker : invokers) {
               try {
                   result = invoker.invoke(invocation);
               } catch (RpcException e) {
                   exception = e;
                   logger.warn(e.getMessage(), e);
               } catch (Throwable e) {
                   exception = new RpcException(e.getMessage(), e);
                   logger.warn(e.getMessage(), e);
               }
           }
           if (exception != null) {
               throw exception;
           }
           return result;
       }
}

2.LoadBalance

• 看下主接口

   /**
* 負(fù)載均衡-四種負(fù)載均衡策略
* LoadBalance. (SPI, Singleton, ThreadSafe)
* <p>
* <a >Load-Balancing</a>
*
* @see com.alibaba.dubbo.rpc.cluster.Cluster#join(Directory)
*/
@SPI(RandomLoadBalance.NAME)
public interface LoadBalance {

       /**
        * select one invoker in list.
        *
        * @param invokers   invokers.
        * @param url        refer url
        * @param invocation invocation.
        * @return selected invoker.
        */
       @Adaptive("loadbalance")
       <T> Invoker<T> select(List<Invoker<T>> invokers, URL url, Invocation invocation) throws RpcException;
}

默認(rèn)取的是RandomLoadBalance，那我們就以消費(fèi)流程去詳細(xì)解析下這個(gè)負(fù)債均衡策略。

• RandomLoadBalance
  既然是負(fù)債均衡，那就是發(fā)起遠(yuǎn)程調(diào)用時(shí)選擇provider服務(wù)時(shí)發(fā)揮作用，那我們從默認(rèn)的FailoverClusterInvoker.doInvoke進(jìn)入：

  
  
  image.png
  
  

  出現(xiàn)了loadbalance，那就繼續(xù)跟蹤

  
  
  image.png
  
  
  image.png
  
  因?yàn)槲冶镜鼐蛦⒘艘粋€(gè)provider，因此就無(wú)需走負(fù)債均衡了，直接返回，但這里如果provider大于1的話，看上面畫出的重點(diǎn):
  先找到AbstractLoadBalance的select方法:

@Override
   public <T> Invoker<T> select(List<Invoker<T>> invokers, URL url, Invocation invocation) {

       if (invokers == null || invokers.isEmpty()) return null;

       if (invokers.size() == 1) return invokers.get(0);

       // 進(jìn)行選擇，具體的子類實(shí)現(xiàn)，我們這里是RandomLoadBalance
       return doSelect(invokers, url, invocation);
   }

又是鉤子，具體就順其到子類了:

   /**
* random load balance.
* 默認(rèn)的策略
*
* 隨機(jī)，按權(quán)重設(shè)置隨機(jī)概率。
* 在一個(gè)截面上碰撞的概率高，但調(diào)用量越大分布越均勻，而且按概率使用權(quán)重后也比較均勻，有利于動(dòng)態(tài)調(diào)整提供者權(quán)重。
*
* 1.獲取invokers的個(gè)數(shù),并遍歷累加權(quán)重
* 2.若不為第0個(gè),則將當(dāng)前權(quán)重與上一個(gè)進(jìn)行比較,只要有一個(gè)不等則認(rèn)為不等,即:sameWeight=false
* 3.若總權(quán)重>0 且 sameWeight=false 按權(quán)重獲取隨機(jī)數(shù),根據(jù)隨機(jī)數(shù)合權(quán)重相減確定調(diào)用節(jié)點(diǎn)
* 4.sameWeight=true,則均等隨機(jī)調(diào)用
*
* eg:假設(shè)有四個(gè)集群節(jié)點(diǎn)A,B,C,D,對(duì)應(yīng)的權(quán)重分別是1,2,3,4,那么請(qǐng)求到A節(jié)點(diǎn)的概率就為1/(1+2+3+4) = 10%.B,C,D節(jié)點(diǎn)依次類推為20%,30%,40%.
*/
public class RandomLoadBalance extends AbstractLoadBalance {

   public static final String NAME = "random";

   private final Random random = new Random();

   @Override
   protected <T> Invoker<T> doSelect(List<Invoker<T>> invokers, URL url, Invocation invocation) {
       int length = invokers.size(); // Number of invokers 總個(gè)數(shù)
       int totalWeight = 0; // The sum of weights 總權(quán)重
       boolean sameWeight = true; // Every invoker has the same weight? 權(quán)重是否都一樣
       for (int i = 0; i < length; i++) {
           int weight = getWeight(invokers.get(i), invocation);
           totalWeight += weight; // Sum 累計(jì)總權(quán)重
           if (sameWeight && i > 0 && weight != getWeight(invokers.get(i - 1), invocation)) {
               sameWeight = false; // 計(jì)算所有權(quán)重是否都一樣
           }
       }

       // eg: 總權(quán)重為10(1+2+3+4),那么怎么做到按權(quán)重隨機(jī)呢?根據(jù)10隨機(jī)出一個(gè)整數(shù),假如為隨機(jī)出來(lái)的是2.然后依次和權(quán)重相減,比如2(隨機(jī)數(shù))-1(A的權(quán)重) = 1,然后1(上一步計(jì)算的結(jié)果)-2(B的權(quán)重) = -1,此時(shí)-1 < 0,那么則調(diào)用B,其他的以此類推
       if (totalWeight > 0 && !sameWeight) {
           // 如果權(quán)重不相同且權(quán)重大于0.則按總權(quán)重?cái)?shù)隨機(jī)
           // If (not every invoker has the same weight & at least one invoker's weight>0), select randomly based on totalWeight.
           int offset = random.nextInt(totalWeight);
           // 確定隨機(jī)值落在那個(gè)片段上
           // Return a invoker based on the random value.
           for (int i = 0; i < length; i++) {
               offset -= getWeight(invokers.get(i), invocation);
               if (offset < 0) {
                   return invokers.get(i);
               }
           }
       }
       // 如果權(quán)重相同或權(quán)重為0則均等隨機(jī)
       // If all invokers have the same weight value or totalWeight=0, return evenly.
       return invokers.get(random.nextInt(length));
   }
}

當(dāng)前策略的算法在注釋中很清楚了，這里不在細(xì)說(shuō)。其他三種負(fù)債均衡其實(shí)處理方式大致相同，簡(jiǎn)單列一下:

• RoundRobinLoadBalance

輪循，按公約后的權(quán)重設(shè)置輪循比率。

   @Override
   protected <T> Invoker<T> doSelect(List<Invoker<T>> invokers, URL url, Invocation invocation) {
       String key = invokers.get(0).getUrl().getServiceKey() + "." + invocation.getMethodName();
       int length = invokers.size(); // Number of invokers invokers的個(gè)數(shù)
       int maxWeight = 0; // The maximum weight // 最大權(quán)重
       int minWeight = Integer.MAX_VALUE; // The minimum weight 最小權(quán)重
       final LinkedHashMap<Invoker<T>, IntegerWrapper> invokerToWeightMap = new LinkedHashMap<Invoker<T>, IntegerWrapper>();
       int weightSum = 0;
       for (int i = 0; i < length; i++) {
           int weight = getWeight(invokers.get(i), invocation);
           maxWeight = Math.max(maxWeight, weight); // Choose the maximum weight  累計(jì)最大權(quán)重
           minWeight = Math.min(minWeight, weight); // Choose the minimum weight  累計(jì)最小權(quán)重
           if (weight > 0) {
               invokerToWeightMap.put(invokers.get(i), new IntegerWrapper(weight));
               weightSum += weight;
           }
       }
       AtomicPositiveInteger sequence = sequences.get(key);
       if (sequence == null) {
           sequences.putIfAbsent(key, new AtomicPositiveInteger());
           sequence = sequences.get(key);
       }
       int currentSequence = sequence.getAndIncrement();
       if (maxWeight > 0 && minWeight < maxWeight) {  // 如果權(quán)重不一樣
           int mod = currentSequence % weightSum;
           for (int i = 0; i < maxWeight; i++) {
               for (Map.Entry<Invoker<T>, IntegerWrapper> each : invokerToWeightMap.entrySet()) {
                   final Invoker<T> k = each.getKey();
                   final IntegerWrapper v = each.getValue();
                   if (mod == 0 && v.getValue() > 0) {
                       return k;
                   }
                   if (v.getValue() > 0) {
                       v.decrement();
                       mod--;
                   }
               }
           }
       }
       // Round robin 取模循環(huán)
       return invokers.get(currentSequence % length);
   }

• LeastActiveLoadBalance

最少活躍調(diào)用數(shù)，相同活躍數(shù)的隨機(jī)，活躍數(shù)指調(diào)用前后計(jì)數(shù)差。
使慢的提供者收到更少請(qǐng)求，因?yàn)樵铰奶峁┱叩恼{(diào)用前后計(jì)數(shù)差會(huì)越大

舉個(gè)實(shí)際的例子:
A請(qǐng)求接受一個(gè)請(qǐng)求時(shí)計(jì)數(shù)+1，請(qǐng)求完再-1；B請(qǐng)求接受一個(gè)請(qǐng)求時(shí)，計(jì)數(shù)+1,請(qǐng)求完計(jì)數(shù)-1；按照這種邏輯，如果請(qǐng)求中的節(jié)點(diǎn)肯定比沒有請(qǐng)求的計(jì)數(shù)低，因此找計(jì)數(shù)低的服務(wù)處理。場(chǎng)景就是:處理越慢的服務(wù)，計(jì)數(shù)越容易高，因此將后面請(qǐng)求分發(fā)給計(jì)數(shù)低的服務(wù)會(huì)更加友好。

   @Override
   protected <T> Invoker<T> doSelect(List<Invoker<T>> invokers, URL url, Invocation invocation) {
       int length = invokers.size(); // Number of invokers ,invoker總數(shù)
       int leastActive = -1; // The least active value of all invokers ,所有invoker的最小活躍數(shù)
       int leastCount = 0; // The number of invokers having the same least active value (leastActive)  擁有最小活躍數(shù)的Invoker是的個(gè)數(shù)
       int[] leastIndexs = new int[length]; // The index of invokers having the same least active value (leastActive)  擁有最小活躍數(shù)的Invoker的下標(biāo),也就是將最小活躍的invoker集中放入新數(shù)組,以便后續(xù)遍歷
       int totalWeight = 0; // The sum of weights  總權(quán)重
       int firstWeight = 0; // Initial value, used for comparision  初始權(quán)重,用于計(jì)算是否相同
       boolean sameWeight = true; // Every invoker has the same weight value?  是否所有invoker的權(quán)重都相同
       for (int i = 0; i < length; i++) {
           Invoker<T> invoker = invokers.get(i);
           int active = RpcStatus.getStatus(invoker.getUrl(), invocation.getMethodName()).getActive(); // Active number 活躍數(shù)
           int weight = invoker.getUrl().getMethodParameter(invocation.getMethodName(), Constants.WEIGHT_KEY, Constants.DEFAULT_WEIGHT); // Weight
           if (leastActive == -1 || active < leastActive) { // Restart, when find a invoker having smaller least active value.  如果發(fā)現(xiàn)更小的活躍數(shù)則重新開始
               leastActive = active; // Record the current least active value 記錄下最小的活躍數(shù)
               leastCount = 1; // Reset leastCount, count again based on current leastCount 重新統(tǒng)計(jì)最小活躍數(shù)的個(gè)數(shù)
               leastIndexs[0] = i; // Reset  重置小標(biāo)
               totalWeight = weight; // Reset
               firstWeight = weight; // Record the weight the first invoker 重置第一個(gè)權(quán)重
               sameWeight = true; // Reset, every invoker has the same weight value?  重置是否權(quán)重相同標(biāo)識(shí)
           } else if (active == leastActive) { // If current invoker's active value equals with leaseActive, then accumulating.  累計(jì)相同的最小活躍數(shù)
               leastIndexs[leastCount++] = i; // Record index number of this invoker  累計(jì)相同的最小活躍invoker的小標(biāo)
               totalWeight += weight; // Add this invoker's weight to totalWeight. 累加總權(quán)重
               // If every invoker has the same weight?  是否所有權(quán)重一樣
               if (sameWeight && i > 0
                       && weight != firstWeight) {
                   sameWeight = false;
               }
           }
       }
       // assert(leastCount > 0)
       if (leastCount == 1) {
           // 如果只有一個(gè)最小則直接返回
           // If we got exactly one invoker having the least active value, return this invoker directly.
           return invokers.get(leastIndexs[0]);
       }
       if (!sameWeight && totalWeight > 0) {
           // 如果權(quán)重不相同且總權(quán)重大于0,則按總權(quán)重隨機(jī)
           // If (not every invoker has the same weight & at least one invoker's weight>0), select randomly based on totalWeight.
           int offsetWeight = random.nextInt(totalWeight);
           // 按隨機(jī)數(shù)去值
           // Return a invoker based on the random value.
           for (int i = 0; i < leastCount; i++) {
               int leastIndex = leastIndexs[i];
               offsetWeight -= getWeight(invokers.get(leastIndex), invocation);
               if (offsetWeight <= 0)
                   return invokers.get(leastIndex);
           }
       }
       // 如果權(quán)重相同或總權(quán)重為0,則均等隨機(jī)
       // If all invokers have the same weight value or totalWeight=0, return evenly.
       return invokers.get(leastIndexs[random.nextInt(leastCount)]);
   }

• ConsistentHashLoadBalance

一致性 Hash，相同參數(shù)的請(qǐng)求總是發(fā)到同一提供者。當(dāng)某一臺(tái)提供者掛時(shí)，原本發(fā)往該提供者的請(qǐng)求，基于虛擬節(jié)點(diǎn)，平攤到其它提供者，不會(huì)引起劇烈變動(dòng)。

   protected <T> Invoker<T> doSelect(List<Invoker<T>> invokers, URL url, Invocation invocation) {
       String key = invokers.get(0).getUrl().getServiceKey() + "." + invocation.getMethodName();
       int identityHashCode = System.identityHashCode(invokers);
       ConsistentHashSelector<T> selector = (ConsistentHashSelector<T>) selectors.get(key);
       if (selector == null || selector.identityHashCode != identityHashCode) {
           selectors.put(key, new ConsistentHashSelector<T>(invokers, invocation.getMethodName(), identityHashCode));
           selector = (ConsistentHashSelector<T>) selectors.get(key);
       }
       return selector.select(invocation);
   }

具體相關(guān)算法:
http://en.wikipedia.org/wiki/Consistent_hashing

3.Router

• 請(qǐng)求被路由到哪個(gè)服務(wù)器，靠的就是路由啦，先看下主接口:

   /**
* Router. (SPI, Prototype, ThreadSafe)
* <p>
* <a >Routing</a>
*
* @see com.alibaba.dubbo.rpc.cluster.Cluster#join(Directory)
* @see com.alibaba.dubbo.rpc.cluster.Directory#list(Invocation)
*/
public interface Router extends Comparable<Router> {

       /**
        * get the router url.
        *
        * @return url
        */
       URL getUrl();
   
       /**
        * route.
        *
        * @param invokers
        * @param url        refer url
        * @param invocation
        * @return routed invokers
        * @throws RpcException
        */
       <T> List<Invoker<T>> route(List<Invoker<T>> invokers, URL url, Invocation invocation) throws RpcException;
}

核心方法已經(jīng)出現(xiàn)了。我們還是按照原有思路debug一下：

image.png

image.png

image.png

OK ，路由核心出現(xiàn)了，上面方法做了兩件事:
- 1.RegistryDirectory doList(invocation)將所有可用的invokers根據(jù)參數(shù)條件篩選出來(lái)；
- 2.根據(jù)路由規(guī)則，將directory中篩選出來(lái)的invokers進(jìn)行過濾，比如MockInvokersSelector將所有mock invokers過濾掉。

image.png

image.png

過濾出來(lái)的invokers再返回即完成路由操作。路由執(zhí)行大體流程就是如此，接下來(lái)列一下幾個(gè)路由策略：

• ScriptRouter

  腳本路由規(guī)則 支持 JDK 腳本引擎的所有腳本，比如：javascript, jruby, groovy 等，通過 type=javascript 參數(shù)設(shè)置腳本類型，缺省為 javascript。

  @Override
  @SuppressWarnings("unchecked")
  public <T> List<Invoker<T>> route(List<Invoker<T>> invokers, URL url, Invocation invocation) throws RpcException {
      try {
          List<Invoker<T>> invokersCopy = new ArrayList<Invoker<T>>(invokers);
          Compilable compilable = (Compilable) engine;
          Bindings bindings = engine.createBindings();
          bindings.put("invokers", invokersCopy);
          bindings.put("invocation", invocation);
          bindings.put("context", RpcContext.getContext());
          CompiledScript function = compilable.compile(rule);
          Object obj = function.eval(bindings);
          if (obj instanceof Invoker[]) {
              invokersCopy = Arrays.asList((Invoker<T>[]) obj);
          } else if (obj instanceof Object[]) {
              invokersCopy = new ArrayList<Invoker<T>>();
              for (Object inv : (Object[]) obj) {
                  invokersCopy.add((Invoker<T>) inv);
              }
          } else {
              invokersCopy = (List<Invoker<T>>) obj;
          }
          return invokersCopy;
      } catch (ScriptException e) {
          //fail then ignore rule .invokers.
          logger.error("route error , rule has been ignored. rule: " + rule + ", method:" + invocation.getMethodName() + ", url: " + RpcContext.getContext().getUrl(), e);
          return invokers;
      }
  }
  

• ConditionRouter

條件路由: 根據(jù)dubbo管理控制臺(tái)配置的路由規(guī)則來(lái)過濾相關(guān)的invoker,這里會(huì)實(shí)時(shí)觸發(fā)RegistryDirectory類的notify方法，通知本地重建invokers

```
@Override
public <T> List<Invoker<T>> route(List<Invoker<T>> invokers, URL url, Invocation invocation) throws RpcException {
    if (invokers == null || invokers.isEmpty()) {
        return invokers;
    }
    try {
        if (!matchWhen(url, invocation)) {
            return invokers;
        }
        List<Invoker<T>> result = new ArrayList<Invoker<T>>();
        if (thenCondition == null) {
            logger.warn("The current consumer in the service blacklist. consumer: " + NetUtils.getLocalHost() + ", service: " + url.getServiceKey());
            return result;
        }
        for (Invoker<T> invoker : invokers) {
            if (matchThen(invoker.getUrl(), url)) {
                result.add(invoker);
            }
        }
        if (!result.isEmpty()) {
            return result;
        } else if (force) {
            logger.warn("The route result is empty and force execute. consumer: " + NetUtils.getLocalHost() + ", service: " + url.getServiceKey() + ", router: " + url.getParameterAndDecoded(Constants.RULE_KEY));
            return result;
        }
    } catch (Throwable t) {
        logger.error("Failed to execute condition router rule: " + getUrl() + ", invokers: " + invokers + ", cause: " + t.getMessage(), t);
    }
    return invokers;
}
```

OK 路由基本就分析到這里；

4.Directory

• 這個(gè)在consumer中已經(jīng)分析過了，簡(jiǎn)單看看官網(wǎng)描述:

  Directory 代表多個(gè) Invoker，可以把它看成 List<Invoker> ,但與 List 不同的是，它的值可能是動(dòng)態(tài)變化的，比如注冊(cè)中心推送變更

   /**
* Directory. (SPI, Prototype, ThreadSafe)
* <p>
* <a >Directory Service</a>
*
* Directory 代表多個(gè) Invoker，可以把它看成 List<Invoker> ，但與 List 不同的是，它的值可能是動(dòng)態(tài)變化的，比如注冊(cè)中心推送變更
*
* @see com.alibaba.dubbo.rpc.cluster.Cluster#join(Directory)
*/
public interface Directory<T> extends Node {

       /**
        * get service type.
        *
        * @return service type.
        */
       Class<T> getInterface();
   
       /**
        * list invokers.
        *
        * @return invokers
        */
       List<Invoker<T>> list(Invocation invocation) throws RpcException;
}

而此處list方法的核心邏輯也是在分析Route中就已經(jīng)見過了，不在分析；
Directory能夠動(dòng)態(tài)根據(jù)注冊(cè)中心維護(hù)Invokers列表，是因?yàn)橄嚓P(guān)Listener在被notify之后會(huì)觸發(fā)methodInvokerMap和urlInvokerMap等緩存的相關(guān)變動(dòng)；最后在list方法中也就實(shí)時(shí)取出了最新的invokers；看下之前的流程就清楚了；

• StaticDirectory

構(gòu)造方法傳入invokers,因此這個(gè)Directory的invokers是不會(huì)動(dòng)態(tài)變化的,使用場(chǎng)景不多；

public StaticDirectory(List<Invoker<T>> invokers) {
       this(null, invokers, null);
   }

   public StaticDirectory(List<Invoker<T>> invokers, List<Router> routers) {
       this(null, invokers, routers);
   }

   public StaticDirectory(URL url, List<Invoker<T>> invokers) {
       this(url, invokers, null);
   }

   public StaticDirectory(URL url, List<Invoker<T>> invokers, List<Router> routers) {
       super(url == null && invokers != null && !invokers.isEmpty() ? invokers.get(0).getUrl() : url, routers);
       if (invokers == null || invokers.isEmpty())
           throw new IllegalArgumentException("invokers == null");
       this.invokers = invokers;
   }

• RegistryDirectory

根據(jù)注冊(cè)中心的推送變更，動(dòng)態(tài)維護(hù)invokers列表；

整個(gè)集群大致模塊就到這里。