本文從CSDN上轉(zhuǎn)移過來(lái)：
http://blog.csdn.net/mounty_fsc/article/details/51085654

Caffe中，Blob，Layer，Net，Solver是最為核心的類，以下介紹這幾個(gè)類，Solver將在下一節(jié)介紹。

1 Blob

1.1 簡(jiǎn)介

Blob是：

1. 對(duì)待處理數(shù)據(jù)帶一層封裝，用于在Caffe中通信傳遞。
2. 也為CPU和GPU間提供同步能力
3. 數(shù)學(xué)上，是一個(gè)N維的C風(fēng)格的存儲(chǔ)數(shù)組
   總的來(lái)說(shuō)，Caffe使用Blob來(lái)交流數(shù)據(jù)，其是Caffe中標(biāo)準(zhǔn)的數(shù)組與統(tǒng)一的內(nèi)存接口，它是多功能的，在不同的應(yīng)用場(chǎng)景具有不同的含義，如可以是：batches of images, model parameters, and derivatives for optimization等。

1.2 源代碼

/** 
 * @brief A wrapper around SyncedMemory holders serving as the basic 
 *        computational unit through which Layer%s, Net%s, and Solver%s 
 *        interact. 
 * 
 * TODO(dox): more thorough description. 
 */  
template <typename Dtype>  
class Blob {  
 public:  
  Blob()  
       : data_(), diff_(), count_(0), capacity_(0) {}  
  
  /// @brief Deprecated; use <code>Blob(const vector<int>& shape)</code>.  
  explicit Blob(const int num, const int channels, const int height,  
      const int width);  
  explicit Blob(const vector<int>& shape);  
  
  .....  
  
 protected:  
  shared_ptr<SyncedMemory> data_;  
  shared_ptr<SyncedMemory> diff_;  
  shared_ptr<SyncedMemory> shape_data_;  
  vector<int> shape_;  
  int count_;  
  int capacity_;  
  
  DISABLE_COPY_AND_ASSIGN(Blob);  
};  // class Blob  

注：此處只保留了構(gòu)造函數(shù)與成員變量。

說(shuō)明：

1. Blob在實(shí)現(xiàn)上是對(duì)SyncedMemory（見1.4部分）進(jìn)行了一層封裝。
2. shape_為blob維度，見1.3部分
3. data_為原始數(shù)據(jù)
4. diff_為梯度信息
5. count為該blob的總?cè)萘浚磾?shù)據(jù)的size），函數(shù)count(x,y)（或count(x)）返回某個(gè)切片[x,y]（[x,end]）內(nèi)容量，本質(zhì)上就是shape[x]shape[x+1]....*shape[y]的值

1.3 Blob的shape

由源代碼中可以注意到Blob有個(gè)成員變量：vector<ini> shape_
其作用：

1. 對(duì)于圖像數(shù)據(jù)，shape可以定義為4維的數(shù)組(Num, Channels, Height, Width)或(n, k, h, w)，所以Blob數(shù)據(jù)維度為nkh*w，Blob是row-major保存的，因此在(n, k, h, w)位置的值物理位置為((n * K + k) * H + h) * W + w。其中Number是數(shù)據(jù)的batch size，對(duì)于256張圖片為一個(gè)training batch的ImageNet來(lái)說(shuō)n = 256;Channel是特征維度，如RGB圖像k = 3
2. 對(duì)于全連接網(wǎng)絡(luò)，使用2D blobs (shape (N, D))，然后調(diào)用InnerProductLayer
3. 對(duì)于參數(shù)，維度根據(jù)該層的類型和配置來(lái)確定。對(duì)于有3個(gè)輸入96個(gè)輸出的卷積層，F(xiàn)ilter核 11 x 11，則blob為96 x 3 x 11 x 11. 對(duì)于全連接層，1000個(gè)輸出，1024個(gè)輸入，則blob為1000 x 1024.

1.4 SyncedMemory

由1.2知，Blob本質(zhì)是對(duì)SyncedMemory的再封裝。其核心代碼如下：

/** 
 * @brief Manages memory allocation and synchronization between the host (CPU) 
 *        and device (GPU). 
 * 
 * TODO(dox): more thorough description. 
 */  
class SyncedMemory {  
 public:  
...  
 const void* cpu_data();  
  const void* gpu_data();  
  void* mutable_cpu_data();  
  void* mutable_gpu_data();  
...  
 private:  
...  
  void* cpu_ptr_;  
  void* gpu_ptr_;  
...  
};  // class SyncedMemory  

Blob同時(shí)保存了data_和diff_,其類型為SyncedMemory的指針。
對(duì)于data_(diff_相同),其實(shí)際值要么存儲(chǔ)在CPU(cpu_ptr_)要么存儲(chǔ)在GPU(gpu_ptr_),有兩種方式訪問CPU數(shù)據(jù)(GPU相同)：

1. 常量方式,void* cpu_data()，其不改變cpu_ptr_指向存儲(chǔ)區(qū)域的值。

2. 可變方式，void* mutable_cpu_data()，其可改變cpu_ptr_指向存儲(chǔ)區(qū)值。
   以mutable_cpu_data()為例

   void* SyncedMemory::mutable_cpu_data() {
     to_cpu();
     head_ = HEAD_AT_CPU;
     return cpu_ptr_;
   }
   
   inline void SyncedMemory::to_cpu() {
     switch (head_) {
     case UNINITIALIZED:
       CaffeMallocHost(&cpu_ptr_, size_, &cpu_malloc_use_cuda_);
       caffe_memset(size_, 0, cpu_ptr_);
       head_ = HEAD_AT_CPU;
       own_cpu_data_ = true;
       break;
     case HEAD_AT_GPU:
   #ifndef CPU_ONLY
       if (cpu_ptr_ == NULL) {
         CaffeMallocHost(&cpu_ptr_, size_, &cpu_malloc_use_cuda_);
         own_cpu_data_ = true;
       }
       caffe_gpu_memcpy(size_, gpu_ptr_, cpu_ptr_);
       head_ = SYNCED;
   #else
       NO_GPU;
   #endif
       break;
     case HEAD_AT_CPU:
     case SYNCED:
       break;
     }
   }
   

說(shuō)明：

1. 經(jīng)驗(yàn)上來(lái)說(shuō)，如果不需要改變其值，則使用常量調(diào)用的方式，并且，不要在你對(duì)象中保存其指針。為何要這樣設(shè)計(jì)呢，因?yàn)檫@樣涉及能夠隱藏CPU到GPU的同步細(xì)節(jié)，以及減少數(shù)據(jù)傳遞從而提高效率，當(dāng)你調(diào)用它們的時(shí)候，SyncedMem會(huì)決定何時(shí)去復(fù)制數(shù)據(jù)，通常情況是僅當(dāng)gnu或cpu修改后有復(fù)制操作，引用1官方文檔中有一個(gè)例子說(shuō)明何時(shí)進(jìn)行復(fù)制操作。
2. 調(diào)用mutable_cpu_data()可以讓head轉(zhuǎn)移到cpu上
3. 第一次調(diào)用mutable_cpu_data()是UNINITIALIZED將執(zhí)行9到14行，將為cpu_ptr_分配host內(nèi)存
4. 若head從gpu轉(zhuǎn)移到cpu，將把數(shù)據(jù)從gpu復(fù)制到cpu中

2 Layer

2.1 簡(jiǎn)介

Layer是Caffe的基礎(chǔ)以及基本計(jì)算單元。Caffe十分強(qiáng)調(diào)網(wǎng)絡(luò)的層次性，可以說(shuō)，一個(gè)網(wǎng)絡(luò)的大部分功能都是以Layer的形式去展開的，如convolute,pooling,loss等等。
在創(chuàng)建一個(gè)Caffe模型的時(shí)候，也是以Layer為基礎(chǔ)進(jìn)行的，需按照src/caffe/proto/caffe.proto中定義的網(wǎng)絡(luò)及參數(shù)格式定義網(wǎng)絡(luò) prototxt文件（需了解google protocol buffer）

2.2 Layer與Blob的關(guān)系

如圖，名為conv1的Layer 的輸入是名為data的bottom blob，其輸出是名為conv1的top blob。

其protobuff定義如下，一個(gè)layer有一個(gè)到多個(gè)的top和bottom，其對(duì)應(yīng)于blob

layer {  
      name: "conv1"  
      type: "Convolution"  
      bottom: "data"  
      top: "conv1"  
     ....  
    }  

2.3 源代碼

 /** 
     * Layer%s must implement a Forward function, in which they take their input 
     * (bottom) Blob%s (if any) and compute their output Blob%s (if any). 
     * They may also implement a Backward function, in which they compute the error 
     * gradients with respect to their input Blob%s, given the error gradients with 
     * their output Blob%s. 
     */  
    template <typename Dtype>  
    class Layer {  
     public:  
      /** 
       * You should not implement your own constructor. Any set up code should go 
       * to SetUp(), where the dimensions of the bottom blobs are provided to the 
       * layer. 
       */  
      explicit Layer(const LayerParameter& param)  
        : layer_param_(param), is_shared_(false) {  
    ...  
        }  
      virtual ~Layer() {}  
      
      /** 
       * @brief Implements common layer setup functionality. 
       * @param bottom the preshaped input blobs 
       * @param top 
       *     the allocated but unshaped output blobs, to be shaped by Reshape 
       */  
      void SetUp(const vector<Blob<Dtype>*>& bottom,  
          const vector<Blob<Dtype>*>& top) {  
    ...  
      }  
      
      ...  
      
      /** 
       * @brief Given the bottom blobs, compute the top blobs and the loss. 
       * \return The total loss from the layer. 
       * 
       * The Forward wrapper calls the relevant device wrapper function 
       * (Forward_cpu or Forward_gpu) to compute the top blob values given the 
       * bottom blobs.  If the layer has any non-zero loss_weights, the wrapper 
       * then computes and returns the loss.
       * 
       * Your layer should implement Forward_cpu and (optionally) Forward_gpu. 
       */  
      inline Dtype Forward(const vector<Blob<Dtype>*>& bottom,  
          const vector<Blob<Dtype>*>& top);  
      
      /** 
       * @brief Given the top blob error gradients, compute the bottom blob error 
       *        gradients. 
       * 
       * @param top 
       *     the output blobs, whose diff fields store the gradient of the error 
       *     with respect to themselves 
       * @param propagate_down 
       *     a vector with equal length to bottom, with each index indicating 
       *     whether to propagate the error gradients down to the bottom blob at 
       *     the corresponding index 
       * @param bottom 
       *     the input blobs, whose diff fields will store the gradient of the error 
       *     with respect to themselves after Backward is run 
       * 
       * The Backward wrapper calls the relevant device wrapper function 
       * (Backward_cpu or Backward_gpu) to compute the bottom blob diffs given the 
       * top blob diffs. 
       * 
       * Your layer should implement Backward_cpu and (optionally) Backward_gpu. 
       */  
      inline void Backward(const vector<Blob<Dtype>*>& top,  
          const vector<bool>& propagate_down,  
          const vector<Blob<Dtype>*>& bottom);  
      
     ...  
      
     protected:  
      /** The protobuf that stores the layer parameters */  
      LayerParameter layer_param_;  
      /** The phase: TRAIN or TEST */  
      Phase phase_;  
      /** The vector that stores the learnable parameters as a set of blobs. */  
      vector<shared_ptr<Blob<Dtype> > > blobs_;  
      /** Vector indicating whether to compute the diff of each param blob. */  
      vector<bool> param_propagate_down_;  
      
      /** The vector that indicates whether each top blob has a non-zero weight in 
       *  the objective function. */  
      vector<Dtype> loss_;  
      
      virtual void Forward_cpu(const vector<Blob<Dtype>*>& bottom,  
          const vector<Blob<Dtype>*>& top) = 0;  
      
      virtual void Forward_gpu(const vector<Blob<Dtype>*>& bottom,  
          const vector<Blob<Dtype>*>& top) {  
        // LOG(WARNING) << "Using CPU code as backup.";  
        return Forward_cpu(bottom, top);  
      }  
      
      virtual void Backward_cpu(const vector<Blob<Dtype>*>& top,  
          const vector<bool>& propagate_down,  
          const vector<Blob<Dtype>*>& bottom) = 0;  
      
      virtual void Backward_gpu(const vector<Blob<Dtype>*>& top,  
          const vector<bool>& propagate_down,  
          const vector<Blob<Dtype>*>& bottom) {  
        // LOG(WARNING) << "Using CPU code as backup.";  
        Backward_cpu(top, propagate_down, bottom);  
      }  
      
    ...  
      
     };  // class Layer  

說(shuō)明：每一層定義了三種操作

1. Setup：Layer的初始化
2. Forward：前向傳導(dǎo)計(jì)算，根據(jù)bottom計(jì)算top，調(diào)用了Forward_cpu（必須實(shí)現(xiàn)）和Forward_gpu（可選，若未實(shí)現(xiàn)，則調(diào)用cpu的）
3. Backward：反向傳導(dǎo)計(jì)算，根據(jù)top計(jì)算bottom的梯度，其他同上

2.4 派生類分類

在Layer的派生類中，主要可以分為Vision Layers

• Vision Layers
  Vison 層主要用于處理視覺圖像相關(guān)的層，以圖像作為輸入，產(chǎn)生其他的圖像。其主要特點(diǎn)是具有空間結(jié)構(gòu)。
  包含Convolution(conv_layer.hpp)、Pooling(pooling_layer.hpp)、Local Response Normalization(LRN)(lrn_layer.hpp)、im2col等，注：老版本的Caffe有頭文件include/caffe/vision_layers.hpp，新版本中用include/caffe/layer/conv_layer.hpp等取代
• Loss Layers
  這些層產(chǎn)生loss，如Softmax(SoftmaxWithLoss)、Sum-of-Squares / Euclidean(EuclideanLoss)、Hinge / Margin(HingeLoss)、Sigmoid Cross-Entropy(SigmoidCrossEntropyLoss)、Infogain(InfogainLoss)、Accuracy and Top-k等
• Activation / Neuron Layers
  元素級(jí)別的運(yùn)算，運(yùn)算均為同址計(jì)算（in-place computation，返回值覆蓋原值而占用新的內(nèi)存）。如：ReLU / Rectified-Linear and Leaky-ReLU(ReLU)、Sigmoid(Sigmoid)、TanH / Hyperbolic Tangent(TanH)、Absolute Value(AbsVal)、Power(Power)、BNLL(BNLL)等
• Data Layers
  網(wǎng)絡(luò)的最底層，主要實(shí)現(xiàn)數(shù)據(jù)格式的轉(zhuǎn)換，如：Database(Data)、In-Memory(MemoryData)、HDF5 Input(HDF5Data)、HDF5 Output(HDF5Output)、Images(ImageData)、Windows(WindowData)、Dummy(DummyData)等
• Common Layers
  Caffe提供了單個(gè)層與多個(gè)層的連接。如：Inner Product(InnerProduct)、Splitting(Split)、Flattening(Flatten)、Reshape(Reshape)、Concatenation(Concat)、Slicing(Slice)、Elementwise(Eltwise)、Argmax(ArgMax)、Softmax(Softmax)、Mean-Variance Normalization(MVN)等

注，括號(hào)內(nèi)為L(zhǎng)ayer Type,沒有括號(hào)暫缺信息，詳細(xì)咱見引用2

3 Net

3.1 簡(jiǎn)介

一個(gè)Net由多個(gè)Layer組成。一個(gè)典型的網(wǎng)絡(luò)從data layer（從磁盤中載入數(shù)據(jù)）出發(fā)到loss layer結(jié)束。如圖是一個(gè)簡(jiǎn)單的邏輯回歸分類器。

<img width="100" height="50" src="http://upload-images.jianshu.io/upload_images/1867031-00b800dbcc1abe37?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240">
如下定義：

name: "LogReg"
layer {
  name: "mnist"
  type: "Data"
  top: "data"
  top: "label"
  data_param {
    source: "input_leveldb"
    batch_size: 64
  }
}
layer {
  name: "ip"
  type: "InnerProduct"
  bottom: "data"
  top: "ip"
  inner_product_param {
    num_output: 2
  }
}
layer {
  name: "loss"
  type: "SoftmaxWithLoss"
  bottom: "ip"
  bottom: "label"
  top: "loss"
}

3.2 源代碼

/**
 * @brief Connects Layer%s together into a directed acyclic graph (DAG)
 *        specified by a NetParameter.
 *
 * TODO(dox): more thorough description.
 */
template <typename Dtype>
class Net {
 public:
...
  /// @brief Initialize a network with a NetParameter.
  void Init(const NetParameter& param);
...

  const vector<Blob<Dtype>*>& Forward(const vector<Blob<Dtype>* > & bottom,
      Dtype* loss = NULL);
...
  /**
   * The network backward should take no input and output, since it solely
   * computes the gradient w.r.t the parameters, and the data has already been
   * provided during the forward pass.
   */
  void Backward();
  ...
  Dtype ForwardBackward(const vector<Blob<Dtype>* > & bottom) {
    Dtype loss;
    Forward(bottom, &loss);
    Backward();
    return loss;
  }
...

 protected:
  ...
  /// @brief The network name
  string name_;
  /// @brief The phase: TRAIN or TEST
  Phase phase_;
  /// @brief Individual layers in the net
  vector<shared_ptr<Layer<Dtype> > > layers_;
  /// @brief the blobs storing intermediate results between the layer.
  vector<shared_ptr<Blob<Dtype> > > blobs_;
  vector<vector<Blob<Dtype>*> > bottom_vecs_;
  vector<vector<Blob<Dtype>*> > top_vecs_;
  ...
  /// The root net that actually holds the shared layers in data parallelism
  const Net* const root_net_;
};
}  // namespace caffe

說(shuō)明：

1. Init中，通過創(chuàng)建blob和layer搭建了整個(gè)網(wǎng)絡(luò)框架，以及調(diào)用各層的SetUp函數(shù)。
2. blobs_存放這每一層產(chǎn)生的blobls的中間結(jié)果，bottom_vecs_存放每一層的bottom blobs，top_vecs_存放每一層的top blobs

參考文獻(xiàn)：
[1].http://caffe.berkeleyvision.org/tutorial/net_layer_blob.html
[2].http://caffe.berkeleyvision.org/tutorial/layers.html
[3].https://yufeigan.github.io
[4].https://www.zhihu.com/question/27982282