介紹
先前的教程展示了一個(gè)簡(jiǎn)單的線性模型，對(duì)MNIST數(shù)據(jù)集中手寫數(shù)字的識(shí)別率達(dá)到了91%。
在這個(gè)教程中，我們會(huì)在TensorFlow中實(shí)現(xiàn)一個(gè)簡(jiǎn)單的卷積神經(jīng)網(wǎng)絡(luò)，它能達(dá)到大約99%的分類準(zhǔn)確率，如果你做了一些建議的練習(xí)，準(zhǔn)確率還可能更高。
卷積神經(jīng)網(wǎng)絡(luò)在一張輸入圖片上移動(dòng)一個(gè)小的濾波器。這意味著在遍歷整張圖像來識(shí)別模式時(shí)，要重復(fù)使用這些濾波器。這讓卷積神經(jīng)網(wǎng)絡(luò)在擁有相同數(shù)量的變量時(shí)比全連接網(wǎng)絡(luò)（Fully-Connected）更強(qiáng)大，也讓卷積神經(jīng)網(wǎng)絡(luò)訓(xùn)練得更快。
你應(yīng)該熟悉基本的線性代數(shù)、Python和Jupyter Notebook編輯器。如果你是TensorFlow新手，在本教程之前應(yīng)該先學(xué)習(xí)第一篇教程。
流程圖
下面的圖表直接顯示了之后實(shí)現(xiàn)的卷積神經(jīng)網(wǎng)絡(luò)中數(shù)據(jù)的傳遞。
from IPython.display import Image
Image('images/02_network_flowchart.png')
輸入圖像在第一層卷積層里使用權(quán)重過濾器處理。結(jié)果在16張新圖里，每張代表了卷積層里一個(gè)過濾器（的處理結(jié)果）。圖像經(jīng)過降采樣，分辨率從28x28減少到14x14。
16張小圖在第二個(gè)卷積層中處理。這16個(gè)通道以及這層輸出的每個(gè)通道都需要一個(gè)過濾權(quán)重。總共有36個(gè)輸出，所以在第二個(gè)卷積層有16 x 36 = 576個(gè)濾波器。輸出圖再一次降采樣到7x7個(gè)像素。
第二個(gè)卷積層的輸出是36張7x7像素的圖像。它們被轉(zhuǎn)換到一個(gè)長(zhǎng)為7 x 7 x 36 = 1764的向量中去，它作為一個(gè)有128個(gè)神經(jīng)元（或元素）的全連接網(wǎng)絡(luò)的輸入。這些又輸入到另一個(gè)有10個(gè)神經(jīng)元的全連接層中，每個(gè)神經(jīng)元代表一個(gè)類別，用來確定圖像的類別，即圖像上的數(shù)字。
卷積濾波一開始是隨機(jī)挑選的，因此分類也是隨機(jī)完成的。根據(jù)交叉熵（cross-entropy）來測(cè)量輸入圖預(yù)測(cè)值和真實(shí)類別間的錯(cuò)誤。然后優(yōu)化器用鏈?zhǔn)椒▌t自動(dòng)地將這個(gè)誤差在卷積網(wǎng)絡(luò)中傳遞，更新濾波權(quán)重來提升分類質(zhì)量。這個(gè)過程迭代了幾千次，直到分類誤差足夠低。
這些特定的濾波權(quán)重和中間圖像是一個(gè)優(yōu)化結(jié)果，和你執(zhí)行代碼所看到的可能會(huì)有所不同。
注意，這些在TensorFlow上的計(jì)算是在一部分圖像上執(zhí)行，而非單獨(dú)的一張圖，這使得計(jì)算更有效。也意味著在TensorFlow上實(shí)現(xiàn)時(shí)，這個(gè)流程圖實(shí)際上會(huì)有更多的數(shù)據(jù)維度。
卷積層
下面的圖片展示了在第一個(gè)卷積層中處理圖像的基本思想。輸入圖片描繪了數(shù)字7，這里顯示了它的四張拷貝，我們可以很清晰的看到濾波器是如何在圖像的不同位置移動(dòng)。在濾波器的每個(gè)位置上，計(jì)算濾波器以及濾波器下方圖像像素的點(diǎn)乘，得到輸出圖像的一個(gè)像素。因此，在整張輸入圖像上移動(dòng)時(shí)，會(huì)有一張新的圖像生成。
紅色的濾波權(quán)重表示濾波器對(duì)輸入圖的黑色像素有正響應(yīng)，藍(lán)色的代表有負(fù)響應(yīng)。
在這個(gè)例子中，很明顯這個(gè)濾波器識(shí)別數(shù)字7的水平線段，在輸出圖中可以看到它對(duì)線段的強(qiáng)烈響應(yīng)。
Image('images/02_convolution.png')
濾波器遍歷輸入圖的移動(dòng)步長(zhǎng)稱為stride。在水平和豎直方向各有一個(gè)stride。
在下面的源碼中，兩個(gè)方向的stride都設(shè)為1，這說明濾波器從輸入圖像的左上角開始，下一步移動(dòng)到右邊1個(gè)像素去。當(dāng)濾波器到達(dá)圖像的右邊時(shí)，它會(huì)返回最左邊，然后向下移動(dòng)1個(gè)像素。持續(xù)這個(gè)過程，直到濾波器到達(dá)輸入圖像的右下角，同時(shí)，也生成了整張輸出圖片。
當(dāng)濾波器到達(dá)輸入圖的右端或底部時(shí)，它會(huì)用零（白色像素）來填充。因?yàn)檩敵鰣D要和輸入圖一樣大。
此外，卷積層的輸出可能會(huì)傳遞給修正線性單元（ReLU），它用來保證輸出是正值，將負(fù)值置為零。輸出還會(huì)用最大池化(max-pooling)進(jìn)行降采樣，它使用了2x2的小窗口，只保留像素中的最大值。這讓輸入圖分辨率減小一半，比如從28x28到14x14。
第二個(gè)卷積層更加復(fù)雜，因?yàn)樗?6個(gè)輸入通道。我們想給每個(gè)通道一個(gè)單獨(dú)的濾波，因此需要16個(gè)。另外，我們想從第二個(gè)卷積層得到36個(gè)輸出，因此總共需要16 x 36 = 576個(gè)濾波器。要理解這些如何工作可能有些困難。
導(dǎo)入
%matplotlib inline
import matplotlib.pyplot as plt
import tensorflow as tf
import numpy as np
from sklearn.metrics import confusion_matrix
import time
from datetime import timedelta
import math
使用Python3.5（Anaconda）開發(fā)，TensorFlow版本是
tf.__version__
'1.0.0-rc0'
神經(jīng)網(wǎng)絡(luò)配置
方便起見，在這里定義神經(jīng)網(wǎng)絡(luò)的配置，你可以很容易找到或改變這些數(shù)值，然后重新運(yùn)行Notebook。
# Convolutional Layer 1.
filter_size1 = 5 # Convolution filters are 5 x 5 pixels.
num_filters1 = 16 # There are 16 of these filters.
# Convolutional Layer 2.
filter_size2 = 5 # Convolution filters are 5 x 5 pixels.
num_filters2 = 36 # There are 36 of these filters.
# Fully-connected layer.
fc_size = 128 # Number of neurons in fully-connected layer.
載入數(shù)據(jù)
MNIST數(shù)據(jù)集大約12MB，如果沒在文件夾中找到就會(huì)自動(dòng)下載。
from tensorflow.examples.tutorials.mnist import input_data
data = input_data.read_data_sets('data/MNIST/', one_hot=True)
Extracting data/MNIST/train-images-idx3-ubyte.gz
Extracting data/MNIST/train-labels-idx1-ubyte.gz
Extracting data/MNIST/t10k-images-idx3-ubyte.gz
Extracting data/MNIST/t10k-labels-idx1-ubyte.gz
現(xiàn)在已經(jīng)載入了MNIST數(shù)據(jù)集，它由70,000張圖像和對(duì)應(yīng)的標(biāo)簽（比如圖像的類別）組成。數(shù)據(jù)集分成三份互相獨(dú)立的子集。我們?cè)诮坛讨兄挥糜?xùn)練集和測(cè)試集。
print("Size of:")
print("- Training-set:\t\t{}".format(len(data.train.labels)))
print("- Test-set:\t\t{}".format(len(data.test.labels)))
print("- Validation-set:\t{}".format(len(data.validation.labels)))
Size of:
-Training-set: 55000
-Test-set: 10000
-Validation-set: 5000
類型標(biāo)簽使用One-Hot編碼，這意外每個(gè)標(biāo)簽是長(zhǎng)為10的向量，除了一個(gè)元素之外，其他的都為零。這個(gè)元素的索引就是類別的數(shù)字，即相應(yīng)圖片中畫的數(shù)字。我們也需要測(cè)試數(shù)據(jù)集類別數(shù)字的整型值，用下面的方法來計(jì)算。
data.test.cls = np.argmax(data.test.labels, axis=1)
數(shù)據(jù)維度
在下面的源碼中，有很多地方用到了數(shù)據(jù)維度。它們只在一個(gè)地方定義，因此我們可以在代碼中使用這些數(shù)字而不是直接寫數(shù)字。
# We know that MNIST images are 28 pixels in each dimension.
img_size = 28
# Images are stored in one-dimensional arrays of this length.
img_size_flat = img_size * img_size
# Tuple with height and width of images used to reshape arrays.
img_shape = (img_size, img_size)
# Number of colour channels for the images: 1 channel for gray-scale.
num_channels = 1
# Number of classes, one class for each of 10 digits.
num_classes = 10
用來繪制圖片的幫助函數(shù)
這個(gè)函數(shù)用來在3x3的柵格中畫9張圖像，然后在每張圖像下面寫出真實(shí)類別和預(yù)測(cè)類別。
def plot_images(images, cls_true, cls_pred=None):
assert len(images) == len(cls_true) == 9
# Create figure with 3x3 sub-plots.
fig, axes = plt.subplots(3, 3)
fig.subplots_adjust(hspace=0.3, wspace=0.3)
for i, ax in enumerate(axes.flat):
# Plot image.
ax.imshow(images[i].reshape(img_shape), cmap='binary')
# Show true and predicted classes.
if cls_pred is None:
xlabel = "True: {0}".format(cls_true[i])
else:
xlabel = "True: {0}, Pred: {1}".format(cls_true[i], cls_pred[i])
# Show the classes as the label on the x-axis.
ax.set_xlabel(xlabel)
# Remove ticks from the plot.
ax.set_xticks([])
ax.set_yticks([])
# Ensure the plot is shown correctly with multiple plots
# in a single Notebook cell.
plt.show()
繪制幾張圖像來看看數(shù)據(jù)是否正確
# Get the first images from the test-set.
images = data.test.images[0:9]
# Get the true classes for those images.
cls_true = data.test.cls[0:9]
# Plot the images and labels using our helper-function above.
plot_images(images=images, cls_true=cls_true)
TensorFlow圖
TensorFlow的全部目的就是使用一個(gè)稱之為計(jì)算圖（computational graph）的東西，它會(huì)比直接在Python中進(jìn)行相同計(jì)算量要高效得多。TensorFlow比Numpy更高效，因?yàn)門ensorFlow了解整個(gè)需要運(yùn)行的計(jì)算圖，然而Numpy只知道某個(gè)時(shí)間點(diǎn)上唯一的數(shù)學(xué)運(yùn)算。
TensorFlow也能夠自動(dòng)地計(jì)算需要優(yōu)化的變量的梯度，使得模型有更好的表現(xiàn)。這是由于圖是簡(jiǎn)單數(shù)學(xué)表達(dá)式的結(jié)合，因此整個(gè)圖的梯度可以用鏈?zhǔn)椒▌t推導(dǎo)出來。
TensorFlow還能利用多核CPU和GPU，Google也為TensorFlow制造了稱為TPUs（Tensor Processing Units）的特殊芯片，它比GPU更快。
一個(gè)TensorFlow圖由下面幾個(gè)部分組成，后面會(huì)詳細(xì)描述：
- 占位符變量（Placeholder）用來改變圖的輸入。
- 模型變量（Model）將會(huì)被優(yōu)化，使得模型表現(xiàn)得更好。
- 模型本質(zhì)上就是一些數(shù)學(xué)函數(shù)，它根據(jù)Placeholder和模型的輸入變量來計(jì)算一些輸出。
- 一個(gè)cost度量用來指導(dǎo)變量的優(yōu)化。
- 一個(gè)優(yōu)化策略會(huì)更新模型的變量。
另外，TensorFlow圖也包含了一些調(diào)試狀態(tài)，比如用TensorBoard打印log數(shù)據(jù)，本教程不涉及這些。
創(chuàng)建新變量的幫助函數(shù)
函數(shù)用來根據(jù)給定大小創(chuàng)建TensorFlow變量，并將它們用隨機(jī)值初始化。需注意的是在此時(shí)并未完成初始化工作，僅僅是在TensorFlow圖里定義它們。
def new_weights(shape):
return tf.Variable(tf.truncated_normal(shape, stddev=0.05))
def new_biases(length):
return tf.Variable(tf.constant(0.05, shape=[length]))
創(chuàng)建卷積層的幫助函數(shù)
這個(gè)函數(shù)為TensorFlow在計(jì)算圖里創(chuàng)建了新的卷積層。這里并沒有執(zhí)行什么計(jì)算，只是在TensorFlow圖里添加了數(shù)學(xué)公式。
假設(shè)輸入的是四維的張量，各個(gè)維度如下：
- 圖像數(shù)量
- 每張圖像的Y軸
- 每張圖像的X軸
- 每張圖像的通道數(shù)
輸入通道可能是彩色通道，當(dāng)輸入是前面的卷積層生成的時(shí)候，它也可能是濾波通道。
輸出是另外一個(gè)4通道的張量，如下：
- 圖像數(shù)量，與輸入相同
- 每張圖像的Y軸。如果用到了2x2的池化，是輸入圖像寬高的一半。
- 每張圖像的X軸。同上。
- 卷積濾波生成的通道數(shù)。
def new_conv_layer(input, # The previous layer.
num_input_channels, # Num. channels in prev. layer.
filter_size, # Width and height of each filter.
num_filters, # Number of filters.
use_pooling=True): # Use 2x2 max-pooling.
# Shape of the filter-weights for the convolution.
# This format is determined by the TensorFlow API.
shape = [filter_size, filter_size, num_input_channels, num_filters]
# Create new weights aka. filters with the given shape.
weights = new_weights(shape=shape)
# Create new biases, one for each filter.
biases = new_biases(length=num_filters)
# Create the TensorFlow operation for convolution.
# Note the strides are set to 1 in all dimensions.
# The first and last stride must always be 1,
# because the first is for the image-number and
# the last is for the input-channel.
# But e.g. strides=[1, 2, 2, 1] would mean that the filter
# is moved 2 pixels across the x- and y-axis of the image.
# The padding is set to 'SAME' which means the input image
# is padded with zeroes so the size of the output is the same.
layer = tf.nn.conv2d(input=input,
filter=weights,
strides=[1, 1, 1, 1],
padding='SAME')
# Add the biases to the results of the convolution.
# A bias-value is added to each filter-channel.
layer += biases
# Use pooling to down-sample the image resolution?
if use_pooling:
# This is 2x2 max-pooling, which means that we
# consider 2x2 windows and select the largest value
# in each window. Then we move 2 pixels to the next window.
layer = tf.nn.max_pool(value=layer,
ksize=[1, 2, 2, 1],
strides=[1, 2, 2, 1],
padding='SAME')
# Rectified Linear Unit (ReLU).
# It calculates max(x, 0) for each input pixel x.
# This adds some non-linearity to the formula and allows us
# to learn more complicated functions.
layer = tf.nn.relu(layer)
# Note that ReLU is normally executed before the pooling,
# but since relu(max_pool(x)) == max_pool(relu(x)) we can
# save 75% of the relu-operations by max-pooling first.
# We return both the resulting layer and the filter-weights
# because we will plot the weights later.
return layer, weights
轉(zhuǎn)換一個(gè)層的幫助函數(shù)
卷積層生成了4維的張量。我們會(huì)在卷積層之后添加一個(gè)全連接層，因此我們需要將這個(gè)4維的張量轉(zhuǎn)換成可被全連接層使用的2維張量。
def flatten_layer(layer):
# Get the shape of the input layer.
layer_shape = layer.get_shape()
# The shape of the input layer is assumed to be:
# layer_shape == [num_images, img_height, img_width, num_channels]
# The number of features is: img_height * img_width * num_channels
# We can use a function from TensorFlow to calculate this.
num_features = layer_shape[1:4].num_elements()
# Reshape the layer to [num_images, num_features].
# Note that we just set the size of the second dimension
# to num_features and the size of the first dimension to -1
# which means the size in that dimension is calculated
# so the total size of the tensor is unchanged from the reshaping.
layer_flat = tf.reshape(layer, [-1, num_features])
# The shape of the flattened layer is now:
# [num_images, img_height * img_width * num_channels]
# Return both the flattened layer and the number of features.
return layer_flat, num_features
創(chuàng)建一個(gè)全連接層的幫助函數(shù)
這個(gè)函數(shù)為TensorFlow在計(jì)算圖中創(chuàng)建了一個(gè)全連接層。這里也不進(jìn)行任何計(jì)算，只是往TensorFlow圖中添加數(shù)學(xué)公式。
輸入是大小為[num_images, num_inputs]的二維張量。輸出是大小為[num_images, num_outputs]的2維張量。
def new_fc_layer(input, # The previous layer.
num_inputs, # Num. inputs from prev. layer.
num_outputs, # Num. outputs.
use_relu=True): # Use Rectified Linear Unit (ReLU)?
# Create new weights and biases.
weights = new_weights(shape=[num_inputs, num_outputs])
biases = new_biases(length=num_outputs)
# Calculate the layer as the matrix multiplication of
# the input and weights, and then add the bias-values.
layer = tf.matmul(input, weights) + biases
# Use ReLU?
if use_relu:
layer = tf.nn.relu(layer)
return layer
占位符 （Placeholder）變量
Placeholder是作為圖的輸入，每次我們運(yùn)行圖的時(shí)候都可能會(huì)改變它們。將這個(gè)過程稱為feeding placeholder變量，后面將會(huì)描述它。
首先我們?yōu)檩斎雸D像定義placeholder變量。這讓我們可以改變輸入到TensorFlow圖中的圖像。這也是一個(gè)張量（tensor），代表一個(gè)多維向量或矩陣。數(shù)據(jù)類型設(shè)置為float32，形狀設(shè)為[None, img_size_flat]，None代表tensor可能保存著任意數(shù)量的圖像，每張圖象是一個(gè)長(zhǎng)度為img_size_flat的向量。
x = tf.placeholder(tf.float32, shape=[None, img_size_flat], name='x')
卷積層希望x被編碼為4維張量，因此我們需要將它的形狀轉(zhuǎn)換至[num_images, img_height, img_width, num_channels]。注意img_height == img_width == img_size，如果第一維的大小設(shè)為-1， num_images的大小也會(huì)被自動(dòng)推導(dǎo)出來。轉(zhuǎn)換運(yùn)算如下：
x_image = tf.reshape(x, [-1, img_size, img_size, num_channels])
接下來我們?yōu)檩斎胱兞縳中的圖像所對(duì)應(yīng)的真實(shí)標(biāo)簽定義placeholder變量。變量的形狀是[None, num_classes]，這代表著它保存了任意數(shù)量的標(biāo)簽，每個(gè)標(biāo)簽是長(zhǎng)度為num_classes的向量，本例中長(zhǎng)度為10。
y_true = tf.placeholder(tf.float32, shape=[None, 10], name='y_true')
我們也可以為class-number提供一個(gè)placeholder，但這里用argmax來計(jì)算它。這里只是TensorFlow中的一些操作，沒有執(zhí)行什么運(yùn)算。
y_true_cls = tf.argmax(y_true, dimension=1)
卷積層1
創(chuàng)建第一個(gè)卷積層。將x_image當(dāng)作輸入，創(chuàng)建num_filters1個(gè)不同的濾波器，每個(gè)濾波器的寬高都與 filter_size1相等。最終我們會(huì)用2x2的max-pooling將圖像降采樣，使它的尺寸減半。
layer_conv1, weights_conv1 = \
new_conv_layer(input=x_image,
num_input_channels=num_channels,
filter_size=filter_size1,
num_filters=num_filters1,
use_pooling=True)
檢查卷積層輸出張量的大小。它是（？,14, 14, 16），這代表著有任意數(shù)量的圖像（？代表數(shù)量），每張圖像有14個(gè)像素的寬和高，有16個(gè)不同的通道，每個(gè)濾波器各有一個(gè)通道。
layer_conv1
卷積層2
創(chuàng)建第二個(gè)卷積層，它將第一個(gè)卷積層的輸出作為輸入。輸入通道的數(shù)量對(duì)應(yīng)著第一個(gè)卷積層的濾波數(shù)。
layer_conv2, weights_conv2 = \
new_conv_layer(input=layer_conv1,
num_input_channels=num_filters1,
filter_size=filter_size2,
num_filters=num_filters2,
use_pooling=True)
核對(duì)一下這個(gè)卷積層輸出張量的大小。它的大小是（？， 7， 7， 36）,其中？也代表著任意數(shù)量的圖像，每張圖有7像素的寬高，每個(gè)濾波器有36個(gè)通道。
layer_conv2
轉(zhuǎn)換層
這個(gè)卷積層輸出一個(gè)4維張量。現(xiàn)在我們想將它作為一個(gè)全連接網(wǎng)絡(luò)的輸入，這就需要將它轉(zhuǎn)換成2維張量。
layer_flat, num_features = flatten_layer(layer_conv2)
這個(gè)張量的大小是（？， 1764），意味著共有一定數(shù)量的圖像，每張圖像被轉(zhuǎn)換成長(zhǎng)為1764的向量。其中1764 = 7 x 7 x 36。
layer_flat
num_features
1764
全連接層 1
往網(wǎng)絡(luò)中添加一個(gè)全連接層。輸入是一個(gè)前面卷積得到的被轉(zhuǎn)換過的層。全連接層中的神經(jīng)元或節(jié)點(diǎn)數(shù)為fc_size。我們可以用ReLU來學(xué)習(xí)非線性關(guān)系。
layer_fc1 = new_fc_layer(input=layer_flat,
num_inputs=num_features,
num_outputs=fc_size,
use_relu=True)
全連接層的輸出是一個(gè)大小為（？，128）的張量，？代表著一定數(shù)量的圖像，并且fc_size == 128。
layer_fc1
全連接層 2
添加另外一個(gè)全連接層，它的輸出是一個(gè)長(zhǎng)度為10的向量，它確定了輸入圖是屬于哪個(gè)類別。這層并沒有用到ReLU。
layer_fc2 = new_fc_layer(input=layer_fc1,
num_inputs=fc_size,
num_outputs=num_classes,
use_relu=False)
layer_fc2
預(yù)測(cè)類別
第二個(gè)全連接層估算了輸入圖有多大的可能屬于10個(gè)類別中的其中一個(gè)。然而，這是很粗略的估計(jì)并且很難解釋，因?yàn)閿?shù)值可能很小或很大，因此我們會(huì)對(duì)它們做歸一化，將每個(gè)元素限制在0到1之間，并且相加為1。這用一個(gè)稱為softmax的函數(shù)來計(jì)算的，結(jié)果保存在y_pred中。
y_pred = tf.nn.softmax(layer_fc2)
類別數(shù)字是最大元素的索引。
y_pred_cls = tf.argmax(y_pred, dimension=1)
優(yōu)化損失函數(shù)
為了使模型更好地對(duì)輸入圖像進(jìn)行分類，我們必須改變weights和biases變量。首先我們需要對(duì)比模型y_pred的預(yù)測(cè)輸出和期望輸出的y_true，來了解目前模型的性能如何。
交叉熵（cross-entropy）是在分類中使用的性能度量。交叉熵是一個(gè)常為正值的連續(xù)函數(shù)，如果模型的預(yù)測(cè)值精準(zhǔn)地符合期望的輸出，它就等于零。因此，優(yōu)化的目的就是通過改變網(wǎng)絡(luò)層的變量來最小化交叉熵。
TensorFlow有一個(gè)內(nèi)置的計(jì)算交叉熵的函數(shù)。這個(gè)函數(shù)內(nèi)部計(jì)算了softmax，所以我們要用layer_fc2的輸出而非直接用y_pred,因?yàn)?code>y_pred上已經(jīng)計(jì)算了softmax。
cross_entropy = tf.nn.softmax_cross_entropy_with_logits(logits=layer_fc2,
labels=y_true)
我們?yōu)槊總€(gè)圖像分類計(jì)算了交叉熵，所以有一個(gè)當(dāng)前模型在每張圖上表現(xiàn)的度量。但是為了用交叉熵來指導(dǎo)模型變量的優(yōu)化，我們需要一個(gè)額外的標(biāo)量值，因此簡(jiǎn)單地利用所有圖像分類交叉熵的均值。
cost = tf.reduce_mean(cross_entropy)
優(yōu)化方法
既然我們有一個(gè)需要被最小化的損失度量，接著就可以建立優(yōu)化一個(gè)優(yōu)化器。這個(gè)例子中，我們使用的是梯度下降的變體AdamOptimizer。
優(yōu)化過程并不是在這里執(zhí)行。實(shí)際上，還沒計(jì)算任何東西，我們只是往TensorFlow圖中添加了優(yōu)化器，以便之后的操作。
optimizer = tf.train.AdamOptimizer(learning_rate=1e-4).minimize(cost)
性能度量
我們需要另外一些性能度量，來向用戶展示這個(gè)過程。
這是一個(gè)布爾值向量，代表預(yù)測(cè)類型是否等于每張圖片的真實(shí)類型。
correct_prediction = tf.equal(y_pred_cls, y_true_cls)
上面的計(jì)算先將布爾值向量類型轉(zhuǎn)換成浮點(diǎn)型向量，這樣子False就變成0，True變成1，然后計(jì)算這些值的平均數(shù)，以此來計(jì)算分類的準(zhǔn)確度。
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
運(yùn)行TensorFlow
創(chuàng)建TensorFlow會(huì)話（session）
一旦創(chuàng)建了TensorFlow圖，我們需要?jiǎng)?chuàng)建一個(gè)TensorFlow會(huì)話，用來運(yùn)行圖。
session = tf.Session()
初始化變量
我們需要在開始優(yōu)化weights和biases變量之前對(duì)它們進(jìn)行初始化。
session.run(tf.global_variables_initializer())
用來優(yōu)化迭代的幫助函數(shù)
在訓(xùn)練集中有50,000張圖。用這些圖像計(jì)算模型的梯度會(huì)花很多時(shí)間。因此我們利用隨機(jī)梯度下降的方法，它在優(yōu)化器的每次迭代里只用到了一小部分的圖像。
如果內(nèi)存耗盡導(dǎo)致電腦死機(jī)或變得很慢，你應(yīng)該試著減少這些數(shù)量，但同時(shí)可能還需要更優(yōu)化的迭代。
train_batch_size = 64
函數(shù)執(zhí)行了多次的優(yōu)化迭代來逐步地提升網(wǎng)絡(luò)層的變量。在每次迭代中，從訓(xùn)練集中選擇一批新的數(shù)據(jù)，然后TensorFlow用這些訓(xùn)練樣本來執(zhí)行優(yōu)化器。每100次迭代會(huì)打印出相關(guān)信息。
# Counter for total number of iterations performed so far.
total_iterations = 0
def optimize(num_iterations):
# Ensure we update the global variable rather than a local copy.
global total_iterations
# Start-time used for printing time-usage below.
start_time = time.time()
for i in range(total_iterations,
total_iterations + num_iterations):
# Get a batch of training examples.
# x_batch now holds a batch of images and
# y_true_batch are the true labels for those images.
x_batch, y_true_batch = data.train.next_batch(train_batch_size)
# Put the batch into a dict with the proper names
# for placeholder variables in the TensorFlow graph.
feed_dict_train = {x: x_batch,
y_true: y_true_batch}
# Run the optimizer using this batch of training data.
# TensorFlow assigns the variables in feed_dict_train
# to the placeholder variables and then runs the optimizer.
session.run(optimizer, feed_dict=feed_dict_train)
# Print status every 100 iterations.
if i % 100 == 0:
# Calculate the accuracy on the training-set.
acc = session.run(accuracy, feed_dict=feed_dict_train)
# Message for printing.
msg = "Optimization Iteration: {0:>6}, Training Accuracy: {1:>6.1%}"
# Print it.
print(msg.format(i + 1, acc))
# Update the total number of iterations performed.
total_iterations += num_iterations
# Ending time.
end_time = time.time()
# Difference between start and end-times.
time_dif = end_time - start_time
# Print the time-usage.
print("Time usage: " + str(timedelta(seconds=int(round(time_dif)))))
用來繪制錯(cuò)誤樣本的幫助函數(shù)
函數(shù)用來繪制測(cè)試集中被誤分類的樣本。
def plot_example_errors(cls_pred, correct):
# This function is called from print_test_accuracy() below.
# cls_pred is an array of the predicted class-number for
# all images in the test-set.
# correct is a boolean array whether the predicted class
# is equal to the true class for each image in the test-set.
# Negate the boolean array.
incorrect = (correct == False)
# Get the images from the test-set that have been
# incorrectly classified.
images = data.test.images[incorrect]
# Get the predicted classes for those images.
cls_pred = cls_pred[incorrect]
# Get the true classes for those images.
cls_true = data.test.cls[incorrect]
# Plot the first 9 images.
plot_images(images=images[0:9],
cls_true=cls_true[0:9],
cls_pred=cls_pred[0:9])
繪制混淆（confusion）矩陣的幫助函數(shù)
def plot_confusion_matrix(cls_pred):
# This is called from print_test_accuracy() below.
# cls_pred is an array of the predicted class-number for
# all images in the test-set.
# Get the true classifications for the test-set.
cls_true = data.test.cls
# Get the confusion matrix using sklearn.
cm = confusion_matrix(y_true=cls_true,
y_pred=cls_pred)
# Print the confusion matrix as text.
print(cm)
# Plot the confusion matrix as an image.
plt.matshow(cm)
# Make various adjustments to the plot.
plt.colorbar()
tick_marks = np.arange(num_classes)
plt.xticks(tick_marks, range(num_classes))
plt.yticks(tick_marks, range(num_classes))
plt.xlabel('Predicted')
plt.ylabel('True')
# Ensure the plot is shown correctly with multiple plots
# in a single Notebook cell.
plt.show()
展示性能的幫助函數(shù)
函數(shù)用來打印測(cè)試集上的分類準(zhǔn)確度。
為測(cè)試集上的所有圖片計(jì)算分類會(huì)花費(fèi)一段時(shí)間，因此我們直接用這個(gè)函數(shù)來調(diào)用上面的結(jié)果，這樣就不用每次都重新計(jì)算了。
這個(gè)函數(shù)可能會(huì)占用很多電腦內(nèi)存，這也是為什么將測(cè)試集分成更小的幾個(gè)部分。如果你的電腦內(nèi)存比較小或死機(jī)了，就要試著降低batch-size。
# Split the test-set into smaller batches of this size.
test_batch_size = 256
def print_test_accuracy(show_example_errors=False,
show_confusion_matrix=False):
# Number of images in the test-set.
num_test = len(data.test.images)
# Allocate an array for the predicted classes which
# will be calculated in batches and filled into this array.
cls_pred = np.zeros(shape=num_test, dtype=np.int)
# Now calculate the predicted classes for the batches.
# We will just iterate through all the batches.
# There might be a more clever and Pythonic way of doing this.
# The starting index for the next batch is denoted i.
i = 0
while i < num_test:
# The ending index for the next batch is denoted j.
j = min(i + test_batch_size, num_test)
# Get the images from the test-set between index i and j.
images = data.test.images[i:j, :]
# Get the associated labels.
labels = data.test.labels[i:j, :]
# Create a feed-dict with these images and labels.
feed_dict = {x: images,
y_true: labels}
# Calculate the predicted class using TensorFlow.
cls_pred[i:j] = session.run(y_pred_cls, feed_dict=feed_dict)
# Set the start-index for the next batch to the
# end-index of the current batch.
i = j
# Convenience variable for the true class-numbers of the test-set.
cls_true = data.test.cls
# Create a boolean array whether each image is correctly classified.
correct = (cls_true == cls_pred)
# Calculate the number of correctly classified images.
# When summing a boolean array, False means 0 and True means 1.
correct_sum = correct.sum()
# Classification accuracy is the number of correctly classified
# images divided by the total number of images in the test-set.
acc = float(correct_sum) / num_test
# Print the accuracy.
msg = "Accuracy on Test-Set: {0:.1%} ({1} / {2})"
print(msg.format(acc, correct_sum, num_test))
# Plot some examples of mis-classifications, if desired.
if show_example_errors:
print("Example errors:")
plot_example_errors(cls_pred=cls_pred, correct=correct)
# Plot the confusion matrix, if desired.
if show_confusion_matrix:
print("Confusion Matrix:")
plot_confusion_matrix(cls_pred=cls_pred)
優(yōu)化之前的性能
測(cè)試集上的準(zhǔn)確度很低，這是由于模型只做了初始化，并沒做任何優(yōu)化，所以它只是對(duì)圖像做隨機(jī)分類。
print_test_accuracy()
Accuracy on Test-Set: 10.9% (1093 / 10000)
1次迭代后的性能
做了一次優(yōu)化后，此時(shí)優(yōu)化器的學(xué)習(xí)率很低，性能其實(shí)并沒有多大提升。
optimize(num_iterations=1)
Optimization Iteration: 1, Training Accuracy: 6.2%
Time usage: 0:00:00
print_test_accuracy()
Accuracy on Test-Set: 13.0% (1296 / 10000)
100次迭代優(yōu)化后的性能
100次優(yōu)化迭代之后，模型顯著地提升了分類的準(zhǔn)確度。
optimize(num_iterations=99) # We already performed 1 iteration above.
Time usage: 0:00:00
print_test_accuracy(show_example_errors=True)
Accuracy on Test-Set: 66.6% (6656 / 10000)
Example errors:
1000次優(yōu)化迭代后的性能
1000次優(yōu)化迭代之后，模型在測(cè)試集上的準(zhǔn)確度超過了90%。
optimize(num_iterations=900) # We performed 100 iterations above.
Optimization Iteration: 101, Training Accuracy: 71.9%
Optimization Iteration: 201, Training Accuracy: 76.6%
Optimization Iteration: 301, Training Accuracy: 71.9%
Optimization Iteration: 401, Training Accuracy: 85.9%
Optimization Iteration: 501, Training Accuracy: 89.1%
Optimization Iteration: 601, Training Accuracy: 95.3%
Optimization Iteration: 701, Training Accuracy: 90.6%
Optimization Iteration: 801, Training Accuracy: 92.2%
Optimization Iteration: 901, Training Accuracy: 95.3%
Time usage: 0:00:03
print_test_accuracy(show_example_errors=True)
Accuracy on Test-Set: 93.1% (9308 / 10000)
Example errors:
10,000次優(yōu)化迭代后的性能
經(jīng)過10,000次優(yōu)化迭代后，測(cè)試集上的分類準(zhǔn)確率高達(dá)99%。
optimize(num_iterations=9000) # We performed 1000 iterations above.
Optimization Iteration: 1001, Training Accuracy: 98.4%
Optimization Iteration: 1101, Training Accuracy: 93.8%
Optimization Iteration: 1201, Training Accuracy: 92.2%
Optimization Iteration: 1301, Training Accuracy: 95.3%
Optimization Iteration: 1401, Training Accuracy: 93.8%
Optimization Iteration: 1501, Training Accuracy: 93.8%
Optimization Iteration: 1601, Training Accuracy: 92.2%
Optimization Iteration: 1701, Training Accuracy: 92.2%
Optimization Iteration: 1801, Training Accuracy: 89.1%
Optimization Iteration: 1901, Training Accuracy: 95.3%
Optimization Iteration: 2001, Training Accuracy: 93.8%
Optimization Iteration: 2101, Training Accuracy: 98.4%
Optimization Iteration: 2201, Training Accuracy: 92.2%
Optimization Iteration: 2301, Training Accuracy: 95.3%
Optimization Iteration: 2401, Training Accuracy: 100.0%
Optimization Iteration: 2501, Training Accuracy: 96.9%
Optimization Iteration: 2601, Training Accuracy: 93.8%
Optimization Iteration: 2701, Training Accuracy: 100.0%
Optimization Iteration: 2801, Training Accuracy: 95.3%
Optimization Iteration: 2901, Training Accuracy: 95.3%
Optimization Iteration: 3001, Training Accuracy: 96.9%
Optimization Iteration: 3101, Training Accuracy: 96.9%
Optimization Iteration: 3201, Training Accuracy: 95.3%
Optimization Iteration: 3301, Training Accuracy: 96.9%
Optimization Iteration: 3401, Training Accuracy: 98.4%
Optimization Iteration: 3501, Training Accuracy: 100.0%
Optimization Iteration: 3601, Training Accuracy: 98.4%
Optimization Iteration: 3701, Training Accuracy: 95.3%
Optimization Iteration: 3801, Training Accuracy: 95.3%
Optimization Iteration: 3901, Training Accuracy: 95.3%
Optimization Iteration: 4001, Training Accuracy: 100.0%
Optimization Iteration: 4101, Training Accuracy: 93.8%
Optimization Iteration: 4201, Training Accuracy: 95.3%
Optimization Iteration: 4301, Training Accuracy: 100.0%
Optimization Iteration: 4401, Training Accuracy: 96.9%
Optimization Iteration: 4501, Training Accuracy: 100.0%
Optimization Iteration: 4601, Training Accuracy: 100.0%
Optimization Iteration: 4701, Training Accuracy: 100.0%
Optimization Iteration: 4801, Training Accuracy: 98.4%
Optimization Iteration: 4901, Training Accuracy: 98.4%
Optimization Iteration: 5001, Training Accuracy: 98.4%
Optimization Iteration: 5101, Training Accuracy: 100.0%
Optimization Iteration: 5201, Training Accuracy: 95.3%
Optimization Iteration: 5301, Training Accuracy: 96.9%
Optimization Iteration: 5401, Training Accuracy: 100.0%
Optimization Iteration: 5501, Training Accuracy: 100.0%
Optimization Iteration: 5601, Training Accuracy: 100.0%
Optimization Iteration: 5701, Training Accuracy: 96.9%
Optimization Iteration: 5801, Training Accuracy: 98.4%
Optimization Iteration: 5901, Training Accuracy: 100.0%
Optimization Iteration: 6001, Training Accuracy: 95.3%
Optimization Iteration: 6101, Training Accuracy: 96.9%
Optimization Iteration: 6201, Training Accuracy: 100.0%
Optimization Iteration: 6301, Training Accuracy: 96.9%
Optimization Iteration: 6401, Training Accuracy: 100.0%
Optimization Iteration: 6501, Training Accuracy: 98.4%
Optimization Iteration: 6601, Training Accuracy: 98.4%
Optimization Iteration: 6701, Training Accuracy: 95.3%
Optimization Iteration: 6801, Training Accuracy: 100.0%
Optimization Iteration: 6901, Training Accuracy: 98.4%
Optimization Iteration: 7001, Training Accuracy: 95.3%
Optimization Iteration: 7101, Training Accuracy: 100.0%
Optimization Iteration: 7201, Training Accuracy: 100.0%
Optimization Iteration: 7301, Training Accuracy: 100.0%
Optimization Iteration: 7401, Training Accuracy: 100.0%
Optimization Iteration: 7501, Training Accuracy: 100.0%
Optimization Iteration: 7601, Training Accuracy: 96.9%
Optimization Iteration: 7701, Training Accuracy: 98.4%
Optimization Iteration: 7801, Training Accuracy: 95.3%
Optimization Iteration: 7901, Training Accuracy: 100.0%
Optimization Iteration: 8001, Training Accuracy: 100.0%
Optimization Iteration: 8101, Training Accuracy: 98.4%
Optimization Iteration: 8201, Training Accuracy: 98.4%
Optimization Iteration: 8301, Training Accuracy: 100.0%
Optimization Iteration: 8401, Training Accuracy: 96.9%
Optimization Iteration: 8501, Training Accuracy: 98.4%
Optimization Iteration: 8601, Training Accuracy: 98.4%
Optimization Iteration: 8701, Training Accuracy: 100.0%
Optimization Iteration: 8801, Training Accuracy: 100.0%
Optimization Iteration: 8901, Training Accuracy: 98.4%
Optimization Iteration: 9001, Training Accuracy: 95.3%
Optimization Iteration: 9101, Training Accuracy: 100.0%
Optimization Iteration: 9201, Training Accuracy: 100.0%
Optimization Iteration: 9301, Training Accuracy: 96.9%
Optimization Iteration: 9401, Training Accuracy: 96.9%
Optimization Iteration: 9501, Training Accuracy: 98.4%
Optimization Iteration: 9601, Training Accuracy: 100.0%
Optimization Iteration: 9701, Training Accuracy: 96.9%
Optimization Iteration: 9801, Training Accuracy: 98.4%
Optimization Iteration: 9901, Training Accuracy: 98.4%
Time usage: 0:00:26
print_test_accuracy(show_example_errors=True,
show_confusion_matrix=True)
Accuracy on Test-Set: 98.8% (9880 / 10000)
Example errors:
Confusion Matrix:
[[ 973 0 1 0 0 1 1 0 3 1]
[ 0 1129 2 1 0 0 1 1 1 0]
[ 1 2 1023 2 0 0 0 2 2 0]
[ 1 0 1 1002 0 3 0 1 2 0]
[ 0 1 0 0 974 0 1 0 2 4]
[ 2 0 0 3 0 882 2 0 1 2]
[ 4 1 0 0 1 4 948 0 0 0]
[ 1 4 11 2 0 0 0 1004 2 4]
[ 3 0 4 2 1 2 0 0 960 2]
[ 3 4 1 0 7 5 0 2 2 985]]
權(quán)重和層的可視化
為了理解為什么卷積神經(jīng)網(wǎng)絡(luò)可以識(shí)別手寫數(shù)字，我們將會(huì)對(duì)卷積濾波和輸出圖像進(jìn)行可視化。
繪制卷積權(quán)重的幫助函數(shù)
def plot_conv_weights(weights, input_channel=0):
# Assume weights are TensorFlow ops for 4-dim variables
# e.g. weights_conv1 or weights_conv2.
# Retrieve the values of the weight-variables from TensorFlow.
# A feed-dict is not necessary because nothing is calculated.
w = session.run(weights)
# Get the lowest and highest values for the weights.
# This is used to correct the colour intensity across
# the images so they can be compared with each other.
w_min = np.min(w)
w_max = np.max(w)
# Number of filters used in the conv. layer.
num_filters = w.shape[3]
# Number of grids to plot.
# Rounded-up, square-root of the number of filters.
num_grids = math.ceil(math.sqrt(num_filters))
# Create figure with a grid of sub-plots.
fig, axes = plt.subplots(num_grids, num_grids)
# Plot all the filter-weights.
for i, ax in enumerate(axes.flat):
# Only plot the valid filter-weights.
if i<num_filters:
# Get the weights for the i'th filter of the input channel.
# See new_conv_layer() for details on the format
# of this 4-dim tensor.
img = w[:, :, input_channel, i]
# Plot image.
ax.imshow(img, vmin=w_min, vmax=w_max,
interpolation='nearest', cmap='seismic')
# Remove ticks from the plot.
ax.set_xticks([])
ax.set_yticks([])
# Ensure the plot is shown correctly with multiple plots
# in a single Notebook cell.
plt.show()
繪制卷積層輸出的幫助函數(shù)
def plot_conv_layer(layer, image):
# Assume layer is a TensorFlow op that outputs a 4-dim tensor
# which is the output of a convolutional layer,
# e.g. layer_conv1 or layer_conv2.
# Create a feed-dict containing just one image.
# Note that we don't need to feed y_true because it is
# not used in this calculation.
feed_dict = {x: [image]}
# Calculate and retrieve the output values of the layer
# when inputting that image.
values = session.run(layer, feed_dict=feed_dict)
# Number of filters used in the conv. layer.
num_filters = values.shape[3]
# Number of grids to plot.
# Rounded-up, square-root of the number of filters.
num_grids = math.ceil(math.sqrt(num_filters))
# Create figure with a grid of sub-plots.
fig, axes = plt.subplots(num_grids, num_grids)
# Plot the output images of all the filters.
for i, ax in enumerate(axes.flat):
# Only plot the images for valid filters.
if i<num_filters:
# Get the output image of using the i'th filter.
# See new_conv_layer() for details on the format
# of this 4-dim tensor.
img = values[0, :, :, i]
# Plot image.
ax.imshow(img, interpolation='nearest', cmap='binary')
# Remove ticks from the plot.
ax.set_xticks([])
ax.set_yticks([])
# Ensure the plot is shown correctly with multiple plots
# in a single Notebook cell.
plt.show()
輸入圖像
繪制圖像的幫助函數(shù)
def plot_image(image):
plt.imshow(image.reshape(img_shape),
interpolation='nearest',
cmap='binary')
plt.show()
如下所示，繪制一張測(cè)試集中的圖像。
image1 = data.test.images[0]
plot_image(image1)
繪制測(cè)試集里的另一張圖像。
image2 = data.test.images[13]
plot_image(image2)
卷積層 1
現(xiàn)在繪制第一個(gè)卷積層的濾波權(quán)重。
其中正值權(quán)重是紅色的，負(fù)值為藍(lán)色。
plot_conv_weights(weights=weights_conv1)
將這些卷積濾波添加到第一張輸入圖像，得到以下輸出，它們也作為第二個(gè)卷積層的輸入。注意這些圖像被降采樣到14 x 14像素，即原始輸入圖分辨率的一半。
plot_conv_layer(layer=layer_conv1, image=image1)
下面是將卷積濾波添加到第二張圖像的結(jié)果。
plot_conv_layer(layer=layer_conv1, image=image2)
從這些圖像很難看出卷積濾波的作用是什么。顯然，它們生成了輸入圖像的一些變體，就像光線從不同角度打到圖像上并產(chǎn)生陰影一樣。
卷積層 2
現(xiàn)在繪制第二個(gè)卷積層的濾波權(quán)重。
第一個(gè)卷積層有16個(gè)輸出通道，代表著第二個(gè)卷基層有16個(gè)輸入。第二個(gè)卷積層的每個(gè)輸入通道也有一些權(quán)重濾波。我們先繪制第一個(gè)通道的權(quán)重濾波。
同樣的，正值是紅色，負(fù)值是藍(lán)色。
plot_conv_weights(weights=weights_conv2, input_channel=0)
第二個(gè)卷積層共有16個(gè)輸入通道，我們可以同樣地畫出其他圖像。這里我們畫出第二個(gè)通道的圖像。
plot_conv_weights(weights=weights_conv2, input_channel=1)
由于這些濾波是高維度的，很難理解它們是如何應(yīng)用的。
給第一個(gè)卷積層的輸出加上這些濾波，得到下面的圖像。
這些圖像被降采樣至7 x 7的像素，即上一個(gè)卷積層輸出的一半。
plot_conv_layer(layer=layer_conv2, image=image1)
這是給第二張圖像加上濾波權(quán)重的結(jié)果。
plot_conv_layer(layer=layer_conv2, image=image2)
從這些圖像來看，似乎第二個(gè)卷積層會(huì)檢測(cè)輸入圖像中的線段和模式，這對(duì)輸入圖中的局部變化不那么敏感。
關(guān)閉TensorFlow會(huì)話
現(xiàn)在我們已經(jīng)用TensorFlow完成了任務(wù)，關(guān)閉session，釋放資源。
# This has been commented out in case you want to modify and experiment
# with the Notebook without having to restart it.
# session.close()
總結(jié)
我們看到卷積神經(jīng)網(wǎng)絡(luò)在識(shí)別手寫數(shù)字上的表現(xiàn)要比教程#01中簡(jiǎn)單線性模型要好得多。卷積神經(jīng)網(wǎng)絡(luò)可能達(dá)到99%的分類準(zhǔn)確率，如果你做一些調(diào)整，還可能表現(xiàn)得更好，而簡(jiǎn)單線性模型只有91%的正確率。
然而，卷積神經(jīng)網(wǎng)絡(luò)實(shí)現(xiàn)起來更復(fù)雜，并且光看權(quán)重濾波也不好理解為什么它能奏效或者失敗。
因此我們需要一個(gè)更簡(jiǎn)單的實(shí)現(xiàn)卷積神經(jīng)網(wǎng)絡(luò)的方式，同時(shí)也要尋找一種更好的方法來對(duì)它們內(nèi)部工作原理進(jìn)行可視化。
練習(xí)
下面使一些可能會(huì)讓你提升TensorFlow技能的一些建議練習(xí)。為了學(xué)習(xí)如何更合適地使用TensorFlow，實(shí)踐經(jīng)驗(yàn)是很重要的。
在你對(duì)這個(gè)Notebook進(jìn)行修改之前，可能需要先備份一下。
- 如果你不改變?nèi)魏螀?shù)，多次運(yùn)行Notebook，會(huì)得到完成一樣的結(jié)果嗎？隨機(jī)性的來源是什么？
- 再進(jìn)行10,000次優(yōu)化。結(jié)果有變好么？
- 改變優(yōu)化器的學(xué)習(xí)率。
- 改變層次的屬性，比如卷積濾波器數(shù)量、濾波器的大小、全連接層中的神經(jīng)元數(shù)量等等。
- 在全連接層之后添加一個(gè)drop-out層。在計(jì)算分類準(zhǔn)確率的時(shí)候，drop-out層可能為0，因此你需要一個(gè)placeholder變量。
- 改變ReLU和max-pooling的順序。它的計(jì)算結(jié)果相同么？最快的計(jì)算方法是什么？節(jié)省了多少計(jì)算量？這也適用于Sigmoid-function和average-pooling嗎？
- 添加一個(gè)或多個(gè)卷積層和全連接層。這對(duì)性能有幫助嗎？
- 能得到良好結(jié)果的最小可能配置是什么？
- 試著在最后一個(gè)全連接層中使用ReLU。性能有變化嗎？為什么？
- 卷積層里不用pooling。這對(duì)分類準(zhǔn)確率和訓(xùn)練時(shí)間有影響嗎？
- 在卷積層里用2x2的stride代替max-pooling？有什么變化嗎？
- 不看源碼，自己重寫程序。
- 向朋友解釋程序如何工作。
