歡迎Follow我的GitHub，關(guān)注我的簡(jiǎn)書(shū)
自編碼器，使用稀疏的高階特征重新組合，來(lái)重構(gòu)自己，輸入與輸出一致。
TensorFlow框架的搭建方法，參考
源碼，同時(shí)，復(fù)制autoencoder_models的模型文件。
本文源碼的GitHub地址
工程配置
下載Python的依賴庫(kù)：scikit-learn==0.19.0、scipy==0.19.1、sklearn==0.0
scipy
如果安裝scipy出錯(cuò)，則把scipy==0.19.1寫入requestments.txt，再安裝，錯(cuò)誤如下：
THESE PACKAGES DO NOT MATCH THE HASHES FROM THE REQUIREMENTS FILE. If you have updated the package versions, please update the hashes. Otherwise, examine the package contents carefully; someone may have tampered with them.
scipy from http://mirrors.aliyun.com/pypi/packages/63/68/c5098f3b6034e69d187e3f2e989f462143d9f8b524f5a4f9e13c4a6f5f47/scipy-0.19.1-cp27-cp27m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl#md5=72415e8da753eea97eb9820602931cb5:
Expected md5 72415e8da753eea97eb9820602931cb5
Got 073584eb2c597bbfb82a5865b7055787
或者，直接編寫requestments.txt，全部安裝
pip install -r requirements.txt
matplotlib
安裝matplotlib
pip install matplotlib -i http://mirrors.aliyun.com/pypi/simple --trusted-host mirrors.aliyun.com
如果安裝matplotlib報(bào)錯(cuò)，如下：
RuntimeError: Python is not installed as a framework. The Mac OS X backend will not be able to function correctly if Python is not installed as a framework. See the Python documentation for more information on installing Python as a framework on Mac OS X. Please either reinstall Python as a framework, or try one of the other backends. If you are using (Ana)Conda please install python.app and replace the use of 'python' with 'pythonw'. See 'Working with Matplotlib on OSX' in the Matplotlib FAQ for more information.
則執(zhí)行Shell命令
cd ~/.matplotlib
touch matplotlibrc
導(dǎo)入matplotlib
import matplotlib
matplotlib.use('TkAgg')
import matplotlib.pyplot as plt
opencv
opencv的導(dǎo)入庫(kù)是cv2，安裝是opencv-python
sudo pip install opencv-python -i http://mirrors.aliyun.com/pypi/simple --trusted-host mirrors.aliyun.com
導(dǎo)入cv2，如果直接使用import cv2，則無(wú)法自動(dòng)補(bǔ)全，導(dǎo)入時(shí)應(yīng)該使用：
import cv2.cv2 as cv2
圖片存儲(chǔ)
獲取MNIST的圖片源，test表示測(cè)試集，train表示訓(xùn)練集，images表示圖片集，labels表示標(biāo)簽集。images的數(shù)據(jù)類型是ndarry，784維；labels的數(shù)據(jù)類型也是ndarray，one-hot類型。
# 加載數(shù)據(jù)
mnist = input_data.read_data_sets('MNIST_data', one_hot=True)
images = mnist.test.images # 圖片
labels = mnist.test.labels # 標(biāo)簽
將784維的一階矩陣轉(zhuǎn)換為28維的二階圖片，將one-hot標(biāo)簽轉(zhuǎn)換為數(shù)字（0~9），存儲(chǔ)test的前100張圖片。
# 存儲(chǔ)圖片
size = len(labels)
for i in range(size):
pxl = np.array(images[i]) # 像素
img = pxl.reshape((28, 28)) # 圖片
lbl = np.argmax(labels[i]) # 標(biāo)簽
misc.imsave('./IMAGE_data/test/' + str(i) + '_' + str(lbl) + '.png', img) # scipy的存儲(chǔ)模式
if i == 100:
break
合并100張圖片為一張圖片，便于做對(duì)比。
# 合并圖片
large_size = 28 * 10
large_img = Image.new('RGBA', (large_size, large_size))
paths_list, _, __ = listdir_files('./IMAGE_data/test/')
for i in range(100):
img = Image.open(paths_list[i])
loc = ((int(i / 10) * 28), (i % 10) * 28)
large_img.paste(img, loc)
large_img.save('./IMAGE_data/merged.png')
圖片的三種存儲(chǔ)方式：scipy、matplotlib（含坐標(biāo)）、opencv。
# 其他的圖片存儲(chǔ)方式
pixel = np.array(images[0]) # 784維的數(shù)據(jù)
label = np.argmax(labels[0]) # 找到標(biāo)簽
image = pixel.reshape((28, 28)) # 轉(zhuǎn)換成28*28維的矩陣
# -------------------- scipy模式 -------------------- #
misc.imsave('./IMAGE_data/scipy.png', image) # scipy的存儲(chǔ)模式
# -------------------- scipy模式 -------------------- #
# -------------------- matplotlib模式 -------------------- #
plt.gray() # 轉(zhuǎn)變?yōu)榛叶葓D片
plt.imshow(image)
plt.savefig("./IMAGE_data/plt.png")
# plt.show()
# -------------------- matplotlib模式 -------------------- #
# -------------------- opencv模式 -------------------- #
image = image * 255 # 數(shù)據(jù)是0~1的浮點(diǎn)數(shù)
cv2.imwrite("./IMAGE_data/opencv.png", image)
# cv2.imshow('hah', pixels)
# cv2.waitKey(0)
# -------------------- opencv模式 -------------------- #
自編碼器
讀取MNIST的數(shù)據(jù)
mnist = input_data.read_data_sets('MNIST_data', one_hot=True)
將訓(xùn)練數(shù)據(jù)與測(cè)試數(shù)據(jù)標(biāo)準(zhǔn)化
X_train, X_test = standard_scale(mnist.train.images, mnist.test.images)
以訓(xùn)練數(shù)據(jù)為標(biāo)準(zhǔn)，計(jì)算均值和標(biāo)準(zhǔn)差，然后處理訓(xùn)練數(shù)據(jù)與測(cè)試數(shù)據(jù)。
def standard_scale(X_train, X_test):
preprocessor = prep.StandardScaler().fit(X_train)
X_train = preprocessor.transform(X_train)
X_test = preprocessor.transform(X_test)
return X_train, X_test
在StandardScaler中，mean_表示均值矩陣，與圖片維數(shù)一致；scale_表示標(biāo)準(zhǔn)差，也與圖片維數(shù)一致；矩陣中每一個(gè)數(shù)字都減去對(duì)應(yīng)的均值，除以對(duì)應(yīng)的標(biāo)準(zhǔn)差。
self.scale_ = _handle_zeros_in_scale(np.sqrt(self.var_))
X -= self.mean_
X /= self.scale_
設(shè)置訓(xùn)練參數(shù)：n_samples全部樣本個(gè)數(shù)，training_epochs迭代次數(shù)，batch_size批次的樣本數(shù)，display_step顯示步數(shù)。
n_samples = int(mnist.train.num_examples)
training_epochs = 20
batch_size = 128
display_step = 1
AdditiveGaussianNoiseAutoencoder，簡(jiǎn)稱AGN，加高斯噪聲的自動(dòng)編碼器。n_input輸入節(jié)點(diǎn)數(shù)，與圖片維數(shù)相同，784維；n_hidden隱含層的節(jié)點(diǎn)數(shù)，需要小于輸入節(jié)點(diǎn)數(shù)，200維；transfer_function激活函數(shù)，tf.nn.softplus；optimizer優(yōu)化器，AdamOptimizer，學(xué)習(xí)率是0.001；scale噪聲系數(shù)，0.01。
autoencoder = AdditiveGaussianNoiseAutoencoder(
n_input=784, n_hidden=200, transfer_function=tf.nn.softplus,
optimizer=tf.train.AdamOptimizer(learning_rate=0.001), scale=0.01)
關(guān)于激活函數(shù)softplus的原理如下：
mat = [1., 2., 3.] # 需要使用小數(shù)
# softplus: [ln(e^1 + 1), ln(e^2 + 1), ln(e^3 + 1)]
print tf.Session().run(tf.nn.softplus(mat))
random_normal生成隨機(jī)的正態(tài)分布數(shù)組
rn = tf.random_normal((100000,)) # 一行，指定seed，防止均值的時(shí)候隨機(jī)
mean, variance = tf.nn.moments(rn, 0) # 計(jì)算均值和方差，預(yù)期均值約等于是0，方差是1
print tf.Session().run(tf.nn.moments(rn, 0))
AdditiveGaussianNoiseAutoencoder的構(gòu)造器
def __init__(self, n_input, n_hidden, transfer_function=tf.nn.softplus, optimizer=tf.train.AdamOptimizer(),
scale=0.1):
self.n_input = n_input # 輸入的節(jié)點(diǎn)數(shù)
self.n_hidden = n_hidden # 隱含層節(jié)點(diǎn)數(shù)，小于輸入節(jié)點(diǎn)數(shù)
self.transfer = transfer_function # 激活函數(shù)
self.scale = tf.placeholder(tf.float32) # 系數(shù)，待訓(xùn)練的參數(shù)，初始的feed數(shù)據(jù)是training_scale
self.training_scale = scale # 高斯噪聲系數(shù)
network_weights = self._initialize_weights() # 初始化權(quán)重系數(shù)，輸入層w1/b1，輸出層w2/b2
self.weights = network_weights # 權(quán)重
# model
self.x = tf.placeholder(tf.float32, [None, self.n_input]) # 需要feed的數(shù)據(jù)
self.hidden = self.transfer(tf.add(tf.matmul(self.x + scale * tf.random_normal((n_input,)),
self.weights['w1']),
self.weights['b1']))
self.reconstruction = tf.add(tf.matmul(self.hidden, self.weights['w2']), self.weights['b2'])
# cost，0.5*(x - x_)^2，求和
self.cost = 0.5 * tf.reduce_sum(tf.pow(tf.subtract(self.reconstruction, self.x), 2.0))
self.optimizer = optimizer.minimize(self.cost)
init = tf.global_variables_initializer()
self.sess = tf.Session()
self.sess.run(init) # 執(zhí)行圖
random_normal隨機(jī)生成矩陣，參數(shù)(n_input,)，n_input行1列，均值為0，方差為1，tf.nn.moments，返回均值和方差。
rn = tf.random_normal((100000,)) # 一行，指定seed，防止均值的時(shí)候隨機(jī)
mean, variance = tf.nn.moments(rn, 0) # 計(jì)算均值和方差，預(yù)期均值約等于是0，方差是1
print tf.Session().run(tf.nn.moments(rn, 0))
初始化權(quán)重，分為兩層，將n_input維的數(shù)據(jù)轉(zhuǎn)換為n_hidden維的數(shù)據(jù)，再反向轉(zhuǎn)換回去。初始權(quán)重初始化使用xavier_initializer（澤維爾初始化器），權(quán)重的均值為1，方差為1/(n_input+n_hidden)。
def _initialize_weights(self):
all_weights = dict()
# 使用xavier_initializer初始化
all_weights['w1'] = tf.get_variable("w1", shape=[self.n_input, self.n_hidden],
initializer=tf.contrib.layers.xavier_initializer())
all_weights['b1'] = tf.Variable(tf.zeros([self.n_hidden], dtype=tf.float32))
all_weights['w2'] = tf.Variable(tf.zeros([self.n_hidden, self.n_input], dtype=tf.float32))
all_weights['b2'] = tf.Variable(tf.zeros([self.n_input], dtype=tf.float32))
return all_weights
訓(xùn)練模型，輸出每個(gè)輪次的平均avg_cost，
for epoch in range(training_epochs):
avg_cost = 0.
total_batch = int(n_samples / batch_size)
# Loop over all batches
for i in range(total_batch):
batch_xs = get_random_block_from_data(X_train, batch_size)
# Fit training using batch data
cost = autoencoder.partial_fit(batch_xs)
# Compute average loss
avg_cost += cost / n_samples * batch_size
# Display logs per epoch step
if epoch % display_step == 0:
print("Epoch:", '%04d' % (epoch + 1), "cost=", "{:.9f}".format(avg_cost))
print("Total cost: " + str(autoencoder.calc_total_cost(X_test)))
隨機(jī)獲取起始位置，取區(qū)塊大小的一批數(shù)據(jù)。
def get_random_block_from_data(data, batch_size):
start_index = np.random.randint(0, len(data) - batch_size) # 隨機(jī)獲取區(qū)塊
return data[start_index:(start_index + batch_size)] # batch_size大小的區(qū)塊
調(diào)用autoencoder的partial_fit，向算法Feed數(shù)據(jù)，數(shù)據(jù)就是批次數(shù)據(jù)，高斯噪聲系數(shù)使用默認(rèn)。
def partial_fit(self, X):
cost, opt = self.sess.run((self.cost, self.optimizer),
feed_dict={self.x: X, self.scale: self.training_scale})
return cost
最終輸出整個(gè)測(cè)試集X_test的Cost值。
print("Total cost: " + str(autoencoder.calc_total_cost(X_test)))
原圖像的效果（100張）：
自編碼器的效果（100張）：
OK，that‘s all! Enjoy it!
