(全文Google翻譯,自學使用)
本指南讓您開始在TensorFlow中編程。 在使用本指南之前,請安裝TensorFlow。 為了充分利用本指南,您應該了解以下內容:
This guide gets you started programming in TensorFlow. Before using this guide,install TensorFlow. To get the most out of this guide, you should know the following:
- 如何用Python編程。
- 至少了解一點數組。
- 理想的是,了解關于機器學習的東西。 但是,如果您對機器學習知之甚少,那么這仍然是您應該閱讀的第一本指南。
- How to program in Python.
- At least a little bit about arrays.
- Ideally, something about machine learning. However, if you know little or nothing about machine learning, then this is still the first guide you should read.
TensorFlow提供了多個API。 最低級別的API - TensorFlow核心 - 為您提供完整的編程控制。 我們推薦TensorFlow Core用于機器學習研究人員和其他需要對其模型進行精細控制的人員。 更高層次的API建立在TensorFlow核心之上。 這些更高級別的API通常比TensorFlow Core更容易學習和使用。 另外,更高級別的API使得不同用戶之間的重復任務更容易和更一致。 像tf.estimator這樣的高級API可以幫助您管理數據集,估算器,培訓和推理。
TensorFlow provides multiple APIs. The lowest level API--TensorFlow Core-- provides you with complete programming control. We recommend TensorFlow Core for machine learning researchers and others who require fine levels of control over their models. The higher level APIs are built on top of TensorFlow Core. These higher level APIs are typically easier to learn and use than TensorFlow Core. In addition, the higher level APIs make repetitive tasks easier and more consistent between different users. A high-level API like tf.estimator helps you manage data sets, estimators, training and inference.
本指南從TensorFlow核心教程開始。 稍后,我們演示如何在tf.estimator中實現相同的模型。 了解TensorFlow核心原則將給你一個很好的思維模型,說明在使用更緊湊的更高級別的API時,內部工作是如何工作的。
This guide begins with a tutorial on TensorFlow Core. Later, we demonstrate how to implement the same model in tf.estimator. Knowing TensorFlow Core principles will give you a great mental model of how things are working internally when you use the more compact higher level API.
張量(Tensors)
TensorFlow中的數據中心單位是張量。 一個張量由一組形成任意數量維數的原始值組成。 張量的rank是它的維數。 這里是一些張量的例子:
The central unit of data in TensorFlow is the tensor. A tensor consists of a set of primitive values shaped into an array of any number of dimensions. A tensor's rank is its number of dimensions. Here are some examples of tensors:
3 # a rank 0 tensor; a scalar with shape []
[1., 2., 3.] # a rank 1 tensor; a vector with shape [3]
[[1., 2., 3.], [4., 5., 6.]] # a rank 2 tensor; a matrix with shape [2, 3]
[[[1., 2., 3.]], [[7., 8., 9.]]] # a rank 3 tensor with shape [2, 1, 3]
TensorFlow核心教程(TensorFlow Core tutorial)
導入Tensorflow(Importing TensorFlow)
TensorFlow程序的規范導入語句如下所示:
The canonical import statement for TensorFlow programs is as follows:
import tensorflow as tf
這使得Python可以訪問所有TensorFlow的類,方法和符號。 大多數文檔假定您已經完成了這個工作。
This gives Python access to all of TensorFlow's classes, methods, and symbols. Most of the documentation assumes you have already done this.
計算圖(The Computational Graph)
你可能會想到TensorFlow核心程序由兩個獨立的部分組成:
- 構建計算圖。
- 運行計算圖。
You might think of TensorFlow Core programs as consisting of two discrete sections:
- Building the computational graph.
- Running the computational graph.
計算圖是排列成由節點組成的圖的一系列TensorFlow操作。 我們來構建一個簡單的計算圖。 每個節點將零個或多個張量作為輸入,并生成張量作為輸出。 一種節點的類型是一個常量。 像所有的TensorFlow常量一樣,它不需要輸入,而是輸出一個內部存儲的值。 我們可以創建兩個浮點張量node1
和node2
,如下所示:
A computational graph is a series of TensorFlow operations arranged into a graph of nodes. Let's build a simple computational graph. Each node takes zero or more tensors as inputs and produces a tensor as an output. One type of node is a constant. Like all TensorFlow constants, it takes no inputs, and it outputs a value it stores internally. We can create two floating point Tensors
node1
andnode2
as follows:
node1 = tf.constant(3.0, dtype=tf.float32)
node2 = tf.constant(4.0) # also tf.float32 implicitly
print(node1, node2)
最后的打印聲明產生
The final print statement produces
Tensor("Const:0", shape=(), dtype=float32) Tensor("Const_1:0", shape=(), dtype=float32)
請注意,打印節點不會像預期的那樣輸出值3.0和4.0。 相反,它們是在評估時分別產生3.0和4.0的節點。 為了實際評估節點,我們必須在會話中運行計算圖。 會話封裝了TensorFlow運行時的控制和狀態。
Notice that printing the nodes does not output the values 3.0 and 4.0 as you might expect. Instead, they are nodes that, when evaluated, would produce 3.0 and 4.0, respectively. To actually evaluate the nodes, we must run the computational graph within a session. A session encapsulates the control and state of the TensorFlow runtime.
以下代碼創建一個Session對象,然后調用其run方法來運行足夠的計算圖來評估node1和node2。 通過在會話中運行計算圖如下:
The following code creates a Session object and then invokes its run method to run enough of the computational graph to evaluate node1 and node2. By running the computational graph in a session as follows:
sess = tf.Session()
print(sess.run([node1, node2]))
我們看到了3.0和4.0的預期值:
we see the expected values of 3.0 and 4.0:
[3.0, 4.0]
我們可以通過將Tensor節點與操作相結合來構建更復雜的計算(操作也是節點)。 例如,我們可以添加我們的兩個常量節點,并產生一個新的圖形如下:
We can build more complicated computations by combining Tensor nodes with operations (Operations are also nodes). For example, we can add our two constant nodes and produce a new graph as follows:
from __future__ import print_function
node3 = tf.add(node1, node2)
print("node3:", node3)
print("sess.run(node3):", sess.run(node3))
最后兩個打印語句產生
The last two print statements produce
node3: Tensor("Add:0", shape=(), dtype=float32)
sess.run(node3): 7.0
TensorFlow提供了一個名為TensorBoard的實用程序,可以顯示計算圖的圖片。 下面是一個屏幕截圖,展示了TensorBoard如何將圖形可視化:
TensorFlow provides a utility called TensorBoard that can display a picture of the computational graph. Here is a screenshot showing how TensorBoard visualizes the graph:
就目前來看,這張圖并不是特別有趣,因為它總是會產生一個不變的結果。 圖可以參數化來接受外部輸入,稱為占位符。 占位符是稍后提供值的承諾。
As it stands, this graph is not especially interesting because it always produces a constant result. A graph can be parameterized to accept external inputs, known as placeholders. A placeholder is a promise to provide a value later.
a = tf.placeholder(tf.float32)
b = tf.placeholder(tf.float32)
adder_node = a + b # + provides a shortcut for tf.add(a, b)
前面的三行有點像一個函數或lambda,我們在其中定義兩個輸入參數(a和b),然后對它們進行操作。 我們可以通過使用run方法的feed_dict參數將多個輸入的具體值提供給占位符來評估這個圖形。
The preceding three lines are a bit like a function or a lambda in which we define two input parameters (a and b) and then an operation on them. We can evaluate this graph with multiple inputs by using the feed_dict argument to the run method to feed concrete values to the placeholders:
print(sess.run(adder_node, {a: 3, b: 4.5}))
print(sess.run(adder_node, {a: [1, 3], b: [2, 4]}))
結果輸出
resulting in the output
7.5
[ 3. 7.]
在TensorBoard中,圖看起來像這樣:
In TensorBoard, the graph looks like this:
我們可以通過添加另一個操作來使計算圖更加復雜。 例如,
We can make the computational graph more complex by adding another operation. For example,
add_and_triple = adder_node * 3.
print(sess.run(add_and_triple, {a: 3, b: 4.5}))
產生的輸出
produces the output
22.5
在TensorBoard中,上面的計算圖如下所示:
The preceding computational graph would look as follows in TensorBoard:
在機器學習中,我們通常需要一個可以進行任意輸入的模型,比如上面的模型。 為了使模型可訓練,我們需要能夠修改圖形以獲得具有相同輸入的新輸出。 變量允許我們將可訓練參數添加到圖形中。 它們由一個類型和初始值所構成:
In machine learning we will typically want a model that can take arbitrary inputs, such as the one above. To make the model trainable, we need to be able to modify the graph to get new outputs with the same input. Variables allow us to add trainable parameters to a graph. They are constructed with a type and initial value:
W = tf.Variable([.3], dtype=tf.float32)
b = tf.Variable([-.3], dtype=tf.float32)
x = tf.placeholder(tf.float32)
linear_model = W*x + b
當你調用tf.constant
時,常量被初始化,并且它們的值永遠不會改變。 相反,變量在您調用tf.Variable
時不會被初始化。 要初始化TensorFlow程序中的所有變量,您必須顯式調用一個特殊的操作,如下所示:
Constants are initialized when you call tf.constant
, and their value can never change. By contrast, variables are not initialized when you call tf.Variable. To initialize all the variables in a TensorFlow program, you must explicitly call a special operation as follows:
init = tf.global_variables_initializer()
sess.run(init)
實現init是TensorFlow子圖的一個句柄,初始化所有的全局變量是非常重要的。 在我們調用sess.run之前,變量是未初始化的。
由于x是一個占位符,我們可以同時為x的多個值計算linear_model,如下所示:
It is important to realize
init
is a handle to the TensorFlow sub-graph that initializes all the global variables. Until we callsess.run
, the variables are uninitialized.
Since x is a placeholder, we can evaluate linear_model for several values of x simultaneously as follows:
print(sess.run(linear_model, {x: [1, 2, 3, 4]}))
產生輸出
to produce the output
[ 0. 0.30000001 0.60000002 0.90000004]
我們已經創建了一個模型,但我們不知道它有多好。 為了評估訓練數據模型,我們需要一個占位符來提供所需的值,并且我們需要編寫一個損失函數。
We've created a model, but we don't know how good it is yet. To evaluate the model on training data, we need a y placeholder to provide the desired values, and we need to write a loss function.
損失函數用于衡量當前模型距離提供的數據有多遠。 我們將使用線性回歸的標準損失模型,它將當前模型和提供的數據之間的差值的平方相加。 linear_model - y
創建一個向量,其中每個元素是相應示例的誤差增量。 我們調用tf.square
來進行平方。 然后,我們求和所有的平方誤差來創建一個標量,用·tf.reduce_sum`來獲得所有例子的誤差:
A loss function measures how far apart the current model is from the provided data. We'll use a standard loss model for linear regression, which sums the squares of the deltas between the current model and the provided data.
linear_model - y
creates a vector where each element is the corresponding example's error delta. We calltf.square
to square that error. Then, we sum all the squared errors to create a single scalar that abstracts the error of all examples usingtf.reduce_sum
:
y = tf.placeholder(tf.float32)
squared_deltas = tf.square(linear_model - y)
loss = tf.reduce_sum(squared_deltas)
print(sess.run(loss, {x: [1, 2, 3, 4], y: [0, -1, -2, -3]}))
產生損失價值
producing the loss value
23.66
我們可以通過將W和b的值重新賦值為-1和1的完美值來手動改進。變量初始化為提供給tf.Variable
的值,但可以使用tf.assign
等操作進行更改。 例如,W = -1和b = 1是我們模型的最佳參數。 我們可以相應地改變W和b:
We could improve this manually by reassigning the values of
W
andb
to the perfect values of -1 and 1. A variable is initialized to the value provided totf.Variable
but can be changed using operations liketf.assign
. For example, W=-1 and b=1 are the optimal parameters for our model. We can change W and b accordingly:
fixW = tf.assign(W, [-1.])
fixb = tf.assign(b, [1.])
sess.run([fixW, fixb])
print(sess.run(loss, {x: [1, 2, 3, 4], y: [0, -1, -2, -3]}))
最后的打印輸出顯示現在的損失是零。
The final print shows the loss now is zero.
0.0
我們猜測了W和b的“完美”值,但是機器學習的重點是自動找到正確的模型參數。 我們將在下一節展示如何完成這個。
We guessed the "perfect" values of W and b, but the whole point of machine learning is to find the correct model parameters automatically. We will show how to accomplish this in the next section.
tf.train API
機器學習的完整討論超出了本教程的范圍。 但是,TensorFlow提供了優化器,可以逐漸改變每個變量,以最大限度地減少損失函數。 最簡單的優化器是梯度下降。 它根據相對于該變量的損失導數的大小來修改每個變量。 一般來說,手動計算符號派生是繁瑣和容易出錯的。 因此,只要使用函數tf.gradients描述模型,TensorFlow就可以自動生成衍生物。 為了簡單起見,優化程序通常會為您執行此操作。 例如,
A complete discussion of machine learning is out of the scope of this tutorial. However, TensorFlow provides optimizers that slowly change each variable in order to minimize the loss function. The simplest optimizer is gradient descent. It modifies each variable according to the magnitude of the derivative of loss with respect to that variable. In general, computing symbolic derivatives manually is tedious and error-prone. Consequently, TensorFlow can automatically produce derivatives given only a description of the model using the function tf.gradients. For simplicity, optimizers typically do this for you. For example,
optimizer = tf.train.GradientDescentOptimizer(0.01)
train = optimizer.minimize(loss)
sess.run(init) # reset values to incorrect defaults.
for i in range(1000):
sess.run(train, {x: [1, 2, 3, 4], y: [0, -1, -2, -3]})
print(sess.run([W, b]))
最終的模型參數:
results in the final model parameters:
[array([-0.9999969], dtype=float32), array([ 0.99999082], dtype=float32)]
現在我們已經完成了機器學習! 雖然這個簡單的線性回歸模型不需要太多的TensorFlow核心代碼,但是更復雜的模型和方法將數據提供給模型需要更多的代碼。 因此,TensorFlow為常見的模式,結構和功能提供更高層次的抽象。 我們將在下一節學習如何使用這些抽象。
Now we have done actual machine learning! Although this simple linear regression model does not require much TensorFlow core code, more complicated models and methods to feed data into your models necessitate more code. Thus, TensorFlow provides higher level abstractions for common patterns, structures, and functionality. We will learn how to use some of these abstractions in the next section.
完整的程序(Complete program)
完成的可訓練線性回歸模型如下所示:
The completed trainable linear regression model is shown here:
import tensorflow as tf
# Model parameters
W = tf.Variable([.3], dtype=tf.float32)
b = tf.Variable([-.3], dtype=tf.float32)
# Model input and output
x = tf.placeholder(tf.float32)
linear_model = W*x + b
y = tf.placeholder(tf.float32)
# loss
loss = tf.reduce_sum(tf.square(linear_model - y)) # sum of the squares
# optimizer
optimizer = tf.train.GradientDescentOptimizer(0.01)
train = optimizer.minimize(loss)
# training data
x_train = [1, 2, 3, 4]
y_train = [0, -1, -2, -3]
# training loop
init = tf.global_variables_initializer()
sess = tf.Session()
sess.run(init) # reset values to wrong
for i in range(1000):
sess.run(train, {x: x_train, y: y_train})
# evaluate training accuracy
curr_W, curr_b, curr_loss = sess.run([W, b, loss], {x: x_train, y: y_train})
print("W: %s b: %s loss: %s"%(curr_W, curr_b, curr_loss))
當運行時,它將生成:
When run, it produces
W: [-0.9999969] b: [ 0.99999082] loss: 5.69997e-11
請注意,損失是非常小的數字(非常接近零)。 如果你運行這個程序,你的損失可能與上述損失不完全一樣,因為模型是用偽隨機值初始化的。
Notice that the loss is a very small number (very close to zero). If you run this program, your loss may not be exactly the same as the aforementioned loss because the model is initialized with pseudorandom values.
這個更復雜的程序仍然可以在TensorBoard中可視化
This more complicated program can still be visualized in TensorBoard
tf.estimator
tf.estimator是一個高級的TensorFlow庫,它簡化了機器學習的機制,包括:
- 運行訓練循環
- 運行評估循環
- 管理數據集
tf.estimator定義了許多常見的模型。
tf.estimator
is a high-level TensorFlow library that simplifies the mechanics of machine learning, including the following:
- running training loops
- running evaluation loops
- managing data sets
tf.estimator defines many common models.
基本用法(Basic usage)
注意線性回歸程序在tf.estimator中變得簡單多了:
Notice how much simpler the linear regression program becomes with
tf.estimator
:
# NumPy is often used to load, manipulate and preprocess data.
import numpy as np
import tensorflow as tf
# Declare list of features. We only have one numeric feature. There are many
# other types of columns that are more complicated and useful.
feature_columns = [tf.feature_column.numeric_column("x", shape=[1])]
# An estimator is the front end to invoke training (fitting) and evaluation
# (inference). There are many predefined types like linear regression,
# linear classification, and many neural network classifiers and regressors.
# The following code provides an estimator that does linear regression.
estimator = tf.estimator.LinearRegressor(feature_columns=feature_columns)
# TensorFlow provides many helper methods to read and set up data sets.
# Here we use two data sets: one for training and one for evaluation
# We have to tell the function how many batches
# of data (num_epochs) we want and how big each batch should be.
x_train = np.array([1., 2., 3., 4.])
y_train = np.array([0., -1., -2., -3.])
x_eval = np.array([2., 5., 8., 1.])
y_eval = np.array([-1.01, -4.1, -7, 0.])
input_fn = tf.estimator.inputs.numpy_input_fn(
{"x": x_train}, y_train, batch_size=4, num_epochs=None, shuffle=True)
train_input_fn = tf.estimator.inputs.numpy_input_fn(
{"x": x_train}, y_train, batch_size=4, num_epochs=1000, shuffle=False)
eval_input_fn = tf.estimator.inputs.numpy_input_fn(
{"x": x_eval}, y_eval, batch_size=4, num_epochs=1000, shuffle=False)
# We can invoke 1000 training steps by invoking the method and passing the
# training data set.
estimator.train(input_fn=input_fn, steps=1000)
# Here we evaluate how well our model did.
train_metrics = estimator.evaluate(input_fn=train_input_fn)
eval_metrics = estimator.evaluate(input_fn=eval_input_fn)
print("train metrics: %r"% train_metrics)
print("eval metrics: %r"% eval_metrics)
當運行時,它產生像下面的輸出:
When run, it produces something like
train metrics: {'average_loss': 1.4833182e-08, 'global_step': 1000, 'loss': 5.9332727e-08}
eval metrics: {'average_loss': 0.0025353201, 'global_step': 1000, 'loss': 0.01014128}
請注意我們的評估數據是如何有更高的損失,但仍然接近于零。 這意味著我們正在正確地學習。
train metrics: {'average_loss': 1.4833182e-08, 'global_step': 1000, 'loss': 5.9332727e-08}
eval metrics: {'average_loss': 0.0025353201, 'global_step': 1000, 'loss': 0.01014128}
自定義模型(A custom model)
tf.estimator不會將您鎖定到其預定義的模型中。 假設我們想創建一個沒有內置到TensorFlow中的自定義模型。 我們仍然可以保留高級抽象的數據集,喂食,訓練等。 為了說明,我們將展示如何使用我們對低級別TensorFlow API的知識來實現我們自己的LinearRegressor的等效模型。
tf.estimator does not lock you into its predefined models. Suppose we wanted to create a custom model that is not built into TensorFlow. We can still retain the high level abstraction of data set, feeding, training, etc. of tf.estimator. For illustration, we will show how to implement our own equivalent model to LinearRegressor using our knowledge of the lower level TensorFlow API.
要定義一個與tf.estimator一起工作的自定義模型,我們需要使用tf.estimator.Estimator。 tf.estimator.LinearRegressor實際上是tf.estimator.Estimator的一個子類。 我們只需要提供一個函數model_fn來告訴tf.estimator如何評估預測,訓練步驟和損失,而不是Estimator的子分類。 代碼如下:
To define a custom model that works with tf.estimator, we need to use tf.estimator.Estimator. tf.estimator.LinearRegressor is actually a sub-class of tf.estimator.Estimator. Instead of sub-classing Estimator, we simply provide Estimator a function model_fn that tells tf.estimator how it can evaluate predictions, training steps, and loss. The code is as follows:
import numpy as np
import tensorflow as tf
# Declare list of features, we only have one real-valued feature
def model_fn(features, labels, mode):
# Build a linear model and predict values
W = tf.get_variable("W", [1], dtype=tf.float64)
b = tf.get_variable("b", [1], dtype=tf.float64)
y = W*features['x'] + b
# Loss sub-graph
loss = tf.reduce_sum(tf.square(y - labels))
# Training sub-graph
global_step = tf.train.get_global_step()
optimizer = tf.train.GradientDescentOptimizer(0.01)
train = tf.group(optimizer.minimize(loss),
tf.assign_add(global_step, 1))
# EstimatorSpec connects subgraphs we built to the
# appropriate functionality.
return tf.estimator.EstimatorSpec(
mode=mode,
predictions=y,
loss=loss,
train_op=train)
estimator = tf.estimator.Estimator(model_fn=model_fn)
# define our data sets
x_train = np.array([1., 2., 3., 4.])
y_train = np.array([0., -1., -2., -3.])
x_eval = np.array([2., 5., 8., 1.])
y_eval = np.array([-1.01, -4.1, -7., 0.])
input_fn = tf.estimator.inputs.numpy_input_fn(
{"x": x_train}, y_train, batch_size=4, num_epochs=None, shuffle=True)
train_input_fn = tf.estimator.inputs.numpy_input_fn(
{"x": x_train}, y_train, batch_size=4, num_epochs=1000, shuffle=False)
eval_input_fn = tf.estimator.inputs.numpy_input_fn(
{"x": x_eval}, y_eval, batch_size=4, num_epochs=1000, shuffle=False)
# train
estimator.train(input_fn=input_fn, steps=1000)
# Here we evaluate how well our model did.
train_metrics = estimator.evaluate(input_fn=train_input_fn)
eval_metrics = estimator.evaluate(input_fn=eval_input_fn)
print("train metrics: %r"% train_metrics)
print("eval metrics: %r"% eval_metrics)
運行時產生
When run, it produces
train metrics: {'loss': 1.227995e-11, 'global_step': 1000}
eval metrics: {'loss': 0.01010036, 'global_step': 1000}
請注意,自定義的model_fn()函數的內容與來自較低級別API的手動模型訓練循環非常相似。
Notice how the contents of the custom model_fn() function are very similar to our manual model training loop from the lower level API.
下一步(Next steps)
現在您已經掌握了TensorFlow的基礎知識。 我們還有更多的教程可供您了解更多信息。 如果您是機器學習的初學者,請參閱MNIST初學者,否則請參閱Deep MNIST專家。
Now you have a working knowledge of the basics of TensorFlow. We have several more tutorials that you can look at to learn more. If you are a beginner in machine learning see MNIST for beginners, otherwise see Deep MNIST for experts.
https://www.tensorflow.org/get_started/get_started#the_computational_graph