Large-scale Linear Models with TensorFlow#

TensorFlow下的大規模線性模型#

The tf.learn API provides (among other things) a rich set of tools for working with linear models in TensorFlow. This document provides an overview of those tools. It explains:

• what a linear model is.
• why you might want to use a linear model.
• how tf.learn makes it easy to build linear models in TensorFlow.
• how you can use tf.learn to combine linear models with deep learning to get the advantages of both.

在Tensorflow里，tf.learn API 提供一個豐富的用于線性模型的工具集。這篇文檔提供一個關于這些工具的概述，解釋了：

• 什么是線性模型。
• 為什么你可能會想使用線性模型。
• 如果在 Tensorflow 內使 tf.learn 方便的建立線性模型。
• 如何使用 tf.learn 將線性模型和深度學習進行結合以獲得其優勢。

Read this overview to decide whether the tf.learn linear model tools might be useful to you. Then do the Linear Models tutorial to give it a try. This overview uses code samples from the tutorial, but the tutorial walks through the code in greater detail.

閱讀這篇概述以決定 tf.learn 線性模型工具是否對你有用。然后嘗試一下 線性模型教程。這篇概述會使用該教程的示例代碼，并詳細介紹這些代碼。

To understand this overview it will help to have some familiarity with basic machine learning concepts, and also with tf.learn.

理解這篇概述，它將有助于你熟悉基本的機器學習的概念和 tf.learn.

Contents

• Large-scale Linear Models with TensorFlow
• What is a linear model?
• Why would you want to use a linear model?
• How does tf.learn help you build linear models?
  • Feature columns and transformations
  • Linear estimators
  • Wide and deep learning

目錄

• TensorFlow下的大規模線性模型
• 什么是線性模型？
• 為什么你想使用線性模型？
• tf.learn如何幫助你建立線性模型？
  • 特征列和轉換
  • 線性估計器
  • 廣度和深度學習

What is a linear model?##

什么是線性模型？##

A linear model uses a single weighted sum of features to make a prediction. For example, if you have data on age, years of education, and weekly hours of work for a population, you can learn weights for each of those numbers so that their weighted sum estimates a person's salary. You can also use linear models for classification.

一個線性模型使用特征的單個加權和來進行預測。例如，如果你有一些人口的年齡，受教育程度和每周工作時間的數據。你可以學習到每個參數，并使其的和來估計一個人的薪水。你也同樣可以使用線性模型來進行分類。

Some linear models transform the weighted sum into a more convenient form. For example, logistic regression plugs the weighted sum into the logistic function to turn the output into a value between 0 and 1. But you still just have one weight for each input feature.

一些線性模型會將加權和轉換為一個更方便的形式。例如，logistic 回歸 注入加權和進 logistic函數中以將輸出轉換為0和1之間的值。但是你依然需要為每一個輸入的特征提供一個權重。

Why would you want to use a linear model?##

為什么你想使用線性模型？##

Why would you want to use so simple a model when recent research has demonstrated the power of more complex neural networks with many layers?

為什么你要在最近的研究展示了多層復雜的神經網絡的（強大）能力的情況下，使用一個如此簡單的模型？

Linear models:

• train quickly, compared to deep neural nets.
• can work well on very large feature sets.
• can be trained with algorithms that don't require a lot of fiddling with learning rates, etc.
• can be interpreted and debugged more easily than neural nets. You can examine the weights assigned to each feature to figure out what's having the biggest impact on a prediction.
• provide an excellent starting point for learning about machine learning.
• are widely used in industry.

線性模型:

• 相比于深度神經網絡有著更快的訓練速度。
• 能夠在大量特征集下表現優秀。
• 能夠使用不需要大量的學習速率的算法進行訓練。
• 比起神經網絡，容易進行解釋和調試。你可以為每個特征分配權重以獲得其對于預測有著多大的影響。
• 為學習機器學習提供了一個極好的起步。
• 廣泛應用于工業。

How does tf.learn help you build linear models?##

tf.learn如何幫助你建立線性模型？###

You can build a linear model from scratch in TensorFlow without the help of a special API. But tf.learn provides some tools that make it easier to build effective large-scale linear models.

在Tensorflow下，你能夠在不借助一些特別的API的情況下，從頭開始建立一個線性模型。并且 tf.learn 提供一些工具使得能夠的建立一個大規模線性模型。

Feature columns and transformations###

特征列和轉換###

Much of the work of designing a linear model consists of transforming raw data into suitable input features. tf.learn uses the FeatureColumn abstraction to enable these transformations.

設計一個線性模型有許多工作，包括轉換原始數據成一個適合的輸入特征。tf.learn 使用 FeatureColumn 來抽象使用這些變換。

A FeatureColumn represents a single feature in your data. A FeatureColumn may represent a quantity like 'height', or it may represent a category like 'eye_color' where the value is drawn from a set of discrete possibilities like {'blue', 'brown', 'green'}.

一個FeatureColumn 在你的數據中表示一個單個特征。 一個 FeatyreColumn 可能會表示成 '高度' 的數量，或者它可能表示成一個 '眼睛顏色' 的目錄并且其值可能會被繪制成一個像{'藍色', '棕色', '綠色'}的離散可能性值。

In the case of both continuous features like 'height' and categorical features like 'eye_color', a single value in the data might get transformed into a sequence of numbers before it is input into the model. The FeatureColumn abstraction lets you manipulate the feature as a single semantic unit in spite of this fact. You can specify transformations and select features to include without dealing with specific indices in the tensors you feed into the model.

在這兩種連續特征，像 '高度' 和 像 '眼睛顏色' 的 ‘分類特征’ 情況下，單個值在輸入這個模型前可能會被轉換成一個數字序列。抽象出的FeatureColumn 使得你能夠像操作語義一樣來操作這些特征。你可以在不需要處理輸入模型的張量中的特定指數的情況下，指定轉換方式并選擇要進行轉換的特征。

Sparse columns####

稀疏列####

Categorical features in linear models are typically translated into a sparse vector in which each possible value has a corresponding index or id. For example, if there are only three possible eye colors you can represent 'eye_color' as a length 3 vector: 'brown' would become [1, 0, 0], 'blue' would become [0, 1, 0] and 'green' would become [0, 0, 1]. These vectors are called "sparse" because they may be very long, with many zeros, when the set of possible values is very large (such as all English words).

分類特征在線性模型通常被轉化成一個稀疏向量，并且其對應值都有一個對應的序列或者id。例如，如果對于眼睛顏色只存在三種可能性的情況下，你可以表示‘眼睛顏色’為一個長度為3的向量：'棕色'應為[1, 0, 0], '藍色'應為[0, 1, 0]然后 '綠色' 為 [0, 0, 1]。當可能值的集合非常大時（如所有英語單詞），這些向量被稱為“”稀疏向量”，因為他們可能很長但是存在很多0值。

While you don't need to use sparse columns to use tf.learn linear models, one of the strengths of linear models is their ability to deal with large sparse vectors. Sparse features are a primary use case for the tf.learn linear model tools.

當你不需要使用稀疏來使用 tf.learn 的線性模型， 線性模型的其中一個優勢是他有能力去處理大規模的稀疏向量。稀疏特征是 tf.learn 線性模型工具的一個主要用例。

Encoding sparse columns#####

編碼稀疏列#####

FeatureColumn handles the conversion of categorical values into vectors automatically, with code like this:
eye_color = tf.contrib.layers.sparse_column_with_keys( column_name="eye_color", keys=["blue", "brown", "green"])

稀疏列主要用于自動將分類值轉化為向量值，其代碼像：
eye_color = tf.contrib.layers.sparse_column_with_keys( column_name="eye_color", keys=["blue", "brown", "green"])

where eye_color is the name of a column in your source data.

eye_color是你的源數據中一列的名稱。

You can also generate FeatureColumns for categorical features for which you don't know all possible values. For this case you would use sparse_column_with_hash_bucket(), which uses a hash function to assign indices to feature values.
education = tf.contrib.layers.sparse_column_with_hash_bucket("education", hash_bucket_size=1000)

當你不知道所有的可能值時，你同樣能夠為分類特征生成FeatureColumns。在這種情況下你應該使用sparse_column_with_hash_bucket()，其使用hash函數去為特征值分類序列。
education = tf.contrib.layers.sparse_column_with_hash_bucket("education", hash_bucket_size=1000)

Feature Crosses#####

交叉特征#####

Because linear models assign independent weights to separate features, they can't learn the relative importance of specific combinations of feature values. If you have a feature 'favorite_sport' and a feature 'home_city' and you're trying to predict whether a person likes to wear red, your linear model won't be able to learn that baseball fans from St. Louis especially like to wear red.

因為線性模型是給每個單獨的特征分配獨立的權重，所以他們無法學習特征在特定組合的重要性。如果你有一個特征 '最喜歡的運動' 和特征 '居住城市'，然后并且你試圖想要推斷一個人在什么情況下會穿紅色的衣服。你的線性模型將沒有不會學習到來自St.Louis的棒球粉絲是很喜歡穿紅色系的衣服。

You can get around this limitation by creating a new feature 'favorite_sport_x_home_city'. The value of this feature for a given person is just the concatenation of the values of the two source features: 'baseball_x_stlouis', for example. This sort of combination feature is called a feature cross.

你可以通過創建一個新的叫‘最喜歡的運動x居住城市’ 特征來克服這個限制。這個特征的值對一個給定的人（的數據）來說只是關聯了兩個源特征的值。例如，這種組合特征被稱為交叉特征。

The crossed_column() method makes it easy to set up feature crosses:
sport = tf.contrib.layers.sparse_column_with_hash_bucket("sport", hash_bucket_size=1000) city = tf.contrib.layers.sparse_column_with_hash_bucket("city", hash_bucket_size=1000) sport_x_city = tf.contrib.layers.crossed_column([sport, city], hash_bucket_size=int(1e4))

crossed_column()函數能夠很容易就建立出交叉特征。

Continuous columns####

連續列####

You can specify a continuous feature like so:
age = tf.contrib.layers.real_valued_column("age")

你像這樣可以指定一個連續特征：

Although, as a single real number, a continuous feature can often be input directly into the model, tf.learn offers useful transformations for this sort of column as well.

盡管對于一個實數，一個連續的特征一般是能夠直接輸入到模型中的。tf.learn也提供對連續列進行轉換。

Bucketization####

桶化####

Bucketization turns a continuous column into a categorical column. This transformation lets you use continuous features in feature crosses, or learn cases where specific value ranges have particular importance.

桶化轉換一個連續列為分類列。這種轉換會讓你在交叉特征中使用連續特征，或者是學習對于特定的范圍值能夠造成的重大影響。

Bucketization divides the range of possible values into subranges called buckets:
age_buckets = tf.contrib.layers.bucketized_column( age, boundaries=[18, 25, 30, 35, 40, 45, 50, 55, 60, 65])

桶化會將可能值的范圍劃分為一系列小的子范圍，這樣的子范圍被稱之為桶：

The bucket into which a value falls becomes the categorical label for that value.

其桶內新添加的值將成為該值的分類標簽

Input function####

輸入函數####

FeatureColumns provide a specification for the input data for your model, indicating how to represent and transform the data. But they do not provide the data itself. You provide the data through an input function.

FeatureColumns為你的模型輸入提供了一個規范格式，表面如何去表示和轉換這些數據。但是其本身不會提供這些數據。你需要通過一個輸入函數來提供這些數據。

The input function must return a dictionary of tensors. Each key corresponds to the name of a FeatureColumn. Each key's value is a tensor containing the values of that feature for all data instances. See Building Input Functions with tf.contrib.learn for a more comprehensive look at input functions, and input_fn in the linear models tutorial code for an example implementation of an input function.

這個輸入函數必須返回一個張量的目錄。每一個鍵值對應一個FeatureColumn的名字。每一個鍵值對應的值是一個包含所有數據實例中某類特征值的張量。詳情可見使用tf.contrib.learn建立輸入函數以便更全面的了解輸入函數，并且在線性模型教程代碼中有引入了一個輸入函數的實例。

The input function is passed to the fit() and evaluate() calls that initiate training and testing, as described in the next section.

輸入函數通過調用fit()和evaluate()來發起訓練和測試，將會在下一節介紹他們。

Linear estimators###

線性估計器###

tf.learn's estimator classes provide a unified training and evaluation harness for regression and classification models. They take care of the details of the training and evaluation loops and allow the user to focus on model inputs and architecture.

tf.learn的估計器類為回歸和分類模型提供了一個統一的訓練和評估方法。他們主要處理訓練和評估循環的細節及其允許用戶能夠更專注于模型輸入與結構。

To build a linear estimator, you can use either the tf.contrib.learn.LinearClassifier estimator or the tf.contrib.learn.LinearRegressor estimator, for classification and regression respectively.

為了建立一個線性估計器，你可以使用分別用于分類和回歸的tf.contrib.learn.LinearClassifier估計器或tf.contrib.learn.LinearRegressor估計器。

As with all tf.learn estimators, to run the estimator you just:

• Instantiate the estimator class. For the two linear estimator classes, you pass a list of FeatureColumns to the constructor.
• Call the estimator's fit() method to train it.
• Call the estimator's evaluate() method to see how it does.
  For example:
  e = tf.contrib.learn.LinearClassifier(feature_columns=[ native_country, education, occupation, workclass, marital_status, race, age_buckets, education_x_occupation, age_buckets_x_race_x_occupation], model_dir=YOUR_MODEL_DIRECTORY)
e.fit(input_fn=input_fn_train, steps=200)
# Evaluate for one step (one pass through the test data).
results = e.evaluate(input_fn=input_fn_test, steps=1)
# Print the stats for the evaluation.for key in sorted(results):
print "%s: %s" % (key, results[key])

所有的tf.learn估計器，只需要以下幾步就可以運行：

• 實例化估計器類。對于這兩種估計器類，需要傳遞一個FeatureColumns列表到其構造器中。
• 調用估計器的fit()方法去訓練它。
• 調用估計器的evaluate()方法去觀察其運行。
  例如：

Wide and deep learning##

廣度和深度學習##

The tf.learn API also provides an estimator class that lets you jointly train a linear model and a deep neural network. This novel approach combines the ability of linear models to "memorize" key features with the generalization ability of neural nets. Use tf.contrib.learn.DNNLinearCombinedClassifier to create this sort of "wide and deep" model:
e = tf.contrib.learn.DNNLinearCombinedClassifier( model_dir=YOUR_MODEL_DIR, linear_feature_columns=wide_columns, dnn_feature_columns=deep_columns, dnn_hidden_units=[100, 50])

tf.learn API同樣提供一個估計器類使得讓你聯合訓練一個線性模型和一個深度神經網絡。這種新穎的方法結合了線性模型“記憶”關鍵特征與神經網絡的泛化能力。使用tf.contrib.learn.DNNLinearCombinedClassifier去創建這種“廣度且深度”的模型：

For more information, see the Wide and Deep Learning tutorial.

欲了解更多信息，可以查看這個廣度和深度學習教程.