• Dataset transformations| 數(shù)據(jù)轉(zhuǎn)換

• Combining estimators|組合學(xué)習(xí)器
• Feature extration|特征提取
• Preprocessing data|數(shù)據(jù)預(yù)處理

<p id='1'>1 Dataset transformations</p>

scikit-learn provides a library of transformers, which may clean (see Preprocessing data), reduce (see Unsupervised dimensionality reduction), expand (see Kernel Approximation) or generate (see Feature extraction) feature representations.

scikit-learn 提供了數(shù)據(jù)轉(zhuǎn)換的模塊，包括數(shù)據(jù)清理、降維、擴(kuò)展和特征提取。

Like other estimators, these are represented by classes with fit method, which learns model parameters (e.g. mean and standard deviation for normalization) from a training set, and a transform method which applies this transformation model to unseen data. fit_transform may be more convenient and efficient for modelling and transforming the training data simultaneously.

scikit-learn模塊有3種通用的方法：fit(X,y=None)、transform(X)、fit_transform(X)、inverse_transform(newX)。fit用來(lái)訓(xùn)練模型；transform在訓(xùn)練后用來(lái)降維；fit_transform先用訓(xùn)練模型，然后返回降維后的X；inverse_transform用來(lái)將降維后的數(shù)據(jù)轉(zhuǎn)換成原始數(shù)據(jù)。

<p id='1.1'>1.1 combining estimators</p>

• <p id='1.1.1'>1.1.1 Pipeline:chaining estimators</p>

Pipeline 模塊是用來(lái)組合一系列估計(jì)器的。對(duì)固定的一系列操作非常便利，如：同時(shí)結(jié)合特征選擇、數(shù)據(jù)標(biāo)準(zhǔn)化、分類。

• Usage|使用
  代碼：

from sklearn.pipeline import Pipeline  
from sklearn.svm import SVC 
from sklearn.decomposition import PCA
from sklearn.pipeline import make_pipeline
#define estimators
#the arg is a list of (key,value) pairs,where the key is a string you want to give this step and value is an estimators object
estimators=[('reduce_dim',PCA()),('svm',SVC())]  
#combine estimators
clf1=Pipeline(estimators)
clf2=make_pipeline(PCA(),SVC())  #use func make_pipeline() can do the same thing
print(clf1,'\n',clf2) 

輸出：

Pipeline(steps=[('reduce_dim', PCA(copy=True, n_components=None, whiten=False)), ('svm',           SVC(C=1.0, cache_size=200, class_weight=None, coef0=0.0,
  decision_function_shape=None, degree=3, gamma='auto', kernel='rbf',
  max_iter=-1, probability=False, random_state=None, shrinking=True,
  tol=0.001, verbose=False))]) 
 Pipeline(steps=[('pca', PCA(copy=True, n_components=None, whiten=False)), ('svc', SVC(C=1.0, cache_size=200, class_weight=None, coef0=0.0,
  decision_function_shape=None, degree=3, gamma='auto', kernel='rbf',
  max_iter=-1, probability=False, random_state=None, shrinking=True,
  tol=0.001, verbose=False))])

可以通過(guò)set_params()方法設(shè)置學(xué)習(xí)器的屬性，參數(shù)形式為<estimator>_<parameter>

clf.set_params(svm__C=10)

上面的方法在網(wǎng)格搜索時(shí)很重要：

from sklearn.grid_search import GridSearchCV
params = dict(reduce_dim__n_components=[2, 5, 10],svm__C=[0.1, 10, 100])
grid_search = GridSearchCV(clf, param_grid=params)

上面的例子相當(dāng)于把pipeline生成的學(xué)習(xí)器作為一個(gè)普通的學(xué)習(xí)器，參數(shù)形式為<estimator>_<parameter>。

• Note|說(shuō)明
  1.可以使用dir()函數(shù)查看clf的所有屬性和方法。例如step屬性就是每個(gè)操作步驟的屬性。
  如

('reduce_dim', PCA(copy=True, n_components=None, whiten=False))

2.調(diào)用pipeline生成的學(xué)習(xí)器的fit方法相當(dāng)于依次調(diào)用其包含的所有學(xué)習(xí)器的方法，transform輸入然后把結(jié)果扔向下一步驟。pipeline生成的學(xué)習(xí)器有著它包含的學(xué)習(xí)器的所有方法。如果最后一個(gè)學(xué)習(xí)器是分類，那么生成的學(xué)習(xí)器就是分類，如果最后一個(gè)是transform，那么生成的學(xué)習(xí)器就是transform，依次類推。

• <p id='1.1.2'> 1.1.2 FeatureUnion: composite feature spaces</p>

與pipeline不同的是FeatureUnion只組合transformer，它們也可以結(jié)合成更復(fù)雜的模型。

FeatureUnion combines several transformer objects into a new transformer that combines their output. AFeatureUnion takes a list of transformer objects. During fitting, each of these is fit to the data independently. For transforming data, the transformers are applied in parallel, and the sample vectors they output are concatenated end-to-end into larger vectors.

• Usage|使用
  代碼：

from sklearn.pipeline import FeatureUnion   
from sklearn.decomposition import PCA
from sklearn.decomposition import KernelPCA
from sklearn.pipeline import make_union
#define transformers
#the arg is a list of (key,value) pairs,where the key is a string you want to give this step and value is an transformer object
estimators=[('linear_pca)',PCA()),('Kernel_pca',KernelPCA())]  
#combine transformers
clf1=FeatureUnion(estimators)
clf2=make_union(PCA(),KernelPCA())
print(clf1,'\n',clf2) 
print(dir(clf1))

輸出：

FeatureUnion(n_jobs=1,
       transformer_list=[('linear_pca)', PCA(copy=True, n_components=None, whiten=False)), ('Kernel_pca', KernelPCA(alpha=1.0, coef0=1, degree=3, eigen_solver='auto',
     fit_inverse_transform=False, gamma=None, kernel='linear',
     kernel_params=None, max_iter=None, n_components=None,
     remove_zero_eig=False, tol=0))],
       transformer_weights=None) 
 FeatureUnion(n_jobs=1,
       transformer_list=[('pca', PCA(copy=True, n_components=None, whiten=False)), ('kernelpca', KernelPCA(alpha=1.0, coef0=1, degree=3, eigen_solver='auto',
     fit_inverse_transform=False, gamma=None, kernel='linear',
     kernel_params=None, max_iter=None, n_components=None,
     remove_zero_eig=False, tol=0))],
       transformer_weights=None)

可以看出FeatureUnion的用法與pipeline一致

• Note|說(shuō)明

(A [FeatureUnion
](http://scikit- learn.org/stable/modules/generated/sklearn.pipeline.FeatureUnion.html#sklearn.pipeline.FeatureUn ion) has no way of checking whether two transformers might produce identical features. It only produces a union when the feature sets are disjoint, and making sure they are is the caller’s responsibility.)

Here is a example python source code:[feature_stacker.py](http://scikit-learn.org/stable/_downloads/feature_stacker.py)

<p id='1.2'>1.2 Feature extraction</p>

The sklearn.feature_extraction
module can be used to extract features in a format supported by machine learning algorithms from datasets consisting of formats such as text and image.

skilearn.feature_extraction模塊是用機(jī)器學(xué)習(xí)算法所支持的數(shù)據(jù)格式來(lái)提取數(shù)據(jù)，如將text和image信息轉(zhuǎn)換成dataset。
Note:
Feature extraction（特征提取）與Feature selection（特征選擇）不同，前者是用來(lái)將非數(shù)值的數(shù)據(jù)轉(zhuǎn)換成數(shù)值的數(shù)據(jù)，后者是用機(jī)器學(xué)習(xí)的方法對(duì)特征進(jìn)行學(xué)習(xí)（如PCA降維）。

• <p id='1.2.1'>1.2.1 Loading features from dicts</p>

The class DictVectorizer
can be used to convert feature arrays represented as lists of standard Python dict
objects to the NumPy/SciPy representation used by scikit-learn estimators.
Dictvectorizer類用來(lái)將python內(nèi)置的dict類型轉(zhuǎn)換成數(shù)值型的array。dict類型的好處是在存儲(chǔ)稀疏數(shù)據(jù)時(shí)不用存儲(chǔ)無(wú)用的值。

代碼：

measurements=[{'city': 'Dubai', 'temperature': 33.}
,{'city': 'London', 'temperature':12.}
,{'city':'San Fransisco','temperature':18.},]
from sklearn.feature_extraction import DictVectorizer
vec=DictVectorizer()
x=vec.fit_transform(measurements).toarray()
print(x)
print(vec.get_feature_names())```
輸出：

[[ 1. 0. 0. 33.]
[ 0. 1. 0. 12.]
[ 0. 0. 1. 18.]]
['city=Dubai', 'city=London', 'city=San Fransisco', 'temperature']
[Finished in 0.8s]

* ###<p id='1.2.2'>1.2.2 Feature hashing</p>
* ###<p id='1.2.3'>1.2.3 Text feature extraction</p>
* ###<p id='1.2.4'>1.2.4 Image feature extraction</p>
以上三小節(jié)暫未考慮（設(shè)計(jì)到語(yǔ)言處理及圖像處理）[見(jiàn)官方文檔][官方文檔]
[官方文檔]: http://scikit-learn.org/stable/data_transforms.html

##<p id='1.3'>1.3 Preprogressing data</p>
>The sklearn.preprocessing
 package provides several common utility functions and transformer classes to change raw feature vectors into a representation that is more suitable for the downstream estimators

sklearn.preprogressing模塊提供了幾種常見(jiàn)的數(shù)據(jù)轉(zhuǎn)換，如標(biāo)準(zhǔn)化、歸一化等。
* ###<p id='1.3.1'>1.3.1 Standardization, or mean removal and variance scaling</p>
>**Standardization** of datasets is a **common requirement for many machine learning estimators** implemented in the scikit; they might behave badly if the individual features do not more or less look like standard normally distributed data: Gaussian with **zero mean and unit variance**.

 很多學(xué)習(xí)算法都要求事先對(duì)數(shù)據(jù)進(jìn)行標(biāo)準(zhǔn)化，如果不是像標(biāo)準(zhǔn)正太分布一樣0均值1方差就可能會(huì)有很差的表現(xiàn)。

 * Usage|用法

 代碼：
```python
from sklearn import preprocessing
import numpy as np
X = np.array([[1.,-1., 2.], [2.,0.,0.], [0.,1.,-1.]])
Y=X
Y_scaled = preprocessing.scale(Y)
y_mean=Y_scaled.mean(axis=0) #If 0, independently standardize each feature, otherwise (if 1) standardize each sample|axis=0 時(shí)求每個(gè)特征的均值，axis=1時(shí)求每個(gè)樣本的均值
y_std=Y_scaled.std(axis=0)
print(Y_scaled)
scaler= preprocessing.StandardScaler().fit(Y)#用StandardScaler類也能完成同樣的功能
print(scaler.transform(Y))

輸出：

[[ 0.         -1.22474487  1.33630621]
 [ 1.22474487  0.         -0.26726124]
 [-1.22474487  1.22474487 -1.06904497]]
[[ 0.         -1.22474487  1.33630621]
 [ 1.22474487  0.         -0.26726124]
 [-1.22474487  1.22474487 -1.06904497]]
[Finished in 1.4s]

• Note|說(shuō)明
  1.func scale
  2.class StandardScaler
  3.StandardScaler 是一種Transformer方法，可以讓pipeline來(lái)使用。
  MinMaxScaler （min-max標(biāo)準(zhǔn)化[0,1]）類和MaxAbsScaler（[-1,1]）類是另外兩個(gè)標(biāo)準(zhǔn)化的方式，用法和StandardScaler類似。
  4.處理稀疏數(shù)據(jù)時(shí)用MinMax和MaxAbs很合適
  5.魯棒的數(shù)據(jù)標(biāo)準(zhǔn)化方法（適用于離群點(diǎn)很多的數(shù)據(jù)處理）：

the median and the interquartile range often give better results

用中位數(shù)代替均值（使均值為0），用上四分位數(shù)-下四分位數(shù)代替方差（IQR為1？）。

• <p id='1.3.2'>1.3.2 Impution of missing values|缺失值的處理</p>

• Usage
  代碼：

import scipy.sparse as sp
from sklearn.preprocessing import Imputer
X=sp.csc_matrix([[1,2],[0,3],[7,6]])
imp=preprocessing.Imputer(missing_value=0,strategy='mean',axis=0)
imp.fit(X)
X_test=sp.csc_matrix([[0, 2], [6, 0], [7, 6]])
print(X_test)
print(imp.transform(X_test))

輸出：

  (1, 0)    6
  (2, 0)    7
  (0, 1)    2
  (2, 1)    6
[[ 4.          2.        ]
 [ 6.          3.66666675]
 [ 7.          6.        ]]
[Finished in 0.6s]

• Note
  1.scipy.sparse是用來(lái)存儲(chǔ)稀疏矩陣的
  2.Imputer可以用來(lái)處理scipy.sparse稀疏矩陣

• <p id='1.3.3'>1.3.3 Generating polynomial features</p>

• Usage
  代碼：

import numpy as np
from sklearn.preprocessing import PolynomialFeatures
X=np.arange(6).reshape(3,2)
print(X)
poly=PolynomialFeatures(2)
print(poly.fit_transform(X))

輸出：

[[0 1]
 [2 3]
 [4 5]]
[[  1.   0.   1.   0.   0.   1.]
 [  1.   2.   3.   4.   6.   9.]
 [  1.   4.   5.  16.  20.  25.]]
[Finished in 0.8s]

• Note
  生成多項(xiàng)式特征用在多項(xiàng)式回歸中以及多項(xiàng)式核方法中 。

• <p id='1.3.4'>1.3.4 Custom transformers</p>

這是用來(lái)構(gòu)造transform方法的函數(shù)

• Usage:
  代碼：

import numpy as np
from sklearn.preprocessing import FunctionTransformer
transformer = FunctionTransformer(np.log1p)
x=np.array([[0,1],[2,3]])
print(transformer.transform(x))

輸出：

[[ 0.          0.69314718]
 [ 1.09861229  1.38629436]]
[Finished in 0.8s]

• Note

For a full code example that demonstrates using a FunctionTransformer
to do custom feature selection, see Using FunctionTransformer to select columns