原文鏈接:http://www.cnblogs.com/cv-pr/p/7081861.html
Logistic回歸是機器學習中非常經典的一個方法，主要用于解決二分類問題，它是多分類問題softmax的基礎，而softmax在深度學習中的網絡后端做為常用的分類器，接下來我們將從原理和實現來闡述該算法的思想。
1.原理
擬合的效果圖如下所示：
w
迭代的梯度變化非常快，五次就能達到非常好的結果，如下所示：
[ 4.54704357]
[ 0.19111694]
[ 0.2380104]
[ 0.01743344]
[? 8.45306379e-05]
[? 1.95907862e-09]
[? 1.90137901e-16]
[? 1.90137901e-16]
[? 1.90137901e-16]
[? 1.90137901e-16]
我們使用了python實現Logistic回歸，注意：我們這里對H
是直接的求逆，如果特征維度很高的情況下，這會消耗較大的計算亮，因此我們可以采用更有效的求解方法，如Cholesky分解法，最后貼上馬農最愛的代碼：
import numpy as np
from matplotlib import pyplot as plt
class LogisticClassifier:
def __init__(self):
print("init");
def logistic(self,Xa,wa):
val = 1/(1+np.exp(-Xa.dot(wa)));
return val;
def train(self,X,t,iter_num):
print("start to training");
Xa = np.array(X)
xsize = Xa.shape
dim = xsize[1]+1
num = xsize[0]
Xa = np.c_[Xa,np.ones([num,1])]
ta = np.array(t)
print dim,num
wa = 0.5*np.ones([dim,1])
for it in range(iter_num):
ya = self.logistic(Xa,wa)
deriv_wa = Xa.T.dot(ya-ta)
R = np.diag((ya*(1-ya)).flat)
H = Xa.T.dot(R).dot(Xa)
delta_w = np.linalg.inv(H).dot(deriv_wa)
wa = wa - delta_w;
print np.linalg.norm(delta_w.T, 2, 1)
#print wa
return wa
if __name__ == "__main__":
print ('This is main of module "hello.py"')
logCls = LogisticClassifier();
#construct data
X = [[0.5],[0.75],[1],[1.25],[1.5],[1.75],[1.75],[2],[2.25],[2.5],[2.75],[3],[3.25],[3.5],[4],[4.25],[4.5],[4.75],[5],[5.5]]
t = [[0],[0],[0],[0],[0],[0],[1],[0],[1],[0],[1],[0],[1],[0],[1],[1],[1],[1],[1],[1]]
iter_num = 10;
#training weight
w = logCls.train(X, t, iter_num)
print ("learned weight:\n")
print w
#draw and show the result
pos_t = [x for i, x in enumerate(t) if x == [1]]
pos_X = [X[i] for i, x in enumerate(t) if x == [1]]
neg_t = [x for i, x in enumerate(t) if x == [0]]
neg_X = [X[i] for i, x in enumerate(t) if x == [0]]
plt.scatter(pos_X,pos_t,color="r",marker='o',s = 100)
plt.scatter(neg_X,neg_t,color="g",marker='o',s = 100)
Xfitted? = np.array(np.linspace(0,6,100))
XfittedC = np.c_[Xfitted,np.ones([100,1])]
Yfitted = logCls.logistic(XfittedC, w)
plt.plot(Xfitted.flat,Yfitted.flat,color="b",linewidth= 5)
#reset the axes
ax = plt.gca()
#no bonding box
ax.spines['top'].set_color('none')
ax.spines['right'].set_color('none')
#set as zero
ax.xaxis.set_ticks_position('bottom')
ax.spines['bottom'].set_position(('data',0.5))
ax.yaxis.set_ticks_position('left')
ax.spines['left'].set_position(('data',3))
plt.xlabel("X",fontsize="xx-large")
plt.ylabel("t",fontsize="xx-large")
plt.title("Logistic method,learned weight:[%f,%f]"%(w[0],w[1]),fontsize="xx-large")
plt.legend(["Fitted function","Postive Samples","Negative Samples"],fontsize="xx-large",loc='upper left');
plt.show()
3.參考資料
[1].Logistic回歸與梯度下降法
[2].Logistic回歸與牛頓迭代法
Make Change - Focus on Computer Vision and Pattern Recognition
版權聲明：本文為博主原創文章，未經博主允許不得轉載
