urllib模塊

方法

urllib.request.urlopen(網(wǎng)址)

1. 打開一個網(wǎng)址，返回一個請求對象(request)

request.read()

1. 返回值是二進制的

request.readline()

1. 返回值是二進制的
2. 需要用 decode("utf-8")進行解碼

抓取天涯郵箱，逐行讀取

import re
import urllib
import urllib.request

mailRegex = re.compile("([A-Z0-9._%+-]+@[A-Z0-9.-]+\.[A-Z]{2,4})", re.IGNORECASE)
mailList = []
for line in urllib.request.urlopen("http://bbs.tianya.cn/m/post-140-393974-6.shtml"):
    myList = mailRegex.findall(line.decode("utf-8"))
    if myList:
        mailList.extend(myList)

print(mailList)

抓取天涯郵箱，全部讀取

import re
import urllib
import urllib.request

mailRegex = re.compile("([A-Z0-9._%+-]+@[A-Z0-9.-]+\.[A-Z]{2,4})", re.IGNORECASE)
mailList = []
myStr = urllib.request.urlopen("http://bbs.tianya.cn/m/post-140-393974-6.shtml").read()
mailList.extend(mailRegex.findall(myStr.decode("utf-8")))
print(mailList)

股票代碼以及股票所在公司名抓取

下載

tup1 = urllib.request.urlretrieve("網(wǎng)址","1.jpg")

urlretrieve()

1. 參數(shù)1：網(wǎng)址
2. 參數(shù)2：目標地址
3. 返回值一個元組("存儲地址", "HttpMessage實例")

股票數(shù)據(jù)下載

url="http://quotes.money.163.com/service/chddata.html?code=1300133&end=20130523&fields=TCLOSE;HIGH;LOW;TOPEN;LCLOSE;CHG;PCHG;TURNOVER;VOTURNOVER;VATURNOVER;TCAP;MCAP"

鏈接分析

1. code=1300133,1代表深市,0代表滬市,后面接著的是股票代碼
2. end=20130523,股票交易日期

下載python庫

xml支持

pip install lxml    # lxml支持

處理數(shù)據(jù)

pip install pandas  # 處理數(shù)據(jù)

科學計算

pip install numpy   # 科學計算

財經(jīng)數(shù)據(jù)接口

pip install tushare # 財經(jīng)數(shù)據(jù)接口