Xc.Li
Apr.2016

Python部分

應(yīng)用Python的BeautifulSoup庫(kù)和RegEx獲取圖書館網(wǎng)頁(yè)上的座位信息。連續(xù)多天在相同時(shí)間獲取數(shù)據(jù)，便于橫向比較。
簡(jiǎn)單來(lái)說(shuō)就是在每天7：00~23：00之間在每個(gè)小時(shí)的00，10，20，30，40，50分鐘獲取圖書館當(dāng)時(shí)所有樓層的上座情況，然后保存到一個(gè)文件里。
獲取數(shù)據(jù)的設(shè)計(jì)思路參考(這個(gè)是上學(xué)期編的程序，略有不同)：用python自動(dòng)獲取圖書館空座信息
獲得的數(shù)據(jù)保存為csv格式，數(shù)據(jù)格式：

date.csv
'time', dec_time,一樓在用，一樓空閑，二樓在用，二樓空閑...六樓在用，六樓空閑
例如：
"D:\OneDrive\Python\data\lib\Apr13.csv"
'22:30:07', 22.5, 19, 17, 150, 172, 203, 258, 153, 219, 135, 214, 34, 22

Stata部分

csv文件導(dǎo)入到Stata：
可以生存一張描繪所有樓層各個(gè)時(shí)間的圖像，生成dta文件，便于進(jìn)一步處理

Apr13.png

change-format.do
clear
local date Apr13
local day Apr13_Wed3
infile str8 `day'time dec_time `day'l1a `day'l1b `day'l2a `day'l2b `day'l3a `day'l3b `day'l4a `day'l4b `day'l5a `day'l5b `day'l6a `day'l6b using D:\OneDrive\Python\data\lib\\`date'.csv
graph twoway line `day'l1a dec_time ||line `day'l2a dec_time || line `day'l3a dec_time || line `day'l4a dec_time || line `day'l5a dec_time || line `day'l6a dec_time, xlabel(7(1)23) ylabel(0(30)350)
save "D:\lib-data\\`date'.dta"

相關(guān)的python程序：

kol模塊：自動(dòng)切換Wifi網(wǎng)絡(luò)保證在線
timer模塊：每五分鐘運(yùn)行一次命令的實(shí)現(xiàn)
writocsv模塊：見(jiàn)下文

主程序

# get library seats
# Designed By Xc.Li
import urllib
import BeautifulSoup
import re
import time
from os import system

import kol
import writetocsv
import timer


def find_number(keyword):
    for tag in tags:
        s_tag = str(tag)
        flag = re.match(keyword, s_tag)
        if flag is not None:
            return s_tag[54:-18]

while True:
    timer.func(10)
    kol.func()
    url = 'http://lib.ecust.edu.cn:8081/gateseat/lrp.aspx'
    html = urllib.urlopen(url).read()
    soup = BeautifulSoup.BeautifulSoup(html)
    tags = soup('span')

    output = list()
    output.append(time.ctime()[11:-5])
    # time in decimal
    hour = int(time.ctime()[11:13])
    minute = int(time.ctime()[14:16])
    decimal_time = round(hour + minute/60.0, 2)
    output.append(decimal_time)

    for floor in [1, 2, 3, 4, 5, 6]:
        # print 'Floor:', floor
        keyword = '^<span id="Label' + str(floor) + 'f1' + '".*>'
        # print 'Used:', find_number(keyword)
        output.append(int(find_number(keyword)))
        keyword = '^<span id="Label' + str(floor) + 'f2' + '".*>'
        # print 'Left:', find_number(keyword)
        output.append(int(find_number(keyword)))
    print output
    str_output = str(output)[1:-1]
    writetocsv.write('lib', 'default', str_output)
    time.sleep(60)
    system('cls')
    # auto-quit
    if hour == 22 and minute > 29:
        print "program ends!"
        writetocsv.write('lib', 'default', "program end at:%s" % time.ctime())
        time.sleep(5)
        break

writetocsv模塊

# Write to CSV
# Designed By Xc.Li @ Mar.2016
import time
import os


def write(dirname, filename, content):
    if filename == "default":
        filename = time.ctime()[4:7]+time.ctime()[8:10]
    file_name = "D:\OneDrive\Python\\data\%s\%s.csv" % (dirname, filename)
    dir_name = "D:\OneDrive\Python\\data\%s" % dirname
    try:
        tf = open(file_name, 'a')
    except:
        os.system("mkdir "+dir_name)
        tf = open(file_name, 'a')
    tf.write(content)
    tf.write('\n')
    tf.close()