前言
嗨嘍！大家好呀，這里是魔王~**
本次必備素材:
- wkhtmltopdf [軟件]
- 素材代碼
第三方庫：
- requests >>> pip install requests
- parsel >>> pip install parsel
- pdfkit >>> pip install pdfkit
開發環境：
- 版 本：python3.8
- 編輯器：pycharm
win + R 輸入cmd 輸入安裝命令 pip install 模塊名 如果出現爆紅 可能是因為 網絡連接超時 切換國內鏡像源
采集流程:
一. 分析想要數據內容, 可以從哪里獲取
通過開發者工具進行抓包分析, 分析之后可得, 我們想要數據內容其實就請求導航欄url地址即可
二. 代碼實現步驟:
獲取多個文章內容(獲取所有文章url地址)
- 發送請求, 對于文章目錄頁面發送請求
- 獲取數據, 獲取網頁源代碼數據 文本數據
- 解析數據, 提取文章url地址
獲取文章內容代碼
- 發送請求, 對于url地址發送請求
- 獲取數據, 獲取網頁源代碼數據
- 解析數據, 提取文章內容
- 保存數據, 先保存成html文件, 再把html文件轉成PDF
代碼
# import requests # 數據請求模塊
# import parsel # 數據解析模塊
# import re # 正則表示
# import pdfkit
# import subprocess
# for page in range(4, 6):
# url = f'https://blog.csdn.net/fei347795790/article/list/{page}' # 確定請求網址
# # headers 請求頭, 主要用于偽裝python, 防止程序被服務器識別出來
# headers = {
# 'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.4896.88 Safari/537.36'
# }
# # 用requests模塊里面get方式發送請求
# response = requests.get(url=url, headers=headers)
# # print(response.text) # <Response [200]> 響應對象 200 表示請求成功
# selector = parsel.Selector(response.text) # <Selector xpath=None data='<html lang="zh-CN">\n<head>\n <meta ...'> 返回對象
# # css 是解析方式之一 根據標簽屬性內容提取數據 a::attr(href) 獲取a標簽里面href屬性
# href = selector.css('#articleMeList-blog > div.article-list > div > h4 > a::attr(href)').getall()
# # print(href)
# for index in href:
# try:
# print(index)
# html_data = requests.get(url=index, headers=headers).text
# selector_1 = parsel.Selector(html_data)
# title = selector_1.css('#articleContentId::text').get()
# cmd = f'C:\\01-Software-installation\\wkhtmltopdf\\bin\\wkhtmltopdf.exe {index} pdf_1\\{title}.pdf'
# subprocess.run(cmd, shell=True)
# except Exception as e:
# print(e)
import requests
url = 'https://blog.csdn.net/phoenix/web/v1/comment/submit'
like_url = 'https://blog.csdn.net//phoenix/web/v1/article/like'
headers = {
'cookie': 'uuid_tt_dd=10_29360288410-1640936706807-857482; __gads=ID=1a4feb23074a3469-22da76a196cf0001:T=1640936708:RT=1640936708:S=ALNI_MawGCakjM400IbVY204TvKfKLhDlg; Hm_lvt_e5ef47b9f471504959267fd614d579cd=1645514550; __gpi=UID=0000049689281fe2:T=1649317424:RT=1649317424:S=ALNI_MYlX9R83NQ5EzlFY5UgNF09G45dPw; c_dl_prid=-; c_dl_rid=1650090830371_447095; c_dl_fref=https://so.csdn.net/so/search; c_dl_fpage=/download/qq_43651710/10848772; c_dl_um=distribute.pc_search_result.none-task-blog-2%7Evipall%7Efirst_rank_ecpm_v1%7Erank_v31_ecpm-1-114898691.nonecase; dc_session_id=10_1650262926080.949004; c_first_ref=www.baidu.com; c_first_page=https%3A//blog.csdn.net/fei347795790/category_11731395.html; c_segment=10; Hm_lvt_6bcd52f51e9b3dce32bec4a3997715ac=1650090803,1650095679,1650112607,1650262927; firstDie=1; hide_login=1; dc_sid=70fca81ac8fa563314905c0e38f533b9; unlogin_scroll_step=1650263871780; c_pref=default; SESSION=eb13b53e-41e8-43e0-aa6c-54811bb65d0c; c_ref=https%3A//blog.csdn.net/fei347795790/article/details/110070943; ssxmod_itna=Qqfx2DB7D=DQexCq0LpO8D9i8DORYYrQN7Yd7DlOiQxA5D8D6DQeGTTRdY=T1zCep+uDQDRgyfKlFpO2GWKk7YawWsUnO4GLDmKDyKA=ueDxOq0rD74irDDxD39D7PGmDiWZRD72=1lSgK8DWKDKx0kDY5Dw=AGDiPD7gFeCB9w1g911pBGd4D1qCvxKBKD9x0CDlPxf9GkDDyf69isyo3EDmb3A1BhDCKDjg71s6YDUeysgaFU/j0aAnT5YQxxLQi4Kg0Dt=2DK2GYGQpN1nredjDxfsrFTnTqDDpxpywx4D===; ssxmod_itna2=Qqfx2DB7D=DQexCq0LpO8D9i8DORYYrQN7YdD6h8iQD0vxLx03qKru2d+UOqcnUg8xhCDRoHKH1SQqrUY0iFWAxm=RhDFIOD8xod7VS8Bv0+m23mlQcq+912jIp1r/8bM1z9ZgSyzg5CKBhHsmH8BeHiq8wHMDp1prTH5eoO5FE83p976COKCP57q35OWchz=iuDVBi5KB4GeDIbenWenPaKBYrmQWWek4qqcAFWKnxt0/M=u0pK0nDH5M+rPa1eVQQxRaZDREMbBYBbi5mb17K13xzFV+en8OpHAqw+pp5dK4=R7caLRTTSb5K91ea5UFt8D4QRiIhqRrfRvY+eu3qEY9QQR0z44fK=RGxd4eDPiR+10hu+FCIxaBe1Ue=QB7YnpQc/FwEWvP=mO+4sAHn95OQwbC9H/p+mTa9E/lIP2bcWFk+mwB9N/Ej9ID2xYE+aLSiPkWWT=iiK+aT0bWKAsYGdWnDDgDcIQr4ORGCBGmQPG7O2Y7VmmARgGWWKoqszEmiwB0m7gWRz91N+QE4wXTt78wCo3LWZRxCkoO7m1KT4rmvfKxZ+NITqbgw/hrixDKd9D7=DYFqeD===; UserName=weixin_43239784; UserInfo=b58cf84406a84acebf2c3f36442f1c59; UserToken=b58cf84406a84acebf2c3f36442f1c59; UserNick=%E6%97%A0%E9%9B%A8%E0%B8%88%E0%B8%B8%E0%B9%8A%E0%B8%9A; AU=1D5; UN=weixin_43239784; BT=1650268841955; p_uid=U010000; c_page_id=default; dc_tos=raizmv; log_Id_pv=153; Hm_lpvt_6bcd52f51e9b3dce32bec4a3997715ac=1650268904; Hm_up_6bcd52f51e9b3dce32bec4a3997715ac=%7B%22islogin%22%3A%7B%22value%22%3A%221%22%2C%22scope%22%3A1%7D%2C%22isonline%22%3A%7B%22value%22%3A%221%22%2C%22scope%22%3A1%7D%2C%22isvip%22%3A%7B%22value%22%3A%220%22%2C%22scope%22%3A1%7D%2C%22uid_%22%3A%7B%22value%22%3A%22weixin_43239784%22%2C%22scope%22%3A1%7D%7D; Hm_ct_6bcd52f51e9b3dce32bec4a3997715ac=6525*1*10_29360288410-1640936706807-857482!5744*1*weixin_43239784; log_Id_view=478; log_Id_click=110',
'origin': 'https://blog.csdn.net',
'referer': 'https://blog.csdn.net/fei347795790/article/details/110070943',
'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.4896.88 Safari/537.36',
'x-requested-with': 'XMLHttpRequest',
'x-tingyun-id': 'im-pGljNfnc;r=268943811',
}
data = {
'commentId': '',
'content': '自游老師真帥',
'articleId': '124196275',
}
like_data = {
'articleId': '110070943'
}
# response = requests.post(url=url, data=data, headers=headers)
response = requests.post(url=like_url, data=like_data, headers=headers)
print(response)
尾語
好了，我的這篇文章寫到這里就結束啦！
有更多建議或問題可以評論區或私信我哦！一起加油努力叭(? ?_?)?
喜歡就關注一下博主，或點贊收藏評論一下我的文章叭！！！
