Scrapy的Item是進(jìn)行數(shù)據(jù)保存不可缺少的步驟，通過它進(jìn)行數(shù)據(jù)的整理并通過Pipelines進(jìn)行數(shù)據(jù)的數(shù)據(jù)庫保存，圖片下載等，它只有一種類型scrapy.Field()。
由于需要添加一個封面圖，對上面的爬蟲添加一個front_image_url字段對parse函數(shù)進(jìn)行修改

    def parse(self, response):
        """
        1、獲取文章列表頁url并交給scrapy進(jìn)行解析
        2、獲取下一個文章列表頁
        """
        article_list = response.css('#archive .floated-thumb .post-thumb a')
        for article in article_list:
            image_url = article.css("img::attr(src)").extract_first("")
            article = article.css("::attr(href)").extract_first("")
            yield Request(url=parse.urljoin(response.url, article), meta={"front_image_url":image_url}, callback=self.parse_detail)
            print(parse.urljoin(response.url, article))

其中的meta字段是傳遞值的方法。在調(diào)試時返回的response中會出現(xiàn)meta的內(nèi)容，它是一個字典，故在傳遞時可以直接通過 response.meta['front_image_url']進(jìn)行引用（也可以使用get的方法，附默認(rèn)值防止出現(xiàn)異常）：

Paste_Image.png

 def parse_detail(self, response):
        front_image_url = response.meta.get("front_image_url", "") #文章封面圖
        title = response.css('div.entry-header h1::text').extract_first("0")
        create_date = response.css('p.entry-meta-hide-on-mobile::text').extract()[0].replace('·','').strip()
        fav_nums = response.css("span.bookmark-btn::text").extract_first("0")
        praise_nums = response.css("div.post-adds h10::text").extract_first("0")

在Items.py中，定義一個Item并聲明它的字段內(nèi)容：

class JobBoleArticleItem(scrapy.Item):
    title = scrapy.Field()
    create_date = scrapy.Field()
    url = scrapy.Field()  # 保存url
    url_object_id = scrapy.Field() # 保存url的md5值去重等
    front_image_url = scrapy.Field()
    front_image_path = scrapy.Field() # 保存封面圖的本地路徑
    praise_nums = scrapy.Field()
    comment_nums = scrapy.Field()
    fav_nums = scrapy.Field()
    tags = scrapy.Field()
    content = scrapy.Field()

在jobbole.py中from articlespider.items import JobBoleArticleItem進(jìn)行引用
并在parse_detail中進(jìn)行初始化article_item = JobBoleArticleItem()，總體代碼如下：

    def parse_detail(self, response):
        article_item = JobBoleArticleItem()
        front_image_url = response.meta.get("front_image_url", "") #文章封面圖
        title = response.css('div.entry-header h1::text').extract_first("0")
        create_date = response.css('p.entry-meta-hide-on-mobile::text').extract()[0].replace('·','').strip()
        fav_nums = response.css("span.bookmark-btn::text").extract_first("0")
        praise_nums = response.css("div.post-adds h10::text").extract_first("0")
        match_re = re.match(".*?(\d+).*", fav_nums)
        if match_re:
            fav_nums = match_re.group(1)
        else:
            fav_nums = 0
        comment_nums = response.css("a[href='#article-comment'] span::text").extract_first("0")
        match_re = re.match(".*?(\d+).*", comment_nums)
        if match_re:
            comment_nums = match_re.group(1)
        else:
            comment_nums = 0
        content = response.css(".entry").extract_first("0")
        tag_list = response.css('p.entry-meta-hide-on-mobile a::text').extract()
        tag_list = [e for e in tag_list if not e.strip().endswith("評論")]
        tags = ",".join(tag_list)
        article_item['title'] = title
        article_item['url'] = response.url
        article_item['create_date'] = create_date
        article_item['front_image_url'] = front_image_url
        article_item['praise_nums'] = praise_nums
        article_item['comment_nums'] = comment_nums
        article_item['fav_nums'] = fav_nums
        article_item['tags'] = tags
        article_item['content'] = content
        yield article_item

最后一句yield article_item會自動提交到settings中的ITEM_PIPELINES進(jìn)行處理。
此時在pipelines.py中設(shè)置斷點(diǎn)調(diào)試，可以看到article_item中的值已經(jīng)傳遞到這里了。

Paste_Image.png