一、前言

從現在開始會偶爾寫一些付費的內容，之所以要寫付費內容，主要基于以下幾點，占比權重依次降低。
1、我喜歡錢。
2、建立有用知識的門檻，把有用的東西給予真正需要的人。
3、通過付費倒逼我寫出更好的文章，如果文章不好你可以盡情吐槽，前提是你花了錢。

二、抖音介紹

抖音現在真的是越來越火，我們公司的“一面APP”也想做對抖音APP的數據抓取。基于此，我下載抖音，開始研究它的HTTP請求內容，后來我發現它有毒，越刷越上癮，總感覺下一個視頻一定會更精彩，根本停不下來，似乎早已忘記當初為什么下載抖音...
當然，最后我還是控制著了。

三、獲取抖音中某個“挑戰”下熱門視頻及單個視頻的URL

關于對APP進行抓包的方法，在這我不做講解，這個方法在網上可以找到很多相關資料（收費價值不大）。

如果我們要獲取抖音中某個“挑戰”下熱門視頻，大概的步驟如下：
1）配置好Fiddler或Charles（我這邊是使用Charles），通過Charles可以對手機APP進行抓包，如下圖1所示；

圖1

2）在抖音APP上進行挑戰的內容搜索，例如搜索“絕地求生”，如下圖2所示；

圖2

3）點擊“絕地求生”挑戰，查看Charles中的HTTP請求數據，找出獲取挑戰下視頻的API，如下圖3所示；

圖3

我們發現在我們點擊“絕地求生”進入這個挑戰的詳情頁面時，抖音APP向服務端申請了兩個接口請求。
一個是detail，它里面包含的是這個挑戰的一些具體信息，因為我需要的是視頻信息，所以這個我們先不管。
另一個是aweme，它的接口內容如下：

https://aweme.snssdk.com/aweme/v1/challenge/aweme/?iid=30373511894&device_id=35781128184&os_api=18&app_name=aweme&channel=App%20Store&idfa=811A8841-030F-4AEA-B934-C2A56489C32D&device_platform=iphone&build_number=17805&vid=A4BB3AF4-7981-4805-995B-78419881DC11&openudid=99bf2b608173e21cefd562496d2cf21fa8eba580&device_type=iPhone7,2&app_version=1.7.8&version_code=1.7.8&os_version=11.3&screen_width=750&aid=1128&ac=WIFI&ch_id=1574030716416014&count=21&cursor=0&pull_type=2&query_type=0&type=5&mas=001469ed4a7a3f61de046e21c8c7ae8bcb64a983471da18ec0c8d1&as=a1c5206d9e676a38ee4960&ts=1524500606

同時通過查看這個接口返回的JSON數據，也確定這個是獲取視頻的API，如下是返回的JSON視頻數據（只保留aweme_list中的第一條數據，用于分析）：

{
  "has_more" : 1,
  "cursor" : 21,
  "rid" : "201804240023290100100511301210FF",
  "status_code" : 0,
  "extra" : {
    "logid" : "201804240023290100100511301210FF",
    "now" : 1524500609305,
    "fatal_item_ids" : []
  },
  "aweme_list" : [
    {
      "text_extra" : [],
      "status" : {
        "is_delete" : false,
        "allow_share" : true,
        "is_private" : false,
        "allow_comment" : true,
        "private_status" : 0,
        "with_goods" : false
      },
      "author_user_id" : 60978405851,
      "is_hash_tag" : 0,
      "create_time" : 1501112821,
      "aweme_type" : 0,
      "cha_list" : [
        {
          "schema" : "aweme:\/\/aweme\/challenge\/detail?cid=1574030716416014",
          "user_count" : 0,
          "author" : {},
          "cha_name" : "絕地求生",
          "type" : 0,
          "desc" : "吃雞吃雞！今晚吃雞！",
          "cid" : "1574030716416014"
        }
      ],
      "region" : "CN",
      "user_digged" : 0,
      "share_info" : {
        "share_weibo_desc" : "#在抖音，記錄美好生活#空氣泳",
        "share_title" : "空氣泳",
        "share_url" : "https:\/\/www.iesdouyin.com\/share\/video\/6447230455191178509\/?region=CN&mid=6443685673772944141&titleType=title",
        "share_desc" : "在抖音，記錄美好生活"
      },
      "video" : {
        "dynamic_cover" : {
          "url_list" : [
            "https:\/\/p3.pstatp.com\/obj\/30b20017abc8da2a1379",
            "https:\/\/pb9.pstatp.com\/obj\/30b20017abc8da2a1379",
            "https:\/\/pb3.pstatp.com\/obj\/30b20017abc8da2a1379"
          ],
          "uri" : "30b20017abc8da2a1379"
        },
        "play_addr_lowbr" : {
          "url_list" : [
            "https:\/\/aweme.snssdk.com\/aweme\/v1\/play\/?video_id=fe29c6e0c74a40f6baf22400552d6d2f&line=0&ratio=720p&media_type=4&vr_type=0&test_cdn=None&improve_bitrate=0",
            "https:\/\/api.amemv.com\/aweme\/v1\/play\/?video_id=fe29c6e0c74a40f6baf22400552d6d2f&line=0&ratio=720p&media_type=4&vr_type=0&test_cdn=None&improve_bitrate=0",
            "https:\/\/aweme.snssdk.com\/aweme\/v1\/play\/?video_id=fe29c6e0c74a40f6baf22400552d6d2f&line=1&ratio=720p&media_type=4&vr_type=0&test_cdn=None&improve_bitrate=0",
            "https:\/\/api.amemv.com\/aweme\/v1\/play\/?video_id=fe29c6e0c74a40f6baf22400552d6d2f&line=1&ratio=720p&media_type=4&vr_type=0&test_cdn=None&improve_bitrate=0"
          ],
          "uri" : "fe29c6e0c74a40f6baf22400552d6d2f"
        },
        "width" : 540,
        "ratio" : "720p",
        "play_addr" : {
          "url_list" : [
            "https:\/\/aweme.snssdk.com\/aweme\/v1\/play\/?video_id=fe29c6e0c74a40f6baf22400552d6d2f&line=0&ratio=720p&media_type=4&vr_type=0&test_cdn=None&improve_bitrate=0",
            "https:\/\/api.amemv.com\/aweme\/v1\/play\/?video_id=fe29c6e0c74a40f6baf22400552d6d2f&line=0&ratio=720p&media_type=4&vr_type=0&test_cdn=None&improve_bitrate=0",
            "https:\/\/aweme.snssdk.com\/aweme\/v1\/play\/?video_id=fe29c6e0c74a40f6baf22400552d6d2f&line=1&ratio=720p&media_type=4&vr_type=0&test_cdn=None&improve_bitrate=0",
            "https:\/\/api.amemv.com\/aweme\/v1\/play\/?video_id=fe29c6e0c74a40f6baf22400552d6d2f&line=1&ratio=720p&media_type=4&vr_type=0&test_cdn=None&improve_bitrate=0"
          ],
          "uri" : "fe29c6e0c74a40f6baf22400552d6d2f"
        },
        "cover" : {
          "url_list" : [
            "https:\/\/p1.pstatp.com\/aweme\/300x400\/30b600031f0eca5498aa.jpeg",
            "https:\/\/pb3.pstatp.com\/aweme\/300x400\/30b600031f0eca5498aa.jpeg",
            "https:\/\/pb3.pstatp.com\/aweme\/300x400\/30b600031f0eca5498aa.jpeg"
          ],
          "uri" : "300x400\/30b600031f0eca5498aa"
        },
        "height" : 960,
        "bit_rate" : [
          {
            "bit_rate" : 4325045,
            "gear_name" : "normal",
            "quality_type" : 0
          }
        ],
        "origin_cover" : {
          "url_list" : [
            "http:\/\/p1.pstatp.com\/large\/30b600031f0eca5498aa.jpeg",
            "http:\/\/pb3.pstatp.com\/large\/30b600031f0eca5498aa.jpeg",
            "http:\/\/pb3.pstatp.com\/large\/30b600031f0eca5498aa.jpeg"
          ],
          "uri" : "large\/30b600031f0eca5498aa"
        },
        "duration" : 3820,
        "download_addr" : {
          "url_list" : [
            "https:\/\/aweme.snssdk.com\/aweme\/v1\/play\/?video_id=fe29c6e0c74a40f6baf22400552d6d2f&line=0&ratio=720p&watermark=1&media_type=4&vr_type=0&test_cdn=None&improve_bitrate=0",
            "https:\/\/api.amemv.com\/aweme\/v1\/play\/?video_id=fe29c6e0c74a40f6baf22400552d6d2f&line=0&ratio=720p&watermark=1&media_type=4&vr_type=0&test_cdn=None&improve_bitrate=0",
            "https:\/\/aweme.snssdk.com\/aweme\/v1\/play\/?video_id=fe29c6e0c74a40f6baf22400552d6d2f&line=1&ratio=720p&watermark=1&media_type=4&vr_type=0&test_cdn=None&improve_bitrate=0",
            "https:\/\/api.amemv.com\/aweme\/v1\/play\/?video_id=fe29c6e0c74a40f6baf22400552d6d2f&line=1&ratio=720p&watermark=1&media_type=4&vr_type=0&test_cdn=None&improve_bitrate=0"
          ],
          "uri" : "fe29c6e0c74a40f6baf22400552d6d2f"
        },
        "has_watermark" : true
      },
      "is_vr" : false,
      "vr_type" : 0,
      "share_url" : "https:\/\/www.douyin.com\/share\/video\/6447230455191178509\/?mid=6443685673772944141",
      "sort_label" : "",
      "music" : {
        "cover_hd" : {
          "url_list" : [
            "http:\/\/p3.pstatp.com\/live\/1080x1080\/2bd9005342019b56cf7e.jpeg"
          ],
          "uri" : "2bd9005342019b56cf7e"
        },
        "cover_large" : {
          "url_list" : [
            "http:\/\/p3.pstatp.com\/live\/720x720\/2bd9005342019b56cf7e.jpeg"
          ],
          "uri" : "2bd9005342019b56cf7e"
        },
        "status" : 1,
        "extra" : "{\"has_edited\": 0}",
        "user_count" : 0,
        "title" : "Iceland",
        "duration" : 32,
        "play_url" : {
          "url_list" : [
            "http:\/\/p1.pstatp.com\/obj\/29c70000654ea97a6169",
            "http:\/\/p9.pstatp.com\/obj\/29c70000654ea97a6169",
            "http:\/\/p3.pstatp.com\/obj\/29c70000654ea97a6169"
          ],
          "uri" : "29c70000654ea97a6169"
        },
        "mid" : "6443685673772944141",
        "offline_desc" : "",
        "schema_url" : "",
        "cover_medium" : {
          "url_list" : [
            "http:\/\/p3.pstatp.com\/live\/200x200\/2bd9005342019b56cf7e.jpeg"
          ],
          "uri" : "2bd9005342019b56cf7e"
        },
        "is_original" : false,
        "id" : 6443685673772944141,
        "cover_thumb" : {
          "url_list" : [
            "http:\/\/p3.pstatp.com\/live\/100x100\/2bd9005342019b56cf7e.jpeg"
          ],
          "uri" : "2bd9005342019b56cf7e"
        },
        "audio_track" : {
          "url_list" : [
            "https:\/\/p1.pstatp.com\/obj\/46bd000dbd65c3dfd8ba",
            "https:\/\/pb3.pstatp.com\/obj\/46bd000dbd65c3dfd8ba",
            "https:\/\/pb3.pstatp.com\/obj\/46bd000dbd65c3dfd8ba"
          ],
          "uri" : "46bd000dbd65c3dfd8ba"
        },
        "effects_data" : {
          "url_list" : [
            "https:\/\/p1.pstatp.com\/obj\/2efe00235ebf732bb56f",
            "https:\/\/pb3.pstatp.com\/obj\/2efe00235ebf732bb56f",
            "https:\/\/pb3.pstatp.com\/obj\/2efe00235ebf732bb56f"
          ],
          "uri" : "2efe00235ebf732bb56f"
        },
        "source_platform" : 22,
        "author" : "FlyBoy,Gavrielle",
        "id_str" : "6443685673772944141"
      },
      "is_top" : 0,
      "aweme_id" : "6447230455191178509",
      "cmt_swt" : false,
      "risk_infos" : {
        "warn" : false,
        "content" : "",
        "risk_sink" : false,
        "type" : 0
      },
      "label_origin_author" : {
        "url_list" : [
          "https:\/\/p3.pstatp.com\/obj\/216a001ff805affc1dab",
          "https:\/\/pb9.pstatp.com\/obj\/216a001ff805affc1dab",
          "https:\/\/pb3.pstatp.com\/obj\/216a001ff805affc1dab"
        ],
        "uri" : "216a001ff805affc1dab"
      },
      "label_origin_author_text" : "發起者",
      "label_top" : {
        "url_list" : [
          "https:\/\/p3.pstatp.com\/obj\/c150000f34767e2cb56",
          "https:\/\/pb9.pstatp.com\/obj\/c150000f34767e2cb56",
          "https:\/\/pb3.pstatp.com\/obj\/c150000f34767e2cb56"
        ],
        "uri" : "c150000f34767e2cb56"
      },
      "is_ads" : false,
      "statistics" : {
        "digg_count" : 19578,
        "aweme_id" : "6447230455191178509",
        "share_count" : 283,
        "play_count" : 1296122,
        "comment_count" : 100
      },
      "video_labels" : [],
      "rate" : 12,
      "bodydance_score" : 0,
      "desc" : "空氣泳",
      "author" : {
        "special_lock" : 0,
        "is_binded_weibo" : false,
        "shield_follow_notice" : 0,
        "nickname_lock" : 0,
        "avatar_larger" : {
          "url_list" : [
            "https:\/\/p1.pstatp.com\/aweme\/1080x1080\/38c90028f622dd195aaf.jpeg",
            "https:\/\/pb3.pstatp.com\/aweme\/1080x1080\/38c90028f622dd195aaf.jpeg",
            "https:\/\/pb3.pstatp.com\/aweme\/1080x1080\/38c90028f622dd195aaf.jpeg"
          ],
          "uri" : "38c90028f622dd195aaf"
        },
        "follow_status" : 0,
        "with_commerce_entry" : false,
        "original_music_qrcode" : null,
        "authority_status" : 0,
        "is_ad_fake" : false,
        "prevent_download" : false,
        "enterprise_verify" : false,
        "verification_type" : 1,
        "app_id" : 1128,
        "weibo_url" : "",
        "community_discipline_status" : 0,
        "risk_flag" : 0,
        "need_recommend" : 1,
        "live_rec_level" : 0,
        "unique_id" : "MysticzZ",
        "hide_location" : false,
        "enterprise_verify_reason" : "",
        "short_id" : "17612748",
        "account_region" : "",
        "star_use_new_download" : false,
        "signature" : "頹",
        "avatar_medium" : {
          "url_list" : [
            "https:\/\/p1.pstatp.com\/aweme\/720x720\/38c90028f622dd195aaf.jpeg",
            "https:\/\/pb3.pstatp.com\/aweme\/720x720\/38c90028f622dd195aaf.jpeg",
            "https:\/\/pb3.pstatp.com\/aweme\/720x720\/38c90028f622dd195aaf.jpeg"
          ],
          "uri" : "38c90028f622dd195aaf"
        },
        "verify_info" : "",
        "create_time" : 1495648525,
        "story_open" : false,
        "region" : "CN",
        "hide_search" : false,
        "avatar_thumb" : {
          "url_list" : [
            "https:\/\/p1.pstatp.com\/aweme\/100x100\/38c90028f622dd195aaf.jpeg",
            "https:\/\/pb3.pstatp.com\/aweme\/100x100\/38c90028f622dd195aaf.jpeg",
            "https:\/\/pb3.pstatp.com\/aweme\/100x100\/38c90028f622dd195aaf.jpeg"
          ],
          "uri" : "38c90028f622dd195aaf"
        },
        "school_poi_id" : "",
        "update_before" : 0,
        "shield_comment_notice" : 0,
        "original_music_cover" : null,
        "shield_digg_notice" : 0,
        "bind_phone" : "",
        "live_verify" : 0,
        "birthday" : "1999-03-28",
        "school_type" : 0,
        "uid" : "60978405851",
        "avatar_uri" : "38c90028f622dd195aaf",
        "weibo_schema" : "",
        "is_verified" : true,
        "custom_verify" : "",
        "commerce_user_level" : 0,
        "gender" : 1,
        "has_register_notice" : 0,
        "reflow_page_gid" : 0,
        "reflow_page_uid" : 0,
        "nickname" : "Atlantis.",
        "weibo_verify" : "",
        "language" : "zh-Hans",
        "share_qrcode_uri" : "216a0035fc39c9e9376f",
        "room_id" : 0,
        "constellation" : 1,
        "school_name" : "",
        "weibo_name" : ""
      },
      "is_relieve" : false
    },
    {
        ...第二條數據
    }
    ...其他更多的數據
  ]
}

4）對這個API進行分析（破解或者繞過API的加密），找出可以直接獲取JSON數據的純種API；
API的內容為：
https://aweme.snssdk.com/aweme/v1/challenge/aweme/?iid=30373511894&device_id=35781128184&os_api=18&app_name=aweme&channel=App%20Store&idfa=811A8841-030F-4AEA-B934-C2A56489C32D&device_platform=iphone&build_number=17805&vid=A4BB3AF4-7981-4805-995B-78419881DC11&openudid=99bf2b608173e21cefd562496d2cf21fa8eba580&device_type=iPhone7,2&app_version=1.7.8&version_code=1.7.8&os_version=11.3&screen_width=750&aid=1128&ac=WIFI&ch_id=1574030716416014&count=21&cursor=0&pull_type=2&query_type=0&type=5&mas=001469ed4a7a3f61de046e21c8c7ae8bcb64a983471da18ec0c8d1&as=a1c5206d9e676a38ee4960&ts=1524500606。
為了便于分析，我們先把一些無關的手機設備和版本信息除掉： https://aweme.snssdk.com/aweme/v1/challenge/aweme/?ch_id=1574030716416014&count=21&cursor=0&pull_type=2&query_type=0&type=5&as=a1c5206d9e676a38ee4960&ts=1524500606
它現在只剩下ch_id、count、cursor、pull_type、query_type、type、as以及ts幾個參數，其中pull_type、query_type和type三個是固定的，不用管。cursor和count是獲取的數據的開始和數量，ch_id應該是挑戰的唯一標示。接下來就是as和ts兩個參數，通過多次實驗，我們發現這個API使用了as和cp參數進行了加密處理，也就是說我們無法使用這個API直接獲取到API的數據，需要對as、ts的加密算法進行破解。當然，上面我也提過，面對這個情況，要么破解算法要么繞過加密，原則上是能夠繞過的就不要選擇破解，因為破解的難度很大，需要逆向，這個以后再講（我也沒做過）。
APP或PC上破解不了的接口，可以嘗試使用m站。這個是我們的一貫原則。
我們發現抖音中的挑戰有一個分享按鈕，我們試著分享出這個H5頁面來進行分析，分享之后的URL為：
https://www.iesdouyin.com/share/challenge/1574030716416014?utm_campaign=client_share&app=aweme&utm_medium=ios&iid=30373511894&utm_source=weixin&tt_from=weixin&utm_source=weixin&utm_medium=aweme_ios&utm_campaign=client_share&uid=58875748483&did=30373511894
在瀏覽器上打開這個URL，通過分析它的請求，只看XHR，見下圖4：