app小程序手机端Python爬虫实战18-通过mitmproxy解析短视频App返回数据

uiautomator 虚幻 385℃ 0评论

作者:虚坏叔叔
博客:https://xuhss.com

早餐店不会开到晚上,想吃的人早就来了!?

通过mitmproxy解析短视频App返回数据

13eada7a17af4e208aaeb521bfcb243a - app小程序手机端Python爬虫实战18-通过mitmproxy解析短视频App返回数据

一、电脑端证书配置

1.1 首先cmd运行一下以下命令,以便在windows系统中产生CA证书

mitmdump

1.2 接着在.mitmproxy目录找到CA证书,如下图所示:

watermark,type ZmFuZ3poZW5naGVpdGk,shadow 10,text aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZlaXl1Njg=,size 16,color FFFFFF,t 70 - app小程序手机端Python爬虫实战18-通过mitmproxy解析短视频App返回数据

1.3 双击mitmproxy-ca.p12进入导入证书的页面,点击“下一步”

watermark,type ZmFuZ3poZW5naGVpdGk,shadow 10,text aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZlaXl1Njg=,size 16,color FFFFFF,t 70 - app小程序手机端Python爬虫实战18-通过mitmproxy解析短视频App返回数据

1.4 不用输入密码,直接“下一步”

watermark,type ZmFuZ3poZW5naGVpdGk,shadow 10,text aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZlaXl1Njg=,size 16,color FFFFFF,t 70 - app小程序手机端Python爬虫实战18-通过mitmproxy解析短视频App返回数据

1.5 选择“将所有的证书都放入下列存储”,接着选择“受信任的根证书颁发机构”

watermark,type ZmFuZ3poZW5naGVpdGk,shadow 10,text aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZlaXl1Njg=,size 16,color FFFFFF,t 70 - app小程序手机端Python爬虫实战18-通过mitmproxy解析短视频App返回数据

1.6最后,弹出警告窗口,直接点击“是”

watermark,type ZmFuZ3poZW5naGVpdGk,shadow 10,text aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZlaXl1Njg=,size 16,color FFFFFF,t 70 - app小程序手机端Python爬虫实战18-通过mitmproxy解析短视频App返回数据

二、fiddler和手机的配置

可以查看这篇文章配置:

https://blog.csdn.net/biggbang/article/details/126254360

如果要抓取https包,还需要安装这个证书:

启动mitmproxy之后,手机网打开网址:mitm.it ,选择匹配的证书下载安装(见下图),

9f9ef68316da4e828397f2c73d4e3876 - app小程序手机端Python爬虫实战18-通过mitmproxy解析短视频App返回数据

安装完成后,可以在这里看到证书(可以看到这里我安装了2个证书):

设置=》安全=》信任的凭据=》用户

86999da64e7f4ce38f35b6e0060e8382 - app小程序手机端Python爬虫实战18-通过mitmproxy解析短视频App返回数据

三、抖音版本

必须是10.0.0 否则抓取不到明文数据

34cfde6490104e338cd765d3ba00a750 - app小程序手机端Python爬虫实战18-通过mitmproxy解析短视频App返回数据

四、设置只抓取手机包

点击capturing,fiddler就只会抓取手机报

fca4c641b6e6469081bc76a3257de345 - app小程序手机端Python爬虫实战18-通过mitmproxy解析短视频App返回数据

五、抓取数据包

操作抖音,进入到用户界面,可以看到有如下2个数据包:

fd64384f746744a2838182d467317232 - app小程序手机端Python爬虫实战18-通过mitmproxy解析短视频App返回数据

5.1 个人信息页接口

可以看到这里返回了一个json数据,这个加密了,所以用户数据获取不到了

0dea677c507342ab9ac79ba372d279ac - app小程序手机端Python爬虫实战18-通过mitmproxy解析短视频App返回数据

返回的数据

{
    "status_code": 3072,
    "status_msg": "",
    "log_pb": {
        "impr_id": "202208131543130102121460870CCF611C"
    }
}

请求头:

https://aweme-eagle.snssdk.com/aweme/v1/user/?user_id=4195336996258260&retry_type=no_retry&iid=4051078915568445&device_id=518619881800254&ac=wifi&channel=xiaoshangdian_douyin_and19&aid=1128&app_name=aweme&version_code=180&version_name=1.8.0&device_platform=android&ssmix=a&device_type=OPPO+R17+Pro&device_brand=OPPO&language=zh&os_api=22&os_version=5.1.1&uuid=866174451891945&openudid=13bcc1f0e3ba8eb4&manifest_version_code=180&resolution=1080*1920&dpi=320&update_version_code=1800&_rticket=1660376593582&ts=1660376593&as=aa92ef2e4862f7561192ef&cp=2e9892ef2e4892ef2e9032&mas=01735323139973599973a3b9b979d055b7599973a3b3b35393d359 

5.2 视频列表接口

3bfc946859504ee88723d77d9799839b - app小程序手机端Python爬虫实战18-通过mitmproxy解析短视频App返回数据

返回的数据

{
    "aweme_list": [{
                "ad_candidates": null,
                "anchor_info": {
                    "display_info": {
                        "after_play_ms": 0,
                        "after_play_times": 0,
                        "time_slices": null
                    },
                    "extra": "",
                    "icon": {
                        "height": 720,
                        "uri": "obj/32e6000117db7e6330650",
                        "url_list": ["http://p26-sign.douyinpic.com/obj/32e6000117db7e6330650.webp?x-expires=1660546800\u0026x-signature=5iC83w%2FLwabiB3AynnOZQ2EK1CU%3D\u0026from=1336182360", "http://p6-sign.douyinpic.com/obj/32e6000117db7e6330650.webp?x-expires=1660546800\u0026x-signature=kIrKIZTZwfbXx%2BYu1RVpnSlfsAI%3D\u0026from=1336182360", "http://p9-sign.douyinpic.com/obj/32e6000117db7e6330650.webp?x-expires=1660546800\u0026x-signature=9kp7MR7r6NTfltR6azIgsUJv%2Bzc%3D\u0026from=1336182360", "http://p26-sign.douyinpic.com/obj/32e6000117db7e6330650.jpeg?x-expires=1660546800\u0026x-signature=aVnphFhO4c4AvTj5xK%2F4CbAvEBI%3D\u0026from=1336182360"],
                        "width": 720
                    },
                    "id": "32_120791",
                    "title": "西瓜视频 | 查看剧名和完整全片",
                    "title_tag": "",
                    "type": 12,
                    "web_url": "https://ugh5.cn/ug/armor/533753ac7e8f552af0a19d3c8eafe60a/?hide_nav_bar=1\u0026should_full_screen=1\u0026from=singlemessage\u0026aid=32\u0026append_common_params=1\u0026gid=7012095819830526501\u0026aweme_item_id=7112715299929050398"
                },
                "anchors": null,
                "author": {
                    "accept_private_policy": false,
                    "account_region": "",
                    "ad_cover_url": null,
                    "apple_account": 0,
                    "authority_status": 0,
                    "avatar_168x168": {
                        "height": 720,
                        "uri": "168x168/aweme-avatar/tos-cn-avt-0015_1c4e7829846e3cef45ca5b0944d002be",
                        "url_list": ["https://p26.douyinpic.com/img/aweme-avatar/tos-cn-avt-0015_1c4e7829846e3cef45ca5b0944d002be~c5_168x168.webp?from=116350172\u0026se=false\u0026biz_tag=feed_avatar\u0026l=202208131555470102121940511DC9C159", "https://p3.douyinpic.com/img/aweme-avatar/tos-cn-avt-0015_1c4e7829846e3cef45ca5b0944d002be~c5_168x168.webp?from=116350172\u0026se=false\u0026biz_tag=feed_avatar\u0026l=202208131555470102121940511DC9C159", "https://p9.douyinpic.com/img/aweme-avatar/tos-cn-avt-0015_1c4e7829846e3cef45ca5b0944d002be~c5_168x168.webp?from=116350172\u0026se=false\u0026biz_tag=feed_avatar\u0026l=202208131555470102121940511DC9C159", "https://p26.douyinpic.com/img/aweme-avatar/tos-cn-avt-0015_1c4e7829846e3cef45ca5b0944d002be~c5_168x168.jpeg?from=116350172\u0026se=false\u0026biz_tag=feed_avatar\u0026l=202208131555470102121940511DC9C159"],
                        "width": 720
                    },
                    "avatar_300x300": {
                        "height": 720,
                        "uri": "300x300/aweme-avatar/tos-cn-avt-0015_1c4e7829846e3cef45ca5b0944d002be",
                        "url_list": ["https://p6.douyinpic.com/img/aweme-avatar/tos-cn-avt-0015_1c4e7829846e3cef45ca5b0944d002be~c5_300x300.webp?from=116350172\u0026se=false\u0026biz_tag=feed_avatar\u0026l=202208131555470102121940511DC9C159", "https://p11.douyinpic.com/img/aweme-avatar/tos-cn-avt-0015_1c4e7829846e3cef45ca5b0944d002be~c5_300x300.webp?from=116350172\u0026se=false\u0026biz_tag=feed_avatar\u0026l=202208131555470102121940511DC9C159", "https://p26.douyinpic.com/img/aweme-avatar/tos-cn-avt-0015_1c4e7829846e3cef45ca5b0944d002be~c5_300x300.webp?from=116350172\u0026se=false\u0026biz_tag=feed_avatar\u0026l=202208131555470102121940511DC9C159", "https://p6.douyinpic.com/img/aweme-avatar/tos-cn-avt-0015_1c4e7829846e3cef45ca5b0944d002be~c5_300x300.jpeg?from=116350172\u0026se=false\u0026biz_tag=feed_avatar\u0026l=202208131555470102121940511DC9C159"],
                        "width": 720
                    },
                    "avatar_larger": {
                        "height": 720,
                        "uri": "1080x1080/aweme-avatar/tos-cn-avt-0015_1c4e7829846e3cef45ca5b0944d002be",
                        "url_list": ["https://p26.douyinpic.com/aweme/1080x1080/aweme-avatar/tos-cn-avt-0015_1c4e7829846e3cef45ca5b0944d002be.webp?from=116350172\u0026se=false\u0026biz_tag=feed_avatar\u0026l=202208131555470102121940511DC9C159", "https://p9.douyinpic.com/aweme/1080x1080/aweme-avatar/tos-cn-avt-0015_1c4e7829846e3cef45ca5b0944d002be.webp?from=116350172\u0026se=false\u0026biz_tag=feed_avatar\u0026l=202208131555470102121940511DC9C159", "https://p11.douyinpic.com/aweme/1080x1080/aweme-avatar/tos-cn-avt-0015_1c4e7829846e3cef45ca5b0944d002be.webp?from=116350172\u0026se=false\u0026biz_tag=feed_avatar\u0026l=202208131555470102121940511DC9C159", "https://p26.douyinpic.com/aweme/1080x1080/aweme-avatar/tos-cn-avt-0015_1c4e7829846e3cef45ca5b0944d002be.jpeg?from=116350172\u0026se=false\u0026biz_tag=feed_avatar\u0026l=202208131555470102121940511DC9C159"],
                        "width": 720
                    },
                    "avatar_medium": {
                        "height": 720,
                        "uri": "720x720/aweme-avatar/tos-cn-avt-0015_1c4e7829846e3cef45ca5b0944d002be",
                        "url_list": ["https://p9.douyinpic.com/aweme/720x720/aweme-avatar/tos-cn-avt-0015_1c4e7829846e3cef45ca5b0944d002be.webp?from=116350172\u0026se=false\u0026biz_tag=feed_avatar\u0026l=202208131555470102121940511DC9C159", "https://p3.douyinpic.com/aweme/720x720/aweme-avatar/tos-cn-avt-0015_1c4e7829846e3cef45ca5b0944d002be.webp?from=116350172\u0026se=false\u0026biz_tag=feed_avatar\u0026l=202208131555470102121940511DC9C159", "https://p6.douyinpic.com/aweme/720x720/aweme-avatar/tos-cn-avt-0015_1c4e7829846e3cef45ca5b0944d002be.webp?from=116350172\u0026se=false\u0026biz_tag=feed_avatar\u0026l=202208131555470102121940511DC9C159", "https://p9.douyinpic.com/aweme/720x720/aweme-avatar/tos-cn-avt-0015_1c4e7829846e3cef45ca5b0944d002be.jpeg?from=116350172\u0026se=false\u0026biz_tag=feed_avatar\u0026l=202208131555470102121940511DC9C159"],
                        "width": 720
                    },
                    "avatar_thumb": {
                        "height": 720,
                        "uri": "100x100/aweme-avatar/tos-cn-avt-0015_1c4e7829846e3cef45ca5b0944d002be",
                        "url_list": ["https://p26.douyinpic.com/aweme/100x100/aweme-avatar/tos-cn-avt-0015_1c4e7829846e3cef45ca5b0944d002be.webp?from=116350172\u0026se=false\u0026biz_tag=feed_avatar\u0026l=202208131555470102121940511DC9C159", "https://p11.douyinpic.com/aweme/100x100/aweme-avatar/tos-cn-avt-0015_1c4e7829846e3cef45ca5b0944d002be.webp?from=116350172\u0026se=false\u0026biz_tag=feed_avatar\u0026l=202208131555470102121940511DC9C159", "https://p9.douyinpic.com/aweme/100x100/aweme-avatar/tos-cn-avt-0015_1c4e7829846e3cef45ca5b0944d002be.webp?from=116350172\u0026se=false\u0026biz_tag=feed_avatar\u0026l=202208131555470102121940511DC9C159", "https://p26.douyinpic.com/aweme/100x100/aweme-avatar/tos-cn-avt-0015_1c4e7829846e3cef45ca5b0944d002be.jpeg?from=116350172\u0026se=false\u0026biz_tag=feed_avatar\u0026l=202208131555470102121940511DC9C159"],
                        "width": 720
                    },
                    "avatar_uri": "aweme-avatar/tos-cn-avt-0015_1c4e7829846e3cef45ca5b0944d002be",
                    "aweme_count": 0,
                    "bind_phone": "",
                    "birthday": "",
                    "cha_list": null,
                    "comment_filter_status": 0,
                    "comment_setting": 0,
                    "commerce_user_level": 0,
                    "common_interest": null,
                    "constellation": 7,
                    "cover_url": [{
                        "height": 720,
                        "uri": "c8510002be9a3a61aad2",
                        "url_list": ["https://p26.douyinpic.com/obj/c8510002be9a3a61aad2?from=116350172", "https://p3.douyinpic.com/obj/c8510002be9a3a61aad2?from=116350172", "https://p9.douyinpic.com/obj/c8510002be9a3a61aad2?from=116350172"],
                        "width": 720
                    }],
                    "create_time": 0,
                    "custom_verify": "",
                    "cv_level": "",
                    "download_prompt_ts": 0,
                    "download_setting": -1,
                    "duet_setting": 0,
                    "enterprise_verify_reason": "",
                    "favoriting_count": 0,
                    "fb_expire_time": 0,
                    "follow_status": 0,
                    "follower_count": 0,
                    "follower_status": 0,
                    "followers_detail": null,
                    "following_count": 0,
                    "gender": 1,
                    "geofencing": null,
                    "google_account": "",
                    "has_email": false,
                    "has_facebook_token": false,
                    "has_insights": false,
                    "has_orders": false,
                    "has_twitter_token": false,
                    "has_unread_story": false,
                    "has_youtube_token": false,
                    "hide_location": false,
                    "hide_search": false,
                    "ins_id": "",
                    "is_ad_fake": false,
                    "is_binded_weibo": false,
                    "is_block": false,
                    "is_discipline_member": false,
                    "is_gov_media_vip": false,
                    "is_phone_binded": false,
                    "is_series_user": false,
                    "is_star": false,
                    "is_verified": true,
                    "item_list": null,
                    "language": "zh-Hans",
                    "live_agreement": 0,
                    "live_agreement_time": 0,
                    "live_commerce": false,
                    "live_status": 0,
                    "live_verify": 0,
                    "location": "",
                    "need_recommend": 0,
                    "neiguang_shield": 0,
                    "new_story_cover": null,
                    "nickname": "高清剪影员",
                    "platform_sync_info": null,
                    "prevent_download": false,
                    "react_setting": 0,
                    "reflow_page_gid": 0,
                    "reflow_page_uid": 0,
                    "region": "CN",
                    "relative_users": null,
                    "room_id": 0,
                    "school_name": "",
                    "school_poi_id": "",
                    "school_type": 0,
                    "sec_uid": "MS4wLjABAAAAgZrKYW3RShKlmlmXE6h5fT_uXKkr0WfKNhyCMb1E7g0",
                    "secret": 0,
                    "share_info": {
                        "share_desc": "",
                        "share_qrcode_url": {
                            "height": 720,
                            "uri": "670c00039f87dc8c6ad8",
                            "url_list": ["https://p26-sign.douyinpic.com/obj/670c00039f87dc8c6ad8?x-expires=1660395600\u0026x-signature=z3n60tJS720QdABYZfMBXYS5z94%3D\u0026from=116350172", "https://p3-sign.douyinpic.com/obj/670c00039f87dc8c6ad8?x-expires=1660395600\u0026x-signature=EN%2FXG819fto2b3flevsxP5IPUqQ%3D\u0026from=116350172", "https://p6-sign.douyinpic.com/obj/670c00039f87dc8c6ad8?x-expires=1660395600\u0026x-signature=fh6zxv05CsGE51M%2B3GcKwVHNPUs%3D\u0026from=116350172"],
                            "width": 720
                        },
                        "share_title": "",
                        "share_title_myself": "",
                        "share_title_other": "",
                        "share_url": "",
                        "share_weibo_desc": ""
                    },
                    "share_qrcode_uri": "670c00039f87dc8c6ad8",
                    "shield_comment_notice": 0,
                    "shield_digg_notice": 0,
                    "shield_follow_notice": 0,
                    "short_id": "376330109",
                    "show_image_bubble": false,
                    "signature": "? 一首音乐 一部电影 品百味人生;\n\n? 带你走近电影世界,从此不再剧荒;",
                    "signature_display_lines": 5,
                    "special_lock": 1,
                    "status": 1,
                    "story_count": 0,
                    "story_open": false,
                    "sync_to_toutiao": 0,
                    "total_favorited_new": 0,
                    "tw_expire_time": 0,
                    "twitter_id": "",
                    "twitter_name": "",
                    "type_label": null,
                    "uid": "61289731898",
                    "unique_id": "dapengv168",
                    "unique_id_modify_time": 1660377349,
                    "user_canceled": false,
                    "user_mode": 0,
                    "user_period": 0,
                    "user_rate": 1,
                    "verification_type": 1,
                    "verify_info": "",
                    "video_icon": {
                        "height": 720,
                        "uri": "",
                        "url_list": null,
                        "width": 720
                    },
                    "weibo_name": "",
                    "weibo_schema": "",
                    "weibo_url": "",
                    "weibo_verify": "",
                    "with_commerce_entry": false,
                    "with_dou_entry": false,
                    "with_fusion_shop_entry": true,
                    "with_shop_entry": false,
                    "youtube_channel_id": "",
                    "youtube_channel_title": "",
                    "youtube_expire_time": 0
                },
                "author_user_id": 61289731898,
                "aweme_control": {
                    "can_comment": true,
                    "can_forward": true,
                    "can_share": true,
                    "can_show_comment": true
                },
                "aweme_id": "7112715299929050398",
                "aweme_type": 0,
                "bodydance_score": 0,
                "cha_list": [{
                    "author": {
                        "ad_cover_url": null,
                        "cha_list": null,
                        "common_interest": null,
                        "cover_url": null,
                        "followers_detail": null,
                        "geofencing": null,
                        "item_list": null,
                        "new_story_cover": null,
                        "platform_sync_info": null,
                        "relative_users": null,
                        "type_label": null
                    },
                    "cha_attrs": null,
                    "cha_name": "马丽",
                    "cid": "1579411712356429",
                    "collect_stat": 0,
                    "connect_music": null,
                    "desc": "1982年6月28日出生于辽宁省丹东,先后毕业于中央戏剧学院表演系、北京大学戏剧研究所,中国内地女演员\n2005年马丽加入开心麻花舞台喜剧团队\n2006年参演话剧《建筑大师》\n2009年主演舞台剧《江湖学院》。\n2010年参演喜剧电影《决战刹马镇》,同年与何炅合作小品《超幸福鞋垫》,之后成功推出“超幸福”系列小品\n2012年出演家庭情感剧《AA制生活》\n2013年首次参加中央电视台春节联欢晚会,与沈腾合作小品《今天的幸福2》,饰演马冬梅\n2014年再次参加央视春晚并表演小品《扶不扶》\n2015年参加央视春晚并表演小品《投其所好》,9月30日领衔主演的青春爱情喜剧电影《夏洛特烦恼》上映\n2017年9月30日领衔主演的喜剧电影《羞羞的铁拳》上映。",
                    "is_challenge": 0,
                    "is_commerce": false,
                    "is_pgcshow": false,
                    "schema": "aweme://aweme/challenge/detail?cid=1579411712356429",
                    "share_info": {
                        "bool_persist": 0,
                        "share_desc": "1982年6月28日出生于辽宁省丹东,先后毕业于中央戏剧学院表演系、北京大学戏剧研究所,中国内地女演员\n2005年马丽加入开心麻花舞台喜剧团队\n2006年参演话剧《建筑大师》\n2009年主演舞台剧《江湖学院》。\n2010年参演喜剧电影《决战刹马镇》,同年与何炅合作小品《超幸福鞋垫》,之后成功推出“超幸福”系列小品\n2012年出演家庭情感剧《AA制生活》\n2013年首次参加中央电视台春节联欢晚会,与沈腾合作小品《今天的幸福2》,饰演马冬梅\n2014年再次参加央视春晚并表演小品《扶不扶》\n2015年参加央视春晚并表演小品《投其所好》,9月30日领衔主演的青春爱情喜剧电影《夏洛特烦恼》上映\n2017年9月30日领衔主演的喜剧电影《羞羞的铁拳》上映。",
                        "share_quote": "",
                        "share_signature_desc": "",
                        "share_signature_url": "",
                        "share_title": "我在抖音参与话题讨论#马丽 ",
                        "share_title_myself": "",
                        "share_title_other": "",
                        "share_url": "https://www.iesdouyin.com/share/challenge/1579411712356429/?u_code=0",
                        "share_weibo_desc": "我在抖音参与话题讨论#马丽 "
                    },
                    "sub_type": 0,
                    "type": 1,
                    "user_count": 0,
                    "view_count": 0
                }],
                "challenge_position": null,
                "chapter_list": null,
                "city": "0",
                "cmt_swt": false,
                "collect_stat": 0,
                "comment_list": null,
                "comment_permission_info": {
                    "can_comment": true,
                    "comment_permission_status": 0,
                    "item_detail_entry": false,
                    "press_entry": false,
                    "toast_guide": false
                },
                "commerce_config_data": null,
                "create_time": 1656058084,
                "desc": "#沈腾 #马丽 爆笑喜剧来袭!点击上方(红色按钮)可免费看完整版 #搞笑",
                "desc_language": "zh",
                "digg_lottie": {
                    "can_bomb": 0,
                    "lottie_id": ""
                },
                "dislike_dimension_list": null,
                "distance": "",
                "distribute_type": 2,
                "duet_aggregate_in_music_tab": false,
                "duration": 113271,
                "enable_top_view": true,
                "follow_up_type": 0,
                "geofencing": null,
                "geofencing_regions": null,
                "group_id": "7112715299929050398",
                "guide_btn_type": 0,
                "has_vs_entry": false,
                "hide_mix_entry": true,
                "hybrid_label": null,
                "image_infos": null,
                "images": null,
                "img_bitrate": null,
                "impression_data": {
                    "group_id_list_a": [7068954119171624192, 7076324559435107584, 7063046750663101696, 7058960565942684931, 7081615655060917508, 7074068728769481987, 7076685367167962371, 7059581426647239943, 7059416051171331336, 7062537319684771080, 7013382985508506888, 7059630019806629127, 7085547286473755916, 7081645698151730445, 7064550317395938573, 7067107269065835015],
                    "group_id_list_b": [7076324559435107584, 7068954119171624192, 7063046750663101696, 7058960565942684931, 7081615655060917508, 7074068728769481987, 7076685367167962371, 7059581426647239943, 7059416051171331336, 7062537319684771080, 7013382985508506888, 7059630019806629127, 7085547286473755916, 7081645698151730445, 7064550317395938573, 7067107269065835015],
                    "similar_id_list_a": null,
                    "similar_id_list_b": null
                },
                "interaction_stickers": null,
                "is_ads": false,
                "is_fantasy": false,
                "is_force_placed": false,
                "is_hash_tag": 1,
                "is_image_beat": false,
                "is_in_scope": false,
                "is_pgcshow": false,
                "is_relieve": false,
                "is_story": 0,

请求头

GET https://aweme-eagle.snssdk.com/aweme/v1/feed/?type=0&max_cursor=0&min_cursor=0&count=6&volume=0.7333333333333333&pull_type=1&need_relieve_aweme=0&ts=1660377347&app_type=lite&os_api=22&device_type=OPPO%20R17%20Pro&device_platform=android&ssmix=a&iid=4051078915568445&manifest_version_code=180&dpi=320&uuid=866174451891945&version_code=180&app_name=aweme&version_name=1.8.0&openudid=13bcc1f0e3ba8eb4&device_id=518619881800254&resolution=1080*1920&os_version=5.1.1&language=zh&device_brand=OPPO&ac=wifi&update_version_code=1800&aid=1128&channel=xiaoshangdian_douyin_and19&_rticket=1660377347788&as=aa9cbb4e4862f759039cbb&cp=4e989cbb4e489cbb4e9032&mas=017353231399137979f9a3b9b979d0a5ff7979f9a3f333a393d359 HTTP/1.1
Host: aweme-eagle.snssdk.com
Connection: keep-alive
Cookie: install_id=4051078915568445; ttreq=1$9af50f38bbd6a2b7c9e96b4c8275ba7212362034; odin_tt=703eef972eedaedcedd2e227c9bf32058b748c100b12103a362a6c7b808bdd8f11478effe94b0fc202046d32851411804159e33fd38e2aeeeab570b39df8e7194f6f1519bdeeed846275a309678d55ce
Accept-Encoding: gzip
X-SS-REQ-TICKET: 1660377347788
User-Agent: com.ss.android.ugc.aweme/100001 (Linux; U; Android 5.1.1; zh_CN; OPPO R17 Pro; Build/NMF26X; Cronet/58.0.2991.0)
X-Gorgon: 030000000001b18c2cbcc95775b045452115ea6be3a14cbcb73b
X-Khronos: 1660377347

六、使用mitmproxy对返回数据进行解析

创建decode_douyin.py

#个人信息页接口
#https://aweme-eagle.snssdk.com/aweme/v1/user/?user_id
#滑动视频接口
#https://aweme-eagle.snssdk.com/aweme/v1/feed/?type
import json

def response(flow):
    """10版本抖音app返回数据解析"""
    # 滑动视频接口
    if 'https://aweme-eagle.snssdk.com/aweme/v1/feed/' in flow.request.url:
        #使用json来加载响应数据
        video_response = json.loads(flow.response.text)
        video_list = video_response.get("aweme_list")
        for item in video_list:
            print(item.get("desc"), "")

    # 发布者页面
    if 'https://aweme-eagle.snssdk.com/aweme/v1/user/?user_id' in flow.request.url:
        video_response = json.loads(flow.response.text)
        print(video_response.get("status_code"), "")

七、开启运行数据解析文件

mitmdump -s decode_douyin.py -p 8889

此时就可以在页面中u单上看到打印出来的视频描述数据:

887ce7d0ffb342faa4e575e7f309d06e - app小程序手机端Python爬虫实战18-通过mitmproxy解析短视频App返回数据

? 往期优质文章分享

? 优质教程分享 ?

  • ?如果感觉文章看完了不过瘾,可以来我的其他 专栏 看一下哦~
  • ?比如以下几个专栏:Python实战微信订餐小程序、Python量化交易实战、C++ QT实战类项目 和 算法学习专栏
  • ?可以学习更多的关于C++/Python的相关内容哦!直接点击下面颜色字体就可以跳转啦!
学习路线指引(点击解锁) 知识定位 人群定位
? Python实战微信订餐小程序 ? 进阶级 本课程是python flask+微信小程序的完美结合,从项目搭建到腾讯云部署上线,打造一个全栈订餐系统。
?Python量化交易实战 ? 入门级 手把手带你打造一个易扩展、更安全、效率更高的量化交易系统
❤️ Python&C++混合编程❤️ 难度偏高 分享学习Python C++混合编程知识!

? 资料白嫖,温馨提示 ?

关注下面卡片即刻获取更多编程知识,包括各种语言学习资料,上千套PPT模板和各种游戏源码素材等等资料。更多内容可自行查看哦!

2e3d1f57d8cc4b689c0ec284120b1acc - app小程序手机端Python爬虫实战18-通过mitmproxy解析短视频App返回数据

转载请注明:虚坏叔叔 » app小程序手机端Python爬虫实战18-通过mitmproxy解析短视频App返回数据

喜欢 (5)

您必须 登录 才能发表评论!