在B站上观看北京理工大学的爬虫慕课,在爬取淘宝商品价格和名称时出现了只出表头信息的结果。

经过在视频下方评论区的方法发现依然有错,最后发现是自己的拼写错误:joy:
发话不多说直接上教程:(我使用的是chrome浏览器)
在淘宝页面右键,点击检查:
然后在源代码区进行以下步骤:
把复制的内容放到如下位置:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
| def getHTML(url): try: print('解析开始') headers = { 'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36', 'cookie': 'miid=1204848422178730713; thw=cn; cna=/FjUFFH98VYCAbfHca7IzgoX; tracknick=%5Cu51B0%5Cu6CB3%5Cu4E16%5Cu7EAA%5Cu98DE; tg=0; hng=CN%7Czh-CN%7CCNY%7C156; enc=S9dNZWnBqTugKoqk7f9MpczA8QNKFdbWP8Dab4FEeTI43rFE%2B%2FrwSE7PnLJF%2BRA5yy70RD7m9rnCJyj72bByLg%3D%3D; t=23bd277878acd720dad699e1cc1f655d; cookie2=1266fc62e2cc3c4136c7085328db7985; v=0; _tb_token_=3fe6ee4e766b8; alitrackid=www.taobao.com; lastalitrackid=www.taobao.com; _samesite_flag_=true; sgcookie=EvIOOHQLzH3Oo804wQlio; unb=2278212962; uc3=lg2=VFC%2FuZ9ayeYq2g%3D%3D&nk2=0ryj1fYS8gv0fQ%3D%3D&vt3=F8dBxdAUzLP9Oq23c0M%3D&id2=UUpmlOw4s1nHfg%3D%3D; csg=f123a494; lgc=%5Cu51B0%5Cu6CB3%5Cu4E16%5Cu7EAA%5Cu98DE; cookie17=UUpmlOw4s1nHfg%3D%3D; dnk=%5Cu51B0%5Cu6CB3%5Cu4E16%5Cu7EAA%5Cu98DE; skt=a5e20976396f587e; existShop=MTU4NjA4MzcyMA%3D%3D; uc4=id4=0%40U2gsGA%2BPaSbBR0Lnefhgm24jPZLA&nk4=0%400A28Ox7kSTkcIvypO6o%2FqsLX5wQe; _cc_=VT5L2FSpdA%3D%3D; _l_g_=Ug%3D%3D; sg=%E9%A3%9E2f; _nk_=%5Cu51B0%5Cu6CB3%5Cu4E16%5Cu7EAA%5Cu98DE; cookie1=VAn7B4bWBvYsc6mrUjgaw%2F%2F8dWi4qfHRI6oPI3CZaYI%3D; tfstk=c47CBgD9vvDCt6DJ59NwYM0dAv8RZXe6sk9dOGUC18n-cLCCix02nXEhqA09DC1..; mt=ci=3_1; JSESSIONID=E9C7BE989C9A4781DC9BDF55EE086724; l=dBLuPSIcqWzW1J7tBOCgqDBHHA7TYIRAgulLJpFvi_5Iw6Y1A2QOo1N4MFv6cjWft28B4z6vzNe9-etlsn4pJA--g3fydxDc.; isg=BNDQjY51utpNoGWOeaq5eOihoR4imbTj3pjtcMqhxyv-BXCvcqgLcyd_3c3l1Wy7; uc1=cookie16=U%2BGCWk%2F74Mx5tgzv3dWpnhjPaQ%3D%3D&cookie21=W5iHLLyFeYZ1WM9hVnmS&cookie15=URm48syIIVrSKA%3D%3D&existShop=false&pas=0&cookie14=UoTUPOT6s4ShJw%3D%3D&tag=8&lng=zh_CN', } h = requests.get(url, headers = headers) h.raise_for_status() h.encoding = h.apparent_encoding return h.text except: return 'html解析失败'
|
修改划线位置,把自己的user-agent和cookie粘贴即可。

至此,若代码没有拼写等低级错误就会成功输出正确信息。

我自己就是因为拼写错误只输出了表头。

另外一个检查错误的技巧就是巧用print()函数,来检查代码执行到哪里出错,以及是否输出正确信息。我就是通过这个方法一步步查找到plt得到的返回值为零,然后发现price拼写错误了。
谢谢阅读:happy: