首页 > IT职业 > IT生活 > 利用python编写爬虫爬取淘宝奶粉部分数据.1
import requests
import re
import pymysql.cursors
def getHTMLText(url):
kv = {'User-Agent':'Mozilla/5.0'} #反反爬虫
try:
r = requests.get(url, headers = kv, timeout = 30)
r.raise_for_status()
r.encoding = r.apparent_encoding
return r.text
except:
print("error1")
def parsePage(ilt,html):
try:
url = start_url + '&s=' + str(44*i)
html = getHTMLText(url)
parsePage(infoList,html)
except:
continue
printGoodsList(infoList) main()
来自 “ ITPUB博客 ” ,链接:https://blog.itpub.net/2249/viewspace-2799649/ ,如需转载,请注明出处,否则将追究法律责任。