Join GitHub today
GitHub is home to over 50 million developers working together to host and review code, manage projects, and build software together.
Sign upPython crawler keeps crawling the content of the first page repeatedly #2379
Comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
import requests import urllib from lxml import html #需要爬数据的网址 j=0 #!s-p3 for k in range(1,20): url='https://www.duitang.com/search/?kw=%E6%AD%A3%E5%A4%AA&type=feed'+'#!s-p'+str(k) page=requests.Session().get(url) tree=html.fromstring(page.text) result=tree.xpath('//a[@class="a"]//img/@src') #获取需要的数据 for i in result: urllib.request.urlretrieve(i,'C://Users//FangJZ//Desktop//duitan//'+(str(j))+'.jpg') j=j+1 print('true')