技术:python3.5 + requests
运行环境:python3.5 + windows 10
demo功能:提供一个python requests 模块(http网络请求)使用demo。 抓取百度美女
安装requests 模块
1. download the source code
2. 进入requests目录, 执行 python setup.py install, 直到运行成功
# -*- coding: utf-8 -*-
import sys, json, os, uuid, time
import requests
def downloadImageFile(imgUrl):##根据图片地址, 下载图片到本地
local_filename = imgUrl.split('/')[-1]
print("Download Image File=", local_filename)
#extension = os.path.splitext(json_string['newslist'][0]['picUrl'])[1]
r = requests.get(imgUrl, stream=True) # here we need to set stream = True parameter
with open('D:\\images\\'+local_filename, 'wb') as f:
for chunk in r.iter_content(chunk_size=1024):
if chunk: # filter out keep-alive new chunks
f.write(chunk)
f.flush()
f.close()
return local_filename
count=905
counter=0
while 1:
if(counter>=600):
time.sleep(1)
url='http://image.baidu.com/search/avatarjson?tn=resultjsonavatarnew&ie=utf-8&word=%E7%BE%8E%E5%A5%B3&cg=girl&pn='+str(30*count)+'&rn=30&itg=0&z=0&fr=&width=&height=&lm=-1&ic=0&s=0&st=-1&gsm=960000000096'
count=count+1;
req = requests.get(url)
json_string = json.loads(req.text)
for i in range(1, len(json_string['imgs'])):
try:
downloadImageFile(json_string['imgs'][i-1]['objURL'])
except Exception as e:
print(e)
counter=counter+1
print(str(counter) +'at '+ str(time.time()))
time.sleep(0.001)
结果自己看吧, 我已经下载了好多, 只是百度有下载限制
欢迎添加微信,互相学习↑↑↑ -_-
白老虎
programming is not only to solve problems, ways to think
grafana 级连 菜单 templating (variables) 配置
rocketmq 集群搭建 (2master + 2slave + 2namesrv)
AI 机器人 抓取 微信 聊天中的 百度网盘 分享地址和密码