Skip to main content

Little Red Book notes, home page, detailed page crawler

Project description

Spider_XHS

image

小红书个人主页图片和视频无水印爬取

效果图

image

image

image

image

运行环境

Python环境

NodeJS环境

运行方法:把你想要的id全部放到列表里




多用户下载(下载用户列表所有的笔记)(需登录cookie)

python muti-note.py

多笔记下载(下载笔记列表里所有的笔记)(需登录cookie)

python one-note.py

下载搜索内容(需登录cookie)

python search-note.py

日志

  1. 23/08/08 first commit

  2. 23/09/13 【api更改params增加两个字段】修复图片无法下载,有些页面无法访问导致报错。

  3. 23/09/16 【较大视频出现编码问题】修复视频编码问题,加入异常处理。

  4. 23/09/18 代码重构,加入失败重试。

  5. 23/09/19 新增下载搜索结果功能

注意事项

本项目仅供学习与交流,侵权必删

关于muti-note和one-note

  1. 这两个必须登录,获取cookie,不然无法获取所有笔记

  2. muti-note处理的是个人主页 https://www.xiaohongshu.com/user/profile/6185ce66000000001000705b

  3. one-note处理的是笔记详细页 https://www.xiaohongshu.com/explore/64d06670000000000800fb4a

  4. search-note处理的是搜索结果

other

  1. 自行将cookies放到目录下cookies.txt中,去设置里的应用程序里找或者网络请求里找,需要哪些可以参考cookie.txt文件。

  2. 可采用以下方法获取cookie,并运行对应文件。

image

image

  1. 欢迎star,不时更新。

  2. 有问题可以加QQ或者微信交流(992822653)

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

xhs_spider-1.0.0.tar.gz (11.0 kB view details)

Uploaded Source

File details

Details for the file xhs_spider-1.0.0.tar.gz.

File metadata

  • Download URL: xhs_spider-1.0.0.tar.gz
  • Upload date:
  • Size: 11.0 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.9.18

File hashes

Hashes for xhs_spider-1.0.0.tar.gz
Algorithm Hash digest
SHA256 a66aee665165657436938fd8d8e17d0751bf09f085262d76598decc5c927b0e6
MD5 1ae081b7591d4ead5c000e64b591eec7
BLAKE2b-256 e38f9579a247ff00298fc2bea37e39d2ee7ce3fcfe5c41e6cd1ded4b06908772

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page