Skip to main content

Nasy Crawler Framework -- Never had such a pure crawler.

Project description

Table of Contents


Never had such a pure crawler like this nacf.

Although I often write crawlers, I don’t like to use huge frameworks, such as scrapy, but prefer simple requests+bs4 or more general requests_html. However, these two are inconvenient for a crawler. E.g. Places, such as error retrying or parallel crawling, need to be handwritten by myself. It is not very difficult to write it while writing too much can be tedious. Hence I started writing this nacf (Nasy Crawler Framework), hoping to simplify some error retrying or parallel writing of crawlers.


Table 1: Packages
Package Version Description
requests-html 0.10.0 HTML Parsing for Humans.
nalude 0.3.0 A standard module. Inspired by Haskell’s Prelude.


see tests.

Development Process

DONE Http Functions

CLOSED: <Thu Feb 28 20:51:00 2019>


CLOSED: <Tue Dec 25 17:36:00 2018>


CLOSED: <Thu Feb 28 20:44:00 2019>


CLOSED: <Thu Feb 28 20:51:00 2019>

DONE Fix an error from inspect.Parameter which caused the function parallel down. :err:1:

CLOSED: <Wed Dec 26 20:26:00 2018>


NEXT Usage



Version 1.0.2

  • Data: <Sun Mar 10, 2019>
  • Changes: Update nalude.

Version 1.0.1

  • Data: <Sun Mar 10, 2019>
  • Changes: Update requests-html.

Version 1.0.0

  • Data: <Thu Feb 28, 2019>
  • Changes: Now, old HTTP methods (get and post) cannot accept multiple URLs. Instead, we can use gets and posts.
  • Adds: - nacf.html
    • nacf.json
    • nacf.gets
    • nacf.posts
  • Includes: - nalude

Version 0.1.2

  • Data: <Wed Dec 26, 2018>
  • Fixed: inspect.Parameter error in last version.

Version 0.1.1

  • Data: <Wed Dec 26, 2018>
  • Ignored: An error caused by inspect.Parameter
  • Help Wanted: Can someone help me about the Parameter?

Version 0.1.0

  • Date: <Sun Dec 23, 2018>
  • Commemorate Version: First Version
    • Basic Functions.

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

nacf-1.0.3.tar.gz (17.9 kB view hashes)

Uploaded source

Built Distribution

nacf-1.0.3-py3-none-any.whl (20.8 kB view hashes)

Uploaded py3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page