Skip to main content

Nasy Crawler Framework -- Never had such a pure crawler.

Project description

Table of Contents

Prologue

Never had such a pure crawler like this nacf.

Although I often write crawlers, I don’t like to use huge frameworks, such as scrapy, but prefer simple requests+bs4 or more general requests_html. However, these two are inconvenient for a crawler. E.g. Places, such as error retrying or parallel crawling, need to be handwritten by myself. It is not very difficult to write it while writing too much can be tedious. Hence I started writing this nacf (Nasy Crawler Framework), hoping to simplify some error retrying or parallel writing of crawlers.

Packages

Table 1: Packages
Package Version Description
requests-html 0.9.0 HTML Parsing for Humans.
nalude 0.2.0 A standard module. Inspired by Haskell’s Prelude.

Development Process

DONE Http Functions

CLOSED: <Thu Feb 28 20:51:00 2019>

DONE Get

CLOSED: <Tue Dec 25 17:36:00 2018>

DONE Post

CLOSED: <Thu Feb 28 20:44:00 2019>

DONE Bugs

CLOSED: <Thu Feb 28 20:51:00 2019>

DONE Fix an error from inspect.Parameter which caused the function parallel down. :err:1:

CLOSED: <Wed Dec 26 20:26:00 2018>

NEXT Docs

NEXT Usage

Epoligue

History

Version 1.0.0

  • Data: <Thu Feb 28, 2019>
  • Changes: Now, old HTTP methods (get and post) cannot accept multiple URLs. Instead, we can use gets and posts.
  • Adds: - nacf.html
    • nacf.json
    • nacf.gets
    • nacf.posts
  • Includes: - nalude

Version 0.1.2

  • Data: <Wed Dec 26, 2018>
  • Fixed: inspect.Parameter error in last version.

Version 0.1.1

  • Data: <Wed Dec 26, 2018>
  • Ignored: An error caused by inspect.Parameter
  • Help Wanted: Can someone help me about the Parameter?

Version 0.1.0

  • Date: <Sun Dec 23, 2018>
  • Commemorate Version: First Version
    • Basic Functions.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for nacf, version 1.0.1
Filename, size File type Python version Upload date Hashes
Filename, size nacf-1.0.1-py3-none-any.whl (41.2 kB) File type Wheel Python version py3 Upload date Hashes View
Filename, size nacf-1.0.1.tar.gz (14.8 kB) File type Source Python version None Upload date Hashes View

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring DigiCert DigiCert EV certificate Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page