spcrawler 1.0.1

spcrawler: A command-line tool to backup Sharepoint public installations data from open API endpoint

spcrawler is a command-line tool to backup Sharepoint public installation data from open API endpoint It uses Sharepoint API located at “/_api/web” and dumps all data and resources.

Contents

• 1 Main features

• 2 Installation

  • 2.1 Any OS

  • 2.2 Python version

• 3 Usage

• 4 Commands

  • 4.1 Ping command

  • 4.2 Walk command

  • 4.3 Dump command

1 Main features

• Metadata extraction

• Download all files (resources) from Sharepoint installation

2 Installation

2.1 Any OS

A universal installation method (that works on Windows, Mac OS X, Linux, …, and always provides the latest version) is to use pip:

# Make sure we have an up-to-date version of pip and setuptools:
$ pip install --upgrade pip setuptools

$ pip install --upgrade spcrawler

(If pip installation fails for some reason, you can try easy_install spcrawler as a fallback.)

2.2 Python version

Python version 3.6 or greater is required.

3 Usage

Synopsis:

$ spcrawler [command] [flags]

See also python -m spcrawler and spcrawler [command] --help for help for each command.

4 Commands

4.1 Ping command

Pings API endpoint located at url + “/_api/web” and returns OK if it’s available.

Ping asutk.ru API endpoint

$ spcrawler ping --url https://asutk.ru

4.2 Walk command

Lists objects in Sharepoint installation

Walks over FA.ru website objects

$ spcrawler walk --url http://fa.ru

4.3 Dump command

Dumps all objects/lists/data from API to JSON lines files. Stores all data to local path “domainname/data”

Dumps data from FA.ru (Financial university) website

$ spcrawler dump --url http://fa.ru