Scrapy Settings Log
An extension that allows a user to display all or some of their scrapy spider settings at runtime. It will add a logline with some or all settings for your spider in JSON compatible format.
Install
pip install scrapy-settings-log
Quick-Start
Add the following to your settings:
EXTENSIONS = {
'scrapy_settings_log.SpiderSettingsLogging': 999,
}
SETTINGS_LOGGING_ENABLED = True
When you run your spider you will see a log like below when spider is closing:
[scrapy_settings_log] INFO: {"SETTINGS_LOGGING_ENABLED": true, ...}
Additional Options
SETTINGS_LOGGING_REGEX- Add a regular expression to only show some settings - for exampleSETTINGS_LOGGING_REGEX = "SPIDERMON"will show settings with SPIDERMON in their name.SETTINGS_LOGGING_INDENT- Add indentation to make log more human-readable.MASKED_SENSITIVE_SETTINGS_ENABLED- Default isTrue- if settings logging is enabled it will mask the value of settings that may be sensitive (password, apikey). For example AWS_SECRET_ACCESS_KEY will have their value shown as **********
Advanced
Subclass and override the output_settings method if you want the settings to be reported in another way.
from scrapy_settings_log import SpiderSettingsLogging
class CustomSettingsLogger(SpiderSettingsLogging):
def output_settings(self, settings: dict, spider: scrapy.Spider):
# custom code here
Metadata
Release files for scrapy-settings-log 1.4.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| scrapy_settings_log-1.4.0.tar.gz | 16.4 kB | Without provenance | Details for scrapy_settings_log-1.4.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| scrapy_settings_log-1.4.0-py3-none-any.whl | Python 3 | none | any | Details for scrapy_settings_log-1.4.0-py3-none-any.whl |
Total release size: 32.0 kB
Release files / scrapy_settings_log-1.4.0.tar.gz
| Download URL | scrapy_settings_log-1.4.0.tar.gz |
|---|---|
| Size | 16.4 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
a7dbac1be30e5eb92d05bd9f7812202702459309ea4c06de0353f29339d497dc
|
|
BLAKE2b-256 checksum How to use checksums |
7c2f338af11430818db3e1bf879f6fe6c4cc4e1ff128d28de4faa7d572a0c62a
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/5.1.1 CPython/3.11.9
|
Release files / scrapy_settings_log-1.4.0-py3-none-any.whl
| Download URL | scrapy_settings_log-1.4.0-py3-none-any.whl |
|---|---|
| Size | 15.6 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
881bc113cb30226e2d7db961febee48a438eecb4ed42ab49c34b2ede88a929be
|
|
BLAKE2b-256 checksum How to use checksums |
eea6c7b36ab18351ca85ff474212d577c2470b442dd1db5a421b61e29d8e0a27
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/5.1.1 CPython/3.11.9
|
Security for scrapy-settings-log 1.4.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.