Skip to main content

CSV Processor

Project description

PyPI CI Codecov Documentation Supported Python versions License

Generic CSV Processing for Django Apps

Overview

This library provides a CSVProcessor class which should be subclassed to implement your own per-row processing of CSV files. At minimum, override process_row(row).

The mixins support optional checksums of arbitrary columns, and asynchronous processing of files using Celery.

Documentation

Super CSV is a stand alone library that can be used for CSV management, both syncronous and async.

Testing

make requirements

Will install all prerequisites.

make test

Runs the unit tests in local environment.

Using with Docker Devstack

Prerequisite: Have your Open edX https://github.com/openedx/devstack properly installed. Note: When you see “from inside the lms” below, it means that you’ve run make lms-shell from your devstack directory and are on a command prompt inside the LMS container.

  1. Clone this repo into ../src/ directory (relative to your “devstack” repo location). This will mount the directory in a way that is accessible to the lms container.

  2. Clone inside the lms, uninstall super-csv and reinstall your local copy. You can run the following line from inside the lms, or from the host machine run make install-local. This is necessary if one wants to use latest version for testing/development purposes:

    pip uninstall super-csv -y; pip install -e /edx/src/super-csv
  3. Setup dev environment (since virtual environments are recommended for python development, here is an example of using virtualenv. Other tools would work as well). By default, edx containers come with virtualenv preinstalled

    cd /edx/src/super-csv
    virtualenv super-csv-env
    source super-csv-env/bin/activate
    make requirements
  4. That’s it.

License

The code in this repository is licensed under the Apache v2 License unless otherwise noted.

Please see LICENSE.txt for details.

How To Contribute

Contributions are very welcome.

Please read How To Contribute for details.

The pull request description template should be automatically applied if you are creating a pull request from GitHub. Otherwise you can find it at PULL_REQUEST_TEMPLATE.md.

The issue report template should be automatically applied if you are creating an issue on GitHub as well. Otherwise you can find it at ISSUE_TEMPLATE.md.

Reporting Security Issues

Please do not report security issues in public. Please email security@openedx.org.

Getting Help

Have a question about this repository, or about Open edX in general? Please refer to this list of resources if you need any assistance.

Change Log

Unreleased

[4.1.0] - 2025-04-24

  • Added django52 support.

[3.2.0] - 2023-07-31

  • Added support of python 3.11 and 3.12

  • Dropped django32 support.

[3.1.0] - 2023-07-31

[3.0.1] - 2022-02-17

  • fix: encode the csv data before save

[3.0.0] - 2022-02-17

  • Fix dev.in to pull from ci.txt rather than travis.txt.

  • Dropped Django22, 30 and 31 support.

  • Added Django40 support.

[2.1.1] - 2021-09-01

  • Fix missing file error for bulk grade. Some of the files were expired but the code still expect to read it without try/catch.

[2.1.0] - 2020-07-26

  • Added Django 3.0, 3.1 & 3.2 Support

[2.0.1] - 2020-03-12

  • Update write_file and get_iterator to allow row/column overrides

[2.0.0] - 2020-12-23

  • Removed python 3.5 support

  • Adds support for celery 5

[1.1.0] - 2020-12-02

  • Add code_owner custom attribute for monitoring celery tasks.

[1.0.3] - 2020-11-20

  • Updated travis badge in README.rst to point to travis-ci.com

[1.0.2] - 2020-09-14

  • Move to Apache License

[1.0.1] - 2020-09-14

  • Minor updates to attrs, code-annotations, and freezegun dependencies

[1.0.0] - 2020-09-02

  • Upgrade celery to 4.2.2

[0.9.9] - 2020-05-28

  • Removed caniusepython3 & python_2_unicode_compatible packages

[0.9.8] - 2020-05-04

  • dropped support for Django version less than 2.2 and Added support for python 3.8

[0.9.7] - 2020-03-05

  • Remove django-celery dependency

[0.9.6] - 2019-11-20

  • Django 2.2 support

[0.9.5] - 2019-10-08

  • Decode the input file before passing to DictReader if necessary

[0.9.4] - 2019-09-24

  • Let the DeferrableMixin.save() method take an optional operating_user parameter.

[0.9.3] - 2019-09-20

  • Commit after CSVOperation creation so that async celery tasks can find the operation record when they start.

[0.9.2] - 2019-09-17

  • If a class inheriting from DeferrableMixin has a field user_id, use that user for the CSVOperation

[0.9.1] - 2019-07-19

  • Reverses the ordering of CSV operation history rows

[0.8] - 2019-07-22

  • Changes checksum column to avoid CSV quoting issues

[0.7.1] - 2019-07-19

  • Exposes additional fields in serialized history of operations re:degree of success of the operation

[0.5.0] - 2019-07-02

  • Exposes methods for serializing history of operations for particular processors.

[0.1.0] - 2019-05-15

Added

  • First release on PyPI.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

super_csv-4.1.0.tar.gz (24.8 kB view details)

Uploaded Source

Built Distribution

super_csv-4.1.0-py2.py3-none-any.whl (21.7 kB view details)

Uploaded Python 2Python 3

File details

Details for the file super_csv-4.1.0.tar.gz.

File metadata

  • Download URL: super_csv-4.1.0.tar.gz
  • Upload date:
  • Size: 24.8 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.1.0 CPython/3.12.9

File hashes

Hashes for super_csv-4.1.0.tar.gz
Algorithm Hash digest
SHA256 54d7dd757782ceb56775dfc964f89b87ff345796f279e0a364effd9caf60b250
MD5 c1b3e37a70ce0445c43ed55a1947caa2
BLAKE2b-256 746a26c8dc1a3d0f2b703690ba057983c6cf0824a7f81a15f49eeb62274ee92c

See more details on using hashes here.

File details

Details for the file super_csv-4.1.0-py2.py3-none-any.whl.

File metadata

  • Download URL: super_csv-4.1.0-py2.py3-none-any.whl
  • Upload date:
  • Size: 21.7 kB
  • Tags: Python 2, Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.1.0 CPython/3.12.9

File hashes

Hashes for super_csv-4.1.0-py2.py3-none-any.whl
Algorithm Hash digest
SHA256 6052e0ca57c081d2e90a6806864bfa01b2345322b2a3e734d3a514f077ff31b8
MD5 248c542b97bd0cc28a563f8f68716ec6
BLAKE2b-256 49935e05f46c8cfe9d86d1ec1c62228c1b9cb73ffccf6e6f4d0641552f6209fe

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page