mayan-importer 3.5

Mayan EDMS importer

Description

Mayan EDMS app to migrate files from external sources.

License

This project is open sourced under the Apache 2.0 License.

Installation

1. Install from PyPI in the same virtualenv where you installed Mayan EDMS. Or if using the Docker image, pass mayan-importer to the MAYAN_PIP_INSTALLS environment variable.

   pip install mayan-importer

2. Add importer to the COMMON_EXTRA_APPS setting, either as an environment variable, from a Python settings modules, or from the UI via the config.yaml configuration file.

   Python settings module example:

   INSTALLED_APPS += (
       'importer',
   )

3. Run the migrations for the app (not required for the Docker image):

   mayan-edms.py migrate

Requirements

• dropbox - Dropbox Python SDK (https://www.dropbox.com/developers/documentation/python#install)

• Mayan EDMS version 3.4

Usage

• Create an app in Dropbox’s App Console (https://www.dropbox.com/developers/apps/).

• Generate an Access Token with no expiration.

• Create a credential instance in the Mayan EDMS Importer app and enter the Access Token.

• Create an Import Setup that will filter the files to fetch from Dropbox.

• Click the “Populate” button and check that the item count is correct.

• Click the “Process” button to start the import process.

3.3 (2024-05-27)

• Support OAuth2 refresh tokens.

• Re-add support for computer team administrator API ID.

• Split views.

• Update method names.

3.2 (2024-05-04)

• Use Dropzone widget.

• Add migration reversing functions.

• Show unknown state labels.

3.1.1 (2024-05-04)

• Fix filer load.

3.1 (2024-05-03)

• Improve navigation.

• Improve processing condition logic.

• Split modules.

• Move common state code into a mixin.

• Add fieldsets.

3.0 (2024-04-01)

• Refactor app.

• Remove import setup action.

• Use source metadata to store import metadata.

• Don’t deserialize import item source column.

• Use upstream backend, dynamic forms, credentials mixins, model mixins, error log.

• Use fieldsets.

• Bring code standards up to series 4.5.

• Support processing and deleting individual setup item entries.

• Improve import setup and import setup item state handling.

• Improve navigation, add return links.

2.5.2 (2024-02-22)

• Fix credential get_backend usage.

• Code styles updates.

• Add view icons.

2.5.1 (2024-02-20)

• Fix OAuthAccessToken credential use to CredentialBackendAccessToken.

2.5 (2024-02-20)

• Update import for Mayan EDMS version 4.5.

2.4 (2024-01-24)

• Update import for Mayan EDMS version 4.4.

2.3.1 (2023-06-12)

• Require credentials version 1.4.1.

• Support new event manager module location.

2.3 (2022-09-08)

• Split models module.

• Support item time buffer.

2.2 (2022-09-03)

• Add support for editing import setup items.

• Add new events.

• Expand tests.

• Sort import setup items by identifier.

2.1 (2022-08-19)

• Improve test rig. Add test project using upstream Mayan. Add make file targets and development settings.

• Fix API view action method to accept the new serializer argument.

• Fix ModelFiler deleting all instance. The queryset of instances to delete now uses filter.

2.0 (2022-08-08)

• Remove the view ImportSetupItemSaveDownloadView.

• Reorganize menus and navigation.

• Rename load, prepare, and save links.

• Remove success URL redirection from views.

• Update save to use the Download file and messaging systems.

1.20 (2022-08-06)

• Update SharedUploadedFile model loading. App changed from common to storage.

1.19 (2022-01-25)

• Update for Mayan EDMS 4.2.

• Require credentials 1.4.

1.18 (2021-06-17)

• Use a dedicated worker.

1.17 (2021-06-15)

• Use document label as filename.

1.16 (2021-06-12)

• Use the new document file task to speed up imports.

• Rename the “Populate” strings to “Prepare” for clarity.

1.15 (2021-06-08)

• Remove document creation transaction.

1.14 (2021-06-07)

• Update new document creation interface.

1.13 (2021-06-03)

• Update for version 4.0.

1.12.1 (2021-02-23)

• Require credentials 1.2 version.

• Fix backend SourceColumn label.

1.12 (2020-12-27)

• Add database backed logging to import setups.

• Add events for tracking start and end of the import setup population and processing.

• Improve import setup state tracking.

• Improve tests and add event testing.

1.11 (2020-12-17)

• Add API.

1.10.2 (2020-10-05)

• Fix context variable typo.

1.10.1 (2020-10-05)

• Revert usage of task_upload_new_version. Version processing is now done as part of the same code context as the import.

1.10.0 (2020-10-05)

• Support disabling import setup state update to workaround overloaded databases.

• Keep track of the documents created from an import setup item.

• Use document task_upload_new_version to process the document version as a separate code context.

• Use a queryset iterator when launching the processing tasks of the import setup items to save memory.

1.9.0 (2020-10-04)

• Add cabinet action. This action creates a cabinet structure from a path like value.

• Execute only enabled actions.

• Rename the modules of the test importer and import setup actions.

• Fix grammatical errors.

1.8.0 (2020-10-04)

• Index the import setup item state field.

• Check the state of the import setup item before processing.

1.7.0 (2020-10-03)

• Remove metadata mapping field.

• Add import setup actions. These are execute after the document is imported. Add an import setup action to assign a metadata value from a template.

• Backport the templating_tags from version 3.5.

1.6.0 (2020-09-30)

• Fix “off-by-one” process size issue.

• Delete shared uploaded file after creating document to keep the shared_files folder size small.

• Update import setup clear, populating, and process views to work on single or multiple items.

1.5.0 (2020-09-25)

• Add model filer to load and save models from and to CSV.

1.4.0 (2020-09-24)

• Add import setup item completion event.

• Commit the import setup executed event when the execute method is called instead of the get get_backend_intsance.

• Add more tests.

• Rename fields and models for clarity. Item metadata field renamed to ‘data’ to avoid confusion with document metadata.

• Event, permission, and action named “Execute” is now “Process”.

• Filter items by regular expressions during population and also during processing.

• Add team_admin_id field to the Dropbox backend to avoid an extra API call for each item to be imported.

• Multiple values are now cached for higher performance.

• Process and clear links are disabled for empty import setups.

• Smarter backend import error exclusion.

• Automatic backend keyword argument setup from dynamic fields.

• Support import item fields as attributes or dictionary keys.

1.3.0 (2020-09-23)

• Add support to process individual items.

• Add background task support for individual items. Each item is now processed independently and in parallel.

• Add thousand comma separator to the progress summary column.

1.2.0 (2020-09-22)

• Support Dropbox Team admin access.

• Add import setup state field.

• Add import setup item list view.

• Add import setup item delete view.

1.1.0 (2020-09-08)

• Convert app into a general import app. Dropbox code moved into a separate importers module.

1.0.2 (2020-09-07)

• Update absolute imports to self.

1.0.1 (2020-09-07)

• Update absolute imports to the Credentials app.

1.0.0 (2020-09-01)

• Initial release