A tool allowing you to migrate data from legacy ILS:s (Library systems) into FOLIO LSP
Project description
FOLIO Migration Tools
A toolkit that enables you to migrate data over from a legacy ILS system into FOLIO LSP
The scripts requires a FOLIO tenant with reference data properly set up. The script will throw messages telling what reference data is missing.
When the files have been created, post them to FOLIO using the service_tools set of programs. Preferably BatchPoster
FOLIO migration process
This repo plays the main part in a process using many tools. The process itself is documented in more detail, including example configuration files, at This template repository In order to perform migrations according to this process, you need to clone or make the following repositories:
Mapping files
The scripts also relies on a folder with a set of mapping files. There is a template repository with examples of the files needed and some documentation around it in the Readme. There is also a web tool for creating mapping files from delimited source files
Bib records to Invcentory and SRS records
MARC mapping for Bib level records is based on the mapping-rules residing in a FOLIO tenant. Read more on this in the Readme in the Source record manager Module repo.
The trigger for this process it the main_bibs.py. In order to see what parameters are needed, just do pipenv run python main_bibs.py -h
In the picture above, you can se the files needed and the files created as part of the proces.
MFHD-to-Inventory
Mapping rules
This processing does not store the MARC records anywhere since this is not available in FOLIO yet (Planned for the Kiwi release). Only FOLIO Holdings records are created. MFHD-to-Inventory mapping also relies on mapping based on a similar JSON structure. This is not stored in the tenant and must be maintained by you. A template/example is available in migration_repo_template
If you do not have MFHD records available, you can build a mapping file this web tool from the Item data. This will generate Holdings records to connect to the items. There are two scripts, depending on what source data you have: main_holdings_csv.py and main_holdings_marc.py
Location mapping
For holdings mapping, you also need to map legacy locations to FOLIO locations. An example map file is available at migration_repo_template
Items-to-Inventory
Items-to-Inventory mapping is based on a json structure where the CSV headers are matched against the target fields in the FOLIO items. To create a mapping file, use the web tool.
Tests
There is a test suite for Bibs-to-Instance mapping. You need to add arguments in order to run it against a FOLIO environment. The tests are run against the latest FOLIO Bugfest environment as part of the commit process in this repo.
Running the tests for the Rules mapper
- Install the packages in the Pipfile
- Run
clear; pipenv run pytest -v --log-level=DEBUG --password PASSWORD --tenant_id TENANT_ID --okapi_url OKAPI URL --username USERNAME
Running the scripts
For information on what files are needed and produced by the toolkit, refer to the documentation and example files in the template repository.
All Migration tasks
pipenv run python main.py CONFIGURATION_FILE_PATH MIGRATION_TASK_NAME
The above will fetch the mapping-rules from the FOLIO tenant specified and transform the supplied MARC21 record files into FOLIO Instance and SRS records.
Example:
pipenv run python main.py ~/code/migration_repo_template/mapping_files/exampleConfiguration.json transform_bibs
Explanation
--okapi_password The password to the FOLIO tenant configured in your configuration file
--base_folder The base folder for the library you are migrating from. This should ideally be a git repository created from the migration_repo_template
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for folio_migration_tools-1.0.2.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3044a3eccf247e8b8d723e0ccea1587798b0b9502065efb33027f9d8e2cbc0a7 |
|
MD5 | 4f87c91946cbf50b4b2b7a043d8de773 |
|
BLAKE2b-256 | af28876e1ee3c6fa4757cdca8d0d9cba6d3e17a165d23a222477502d257ccb61 |
Hashes for folio_migration_tools-1.0.2-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e885d9810b3287ba38fde34dbb0f9e9cdadcc09c1e0c64ab53b689fa5072f031 |
|
MD5 | 026a6cdd54836fc1435b36c4d5caa1de |
|
BLAKE2b-256 | 077ce15382c7409622c74bc27b564090b676fd6ff8e446aff75fc5653bafcd35 |