Ed-Fi LMS Toolkit Harmonizer
Project description
lms-harmonizer
Utility / process for linking LMS data with SIS data in an Ed-Fi ODS. Assumes that LMS data have been retrieved from an LMS via an LMS Extractors and uploaded into tables in the lms schema via LMS Data Store Loader.
Harmonization Process
The primary duty of the LMS Harmonizer is to match Student and Sections found in the data extracted from upstream Learning Management Systems (LMS) with the same entities in an Ed-Fi ODS database, which are sourced from a Student Information System (SIS). These systems often do not have perfect alignment, so that the "harmonization" process used by this tool is imperfect and may need customization to fit your implementation.
Any LMS records that cannot be matched to SIS records are considered "exceptions"; you can generate CSV exports listing the exceptions by using one of the command line options described below. With that information, you can then either update the LMS source system by providing it with additional information, or you can customize this process.
The out-of-the box solution has the following mapping logic:
Student Mapping
For Canvas and Schoology, the default assumption is that the SIS's unique
identifier for the student, which has been loaded into the
Student.StudentUniqueId
field via the Ed-FI ODS / PAI, has also been loaded
into the LMS in the following field:
- Canvas:
sis_user_id
(API documentation) - Schoology:
school_uid
(API documentation)
Google Classroom does not have such a field. Instead, the Harmonizer assumes that the student's email address used in Google Classroom is also recorded in the SIS and loaded into the ODS. The Harmonizer then does a simple matching of records by that email address, under the assumption that no two students have the same email address.
Section Mapping
The solution currently assumes that the SIS has a globally-unique identifier,
which has been loaded into the Section.SectionIdentifier
field in the ODS /
API. This same value is then assumed to be loaded into the following field for each LMS:
- Canvas:
sis_section_id
on a course (API documentation) - Google Classroom:
course.aliases
object (API documentation) - Schoology:
section_school_code
on a course section (API documentation)
Customization
If your implementation does not match the logic described above, then you can clone this repository and modify the stored procedures to fit alternate logic.
Getting Started
-
Requires Python 3.9+ and Poetry.
-
Requires that you have the ODS/API Suite 3, Version 5.2 or Version 5.3 and have installed the Ed-Fi-LMS extension.
-
Install required Python packages:
poetry install
-
Create a
.env
file, or plan to pass all configuration via command line arguments. See .env.example.
Configuration
Supported parameters:
Description | Required | Command Line Argument | Environment Variable |
---|---|---|---|
DB Engine | no | -g or --engine (mssql |
postgresql) |
DB Server | yes | -s or --server |
DB_SERVER |
DB Port | no (default: 1433) | --port |
DB_PORT |
DB Name | yes | -d or --dbname |
DB_NAME |
Exceptions report directory | no (no default) | -e or --exceptions-report-directory |
EXCEPTIONS_REPORT_DIRECTORY |
DB Username ** | no (no default) | -u or --username |
DB_USERNAME |
DB Password ** | no (no default) | -p or --password |
DB_PASSWORD |
Use integrated security ** | no (default: false) | -i or --useintegratedsecurity |
USE_INTEGRATED_SECURITY |
Log level* | no (default: INFO) | -l or --log-level |
LOG_LEVEL |
Encrypt db connection | no (default: False) | -n or --encrypt |
ENCRYPT_SQL_CONNECTION |
Trust db server certificate | no (default: False) | -t or --trust-certificate |
TRUST_SERVER_CERTIFICATE |
* Valid values for the optional log level:
- DEBUG
- INFO(default)
- WARNING
- ERROR
- CRITICAL
** If using integrated security, DB Username and password won't be required, otherwise they are required.
Running the Tool
For detailed help, execute poetry run python edfi_lms_harmonizer -h
.
Sample call using full integrated security, loading from the sample files directory:
poetry run python edfi_lms_harmonizer --server localhost --dbname lms_toolkit --useintegratedsecurity
Developer Notes
Dev Operations
- Style check:
poetry run flake8
- Static typing check:
poetry run mypy .
- Run unit tests:
poetry run pytest tests
- Run unit tests with code coverage:
poetry run coverage run -m pytest tests
- View code coverage:
poetry run coverage report
- Run SQL Server integration tests:
poetry run pytest tests_integration_mssql
- Run PostgreSQL integration tests:
poetry run pytest tests_integration_pgsql
Also see build.py for use of the build script.
Integration Testing
See Integration test setup instructions (SQL Server | PostgreSQL) for information on configuring integration testing to work in various environments.
Legal Information
Copyright (c) 2022 Ed-Fi Alliance, LLC and contributors.
Licensed under the Apache License, Version 2.0 (the "License").
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
See NOTICES for additional copyright and license notifications.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for edfi-lms-harmonizer-1.2.0.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 337fb88bb6a05ae73f08f5fb3f142e06435de44b497dcde3eba3242d0481b659 |
|
MD5 | b00b9bbfc4f8a394dd263b656f618a65 |
|
BLAKE2b-256 | cdba92e984822fd225359d1387076938aeb680ccca3a46978f7816539c328b4d |
Hashes for edfi_lms_harmonizer-1.2.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | dda4b7472d6097cfec8fd5f77d7641e9873abe7fbcf51b06d502c2852c851cc1 |
|
MD5 | beb17b26ff6f734c7655feb06d8461aa |
|
BLAKE2b-256 | b1208c1e4367000b7dfef85f0e7b924fd2777e9badad9c056d3c5c4c61b3b511 |