pvsite-datamodel
⚠️ Note this repo will soon be deprecated in favour of a new Data Platform
Database schema specification for PV Site data.
Repository structure
pvsite_datamodel:
read: # Sub package containing modules for reading from the database
write: # Sub package containing modules for writing to the database
- connection.py # Class for connecting to the database
- sqlmodels.py # SQLAlchemy definitions of table schemas
tests: # External tests package
Top-level functions
Classes specifying table schemas:
- APIRequestSQL
- GenerationSQL
- ForecastSQL
- ForecastValueSQL
- MLModelSQL
- UserSQL
- LocationSQL
- LocationGroupSQL
- StatusSQL
- ClientSQL
Database connection objects:
- DatabaseConnection
Read and write package functions
- Read function currently accessible via
from pvsite_datamodel.read import <func>. - Write function Currently accessible via
from pvsite_datamodels.write import <func>.
| Read Package Functions | Write Package Functions |
|---|---|
get_user_by_email |
insert_forecast_values |
get_pv_generation_by_sites |
insert_generation_values |
get_site_by_uuid |
create_site |
get_site_by_client_site_id |
create_site_group |
get_site_by_client_site_name |
create_user |
get_sites_by_client_name |
create_client |
get_all_sites |
make_fake_site |
get_sites_by_country |
add_site_to_site_group |
get_site_group_by_name |
change_user_site_group |
get_latest_status |
update_user_site_group |
get_latest_forecast_values_by_site |
edit_site |
get_client_by_name |
edit_client |
assign_site_to_client |
|
delete_site |
|
delete_user |
|
delete_site_group |
|
| TODO update table |
Local Repository Setup(Linux)
This guide walks you through setting up the repository locally, installing dependencies. Follow the steps carefully to get your development environment up and running.
Pre-requisite:
Use uv for virtual environment management.
- Install Pgadmin4 for database management
Database Setup
Follow these steps to set up the database locally:
-
Install PostgreSQL: Download and install PostgreSQL on your system : Download PostgreSQL for Linux
-
Start PostgreSQL Service:
sudo service postgresql start
- Check PostgreSQL Status: Ensure the service is running properly:
sudo service postgresql status
- Get Your Local IP Address: Copy the IP address to configure your connection:
hostname -I
-
Create the Database: Use pgAdmin 4 or the command line to create your database.
-
Run Migrations with Alembic: Export the database URL and apply migrations:
export DB_URL="postgresql://<username>:<password>@<your_ip>:5432/<your_database>"
- Note: Replace the placeholders with your actual database credentials.
- Run the migrations:
uv run alembic upgrade head
Steps to setup the repository locally
- Fork & Clone the Repository:
git clone https://github.com/openclimatefix/pv-site-datamodel.git
- Navigate to the Repository:
cd pv-site-datamodel
- Install Dependencies:
uv sync
- Activate Virtual Environment:
source .venv/bin/activate
- Set Database URL:
Replace the placeholder with your actual database connection string.
export DB_URL="postgresql://<username>:<password>@<your_ip>:5432/<your_database>"
- Make Code Changes:
Modify the necessary code files as needed.
- Format the Code:
make format
- Lint the Code:
make lint
Running the tests
Run the following command to execute the test suite:
make test
PVSite Database Schema
---
title: SQLAlchemy relationships
---
classDiagram
class UserSQL{
+ user_uuid : UUID ≪ PK ≫
+ email : String(255) ≪ U ≫
+ location_group_uuid : UUID ≪ FK ≫
}
class LocationGroupSQL{
+ location_group_uuid : UUID ≪ PK ≫
+ location_group_name : String ≪ U ≫
+ service_level : Integer ≪ U ≫
}
class LocationGroupLocationSQL{
+ location_group_location_uuid : UUID ≪ PK ≫
+ location_group_uuid : UUID ≪ FK ≫
+ location_uuid : UUID ≪ FK ≫
}
class LocationLocationSQL{
+ location_location_uuid : UUID ≪ PK ≫
+ location_parent_uuid : UUID ≪ FK ≫
+ location_child_uuid : UUID ≪ FK ≫
}
class LocationSQL{
+ location_uuid : UUID ≪ PK ≫
+ client_location_id : Integer
+ client_location_name : String
+ country : String ≪ D ≫
+ region : String
+ location_type: Enum ≪ D ≫
+ location_metadata : String
+ dno : String
+ gsp : String
+ asset_type : Enum ≪ D ≫
+ orientation : Float
+ tilt : Float
+ latitude : Float
+ longitude : Float
+ capacity_kw : Float
+ inverter_capacity_kw : Float
+ module_capacity_kw : Float
+ ml_id : Integer ≪ U ≫
+ client_uuid : UUID ≪ FK ≫
+ ml_model_uuid : UUID ≪ FK ≫
}
class ClientSQL{
+ client_uuid : UUID ≪ PK ≫
+ client_name : String
}
class GenerationSQL{
+ generation_uuid : UUID ≪ PK ≫
+ location_uuid : UUID ≪ FK ≫
+ generation_power_kw : Float
+ start_utc : DateTime
+ end_utc : DateTime
}
class ForecastSQL{
+ forecast_uuid : UUID ≪ PK ≫
+ location_uuid : UUID ≪ FK ≫
+ timestamp_utc : DateTime
+ forecast_version : String
}
class ForecastValueSQL{
+ forecast_value_uuid : UUID ≪ PK ≫
+ start_utc : DateTime
+ end_utc : DateTime
+ forecast_power_kw : Float
+ horizon_minutes : Integer
+ forecast_uuid : UUID ≪ FK ≫
}
class StatusSQL{
+ status_uuid : UUID ≪ PK ≫
+ status : String
+ message : String
}
class InverterSQL{
+ inverter_uuid : UUID ≪ PK ≫
+ site_uuid : UUID ≪ FK ≫
}
class APIRequestSQL{
+ uuid : UUID ≪ PK ≫
+ url : String
+ user_uuid : UUID ≪ FK ≫
}
class MLModelSQL{
+ uuid : UUID ≪ PK ≫
+ mode_name : String
+ model_version : UUID ≪ FK ≫
}
UserSQL "1" -- "N" LocationGroupSQL : belongs_to
LocationGroupLocationSQL "N" -- "1" LocationSQL : contains
LocationGroupSQL "1" -- "N" LocationGroupLocationSQL : contains
LocationSQL "1" -- "N" GenerationSQL : generates
LocationSQL "1" -- "N" ForecastSQL : forecasts
LocationSQL "N" -- "0" MLModelSQL : ml_model
ForecastSQL "1" -- "N" ForecastValueSQL : contains
MLModelSQL "1" -- "N" ForecastValueSQL : forecasts
LocationSQL "1" -- "N" InverterSQL : contains
UserSQL "1" -- "N" APIRequestSQL : performs_request
ClientSQL "1" -- "N" LocationSQL : owns
LocationSQL "1" -- "N" LocationLocationSQL : contains
LocationLocationSQL "N" -- "1" LocationSQL : contains
class Legend{
UUID: Universally Unique Identifier
PK: Primary Key
FK: Foreign Key
U: Unique Constraint
D: Default Value
}
Multiple Clients
We have the ability to have these different scenarios
- one user - can add or view one site
- one user, can add or view multiple sites
- Two users (for example from the sample company), want to look at one site
- Two users, wanting to look at multiple sites (could be added by another user). Any user from site group can add a site.
- OCF user want to see everything (admin)
Solution
Location can mean site or region.
graph TD;
User-- N:1 -->LocationGroup;
LocationGroup-- N:N -->Location;
- One
useris in oneloationgroup. Each location group can have multiple users. - Each
locationroupcontains multiplelocations. Onelocationcan be in multiplelocationgroups
1. one user - one site
graph TD;
A(User=Alice)-->B(LocationGroup=Alice1);
B --> C(Site);
2. one user - two sites
graph TD;
A(User=Alice)-->B(LocationGroup=Alice1);
B --> C1(Site1);
B --> C2(Site2);
3. Two users - one site
graph TD;
A1(User=Alice)-->B(LocationGroup);
A2(User=Bob)-->B(LocationGroup);
B --> C1(Site1);
4. Two users - two site
graph TD;
A1(User=Alice)-->B(LocationGroup);
A2(User=Bob)-->B(LocationGroup);
B --> C1(Site1);
B --> C2(Site2);
5. OCF can see everything
graph TD;
A1(User=Alice)-->B(LocationGroup1);
A2(User=Bob)-->B(LocationGroup1);
A3(User=OCF)-->B2(LocationGroup2);
B --> C1(Site1);
B --> C2(Site2);
B2 --> C1(Site1);
B2 --> C2(Site2);
B2 --> C3(Site3);
Database migrations using alembic
Contributors ✨
Thanks goes to these wonderful people (emoji key):
This project follows the all-contributors specification. Contributions of any kind welcome!
Metadata
Release files for pvsite-datamodel 1.2.15
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| pvsite_datamodel-1.2.15.tar.gz | 39.9 kB | Without provenance | Details for pvsite_datamodel-1.2.15.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| pvsite_datamodel-1.2.15-py3-none-any.whl | Python 3 | none | any | Details for pvsite_datamodel-1.2.15-py3-none-any.whl |
Total release size: 81.9 kB
Release files / pvsite_datamodel-1.2.15.tar.gz
| Download URL | pvsite_datamodel-1.2.15.tar.gz |
|---|---|
| Size | 39.9 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
b34dce83483ea5917f9b78881d1fd1f119f1887d92ea64f74be9cd4d1b1ffa87
|
|
BLAKE2b-256 checksum How to use checksums |
3c5ba98aab0935c81bb07c1b327f00a300aa13517e9569bfcb139d0dfc2050c5
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.11.3 {"installer":{"name":"uv","version":"0.11.3","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
Release files / pvsite_datamodel-1.2.15-py3-none-any.whl
| Download URL | pvsite_datamodel-1.2.15-py3-none-any.whl |
|---|---|
| Size | 41.9 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
383d536cb5167eb6c867d840df2fd96daae3edcb2198f79bf791b68af0ec7042
|
|
BLAKE2b-256 checksum How to use checksums |
db82327998ca1ec9d80662a6bd7ebf9537b27c03c7ad917659fa0c0a185947e2
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.11.3 {"installer":{"name":"uv","version":"0.11.3","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
Security for pvsite-datamodel 1.2.15
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.