pvsite-datamodel
:warning: Note this repo will soon be deprecated in favour of a new Data Platform
Database schema specification for PV Site data.
Repository structure
pvsite_datamodel:
read: # Sub package containing modules for reading from the database
write: # Sub package containing modules for writing to the database
- connection.py # Class for connecting to the database
- sqlmodels.py # SQLAlchemy definitions of table schemas
tests: # External tests package
Top-level functions
Classes specifying table schemas:
- APIRequestSQL
- GenerationSQL
- ForecastSQL
- ForecastValueSQL
- MLModelSQL
- UserSQL
- LocationSQL
- LocationGroupSQL
- StatusSQL
- ClientSQL
Database connection objects:
- DatabaseConnection
Read and write package functions
- Read function currently accessible via
from pvsite_datamodel.read import <func>. - Write function Currently accessible via
from pvsite_datamodels.write import <func>.
| Read Package Functions | Write Package Functions |
|---|---|
get_user_by_email |
insert_forecast_values |
get_pv_generation_by_sites |
insert_generation_values |
get_site_by_uuid |
create_site |
get_site_by_client_site_id |
create_site_group |
get_site_by_client_site_name |
create_user |
get_sites_by_client_name |
create_client |
get_all_sites |
make_fake_site |
get_sites_by_country |
add_site_to_site_group |
get_site_group_by_name |
change_user_site_group |
get_latest_status |
update_user_site_group |
get_latest_forecast_values_by_site |
edit_site |
get_client_by_name |
edit_client |
assign_site_to_client |
|
delete_site |
|
delete_user |
|
delete_site_group |
|
| TODO update table |
Local Repository Setup(Linux)
This guide walks you through setting up the repository locally, installing dependencies. Follow the steps carefully to get your development environment up and running.
Pre-requisite:
Use uv for virtual environment management.
- Install Pgadmin4 for database management
Database Setup
Follow these steps to set up the database locally:
-
Install PostgreSQL: Download and install PostgreSQL on your system : Download PostgreSQL for Linux
-
Start PostgreSQL Service:
sudo service postgresql start
- Check PostgreSQL Status: Ensure the service is running properly:
sudo service postgresql status
- Get Your Local IP Address: Copy the IP address to configure your connection:
hostname -I
-
Create the Database: Use pgAdmin 4 or the command line to create your database.
-
Run Migrations with Alembic: Export the database URL and apply migrations:
export DB_URL="postgresql://<username>:<password>@<your_ip>:5432/<your_database>"
- Note: Replace the placeholders with your actual database credentials.
- Run the migrations:
uv run alembic upgrade head
Steps to setup the repository locally
- Fork & Clone the Repository:
git clone https://github.com/openclimatefix/pv-site-datamodel.git
- Navigate to the Repository:
cd pv-site-datamodel
- Install Dependencies:
uv sync
- Activate Virtual Environment:
source .venv/bin/activate
- Set Database URL:
Replace the placeholder with your actual database connection string.
export DB_URL="postgresql://<username>:<password>@<your_ip>:5432/<your_database>"
- Make Code Changes:
Modify the necessary code files as needed.
- Format the Code:
make format
- Lint the Code:
make lint
Running the tests
Run the following command to execute the test suite:
make test
PVSite Database Schema
---
title: SQLAlchemy relationships
---
classDiagram
class UserSQL{
+ user_uuid : UUID ≪ PK ≫
+ email : String(255) ≪ U ≫
+ location_group_uuid : UUID ≪ FK ≫
}
class LocationGroupSQL{
+ location_group_uuid : UUID ≪ PK ≫
+ location_group_name : String ≪ U ≫
+ service_level : Integer ≪ U ≫
}
class LocationGroupLocationSQL{
+ location_group_location_uuid : UUID ≪ PK ≫
+ location_group_uuid : UUID ≪ FK ≫
+ location_uuid : UUID ≪ FK ≫
}
class LocationLocationSQL{
+ location_location_uuid : UUID ≪ PK ≫
+ location_parent_uuid : UUID ≪ FK ≫
+ location_child_uuid : UUID ≪ FK ≫
}
class LocationSQL{
+ location_uuid : UUID ≪ PK ≫
+ client_location_id : Integer
+ client_location_name : String
+ country : String ≪ D ≫
+ region : String
+ location_type: Enum ≪ D ≫
+ location_metadata : String
+ dno : String
+ gsp : String
+ asset_type : Enum ≪ D ≫
+ orientation : Float
+ tilt : Float
+ latitude : Float
+ longitude : Float
+ capacity_kw : Float
+ inverter_capacity_kw : Float
+ module_capacity_kw : Float
+ ml_id : Integer ≪ U ≫
+ client_uuid : UUID ≪ FK ≫
+ ml_model_uuid : UUID ≪ FK ≫
}
class ClientSQL{
+ client_uuid : UUID ≪ PK ≫
+ client_name : String
}
class GenerationSQL{
+ generation_uuid : UUID ≪ PK ≫
+ location_uuid : UUID ≪ FK ≫
+ generation_power_kw : Float
+ start_utc : DateTime
+ end_utc : DateTime
}
class ForecastSQL{
+ forecast_uuid : UUID ≪ PK ≫
+ location_uuid : UUID ≪ FK ≫
+ timestamp_utc : DateTime
+ forecast_version : String
}
class ForecastValueSQL{
+ forecast_value_uuid : UUID ≪ PK ≫
+ start_utc : DateTime
+ end_utc : DateTime
+ forecast_power_kw : Float
+ horizon_minutes : Integer
+ forecast_uuid : UUID ≪ FK ≫
}
class StatusSQL{
+ status_uuid : UUID ≪ PK ≫
+ status : String
+ message : String
}
class InverterSQL{
+ inverter_uuid : UUID ≪ PK ≫
+ site_uuid : UUID ≪ FK ≫
}
class APIRequestSQL{
+ uuid : UUID ≪ PK ≫
+ url : String
+ user_uuid : UUID ≪ FK ≫
}
class MLModelSQL{
+ uuid : UUID ≪ PK ≫
+ mode_name : String
+ model_version : UUID ≪ FK ≫
}
UserSQL "1" -- "N" LocationGroupSQL : belongs_to
LocationGroupLocationSQL "N" -- "1" LocationSQL : contains
LocationGroupSQL "1" -- "N" LocationGroupLocationSQL : contains
LocationSQL "1" -- "N" GenerationSQL : generates
LocationSQL "1" -- "N" ForecastSQL : forecasts
LocationSQL "N" -- "0" MLModelSQL : ml_model
ForecastSQL "1" -- "N" ForecastValueSQL : contains
MLModelSQL "1" -- "N" ForecastValueSQL : forecasts
LocationSQL "1" -- "N" InverterSQL : contains
UserSQL "1" -- "N" APIRequestSQL : performs_request
ClientSQL "1" -- "N" LocationSQL : owns
LocationSQL "1" -- "N" LocationLocationSQL : contains
LocationLocationSQL "N" -- "1" LocationSQL : contains
class Legend{
UUID: Universally Unique Identifier
PK: Primary Key
FK: Foreign Key
U: Unique Constraint
D: Default Value
}
Multiple Clients
We have the ability to have these different scenarios
- one user - can add or view one site
- one user, can add or view multiple sites
- Two users (for example from the sample company), want to look at one site
- Two users, wanting to look at multiple sites (could be added by another user). Any user from site group can add a site.
- OCF user want to see everything (admin)
Solution
Location can mean site or region.
graph TD;
User-- N:1 -->LocationGroup;
LocationGroup-- N:N -->Location;
- One
useris in oneloationgroup. Each location group can have multiple users. - Each
locationroupcontains multiplelocations. Onelocationcan be in multiplelocationgroups
1. one user - one site
graph TD;
A(User=Alice)-->B(LocationGroup=Alice1);
B --> C(Site);
2. one user - two sites
graph TD;
A(User=Alice)-->B(LocationGroup=Alice1);
B --> C1(Site1);
B --> C2(Site2);
3. Two users - one site
graph TD;
A1(User=Alice)-->B(LocationGroup);
A2(User=Bob)-->B(LocationGroup);
B --> C1(Site1);
4. Two users - two site
graph TD;
A1(User=Alice)-->B(LocationGroup);
A2(User=Bob)-->B(LocationGroup);
B --> C1(Site1);
B --> C2(Site2);
5. OCF can see everything
graph TD;
A1(User=Alice)-->B(LocationGroup1);
A2(User=Bob)-->B(LocationGroup1);
A3(User=OCF)-->B2(LocationGroup2);
B --> C1(Site1);
B --> C2(Site2);
B2 --> C1(Site1);
B2 --> C2(Site2);
B2 --> C3(Site3);
Database migrations using alembic
Contributors ✨
Thanks goes to these wonderful people (emoji key):
This project follows the all-contributors specification. Contributions of any kind welcome!
1 maintainer
Metadata
1 maintainer
Release files for pvsite-datamodel 1.2.15
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| pvsite_datamodel-1.2.15.tar.gz | 39.9 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| pvsite_datamodel-1.2.15-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 81.9 kB
Release files / pvsite_datamodel-1.2.15.tar.gz
| Download URL | pvsite_datamodel-1.2.15.tar.gz |
|---|---|
| Size | 39.9 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
b34dce83483ea5917f9b78881d1fd1f119f1887d92ea64f74be9cd4d1b1ffa87
|
|
BLAKE2b-256 checksum How to use checksums |
3c5ba98aab0935c81bb07c1b327f00a300aa13517e9569bfcb139d0dfc2050c5
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.11.3 {"installer":{"name":"uv","version":"0.11.3","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
Release files / pvsite_datamodel-1.2.15-py3-none-any.whl
| Download URL | pvsite_datamodel-1.2.15-py3-none-any.whl |
|---|---|
| Size | 41.9 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
383d536cb5167eb6c867d840df2fd96daae3edcb2198f79bf791b68af0ec7042
|
|
BLAKE2b-256 checksum How to use checksums |
db82327998ca1ec9d80662a6bd7ebf9537b27c03c7ad917659fa0c0a185947e2
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.11.3 {"installer":{"name":"uv","version":"0.11.3","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
