quest-snl 2.0.0rc4

Sandia National Laboratories Energy Storage Application Platform

QuESt 2.0.b: Open-source Platform for Energy Storage Analytics

Current release version: 2.0

Release date: March, 2024

Contact

For issues and feedback we would appreciate it if you could use the "Issues" feature of this repository. This helps others join the discussion and helps us keep track of and document issues.

Email

Project maintainer (Tu Nguyen) @sandia.gov: tunguy

Table of contents

• Introduction
• App Hub
• Workspace
• QuESt GPT
• Innovations
• Uniqueness
• Advantages
• Installation
• Contact
• Feedback
• References

What is it?

QuESt 2.0 is an evolved version of the original QuESt, an open-source Python software designed for energy storage (ES) analytics. It transforms into a platform providing centralized access to multiple tools and improved data analytics, aiming to simplify ES analysis and democratize access to these tools.

Currently, QuESt 2.0 includes three main components:

The App Hub

The QuESt App Hub operates similarly to an app store, offering access points to a multitude of applications. Currently, various energy storage analytics tools have been available on QuESt App hub. For example:

• QuESt Data Manager manages the acquisition of data.

• QuESt Valuation estimates the potential revenue generated by energy storage systems when providing ancillary services in the electricity markets.

• QuESt BTM (Behind-The-Meter) calculates the cost savings for time-of-use and net energy metering customers utilizing behind-the-meter energy storage systems.

• QuESt Technology Selection supports in selecting the appropriate energy storage technology based on specific applications and requirements.

• QuESt Performance evaluates the performance of energy storage systems in different climatic conditions.

• QuESt Microgrid supports microgrid design and simulation considering energy storage as a key component.

It has been designed with key features to improve user experience and application management:

• User-Friendly Access: Users can easily find and install applications that suit their specific needs.

• Isolated Environments: Upon installation, each application creates an isolated environment. This ensures that applications run independently, preventing conflicts, and enhancing stability.

• Simultaneous Operation: Multiple applications can be installed and operated simultaneously, allowing users to leverage different tools without interference.

The Workspace

The QuESt Workspace provides an integrated environment where users can create workflows by assembling multiple applications into a coherent process. It enhances the platform's usability and efficiency through several mechanisms:

• Integration of Applications: Users can create work processes that integrate multiple apps by assembling pipelines using plugin extensions. This modular approach allows for the flexible composition of analytics workflows tailored to specific needs.

• Workflow Management: The workspace supports the selection, assembly, connection, and post-processing of data and tools. This structured approach streamlines the analytics process, from data preparation to visualization, making it easier to manage and understand.

QuESt GPT

QuESt GPT represents a leap forward in data analytics within the platform, utilizing generative AI (specifically Large Language Models, or LLM) for data characterization and visualization:

• Data Insights: Users can select datasets and ask questions about the data, with QuESt GPT providing insights based on the data's characteristics. This interaction model simplifies complex data analysis, making it accessible to users without deep technical expertise.

• Utilization of LLMs: By leveraging advanced open-source LLMs such as OpenAi’s GPT-4 and Meta’s Llama2, QuESt GPT can perform sophisticated data analytics tasks, such as characterizing and visualizing large datasets. This enables users to gain deeper insights from their data, supporting more informed decision-making at no costs.

What are the key innovations of QuESt 2.0?

QuESt 2.0 facilitates the advancement of energy storage technology by making powerful analytics tools accessible to all energy storage stakeholders, aligning with DOE’s energy storage program goals. The platform standardizes data and program structures, integrates applications seamlessly, and utilizes generative AI for advanced analytics, simplifying user interaction and enabling deeper insights from diverse data sources. This positions QuESt 2.0 as a pioneering platform in the energy storage domain, with the potential to significantly impact both the field and the broader energy landscape. Specifically, the key innovations of QuESt 2.0 include:

1. Integration and Usability: At its core, QuESt 2.0 revolutionizes how energy storage analytics are performed by providing a seamless, user-friendly platform that integrates multiple applications developed by independent developers. This allows for a more cohesive and efficient user experience, significantly lowering the learning curve for users at various levels of expertise.
2. AI-powered Data Analytics: The incorporation of QuESt GPT, utilizing Large Language Models (LLM), represents a significant technological leap forward. This feature enables users to perform more sophisticated data analytics, providing deeper insights from diverse data sources. It allows users to interact with data in an intuitive way, asking questions and receiving insights, which democratizes access to complex data analysis.
3. Complex Workflows: The QuESt Workspace and the QuESt App Hub enhance the platform's capability to support complex analytical workflows. Users can integrate multiple applications into a single process, creating efficient pipelines for data analysis. The users can run their workflows locally or schedule them to run on cloud services (e.g., AWS, Azure..)

How is QuESt 2.0 different from the other tools in Energy Storage Analytics?

QuESt 2.0 distinguishes itself in the crowded space of energy storage analytics tools by offering a unified platform rather than a collection of individual tools. While there are numerous tools available, these tend to focus on specific aspects of energy storage analysis and lack the integration and broad applicability that QuESt 2.0 provides.

Key Competitive Advantages of QuESt 2.0:

• Unified Platform: Unlike individual tools that offer limited capabilities in specific topics, QuESt 2.0 serves as a comprehensive platform that integrates multiple applications for energy storage analytics. This integration allows users to access a wide range of tools and functionalities within a single environment, streamlining the workflow and enhancing productivity.

• Advanced Data Analytics: With the inclusion of QuESt GPT for advanced data analytics, QuESt 2.0 utilizes generative AI to provide deeper insights from diverse data sources. This capability is not commonly found in other tools, giving QuESt 2.0 a significant technological edge.

• Flexibility and Scalability: QuESt 2.0's modular design allows for the easy integration of new tools and applications, ensuring that the platform remains relevant and adaptable to future developments in energy storage analytics.

• Collaborative Development: As an open-source platform, QuESt 2.0 benefits from contributions by a wide community of developers, enhancing its capabilities and ensuring continuous improvement.

How to download QuESt?

QuESt is currently available on Github at: https://github.com/sandialabs/snl-quest.

Installation Instructions for QuESt

Easy Installation (Recommended)

For Windows Users:

Prerequisites

GitHub: You need Git installed on your system to clone the repository. If you don’t have Git installed, you can follow these instructions to install it.

1. Click on the releases located at the top of this page on the right side as shown in the picture below. (Opening this page in a new tab may be helpful to follow along with the instructions)
2. On the releases page, download the quest_win.zip file found in the assets of QuESt2.0.b pre-release.
3. Extract the downloaded file.
4. Navigate inside the extracted directory and double click on start.bat. This will setup QuESt and launch it when it is finished. For future uses of QuESt users can double click the start.bat file to launch QuESt.

For Mac Users

Prerequisites

1. GitHub: You need Git installed on your system to clone the repository. If you don’t have Git installed, you can follow these instructions to install it.

2. GLPK (GNU Linear Programming Kit): You need to install GLPK. You can do this using either Homebrew or Conda. If you don’t have either installed, follow the instructions below:

   • Installing GLPK using Homebrew:

     1. Open the Terminal.
     2. Run the following command to install Homebrew:

        /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
        

     3. After installation, you can install GLPK using:

        brew install glpk
        

   • Installing GLPK using Conda:

     1. After installing Conda (Instructions below), you can install GLPK using:

        conda install -c conda-forge glpk
        

3. Conda:

   1. You can install Miniconda by following these instructions.

4. Apple Silicon (M1/M2/Mxx) Setup Instructions

If you have an M-type architecture (Apple Silicon), follow these steps to set up your environment.

---

1. Install Rosetta 2

Rosetta 2 allows you to run Intel-based applications on Apple Silicon. To install Rosetta 2:

1. Open the Terminal.

2. Run the following command:

   softwareupdate --install-rosetta --agree-to-license
   

   Close the Terminal completely by using Command + Q or selecting Quit Terminal from the menu.

3. Configure the Terminal to Open Using Rosetta

To ensure that the Terminal runs in Rosetta mode:

Open Finder and navigate to Applications > Utilities.
Locate the Terminal application.
Right-click on Terminal and select Get Info.
In the Get Info window, check the box that says Open using Rosetta.
Close the Get Info window.

3. Open a Terminal with Rosetta Active

After configuring the Terminal to open using Rosetta:

Open the Terminal again.
Verify that Rosetta is active by running the following command:

  ```bash
   arch
  ```

You should see i386 as the output, indicating that the Terminal is running in Rosetta mode.

4. Install Intel Homebrew

With Rosetta active, install Intel Homebrew:

Run the following command:

  ```bash
  arch -x86_64 /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
  ```


Follow the on-screen instructions to complete the installation.

5. Install Python 3.9 (Intel Version)

After installing Intel Homebrew, use it to install Python 3.9:

Run the following command:
  ```bash
  arch -x86_64 /usr/local/bin/brew install python@3.9
  ```
Verify the installation by checking the Python version and architecture:

    arch -x86_64 /usr/local/opt/python@3.9/bin/python3.9 -c "import platform; print('Python Version:', platform.python_version()); print('Architecture:', platform.machine())"

You should see output similar to:

Python Version: 3.9.x
Architecture: x86_64

6. Create a folder on the Desktop called quest_folder

7. Open a terminal at the folder by right clicking on the folder and selecting open terminal at folder.

8. Clone the QuESt repository using the following command:

   git clone -b QuESt_2.0.c https://github.com/sandialabs/snl-quest.git
   

9. Navigate inside the cloned repository from the terminal using the command:

      cd snl-quest
   

10. Set up the environment for QuESt by running the following command:

   conda create -n quest python=3.9

When prompted type y and enter.

11. Activate the environment that has just been created using the command:

   conda activate quest

12. With the environment active and from inside the snl-quest directory set up the dependencies for QuESt using the command:

pip install -e .[dev]

13. Now QuESt is set up and can be run using the command:

python -m quest

Future runs of QuESt

1. After the environment has been set up open a terminal inside the snl-quest folder and activate the environment using the command:

   conda activate quest
   

2. Run QuESt with the command:

   python -m quest
   

Installing QuESt using the executable

1. Click on the releases located at the top of this page on the right side.

2. Download the quest_mac.zip file found in the assets of the QuESt 2.0.b pre-release.

3. Extract the downloaded file.

4. Activate Rosetta 2 (for Apple Silicon users):

   • Navigate to Finder > Applications > Utilities and locate Terminal.
   • Right-click on Terminal and select Get Info.
   • Check the box that says Open using Rosetta.
   • If you need to install Rosetta you can run this line from the terminal:

   softwareupdate --install-rosetta --agree-to-license
   

5. Open the Terminal after Rosetta has been activated.

6. Navigate to the directory where QuESt is located. You can do this using the cd command. For example:

   cd /path/to/quest_directory
   

7. Make the start.sh script executable and launch QuESt by running:

   chmod +x start.sh && ./start.sh
   

For Linux Users:

1. Click on the code button at the top of the page and then select "Download ZIP" from the dropdown.

2. Go to the location of the downloaded ZIP folder (default is Downloads).

3. Extract the ZIP folder to your desired location (e.g., Desktop).

4. Install python (3.9.13)

Installing Python

1. Go to python.org.

2. Download a version of Python suitable for your operating system (3.9.x recommended).

3. Follow the installation instructions provided on the website.

4. Install git

Installing Git

• Visit git-scm.com to download Git for your operating system.
• Follow the installation instructions provided on the website.

6. Open a terminal inside the installed quest directory and create a virtual env.

Setting Up a Virtual Environment

1. Install virtualenv (if not already installed):

   python3 -m pip install virtualenv
   

2. Create a virtual environment:

   virtualenv env
   

3. Activate the virtual environment:

   source env/bin/activate
   

4. With the virtual env activated we can now setup the environment for QuESt with the following command:

   python3 -m pip install -e .[dev]
   

5. Run QuESt with the following command:

    ```bash
    python3 -m quest
    ```
   

Installation for Developers:

Prerequisites

• Python (3.9.x recommended) installed on your system
• Git installed on your system

Installing Python

1. Go to python.org.
2. Download a version of Python suitable for your operating system (3.9.x recommended).
3. Follow the installation instructions provided on the website.

Installing Git

• Visit git-scm.com to download Git for your operating system.
• Follow the installation instructions provided on the website.

Setting Up a Virtual Environment

1. Install virtualenv (if not already installed):

   pip install virtualenv
   

2. Create a virtual environment:

   virtualenv env
   

3. Activate the virtual environment:

   • On Windows:

     .\env\Scripts\activate
     

   • On Linux/macOS:

     source env/bin/activate
     

Installing QuESt with Git

1. Clone the Repository:

   git clone <repository_url>
   

   Replace <repository_url> with the URL of the QuESt GitHub repository.

2. Navigate to the QuESt Directory:

   cd <repository_name>
   

   Replace <repository_name> with the name of the directory where QuESt was cloned.

3. Install Dependencies:

   pip install -e .[dev]
   

Installing QuESt via Pip

1. Create a Virutal environment:

   python -m venv venv
   

2. Activate the virual environment

   Windows:

   venv\Scripts\activate
   

   Mac:

   source venv/bin/activate
   

3. Install QuESt:

   pip install quest-snl[dev]
   

4. Run QuESt:

   quest
   

Running QuESt

Easy Version:

1. If you have followed the Easy Installation steps you can now double click on the start.bat if you are on windows or on the unix_entry if you are on mac or linux to launch quest. Some unix based systems may require users to always use the terminal to run the unix_entry.sh script to launch QuESt.

For Developers:

1. Run QuESt:
   • Once the dependencies are installed, ensure you have navigated to the directory where QuESt is installed and the virtual environment is activated. You can run QuESt using the following command:
     • On Windows:

       python -m quest
       

     • On Linux/macOS:

       python3 -m quest
       

     • Alternatively quest may be run any directory as long as the venv is active using the terminal command:

       quest
       

Deactivating the Virtual Environment

1. Deactivate the Virtual Environment:

   deactivate
   

   This will return you to your system's default Python environment.

References

Nguyen, Tu A., David A. Copp, and Raymond H. Byrne. "Stacking Revenue of Energy Storage System from Resilience, T&D Deferral and Arbitrage." 2019 IEEE Power & Energy Society General Meeting (PESGM). IEEE, 2019.

Byrne, Raymond H., Tu A. Nguyen, and Ricky J. Concepcion. "Opportunities for Energy Storage in CAISO." 2018 IEEE Power & Energy Society General Meeting (PESGM). IEEE, 2018. Available online.

Byrne, Raymond H., Tu Anh Nguyen, and Ricky James Concepcion. Opportunities for Energy Storage in CAISO. No. SAND2018-5272C. Sandia National Lab.(SNL-NM), Albuquerque, NM (United States), 2018. Available online.

Concepcion, Ricky J., Felipe Wilches-Bernal, and Raymond H. Byrne. "Revenue Opportunities for Electric Storage Resources in the Southwest Power Pool Integrated Marketplace." 2018 IEEE Power & Energy Society General Meeting (PESGM). IEEE, 2018. Available online.

Wilches-Bernal, Felipe, Ricky J. Concepcion, and Raymond H. Byrne. "Electrical Energy Storage Participation in the NYISO Electricity and Frequency Regulation Markets." 2018 IEEE Power & Energy Society General Meeting (PESGM). IEEE, 2018.

Nguyen, Tu A., and Raymond H. Byrne. "Maximizing the cost-savings for time-of-use and net-metering customers using behind-the-meter energy storage systems." 2017 North American Power Symposium (NAPS). IEEE, 2017. Available online.

Nguyen, Tu A., et al. "Maximizing revenue from electrical energy storage in MISO energy & frequency regulation markets." 2017 IEEE Power & Energy Society General Meeting. IEEE, 2017. Available online.

Byrne, Raymond H., Ricky J. Concepcion, and César A. Silva-Monroy. "Estimating potential revenue from electrical energy storage in PJM." 2016 IEEE Power and Energy Society General Meeting (PESGM). IEEE, 2016. Available online.

Byrne, Raymond H., et al. "The value proposition for energy storage at the Sterling Municipal Light Department." 2017 IEEE Power & Energy Society General Meeting. IEEE, 2017. Available online.

Byrne, Raymond H., et al. "Energy management and optimization methods for grid energy storage systems." IEEE Access 6 (2017): 13231-13260. Available online.

Byrne, Raymond H., and César A. Silva-Monroy. "Potential revenue from electrical energy storage in ERCOT: The impact of location and recent trends." 2015 IEEE Power & Energy Society General Meeting. IEEE, 2015. Available online.