schema-miner 2.0.0

A Human-in-the-Loop Workflow for Scientific Schema Mining with Large Language Models

LLMs4SchemaDiscovery: A Human-in-the-Loop Workflow for Scientific Schema Mining with Large Language Models

Schema-Miner is an open-source framework for scientific schema mining. It combines Large Language Models (LLMs) with human-in-the-loop refinement to extract, and semantically ground schema properties from unstructured text. Documentation and usage guides are available at schema-miner.readthedocs.io.

🧪 Installation

Install the package directly from PyPI:

pip install schema-miner

If you are working with the source code directly, install dependencies from requirements.txt:

git clone https://github.com/sciknoworg/schema-miner.git
cd schema-miner
pip install -r requirements.txt
pip install -e .

⚙️ System Requirements

Running with OpenAI models (e.g., GPT-4o, GPT-4-turbo) requires no special hardware beyond a basic system with internet access, since inference is API-based. For open-source models (e.g., Llama 3.1 8B), local execution is possible on CPU but slow; for practical performance, a GPU with sufficient VRAM (per model specifications) is strongly recommended.

For more details, please check the documentation here.

🚀 Quick Start

For a quick start, see the provided example notebooks highlighting the overall workflows of the schema-miner.

	Notebook
1	Schema Mining With LLMs and expert Example
2	Schema Ontology Grounding Example

📚 Citing this Work

If you use this repository in your research or applications, please cite the appropriate paper(s):

• Schema-Miner (for schema discovery/mining only):

  Sameer Sadruddin, Jennifer D’Souza, Eleni Poupaki, Alex Watkins, Hamed Babaei Giglou, Anisa Rula, Bora Karasulu, Sören Auer, Adrie Mackus, and Erwin Kessels. LLMs4SchemaDiscovery: A Human-in-the-Loop Workflow for Scientific Schema Mining with Large Language Models. In The Semantic Web – ESWC 2025, Springer, Cham, pp. 244–261. https://doi.org/10.1007/978-3-031-94578-6_14

  📌 BibTeX

  @InProceedings{10.1007/978-3-031-94578-6_14,
    author    = {Sadruddin, Sameer and D'Souza, Jennifer and Poupaki, Eleni and Watkins, Alex and Babaei Giglou, Hamed and Rula, Anisa and Karasulu, Bora and Auer, S{\"o}ren and Mackus, Adrie and Kessels, Erwin},
    editor    = {Curry, Edward and Acosta, Maribel and Poveda-Villal{\'o}n, Maria and van Erp, Marieke and Ojo, Adegboyega and Hose, Katja and Shimizu, Cogan and Lisena, Pasquale},
    title     = {LLMs4SchemaDiscovery: A Human-in-the-Loop Workflow for Scientific Schema Mining with Large Language Models},
    booktitle = {The Semantic Web},
    year      = {2025},
    publisher = {Springer Nature Switzerland},
    address   = {Cham},
    pages     = {244--261},
    isbn      = {978-3-031-94578-6},
  }
  

• Schema-Miner^pro (for schema mining with QUDT grounding / ontology grounding):

  Sameer Sadruddin, Jennifer D’Souza, Eleni Poupaki, Alex Watkins, Bora Karasulu, Sören Auer, Adrie Mackus, and Erwin Kessels. SCHEMA-MINER^pro: Agentic AI for Ontology Grounding over LLM-Discovered Scientific Schemas in a Human-in-the-Loop Workflow. In Semantic Web Journal. https://www.semantic-web-journal.net/system/files/swj3871.pdf

  📌 BibTeX

  @InProceedings{10.1007/978-3-031-94578-6_14,
    author    = {Sadruddin, Sameer and D'Souza, Jennifer and Poupaki, Eleni and Watkins, Alex and Karasulu, Bora and Auer, S{\"o}ren and Mackus, Adrie and Kessels, Erwin},
    title     = {SCHEMA-MINERpro: Agentic AI for Ontology Grounding over LLM-Discovered Scientific Schemas in a Human-in-the-Loop Workflow},
    journal = {Semantic Web Journal},
    year      = {2025},
  }
  

👥 Contact & Contributions

We’d love to hear from you! Whether you're interested in collaborating on schema-miner or have ideas to extend its capabilities, feel free to reach out:

• Collaboration inquiries: Contact Jennifer D'Souza at jennifer.dsouza [at] tib.eu

• Development questions or bug reports: Please open an issue right here in the repository or get in touch with the lead developer Sameer Sadruddin at sameer.sadruddin [at] tib.eu

Let’s build better schema-mining tools—together!

📃 License

This work is licensed under a MIT License

🔗 Links

Source Code: https://github.com/sciknoworg/schema-miner

Documentation: https://schema-miner.readthedocs.io/en/latest/

Issues: https://github.com/sciknoworg/schema-miner/issues