Extensible Next.js/DeepWiki content extractor with zero external dependencies
Project description
deepwiki-to-md
English README. 日本語はこちら → README_JP.md
Zero-dependency CLI and Python library to extract Markdown from Next.js/DeepWiki HTML. Includes a small search helper for public repository indexes and an optional chat helper.
- CLI:
deepwiki-to-md - Requirements: Python 3.8+
- Dependencies: Standard library only (optional extras for dev/docs)
Install
pip install deepwiki-to-md
Usage
- From local HTML/string (CLI and Python):
# CLI
echo "<html>...</html>" | deepwiki-to-md
# Python API
from deepwiki_to_md import ContentExtractor
html = """
<!doctype html>
<html>...</html>
"""
extractor = ContentExtractor()
md = extractor.extract_from_html(html)
print(md)
- From URL (files are saved only when the input is a URL):
# CLI
# Files under .deepwiki are created only for URL input
deepwiki-to-md https://deepwiki.com/microsoft/vscode/some-page --path ./.deepwiki
# Python API (same behavior as the CLI)
from deepwiki_to_md import ContentExtractor, save_markdown_to_library
url = "https://deepwiki.com/microsoft/vscode/some-page"
base_dir = "./.deepwiki" # equivalent to --path (optional)
extractor = ContentExtractor()
md = extractor.extract_from_url(url)
result = save_markdown_to_library(md, url, base_dir)
print("saved files:")
for p in result["saved_files"]:
print(" -", p)
print("library index:", result["library_file"]) # .deepwiki/<username>/<library>.md
- Search public repository indexes:
# CLI (JSON by default)
deepwiki-to-md --search "Gemini"
# Human-readable development-log style
deepwiki-to-md --search "Gemini" --devlog
# Python API (same search capability)
from search_repository import search_repositories, API_URL
print(API_URL) # => https://api.devin.ai/ada/list_public_indexes
result = search_repositories("Gemini")
indices = result.get("indices", [])
print("indices:", len(indices))
License
MIT License
More documentation
- Library reference (includes both Python API and CLI examples): deepwiki_to_md.md
Chat (Devin API) result object: ChatResult
The chat helper (src/chat.py) returns a ChatResult object instead of a plain dict.
-
Highlights
- Inherits from dict → works with json.dumps(result) directly.
- Convenient attribute access (e.g., result.response_message) and to_dict().
- print(result) shows a human-readable summary.
-
Main properties
- sent_message: str
- response_message: Optional[str]
- status_code: Any
- reference_files: List[str]
- reference_file_contents: Dict[str, str]
-
Example (excerpt)
import asyncio
import json
from chat import load_or_create_config, send_chat_message, ChatResult
async def main() -> None:
config = load_or_create_config('./config.json')
if not config:
raise SystemExit('config missing')
result: ChatResult = await send_chat_message(
wiki_url='https://deepwiki.com/microsoft/vscode',
message='What is the purpose of this repository?',
config=config,
use_deep_research=False,
)
print(result) # human-readable summary via __str__
print(result.response_message) # attribute access
print(json.dumps(result, indent=2, ensure_ascii=False)) # still a dict
if __name__ == '__main__':
asyncio.run(main())
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
deepwiki_to_md-2.0.2.tar.gz
(24.5 kB
view details)
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file deepwiki_to_md-2.0.2.tar.gz.
File metadata
- Download URL: deepwiki_to_md-2.0.2.tar.gz
- Upload date:
- Size: 24.5 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/6.2.0 CPython/3.11.9
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
e41efa67c31461c5a03d92ac76a7c414780af4dcc455c32c662df515a0f05efc
|
|
| MD5 |
966b582a92cc59fb74df8673566507a2
|
|
| BLAKE2b-256 |
9b8783d30e2a253bba73781ce6e6d6e6f7800b0b78cadb8161406b386318cc9e
|
File details
Details for the file deepwiki_to_md-2.0.2-py3-none-any.whl.
File metadata
- Download URL: deepwiki_to_md-2.0.2-py3-none-any.whl
- Upload date:
- Size: 23.8 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/6.2.0 CPython/3.11.9
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
9b8cb1657190f38fd4f583a4503981f13f0805a732378030e6595f4d12fca996
|
|
| MD5 |
3fe70286aecded1cf0fa618c977d5ae5
|
|
| BLAKE2b-256 |
b00beada1234f673b4e36dd0c28f7f595cff37fd62e317d1b70f46d7272d8312
|
