note-to-json 0.2.3

Convert Markdown or text files to structured JSON, offline.

Note to JSON

Convert Markdown or text files to structured JSON, offline.

Features

• Privacy-first: All processing happens locally, no data sent to external services
• Flexible input: Supports Markdown, plain text, and JSON files
• Automatic encoding detection: Handles UTF-8, UTF-16, and other encodings
• Batch processing: Process multiple files with glob patterns
• Resilient parsing: Graceful handling of malformed inputs and encoding issues
• Progress reporting: Detailed feedback for batch operations
• Error recovery: Continue processing even when some files fail

Installation

pip install note-to-json

Quick Start

# Convert a single file
note2json input.md

# Convert multiple files
note2json *.md

# Output to STDOUT
note2json input.md --stdout

# Pretty-print JSON
note2json input.md --stdout --pretty

CLI Usage

Basic Commands

note2json [OPTIONS] INPUT_FILE(S)

Options

• -o, --output PATH: Specify output file path
• --stdout: Print JSON to STDOUT instead of writing to file
• --pretty: Pretty-print JSON with 2-space indentation
• --stdin: Read input from STDIN instead of files
• --input-format {auto,md,txt,json}: Specify input format (default: auto)
• --no-emoji: Disable emoji in status output
• --continue-on-error: Continue processing remaining files even if some fail
• --verbose: Show detailed progress information
• --retry-failed: Automatically retry failed files with different strategies

Input Formats

• auto (default): Automatically detect format based on content
• md/txt: Parse as Markdown/plain text
• json: Parse as JSON (with schema validation)

Examples

# Parse to default output file
note2json input.md                    # → input.parsed.json

# Parse to custom output file
note2json input.md -o output.json     # → output.json

# Parse to STDOUT
note2json input.md --stdout           # → prints to terminal

# Pretty-print to STDOUT
note2json input.md --stdout --pretty  # → formatted JSON

# Process multiple files
note2json *.md                        # → individual .parsed.json files

# Continue on errors
note2json *.md --continue-on-error    # → process all files, report failures

# Retry failed files automatically
note2json *.md --retry-failed         # → retry failed files with different strategies

# Show progress
note2json *.md --verbose              # → detailed progress information

# Read from STDIN (Windows)
type data.json | note2json --stdin --input-format json --stdout

# Read from STDIN (macOS/Linux)
cat data.json | note2json --stdin --input-format json --stdout

Resilience Features

Error Handling

The CLI provides robust error handling with clear, actionable error messages:

• Encoding issues: Automatic fallback to multiple encoding detection methods
• Malformed inputs: Graceful degradation with automatic validation fixes
• Batch processing: Continue processing even when individual files fail
• Detailed reporting: Comprehensive error summaries with categorization
• Actionable advice: Specific suggestions for fixing common issues
• Retry strategies: Automatic retry with different parsing approaches

Error Types

• Missing files: Exit code 2
• Parsing errors: Exit code 3
• Encoding errors: Detailed information about attempted encodings
• Validation errors: Automatic fixing of common schema issues
• Format mismatches: Clear guidance on input format selection
• Retry failures: Information when all retry strategies fail

Enhanced Error Messages

Error messages now include specific, actionable advice:

# Example of enhanced error message with advice
Error: Schema validation failed at 'title': 'None' is not of type 'string'
💡 Advice: Add the missing required field 'title'

Retry Logic

Use --retry-failed to automatically attempt processing failed files with different strategies:

note2json *.md --retry-failed

The retry system will:

1. Format switching: Try different input formats (txt, json, auto)
2. Raw text processing: Fall back to basic text extraction
3. Schema relaxation: Create minimal valid structures when possible
4. Detailed reporting: Show which retry strategy succeeded

Continue on Error

Use --continue-on-error to process all files even when some fail:

note2json *.md --continue-on-error

This will:

• Process all files that can be parsed
• Report failures with detailed error messages
• Provide a summary of successful vs. failed files
• Exit with appropriate error code

Enhanced Progress Reporting

Use --verbose for detailed progress information with time estimation:

note2json *.md --verbose

Shows:

• Current file being processed
• Progress counter (e.g., [3/10])
• Visual progress bar with percentage
• Estimated time remaining (ETA)
• Summary of results
• Error breakdown by type
• Troubleshooting tips for common issues

Output Schema

The tool outputs structured JSON with the following schema:

{
  "title": "string",
  "timestamp": "ISO 8601 date-time",
  "raw_text": "string",
  "plain_text": "string",
  "tags": ["string"],
  "headers": ["string"],
  "date": "string (optional)",
  "tone": "string (optional)",
  "summary": "string (optional)",
  "reflections": ["string (optional)"]
}

Input Format Support

Markdown/Text

• Headers: Extracts # Title as headers
• Metadata: Parses **Date:**, **Tags:**, **Tone:** fields
• Summary: Extracts content between **Summary:** and ---
• Reflections: Extracts bullet points after **Core Reflections:**

JSON

• Schema validation: Ensures output matches required schema
• Auto-normalization: Converts arbitrary JSON to schema format
• Format detection: Automatically identifies JSON vs. text content

Encoding Support

• UTF-8: Standard encoding with BOM support
• UTF-16: Little-endian and big-endian variants
• Fallback detection: Uses chardet for automatic encoding detection
• Error handling: Graceful degradation with detailed error reporting

Development

Installation

git clone https://github.com/Mugiwara555343/note2json.git
cd note2json
pip install -e .

Testing

# Run all tests
pytest

# Run integration tests only
pytest -m integration

# Run with coverage
pytest --cov=note_to_json

Code Quality

# Format code
black note_to_json/ tests/

# Sort imports
isort note_to_json/ tests/

# Run pre-commit hooks
pre-commit run --all-files

Contributing

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Add tests for new functionality
5. Ensure all tests pass
6. Submit a pull request

License

MIT License - see LICENSE file for details.

Changelog

v0.2.2

• Resilience improvements: Better error handling and recovery
• Continue on error: Process remaining files even when some fail
• Progress reporting: Detailed feedback for batch operations
• Enhanced encoding detection: Fallback mechanisms and better error messages
• Validation fixes: Automatic correction of common schema issues
• Error categorization: Grouped error reporting for better analysis

v0.2.1

• Improved encoding detection
• Better error messages
• Enhanced JSON passthrough

v0.2.0

• Added JSON input support
• Improved encoding handling
• Better error reporting

v0.1.0

• Initial release
• Basic Markdown parsing
• CLI interface