Categorizer

Categorizer is a simple tool which you can use to categorise your string records into predefined -nested- categories using the power of LLMs.

• upload your categories and subcategories ( read from yaml file or create them on the fly )
• initialize LEC. You can use different modes and your own task-specific prompts if you like
• Run the LEC and it will output a dataframe with all categories and subcategories and llm's reasoning to select them
• It is possible to leave notes for LLM for each category to help him categorize with more accuracy
• You can also use included naive classification method which supports regex based or keyword matching mechanism to reduce the LLM compute

Here is some benchmarking you to understand better.

category depth category combination size allowed retry

Usage

lec = LLMEnhancedClassifier(
        llm_model=llm_model,
        llm_refiner_model=llm_refiner,
        categories_yaml_path='categories.yaml',
        meta_patterns_yaml_path='bank_patterns.yaml',
        subcategory_level=2  # Change this value to set the number of subcategories (max 4)
    )
    
lec.load_records(df)
 df = lec.classify_lvl_by_lvl()