nlptoolkit-dictionary-cy 1.0.37

Simple Dictionary Processing

Turkish Dictionary ============

    This resource is a dictionary of Modern Turkish, comprised of the definitions of over 50.000 individual entries. Each entry is matched with its corresponding synset (set of synonymous words and expressions) in the Turkish WordNet, KeNet.
    
    The bare-forms in the lexicon consists of nouns, adjectives, verbs, adverbs, shortcuts, etc. Each bare-form appears the same in the lexicon except verbs. Since the bare-forms of the verbs in Turkish do not have the infinitive affix ‘mAk’, our lexicon includes all verbs without the infinitive affix. The bare-forms with diacritics are included in two forms, with and without diacritics. For example, noun ‘rüzgar’ appear both as ‘rüzgar’ and ‘rüzgâr’.
    
    Special markers are included as bare-forms such as doc, s, etc.
    
    Some compound words are included in their affixed form. For instance, ‘acemlalesi’ appears as it is, but not as ‘acemlale’.
    
    Foreign words, especially proper noun foreign words, are included, so that the system can easily recognize them as proper nouns. For instance, the words ‘abbott’, ‘abbigail’ are example foreign proper nouns. Including foreign proper nouns, there are 19,000 proper nouns in our lexicon.
    
    From derivational suffixes, we only include words which has taken -lI, -sIz, -CI, -lIk, and -CIlIk derivational affixes. For example, the bare-forms ‘abacı’, ‘abdallık’, ‘abdestli’ and ‘abdestlilik’, are included, since they have taken one or more derivational affixes listed above.
    
    Each bare-form has a set of attributes. For instance, ‘abacı’ is a noun, therefore, it includes CL_ISIM attribute. Similarly, ‘abdestli’ is an adjective, which includes IS_ADJ attribute. If the bare-form has homonyms with different part of speech tags, all corresponding attributes are included.
    
    |Name|Purpose|
    |---|---|
    |CL ISIM, CL FIIL, IS_OA|Part of speech tag(s)|
    |IS_DUP|Part of a duplicate form|
    |IS_KIS|Abbreviation, which does not obey vowel harmony while taking suffixes.|
    |IS_UU, IS_UUU|Does not obey vowel harmony while taking suffixes.|
    |IS_BILES|A portmanteau word in affixed form, such as ‘adamotu’|
    |IS_B_SI|A portmanteau word ending with ‘sı’, such as ‘acemlalesi’|
    |IS_CA|Already in a plural form, therefore can not take plural suffixes such as ‘ler’ or ‘lar’.|
    |IS_ST|The second consonant undergoes a resyllabification.|
    |IS_UD, IS_UDD, F_UD|Includes vowel epenthesis.|
    |IS_KG|Ends with a ‘k’, and when it is followed by a vowel-initial suffix, the final ‘k’ is replaced with a ‘g’.|
    |IS_SD, IS_SDD, F_SD|Final consonant gets devoiced during vowel-initial suffixation.|
    |F GUD, F_GUDO|The verb bare-form includes vowel reduction.|
    |F1P1, F1P1-NO-REF|A verb, and depending on this attribute, the verb can (or can not) take causative suffix, factitive suffix, passive suffix etc.|
    
    Simple Web Interface
    ============
    [Turkish Dictionary Search Link 1](http://104.247.163.162/nlptoolkit/turkish-dictionary.html) [Turkish Dictionary Search Link 2](https://starlangsoftware.github.io/nlptoolkit-web-simple/turkish-dictionary.html)
    
    [Turkish MorphoLex Search Link 1](http://104.247.163.162/nlptoolkit/turkish-morphological-lexicon.html) [Turkish MorphoLex Search Link 2](https://starlangsoftware.github.io/nlptoolkit-web-simple/turkish-morphological-lexicon.html)
    
    Video Lectures
    ============
    
    [<img src="https://github.com/StarlangSoftware/Dictionary/blob/master/video1.jpg" width="50%">](https://youtu.be/10iAqbfsA2A)[<img src="https://github.com/StarlangSoftware/Dictionary/blob/master/video2.jpg" width="50%">](https://youtu.be/C-_TZDkFwzQ)
    
    For Developers
    ============
    You can also see [Python](https://github.com/starlangsoftware/Dictionary-Py), [Java](https://github.com/starlangsoftware/Dictionary), [C++](https://github.com/starlangsoftware/Dictionary-CPP), [C](https://github.com/starlangsoftware/Dictionary-C), [Swift](https://github.com/starlangsoftware/Dictionary-Swift), [Js](https://github.com/starlangsoftware/Dictionary-Js), [Php](https://github.com/starlangsoftware/Dictionary-Php), or [C#](https://github.com/starlangsoftware/Dictionary-CS) repository.
    
    ## Requirements
    
    * [Python 3.7 or higher](#python)
    * [Git](#git)
    
    ### Python 
    
    To check if you have a compatible version of Python installed, use the following command:
    
        python -V
        
    You can find the latest version of Python [here](https://www.python.org/downloads/).
    
    ### Git
    
    Install the [latest version of Git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git).
    
    ## Pip Install
    
    	pip3 install NlpToolkit-Dictionary-Cy
    
    ## Download Code
    
    In order to work on code, create a fork from GitHub page. 
    Use Git for cloning the code to your local or below line for Ubuntu:
    
    	git clone <your-fork-git-link>
    
    A directory called Dictionary will be created. Or you can use below link for exploring the code:
    
    	git clone https://github.com/starlangsoftware/Dictionary-Cy.git
    
    ## Open project with Pycharm IDE
    
    Steps for opening the cloned project:
    
    * Start IDE
    * Select **File | Open** from main menu
    * Choose `DataStructure-CY` file
    * Select open as project option
    * Couple of seconds, dependencies will be downloaded. 
    
    Detailed Description
    ============
    
    + [TxtDictionary](#txtdictionary)
    + [TxtWord](#txtword)
    + [SyllableList](#syllablelist)
    
    ## TxtDictionary
    
    Dictionary is used in order to load Turkish dictionary or a domain specific dictionary. In addition, misspelled words and the true forms of the misspelled words can also be loaded.
    
    To load the Turkish dictionary and the misspelled words dictionary,
    
    	a = TxtDictionary()
    	
    To load the domain specific dictionary and the misspelled words dictionary,
    
    	TxtDictionary(self, fileName=None, misspelledFileName=None)
    
    And to see if the dictionary involves a specific word, getWord is used.
    
    	getWord(self, name: str) -> Word
    
    ## TxtWord
    
    The word features: To see whether the TxtWord class of the dictionary is a noun or not,
    
    	isNominal(self) -> bool
    
    To see whether it is an adjective,
    
    	isAdjective(self) -> bool
    
    To see whether it is a portmanteau word,
    
    	isPortmanteau(self) -> bool
    
    To see whether it obeys vowel harmony,
    
    	notObeysVowelHarmonyDuringAgglutination(self) -> bool
    
    And, to see whether it softens when it get affixes, the following is used.
    
    	rootSoftenDuringSuffixation(self) -> bool
    
    ## SyllableList
    
    To syllabify the word, SyllableList class is used.
    
    	SyllableList(self, word: str)
    
    # Cite
    
    	@inproceedings{yildiz-etal-2019-open,
        	title = "An Open, Extendible, and Fast {T}urkish Morphological Analyzer",
        	author = {Y{\i}ld{\i}z, Olcay Taner  and
          	Avar, Beg{\"u}m  and
          	Ercan, G{\"o}khan},
        	booktitle = "Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2019)",
        	month = sep,
        	year = "2019",
        	address = "Varna, Bulgaria",
        	publisher = "INCOMA Ltd.",
        	url = "https://www.aclweb.org/anthology/R19-1156",
        	doi = "10.26615/978-954-452-056-4_156",
        	pages = "1364--1372",
    	}
    
    For Contibutors
    ============
    
    ### Setup.py file
    1. Do not forget to set package list. All subfolders should be added to the package list.
    ```
        packages=['Classification', 'Classification.Model', 'Classification.Model.DecisionTree',
                  'Classification.Model.Ensemble', 'Classification.Model.NeuralNetwork',
                  'Classification.Model.NonParametric', 'Classification.Model.Parametric',
                  'Classification.Filter', 'Classification.DataSet', 'Classification.Instance', 'Classification.Attribute',
                  'Classification.Parameter', 'Classification.Experiment',
                  'Classification.Performance', 'Classification.InstanceList', 'Classification.DistanceMetric',
                  'Classification.StatisticalTest', 'Classification.FeatureSelection'],
    ```
    2. Package name should be lowercase and only may include _ character.
    ```
        name='nlptoolkit_math',
    ```
    3. Package data should be defined and must ibclude pyx, pxd, c and py files.
    ```
        package_data={'NGram': ['*.pxd', '*.pyx', '*.c', '*.py']},
    ```
    4. Setup should include ext_modules with compiler directives.
    ```
        ext_modules=cythonize(["NGram/*.pyx"],
                              compiler_directives={'language_level': "3"}),
    ```
    
    ### Cython files
    1. Define the class variables and class methods in the pxd file.
    ```
    cdef class DiscreteDistribution(dict):
    
        cdef float __sum
    
        cpdef addItem(self, str item)
        cpdef removeItem(self, str item)
        cpdef addDistribution(self, DiscreteDistribution distribution)
    ```
    2. For default values in class method declarations, use *.
    ```
        cpdef list constructIdiomLiterals(self, FsmMorphologicalAnalyzer fsm, MorphologicalParse morphologicalParse1,
                                   MetamorphicParse metaParse1, MorphologicalParse morphologicalParse2,
                                   MetamorphicParse metaParse2, MorphologicalParse morphologicalParse3 = *,
                                   MetamorphicParse metaParse3 = *)
    ```
    3. Define the class name as cdef, class methods as cpdef, and \_\_init\_\_ as def.
    ```
    cdef class DiscreteDistribution(dict):
    
        def __init__(self, **kwargs):
            """
            A constructor of DiscreteDistribution class which calls its super class.
            """
            super().__init__(**kwargs)
            self.__sum = 0.0
    
        cpdef addItem(self, str item):
    ```
    4. Do not forget to comment each function.
    ```
        cpdef addItem(self, str item):
            """
            The addItem method takes a String item as an input and if this map contains a mapping for the item it puts the
            item with given value + 1, else it puts item with value of 1.
    
            PARAMETERS
            ----------
            item : string
                String input.
            """
    ```
    5. Function names should follow caml case.
    ```
        cpdef addItem(self, str item):
    ```
    6. Local variables should follow snake case.
    ```
    	det = 1.0
    	copy_of_matrix = copy.deepcopy(self)
    ```
    7. Variable types should be defined for function parameters, class variables.
    ```
        cpdef double getValue(self, int rowNo, int colNo):
    ```
    8. Local variables should be defined with types.
    ```
        cpdef sortDefinitions(self):
            cdef int i, j
            cdef str tmp
    ```
    9. For abstract methods, use ABC package and declare them with @abstractmethod.
    ```
        @abstractmethod
        def train(self, train_set: list[Tensor]):
            pass
    ```
    10. For private methods, use __ as prefix in their names.
    ```
        cpdef list __linearRegressionOnCountsOfCounts(self, list countsOfCounts)
    ```
    11. For private class variables, use __ as prefix in their names.
    ```
    cdef class NGram:
        cdef int __N
        cdef double __lambda1, __lambda2
        cdef bint __interpolated
        cdef set __vocabulary
        cdef list __probability_of_unseen
    ```
    12. Write \_\_repr\_\_ class methods as toString methods
    13. Write getter and setter class methods.
    ```
        cpdef int getN(self)
        cpdef setN(self, int N)
    ```
    14. If there are multiple constructors for a class, define them as constructor1, constructor2, ..., then from the original constructor call these methods.
    ```
    cdef class NGram:
    
        cpdef constructor1(self, int N, list corpus):
        cpdef constructor2(self, str fileName):
        def __init__(self,
                     NorFileName,
                     corpus=None):
            if isinstance(NorFileName, int):
                self.constructor1(NorFileName, corpus)
            else:
                self.constructor2(NorFileName)
    ```
    15. Extend test classes from unittest and use separate unit test methods.
    ```
    class NGramTest(unittest.TestCase):
    
        def test_GetCountSimple(self):
    ```
    16. For undefined types use object as type in the type declarations.
    ```
    cdef class WordNet:
    
        cdef object __syn_set_list
        cdef object __literal_list
    ```
    17. For boolean types use bint as type in the type declarations.
    ```
    	cdef bint is_done
    ```
    18. Enumerated types should be used when necessary as enum classes, and should be declared in py files.
    ```
    class AttributeType(Enum):
        """
        Continuous Attribute
        """
        CONTINUOUS = auto()
        """
    ```
    19. Resource files should be taken from pkg_recources package.
    ```
    	fileName = pkg_resources.resource_filename(__name__, 'data/turkish_wordnet.xml')
    ```