pyaiml21.utils.text_preprocessors

Utilities to preprocess user’s input.

Functions

is_cjk

Return True if given character is CJK character.

normalize_cjk_user_input

Perform CJK normalisation, split to sentences and each to words.

normalize_user_input

Perform latin-alphabet normalisation, split to sentences and each to words.

split_sentences

Split text s to its sentences.