aito.schema.AitoTokenNgramAnalyzerSchema

class aito.schema.AitoTokenNgramAnalyzerSchema(source: aito.schema.AitoAnalyzerSchema, min_gram: int, max_gram: int, token_separator: str = None)

Aito TokenNGramAnalyzer schema

Parameters

  • source (AitoAnalyzerSchema) – the source analyzer to generate features before being combined into n-grams

  • min_gram (int) – the minimum length of characters in a feature

  • max_gram (int) – the maximum length of characters in a feature

  • token_separator (str, defaults to ' ') – the string used to join the features of the source analyzer

Methods

from_deserialized_object(obj)

create an AitoSchema object from a JSON deserialized object

from_json_string(json_string, **kwargs)

create an AitoSchema object from a JSON string

infer_from_samples(samples[, max_sample_size])

Infer an analyzer from the given samples

to_json_serializable()

convert the AitoSchema object to a json serializable object (dict, in most case)

to_json_string(**kwargs)

the AitoSchema object as a JSON string

Attributes

analyzer_type

comparison_properties

iterable of the properties that will be used for comparison with another schema object of the same type

token_separator

the string to join the features

type

the type of the schema component