Project Lucy has retired. For details please refer to its Attic page.
Lucy::Analysis::Analyzer – C API Documentation
Apache Lucy™

Lucy::Analysis::Analyzer

parcel Lucy
class variable LUCY_ANALYZER
struct symbol lucy_Analyzer
class nickname lucy_Analyzer
header file Lucy/Analysis/Analyzer.h

Name

Lucy::Analysis::Analyzer – Tokenize/modify/filter text.

Description

An Analyzer is a filter which processes text, transforming it from one form into another. For instance, an analyzer might break up a long text into smaller pieces (RegexTokenizer), or it might perform case folding to facilitate case-insensitive search (Normalizer).

Functions

init
lucy_Analyzer*
lucy_Analyzer_init(
    lucy_Analyzer *self
);

Abstract initializer.

Methods

Transform (abstract)
lucy_Inversion* // incremented
lucy_Analyzer_Transform(
    lucy_Analyzer *self,
    lucy_Inversion *inversion
);

Take a single Inversion as input and returns an Inversion, either the same one (presumably transformed in some way), or a new one.

inversion

An inversion.

Transform_Text
lucy_Inversion* // incremented
lucy_Analyzer_Transform_Text(
    lucy_Analyzer *self,
    cfish_String *text
);

Kick off an analysis chain, creating an Inversion from string input. The default implementation simply creates an initial Inversion with a single Token, then calls Transform(), but occasionally subclasses will provide an optimized implementation which minimizes string copies.

text

A string.

Split
cfish_Vector* // incremented
lucy_Analyzer_Split(
    lucy_Analyzer *self,
    cfish_String *text
);

Analyze text and return an array of token texts.

text

A string.

Dump
cfish_Obj* // incremented
lucy_Analyzer_Dump(
    lucy_Analyzer *self
);

Dump the analyzer as hash.

Subclasses should call Dump() on the superclass. The returned object is a hash which should be populated with parameters of the analyzer.

Returns: A hash containing a description of the analyzer.

Load
cfish_Obj* // incremented
lucy_Analyzer_Load(
    lucy_Analyzer *self,
    cfish_Obj *dump
);

Reconstruct an analyzer from a dump.

Subclasses should first call Load() on the superclass. The returned object is an analyzer which should be reconstructed by setting the dumped parameters from the hash contained in dump.

Note that the invocant analyzer is unused.

dump

A hash.

Returns: An analyzer.

Inheritance

Lucy::Analysis::Analyzer is a Clownfish::Obj.