Lucy::Analysis::Analyzer - Tokenize/modify/filter text.
# Abstract base class.
An Analyzer is a filter which processes text, transforming it from one form into another. For instance, an analyzer might break up a long text into smaller pieces (RegexTokenizer), or it might perform case folding to facilitate case-insensitive search (Normalizer).
package MyAnalyzer; use base qw( Lucy::Analysis::Analyzer ); our %foo; sub new { my $self = shift->SUPER::new; my %args = @_; $foo{$$self} = $args{foo}; return $self; }
Abstract constructor. Takes no arguments.
my $inversion = $analyzer->transform($inversion);
Take a single Inversion as input and returns an Inversion, either the same one (presumably transformed in some way), or a new one.
my $inversion = $analyzer->transform_text($text);
Kick off an analysis chain, creating an Inversion from string input. The default implementation simply creates an initial Inversion with a single Token, then calls transform(), but occasionally subclasses will provide an optimized implementation which minimizes string copies.
my $arrayref = $analyzer->split($text);
Analyze text and return an array of token texts.
my $obj = $analyzer->dump();
Dump the analyzer as hash.
Subclasses should call dump() on the superclass. The returned object is a hash which should be populated with parameters of the analyzer.
Returns: A hash containing a description of the analyzer.
my $obj = $analyzer->load($dump);
Reconstruct an analyzer from a dump.
Subclasses should first call load() on the superclass.
The returned object is an analyzer which should be reconstructed by setting the dumped parameters from the hash contained in dump
.
Note that the invocant analyzer is unused.
Returns: An analyzer.
Lucy::Analysis::Analyzer isa Clownfish::Obj.
Copyright © 2010-2015 The Apache Software Foundation, Licensed under the
Apache License, Version 2.0.
Apache Lucy, Lucy, Apache, the Apache feather logo, and the Apache Lucy project logo are trademarks of The
Apache Software Foundation. All other marks mentioned may be trademarks or registered trademarks of their
respective owners.