org.grobid.core.lexicon

Class FastMatcher

java.lang.Object

org.grobid.core.lexicon.FastMatcher

public final class FastMatcher
extends java.lang.Object

Class for fast matching of word sequences over text stream.

Constructor Summary

Constructors

Constructor and Description
FastMatcher()
FastMatcher(java.io.File file)
FastMatcher(java.io.File file, Analyzer analyzer)
FastMatcher(java.io.File file, Analyzer analyzer, boolean caseSensitive)
FastMatcher(java.io.InputStream is)
FastMatcher(java.io.InputStream is, Analyzer analyzer)
FastMatcher(java.io.InputStream is, Analyzer analyzer, boolean caseSensitive)

Method Summary

Modifier and Type	Method and Description
int	loadTerm(java.lang.String term, Analyzer analyzer) Load a term to the fast matcher, by default the standard delimiters will be ignored
int	loadTerm(java.lang.String term, Analyzer analyzer, boolean ignoreDelimiters) Load a term to the fast matcher, by default the loading will be case sensitive
int	loadTerm(java.lang.String term, Analyzer analyzer, boolean ignoreDelimiters, boolean caseSensitive) Load a term to the fast matcher
int	loadTerms(java.io.File file) Load a set of terms to the fast matcher from a file listing terms one per line
int	loadTerms(java.io.File file, Analyzer analyzer, boolean caseSensitive) Load a set of terms to the fast matcher from a file listing terms one per line
int	loadTerms(java.io.File file, boolean caseSensitive) Load a set of terms to the fast matcher from a file listing terms one per line
int	loadTerms(java.io.InputStream is, Analyzer analyzer, boolean caseSensitive) Load a set of term to the fast matcher from an input stream
java.util.List<OffsetPosition>	matchCharacter(java.lang.String text) Gives the character positions within a text where matches occur.
java.util.List<OffsetPosition>	matchCharacter(java.lang.String text, boolean caseSensitive) Gives the character positions within a text where matches occur.
java.util.List<OffsetPosition>	matchCharacterLayoutToken(java.util.List<LayoutToken> tokens) Gives the character positions within a tokenized text where matches occur.
java.util.List<OffsetPosition>	matchCharacterLayoutToken(java.util.List<LayoutToken> tokens, boolean caseSensitive) Gives the character positions within a tokenized text where matches occur.
java.util.List<OffsetPosition>	matcherPairs(java.util.List<Pair<java.lang.String,java.lang.String>> tokens) Identify terms in a piece of text and gives corresponding token positions.
java.util.List<OffsetPosition>	matcherPairs(java.util.List<Pair<java.lang.String,java.lang.String>> tokens, boolean caseSensitive) Identify terms in a piece of text and gives corresponding token positions.
java.util.List<OffsetPosition>	matchLayoutToken(java.util.List<LayoutToken> tokens) Identify terms in a piece of text and gives corresponding token positions.
java.util.List<OffsetPosition>	matchLayoutToken(java.util.List<LayoutToken> tokens, boolean ignoreDelimiters, boolean caseSensitive) Identify terms in a piece of text and gives corresponding token positions.
java.util.List<OffsetPosition>	matchToken(java.lang.String text) Identify terms in a piece of text and gives corresponding token positions.
java.util.List<OffsetPosition>	matchToken(java.lang.String text, boolean caseSensitive) Identify terms in a piece of text and gives corresponding token positions.
protected java.lang.String	processToken(java.lang.String token) Process token, if different than @newline

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

FastMatcher

public FastMatcher()

FastMatcher

public FastMatcher(java.io.File file)

FastMatcher

public FastMatcher(java.io.File file,
                   Analyzer analyzer)

FastMatcher

public FastMatcher(java.io.File file,
                   Analyzer analyzer,
                   boolean caseSensitive)

FastMatcher

public FastMatcher(java.io.InputStream is)

FastMatcher

public FastMatcher(java.io.InputStream is,
                   Analyzer analyzer)

FastMatcher

public FastMatcher(java.io.InputStream is,
                   Analyzer analyzer,
                   boolean caseSensitive)

Method Detail

loadTerms

public int loadTerms(java.io.File file)
              throws java.io.IOException

Load a set of terms to the fast matcher from a file listing terms one per line

Throws:

java.io.IOException

loadTerms

public int loadTerms(java.io.File file,
                     boolean caseSensitive)
              throws java.io.IOException

Load a set of terms to the fast matcher from a file listing terms one per line

Throws:

java.io.IOException

loadTerms

public int loadTerms(java.io.File file,
                     Analyzer analyzer,
                     boolean caseSensitive)
              throws java.io.IOException

Load a set of terms to the fast matcher from a file listing terms one per line

Throws:

java.io.IOException

loadTerms

public int loadTerms(java.io.InputStream is,
                     Analyzer analyzer,
                     boolean caseSensitive)
              throws java.io.IOException

Load a set of term to the fast matcher from an input stream

Throws:

java.io.IOException

loadTerm

public int loadTerm(java.lang.String term,
                    Analyzer analyzer)

Load a term to the fast matcher, by default the standard delimiters will be ignored

loadTerm

public int loadTerm(java.lang.String term,
                    Analyzer analyzer,
                    boolean ignoreDelimiters)

Load a term to the fast matcher, by default the loading will be case sensitive

loadTerm

public int loadTerm(java.lang.String term,
                    Analyzer analyzer,
                    boolean ignoreDelimiters,
                    boolean caseSensitive)

Load a term to the fast matcher

matchToken

public java.util.List<OffsetPosition> matchToken(java.lang.String text)

Identify terms in a piece of text and gives corresponding token positions. All the matches are returned.

Parameters:

text: - the text to be processed

Returns:

the list of offset positions of the matches, an empty list if no match have been found

matchToken

public java.util.List<OffsetPosition> matchToken(java.lang.String text,
                                                 boolean caseSensitive)

Identify terms in a piece of text and gives corresponding token positions. All the matches are returned.

Parameters:

text: - the text to be processed

caseSensitive: - ensure case sensitive matching or not

Returns:

the list of offset positions of the matches, an empty list if no match have been found

matchLayoutToken

public java.util.List<OffsetPosition> matchLayoutToken(java.util.List<LayoutToken> tokens)

Identify terms in a piece of text and gives corresponding token positions. All the matches are returned. Here the input is a list of LayoutToken object.

Parameters:

tokens - the text to be processed as a list of LayoutToken objects

Returns:

the list of offset positions of the matches, an empty list if no match have been found

matchLayoutToken

public java.util.List<OffsetPosition> matchLayoutToken(java.util.List<LayoutToken> tokens,
                                                       boolean ignoreDelimiters,
                                                       boolean caseSensitive)

Identify terms in a piece of text and gives corresponding token positions. All the matches are returned. Here the input is a list of LayoutToken object.

Parameters:

tokens - the text to be processed as a list of LayoutToken objects

ignoreDelimiters - if true, ignore the delimiters in the matching process

caseSensitive: - ensure case sensitive matching or not

Returns:

the list of offset positions of the matches, an empty list if no match have been found

matchCharacter

public java.util.List<OffsetPosition> matchCharacter(java.lang.String text)

Gives the character positions within a text where matches occur.

By iterating over the OffsetPosition and applying substring, we get all the matches.

All the matches are returned.

Parameters:

text: - the text to be processed

caseSensitive: - ensure case sensitive matching or not

Returns:

the list of offset positions of the matches referred to the input string, an empty list if no match have been found

matchCharacter

public java.util.List<OffsetPosition> matchCharacter(java.lang.String text,
                                                     boolean caseSensitive)

Gives the character positions within a text where matches occur.

By iterating over the OffsetPosition and applying substring, we get all the matches.

All the matches are returned.

Parameters:

text: - the text to be processed

caseSensitive: - ensure case sensitive matching or not

Returns:

the list of offset positions of the matches referred to the input string, an empty list if no match have been found

matchCharacterLayoutToken

public java.util.List<OffsetPosition> matchCharacterLayoutToken(java.util.List<LayoutToken> tokens)

Gives the character positions within a tokenized text where matches occur.

All the matches are returned.

Parameters:

tokens - the text to be processed as a list of LayoutToken objects

Returns:

the list of offset positions of the matches referred to the input string, an empty list if no match have been found

matchCharacterLayoutToken

public java.util.List<OffsetPosition> matchCharacterLayoutToken(java.util.List<LayoutToken> tokens,
                                                                boolean caseSensitive)

Gives the character positions within a tokenized text where matches occur.

All the matches are returned.

Parameters:

tokens - the text to be processed as a list of LayoutToken objects

caseSensitive - ensure case sensitive matching or not

Returns:

the list of offset positions of the matches referred to the input string, an empty list if no match have been found

matcherPairs

public java.util.List<OffsetPosition> matcherPairs(java.util.List<Pair<java.lang.String,java.lang.String>> tokens)

Identify terms in a piece of text and gives corresponding token positions. All the matches are returned. This case correspond to text from a trainer, where the text is already tokenized with some labeled that can be ignored.

Parameters:

tokens: - the text to be processed

Returns:

the list of offset positions of the matches, an empty list if no match have been found

matcherPairs

public java.util.List<OffsetPosition> matcherPairs(java.util.List<Pair<java.lang.String,java.lang.String>> tokens,
                                                   boolean caseSensitive)

Identify terms in a piece of text and gives corresponding token positions. All the matches are returned. This case correspond to text from a trainer, where the text is already tokenized with some labeled that can be ignored.

Parameters:

tokens: - the text to be processed

caseSensitive: - ensure case sensitive matching or not

Returns:

the list of offset positions of the matches, an empty list if no match have been found

processToken

protected java.lang.String processToken(java.lang.String token)

Process token, if different than @newline