org.grobid.core.engines

Class Segmentation

java.lang.Object

org.grobid.core.engines.AbstractParser

org.grobid.core.engines.Segmentation

All Implemented Interfaces:

java.io.Closeable, java.lang.AutoCloseable, GenericTagger

public class Segmentation
extends AbstractParser

Realise a high level segmentation of a document into cover page, document header, page footer, page header, document body, bibliographical section, each bibliographical references in the biblio section and finally the possible annexes.

Field Summary

Fields inherited from class org.grobid.core.engines.AbstractParser

analyzer, cntManager

Constructor Summary

Constructors

Constructor and Description
Segmentation() TODO some documentation...

Method Summary

Modifier and Type	Method and Description
void	close()
void	createBlankTrainingData(java.io.File file, java.lang.String pathFullText, java.lang.String pathTEI, int id) Get the content of the pdf and produce a blank training data TEI file, i.e.
void	createTrainingSegmentation(java.lang.String inputFile, java.lang.String pathFullText, java.lang.String pathTEI, int id) Process the content of the specified pdf and format the result as training data.
java.lang.String	getAllLinesFeatured(Document doc) Addition of the features at line level for the complete document.
Document	prepareDocument(Document doc)
Document	processing(DocumentSource documentSource, GrobidAnalysisConfig config) Segment a PDF document into high level zones: cover page, document header, page footer, page header, body, page numbers, biblio section and annexes.
Document	processing(java.lang.String text)
java.lang.StringBuffer	trainingExtraction(java.lang.String result, java.util.List<LayoutToken> tokenizations, Document doc) Extract results from a labelled full text in the training format without any string modification.

Methods inherited from class org.grobid.core.engines.AbstractParser

label, label

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Segmentation

public Segmentation()

TODO some documentation...

Method Detail

processing

public Document processing(DocumentSource documentSource,
                           GrobidAnalysisConfig config)

Segment a PDF document into high level zones: cover page, document header, page footer, page header, body, page numbers, biblio section and annexes.

Parameters:

documentSource - document source

Returns:

Document object with segmentation information

processing

public Document processing(java.lang.String text)

prepareDocument

public Document prepareDocument(Document doc)

getAllLinesFeatured

public java.lang.String getAllLinesFeatured(Document doc)

Addition of the features at line level for the complete document.

This is an alternative to the token level, where the unit for labeling is the line - so allowing faster processing and involving less features. Lexical features becomes line prefix and suffix, the feature text unit is the first 10 characters of the line without space. The dictionary flags are at line level (i.e. the line contains a name mention, a place mention, a year, etc.) Regarding layout features: font, size and style are the one associated to the first token of the line.

createTrainingSegmentation

public void createTrainingSegmentation(java.lang.String inputFile,
                                       java.lang.String pathFullText,
                                       java.lang.String pathTEI,
                                       int id)

Process the content of the specified pdf and format the result as training data.

Parameters:

inputFile - input file

pathFullText - path to fulltext

pathTEI - path to TEI

id - id

createBlankTrainingData

public void createBlankTrainingData(java.io.File file,
                                    java.lang.String pathFullText,
                                    java.lang.String pathTEI,
                                    int id)

Get the content of the pdf and produce a blank training data TEI file, i.e. a text only TEI file without any tags. This is usefull to start from scratch the creation of training data at the same level as the segmentation parser.

Parameters:

inputFile - input file

pathFullText - path to fulltext

pathTEI - path to TEI

id - id

trainingExtraction

public java.lang.StringBuffer trainingExtraction(java.lang.String result,
                                                 java.util.List<LayoutToken> tokenizations,
                                                 Document doc)

Extract results from a labelled full text in the training format without any string modification.

Parameters:

result - reult

tokenizations - toks

Returns:

extraction

close

public void close()
           throws java.io.IOException

Specified by:

close in interface java.io.Closeable

Specified by:

close in interface java.lang.AutoCloseable

Overrides:

close in class AbstractParser

Throws:

java.io.IOException