Tokenize Paragraph

[peddinti]

Hi it appears i can only tokenize a sentence but not a paragraph. I have a paragraph and would like to tokenize it, how can i do that? i am also not able to get the sentences in a paragraph to be able to successively tokenize the sentences

[louismullie]

You would need to segment the paragraph first, and then you can tokenize the sentences.

[louismullie]

See the following section from the manual: https://github.com/louismullie/treat/wiki/Manual#segmenters