[developers] Question about RMRS cfrom/cto in PET

[Ben Waldron]

Berthold Crysmann wrote:
> Hi Yi,
>
> try experimenting with the -tok parameter: the yy_counts and 
> xml_counts parameter should give you character spans.
> You need to provide the input in yy or Pic format, though.
If you are providing a character string as input (rather than 
preprocessed YY, PIC or SMAF input) try the -tok=fsr parameter. This 
will activate the FSPP tokeniser (which is used by the LKB) and 
behaviour should be identical to that you get when running the LKB.

- Ben
> Standard string input appears to give token positions.... Maybe we 
> could have this added as an option?