[developers] Perl FSR tokenisation tool
bec.dridan at gmail.com
Thu Oct 2 12:02:09 CEST 2008
As I mentioned in Japan, I have Perl code that I use to tokenise
sentences as expected by the grammars (or as defined by the
preprocessor.fsr files) before I feed them to PET. I believe this code
will be obsoleted by the chart-mapping changes, but I've made it
available to anyone who needs that functionality now, before the
chart-mapping version of PET is mainstream release.
As we also discussed in Japan, I've created a page on the wiki
(http://wiki.delph-in.net/moin/ToolsTop) to list tools and scripts of
this sort. If anyone else has scripts they want to share, maybe they
could link them from here. The wiki page should also be linked from
somewhere I guess.
More information about the developers