[developers] PET regression testing

[Francis Bond]

G'day,

sorry for the slow reply, I was on holiday.

> Dimensions i can think of are:
>
> - packing / no packing (or even different variants)?
> - input modes? (including character conversion)
> - morphology
> - chart mapping
> - mrs construction
> - partial/full unpacking
>
> It would also be nice to have synthetic grammars to run more unit-like
> tests for, e.g., type hierarchy & unification, FS inheritance &
> unification, etc.
>
> Are there things i have missed?


lkb vs pet  although maybe they have diverged so much that this would
be uninformative.


> I've started with some Java code (because that's something that is
> available free-of-charge on every modern computer) that gives me
> efficient access to tsdb skeletons. What i need now is a set of
> grammars and correlated input items that i can process without
> additional machinery, the more, the better. For the first version, i'd
> suggest to just check the number of passive edges for full parsing
> without any packing and resource restrictions, which means that test items
> have to be of moderate size.

The MRS test suites sound good for this, they are typically only 100+
sentences, are bundled with most grammars, and have good coverage.
For most grammars they are in grammar_name/tsdb/skeletons/mrs; for the
erg in lkb/src/tsdb/skeletons/english/mrs.   I can also point to
online versions from logon svn if that is easier.

To just extract the sentences (as I am sure you know): cut -d@ -f7
path_ending_in_mrs/item

> The rest could then be added step by step (maybe some help out there?).

Of course.

-- 
Francis Bond <http://www3.ntu.edu.sg/home/fcbond/>
Division of Linguistics and Multilingual Studies
Nanyang Technological University