[developers] [delph-in] questions about PET vs. LKB

[Berthold Crysmann]

Dear Sanghoun, 

it seems that one of your setups uses VPM (variable property mapping),
while the other does not. Look for files/configuration option mentioning
vpm. If it is missing on one side, you might want to consult some
grammar that uses it systematically to discover what the right calls
are, e.g ERG, GG (logon version) etc.  

Cheers, 

B

On Thu, 2012-06-21 at 12:10 -0700, Sanghoun Song wrote:
> Dear folks,
> 
> Developing the matrix system, we've tried to speed up the regression
> tests that verify each change works correctly within the current
> system implemented so far. (More info here:
> http://moin.delph-in.net/MatrixRegressionTesting)
> 
> What we've done for speeding up includes:
> (1) using PET (cheap) to create each profile with the tsdbdump option
> (2) kicking off tsdb++ only once and comparing the whole results to
> gold at the same time (i.e., tsdb++ is loaded only for comparison
> between [current] and [gold]).
> 
> The procedure itself basically works, and also the regression tests
> work much faster. However, we have run into difficulty regarding the
> MRS format output by PET.  The gold profiles were made with the LKB
> (and we will continue to use the LKB for this purpose, as well as
> interactive exploration of the regression tests, so we can't simply
> switch everything to PET).
> 
> PET
> [ LTOP: handle1 INDEX: event2 RELS: < [ "_n1_n_rel"<0:0> LBL: handle3
> ARG0: ref-ind4 ] [ "exist_q_rel"<0:0> LBL: handle5 ARG0: ref-ind4
> RSTR: handle6 BODY: handle7 ] [ "_iv_v_rel"<0:0> LBL: handle8 ARG0:
> event2 ARG1: ref-ind4 ] > HCONS: < handle6 qeq handle3 > ]
> 
> LKB
> [ LTOP: h1 INDEX: e2 [ e SF: PROP-OR-QUES E.TENSE: TENSE E.ASPECT:
> ASPECT E.MOOD: MOOD ] RELS: < [ "_n1_n_rel" LBL: h3 ARG0: x4 [ x
> SPECI: BOOL COG-ST: COG-ST PNG: PNG ] ] [ "exist_q_rel" LBL: h5 ARG0:
> x4 RSTR: h6 BODY: h7 ] [ "_iv_v_rel" LBL: h8 ARG0: e2 ARG1: x4 ] >
> HCONS: < h6 qeq h3 > ]
> 
> There are three different things between these two.
> (1) handle1 vs. h1, etc.
> (2) LKB uses more specific structure for events and individuals like
> [ e SF: PROP-OR-QUES E.TENSE: TENSE E.ASPECT: ASPECT E.MOOD: MOOD ]
> (3) PET specifies some pointers like <0:0>, but we confirmed that it
> didn't matter in comparison on tsdb++.
> 
> Is there some set of flags we can give to cheap to make the PET output
> match the LKB's?  (This seems relevant to the discussion on "Standard format
> specification for MRS" at the up-coming Summit.)
> 
> A second issue that came up as we tried to migrate to PET for our regression
> tests concerns flop.  Specifically, certain characters (such as %) in instance
> identifiers will cause flop to produce an empty .grm file without giving any
> indication that there was an error.  (These same files work fine with the LKB.)
> 
> In this case, we can change our naming scheme to avoid the problematic
> characters, but we wanted to report the issue anyway, as it seems like flop
> should at least indicate that there was a problem, and which instances were
> problematic.  We've submitted a ticket on the PET trac, but wanted to mention
> it here as well, in case that trac is not currently monitored.
> 
> 
> Thanks,
> Sanghoun (for the Matrix developers group)
> 




-- 
Berthold Crysmann 
CNRS - Laboratoire de Linguistique Formelle (UMR 7110) - Paris Diderot
Adresse postale: Case 7031, 5 rue Thomas Mann, F-75205 Paris Cedex 13
Adresse géographique: 175, rue du Chevaleret, F-75013 Paris