[developers] A few LOGON questions ...

Tue Aug 4 07:35:35 CEST 2009

hi eric,

:quantum specifies how much time a client processor can spend  
processing one item before [incr tsdn()] will force-terminate it.   
with such a large transfer grammar you're possibly seeing many  
more :quantum time-outs, which may account for the drop in coverage.  
could you inspect the fan-out file to quantify which process(es) time 
(s) out frequently?  however, allowing transfer more time might not  
help much: the process is an exhaustive breadth-first search with very  
limited ambiguity packing.  lots more transfer ambiguity inevitably  
leads to exponential growth in combinatorics :-(.

as regards the fan-out beam, there've been no changes in quite some  
time, and five sounds like the right default setting.  it should work  
to use the --limit option to the ‘batch’ script to change the  
branching factor.  in fact, the following allows setting it per  
component too:

   ./batch --limit 2:3:4:10

the first numbers three control parsing, transfer, and generation.   
the last number determines how many outputs are retained after global  
re-ranking.

good luck with your runs :-), oe

On Aug 4, 2009, at 5:19 AM, Eric Nichols <ericnichols79 at gmail.com>  
wrote:

> Greetings,
>
> I have a few follow-up questions about the current LOGON Barcelona  
> snapshot.
> A little background: I am working on cleaning up automatically  
> acquired MTRs
> using feedback cleaning -- calculating the individual BLEU (or NEVA)  
> score
> contribution of an individual rule and throwing away those with  
> negative
> contributions -- so I am running LOGON with over 100K of translation  
> rules.
>
> As I mentioned in my prior mail, translating with such a large  
> number of rules
> can cause LOGON to time out with its default wait settings. I got  
> LOGON to
> successfully load my large set of MTRs by setting wait to 3000. This  
> appeared
> to get translation working, but our transfer coverage dropped from  
> 45% to 30%
> (seev http://wiki.delph-in.net/moin/MtJaenTanaka). Looking at the  
> fans, I noticed
> that the PVM client seems to be exiting a lot. Is there any other  
> setting that controls
> timeouts. Francis asked about the difference between wait and  
> quantum on the
> list earlier. Is that relevant here?
>
> [11:45:26] (1000020) |あなた を 見 て いる と 私 は あな 
> た の お父さん を 思い出し ます 。| --- 5 (0.05| 
> 0.04:0.05 s) <79
> :1125> {1718:980} (8.6m).
> |
> |-[920.87] # 0 {10.00} --- error: PVM client exit <4000d>.
> |
> |-[1819.40] # 1 {10.00} --- error: PVM client exit <40017>.
> ||-[2726.81] # 2 {9.00} --- error: PVM client exit <4001b>.
> |
> |-[3635.84] # 3 {9.00} --- error: PVM client exit <4001f>.
> |
> |-[4549.08] # 4 {8.00} --- error: PVM client exit <40023>.
> |
> |< |あなた を 見 て いる と 私 は あなた の お父さ 
> ん を 思い出し ます 。| (1000020) --- 5 x 0 x 0 = 0
> |@ |You remind me of your father.|
> |= 3:0 of 3 {100.0+0.0}; 2:0 of 3:0 {66.7 0.0}; 2:0 of 2:0 {100.0  
> 0.0} @ 2 of 3 {66.7} <0.05 0.08|0.09 0.13>.
>
> Also, I noticed that in the fan now, only 5 of the translations are  
> listed at the end of
> a translation block with the source and reference sentence. Is this  
> an intentional
> change or a regression? Does it effect how the NEVA and oracles are  
> computed?
>
> Eric
>
> [21:08:46] (1002140) |彼女 は 彼 に 間違っ た 印象 を 抱 
> い た 。| --- 5 (0.02|0.01:0.02 s) <42:450> {631:480} (5.0m).
> |
> |-[34.78] # 0 {5.00} --- 30 (34.25:12.46|0.00:0.00 s) <:608> (111m  
> 9.4g 104k= 9.5g) [1].
> | |
> | |-[38.83] # 0 {-816.20} --- (3.52|3.52 s) <198:4095> {17613:3586}  
> (48m 4.1g 0= 4.1g) [0].
> | |
> | |-[42.88] # 1 {-816.20} --- (3.73|3.73 s) <198:4095> {17613:3586}  
> (48m 4.1g 0= 4.1g) [0].
> | |
> | |-[48.95] # 2 {-927.81} --- (4.96:1.56|4.96 s) <178:4031>  
> {17469:3538} (46m 3.9g 0= 3.9g) [1].
> | |
> | |-[53.00] # 3 {-927.81} --- (3.41|3.41 s) <178:4031> {17469:3538}  
> (46m 3.9g 0= 3.9g) [0].
> | |
> | |-[55.80] # 4 {-1045.05} --- 4 (2.59|0.00:2.59 s) <192:3043>  
> {12339:2695} (38m 2.8g 0= 2.8g) [0].
> | |   |She bore an impression that was mistaken in him.| {-71.88| 
> 1.01|22.89|0.00|0.00} <0.08>
> | |   |She bore an impression that was mistaken at him.| {-74.21| 
> 0.39|22.89|0.00|0.00} <0.08>
> | |   |She bore an impression, that was mistaken on him.| {-70.18| 
> 0.12|22.89|0.00|0.00} <0.08>
> | |   |She bore an impression that was mistaken on him.| {-73.60| 
> 0.08|22.89|0.00|0.00} <0.08>
> |
> |-[58.18] # 1 {5.00} --- error: unknown transfer predicates:  
> |"ja:rel_p_rel"|.
> |
> |-[58.24] # 2 {5.00} --- error: unknown transfer predicates:  
> |"ja:rel_p_rel"|.
> |
> |-[116.81] # 3 {5.00} --- 30 (57.78:36.29|0.00:0.00 s) <:608> (111m  
> 9.4g 0= 9.5g) [2].
> | |
> | |-[120.86] # 0 {-800.41} --- 3 (2.89|0.00:2.89 s) <198:3537>  
> {15972:3107} (43m 3.3g 0= 3.3g) [0].
> | |   |She bore a general impression that was mistaken in him.|  
> {-77.55|-1.13|33.33|0.00|0.00} <0.10>
> | |   |She bore a general impression that was mistaken on him.|  
> {-79.29|-1.83|33.33|0.00|0.00} <0.10>
> | |   |She bore a general impression that was mistaken at him.|  
> {-79.91|-2.06|33.33|0.00|0.00} <0.10>
> | |
> | |-[127.21] # 1 {-800.41} --- 3 (4.39:1.51|0.00:4.39 s) <198:3537>  
> {15972:3107} (43m 3.3g 0= 3.3g) [1].
> | |   |She bore a general impression that was incorrect in him.|  
> {-78.64|-1.13|33.33|0.00|0.00} <0.10>
> | |   |She bore a general impression that was incorrect on him.|  
> {-80.99|-1.83|33.33|0.00|0.00} <0.10>
> | |   |She bore a general impression that was incorrect at him.|  
> {-79.33|-2.06|33.33|0.00|0.00} <0.10>
> | |
> | |-[132.61] # 2 {-915.38} --- 3 (2.86|0.00:2.86 s) <178:3377>  
> {15828:2963} (41m 3.1g 0= 3.2g) [0].
> | |   |She hugged a general impression that was mistaken in him.|  
> {-79.99|-1.40|33.33|0.00|0.00} <0.10>
> | |   |She hugged a general impression that was mistaken on him.|  
> {-81.73|-2.10|33.33|0.00|0.00} <0.10>
> | |   |She hugged a general impression that was mistaken at him.|  
> {-82.34|-2.33|33.33|0.00|0.00} <0.10>
> | |
> | |-[137.37] # 3 {-915.38} --- 3 (2.87|0.00:2.87 s) <178:3377>  
> {15828:2963} (41m 3.1g 0= 3.2g) [0].
> | |   |She hugged a general impression that was incorrect in him.|  
> {-81.07|-1.40|33.33|0.00|0.00} <0.10>
> | |   |She hugged a general impression that was incorrect on him.|  
> {-83.42|-2.10|33.33|0.00|0.00} <0.10>
> | |   |She hugged a general impression that was incorrect at him.|  
> {-81.77|-2.33|33.33|0.00|0.00} <0.10>
> | |
> | |-[141.44] # 4 {-1014.13} --- 4 (2.33|0.00:2.33 s) <192:2863>  
> {12054:2560} (36m 2.4g 0= 2.5g) [0].
> | |   |She bore an impression that was mistaken in him.| {-71.88| 
> 1.72|22.89|0.00|0.00} <0.08>
> | |   |She bore an impression that was mistaken on him.| {-73.60| 
> 1.02|22.89|0.00|0.00} <0.08>
> | |   |She bore an impression that was mistaken at him.| {-74.21| 
> 0.79|22.89|0.00|0.00} <0.08>
> | |   |She bore an impression which was mistaken on him.|  
> {-70.82|-0.32|22.89|0.00|0.00} <0.08>
> |
> |-[143.53] # 4 {4.00} --- error: unknown transfer predicates:  
> |"ja:_machigau_v_1_rel"|.
> |
> |< |彼女 は 彼 に 間違っ た 印象 を 抱い た 。|  
> (1002140) --- 5 x 60 x 20 = 17 [20]
> |@ |She got a false impression of him.|
> |> |She bore an impression that was mistaken in him.| {7.61e-1}  
> <0.08> (3:4:0).
> |> |She bore an impression, that was mistaken on him.| {7.60e-1}  
> <0.08> (0:4:2).
> |> |She bore an impression which was mistaken on him.| {7.51e-1}  
> <0.08> (3:4:3).
> |> |She bore an impression that was mistaken on him.| {6.79e-1}  
> <0.08> (3:4:1).
> |> |She bore a general impression that was mistaken in him.|  
> {6.51e-1} <0.10> (3:0:0).
> |= 49:0 of 65 {75.4+0.0}; 20:0 of 49:0 {40.8 0.0}; 10:0 of 20:0  
> {50.0 0.0} @ 10 of 65 {15.4} <0.01 0.09|0.03 0.20>.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.delph-in.net/archives/developers/attachments/20090804/ae636549/attachment.html>