[developers] token-level ambiguity in the LKB parser

[Stephan Oepen]

hi ann and ben,

there was a bug in SPPP resulting in invalid chart vertices when there
was token-level ambiguity in the input (as used in the korean grammar).
looking into the problem, i noticed that add-morpho-stem-edge() was in
fact called with some of its arguments out-of-order.

anyway, i am writing to

  - warn you that i changed add-token-edge(), so as to return the edge
    just created; looking at current callers, nobody seems to care (but
    it is a change in API, after all).

  - ask you to have a look at the token chart in the attachment to see
    whether you would expect this kind of token-level ambiguity to work
    in the LKB parser: tokens 3 + 5, jointly have the same span as 7; i
    think it should, but some re-assurance would be welcome.

yang, i was unable to get a full parse from your `sppp1' sample, but it
seems quite a few things get built.  could you maybe test whether there
actually should be one or more full parses here (if so how)?  the chart
i get from the new code is attached below (a build is underway).

finally, woodley, in case you read this far: with token-level ambiguity
the LUI chart display will need extending as regards `decoration' with
surface elements at the bottom.  it would be tempting to just send you

  #E[-0 -1 0 -1 "존이" "" []]
  #E[-1 -2 1 -1 "ë°¥" "" []]
  #E[-1 -3 2 -1 "밥이나" "" []]
  #E[-2 -3 3 -1 "이나" "" []]
  #E[-3 -4 4 -1 "빵을" "" []]
  #E[-4 -5 5 -1 "먹었다" "" []]

and have the tokens organize themselves in multiple rows.  do you think
future LUI releases might include support for this?

                                                          best  -  oe

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+++ Universitetet i Oslo (IFI); Boks 1080 Blindern; 0316 Oslo; (+47) 2285 7989
+++     CSLI Stanford; Ventura Hall; Stanford, CA 94305; (+1 650) 723 0515
+++       --- oe at csli.stanford.edu; oe at hf.uio.no; stephan at oepen.net ---
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

-------------- next part --------------
A non-text attachment was scrubbed...
Name: korean.tchart
Type: application/octet-stream
Size: 568 bytes
Desc: not available
URL: <http://lists.delph-in.net/archives/developers/attachments/20060130/58f29c20/attachment.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: korean.chart
Type: application/octet-stream
Size: 2201 bytes
Desc: not available
URL: <http://lists.delph-in.net/archives/developers/attachments/20060130/58f29c20/attachment-0001.obj>