[developers] Re: pending/367: (itsdb) certain test suite items kill Lisp

[Stephan Oepen]

hi again (copying `developers' because of potential broader interest),

> I've been having trouble processing the full CSLI test suite when using 
> the default *maximum-number-of-edges* (100000). Certain test suite items 
> (fairly regularly, but not always) cause Lisp to die horribly, as copied 
> below. Decreasing *maximum-number-of-edges* to 10000 reliably fixes the 
> problem. Problem occurs on all (3) machines tried so far, with both 
> Allegro 6.2 and Allegro 7.0.

> LKB(11): (let ((*maximum-number-of-edges* 100000))
>            (do-parse-tty "Chiang bet Devito a dollar that Browne
                            interviewed Abrams and a dollar that Abrams
                            would be hired."))

  [...]

> gc-after-hook(): local; new: 145369608; old: 0; pending: 48; efficiency: 37.
>  
> system error (gsgc): Object already pointing to target newspace half: 
> The internal data structures in the running Lisp image have been
> corrupted and execution cannot continue.  Check all foreign functions
> and any Lisp code that was compiled with high speed and/or low safety,
> as these are two common sources of this failure.  If you cannot find
> anything incorrect in your code you should contact technical support
> for Allegro Common Lisp, and we will try to help determine whether
> this is a coding error or an internal bug.
> Would you like to dump core for debugging before exiting(y or n)? y

generally, i would say that a maximum of 100,000 edges is way to many
when using the ERG and the LKB parser(s); in generation, we usually go
with a maximum of 25,000 these days, and that might be a good value in
parsing too.  my [incr tsdb()] default of 100,000, apparently, assumes 
that a majority of users use PET in batch parsing.  you could use your
`~/.tsdbrc' to customize for your needs, e.g.

  (setf *tsdb-maximal-number-of-edges* 25000)

the gsgc() problem you experience, unfortunately, is long-standing in
Allegro CL and has to do with heap placement and the system failing to
detect that newspace growth can collapse with the C heap location.  it
is possible to allow the Lisp more room to grow by trying to maximize
the distance between the Lisp and C heap base addresses, though at the
cost of leaving less room for foreign objects (and requesting address
ranges that may be unavailable in some environments, notably EM64T in
32-bit compatibility mode).  some time ago, i checked in my build file
for custom Allegro images as `.../etc/bclim.lisp' in the LKB tree, and
you could try using that (it will allow up to some 1.8 gbyte Lisp heap
size).

          apologies, i wish something could be done about this  -  oe

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+++ Universitetet i Oslo (ILN); Boks 1102 Blindern; 0317 Oslo; (+47) 2285 7989
+++     CSLI Stanford; Ventura Hall; Stanford, CA 94305; (+1 650) 723 0515
+++       --- oe at csli.stanford.edu; oe at hf.uio.no; stephan at oepen.net ---
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++