[developers] Error when training generation model

Stephan Oepen oe at ifi.uio.no
Wed Oct 12 18:30:41 CEST 2011


hi dominikus,

this much sounds like you (i.e. the [incr tsdb()] code involved
in training a realization ranking model) triggers a known bug
in Allegro CL, that i recently observed after upgrading to ACL
8.2.  i attach an explanation of the problem to this message
below, and would encourage those (few) among us coding in
Lisp to take a closer look.

in a nutshell, there is stricter type checking now, and a bug
in the loop() macro trips this up.  in case you have available
a license for ACL 8.2, you could work around this by running
LOGON from the sources (and following the instructions in
the attachment to make the compiler accept the 'white lie' in
the loop() initialization).

if you need an updated LOGON run-time binary, i hope to fix
this problem in the near future, but cannot promise i will get
to it before the end of the month.

all best, oe


On Mon, Oct 10, 2011 at 09:54, Dominikus Wetzel <dwetzel at coli.uni-sb.de> wrote:
> Hi all,
>
> I want to train a generation reranking model following the steps described
> here: http://moin.delph-in.net/LogonModeling "Generation model from
> non-treebanked underspecified MRSs". When performing the feature caching
> step (./load --binary fc.g.lisp), the process is terminated with the
> following error message:
>
> Error: about to bind C to NIL, which is not of type CHARACTER.
>  [condition type: TYPE-ERROR]
>
> Thanks very much in advance for any help.
> Best wishes,
> Dominikus
>
>
>
> This is some relevant context output:
>
> [...]
>
> TSNLP(3): ; Loading /home/dwetzel/opt/logon/lingo/redwoods/fc.g.lisp
> ;   Loading /home/dwetzel/opt/logon/lingo/redwoods/generation.lisp
> install-gc-strategy(): disabling tenure; global garbage collection ...
> [15:37:05] gc-after-hook(): {L#37 N=784 O=46M E=74%} [S=2.3G R=694M].
> 78120432 bytes have been tenured, next gc will be global.
> See the documentation for variable EXCL:*GLOBAL-GC-BEHAVIOR* for more
> information.
> [15:37:06] gc-after-hook(): {G#37 N=1008 O=0 E=0%} [S=2.3G R=694M].
> [15:37:06] gc-after-hook(): {L#38 N=4.4K O=0 E=0%} [S=2.3G R=694M].
> [15:37:06] gc-after-hook(): {L#39 N=4.4K O=0 E=0%} [S=2.3G R=694M].
>  done.
> [15:37:06] operate-on-profiles(): reading `erg/1010/rtc006/11-10-10/lkb'
> [15:37:06] operate-on-profiles(): running `erg/1010/rtc006/11-10-10/lkb'
> [30009000 - 30009200|.
> [15:37:30] open-fc(): new BDB `fc.bdb'.
> Error: about to bind C to NIL, which is not of type CHARACTER.
>  [condition type: TYPE-ERROR]
>
> Restart actions (select using :continue):
>  0: store the value anyway.
>  1: retry the load of fc.g.lisp
>  2: skip loading fc.g.lisp
>  3: recompile /home/dwetzel/opt/logon/lingo/redwoods/fc.g.lisp
>  4: Return to Top Level (an "abort" restart).
>  5: Abort entirely from this (lisp) process.
> [1] TSNLP(4): :pop
> TSNLP(4): EOF
> Really exit lisp [n]?
> close-connection(): `erg/1010/rtc006/11-10-10/lkb' expiry.
-------------- next part --------------
Date: Fri, 16 Sep 2011 10:43:52 -0700
From: Duane Rettig <duane at franz.com>
To: oe at ifi.uio.no
Subject: Re: [spr38517] weird (to me) compiler behavior after migrating to ACL 8.2
Cc: support at franz.com

Stephan,

>> i just noticed that after transitioning to 8.2 a piece of our code breaks
>> when using somewhat peculiar compiler settings.  this used to work
>> okay in earlier ACL versions, and i cannot say immediately whether
>> our code is to blame or the ACL 8.2 compiler.
>> 
>> i attach the relevant code below.  invoking the compiled function, i
>> get a type error
>> 
>>   CL-USER(9): (xml-escape-string "foo")
>>   Error: about to bind C to NIL, which is not of type CHARACTER.
>>     [condition type: TYPE-ERROR]
>> 
>> could someone please comment on this problem?
>> 
>> we have been using (speed 1) (safety 3) (space 1) (debug 3) for our
>> run-time images for many years.  i cannot say i have thought very
>> hard about these choices.  just now, for example, i am unsure of
>> (debug 3) in this context, as we do not expect actual debugging (of
>> Lisp code) to happen in this images.  but safety, on the other hand,
>> does seem relevant.  do the above seem particularly weird to you?

Well, yes, in some ways, but it is a known problem we are working on
for 9.0, and we have put out some patches for 8.2 for extreme cases.

The history is this:  In 8.2 we introduced a new compiler-switch:
comp:verify-type-declarations-switch, which provides explicit type
checks at various places where declared variables are bound or
setq'd in the code.  This switch returns true when safety is 2 or 3.
This switch, in combinaion with the other optimization compiler
switches, allows you to set your speed and safety to relatively high
levels, and to get the benefit of both strong type chechs and
efficient code.  So if speed is 3 and safety is 2, then the compiler
trusts declarations, but also verifies them at binding or setq
points. so you get the best of both worlds.

We had known already that this switch would force some programming
changes on users; those who used this switch (i.e. declared
optimization levels 2 or 3 for safety) would need to be vigilant
against the kind of programming where a "white lie" was told to the
compiler:

 (let ((x nil))
   (declare (fixnum x))

    ...

    (setq x (length y))

     ...

Such code is seen often, but is caught with this new switch.


What we hadn't counted on was that many loop constructs use this white
lie to perform pre-initializations, especially in situations where
there are explicit or implicit type declarations on the loop
variables.  The example you have shown is one of these.  I will look
into fixing the loop macro so that it doesn't initialize the variable
to a value of the rong type.


Meanwhile, you can work around the problem generally by setting the
safety level down to 1,  or specifically by wrapping the defun of
xml-escape-string with the following

(compiler-let ((comp::verify-type-declarations-switch nil))
 ...
)

The latter requires explicit source code changes wherever you get this
problem, but from your description I suspect you want the higher type
checking when it works correctly, so it might be a better tradeoff for
you.

If you don't like either of the two above solutions, you can always
simulate the pre-8.2 behaviors with a more drastic move; in some
initialization code, evaluate this:

(setq comp::verify-type-declarations-switch nil)

which will forever turn off the compilation of these extra compiler
checks.

Let me know if you have further questions about this.

Duane


More information about the developers mailing list