[developers] ACL 7.0 issues (memory and encodings)

[Francis Bond]

G'day,

Stephan reports:
> apparently running PET as an [incr tsdb()] client using Allegro CL 7.0
> is, put mildly, unreliable.  dan reports:
>
>> Thanks for setting up the PET machinery on the shiny new machines.  I made
>> a few attempts to use ld for batch processing on the rondane data, but
>> unfortunately the Lisp process eventually just goes into a terminal tail
>> spin, silently spinning at full tilt for about three minutes and then
>> dying equally silently.  I get about a third of the way through that
>> data set, and it doesn't seem to matter whether I use one, two, or four
>> CPUs (though I'm sure erik cared when I tried four) - same silent death.
>
> i can reproduce the problem and believe that it is related to:
>
>   http://www.franz.com/support/documentation/7.0/doc/implementation.htm#arrays-1
>
> the foreign function wrapper to PVM manipulates static arrays at a very
> low level.  specifially, it is using .array-total-size-limit. to check
> for buffer limits.  i suspect what could be happening is exhaustion of
> the (ACL) C heap, i.e. where with older ACL versions (and smaller array
> limits) you would have seen the [incr tsdb()] error "PVM message buffer
> overflow", you can now require up to 32 times more static space on the
> Lisp side.  quite likely, our standard image layout (moving the C heap
> way up to the top to allow maximum Lisp heap growth) is not up for such
> demands. 

We had similar problems and solved them by using ACL 7.0 with the
default image: 
 (setq fi:common-lisp-image-name "/raid3/project/delphin/acl70/clim")
 (setq fi:common-lisp-image-file "/raid3/project/delphin/acl70/clim.dxl")

This has been working fine with cheap, as well as updating and
normalizing. Well, it runs out of memory updating every now and then
updating, but ACL 6.2 did that too.


On a semi-related note, we had some troubles with the latest eli and ACL
7.0  and getting the encodings we wanted.  eli insists on 
;; Setting (stream-external-format *terminal-io*) to :emacs-mule.
and we want it to be :euc.

Our solution is to 
  (setf excl:*default-external-format*  
 (setf (stream-external-format *terminal-io*) :euc))
which only seems to work from inside the *common-lisp* buffer.

We thus call lisp as follows:

(defun acl70 (&optional prefix)
  (setq lkb-tmp-dir "/tmp")
  (interactive "P")
  (load "/project/delphin/acl70/eli/fi-site-init")
  (setq fi:common-lisp-image-name "/project/delphin/acl70/clim")
  (setq fi:common-lisp-image-file "/project/delphin/acl70/clim.dxl")
  (setq fi:common-lisp-directory "/project/delphin/acl70")
  (setq fi:common-lisp-image-arguments 
    (list 
     "-locale" "japan.EUC"
     "-qq"
     "-L" "/project/delphin/cl-init.cl"))
  (fi:common-lisp)
  (japanify "*common-lisp*" 'euc-jp)
    (insert "(setf excl:*default-external-format*
               (setf (stream-external-format *terminal-io*) :euc))")
  )

and then hit return!  Ugly but workable.

If anyone has a more elegant solution then please let us know.

-- 
Francis Bond  <www.kecl.ntt.co.jp/icl/mtg/members/bond/>
NTT Communication Science Laboratories | Machine Translation Research Group