[developers] [incr() tsdb] batch batch processing

[Francis Bond]

G'day,

On 30 November 2010 07:13, Michael Wayne Goodman
<goodmami at u.washington.edu> wrote:
> Hi folks,
>
> I'm trying to increase the speed of multiple calls to [incr tsdb()]'s
> batch processing, and could use some help.
>
> Background: Part of Matrix development involves running a series of
> regression tests where we customize grammars and compare (using [incr
> tsdb()]) their parsing results to a gold standard. We have well over a
> hundred test grammars, and running the whole set takes some time.
>
> While we're switching to parsing with PET, we found that most of the
> time is spent loading LOGON's default Index.lsp. How can we prevent
> [incr tsdb()] from loading this file on startup? I've had success
> moving Index.lsp so it cannot be found, but this is not a very good
> solution.

    *       To set the database home:

(tsdb:tsdb :home "/home/oe/src/itsdb/src/tsdb/home")

    *       To set the location of skeletons

(tsdb:tsdb :skeletons "/home/oe/src/lkb/src/tsdb/skeletons/english")


If you point the skeletons to a directory with none, then it will be
quicker :-).

You can put these in your .tsdbrc.  See:
http://wiki.delph-in.net/moin/ItsdbCustomization

> Also, do you have any tips for running batches of batch parsing jobs?
> Specifically, can we load and parse all grammars within a single
> running lisp instance? Currently we are sending lisp commands to the
> LOGON scripts for each grammar separately, meaning all the time spent
> starting up [incr tsdb()] happens for each grammar.
>
> Thanks,

If the MRS globals and other settings are compatible, then you can do
this, but our experience was it can be the source of extremely hard to
track down weirdness, so it is not recommended.  For different
versions of the same grammar though, it should be OK.


-- 
Francis Bond <http://www3.ntu.edu.sg/home/fcbond/>
Division of Linguistics and Multilingual Studies
Nanyang Technological University