[developers] [incr() tsdb] batch batch processing

[Francis Bond]

G'day,

>> If you point the skeletons to a directory with none, then it will be
>> quicker :-).
>
> But I think we need to point the skeletons to a directory with
> skeletons, because we are using them to create profiles.

If you are pointing itsdb just to the Index.lsp you want, then it will
be as quick as it can be.


> They are different grammars, but they all have the same collateral
> files, since they're all generated by the customization system.
>
> The question is how do we do it?

logon/batch or logon/parse are just creating a lisp file and piping it to lisp.

You can see the file being created by prepending --cat:
---
~/logon/batch --cat --jaen mrs
(setf (system:getenv "DISPLAY") nil)
(pushnew :logon *features*)
(load "/home/bond/logon/lingo/lkb/src/general/loadup.lisp")
(pushnew :lkb *features*)
(compile-system "tsdb")
(tsdb:tsdb :skeleton
             "/home/bond/logon/dfki/jacy/tsdb/skeletons")
(setf tsdb::*tsdb-maximal-number-of-analyses* 5)
(setf tsdb::*tsdb-exhaustive-p* nil)
(tsdb:tsdb :cpu :ja2en :file t :count 1 :error :exit :wait 1200)
(setf target
         (format
          nil "~a~a"
          (tsdb::suggest-test-run-directory "mrs" :absolute nil)
          ""))
(tsdb:tsdb :create target :skeleton "mrs")
(tsdb::tsdb-do-process target :type :translate :overwrite t)
(pvm::pvm_quit)
(sleep 1)
(excl:exit)
---

You can copy this file, and add in extra bits of parsing:

For example, call another cpu and parse again:
(tsdb:tsdb :cpu :ja2en3 :file t :count 1 :error :exit :wait 1200)
(setf target
         (format
          nil "~a~a"
          (tsdb::suggest-test-run-directory "mrs2" :absolute nil)
          ""))
(tsdb:tsdb :create target :skeleton "mrs")
(tsdb::tsdb-do-process target :type :translate :overwrite t)

This could go just before the (pvm::pvm_quit).

You then pass this to lisp:

${LOGONROOT}/bin/logon --binary  yourlispfile -I base -locale
ja_JP.UTF-8 -qq 2>&1 | tee logfile



-- 
Francis Bond <http://www3.ntu.edu.sg/home/fcbond/>
Division of Linguistics and Multilingual Studies
Nanyang Technological University