[developers] LKB-FOS now includes [incr tsdb()]

Mon Jun 1 12:04:01 CEST 2020

Hi Francis,

Thanks for the report and suggestions about LD_LIBARY_PATH. I put a hint about what to do in the README, but it's buried in dense text so not easy to find ("LD_LIBARY_PATH must include <path-to-lkb_fos>/lib/linux.x86.64"). It's a good ides to have a couple of sentences about this in LkbFos.

Here's what LKB-FOS does on startup:

* it finds the absolute path to the lkb_fos directory, and from that, it constructs a path to where it thinks bdb.so should be (e.g. on my system this is /home/ubuntu/Documents/delphin/lkb_fos/src/tsdb/linux.x86.64/bdb.so)

* it attempts to load bdb.so from this path as a shared library

* if there's an error in this load, it's either because the user has moved bdb.so, or libdb-4.2.so (which bdb.so depends on) can't be found

* libdb-4.2.so will be pulled in if LD_LIBARY_PATH points to lkb_fos/lib/linux.x86.64

Later in a session, if you start up the [incr tsdb()] podium it tries to load libtermcap.so.2; on my system this is in a standard Linux shared library directory so is picked up fine. But evidently this isn't the case for everyone. So to avoid installation fuss, I was just going to provide it in lkb_fos/lib/linux.x86.64 - the file is very small. LOGON also takes this approach. I'll do this in the next LKB-FOS release.

If DELPHINHOME is set as recommended, then I think the following should be sufficient:

export LD_LIBRARY_PATH=$DELPHINHOME/lkb_fos/lib/linux.x86.64:$LD_LIBRARY_PATH

And for the moment, some users will also need to execute:

ln -s <some system library directory>/libncurses.so.5.7 .../lkb_fos/lib/linux.x86.64/libtermcap.so.2

Does this look reasonable? If so I'll update LkbFos.

John

On 1 Jun 2020, at 05:35, Francis Bond <bond at ieee.org<mailto:bond at ieee.org>> wrote:

Hi,

to get it working on ubuntu 18.04.4, I had to make some libraries visible:
bdb.so, libdb-4.2.so<http://libdb-4.2.so>, libtermcap.so.2

I did this with:
export LD_LIBRARY_PATH=/home/bond/delphin/lkb_fos.2020/src/tsdb/linux.x86.64:/home/bond/delphin/lkb_fos.2020/lib/linux.x86.64

I also had to link libtermcap.so.2 -> /lib/x86_64-linux-gnu/libncurses.so.5.9

Maybe we could have a symbolic link from src/tsdb/linux.x86.64/bdb.so to lib/linux.x86.64/bdb.so
so that we only have to point to a single directory?

Should I make an installation section in LkbFos and add the notes there?

On Fri, May 22, 2020 at 5:39 AM John Carroll <J.A.Carroll at sussex.ac.uk<mailto:J.A.Carroll at sussex.ac.uk>> wrote:
Hi all,

I've just released a new version of LKB-FOS. The main change is that the Linux version includes all of the non-LOGON parts of [incr tsdb()]. The podium runs, and I believe that all of its menu commands are working correctly. I've created a foreign function interface in SBCL for the BDB C program, so training maxent models also works. Anything that's at all CPU-intensive runs a lot quicker than in the LOGON run-time binary.

For macOS, I haven't made a serious attempt at recompiling the core [incr tsdb()] C programs (tsdb, swish++), so there's not much of it that works - the main useful exception being reading and applying maxent models (e.g. as described at the end of http://moin.delph-in.net/LkbGeneration<http://moin.delph-in.net/LkbGeneration>).

No LOGON-specific functionality is available (i.e. source code enabled by the :logon feature), which means that PVM, WWW demo, SVMs and language models, external MT system interfaces etc are missing. If anyone particularly wants one of these features in LKB-FOS, it should be possible now there's a solid foundation to start from.

BTW, below is a relevant posting to the developers list by Stephan in 2006. The previous posting in that thread was over-optimistic: a number of issues (which I won't bore this list with) made the port to SBCL harder than one might have expected. Anyway, I'm pleased to have made progress on this issue 14 years on!

All the best,

John

PS The new LKB-FOS contains many other improvements - please see the README. Download link at http://moin.delph-in.net/LkbFos<http://moin.delph-in.net/LkbFos>

> http://lists.delph-in.net/archives/developers/2006/000632.html<http://lists.delph-in.net/archives/developers/2006/000632.html>
>
> [developers] SBCL port
> Stephan Oepen oe at csli.Stanford.EDU<http://csli.Stanford.EDU>
> Mon Oct 30 11:23:05 CET 2006
>
> howdy,
>
> > But I expect a port would not be too difficult to achieve for either
> > of these systems. Stephan, what do you think?
>
> [incr tsdb()] makes fairly central use of foreign functions, which are
> non-standard.  also, the [incr tsdb()] GUI depends on threads, which in
> SBCL are just barely available (in a way different from the traditional
> MP package), and only for Linux on x86 and AMD64 currently.  i have no
> current plans to port [incr tsdb()] to other Lisps, and personally i am
> not too keen on getting other developers involved in that right now.  i
> would want to review patches to [incr tsdb()] code so as to make sure i
> can maintain its overall design.  these days i am afraid i have no time
> for such activity.
>
> the LOGON MT architecture is an extension to [incr tsdb()], i.e it has
> inherited the same constraints on cross-platform portability.  however,
> we are about to release a complete run-time edition of LOGON, such that
> people will be able to get full functionality without their own license
> for Allegro CL.
>
> more high-level, SBCL does look like a Lisp going the right direction.
> but before it makes sense for us to make the coordinated effort towards
> supporting the breadth of DELPH-IN software on a new Lisp, we should be
> sure of our minimum requirements.  the following come to my mind:
>
>   (1) stable, efficient, actively maintained ANSI CL implementation
>   (2) UniCode strings, including full external format support
>   (3) cross-platform availability
>   (4) multi-processing, preferably with Lisp control of scheduler
>   (5) foreign function interface
>   (6) high-level OS interface: run-shell-command(), sockets, et al.
>
> SBCL appears to have all of the above but (4).  i know CMU-CL used to
> include the traditional MP package, but i have no idea about the other
> desiderata there.
>
>                                                           best  -  oe
>
> +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> +++ Universitetet i Oslo (IFI); Boks 1080 Blindern; 0316 Oslo; (+47) 2284 0125
> +++     CSLI Stanford; Ventura Hall; Stanford, CA 94305; (+1 650) 723 0515
> +++       --- oe at csli.stanford.edu<http://csli.stanford.edu>; oe at ifi.uio.no<http://ifi.uio.no>; stephan at oepen.net<http://oepen.net> ---
> +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

--
Francis Bond <http://www3.ntu.edu.sg/home/fcbond/<http://www3.ntu.edu.sg/home/fcbond/>>
Division of Linguistics and Multilingual Studies
Nanyang Technological University

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.delph-in.net/archives/developers/attachments/20200601/0070579e/attachment.html>