[developers] Web interfaces

Sun May 30 08:36:14 UTC 2021

Hey folks,

I'm not active in the DELPH-IN world these days, but happy to offer what
context I can regarding delphin-viz and offer some of the thoughts I have
previously had around how DELPH-IN visualisation could be made more
accessible.

As Michael's covered, delphin-viz is made up of a number of SVG-based
visualisations, and all consume the JSON responses produced by the ERG API
<http://moin.delph-in.net/wiki/ErgApi>.

   - *mrs.js*: AVM style rendering of MRS analyses based on the svg.js
   <https://svgdotjs.github.io/>. I created this in order to match the
   visualization capabilities of LOGON, rather than from any particular
   use-case motivated reason. Makes sense that people's intuitions around its
   utility have moved on.
   - *dmrs.js: *A super slick DMRS visualisation that Michael developed
   using D3.js <https://d3js.org/>
   - *tree.js: *Derivation tree visualisations that I adapted from code
   originally written by Woodley (in the Full Forest Treebanker?). This
   doesn't use any SVG library, just vanilla JavaScript. The main limitation
   that I was hoping to improve for this visualisation is that it only
   supports showing the verbose identifiers of nodes (eg sb-hd_mc_c),
   rather than the more accessible labels, such as S, VP, NP. From memory,
   this limitation was from the labels not being available within the ERG API
   (or maybe just existing implementations?), so this feature would need to be
   added there first.

The demo <https://delph-in.github.io/delphin-viz/demo> included in that
repo is a pretty simple web app built on JQuery that queries the chosen
grammar/webserver from those it knows about with a string to parse, and
then renders the different analyses returned with the visualisations
selected. I previously had an ERG API server up and running with my own
grammars using Michael's PyDelphin web server
<https://pydelphin.readthedocs.io/en/latest/api/delphin.web.server.html>,
which I agree is the easiest way to spin one up yourself.

One notable limitation of the demo (and the underlying visualisation) is
that there's no integration between them. It would be nice to hover over an
identifier in one visualisation and have the corresponding objects become
highlighted in other visible visualisations for that analysis.

My vision was that the demo was more of a proof of concept for how those
individual javascript libraries could be integrated into a web page by
people wanting to create DELPH-IN tools/resources, while potentially also
being a useful tool itself. It seems like its value to date has been more
from the latter of these two use-cases, which makes sense as it really
needs a lot more work to make it a reusable and accessible library for
downstream users.

Here's the thoughts I had assembled around a possible pathway towards the
vision of a set of DELPH-IN visualisation components that could be readily
consumed by web apps for research tools or communication purposes.

   - continue with the client-server based approach of consuming JSON from
   the ERG API. With the existing documented API and options for spinning up a
   server already, this just makes a lot of sense to me.
   - Pick a single SVG library that the visualisations are all built on to
   make it easier for people to contribute (D3.js seems like an obvious
   candidate)
   - Lean further into the idea of modular JavaScript visualisations, one
   for each viz type, by creating components for each which have their own
   documented interfaces.
   - Try to support Python users to consume these components. I don't think
   it's realistic to require the likely users of the library to go and learn
   HTML + CSS + JavaScript to build out a simple interface if they don't
   already have familiarity. And even if they do, the friction of spinning up
   another dev environment and context shifting into that world would be a
   disincentive for their use.

One strategy that I'm partial to for meeting these goals would be for
visualisation developers to create a React component for each visualisation
type. Once a sensible interface has been defined for these components, they
can then be readily converted into Plotly Dash <https://plotly.com/dash/>
Components. Dash is a library that essentially provides Python folks with
the ability to create interactive data-oriented web apps in Python by
defining page layouts and callback functions all in Python. Once the
delphin-viz React components have been created and converted
<https://github.com/plotly/dash-component-boilerplate> into Python classes
and wrapped up into Python package, consumers could simply `pip
install delphin-dash` use the PyDelphin client to query an existing server,
and then drop the viz components they want into the desired location on the
page, wiring up the server response to the components' inputs. This way,
the need to know JavaScript/React is restricted to those contributing to
the components themselves, but they could be used by people with a bit of
Python familiarity (and perhaps a bit of CSS knowledge to customise the
layout). One catch is that D3.js and React need a bit of management for
them to co-exist, however the Dash folks have some strategies
<https://dash.plotly.com/d3-react-components> for this.

Unfortunately I don't have bandwidth for driving any of this work, however
if there was any interest in exploring the Dash strategy I mentioned, I'd
be more than happy to discuss further with folks, and perhaps help out here
and there.

Cheers,
Ned

On Mon, 10 May 2021 at 12:46, goodman.m.w at gmail.com <goodman.m.w at gmail.com>
wrote:

> Regarding web APIs: If your needs are responding to dynamic parsing
> requests or serving the contents of stored parses, PyDelphin already
> provides this functionality. For serving stored parses from [incr tsdb()]
> profiles, it only transforms token lists (p-input or p-tokens), MRSs, and
> derivations into JSON representations, and the remaining fields are
> returned as text, but this is configurable if you have a suitable
> transformation function (in Python, alas). It does not have a scheme for
> identifying substructures, as Stephan described. I'm happy to discuss API
> design in case you have other needs.
>
> Regarding visualizations, while delphin-viz uses the AVM-like
> visualization of MRSs, I agree with Stephan that it is not ideal. In
> general, however, I'd prefer a node-and-edge graphical representation of
> MRS over the compact textual one. The DOT-based layouts are good when you
> want to focus on the semantic graph by itself. I like arc diagrams when
> comparing the semantics to the original sentence as it keeps the word order
> intact: I attach a screenshot of the MRS adaption of the DMRS arc diagrams
> I used in my dissertation (basically the same as those for DMRS, but with
> variables). Unfortunately I only have LaTeX macros for these. D3.js has
> "force-directed" graph layouts (those springy, animated ones you've
> probably seen around the web). You can also use D3's SVG primitives to draw
> nodes and edges, but then you're on your own for the layout (this is
> essentially how I do the delphin-viz one).
>
> On Sun, May 9, 2021 at 4:39 PM Stephan Oepen <oe at ifi.uio.no> wrote:
>
>> hi alexandre,
>>
>> i developed the visualizations in the LOGON on-line demonstrator (e.g.
>> http://erg.delph-in.net) in 2002, hence they are pure dynamic HTML
>> before SVG became generally available.  the HTML generation is part of the
>> lisp code of the LKB and [incr tsdb()].
>>
>> i am not sure there is much to learn here, in part because i personally
>> consider some of the visualization choices out of date: for MRS, i have
>> come to consider the AVM-like rendering misleading and today would advocate
>> a more compact format, as we use in the ERG Semantic Documentation (e.g.
>> http://moin.delph-in.net/wiki/ErgSemantics/Essence).  for EDS, i would
>> nowadays favor a graphical rendering over the textual one, e.g. via the DOT
>> layout engine, as implemented in mtool (e.g.
>> http://mrp.nlpl.eu/2020/index.php?page=12).
>>
>> only for DM, i am still fairly happy with the visualization used on the
>> LOGON demonsrator for the ERG.  this is rendered client-side, using an
>> extension of the Brat JavaScript library to support the SDP 2015 tabular
>> format.  however, for DM we currently lack a conversion path outside [incr
>> tsdb()], and even what is there has hardly been maintained since angelina
>> ivanova developed the original converter in 2014. or so.
>>
>> visualization in the WSI web application, in turn, was developed by milen
>> kouylekov, in part as a mimicry of the HTML renderings from LOGON.  in some
>> cases, i believe, the rendering is done in the client (in JavaScript), e.g.
>> for the SDP graphs (including DM) and probably EDS.  for MRS, i believe
>> milen may actually pre-compute the HTML on the server, in which case this
>> would be part of the WSI Java code base.  again, i am not sure it will pay
>> off to try and re-use much of these parts.
>>
>> i generally buy into the delphin-viz philosophy: communicate various
>> types of representations in JSON and do the rendering in JavaScript on the
>> client, with liberal use of SVG.  i believe further development in this
>> paradigm has the highest potential for reuse.  however, it may limit layout
>> sophistication: for various types of semantic graphs, i have grown very
>> fond of the DOT language and GraphViz rendering engines.  last i checked
>> there were multiple initiatives to support that framework for in-browser
>> rendering, but none appeared to be reasonably mature yet.
>>
>> finally, in the WSI context you will ultimately want more than just the
>> visualization of an object: in my use of the service, i have found it
>> essential to highlight the parts of the structure matched by a given query,
>> e.g. MRS variables, predicates, roles, etc.  i am pretty sure milen applies
>> the color-highlighting in the client, e.g. when toggling between multiple
>> possible matches, based on (sets of) a list of sub-structure identifiers
>> that match constituents of the query.  to do this, the query processor
>> (server) and renderer (client) need a common scheme of referring to
>> sub-structures.
>>
>> best wishes, oe
>>
>>
>> On Fri, 7 May 2021 at 03:46 Alexandre Rademaker <arademaker at gmail.com>
>> wrote:
>>
>>> Hi all,
>>>
>>> I want to understand the alternatives we have for presenting semantic
>>> and syntactic structures on a web page. This is part of our current effort
>>> to rewrite the WSI interface. The goal is to have something similar to the
>>> last version of WSI that I had access and I have it running with sample
>>> data at https://wsi.mybluemix.net/demo/.
>>>
>>> This instance has MRS and EDS. I could not import DMRS, but I remember
>>> that I saw somewhere another instance of WSI exposing DMRS. Does anyone
>>> know where?
>>>
>>> Anyway, we already have a new WQL to SPARQL transformation in place, and
>>> now we are moving to the web interface part. The WSI code is quite
>>> complicated using a lot of javascript. I could not understand how the
>>> EDS/MRSs are transmitted from the Jean to the browser.
>>>
>>> But even if I ignore that and design a new strategy, I would like to
>>> reuse the alternatives available to present MRS, DMRS, and EDS in the
>>> browser. Besides the WSI interface, we have:
>>>
>>> 1. http://chimpanzee.ling.washington.edu/demophin/erg/parse displays
>>> only DMRS. Who is the author of the dmrs.js? The DMRSs are transformed into
>>> SVG…
>>>
>>> 2. http://delph-in.github.io/delphin-viz/demo/. It shows derivation
>>> trees, MRS and DRMSs. Uses https://github.com/delph-in/delphin-viz, it
>>> is the state-of-the-art solution for displaying DELPH-IN structures? I
>>> didn't find documentation; how should my backend provide the data to allow
>>> this code to load/parse and display them on a page?
>>>
>>> 3. http://erg.delph-in.net/logon. It shows trees (not the same as
>>> above), dm, MRS, EDS. The DM is not the DMRS… The tree, EDS, and MRS are
>>> HTML tables produced on the server-side by some code; the DM is transformed
>>> in SVG, but how? The backend is Lisp. I would love to understand all the
>>> loading steps of the logon scripts! I suppose it is using LKB with the ERG
>>> 1214, but the www bash script is almost impossible to read... anyway, for
>>> WSI, I don't need to call a parser but to retrieve the structures from an
>>> RDF triplestore and display them.
>>>
>>> Does anyone have any direction? It would help me a lot!
>>>
>>> Best,
>>> Alexandre
>>>
>>>
>>>
>
> --
> -Michael Wayne Goodman
>

-- 
nedned.net
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.delph-in.net/archives/developers/attachments/20210530/b1d5bdb3/attachment-0001.htm>