<div dir="ltr"><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">Oops, I accidentally replied off-list to Stephan and Alexandre. I'll summarize my responses (no longer inline to the main thread; sorry!)<br></div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif"><br></div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">Alexandre wrote:</div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">> Is this the generic python split method from strings? Is it safe? What
about possible non whitespace exceptions on the tokenisation?</div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif"><br></div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">It is. This is just an ad hoc fix for the Xigt codebase to keep Emily's group moving along with their research. We're hoping to make a more robust version eventually.</div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif"><br></div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">Stephan wrote:</div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">> <span class="gmail-im">this results in a flag-like behavior for the processing
commands: skipping over (what you might call ‘noise’) items which lack
linguistic content.</span></div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif"><span class="gmail-im"><br></span></div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif"><span class="gmail-im">I thought that's what i-wf = 2 is for? <a href="http://moin.delph-in.net/ItsdbReference#Well_Formedness_.28i-wf.29" target="_blank">http://moin.delph-in.net/ItsdbReference#Well_Formedness_.28i-wf.29</a></span></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, Apr 12, 2019 at 8:09 AM Stephan Oepen <<a href="mailto:oe@ifi.uio.no">oe@ifi.uio.no</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><div dir="auto">my apologies for the opaque inside joke, alexandre! some of us used to work at a start-up (called YY Technologies) a few years ago, developing the premier email auto response solution at the time. hence support for mbox files was relevant at least back then :-). and the ERG treebanks still include four profiles with e-commerce emails.</div></div><div dir="auto"><br></div><div dir="auto">regarding interpretation of the i-length field, i would maybe argue that the two use cases ultimately are the same meaning: the field quantifies the length of linguistic content; when there is nothing worse parsing in an item, its ‘linguistic length’ is zero (or maybe -1, not quite sure just now). but, yes, this results in a flag-like behavior for the processing commands: skipping over (what you might call ‘noise’) items which lack linguistic content.</div><div dir="auto"><br></div><div dir="auto">best, oe</div><div dir="auto"><br></div><div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, 11 Apr 2019 at 19:31 Alexandre Rademaker <<a href="mailto:arademaker@gmail.com" target="_blank">arademaker@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hi Stephan,<br>
<br>
> On 11 Apr 2019, at 13:52, Stephan Oepen <<a href="mailto:oe@ifi.uio.no" target="_blank">oe@ifi.uio.no</a>> wrote:<br>
> <br>
> hi emily and mike,<br>
> <br>
> the [incr tsdb()] import facilities support some mixed-content document formats, notably the un*x mbox format (you can guess when that was useful functionality).<br>
<br>
Sorry, I don’t! Do you mean that mbox files can be imported to profiles directly? So far, I always thought that a profile itens are all sentences or phrases subject to be analysed by a grammar.<br>
<br>
> to represent all data in the profile while not pretending that there is linguistic content (worth sending to the parser) in email headers, the corresponding items are marked as i-length = -1 (or maybe 0, not quite sure). this is the reason for the ‘Process | ...’ commands to require a non-zero, positive length ... in other words a reassurance that there actually is linguistic content in the item. in this regard, i-length (like i-id, i-input, and possibly i-wf as well) is a mandatory field in the item relation.<br>
<br>
So i-length has two meanings, it is at the same time the length of the input (in tokens) but also a flag. The -1 has special meaning, right? <br>
<br>
That is, what you are saying is that a profile can also accommodate noise data and we can explicit use the i-length to mark what itens are relevant for processing. Is that right?<br>
<br>
Best,<br>
Alexandre<br>
<br>
<br>
</blockquote></div></div>
</blockquote></div><br clear="all"><br>-- <br><div dir="ltr" class="gmail_signature">-Michael Wayne Goodman</div>