<div dir="ltr"><div dir="ltr"><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">Hi Stephan, all,</div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif"><br></div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">Thanks for coming back to the thread.<br></div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif"><br></div></div><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sun, Oct 31, 2021 at 12:47 AM Stephan Oepen <<a href="mailto:oe@ifi.uio.no">oe@ifi.uio.no</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><span class="gmail_default" style="font-family:arial,helvetica,sans-serif">[...</span><span class="gmail_default" style="font-family:arial,helvetica,sans-serif">]</span><br> > Here I'd expect "abb" as the output because the second occurrence of "a" should have different characterization as the input string.<br> <br> why exactly do you think duplicating the masked capture group should be blocked? if i read your rules right, a full masked region is matched by the replacement rule here. so both copies in the output remain ‘intact’, in the sense of preserving the masked content and its characterization, i would think?<br> <br> similarly, i do not immediately see on which grounds we should block swapping around two complete masked regions? i wonder whether your current code actually operationalizes sufficient notions of preserving masked sub-strings?<br clear="all"></blockquote></div><div><br></div><div><div style="font-family:arial,helvetica,sans-serif" class="gmail_default">For both of these cases, it is because I understood masking as working within the framework we have for REPP characterization and not preempting or co-opting the backreferences in rewrite rules. From Footnote 9 in Dridan and Oepen, 2012, in reference to characterization of capture groups:</div><div style="font-family:arial,helvetica,sans-serif" class="gmail_default"><br></div><div style="font-family:arial,helvetica,sans-serif" class="gmail_default"> 9 If capture group references are used out-of-order, however,<br> the per-group linkage is no longer well-defined, and we resort<br> to the maximum-span ‘union’ of boundary points (see below).</div><br></div><div><div style="font-family:arial,helvetica,sans-serif" class="gmail_default">I recall that, during the development of PyDelphin's repp module, I confirmed this statement with tests against the C++ REPP implementation (unfortunately I'm having trouble compiling it at the moment so I'm going from memory). So the following would have fully traceable characterization for all backreferences on the RHS:</div><div style="font-family:arial,helvetica,sans-serif" class="gmail_default"><br></div><div style="font-family:arial,helvetica,sans-serif" class="gmail_default"> !(ab)(cd)(ef) \1 \2 \3</div><div style="font-family:arial,helvetica,sans-serif" class="gmail_default"> !(ab)(cd)(ef) \1 \2</div><div style="font-family:arial,helvetica,sans-serif" class="gmail_default"> !(ab)(cd)(ef) \1</div><div style="font-family:arial,helvetica,sans-serif" class="gmail_default"><br></div><div style="font-family:arial,helvetica,sans-serif" class="gmail_default">But the following would not:</div><div style="font-family:arial,helvetica,sans-serif" class="gmail_default"><br></div><div style="font-family:arial,helvetica,sans-serif" class="gmail_default"> !(ab)(cd)(ef) \1 \3 \2 (out of order)<br></div><div style="font-family:arial,helvetica,sans-serif" class="gmail_default"> !(ab)(cd)(ef) \1 \1 \2 (not strictly increasing)<br></div><div style="font-family:arial,helvetica,sans-serif" class="gmail_default"> !(ab)(cd)(ef) \1 \3 (gap before \3)</div><div style="font-family:arial,helvetica,sans-serif" class="gmail_default"> !(ab)(cd)(ef) \2 \3 (gap before \2)</div><div style="font-family:arial,helvetica,sans-serif" class="gmail_default"><br></div><div style="font-family:arial,helvetica,sans-serif" class="gmail_default">For instance:</div><div style="font-family:arial,helvetica,sans-serif" class="gmail_default"><br></div><div style="font-family:arial,helvetica,sans-serif" class="gmail_default"> $ cat test.rpp <br> !(ab)(cd)(ef) \1 \3 \2<br> $ delphin repp -m test.rpp -f triple <<<"abcdef"<br> (0, 2, ab)<br> (2, 6, ef)<br> (2, 6, cd)<br></div><div style="font-family:arial,helvetica,sans-serif" class="gmail_default"><br></div><div style="font-family:arial,helvetica,sans-serif" class="gmail_default">Note that the CFROM:CTO values for ef and cd are *not* 4:6 and 2:4, respectively. So if we attempt to duplicate a mask or swap two mask positions, then the rewrite rule would not preserve characterization for at least the second backreference and, thus, the rule would be blocked.</div><div style="font-family:arial,helvetica,sans-serif" class="gmail_default"><br></div><div style="font-family:arial,helvetica,sans-serif" class="gmail_default">Side note: since, normally, the question of which backreferences in a rule's RHS are fully traceable is not sensitive to the input of the rewrite rule, my implementation analyzes the RHS at model-load time and not rule-application time. If we declare that masks don't play by the normal rules, then I could no longer perform this optimization as it's unknown a priori whether a rewrite rule will be applied to masked or unmasked content.</div><div style="font-family:arial,helvetica,sans-serif" class="gmail_default"><br></div><div style="font-family:arial,helvetica,sans-serif" class="gmail_default">Mainly for the logical reason above (concerning characterization and rule-blocking) and secondly for the implementational issue and that it's dubious that masks will ever be duplicated or swapped in practice, I'm quite happy to simply state that such operations are blocked. Do you have a strong case for exempting masked regions from the normal characterization behavior?</div><div style="font-family:arial,helvetica,sans-serif" class="gmail_default"><br></div><div style="font-family:arial,helvetica,sans-serif" class="gmail_default">Finally, I think there might be a regex hack that allows for duplicated masked regions, at least. The following captures the same span twice:<br></div><div style="font-family:arial,helvetica,sans-serif" class="gmail_default"><br></div><div style="font-family:arial,helvetica,sans-serif" class="gmail_default"> !((abc)) \1 \2</div><div style="font-family:arial,helvetica,sans-serif" class="gmail_default"><br></div><div style="font-family:arial,helvetica,sans-serif" class="gmail_default">This doesn't work in my current implementation (possibly due to a bug), so for now it's just theoretical.<br></div></div><div><br></div>-- <br><div dir="ltr" class="gmail_signature">-Michael Wayne Goodman</div></div>