Thank you!<div><br></div><div>Megan<br><br><div class="gmail_quote">On Fri, Feb 1, 2013 at 1:10 AM, Yi Zhang <span dir="ltr"><<a href="mailto:yzhang@coli.uni-sb.de" target="_blank">yzhang@coli.uni-sb.de</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word"><div><br><div>Begin forwarded message:</div><br><blockquote type="cite"><div style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px">
<span style="font-family:'Helvetica';font-size:medium;color:rgba(0,0,0,1.0)"><b>From: </b></span><span style="font-family:'Helvetica';font-size:medium">Yi Zhang <<a href="mailto:yizhang@dfki.de" target="_blank">yizhang@dfki.de</a>><br>
</span></div><div style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px"><span style="font-family:'Helvetica';font-size:medium;color:rgba(0,0,0,1.0)"><b>Subject: </b></span><span style="font-family:'Helvetica';font-size:medium"><b>Re: [developers] a couple of DeepBank questions</b><br>
</span></div><div style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px"><span style="font-family:'Helvetica';font-size:medium;color:rgba(0,0,0,1.0)"><b>Date: </b></span><span style="font-family:'Helvetica';font-size:medium">February 1, 2013 10:09:04 AM GMT+01:00<br>
</span></div><div style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px"><span style="font-family:'Helvetica';font-size:medium;color:rgba(0,0,0,1.0)"><b>To: </b></span><span style="font-family:'Helvetica';font-size:medium">Megan Schneider <<a href="mailto:caelum@gmail.com" target="_blank">caelum@gmail.com</a>><br>
</span></div><div style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px"><span style="font-family:'Helvetica';font-size:medium;color:rgba(0,0,0,1.0)"><b>Cc: </b></span><span style="font-family:'Helvetica';font-size:medium"><a href="mailto:developers@delph-in.net" target="_blank">developers@delph-in.net</a><br>
</span></div><div><div class="h5"><br><div style="word-wrap:break-word"><div>hi Megan,</div><div><br></div><blockquote type="cite">1) How do the DeepBank sentence identifiers map to the Penn Treebank? (<span style="color:rgb(34,34,34);font-size:13px;font-family:arial,sans-serif">20020005</span><span style="color:rgb(34,34,34);font-size:13px;font-family:arial,sans-serif"> appears to map to the 5th sentence in RAW/</span><span style="color:rgb(34,34,34);font-size:13px;font-family:arial,sans-serif">parsed/mrg/wsj/00/wsj_0020.mrg and </span><span style="color:rgb(34,34,34);font-size:13px;font-family:arial,sans-serif">20001001 appears to map to the first sentence in 00/wsj_0001.mrg from looking at where the sentences in question exist)</span><div>
<font color="#222222" face="arial, sans-serif"><br></font></div></blockquote><div>yes, your observation is right. the sentence identifiers in DeepBank are 8-digit integers, always starting (from left) with "2", followed by 4 digits corresponding to the file name in the PTB (e.g. 0234 is from the file 02/wsj_0234.mrg), and ends with 3 digits corresponding to the sentence number in that file (starting from 1). </div>
<div><br></div><br><blockquote type="cite"><div><font color="#222222" face="arial, sans-serif">2) Does anyone have a version of the Penn Treebank which is limited to only those parses/sentences also contained in DeepBank?</font></div>
<div><font color="#222222" face="arial, sans-serif"><br></font></div><div><font color="#222222" face="arial, sans-serif"><br></font></div></blockquote><div>you can find a simple perl script from the following link, which will select and print the subpart of PTB (in original .mrg format) according to a list of sentence ids. </div>
<div><a href="http://www.coli.uni-saarland.de/~yzhang/files/select-ptb-with-iid.pl" target="_blank">http://www.coli.uni-saarland.de/~yzhang/files/select-ptb-with-iid.pl</a></div><div><br></div><div>a simple way of getting the id list is the following command line (assuming you are doing it on the DeepBank release 0.9, which contains thinned tsdb profiles):</div>
<div> </div><div> $ for i in deepbank-0.9/tsdb/*.1; do zcat $i/result.gz | cut -f 1 -d@ >> id-list.txt; done</div><div><br></div><div>afterwards, run the perl script:</div><div> $ perl <a href="http://select-ptb-with-iid.pl" target="_blank">select-ptb-with-iid.pl</a> id-list.txt penntreebank3/parsed/mrg/wsj/ > ptb-deepbank-0.9.mrg</div>
<div><br></div><div>best,</div><div>yi</div></div></div></div></blockquote></div><br></div></blockquote></div><br></div>