<html><head><meta http-equiv="Content-Type" content="text/html charset=iso-8859-1"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><div><br><div>Begin forwarded message:</div><br class="Apple-interchange-newline"><blockquote type="cite"><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px;"><span style="font-family:'Helvetica'; font-size:medium; color:rgba(0, 0, 0, 1.0);"><b>From: </b></span><span style="font-family:'Helvetica'; font-size:medium;">Yi Zhang <<a href="mailto:yizhang@dfki.de">yizhang@dfki.de</a>><br></span></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px;"><span style="font-family:'Helvetica'; font-size:medium; color:rgba(0, 0, 0, 1.0);"><b>Subject: </b></span><span style="font-family:'Helvetica'; font-size:medium;"><b>Re: [developers] a couple of DeepBank questions</b><br></span></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px;"><span style="font-family:'Helvetica'; font-size:medium; color:rgba(0, 0, 0, 1.0);"><b>Date: </b></span><span style="font-family:'Helvetica'; font-size:medium;">February 1, 2013 10:09:04 AM GMT+01:00<br></span></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px;"><span style="font-family:'Helvetica'; font-size:medium; color:rgba(0, 0, 0, 1.0);"><b>To: </b></span><span style="font-family:'Helvetica'; font-size:medium;">Megan Schneider <<a href="mailto:caelum@gmail.com">caelum@gmail.com</a>><br></span></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px;"><span style="font-family:'Helvetica'; font-size:medium; color:rgba(0, 0, 0, 1.0);"><b>Cc: </b></span><span style="font-family:'Helvetica'; font-size:medium;"><a href="mailto:developers@delph-in.net">developers@delph-in.net</a><br></span></div><br><meta http-equiv="Content-Type" content="text/html charset=iso-8859-1"><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><div>hi Megan,</div><div><br></div><blockquote type="cite">1) How do the DeepBank sentence identifiers map to the Penn Treebank? (<span style="background-color:rgb(255,255,255);color:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px">20020005</span><span style="background-color:rgb(255,255,255);color:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px"> appears to map to the 5th sentence in RAW/</span><span style="background-color:rgb(255,255,255);color:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px">parsed/mrg/wsj/00/wsj_0020.mrg and </span><span style="background-color:rgb(255,255,255);color:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px">20001001 appears to map to the first sentence in 00/wsj_0001.mrg from looking at where the sentences in question exist)</span><div>
<font color="#222222" face="arial, sans-serif"><br></font></div></blockquote><div>yes, your observation is right. the sentence identifiers in DeepBank are 8-digit integers, always starting (from left) with "2", followed by 4 digits corresponding to the file name in the PTB (e.g. 0234 is from the file 02/wsj_0234.mrg), and ends with 3 digits corresponding to the sentence number in that file (starting from 1). </div><div><br></div><br><blockquote type="cite"><div><font color="#222222" face="arial, sans-serif">2) Does anyone have a version of the Penn Treebank which is limited to only those parses/sentences also contained in DeepBank?</font></div>
<div><font color="#222222" face="arial, sans-serif"><br></font></div><div><font color="#222222" face="arial, sans-serif"><br></font></div></blockquote><div>you can find a simple perl script from the following link, which will select and print the subpart of PTB (in original .mrg format) according to a list of sentence ids. </div><div><a href="http://www.coli.uni-saarland.de/~yzhang/files/select-ptb-with-iid.pl">http://www.coli.uni-saarland.de/~yzhang/files/select-ptb-with-iid.pl</a></div><div><br></div><div>a simple way of getting the id list is the following command line (assuming you are doing it on the DeepBank release 0.9, which contains thinned tsdb profiles):</div><div> </div><div> $ for i in deepbank-0.9/tsdb/*.1; do zcat $i/result.gz | cut -f 1 -d@ >> id-list.txt; done</div><div><br></div><div>afterwards, run the perl script:</div><div> $ perl select-ptb-with-iid.pl id-list.txt penntreebank3/parsed/mrg/wsj/ > ptb-deepbank-0.9.mrg</div><div><br></div><div>best,</div><div>yi</div></div></blockquote></div><br></body></html>