[itsdb] bitext import

[Francis Bond]

G'day,

bitext import fails if there are more than 9 reference translations.
The IWSLT competition goes for 16....

I attach a small text file that shows the problem --- if you do bitext
import (either as part of a batch, or through the itsdb menu) then it
gets very confused.


-- 
Francis Bond <http://www2.nict.go.jp/x/x161/en/member/bond/>
NICT Computational Linguistics Group
-------------- next part --------------
???????????? ?????? ?????? ??? 
I have a throbbing pain.
I am experiencing a throbbing pain.
I am suffering from a throbbing pain.
I am feeling a throbbing pain.
It is a throbbing pain.
It's throbbing and it really hurts.
It's painful and it's throbbing.
It's throbbing with pain.
It's in throbbing pain.
It hurts so much it's throbbing.
I've got a throbbing pain.
I can feel a throbbing pain.
I am suffering from a throbbing pain.
I am experiencing a throbbing pain.
I have a painful throbbing.
I feel a painful throbbing.

?????? ??? ??? ??? ?????? ??? 
It is August fifteenth.
August fifteenth.
The fifteenth of August.
The date is the fifteenth of August.
It's the fifteenth of August.
It is the fifteenth day of August.
That would be August fifteenth.
August fifteenth.
On August fifteenth.
That's August fifteenth.
August the fifteenth.
It's the fifteenth of August.
The fifteenth of August.
On the fifteenth of August.
On August the fifteenth.
August fifteenth.

?????? ??? ?????? ??? ?????? ?????? ??? ??? 
Could you send this to Japan?
Could you ship this to Japan?
Could you send it to Japan?
Please send this to Japan for me.
Can you have this sent to Japan for me?
Would you be able to send this to Japan for me?
Can I have this sent to Japan?
Would it be possible to ship it to Japan?
Can this be sent to Japan?
Could I have this mailed to Japan?
Forward this to Japan, please.
Would you mind sending this to Japan for me?
Could I have this sent to Japan?
Could you ship this to Japan?
Would you mind having this shipped to Japan?
Could I get this sent to Japan?