Neural Lattice Search for Domain Adaptation in Machine Translation

Transcription

1 Neural Lattice Search for Domain Adaptation in Machine Translation Huda Khayrallah, Gaurav Kumar Kevin Duh, Matt Post, Philipp Koehn This talk was presented at IJCNLP 2017 It is based on this paper: bib:

2 Neural Lattice Search for Domain Adaptation in Machine Translation Huda Khayrallah, Gaurav Kumar Kevin Duh, Matt Post, Philipp Koehn

3 combine adequacy of PBMT with fluency of NMT 3

4 use PBMT to constrain the search space of NMT 4

5 Source die brötchen sind warm PBMT the Lattice bread is buns are warm 5

6 the Source die brötchen sind warm Lattice bread is buns are warm Neural Lattice Search Target the buns are warm 6

7 die brötchen sind warm the buns are warm bread is the buns are warm 7

8 die brötchen sind warm bread is the buns are warm 8

9 die brötchen sind warm bread is the buns are warm

10 die brötchen sind warm bread is the buns are warm the

11 die brötchen sind warm bread is the buns are warm the

12 die brötchen sind warm bread is the buns are warm the

13 die brötchen sind warm bread is the buns are warm the buns

14 die brötchen sind warm bread is the buns are warm the buns

15 die brötchen sind warm bread is the buns are warm the buns

16 die brötchen sind warm bread is the buns are warm the buns

17 die brötchen sind warm bread is the buns are warm the buns warm

18 die brötchen sind warm bread is the buns are warm warm warm the buns warm warm warm warm warm

19 Experiments 19

20 Setting: Domain adaptation Small in-domain IT, Medical, Koran, Subtitles PBMT outperforms NMT Large out-of-domain parliamentary proceedings (WMT) NMT outperforms PBMT 20

21 Setting: Domain adaptation NMT in-domain out-of-domain PBMT in-domain out-of-domain 21

22 IT Results BLEU PBMT (in) NMT (out) n-best Lattice Search 22

23 Results BLEU IT Koran Medical Subtitle PBMT (in) NMT (out) n-best lattice search 23

24 Conclusion Lattice search > n-best rescoring Use in-domain PBMT to constrain search space NMT can be in- or out-of-domain Code: github.com/khayrallah/nematus-lattice-search 24

25 Thanks! This material is based upon work supported in part by the Defense Advanced Research Projects Agency (DARPA) under Contract No. HR C Any opinions, findings and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the Defense Advanced Research Projects Agency (DARPA). 25

26 Neural Lattice Search for Domain Adaptation in Machine Translation Huda Khayrallah, Gaurav Kumar Kevin Duh, Matt Post, Philipp Koehn {huda, gkumar, kevinduh, post, code: github.com/khayrallah/nematus-lattice-search

27 27

28 28

29 29

30 Corpus Sizes Corpus Words Sentences W/S Medical 14,301,472 1,104, IT 3,041, ,817 9 Koran 9,848, , Subtitles 114,371,754 13,873,398 8 EuroParl 113,165,079 4,562,

31 How much text do we have? BLEU IT Koran Medical Phrase-Based with Big LM Phrase-Based Neural Subtitles & WMT Corpus size (English words) [Koehn & Knowles 2017] 31

32 Corpus Sizes words Medical IT Koran Subtitles EuroParl words 32

33 IT Baselines BLEU PBMT (in) PBMT (out) NMT (in) NMT (out) 33

34 34

35 Source Versionsinformationen ausgeben und beenden Reference output version information and exit PBMT Spend version information and end NMT Spend and end versionary information lattice Print version information and exit 35

36 Results BLEU IT Medical Koran Subtitle PBMT (in) NMT (in) N-best Hybrid Lattice 36

37 Stack Based Decoding Stacks based on number of target words translated Keep track of: Score Current lattice node Current neural state incoming arc length 37