Unsupervised Russian POS tagging with appropriate context

Li Yang; Erik Peterson; John Chen; Yana Petrova; Rohini Srihari

doi:10.1007/978-3-642-23538-2_54

Unsupervised Russian POS tagging with appropriate context

Li Yang
, Erik Peterson
, John Chen
, Yana Petrova
, Rohini Srihari

Department of Computer Science and Engineering

Janya Inc.
SUNY Buffalo

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

While adopting the contextualized hidden Markov model (CHMM) framework for unsupervised Russian POS tagging, we investigate the possibility of utilizing the left, right, and unambiguous context in the CHMM framework. We propose a backoff smoothing method that incorporates all three types of context into the transition probability estimation during the expectation-maximization process. The resulting model with this new method achieves overall and disambiguation accuracies comparable to a CHMM using the classic backoff smoothing method for HMM-based POS tagging from [17].

Original language	English
Title of host publication	Text, Speech and Dialogue - 14th International Conference, TSD 2011, Proceedings
Pages	427-433
Number of pages	7
DOIs	https://doi.org/10.1007/978-3-642-23538-2_54
State	Published - 2011
Event	14th International Conference on Text, Speech and Dialogue, TSD 2011 - Pilsen, Czech Republic Duration: Sep 1 2011 → Sep 5 2011

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	6836 LNAI
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	14th International Conference on Text, Speech and Dialogue, TSD 2011
Country/Territory	Czech Republic
City	Pilsen
Period	09/1/11 → 09/5/11

Keywords

CHMM
expectation-maximization (EM)
left
right
transition probability
unambiguous context
unsupervised Russian part-of-speech tagging

Access to Document

10.1007/978-3-642-23538-2_54

Cite this

Yang, L., Peterson, E., Chen, J., Petrova, Y., & Srihari, R. (2011). Unsupervised Russian POS tagging with appropriate context. In Text, Speech and Dialogue - 14th International Conference, TSD 2011, Proceedings (pp. 427-433). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 6836 LNAI). https://doi.org/10.1007/978-3-642-23538-2_54

@inproceedings{f914ed267bd24b0380e4ead223fe8033,

title = "Unsupervised Russian POS tagging with appropriate context",

abstract = "While adopting the contextualized hidden Markov model (CHMM) framework for unsupervised Russian POS tagging, we investigate the possibility of utilizing the left, right, and unambiguous context in the CHMM framework. We propose a backoff smoothing method that incorporates all three types of context into the transition probability estimation during the expectation-maximization process. The resulting model with this new method achieves overall and disambiguation accuracies comparable to a CHMM using the classic backoff smoothing method for HMM-based POS tagging from [17].",

keywords = "CHMM, expectation-maximization (EM), left, right, transition probability, unambiguous context, unsupervised Russian part-of-speech tagging",

author = "Li Yang and Erik Peterson and John Chen and Yana Petrova and Rohini Srihari",

year = "2011",

doi = "10.1007/978-3-642-23538-2\_54",

language = "English",

isbn = "9783642235375",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

pages = "427--433",

booktitle = "Text, Speech and Dialogue - 14th International Conference, TSD 2011, Proceedings",

note = "14th International Conference on Text, Speech and Dialogue, TSD 2011 ; Conference date: 01-09-2011 Through 05-09-2011",

}

Yang, L, Peterson, E, Chen, J, Petrova, Y & Srihari, R 2011, Unsupervised Russian POS tagging with appropriate context. in Text, Speech and Dialogue - 14th International Conference, TSD 2011, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 6836 LNAI, pp. 427-433, 14th International Conference on Text, Speech and Dialogue, TSD 2011, Pilsen, Czech Republic, 09/1/11. https://doi.org/10.1007/978-3-642-23538-2_54

Unsupervised Russian POS tagging with appropriate context. / Yang, Li; Peterson, Erik; Chen, John et al.
Text, Speech and Dialogue - 14th International Conference, TSD 2011, Proceedings. 2011. p. 427-433 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 6836 LNAI).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Unsupervised Russian POS tagging with appropriate context

AU - Yang, Li

AU - Peterson, Erik

AU - Chen, John

AU - Petrova, Yana

AU - Srihari, Rohini

PY - 2011

Y1 - 2011

N2 - While adopting the contextualized hidden Markov model (CHMM) framework for unsupervised Russian POS tagging, we investigate the possibility of utilizing the left, right, and unambiguous context in the CHMM framework. We propose a backoff smoothing method that incorporates all three types of context into the transition probability estimation during the expectation-maximization process. The resulting model with this new method achieves overall and disambiguation accuracies comparable to a CHMM using the classic backoff smoothing method for HMM-based POS tagging from [17].

AB - While adopting the contextualized hidden Markov model (CHMM) framework for unsupervised Russian POS tagging, we investigate the possibility of utilizing the left, right, and unambiguous context in the CHMM framework. We propose a backoff smoothing method that incorporates all three types of context into the transition probability estimation during the expectation-maximization process. The resulting model with this new method achieves overall and disambiguation accuracies comparable to a CHMM using the classic backoff smoothing method for HMM-based POS tagging from [17].

KW - CHMM

KW - expectation-maximization (EM)

KW - left

KW - right

KW - transition probability

KW - unambiguous context

KW - unsupervised Russian part-of-speech tagging

UR - https://www.scopus.com/pages/publications/80052772895

U2 - 10.1007/978-3-642-23538-2_54

DO - 10.1007/978-3-642-23538-2_54

M3 - Conference contribution

AN - SCOPUS:80052772895

SN - 9783642235375

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 427

EP - 433

BT - Text, Speech and Dialogue - 14th International Conference, TSD 2011, Proceedings

T2 - 14th International Conference on Text, Speech and Dialogue, TSD 2011

Y2 - 1 September 2011 through 5 September 2011

ER -

Yang L, Peterson E, Chen J, Petrova Y, Srihari R. Unsupervised Russian POS tagging with appropriate context. In Text, Speech and Dialogue - 14th International Conference, TSD 2011, Proceedings. 2011. p. 427-433. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-642-23538-2_54

Unsupervised Russian POS tagging with appropriate context

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this