Publications of the Statistical Speech Technology Group Beckman Institute, University of Illinois at Urbana-Champaign

• Speech-to-Text: Acoustic Features | Acoustic Modeling | Audiovisual Speech Recognition | Pronunciation Modeling
• Speech-to-Meaning: Information Retrieval | Meta-Linguistics: Emotion, Cognitive State | Task Semantics | Automatic Recognition of Prosody
• Human-Computer Interaction: Text-to-Speech | Second-Language Acquisition | Tutoring Systems | Disability
• Computer Audition: Noise and Reverberation | Speech Coding | Speaker Recognition | Acoustic Event Detection
• Computer Cognition: Computer Vision | Machine Learning
• Human Speech and Language: Acoustic Physiology | Neurophysiology | Speech Perception | Distinctive Features | Formant Tracking | Prosody Analysis
• Cartoons

Speech Recognition

Acoustic Features

• Sarah Borys, An SVM Front End Landmark Speech Recognition System, M.S. Thesis, 2008.
• Bryce Lobdell, Mark Hasegawa-Johnson, and Jont B. Allen, Human Speech Perception and Feature Extraction, Interspeech 2008
• Ming Liu, Xi Zhou, Mark Hasegawa-Johnson, Thomas S. Huang, and Zhengyou Zhang, "Frequency Domain Correspondence for Speaker Normalization," in Proc. Interspeech, Antwerp, August, 2007.
• Xi Zhou, Yu Fun, Ming Liu, Mark Hasegawa-Johnson, and Thomas Huang, "Robust Analysis and Weighting on MFCC Components for Speech Recognition and Speaker Identification," ICME 2007 (VACE NBCHC060160; NSF 0426627).
• Sarah Borys and Mark Hasegawa-Johnson, "Distinctive Feature Based SVM Discriminant Features for Improvements to Phone Recognition on Telephone Band Speech." ISCA Interspeech, October 2005 (NSF 0132900).
• Mark Hasegawa-Johnson, Sarah Borys and Ken Chen, ``Experiments in Landmark-Based Speech Recognition.'' Sound to Sense: Workshop in Honor of Kenneth N. Stevens, June, 2004 (NSF 0132900).
• M. Kamal Omar and Mark Hasegawa-Johnson, Model Enforcement: A Unified Feature Transformation Framework for Classification and Recognition, IEEE Transactions on Signal Processing, vol. 52, no. 10, pp. 2701-2710, 2004 (NSF 0132900).
• Stefan Geirhofer, Feature Reduction with Linear Discriminant Analysis and its Performance on Phoneme Recognition. Undergraduate research project.
• Mohamed Kamal Mahmoud Omar, Acoustic Feature Design for Speech Recognition: A Statistical Information-Theoretic Approach. Ph.D. Thesis, 2003.
• M. Kamal Omar and Mark Hasegawa-Johnson, Approximately Independent Factors of Speech Using Nonlinear Symplectic Transformation, IEEE Transactions on Speech and Audio Processing, vol. 11, no. 6, pp. 660-671, 2003 (NSF 0132900).
• M. Omar and M. Hasegawa-Johnson, Non-Linear Independent Component Analysis for Speech Recognition, International Conference on Computer, Communication and Control Technologies (CCCT '03), 2003 (NSF 0132900).
• M. Omar and M. Hasegawa-Johnson, Strong-Sense Class-Dependent Features for Statistical Recognition, IEEE Workshop on Statistical Signal Processing, St. Louis, MO, 2003, 473-476 (NSF 0132900).
• M. K. Omar and M. Hasegawa-Johnson, Maximum Conditional Mutual Information Projection For Speech Recognition, Interspeech, September, 2003, 505-508 (NSF 0132900).
• M. K. Omar and M. Hasegawa-Johnson, Non-Linear Maximum Likelihood Feature Transformation For Speech Recognition, Interspeech, September, 2003, 2497-2500 (NSF 0132900).
• M. Hasegawa-Johnson, Finding the Best Acoustic Measurements for Landmark-Based Speech Recognition, Accumu Magazine, Kyoto Computer Gakuin, Kyoto, Japan, 2002 (NSF 0132900).
• M. Omar, K. Chen, M. Hasegawa-Johnson and Y. Brandman, An Evaluation of using Mutual Information for Selection of Acoustic-Features Representation of Phonemes for Speech Recognition, Interspeech, Denver, CO, September 2002, pp. 2129-2132 (Phonetact, Inc.).
• Z. Jing and M. Hasegawa-Johnson, Auditory-Modeling Inspired Methods of Feature Extraction for Robust Automatic Speech Recognition, ICASSP Student Session, May 2002, IV:4176 (NSF 0132900).
• M. K. Omar and M. Hasegawa-Johnson, "Maximum Mutual Information Based Acoustic Features Representation of Phonological Features for Speech Recognition," ICASSP, May 2002, I:81-84.
• Zhinian Jing, Voice Index and Frame Index for Recognition of Digits in Speech Background. M.S. Thesis, 2002.
• W. Gunawan and M. Hasegawa-Johnson, "PLP Coefficients can be Quantized at 400 bps," ICASSP, Salt Lake City, UT, pp. 2.2.1-4, 2001.

Acoustic Modeling

• Jui-Ting Huang, Xi Zhou, Mark Hasegawa-Johnson and Thomas Huang, Kernel Metric Learning for Phonetic Classification, ASRU 2009 (NSF 0703624 and 0534133)
• Xiaodan Zhuang, Hosung Nam, Mark Hasegawa-Johnson, Louis Goldstein, and Elliot Saltzman, Articulatory Phonological Code for Word Recognition, Interspeech, 34549:1-4, Brighton, September 2009 (NSF 0703624)
• Bowon Lee and Mark Hasegawa-Johnson, A Phonemic Restoration Approach for Automatic Speech Recognition with Highly Nonstationary Background Noise, DSP in Cars workshop, Dallas, July 2009
• Jui-Ting Huang and Mark Hasegawa-Johnson, On semi-supervised learning of Gaussian mixture models for phonetic classification, NAACL HLT Workshop on Semi-Supervised Learning, 2009 (NSF 0534106 and NSF 0703624).
• Jui-Ting Huang and Mark Hasegawa-Johnson, Maximum Mutual Information Estimation with Unlabeled Data for Phonetic Classification. Proc. Interspeech 2008 (NSF 0534133).
• Xiaodan Zhuang, Hosung Nam, Mark Hasegawa-Johnson, Louis Goldstein, and Elliot Saltzman, The Entropy of Articulatory Phonological Code: Recognizing Gestures from Tract Variables, Interspeech 2008 (NSF 0703624, NSF 0703782, NIH DC02717).
• Arthur Kantor and Mark Hasegawa-Johnson, Stream Weight Tuning in Dynamic Bayesian Networks, Proc. ICASSP 2008 (NSF 0703624).
• Bowon Lee, Robust Speech Recognition in a Car Using a Microphone Array. Ph.D. thesis, 2006.
• Chitturi, R. and Hasegawa-Johnson, M. Novel Entropy-Based Moving Average Refiners for HMM Landmarks. Interspeech, September 2006 (NSF 0132900).
• Mark Hasegawa-Johnson, James Baker, Sarah Borys, Ken Chen, Emily Coogan, Steven Greenberg, Amit Juneja, Katrin Kirchhoff, Karen Livescu, Srividya Mohan, Jennifer Muller, Kemal Sönmez, and Tianyu Wang, "Landmark-Based Speech Recognition: Report of the 2004 Johns Hopkins Summer Workshop." ICASSP, March 2005 (NSF 0121285).
• Yeojin Kim and Mark Hasegawa-Johnson, Phonetic Segment Rescoring Using SVMs. Midwest Computational Linguistics Colloquium, Columbus, OH, 2005 (NSF 0132900).
• Mark Hasegawa-Johnson, James Baker, Steven Greenberg, Katrin Kirchhoff, Jennifer Muller, Kemal Sonmez, Sarah Borys, Ken Chen, Amit Juneja, Katrin Kirchhoff, Karen Livescu, Srividya Mohan, Emily Coogan, and Tianyu Wang, Landmark-Based Speech Recognition: Report of the 2004 Johns Hopkins Summer Workshop. technical report of the Johns Hopkins Center for Language and Speech Processing, 2005 (NSF 0121285).
• Mark Hasegawa-Johnson, Landmark-Based Speech Recognition: The Marriage of High-Dimensional Machine Learning Techniques with Modern Linguistic Representations, talk given at Tsinghua University, October 2004 (NSF 0132900).
• Ameya Deoras and Mark Hasegawa-Johnson, "A Factorial HMM Approach to Robust Isolated Digit Recognition in Background Music." Interspeech, October, 2004 (NSF 0132900).
• Ameya Deoras and Mark Hasegawa-Johnson, A Factorial HMM Approach to Simultaneous Recognition of Isolated Digits Spoken by Multiple Talkers on One Audio Channel, ICASSP 2004 (NSF 0132900).
• Yanli Zheng and Mark Hasegawa-Johnson, Acoustic segmentation using switching state Kalman Filter, ICASSP 2003, April 2003, I:752-755 (NSF 0132900).
• Ameya Deoras, A Factorial HMM Approach to Robust Isolated Digit Recognition in Non-Stationary Noise. B.S. Thesis, 2003.
• M. K. Omar, M. Hasegawa-Johnson and S. E. Levinson, Gaussian Mixture Models of Phonetic Boundaries for Speech Recognition, ASRU 2001 (NSF 0132900).
• M. Hasegawa-Johnson, Multivariate-State Hidden Markov Models for Simultaneous Transcription of Phones and Formants, ICASSP, Istanbul, pp. 1323-26, 2000

Audiovisual Speech Recognition

• Thomas S. Huang, Mark A. Hasegawa-Johnson, Stephen M. Chu, Zhihong Zeng, and Hao Tang, Sensitive Talking Heads, IEEE Signal Processing Magazine 26(4):67-72, July 2009
• Mark Hasegawa-Johnson, Karen Livescu, Partha Lal and Kate Saenko, Audiovisual Speech Recognition with Articulator Positions as Hidden Variables, in Proc. International Congress on Phonetic Sciences (ICPhS), Saarbrücken, August, 2007 (NSF 0121285).
• Mark Hasegawa-Johnson, Audio-Visual Speech Recognition: Audio Noise, Video Noise, and Pronunciation Variability, talk given to the Signal Processing Society, IEEE Japan, June 2007 (NSF 0534106; NIH DC008090A).
• Yun Fu, Xi Zhou, Ming Liu, Mark Hasegawa-Johnson, and Thomas S. Huang, Lipreading by Locality Discriminant Graph, IEEE International Conference on Image Processing (ICIP) 2007 (VACE NBCHC060160; NSF 0426627).
• Karen Livescu, Ozgur Cetin, Mark Hasegawa-Johnson, Simon King, Chris Bartels, Nash Borges, Arthur Kantor, Partha Lal, Lisa Yung, Ari Bezman, Stephen Dawson-Haggerty, Bronwyn Woods, Joe Frankel, Matthew Magimai-Doss, and Kate Saenko, Articulatory Feature-Based Methods for Acoustic and Audio-Visual Speech Recognition: Summary from the 2006 JHU Summer Workshop. ICASSP, May 2007 (NSF 0121285).
• Karen Livescu, Özgür Çetin, Mark Hasegawa-Johnson, Simon King, Chris Bartels, Nash Borges, Arthur Kantor, Partha Lal, Lisa Yung, Ari Bezman, Stephen Dawson-Hagerty, Bronwyn Woods, Joe Frankel, Mathew Magimai-Doss, and Kate Saenko, Articulatory-Feature-Based Methods for Acoustic and Audio-Visual Speech Recognition: 2006 JHU Summer Workshop Final Report. Johns Hopkins Center for Language and Speech Processing, 2007 (NSF 0121285).
• Mark Hasegawa-Johnson, Object Tracking and Asynchrony in Audio-Visual Speech Recognition. talk given to the Artificial Intelligence, Vision, and Robotics seminar series, August, 2006 (NSF 0534106; NIH DC008090A).
• Mark Hasegawa-Johnson, Dealing with Acoustic Noise. Part IIII: Video. tutorial presentation given at WS06, Center for Language and Speech Processing, July 2006 (NSF 0121285).
• Camille Goudeseune and Bowon Lee, AVICAR: Audio-Visual Speech Recognition in a Car Environment. Promotional Film, 2006 (Motorola RPS19).
• Bowon Lee, Mark Hasegawa-Johnson, Camille Goudeseune, Suketu Kamdar, Sarah Borys, Ming Liu, and Thomas Huang, AVICAR: Audio-Visual Speech Corpus in a Car Environment. Interspeech, October 2004 (Motorola RPS19).
• S.E. Levinson, T.S. Huang, M.A. Hasegawa-Johnson, K. Chen, S. Chu, A. Garg, Z. Jing, D. Li, J. Lin, M. Omar and Z. Wen, Multimodal Dialog Systems Research at Illinois, ARPA Workshop on Multimodal Speech Recognition and SPINE, June, 2002 (NSF 0132900).

Pronunciation Modeling

• Karen Livescu, Özgür Çetin, Mark Hasegawa-Johnson, Simon King, Chris Bartels, Nash Borges, Arthur Kantor, Partha Lal, Lisa Yung, Ari Bezman, Stephen Dawson-Hagerty, Bronwyn Woods, Joe Frankel, Mathew Magimai-Doss, and Kate Saenko, Articulatory-Feature-Based Methods for Acoustic and Audio-Visual Speech Recognition: 2006 JHU Summer Workshop Final Report. Johns Hopkins Center for Language and Speech Processing, 2007 (NSF 0121285).
• Karen Livescu, Ozgur Cetin, Mark Hasegawa-Johnson, Simon King, Chris Bartels, Nash Borges, Arthur Kantor, Partha Lal, Lisa Yung, Ari Bezman, Stephen Dawson-Haggerty, Bronwyn Woods, Joe Frankel, Matthew Magimai-Doss, and Kate Saenko, Articulatory Feature-Based Methods for Acoustic and Audio-Visual Speech Recognition: Summary from the 2006 JHU Summer Workshop. ICASSP, May 2007 (NSF 0121285).
• Ken Chen and Mark Hasegawa-Johnson, Modeling pronunciation variation using artificial neural networks for English spontaneous speech. Interspeech, October 2004 (NSF 0414117).

Speech-to-Meaning

Information Retrieval

Xiaodan Zhuang, Jui-Ting Huang, and Mark Hasegawa-Johnson, Speech Retrieval in Unknown Languages: a Pilot Study, NAACL HLT Cross-Lingual Information Access Workshop (CLIAWS), 2009 (NSF 0534106 and NSF 0703624)

Meta-Linguistics and Emotion

• Hao Tang, Stephen M. Chu, Mark Hasegawa-Johnson, Thomas S. Huang, Emotion Recognition from Speech via Boosted Gaussian Mixture Models, 2009 International Conference on Multimedia & Expo (ICME'09; NIH R21 DC008090 A)
• Tong Zhang, Mark Hasegawa-Johnson and Stephen E. Levinson, Cognitive State Classification in a spoken tutorial dialogue system, Speech Communication 48(6):616-632, 2006(NSF 0085980).
• Tong Zhang, Mark Hasegawa-Johnson, and Stephen E. Levinson, Children's Emotion Recognition in an Intelligent Tutoring Scenario. Interspeech, October, 2004 (NSF 0085980).
• Tong Zhang, Mark Hasegawa-Johnson, and Stephen E. Levinson, An empathic-tutoring system using spoken language, Australian conference on computer-human interaction (OZCHI), 2003 (NSF 0085980).
• Tong Zhang, Mark Hasegawa-Johnson, and Stephen E. Levinson, Mental State Detection of Dialogue System Users via Spoken Language, ISCA/IEEE Workshop on Spontaneous Speech Processing and Recognition (SSPR), April 2003, MAP17.1-4 (NSF 0085980).

Task Semantics

• Tong Zhang, Mark Hasegawa-Johnson and Stephen E. Levinson, "Extraction of Pragmatic and Semantic Salience from Spontaneous Spoken English," Speech Communication, 2007 (NSF 0085980).
• Tong Zhang, Mark Hasegawa-Johnson, and Stephen E. Levinson, A Hybrid Model for Spontaneous Speech Understanding. AAAI 2005 (NSF 0085980).
• Tong Zhang, Mark Hasegawa-Johnson and Stephen E. Levinson, Automatic detection of contrast for speech understanding. Interspeech, October, 2004 (NSF 0085980).
• Yuexi Ren, Mark Hasegawa-Johnson and Stephen E. Levinson. "Semantic analysis for a speech user interface in an intelligent-tutoring system", Intl. Conf. on Intelligent User Interfaces. Madeira, Portugal, 2004 (NSF 0085980).

Automatic Recognition of Prosody

• Jui-Ting Huang and Mark Hasegawa-Johnson, Unsupervised Prosodic Break Detection in Mandarin Speech, SpeechProsody 2008 (NSF 0534133).
• Xiaodan Zhuang and Mark Hasegawa-Johnson, Towards Interpretation of Creakiness in Switchboard, SpeechProsody 2008 (NSF 0414117).
• Taejin Yoon, Jennifer Cole, and Mark Hasegawa-Johnson, Detecting Non-Modal Phonation in Telephone Speech, SpeechProsody, 2008 (NSF 0414117).
• Taejin Yoon, A Predictive Model of Prosody Through Grammatical Interface: A Computational Approach, Ph.D. Thesis, 2007.
• Ken Chen, Mark Hasegawa-Johnson and Jennifer Cole, A Factored Language Model for Prosody-Dependent Speech Recognition, Speech Synthesis and Recognition, Vedran Kordic, Ed., Advanced Robotic Systems, 2007 (NSF 0132900).
• Mark Hasegawa-Johnson, Jennifer Cole, Ken Chen, Partha Lal, Amit Juneja, Taejin Yoon, Sarah Borys, and Xiaodan Zhuang, Prosodically Organized Automatic Speech Recognition. Linguistic Processes in Spontaneous Speech, Academica Sinica, Taiwan, November 2006 (NSF 0414117; NSF 0121285).
• Mark Hasegawa-Johnson, Phonology and the Art of Automatic Speech Recognition, Director's Seminar Series, Beckman Institute, University of Illinois at Urbana-Champaign, November 2006 (NSF 0414117).
• Taejin Yoon, Xiaodan Zhuang, Jennifer Cole, and Mark Hasegawa-Johnson, Voice Quality Dependent Speech Recognition. Midwest Computational Linguistics Colloquium, Urbana, IL, 2006 (NSF 0414117).
• Ken Chen, Mark Hasegawa-Johnson, Aaron Cohen, Sarah Borys, Sung-Suk Kim, Jennifer Cole and Jeung-Yoon Choi, Prosody Dependent Speech Recognition on Radio News Corpus of American English. IEEE Transactions on Speech and Audio Processing, 14(1):232-245, 2006 (NSF 0132900).
• Cole, Jennifer, Mark Hasegawa-Johnson, Chilin Shih, Eun-Kyung Lee, Heejin Kim, H. Lu, Yoonsook Mo, Tae-Jin Yoon. (2005). Prosodic Parallelism as a Cue to Repetition and Hesitation Disfluency, Proceedings of DISS'05 (An ISCA Tutorial and Research Workshop), Aix-en-Provence, France, pp. 53-58 (NSF 0414117).
• Mark Hasegawa-Johnson, Ken Chen, Jennifer Cole, Sarah Borys, Sung-Suk Kim, Aaron Cohen, Tong Zhang, Jeung-Yoon Choi, Heejin Kim, Taejin Yoon, and Sandra Chavarria, Simultaneous Recognition of Words and Prosody in the Boston University Radio Speech Corpus. Speech Communication 46(3-4):418-439, 2005 (NSF 0132900).
• Yoon, Tae-Jin, Cole, Jennifer, Mark Hasegawa-Johnson, and Chilin Shih. Detecting Non-modal Phonation in Telephone Speech. Unpublished manuscript, 2005 (NSF 0414117).
• Yoon, Tae-Jin, Cole, Jennifer, Mark Hasegawa-Johnson, and Chilin Shih. (2005). Acoustic correlates of non-modal phonation in telephone speech, The Journal of the Acoustical Society of America 117(4), p. 2621 (NSF 0414117).
• Sarah Borys, Mark Hasegawa-Johnson, Ken Chen, and Aaron Cohen, Modeling and Recognition of Phonetic and Prosodic Factors for Improvements to Acoustic Speech Recognition Models. Interspeech, October, 2004 (NSF 0132900).
• Mark Hasegawa-Johnson, Speech Recognition Models of the Interdependence Among Syntax, Prosody, and Segmental Acoustics, talk given at Tsinghua University, October 2004 (NSF 0414117).
• Mark Hasegawa-Johnson, Jennifer Cole, Chilin Shih, Ken Chen, Aaron Cohen, Sandra Chavarria, Heejin Kim, Taejin Yoon, Sarah Borys, and Jeung-Yoon Choi, Speech Recognition Models of the Interdependence Among Syntax, Prosody, and Segmental Acoustics, Human Language Technologies: Meeting of the North American Chapter of the Association for Computational Linguistics (HLT/NAACL), Workshop on Higher-Level Knowledge in Automatic Speech Recognition and Understanding, May, 2004 (NSF 0414117).
• Ken Chen and Mark Hasegawa-Johnson, How Prosody Improves Word Recognition, SpeechProsody 2004, Nara, Japan, March 2004, 583-586 (NSF 0132900).
• Aaron Cohen, A Survey of Machine Learning Methods for Predicting Prosody in Radio Speech. M.S. Thesis, 2004.
• Ken Chen, Mark Hasegawa-Johnson, Aaron Cohen, and Jennifer Cole, A Maximum Likelihood Prosody Recognizer, SpeechProsody 2004, Nara, Japan, March 2004, 509-512 (NSF 0132900; Illinois CRI).
• Ken Chen and Mark Hasegawa-Johnson, An Automatic Prosody Labeling System Using ANN-Based Syntactic-Prosodic Model and GMM-Based Acoustic-Prosodic Model, ICASSP 2004 (NSF 0132900; Illinois CRI).
• Sung-Suk Kim, Mark Hasegawa-Johnson, and Ken Chen, Automatic Recognition of Pitch Movements Using Multilayer Perceptron and Time-Delay Recursive Neural Network, IEEE Signal Processing Letters 11(7):645-648, 2004(NSF 0132900; Illinois CRI).
• Yuexi Ren, Sung-Suk Kim, Mark Hasegawa-Johnson, and Jennifer Cole, Speaker-Independent Automatic Detection of Pitch Accent, SpeechProsody 2004, Nara, Japan, March 2004, 521-524 (NSF 0085980).
• Ken Chen, Mark Hasegawa-Johnson and Sung-Suk Kim, An Intonational Phrase Boundary and Pitch Accent Dependent Speech Recognizer. International Conference on Systems, Cybernetics, and Intelligence, 2003 (Illinois CRI).
• Ken Chen and Mark Hasegawa-Johnson, ``Improving the robustness of prosody dependent language modeling based on prosody syntax cross-correlation.'' ASRU, 2003 (Illinois CRI).
• Ken Chen, Mark Hasegawa-Johnson and Jennifer Cole, "Prosody Dependent Speech Recognition on Radio News," IEEE Workshop on Statistical Signal Processing, St. Louis, MO, 2003 (Illinois CRI).
• K. Chen, M. Hasegawa-Johnson, A. Cohen, S. Borys, and J. Cole, Prosody Dependent Speech Recognition with Explicit Duration Modelling at Intonatinal Phrase Boundaries. Interspeech, September, 2003, 393-396 (Illinois CRI).
• Sarah Borys, Recognition of Prosodic Factors and Detection of Landmarks for Improvements to Continuous Speech Recognition Systems. B.S. Thesis, 2003.
• Sarah Borys, Mark Hasegawa-Johnson and Jennifer Cole, The Importance of Prosodic Factors in Phoneme Modeling with Applications to Speech Recognition, ACL Student Session, 2003 (NSF 0132900).
• Sarah Borys, Mark Hasegawa-Johnson and Jennifer Cole, Prosody as a Conditioning Variable in Speech Recognition, Illinois Journal of Undergraduate Research, 2003 (Illinois CRI).

Human-Computer Interaction

Speech Synthesis

• Thomas S. Huang, Mark A. Hasegawa-Johnson, Stephen M. Chu, Zhihong Zeng, and Hao Tang, Sensitive Talking Heads, IEEE Signal Processing Magazine 26(4):67-72, July 2009
• Hao Tang, Yun Fu, Jilin Tu, Mark Hasegawa-Johnson, and Thomas S. Huang, Humanoid Audio-Visual Avatar with Emotive Text-to-Speech Synthesis, IEEE Trans. Multimedia 10(6):969-981, 2008
• Hao Tang, Xi Zhou, Matthias Odisio, Mark Hasegawa-Johnson, and Thomas Huang, Two-Stage Prosody Prediction for Emotional Text-to-Speech Synthesis, Interspeech 2008 (VACE; NSF 0426227).
• Hao Tang, Yun Fu, Jilin Tu, Thomas Huang, and Mark Hasegawa-Johnson, EAVA: A 3D Emotive Audio-Visual Avatar, IEEE Workshop on Applications of Computer Vision (IEEE WACV '08), 2008 (VACE; NSF 0426227).

Second-Language Acquisition

• Su-Youn Yoon, Richard Sproat, and Mark Hasegawa-Johnson, Automated Pronunciation Scoring using Confidence Scoring and Landmark-based SVM, Interspeech 80100:1-4, Brighton, September 2009
• Su-Youn Yoon, Mark Hasegawa-Johnson and Richard Sproat, Automated Pronunciation Scoring for L2 English Learners, CALICO workshop, 2009
• Su-Youn Yoon, Lisa Pierce, Amanda Huensch, Eric Juul, Samantha Perkins, Richard Sproat, and Mark Hasegawa-Johnson, Construction of a rated speech corpus of L2 learners' speech, CALICO journal, 2009
• Su-Youn Yoon, Lisa Pierce, Amanda Huensch, Eric Juul, Samantha Perkins, Richard Sproat, and Mark Hasegawa-Johnson, "Construction of a rated speech corpus of L2 learners' speech," CALICO workshop, 2008

Tutoring

• Tong Zhang, Mark Hasegawa-Johnson and Stephen E. Levinson, Cognitive State Classification in a spoken tutorial dialogue system, Speech Communication 48(6):616-632, 2006(NSF 0085980).
• Tong Zhang, Mark Hasegawa-Johnson, and Stephen E. Levinson, Children's Emotion Recognition in an Intelligent Tutoring Scenario. Interspeech, October, 2004 (NSF 0085980).
• Tong Zhang, Mark Hasegawa-Johnson, and Stephen E. Levinson, An empathic-tutoring system using spoken language, Australian conference on computer-human interaction (OZCHI), 2003 (NSF 0085980).
• M. Hasegawa-Johnson, J. Cha, S. Pizza and K. Haker, CTMRedit: A case study in human-computer interface design, International Conference On Public Participation and Information Tech., Lisbon, pp. 575-584, 1999 (NIH DC0032301).

Disability

• Harsh Vardhan Sharma, Mark Hasegawa-Johnson, Jon Gunderson, and Adrienne Perlman, Universal Access: Speech Recognition for Talkers with Spastic Dysarthria, Interspeech 42862:1-4, Brighton, September 2009 (NIH R21 DC008090A)
• Harsh Vardhan Sharma, Universal Access: Experiments in Automatic Recognition of Dysarthric Speech, M.S. Thesis, 2008 (NSF 0534106).
• Heejin Kim, Mark Hasegawa-Johnson, Adrienne Perlman, Jon Gunderson, Thomas Huang, Kenneth Watkin, and Simone Frame, Dysarthric Speech Database for Universal Access Research, Interspeech 2008 (NSF 0534106; NIH DC008090A).
• Weimo Zhu, Mark Hasegawa-Johnson, Karen Chapman-Novakofski, and Arthur Kantor, Cellphone-Based Nutrition E-Diary. National Nutrient Database Conference, 2007 (Robert Wood Johnson Foundation).
• Weimo Zhu, Mark Hasegawa-Johnson, Arthur Kantor, Dan Roth, Yong Gao, Youngsik Park, and Lin Yang, "E-coder for Automatic Scoring Physical Activity Diary Data: Development and Validation." ACSM, 2007 (Robert Wood Johnson Foundation).
• Mark Hasegawa-Johnson, Jonathan Gunderson, Adrienne Perlman, and Thomas Huang, HMM-Based and SVM-Based Recognition of the Speech of Talkers with Spastic Dysarthria, ICASSP, May 2006 (NSF 0534106; NIH DC008090A).
• Weimo Zhu, Mark Hasegawa-Johnson, and Mital Arun Gandhi, Accuracy of Voice-Recognition Technology in Collecting Behavior Diary Data. Association of Test Publishers (ATP): Innovations in Testing, March 2005 (Robert Wood Johnson Foundation).

Computer Audition

• J. Beauchamp, H. Taube, S. Tipei, S. Wyatt, L. Haken and M. Hasegawa-Johnson, "Acoustics, Audio, and Music Technology Education at the University of Illinois," JASA, 110(5):2961, 2001.

Noise and Reverberation

• Lae-Hoon Kim and Mark Hasegawa-Johnson, Optimal Multi-Microphone Speech Enhancement in Cars, DSP in Cars workshop, Dallas, July 2009 (NSF 0803219 and 0807329)
• Lae-Hoon Kim, Mark Hasegawa-Johnson, Jun-Seok Lim, and Koeng-Mo Sung, Acoustic model for robustness analysis of optimal multipoint room equalization, JASA 123(4):2043-2053, 2008.
• Lae-Hoon Kim and Mark Hasegawa-Johnson, Optimal Speech Estimator Considering Room Response as well as Additive Noise: Different Approaches in Low and High Frequency Range, ICASSP 2008.
• Bowon Lee and Mark Hasegawa-Johnson, Minimum Mean Squared Error A Posteriori Estimation of High Variance Vehicular Noise, in 2007 Biennial on DSP for In-Vehicle and Mobile Systems, Istanbul, June, 2007 (Motorola RPS19; NSF 0534106).
• Bowon Lee, Robust Speech Recognition in a Car Using a Microphone Array. Ph.D. thesis, 2006.
• Mark Hasegawa-Johnson, Dealing with Acoustic Noise. Part II: Beamforming. tutorial presentation given at WS06, Center for Language and Speech Processing, July 2006 (NSF 0121285).
• Mark Hasegawa-Johnson, Dealing with Acoustic Noise. Part I: Spectral Estimation. tutorial presentation given at WS06, Center for Language and Speech Processing, July 2006 (NSF 0121285).
• Lae-Hoon Kim and Mark Hasegawa-Johnson, Generalized Optimal Multi-Microphone Speech Enhancement Using Sequential Minimum Variance Distortionless Response (MVDR) Beamforming and Postfiltering, ICASSP, May 2006.
• Lae-Hoon Kim and Mark Hasegawa-Johnson, Generalized multi-microphone spectral amplitude estimation based on correlated noise model. 119th Convention of the Audio Engineering Society, New York, October 2005.
• Mital Gandhi and Mark Hasegawa-Johnson, Source Separation using Particle Filters. Interspeech, October 2004 (NSF 0132900).
• Bowon Lee, Mark Hasegawa-Johnson, and Camille Goudeseune, Open Loop Multichannel Inversion of Room Impulse Response, JASA 113(4):2202-3, 2003 (NSF 0132900).

Speech Coding

• M. Hasegawa-Johnson and A. Alwan, Speech Coding: Fundamentals and Applications, Wiley Encyclopedia of Telecommunications and Signal Processing, J. Proakis, Ed., Wiley and Sons, NY, December 2002 (NSF 0132900).
• W. Gunawan and M. Hasegawa-Johnson, "PLP Coefficients can be Quantized at 400 bps," ICASSP, Salt Lake City, UT, pp. 2.2.1-4, 2001.
• Mark Hasegawa-Johnson and T. Taniguchi, "On-line and off-line computational reduction techniques using backward filtering in CELP speech coders," IEEE Transactions Acoustics, Speech, and Signal Processing, vol. 40, pp. 2090-2093, 1992 (Fujitsu).
• M. A. Johnson and T. Taniguchi, "Low-complexity multi-mode VXC using multi-stage optimization and mode selection," ICASSP, Toronto, Canada, pp. 221-224, 1991 (Fujitsu).
• T. Taniguchi, M. A. Johnson, and Y. Ohta, Pitch sharpening for perceptually improved CELP, and the sparse-delta codebook for reduced computation, ICASSP, Toronto, Canada, pp. 241-244, 1991 (Fujitsu).
• T. Taniguchi, F. Amano, and M. A. Johnson, "Improving the performance of CELP-based speech coding at low bit rates," International Symposium on Circuits and Systems, Singapore, 1991 (Fujitsu).
• M. A. Johnson and T. Taniguchi, "Computational reduction in sparse-codebook CELP using backward-weighting of the input," Institute of Electr., Information, and Comm. Eng. Symposium, DSP 90-15, Hakata, 61-66, 1990 (Fujitsu).
• T. Taniguchi, M. A. Johnson, and Y. Ohta, "Multi-vector pitch-orthogonal LPC: quality speech with low complexity at rates between 4 and 8 kbps," ICSLP, Kobe, pp. 113-116, 1990 (Fujitsu).
• M. A. Johnson and T. Taniguchi, "Pitch-orthogonal code-excited LPC," IEEE Global Telecommunications Conference (GLOBECOM), San Diego, CA, pp. 542-546, 1990 (Fujitsu).

Speaker Recognition

• Ming Liu, Xi Zhou, Mark Hasegawa-Johnson, Thomas S. Huang, and Zhengyou Zhang, "Frequency Domain Correspondence for Speaker Normalization," in Proc. Interspeech, Antwerp, August, 2007.
• Xi Zhou, Yu Fun, Ming Liu, Mark Hasegawa-Johnson, and Thomas Huang, Robust Analysis and Weighting on MFCC Components for Speech Recognition and Speaker Identification, ICME 2007 (VACE NBCHC060160; NSF 0426627).
• Ming Liu, Zhengyou Zhang, Mark Hasegawa-Johnson, and Thomas Huang, Exploring Discriminative Learning for Text-Independent Speaker Recognition, ICME 2007 (NSF 0426627).

Acoustic Event Detection

• Mark Hasegawa-Johnson, Xiaodan Zhuang, Xi Zhou, Camille Goudeseune, and Thomas Huang, Adaptation of Tandem HMMs for Non-Speech Audio Event Detection, Spring Meeting of the ASA (NSF 08-03219 and 08-07329).
• Mark Hasegawa-Johnson, Xiaodan Zhuang, and Xi Zhou, Adaptation of tandem HMMs for non-speech audio event detection, Spring Meeting of the ASA (NSF 08-03219 and 08-07329).
• Xiaodan Zhuang, Jing Huang, Gerasimos Potamianos and Mark Hasegawa-Johnson, Acoustic Fall Detection using Gaussian Mixture Models and GMM Supervectors, ICASSP 2009 (NetCarity).
• Xiaodan Zhuang, Xi Zhou, Thomas S. Huang and Mark Hasegawa-Johnson, Feature Analysis and Selection for Acoustic Event Detection, ICASSP 2008 (VACE; NSF 0414117; NSF 0534106).
• Xi Zhou, Xiaodan Zhuang, Ming Lui, Hao Tang, Mark Hasegawa-Johnson and Thomas Huang, HMM-Based Acoustic Event Detection with AdaBoost Feature Selection, Proc. CLEAR Evaluation and Workshop (Classification of Events, Activities, and Relationships), Baltimore, May, 2007 (VACE; NSF 0414117; NSF 0534106).
• David Petruncio, Evaluation of Various Features for Music Genre Classification with Hidden Markov Models. B.S. Thesis, 2002.

Computer Cognition

Computer Vision

• Xiaodan Zhuang, Xi Zhou, Mark A. Hasegawa-Johnson and Thomas S. Huang, Efficient Object Localization with Gaussianized Vector Representation, IMCE 2009 (NSF 0803219).
• Xiaodan Zhuang, Xi Zhou, Mark Hasegawa-Johnson, and Thomas Huang, Face Age Estimation Using Patch-based Hidden Markov Model Supervectors, ICPR 2008 (NSF 0534106; VACE).
• Xi Zhou, Xiaodan Zhuang, Hao Tang, Mark Hasegawa-Johnson, and Thomas Huang, A Novel Gaussianized Vector Representation for Natural Scene Categorization, ICPR 2008 (NSF 0534106; VACE).
• Xi Zhou, Xiaodan Zhuang, Shuicheng Yan, Shih-Fu Chang, Mark Hasegawa-Johnson, and Thomas S. Huang, SIFT-Bag Kernel for Video Event Analysis, ACM Multimedia 2008 (NSF 0534106; VACE).

Machine Learning

• Jui-Ting Huang and Mark Hasegawa-Johnson, On semi-supervised learning of Gaussian mixture models for phonetic classification, NAACL HLT Workshop on Semi-Supervised Learning, 2009 (NSF 0534106 and NSF 0703624).
• Mark Hasegawa-Johnson, Tutorial: Pattern Recognition in Signal Processing, Spring Meeting of the ASA (NSF 08-03219 and 08-07329).
• Yang Li, Incremental Training and Growth of Artificial Neural Networks, M.S. Thesis, 2008 (NSF 0534106).
• M. Kamal Omar and Mark Hasegawa-Johnson, Model Enforcement: A Unified Feature Transformation Framework for Classification and Recognition, IEEE Transactions on Signal Processing, vol. 52, no. 10, pp. 2701-2710, 2004 (NSF 0132900).

Human Speech and Language

Acoustic Physiology

• Mark Hasegawa-Johnson, Shamala Pizza, Abeer Alwan, Jul Cha, and Katherine Haker, Vowel Category Dependence of the Relationship Between Palate Height, Tongue Height, and Oral Area, Journal of Speech, Language, and Hearing Research, vol. 46, no. 3, pp. 738-753, 2003 (NIH DC0032301).
• Yanli Zheng, Mark Hasegawa-Johnson, and Shamala Pizza, PARAFAC Analysis of the Three dimensional tongue Shape, Journal of the Acoustical Society of America, vol. 113, no. 1, pp. 478-486, January 2003 (NIH DC0032301).
• Mark Hasegawa-Johnson, Line Spectral Frequencies are the Poles and Zeros of a Discrete Matched-Impedance Vocal Tract Model, Journal of the Acoustical Society of America, vol. 108, no. 1, pp. 457-460, 2000 (NIH DC0032301).
• Y. Zheng and M. Hasegawa-Johnson, Three Dimensional Tongue shape Factor Analysis, American Speech-Language Hearing Association National Convention, Washington, DC, 2000. Published in the magazine ASHA Leader, 5(16):144 (NIH 0032301).
• M. Hasegawa-Johnson, Preliminary Work and Proposed Continuation: Imaging of Speech Anatomy and Behavior. Talk given at the Universities of Illinois Inter-campus Biomedical Imaging Forum, 2001 (NIH 0032301).
• M. Hasegawa-Johnson, J. Cha and K. Haker, CTMRedit: A Matlab-based tool for segmenting and interpolating MRI and CT images in three orthogonal planes, 21st Annual International Conference of the IEEE/EMBS Society, pp. 1170. 1999 (NIH 0032301).
• M. Hasegawa-Johnson, "Combining magnetic resonance image planes in the Fourier domain for improved spatial resolution." International Conference On Signal Processing Applications and Technology, Orlando, FL, pp. 81.1-5, 1999 (NIH 0032301)
• Mark Hasegawa-Johnson, Electromagnetic Exposure Safety of the Carstens Articulograph AG100, Journal of the Acoustics Society of America, vol. 104, pp. 2529-2532, 1998 (NIH 0032301).
• M. A. Johnson, "Using beam elements to model the vocal fold length in breathy voicing," JASA 91:2420-2421, 1992.

Neurophysiology

• Soo-Eun Chang, Nicoline Ambrose, Kirk Erickson, and Mark Hasegawa-Johnson, "Brain Anatomy Differences in Childhood Stuttering." Neuroimage, in press (NIH DC05210, Illinois Research Board).
• Soo-Eun Chang, Kirk I. Erickson, Nicoline G. Ambrose, Mark Hasegawa-Johnson, and C.L. Ludlow, "Deficient white matter development in left hemisphere speech-language regions in children who stutter." Society for Neuroscience, Atlanta, GA, 2006 (NIH DC05210, Illinois Research Board).
• Soo-Eun Chang, Nicoline Ambrose, and Mark Hasegawa-Johnson, "An MRI (DTI) study on children with persistent developmental stuttering." 2004 ASHA Convention, American Speech Language and Hearing Association, November, 2004 (Illinois Research Board).

Speech Perception

• Bryce Lobdell, Models of Human Phone Transcription in Noise Based on Intelligibility Predictors, Bryce Lobdell, Ph.D. Thesis, 2009
• Yoonsook Mo, Jennifer Cole and Mark Hasegawa-Johnson, How do ordinary listeners perceive prosodic prominence? Syntagmatic vs. Paradigmatic comparison. Spring Meeting of the ASA, 2009 (NSF 0703624)
• Bryce Lobdell, Mark Hasegawa-Johnson, and Jont B. Allen, Human Speech Perception and Feature Extraction, Interspeech 2008
• Mark Hasegawa-Johnson, Bayesian Learning for Models of Human Speech Perception, IEEE Workshop on Statistical Signal Processing, St. Louis, MO, 2003, 393-396(NSF 0132900).
• S. Takayanagi, M. Hasegawa-Johnson, L. S. Eisner and A. Schaefer-Martinez, Information theory and variance estimation techniques in the analysis of category rating data and paired comparisons. JASA, 102:3091, 1997

Distinctive Features

• Xiaodan Zhuang, Hosung Nam, Mark Hasegawa-Johnson, Louis Goldstein, and Elliot Saltzman, Articulatory Phonological Code for Word Recognition, Interspeech, 34549:1-4, Brighton, September 2009 (NSF 0703624)
• Sarah Borys, An SVM Front End Landmark Speech Recognition System, M.S. Thesis, 2008.
• Xiaodan Zhuang, Hosung Nam, Mark Hasegawa-Johnson, Louis Goldstein, and Elliot Saltzman, The Entropy of Articulatory Phonological Code: Recognizing Gestures from Tract Variables, Interspeech 2008 (NSF 0703624, NSF 0703782, NIH DC02717).
• Yoonsook Mo, Temporal, spectral evidence of devoiced vowels in Korean, in Proc. International Congress on Phonetic Sciences (ICPhS), Saarbrücken, August, 2007.
• Chitturi, R. and Hasegawa-Johnson, M. "Novel Time-Domain Multi-class SVMs for Landmark Detection." Interspeech, September 2006.
• M. Hasegawa-Johnson, "Time-Frequency Distribution of Partial Phonetic Information Measured Using Mutual Information," Interspeech IV:133-136, Beijing, 2000.
• M. A. Hasegawa-Johnson, "Burst spectral measures and formant frequencies can be used to accurately discriminate stop place of articulation," JASA, 98:2890, 1995
• Mark A. Johnson, "A mapping between trainable generalized properties and the acoustic correlates of distinctive features," MIT Speech Communication Group Working Papers, vol. 9, pp. 94-105, 1994.
• M. Johnson, "Automatic context-sensitive measurement of the acoustic correlates of distinctive features," ICSLP, Yokohama, pp. 1639-1643, 1994
• M. A. Johnson, "A mapping between trainable generalized properties and the acoustic correlates of distinctive features," JASA, vol. 94, p. 1865, 1993.

Formant Tracking

• İ. Yücel Özbek, Mark Hasegawa-Johnson, and Yübeccel Demirekler, Formant Trajectories for Acoustic-to-Articulatory Inversion, Interspeech 95957:1-4, Brighton, September 2009
• Yanli Zheng, Feature Extraction and Acoustic Modeling for Speech Recognition. Ph.D. Thesis, 2005.
• Yanli Zheng and Mark Hasegawa-Johnson, Stop Consonant Classification by Dynamic Formant Trajectory. Interspeech, October, 2004 (NSF 0132900).
• Yanli Zheng and Mark Hasegawa-Johnson, Formant Tracking by Mixture State Particle Filter, ICASSP 2004 (NSF 0132900).
• Y. Zheng and M. Hasegawa-Johnson, Particle Filtering Approach to Bayesian Formant Tracking, IEEE Workshop on Statistical Signal Processing, September, 2003, 581-584 (NSF 0132900).

Prosody Analysis

• Yoonsook Mo, Jennifer Cole and Mark Hasegawa-Johnson, Prosodic effects on vowel production: evidence from formant structure, Interspeech 19096:1-4, Brighton, September 2009 (NSF 0703624)
• Yoonsook Mo, Jennifer Cole and Mark Hasegawa-Johnson, How do ordinary listeners perceive prosodic prominence? Syntagmatic vs. Paradigmatic comparison. Spring Meeting of the ASA, 2009 (NSF 0703624)
• Taejin Yoon, Jennifer Cole and Mark Hasegawa-Johnson, On the edge: Acoustic cues to layered prosodic domains, in Proc. International Congress on Phonetic Sciences (ICPhS) Saarbrücken, August, 2007 (NSF 0414117).
• Taejin Yoon, Jennifer Cole and Mark Hasegawa-Johnson, On the edge: Acoustic cues to layered prosodic domains. 81st Annual Meeting of the Linguistic Society of America, Anaheim, CA, January 5, 2007 (NSF 0414117).
• Jennifer Cole, Heejin Kim, Hansook Choi, and Mark Hasegawa-Johnson, "Prosodic effects on acoustic cues to stop voicing and place of articulation: Evidence from Radio News speech." J Phonetics 35:180-209, 2007 (NSF 0414117).
• Kim, H., Yoon, T., Cole, J., and Hasegawa-Johnson, M. Acoustic differentiation of L- and L-L% in Switchboard and Radio News speech. Proceedings of Speech Prosody 2006, Dresden (NSF 0414117).
• Taejin Yoon, "Mapping Syntax and Prosody." Midwest Computational Linguistics Colloquium, Columbus, OH, 2005 (NSF 0414117).
• Jeung-Yoon Choi, Mark Hasegawa-Johnson, and Jennifer Cole, "Finding Intonational Boundaries Using Acoustic Cues Related to the Voice Source." Journal of the Acoustical Society of America 118(4):2579-88, 2005 (Illinois CRI).
• Cole, Jennifer, Mark Hasegawa-Johnson, Chilin Shih, Eun-Kyung Lee, Heejin Kim, H. Lu, Yoonsook Mo, Tae-Jin Yoon. (2005). "Prosodic Parallelism as a Cue to Repetition and Hesitation Disfluency," Proceedings of DISS'05 (An ISCA Tutorial and Research Workshop), Aix-en-Provence, France, pp. 53-58 (NSF 0414117).
• Taejin Yoon, Sandra Chavarria, Jennifer Cole, and Mark Hasegawa-Johnson, Intertranscriber Reliability of Prosodic Labeling on Telephone Conversation Using ToBI. Interspeech, October, 2004 (Illinois CRI).
• Tae-Jin Yoon, Heejin Kim, and Sandra Chavarría. "Local Acoustic Cues Distinguishing Two Levels of prosodic Phrasing: Speech Corpus Evidence," Lab phon 9, University of Illinois at Urbana-Champaign, 2004 (Illinois CRI).
• Heejin Kim, Jennifer Cole, Hansook Choi, and Mark Hasegawa-Johnson, The Effect of Accent on Acoustic Cues to Stop Voicing and Place of Articulation in Radio News Speech, SpeechProsody 2004, Nara, Japan, March 2004, 29-32 (Illinois CRI).
• Sandra Chavarria, Taejin Yoon, Jennifer Cole, and Mark Hasegawa-Johnson, Acoustic differentiation of ip and IP boundary levels: Comparison of L- and L-L% in the Switchboard corpus, Speech Prosody 2004, Nara, Japan, March 2004, 333-336 (Illinois CRI).
• J. Cole, H. Choi, H. Kim, and M. Hasegawa-Johnson, The Effect of Accent on the Acoustic Cues to Stop Voicing in Radio News Speech, Proceedings of the International Congress of Phonetic Sciences, Barcelona, Spain, August, 2003 (Illinois CRI).
• Mark A. Johnson, "Analysis of durational rhythms in two poems by Robert Frost," MIT Speech Communication Group Working Papers, vol. 8, pp. 29-42, 1992.

Cartoons

• Lovable Indestructible Grad Student of Chaos, the cartoons.
• Lovable Indestructible Grad Student of Chaos, the Ph.D. thesis