Contemporary Journal of Social Science Review • pp. 1143-1170
ANALYZING LEXICAL COMPLEXITY IN LEARNER CORPORA:A CORPUSDRIVEN APPROACH USING PART-OF-SPEECH TAGGING AND DEPENDENCY PARSING
Abstract
Lexical complexity is a crucial component of second language (L2) proficiency, encompassing the range,
sophistication, and density of vocabulary used by learners. This study examines lexical complexity in learner
corpora through a corpus-driven methodology leveraging part-of-speech (POS) tagging and dependency parsing.
The objectives are to (a) operationalize lexical complexity into measurable indices (lexical density, diversity, and
sophistication), (b) utilize POS tagging to automatically identify and categorize lexical items, and (c) apply
dependency parsing to incorporate syntactic context into lexical complexity analysis. The study analyzes a corpus
of 300 L2 English essays from intermediate and advanced learners. Results show that advanced learners use a
higher proportion of low-frequency “sophisticated” words and exhibit greater lexical diversity than intermediate
learners, although lexical density (content word ratio) remains comparable across proficiency levels.
Dependency-based metrics (e.g., average dependency length, noun modifier counts) provided additional insights
into how learners deploy complex lexico-syntactic structures. The findings highlight a positive correlation
between lexical complexity indices and writing quality scores, with lexical sophistication and diversity emerging
as significant predictors of human-rated proficiency. The study fills a methodological gap by integrating NLP
tools in learner corpus research to yield a multi-dimensional profile of lexical complexity. Implications are
discussed for L2 writing pedagogy, automated writing evaluation, and the development of hybrid computational
models for L2 complexity. This research underscores the value of combining POS tagging and dependency parsing
in corpus analyses to obtain granular, robust measures of lexical complexity in learner language
Record
- Identifier
- https://doi-services.com/1102/3231
- Published
- 2025-10-05
- Version
- 1