- Research Article
- 10.18255/1818-1015-2026-1-6-29
Multi-tier linguistic feature engineering for CEFR classification: a comprehensive analysis of deterministic and machine learning-based features
- Mar 16, 2026
- Modeling and Analysis of Information Systems
- Tendai M Chikake + 2 more +2
We analyzed 133 linguistic features for automated proficiency classification under the Common European Framework of Reference (CEFR) in a two-tier architecture: deterministic Tier 1 (lexical, morphological, and syntactic measures) and machine-learning-based Tier 2 (semantic coherence, topic structure, cohesion, and error-analysis signals). Experiments were conducted on a corpus of 3,205 learner texts from mixed sources, with triangulated validation against expert-verified Cambridge examination subsets. The materials were collected in 2022--2025 and included a substantial institutional corpus of over 3,000 essays and other writing texts produced by students of Moscow Institute of Physics and Technology (MIPT) studying English as a foreign language and regularly assessed by our AI-powered testing system ISTOK (Intelligent System for Testing General Language Competencies). Feature matrices were standardized after missing-value handling (fold-local median imputation for cross-validation and zero-fill for held-out reporting). In supervised evaluation, the best Tier 1+2 model reaches 66.72% exact accuracy (macro F1 = 0.69) and 94.53% adjacent accuracy (within one CEFR level) on a 3,198-sample CEFR-labeled benchmark split; an extended comparison including preliminary Tier 3 features achieves 67.50%. Unsupervised analyses show strong structure for extreme levels (A1 99.5% purity; C2 82.4% purity) and moderate alignment with professional Cambridge labels (Adjusted Rand Index = 0.303). We report block ablations and compact subset searches, with strongest signals from morphological complexity and lexical sophistication, and consistent incremental gains from error-based features. The results provide a validated, interpretable feature inventory and practical guidance for feature selection in automated language assessment systems.
Read more