Classting R&D Blog

Abstract. With an increasing interest in personalized learning, active research is being conducted on knowledge tracing to predict the learner’s knowledge state. Recently, studies have attempted to improve the performance of the knowledge tracing model by incorporating various types of side information. We propose a knowledge tracing method that utilizes the learner’s language proficiency as side information. Language proficiency is a key component of comprehending a question’s text and is known to be closely related to students’ academic performance. In this study, language proficiency was defined with Elo rating score and time window features, and was used in the knowledge tracing task. The dataset used in this study contains 54,470 students and 7,619,040 interactions, which were collected from a real-world online-learning platform. We conducted a correlation analysis to determine whether the language proficiency information of students was related to their ability to solve math word problems. In addition, we examined the effect of incorporating the language proficiency information on the knowledge tracing models using various baseline models. The analysis revealed a high correlation between the length of word problems and students’ language proficiency. Furthermore, in experiments with various baseline models, utilizing the language proficiency information improved the knowledge tracing model’s performance. Finally, when language proficiency information was incorporated, the cold start problem of the knowledge tracing model was mitigated. The findings of this study can be used as a supplement for educational instruction.

As interest in personalized online learning grows, the research in the field of knowledge tracing (KT) to model a learner’s knowledge state also increases. Many studies have recently been conducted to improve the performance of the KT model by using various side information, such as response time [31,33], the number of attempts [10,43], question text [21,36], and relationship between concepts [9,23].

On the other hand, previous studies have found that the academic achievement in one subject is closely related to achievement in other subjects. For example, math grades are significantly correlated with science grades [16,39], while English grades have a significant impact on math and science grades [3,4]. As a result, if learning data from multiple subjects in which students participated can be collected, problem-solving information from one subject can be used to predict achievement in other subjects.

In particular, if we collect problem-solving performance data for students’ first language subjects (e.g., Korean, English, French, etc.), we can estimate their language proficiency (LP) information. LP is an important factor when learners acquire domain knowledge, and it has been shown to have a significant impact on academic performance in a variety of subjects [6,15,30,35]. Moreover, LP is an important factor in the learner’s problem solving. This is owing to the fact that learners must be able to read and comprehend the problem to complete it correctly [6]. For example, in mathematics, students with low LP are known to struggle with long word problems [37]. Therefore, LP information extracted from students’ problem-solving data of language subjects is likely to be useful for the KT task to predict their future performance in other subjects.

However, despite the fact that LP is an important factor in predicting students’ academic performance, to our knowledge, no study in the KT field has used this information. In addition, studies that used academic achievement data in one subject to predict the student’s knowledge state in another subject have not been sufficiently conducted.

In this study, we propose a KT method using LP information extracted from Korean problem solving data from students. First, we analyzed whether LP information was related to students’ math word problem performance. In addition, through experiments with real-world datasets, we predicted the knowledge state using students’ LP information and investigated whether using this information was effective in improving the performance of the KT model. Furthermore, we investigated whether using LP information can help mitigate the cold start problem.

As a result, the following research questions (RQs) were posed in this study, in an attempt to find answers.

Data matters. Accordingly, data augmentation has been an important ingredient for boosting performances of learned models. Prior data augmentation methods for few-shot text classification have led to great performance boosts. However, they have not been designed to capture the intricate compositional structure of natural language. As a result, they fail to generate samples with plausible and diverse sentence structures. Motivated by this, we present the data augmentation using lexicalized probabilistic context-free grammars that generates augmented samples with diverse syntactic structures with plausible grammar.

The Lexicalized PCFG parse trees consider both the constituents and dependencies to produce a syntactic frame that maximizes a variety of word choices in a syntactically preservable manner without specific domain experts. Experiments on few-shot text classification tasks demonstrate that ALP enhances many state-of-the-art classification methods.

We delve into the train-val splitting methodologies when a data augmentation method comes into play. We argue empirically that the traditional splitting of training and validation sets is sub-optimal compared to our novel augmentation-based splitting strategies that further expand the training split with the same number of labeled data. Taken together, our contributions on the data augmentation strategies yield a strong training recipe for few-shot text classification tasks.

오늘은 교육 전문가들의 의견을 반영하여, 실용적으로 실제 교육 현장에서 활용하기 용이한 학습 성취 예측 방법을 제안한 연구를 소개하도록 하겠습니다.

학생의 학습 성취를 미리 예측할 수 있다면, 적절한 교육적인 조치를 취함으로써 학습의 효과를 높이는 데 기여할 수 있습니다. 이러한 기대 하에 학생의 학습 성취를 예측하기 위한 연구들이 오랜 시간 꾸준히 진행되어 왔습니다. 학습자 해당 분야에 대한 선행 연구들을 분석하면 다음과 같은 특징을 발견할 수 있습니다.

오늘 소개드릴 연구에서는, 이렇게 꾸준히 진행되어 온 학습 성취 예측 기술에 대해 한 가지 의문을 제기합니다. “과연 이 기술들을 실제 교육 현장에서 사용할 수 있을까?”

Personality detection identifies a person's characteristic patterns in the online text he or she creates. Active participants on social media yield a considerable amount of online posts implying their psychological status. Having emerged personalized systems using online postings, natural language processing tasks requires psycholinguistic knowledge to detect individuals' personality traits. This emerging task is currently in demand for extensive application scenarios such as personalized recommendation systems, dialogue systems, and more.

Yet prior personality detection has been under-explored and restricted. Existing methods require resource-intensive and time-consuming professional tags to infer personality traits using ground-truth information. Those using deep neural networks have still focused on extra resources of lexical features in addition to the labeled training data. The performance heavily relying on such manual resources as Linguistic Inquiry and Word Count (LIWC) and Medical Research Council (MRC) restricts the pre-trained model's psycholinguistic proficiency and constrains the method in words only specified in the dictionaries. Not only utilizing the dictionaries but pre-trained language models are not in proper use.

We have three assumptions regarding pre-trained language models (PLMs) to minimize these issues. First, personality detection methods utilizing pre-trained language models (PLMs) rely on shortcutted lexical cues than understanding the knowledge in texts. Second, those lexical cues harm PLMs to perform well on out-of-distribution or unseen examples. Third, PLMs are capable of acknowledging the causal relations across out-of-distribution datapoints. We aim to generalize models well to unseen data by inducing models to emerge relevant knowledge through causal relations

Most work on personality detection studies the model performance with an evaluation metric such as Big Five models. The Big Five models utilize four different psychological attributes of analyzing how one gains energy (extraversion), how one processes information (openness to experience), how one makes decisions (agreeableness), and how one presents themselves to the outside world (conscientiousness), and how one he feels anxiety and depression (neuroticism).