Menu
×
John L. Dart Library
9 a.m. - 5 p.m.
Phone: (843) 722-7550
West Ashley Library
9 a.m. - 5 p.m.
Phone: (843) 766-6635
Folly Beach Library
9 a.m. - 2 p.m.
*open the 2nd and 4th Saturday
*open the 2nd and 4th Saturday
Phone: (843) 588-2001
Edgar Allan Poe/Sullivan's Island Library
Closed for renovations
Phone: (843) 883-3914
Wando Mount Pleasant Library
9 a.m. - 5 p.m.
Phone: (843) 805-6888
Village Library
9 a.m. - 1 p.m.
Phone: (843) 884-9741
St. Paul's/Hollywood Library
9 a.m. - 5 p.m.
Phone: (843) 889-3300
Otranto Road Library
9 a.m. - 5 p.m.
Phone: (843) 572-4094
Mt. Pleasant Library
9 a.m. – 5 p.m.
Phone: (843) 849-6161
McClellanville Library
9 a.m. – 1 p.m.
Phone: (843) 887-3699
Keith Summey North Charleston Library
9 a.m. - 5 p.m.
Phone: (843) 744-2489
John's Island Library
9 a.m. - 5 p.m.
Phone: (843) 559-1945
Hurd/St. Andrews Library
9 a.m. - 5 p.m.
Phone: (843) 766-2546
Miss Jane's Building (Edisto Library Temporary Location)
9 a.m. – 1 p.m.
Phone: (843) 869-2355
Dorchester Road Library
9 a.m. - 5 p.m.
Phone: (843) 552-6466
Baxter-Patrick James Island
9 a.m. - 5 p.m.
Phone: (843) 795-6679
Main Library
9 a.m. - 5 p.m.
Phone: (843) 805-6930
Bees Ferry West Ashley Library
9 a.m. - 5 p.m.
Phone: (843) 805-6892
Mobile Library
Closed
Phone: (843) 805-6909
Today's Hours
John L. Dart Library
9 a.m. - 5 p.m.
Phone: (843) 722-7550
West Ashley Library
9 a.m. - 5 p.m.
Phone: (843) 766-6635
Folly Beach Library
9 a.m. - 2 p.m.
*open the 2nd and 4th Saturday
*open the 2nd and 4th Saturday
Phone: (843) 588-2001
Edgar Allan Poe/Sullivan's Island Library
Closed for renovations
Phone: (843) 883-3914
Wando Mount Pleasant Library
9 a.m. - 5 p.m.
Phone: (843) 805-6888
Village Library
9 a.m. - 1 p.m.
Phone: (843) 884-9741
St. Paul's/Hollywood Library
9 a.m. - 5 p.m.
Phone: (843) 889-3300
Otranto Road Library
9 a.m. - 5 p.m.
Phone: (843) 572-4094
Mt. Pleasant Library
9 a.m. – 5 p.m.
Phone: (843) 849-6161
McClellanville Library
9 a.m. – 1 p.m.
Phone: (843) 887-3699
Keith Summey North Charleston Library
9 a.m. - 5 p.m.
Phone: (843) 744-2489
John's Island Library
9 a.m. - 5 p.m.
Phone: (843) 559-1945
Hurd/St. Andrews Library
9 a.m. - 5 p.m.
Phone: (843) 766-2546
Miss Jane's Building (Edisto Library Temporary Location)
9 a.m. – 1 p.m.
Phone: (843) 869-2355
Dorchester Road Library
9 a.m. - 5 p.m.
Phone: (843) 552-6466
Baxter-Patrick James Island
9 a.m. - 5 p.m.
Phone: (843) 795-6679
Main Library
9 a.m. - 5 p.m.
Phone: (843) 805-6930
Bees Ferry West Ashley Library
9 a.m. - 5 p.m.
Phone: (843) 805-6892
Mobile Library
Closed
Phone: (843) 805-6909
Patron Login
menu
Item request has been placed!
×
Item request cannot be made.
×
Processing Request
Triangulating Natural Language Processing (NLP)-Based Analysis of Rater Comments and Many-Facet Rasch Measurement (MFRM): An Innovative Approach to Investigating Raters' Application of Rating Scales in Writing Assessment
Item request has been placed!
×
Item request cannot be made.
×
Processing Request
- Author(s): Huiying Cai (ORCID
Huiying Cai (ORCID 0000-0001-5110-4553 ); Xun Yan (ORCIDXun Yan (ORCID 0000-0002-4544-4938 )- Language:
English- Source:
Language Testing. 2024 41(2):384-411.- Publication Date:
2024- Document Type:
Journal Articles
Reports - Research - Language:
- Additional Information
- Availability: SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: [email protected]; Web site: https://sagepub.com
- Peer Reviewed: Y
- Source: 28
- Education Level: Higher Education
Postsecondary Education - Subject Terms:
- Accession Number: 10.1177/02655322231210231
- ISSN: 0265-5322
1477-0946 - Abstract: Rater comments tend to be qualitatively analyzed to indicate raters' application of rating scales. This study applied natural language processing (NLP) techniques to quantify meaningful, behavioral information from a corpus of rater comments and triangulated that information with a many-facet Rasch measurement (MFRM) analysis of rater scores. The data consisted of ratings on 987 essays by 36 raters (a total of 3948 analytic scores and 1974 rater comments) on a post-admission English Placement Test (EPT) at a large US university. We computed a set of comment-based features based on the analytic components and evaluative language the raters used to infer whether raters were aligned to the scale. For data triangulation, we performed correlation analyses between the MFRM measures of rater performance and the comment-based measures. Although the EPT raters showed overall satisfactory performance, we found meaningful associations between rater comments and performance features. In particular, raters with higher precision and fit to what the Rasch model predicts used more analytic components and used evaluative language more similar to the scale descriptors. These findings suggest that NLP techniques have the potential to help language testers analyze rater comments and understand rater behavior.
- Abstract: As Provided
- Publication Date: 2024
- Accession Number: EJ1419103
- Availability:
Contact CCPL
Copyright 2022 Charleston County Public Library Powered By EBSCO Stacks 3.3.0 [350.3] | Staff Login
No Comments.