|
Citation
|
Download
|
|
|
The Main Papers and the Assistments Project
|
|
The following paper describes two main results about the Assistment's being good at
1) effective at promoting learning and 2) also at reliable at assessing.
This is the first paper to read about the Assistment project.
|
|
Razzaq, L., Feng, M., Nuzzo-Jones, G., Heffernan, N.T., Koedinger, K. R., Junker, B., Ritter, S., Knight, A., Aniszczyk, C., Choksey, S., Livak, T., Mercado, E., Turner, T.E., Upalekar. R, Walonoski, J.A., Macasek. M.A., Rasmussen, K.P. (2005). The Assistment Project: Blending Assessment and Assisting. In C.K. Looi, G. McCalla, B. Bredeweg, & J. Breuker (Eds.) Proceedings of the 12th International Conference on Artificial Intelligence In Education, 555-562. Amsterdam: ISO Press.
|
[ PDF ] [ DOC ]
[ Visual Presentation ]
|
| | |
|
The following book chapter gives an overview of the whole project, introducing system architechture, builder, content development and usage, reporting system and data analysis.
|
|
Razzaq, Feng, Heffernan, Koedinger, Nuzzo-Jones, Junker, Macasek, Rasmussen, Turner & Walonoski (2007).
Blending Assessment and Instructional Assistance. In Nadia Nedjah, Luiza deMacedo Mourelle, Mario Neto Borges and Nival Nunesde Almeida (Eds).
Intelligent Educational Machines within the Intelligent Systems Engineering Book Series . pp.23-49.
(see http://www.isebis.eng.uerj.br/).
Springer Berlin / Heidelberg.
|
[ PDF ] [ DOC ]
|
|
|
|
Assessment Results of Predicting State Test Scores
|
| | |
|
The following papers address the assessment challenge in the Assistment system and describes our effort to improve the precision of our prediction of student's test scores.
We showed in these papers that we can improve our prediction of student's MCAS score by utilizing the amount of assistance they need in the system and by tracking students' learning over time.
|
|
Feng, M., Heffernan, N.T., & Koedinger, K.R. (2006a). Addressing the Testing Challenge with a Web-Based E-Assessment System that Tutors as it Assesses. Proceedings of the Fifteenth International World Wide Web Conference. pp. 307-316. New York, NY: ACM Press. 2006. ISBN:1-59593-332-9. Nominated for "Best Student Paper".
|
[ PDF ] [ DOC ]
|
|
Feng, M., Heffernan, N.T., & Koedinger, K.R. (2006b). Predicting state test scores better with intelligent tutoring systems: developing metrics to measure assistance required. In Ikeda, Ashley & Chan (Eds.). Proceedings of the 8th International Conference on Intelligent Tutoring Systems. Springer-Verlag: Berlin. pp. 31-40. 2006.
|
[ PDF ] [ DOC ]
|
|
Anozie N., & Junker B. W. (2006). Predicting end-of-year accountability assessment scores from monthly student records in an online tutoring system. In Beck, J., Aimeur, E., & Barnes, T. (Eds). Educational Data Mining: Papers from the AAAI Workshop. Menlo Park, CA: AAAI Press. pp. 1-6. Technical Report WS-06-05.
|
[ PDF ] [ DOC ]
|
|
Ayers E., & Junker B. W. (2006). Do skills combine additively to predict task difficulty in eighth grade mathematics? In Beck, J., Aimeur, E., & Barnes, T. (Eds). Educational Data Mining: Papers from the AAAI Workshop. Menlo Park, CA: AAAI Press. pp. 14-20. Technical Report WS-06-05.
|
[ PDF ] [ DOC ]
|
|
|
The following paper summarizes several related works on the assessment aspect of the project.
|
|
Junker, B. W. (2006). Using On-line Tutoring Records to Predict End-of-Year Exam Scores: Experience with the ASSISTments Project and MCAS 8th Grade Mathematics. To appear in Lissitz, R. W. (Ed.), Assessing and modeling cognitive development in school: intellectual growth and standard settings. Maple Grove, MN: JAM Press.
|
[ PDF ] [ DOC ]
|
|
|
The ASSISTment system tracks 98 different math skills. These papers, using two different modeling methods, show that using finer-grained skill model allow for better predictions of students real MCAS scores.
|
|
Pardos, Z. A., Heffernan, N. T., Anderson, B., & Heffernan C. (2006). Using Fine-Grained Skill Models to Fit Student Performance with Bayesian Networks. Workshop in Educational Data Mining held at the 8th International Conference on Intelligent Tutoring Systems. Taiwan. 2006.
|
[PDF] [DOC]
|
|
Feng, M., Heffernan, N. T., Mani, M., & Heffernan, C. (2006). Using Mixed-Effects Modeling to Compare Different Grain-Sized Skill Models. In Beck, J., Aimeur, E., & Barnes, T. (Eds). Educational Data Mining: Papers from the AAAI Workshop. Menlo Park, CA: AAAI Press. pp. 57-66. Technical Report WS-06-05. ISBN 978-1-57735-287-7.
|
[PDF] [DOC]
|
|
Pardos, Z. A., Heffernan, N. T., Anderson, B. & Heffernan, C. (2007). The effect of model granularity on student performance prediction using Bayesian networks. The International User Modeling Conference 2007.
|
[PDF] [DOC]
|
|
Pardos, Z., Feng, M. & Heffernan, N. T. & Heffernan-Lindquist, C. (2007) Analyzing fine-grained skill models using bayesian and mixed effect methods. In Luckin & Koedinger (Eds.) Proceedings of the 13th Conference on Artificial Intelligence in Education. IOS Press. pp. 626-628.
|
[PDF] [DOC]
|
|
Feng, M. & Heffernan, N. T. (2007). Assessing Students’ Performance: Item Difficulty Parameter vs. Skill Learning Tracking. Paper presented at the National Council on Educational Measurement 2007 Annual Conference, Chicago.
|
[PDF] [DOC ]
|
|
|
Related Papers
|
|
These papers describe some learning experiments we ran in the system.
|
|
Razzaq, L., Heffernan, N.T. (2006). Scaffolding vs. hints in the Assistment System. In Ikeda, Ashley & Chan (Eds.). Proceedings of the 8th International Conference on Intelligent Tutoring Systems. Springer-Verlag: Berlin. pp. 635-644. 2006.
|
[ PDF ] [ DOC ]
|
|
Razzaq, L., Heffernan, N.T., Lindeman, R.W. (2007). What level of tutor interaction is best?. In Luckin & Koedinger (Eds) Proceedings of the 13th Conference on Artificial Intelligence in Education. Amsterdam, Netherlands: IOS Press.
|
[ PDF ] [ DOC ]
|
|
|
These papers describe the reports that teacher can get from the system.
|
|
Feng, Mingyu, Heffernan, N.T. (2005). Informing Teachers
Live about Student Learning: Reporting in the Assistment
System. the 12th International Conference on Artificial
Intelligence in Education 2005 Workshop on Usage
Analysis in Learning Systems, 2005, Amsterdam.
|
[ PDF ] [ DOC ]
|
| Feng, M., Heffernan, N.T. (2006). Informing Teachers Live about Student Learning: Reporting in the Assistment System. To be published in Technology, Instruction, Cognition, and Learning Journal Vol. 3. Old City Publishing, Philadelphia, PA. 2006. |
[ PDF ] [ DOC ]
|
| Feng, M. & Heffernan, N. (2007). Towards Live Informing and Automatic Analyzing of Student Learning: Reporting in ASSISTment System. Journal of Interactive Learning Research. 18 (2), pp. 207-230. Chesapeake, VA: AACE. |
[ PDF ] [ DOC ]
|
|
|
The paper and poster below describes the architecture of the Assistment System.
|
|
Nuzzo-Jones, G., Walonoski, J.A., Heffernan, N.T., Livak, T. (2005). The eXtensible Tutor Architecture: A New Foundation for ITS. In C.K. Looi, G. McCalla, B. Bredeweg, & J. Breuker (Eds.) Proceedings of the 12th International Conference on Artificial Intelligence In Education, 902-904. Amsterdam: ISO Press.
|
[ PDF ] [ DOC ]
[ Visual Presentaion ]
|
|
Nuzzo-Jones, G., Walonoski, J.A., Heffernan, N.T., Livak, T. (2005). The eXtensible Tutor Architecture: A New Foundation for ITS.
In Proceedings of the 12th International Conference on Artificial Intelligence in Education 2005
Workshop on Adaptive Systems for Web-Based Education: Tools and Reusability, Amsterdam
|
[ PDF ] [ DOC ]
|
|
|
The paper and poster below are about the "Builder" tool that teacher's can use to create Assistments quickly and easily.
|
|
Turner, T.E., Macasek, M.A., Nuzzo-Jones, G., Heffernan, N..T, Koedinger, K. (2005). The Assistment Builder: A Rapid Development Tool for ITS. In C.K. Looi, G. McCalla, B. Bredeweg, & J. Breuker (Eds.) Proceedings of the 12th International Conference on Artificial Intelligence In Education, 929-931. Amsterdam: ISO Press.
|
[ PDF ] [ DOC ]
[ Visual Presentation ]
|
|
Turner, T.E., Macasek, M.A., Nuzzo-Jones, G., Heffernan, N..T, Koedinger, K. (2005). The Assistment Builder: A Rapid
Development Tool for ITS. Poster in the 12th International Conference on Artificial Intelligence in Education 2005
Workshop on Adaptive Systems for Web-Based Education: Tools and Reusability, Amsterdam
|
[ PDF ] [ DOC ]
[ Visual Presentaion ]
|
|
Heffernan N.T., Turner T.E., Lourenco A.L.N., Macasek M.A., Nuzzo-Jones G., Koedinger K.R., The ASSISTment Builder: Towards an Analysis of Cost Effectiveness of ITS creation, Accepted by FLAIRS2006, Florida, USA (2006).
|
[ PDF ] [ DOC ]
[ Visual Presentation ]
|
|
|
The papers below is about the detection and prevention of off-task "gaming" behavior in intelligent tutoring systems.
|
|
Walonoski, J., Heffernan, N.T. (2006). Detection and Analysis of Off-Task Gaming Behavior in Intelligent Tutoring Systems. In Ikeda, Ashley & Chan (Eds.). Proceedings of the 8th International Conference on Intelligent Tutoring Systems. Springer-Verlag: Berlin. pp. 382-391. 2006
|
[ PDF ]
[ Visual Presentation ]
|
|
Walonoski, J., Heffernan, N. T. (2006). Prevention of Off-Task Gaming Behavior in Intelligent Tutoring Systems. In Ikeda, Ashley & Chan (Eds.). Proceedings of the 8th International Conference on Intelligent Tutoring Systems. Springer-Verlag: Berlin. pp. 722-724. 2006
|
[PDF]
|
|
|
These two paper talks about methods of quickly tagging questions
with skills based upon the words in the questions.
|
|
Carolyn Rose, Pinar Donmez, Gahgene Gweon, Andrea Knight, Brian Junker, William Cohen, Kenneth Koedinger, Neil Heffernan (2005).
Automatic and Semi-Automatic Skill Coding With a View Towards Supporting On-Line Assessment. the 12th International Conference on Artificial Intelligence in Education 2005, Amsterdam
|
[ PDF ] [ DOC ]
|
| Kardian, K., Heffernan, N.T. (2006). Knowledge Engineering for Intelligent Tutoring Systems: Assessing Semi-automatic Skill Encoding Methods. In Ikeda, Ashley & Chan (Eds.). Proceedings of the 8th International Conference on Intelligent Tutoring Systems. Springer-Verlag: Berlin. pp. 735-737 |
[PDF] [DOC]
|
|
|
This paper describes an attempt to allow the Assistment to grade a student's "effort" by paying attention to how many hints they
ask for, how quick they respond to an item, and what skill we think they have mastered.
Feng, M., Heffernan, N.T., & Koedinger, K.R. (2005). Looking for Sources of Error in Predicting Student's Knowledge. In Beck. J (Eds). Educational Data Mining: Papers from the 2005 AAAI Workshop. Menlo Park, California: AAAI Press. pp. 54-61 Technical Report WS-05-02. ISBN 978-1-57735-238-9
|
[ PDF ] [ DOC ]
|
|
Other related publications
|
|
The proposal for a symposium at NCME Annual Conference 2007 (Accepted)
|
|
Junker B.(2007). On-demand learning-embedded benchmark assessment using classroom-accessible technology
|
[ PDF ] [ DOC ]
|
|
|
The following paper presented a preliminary study trying to find out if bugs tranfer between schools.
|
|
Weitz, R., Heffernan, N. T., Kodaganallur, V. & Rosenthal, D. (2007). The distribution of student errors across schools: An initial study. In Luckin & Koedinger (Eds.) Proceedings of the 13th Conference on Artificial Intelligence in Education. IOS Press. pp 671-673.
|
[PDF] [DOC ]
|
|
|
