 |
JACK - A System for Computer-Aided Assessments and Exercises
JACK is a server-based system for the management of computer-aided assessments and exercises and for automated grading. The current version 2.0 is based on a flexible architecture that is capable of handling different kinds of exercises, checking mechanisms and presentation frontends. At the moment, JACK is used for checking Java exercises by means of static and dynamic tests and for visualizing data structures from exercise solutions. Extensions for other programming languages, design artifacts like UML diagrams or general assessment techniques like multiple choice are planned for future versions.
News about JACK
- JACK completed it's sixth year of service in winter term 2011/12 and usage was higher than ever before! Programming exams with over 400 students have been conducted and a total amount of almost 12'000 solutions of programming exercises have been checked.
- Two papers dealing with different aspects of JACK have been accepted for ITiCSE 2011 in Darmstadt.
- JACK demo server is back online at http://jack-demo.s3.uni-due.de/.
Features
- Teacher's interface for managing exercises, setting up examinations and reviewing results.
- Free student's access by matriculation number for self-training exercises and result review.
- Personalized student's access via generated TANs for examinations with designated tasks for each student.
- Extensive automated checks for each submitted solution with textual feedback for each detected flaw in programming exercises.
- Flexible Java exercises with custom code templates, custom code snipplets that may not be changed and requests for source files that are not provided as template.
- Statistics for submitted solutions, participating students and different results.
- Search by matriculation number or results.
- Eclipse-plugin for easy access to Java exercises and easy setup of computer pools and system environments for examinations.
Technology
JACK is a module based application designed for JBOSS application servers. An intelligent architecture ensures reliable and fast service in all situations:
- A frontend and database server offers the management and submitting service and stores everything immediately in a PostgresSQL database.
- A backend system based on OSGi performs the automated checks for Java exercises in a separate sandbox JVM, so no malicious solution can tamper with the system and destroy stored data. Moreover, no problems with the backend can disturb students from uploading solutions via the frontend server, so the proper operation of the system during assessment sessions can be assured.
- Additional backend instances can be used in parallel without restrictions to process more results at the same time.
Checks for Java exercises are performed with different technologies to be as precises as possible and to give valuable textual feedback to the students:
- Static checks based on the abstract syntax graph of an submitted solution check e.g. for correct use of inheritance structures, local loop variables or recursive calls.
- Static checks are rule based by means of graph transformation and may perform preparation steps to handle syntactically different but semantically equal statements.
- Dynamic checks execute tests with defined inputs to check for correct output.
- Infinite loops and recursions in solutions are canceled automatically in dynamic checks. Typical exceptions are caught and included in the list of detected flaws with additional comments.
In both cases, each rule and each test can be provided with an individual comment and explanation for the feedback to the students. Teachers can review the results and add manual results and comments when JACK made a wrong decision.
Experiences
JACK is used at the University of Duisburg-Essen since 2006 to support lectures on programming, algorithms and data structures for first year students. Attestations with more than 400 participants were successfully organized and more than 31'000 solutions have been checked so far.
The time needed for manual preparation and post processing of exercises and attestations could be reduced to a minimum by using JACK. Hence, more time could be invested in high quality and appealing exercises, that motivate students engage themselves in Java programming. At least as much or even more than any manual correction or tutorial, direct automated feedback on the self-training server motivated the students to submit solutions and correct detected flaws. More than 75% of all students using JACK accepted the automated feedback as helpful and 89% voted for more e-learning opportunities in our lectures.
Assessment sessions for programming exercises could easily be arranged by using the Eclipse-plugin and TAN login in a very restricted system environment that prevented any attempt of deception or manipulation.
Screenshots
 | Auto-generated feedback for a programming exercise in the student's view. |
 | Detailed result statistics for an exercise in the teacher's view. |
 | Visualization of a data structure as created by JACK. |
Publications
- Michael Striewe, Michael Goedicke: Studentische Interaktion mit automatischen Pruefungssystemen, in: Proceedings of "DeLFI 2011 - Die 9. E-Learning Fachtagung Informatik", Dresden, 2011
- Michael Striewe, Michael Goedicke: Using Run Time Traces in Automated Programming Tutoring, in: 16th Annual Conference on Innovation and Technology in Computer Science Education, Darmstadt, pages 303-307, 2011
- Michael Striewe, Michael Goedicke: Automated Checks on UML Diagrams, in: 16th Annual Conference on Innovation and Technology in Computer Science Education, Darmstadt, pages 38-42, 2011
- Michael Striewe, Michael Goedicke: Feedback-Moeglichkeiten in automatischen Pruefungssystemen, in: Proceedings of "DeLFI 2010 - Die 8. E-Learning Fachtagung Informatik", Duisburg, pages 85-96, 2010
- Michael Striewe, Michael Goedicke: Visualizing Data Structures in an E-Learning System, in: Proceedings of the 2nd International Conference on Computer Supported Education (CSEDU) 2010, Valencia, Spain, volume 1, pages 172-179, 2010
- Michael Striewe, Michael Goedicke: Effekte automatischer Bewertungen fuer Programmieraufgaben in Uebungs- und Pruefungssituationen, in: Proceedings of "DeLFI 2009 - Die 7. E-Learning Fachtagung Informatik", Berlin, pages 223-234, 2009
- Michael Striewe, Moritz Balz, Michael Goedicke: A Flexible and Modular Software Architecture for Computer Aided Assessments and Automated Marking, in: Proceedings of the 1st International Conference on Computer Supported Education 2009, Lisboa, Portugal, volume 2, pages 54-61, 2009
- Michael Goedicke, Michael Striewe, Moritz Balz: Computer Aided Assessments and Programming Exercises with JACK, ICB Report No 28, Institute for Computer Science and Business Information Systems, University of Duisburg-Essen, 2008
- Carsten Koellmann, Michael Goedicke: A Specification Language for Static Analysis of Student Exercises, in: Proceedings of the 23rd International Conference on Automated Software Engineering, Aquila, Italy, 2008
- Carsten Koellmann, Michael Goedicke: Automation of Java Code Analysis for Programming Exercises, in: Proceedings of the 3rd International Workshop on Graph based Tools (GraBaTs), co-located at the 3rd International Conference on Graph Transformation (ICGT). Natal, Brazil, 2006
Students' Theses and Projects in the Context of JACK
- Martin Filipczyk: Heuristische Erkennung von Endlosschleifen in laufenden Programmen, Project Work, finished in 2012
- Sebastian Okon: Visualisierung von Programmtraces in einem eLearning-System, Bachelor Thesis, finished in 2011
- Alexander Jung: Bewertung der Eignung von Software-Metriken für den Einsatz in einem automatisierten Prüfsystem, Bachelor Thesis, finished in 2011
- Zongyuan Li: Semi-Automatische Generierung von Pruefregeln fuer ein E-Learning-Werkzeug, Bachelor Thesis, finished in 2011
- Martin Filipczyk: Mustersuche in graphbasierten Repräsentationen von Programmen , Bachelor Thesis, finished in 2010
- Mobasher Ullah: Visualisierung von Objektstrukturen aus Übungsaufgaben zur Programmierung, Bachelor Thesis, finished in 2009
- Alexander Trops: Manipulation von Java-Quellcode durch Transformation des Syntaxgraphen zur Verbesserung des dynamischen Model Checking, Master Thesis, finished in 2008
Contact Information
|
