A research on innovative management of teaching assessment- using I.T. to implement students\' on-line peer review of learning outcomes through rubric

The assessment of a student’s learning outcome in a particular course is aimed not only to evaluate the learning effects but also to actively provide the student with immediate feedback to improve performance. However, the conventional methods for assessment are mostly concerned with the learning effects. In this study, the free software of General Public License is adopted to develop a database platform to let students of different learning teams conduct a peer review online, both quantitatively and qualitatively. With the system, inter and intra team peer reviews are also implemented. The findings help facilitate the conduct of online rubric quantitative and qualitative reviews, regardless of self or mutual ones. A multidimensional evaluation of learning outcome is therefore achieved. More crucially, the finding offer students an opportunity to drill down their learning effects, to understand current learning status, and to be acquainted with the feedback or comments made by peers. It also helps the learners modify their pace as well as strategies to eventually better their learning outcome.


Introduction
The quality of university students are commonly suspected in modern society due to the rapid increase in the number of higher education institution; therefore, the learning outcome and the assessment of students' core abilities have become the main concerns worldwide [1]. To improve the learning outcome through various kinds of strategies and skills has also become a trend for colleges and universities in the management of teaching quality. Education theories have suggested that the assessment of a student's learning outcome(SLO) is aimed not only to give comments on a student's learning progress but also acts as a mechanism to advise how to rectify learning strategies and activities in order to improve the current learning situation.
Conventionally, in school education, teachers are usually the ones who evaluate SLO through a summative assessment. This method is seen too subjective because of lack of unbiased criteria. Also, the teachers might be influenced by personal liking or disliking towards particular students. Furthermore, it ignores the subjectivity and the initiative of the students, which will finally write off their characteristics and creativity in learning [2].
In addition, the traditional method of assessment, in a summative form, gives no feedback to the students for improving the quality of learning. This seems having greatly degraded the main functions and the perspectives of the assessment. Based on the reasons mentioned, in consideration of increasing self-improvement and learning effects of the students, this research develops an online database platform with the aims not only for teachers to easily conduct an assessment that emphasizes equally on both formative and summative evaluations but also for students to carry out intra and inter team peer to peer reviews. Through the process and the results of such an assessment, it is hoped that the students are able to understand their weaknesses in learning and eventually make improvement.

Literature review 2.1 Assessment of students' learning outcomes
From a macro viewpoint to look at the learning outcomes of the students, scholars suggested in 2010 that SLO refers to the knowledge learned in school and the ability to apply these knowledge to various situations after completion of the course or graduation [3]. On the other hand, in any kind of courses, learning outcome mainly focuses on the performance of the students, especially the objectives of the learning and the core abilities achieved after completion of a particular course [4]. To further explain this, SLO can be properly examined after each session of learning. The results are then used both by the teacher to adjust teaching plan and design, and by the students to review their learning strategies and paces.
The assessment of learning outcome is usually classified into direct and indirect ones. The former one reviews a student's knowledge as well as mental state according to his or her learning status, skills, attitude, and behaviour through a set of test design, psychological scale, or questionnaire. Certificate test, in class examination, doing reports on special topics, and capstone course are common tools to assess. The indirect one, on the other hand, utilizes questionnaire, alumni follow-up, and employer's satisfaction with the alumni to evaluate learning outcomes in a long term. It is noticeably that the assessment of learning outcome should not focus merely on the evaluation itself. The ways of teaching, course syllabus, and the methods conducting the assessment should be taken into consideration as well [3]. Due to the discrepancy in the levels of different courses, the assessment of learning outcome should be designed based on the level of the course to make it more specific and targeted in order to avoid evaluation errors. In other words, courses might be categorized into three levels -preliminary, intermediate, and advanced. For preliminary students, their abilities are shown in describing, memorizing, and didactic. The intermediate students should focus their abilities in application, presentation, explanation, analysis, making comments, comparison, and problem-solving. The advanced course, meanwhile, would concentrate on cultivating students' initiative abilities in creativity, planning, administration, and construction.

Rubric
The term 'rubric', a Latin word, originally derived in the 13th century or even earlier days, which refers to the headlines or the highlights of important meanings in religion marked by red ochre or chalk in manuscripts. At that time, red colour pens were only used to make remarks in religious books or the authoritative publications. The term was later defined as a brief and authoritative criterion. Nowadays, 'rubric' is commonly defined as the red mark, the title, the prompt, or the instruction in books or on exam papers. It is also utilized in the process of teaching assessment.
In teaching activities, 'rubric' is a method of assessment that enhances learning activity [5]. It is also a structured evaluating standard with the characteristics of flexibility, strong operability, and oriented process [2]. As rubric was gradually developed into an evaluation tool, it has become a terminology for education assessment, standing for the scoring guide to evaluate students' learning quality in learning process [6]. It usually gives the definition of learning quality and achievement, and specifies the strategies and the standard of the assessment. It is usually shown in a form of a table, enabling teachers to mark and students to plan for future learning [7].

Implementation of paper-based rubric on peer review
To take the researcher of this paper as example, a paper-based rubric was ever conducted for peer to peer reviews. This experience gave an important reference to the development of the database platform online. The details are shown as follows.

Grouping
Prior to the commencement of the class, the students were grouped into different learning teams to prepare together for the assigned report and presentation. The number of each team varied, according to the total number of the participants. The students of the same team were seated adjacently. In this study, seven teams were formed, with four to seven members in each one.

Preparation of paper-based rubric scale
Generally speaking, to conduct a typical rubric assessment, a form of peer to peer review is needed. In the case of the study, three forms were designed, including the ones for 'inter team peer review', 'selfreview', and the 'assessment of the exercise.' The inter team peer review form was used to evaluate the performance of the students on stage. The second one enabled the one in the reporting team to review the performances of him/herself and his or her teammates. The form of 'assessment of the assigned exercise' was used by students of the reporting team to evaluate the achievement of the students out of their team, based on the scores gained in the assigned exercise.
The intra team peer review form is composed of a quantitative scoring assessment and a section for making qualitative comments. The scale for the quantitative scoring assessment is a cross matrix. Within each cell of the matrix, the lowest standard of the score obtained by the subject is recorded, namely 'rubric'. The teacher can design the assessment items in accordance with the goals of each course and create a scale of four or five levels. In the case having been discussed so far, six items of assessment were made, including the content of the report, the application and the example(s) used, the presentation skills, the design of the presentation, the composure on stage, interactivity, and collaboration. Each item was graded based on a five-level scale-'5' for 'excellent', '4' for 'good', '3' for 'fair', '2' for 'needing improvement', and '1' for 'poor'. As for the part of the qualitative comment, each reviewer was required to write down his or her subjective opinion(s) and feeling of the report or suggestions to the subject with at least 20 words.
The self-review form also includes sections of quantitative scoring and qualitative comment. The quantitative scoring form is the same as that of the intra team peer review form. However, the comment part is different. While the reviewer and the subject are different individuals, the comment is made by the reviewer to give the subject pieces of advice. On the contrary, the comment will act like a learning feedback if the reviewer and the subject are the same one. In the case discussed, students were required to write down the self-learning feedback, which should comprise the items such as the favourite part of the course, the most beneficial part received from the course, and the part(s) of the course that need(s) to improve.
The evaluation of students' grades for doing the assigned exercise should be principally included in the content of the 'assessment of the assigned exercise.' In the case, the items on the assessment list included 'the timing used for answering the questions and the correct rate of the answers. Each item was also graded according to the five-level policy.

To convert scores on five-level scale to centesimal grades
In general, centesimal grading policy is widely adopted in Taiwan's education system to evaluate SLO. Therefore, to convert the scores on the five-level scale to centesimal grades is essential. From level one to four, in the case of the study carried out by the researcher, each level was made equivalent to 12 grades in the hundred mark system. Based on this principle, a score between 5.0 and 4.1 is converted to one between 100 and 89.2; between 4.0 and 3.1 calculated as one between 88.0 and 77.2; between 3.0 and 2.1 converted to a score between 76 and 65.2; between 2.0 to 1.1 scored between 64 and 53.2. The score between 1.0 and 0 was converted to the one between 52 and zero.

Calculation of the grade of general SLO
Principally, the ratio of the scores of peer to peer review, self-review, and the exercise work in final grade can be determined either by the teacher or based on the discussion of the teacher with the students. In the previous study, it was firstly proposed by the teacher, and the decision was finally made after discussion with the students. According to the consensus reached, peer to peer review, self-review, and the score for the exercise work counted for 40%, 20%, and 40%, respectively. Fig. 1 illustrates the research architecture of this work. In the research, SLO, the assessment standard of rubric, and literature reviews of several case studies related to the implementation of rubric were firstly examined. Next, a class in which the researcher had ever conducted flipped classroom and rubric assessment was chosen as the object of study. Then, a discussion with the students on how to improve the items on the assessment forms took place after the assessment forms used for the previous rubric were collected and reviewed. After this, a platform and tools for system development were chosen to conduct demand analysis, function design, program coding for the prototype system. The process of testing was finally conducted. In consideration of the development costs as well as the features of the internet, i.e. being able to be used for multiple work by many people, the free software of General Public License (GPL) is adopted to develop a system platform, using the tools of Ubuntu Linux as operation system, MySQL as database server, Apache 2 as web server, and PHP7, Java script, and HTML5 for program coding and the design of operation interface. The structure of the system platform is shown in Fig. 2 Fig. 3 represents the design of the system functions with UML use case diagram. The functions of the platform developed can be divided into two kinds-one for the teacher and the other for the students. As for the teacher's one, it helps the users to assign report team, adjust learning groups, inquire about the team members, the learning quantitative feedback or check the qualitative feedback of peer to peer reviews, and enquire about the grades of the students. The teacher can also calculate the general grades of the students. The later one has the functions of join learning team, peer to peer review( inter review) online, self-review (intra review) online, peer to peer review of the exercise work online, real time inquiry about individual learning outcome, and inquiry about the qualitative feedback from other members. Through this platform, teachers can easily implement rubric online for a real time multiple assessment to examine SLO in both qualitative and quantitative ways. On the other hand, following the completion of group report, students can not only carry out empathic learning and inter group peer reviews but also inquire about the grades of their report or the assigned exercise work. They can also make qualitative comment to themselves and other students.

Procedures and activities in the initial state
The platform can be administered by the computer centre to be used by faculties and students of the whole institution. It can be also kept by any department for its members or by individual teacher for his or her own use. Prior to a new semester, the data of the previous semester will be exported by the administrator from the rear end of the database by using MySQL Workbench for archive. The data in the database will be deleted, too, in order for the teachers to use the system in the coming new semester.
After the names of the teacher and the students are confirmed, the administrator will import the information about the teachers, the courses, and the students into the database in the format of CSV file. The professor's information should include the professor ID, name, the department the professor belongs to, and the password. The file of the courses information includes the titles of the class and course, the ID of professor who offer the course, the course ID, the year and semester the course belongs to. The file of the student information includes the class and course title, the ID, name and password of student who join the course, and the group number jointed. The file formats are shown in following table.

Table 1. Examples of the information in CVS file format to be imported
Professor information format: ID, name, the department belonged, and password Course information format: Class and course titles, professor's ID, course ID, year and the semester Student information format: Class and course titles, the ID, name, password of student, and the group number

Procedure and activities in general status of use
The procedure and activities of using this system in classroom for teacher are described as follows. First, personal account and password have to be entered to log in. Then, using the drop down menu to choose the course title. At the beginning of class, the teacher must require three to five students to form a learning team and login to the system for cooperative study. After the student completes the study team, the teacher assigns the study topic to each study group. After a period of teaching, students need to submit a study experience report. In class, the teacher must specify the day's report team using the 'assign report team' functional menu, then the report team members will be able to come to the stage to make report, while other students will be able to conduct an online peer review for the reporting student based on the evaluation criteria which the teacher previously defined in the system. After the report team completes the report, each student in the report group have to conduct a self-assessment. In addition, the teacher can regroup the students upon request by using the function menu of 'team adjustment', and the function menu of 'team member inquiry', 'review inquiry', 'score inquiry', and 'score summary' are used for checking the member list of each team, the mutual reviews, scores of peer review and exercise work, and the grades of all students for each learning item, respectively.
For the students, initially, they need to be grouped and adjust the seating in classroom in accordance with the teacher's order. Next, they have to be connected with the internet by using PC or mobile device to choose the course participated in. After this, students have to log in the system using their account and passwords. The function of 'join learning team' must be used for the students who use the system for the first time to join any particular team. In classroom, the learning group is divided into two sub groups-the one to report and the one not doing so. The function of 'inter review' is then used by the group not delivering report on stage to grade the students delivering their presentation. A subjective review or comment in at least 20 words will also be given afterwards. On the other hand, in the wake of the presentation and the teacher's comment on the report, the team members of the report group use the 'intra review' function to assess oneself as well as his or her teammates, based on the five-grade scoring criterion.

IMETI 2017
As for the part of self-review, in addition to a quantitative score, a learning feedback and a subjective comment on the teammates, both written in more than 20 words, should be made. Besides, after the exercise questions are revealed and explained, the report team members will assess the work done by the students not reporting by using the function of 'exercise review'. Moreover, through the connection with the system online, students are able to enquire about the grades of peer to peer review, self-review, and the exercise work, as well as the up-to-date summary of their scores. The qualitative comment of oneself is also acquirable.

Data demand analysis
In this system, a professor can instruct several courses in a semester, and the data of the professor is stated in the table of 'professors,' specifying the information such as 'professor_id' (in the form of character with a maximum length of 20 letters), 'name' (character form, less than 12 letters), the 'department' (character form, less than 24 letters), and 'password' of a maximum length of 15 letters.
The course data is stored in the table of 'courses'. Each course must be taught by at least one professor. The 'courses' table should state 'class_course_name' (in characters, not exceeding 45 letters), 'course_id' (in characters with a maximum length of 20 letters), and 'professor_id' (in characters, not exceeding 20 letters). The 'academic_year' is recorded in the format of integer with three digits. The Arabic numeral 1 refers to the first semester and 2 stands for the second semester.
The data of the students is stored in the table of 'students'. Every student must take part in at least one course, and one can keep several grades of intra review, inter review, and the assigned exercise. Specifying the information such as 'student_id' (in characters with a maximum length of 12 letters), 'class_course_name' (in characters, less than 45 letters), 'student_name' (in characters, not exceeding 15 letters), 'password' (in characters, not exceeding 15 letters), and 'group_id' is stated in integer format, with two digits at most.
The score of inter review is stored in the table of 'inter_review_score'. Each score belongs to a particular student, too. Information including 'class_course_name' (in characters with a maximum length of 45 letters), 'intra_review_id' (in integer form with a maximum length of two digits), 'owner_id' (in characters with a maximum length of 12 letters), 'giver_id' (in characters with 12 letters at most), score 'item_1' to 'item_6' (in integers, not exceeding two digits), 'comments' (in characters not exceeding 1,000 words), and 'review_date' (in date format).
The score of student's exercise work is stored in the table of 'exercise_score', and each grade belongs to one student. The data recorded includes 'class_course_name' (in characters with a maximum length of 45 letters), 'exercise_id' (in the form of integer, not exceeding two digits), 'owner_id' (in characters with a maximum length of 12 letters), and 'giver_id' (in characters, with a maximum length of 12 letters). Score 'item 1' and 'item2' are both recorded in integer, with a maximum length of three digits. Finally, 'review_date' is stated in date format.
The score of student's intra review is stored in the table of 'intra_review_score'. Each item must belong to a particular student only. Information including 'class_course_name' (in characters with a maximum length of 45 letters), 'intra_review_id' (in the form of integer with two digits), 'owner_id' (in characters, not exceeding 12 letters), 'giver_id' (in characters with 12 characters at most), score 'item_1' to 'item_6' (in integers, not exceeding two digits), and the learning feedback are recorded in 'comments' field(in characters with a maximum length of 1,000 words), 'review_date' stated with date format.

Database ERD design
Based on the demand analysis of the data, MySQL Workbench is used for the system database ERD design, which is shown below in Fig. 4.

Online test
Following the completion of the prototype system, the researcher, according to the default work flow and the rules of the system, conducted an online platform testing program in a post graduate class. Due to the limited space of the paper, this section illustrates briefly the program modules relations, self and mutual reviews, and the inquiry about scores. Students' feedback is also shown as follows.

Program module relations and function menu
The relations between the program modules operated by the teacher and its linking program modules are illustrated in Fig. 5. Fig. 6 describes the one used by the students. In the centre of the figure is the name of the program module. The ones on the left are the modules linked to the main module, and the right ones are the modules that can be linked to from the operation item.  Fig. 7 is a screenshot on the operation menu for the student after successful logging in, and the user's name is shown on the screen to create a friendly environment. On the platform, the function menu is classified into several items, such as 'Join Learning Team', 'Inter Review', 'Intra Review', 'Exercise Review', 'Reviews Inquiry' and 'Scores Inquiry'. Fig. 8 is a screenshot on the operation menu for the teacher. Similarly, the teacher's name is shown above the logo of the platform after logging in. The function menu for the teacher is categorized into 'Assign Report Team', 'Team Adjusting', inquiry about 'team members', 'student peer reviews', 'individual student scores' as well as 'Score Summary' of all students.

Self and mutual reviews of the students
After the teacher has assigned the current report team, the students are able to conduct mutual and selfreviews on the reports on the platform online. Fig. 9 shows the operation interface for peer to peer mutual review. At the beginning, the reviewer clicks the name of the subject, and the information of the subject, the reviewer's name, and the date will be subsequently shown on the screen. By using the radio button, the reviewer grades the subjects on a five-level scale-excellent, good, fair, needing improvement, and poor(the default grade is fair), based on the content of the report, the application and the example(s) used, the imposing manner and style on stage, the design of the presentation, interaction with audience, and collaboration with team members. Finally, a comprehensive comment at least 20 words is made. Similar to Fig. 9, Fig. 10 is the operation interface for self-review. The major difference is that on this interface learning feedback of at least 20 words are required to be keyed in.

Inquiry examples
The inquiry function of this platform is divided into two groups-the one for teacher and the other one for student. The teacher can inquire individual student's information in different learning team, his or her score received and its scorer, the reviews and the reviewer, and the total scores obtained by the whole students. The students, on the other hand, can find out the reviews and the average score given, and then drill down the original score of each item. Fig.11 is screenshot example to show the results of teacher's inquiring the grades of all students, and Fig.12 is screenshot example of student's inquiring peer reviews.

Feedback opinions
This section makes excerpts from the opinions of three students who ever used the online platform, hoping their ideas serve as references for examining the research purpose and further improvement of the system. Student 2: Using this platform, I could know my scores and the reviews given by other students right away. This helps me do self-review as well. If the number of the participants is small, the effect of this system may weaken. It is easy for us to know who scored us. To benefit each other, we might give a good mark, and the score received/given might be unreal. Student 3: This is a very good evaluation system. We can not only mark and make comments anonymously on the learning outcomes of the students of other teams but also conduct self and intralearning team peer reviews. As for giving reviews, it requires careful listening to each report, or it must be hard for us to write anything about it. From this point of view, I suggest the system be widely used by undergraduate students, which will help them concentrate more in class and let them perceive the views or opinions of others on their works.

Conclusion
Based on this research, it is suggested that learning methods are facilitated through the real time function of intra and inter peer reviews on this platform. By using PC or mobile device to be connected online, they can conduct quantitative intra and inter peer reviews, make qualitative comments and learning feedbacks, inquire about personal score of the report, check the quantitative grade of the exercise, drill down the original score of each report, and acquire the qualitative advice on learning made by others. All of these serve as feedback to self-learning status and provide the students with valuable references to adjust learning strategies and activities, and eventually improve current learning situation.
Through this database platform online, on the other hand, teachers can easily conduct multiple rubric assessments, regardless of intra or inter reviews of the students in either general or computer classroom. They can also quickly examine the scores of each report done by a particular student, drill down in discovery of the original score, check the homework and its score and count the result, and finally understand the learning state of the students and make adjustment to the teaching plan.

Future works
Although the prototype system proposed in this study can achieve online peer to peer review and selfassessment in classroom teaching, however, the function of the system still needs to be continually improved through constant feedback from users, so as to achieve a smooth state more. Therefore, the follow-up work of this research includes promoting the application of this prototype system through different levels of teaching seminars and working with teachers who use the system to continuously collect feedback from teachers and students as a reference for continuous system revision and improvement.