Methodology Article | | Peer-Reviewed

Computing Tutors Performance Indicator in Virtual Learning Environment

Received: 29 January 2024    Accepted: 12 February 2024    Published: 2 April 2024
Views:       Downloads:
Abstract

Quality assurance has become an urgent necessity in all spheres of life, including education and learning in all stages from primary school to higher education, since it constitutes the core of development in the new era of knowledge economy. The aim of this paper is to present a new methodology applied to ensure quality of learning process within virtual learning environments. Our case study focused on the model of quality control adopted by Syrian Virtual University (SVU), which is pioneer in the field of virtual learning in the Middle East and the Arab region. The methodology is based on SET methodology (Students Evaluation Test) and defined by the assessments of each of the four pillars of learning process: (1) Assessment of tutors performed by learners; (2) Assessment of tutors performed by the administration; (3) Assessment of tutors performed by their peers; (4) Self-assessment of tutors. The last two procedures have not been yet implemented within SVU, while other procedures are in continuous development in order to improve their results. The importance of this model comes from its flexibility, its appliance since 2018 in a virtual education establishment (Syrian Virtual University), its possible generalization to any similar environment with virtual academic programs.

Published in Higher Education Research (Volume 9, Issue 2)
DOI 10.11648/j.her.20240902.12
Page(s) 38-44
Creative Commons

This is an Open Access article, distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution and reproduction in any medium or format, provided the original work is properly cited.

Copyright

Copyright © The Author(s), 2024. Published by Science Publishing Group

Keywords

Virtual Learning, E-Learning, On-Line Education, Quality Control in Virtual Learning, Tutors Performance Indicator (TPI), Students Evaluation of Teaching (SET), Syrian Virtual University (SVU)

1. Introduction
Established in 2002, the Syrian Virtual University (SVU) is the first virtual university in the Arab world and the Middle East region. It belongs to the 4 the generation universities . It is the 5th public state university in Syria among 8 state universities and 23 private universities established later on. The learning process in SVU environment is independent of time and space. However, the learning process is similar to traditional learning process where students are able to attend sessions, provide assignments by means of e-learning tools, and follow their evaluation in one of SVU exam centers spread inside Syria and worldwide. The Syrian Virtual University has grown considerably with a steady increase in the number of students starting in 2003 with 110 students registered one academic program and reaching 38,649 students actually registered in three faculties and more than 12 academic programs, ranging from bachelor to master degrees in any field that could be learned virtually. During its journey, the Syrian Virtual University has developed its own working mechanisms to ensure its credibility as an academic institution using the virtual model of learning (E-learning model that simulates numerically all traditional learning components ). Thus, SVU has developed its own digital campus adapted to its needs. It was also a pioneer -since 2002- in developing and using numeric teaching tools of our time (Learning Management Systems, Virtual Classroom, Video Conferencing). Moreover, SVU has developed during two decades its own Assessment Management & Control System that links all SVU Exam centers to the headquarters in Damascus and ensures a strict follow-up and monitoring of its exam centers distributed in 33 different locations in Syria (17 exam centers), and in most Arab countries, in addition to Germany, Russia, Turkey and Austria (16 exam centers). The system allows SVU students to achieve their exams worldwide simultaneously within SVU exam centers. This modern and ambitious learning model, which currently regroup more than 45,000 students and 700 teachers, helped thousands of Syrian students to overcome the problems caused by the war that has ravaged Syria since 2011, and the problems of lockdown caused by COVID-19 pandemic since 2020. However, such learning model needs a quality assurance process that ensures it accreditation comparing with traditional forms of learning, and facing the mistrust declared by some actors of the learning system. The aim of this paper is to present a new methodology applied to ensure quality of learning process within virtual learning environments. Our case study focused on the model of quality assurance adopted by Syrian Virtual University (SVU), which is pioneer in the field of virtual learning in the Middle East and the Arab region. The methodology is defined by four assessments of the four pillars of learning process : (1) Assessment of tutors performed by learners; (2) Assessment of tutors performed by the administration; (3) Assessment of tutors performed by their peers; (4) Self-assessment of tutors. The last two procedures have not been yet implemented, while other procedures are in continuous development in order improve their results. In the following we start by mentioning the research concerns and objectives. We present in the second part the virtual learning environment at Syrian Virtual University. The third part presents the evaluation process in a virtual learning environment. The final one presents the application scale of the system.

2. Research Concerns and Objectives

As a result of its expansion and development, the Syrian Virtual University (SVU) needs to monitor the progress of its learning process and apply quality standards through a set of procedures that are constantly reviewed and improved ensuring reliable virtual learning process. Thus, an integrated quality assurance system is placed to ensure a high academic level correlated to the needs of the target markets and equivalent to traditional learning system in the disciplines that can be delivered virtually. In this paper we introduce the quality control model applied actually at Syrian Virtual University. This model is composed of two components: the first is serving as actual quality control process of the learning system at SVU. The second is under development and could be applied in the near future. The importance of this model comes from its flexibility and the application of the first component since 2020 in a completely virtual environment (Syrian Virtual University), and the possibility to generalize its application to any similar environment with online academic programs.

3. The Virtual Learning Model at SVU

Before proceeding to the presentation of the quality control system at SVU, it is necessary to review the components of the learning process in Syrian Virtual University. The learning process simulates the traditional process of learning (even outperforms it for any academic cursus that could be learned virtually). It consists of six pillars:
1. Learning Content using Learning Management System (LMS) that allows students to use pre-prepared content and enables them to upload activities and participate in forums.
2. Online sessions and offline recorded sessions simulating virtual classroom. This system also allows students to re-attend sessions at any time out of their schedule.
3. Social Network SVU Averroes intended to propose a social learning network for SVU students. The network aims to move the learning process toward new basis of self and cooperative learning to use the concept of social networks.
4. Evaluation mechanisms based on the assessment of students’ activity during online sessions, students’ interaction in SVU social network, and the offline assignments proposed by tutors and delivered by students via LMS.
5. AMS Assessment Management System: where the student submits his exams electronically through multiple-choice questions or essays, and the job and exam are marked. It’s worth noticing, that SVU students are obliged to pass their exam in a fixed exam centers spread in different Syrian cities and worldwide as mentioned in the introduction.
Figure 1. SVU Virtual learning Environment.
4. Evaluation of Learning
The evaluation process is no less important in virtual learning establishment than in traditional learning one. On the contrary, due to the special nature of virtual learning, and the dose of self-learning part (which leads us to insist on “learning” term instead of “education”), and the physical distances separating administration, tutors and students, the application of a more accurate evaluation becomes an urgent necessity . It should also include all parts of the learning process: simultaneous sessions, outclass jobs, exams, and others.
The virtual learning environment does not depend entirely on simultaneous sessions, as it is a self-learning environment for the student with the help of scientific content that must be prepared interactively and thoughtfully, which led to the emergence of new factors affecting the assessment process.
Virtual learning is distinguished by preserving all the movements that take place in all corners of the learning process within its electronic systems, which helps in the assessment process to a large extent. It opens new horizons for those in charge of the subject of assessment and quality testing that can be used at any time.
5. Quality Assurance in a Virtual Learning Environment
Due to its virtual nature, SVU adopt a new form of quality assurance for its learning process based on the same rules adopted in any traditional learning process with major modifications related to the nature of virtual learning. A previous study was conducted dealing with SET (Students Evaluation of Teachers) and the different factors related to. In order to reach comprehensive value for teacher performance assessment we move forward in the present research to new process of evaluation of teaching’s quality based on semester measurement with 4 components (Student evaluation of teachers; Administration evaluation of teachers; Peer evaluation; and Self-evaluation). A new indicator is computed called Tutor Performance Indicator (TPI).

5.1. TPI Computing Methodology

TPI is a multidimensional indicator which reflects a combination of the student assessment related teacher’s performance and teacher’s commitment to good practices in virtual teaching (Online and offline Sessions management; Course management; Student’s follow-up; and Exams). The value of this indicator varies between 0 (worst) and 100 (best).
The objective of TPI’s measuring is to improve tutoring process at Syrian Virtual University by monitoring tutor’s performance. This indicator certainly has secondary benefits for program directors and SVU administration regarding the sustainability of tutor’s contract with SVU.
The information for good practice is generally taken electronically except for the control of sessions content since their monitoring is difficult to evaluate without human intervention. The decision concerning the weight of each practice is determined each year by the quality control team and University's Board of Directors.
The performance score is computed for each teacher where one could get a very good evaluation mark by achieving an average of high student assessment and high internal assessment. Thus, a teacher with an outstanding teaching performance can have a low evaluation mark if he/she commits insufficient good practices in course management, session management, or the like.
To compute tutor’s performance indicator for each tutor (TPI), five independent teams composed of staff members and quality consultants have special authorizations to extract the necessary data in order to build the following five evaluation reports. Each report contains the measurement of multiple sub indicators used for the computation of Tutor’s TPI:
Table 1. Reports and Sub-indicators.

Report

Data Source

Coverage

Period

Report Author

Supervisor

Decision Authority

CME

LMS

All tutors

Before exams

Program director’s assistant

QA Supervisor

Scientific affairs council

SME

Sessions

All tutors

Before exams

QA consultant

QA Supervisor

Scientific affairs council

EXM

Exams

All tutors

After exams

Exam director

QA Supervisor

Scientific affairs council

SET1

Questionnaire

All tutors

Middle of semester

Survey system admin

QA Supervisor

Scientific affairs council

SET2

Questionnaire

All tutors

End of semester

Survey system admin

QA Supervisor

Scientific affairs council

QNL

Virtual class

Samples

End of semester

QA consultant

QA Supervisor

University Council

CMP

All Sources

Samples

After request

QA Supervisor

SVU’s Vice President

University Council

*CME: Classes Management Evaluation
*CMP: Complaints
It’s worth noting that the period of report issuing and publishing is fixed to increase the credibility of the report especially for those who depends on students’ evaluation of tutor’s performance.

5.2. Measurement Sub Indicators

The list of measurement sub indicators varies from one semester to the other as some negative practices completely disappear and new negative practices emerge. However, the following is a list of the practices observed since 2018 . These sub indicators represent good and bad practices are mentioned in the following tables with the weight of each one. This list of items is adopted by SVU administration during academic year 2022/2023.
Table 2. Sub indicators related to courses’ management (CME).

#

Items to be measures

Weight

Organized work plan delivered to students at the beginning of the semester

10

Respect of the criteria mentioned in cursus’ definition documents

10

LMS course’s page is well structured, complete and follows SVU and quality matters standards

5

Detailed correction scheme of assignments is delivered to students

5

The correction scale of the assignment is uploaded and well explained

5

Follow-up of students’ complaint about assignment correction at time

5

Total

40

Table 3. Sub indicators related to sessions’ management (SME).

#

Items to be measures

Weight

Periodicity and Commitment.

10

Sessions’ schedule and organization (delivery and upload time plan)

10

Number of attended students

5

Total

25

Table 4. Sub indicators related to exams’ management (EXM).

#

Items to be measures

Weight

Comprehensive and well-structured exam

10

Assessment respects the criteria mentioned in cursus’ definition documents

15

Questions’ editing

5

Questions’ uploading and formatting

5

Total

35

Table 5. Sub indicators related to online activities (ONL).

#

Items to be measures

Weight

Interactivity

20

Using good practices of presentations

15

Assessment respects the criteria mentioned in cursus’ definition documents

15

Total

50

Table 6. TPI Computation.

#

Report

Publishing date

Grade

CME

Before exams’ period

40

SME

Before exams’ period

25

EXM

Ater exams’ period

35

ONL

Before exams’ period

25

SET1

Before exams’ period

25

SET2

After exams’ period

50

Total

200

6. Results
During the last 5 years 420 tutors are evaluated each semester (twice by year) using the above-described. The evaluation is performed by a team composed of 16 members who constitutes the quality control service at SVU.
The data is generated by 24 academic programs running actually at SVU. 8 bachelor programs delivered to approximatively 17000 students in total where each program contains in average 50 courses; and 16 master programs delivered to approximatively 6000 students in total where each program contains in average 50 courses.
Each semester, a global report is generated describing bad and good practices and making suggestion to ameliorate the quality of service and proposing advices to adjust the tutoring practices.
A global guide of quality control measurement Is published and updated each semester at
https://www.svuonline.org/en/quality/quality-assurance
A list of best tutors is published on SVU’s website in a special page such as the one published actually:
https://www.svuonline.org/en/quality/f22-top-tutors
Moreover, a training is delivered to tutors who had high percentage of bad practices (with grades less than 100/200), in order to adjust their errors. This gives any tutor the opportunity to continue as member of SVU staff.
7. Discussion
We present in this paper a new methodology for computing tutors performance indicator for virtual learning environment. The methodology is based on the computation of SET (Student’s evaluation of teaching) in traditional learning. However, our methodology tries to adapt the SET’s model of computation in order to be suitable for virtual learning. SET is computed generally regarding to 4 components: (1) Assessment of tutors performed by learners; (2) Assessment of tutors performed by the administration; (3) Assessment of tutors performed by their peers; (4) Self-assessment of tutors. We adapt some components from the parts (1) and (2) to become suitable in the case of virtual learning environment during the computation of tutors’ performance indicator at Syrian Virtual University; However, many difficulties arises during this adaptation related to the lake of direct physical contact between tutors and students during the online sessions which could be held without video conferencing using the white board with voice interaction. This lake of video contact could influence some factors during students’ judgment. An improvement of such factors could be done by forcing the use of video conferencing during few online sessions of the course which could be impractical in virtual learning. In the near future, we should improve the (TPI) by adding the two last components of the evaluation in order to establish a complete functional model of TPI evaluation for distant and virtual learning.
8. Conclusions
Computing Tutors Performance Indicator has become a necessity for evaluating the quality of courses in any academic institution. The importance of such indicators increases in n institution based on virtual learning since all contacts between students and tutors, and between students and the administration become online and distant. Thus, it is crucial to develop a new methodology to evaluate such indicator in such a virtual environment. This development is based on the methodology of TPI computation in a traditional learning environment and adapted to the virtual learning model since all the component of the model are virtual.
Abbreviations
SVU: Syrian Virtual University
SET: Students Evaluation of Teaching
TPI: Tutors Performance Indicator
Author Contributions
Khalil Ajami is the sole author. The author read and approved the final manuscript.
Conflicts of Interest
The authors declare no conflicts of interest.
References
[1] Herbert W. Marsh, Lawrence A. Making students' evaluations of teaching effectiveness effective: The critical issues of validity, bias, and utility. American Psychologist 52(11), November 1997,
[2] Badri, M. A., Abdulla, M., Kamali, M. A., & Dodeen, H. Identifying potential biasing variables in student evaluation of teaching in a newly accredited business program in the UAE. International Journal of Educational Management, January 2006. 20(1), 43-59.
[3] Clifford Nowell, Lewis R. Gale, Bruce Handley John B. Goddard. Assessing faculty performance using student evaluations of teaching in an uncontrolled setting. Assessment & Evaluation in Higher Education 35(4): 463-475. July 2010.
[4] Stuart Palmer, (2011), An institutional study of the influence of ‘onlineness’ on student evaluation of teaching in a dual mode Australian university. 28th Annual Conference of the Australasian Society for Computers in Learning in Tertiary Education. December 2011.
[5] Mary R. Hedges, Don J. Webber, Using student evaluations to improve individual and department teaching qualities. Pages 323-339. Research in Post-Compulsory Education, Published online, 07 July 2014.
[6] Alauddin, Mohammad, Does the student evaluation of teaching instrument Really measure instructors' teaching effectiveness? An econometric analysis of Students’ perceptions in economics courses. Economic Analysis and Policy 44(2), July 2014.
[7] Khalil, Ajami, Ola, Haidar. Applying students’ evaluation of teaching (SET) in virtual learning environment (Case Study: Syrian Virtual University). International Journal of Teaching and Education (IJOTE) 6, 2 (2018), 24–45.
[8] Khalil Ajami. and Ola Haidar. SVU-VOSE: A hybrid model of a virtual, open, and socially driven learning environment. eLearn Magazine Issue 5, page 10, 2019, ACM New York, NY, USA EISSN: 1535-394X,
[9] Esther Huertas & al. Considerations for quality assurance of e-learning provision. Report from the ENQA Working Group VIII on quality assurance and e-learning. European Association for Quality Assurance in Higher Education AISBL 2018, Brussels. ISBN 978-952-5539-88-2 (web publication), ISSN 1458-1051.
[10] Ron Oliver. Exploring benchmarks and standards for assuring quality online teaching and learning in higher education. Proceedings of the 16th Open and Distance Learning Association of Australia Biennial Forum. Canberra Australia. 2003.
[11] Pawlowski, J. M. The Quality Adaptation Module: Adaptation of the Quality Standard ISO-IEC 19796-1 for Learning, Education and Training. Educational Technology and Society, 10(2), 3-16. 2007.
[12] Quality Matters (QM) Rubric Standards 2011-2013 edition.
[13] Shattuck, K., & Diehl, W. C (2011). Scholarly research that informed and supported the development of the 2011-2013 Quality Matters in Higher Education Rubric.
[14] Alla Lapteva, Valerii Efimov. New Generation of Universities. University 4.0. Journal of Siberian Federal University. Humanities & Social Sciences 11 (2016 9) 2681-2696. November 2016.,
[15] Ceren Çubukçu Çerasi. DIGITAL TRANSFORMATION TO UNIVERSITY 4.0: A ROADMAP. Uludağ University Journal of The Faculty of Engineering. 2020.
[16] Creative Commons organization, “Creative commons licenses and Tools (CC) Copyright 2019”. Available from
Cite This Article
  • APA Style

    Ajami, K. (2024). Computing Tutors Performance Indicator in Virtual Learning Environment . Higher Education Research, 9(2), 38-44. https://doi.org/10.11648/j.her.20240902.12

    Copy | Download

    ACS Style

    Ajami, K. Computing Tutors Performance Indicator in Virtual Learning Environment . High. Educ. Res. 2024, 9(2), 38-44. doi: 10.11648/j.her.20240902.12

    Copy | Download

    AMA Style

    Ajami K. Computing Tutors Performance Indicator in Virtual Learning Environment . High Educ Res. 2024;9(2):38-44. doi: 10.11648/j.her.20240902.12

    Copy | Download

  • @article{10.11648/j.her.20240902.12,
      author = {Khalil Ajami},
      title = {Computing Tutors Performance Indicator in Virtual Learning Environment
    },
      journal = {Higher Education Research},
      volume = {9},
      number = {2},
      pages = {38-44},
      doi = {10.11648/j.her.20240902.12},
      url = {https://doi.org/10.11648/j.her.20240902.12},
      eprint = {https://article.sciencepublishinggroup.com/pdf/10.11648.j.her.20240902.12},
      abstract = {Quality assurance has become an urgent necessity in all spheres of life, including education and learning in all stages from primary school to higher education, since it constitutes the core of development in the new era of knowledge economy. The aim of this paper is to present a new methodology applied to ensure quality of learning process within virtual learning environments. Our case study focused on the model of quality control adopted by Syrian Virtual University (SVU), which is pioneer in the field of virtual learning in the Middle East and the Arab region. The methodology is based on SET methodology (Students Evaluation Test) and defined by the assessments of each of the four pillars of learning process: (1) Assessment of tutors performed by learners; (2) Assessment of tutors performed by the administration; (3) Assessment of tutors performed by their peers; (4) Self-assessment of tutors. The last two procedures have not been yet implemented within SVU, while other procedures are in continuous development in order to improve their results. The importance of this model comes from its flexibility, its appliance since 2018 in a virtual education establishment (Syrian Virtual University), its possible generalization to any similar environment with virtual academic programs.
    },
     year = {2024}
    }
    

    Copy | Download

  • TY  - JOUR
    T1  - Computing Tutors Performance Indicator in Virtual Learning Environment
    
    AU  - Khalil Ajami
    Y1  - 2024/04/02
    PY  - 2024
    N1  - https://doi.org/10.11648/j.her.20240902.12
    DO  - 10.11648/j.her.20240902.12
    T2  - Higher Education Research
    JF  - Higher Education Research
    JO  - Higher Education Research
    SP  - 38
    EP  - 44
    PB  - Science Publishing Group
    SN  - 2578-935X
    UR  - https://doi.org/10.11648/j.her.20240902.12
    AB  - Quality assurance has become an urgent necessity in all spheres of life, including education and learning in all stages from primary school to higher education, since it constitutes the core of development in the new era of knowledge economy. The aim of this paper is to present a new methodology applied to ensure quality of learning process within virtual learning environments. Our case study focused on the model of quality control adopted by Syrian Virtual University (SVU), which is pioneer in the field of virtual learning in the Middle East and the Arab region. The methodology is based on SET methodology (Students Evaluation Test) and defined by the assessments of each of the four pillars of learning process: (1) Assessment of tutors performed by learners; (2) Assessment of tutors performed by the administration; (3) Assessment of tutors performed by their peers; (4) Self-assessment of tutors. The last two procedures have not been yet implemented within SVU, while other procedures are in continuous development in order to improve their results. The importance of this model comes from its flexibility, its appliance since 2018 in a virtual education establishment (Syrian Virtual University), its possible generalization to any similar environment with virtual academic programs.
    
    VL  - 9
    IS  - 2
    ER  - 

    Copy | Download

Author Information