Usefulness of feedback in e-learning from the students’ perspective

Purpose: Functionality of feedback in pedagogical processes has been broadly analyzed in face-to-face learning, although to a lesser extent than in the on-line learning. Narciss (2004, 2008) distinguishes two dimensions within the feedback, the semantic dimension and the structural dimension. This article aims to analyze, from the student’s perspective, the semantic dimension of feedback in a virtual learning environment (VLE). Firstly, we analyze the importance that VLE students give to feedback and its degree of personalization. Later, the usefulness that students deem feedback should have, paying special attention to each of its semantic subdimensions (Narciss, 2004). Design/methodology/approach: A survey was conducted among students of Business Administration degree of the Universitat Oberta de Catalunya (UOC). 182 students took part, separated into two groups (pilot and control). Findings: It has been proved that 90% of students give a great or very great importance to reception of feedback from their tutors, a relevance they consider to be higher than the one within a face-to-face environment. This percentage is around 75% with regard to the importance given to the level of personalization of feedback. The development of a factor analysis has revealed that usefulness of personalized feedback perceived by the students can be subsumed under two large dimensions: the one that facilitates learning (related to its semantic dimension) and the motivational one (by


Introduction
In the framework of the adaptation of new degrees to the European Higher Education Area (EHEA), the student becomes the protagonist of his/her learning process, which is carried out by fulfilling learning activities.In this context, as shown in the UOC's (Universitat Oberta de Catalunya) education model, the function of accompaniment and personalized relationship between teacher and learner is essential in order for each student to really attain the objectives and the competencies associated to each subject in particular, and the program in general.
The UOC is a totally virtual university.One of the basic characteristics of its educative pattern is to encourage a range of continuous assessment activities to every subject that is part of the diverse learning programs.The underlying idea is that the student attains the objective and competencies inherent to each subject by carrying out the different proposed activities, jointly with a set of multiplatform materials which go along with the learning process.These continuous evaluation activities are called Continuous Assessment Activities (PAC, for its acronym in Catalan).
The elaboration of these PACs is, therefore, the axis of the student's work.Each activity is evaluated by the tutor with a mark, which is sent to a private space only shared by the tutor and the learner.Furthermore, as these PACs are evaluated systematically and rigorously, they become the most direct and automatic channel for the student to receive a clear information on whether he/she is attaining the previewed goals or not, as well as identification of the mistakes he/she is making and what he/she is lacking.
Nevertheless, one of the aspects of the UOC's learning model that students were more dissatisfied with in recent years is the absence of a personalized feedback to the continuous assessment activities they have submitted (Martínez, Juan & Castan, 2010).Existent literature shows difficulties arisen when giving feedback in teaching and learning processes.One of such difficulties is related to the contents of this feedback.This is about deciding which is the adequate feedback and which the best way to make it available to the student, since the student's aim is to do his/her best in developing his/her learning process (Gibbs & Simpson, 2004).Few are the authors that have studied this aspect within virtual learning environments until now, for instance Espasa (2008Espasa ( , 2010)), Guasch, Espasa and Álvarez (2010) and Álvarez, Espasa and Guasch (2011).And here is where our study is placed.Taking as starting point the definition of the concept of feedback given by Susanne Narciss in her multiple scientific works (Narciss, 2004(Narciss, , 2008;;Narciss & Huth, 2004, 2006;Narciss, Körndle, Reimann & Müller, 2004), this study is focused on the semantic dimension of feedback.This work is organized as follows: in the first part we present the theoretical framework that served us to think and design the pilot test.In the second part we describe the pilot test's implementation and we announce the first results.We close with some conclusions and the future lines of investigation opened out of this first experience.

Theoretical framework
Using feedback in the pedagogical process has been broadly studied.Existing literature shows the difficulties arisen both from the teacher's perspective and the student's perspective, giving or receiving feedback in the learning process in an e-learning environment.Such difficulties are linked basically to the high ratio students-teacher in the virtual classrooms, that increases the follow-up task of the learning that the teacher could do (see Buchanan, 2000;Ley, 1999).
Other existent difficulties are related to the contents of the feedback, that is to say, how and which feedback is provided to the student so as for him/her to make the best of it in his/her learning process, and to make sure that it has been the appropriate feedback (Gibbs & Simpson, 2004).This issue has not been too studied until now and even less in e-learning environments.The present work intends to explore how and which is the feedback to be provided, in order to improve its efficiency and efficacy in virtual classrooms with a high number of students.With this aim, different multimedia technologies are used (video, audio, screen captures, etc.) as a complement or substitute to written language.This aspect has not been studied yet and is the main contribution of this article.
Taking as starting point the definition of the concept of feedback contained in works by Narciss and her collaborators (Narciss, 2004(Narciss, , 2008;;Narciss & Huth, 2004, 2006;Narciss et al, 2004), the aim of this article is focused on its semantic and structural dimension.

Semantic dimension
The semantic dimension of feedback refers to the contents it transmits, and has been studied in non-virtual environments by several authors such as Kluger and DeNisi (1996), Kramarski and Zeichner (2001), Kulhavy and Stock (1989), Mason and Brunning (2001) and Narciss (2004).In virtual environments it has been tackled by Espasa (2008).Due to the objectives and characteristics of our study, we use the taxonomy elaborated by this author, which includes four semantic sub-dimensions of feedback (from the lesser to the major degree of complexity) that are not exclusive amongst them.Such sub-dimensions are: • S 1 .Error identification and correction: the feedback informs about the correction of errors made.
• S2.Correct answer: the feedback shows the correct answer or gives the right solution.
• S3.Task improvement: the feedback recommends contents and strategies in order to improve the work, or the test that has been submitted.
• S 4 .In-depth information: the feedback suggests in-depth information in order to continue to move forward in learning in the future.
Out of these four sub-dimensions, our analysis will focus on the first three (error identification and correction, correct answer and task improvement) as they are the most inherent to initial subjects in the framework of graduate studies.
In order to improve the learning process feedback requires (Kulhavy & Stock, 1989;Espasa, 2010;Mory, 2004;Gibbs & Simpson, 2004) the following elements 1 or/and 2, where the teacher assesses on whether the answer is correct or not (Verification feedback).And 3, where the teacher offers guidelines towards the correct answer (Formative feedback).
In most of the subjects that make up the UOC's Degree in Business Administration, feedback made by the tutor includes the following sub-dimensions: • ( 1 ) Guidelines to the answer: once the term to submit the continuous assessment activities is over, a correct or oriented answer is provided (or rather "a correct solution" among the possible) to the whole group of students in the classroom.Thus, students can carry out a self-assessment activity, comparing this solution to the contents of the activity they submitted.This category of feedback fully coincides with sub-dimension 2 of the feedback's semantic dimension.
• ( 2 ) Specific feedback: In the virtual classroom each tutor has a tool (continuous assessment registry -RAC-) that allows the sending of (written) comments to each student.But often, groups are of 70 students or more and there is the commitment to correct and comment on the activity in a term no longer than a week.So, this makes a personalized input to every student very costly (difficult) since it is strongly time intensive.Instead, a comment on the most frequent and common mistakes done by the students in their activities is provided in the virtual blackboard of the classroom (public space).Despite it does not individualize the comment; obviously we can assimilate this kind of feedback to the first semantic sub-dimension.
• (3) Answer to doubts: Moreover, each tutor answers doubts raised by the students on the activities and contents they refer to.This could be labeled as a reactive, nonproactive feedback.The aim of this kind of input is to improve the work to be done by students, coinciding with the third sub-dimension previously described.
Therefore, according to the literature to improve the learning process and to motivate students the implementation of personalized inputs becomes a key factor, in order to offer a differential treatment according to their own diversity.
On the contrary, elaborating personalized inputs of four or more continuous assessment activities by the tutors, that have classrooms of approximately 70 students, is often a nonsustainable task due to the time it requires taking into account that the estimate and demanded dedication of tutors is a maximum of two hours per day.

Structural dimension
As regards the structural dimension of feedback, it refers to the presentation or the feedback's shape in a concrete context.Considering contributions made by Narciss and Huth (2006), others such as Hyland (2001Hyland ( , 2003) ) and Espasa (2008) in an e-learning environment of higher education such as the university we are analysing, the main elements that define the feedback's structural dimension are:

Agents that take part in the feedback processes
One of the elements inherent to feedback are the members that take part of its processes.This research has been developed by using a technological platform that integrates among others two applications that make it possible to provide the student with feedback.The first one is the continuous evaluation registry (RAC, for its acronym in Catalan), a space in the virtual classroom where the teacher inserts the marks of the continuous evaluation and facilitates the personalized feedback (which is sent to a private space only shared by teacher and learner).
The second one is the calendar and it can be found in the planning space of the virtual classroom.There, the student can accede to the solutions of the continuous evaluation tests (PAC), prepared by the teacher and published one day after the students have submitted their solutions.

Space of virtual communication where the feedback processes take place
Another component is the virtual space where the feedback takes place, be it public or private.
In this study we analyse the effects of feedback in a private space (from now on personalized feedback).Authors such as Rice, Mousley and Davis (1994) have proved that personalized feedback (private spaces) has advantages over feedback in public spaces because it fits better to the needs and characteristics of each student.

Moment of feedback
To provide immediate or differed feedback has been the object of discussion in several researches on this issue.It is remarkable the proposal made by Dempsey and Wager (1988), that define some categories used in this work.The first one is immediate feedback (it provides the student with information on the correct answer) and in this context it corresponds to the PACs' solutions, published just one day after the students submit their answers.The second one is differed feedback (where the errors made are commented on, some contents are recommended as well as strategies to improve the process of learning), that gives added value to the solution of the PAC.

Extension and form of the feedback
Another special feature of feedback in a virtual environment is its extension.It is tightly linked to the semantic dimension of feedback and to the kind of knowledge on which the feedback is given (depending on whether it is more abstract or conceptual or more applied and procedural).In order for the feedback to be effective it must give enough information to university students so as for them to attain the aims of learning previewed.
The feedback can have different shapes: text and/or technological multimedia (audio and video recordings, screenshots with audio as well as the possibility to upload compressed videos).The student receives it in the RAC as an attached file or in the message itself.
Following Narciss and her collaborators (Narciss 2004(Narciss , 2008;;Narciss & Huth, 2004, 2006;Narciss et al., 2004), in this article we also state that there are other dimensions apart from the semantic and the structural dimension with influence on the feedback processes, as in the case of the pedagogical design.Following we explain the pedagogical framework in which this study has been carried out.
Our project analyses the efficiency, from the student and the tutor point of view, of different ways (text, video, screen-shots, annotated PAC) to give feedback to the students' continuous assessment activities in order to enforce subdimension 3.This article shows results obtained from the student's viewpoint.

Methodology
The study of the feedback impact and the analysis of hypotheses raised will be based on answers to surveys given by students that take part in the pilot tests, as well as academic results.With the specific aim already mentioned, a pilot test was designed in order to analyze the results and the students' assessment on the use of multimedia tools to provide personalized feedback.The pilot test's design has been double-sided.On the one hand, the technical side that focused on making feedback feasible by the tutor, and, on the other, the teaching side that focused on how to make it have a major impact on the student.
When implementing the pilot test, these criteria were followed: • Eight subjects as different as possible were selected, so that implementation of multimedia feedback could be evaluated according to the typology of the subject of the Business Administration Degree.
• The pilot test focused on one of the subject's classrooms in order to check its real impact, while other control classrooms of the same subject were maintained.
• Previous information from the involved students was searched in order to analyze their profile, preferences and predisposition to receive a better feedback and the importance they give to its quantity and quality, among other aspects.
• Bearing in mind the diverse technical existing options when giving feedback (video, audio and video or commented screen-shots), we opted for trying different tools in different subjects so that these could better adapt to the subject's needs.
• Feedback in the pilot test was only implemented in specific evaluation tests, searching to maximize its usefulness.In the rest of tests the option was for written feedback.
• In all cases, feedback is linked to the correction of evaluation tests.
• We also take into account the size of the group due to the fact that this is a critical point.
Taking these elements into account, we have also implemented a set of technological innovations with the objective of making the elaboration of multimedia feedback (both video and audio) easy.To do that, an integration of LANG blog was developed in this application of mark registration.Thus, in a short space of time the options of audio and video recording have been incorporated to the continuous evaluation registry as well as the option to upload a file to those tutors that opted for screen-shooting.Although this agile integration makes implementation of the pilot test of multimedia feedback possible, it is not a scalable or accurate solution in the long term.The objective was to allow a quick and easy sound and video recording as well as its editing and ulterior visualization by the students.

Results
The UOC students' profile has modified substantially in recent times.The tendency shows clearly that the average age has significantly lowered as a result of changes that society is experimenting.
Such changes also imply a variation on the approach adopted by students when following their learning process.Longer working days and less available time to devote to studying, has limited the students' participation in the different spaces they have to that aim.In this sense, one of the elements of the most importance some years ago was feedback amongst students from their interventions in the forum under the teacher's guidance and coordination, is now secondary.
As said, these students work mostly full-time, and usually have a low (less than 8 hours per week) dedication to subjects as well as a big diversity as regards the number of subjects they have signed on each semester.
The process followed has been a first survey at the beginning of the year, with the aim to detect and describe perceptions students have about feedback before it really takes place, and a second survey once the subject has finished, where impact got from implementing this feedback is compared to the effect it might have had.
The main results obtained are: • The fact of receiving feedback from the teacher-consultant is valued as very important.
Apart from the fact that access to solutions is already a first feedback, students value very highly that they are commented and adapted to each student's specificities.In this sense, it is highly appreciated that the teacher's feedback focusses on particular aspects where the student has had some difficulties and skips those correctly solved by the student.Personalization of feedback is double-sided.Apart from the elements already mentioned, when trying to encourage continuity of the learning process, motivational aspects must be taken into account.
• Quality of the feedback received is significantly valued.On the contrary not much importance is given to quantity neither to the tool through which it is received (RAC, board, mailbox, etc.), nor to the format or the moment they received the feedback.Only an approximate 18% of students agree or totally agree that there is no need of personalized feedback if they already have standard complete solutions to PACs' questions.
They think this feedback must help them to improve their answers in future PACs (more than 90%), by explaining concretely the mistakes made.
On the other hand it is to be noted that although they think it is good to receive feedback as soon as possible once they have submitted the continuous assessment test, almost 60% of students would agree on receiving it later if this was personalized.Once the semester is over and teachers have implemented personalized feedback, based on the criteria and methodology mentioned in the introduction, we will analyze the results of surveys that will be answered by students.
The main results obtained are the following: • All students perceive they are receiving feedback, both those in the control classroom and those in the pilot classroom.None of them considers he/she has received excessive feedback, and only an approximate 10% thinks it is insufficient.This percentage is reduced to less than half in the pilot classroom.This shows that UOC already gives high importance to feedback, although it is not completely personalized and students perceive it that way.Nevertheless the satisfaction degree is higher in the pilot test than in the control classroom; around 80% of students in the pilot test and 70% in the control classroom are satisfied or very satisfied.This reveals that the way towards personalization of feedback helps to give a higher satisfaction to students.• It can be observed that there is a significant difference between importance given to feedback by students of control classrooms and by those of the pilot, the one given by the latter being higher.Seemingly, taking part in the pilot test has allowed them to foresee potentialities and positive effects of feedback in this context.• Finally, it is to be outlined that if we focus on usefulness of personalized feedback perceived by students, we observe that the pilot test's students value, above all and significantly more than the control classrooms students, that it has served them to identify mistakes made in the PAC (almost 80% to 60%).Therefore, using personalized feedback through multimedia tools facilitates identifying mistakes made by the student in his/her learning process, first semantic dimension of feedback, and hopefully this will contribute to improve such process.
It is also remarkable that students that received this personalized feedback think it has served them particularly to have an easier communication with consultants, to clarify specific doubts about the subject and not to give it up.These aspects highlight the importance of personalized feedback.A further step on the feedback application impact analysis in classrooms is to see what the students' satisfaction is, once it has been implemented.Presuming that results of the satisfaction variable move within a scale of 1 (min) to 5 (max) we can see that there is a higher satisfaction among students that received a personalized feedback although the difference (see Table 1).Parameters included in this exercise are not statistically significant and this can be due, in some cases, to the reduce number of observations they have.In any case, we think it is still relevant to show existent discrepancies since they can show the current tendencies.As already mentioned, the first factor, defined as Semantic Dimension, includes variables related to identification, answer and learning of the students' doubts, regarding the subject's basic concepts.The second resulting factor, named Motivation and Communication factor, gathers together those motivational variables that, though with no direct impact on the evaluation results, do have an incidence on the continuation of studies by the student, with direct results on the rate of students that follow continuous assessment.In on-line studies, this dimension is critical since one of the main elements that affect continuation of studies is, precisely, to impel students' motivation all along the year so they can overcome difficulties as they appear during the semester.future assessment tests (third semantic dimension) as well as facilitate detection of mistakes made (first semantic dimension).This fact shows the need to go deeper into personalized feedback since this would be the way to deal with all semantic dimensions of feedback.

Group
• On the other hand, they outline the motivational effect of feedback, this aspect becoming a deterrent to dropout the subject.
Following, assessment given by students to feedback received in the pilot test is shown: • In no case, students consider excessive or inaccurate the feedback received.Instead, some of them considered it was insufficient.
• Students think that feedback received has served them particularly to easy communication with consultants, to clarify specific doubts about the subject and not to give it up.
• Students showed very "practical" in the sense that they value, above all, usefulness of the feedback received, much more than the kind of feedback or the channel (audio, video, etc.) used.In fact, regardless of the channel used, the fact to be able to consult any time feedback given to a particular activity and, even, the possibility to print it out is positively valued by students.
• Implementing this kind of feedback leads the student to become more demanding and not to value written feedback so much.He/she claims more often to have a personalized and multimedia feedback.
From the student's perspective, it is necessary to go deeply into personalization of feedback, since, thus, all semantic dimensions of feedback could be addressed, and presently this aspect is not well covered.On the other hand, implementing feedback has a clear motivational effect over the student, which had not been sufficiently evidenced by other researches.Such effect needs not to be underestimated, particularly within an on-line environment, where the dropout rate is usually high.This personalized feedback helps to increase the student's feeling that he/she is being accompanied during the learning process, thus reducing the feeling of loneliness that is usually associated with this learning type.
In short, implementing personalized feedback has a relevant impact on the student, who values it because it makes his/her learning process easier, it motivates him/her not to give up and encourages communication with the teacher-consultant.
Figure 1.Value importance of the following aspects Figure 2. Show your level of conformity with the following statements

Figure 3 .
Figure 3. My level of satisfaction with personalized feedback I received is

Figure 4 .
Figure 4. Value if you consider more or less adequate and useful each of the following formats of feedback according to the subject's characteristics Scale: 1 minimum to 5 maximum

Figure 5 .
Figure 5. Compared to a face-to-face course, I think that feedback is more important in an on-line course Figure 6.Personalized feedback I received has served me to...

Table 1 .
My level of satisfaction with personalized feedback isIn the same line, it will be relevant to analyze what is behind these possible differences, looking in detail at which elements were useful to students after receiving the personalized feedback.To that end, we proposed the following answers to the question: Complete the statement: personalized feedback that I received has served me to: Clarify specific doubts about the PAC, Identify mistakes made in the PAC, Know how not to fall in the same mistakes made in the PAC, Direct me on how to solve next PACs, Understand better the subject's basic concepts, Complement the subject's teaching material, Motivate me to study and make the PACs, Do not give the subject up, Feel more supported by the consultant and Have an easier communication with the consultant.Results of the territorial analyses show two satisfaction factors (see Table2), which explain 80% of the total variance.The first factor, that explains 70% of the variance, includes variables of concepts and material and we will define it as Semantic Dimension factor.The second factor, that explains 10% of the variance, includes variables of motivation and communication with the consultant and will be defined as Motivation and Communication factor.
Note: Extraction method: Analysis of main components.Rotation method: Normalization Varimax with Kaiser.Rotation converged on 3 iterations.

Table 2 .
Matrix of rotated components