Overview of e-portfolios and e-assessment in teacher education


This section of the website looks at both e-assessment and e-portfolios with a particular emphasis on applications to initial teacher education. A number of case studies are included which are indicative of how different institutions are exploring and developing their thinking in these areas. In essence e-assessment is about measuring some facet of the learner whilst an e-portfolio is about the learner compiling an electronic record of their learning experience. Often in teacher education the compilation of an e-portfolio plus a reflective journal constitutes the assessment task. Hence this section deals with both topics as one, recognising that in HE the distinction is not always clear.
Aspects of this topic are covered in the following sections:

Definitions
Characteristics of today's learners
E-portfolios in the HE environment
E-assessment in the HE environment
Scenario - the HE reality
Case studies
References
E-portfolio frameworks


author: Roger Keeling

Definitions


Definitions of e-portfolios and e-assessment can be found on the e-futures site (a QCA initiative)
www.efutures.org/graphics/e_portfolios.html. From this site it is also possible to download “e-assessment: Guide to effective practice” (this also covers e-portfolios).

The QCA view on e-assessment is summarised in the statement below:
“QCA believes that e-assessment and e-learning plays an important role in the future of education and training. QCA wants to ensure that technology plays a key role in the learning and assessment systems of the future. Technology must provide a valid and reliable learning platform, it must be robust and accessible. Above all, it must improve and enrich education.”
www.qca.org.uk/qca_6708.aspx

Another definition is provided at: www.eportfolios.ac.uk/definition. This site aims to provide a community resource for those interested in e-portfolios and Personal Development Planning (PDP). It documents some interesting ideas beyond the field of teacher education. It is based in the Medical School at the University of Newcastle upon Tyne.

Becta also focus on e-portfolios on their Government and Partners website. Becta don’t offer a succinct definition as such but list some of the elements which the process of developing an e-portfolio should encompass. Click on the link below to read the list of elements, together with links to different aspects of e-portfolios.
http://partners.becta.org.uk/index.php?section=pv&catcode=_pv_ep_02

It appears that in HE more emphasis has been put on e-portfolios than on e-assessment. This is partly a consequence of students needing to show evidence of attainment and meeting standards. By comparison school examination boards are actively exploring e-assessment, driven by the need to test cohorts of thousands of pupils over a national area in a standardised and robust manner. This section will not focus on the school environment but instead will look at what is happening in institutes of HE, and in particular the pedagogical gains that these developments offer.

author: Roger Keeling

Characteristics of today's learners


Initially we should ask why we are going down this route at all and to answer this we need to look at the characteristics of today’s learners. Traditional assessment has served us well but we are going through a cultural change. The traditional lecture is detached from reality. Students expect more interactivity, they expect tutors to use the tools that are natural to today’s learners. Contemporary skills are not listening and note taking, but are based around collaboration, discussion, communication, problem solving and flexibility. With students having instant access to Google or Wikipedia, memorisation is no longer the essential skill that it was in the past. Students of today are used to: Passive learning is no longer de rigueur. Learning has to be active, with authentic tasks and goal oriented. In fact assessment 2.0 has to be: All this points to the fact that in such a rapidly changing environment HE has to reconsider the teaching, learning and assessment environments.

In the area of e-assessment, Dr Randy Bennett’s presentation at the Institute of Educational Assessors National Conference, May 2007 provides a thoughtful insight as to where we are at present and the potential gains.

He describes three stages in the development of e-assessment.
  1. The first generation focuses on infrastructure and automating existing processes.
  2. The second generation concerns advances in technology and measurement science, together with insights into learning, but still serves “institutional constituencies”.
  3. The third generation (and we are not there yet) is about reinvention. Learning and assessment, both at a distance and on demand, is a challenge still to be solved. It is about a complete radical rethink focused on the needs of the learners.
In fact, the first generation is firmly established and is evidenced by the driving theory test and the trainee teachers test for literacy and numeracy. The second generation is more problematic. The emphasis should be put on the testing process rather than the marking convenience; enabling the students to demonstrate what they have learnt.

Bennett identifies 5 key points where the internet has features that will prove invaluable for e-assessment:
author: Roger Keeling

E-portfolios in the HE environment


E-portfolios enable tutors to track, monitor, assess and communicate with students, but to be really worthwhile they must also offer a real benefit to the student. From the student viewpoint they can offer: As an aside, one form of reflection is through the medium of patchwork texts. The characteristics of a patchwork text assignment are that it is: For details of this approach see:
http://web.anglia.ac.uk/richardwinter/Web_ContextPatchworkText.doc
http://education.guardian.co.uk/egweekly/story/0,5500,973742,00.html
Winter, R (2003) Contextualising the Patchwork Text: Addressing problems of Coursework assessment in HE. Innovations in Education and Teaching International 40(2)

The crucial issue about an e-portfolio is that it should be more than a system of on-line storage. Overall, it must be seen to have some real long term worth and value to the ‘owner’. It affords the learner the following benefits: JISC ran a project entitled ‘The Learner’s Voice’ (2007) and as part of it groups of students each made a short video about their experiences of some aspects of e-learning. Jenny and Emma each talk about their e-portfolio and what it means to them. The video lasts about 5 minutes. To view it click here.

author: Roger Keeling

E-assessment in the HE environment


Two Approaches to e-assessment
There are two approaches that are commonly recognised. The first is to provide some form of computerised test that can be marked instantly by the computer and hence give the student immediate feedback. The ICT skills test that all students have to take is an example of this. The second approach is where the assessment is completed at a PC but the results have to be hand marked by the tutor. This is normally applicable in circumstances where students are asked for a free text answer based upon their opinion, interpretation or analysis.

Why PC-based assessment?
Regarding this second approach it might be reasonable to ask why is this a PC based form of assessment (after all, students have hand written essays for decades)? Most students will give the answer that a keyboard, and not handwriting, is the natural form of their written expression. They are used to drafting and redrafting and hand writing one hour timed essays is an anathema to them. From the tutor’s point of view the outcome is also typed so that he/she can concentrate on the content of the answer as opposed to spending half of their time trying to decode the handwriting. Even in free text answers that the tutor has to mark manually, it is possible to provide instant feedback in the form of model answers. Students will then be able, whilst fresh in their memory, to reflect on whether or not they have considered the key points.

Assessment anywhere, anytime?
In principle this assessment could be anywhere, anytime, but in practice this is not often the case. Society is not yet ready for a system whereby the student can take an assessment in her kitchen on a Saturday afternoon whilst the rest of the family are at the Emirates Stadium watching Arsenal. However, this can be the case if the assessment is for formative and not summative purposes. The TDA Numeracy Test, for example, can be practiced from home, but the definitive test has to be undertaken in a secure and supervised environment.

Although “anywhere, anytime” is a principle that we are still working towards, if a secure environment can be established then “anytime” can be implemented now. Tests have to be set up so that there is a large question bank and/or, where applicable, randomised questions. This will ensure that one student who takes the test early can not give an added advantage to their peers. It also has the advantage that resits can be taken at any time once the tutor thinks that the student has spent appropriate time and effort addressing their weaknesses.

The route map for e-assessment looks something like this:

Where we are now Potential Advantages
Current problems are:

1. Imitates traditional assessment

2. Reproduces the paper experience

3. Limited question types (usually multiple choice questions (MCQ), short answer questions (SAQ) and extended answer questions (EAQ))

4. Individualistic/competitive/anti collaborative

5. Not delivering contemporary skills (collaboration, problem solving, flexibility)

6. Encourages memorisation and not understanding

7. Assessment still tends to drive teaching and learning - ‘teaching to the test’

8. It is also inflexible – one size fits all (not personalised)
 1. Instant results – very effective for formative feedback (but feedback may need the interpretation of a teacher in order to really benefit the student). The feedback might also suggest links to e-learning facilities

2. On demand and on-line (candidates take an exam when they are ready and not when the tutor says they have to be ready)

3. Very good for the presentation of simulated environments – (ie. the test may include video or audio based questions or very large data sets). It could also involve images which allow manipulation and magnification (ie. Google maps)

4. Enables students to demonstrate their visual and kinaesthetic understanding more effectively

5. Randomisation of questions and/or a very large question bank can lead to an ‘infinite’ set of different test papers – going beyond ‘one size fits all’

6. Provides opportunity for adaptive tests that respond to the student’s answers

7. Some logistical gains (eg. specialisation of markers in a limited number of questions, reduction in clerical errors, elimination of paper distribution, real time monitoring during the marking process)

8. Flexibility as to when resits can be taken

9. Objectivity in marking

An excellent summary entitled “How might e-assessment contribute to the changing educational agenda?” is written by Jon Williamson and Marian Sainsbury from the NFER e-assessment research project.
Download Acrobat document: NFER E-assessment White Paper TDA.pdf (39K)

author: Roger Keeling

Scenario - the HE reality


It is approaching the end of May. You have taught three groups of 30 postgraduates each over a period of 10 weeks. They have built up a portfolio of evidence to show what skills, knowledge and understanding they have acquired. They have also written a 2000 word essay reflecting on how their newly acquired knowledge might enhance their teaching in the classroom. You have a phone call from the Exams and Assessment office to say that the students' work is ready for collection. “It is quite bulky, do you want the porters to deliver it to you? There are 8 boxes all together”. Your heart sinks – surely there must be another way!

Some institutions are beginning to look at different solutions to solving this perennial problem. However, whichever solutions are adopted, they should offer more than a convenient way to alleviate assessment bottlenecks for the tutor. The overriding question is whether assessment been used effectively and meaningfully.

The six case studies included in this section are not necessarily exemplary accounts, but are put forward by the tutors involved in order to share experiences and to indicate pointers for future developments.

author: Roger Keeling

Case studies


1. University of Wolverhampton
A paper on “The use of e-portfolios and blogging in Teacher Education” by Julie Hughes from the University of Wolverhampton is available on the JISC InfoNet site:
http://www.jiscinfonet.ac.uk/case-studies/tangible/wolverhampton/index_html1
This case study illustrates an effect on learning, an effect on exam results and an effect on student personal development. It also looks at student satisfaction with both e-learning and innovation in teaching and learning. The paper also focuses on staff satisfaction with e-learning and the hurdles that need to be overcome in this area including changes from policy on the one hand to redesigning learning spaces on the other.

2 University of Worcester.
Moira Savage describes a relatively new project with 20 PGCE students at the University of Worcester. The students have used PebblePAD to compile an e-portfolio of their ICT related work. Moira explains what is included in the portfolio and why she thinks it is superior to the previous paper version. Whilst she also recognises its limitations, its strengths are such that it is here to stay.
Download Acrobat document: worcester.pdf (1306K)

3. Liverpool Hope University
John Sharrock explains how his PGCE students compile an e-portfolio through the medium of the database facility in Moodle. He focuses on the type of information the students are expected to include and some of the problems that present themselves.
Download Word document: liverpool hope.doc (817K)

4. University of Northampton
Gareth Honeyford describes an e-portfolio system that he is pursuing with undergraduates at University of Northampton. This is based upon Powerpoint (or any alternative framework that the student wants to use). In essence it is a series of templates to which the students upload their evidence (and ultimately save as a CD).
Download Word document: northampton.doc (92K)

5. University of Southampton
John Woollard describes a bespoke open source, but relatively long established, system that they have been using at the University of Southampton (initiated through a TDA grant in 2001). John discusses the affordances that the system offers and also highlights how they would like to see the system developing in the future.
Download Word document: southampton.doc (112K)

6. University of Wolverhampton
Sarah Weatherhead describes how the use of PebblePAD with early years students has enabled them to communicate better, to be more reflective and to be more confident users of the technology. Moreover there has been a significant shift in how they perceive teaching and learning.
Download Acrobat document: wolverhampton.pdf (1860K)

All these case studies have an element of experimentation about them, but all recognise that e-portfolios are the way to go and are superior to the paper version and the associated overloaded files where quantity seems to take precedent over quality. There are distinct affordances in terms of students being able to provide evidence in different formats rather than just the written. There are opportunities for on-going formative assessment, sometimes by peers and sometimes by tutors. E-portfolios also offer the latter a monitoring facility which enables tutors to intervene much quicker in cases where they think particular students need help or prompting. Sharing and collaboration can be much enhanced. RSS feeds can also alert tutors to the fact that a student has updated their portfolio.

However, there are still problems to be overcome. Some are legalistic in terms of data protection (ie. including video of children at work for example) and some are technical in terms of transporting the e-portfolio to a different platform once the student’s course is complete.

author: Roger Keeling

References


JISC is actively involved in the area of e-portfolios and a summary of activities up to September 2007 is provided at:
www.jisc.ac.uk/publications/publications/eportfoliooverviewv2.aspx

Effective Practice with e-Assessment (JISC)
www.jisc.ac.uk/media/documents/themes/elearning/effpraceassess.pdf

e-Assessment: An overview of JISC activities (June 2006)
www.jisc.ac.uk/uploaded_documents/ACFC6B.pdf

e-Assesssment Glossary
www.jisc.ac.uk/assessment.html

e-Portfolios: What institutions really need to know
www.jisc.ac.uk/media/documents/publications/eportfoliosbpv1.doc

Roadmap for e-assessment: Report for JISC can be downloaded from here
www.jisc.ac.uk/elp_assessment.html

Becta’s gateway page that leads to a number of articles on e-portfolios
http://partners.becta.org.uk/index.php?section=pv&catcode=_pv_ep_02

Becta - Impact of e-portfolios on learning
http://partners.becta.org.uk/index.php?section=rh&catcode=_re_rp_02&rid=14007
This report, largely compiled by a group of researchers at the University of Nottingham, can be downloaded but it is 111 pages.

ESCalate Education Subject Centre of the HE Academy Network - Transferabililty of e-portfolios in Education
http://escalate.ac.uk/eportfolios
The aim of this project is to undertake an investigation of existing e-portfolio models and their use within the field of Education and to identify common ground which might be used to introduce a measure of standardisation and transferability.

UK Centre for Legal Education - Getting Started with e-portfolios
http://www.ukcle.ac.uk/resources/ict/eportfolios.html
A very practical document from the UK Centre for Legal Education (University of Warwick). It spells out what to look out for, tips and hints, checklist of benefits and case studies.

Nesta Futurelab (Ridgway J, McCusker S and Pead D) 2005 Literature review of e-assessment Research Report 10 Nesta Futurelab. Updated by Martin Ripley 2007 – available at:
http://www.futurelab.org.uk/resources/publications_reports_articles/literature_reviews/Literatur
e_Review204

A useful Wiki on the “e-portfolios and the personalisation agenda” has been initiated by Kevin Brace and is available at:
http://rsc-elearn.pbwiki.com/eportfolios
There are numerous links including ones to e-portfolio tools, e-portfolio basics and another section on examples of exemplary e-portfolios. If you can’t find what you want on this site there will undoubtedly be a link to another site that may well contain the answer.

Another page that will send you off in numerous directions is:
http://del.icio.us/deliboy/eportfolios

author: Roger Keeling

E-portfolio frameworks


In addition to the facilities built into VLEs such as Blackboard and Moodle, there are more specific e-portfolio packages:

Profile
www.profile.ac.uk
”Profile.ac.uk is a web-based system which provides a tool to capture, track and assess student learning remotely. The system provides students with individual, secure electronic-portfolios which contain web-forms to guide students through any educational process”

PebblePad
www.pebblepad.co.uk
PebblePad has been designed with the learner at the centre of the system. It provides scaffolding to help users create records of learning, achievement and aspiration and has a reflective structure underpinning all of its core elements. PebblePad supports personal learning whilst providing a powerful suite of tools to improve learning in institutional contexts. Conversation, communication and collaboration is easy in PebblePad; items can be shared with trusted individuals, published to group pages or made public to the world-wide web.”

Mahara
http://www.mahara.org/
“Mahara is an open source e-portfolio, weblog, resume builder and social networking system, connecting users and creating online learner communities. Mahara is designed to provide users with the tools to demonstrate their life-long learning, skills and development over time to selected audiences.”
ULCC are running a Mahara Evaluation at http://portfolio.evaluation.ulcc.ac.uk/mahara

MAPS
http://www.maps-ict.com/fe/index.htm
Managed e-portfolios with particular reference to work in schools and colleges.

e-Folio
http://efolio.project.mnscu.edu/index.asp?Type=NONE&SEC={116D5481-4358-483D-B413-309B7E654CD7}
“Our industry-leading online e-portfolio management systems empower career seekers, students, educators, and institutions to organize, manage, and display their academic and career information to enhance education, promote achievement, and advance careers.”
Available in the UK from http://www.maximise-ict.co.uk/


author: Roger Keeling