usecases :: Blog

http://www.pontydysgu.org/2008/10/open-accreditation-a-model/

Can we develop an Open Accreditation system.  What would we be looking for. In this post Jenny Hughes looks at criteria for a robust and effective cccreditation system.

An accreditation system depends on the robustness of the assessment system on which it is based.

Imagine you were in a shop that sold accreditation / assessment systems ‘off-the-peg” – what would criteria would you use if you went in to buy one?

Reliability

Reliability is a measure of consistency. A robust assessment system should be reliable; that is, it should be based on an assesssment process that yields the same results irrespective of who is conducting it or the environmental conditions under which it is taking place. Intra-tester reliability simply means that if the same asessor is assessing performance his or her judgement should be consistent and not influenced by, for example, another learner they might have just assessed or whether they feel unwell or just in a bad mood! Inter-tester reliability means that if two different assessors were given exactly the same questions, data collection tools, output data and so on, their conclusions should also be the same. Extra-tester reliability means that the assessor’s conclusions should not be influenced by extraneous circumstances, which should have no bearing on the assessment object.

Validity

Validity is a measure of ‘appropriateness’ or ‘fitness for purpose’. There are three sorts of validity. Face validity implies a match between what is being assessed or tested and how that is being done. For example, if you are assessing how well someone can bake a cake or drive a car then you would probably want them to actually do it rather than write an essay about it! Content validity means that what you are testing is actually relevant, meaningful and appropriate and there is a match between what the learner is setting out to do and what is being assessed. If an assessment system has predictive validity it means that the results are still likely to hold true even under conditions that are different from the test conditions. For example, performance assessment of airline pilots who are trained to cope with emergency situations on a simulator must be very high on predictive validity.

Replicability

Ideally an assessment should be carried out and documented in a way which is transparent and which allows the assessment to be replicated by others to achieve the same outcomes. Some ‘subjectivist’ approaches to assessment would disagree, however.

Transferability

Although each assessment should be designed around a particular piece of learning, a good assessment  system is one which could be adapted for similar  situations or could be extended easily to new activities. That is, if your situation evolves and changes over a period of time in response to need, it would be useful if you didn’t have to rethink your entire assessment system. Transferability is about the shelf-life of the assessment and also about maximising its usefulness

Credibility

People actually have to believe in yourassessment! It needs to be authentic, honest, transparent and ethical. If you have even one group of stakeholders questioning the rigour of the assessment process or doubting the results or challenging the validity of the conclusions, the assessment loses credibility and is not worth doing.

Practicality

This means simply that however sophisticated and technically sound the assessment is, if it takes too much of people’s time or costs too much or is cumbersome to use or the products are inappropriate then it is not a good assessment system !

Comparability

Although an assessment system should be customised to meet the needs of particular learning events, a good assessment system should also take into account the wider assessment ‘environment’ in which the learning is located. For example, if you are working in an environment where assessment is normally carried out by particular people (e.g teachers, lecturers) in a particular institution (e.g school or university) where ‘criteria reference assessment is the norm, then if you undertake a radically different type of assessment you may find that your audience will be less receptive and your results less acceptable. Similarly, if the learning that is being assessed is part of a wider system and everyone else is using a different system then this could mean that your input is ignored simply because it is too difficult to integrate.

Also, if you are trying to compare performance from one year to the next or compare learning outcomes with other people, then this needs to be taken into account.

http://www.pontydysgu.org/2008/10/jm-coetzee-on-open-accreditatio

Last week I write that with the emergence of open learning in the form of freely available courses, seminars and events on the internet, and with the increasing availability of open educational resources, open accreditation is the final frontier.

This week I will be writing a series of blog pots on accreditation. But first up an excellent quote from J,M Coetzee in his novel ‘Diary of a Bad Year’. Just substitute ‘open online networks for ‘peoples homes’

“In the days when Poland was under communist rule, there were dissidents who conducted night classes in their homes, running seminars on writers and philosophers excluded from the official canon (for example, Plato). No money changed hands though there may have been other forms of payment. if the spirit of the university is to survive, something along these lines may have to come into being in countries where tertiary eduction has been wholly subordinated to business principles. In other words the real university may have to move into peoples homes and grant degrees for which the sole backing will be the names of the scholars who sign the certificates.”

http://www.pontydysgu.org/2008/10/not-a-data-projector-in-sight/

Before travelling to Portugal I helped run a two day workshop on policies and practice in the training of teachers and trainers. The workshop, sponsored by the European Commission brought togther researchers and practitgioners form five different countries. And we ran it as a real workshop. The participants themselves produced the materials. And not a powerpoint in site. It was like a breath of fresh air to be talking, working, developing and sharing ideas. Sometimes I wonder if the ease of use of tools such as Powerpoint have made us lazy and worse has stifled creativity and particpation.

http://www.pontydysgu.org/2008/10/500000-laptops-for-schools-in-p

it has been a busy week. From Thursday to Saturday I was in Braga speaking at a conference for teachers on Web 2.0 technologies for learning. About 250 teachers turned up and worked until eight in the evening. I greatly enjoyed myself (fabulous hospitality) and was impressed by the level of commitment. I also greatly enjoyed the chance ot chat with George Siemens who was also presenting at the conference. More later this week on some of the ideas we discussed.

Back to Portugal. According to Reuters “Portugal’s Socialist government began the roll-out on Tuesday of 500,000 ultra-cheap laptops for school children in a programme that could be extended to Venezuela, the government said.

“The government’s educational technology plan aims to make Portugal one of the top five most technologically advanced countries in Europe,” it said.

Portugal has some of the lowest school achievement levels in western Europe and Socrates has made boosting education a key priority. The government hopes the Magellan project will raise computer access at schools to two students per computer by 2010, up from five this year.

While the computer will be assembled in Portugal by a company called JP Sa Couto, it is based on Intel’s Classmate PC, a cheap computer that has been adopted in various formats in countries such as Brazil and Indonesia.

Venezuelan President Hugo Chavez, who has visited Portugal several times in the past year and is due in Lisbon later this week, has said the Magellan could also be used in Venezuelan schools.”

I was aprticually impresed at the conference with ideas for using computers with younger children. But of coures there are worries. I have no doubt that the kids will know how to use teh computers. But there needs to be a big programme of professional development to ensure the teachers udnerstand how to use teh computers for learning if the full value of the programmee is to be realsied.

http://www.pontydysgu.org/2008/10/what-does-a-personal-learning-e

I’m doing two presentations on Personal Learning Environments this week - one in Braga in Portugal and one on line at Bar Camp Berlin. Regular readers will know these are not my first presentations on PLEs. And lately I have been concerned that the debate has been stalling a little. We have been very good at saying what a PLE is not - it is not an instititional learning system etc. but rather hazy on just what it is. So that is the theme of the all brand new - new words, new pictures presentation. I will release the slides in Friday and try to get an audio version out in next two weeks. One of the points which I am at pains to make is that a PLE is not just a technical infrastruture - indeed it is possible to imanagine a PLE which involves not computers what so ever. Anyway here is a picture Jenny Hughes drew for me of her PLE>

http://www.pontydysgu.org/2008/10/a-month-of-meetings/

last week was spent with my head in a bunch of spreadsheets doing the Pontydysgu accounts.

But this week I’m on the road. tomorrow and Thursday Pontydysgu is organising an international seminar in Newport, Wales on the Training of Trainers in Europe. And Thurdsay night I am off to Braga for a conference on web 2.0, social software and learning. Saturday I am presenting at Bar Camp Berlin through a video tie up from Braga. Then a quick flight to Bremen and on Sunday I am off to Glasgow for a meeting with Careers Scotland. Tuesday night I will be in Maidstone, Kent for a meeting on e-Portfolios. Back to Bremen for three days, then Barcelona. The following week is Seville and perhaps London. Phew. But the real point is that if our paths are crossing I would love to meet up with any of you. Just send me an email saying where you will be.

http://www.pontydysgu.org/2008/10/teachers-nerdyness-and-tech/

Jenny Hughes has had a hard week editing a handbook for teachers on technology. The stress is begining to show. Whilst waiting for partners from the Taccle project (who are producing the handbook) to arrive, she twittered to me this reflection on teachers and nerdyness on an open skype channel :). And teachers - we love you really. LOL

“teachers get nerdy about who sits where in staff room

17:56

and nerdy about cardigans

17:56

and potted plants on the window sill

17:56

and using a ruler to do their register

17:56

and not lending anyone their red pen

17:57

and sandwiches in tupperware containers

17:57

and the Daily Mail

17:58

and whether its morally justified in going on strike when the future of the world is in their hand

17:58

and keeping peppermints in their desk

17:59

and seeig if they can make their suply of sugar paper last longer than anyone else

17:59

and nature tables

18:00

and box pleated skirts

18:00

(thats just the men)

18:00

and having special things for de-icing their windows

18:01

and posting a list of Rules of the Staff Room on the wall

18:02

which always starts with the milk and biscuit buying rota

18:02

and ends with a warning about not leaving dirty cups

18:02

which is underlined 3 times

18:03

with too many exclamation marks

18:03

but computers, I assure you, they are not nerdy about

18:05

Mind you (because Im bored sitting in hotel waiting for foreigners) teachers do have some techy skills

18:09

they are very good with shoe lace technology

18:12

have built in sensors and rapid response programmes for spotting the kid who is going to throw up or piss themselves

18:13

some of us were shit hot on the ballistics of board rubbers

18:14

(you have to be able to get  them to land wood side down on the desk to give kid nasty noisy fright or felt side down to leave them spluttering and coughing in a cloud of chalk dust. Its a high level skill)

18:18

they are competent at every sort of print technology from Caxton, through spirit copiers, Bandas, Roneos and Gestetners with or without wax stencils and including John Bull printing outfits. You can assess competence by seeing how big the pink, purple or green stain on their fingers is.

18:20

the older ones    can still tune the wireless in to Music and Movement (with Daphne Oxenford)

18:22

and some of them can even fit together those plastic shapes that make geodesic domes. The others stuff them in the back of the cupboard and hope the kids don’t find them

18:23

they are really good at measuring the length of the playground with a push along wheely thing and they check it every year just to make sure

18:24

and best of all they have pencil sharpeners whith a handle you turn on their desks. They are awesome.

18:25

so not entirely techno deficient

18:27

it’s just those com…comt…compu things

18:27

on line again tomorrow

18:29

http://www.pontydysgu.org/2008/10/open-learning-the-debate-contin

Continuing the open learning debate….I greatly like this diagramme by George Siemens. I think there is much of merit here. Very happpy to see acknowledgemnt of the importance of self publishing (as opposed to academic reporsitories). However there are a few things missed out.

Firstly if we take congniscence of Jenny Hughes’ defintion of learning as ‘to find and follow a track’ as counterposed to curriculum  from the latin ‘currere’, which means to run or race and ‘curriculum’ as race or racecourse, then instotutions and teacher have an important role in assisting learners in developing their own learning pathways.

A second important  role is that of assessment. But to understand this we need to decouple assessment from acceditation. If we design assessment as a learning process and move from assessemnt of learning to asssessement for learning this could become an integral part of the process of finding and developing learning pathways. This is not so utopian. Serendipitously. The Times newpaper today published an article about innovative assessment in UK universities. The struggle, though, as with self and peer assessment is in assessment having to match accreditation procedures. Without this link, we could open up all jinds of new forms of assessment.

A final point on accreditation. Many learners do not want or require accreditation. Indeed it is the formal accreditation procedures which deters them for signing up for a learning programme. And as Antnio Fini, talking about the home made certificate he got from the OpenEd2007 course, says: “all my connections, blog posts, comments, collective works, presentations, articles related to that experience, are still out there as tangible proofs of this learning. So I could equally put the OpenEd course in my CV and could ask to my supervisor to evaluate all that activity for credit in my PhD, also without that piece of paper!”

Why not put the learners in charge of accreditation. Lets leave it to them to decide how they wish to show what and how they have learned - albeit with support. I once co-ran a course with Jenny Hughes where we offered the particpants their certificate at the start of the courese. They refused! But it did raise the issue of why they were doing the course and how they valued learning. And that is an issue we need to bring to the fore.

http://www.pontydysgu.org/2008/10/have-you-got-something-to-say-a

Every month we are running Sounds of the Bazaar ‘Emerging Mondays’ - a topical live internet radio show combining the best of radio, live chat and occasionally some extravagant social activity inside Second Life. The aim is to bring lively commentary and debate on topical issues to the start of each month.

Upcoming

27th October 2008: Emerging Mondays: the Digital Identity show

‘What is digital identity?’

So what does digital identity mean to you? Do you care? As more of our lives, from personal to professional activities, find their way online how do we cope with managing our fractured and distributed digital presence. Can we ever keep ‘personal’ separate from ‘professional’ when tools and services mash-up our online identities in ways that are beyond our control? What does this mean for the development of new literacies and new services that seek to put our identities back within our grasp.

With interviews, music, strong opinion, poetry, our very own edupunk granny Leila and more. LIVE.

Have you got soemthing to say about identities. Would you liek to come on th programme through a skype or telephone link-up. In depth interviews or just two minute vox spots are all welcome. Tell us a story or read a poem. If you are interested just drop  me an email - graham10 [@] mac.com.

We will be broadcasting *LIVE* from 1800 - 1900 UK time, 1900 - 2000 Central European Time. Links to the programme url and chatroom to follow.

http://www.pontydysgu.org/2008/10/open-learning-is-here-where-nex

First we had open educational resources. This was a step forward but the resources were variable in quality, hard to find and were often tied to courses which made them hard to use for self study. Those issues haven’t gone away but improvements in search technologies and a wider general conciousness about the value of self publishing open resources means it is increasingly easy to find what you want.

And now we are witnessing an explosion in open learning. Of course there are the big publicity happenings like the CCK08 Massive Open Online Course (MOOC)  organised by Stephen Downes and George Siemans on connectivism.

But more important is the flowering of opportunties for learning from many, many diverse sources. One of the best things about Twitter is it opens access to many events going on and opportunities to partiucpate at a distance. Last week I dropped in on a TeachMeet session being organised as part of the Scottish Learning Festival. Someone had ’shouted it out” in Twitter, I followed the link and ended up in a broadcast over the UK Open Universities free Flashmeeting service. There was about twenty or so of us particpating online. Whilst the quality of the video sometimes left something to be desired (and I was stuggling to follow Glaswegian accents) this was more than made up for by the quality and humour in the online chat.

Yesterday morning I recieved this in my email: “You have a live session today with cristinacost on ‘Connecting Online : Sharing Life’s Experiences’. The session will start at 10:00 AM W. Europe Standard Time and is 60 minutes long.”  This is a free course being organised in the WizIQ environment. Sadly I am bogged down in administration and had no time to go. But over the last year there has been an explosion of such open courses and seminars. We are organising one such series oursleves through the Jisc Evolve project.

And this morning Cristina Costa showed me her online bookclub “Living Literature though Exploration.” This more than anything impresses me as to how we have moved towards real open learning through Web 2.0 tools (in this case as simplle as a shared blog and some bookmarks.

However there remain a number of issues.

The last barrier to open learning - and a very complex one - is that of accreditation. Whilst I am sceptical about the Connectivism MOOC, it is raising a number of central questions about open learning, not least that of accreditation. Under the Connectivism course model, only 34 (I think) sdtudents are offically enrolled for accreditation and therefore pay fees. Their fees pay for the costs of the course which is open and free to everyone else. As part of this they get feedback form tutors on course assignments and accreditation at the end of the course. How important is this for learning? And would it be possible for a student to develop a portfolio based on particpation in the course and then claim accreditation elsewhere? Are we moving to a model where learning is open but institutions have a major role in accrediting that learning (presumably through a portfolio model)? Can we develop a concept of open accreditation? And what would that mean?