"This was a landmark project for IStructE; affecting almost all of our members. Distinction understood that from the beginning and went to great lengths to ensure a smooth process. Everyone at Distinction was great at keeping the project moving. Until recently, you could say we’ve been a bit slow to make changes to our technology or digital offerings. However, the success of this project has given key stakeholders the confidence and member backing to drive innovation in other areas of the Institution."
For the world’s largest certified membership organisation (c30,000 members), offering a range of qualifications – all of which require an exam to be sat in hundreds of locations across the globe – the adjudication process was an enormous overhead in both time and money.
Having been in business for over 100 years, with many of its manual processes not having changed for decades, the Institution of Structural Engineers came to us to digitally transform its business.
Transforming its business had a huge impact on the Institution of Structural Engineers.
The move to digital saved them £40,000 per annum in shipping costs alone. The organisation’s capacity to adjudicate exams increased by 300% and the tune taken to mark exams was reduced by 100%.
Often, established businesses have inefficient processes that result from their history and culture, particularly when those processes extend globally. In helping the world’s largest certified membership organisation to improve its examination and marking processes, we helped transform the business, make it fit for the digital age and reduced overheads.
Conducting a thorough diagnosis of the current situation, including conversations with every level of stakeholder and some judicious fact-finding, we recognised that a major obstacle was in a stereotypical view of examiners. Our discovery workshops uncovered that this prejudice was unfounded and, instead of opposition, we found the examiners open to change.
Once all stakeholders were aligned, we took the whole marking process online by automating every stage. With the need to process huge amounts of data quickly and efficiently, we developed a four-part system covering candidate distribution and management, PDF script viewer, adjudication app and a feedback app.
It’s all too easy to think the stress of exams resides solely at the desk of the students, nervously turning over their papers. But for the Institution of Structural Engineers, the stresses of the whole examination process went much deeper.
As the world’s largest certified membership organisation – numbering almost 30,000 – the Institute has for over 100 years offered a range of qualifications, all of which require an exam to be sat in hundreds of locations across the world. The qualifications are a valuable source of income, one which they were looking to maximise, yet one which was costing the Institute far too much to run.
The problem was, the Institute was wrestling with a paper marking process which had remained unchanged for decades. They’d always done it that way, but cracks were beginning to show.
An entirely manual and physical event, this process involved collecting completed A3 exam papers by hand, placing them into boxes of 500, shipping to the UK, anonymising, splitting out into tranches of 20 and posting to individual examiners who, making life more difficult, are not always UK-based. Also requiring the marks of a second examiner, the whole process then had to be repeated. So far, so lengthy.
Faced with such a problematic process – papers went missing, handwriting wasn’t always legible and disparate marks required the extra involvement of a third-party adjudicator. The Institute knew it was time for a change.
The financial benefits of doing so were compelling. There was the desire to avoid the spiralling costs of global postage. Then there was the need to shorten the process from its current nine months to fit in more revenue-earning exams (students were having to wait a full 12 months before being able to re-sit). Owning such an outdated process also reflected poorly on the Institute’s brand.
Conducting a thorough diagnosis of the current situation, including conversations with every level of stakeholder together with a little judicious fact finding, we recognised that a major obstacle was in a stereotypical view of the examiners. These were often in their 60s and 70s, mainly male and recently retired. Consequently, it had always been assumed they would be resistant to change and fearful of something new.
During our workshops, we listened to the views of the examiners, and involved them in all our explorations and possible better ways forward. We removed the fear factor and found the prejudice was unfounded. Instead of finding opposition, our transparent and non-judgmental approach helped the examiners became healthily invested in change for the better.
And, on the first run through of our prototype, their excitement clearly demonstrated they had more in common with the digitally-savvy exam sitters than previously thought possible.
So, what were we suggesting that produced such an about face? Simply that by taking the whole marking process online, by automating each stage, would make the whole journey quicker and pain-free for everyone.
We realised that to manage the large amount of data quickly and efficiently, the system would need to be developed in four parts:
Candidate distribution and management: enabling exam managers to distribute multiple candidates to examiners and then track an individual candidate’s progress.
PDF script viewer: giving examiners a high-resolution PDF view of a script and a simple system for marking and providing feedback on each question.
Adjudication app: automating the distribution and management of by the exam manager.
Feedback app: automatically loading examiners’ comments and marks, before saving, previewing and sending a feedback email directly.
Each of these elements made life simpler and easier for examiners and enabled them to spend much less time on marking and managing each paper.
All combined into one app, the seamless process brought immediate results. With six months cut from the entire process, exams were completed quicker, leading to a threefold increase in fees. Examiners were delighted to move from the cost and hassle of a physical, postage-based system.
In the end, our early involvement proved decisive in that we could involve the audience from the very start and keep the various stakeholders engaged at every stage. Being invested in the vision, the journey to lasting change
Transforming its business had a huge impact on the Institution of Structural Engineers. The move to digital saved them £40,000 per year in shipping costs alone. The organisation’s capacity to adjudicate exams increased by 300% and the time taken to mark exams was reduced by 100%.
