Assessing the English level of university applicants
Many educational institutions face a common dilemma: how to assess the English level of international applicants? For most UK universities, the answer is a simple one – an IELTS score. This was the case for Brunel University up until early 2020. Then, the covid-19 pandemic struck, forcing the closure of IELTS centres the world over. Brunel improvised and adapted by moving rapidly to online testing.
Online Testing: Key considerations
In selecting an online English assessment, Brunel had three key considerations. The first two were for the test to be accurate and reliable. After all, it would be replacing the IELTS – a high stakes exam. Brunel had to be sure that the test would correctly determine the English level of test-takers and would do so in a fair and consistent way. With the world in various degrees of lockdown, a third factor became equally important – test administration. Test-takers had to be able to complete the test anywhere, on their own devices, with minimal administrative work from Brunel’s team.
The Dynamic Placement Test (DPT) proved to be the solution. It is a 30-minute, adaptive and randomised English placement test that can be set up and deployed quickly. Content created by telc language tests ensures the highest possible level of accuracy and reliability; while a solid technical infrastructure means the test can be taken anywhere, on any device. In the words of Ms Yodi Yu, China Recruitment Manager at Brunel, ‘as the administrator, the system is clear and allows me to check which students are doing the test, which ones have completed it, and immediately check the result’.
Remote Testing Set-up
DPT runs on any device - laptop, desktop, tablet or mobile
As it was replacing a comprehensive, 3-hour examination, some modifications were made to the regular DPT set-up. The test would be supplemented by a 40-minute writing task and an online interview to provide a deeper insight into the candidates’ productive skills. It would also be invigilated, remotely. The process was as follows:
Students were split into test groups.
The Admin Panel was used to automatically send out Welcome Emails to all test-takers; this included login details, access instructions and test information (the DPT test-taker video guide).
Students with unreliable internet connections were advised to use the ‘Offline Mode’ of the Dynamic Placement Test.
Ms Yodi Yu used the Admin Panel to monitor the tests in real-time – viewing results and generating certificates once tests were completed.
DPT’s ‘follow up’ feature allowed for candidates to be automatically redirected to a separate writing task upon completion.
Invigilation
The randomised and sophisticated nature of DPT’s question types (such as word placement and sentence reconstruction) makes cheating exceedingly difficult, and this is compounded by the ‘Anomaly Tracker’, which flags any suspicious test-taker behaviour to the administrator. However, in this project Brunel decided to take test security a step further by conducting remote invigilation. Test administrators at Brunel used Zoom (video-conferencing software) to ensure the designated candidates were the ones (and the only ones) taking the test.
Outcome & Key Learnings
After the first few rounds of pilot tests, Yodi shared her key learnings. Some of the test-takers in the first few rounds of pilot tests had unstable internet connections, which caused audio issues for the test’s listening questions. Yodi found the ‘Offline Mode’ of the test a useful feature in preventing these issues. The Offline Mode downloads test content before the test begins and automatically uploads the results once an internet connection is re-established. The test-takers’ experience is identical to that of the ‘Online Mode’ – it simply prevents a poor internet connection from affecting the final result. ‘I would suggest that you let the students download the test at least one day before the test is due to start’, says Yodi.
With regards to DPT’s Admin Panel, Yodi found it “easy to use”. The team at Atlas English provided support in setting up student accounts, but she found the system intuitive: it was easy to login, navigate the Admin Panel, and monitor the students’ results herself.
Adapting to online testing in the midst of the covid-19 pandemic did present challenges. However, it also presented an opportunity to improve the existing assessment system, which is exactly what Brunel did. By adapting and taking advantage of existing resources and technologies, Brunel was able to effectively assess the English level of all applicants from the safety of their own homes.