Maintaining the credibility of degrees

Read the Insight brief online

• Introduction
• How degrees are classified
• Our regulatory approach
• Changes in degree classifications
  
  • Differential effects
  • Effects of the pandemic
  • Explaining the increases
• Perceptions
• What the OfS is doing
• Conclusion

Introduction

The reputation of English higher education depends in part on public confidence that the degrees that universities and colleges award, and the classifications they give to these awards, are reliable and consistent.¹ This means, for instance, that a graduate awarded a first class (1st) or upper second class (2:1) degree in maths in 2020 should have demonstrated broadly the same achievement as someone else with the equivalent qualification, whether they graduated in 2020, 2000 or 1980.

One aspect of the Office for Students’ (OfS’s) role is to ensure that graduates continue to benefit from their higher education, and this means ensuring that they can trust that their 1st or 2:1 continues to hold its value when compared with degrees awarded to the students who follow them. Studies show that there has been a significant increase over the past decade in the numbers of these higher grades awarded by English universities and colleges, and it is essential that graduates, employers and the public have confidence that this reflects genuinely improved performance by a large proportion of students, rather than reductions in the validity and reliability of assessment, or other changes in how classifications are calculated.

In itself, an increase in the numbers of students gaining higher grades is not necessarily a cause for concern. For example, if a university’s teaching improves for any reason, and its requirements for awarding a 1st or 2:1 remain constant, then more students are likely to earn these grades. In this case it is right for their achievement to be recognised in the class of degree awarded. However, it would be wrong if a university’s requirements were to be lowered so that more candidates were awarded a 1st or 2:1 without improvement in performance. The term ‘grade inflation’ is used to describe a situation where an increase in the awarding of higher grades does not reflect an actual increase in student achievement.

Our latest analysis of changes in degree classifications found that, between the academic years 2010-11 and 2020-21, the proportion of 1sts awarded in England has more than doubled, from 15.7 per cent in 2010-11 to 37.9 per cent in 2020-21.² It is possible that some of this growth arises for positive reasons, such as improvements in teaching quality, or greater effort on the part of students. However, without further explanation it is not clear whether, and in which cases, the cause may instead be a relaxing of requirements for achieving these degrees.

As the then Minister of State for Universities, Michelle Donelan, observed in response to our analysis, ‘Unjustifiable increases in the proportion of top degrees being awarded’ are detrimental.³ Our task is to identify where increases are likely to be a result of unjustifiable practices, and to respond accordingly through our regulation.

How degrees are classified

For the past century, British universities and colleges have generally categorised the undergraduate degrees they award into first, upper second, lower second and third classes. Students who do not meet the requirements for one of these may still be able to earn a pass degree or other form of recognition. While one institution will sometimes validate or franchise courses that are delivered by another, for the most part larger institutions act as their own independent degree awarding bodies. In England, degree awarding powers are conferred by the OfS.⁴ They are separate from university status.

There is no single unifying system of the kind seen in schools – whereby, for instance, the Office of Qualifications and Examinations Regulation (Ofqual) bases its expectation for GCSE outcomes on a standard cohort, mitigated by measurements of actual performance in the form of the National Reference Test in maths and English.⁵ Rather, in higher education, each degree awarding body is autonomous, deciding how to assess its own students and calculate the classifications for the degrees they are awarded.

Students are assessed against criteria that are linked to the curriculum of their specific course, developed according to the judgement of the academic staff who teach that subject. All institutions are responsible for ensuring that the awards they make properly reflect student achievement now, and that they retain credibility over time.⁶

Our regulatory approach

The OfS aims to ensure that every student, whatever their background, has a fulfilling experience of higher education that enriches their life and career. For this reason we regulate to ensure the quality of higher education, along with equality of opportunity.

To this end we set minimum requirements for quality, and expect that universities and colleges will meet or exceed them. This baseline represents the performance that students and taxpayers are entitled to expect. If universities or other higher education providers fail to meet it, then we will intervene. Meanwhile, we use influence and incentives to persuade them to exceed it. Our focus is on addressing cases where higher education in England falls short, precisely because we consider that much of it is of high quality and requires no such intervention.

Our strategy for 2022 to 2025 includes the objective that ‘All students, from all backgrounds, can progress into employment, further study, and lead fulfilling lives, in which their qualifications hold their value over time.’ In pursuit of this, one of our goals is that ‘Students are rigorously assessed, and the qualifications they are awarded are credible and comparable to those granted previously.’⁷ In our recent update to the OfS’s regulatory framework, published on 1 May 2022, degree classification is included under ‘Condition B4: Assessments and awards’.⁸ This includes requirements that all institutions ensure that students are assessed effectively through valid and reliable assessments, and that academic regulations are designed such that awards made to students are credible.

Among other things, this means that students must be assessed in an appropriately comprehensive and challenging way, and that the qualifications awarded to students reflect their knowledge and skills. The factors we may take into account in considering the latter point include changes in the numbers and proportions of classifications, both over time and compared with other universities and colleges.

Changes in degree classifications

In the mid-1990s, there was a general move in English universities from a norm-referenced classification system, which measured students’ performance relative to others on the same course, to the current criteria-based system.¹⁴ The norm-referenced system meant that, at only 7.3 per cent, the proportion of graduates in England gaining 1sts in 1995-95 had hardly changed since the 1960s.¹⁵ In contrast, by 2020-21 this figure had become 37.7 per cent.¹⁶

As Figure 1 shows, this proportion grew steadily from 15.7 per cent in 2010-11 until 2017-18, when it stood at 28.9 per cent. During the same period, the proportion gaining 2:2s and below had fallen by 11 percentage points, while the rate of 2:1s decreased much less sharply, by around two percentage points.¹⁷ While this growth was steady on average across the higher education sector in England, at individual universities and colleges it has sometimes been substantially more abrupt, sometimes involving growth of well over five percentage points in a single year.¹⁸

Figure 1: Degree classifications over time

Population: Full-time UK-domiciled undergraduates with classified degrees from higher education providers in England. Source: Higher Education Statistics Agency (HESA) student data, Education and Skills Funding Council Individualised Learner Record (ILR).¹⁹ Note: The y-axis does not start at zero.

In the following year, 2018-19, following our publication of earlier analysis and expressions of concern, the annual increases stalled, with very little change in any of the overall rates.²⁰ However, in 2019-20 there was a large increase in the rate of 1sts to 35.9 per cent, accompanied by a slight decrease in 2:1s and a sharp fall of over five percentage points in 2:2s and below. The extent to which this unprecedented change can be ascribed to the onset of the COVID-19 pandemic is discussed below. Following this, 2020-21 saw a further smaller increase in the proportion of 1sts, with an accompanying decrease in the proportions of other grades.²¹

While the exact proportions differ, the broad pattern of change in England in recent years reflects those recorded in the other nations of the UK.²²

Differential effects

The historical data shows differences in the classifications awarded to different groups of students. For example, as Figure 2 shows, in 2010-11, 44 per cent of black students gained 1sts or 2:1s, compared with 71 per cent of white students – a gap of 27 percentage points. By 2017-18 both figures had risen, to 58.9 per cent and 82.0 per cent respectively, but the gap in attainment had reduced by only a fifth, to 23.1 percentage points.

Figure 2: Gap in degree outcomes (1sts or 2:1s) between white and black students

Population: Full-time UK-domiciled undergraduates with classified degrees from higher education providers in England registered as white or black. This figure excludes students of other ethnicities. Source: HESA student data, ILR.²³ Note: The y-axis does not start at zero.

Meanwhile, as shown in Figure 3, between 2010-11 and 2017-18 the percentage of students with a declared disability achieving 1sts or 2:1s rose from 62.9 to 75.9 per cent, while that for students without a declared disability rose from 67.4 per cent to 78.7 per cent. This attainment gap, though always smaller than that between black and white students, reduced more substantially from 4.5 to 2.8 percentage points.

Figure 3: Gap in degree outcomes (1sts or 2:1s) between disabled and non-disabled students

Population: Full-time UK-domiciled undergraduates with classified degrees from higher education providers in England. Source: HESA student data, ILR.²⁴ Note: The y-axis does not start at zero.

These differences in respective attainment rates held largely steady in 2018-19, but the most recent changes affected them in different ways. Thus, the increase in 2019-20 was even higher among black students than among white students. The proportion of white students gaining a 1st or 2:1 increased by 4.3 percentage points from 2018-19 to 2019-20, while for black students the increase was 8.1 percentage points.²⁵

The same pattern is visible in the attainment of disabled students, though as stated the attainment gap before the pandemic was narrower. The proportion of disabled students who gained a 1st or 2:1 rose by 6.1 percentage points between 2018-19 and 2019-20, while that for non-disabled students rose by 4.9 percentage points.²⁶

The effect of the recent changes has thus been to reduce, though not eliminate, these differences.²⁷ It cannot be right that the chances of achieving a 1st or 2:1 are determined in part by a student’s personal characteristics, and the OfS has consistently sought to ensure that universities and colleges are working to close these longstanding gaps. As things stand, however, they have been reduced by increasing the proportion of higher classifications for black and disabled students, rather than addressing any historical grade inflation for white or non-disabled students, or for students overall.

These changes may nevertheless be a welcome sign of improvement in the sector. However, to be credible to the public, sudden substantial changes to previously intractable attainment gaps need to be explained. Here as elsewhere, it is essential that the classifications awarded are a genuine reflection of student achievement.

By contrast, the gap in attainment of 1sts and 2:1s between mature and younger learners, which at around 10 percentage points in 2018-19 was comparable in size with that between white and black students, narrowed by only 0.2 percentage points in 2019-20.²⁸

As Figure 4 shows, another striking difference applies across the range of students entering higher education with different grades in their A-levels or equivalent qualifications. Long-term increases in 1sts and 2:1s are visible across all sets of qualifications. The differences for certain groupings, predominantly those with the weakest A-level scores, are prominent across the decade.²⁹

Figure 4: Degree attainment by entry qualifications for academic years 2010-11 and 2018-19 to 2020-21

Population: Full-time UK-domiciled undergraduates with classified degrees with the entry qualifications shown, from higher education providers in England with sufficient data for all the years considered. Source: OfS, ‘Analysis of degree classifications over time: Changes in graduate attainment from 2010-11 to 2020-21’, based on HESA student data, ILR.³⁰ Note: The y-axis does not start at zero.

Effects of the pandemic

As noted, the largest overall increase in 1sts took place in the year of onset of the COVID-19 pandemic, when many universities changed aspects of their assessment methods to respond to the exceptional circumstances. However, the lack of change in 2018-19 relative to 2017-18 already represented a departure from previous trends, making it difficult to envisage how attainment might have looked in 2019-20 in the absence of the pandemic.

Not all universities and colleges responded in the same way to the pandemic, and we cannot provide a definitive reason for this increase from the information we currently have. Notably, though, many universities’ pandemic strategies included ‘no detriment’ and ‘safety net’ policies, designed to protect students’ results on the basis of their past assessments and to take into account the impact of the pandemic on the multiple personal factors underlying performance. Universities and colleges also expanded online student support, in many cases moving to entirely online delivery of courses and assessment.³¹

Some of these universities have argued that that, rather than reducing rigour, these policies improved their students’ confidence and reduced their anxiety, leading to a real improvement in achievement and a deserved increase in the degree classifications awarded.³² These arguments have not yet been tested.

The pandemic and its associated lockdowns may have had other indirect effects. For instance, it is not difficult to imagine that the move to wholly online teaching, without the need for physical attendance, could have helped bridge the attainment gap between disabled and non-disabled students; or that mature students, who are more likely to have families and whose caring responsibilities would thus have increased during school and nursery closures, would have seen no comparable jump in attainment.

The most recent growth in higher degree classifications coincides with the pandemic, and we need to avoid any grade inflation arising from those exceptional circumstances becoming a permanent feature of the future classification system. However, the longer-term trend also requires further investigation and explanation.

Explaining the increases

The OfS analysis of the overall increases at 143 registered higher education providers from 2010-11 to 2020-21 used a statistical model to predict the outcomes from 2011-12 onwards, based on the changes in the characteristics of graduates during these years.³³ This model is not intended to be prescriptive and does not represent an ‘expectation’, in the sense of what we would wish to see happen. Rather, it is a description of the state of affairs we would expect to play out based on certain assumptions, including that there were no changes in the proportions of particular classifications awarded to students with different characteristics.

Comparing the actual results with those predicted based on the shifts in demographic proportions reveals the changes that can be regarded as ‘unexplained’ by the model. In calculating the predictions, an assumption is made for the sake of argument that the outcomes for specific demographic groups will remain the same over time. The analysis says nothing about whether or why these might in fact change, and does not imply that historical differences in performance between student groups are acceptable.

The prediction shows the proportion of 1sts falling between 2010-11 and 2018-19, by 2.3 percentage points, whereas in reality it increased by 13.8 percentage points. This means that, according to the model, 16.1 percentage points of the 29.7 per cent obtaining 1sts in 2018-19 represent unexplained change from the outcomes in 2010-11.³⁴ Following the additional changes seen during the pandemic, the figure for 2020-21 shows 22.4 percentage points of 37.9 per cent representing unexplained change. The annual progression is illustrated in Figure 5.

Figure 5: Observed and unexplained sector-level changes in first class degree attainment

Population: Full-time UK-domiciled undergraduates with classified degrees, from higher education providers in England with sufficient data for all the years considered. Source: OfS, ‘Analysis of degree classifications over time: Changes in graduate attainment from 2010-11 to 2020-21’, based on HESA student data, ILR.³⁵

The OfS has been clear that those universities that consistently recruit students from underrepresented groups, whose performance has historically been lower, should improve outcomes for these students rather than tolerating lower ones.³⁶ It may therefore seem anomalous that our analysis applies a methodology that assumes such students’ performances remain constant, and treats improvement in their outcomes as ‘unexplained’. However, even an unprecedented improvement in such performance could only account for a fraction of the unexplained achievement.

A second set of predictions, based on the same statistical model, shows how we would expect the outcomes to look in 2020-21 if attainment gaps had completely closed between students of different sex, ethnicity and disability and area-based disadvantage.³⁷ This shows that, even assuming all graduates attained at the highest levels predicted by the model, some 17.4 percentage points of the 37.9 per cent obtaining 1sts in 2020-21 represent unexplained change.³⁸ While lower than 22.4 percentage points, this proportion is clearly high. Previous analysis confirms that this finding also holds for 2018-19 graduates.³⁹

These results are in line with those of earlier OfS reports carrying out similar analysis, which also found unexplained changes for the years prior to 2020-21.⁴⁰ The analysis concludes that over the period, nearly all the universities and colleges included had significant increases in both the absolute and the unexplained awarding of 1sts, based on a comparison with the higher education sector as a whole in 2010-11.⁴¹

This considerable divergence of the actual results from what would have been expected, even in a best-case scenario, requires explanation. To explain the change convincingly, there would have to be good reasons why all students, of all demographics, had undergone unprecedented improvements in their actual performance up to 2017-18, and again in 2019-20.

In the absence of specific accounts of how this can have happened, the understandable assumption of employers and the public will be that these increases are due to grade inflation in the current system. To maintain confidence in the current degree classification system, and the awards made to students, universities must convincingly demonstrate that students are assessed effectively, that assessment is valid and reliable, and that the qualifications awarded properly reflect students’ knowledge and skills.

Perceptions

As noted, the issue of grade inflation is significant for the reputation of English higher education, if members of the public do not perceive the increases seen since 2010-11 to be credible. And as recent media coverage (and the coverage of our past reports on degree classification) has shown, the opinions it evokes can diverge strongly. Some commentators believe that students’ status as consumers has caused universities to remove rigour from assessment and so driven up degree classifications, making it impossible to discriminate between excellent graduates and those who are merely adequate.⁴² Others have suggested that the OfS risks penalising institutions for genuine improvements in the quality of teaching.⁴³

Some media reports of our recent analysis highlighted the striking fact that, while the rate of achievement of 1sts nearly doubled between 2010-11 and 2020-21 among students entering with grades AAA or above at A-level, for those who entered with grades DDD or below the rate more than quintupled.⁴⁴ However, the latter group represents a very small proportion of the overall student population, between 1 and 2 per cent of the total number of qualifiers. Whether this change represents grade inflation, real improvement or a statistical fluke will bear investigation, but it has little impact on the overall number of 1sts, and thus on the question of their credibility.

Studies before the pandemic suggested that increases in higher classifications since the late 20th century were not confined to the UK, but were observable in, for instance, the USA, Canada and Germany. This was attributed to causes including the commodification of higher education, competition between universities and the linkage of academic tenure for staff to good outcomes. There was less evidence of it in Australia, where there was no common grading system and little public dissemination of grades.⁴⁵

During the pandemic British schools also saw significant increases in grades, after procedures were changed so that GCSE and A-level outcomes relied on teacher assessments rather than externally validated exams. Ofqual has already announced plans to return GCSE and A-level grades to their proportions before the pandemic in 2023, with 2022 representing a transitional year.⁴⁶ The public expectation that universities should do likewise is understandable, and Universities UK has rightly committed to achieving this.

Unwinding the impact of the changes made during the pandemic does not, however, explain or resolve the longer-term trend. The question of credibility therefore remains, entangled perhaps with more fundamental questions about whether the classification system is fit for purpose.

What the OfS is doing

The OfS’s view is that grade inflation is bad for students, graduates and employers, and damages the reputation of English higher education.⁴⁷ This does not mean that we assume all unexplained improvements in degree classification are unmerited, and we understand that different factors were in play during the pandemic.

However, the most disruptive aspects of the pandemic will now no longer be placing pressure on university assessment practices. We therefore expect universities’ requirements for 1sts and 2:1s, and the proportions of students attaining these classifications, to revert to their pre-pandemic levels as Universities UK has agreed. At the same time, we are interested in establishing the causes of the steady growth in higher degree classifications over the years preceding the pandemic.

We will continue to use the data and other information available to us to identify courses and universities that may not be meeting our regulatory requirements relating to assessment and awarding practices. This will include considering how far these longer-term changes in degree classifications can be explained, statistically or otherwise, and thus whether the qualifications awarded to students are credible.⁴⁸

We know that a range of factors might underlie such changes, some of which could be positive and would probably not affect compliance with our conditions. In line with our general regulatory approach, we will focus our investigation and enforcement activity on cases where substantial increases cannot be explained, either by our data analysis or by other evidence, or where the explanations raise concerns.⁴⁹

Our risk-based approach means that we will not investigate every higher education provider where there may be a cause for concern. Instead we will investigate in a targeted way, based on the information available to us. We are willing to intervene to protect the credibility of awards, and would expect to report on the findings of our investigations.

Our expectation is that this approach will encourage all universities and colleges to ensure that their assessment and awarding practices remain rigorous and credible.

Conclusion

In 2019-20, English universities made significant changes to their assessment policies and practices, which coincided with a significant increase in the classifications awarded to students whose lives and circumstances had themselves changed radically. It is right that the sector now takes steps to reverse those pandemic-related increases. However, these changes followed a decade in which the proportions of 1sts awarded had grown considerably, and mostly steadily, and that too requires further exploration and explanation.

Public trust in the consistent value of degrees is essential to ensure that employers can continue to have confidence in the abilities of the graduates they employ. Large increases in higher classifications inevitably erode such confidence, unless they can be explained with evidence showing that they reflect genuine improvement in the achievement of graduates.

There is a clear need for this debate to be informed by further research and inquiry, whose outcomes must be disseminated in a manner accessible to the public at large. We intend, through our investigations, to increase the information available about these trends and the explanations for them, and through our regulation to work to correct them where this is appropriate. This approach is designed to stabilise the public perception of degrees, and to re-establish trust in the credibility of qualifications awarded to students, to the benefit of future graduates, employers and taxpayers alike.