More About Mastery, Depth and High Attainers

 

I am increasingly concerned about NCETM’s notion that ‘stretch and challenge’ should always involve studying the same material in greater depth.

This is becoming increasingly pervasive and resulting in widespread confusion, amongst teachers as well as parents. For example see this Mumsnet thread and this sample of recent Tweets

.

https://twitter.com/FJStarfish/status/646923902824919040

.

Backstory

In April 2015 I published a post on the Gifted Phoenix Blog comparing the one-dimensional approach advocated by the NCETM with an earlier model of top-end differentiation that utilises enrichment (greater breadth), extension (more depth) and acceleration (faster pace) in responding to high attainers’ widely differing needs.

I showed how the NCETM has steadily adjusted its position, to a point where it is now directly at odds with the National Curriculum.

The National Curriculum contains an inclusion statement:

‘Teachers should set high expectations for every pupil. They should plan stretching work for pupils whose attainment is significantly above the expected standard.’

There is flexibility:

‘…to introduce content earlier or later than set out in the programme of study. In addition, schools can introduce key stage content during an earlier key stage, if appropriate.’

Critically, the maths programmes of study say (my emphasis):

‘The expectation is that the majority of pupils will move through the programmes of study at broadly the same pace. However, decisions about when to progress should always be based on the security of pupils’ understanding and their readiness to progress to the next stage. Pupils who grasp concepts rapidly should be challenged through being offered rich and sophisticated problems before any acceleration through new content. Those who are not sufficiently fluent with earlier material should consolidate their understanding, including through additional practice, before moving on.’

Yet the NCETM has progressively shifted its line until what the National Curriculum prescribes for the majority is applied to all learners, regardless of their prior attainment.

By April 2015 its Director was arguing that the statement above:

‘…directly discourages acceleration through content’

It does no such thing.

He claims that teaching for mastery in the Far East, as well as practice to date in England:

‘…suggests that all pupils benefit more from deeper understanding than from acceleration to new material’

He supplies no evidence to support this contention and fails to appreciate the possibility that these two elements need not be mutually exclusive.

This is not a negligible adjustment. As my earlier post notes, the mathematical difference between ‘a majority’ and ‘all’ may be almost 50% of the cohort.

Since I wrote the original post there has been a series of new developments

.

NCETM Assessment Materials

In August 2015 the NCETM published a set of Assessment Materials ‘to help teachers make judgements on the degree to which pupils have acquired mastery of the mathematics curriculum’.

The materials are specific to the primary sector, including a separate booklet for each year from Y1 to Y6. By 1 September NCETM was claiming 35,000 downloads.

This commentary takes the Y6 booklet as an example. The opening introduction begins:

‘In line with the curricula of many high performing jurisdictions, the National Curriculum emphasises the importance of all pupils mastering the content taught each year and discourages the acceleration of pupils into content from subsequent years.’

As shown above, that statement is reading rather more into the National Curriculum than it actually says. As if to prove the point, the introduction goes on to quote it.

In relation to mastery, the materials differentiate between:

• A set of principles and beliefs, including the unexceptionable statement that ‘all pupils are capable of understanding and doing mathematics, given sufficient time’.

• A mastery curriculum advocating ‘one set of mathematical concepts and big ideas for all’:

‘There is no such thing as ‘special needs mathematics’ or ‘gifted and talented mathematics’. Mathematics is mathematics and the key ideas and building blocks are important for everyone.’

• Pedagogical practice to:

‘…keep the class working together on the same topic, whilst at the same time addressing the need for all pupils to master the curriculum and for some to gain greater depth of proficiency and understanding. Challenge is provided by going deeper rather than accelerating into new mathematical content.’

• Achieving mastery of aspects of maths, including being able to apply one’s knowledge ‘in new and unfamiliar situations’.

The section on mastery with greater depth begins by citing Tim Oates:

‘The research for the review of the National Curriculum showed that it should focus on “fewer things in greater depth”, in secure learning which persists, rather than relentless, over-rapid progression.’

This is about the shape of the curriculum as a whole rather than provision for ‘stretch and challenge’ within it.

As noted in my previous post, the National Curriculum Expert Panel, which Oates chaired, understood that it needed to undertake further work on ‘stretch and challenge’:

‘…with respect to children with learning difficulties and those regarded [sic] as high attainers’.

This was commissioned but never published, so creating the policy vacuum that NCETM is now exploiting.

The new materials assert that learners who progress more rapidly will need such stretch and challenge:

‘However, research indicates that these pupils benefit more from enrichment and deepening of content, rather than acceleration into new content. Acceleration is likely to promote superficial understanding, rather than the true depth and rigour of knowledge that is a foundation for higher mathematics.’

They go on to explain that:

‘Developing mastery with greater depth is characterised by pupils’ ability to:

• solve problems of greater complexity (i.e. where the approach is not immediately obvious), demonstrating creativity and imagination

• independently explore and investigate mathematical contexts and structures, communicate results clearly and systematically explain and generalise the mathematics.’

The materials exemplify some tasks and activities that fit this description and which ‘might be used as deepening tasks for pupils who grasp concepts rapidly’.

NCETM is unable to cite the research that justifies its preferred position on breadth, depth and pace.

A footnote refers to a 2012 publication by the independent Advisory Committee for Mathematics Education (ACME). This document is focused on those 5-16 year-olds ‘with the potential to successfully study mathematics at A level or equivalent’.

It too advocates depth in place of acceleration, arguing that this should be embodied in an extension of the National Curriculum programme of study relevant to some 30% of the cohort, with assessment targeted at 20-25% of the cohort.

There is not space here to provide an extended critique of ACME’s paper, but four key points stand out:

• ACME acknowledges that a few exceptional learners ‘may require bespoke provision’. This is elsewhere quantified as ‘the top 1% (or so)’ though no basis is given for this estimate. No attainment benchmark is supplied either. Assuming that A* at GCSE is pitched too low, perhaps one might stretch a point and set it at PISA Level 6, achieved by 3.1% of English learners in PISA 2012 (and almost 31% of their peers in Shanghai).

• The dragon that ACME is slaying is not acceleration per se (by which it means progressing through the curriculum at a faster pace) but poor acceleration practice. It is perfectly feasible to introduce checks and balances that eliminate the shortcomings identified.

• ACME is guilty of treating acceleration (faster pace) and extension (greater depth) as mutually exclusive when, as my previous post explained, both can happily coexist – and often do so in any given task or problem.

• The research evidence is once more conspicuous by its absence. A 2012 Ofsted report is cited rather misleadingly, because it says nothing specific on the topic. We are instructed that acceleration simply doesn’t happen in ‘many of the world’s mathematically most high-performing jurisdictions’, but no evidence is supplied.

NCETM has reproduced unquestioningly the unsubstantiated arguments advanced by ACME. It is pursuing a prescriptive approach to top-end differentiation and advocating it directly to schools. It does not acknowledge the possibility of viable alternatives.

.

Report of the Commission on Assessment Without Levels

The Commission’s delayed Report was published on 17 September 2015.

NCETM produced a press release stating that the Commission had taken evidence from one its staff and claiming that the Report

‘…endorsed the approach embodied by the NCETM’s recently published free materials to help teachers assess pupils’ mastery of mathematical concepts.’

The Report contains a section called ‘Mastery in Assessment’ which also discusses the different ways in which the term is deployed. It distinguishes two related but different pedagogical models:

• The first embodied in ‘the recent promotion of Mathematics Mastery’ (which might or might not include NCETM’s activity), where the term:

‘…denotes a focus on achieving a deeper understanding of fewer topics, through problem-solving, questioning and encouraging deep mathematical thinking. Also sometimes associated with this ‘mastery’ approach is a belief that all children can achieve a high standard and that the purpose of assessment is not differentiation, but ensuring all children have grasped fundamental, necessary content.’

• The second embodied in ‘mastery learning’:

…a specific approach in which learning is broken down into discrete units and presented in logical order. Pupils are required to demonstrate mastery of the learning from each unit before being allowed to move on to the next, with the assumption that all pupils will achieve this level of mastery if they are appropriately supported. Some may take longer and need more help, but all will get there in the end.’

The National Curriculum is said to follow in this second tradition:

‘It is about deep, secure learning for all, with extension of able students (more things on the same topic) rather than acceleration (rapidly moving on to new content). Levels were not consistent with this approach because they encouraged undue pace and progression onto more difficult work while pupils still had gaps in their knowledge or understanding.’

Unusually the Commission cites evidence in support of mastery learning:

‘A large amount of high-quality research has evaluated mastery learning and found consistent and positive impacts on learning (e.g. Kulik et al, 1990; Guskey, 2012)’.

What are we to make of this?

It is clear that the Commission does not endorse the belief within Mathematics Mastery that ‘the purpose of assessment is not differentiation’, since differentiation features in its subsequent principles of in-school summative assessment:

‘How will it be used to support broader progress, attainment and outcomes for the pupils?

For example: how the information provided by the assessment can support the following year’s teacher in differentiating the support given to pupils in the class to achieve the positive outcomes.’

When it comes to mastery learning, the definitions of acceleration (pace) and extension (depth) it employs are simplistic and potentially misleading (see my earlier post for further explanation of these concepts).

As George Constantinides has highlighted, this statement begs important questions about what constitutes ‘rapid’ progression and whether more measured acceleration would be acceptable for at least some high attainers.

It also falls into the same trap as NCETM and ACME, by assuming that pace and depth must be mutually exclusive.

In its response to the Report, the Government feels obliged to clarify the Commission’s statement (while pretending it was right all along):

‘The Commission’s definition of mastery in assessment is particularly helpful and reflects an underpinning principle of the new curriculum that pupils should achieve a secure and deep understanding of the whole curriculum content before being moved on to new content. It is important that schools, and those who support schools, have a clear and singular definition of mastery.’

This broadly reflects the terminology deployed in the National Curriculum. Although ‘the whole curriculum content’ threatens to open a different can of worms, it does not endorse the ‘depth not pace’ formulation espoused by NCETM and now advanced by the Commission.

As George Constantinides says:

‘The fundamental principles outlined are, I think, uncontroversial – ensuring children know their stuff before moving on…this is just good practice in any teaching, new or old curriculum – when combined with providing children enough opportunity to demonstrate mastery….

…Of course children should not be moved onto content until they are secure with prior content. Of course it might be possible to identify lots more content on “the same topic” without straying into content from a later key stage (though I have yet to see good examples of this – publish them, please!) But let’s be clear: the national curriculum does not say that acceleration is unacceptable.’

Finally, the research evidence in support of ‘mastery learning’ needs to be set against the more balanced assessment in the EEF’s Toolkit:

‘Lower attaining pupils may gain more from this strategy than high attaining students, by as much as one or two months’ progress…

… Overall, the evidence base is judged to be of moderate security. There is a large quantity of research on the impact of mastery learning, though much of it is relatively dated and findings are not consistent. In addition, most meta-analyses examining mastery learning use statistical techniques which may inflate the overall effect size so some caution is needed in interpreting the average impact. Having noted these concerns, a more recent study in the US found that mastery learning approaches can increase learning by up to six months in maths for 13-14 year olds, which is consistent with several older studies.’

Elsewhere the Commission repeats its contention that levels fostered the over-emphasis of pace:

‘Too often levels became viewed as thresholds and teaching became focused on getting pupils across the next threshold instead of ensuring they were secure in the knowledge and understanding defined in the programmes of study. Depth and breadth of understanding were sometimes sacrificed in favour of pace.’

This reflects bad practice rather than an inherent problem with the model.

It does not follow that the abolition of levels should lead to the wholesale rejection of faster pace as a differentiation tool for high attainers. Yet that is a real danger in the absence of clear guidance on this issue.

The Commission advocates an approach to assessment that is ‘inclusive of all abilities’ but has nothing of substance to say about high attainers. An expert group has been established to consider assessment for those working below the level of national curriculum tests, but there is no corresponding activity to consider the implications for those at the opposite end of the attainment spectrum.

.

Statutory Interim Frameworks for Teacher Assessment

The Government has also published delayed Frameworks to support teachers in making statutory teacher assessment judgements in 2016.

The Frameworks are interim, in that they apply for a single year only, because the Government is still ‘evaluating options for future years’.

In the response to consultation on the initial version of these frameworks, the Government had suggested that the Commission would help to revise them, but no link is evident in the documents, other than that they were published on the same day. It may be that the frameworks do not fully reflect the Commission’s advice, hence their interim status.

For those assessments where the outcomes of teacher assessment are most significant – KS1 reading, writing and maths and KS2 writing – the frameworks include three different performance categories:

• Working towards the expected standard

• Working at the expected standard and

• Working at greater depth within the expected standard

The remainder – covering KS1 science and KS2 reading, maths and science – have only the single category ‘Working at the expected standard’.

All of the commentary I have seen is obsessing that this ‘expected standard’ has been ratcheted up beyond the expected L4b equivalent to something nearer L5c or even 5b. Teachers and schools are deeply worried about the likely negative impact on their performance statistics.

But they are completely ignoring the impact on their high attainers. The structure of the frameworks mean that, for 2015 at least, there is no provision within statutory teacher assessment for schools to report how many pupils are ‘working beyond the expected standard’.

There is of course nothing to prevent schools from incorporating a ‘working beyond’ category in their own in-school assessment procedures, and to use that within the school and when reporting to parents.

But past experience suggests that, given widespread concerns about teacher workload, the fact that such a standard does not need to be reported will cause it to wither on the vine in many schools.

These frameworks are a direct indication that depth trumps pace where primary ‘stretch and challenge’ is concerned – and that this now applies in reading and writing as well as in maths.

It is important that schools interpret it correctly, to mean ‘depth before pace’ rather than the NCETM’s ‘depth instead of pace’. But I have little confidence that they will do so.

Paradoxically the NCETM’s own assessment materials are now out of kilter with the KS2 maths framework, which makes no provision for working at greater depth. Why?

One assumes that, in evaluating future options, the Government will be considering carefully the likely impact of the 2016 version, including the read-across to end-of-key-stage tests.

The original consultation on primary assessment said that:

‘Key stage 2 national curriculum tests will include challenging material (at least of the standard of the current level 6 test) which all pupils will have the opportunity to answer, without the need for a separate test.’

The draft test frameworks are less specific. For example, the KS2 maths framework says:

‘The end of key stage 2 mathematics test should:

• use appropriate means to allow all pupils to demonstrate their mathematical fluency, solving problems and reasoning.

• provide a suitable challenge for all pupils and give every pupil the opportunity to achieve as high a standard as possible…’

I need professional assistance to judge whether the KS2 sample tests include material ‘at least of the standard of’ L6. But to my untutored eye it seems that all the challenging material is confined to the KS2 programme of study, meaning that the challenge comes exclusively from studying that material in greater depth.

This is different to the former Level 6 tests, for which the guidance stated that:

‘Teachers will need to have covered enough of the key stage 3 programme of study to be assured that a pupil is working at level 6…Questions drawn solely from the key stage 3 programme of study will often be provided with ‘scaffolding’, so that they are accessible to children working at level 6.’

Assuming appropriate safeguards are in place to ensure that high attainers are secure in their prior learning, surely neither tests nor teacher assessment should impose a ceiling lower than in the previous system, otherwise there is a real risk that a National Curriculum designed to raise standards will be seen as ‘dumbing down’ instead?

Do teachers have the capacity and expertise to construct suitable extension tasks, equivalent in challenge to anticipating elements of the KS3 programme of study? Do they understand when and how to combine this with acceleration into the next Key Stage?

.

Last word

In short, it should be perfectly possible to:

• ensure security of understanding before high attainers progress to new content

• prioritise depth over pace (and breadth) when differentiating provision for high attainers, while also retaining opportunities to utilise breadth and pace where appropriate

• recognise those who achieve more through any combination of breadth, depth and pace – assessing their improvements on each dimension (and possibly combining these in a single performance measure).

Given the limitations of the research evidence we should think carefully before curtailing opportunities for high attainers to move at a faster pace, especially where that is most appropriate to their individual needs and circumstances. Flexibility is always preferable to a straitjacket, surely?

We must also beware the trap of assuming that, if we unquestioningly adopt (or even adapt) Shanghai’s practice, we will necessarily increase the percentage of Level 6 performers on future PISA assessments.

Meanwhile teachers and schools need much clearer guidance on these issues.

.

TD

September