Monday, 24 April 2017

Is there a benefit to post-16 remedial policies?

Clémentine Van Effenterre, a researcher at the Paris School of Economics and CVER, reviews the impact of remedial interventions for post-16 students


Remedial interventions in tertiary education are under scrutiny in most OECD countries. They are particularly important in a context of increasing demand for skilled workers. However, they are often costly, and their efficiency in boosting student performance has been questioned. This debate has gained particular relevance in England given recent policy changes that require students who do not get at least a grade C in English or maths in GCSE to repeat exams in these subjects. The low pass rate amongst those who re-sit has raised questions about the sustainability of the policy. What can be done to improve mathematics and English attainment to help students achieving these new requirements? What types of remedial interventions are efficient to address the need of students older than 16? In this context, we have reviewed economic literature on the impact of remedial interventions in tertiary education.

Remediation has gained increasing attention in the recent research in the economics of education, especially in the US, where nearly one-third of first-year college students participate in remedial courses in reading, writing, or mathematics. Traditionally, studies have compared students assigned to remediation to their peers who have not been assigned to these courses, and have found a negative association between remediation and students’ future performance. However, it is difficult to infer causality because the characteristics of those who participate in remedial courses are often different to those of individuals who do not participate, and not all of these differences are easy to capture in surveys. The widespread use of rigorous methodologies and access to new datasets have significantly improved our understanding of the impact of remedial programmes on student outcomes. The most recent studies show that remediation programmes can in principle generate positive results, but often do not. Indeed in principle there may be negative effects that offset positive effects. It is not clear what determines whether remediation programmes are effective or not.

Researchers have tried to open up the ‘black-box’ of remediation, and to investigate the impact of various remedial tools, such as mentoring, peer-mediation, and IT-based approaches. New pedagogic approaches designed to boost students’ outcomes deserve a closer look, although there are few rigorous evaluations assessing their impact. Studies that evaluate mentoring approaches have found evidence of positive effects and interestingly find that face-to-face services cannot easily be replaced by low-cost technology such as text messages. Another interesting finding is that combined approaches (such as academic support services and financial incentives) can be more effective than the provision of one of these services in isolation. It is also important to note that even when interventions find positive effects in the short run, they can quickly fade out in later years. Finally, studies often find the impact of remediation to vary according to students’ characteristics. For example, in certain contexts, women, older students and lower-achieving students have been found to benefit more from remediation services. There is a critical need for more research using rigorous methodologies to understand why certain types of students are more (or less) responsive to certain interventions, and to tailor interventions and pedagogies accordingly.

"Post 16 remedial policies: a literature review", CVER Research Paper 005 is available at http://cver.lse.ac.uk/publications/

Tuesday, 28 March 2017

The decision to undertake an apprenticeship

CVER's Steven McIntosh, from the University of Sheffield, discusses what influences the decision to do an apprenticeship



What is likely to influence the decision-making of young people who are thinking about undertaking an apprenticeship? In this blog I discuss some research we have undertaken in CVER, answering just this question. The data source is the responses to a questionnaire that we developed ourselves, given to a cohort of apprentices at the Advanced Manufacturing Research Centre (AMRC) at the University of Sheffield. These apprentices were surveyed in January-March 2016, having all begun their apprenticeship in September 2015. In total, 61 apprentices responded to our questionnaire (a response rate of around 50%).

The surveyed apprentices are a homogenous group. Almost all are male, white and non-disabled. They are mostly young (two-third were aged 16-18), and their apprenticeships are mostly in either the Maintenance or the Mechanical Manufacturing framework. One source of dissimilarity across apprentices is their socio-economic background, with half having at least one parent in a professional or managerial occupation, and half not. In terms of qualifications obtained prior to their apprenticeship, they are a well-qualified group. Almost all hold GCSEs at grade C or above, including English and Maths. 30% of the apprentices were already qualified to Level 3 before their apprenticeship, either through A levels or vocational qualifications. This is an important point because it means that they would have had other options, such as pursuing the academic route towards a university degree, other good quality technical qualifications, or perhaps employment. What factors, then, led to them choosing an apprenticeship over the other options that they were sufficiently qualified to consider?

I group influences into two primary sources: school, and friends and family. Figures 1 and 2 below illustrate the level of encouragement that the apprentices received from these two sources respectively, with apprentices responding on a five point scale (from ‘encouraged a lot’ to ‘discouraged a lot’). In each case, the first row in the figure shows the proportion in each category amongst all apprentices, with subsequent rows disaggregating the group by two variables of interest.

Figure 1 shows that the apprentices report very little encouragement from their previous school to undertake an apprenticeship. Just 3% said that they had been encouraged a lot by their school, with a further 20% saying that they had been encourage a little. Thus only around a quarter of the sample were encouraged, balanced by a very similar proportion who said that they had been actively discouraged from undertaking an apprenticeship by their school, who presumably advocated other routes that the young people should follow. Over half of the sample report being neither encouraged nor discouraged by their schools. For this group, either information about apprenticeships was offered without advice as to whether such a route was appropriate, or apprenticeships were simply not discussed at all at school.

Disaggregating the sample according to the apprentices’ prior qualification levels, we can see that it is the apprentices already qualified to Level 3 who are particularly likely to have received neither encouragement nor discouragement to follow an apprenticeship. Amongst the lower qualified group, discussions are more likely to have occurred, with a higher proportion in both the encouraged and discouraged groups. This may reflect teachers encouraging the more able amongst the Level 2 qualified group, while discouraging others they consider to be less able to cope. With respect to age group, the numbers suggest more discussion of apprenticeships amongst the younger group (aged 16-18), though again this is evenly split between encouragement and active discouragement.


Figure 1: Encouragement from School for Doing an Apprenticeship



When we look at Figure 2, considering encouragement from family, the contrast with Figure 1 is stark. Over half of the apprentices (56%) were encouraged a lot by their parents to do an apprenticeship, with a further 17% encouraged a little. Only 3% (2 respondents) were actively discouraged by their parents. It seems to be the younger group who have been particularly encouraged by their parents, with the older group receiving less advice either way and being more likely to be left to make their own decision. Disaggregating by socio-economic status makes little difference. If anything, those apprentices from a higher socio-economic background (defined as having at least one parent in a professional or managerial occupation) seem more likely to have been encouraged. Certainly, there is little evidence that parents from a professional background are more likely to try to discourage their children from undertaking an apprenticeship.


Figure 2: Encouragement from Parents for Doing an Apprenticeship


The two figures above therefore show the relative importance of family, compared to school, for encouragement about apprenticeships. The influence of individuals close to the apprentices is further revealed by answers to questions asking them whether they know current or previous apprentices. Almost half know a family member who has undertaken an apprenticeship, and over three-quarters have a friend similarly engaged. Only 12% do not know anyone who has done an apprenticeship. Family and friends therefore seem to be the most important source of information about apprenticeships, a hypothesis confirmed when apprentices were directly asked for the source of their information about apprenticeships.

The survey responses discussed above therefore suggest that, for apprenticeship engagement to spread further amongst young people, then schools and careers advisers have an important role to play in providing information about this route. In this way, young people without personal knowledge of apprenticeships through friends and family can also acquire the knowledge they need to make informed choices about whether apprenticeships are appropriate for them. As more young people undertake an apprenticeship, then the informal networks via friends and family will also expand and develop further, so that apprenticeship may become a more widely accepted, and expected, option for young people, as we see in countries such as Germany.


"The Decision to Undertake an Apprenticeship: A Case Study", CVER Briefing Note 002 is available at http://cver.lse.ac.uk/publications/

Friday, 24 March 2017

The benefits of vocational education for low-achieving school leavers

Vahé Nafilyan, from the Institute for Employment Studies, writes on CVER's latest research paper which looks at a previously neglected group: school leavers starting low level vocational courses


Every year, about 65,000 school leavers start low level vocational courses. As underlined in a report by the House of Lords Select Committee on Social Mobility these young people have received much less attention than those who go on to A-Levels and university and, at the other end of the spectrum, the small minority dropping out of education, employment or training. Although this is a sizeable group (10% of a cohort), their participation in vocational education and labour market outcomes have so far been barely documented.

In a new paper we analyse in detail the school-to-work transition of young people who left school in the summer of 2011 and started low level vocational courses, known in England as ‘Below Level 2’ (BL2). Using newly available linked administrative data, we examine the characteristics of BL2 learners, their learning trajectory and their labour market outcomes.


Young people who start BL2 courses after leaving secondary school are amongst those who have faced the greatest challenges in secondary school. They have extremely weak GCSE results. Nearly all of them failed to achieve five A*-C GCSEs, the typical requirement to start A-Levels; 68.5% of them did not achieve a single A*-C GCSE, a proportion higher than amongst those who left education and got a job (48.6%) or became ‘Not in Employment, Education or Training’ (NEET, 60.7%). BL2 learners were already struggling at school prior to GCSEs. Over half of them performed below the expected level in English and maths at Key Stage 3 (i.e. national examinations taken at 14 years old). In addition, 39.0% of them report having disabilities, learning difficulties or other health problems.


BL2 courses have similar characteristics: most programmes require the completion of 300 to 400 Guided Learning Hours (GLH.), of which 200 to 300 GLH are devoted to the main subject. The main subject for about a third of all young people is Preparation for Life and Work, which focuses on personal and social skills for the purpose of personal growth and further engagement in learning. Amongst the subject areas with a clear vocational focus, the most popular are Construction (23.8%), Engineering (15.5%) and Retail (12.3%).

We find significant progression in education by the BL2 learners. Nearly 80% of BL2 learners made successful transitions leading into continued employment, higher level college-based vocational education or apprenticeships. Three years after leaving secondary school, over half of the BL2 learners achieved a qualification at, or above, Level 2.


However, one in five BL2 learners drop out of education and become and remain NEET. Our analysis suggests that females are at higher risk of becoming NEET than males. Those coming from more disadvantaged areas – as measured by the Index of Multiple Deprivation – are more likely to become NEET compared to those from more privileged backgrounds. Achieving the initial BL2 course is associated with lower risks of becoming NEET.


Looking at labour market outcomes in the 2015 tax year, we find that 59.6% of BL2 learners were doing some paid work, and those who did some paid work earned on average £6,724. Employment rates and earnings are higher for learners who studied Engineering, Construction or Retail compared to Health, Public Service and Care, and Preparation for Life and Work.


Our results also suggest that BL2 learners who achieved their initial vocational learning aim fare better in the labour market than those who did not. After controlling for demographic characteristics, past performance and behaviour at school, as well as college characteristics, achieving the initial BL2 course is associated with a 4.6 percentage point increase in the probability of carrying out some paid work in the 2015 tax year, and with 16.7% higher earnings. Earnings differentials associated with achievement were significantly different across subject areas. Achieving a BL2 course in Health, Public Service and Care, Retail or Engineering are associated with substantially higher earnings compared to achieving a BL2 course in Preparation for Life and Work, the most popular subject area.


The main limitation of our study is that the association between achievement and improved labour market prospects cannot be interpreted as necessarily reflecting a causal effect. While we control for a number of individual and college characteristics, there may be other factors that drive both achievement and labour market outcomes. For instance, less motivated learners may be more at risk of dropping out and become NEET. Conversely, some learners may choose to move on to paid employment or higher level learning, before formally obtaining the qualification.


In our view, the evidence suggests policies should try encourage and support adolescents’ engagement until they successfully gain their (low level) qualification. Evidence on interventions amongst adolescents show that the acquisition of basic skills in numeracy and literacy are greatly valued by employers. But targeting such cognitive skills in isolation is not sufficient for long term impacts on successful labour market performances. Non-cognitive skills such conscientiousness, self-discipline, perseverance, cooperation and willingness to be managed by more senior/adult employees are nearly equally essential ingredients to success on the labour market. The role of training mentors in this process can mimic parents involvement and appears most effective in firms-based environments away from formal schooling (Kautz et al., 2015).



"Young people in low level vocational education: characteristics, trajectories and labour market outcomes", CVER Research Discussion Paper 004 is available at http://cver.lse.ac.uk/publications/


Reference:


Kautz, T., Heckman, J., Diris, R., ter Weel, B. and Borghans, L. (2014). Fostering and Measuring Skills: Improving Cognitive and Non-cognitive Skills to Promote Lifetime Success, OECD Education Working Papers, No. 110, OECD Publishing, Paris. doi: http://dx.doi.org/10.1787/5jxsr7vr78f7-en



Friday, 13 January 2017

How important is providing careers-related information for students?

CVER Director Sandra McNally looks at career advice on offer to students, and what works


The type and quality of education matters for labour market prospects, as reflected in future employment and earnings. There is often dissatisfaction expressed with the careers information and advice provided to students at school and beyond. It’s a matter of common sense (rather than academic study) to say that students do need to have good quality careers information and advice. What isn’t clear is whether cheap information interventions are really going to make the difference for young people as they approach the time where they need to make important decisions. In recent years, there have been a number of economic studies that have used rigorous approaches to test whether simple information interventions actually work. I have reviewed this for a recent IZA World of Labor paper, which focuses on results from 10 evaluations implemented via Randomised Control Trials

Monday, 24 October 2016

Using survey data to estimate the value of vocational qualifications


CVER's Steven McIntosh and Damon Morris, from University of Sheffield, look at the value of vocational qualifications


An important part of the research programme at CVER is to investigate the ‘returns’ to vocational qualifications, that is, the wage premiums earned by individuals who hold such qualifications. The results of such research provide important information to policy-makers about the value the labour market places on qualifications, as well as to individuals making decisions about what courses to pursue. Our work will provide up-to-date evidence on this topic, using a comprehensive range of data sets and methodologies.

Monday, 3 October 2016

Low-achieving teenagers: Evidence from France of the potential of low-cost interventions to clarify educational options

Eric Maurin from the Paris School of Economics is an expert advisor for CVER and presented this work at our recent conference


A simple programme of meetings facilitated by school principals and targeted at low-achieving 15 year olds can help them to identify educational opportunities that fit both their tastes and their academic ability. That is the central finding of a large-scale randomised experiment in Paris, conducted by economic researchers Dominique Goux, Marc Gurgand and Eric Maurin.

Their study, which is forthcoming in the Economic Journal, reveals that the outcome of an intervention in deprived neighbourhoods of Paris has been a very significant reduction in the number of students repeating educational years (‘grade repetition’) and in the number of students dropping out of school altogether. Compared with most existing interventions, this is a very low cost way to help young people who struggle at school to find the educational track most suited to their needs.


Thursday, 15 September 2016

The Benefits of Alternatives to Conventional College: Labor-Market Returns to Proprietary Schooling

Christopher Jepsen, with Peter Mueser and Kyung-Seong Jeon, looks at the labor-market returns to U.S. proprietary schooling (often known as for-profit schooling)


Researchers are increasingly able to estimate the long-term wage returns to education using administrative data. This research reports findings from the US that were presented at the CVER seminar series in 2015 and have recently been published in a discussion paper (Jepsen et al. 2016).


In recent years, U.S. states have drastically reduced funding for education, and public community colleges and universities are particularly hard hit (Phelan, 2014). Proprietary schools (also known as for-profit schools) have been growing dramatically over the last decade, filling a gap in demand for postsecondary education, particularly for low-income and nonwhite individuals. The vast majority of students in this sector pursue vocational qualifications such as certificates and associate’s degrees in areas of study including health, transportation, and trades (i.e. construction, etc.).