The South African Child Support Grant Impact Assessment

The South African:Child Support Grant

Impact Assessment
Evidence from a survey of children, adolescents and their households

The South African Child Support Grant Impact Assessment:
Evidence from a survey of children, adolescents and their households
May 2012
This study was commissioned and funded by the Department of Social Development (DSD), the South African Social Security
Agency (SASSA) and the United Nations Children’s Fund (UNICEF) South Africa. The study was carried out by the Economic
Policy Research Institute (EPRI) in partnership with the International Food Policy Research Institute (IFPRI), as part of a
larger project partnership including the Institute for Development Studies (IDS), Oxford Policy Management (OPM), Reform
Development Consulting (RDC) and Take Note Trading (TNT). This research project is part of a multi-year integrated
qualitative and quantitative impact assessment commissioned by the Department of Social Development, the South African
Social Security Agency and UNICEF South Africa.
Acknowledgements
˜˜ Study Team: Dr. Carolyn Heinrich (Economic Policy Research Institute (EPRI) and University of Texas at Austin), Dr. John
Hoddinott (International Food Policy Research Institute), Dr. Michael Samson (EPRI), Mr. Kenneth Mac Quene (EPRI),
Ms. Ingrid van Niekerk (EPRI), Mr. Bryant Renaud (EPRI).

˜˜ Gratitude to the adult and child respondents for their inputs and contributions.
˜˜ The valuable contribution of the following is also acknowledged: Selwyn Jehoma, Thilde Stevens, Maureen Motepe, Thabani
Buthelezi and Dibolelo Ababio from the Department of Social Development; Eric Musekene, Alice Odhiambo and Rudzani
Takalani, from SASSA; George Laryea-Adjei from UNICEF; Nkechi Obisie-Nmehielle (IOM); Dugan Fraser (Action
Plus); Benjamin Davis (FAO); Ashu Handa (University of North Carolina); Jan Vorster (University of Stellenbosch); Linda
Richter, Vuyiswa Mathambo and Lucia Knight (HSRC); Patrick Chiroro (IRI).
© Department of Social Development/South African Social Security Agency/UNICEF
All care has been taken to ensure that the information is correct and original sources have been indicated for reference and
verification. With an identification of the Department of Social Development, South African Social Security Agency and
UNICEF as source, the document may be freely quoted, reviewed, abstracted, reproduced and translated, in part or in whole,
but not for sale nor for use in conjunction with commercial purposes. Original sources should be acknowledged where indicated
in the publication.
Suggested citation:
DSD, SASSA and UNICEF. 2012. The South African Child Support Grant Impact Assessment: Evidence from a survey of children,
adolescents and their households. Pretoria: UNICEF South Africa
Obtainable free of charge from:
UNICEF, DSD and SASSA
Website: http://www.unicef.org/southafrica; http://www.dsd.gov.za and http://www.sassa.gov.za
Readers are also referred to the complementary research report called “Child Support Grant Evaluation 2010: Qualitative
Research Report” (Stephen Devereux [IDS], Michelle Adato [IFPRI], Rachel Sabates-Wheeler [IDS], Jesse McConnell
[RDC], Elisabeth Becker [IFPRI]) dated June 2011, which presents the qualitative component of the evaluation.

Cover photograph: © UNICEF/Pirozzi
Layout: Handmade Communications (viv@handmadecom.co.za)

The South
African
Child
Support
Grant
Impact
Assessment
Evidence from a survey
of children, adolescents
and their households

The South African Child Support Grant Impact Assessment

Executive Summary
Overview and research questions
The Child Support Grant (CSG) is an important instrument
of social protection in South Africa, reaching over 10 million
South African children each month. This report presents the
findings of a research team’s analysis of a specially designed
survey fielded in rural and urban areas of five South African
provinces, supporting the rigorous impact assessment of how
access to the CSG affects key aspects of child and adolescent
well-being.
The South African Child Support Grant (CSG) was first
introduced in 1998. Over the past 14 years, South Africa’s
social grant programme has evolved into one of the most
comprehensive social protection systems in the developing
world. Expansions to the Child Support Grant’s criteria for
eligibility over this same period include an increase in the
age limit from seven to eighteen years old, and adjustments
to the income threshold to take inflation into account and
improve equity.

Research design, questionnaires and
data
Three questionnaires were designed to gather information on children, adolescents and their households.
Households with participating adolescents were given the
CSG Adolescent Questionnaire and the CSG Household
Questionnaire, while homes with participating young children were given the CSG Young Child Questionnaire and
the CSG Household Questionnaire. In addition, adolescents
completed a confidential, self-administered survey about
their receipt of the CSG, school and work participation,
and their engagement in risky behaviours. The sampling
process took place in two stages. First, a random sample of
locations, defined as the catchment areas for specific paypoints, was drawn from SASSA’s administrative database.
These locations were sampled from each of five provinces:
Eastern Cape, Gauteng, KwaZulu-Natal, Limpopo and
Western Cape. Second, children were randomly selected
from the identified paypoints in order to identify a group of
10-year-olds who enrolled in the CSG programme shortly

after birth, compared to a group enrolled later – at age four
or older. Adolescents were selected around the age cut-off
for eligibility in 2010, including those receiving and not
receiving the CSG. The research team compared the results
of the survey to other national household surveys, including
the 2008 National Income Dynamics Survey (NIDS) and
the 2010 General Household Survey (GHS), and found the
sample largely representative of the corresponding national
populations.

Impact evaluation methodology
The methodology of this evaluation aims to measure causal
programme impacts as the difference between observed outcomes for the beneficiaries and what would have been the
outcomes if this group had not received the Child Support
Grant or received it later versus earlier. The evaluation
strategy controls for factors that might lead to an erroneous
attribution of causality, including individual and household
traits such as poverty status, exposure to shocks, demographic
characteristics and other variables. The evaluation employs
non-experimental approaches rather than a randomised
experiment because there is no practical or legal scope for
randomly allocating grants in South Africa, and the single
cross-sectional survey together with the sample variability
in terms of timing and receipt of grants appropriately supports and strengthens the evaluation approaches adopted for
this study. The main method adopted for this study matches
and compares households receiving the ‘treatment’ (such
as the Child Support Grant from shortly after the child’s
birth) with a comparison group of households with similar
observable characteristics that influence their probability
of application for or receipt of the Child Support Grant.
The study employs extensions of this approach to assess the
impact of the duration of Child Support Grant receipt on
outcomes of interest.

Access to the Child Support Grant
Receipt of the CSG varies over different age groups. Take-up
rates peak for children seven to 10 years in age, while infants

Executive Summary iii

have relatively low take-up rates. Furthermore, youth in
newly-eligible age groups have relatively low take-up rates.
This finding helps explain why adolescents are relatively
less likely to receive the CSG when compared to younger
children. Receipt of the CSG is correlated with multiple
household re-applications as well as household knowledge
of the CSG from formal sources. Generally, relatively poorer
and/or less educated households are more likely to have
received the CSG. In Limpopo, however, adolescents who
first began receiving the CSG between age 10 and 13 years
have significantly lower odds of continuing to receive a CSG
at age 15 or older, an unexpected result which is a subject of
future research.

time of the survey. Children who were enrolled in the CSG
at birth completed significantly more grades of schooling
than children who were enrolled at age six, and achieved
higher scores on a math test.3 Impacts for girls were particularly significant, with early receipt of the CSG increasing
girls’ grade attainment by a quarter of a grade compared to
those receiving the grant only at age 6. The impact largely
resulted from early receipt of the CSG reducing delays in
girls entering school by 27 per cent, with girls enrolling early
obtaining higher scores on math and reading tests. For children whose mothers have less than eight grades of schooling,
the impacts were even greater. Early enrolment in the CSG
raises grade attainment by 10.2 per cent (0.38 grades). The
CSG appears to play a compensatory role for children with
less educated mothers, narrowing the schooling gap between
children whose mothers have less education and those who
have more. In these ways the Child Support Grant promotes
human capital development, improves gender outcomes and
helps to reduce the historical legacy of inequality.

The impact of the Child Support Grant
on children’s health
Analysis of current illness- and health-related expenditures
provides evidence of the impact of the Child Support Grant
on child health. Early enrolment in the CSG reduced the
likelihood of illness (as measured by a 15 day period prior to
the survey), with the effect particularly strong for boys. Boys
enrolled at birth had a 21 per cent likelihood of being ill,
compared to a 30 per cent likelihood for boys enrolled later.4
Children enrolled at birth whose mothers have eight or more
grades of schooling have a significantly lower likelihood of
being ill relative to otherwise comparable children enrolled
at age 6,5 again suggesting that a mother’s education further
complements the Child Support Grant in strengthening
other important impacts, and that these positive impacts are
fairly persistent.6
3.

The mean increase in grades of schooling was 0.14, and the increase in the mean
math test score was 6.0 per cent.

The improvement is 7.7 percentage points, statistically significant at the 10 per
cent level.

This result was significant at the 10% level.

The improvement in the height-for-age z-score is 0.19 standard deviations, a
large impact significant at the five per cent level.

The improvement in this indicator was 8.5 percentage points.

At least from birth to age 10, the age threshold used for this part of the study.

The South African Child Support Grant Impact Assessment

The impact of the Child Support Grant
on time allocation and labour supply
of children

The impact of the Child Support
Grant on schooling outcomes of
adolescents

Analysis of the time allocation and labour supply of 10-yearold children provides evidence of the Child Support Grant’s
impact on the amount of time spent studying, doing chores
or working outside the household. The study finds few
10-year-old children working for pay outside the household.
The timing of CSG enrolment has no statistically significant impact on time spent studying or doing housework.
However, for children in households with no electricity,
early enrolment in the CSG increases the amount of time
spent studying, but the magnitude of this impact is small.

Analysis of adolescent absences from school provides
evidence of the impact of the Child Support Grant on
schooling outcomes for adolescents. Receipt of the CSG
by the household reduces adolescent absences from school,
particularly for male adolescents, even when the household
does not receive the grant specifically for the adolescent.

Variation in receipt of the Child
Support Grant among adolescents
A significant pattern identified in the survey data played
an important role in the evaluation of impacts of the Child
Support Grant on adolescents. Adolescents who first started
receiving the Child Support Grant at an early age (four years
or younger), or more recently at age 14 years or older, are significantly more likely to be in households that are currently
receiving the CSG for the adolescent (at the time of the
survey). On the other hand, a comparatively low proportion
of adolescents who first began receiving the CSG between
the ages of 10 and 13 are in households currently receiving
the grant for them, particularly in the province of Limpopo,
which has one of the highest poverty rates in South Africa.
This study finds that important predictors of successful Child
Support Grant receipt by an adolescent’s caregiver include
(1) application for the grant by the adolescent’s biological
mother, (2) the mother of the adolescent being the head of
the household, (3) adolescent awareness of the availability
of the CSG programme, (4) lower educational attainment
for the household head, (5) persistent re-application for the
CSG in the face of initial rejection.

The impact of the Child Support Grant
on work inside and outside the home
The households in the sample reported fairly similar
responses about the degree to which adolescents worked
inside the home. However, adolescents and their caregivers
reported very different patterns of work outside the home:
the household respondents indicated that only two per cent
of the sampled adolescents work outside the home, while
18.5 per cent of 1,355 adolescents who answered this question indicated that they worked outside the home. Early
receipt of the Child Support Grant (in the first seven years
of life) reduces the likelihood that they will grow up into
adolescents who will work outside the home (as reported
in the adolescent survey). Additionally, there appears to be
a particularly important impact in terms of reduced work
outside of the home for females who received the grant in
early childhood.

The impact of the Child Support Grant
on adolescent risky behaviours
Analysis of adolescent risky behaviours provides evidence of
the Child Support Grant’s impact in significantly reducing
six main risky behaviours – sexual activity, pregnancy, alcohol use, drug use, criminal activity and gang membership.
The evidence documents statistically significant associations
between receipt of the Child Support Grant in adolescence
and:

Executive Summary v

(1) reduced sexual activity and a fewer number of sexual
partners, particularly when the adolescent also received
the grant in early childhood;
(2) reduced pregnancy, again particularly when the adolescent also received the grant in early childhood;
(3) reduced alcohol and drug use, particularly for females,
and with the effect strengthened by early childhood
receipt of the CSG.

Conclusions
The results of this study identify the positive developmental
impact of the Child Support Grant in promoting nutritional,
educational and health outcomes. Early receipt significantly
strengthens a number of these important impacts, providing
an investment in people that reduces multiple dimension
indicators of poverty, promotes better gender outcomes and

reduces inequality. The study also finds that adolescents
receiving the Child Support Grant are more likely to have
some positive educational outcomes, are somewhat less likely
to experience child labour, and are significantly less likely to
engage in behaviours that put their health and well-being at
serious risk. These results convey several key messages:
(1) The Child Support Grant generates positive developmental impact that multiplies its benefits in terms of
directly reducing poverty and vulnerability;
(2) Early enrolment in the Child Support Grant programme
substantially strengthens impacts. Promoting continuous
access to the CSG for eligible children through adolescence would help to maximise the potential benefits of
the grant.
(3) Receipt of the grant by adolescents generates a range
of positive impacts, not least of which is the reduction
in risky behaviours, which in the context of high HIV
prevalence, generates a particularly protective impact.

The South African Child Support Grant Impact Assessment

Table of Contents
Executive

Summary…………………………………………………………………………………………………………………………………………….

ii
PART ONE
Chapter 1 Introduction, Research Questions and Approach…………………………………………………………………………………….. 1
1.1 Introduction
1
1.2 Background
1
1.3 Objectives
3
1.4 Theory of change
3
1.5 Enrolment
7
1.6 Evaluation approach
7
1.7 Organisation of the report
8
PART TWO
Chapter 2 Data Source and Methods………………………………………………………………………………………………………………….. 11
2.1 Introduction
11
2.2 Sampling strategy
11
2.2.1 Different types of PSUs
11
2.2.2 Large and small paypoints
11
2.2.3 Sampling probability
12
2.3 Survey instruments
13
2.4 Survey implementation and data collection
13
Chapter 3 Description and Benchmarking of the Sample………………………………………………………………………………………. 14
3.1 Introduction
14
3.2 Household demographic structure and characteristics
14
3.3 Indicators of wealth
16
3.4 Financial outcomes
19
3.4.1 Introduction
19
3.4.2 Saving behaviours
20
3.4.3 Comparison of banked and un-banked respondents
21
3.4.4 Conclusions
23
PART THREE
Chapter 4 Grant

Access……………………………………………………………………………………………………………………………………. 25
4.1 Introduction
25
4.2 Eligibility criteria
25
4.3 Extension of age threshold
28
4.4 Documentation required
28

Table of Contents vii

4.5
4.6
4.7
4.8

Reasons for not applying
Improvements in grant access
Correlates of early application
Access to other grants

29
30
30
30

Chapter 5 Methods for Assessing the Impact of the Child Support Grant……………………………………………………………….. 34
5.1 Introduction
34
5.2 Double difference methods with randomisation: a ‘gold standard’ approach
34
5.3 Evaluation methods for the CSG: using binary matching methods
36
5.3.1 Overview
36
5.3.2 Constructing treatment and comparison groups with non-random assignment
36
5.3.3 Generalised dose response models
39
5.3.4 Recovering double-difference estimates from a single cross-sectional survey
41
PART FOUR
Chapter 6 The Impact of the CSG on Early Life Outcomes………………………………………………………………………………….. 43
6.1 Introduction
43
6.2 Access and use of preventative health and nutrition care
43
6.3 Impact on anthropometry
47
6.4 Impact on access to pre-schools
48
6.5 Summary
50
Chapter 7 The Impact of the CSG on Schooling………………………………………………………………………………………………….. 51
7.1 Introduction
51
7.2 Schooling
51
7.3 Scores on mathematics tests
59
7.4 Scores on reading and vocabulary tests
63
7.5 Summary
66
Chapter 8 The Impact of the CSG on the Health of 10-Year-Old Children…………………………………………………………….. 67
8.1 Introduction
67
8.2 Current health status
67
8.3 Impact of CSG enrolment on children’s illness
67
8.4 Summary
70
Chapter 9 The Impact of the CSG on Time Allocation and Labour Supply of 10-Year-Old Children………………………….. 71
9.1 Introduction
71
9.2 How children spend their time
71
9.3 Impact of CSG enrolment on children’s time allocation
71
9.4 Summary
73

viii

The South African Child Support Grant Impact Assessment

PART FIVE
Chapter 10 Adolescent and Household Receipt of the Child Support Grant……………………………………………………………. 75
10.1 Samples of adolescents
75
10.2 Definition of adolescent treatment and comparison groups
77
Chapter 11 The Impact of the CSG on Adolescent Schooling Outcomes………………………………………………………………… 82
11.1 Introduction
82
11.2 School enrolment
82
11.3 Absences from school
83
11.4 Grade repetition and grade attainment
84
11.5 Results of the estimation of CSG impacts on schooling outcomes
85
Chapter 12 The Impact of the CSG on Adolescent Work……………………………………………………………………………………… 87
12.1 Introduction
87
12.2 Adolescent behaviours associated with work outside the home
87
12.3 Results of the estimation of CSG impacts on adolescent work outside the household
88
Chapter 13 The Impact of the CSG on Adolescent Risky Behaviours……………………………………………………………………… 91
13.1 Introduction
91
13.2 Results of the estimation of CSG impacts on adolescent sexual activity and pregnancy
93
13.3 Results of the estimation of CSG impacts on adolescent alcohol and drug use
96
13.4 Results of the estimation of CSG impacts on criminal activity and gang membership
99
13.5 Concluding notes for analysis of CSG impacts on adolescents
101
PART SIX
Chapter 14

Conclusions………………………………………………………………………………………………………………………………….. 105
PART SEVEN
Bibliography………………………………………………………………………………………………………………………………………..

………… 109
Appendix 5.1 Estimating Impact using Propensity Scores……………………………………………………………………………………. 112
Appendix 5.2 Estimating Dose-Response Functions…………………………………………………………………………………………… 114
Annex………………………………………………………………………………………………………………………………………………

…………… 117

Table of Contents ix

Tables
Table 2.1
Table 2.2
Table 4.1
Table 5.1
Table 5.2
Table 5.3
Table 6.1
Table 6.2
Table 6.3
Table 6.4
Table 6.5
Table 6.6
Table 6.7
Table 6.8
Table 7.1
Table 7.2
Table 7.3
Table 7.4
Table 7.5a
Table 7.5b
Table 7.5c
Table 7.6
Table 7.7
Table 7.8
Table 7.9
Table 7.10
Table 7.11
Table 7.12
Table 7.13
Table 7.14
Table 7.15
Table 7.16
Table 7.17
Table 7.18
Table 7.19
Table 8.1
Table 8.2

Overview of PSUs in the study
Number of desired respondents per PSU
Correlates of early application
Calculation of the double-difference estimate of average programme effect
Correlates of early CSG enrolment
Dose-response estimates of impact on grade attainment, girls
Possession of Road to Health Card
Correlates of possession of Road to Health Card
Descriptive statistics for growth monitoring
PSM estimates of the impact of early receipt of CSG on growth monitoring
PSM estimates of the impact of vaccination
Impact of early receipt of CSG on anthropometry
Impact of early receipt of CSG on attendance at crèche or nursery schools, full sample
Impact of early receipt of CSG on attendance at crèche or nursery schools, disaggregated by sex and
education level of mother
Highest grade attained by sex
Distribution of entry into Grade 1 by sex
Highest grade attained by entry into Grade 1
Percentage of children repeating a grade by grade, sex and age of entry into Grade 1
Schooling outcomes by child age at first receipt of CSG, all children
Schooling outcomes by child age at first receipt of CSG, girls
Schooling outcomes by child age at first receipt of CSG, boys
Distribution of children by age at first receipt of CSG
Dose-response estimates of impact on grade attainment
Dose-response estimates of impact on grade attainment, girls
Dose-response estimates of impact on grade attainment, boys
Dose-response estimates of impact on grade attainment, children whose mothers have
<8 grades schooling Dose-response estimates of impact on grade attainment, children whose mothers have 8+ grades schooling Dose-response estimates of impact on probability of delayed enrolment by child sex and maternal education Dose-response estimates of impact on probability of grade repetition by child sex and maternal education Scores on selected components of the EGMA by sex and maternal schooling Dose response estimates of impact on EGMA score by child sex and maternal education Dose-response estimates of impact on arithmetic score by child sex and maternal education Dose-response estimates of impact on shape recognition score by child sex and maternal education Dose-response estimates of impact on likelihood of complete reading of story by child sex and maternal education Dose-response estimates of impact on reading comprehension by child sex and maternal education Number of different illnesses by sex Percentage of children with the following symptoms in the last 15 days by sex 12 12 31 35 38 41 43 44 44 45 46 47 49 49 51 51 52 52 52 52 53 53 54 55 55 55 56 57 58 59 60 61 62 64 65 67 68 x The South African Child Support Grant Impact Assessment Table 8.3 Table 8.4 Table 9.1 Table 9.2 Table 9.3 Table 9.4 Table 10.1 Table 10.2 Table 10.3 Table 11.1 Table 12.1 Table 13.1 Table 13.2 Table 13.3 Table 13.4 Table 13.5 Table 13.6 Table 13.7 Table 13.8 Dose-response estimates of impact on probability of illness in last 15 days by child sex and maternal education 68 Dose-response estimates of impact on number of days ill in last 15 days by child sex and maternal education 69 Per cent of children undertaking specified activities by sex 71 Dose-response estimates of impact on study time 72 Dose-response estimates of impact on housework 72 Dose-response estimates of impact on study time and housework, children in households with no electricity 73 Target sample sizes by adolescent age groups 75 Realised samples of adolescents 75 Predictors of CSG receipt by the household and for the adolescent, current and past 80 PSM estimation – comparison of subgroups 86 Dose-response effect results from GPS models of adolescent work outside the home 89 Adolescent reports of main problems teenagers face in community 91 Risky behaviours among South African adolescents as identified in the qualitative component of the CSG impact evaluation 92 PSM estimation of impact of CSG on adolescent sexual activity (outcome: never had sex) 92 Dose-response results from GPS model of adolescent and female adolescent sexual activity (never had sex) 94 Dose-response results from GPS model of female adolescent pregnancy 96 Dose-response results from GPS models of adolescent alcohol use 98 Dose-response results from GPS models of adolescent drug use 98 Dose-response results from GPS models of adolescent criminal activity 100 Figures Figure 1.1 Figure 1.2 Figure 1.3 Figure 1.4 Figure 1.5 Figure 3.1 Figure 3.2 Figure 3.3 Figure 3.4 Figure 3.5 Figure 3.6 Figure 3.7 Figure 3.8 Figure 3.9 Figure 3.10 Figure 3.11 Figure 3.12 Evolution of the Child Support Grant coverage A typology of the Child Support Grant’s social impacts Typology of the Child Support Grant’s developmental impacts The determination of impact of social protection programmes Developmental paths Comparing sub-populations of data Household size (homes with 10-year-olds) Household size (homes with adolescents) Number of children (<18) in household (homes with 10-year-olds) Number of children (<18) in household (homes with adolescents) Access to electricity (homes with 10-year-olds) Access to electricity (homes with adolescents) Homes with corrugated iron/zinc roof (homes with 10-year-olds) Homes with corrugated iron/zinc roof (homes with adolescents) Bank account or savings method by percentage of households Reasons for not having an account by percentage of households Frequency of deposits by percentage of households for different account types 3 4 4 5 6 14 15 15 16 16 17 17 18 18 19 19 20 Table of Contents xi Figure 3.13 Long term insurance, short term insurance and burial plan uptake Figure 3.14 Mean expenditure by banking status Figure 3.15 Expenditure by percentile Figure 3.16 Mean debt by banking status Figure 3.17 Household debt by banking status and percentile Figure 3.18 Household possessions by banking status Figure 3.19 Distribution of distances to money withdrawal point Figure 4.1 Reasons caregivers of poor children eligible for receipt do not apply Figure 4.2 Reasons caregivers delayed application for the CSG (principal reasons) Figure 4.3 Mean number of Old Age Pension Grants per household (homes with 10-year-olds) Figure 4.4 Mean number of Old Age Pensions per household (homes with adolescents) Figure 4.5 Mean number of Disability Grants per household Figure 4.6 Average Amount of Care Dependency Grants per household Figure 5.1 Illustration of the double-difference estimate of average programme effect Figure 5.2 Kernel density of propensity scores by treatment status Figure 5.3 Dose-response graph of impact of age at first receipt of CSG on grade attainment, girls Figure 5.4 Recovering double difference estimates from a single cross-section Figure 6.1 Distribution of age first attended crèche, by sex Figure 7.1 Dose-response graph of impact of age at first receipt of CSG on grade attainment Figure 7.2 Dose-response graph of impact of age at first receipt of CSG on grade attainment, girls Figure 7.3 Dose-response graph of impact of age at first receipt of CSG on grade attainment, boys Figure 9.1 Distribution of time spent studying by sex Figure 10.1 CSG Receipt in households with adolescents Figure 10.2 Adolescent CSG Receipt by age at CSG start Figure 10.3 CSG receipt by household and for adolescent Figure 10.4 Predicting selection into five CSG receipt subgroups Figure 11.1 Agreement between adolescents and their families on school enrolment Figure 11.2 School enrolment rates by treatment group Figure 11.3 Adolescent and household reports of adolescent absences from school Figure 11.4 Number of school absences by treatment group Figure 11.5 Adolescent grade repetition and attainment by treatment subgroup Figure 12.1 Reports of adolescent work outside the home Figure 12.2 Adolescent engagement in risky behaviours by work outside the home Figure 12.3 Probability adolescent works outside household Figure 12.4 Probability female adolescent works outside household Figure 12.5 Probability male adolescent works outside household Figure 13.1 Probability that adolescents never had sexual intercourse Figure 13.2 Probability female adolescents never had sex Figure 13.3 Adolescent number of sex partners Figure 13.4 Probability ever pregnant for female adolescents Figure 13.5 Probability of no adolescent alcohol use Figure 13.6 Probability of no female adolescent alcohol use Figure 13.7 Probability of no male adolescent alcohol use Figure 13.8 Probability of no drug use ever by adolescents Figure 13.9 Probability of no drug use ever by female adolescents Figure 13.10 Probability of no drug use ever by male adolescents Figure 13.11 Probability of no criminal activity, adolescents Figure 13.12 Probability of no criminal activity, adolescent males 21 21 22 22 22 23 23 27 27 31 32 33 33 35 39 40 41 48 53 54 55 71 76 76 77 81 82 83 83 84 84 87 88 90 90 90 95 95 95 96 97 97 97 99 99 99 100 100 xii The South African Child Support Grant Impact Assessment © UNICEF/NYHQ2006-1404/Pirozzi Part 1 Overview Chapter 1 Introduction, Research Questions and Approach 1 Chapter 1 Introduction, Research Questions and Approach 1.1 Introduction The Child Support Grant (CSG) is an important instrument of social protection in South Africa. It is both the largest of South Africa’s social cash transfer programmes7 and one of the government’s ‘most successful social protection interventions.’8 A number of studies9 have contributed to a growing evidence base, demonstrating the successes of the CSG in terms of reducing poverty and promoting human capital development. However, because of its extensive coverage10 and because most prior studies have relied primarily on household-level data, it has been difficult to assess the impact of the CSG on the individuals it is supposed to benefit most, children. Moreover, any attempt to measure causal programme impacts must address the problem of the counterfactual, what outcomes would have been observed had individuals for this group had they not received the programme. All rigorous impact evaluation strategies are designed to identify a method for constructing a proxy for these counterfactual outcomes using information on non-beneficiaries. This requires controlling for the effects of any confounding economic and contextual factors that make programme beneficiaries systematically different from an average non-beneficiary, such as the relative poverty of beneficiaries in targeted programs, exposure to economic shocks, or differences in household characteristics (e.g. demographics, skill levels, or social networks), and affect the impacts of the programme. Impact estimates that imperfectly control for these confounders suffer from ‘selection bias’. This report uses recently collected data derived from a specially designed survey fielded in rural and urban areas of five South African provinces to rigorously assess the impact of how access to the CSG affects key aspects of child and adolescent welfare. Using data from the CSG Evaluation survey, this study demonstrates: i) How early enrolment in the CSG affected the well-being and cognitive development of children compared to children who enrolled in the programme later; ii) The impact of the Child Support Grant on adolescent children. 1.2 Background The South African Child Support Grant (CSG) was first introduced in 199811. Prior to this, the government provided a limited State Maintenance Grant (SMG)12. Applicants for the SMG needed to prove13 that they were the sole provider and caregiver for a child under the age of 17. Parents who were widowed, divorced, had been deserted by their spouse, or had a spouse in jail or other institution that made working impossible were eligible to receive the grant14. A household survey collected in 1990 analysed the impact of this grant and found that only 0.2% of African children were in receipt of State Maintenance Grants, while 1.5% of white children, 4.0% of Indian children and 4.8% of Coloured children received the grant15. Ease of receipt was dependent on location; children living in rural areas were often excluded due to lack of knowledge regarding the grant, inability to travel to application sites, and other administrative problems16. The household survey documented the need for an alternative strategy for targeting poor children that would result in equal access to the grant and increase the number of beneficiaries. In December 1995, the Government of South Africa established the Lund Committee17 to evaluate the current social protection system and provide advice on ways to improve it. The committee recommended18 a Child Support Grant that would reach a greater number of children and families. The 11. McEwen, Kannemeyer, & Woolard, July 2009. 12. Ibid. 7. In terms of the number of participants. (Samson, et al., 2008). 8. Samson, et al., 2008, 6. 14. Kruger, 1998. 9. Agüero, Carter, & Woolard, September 2007; Budlender & Woolard, 2006; Delany, Ismail, Graham, & Ramkisson, June 2008; Makiwane & Udjo, 2006; Samson, et al., 2008. 15. Ibid. 10. 10.8 million (Fact sheet no 2 of 2012: Summary of social grants distribution in South Africa, 2012). 13. Delany, Ismail, Graham, & Ramkisson, June 2008. 16. Ibid. 17. Delany, Ismail, Graham, & Ramkisson, June 2008. 18. Report of the Lund Committee on Child and Family Support, 1996. 2 The South African Child Support Grant Impact Assessment aim was to target and provide the benefit for children in the poorest 30%19 of households. According to these recommendations, the Child Support Grant replaced the State Maintenance Grant in 199820. The modifications made the grant more accessible to caregivers and children and enabled the programme to substantially increase the participation rate. The Child Support Grant was paid to the primary caregiver of the child at a level of R100 per month21. Recipients of the grant were required to pass a means test which was based on household income. Families living in rural areas earning a household income below R800 per month and families in urban areas earning a household income below R1,100 were eligible to receive the grant22. Recipients needed to offer proof of income in order to validate that the household income level being reported was correct23 and, if the caregiver was not the child’s parent, proof that efforts to secure funds from the child’s parents were made unsuccessfully. Due to the low take-up rate of the grant, the application process and means test was again modified in 199924. was set at R1,000 for rural households and R800 for urban households. Since then, the grant amount, and therefore the means threshold, has increased in a stepwise fashion. The grant amount as of April 2012 is R280 per month, and the threshold is set at R33,600 per year for single caregivers and R67,000 per year for married couples29. Since the introduction of the Child Support Grant, modifications to age limit eligibility have also been made. When the grant was first created in 1998, it was limited to children younger than seven years old30. In April 2003, the age limit was increased to include children under the age of nine31. This was further extended in 2004 and 2005 to include children up to the age of eleven and fourteen respectively32. In 2008 children under the age of fifteen33 became eligible to receive the grant. Currently34, a child is eligible until their eighteenth birthday. The means test for the CSG was changed such that grant eligibility was determined based on caregiver’s and spouse’s income as opposed to household income25. The means test remained unchanged in nominal terms from 1999 until 2008. This made it increasingly difficult for caregivers to be considered eligible as the income threshold for eligibility did not increase to keep up with inflation26. In 2008, accepting a study by the Economic Policy Research Institute, the Department of Social Development changed the means test so that the eligibility threshold was equal to 10 times the value of the grant27. The same year, the Child Support Grant benefit was set at R210 per month28 and the income threshold Some conditions associated with the support grant have been eliminated. When the Child Support Grant was first introduced, all recipients were required to participate in ‘development programmes’ and to have their children immunised35. The purpose of these conditions was to encourage parents to take part in activities that would improve their families’ standard of living and to ensure the safety of their children. However, these conditions had created barriers36 to receipt for many poor households. Not all caregivers were able to access the development programmes or could afford the costs associated with getting their children immunised. Children living in poor rural areas where the development programmes were not offered and required costly travel to a health care facility were often excluded37. A 2001 study by the Economic Policy Research Institute38 found 95% of the most vulnerable children excluded from the programme 19. Kruger, 1998. 29. Child Support Grant, 2012. 20. McEwen, Kannemeyer, & Woolard, July 2009. 30. Agüero, Carter, & Woolard, September 2007. 21. Ibid. 31. Delany, Ismail, Graham, & Ramkisson, June 2008. 22. Ibid. 32. Ibid. 23. Delany, Ismail, Graham, & Ramkisson, June 2008. 33. Ibid. 24. Ibid. 34. Child Support Grant, 2012. 25. Ibid. 35. Delany, Ismail, Graham, & Ramkisson, June 2008. 26. Ibid. 36. Samson, et al., 2004. 27. Hall, 2010. 37. Samson, M. J., 2002, 1162. 28. Delany, Ismail, Graham, & Ramkisson, June 2008. 38. Ibid., 1160. Chapter 1 Introduction, Research Questions and Approach 3 in one of the poorest districts in South Africa. The aim of support grants is to target children most in need and these conditions prevented the poorest and most vulnerable children from accessing the grant. The elimination of these conditions increased the take-up rate,39 especially in these poorest areas. South Africa’s social grants have been extremely successful at reducing poverty and promoting economic growth and development43. 1.3 Objectives This study addresses three questions: Figure 1.1 Evolution of the Child Support Grant coverage 12 Children under 18 qualify Millions of children 10 8 6 CSG introduced (children under 7 qualify) Children under 14 qualify Children under 11 qualify 4 Children under 9 qualify 2 2012 (R280) 2011 (R260) 2010 (R250) 2009 (R240) 2008 (R230) 2007 (R200) 2006 (R190) 2005 (R180) 2004 (R170) 2003 (R160) 2002 (R140) 2001 (R110) 2000 (R100) 1999 (R100) 1998 (R100) 0 Question 1: How has early versus late enrolment affected the well-being and development of children? In particular, the study addresses this question in terms of children’s anthropometry, health and schooling, as well as their access and use of preventative health and nutrition care. Question 2: How are critical life course events of adolescents affected by the extension of the CSG? Specifically, this study explores this question in terms of adolescents’ participation in risky behaviour, schooling outcomes, and work inside and outside the home. Question 3: What conditions determine and influence access to the CSG? This study focuses on this question at the point of initial application, by assessing the duration and continuity of receipt at the same time as analysing current access and use. See Annex for sources40 1.4 Theory of change The number of children receiving the grant has increased rapidly in the past 10 years due to the changes made to the age eligibility criterion, the relaxation of the means test and documentation requirements, and the other changes discussed above. SASSA reported 10,789,595 million beneficiaries in 201241. However, exclusion of poor, vulnerable children in the South African population is still a major challenge for the programme, motivating serious consideration of universal provision of the Child Support Grant42. The analysis of the programme impacts originates from a theory of change that recognises the global effectiveness of social cash transfers in tackling poverty and vulnerability for children, while promoting broader developmental impacts. The global evidence base on social cash transfers frames a model for understanding the possible impacts of the Child Support Grant programme. The central arguments for a system of cash grants include: Cash grants are relatively easy to administer (compared to other social policy interventions)44. This is especially the 39. Samson, et al., Forthcoming. 40. See Annex for sources. 41. Fact sheet no 2 of 2012: Summary of social grants distribution in South Africa, 2012). 42. Samson, et al., Forthcoming. 43. Samson, et al., Forthcoming. 44. Samson, van Niekerk, & Mac Quene, Designing and Implementing Social Transfer Programmes, 2006. 4 The South African Child Support Grant Impact Assessment Figure 1.2 A typology of the Child Support Grant’s social impacts Social protection’s social impact Improve human well-being Increase and stabilise consumption Reduce inequality Promote human security Strengthen inclusion Reduce vulnerability to shocks Build a bond between State and citizen Figure 1.3 Typology of the Child Support Grant’s developmental impacts Social protection’s social impact Benefits Security Poverty reduction Risk management Social cohesion Enables job search and investment Fosters peace and political stability Builds human capital Promotes and protects productive assets Social development and pro-poor growth Inclusion Chapter 1 Introduction, Research Questions and Approach 5 case in a country such as South Africa where cash benefits have been administered for a long time and relatively strong administrative systems exist45 to disburse the grants. Cash grants directly reduce poverty of some of the most vulnerable46 and in so doing also reduce inequality. Payment of cash to poor households will reduce the poverty headcount or the poverty gap and also reduce inequality measures because they are typically funded from progressive taxation47 (in national scale programmes). Cash grants therefore directly improve the living standards (consumption) of the poor and increase consumption levels48 of the poor relative to those in higher income groups, directly reducing poverty and inequality. In addition to directly reducing poverty (lower poverty headcounts and poverty gaps)49 cash grants also deal with some of the underlying causes of poverty and in so doing not only provide a safety net (allow people to cope with risk/ provide a minimum income level) but also generate positive dynamics through enabling risks to be mitigated and reduced over time. While poverty reduces resources that provide minimum living standards it also keeps households from consuming more productive50 consumption bundles, participating in economic activities and investing in physical, social, and human capital51 (i.e. education, health, nutrition) assets to ensure future income streams. Cash grants, in addition to funding consumption, enable poor households to make different consumption decisions, participate in productive economic activity and invest52 in the future productivity of the household and household members. – both (1) social and (2) developmental through which the social grant drives change for children, households and the society more broadly. Figure 1.2 illustrates several of social protection’s main social impacts, which represent the achievement of the main objectives of South Africa’s comprehensive system of social security. In addition to furthering these core objectives, the Child Support Grant also serves other secondary but developmental priorities, which furthers the policy objectives of pro-poor and inclusive economic growth.54 In particular, as illustrated in Figure 1.3, the poverty reduction aim of the Child Support Grant also builds, protects and promotes human capital and other productive assets while enabling people to more productively manage risks and shocks, as well as promoting social cohesion. These impacts are influenced by a number of distinct factors, as illustrated in Figure 1.4. Both the policy environment and the social context exert influences on impact, through the choice and design of programmes as well as the transmission mechanisms by which interventions generate impact. In addition, the operations, including the delivery systems, often determine the success or failure of the programme. Figure 1.4 The determination of impact of social protection programmes Operations Complementary institutions Previous studies53 of South Africa’s Child Support Grant as well as global evidence document a typology of pathways 45. Kruger, 1998, 3. 46. Samson, et al., Forthcoming. Policy environment IMPACT Social context 47. Samson, et al., 2004. 48. Rawlings & Rubio, 2005. 49. Samson, et al., Forthcoming. 50. OECD, 2009, 21. 51. Ibid. 52. Ibid., 12. 53. Williams, 2007; Delany, Ismail, Graham, & Ramkisson, June 2008; RDC, 2012; Agüero, Carter, & Woolard, September 2007; Rosa, Leatt, & Hall, 2005; Samson, et al., 2008. 54. For a completely elaborated framework on the linkages between social protection and pro-poor and inclusive economic growth and development, see the OECD Povnet’s policy statement and guidance note on social protection. (OECD, 2009). Morbidity Nutritional status Food intake Dietary diversity Food security Other expenditures for the index child: food, clothing, recreation Cognitive development Utilisation of health services/vaccination Quality of health and education services School progression and learning outcomes School enrolment and attendance Coverage and quality of health and education services Expenditure on human capital services Expenditure on health and education for the index child Expenditure Cash/income Demand factors Supply factors Early pregnancy/ addiction/delinquency Risky/ social behaviours General household expenditure: cigarettes alcohol Coping strategies/ vulnerability to shocks Asset building Saving/investment R240 on a monthly basis Unconditional/to caregivers Participation model: Selection process Grant administration and payment Improved well-being, social development, long term poverty reduction, pro-poor and inclusive economic growth Access to networks/ other social services Household structure Access to credit Remittances Child labour/time allocation of children Labour participation/time allocation of adults/caregivers Intra-household allocation Intra-household power structures Pathways assessed by mainly qualitative methods Pathways assessed by both quantitative and qualitative methods Pathways assessed by mainly quantitative methods LONG TERM OUTCOMES MEDIUM TERM OUTCOMES SHORT TERM OUTCOMES OUTPUTS ACTIVITIES 6 The South African Child Support Grant Impact Assessment Figure 1.5 Developmental paths Chapter 1 Introduction, Research Questions and Approach 7 Increasingly social policy analysts are also recognising the vital role complementary institutions play in determining impact. with a mainly quantitative, qualitative or a combination of qualitative and quantitative approaches. Operational questions include the design and implementation of the Child Support Grant mechanisms, with a particular focus on targeting effectiveness. Operational effectiveness depends critically on administrative capacity and the accessibility of CSG registration and other systems, which in turn depends of the functioning of payments systems and other delivery mechanisms. 1.5 Enrolment Complementary institutions can affect market performance55 (particularly food markets) and supply responses, as well as the availability and quality of human capital services56 (particularly health and education). In addition, appropriate complementary programmes can strengthen livelihoods opportunities and promote access to financial, information and communications services57. These factors operate within both a social and policy context. Impacts are seriously compromised by discrimination and social exclusion58, and heavily influenced by intrahousehold allocation decisions59. In addition, these impacts depend on the interaction between formal instruments and informal social protection systems. Social protection benefits strengthen opportunities for human capital development, but actual nutrition, health, education and other related outcomes will depend on the household demand60 for human capital services. In addition, actual outcomes across the spectrum will depend on poverty dynamics. Figure 1.5 illustrates this theory of change. It represents a tree of effects of the programme by separating the different levels of its influencing strategy according to a Log Frame hierarchy (activities, outputs and outcomes). The different colours indicate the areas of analysis that we expect to cover 55. Samson, van Niekerk, & Mac Quene, 2010. 56. Ibid. 57. Ibid. 58. Ibid. 59. Ibid. 60. Ibid. Whenever a caregiver applies for a social grant at a SASSA office, their information is logged into SOCPEN, a management information system that SASSA uses both to determine eligibility of applying, would-be grant recipients and to keep track of current recipients. Information on location, means, age of child beneficiary, name of caregiver recipient and other data is captured into the system. The sampling process began with the children registered in South Africa’s SOCPEN social grant administrative data system. This population was narrowed down to those children receiving the Child Support Grant. It was further circumscribed to include children only in the Eastern Cape, Gauteng, KwaZulu-Natal, Limpopo and Western Cape provinces. For the ‘young child’ part of the study, the population of interest was defined as children who were born in 2000, with an aim to surveying 10-year-olds. For the ‘adolescent’ part of the study, 15- to 17-year-olds benefiting from the CSG were sampled. In this case an additional comparison group of non-receiving adolescents was also interviewed. 1.6 Evaluation approach As randomisation was precluded from the evaluation design when the Child Support Grant first came into being in 1998, this study employs matching methods to establish attribution of impacts to early versus late enrolment. Participants in the study were separated into ‘treatment’ and ‘control’ groups based on whether or not they were receiving the CSG and for what length of time they had been receiving the grant. Members from each group were matched using a propensity score which in turn was based on observable characteristics. By comparing the members of each matched pair, we were able to estimate what the dosage-response to a specific number of year’s exposure to the CSG was in terms of important 8 The South African Child Support Grant Impact Assessment outcome such as schooling. The process is described in detail in Chapter 5 and the technical annex. 1.7 Organisation of the report This report has been divided into six parts: Part 1: Overview The overview has introduced the research questions and policy issues surrounding the CSG and this report’s impact estimates. It has furthermore briefly described the manner in which the survey and ensuing analysis were executed. Part 2: Research design, validity and baseline data The second part of the report will provide a more detailed look at the survey methodology, including how sampled households were selected, what the questionnaire process involved and the process of data collection and entry. This part continues on to describe basic demographic and other baseline indicators based on the resulting data, comparing them to other national studies for reference and to establish validity. Part 3: Grant access and methods for estimating impacts The third part of the report draws on qualitative and quantitative work to indicate how accessible the CSG is to South African children in need. It identifies barriers and strengths of the grant, and provides some indications for how CSG recipients interact with other social assistance grants. This part of the report concludes with a technical chapter on the methodology used to estimate impacts of the CSG on surveyed children and their families. Part 4: Impacts of the CSG on young children The fourth part of the report focuses on important impacts of the CSG on young South African children’s livelihoods, growth and human capital development. Through examining young children’s health, schooling and labour behaviour through the lens of different ages of enrolment, we were able to observe statistically significant impacts of the CSG on young children’s development. These effects were then disaggregated based on characteristics of the child (in terms of sex) and of the mother (in terms of education attained). Part 5: Receipt of the CSG and impacts on adolescents Receipt of the CSG among adolescents’ households was defined based on whether the household had ever received a CSG (either specifically for the adolescent or for another household member) and whether or not the sampled adolescent was currently receiving the CSG. Sampling children just above and below the age cut-off for the CSG at the time of the survey contributed to variation in receipt of the CSG among adolescents of the same age, and by examining their CSG receipt over time, we were able to also estimate impacts of the CSG based on differing amounts of ‘exposure’ to the grant. The adolescent outcomes that were examined include schooling outcomes, work in and outside the home, and participation in risky behaviours (sexual activity and pregnancy, drug and alcohol use, criminal and gang activity). Part 6: Conclusions and areas of further research Using the quantitative and qualitative findings of this study, the sixth part of this report synthesises findings into key policy implications. Chapter 1 Introduction, Research Questions and Approach 9 10 The South African Child Support Grant Impact Assessment © UNICEF Part 2 Research, validity and baseline data Chapter 2 Data Source and Methods 11 Chapter 2 Data Source and Methods 2.1 Introduction 2.2.1 Different types of PSUs This chapter describes the sampling process employed for the study. The sample is based on identifying two groups of households: (1) those receiving the Child Support Grant for a child approximately 10 years of age (supporting the analysis in Chapters 6, 7, 8 and 9), and (2) households which include an adolescent, either receiving or not receiving the Child Support Grant. In stage 1 of the sampling study, PSUs consist of paypoints and their catchment areas. Not every recipient of the CSG receives their grant in the same way. Recipients may collect their benefit from cash paypoints, bank accounts, Sekululas or other special types of electronic transfer, or through the Post Office64. For selected cash points, enumerators recruited participants at the pay location on pay-out day. For the other paypoint types, addresses on record in SOCPEN were used65. Occasionally, a combination of these approaches was used. For the first group, the sample includes two further subgroups: (A) households with children born in 2000 with those children (born in 2000) beginning to receive the Child Support Grant in the first 18 months after birth and (b) households with children born in 2000 but with those children beginning to receive the grant only between the ages of four and nine years. For the second group, the sample includes four further subgroups: (C) households with children born in 1995 (referred to as ‘15-year-olds’) receiving the Child Support Grant, (D) households with children born in 1994 (referred to as ‘16-year-olds’) receiving grant, (E) households with children born in 1994 (also referred to as ‘16-year-olds’) but not receiving the Child Support Grant, and (F) children born in 1993 (‘17-year-olds’) not receiving the grant. 2.2 Sampling strategy This survey covered five provinces in South Africa and used a two-stage sampling procedure. Stage 1 consisted of selecting geographical areas (Primary Sampling Units or PSUs) with a probability of selection proportionate to their size (PPS)61. From these selected PSUs, individual beneficiaries are chosen at random from the sub-set of beneficiaries on SASSA’s SOCPEN62 system that fit the age and receipt criteria in stage two of the sampling process63. 2.2.2 Large and small paypoints Pre-testing revealed that a selected cash point must have at least 27 10-year-old beneficiaries receiving the CSG to recruit the desired 16 respondents on the day of grant disbursement66. For other paypoint types (i.e. PSUs where addresses were used instead of physically meeting respondents at the paypoint on payday), 40 beneficiaries would be needed to recruit the required 16 youths. This is because the addresses held in SOCPEN were often outdated, vague67 or otherwise not useful. In the cases where a paypoint had fewer than the pre-testdetermined amount of beneficiaries, it was called a ‘small’ paypoint, and grouped with other small paypoints68. In cases where these groups of paypoints still did not carry the required minimum number of relevant beneficiaries, they were further grouped into ‘super-bundles’. These grouped paypoints’ beneficiaries were recruited by address, instead of on-site recruitment at the paypoint on disbursement day, the strategy used at ‘large’ paypoints. Table 2.1 denotes the total number of each type of PSU used in this study. 64. In Limpopo the Post Office also works as a payment agency. (Riemenschneider, 2011. 61. Riemenschneider, 2011. 65. Riemenschneider, 2011. 62. Administrative Management Information System that keeps track of SASSA social grant beneficiaries. 66. Ibid. 67. For example, addresses were listed at “next to the X river”. 63. Riemenschneider, 2011. 68. Riemenschneider, 2011. 12 The South African Child Support Grant Impact Assessment Table 2.1 Overview of PSUs in the study69 PSU Type 2.2.3 Sampling probability Number of PSUs In mathematical terms the probability of selection is given by71: Large bank paypoints (at least 40 relevant beneficiaries per paypoint) 22 Large cash paypoints (with at least 27 relevant beneficiaries per paypoint) 40 Post Office Limpopo 1 with n^ i = estimated size of PSU i Sekulula 28 Cash paypoint groups with at least 27 relevant beneficiaries per group 6 sum(n^ j) = sum of estimated size of all PSUs = relevant population (e.g. 9010 in the case of large bank paypoints) Super-bundles of cash paypoint groups (where each separate group has 1–26 relevant beneficiaries) 2 p2 i = m i /n i = number of respondents in PSU i divided by the (actual real) size of PSU i Bank paypoint bundles 4 Probability of selection = p1*p272 Total p1 i = k*n^i/sum(n^j) k = number of PSUs to be selected 103 Accordingly, several members of each sub-population in the study were to be recruited at each PSU. Table 2.2 below indicates how many members from each sub-population were supposed to be interviewed at each PSU. Table 2.2 Number of desired respondents per PSU70 Beneficiaries of the CSG Grant: The selection probability per respondent consists of at least 2 terms of selection probabilities Number of respondents per PSU Children born in 2000 (‘10-year-olds’) receiving grant from 0 to 18 months 8 Children born in 2000 (‘10-year-olds’) receiving grant from 5–9 years old. 8 Children born in 1995 (‘15-year-olds’) receiving grant 4 Children born in 1994 (‘16-year-olds’) receiving grant 6 Non-beneficiaries who are eligible to receive the CSG * As an example73, let us take the case of a specific bank paypoint in the Western Cape. This paypoint had 73 relevant 10-year-olds (enrolled 0–12 months, or 5–6 years) in the original list from 26 July 2010. The first stage selection probability (p1) of this PSU was 0.1782. The PSU was selected on the basis of PPS (probability proportionate to size) in the first stage of sampling. In September 2010 we received the lists of 10-year-olds who enrolled for the CSG at the age 0–18 months (segment A, n = 118) and at the age 4–9 years (segment B), with n = 35 10-year-olds for this paypoint, i.e. the paypoint had, in the second sampling stage, 2.1 times more respondents (n = 153) than in the first sampling stage (n = 76). Note also the small size of segment B. 71. Riemenschneider, 2011. 69. Riemenschneider, 2011. 72. Suppose we have 11 PSUs only, one with a size of 1,000 (called T for thousand) and the 10 with a size of 100 (called H, for hundred). There are 2,000 10-yearolds in total. Half live in PSU T. Let’s suppose we select 2 PSUs only, using probability proportionate to size. In that case PSU T is so large that it turns out that it will be selected with certainty. In addition, one of the 10 other PSUs will be selected. Let’s suppose, we do 10 interviews per PSU. Hence, the probability of a 10-year-old being selected in PSU T is 10/1000 = 0.001. The probability of being selected in any of the other PSUs is 1/10*10/100 = 0.1*0.1 = 0.001, i.e. the same. However, now consider that during the second stage of sampling it emerges, that PSU T has in reality 1,200 10-year-olds. In that case the selection probability of a 10-year-old in PSU T is smaller than before (and compared to the other PSUs). It is now 10/1,200. 70. Ibid. 73. Example and calculations from Riemenschneider, 2011. 16-year-olds 4 17-year-olds 6 * For those 16- and 17-year-olds in the study a ‘nearest neighbour’ approach was used. Starting from the selected adolescent households, enumerators spiralled out and recruited nearby adolescents not receiving the CSG. Chapter 2 Data Source and Methods 13 If we assume that we interviewed n = 8 segment A respondents in this PSU, the selection probability of a segment A respondent is as follows: (the quantity 0.1782 multiplied by 8) divided by 118 EQUALS 0.012 2.3 Survey instruments Three questionnaires were filled out by surveyed households and enumerators. One focused on the entire household while the other two focused in depth on the sampled young child or adolescent. In addition, adolescents completed a confidential, self-administered survey. The survey instruments are documented in the project’s Fieldwork Report. 2.4 Survey implementation and data collection The CSG Evaluation survey fielded between October 2010 and March 2011 was specially designed to provide the information needed to undertake this evaluation. It collected detailed information on household characteristics at the time of the child’s birth as well as details about their current living situation and context. These include measures of wealth, household demographic structure, characteristics of the caregiver, location characteristics and access to forms and offices needed to apply for the CSG. These data provide covariates that can be used to match early and late enrolees (explained further in Chapter 5). The CSG Evaluation Survey contained a detailed set of questions on when the household enrolled in the CSG and whether access to the CSG was interrupted, and it also collected detailed information on schooling histories, child time allocation and anthropometry. Children completed tests that covered reading and mathematics skills. In addition, an adolescent questionnaire was designed and administered to a sample of 15-, 16-, and 17-yearolds and their households. The adolescent questionnaire likewise includes questions on demographics, schooling history, labour, time allocation, access to CSG and other questions that permit comparisons of beneficiaries and non-beneficiaries and of youth just above and below the age eligibility cut-off to identify CSG impacts. The confidential, self-administered survey completed by adolescents included questions about their receipt of the CSG, school and work participation, and their engagement in risky behaviours. All of the questionnaires were captured into Stata data (.dta) files. One dataset contains information relating specifically to the sampled young children. The second dataset contains information pertaining to the sampled adolescents. Another two datasets contain information relating to the households of 1) sampled young children or 2) sampled adolescents. Still two more, known as roster datasets, contain the information on other individuals in these households. Finally, the seventh dataset contains information captured by the risky behaviour questionnaire filled out by adolescents. 14 The South African Child Support Grant Impact Assessment Chapter 3 Description and Benchmarking of the Sample 3.2 Household demographic structure and characteristics Dynamics Study (NIDS) and the 2010 General Household Survey (GHS) data populations were compared to the CSG sample population in a multi-stage process to compare observable characteristics of surveyed South Africans. First, the sample of respondents to these surveys was filtered to only those in the same five provinces corresponding to the CSG study sample. Next, two separate datasets were extracted: one with only households containing 15- to 17-year-olds; the other containing only households with 10-year-olds, in order to emulate the two datasets created from the CSG survey. These datasets were further filtered such that they contained information from families receiving the Child Support Grant. At times additional sub-selection was used, limiting results to only responses from Gauteng residents, or for other specific comparisons at the province level. To assess the validity and representativeness of the CSG sample, two other nationally representative studies were examined. The sub-samples from the 2008 National Income The first area of comparison is household demographics. This section compares household structure at each of three levels. The consistency across all three studies indicates that 3.1 Introduction This chapter examines the basic characteristics of households sampled in the CSG survey. The first section examines key demographic indicators alongside two nationally representative studies to place the CSG data within the context of established South African family trends and establish this survey’s validity. The following section describes the sample more in depth, providing a look at indicators of wealth, again side by side with other studies. The third section provides results from the financial module of this survey. Figure 3.1 Comparing sub-populations of data NIDS SIMPLE AVERAGE Population of households with 10-year-olds GHS Population of households with15- to 17-year-olds 5 Provinces population Families receiving the CSG (and in the 5 selected provinces) Sub-population comparison CSG young child data CSG adolescent data Chapter 3 Description and Benchmarking of the Sample 15 Figure 3.2 Household size (homes with 10-year-olds) Figure 3.3 Household size (homes with adolescents) 8 8 7 7 6 6,4 6,3 6,2 6,4 6,4 6,3 6,4 6,5 6,3 6 5 5 4 4 3 3 2 2 1 1 0 All homes with 10-year-olds 5 provinces of CSG survey CSG recipients in 5 provinces 0 6,2 5,9 6,3 All homes with 15 to 17-year-olds NIDS NIDS GHS GHS CSG CSG 6,2 6,0 6,4 5 provinces of CSG survey 6,2 6,1 6,4 CSG recipients in 5 provinces the CSG is representative of the surveyed population – CSG receiving families living in KwaZulu-Natal, Eastern Cape, Western Cape, Limpopo or Gauteng with (at least) A) a 10-year-old or B) a 15- to 17-year-old. pattern holds true nationally and for the five-province level and considering only households receiving the CSG. Once again, at the most specific level, the CSG study family size mean falls in between the GHS and NIDS means. In terms of household size, the data remains relatively constant as the sampled population is broken down into comparison groups for each of the three considered surveys. When each of the total samples are considered, families from the CSG survey with a 10-year-old in the home had slightly more household members (6.4 on average) than families with 10-year-olds from NIDS or GHS (6.2 and 6.3, respectively). Once the selected population is narrowed down to families receiving the CSG located in the five provinces that the CSG survey sampled, average household size generated from the CSG data falls right between the corresponding average from NIDS and GHS. The figures in this section break down household structure by the number of individuals under 18 in the average household. The number of minors in the average South African household generated from each sample was comparable across each of the three studies analysed. While the CSG study average was slightly lower in the CSG-recipient group, it was close to the NIDS and GHS averages. Analysing the CSG and GHS studies, the average household size for homes with adolescents (reported in Figure 3.2) is lower than that of homes with 10-year-olds. This Households with 10-year-olds included a higher average number of minors (individuals under 18) in the household than corresponding homes with adolescents. This pattern was consistent across each level of comparison within each of the three studies considered here. The CSG average number of children in homes with adolescents was on par with the GHS mean, and was only very slightly off from the corresponding average from the NIDS data. 16 The South African Child Support Grant Impact Assessment Figure 3.4 Number of children (<18) in household (homes with 10-year-olds) 4.0 3.5 3,4 3,5 3,4 3,4 3,6 3,5 3,4 3,6 3,5 3.0 2.5 2.0 1.5 1.0 0.5 0.0 All homes with 10-year-olds 5 provinces of CSG survey CSG recipients in 5 provinces NIDS GHS 3.3 Indicators of wealth This section compares selected indicators of wealth that are reported in the same way in each of the three studies. For households with 10-year-olds, families from the CSG data had relatively high access to electricity when compared to less specific sub-populations of the NIDS and GHS data. Once the comparison is limited to a province-specific group of CSG-receiving families, however, the results hold consistently across the studies, with CSG and GHS data almost identical, and the NIDS average very close. The CSG and GHS surveys also produce nearly identical results when examining access to electricity for CSG-receiving households in Gauteng with adolescents. Once again, we see a slightly lower degree of access for families in the NIDS data, possibly due to the earlier collection of NIDS survey data. CSG Figure 3.5 Number of children (<18) in household (homes with adolescents) 3.5 3.0 3,3 3,2 3,3 3,3 3,2 3,3 3,3 3,3 3,4 2.5 2.0 1.5 1.0 0.5 0.0 All homes with 15 to 17-year-olds NIDS GHS CSG 5 provinces of CSG survey CSG recipients in 5 provinces Over half of the sampled population (for all three surveys) reported having a corrugated iron or zinc roof. While slightly more families in the CSG-receiving sub-populations of GHS and NIDS reported having this type of roof than those from the CSG survey, the means for all three surveys were within five percentage points of each other. Comparing the figures in this section, it seems there is relatively little difference in the percentage of families with corrugated roofs between households with 10-year-olds and those with adolescents. Once again, the CSG mean is around five percentage points off from the NIDS number (although the GHS mean has jumped to around 10 percentage points off from the CSG mean). Chapter 3 Description and Benchmarking of the Sample 17 Figure 3.6 Access to electricity (homes with 10-year-olds) 89% 77% CSG NIDS GHS 77% 72% 75% 72% 89% 87% 75% CSG-receiving homes in Gauteng All homes with a Child Support Grant All homes with 10-year-olds Figure 3.7 Access to electricity (homes with adolescents) 93% 88% CSG NIDS GHS 88% 78% 79% 78% 93% 87% 73% CSG-receiving homes in Gauteng All homes with a Child Support Grant All homes with 15- to 17-year-olds 18 The South African Child Support Grant Impact Assessment Figure 3.8 Homes with corrugated iron/zinc roof (homes with 10-year-olds) CSG 59% 59% 59% All homes with 10-year-olds 5 Provinces of CSG survey CSG recipients in 5 provinces NIDS GHS 63% 68% 58% 63% 64% 63% All homes with 10-year-olds 5 Provinces of CSG survey CSG recipients in 5 provinces All homes with 10-year-olds 5 Provinces of CSG survey CSG recipients in 5 provinces Figure 3.9 Homes with corrugated iron/zinc roof (homes with adolescents) CSG 58% 58% 58% All homes with 15-17 year-olds 5 Provinces of CSG survey CSG recipients in 5 provinces NIDS GHS 63% 67% 58% 63% 67% 63% All homes with 15- to 17-year-olds 5 Provinces of CSG survey CSG recipients in 5 provinces All homes with 15- to 17-year-olds 5 Provinces of CSG survey CSG recipients in 5 provinces Chapter 3 Description and Benchmarking of the Sample 19 Figure 3.10 Bank account or savings method by percentage of households 10% Postbank only 76% Bank only 0% 8% 2% 2% 2% 3.4 Financial outcomes74 3.4.1 Introduction This section explores the reported financial behaviour of survey respondents. The financial module of the CSG is very All of the above in depth, and includes hypothetical situations for sampled Bank and Stokvel individuals with which to predict their responses. Due to Postbank and Stokvel these specific and in-depth inquiries, results have not been compared to other surveys. This module was primarily Bank and Postbank designed partly to fill in the gaps left by other South African studies. Stokvel only Figure 3.11 Reasons for not having an account by percentage of households 42% I don't have a job 22% I don't have a regular income 16% I do not have enough money left over after expenses 6% I prefer dealing in cash 6% I don't need a bank account 2% I don't know how to open an account 2% I was declined 1% Other (specify) 1% I don't qualify to open a bank account 1% I don't have the necessary documents 1% The bank, institution or group is too far from where I live 74. This section of the report was produced from a module of the survey funded by the Ford Foundation, with more detailed analysis reported in a separate report. The module is supporting the design and implementation of a pilot that links financial inclusion initiatives to the Child Support Grant programme. The South African Child Support Grant Impact Assessment When asked if respondents save money on their accounts/ in their savings groups, there was a positive response from 33.5% of bank account holders, 38.9% of Postbank account holders, and an extremely high, but not unexpected 93.3% of stokvel/savings group members. This is indicative of the importance of social involvement and social capital in such savings groups, something that does not exist with simple bank or Postbank accounts, and something that has the potential to be incorporated into low income banking or other formal savings mechanisms. The frequency of deposits into the different accounts is shown in Figure 3.12. The mean (median) deposit amounts per month for bank, Postbank and stokvel accounts are R391 (R250), R310 (R225) and R153 (R110), respectively. Interesting to note in the frequency of deposit chart is the high incidence of consistent saving in stokvels, with about 91% saving at least once a month. Also, one can note how the behaviour with bank and Postbank accounts tends to be centred at the extremes, with those who save saving quite regularly, and the majority of the others not saving at all. 91,0% 80 48,1% 32,4% 40 20 0 Never Less than once a year 1-3 times a year 1,0% 5,6% 6,0% 60 1,5% 6,0% 4,2% A mixture of reasons was provided by the 1,251 (42.9%) respondents who did not have an account. These could be broadly categorised into process issues (e.g. don’t know how, don’t have the documents), misperceptions concerning banks and bank supply issues (e.g. bank too far away), and issues concerning misperceptions as to the requirements for opening a bank account (e.g. no income, no job, no money left over after expenses). Approximately 10% of the nonbanked group did not open bank accounts as a result of lack of demand. 100 0,5% 1,4% 2,1% Of the sample of CSG survey respondents, 57.1% (1,691) had personally opened a bank account, Postbank account or an account in a stokvel, society or Umgalelo. The distribution of the type of accounts held indicates that 1,288 (76%) of the 1,691 who had opened accounts had only a bank account, with this number increasing to 1,454 (86%) when including those who have bank accounts in conjunction with one of the other account types. Figure 3.12 Frequency of deposits by percentage of households for different account types 51,9% 39,6% 3.4.2 Saving behaviours 1,5% 20 1-2 times every At least three months once a month Bank (n=1,288) Postbank (n=216) Stokvel (n=199) One also can observe a difference in the number of people who say they do not save, but continue on to say that they deposit regularly. This could be as a result of a discrepancy in what is actually considered ‘saving’, in that ‘saving’ might be considered by some to be something with an ultimate goal, such as making a large purchase. An alternative explanation may be that those who make deposits but claim not to save use their accounts in purely a transactional fashion. Uptake of insurance products was quite low in the sample, with only 2.9% having long term insurance (such as pensions, or retirement plans) and only 1.4% having short term insurance (such as asset insurance). However, over half of all households surveyed (53.8%) had a burial plan or burial insurance. Chapter 3 Description and Benchmarking of the Sample 21 Figure 3.13 Long term insurance, short term insurance and burial plan uptake Figure 3.14 Mean expenditure by banking status 100 Unbanked n = 1214 Banked n = 1616 Per cent 80 60 40 R1174.10 R1321.75 Long term insurance Burial plan 0 Short term insurance 20 Yes No 3.4.3 Comparison of banked and un-banked respondents To assess the potential impact of bank accounts, and to compare characteristics, the sample was divided into those who are banked and those who are unbanked. This was done by separating respondents based on their responses to questions on whether they have a bank account, and comparing means or frequency of positive responses. For many of the variables, a t-statistic was calculated to determine significance, with a 1.96 t-statistic indicating a significant difference within a 95% confidence interval. Expenditure was 12.58% higher for banked respondents, with mean expenditures of R1,174.10 and R1,321.75 for unbanked and banked respondents respectively, a difference yielding a t-statistic of 2.29. A larger difference is seen with regards to debt, with banked households holding 29.6% more debt than unbanked households, a significant difference (t = 2.66). 25.3% of banked households were in debt, compared to 19.4% of unbanked households. Debt could represent an individual’s willingness to invest in productive assets that will ultimately amplify their ability to earn a sustainable and higher income. There is a sizeable split between banked and unbanked debt when looking at the debt by percentile, with debt approximately the same up until the 40th percentile, after which the difference increases. The difference in debt could be a result of several factors. From the point of view that bank accounts have a causal relationship with debt, it is possible to argue that households with bank accounts have greater access to bank loans, as there is increased legitimacy in the view of the bank as a lender. An alternative non-causal explanation could be 22 The South African Child Support Grant Impact Assessment Figure 3.16 Mean debt by banking status Figure 3.15 Expenditure by percentile 2500 Unbanked n = 243 2000 Banked n = 429 1500 1000 R1653.00 500 0 10% 20% 30% 40% 50% 60% 70% 80% R2350.00 90% Unbanked Banked Figure 3.17 Household debt by banking status and percentile 10000 8000 6000 4000 2000 0 10% 20% 30% 40% 50% 60% 70% 80% 90% that the correlation between bank account ownership and expenditure could indicate that the greater wealth of those with bank accounts means they are able to borrow more, as there is more collateral, greater willingness of lenders to lend more to those with higher incomes etc. Or, it would even be possible to say that debt, access to which is increased through bank accounts, allows for higher one-period expenditure, as an explanation for the observed correlation. Analysis of household possessions also showed that banked households were better off. Selected items from the questionnaire are shown in Figure 3.18, with the percentage of households in possession of the item shown by the bar, with the t-statistic for the difference in ownership between banked and unbanked households shown in brackets. Banked Unbanked Worth noting is how consistently banked households have more than the unbanked. There is an increase in the difference the more ‘sophisticated’ the goods are, with inferior goods such as bicycles having an insignificant difference (t = 1.06), while Hi-Fis have a more significant difference (t = 7.18). Cars/motorbikes and computers showed a significant difference. Chapter 3 Description and Benchmarking of the Sample 23 Figure 3.19 Distribution of distances to money withdrawal point Figure 3.18 Household possessions by banking status 62,57 66,51 Radio 30,33 HiFi/CDPlayer/MP3 .015 Density 86,51 91,93 Cell Phone 7,50 10,05 Motorbike or Car kernel = epanechnikov, bandwidth = 10 43,02 6,86 9,64 Computer Kernel density estimate .02 .01 .005 4,31 5,14 Bicycle 0 20 40 60 80 100 Banked Unbanked Another point to note is the level of cell phone ownership. Although the significance of the difference is high, both banked and unbanked households have a very high level of cell phone ownership. Analysis of distances to ATMs or bank branches seems to show almost no correlation between travel duration and uptake of banking products. Figure 3.19 shows the distribution of the distance in minutes of both the banked and unbanked groups. There is no substantial difference. The means for the banked and unbanked groups are 30 and 32 minutes respectively. There was a high concentration around the 20 and 30 minute responses, as would be expected by individuals estimating travel time.75 75. To compensate for the discrete nature of the answers, a bandwidth of 10 was used in the kernel density estimation to have a smoother, more comparable distribution. 0 0 50 100 150 200 Time to ATM or Money Withdrawal Point Unbanked Banked 3.4.4 Conclusions This statistical analysis demonstrates consistent evidence that having a bank account or access to other financial instrument is a signal among CSG recipients of having greater material wealth measured by both a few proxy variables and consumption levels. There is an existing demand for bank accounts among CSG recipients that has gone unmet. Distance to a bank or ATM does not appear to be the cause of this gap between demand and supply, but rather the majority of people indicate that factors like not having a job, not having regular income, or not having enough money left over after expenses make up the majority of reasons for not having a bank account. However, none of these responses prevent someone from having an account. 24 The South African Child Support Grant Impact Assessment © UNICEF/NYHQ2006-1404/Pirozzi Part 3 Grant access and impact estimation methods Chapter 4 Grant Access 25 Chapter 4 Grant Access 4.1 Introduction This chapter analyses how effectively children, youth and their caregivers can access the Child Support Grants for which they are eligible. A major dimension of this is the understanding of beneficiaries, non-beneficiaries and other key informants about the eligibility criteria for the CSG, with a particular focus on the recent extension of the age threshold up to 18 years, and on documentation required during the application process. This chapter draws substantially on qualitative work completed during an earlier phase of the project.76 Fieldworkers asked eligible non-beneficiaries and key informants about reasons for not applying, and this chapter further analyses these reasons. The chapter also presents perceptions about recent improvements in the application process. Respondents offered suggestions for further improvements to the application process, which are discussed in this chapter. This chapter also draws together quantitative analysis of grant access based on national representative household surveys. 4.2 Eligibility criteria Three sets of criteria largely determine eligibility for the Child Support Grant: an income-based means test, age thresholds and nationality. The ‘means test’ assesses the income of the CSG applicant (the child’s biological parent or primary caregiver), the income of their spouse or partner, and the income of the beneficiary (the dependent child). In order for an applicant to quality as eligible, the sum of all three incomes must fall below a threshold that is adjusted every year. In 2010/11, the means test threshold for the CSG was set at R2,500 per month (=R30,000 per annum) for a single caregiver and R5,000 per month (=R60,000 per annum) for a married caregiver plus spouse, plus dependent children. An official from the South African Social Security Agency (SASSA) in Port Elizabeth reported how technology enhancements to the management information system (SOCPEN) were strengthening the means test enforcement process: 76. Devererux, 2011. “The means test is now very strict, because those people who are eligible for the grant do not get the grant and it is being accessed by people who are not eligible for it. For example, many applicants have been government employees and did not disclose that they were working in government institutions. We are now in the process of picking all of them in the system and calling them in to provide us with their bank statements and other documents, such as a letter that states which department they were working for. We have given them a period of three months to provide these documents.”77 While most recipients and non-recipients understand that the government applies a means test for Child Support Grant eligibility, a great deal of confusion and misunderstanding clouds the details, particularly in terms of the income threshold above which a caregiver is not eligible, and how formal sector employment (particularly public sector employment) affects eligibility. The legacy of the CSG’s first decade with an unchanging means test threshold appears to reinforce perceptions of an income threshold that is much lower than the actual one, when in fact the means test has become significantly more inclusive over the past several years. One SASSA official from KwaZulu-Natal reported that employed caregivers frequently believe that any employment rules out eligibility for the grant, without understanding that the current income threshold enables millions of workers in South Africa to be eligible for the Child Support Grant. There is no employment test, just the income threshold, and the income threshold is high enough to include many workers. “Some people don’t apply because they are working so they think they are ineligible, because they heard about the means test, but if they are earning only a little money they might still be eligible so they should apply.”78 Similarly, a woman from rural Limpopo, separated but not divorced from her husband, has not applied for the Child Support Grant because of her misperception that her husband’s government employment rules out her eligibility for the grant. 77. Devereux et al., 2011, [EC-U/KI-1]. (Where italicized brackets such as [EC-U/ FG-7] have been used, they refer to focus group transcriptions from the study’s qualitative component. The first two or three letters refer to the province where the focus group occurred, while the number following FG- indicates which focus group the transcription comes from.) 78. Ibid., [KZN-U/KI-1] 26 The South African Child Support Grant Impact Assessment “My husband left me in the house and was a government employee, now it’s been five years since he left me. His employment under government still affects me and we still have marriage certificate. So it’s hard for me to go and apply, because I heard that when you are married to someone who works you can not apply because it’s on the computer system, and I’m also scared because it will appear on the computer. But I am suffering because I’m still not employed and have no income.”79 The misinformation about eligibility reflects a complexity to the means test and the challenges in ensuring that awareness about changes in the targeting rules are disseminated to populations that are often disconnected from reliable information sources. This is compounded by challenges even for SASSA officials and social workers, who are sometimes the source of inaccurate information. One non-beneficiary from Limpopo reports: “Some situations you find that the mother is not working and the father is working but he does not look after the kids for no apparent reason, but when the mother visits the office to apply, they will tell her that she can’t get the CSG because the father is working.”80 test threshold.82 Both the qualitative and quantitative findings suggest that more effective dissemination of detailed information about CSG eligibility criteria will likely reduce the programme’s targeting errors of exclusion. Administrative rules – tied again to the complexity of the targeting system – can also exclude the youngest caregivers, because a teenage mother cannot be simultaneously both a caregiver applicant (on behalf of her own children) and a beneficiary (in terms of her own mother’s application). A SASSA official from KwaZulu-Natal reports the predicament and one approach to addressing the resulting potential exclusion: “Sometimes you find that a teenager gives birth to a child while she is still under her parents’ supervision and she receives the CSG. Then it becomes problematic because a teenage mother who still receives the CSG cannot receive the CSG for herself and the child. So in those circumstances you as a parent have to decide whether you want to receive your own CSG or for the child because you cannot receive both grants at the same time, instead it gets suspended. The best thing is to allow the parent of the teenage mother to receive both grants until the teenage mother is kicked out of the system, then she can claim back her child’s CSG.”83 This is likely to become an increasingly common circumstance given the increase in the age threshold for eligibility. Likewise, a female recipient from the Eastern Cape reports: “I went to social workers and asked them about the grant and they asked me whether my husband was working or myself, I said no and then they said I qualify to apply.”81 The qualitative work suggests that misunderstanding about the means test and perceptions of the disqualifying impact of employment discourage many eligible caregivers from applying. These qualitative results are reinforced by quantitative analysis of the 2008 National Income Dynamics Survey, which finds corroborating evidence. More than one out of 10 poor eligible caregivers who did not apply for the Child Support Grant reported believing their income was too high – even though in reality their incomes fell below the means A further area of misunderstanding, although not widely addressed by respondents, centres on the citizenship and residency status of applicants. The official eligibility criteria for the Child Support Grant limit the benefit to applicants who are citizens or permanent residents of South Africa, without regards for the status of the beneficiary child. Since the caregiver applies on behalf of the child, it is the status of the caregiver and not the child or spouse that determines eligibility. These subtle details create significant confusion, compounded by the perception that possession of valid documents of citizenship or residency is a requirement for eligibility. One respondent from Lenasia, Gauteng reported circumstances that characterise a significant source of exclusion error in the programme: 79. Devereux et al., 2011, [Lim-R/FG3]. 80. Ibid., [Lim-R/FG3]. 82. Samson et al., 2012. 81. Ibid., [EC-R/FG2]. 83. Devereux et al., 2011, [KZN-P/KI-1]. Chapter 4 Grant Access 27 “They told her she is not a resident and as a non-resident the child won’t be able to get a birth certificate. And she told them ‘I do have an ID’ but they said it doesn’t matter. If the father is a resident and the mother is not, you can’t get the birth certificate and you can’t get the CSG.”84 Analysis of the 2008 National Income Dynamics Survey shows that problems with documents is the number one reason cited by poor eligible caregivers for their decision not to apply for Child Support Grant. As Figure 4.1 below demonstrates, more than one out of four poor eligible caregivers who did not apply for the Child Support Grant reported the lack of documents as the reason for not applying, even though SASSA has developed alternative qualifying procedures that relax the documents requirements.85 Figure 4.1 Reasons caregivers of poor children eligible for receipt do not apply86 Simultaneously, problems with getting the correct documentation were the number one reason why application for the CSG was delayed. Figure 4.2 below suggests that the documents problem, relayed by respondents to the 2008 NIDS study above, still existed by the time this study was conducted in 2010. Therefore, despite the fact that 97 per cent of initial applications by respondents to this survey were approved, this number probably fails to shed light on the amount of would-be recipients that never apply or delay application. A SASSA official87 also stated that applicants that show up to the SASSA office without the proper documents are turned away or screened even before their file is opened, indicating that their attempt to get a grant is not recorded in the administrative data system (SOCPEN). Figure 4.2 Reasons caregivers delayed application for the CSG (principal reasons)88 35 25 38,7% 40 30 30 20 25 85. Samson et al., 2012. 86. Author’s own calculations using NIDS 2008, for more detail on how eligibility was calculated, see Samson, et al., Forthcoming 87. Samson, et al., Forthcoming. 88. Authors’ own calculations using CSG data. 3,8% 2,2% Did not think I was responsible for this 7,3% Time constraints 84. Devereux et al., 2011. [Gau-U/FG-4] Cost of applying too high 7,8% Application process too complicated Caregiver has not heard of CSG . 0 Problems with getting documentation Someone else applied Cost too high Child too old Not child's mother Too complicated Other No info on applying In progress Don't know Income too high No time 5 No documents 0 8,7% 5 Did not know how to apply 10 9,7% 13,8% 15 10 Though application would be unsuccesssful 20 15 28 The South African Child Support Grant Impact Assessment 4.3 Extension of age threshold The qualitative fieldwork found widespread awareness about increases in the age threshold for Child Support Grant eligibility. A SASSA official described the range of activities that the agency employed to promote awareness: “We distribute pamphlets and do road shows, and we go to the local radio station and give information, also to the ‘indunas’ in the rural areas.”89 Respondents report that SASSA’s efforts to communicate the changes have been effective, both through direct communications: “I received a letter from SASSA informing me to come to re-apply.”90 ...and through media information campaigns: “We heard about it on the radio, that a child who is 16 can now receive grant till the age of 18.”91 SASSA also effectively leveraged educational institutions to raise awareness: “Mine said the teachers at school announced that children born in 1994 can register for CSG.” 92 Teenagers aware of the increasing eligibility age threshold encouraged their caregivers to apply (or in some cases reapply), sometimes identifying the value of the resources for meeting educational expenses: “She comes to me and says: ‘Mom, I would really love you to go and apply for CSG money for me, so that you can pay for my things at school’”.93 “I was also told by my child: ‘Mama, go and apply so I can get transport fare to school, because you do not make enough money from vending’”.94 As discussed further in Chapter 10, adolescent awareness of the Child Support Grant proved to be a statistically significant determinant of actual receipt, and this factor contributed to building a model for the rigorous attribution of impact. 4.4 Documentation required As discussed above, the clearest message from the analysis of the 2008 National Income Dynamics Survey is the extent to which problems with documents both deter poor eligible caregivers from applying and lead to the rejection of their applications. The qualitative phase of the project explored the impact of missing documents on the application process. Respondents reported a number of core document requirements for a Child Support Grant application: ˜˜ parents’ ID documents, ˜˜ child’s birth certificate, ˜˜ clinic (immunisation or ‘Road to Health’) card, ˜˜ marriage certificate, and ˜˜ proof of address. Respondents also identified additional documents required under special circumstances: ˜˜ a police affidavit (if any key documents are missing); ˜˜ a letter with the ward councillor’s stamp (if there is no other way of establishing proof of address); ˜˜ mother’s ID document (if the child is registered by someone else, e.g. a grandmother); and ˜˜ proof of (un)employment – for the means test. 89. Devereux et al., 2011. [Lim-R/KI-1] 90. Ibid. [EC-U/FG-7] As reported from EPRI’s previous fieldwork in Mount Frere, Eastern Cape95, the cost and difficulty of securing 91. Ibid. [Lim-P/FG-3] 92. Ibid. [KZN-U/FG-7] 94. Devereux et al., 2011. [KZN-U/FG-7] 93. v. [Lim-P/FG-7] 95. Samson, 2002. Chapter 4 Grant Access 29 the required documents frustrates applicants and sometimes leads them to give up on the application process: “Sometimes they ask you to provide proof of residence, or electricity or water. If you are unemployed or staying in RDP houses you cannot have these things, because we do not pay for water and do not use metered electricity. So you might end up being discouraged to continue trying, because they will not assist you without these documents.”96 Chapter 10’s finding that persistence in re-applying for the grant in the phase of rejection is a statistically significant determinant of CSG receipt confirms the importance of cost and difficulty in accessing the grant, as well as for the attribution strategy of this study. The persistence of the documents problem in the application process has been cited as a complicating factor in attempts to reform the Child Support Grant’s means test.97 The qualitative work undertaken as part of the impact assessment’s larger study helps explain why the problem is so entrenched. For example, the Department of Home Affairs may require the child’s clinic card (‘the Road to Health’ card) in order to provide the child’s birth certificate, and both have at times been required to apply for the Child Support Grant. Respondents frequently cited problems with recalcitrant fathers refusing to support the mothers’ applications for the grant on behalf of the child: “The father of my child is refusing to give me the child’s certificate and I had a certified copy, but these people are refusing to help me; instead they are saying that I should go get the certificate. I ask them how because the father is refusing and they tell me they have no idea how, because we want the certificate otherwise we will not register you.”98 4.5 Reasons for not applying While documents problems and misunderstanding the means test represent two of the three most widely cited reasons in the 2008 National Income Dynamics Survey for poor eligible caregivers not applying for the grant, several other reasons also contributed to the resulting exclusion error. Further analysis of the survey identify a range of other factors, including that the process is too time-consuming or otherwise costly, they have a general lack of awareness of the process, or the process is too complicated. Other factors – such as the caregiver not being the child’s mother or the wrongful perception that the child is too old – reflect a misunderstanding of the application process.99 The qualitative fieldwork as part of this larger study helps to illuminate these findings from household survey analysis. For example, a SASSA official from the Eastern Cape indicated that high transport costs discouraged rural caregivers from applying100, a finding similar to that from EPRI’s initial work in Mount Frere 10 years ago. A respondent in the qualitative study from KwaZulu-Natal reported: “My wife used up a lot of my money during the application process; I ended up spending more for the application than what we were going to get!”101 The qualitative study’s findings about misunderstanding and lack of information echo the results from the quantitative analysis: “Sometimes mothers leave their kids with the grandmother, and then they don’t know who should apply for the grant – the mother or the grandmother – so nobody applies even though the family is poor and should be eligible.” (SASSA official, KwaZulu-Natal)102 A comparison of reasons from the impeding application process cited in the qualitative study with the quantitative analysis shows a high degree of correlation. In both the qualitative and the quantitative analysis, the most common 99. Samson et al., 2012. 96. Devereux et al., 2011. [KZN-/FG 4] 100. Devereux et al., 2011. 97. Samson et al., 2012. 101. Ibid., 2011. [KZN-P/FG-4] 98. Devereux et al., 2011. [KZN-U/FG-3] 102. Ibid., 2011. [KZN-U/KI-1] 30 The South African Child Support Grant Impact Assessment problem excluding poor eligible caregivers was lack of required documents, particularly in the Eastern Cape. Lack of information – including misunderstandings and perceptions of complications – represented a top factor in both studies (particularly in rural areas), as well as problems with the costliness and time-consuming nature of the application process. Corruption was not a significant factor in any of the studies.103 4.6 Improvements in grant access Both the qualitative and quantitative analysis find substantial improvements in the application process over the past 10 years, with resulting reductions in both inclusion and exclusion errors of targeting.104 The qualitative analysis identified several important factors contributing to this improvement:105 (1) The number of required documents has fallen: “It is easy now, but back then it was difficult, because you find that you don’t have the marriage certificate – you separated from your husband, and he left with the documents, and it is going to be a long process to find the documents.” “Now they want the certificates of the child who you are applying for the grant. In olden days you were asked to bring a letter from the counsellor and the school principal to sign, but now it is easier compared to previous years.” (2) SASSA more effectively communicates and publicises information about registration procedures and the documents required: “There are changes – now it is easy. Before, you had to go up and down to collect the documents needed, but now they announce that on that date they will be in a community hall and they mention all the documents.”106) “It is better now than in 2001 because welfare officials come to school. I don’t need to spend money anymore to go to town to apply for the grant.” 103. Samson et al., 2012, Devereux et al., 2011. 104. Samson et al., 2004, 2007, 2012; Devereux et al., 2011. 105. Devereux et al., 2011: (1) [EC-R/FG-2] (2) [EC-U/FG-1], [EC-R/FG-1] (3) [KZN-U/FG-2] (4) [EC-P/FG-2] 106. See Samson, 2002, for examples of the difficulties 10 years ago. (3) Technological improvements have become more rapid: “I have a grandson who is three years old. I came in about 2pm and at the end of that day, I got out with everything that clearly stated when the date that I will get the first payment. The process of application is much faster than before, when I first registered.” (4) Improvements in financial inclusion mechanisms lower the cost to the recipient of accessing the grants: “They ask you if you want to get the grant from the bank, then they transfer the grant to the bank and you withdraw it there at any time during receiving dates.” (5) Corruption does not impede the application process or delivery of benefits: “We should also thank Mr Cele for being a councillor, because since he came in four years ago there is improvement because they fear that he may report them.” 4.7 Correlates of early application Analysis of the CSG data reveals several patterns associated with early application for the CSG. As Table 4.1 shows, one of the most significant factors is whether or not the mother was given the CSG application form when the child was born. This finding suggests that a campaign to distribute applications and/or spread information about the CSG at birth (for instance: in hospitals/maternity wards/midwives’ offices) would have a positive impact on early access to the CSG. 4.8 Access to other grants Households receiving the Child Support Grant are more likely to receive another grant than households who receive no CSG. The distribution of other grants, however, varies depending on the data source used to measure the cross-receipt. The following section compares the sample households’ receipt of other grants to evidence from two other surveys: (1) the 2008 National Income Dynamics Survey and (2) the 2010 General Household Survey. The section Chapter 4 Grant Access 31 is organised by grant, in order of prevalence: first reporting results for the Old Age Pension, then the Disability Grant and finally the Care Dependency Grant. By the time the report reaches the third grant, prevalence within the sample has fallen by an order of magnitude – and other grants besides these three have insignificant representation in the sample and are ignored in this analysis. Old Age Pension Households receiving a Child Support Grant are more likely to also receive an Old Age Pension than any other grant. The mean number of Old Age Pensions received by households in both the young child sample and the adolescent sample are very similar – approximately 0.3 grants. This is very close to the number in the 2010 General Household Survey (GHS) households with 10-year-old children, which is slightly higher. For households in the GHS sample with 10-year-old children in the five provinces of the study and that receive the Child Support Grant, the mean number of Old Age Pensions is 0.34, compared to the comparable point estimator for the CSG sample of 0.27. However, the mean from the 2008 National Income Dynamics Survey (NIDS) is inexplicably about twice as high – with a corresponding mean of 0.67 (for households with 10-year-old children in the five provinces of the study and that receive the Child Support Grant). The NIDS sample is much smaller than the GHS sample, and two years older – but these factors do not persuasively explain the difference. These numbers are compared in Figure 4.3. Figure 4.3 Mean number of Old Age Pension Grants per household (homes with 10-year-olds) 0.8 0.7 Table 4.1 Correlates of early application Variable 0,67 0.6 0.5 Marginal effect Z statistic Child is a boy -0.049 -1.59 Mother’s age (if known) 0.007 3.23** Mother’s age known -0.007 -0.73 Mother’s schooling (if known) 0.019 3.53** 0.1 Mother’s schooling known 0.226 2.24** 0.0 Father lives in community 0.092 2.69** Dwelling has tile roof -0.049 -0.73 Dwelling has electricity -0.025 -0.64 Given application when child born 0.130 2.44** Travel time to SASSA office -0.019 -0.53 Born in KwaZulu-Natal 0.034 0.73 Born in Eastern Cape 0.083 1.49 Born in Western Cape 0.023 0.48 Born in Limpopo 0.220 3.37 Born in urban area 0.036 0.56 Born in peri-urban area 0.004 0.08 Born in rural area 0.021 0.35 Source: Calculated from household survey. *, significant at the 10% level; **, significant at the 5% level; *** significant at the 1% level. 0,67 0,60 0.4 0,34 0.3 0.2 0,27 0,27 All homes with 10-year-olds 0,27 0,28 5 provinces of CSG survey 0,27 CSG recipients in 5 provinces NIDS GHS CSG For households with adolescent children in the GHS sample in the five provinces of the study and that receive the Child Support Grant, the mean number of Old Age Pensions is 0.32, compared to the comparable point estimator for the CSG sample of 0.28 – representing a narrowing of the difference compared to the young child samples. Likewise, the mean from the 2008 National Income Dynamics Survey (NIDS) is also closer, but still inexplicably higher – with a corresponding 32 The South African Child Support Grant Impact Assessment mean of 0.40 (for households with adolescent children in the five provinces of the study and that receive the Child Support Grant). Again, the smaller size of the NIDS sample and its earlier survey period do not persuasively explain the difference. These numbers are compared in Figure 4.4. Disability Grant Households receiving a Child Support Grant are less likely to also receive a Disability Grant than an Old Age Pension, but this is nevertheless the second most prevalent other grant in the sample. The mean number of Disability Grants received by households in both the young child sample and the adolescent sample are very similar – ranging between 0.15 and 0.17 grants. Again, this is very close to the numbers in the 2010 General Household Survey (GHS) households with 10-year-old children, but also for households with adolescents. For households in the GHS sample with 10-year-old children in the five provinces of the study and that receive the Child Support Grant, the mean number of Disability Grants is 0.14, compared to the comparable point estimator for the CSG sample of 0.15. Likewise, for households in the GHS sample with adolescent children in the five provinces of the study and that receive the Child Support Grant, the mean number of Disability Grants is 0.15, compared to the Figure 4.4 Mean number of Old Age Pensions per household (homes with adolescents) 0.40 0,40 0,40 0,40 0.35 0,32 0.30 0.25 0,28 0,28 0,25 0,26 0,28 0.20 0.15 comparable point estimator for the CSG sample of 0.17. However, the corresponding means from the 2008 National Income Dynamics Survey (NIDS) are inexplicably nearly twice as high – with a corresponding mean of 0.27 (for both households with 10-year-old children and households with adolescent children, in the five provinces of the study and that receive the Child Support Grant). Again, the salient factors distinguishing the NIDS sample from the GHS sample do not persuasively explain the difference. These numbers are compared in Figure 4.5. Care Dependency Grant Only between one in every 10 and one in every 25 South African households receiving a CSG also receive a Care Dependency Grant (with the estimates varying widely depending on the data source), with the prevalence of cross-receipt much lower than for Old Age Pension or the Disability Grant. The mean number of Care Dependency Grants received by households in both the young child sample and the adolescent sample are the same – an estimated 0.04 grants. This estimate is only about half the estimates derived from both the 2010 General Household Survey and the 2008 National Income Dynamics survey, which themselves yield similar estimates, at least for households with young children. For households with adolescents, the CSG sample yields an estimate closer to the NIDS estimate, compared to the GHS estimate, reversing the pattern demonstrated by two larger grants (the Old Age Pension and the Disability Grant. However, the sample sizes are fairly low in all three surveys, so it is difficult to draw robust inferences from this analysis. These numbers are compared in Figure 4.6. The main conclusion from this analysis is that it is likely to be even less fruitful to compare cross-receipt of the other smaller grants, because the sample sizes will not support significant comparisons. 0.10 0.05 0.00 All homes with 15 to 17-year-olds NIDS GHS CSG 5 provinces of CSG survey CSG recipients in 5 provinces With the exception of the Care Dependency Grant, the cross-receipt documented in the impact assessment survey is very close to that reported in the 2010 General Household Survey, but very different from the 2008 National Income Dynamics Survey. There is no obvious explanation for this difference. The estimates of cross-receipt prove useful in subsequent chapters in helping to explain households’ participation in the Child Support Grant programme. Chapter 4 Grant Access 33 Figure 4.5 Mean number of Disability Grants per household 0.30 0.30 0.27 0.25 0.27 0.25 0.17 0.17 0.17 0.15 0.12 0.10 0.12 0.15 0,15 0,15 0,12 0.10 0,15 0,14 0,11 0.05 0.05 0.00 0,27 0.20 0.20 0.15 0,27 0,26 0.25 All homes with 15 to 17-year-olds 5 provinces of CSG survey CSG recipients in 5 provinces 0.00 All homes with 10-year-olds NIDS NIDS GHS GHS CSG CSG 5 provinces of CSG survey CSG recipients in 5 provinces Figure 4.6 Average Amount of Care Dependency Grants per household 0.6 1.0 0,60 0,90 0.5 0.4 0,50 0,40 0,40 0,50 0,40 0.8 0,90 0,90 0,90 0,90 0,80 0.6 0.3 0.4 0.2 0.1 0.0 0,10 0,10 0,11 All homes with 15-17-year-olds 5 provinces of CSG survey CSG recipients in 5 provinces 0,40 0,40 0,40 0.2 0.0 All homes with 10-year-olds NIDS NIDS GHS GHS CSG CSG 5 provinces of CSG survey CSG recipients in 5 provinces 34 The South African Child Support Grant Impact Assessment Chapter 5 Methods for Assessing the Impact of the Child Support Grant 5.1 Introduction The goal of an impact evaluation is to measure causal programme impacts as differences in outcomes between the beneficiaries and their counterfactual, that is, a proxy for what outcomes would have been for this group had they not received the programme. All impact evaluation strategies are designed to identify a method for constructing a proxy for these counterfactual outcomes, typically using information on non-beneficiaries. This requires controlling for the effects of confounding economic and contextual factors that make programme beneficiaries systematically different from an average non-beneficiary. These confounding factors can include the relative poverty of beneficiaries in targeted programs, exposure to economic shocks, or differences in household characteristics (e.g. demographics, parental schooling, or social networks) that affect the impacts of the programme. Impact estimates that imperfectly control for these confounders suffer from ‘selection bias’. In this chapter, we describe the methods we have used to assess the impact of the Child Support Grant. We begin by outlining the ‘gold standard’ approach and explain why our approach, by necessity, diverges from this. We then describe the approach we take in this report, matching and enumerating its strengths but also being careful to note its weaknesses. An appendix provides a technical treatment of our methods. 5.2 Double Difference Methods with Randomisation: A ‘Gold Standard’ Approach Central to many impact evaluations is the application of ‘difference-in-differences’ or ‘double difference’ methods to longitudinal data. These methods use baseline data before a programme is implemented and follow-up data after it starts to develop a ‘before and after’ comparison. These data are collected from households or individuals receiving the programme and those that do not (‘with the programme’/’without the programme’). To see why both ‘before/after’ and ‘with/without’ data are valuable, consider the following hypothetical situation. Suppose an evaluation only collected data from beneficiaries, and that in the time between the baseline survey and the follow-up, some adverse event occurred (such as a recession which leads to job losses, affecting some households more than others) that makes these households worse off. In such circumstances, beneficiaries may be worse off – the benefits of the programme being more than offset by the damage inflicted by the recession. Alternatively, suppose that in some part of a country, reforms improve school access and thus change enrolment. These effects would show up in the difference over time in the intervention group, in addition to the effects attributable to the programme. More generally, restricting the evaluation to only ‘before/after’ comparisons makes it impossible to separate programme impacts from the influence of other events that affect beneficiary households. To ensure that our evaluation is not adversely affected by such a possibility, it is necessary to know what these indicators would have looked like had the programme not been implemented. Thus, we need a second dimension to our evaluation design which includes data on households ‘with’ and ‘without’ the programme. To see how the double difference method works, consider Table 5.1107. The columns distinguish between households that were receiving cash benefits at baseline (Group I for intervention) and those that were not (Group C for control group). The rows distinguish between before and after the programme (denoted by subscripts 0 and 1). Consider one outcome of interest – the measurement of school enrolment rates for children aged 9–10. Before the programme, we would expect the average enrolment percentage to be similar for the two groups, so that the difference in enrolment rates (I0 – C0) would be close to zero. Once the programme has been implemented, however, we expect differences to emerge between the groups, so (I1 – C1) will not be zero. The double-difference estimate is obtained by subtracting the 107. Maluccio & Flores, 2005. Chapter 5 Methods for Assessing the Impact of the Child Support Grant 35 pre-existing differences between the groups, (I0 – C0), from the difference after the programme has been implemented, (I1 – C1). Under certain conditions (described below), this design will take into account pre-existing observable or unobservable differences between the two assigned groups, thus generating average programme effect estimates. Table 5.1 Calculation of the double-difference estimate of average programme effect Survey round Intervention Control group group (Group I) (Group C) Difference across groups Follow-up I1 C1 I1 – C1 Baseline I0 C0 I0 – C0 Difference across time I1 – I0 C1 – C0 Double-difference (I1 – C1) – (I0 – C0) The double-difference method can be illustrated graphically, as in Figure 5.1. For an arbitrary indicator measured over time, it is assumed (for the graph) that both the intervention and control groups start at the same level (on the vertical axis). No change in the indicator over time would lead to the outcome depicted by point I0 = C0. If only the intervention Figure 5.1 Illustration of the double-difference estimate of average programme effect Intervention I1 Control C1 group were being followed, one would then naïvely calculate the effect of the programme as I1 – I0. However, as the control group makes clear, there was a trend over time that led to an improvement (in this example) of C1 – C0. Estimates ignoring this would overstate the programme effect. Instead, the correct estimate of the programme effect is I1 – C1; this is the double-difference estimate since I0 = C0. In the case where the trend line for the control group was declining, ignoring that effect would tend to understate the programme effect. Applications of the double-difference method require a strategy for removing the selection bias described above. One way of doing so is to assign access to the programme randomly, which eliminates selection bias and allows for a rigorous evaluation108. The intuition is that if access to the programme is random within a group of similarly eligible households, beneficiary or treatment status cannot be correlated with the outcomes. As a result, any observed differences in outcomes over time between the intervention or treatment group and the experimental comparison group (‘control group’) must be a result of the programme. Finally, in an idealised impact evaluation, all beneficiaries receive exactly the same treatment. For example, the same level of transfers would be delivered to all beneficiaries in the same manner with the timing of these transfers the same for all beneficiaries. Differences in treatments within the beneficiary group make it difficult to disentangle the impact of the intervention from factors that led to differences in the delivery of the treatment. I0 = C0 Baseline Follow-up 108. Heckman, Ichimura, & Todd, 1997. 36 The South African Child Support Grant Impact Assessment 5.3 Evaluation methods for the CSG: using binary matching methods 5.3.1 Overview It is relatively rare for impact evaluations to meet the ideal standards described above, and our evaluation strategy and methods likewise must depart from the gold standard approach for three reasons: ˜˜ There is no scope (legally or practically) for randomly allocating the CSG; ˜˜ Children first obtained access to the CSG at different ages. Some children were enrolled at birth while others did not receive their first CSG until they were eight or even older; and ˜˜ We have a single cross-sectional survey that provides the data for the evaluation. At the same time, the data available to us have a number of strengths. Most importantly, we have detailed information on children and their households at different times in the course of their lives, including the timing and length of their receipt of the CSG. As we describe below, this has proven to be of considerable help in estimating impacts. 5.3.2 Constructing treatment and comparison groups with non-random assignment In the absence of randomisation, we address the problem of selection bias through the use of non-experimental methods. One such potential method is matching, in which we construct a comparison group by ‘matching’ treatment households to comparison group households, based on observable characteristics at the time of their application or enrolment into the programme. The impact of the programme is then estimated as the average difference in the outcomes for each treatment household from a weighted average of outcomes in each similar comparison group household from the matched sample. We note here that the comparison group does not need to be a non-beneficiary of the programme. As in the case of the CSG, where there is variation in the timing and dosage or length of receipt of the intervention, the comparison group can consist of beneficiaries with differing treatment levels or the timing thereof. There are a variety of matching methods that differ in the selection of the matched comparison and in the construction of these weighted average differences in outcomes. One popular approach is propensity score matching (PSM)109. Propensity score matching110 constructs a statistical comparison group by matching observations on beneficiaries to observations on non-beneficiaries (or other beneficiary groups) with similar values of their propensity to receive programme benefits or to receive benefits at a particular time or dosage. Having done so, the difference between mean outcomes for the treatment and comparison groups is calculated and then tested to see whether this difference is statistically significant. This procedure yields an estimate of the average impact of the cash transfer on those that receive it – what is termed in the evaluation literature, the average impact of the treatment on the treated (ATT). The validity of this approach rests in part on two assumptions: ‘conditional mean independence’ – conditional on their observed characteristics, non-participants have the same mean outcomes as participants would have if they did not receive the programme – and that valid matches can be found for all values of the matching variables. If outcomes are independent of programme participation after conditioning on the vector of variables used to construct the propensity scores, then outcomes are independent of programme participation (see Rosenbaum and Rubin, 1983). The version of this estimator based on matching was formalised in Heckman, Ichimura and Todd (1997) and Heckman et al. (1998). Propensity score matching provides reliable, 109. Heckman, Ichimura, & Todd, 1997; Smith & Todd, Reconciling Conflicting Evidence on the Performance of Propensity-Score Matching Methods, 2001; Smith & Todd, Does matching overcome LaLonde’s critique of nonexperimental estimators?, 2005. 110. For a presentation of matching estimators, see Heckman, Ichimura, and Todd (1997), Heckman et al. (1998), and Smith and Todd (2001, 2005) for propensity score matching; see Abadie and Imbens (2006) for covariate and nearest neighbour matching; and see Hirano, Imbens, and Ridder (2003) for regression weighted matching. See Blundell and Costa Dias (2000) for a review of nonexperimental evaluation methods. Frölich (2004) provides a useful comparison of the small sample properties of alternative estimators. Chapter 5 Methods for Assessing the Impact of the Child Support Grant 37 low-bias estimates of programme impact, provided that (i) the same data source is used for participants and non-participants, (ii) the data include meaningful variables capable of identifying programme participation and outcomes, and (iii) participants and non-participants have access to the same markets111. Provided the same survey is administered to treatment and comparison group members, criterion (i) will be satisfied. The baseline surveys should include variables that identify programme participation and outcomes related to child welfare and other outcomes, as required by criterion (ii). Criterion (iii) is satisfied if beneficiary and comparison households are surveyed within the same localities as they are in our survey. However, there will be differences in markets across localities, so it will be helpful to control for these differences by including community level dummy variables and by eliciting some community characteristics from the household survey respondents. Past experience has shown that including a set of community variables of this sort may substantially improve the results of impact estimation based on matching112. Implementing the propensity score matching procedure involves several steps. First, for each outcome, a propensity score is estimated for participation in the programme using a probit or logit model, including both determinants of participation in the programme and factors that affect the outcome. Second, the ‘balancing properties’ of the data are tested by testing whether treatment and comparison observations had the same distribution (mean) of propensity scores and of control variables within groupings (roughly quantiles) of the propensity score. Third, the quality of the match can be improved by ensuring that matches are formed only where the distribution of the density of the propensity scores overlap between treatment and comparison observations, or where the propensity score densities have ‘common 111. Heckman, Ichimura, & Todd, 1997.; Heckman, Ichimura, & Todd, 1998; Heckman J., Ichimura, Smith, & Todd, 1998. 112. de Brauw & Hoddinott, 2011; Gilligan & Hoddinott, Is There Persistence in the Impact of Emergency Food Aid? Evidence on Consumption, Food Security, and Assets in Rural Ethiopia, 2007; Gilligan, Hoddinott, & Taffesse, The Impact of Ethiopia’s Productive Safety Net Programme and its Linkages, 2009. support’113. Common support can be improved by dropping treatment observations whose estimated propensity score is greater than the maximum or less than the minimum of the comparison group propensity scores. Similarly, comparison group observations with a propensity score below the minimum or above the maximum of the treatment observations can be dropped. One shortcoming of this approach is that treatment observations near these cut-points face a potential comparison group with propensity scores that are either all lower or all higher than that of the treatment observation, but this can be taken into account. Finally, if necessary, the 2-5% of treatment observations from the interior of the distribution of the propensity score that have the lowest common support (fewest neighbouring comparison observations) can also be dropped to improve the quality of the match. We illustrate this approach by describing in further detail how we estimate results for Chapter 6, the impact of early CSG enrolment on early life outcomes. As these are attained in the first two years of life, it makes sense to assess the impact in terms of whether or not the child was receiving the CSG earlier or later in life. Accordingly, a child who first received the CSG in the first two years of life is defined as a ‘treated’ child, while children who first received the CSG after age two are ‘comparisons’. In other words, both treatment group and comparison group members receive the CSG at some time in their lives, but the treated children are early recipients. Based on the data collected in the field, we have this information on 1,187 children. Of these, 751 (63.3 per cent) enrolled by the time they turned two while the remaining 436 (36.7 per cent) were enrolled after age two. We include the following covariates that are associated with early enrolment in the CSG and/or the outcomes themselves: child characteristics (sex, relationship to the household head, race), household access to other social grants (whether anyone in the household receives the Old Age Grant or whether another child in the household receives the CSG), maternal characteristics (mother’s age, schooling), paternal characteristics (father’s age known), wealth of the household 113. Heckman, Ichimura, & Todd, 1997; Heckman, Ichimura, & Todd, 1998. 38 The South African Child Support Grant Impact Assessment Table 5.2 Correlates of early CSG enrolment Variables Marginal impact of covariate Standard errors* Child is male -0.049 (0.031) Month of birth 0.005 (0.004) Child is son or daughter of head -0.001 (0.057) Child is grandchild or great-grandchild of head 0.024 (0.060) 0.262*** (0.060) Household has received OAG -0.021 (0.048) Another child in household receives CSG 0.007 (0.033) Mother’s age at birth 0.007*** (0.003) Mother’s age is known -0.069 (0.102) Mother’s years of schooling 0.018*** (0.005) Mother’s education is known 0.255** (0.101) Father’s age is known 0.081** (0.034) Dwelling had tin or tile roof when child born -0.047 (0.071) Dwelling had electricity when child born -0.005 (0.041) 0.137*** (0.048) Time (hours) to travel to nearest SASSA office -0.030 (0.037) Child born in KwaZulu-Natal 0.036 (0.048) Child born in Eastern Cape 0.108** (0.053) Child born in Western Cape 0.132** (0.051) Child born in Limpopo 0.209*** (0.055) Child born in rural locality 0.043 (0.065) Child born in peri-urban locality 0.015 (0.060) Child born in urban locality 0.044 (0.060) Number of years since household experienced death shock 0.003 (0.006) Child is African Mother received CSG application at time of birth *Standard errors are heteroscedasticity robust. at time of birth (dwelling had metal or tile roof, household had electricity), whether the mother was given an application form for the CSG when the child was born, distance to the SASSA office at the time of birth, and location of birth (province, urban or rural locality). Note that we do not observe these variables for all children and so sample size is reduced slightly to 1,010. Table 5.2 shows the marginal effects of these covariates on the likelihood that a child was enrolled in the CSG prior to age two. Two examples illustrate what this means: i) the number ‘0.137’ for ‘Mother received CSG application at time of birth’ means that if the probability that the child was enrolled in the CSG before age two was 13.7 percentage points higher if the mother received an application form for the CSG when the child was born; and ii) the number ‘0.081’ for ‘Mother’s years of schooling’ means that for every additional grade of mother’s schooling, the likelihood that the child was enrolled in the CSG before age two increases by 1.8 percentage points. Both covariates have a statistically significant impact. Figure 5.2 graphs the density functions of the propensity scores by treatment status. Early enrolled children have, on average a higher propensity score, which is what we would expect if the probit has any explanatory power. But importantly, the propensity scores for late and early enrollees show a good deal of overlap. Where this overlap exists, we have two children who have similar propensities (or likelihoods) of being enrolled in the CSG given their observable characteristics, but only one is actually enrolled. Chapter 5 Methods for Assessing the Impact of the Child Support Grant 39 Figure 5.2 Kernel density of propensity scores by treatment status 3 did not present concerns.115 Standard errors of the impact estimates are estimated by bootstrap using 100 replications. For a limited number of outcomes, we increased the number of replications but this did not appreciably alter the results we obtained. 2 5.3.3 Generalised dose response models 1 0 .2 .4 .6 .8 1 Estimated propensity score Late Enrollee Early Enrollee In this example, we match treatment and comparison observations by local linear matching with a tricube kernel using Stata’s psmatch2 command114. Heckman, Ichimura, and Todd (1997) and Smith and Todd (2005) argue in favour of local linear matching over some other matching techniques. Local linear matching performs well in samples with low densities of the propensity score in the interior of the propensity score distribution. In some of the estimates presented here, the number of treatment observations is small, including less than 100. Frölich (2004) provides evidence in support of the finite-sample properties of local linear matching relative to most other matching estimators, with the exception of an infrequently used ridge matching approach. In other analyses in our report (particularly for adolescents), we also use radius matching. Sensitivity analyses comparing the matching estimates with alternative matching strategies 114. Leuven & Sianesi, 2011. A limitation of the approach described above is that it averages across all children within the early and the late enrolment categories. This is informationally inefficient – we are throwing away information on the duration of enrolment (or dosage of treatment). Children first obtained access to the CSG at different ages. While some children were enrolled at birth others did not receive their first CSG until they were eight or even older. For some outcomes, such as those relating to schooling, knowing precisely when the child started to receive the CSG, and thus being able to assess the cumulative impact of the CSG, is of value. Given this, we also use an extension of propensity score matching methods developed by Hirano and Imbens (2004) that allows us to assess the impact of the duration of programme participation on outcomes of interest. They describe this in terms of estimating a ‘dose response function’ where the ‘dose’ here is the number of years a child receives the CSG and the ‘response’ is the impact that that level of transfers has on the outcome of interest. As Hirano and Imbens explain, we cannot simply assess impact through an examination of the relationship between observed duration and outcomes because of the selection bias problem noted above. Because the duration of CSG receipt is not a random 115. For some analyses, we tested the sensitivity of the results by constructing impact estimates using nearest neighbour matching (NNM) (see Abadie and Imbens 2006).The NNM estimator shares many of the properties of the PSM estimator: (i) it relies on the same identifying assumptions ((2)-(3) above), (ii) it matches beneficiaries to one or more non-beneficiaries using pre-programme characteristics, and (iii) it estimates the average impact as the average of the difference in the outcome for each beneficiary from a weighted average of outcomes for matched non-beneficiaries. The differences between NNM and PSM derive primarily from the rule used to select comparable non-beneficiaries, and the weights used to construct the difference in weighted average outcomes. NNM, a form of ‘covariate matching,’ matches beneficiaries to non-beneficiaries based directly on the observable characteristics. Each beneficiary is matched to the group of non-beneficiaries with the smallest average difference in pre-programme characteristics, where this difference is determined using a multi-dimensional metric across all control variables. The South African Child Support Grant Impact Assessment variable, failing to control for factors that affect both the level of transfers that are received and outcomes of interest lead to bias in this estimated relationship. Hirano and Imbens (2004) show how, under certain conditions, an extension of the estimation of the propensity score eliminates the bias in this relationship. In the appendix to this chapter, we describe the technical details associated with this method. Here we provide an example of how to interpret the results of estimating the dose response relationship. Figure 5.3 shows the results of estimating the impact of duration of CSG on grade attainment of girls. The horizontal axis denotes different numbers of years that the child receives the CSG and the vertical axis predicted grade attainment. Starting at age zero (birth), these attainments are calculated for each age at which the child first received the CSG. That is, we calculate the predicted impact of receiving CSG since birth at age one, at age two, and so on. The pink line in Figure 5.3 shows the ‘dose response’; it traces out predicted grade attainment given differing durations of CSG participation. Note, as is the case here, the relationship between duration and grade attainment does not have to be strictly linear; rather the Hirano–Imbens method allows the data to trace out the form of the relationship. Figure 5.3 Dose-response graph of impact of age at first receipt of CSG on grade attainment, girls 4.4 Predicted grade attainment 40 4.2 4 3.8 3.6 Confidence bounds at .95 % level 0 2 4 6 Age at first receipt of CSG 8 Dose response Upper bound Low bound Figure 5.3 shows that a child who was enrolled at birth is predicted to attain just over 4.2 grades of schooling while children enrolled at age six, just prior to the age at which children ordinarily enrol, attain four grades of schooling. An attractive feature of this method is that we can calculate standard errors for these predicted impacts; these are the grey and red curves in Figure 5.3 and show the upper and lower bounds of these predicted effects. These results can also be presented in tabular form, as in Table 5.3 that lists age at first receipt of CSG, the predicted impact at that age and the t-statistic (obtained by dividing the predicted impact by its standard error). Chapter 5 Methods for Assessing the Impact of the Child Support Grant 41 Age at first receipt of CSG Predicted outcome Standard error T-statistic Statistical significance 0 4.27 0.05 79.42 *** 1 4.21 0.04 108.23 *** 2 4.15 0.05 76.58 *** 3 4.10 0.07 57.57 *** 4 4.06 0.08 53.81 *** 5 4.04 0.07 58.09 *** 6 4.02 0.08 50.35 *** 7 3.99 0.11 35.83 *** -0.251 0.10 -2.56 ** Difference between predicted impacts at receipt at age zero and six Source: Calculated from household survey. *, significant at the 10% level; **, significant at the 5% level; *** significant at the 1% level. Sample size is 529. We can use results reported in tables like Table 5.3 to assess the change in impact between receipt of CSG from, say, age zero (birth) and age six. In Table 5.3, this difference is (4.27) – (4.02), which equals 0.25 grades. This shows that a child who received CSG payments since birth is predicted to attain a quarter grade more than a child who was enrolled at age six. Because we calculate the standard errors of these impact estimates, we are able to test the null hypothesis that the predicted grade attainments – in this case, receiving CSG at birth rather than age six – are equal. Where they are unequal, we will reject this null hypothesis. 5.3.4 Recovering double-difference estimates from a single cross-sectional survey One issue remains; we have only a single cross-sectional survey to work with and ideally we should be estimating models of changes over time. But many of our outcomes can be interpreted as cumulative changes since birth. For example, when we talk about grade attainment, we can also describe this as the change in the number of grades attained between age zero (birth) and age 10 (when survey was completed). Figure 5.4, an adaptation of Figure 5.1, illustrates this idea. Figure 5.4 Recovering double difference estimates from a single cross-section A B Grade attainment Table 5.3 Dose-response estimates of impact on grade attainment, girls C Birth Time In Figure 5.4, the star shows grade attainment for children at birth. For all children this is equal to zero. (We put grade attainment of zero at a point above the origin simply to make the graph easier to read). Children who, say, enrol at birth attain ‘A’ grades of schooling by age 10 (the triangle) while children enrolled at say age six attain ‘B’ grades of schooling by this age. A single difference impact of early (at birth) versus late (age six) enrolment equals A – B. But given that all children begin with zero grades of schooling, this is equivalent to the double difference estimate, AC – BC. In this way – for these outcomes – we recover our doubledifference estimator. In doing so, it is helpful to account for changes (‘shocks’) that affect children between birth and age 10. We do so by including covariates that capture such events in our calculation of the propensity score. 42 The South African Child Support Grant Impact Assessment © UNICEF/ L Marinovich Part 4 Impacts of the CSG on Young Children Chapter 6 The Impact of the CSG on Early Life Outcomes 43 Chapter 6 The Impact of the CSG on Early Life Outcomes 6.1 Introduction This chapter assesses the impact of the Child Support Grant on early life experiences and outcomes of children 10 years old at the time of the survey. Three domains are considered: access and use of preventative health and nutrition care; anthropometry; and access to pre-schools. We examine these in turn. For each domain, we begin by describing how the data are constructed and then turn to our basic impact estimates. We then disaggregate these estimates by characteristics of the child (boys/girls) and the mother (by level of education) before summarising our results. 6.2 Access and use of preventative health and nutrition care The Young Child Questionnaire had a section where respondents were requested to show the enumerator the Road to Health Card for the child. These cards were intended to be the source of information on the child’s access to preventative health and nutrition care in early life. Specifically, these cards were supposed to be the source of information on weight monitoring and vaccination histories. However, this proved to be a much more challenging exercise than had been envisaged at the time of the survey design. There were numerous cases where because of their poor condition, dates and measures were completely illegible. There were also numerous instances of cards having only weight information but no vaccination histories or vice versa, which was confirmed by follow up calls to these respondents116. Just under 60 per cent of children still had these cards. Are children whose parents still possess these cards different from those who never had them or who have lost them? If systematic differences exist, we will need to be cautious in interpreting our impact estimates. With this in mind, Table 6.1 shows the percentage of children who had Road to Health Cards, disaggregating this information by child sex and education level of the mother. There is no difference between boys and girls, but children whose mothers had 116. RDC, 2012. eight or more grades of schooling were less likely to have a Health Card. Table 6.1 Possession of Road to Health Card Children with Road to Health Card (%) Number of observations Full sample 58.4 716 Girls 58.1 343 Boys 58.6 373 Mother has <8 grades schooling 63.0 201 Mother has 8+ grades schooling 56.0 515 Source: Young Child Questionnaire To examine a broader set of characteristics, we ran a probit regression to assess whether there are child, parental, household or locational characteristics that are associated with possession of a Health Card. This is shown in Table 6.2. The coefficients have been transformed into marginal effects so that, for example, the number -0.158 means that a child born in KwaZulu-Natal is 15.8 percentage points less likely to have a Health Card, all other characteristics held constant. Robust standard errors have been computed. The striking feature of Table 6.2 is the near absence of statistically significant associations across a wide number of variables. This gives us some confidence that possession of a Health Card, and the information contained within, is not systematically related to characteristics that might also affect the outcomes we are interested in. However, there may be other factors that we cannot observe which have affected the likelihood of retaining this card. Also, because we have a smaller sample, we have less statistical power to detect impacts. Growth monitoring is the regular recording of a child’s weight in the first two years of life by a health care professional. It is seen as a means of detecting whether a child is undernourished or at risk of becoming malnourished. Table 44 The South African Child Support Grant Impact Assessment Table 6.2 Correlates of possession of Road to Health Card Variables Marginal Effects Standard Errors Age at which CSG was first received -0.005 (0.006) Child is a boy 0.003 (0.030) Month of birth 0.003 (0.004) Child is son or daughter of head 0.071 (0.052) Child is grandchild of head 0.095* (0.055) Child is African -0.046 (0.054) Child has not moved since birth 0.032 (0.044) Mother’s age at birth 0.004* (0.002) Mother’s age is known -0.050 (0.098) Mother’s grade of schooling -0.004 (0.005) Mother’s schooling is known 0.134 (0.097) Father’s age is known -0.053 (0.033) Household also receives OAG -0.063 (0.046) Another child receives CSG 0.045 (0.031) At birth, child’s house had tile roof -0.008 (0.066) At birth, child’s house had electricity -0.033 (0.036) Mother received CSG application when child born -0.014 (0.056) Child characteristics Parental and household characteristics Location Child born in KwaZulu-Natal -0.158*** (0.042) Child born in Eastern Cape -0.151*** (0.051) Child born in Western Cape -0.080 (0.052) Child born in Limpopo 0.007 (0.058) Travel time to urban centre at time child born -0.012 (0.025) Death of a family member 0.013 (0.045) Death of a friend that provided economic support -0.116 (0.073) 0.168** (0.082) Job loss 0.048 (0.085) Loss of remittances -0.047 (0.146) Family conflict 0.119 (0.121) Divorce -0.011 (0.106) Early life shocks Illness Notes: Heteroscedastic robust standard errors in parentheses. *, significant at the 10 per cent level; **, significant at the five per cent level; ***, significant at the 1 per cent level. A shock is an event that led to a serious reduction in your asset holdings, caused your household income to fall substantially or resulted in a significant reduction in consumption. Sample size is 1 141. 6.3 provides descriptive statistics on growth monitoring, whether the child was weighed and the number of weighings. Conditional on having a Health Card, 70.2 per cent of children in our sample had their weight measured at least once. On average, they were weighted 5.4 times. There are negligible differences between boys and girls and between mothers with different levels of schooling. Table 6.3 Descriptive statistics for growth monitoring Mean Child’s weight measured Standard Deviation Sample Size Number Mean of weighings Standard Deviation Sample Size All Girls Boys Mother Mother has <8 has 8+ grades grades schooling schooling 0.702 0.706 0.700 0.672 0.714 0.46 0.46 0.46 0.47 0.45 716 343 373 201 515 5.4 5.6 5.3 5.6 5.3 5.6 5.7 5.6 6.3 5.3 716 343 373 201 515 Source: Young Child Questionnaire As these outcomes are attained in the first two years of life, it makes sense to assess the impact in terms of whether or not the child received the CSG earlier or later in life. Accordingly, a child who first received the CSG in the first two years of life is defined as a ‘treated’ child, while children who first received the CSG after age two are ‘controls’. We use Propensity Score Matching to assess impact.117 Results are given in Table 6.4. 117. See Chapter 5 for a description of the matching estimator used here. As Nearest Neighbour Matching produced similar results, they are not reported here. Chapter 6 The Impact of the CSG on Early Life Outcomes 45 Table 6.4 PSM estimates of the impact of early receipt of CSG on growth monitoring Outcome Number of observations Mean values for Impact T-stat Early (treatment) Late (control) Treatments Controls Was weight measured 414 166 0.734 0.657 0.077 1.69* Number measures 414 166 5.69 5.09 0.60 1.07 Was weight measured 205 71 0.73 0.65 0.08 0.79 Number measures 205 71 5.7 4.6 1.1 0.87 Was weight measured 207 87 0.74 0.73 0.01 0.18 Number measures 207 87 5.7 5.0 0.7 0.98 Was weight measured 109 47 0.75 0.73 0.02 0.12 Number measures 109 47 6.6 5.7 0.9 0.64 Was weight measured 298 106 0.72 0.66 0.06 1.01 Number measures 298 106 5.3 4.6 0.7 0.69 All observations Girls Boys Mother has <8 grades schooling Mother has 8+ grades schooling Notes: *, significant at the 10 per cent level; **, significant at the five per cent level; ***, significant at the 1 per cent level. Table 6.4 provides weak evidence that receipt of the CSG led to increased use of growth monitoring services. Across the full sample, children receiving the CSG prior to age two were 7.7 percentage points more likely to have been weighed once. This impact is statistically significant at the 10 per cent level. Impacts appear to be larger for girls than for boys and for children whose mothers have eight or more grades of schooling, but these are imprecisely measured, perhaps because these disaggregated samples are small. We now consider whether the likelihood of vaccination is higher if the child receives the CSG. Our rationale for doing so follows from the following observation: Immunisation coverage is a good indicator of children’s access to preventive programmes. Given that the immunisation schedule is used to tag on a number of other preventive programmes and interventions, for example the developmental screening and vitamin A supplementation, immunisation coverage serves as a good proxy indicator of children’s access to other preventive programmes118. Table 6.5 provides PSM impact estimates of early receipt of the CSG on the likelihood that a child receives vaccines for polio, diphtheria, pertussis and tetanus (DPT), hepatitis and measles. By age 24 months, children should have received five polio vaccinations, three for DPT and hepatitis and one against measles. 118. OrcMacro, 2007. 46 The South African Child Support Grant Impact Assessment The mean values for immunisations shown in Table 6.5 are broader consistent with those found in the most recent South African Demographic Health Survey119. There is no evidence of any impact of the CSG on the likelihood that a child receives any of these vaccines. We experimented with various sample disaggregations and alternative covariate sets for the matching estimator but none of these yielded evidence of statistically significant impacts and for this reason are not reported here. Table 6.5 PSM estimates of the impact of vaccination Vaccine Number of observations Mean values for Impact T-stat Early (treatment) Late (control) Treatments Controls Polio 0 465 201 0.815 0.798 0.017 0.48 Polio 1 465 201 0.843 0.839 0.004 0.13 Polio 2 465 201 0.824 0.806 0.018 0.48 Polio 3 465 201 0.783 0.793 -0.010 -0.28 Number immunisations 465 201 3.265 3.236 0.029 0.23 Proportion of children receiving all immunisations 465 201 0.720 0.718 0.003 0.06 DPT 1 465 201 0.822 0.833 -0.011 -0.32 DPT 2 465 201 0.798 0.787 0.011 0.29 DPT 3 465 201 0.763 0.777 -0.014 -0.38 Number immunisations 465 201 2.383 2.397 -0.014 -0.14 Proportion of children receiving all immunisations 465 201 0.742 0.754 -0.012 -0.31 Hep 1 465 201 0.746 0.733 0.013 0.31 Hep 2 465 201 0.723 0.691 0.031 0.76 Hep 3 465 201 0.695 0.689 0.006 0.15 Number immunisations 465 201 2.163 2.113 0.050 0.43 Proportion of children receiving all immunisations 465 201 0.656 0.650 0.006 0.14 465 201 0.748 0.756 -0.007 -0.20 Polio Diphtheria, pertussis and tetanus Hepatitis Measles Measles Notes: *, significant at the 10 per cent level; **, significant at the five per cent level; ***, significant at the 1 per cent level. 119. OrcMacro, 2007. Chapter 6 The Impact of the CSG on Early Life Outcomes 47 Table 6.6 Impact of early receipt of CSG on anthropometry Outcome Number of observations Mean values for Impact T-stat Early (treatment) Late (control Treatments Controls Stunting 665 277 0.068 0.075 -0.007 -0.40 Height for age z score 665 277 -0.683 -0.755 0.072 1.11 Stunting 330 130 0.048 0.061 -0.012 -0.49 Height for age z score 330 130 -0.534 -0.728 0.194 1.84* Stunting 336 145 0.086 0.093 -0.007 -0.25 Height for age z score 336 145 -0.819 -0.793 -0.026 -0.28 Stunting 163 70 0.092 0.062 0.030 0.88 Height for age z score 163 70 -0.841 -0.771 -0.071 -0.48 Stunting 502 190 0.060 0.084 -0.025 -1.00 Height for age z score 502 190 -0.631 -0.815 0.184 2.56** All observations Girls Boys Mothers with <8 grades of schooling Mother has 8+ grades schooling 6.3 Impact on anthropometry The 2003 South African DHS estimates that 28 per cent of South African children are stunted120, meaning their height given their age and sex is more than two standard deviations below that of the international reference standard121. Lost linear growth in early life typically is not fully regained122. Chronic undernutrition in early life also has extensive adverse neurological consequences123. For this reason, it is of interest to see whether access to the CSG in early life increases the height of beneficiary children. 120. OrcMacro, 2007. 121. Black, Allen, & Bhutta, 2008. 122. Martorell, 1999. 123. See Hoddinott, et al., 2011 for references. At first, this might seem an impossible question to answer. Road to Health cards do not record height so we do not know the heights of the young children in our sample when they were pre-schoolers. However, we do know their current heights because these were measured as part of the survey. We take these measures and convert them to height-for-age z scores (HAZ). Conditional on age and sex, a child with a HAZ of zero has the same height as the median child found in a reference population of well-nourished children. It is well established that between age three and puberty, HAZ scores are remarkably stable. For example, Hoddinott et al (2011) find that in their sample of Guatemalan children, the correlation between HAZ at age three and seven is 0.95. Consequently, we can take the HAZ scores of the children in our sample as a good proxy for their HAZ at age three. 48 The South African Child Support Grant Impact Assessment With this in mind, Table 6.6 shows the PSM estimates of the impact on stunting and HAZ. Table 6.6 shows that there is no impact of early receipt of the CSG on stunting or on HAZ z scores for the full sample. However, it improves anthropometric measures for two sub-samples, girls and children whose mothers have eight or more grades of schooling. These effect sizes, approximately 0.19 standard deviations are large. The disaggregated results by maternal education are consistent with what is found in studies of other cash transfer programmes; see for example Behrman and Hoddinott (2005). One explanation is that the production of child health in early life requires the use of complementary inputs: resources such as food and sanitation as well as the knowledge of how to use these so as to ensure that children grow at a healthy rate. This argument implies that solely providing more resources in the absence of these complementary inputs is not sufficient to improve anthropometric status but when these resources are in place, cash transfers can have a positive impact. The majority of children in this sample, 59.9 per cent, attended some sort of pre-school or crèche. Virtually all of these (i.e. more than 95 per cent) had toys, a toilet and provided a meal. Girls were slightly more likely to attend a crèche than boys, 61.9 versus 58.1 per cent but, as Figure 6.1 shows, boys tended to start attendance at a slightly earlier age. The modal age for first attendance is three years of age and few children start after age four. Figure 6.1 Distribution of age first attended crèche, by sex 120 6 100 80 1 10 1 1 11 11 27 25 26 34 40 20 18 2 8 29 31 23 21 1 8 0 3 Girls 8 Boys 3 All 6.4 Impact on access to pre-schools A recent systematic review124 found that children who attended some type of pre-school or crèche generally scored better on tests of literacy, vocabulary, mathematics, and quantitative reasoning. While the effects of non-formal preschools on child outcomes were typically weaker than the effects of formal pre-schools, some non-formal pre-school programs also produced better early child development outcomes. Children from disadvantaged backgrounds tended to benefit more from pre-schools. There is some evidence that these benefits persist through primary school and into adolescence. Given this evidence, it is of interest to see if the CSG affected the likelihood or duration of attendance at pre-school. 124. Engle, et al., 2011. 4 3 60 0 5 Source: Young Child Questionnaire Table 6.7 shows the impact of receipt of the CSG before the child was two, on the likelihood that the child attended a crèche or nursery school, the age at which they first attended a crèche or nursery school and the duration of attendance. Just over 60 per cent of treatment children attended a preschool facility compared to 64.6 per cent of matched control children. Early receipt of the CSG reduces the likelihood that a child attends a pre-school facility but this impact is not statistically significant. Conditional on attending a preschool, early receipt leads to first attendance at a slightly early age and longer duration of attendance but the magnitudes of these impacts are small and not statistically significant. 2 Chapter 6 The Impact of the CSG on Early Life Outcomes 49 Table 6.7 Impact of early receipt of CSG on attendance at crèche or nursery schools, full sample Outcome Number of observations Mean values for Impact T-stat Early (treatment) Late (control) Treatments Controls Attended crèche or nursery school (=1 if yes) 691 288 0.603 0.646 -0.042 -1.36 Age start attending crèche (yrs) 409 173 3.025 3.120 -0.095 -0.88 Duration of attendance at crèche (yrs) 403 170 2.237 2.079 0.158 1.44 Table 6.8 Impact of early receipt of CSG on attendance at crèche or nursery schools, disaggregated by sex and education level of mother Outcome Number of observations Mean values for Impact T-stat Early (treatment) Late (control) Treatments Controls Attended crèche or nursery school 339 140 0.631 0.660 -0.029 -0.51 Age start attending 209 76 3.118 3.230 -0.112 -0.44 Duration of attendance 206 68 2.173 1.773 0.400 1.69* Attended crèche or nursery school 353 159 0.572 0.629 -0.056 -0.99 Age start attending 200 85 2.950 3.171 -0.221 -1.43 Duration of attendance 197 91 2.292 2.201 0.091 0.42 Girls Boys Mothers with <8 grades of schooling Attended crèche or nursery school 174 73 0.494 0.432 0.062 0.73 Age start attending 83 31 3.174 3.172 0.002 0.01 Duration of attendance 82 30 2.307 2.493 -0.186 -0.33 Attended crèche or nursery school 518 209 0.633 0.702 -0.069 -1.84* Age start attending 323 136 2.982 3.009 -0.027 -0.20 Duration of attendance 318 129 2.214 2.107 0.106 0.73 Mother has 8+ grades schooling 50 The South African Child Support Grant Impact Assessment Table 6.8 disaggregates impacts by sex of child and education levels of the mother. There is a large difference between the likelihood that a child will attend a pre-school if her mother has more than seven grades of schooling (63 and 70 per cent for matched treatment and control children) compared to children whose mothers have less than eight grades of schooling (49 and 43 per cent for matched treatment and control children). Early receipt closes this large gap by six percentage points but the impact is not statistically significant. More generally, there is no systematic pattern to the impacts we observe, the vast majority of which are not statistically significant even at the 10 per cent level. 6.5 Summary This chapter has assessed the impact of the Child Support Grant on early life experiences and outcomes of children 10 years old at the time of the survey. Three domains are considered: access and use of preventative health and nutrition care; anthropometry; and access to pre-schools. We use propensity score matching to assess impact, dividing our sample into two groups: children who received the CSG in the first two years of life; and children who only first received the CSG when they were two years of age or older. We find the following: Early life receipt of the CSG increases the likelihood that a child’s growth is monitored by 7.7 percentage points. However, this impact is imprecisely measured, statistically significant at the 10 per cent level. There is no impact of early receipt of the CSG on the likelihood that a child is immunised. For the full sample, there is no evidence that early receipt of the CSG improves a child’s height given age. However, for children whose mothers have more than eight grades of schooling, early receipt has a large, positive impact, increasing HAZ by 0.19 standard deviations. This impact is statistically significant at the five per cent level. It is important given that early life nutritional status has long term effects on cognitive development. However, no impact is found on children whose mothers have less than eight grades of schooling and these children have, on average, lower HAZ scores. For the full sample, there is no evidence that early receipt of the CSG affects attendance at crèches or nursery schools. There is some evidence that it increases the duration of attendance by girls and reduces it for children whose mothers have eight or more grades of schooling but these impacts are not precisely measured. Chapter 7 The Impact of the CSG on Schooling 51 Chapter 7 The Impact of the CSG on Schooling 7.1 Introduction This chapter assesses the impact of the Child Support Grant on schooling outcomes of children 10 years old at the time of the survey. Three domains are considered: grade attainment; scores on tests of mathematical ability; and scores on tests of reading and vocabulary. For each domain, we begin by describing how the data are constructed and then turn to our basic impact estimates. We then disaggregate these estimates by characteristics of the child (boys/girls) and the mother (by level of education) before summarising our results. 7.2 Schooling As part of the survey instrument, the child’s principal caregiver was asked to provide their schooling history. Specifically, they were asked to complete a timeline, starting with Grade 7/Standard 5 and working backwards in time through to attendance at Reception.125 For children no longer in school, we calculate the highest grade of schooling attained; for children still in school, the highest grade attained is the current grade they are enrolled in. Across all children, mean grade attainment is 4.02 grades. The mean is slightly higher for girls (4.15) than for boys (3.89). Table 7.1 shows the distribution of grade attainment by sex. In this sample, the average child starts Grade 1 at age 5.8 years of age. There is no difference in age starting Grade 1 by whether or not a child had attended a crèche. Girls start slightly younger than boys, at age 5.73 years versus 5.86 years. To assess the extent to which children’s entry into primary school is delayed, we begin by noting that the South African school year begins in January. Children should enter Grade 1 at age five, turning six by 30 June in the year of admission and that enrolment can be delayed if a child is not considered ready to start school.126 This implies that a child enrolling on 125. During pilot testing, we experimented with working forwards from Reception and working backwards from Grade 7; respondents found it easier to recall the data if they worked backwards. In 97.5 per cent of cases, the respondent to these questions was the principle caregiver. 126. See http://www.southafrica.info/services/education/edufacts.htm. time enters school at age five if she is born between January and June and enters school at age six if she is born between July and December. Table 7.2 shows the distribution of children entering early, on time and late by sex. Table 7.1 Highest grade attained by sex Highest Grade Attained Girls Boys All children 1 0.2% 0.5% 0.3% 2 2.2 4.5 3.4 3 14.5 23.0 18.9 4 50.5 49.7 50.1 5 30.3 21.1 25.5 6 2.3 0.9 1.6 Not stated 0.0 0.3 0.2 Source: Household survey Table 7.2 Distribution of entry into Grade 1 by sex Starts Grade 1 Girls Boys All children Early 11.8% 8.5% 10.1% On time 53.2 53.9 53.5 Late 35.0 37.6 36.4 Source: Household survey Children who start late have attained fewer grades of schooling compared to children who started early or on time, see Table 7.3. 52 The South African Child Support Grant Impact Assessment Table 7.3 Highest grade attained by entry into Grade 1 Starts Grade 1 Next, we disaggregate the data by the child’s age when the caregiver first received CSG payments for that child. Results are reported in Table 7.5a. Highest grade attained Early On time Late 1 0.0% 0.0% 0.9% 2 0.8 2.5 5.0 3 10.7 13.1 30.2 4 32.0 51.3 53.8 5 49.1 32.0 9.3 6 7.4 1.1 0.7 Child age at first receipt of CSG (years) Mean highest grade attained 4.52 4.16 3.68 Number of observations 122 647 440 Source: Household survey Even in these early years of schooling, grade repetition occurs. Across grades one through four, 21.5 per cent of children have repeated one grade, and another 5.8 per cent have repeated two or more grades. There is a pronounced gender difference with 32.6 per cent of boys repeating at least one grade compared to 20.9 per cent of girls. Mean grade repetition is 0.26 grades for girls and 0.43 grades for boys. Conditional on age at enrolment in Grade 1, boys are always most likely to repeat at any grade level (see Table 7.4). This higher amount of grade repetition for boys (0.17 grades) accounts for much of the gap in grade attainment (0.26 grades) between boys and girls. Table 7.4 Percentage of children repeating a grade by grade, sex and age of entry into Grade 1 Girls Boys Table 7.5a Schooling outcomes by child age at first receipt of CSG, all children Schooling outcomes Highest grade attained Age started school (mean) Number of grades repeated Proportion of children <1 4.13 5.75 0.31 0.26 1 4.04 5.77 0.35 0.26 2 3.92 5.80 0.41 0.32 3, 4 4.00 5.80 0.40 0.29 5, 6 3.95 5.84 0.40 0.29 7, 8, 9, 10 3.91 5.94 0.28 0.21 Total 4.02 5.80 0.35 0.27 Source: Household survey Table 7.5b Schooling outcomes by child age at first receipt of CSG, girls Schooling outcomes Child age at first receipt of CSG (years) Highest grade attained (mean) Age started school (mean) Number of grades repeated (mean) Proportion of children repeating a grade <1 4.26 5.65 0.26 0.21 1 4.18 5.69 0.23 0.19 2 4.06 5.80 0.22 0.18 Grade Early On time Late Early On time Late 3, 4 4.28 5.63 0.28 0.23 1 18.8% 8.7% 9.3% 24.5% 14.5% 14.9% 5, 6 4.05 5.83 0.40 0.29 2 7.3 7.7 7.8 26.4 11.3 10.2 7, 8, 9, 10 3.92 5.93 0.26 0.20 3 2.9 5.8 2.9 20.7 9.2 4.7 Total 4.14 5.74 0.27 0.22 4 2.9 1.6 1.5 5.7 2.4 1.7 Source: Household survey Chapter 7 The Impact of the CSG on Schooling 53 Schooling outcomes Child age at first receipt of CSG (years) Highest grade attained (mean) Age started school (mean) Number of grades repeated (mean) Proportion of children repeating a grade <1 4.01 5.86 0.37 0.31 1 3.91 5.85 0.47 0.33 2 3.80 5.80 0.58 0.45 3, 4 3.81 5.92 0.48 0.38 5, 6 3.87 5.85 0.41 0.30 7, 8, 9, 10 3.91 5.94 0.31 0.22 Total 3.90 5.86 0.43 0.33 Source: Household survey There is a correlation between earlier receipt of the CSG and schooling attainments. Children in households where the grant was received when the child was very young appear to start school at a slightly younger age. There is no obvious difference in grade repetition when tabulated against duration of receipt of the CSG. The differences in schooling outcomes are slightly more pronounced when we restrict attention to girls (Table 7.5b). But Table 7.5c does not show impact, just a suggestive correlation. Our dose-response model described in Chapter 5, however, does provide an estimate of impact. Table 7.6 shows the number of children in our sample by the age at which they first received the CSG. Note that we have few observations of children who first received the CSG after age seven, so few in fact that we cannot plausibly estimate predicted impacts for those children. Given this, we estimate the dose-response model for children first receiving the CSG at zero to seven years of age. Figure 7.1 graphs the dose-response function and Table 7.7 shows the dose-response estimates of the impact of age of receipt of CSG on grade attainment. Table 7.6 Distribution of children by age at first receipt of CSG Age at first receipt of CSG 0 Number of children Per cent Number of girls Number of boys 281 23.7 138 143 1 362 30.5 180 182 2 108 9.1 52 56 3 63 5.3 24 39 4 44 3.7 19 25 5 97 8.2 51 46 6 82 6.9 34 48 7 76 6.4 39 37 8 43 3.6 20 23 9 26 2.2 15 11 10 5 0.4 3 2 1,187 100.0 575 612 Total Source: Household survey Figure 7.1 Dose-response graph of impact of age at first receipt of CSG on grade attainment 4.2 Predicted grade attainment Table 7.5c Schooling outcomes by child age at first receipt of CSG, boys Confidence bounds at .95 % level 4.1 4 3.9 3.8 0 2 Dose response Upper bound Low bound 4 6 Age at first receipt of CSG 8 The South African Child Support Grant Impact Assessment Table 7.7 Dose-response estimates of impact on grade attainment Age at first receipt of CSG Predicted outcome Standard error T-statistic Statistical significance 0 4.12 0.04 94.41 *** 1 4.07 0.03 132.21 *** 2 3.99 0.04 98.05 *** 3 3.92 0.06 69.34 *** 4 3.90 0.06 63.93 *** 5 3.92 0.05 76.73 *** 6 3.98 0.05 73.74 *** 7 4.03 0.08 49.78 *** Difference between predicted impacts at receipt at age zero and six -0.14 0.06 -2.33 ** whether this is true for grade attainment. Table 7.10 considers children whose mothers have less than eight grades of schooling. In this sub-sample, early enrolment in the CSG raises grade attainment by 0.38 grades, a 10.2 per cent increase. By contrast, Table 7.11 shows that the CSG has no impact on grade attainment on children whose mothers have eight or more grades of schooling. These results suggest that the CSG is playing a compensatory role in narrowing the gap between children whose mothers have not completed primary school and mothers with at least some secondary education. This can also be seen by comparing the predicted outcomes across Tables 7.10 and 7.11. For children enrolled at age six, the difference between predicted grade attainments is 0.41 (3.70 versus 4.11), a difference of nearly half a grade. But for children enrolled at birth, the difference is negligible, 0.06 grades (4.08 versus 4.14). Figure 7.2 Dose-response graph of impact of age at first receipt of CSG on grade attainment, girls Source: Calculated from household survey. *, significant at the 10% level; **, significant at the 5% level; *** significant at the 1% level. Sample size is 1,091. Figure 7.1 and Table 7.7 provide a core finding of this report. Children who were enrolled in the CSG at birth completed 0.14 more grades of schooling than children who were enrolled at age six. This impact is statistically significant. Next, we disaggregate our sample by sex. Figure 7.2 and Table 7.8 shows the impact of the CSG on girls completed schooling while Figure 7.3 and Table 7.9 show the impact on boys. This yields the second major finding in this chapter. When we disaggregate by sex, we find that earlier enrolment in the CSG improved girls’ grade attainment by one quarter of a grade compared to enrolment in the CSG at age six. This is a large impact, given that most children in the sample had only completed four grades of schooling. By contrast, early CSG enrolment had no impact on boys’ grade attainment. In Chapter 6, we noted that for some outcomes, the impact of the CSG differed by maternal education. Here we assess 4.4 Predicted grade attainment 54 Confidence bounds at .95 % level 4.2 4 3.8 3.6 0 2 Dose response Upper bound Low bound 4 6 Age at first receipt of CSG 8 Chapter 7 The Impact of the CSG on Schooling 55 Table 7.8 Dose-response estimates of impact on grade attainment, girls Age at first receipt of CSG Predicted outcome Standard error T-statistic Statistical significance 0 4.27 0.05 79.42 *** 1 4.21 0.04 108.23 2 4.15 0.05 3 4.10 4 Table 7.9 Dose-response estimates of impact on grade attainment, boys Predicted outcome Standard error T-statistic Statistical significance 0 3.99 0.06 70.32 *** *** 1 3.91 0.04 89.89 *** 76.58 *** 2 3.83 0.05 77.16 *** 0.07 57.57 *** 3 3.78 0.07 54.24 *** 4.06 0.08 53.81 *** 4 3.80 0.08 50.16 *** 5 4.04 0.07 58.09 *** 5 3.87 0.06 62.58 *** 6 4.02 0.08 50.35 *** 6 3.95 0.07 53.44 *** 7 3.99 0.11 35.83 *** 7 4.01 0.12 33.33 *** -0.251 0.10 -2.56 ** Difference between predicted impacts at receipt at age zero and six Difference between predicted impacts at receipt at age zero and six Age at first receipt of CSG -0.047 Source: Calculated from household survey. *, significant at the 10% level; **, significant at the 5% level; *** significant at the 1% level. Sample size is 529. Source: Calculated from household survey. *, significant at the 10% level; **, significant at the 5% level; *** significant at the 1% level. Sample size is 562. Figure 7.3 Dose-response graph of impact of age at first receipt of CSG on grade attainment, boys Table 7.10 Dose-response estimates of impact on grade attainment, children whose mothers have <8 grades schooling Confidence bounds at .95 % level Predicted grade attainment 4.2 Age at first receipt of CSG 4 3.8 3.6 0 2 4 Age at first receipt of CSG Dose response Upper bound Low bound 6 8 Predicted outcome Standard error T-statistic Statistical significance 0 4.08 0.08 48.67 *** 1 3.85 0.07 55.67 *** 2 3.67 0.07 49.41 *** 3 3.60 0.09 38.89 *** 4 3.61 0.10 35.06 *** 5 3.66 0.09 39.04 *** 6 3.70 0.10 35.91 *** 7 3.70 0.15 24.28 *** -0.376 0.12 -3.16 Difference between predicted impacts at receipt at age zero and six Source: Calculated from household survey. *, significant at the 10% level; **, significant at the 5% level; *** significant at the 1% level. Sample size is 286. 56 The South African Child Support Grant Impact Assessment Why does earlier receipt of the CSG affect grade attainment? We can think of three channels: (a) by affecting the timing at which the child starts school; (b) by affecting the likelihood of grade repetition once they are in school; and (c) a combination of (a) and (b). Table 7.12 examines how the likelihood of delayed entry is affected by early receipt of the CSG. Approximately 38 per cent of children enrolled in birth are predicted to be enrolled late, as are 41 per cent of children enrolled at age six. This difference is not statistically significant. However, when we disaggregate we find statistically significant impacts for two sub-groups, girls and children whose mothers have less than eight grades of schooling. Early receipt of the CSG reduces the likelihood of delayed entry by 12.5 and 14.8 percentage points respectively. Put another way, early receipt of the CSG reduces delayed school entry of girls by 26.5 per cent and by 31.8 per cent for children whose mothers have less than eight grades of schooling. By contrast, once children are enrolled in school, there is no additional impact of early CSG receipt on school progression. While Table 7.13 shows that girls and children whose mothers have little education are less likely to repeat when they were enrolled in the CSG at birth, the differences are not statistically significant. Table 7.11 Dose-response estimates of impact on grade attainment, children whose mothers have 8+ grades schooling Age at first receipt of CSG Predicted outcome Standard error T-statistic Statistical significance 0 4.14 0.06 70.87 *** 1 4.14 0.04 112.58 *** 2 4.10 0.05 78.56 *** 3 4.05 0.07 60.43 *** 4 4.03 0.07 57.96 *** 5 4.05 0.06 70.32 *** 6 4.11 0.06 63.83 *** 7 4.16 0.10 41.30 *** Difference between predicted impacts at receipt at age zero and six -0.032 Source: Calculated from household survey. *, significant at the 10% level; **, significant at the 5% level; *** significant at the 1% level. Sample size is 805. Chapter 7 The Impact of the CSG on Schooling 57 Table 7.12 Dose-response estimates of impact on probability of delayed enrolment by child sex and maternal education All children Girls Boys Predicted outcome Standard error Statistical significance Predicted outcome Standard error Statistical significance Predicted outcome Standard error Statistical significance 0 0.378 0.029 *** 0.347 0.041 *** 0.397 0.033 *** 1 0.333 0.020 *** 0.308 0.026 *** 0.368 0.025 *** 2 0.332 0.025 *** 0.304 0.031 *** 0.373 0.033 *** 3 0.364 0.029 *** 0.334 0.031 *** 0.396 0.036 *** 4 0.402 0.028 *** 0.386 0.029 *** 0.407 0.038 *** 5 0.421 0.023 *** 0.437 0.031 *** 0.390 0.037 *** 6 0.413 0.028 *** 0.472 0.046 *** 0.351 0.046 *** 7 0.389 0.043 *** 0.488 0.070 *** 0.307 0.063 *** Difference between predicted impacts at receipt at age zero and six 0.036 0.04 0.125 0.057 ** -0.046 0.093 Age at first receipt of CSG Mothers <8 grades schooling Mothers 8+ grades schooling Predicted outcome Standard error Statistical significance Predicted outcome Standard error Statistical significance 0 0.317 0.05 *** 0.401 0.03 *** 1 0.383 0.04 *** 0.316 0.02 *** 2 0.422 0.04 *** 0.299 0.03 *** 3 0.433 0.05 *** 0.334 0.04 *** 4 0.433 0.06 *** 0.382 0.04 *** 5 0.440 0.06 *** 0.403 0.03 *** 6 0.465 0.06 *** 0.386 0.04 *** 7 0.505 0.08 *** 0.347 0.06 *** Difference between predicted impacts at receipt at age zero and six 0.148 0.08 ** -0.015 Age at first receipt of CSG Source: Calculated from household survey. *, significant at the 10% level; **, significant at the 5% level; *** significant at the 1% level. 58 The South African Child Support Grant Impact Assessment Table 7.13 Dose-response estimates of impact on probability of grade repetition by child sex and maternal education All children Girls Boys Predicted outcome Standard error Statistical significance Predicted outcome Standard error Statistical significance Predicted outcome Standard error Statistical significance 0 0.275 0.024 *** 0.202 0.035 *** 0.325 0.041 *** 1 0.259 0.017 *** 0.193 0.021 *** 0.340 0.026 *** 2 0.278 0.022 *** 0.198 0.027 *** 0.365 0.038 *** 3 0.312 0.030 *** 0.217 0.037 *** 0.379 0.045 *** 4 0.333 0.032 *** 0.242 0.039 *** 0.367 0.041 *** 5 0.320 0.026 *** 0.264 0.032 *** 0.331 0.033 *** 6 0.285 0.027 *** 0.279 0.034 *** 0.288 0.041 *** 7 0.245 0.040 *** 0.287 0.055 *** 0.253 0.059 *** Difference between predicted impacts at receipt at age zero and six 0.009 0.077 0.050 -0.036 0.056 Age at first receipt of CSG Mothers <8 grades schooling Mothers 8+ grades schooling Predicted outcome Standard error Statistical significance Predicted outcome Standard error Statistical significance 0 0.327 0.043 *** 0.256 0.029 *** 1 0.365 0.033 *** 0.228 0.020 *** 2 0.392 0.040 *** 0.241 0.023 *** 3 0.404 0.058 *** 0.274 0.030 *** 4 0.406 0.068 *** 0.291 0.037 *** 5 0.404 0.058 *** 0.271 0.031 *** 6 0.407 0.054 *** 0.227 0.029 *** 7 0.417 0.090 *** 0.183 0.039 *** Difference between predicted impacts at receipt at age zero and six 0.080 0.073 Age at first receipt of CSG -0.029 Source: Calculated from household survey. *, significant at the 10% level; **, significant at the 5% level; *** significant at the 1% level. Chapter 7 The Impact of the CSG on Schooling 59 learn. The counting and number identification questions are most discerning in children in very early grades; we included them in our survey instrument as a way of introducing children to the test in a non-threatening manner. Below we focus on children’s responses to the questions on arithmetic, shape recognition, word problems and pattern recognition. Table 7.14 provides mean scores and their standard deviations for selected components of the EGMA. Girls, on average, score slightly higher than boys and children whose mothers have more education score higher than children whose mothers have not completed primary school. 7.3 Scores on mathematics tests As part of the Young Child Questionnaire, children were administered the Early Grade Mathematics Assessment (EGMA), a battery of mathematics-related tests developed by Reubens (2009). These include ‘oral counting’ (count as high as you can in 30 seconds), number identification (point to a number and say what it is), number sequences (completes the following sequence, 23, …, 25), word problems (There are eight children walking to school. Six are boys and the rest are girls. How many girls are walking to school?), addition and subtraction, shape recognition (circles, squares, triangles, rectangles) and pattern recognition. The questions used in our survey were selected to be consistent with what South African children in Grades 1 to 4 would be expected to Tables 7.15, 7.16 and 7.17 show the results of estimating the dose-response model for three of these EGMA outcomes, total scores, arithmetic and shape recognition. Table 7.14 Scores on selected components of the EGMA by sex and maternal schooling Mother’s schooling Arithmetic Shape recognition Word problems Pattern recognition Total across all tests All Boys Girls < 8 grade 8+ grades Maximum score 7.11 6.96 7.27 6.56 7.30 10 (3.07) (3.11) (3.01) (3.22) (2.99) 10.85 10.64 11.08 10.42 11.00 (4.02) (4.09) (3.93) (4.42) (3.87) 3.05 3.00 3.11 2.94 3.09 (1.28) (1.31) (1.24) (1.36) (1.25) 3.74 3.64 3.84 3.48 3.79 (1.62) (1.65) (1.59) (1.80) (1.60) 24.50 24.03 25.01 23.11 24.95 (9.05) (9.14) (8.93) (9.61) (8.82) Source: Calculated from household survey. 14 4 5 33 60 The South African Child Support Grant Impact Assessment Table 7.15 Dose response estimates of impact on EGMA score by child sex and maternal education All children Girls Boys Predicted outcome Standard error Statistical significance Predicted outcome Standard error Statistical significance Predicted outcome Standard error Statistical significance 0 25.20 0.47 *** 25.68 0.57 *** 24.70 0.53 *** 1 24.45 0.37 *** 25.36 0.42 *** 23.66 0.48 *** 2 24.23 0.43 *** 25.37 0.60 *** 23.21 0.69 *** 3 24.43 0.51 *** 25.42 0.68 *** 23.45 0.90 *** 4 24.69 0.55 *** 25.18 0.64 *** 24.12 0.89 *** 5 24.71 0.49 *** 24.53 0.65 *** 24.79 0.67 *** 6 24.43 0.58 *** 23.63 0.92 *** 25.18 0.65 *** 7 23.96 0.89 *** 22.71 1.33 *** 25.27 1.05 *** Difference between predicted impacts at receipt at age zero and six -0.77 0.71 -2.05 1.09 * 0.48 Age at first receipt of CSG Mothers <8 grades schooling Mothers 8+ grades schooling Predicted outcome Standard error Statistical significance Predicted outcome Standard error Statistical significance 0 24.17 0.98 *** 25.50 0.49 *** 1 23.29 0.73 *** 24.84 0.43 *** 2 22.44 0.77 *** 25.03 0.45 *** 3 21.99 0.93 *** 25.60 0.51 *** 4 22.16 1.05 *** 25.86 0.57 *** 5 22.87 0.93 *** 25.45 0.57 *** 6 23.76 0.82 *** 24.54 0.77 *** 7 24.54 1.21 *** 23.48 1.16 *** Difference between predicted impacts at receipt at age zero and six -0.40 Age at first receipt of CSG -0.96 Source: Calculated from household survey. *, significant at the 10% level; **, significant at the 5% level; *** significant at the 1% level. Chapter 7 The Impact of the CSG on Schooling 61 Table 7.16 Dose-response estimates of impact on arithmetic score by child sex and maternal education All children Girls Boys Predicted outcome Standard error Statistical significance Predicted outcome Standard error Statistical significance Predicted outcome Standard error Statistical significance 0 7.50 0.17 *** 7.61 0.20 *** 7.32 0.16 *** 1 7.13 0.14 *** 7.44 0.17 *** 6.89 0.18 *** 2 6.95 0.13 *** 7.33 0.19 *** 6.67 0.23 *** 3 6.95 0.15 *** 7.25 0.21 *** 6.71 0.27 *** 4 7.04 0.17 *** 7.17 0.22 *** 6.90 0.28 *** 5 7.09 0.15 *** 7.04 0.23 *** 7.09 0.23 *** 6 7.06 0.18 *** 6.88 0.32 *** 7.20 0.20 *** 7 6.94 0.28 *** 6.71 0.46 *** 7.18 0.30 *** Difference between predicted impacts at receipt at age zero and six -0.44 0.25 * -0.72 0.36 ** -0.12 0.30 Age at first receipt of CSG Mothers <8 grades schooling Mothers 8+ grades schooling Predicted outcome Standard error Statistical significance Predicted outcome Standard error Statistical significance 0 7.12 0.34 *** 7.64 0.16 *** 1 6.58 0.29 *** 7.32 0.11 *** 2 6.17 0.29 *** 7.25 0.14 *** 3 6.02 0.32 *** 7.36 0.19 *** 4 6.14 0.36 *** 7.44 0.20 *** 5 6.42 0.35 *** 7.38 0.17 *** 6 6.71 0.34 *** 7.18 0.22 *** 7 6.90 0.43 *** 6.90 0.37 *** Difference between predicted impacts at receipt at age zero and six -0.41 0.41 -0.46 0.28 * Age at first receipt of CSG Source: Calculated from household survey. *, significant at the 10% level; **, significant at the 5% level; *** significant at the 1% level. 62 The South African Child Support Grant Impact Assessment Table 7.17 Dose-response estimates of impact on shape recognition score by child sex and maternal education All children Girls Boys Predicted outcome Standard error Statistical significance Predicted outcome Standard error Statistical significance Predicted outcome Standard error Statistical significance 0 10.88 0.21 *** 11.01 0.30 *** 10.78 0.30 *** 1 10.81 0.14 *** 11.22 0.21 *** 10.42 0.25 *** 2 10.88 0.17 *** 11.49 0.22 *** 10.28 0.27 *** 3 11.01 0.25 *** 11.59 0.28 *** 10.42 0.35 *** 4 11.07 0.28 *** 11.38 0.33 *** 10.74 0.40 *** 5 11.01 0.23 *** 10.89 0.34 *** 11.10 0.32 *** 6 10.84 0.24 *** 10.31 0.44 *** 11.36 0.26 *** 7 10.65 0.40 *** 9.80 0.65 *** 11.49 0.40 *** Difference between predicted impacts at receipt at age zero and six -0.04 -0.70 0.52 0.58 0.41 Age at first receipt of CSG Mothers <8 grades schooling Mothers 8+ grades schooling Predicted outcome Standard error Statistical significance Predicted outcome Standard error Statistical significance 0 10.34 0.49 *** 11.04 0.19 *** 1 10.42 0.33 *** 10.92 0.16 *** 2 10.39 0.40 *** 11.12 0.20 *** 3 10.33 0.47 *** 11.39 0.24 *** 4 10.33 0.53 *** 11.46 0.27 *** 5 10.45 0.54 *** 11.24 0.26 *** 6 10.67 0.60 *** 10.82 0.32 *** 7 10.92 0.81 *** 10.37 0.45 *** Difference between predicted impacts at receipt at age zero and six 0.33 -0.22 0.40 Age at first receipt of CSG Source: Calculated from household survey. *, significant at the 10% level; **, significant at the 5% level; *** significant at the 1% level. Chapter 7 The Impact of the CSG on Schooling 63 We begin with Table 7.15. This provides some evidence that children who have been enrolled in the CSG for longer score higher on the EGRA. The magnitude of this impact is small however, children at birth score 0.77 points or three per cent higher than children enrolled at age six. This difference is not statistically significant. When we disaggregate by sex, we find that late enrolled girls score 2.05 points or 8.7 per cent lower than early enrolled girls, and this impact is statistically significant. There is no impact on boys or on children whose mothers have less than eight grades of schooling. Early enrolment boosts the EGRA scores of children whose mothers have completed primary school, but the effect is imprecisely measured. Tables 7.16 and 7.17 disaggregate impacts by test item for the largest components of the EGRA battery of tests, arithmetic and shape recognition. This shows that for the full sample, and for children whose mothers have eight or more grades of schooling, late CSG enrolment (enrolment at age six) lowers arithmetic scores by 6.0 and 6.2 per cent (-0.44 and -0.46 points) respectively, compared to children enrolled at birth or in the first year of life. Both are statistically significant at the 10 per cent level. There is a large impact on girls’ arithmetic scores – early enrolment raises these by 0.72 points or 10.5 per cent. This effect is statistically significant at the five per cent level. By contrast, there is no impact of early CSG enrolment on shape recognition. 7.4 Scores on reading and vocabulary tests Children were administered the Early Grade Reading Assessment (EGRA). EGRA documents early grade reading skills127. Like the EGMA, it consists of a battery of tests. Children begin by doing a timed reading of letters and familiar words. They are then asked to read the following passage and answer questions about it. 127. RTI, 2009. Jabu had a dog. The dog was fat and happy. One day Jabu and the dog went out to play. The little dog ran away and got lost. Jabu was sad but after a while the dog came back. Jabu took the dog home. When they got inside the house Jabu gave the dog a bone. The little dog was tired, so he slept. When the dog woke up, Jabu took the dog outside again to play. As with the EGMA, the first two components were administered largely to ensure that children were comfortable with the test. Here we assess whether early enrolment in the CSG affected the likelihood that children could read this passage and answer questions about it. Just under half of the same (47 per cent) could read this passage in less than one minute and, on average, children provided 2.8 correct answers to questions that tested their reading comprehension. 64 The South African Child Support Grant Impact Assessment Table 7.18 Dose-response estimates of impact on likelihood of complete reading of story by child sex and maternal education All children Girls Boys Predicted outcome Standard error Statistical significance Predicted outcome Standard error Statistical significance Predicted outcome Standard error Statistical significance 0 0.490 0.027 *** 0.545 0.036 *** 0.417 0.046 *** 1 0.462 0.021 *** 0.534 0.027 *** 0.400 0.026 *** 2 0.468 0.024 *** 0.537 0.031 *** 0.420 0.032 *** 3 0.492 0.029 *** 0.537 0.038 *** 0.461 0.043 *** 4 0.507 0.030 *** 0.517 0.043 *** 0.495 0.045 *** 5 0.496 0.024 *** 0.475 0.046 *** 0.503 0.039 *** 6 0.463 0.030 *** 0.422 0.055 *** 0.487 0.047 *** 7 0.423 0.053 *** 0.372 0.072 *** 0.462 0.073 *** Difference between predicted impacts at receipt at age zero and six -0.026 0.043 -0.123 0.060 ** 0.070 Age at first receipt of CSG Mothers <8 grades schooling Mothers 8+ grades schooling Predicted outcome Standard error Statistical significance Predicted outcome Standard error Statistical significance 0 0.479 0.055 *** 0.497 0.038 *** 1 0.426 0.038 *** 0.471 0.018 *** 2 0.433 0.052 *** 0.483 0.028 *** 3 0.473 0.058 *** 0.513 0.038 *** 4 0.499 0.055 *** 0.528 0.038 *** 5 0.477 0.047 *** 0.515 0.030 *** 6 0.412 0.055 *** 0.479 0.035 *** 7 0.330 0.079 *** 0.437 0.055 *** Difference between predicted impacts at receipt at age zero and six -0.067 0.076 Age at first receipt of CSG -0.017 Source: Calculated from household survey. *, significant at the 10% level; **, significant at the 5% level; *** significant at the 1% level. Chapter 7 The Impact of the CSG on Schooling 65 Table 7.19 Dose-response estimates of impact on reading comprehension by child sex and maternal education All children Girls Boys Predicted outcome Standard error Statistical significance Predicted outcome Standard error Statistical significance Predicted outcome Standard error Statistical significance 0 2.95 0.09 *** 3.12 0.13 *** 2.78 0.13 *** 1 2.75 0.07 *** 3.00 0.09 *** 2.52 0.10 *** 2 2.65 0.10 *** 2.91 0.12 *** 2.42 0.14 *** 3 2.67 0.12 *** 2.85 0.16 *** 2.49 0.15 *** 4 2.76 0.13 *** 2.85 0.16 *** 2.66 0.16 *** 5 2.84 0.10 *** 2.86 0.15 *** 2.81 0.15 *** 6 2.87 0.12 *** 2.86 0.19 *** 2.90 0.19 *** 7 2.85 0.19 *** 2.84 0.28 *** 2.90 0.28 *** Difference between predicted impacts at receipt at age zero and six -0.08 -0.26 0.22 Age at first receipt of CSG Mothers <8 grades schooling 0.12 Mothers 8+ grades schooling Predicted outcome Standard error Statistical significance Predicted outcome Standard error Statistical significance 0 2.66 0.22 *** 3.05 0.13 *** 1 2.31 0.14 *** 2.89 0.09 *** 2 2.13 0.15 *** 2.85 0.11 *** 3 2.13 0.18 *** 2.91 0.12 *** 4 2.21 0.20 *** 3.01 0.12 *** 5 2.28 0.19 *** 3.10 0.11 *** 6 2.27 0.21 *** 3.13 0.13 *** 7 2.17 0.32 *** 3.11 0.20 *** Difference between predicted impacts at receipt at age zero and six -0.39 0.25 Age at first receipt of CSG 0.08 Source: Calculated from household survey. *, significant at the 10% level; **, significant at the 5% level; *** significant at the 1% level. 66 The South African Child Support Grant Impact Assessment 7.5 Summary We have covered a lot of ground in this chapter and so a summary is helpful. Briefly, we have used a dose-response model to compare the impact of being enrolled ‘early’ in the CSG (at birth or in the first year of life) or ‘late’ (age six, just before starting school). We consider these impacts for the full sample as well as disaggregating by sex and by the education of the child’s mother. We find: ˜˜ Children who were enrolled in the CSG at birth completed 0.14 more grades of schooling than children who were enrolled at age six. This impact is statistically significant. Early enrolment raises scores on a test of arithmetic by 6.0 per cent. ˜˜ There are large differences in impact when we disaggregate by child sex. Early CSG enrolment has positive impacts on many dimensions of girls’ schooling and learning. Earlier enrolment in the CSG improved girls’ grade attainment by one quarter of a grade compared to enrolment in the CSG at age six. This is a large impact, given that most children in the sample had only completed four grades of schooling. The pathway through which this occurs is the reduction in delayed entry. Early receipt of the CSG reduces delayed school entry of girls by 26.5 per cent. Girls who were enrolled early obtain higher marks on tests of mathematical ability and reading. ˜˜ By contrast, there are no impacts on boys’ schooling or learning. ˜˜ For children whose mothers have less than eight grades of schooling, early enrolment in the CSG raises grade attainment by 0.38 grades, a 10.2 per cent increase. The CSG has no impact on grade attainment of children whose mothers have eight or more grades of schooling. These results suggest that the CSG is playing a compensatory role in narrowing the gap between children whose mothers have not completed primary school and mothers with at least some secondary education. For children enrolled at age six, the difference between predicted grade attainments is 0.41 (3.70 versus 4.11), a difference of nearly half a grade. But for children enrolled at birth, the difference is negligible, 0.06 grades. We observe this effect because early receipt of CSG reduces the likelihood that children from these disadvantaged backgrounds are less likely to enrol late. Chapter 8 The Impact of the CSG on the Health of 10-Year-Old Children 67 Chapter 8 The Impact of the CSG on the Health of 10-Year-Old Children 8.1 Introduction This chapter assesses the impact of the Child Support Grant on current illness and health related expenditures. 8.2 Current health status The primary caregiver was asked if the index child had suffered from a number of specified illnesses in the 15 days prior to the survey. Table 8.1 shows that approximately three-quarters of children were free of illness over this period. Only 3.1 per cent had three or more illnesses. There is no difference in caregiver-reported illness by sex. Table 8.1 Number of different illnesses by sex Number of different illnesses Female Male All children 0 73.1 74.4 73.8 1 16.7 13.9 15.3 2 7.2 8.4 7.8 3 2.3 1.9 2.1 4 0.3 1.1 0.7 5 or more 0.3 0.3 0.3 Source: Survey data The percentage of children with these specified illnesses is reported in Table 8.2. Flu and colds were the most common illnesses, followed by stomach aches and coughs. There is no difference in the reported prevalence of these illnesses by sex. Across the sample of all 10-year-olds, the average child was sick for 1.5 days, with no difference between boys and girls. Conditional on being ill, the average number of days ill in the last 15 days was 5.9 days.128 On average, households spent R9.30 on illness-related expenditures over the recall period. Conditional on a child being 128. Note that this is not the duration of illness. We do not know how many days prior to the last 15 days a child had been sick and there may be children who were still ill at the time of interview. sick, 30.9 per cent of households spent money on medication, tests, fees, transport and other illness-related costs. Among those households where the index child was ill and money was spent treating this illness, median expenditures were R66 and only 13 per cent of households spent more than R200. In households where the index child was ill and where money was spent treating this illness, 79 per cent of households spent money on medication, 35 per cent paid consultation fees and 46 per cent had transport-related expenses. 8.3 Impact of CSG enrolment on children’s illness Tables 8.3 and 8.4 give dose-response estimates of the impact of time of enrolment in the CSG on whether the caregiver reported that the child was ill in the previous 15 days and the number of days they were ill. Our dose-response model shows that 24 per cent of children who were enrolled in the CSG at birth (see the first column in Table 8.3) are predicted to have been ill in the last 15 days, a number slightly smaller than the average for the full sample, 26.2 per cent. By contrast, 28.6 per cent of children who were enrolled at age six are predicted to have been ill over the same period, a difference of 4.5 per cent. The difference in these predicted probabilities of being ill is not statistically significant. However, when we disaggregate, we find statistically significant impacts for two groups. Again, comparing children enrolled at age six with those enrolled at birth, we find that boys who were enrolled later had a higher predicted likelihood of being ill, 30.3 per cent compared to 21.2 per cent for boys enrolled at birth. This difference is statistically significant at the 10 per cent level. When we disaggregate by maternal education, we find that children enrolled at birth and with mothers with eight or more grades of schooling have a predicted likelihood of being ill of 19.6 per cent, 8.5 percentage points lower than comparable children enrolled at age six. There is no impact on children whose mothers have less than eight grades of schooling. Recall that we found this same pattern when we looked at anthropometric outcomes in Chapter 6. These results suggest that the health benefits associated with earlier CSG enrolment persist at least to age 10. 68 The South African Child Support Grant Impact Assessment Table 8.2 Percentage of children with the following symptoms in the last 15 days by sex Sex Fever Chronic diarrhoea Flu or cold Cough Open sores Mouth or throat Ear infection Stomach ache Worms Other Female 4.3 0.7 15.4 3.8 1.8 4.0 1.7 5.9 2.3 1.7 Male 2.7 1.7 16.7 4.7 2.8 3.1 1.4 5.2 3.3 0.9 All 3.5 1.2 16.1 4.3 2.3 3.6 1.5 5.5 2.8 1.3 Source: Survey data Table 8.3 Dose-response estimates of impact on probability of illness in last 15 days by child sex and maternal education All children Girls Boys Predicted outcome Standard error Statistical significance Predicted outcome Standard error Statistical significance Predicted outcome Standard error Statistical significance 0 0.240 0.024 *** 0.264 0.036 *** 0.212 0.033 *** 1 0.267 0.018 *** 0.274 0.021 *** 0.259 0.023 *** 2 0.259 0.020 *** 0.267 0.028 *** 0.255 0.030 *** 3 0.236 0.021 *** 0.253 0.031 *** 0.225 0.034 *** 4 0.227 0.023 *** 0.246 0.033 *** 0.211 0.030 *** 5 0.244 0.023 *** 0.253 0.033 *** 0.237 0.026 *** 6 0.286 0.026 *** 0.270 0.038 *** 0.303 0.044 *** 7 0.340 0.037 *** 0.292 0.056 *** 0.394 0.079 *** Difference between predicted impacts at receipt at age zero and six 0.045 0.006 0.05 0.091 0.06 * Age at first receipt of CSG Mothers <8 grades schooling Mothers 8+ grades schooling Predicted outcome Standard error Statistical significance Predicted outcome Standard error Statistical significance 0 0.320 0.045 *** 0.196 0.026 *** 1 0.277 0.037 *** 0.272 0.019 *** 2 0.262 0.046 *** 0.273 0.023 *** 3 0.270 0.052 *** 0.226 0.026 *** 4 0.286 0.057 *** 0.197 0.024 *** 5 0.295 0.055 *** 0.214 0.023 *** 6 0.289 0.057 *** 0.281 0.033 *** 7 0.271 0.072 *** 0.383 0.053 *** Difference between predicted impacts at receipt at age zero and six -0.031 0.075 0.085 0.039 ** Age at first receipt of CSG Source: Calculated from household survey. *, significant at the 10% level; **, significant at the 5% level; *** significant at the 1% level Chapter 8 The Impact of the CSG on the Health of 10-Year-Old Children 69 Table 8.4 Dose-response estimates of impact on number of days ill in last 15 days by child sex and maternal education All children Girls Boys Predicted outcome Standard error Statistical significance Predicted outcome Standard error Statistical significance Predicted outcome Standard error Statistical significance 0 1.41 0.16 *** 1.37 0.23 *** 1.40 0.28 *** 1 1.51 0.13 *** 1.58 0.17 *** 1.43 0.15 *** 2 1.48 0.16 *** 1.61 0.20 *** 1.37 0.20 *** 3 1.41 0.19 *** 1.54 0.26 *** 1.31 0.27 *** 4 1.43 0.22 *** 1.48 0.29 *** 1.39 0.30 *** 5 1.58 0.21 *** 1.55 0.28 *** 1.62 0.31 *** 6 1.82 0.23 *** 1.72 0.35 *** 1.94 0.36 *** 7 2.09 0.32 *** 1.94 0.49 *** 2.25 0.48 *** Difference between predicted impacts at receipt at age zero and six 0.416 0.312 0.345 0.39 0.536 0.44 Age at first receipt of CSG Mothers <8 grades schooling Mothers 8+ grades schooling Predicted outcome Standard error Statistical significance Predicted outcome Standard error Statistical significance 0 2.01 0.38 *** 1.15 0.21 *** 1 1.86 0.32 *** 1.43 0.13 *** 2 1.86 0.39 *** 1.38 0.17 *** 3 1.96 0.41 *** 1.19 0.21 *** 4 2.10 0.45 *** 1.12 0.21 *** 5 2.17 0.46 *** 1.31 0.20 *** 6 2.15 0.50 *** 1.71 0.29 *** 7 2.07 0.67 *** 2.15 0.44 *** Difference between predicted impacts at receipt at age zero and six 0.138 0.56 0.556 0.32 * Age at first receipt of CSG Source: Calculated from household survey. *, significant at the 10% level; **, significant at the 5% level; *** significant at the 1% level. 70 The South African Child Support Grant Impact Assessment Table 8.4 examines the impact of the duration of CSG enrolment on the number of days that the child is reported to be ill. It is important to remember that with more than 70 per cent of children reporting no illness, there is limited variability in the data which makes discerning impact more difficult. Mindful of this, for the full sample early CSG enrolment reduces the number of days the child is reported being ill by 0.42 days. This impact is larger for boys and for children whose mothers have more than eight grades of schooling, 0.54 and 0.56 days respectively. The impact on children with better educated mothers is statistically significant at the 10 per cent level.129 We made several attempts to estimate the effect of duration of CSG enrolment on illness-related expenditures but could find no evidence of impact. We note, however, that the sample of children for these estimates, less than 300, is small with the result that our estimated standard errors tend to be large. In preliminary work, we considered assessing whether early enrolment in the CSG affected the quality of children’s diets. However, it is not clear why the duration of grant receipt should affect this outcome and RDC (2012) indicated that there were considerable problems with the collection of the data. For these reasons, this was not pursued further. 129. We also assessed if conditional on being reported ill, whether duration of CSG enrolment had an impact on reducing the number of days that the child was reported ill. For the full sample, the impact is large, 0.67 days, but in part because the sample is small (less than 300 observations), not statistically significant. 8.4 Summary This chapter has assessed the impact of the Child Support Grant on aspects of child health. Children enrolled in the CSG early in life are less likely to be experiencing illness in the 15 day period prior to the administration of the survey. This effect is not statistically significant. When we disaggregate, we find statistically significant impacts for two groups. We find that boys who were enrolled later had a higher predicted likelihood of being ill, 30.3 per cent compared to 21.2 per cent for boys enrolled at birth. This difference is statistically significant at the 10 per cent level. Children enrolled at birth and with mothers with eight or more grades of schooling have a predicted likelihood of being ill of 19.6 per cent, 8.5 percentage points lower than comparable children enrolled at age six. There is no impact on children whose mothers have less than eight grades of schooling. Recall that we found this same pattern when we looked at anthropometric outcomes in Chapter 6. These results suggest that the health benefits associated with earlier CSG enrolment persist at least to age 10. We do not find differences in expenditures on health, though we suspect this is partly a consequence of the relatively small number of children who were reported being ill. Chapter 9 The Impact of the CSG on Time Allocation and Labour Supply of 10-Year-Old Children 71 Chapter 9 The Impact of the CSG on Time Allocation and Labour Supply of 10-Year-Old Children 9.1 Introduction This chapter assesses the impact of the Child Support Grant on time allocation and labour supply of 10-year-old children. We consider whether the timing of enrolment in the CSG affects the amount of time spent studying, doing chores or working outside the household. 9.2 How children spend their time Virtually all children in this sample attend school (Table 9.1) and most report studying after school. Girls are more likely to do chores than boys. Few children are reported for family businesses or for pay outside the household. Table 9.1 Per cent of children undertaking specified activities by sex Activity Girls Boys All children 100.0% 99.5% 99.8% Studying after school 88.5 81.6 84.9 Assisting with household chores 64.7 54.7 59.5 Helping with family business 3.3 2.1 2.8 Working for pay outside the household 1.3 1.4 1.4 Attending school 9.3 Impact of CSG enrolment on children’s time allocation Tables 9.2 and 9.3 give dose-response estimates of the impact of time of enrolment in the CSG on the time these 10-year-old children spend studying and assisting with household chores. Tables 9.2 and 9.3 show clearly that earlier or later enrolment for the CSG has no impact on time spent studying or doing housework. This is also true if, as in previous chapters, we disaggregate by sex or by maternal education. However, we also experimented with other disaggregations and this produced one noteworthy finding shown in Table 9.4. For children in households with no electricity, late enrolment in the CSG reduces the amount of time spent studying. The magnitude of this impact is small, causing study time to fall by (-0.286 x 60) 17 minutes per day. One reason why we are unable to uncover impacts pertains to the ‘lumpiness’ of the data. Figure 9.1 gives an example of this, showing the distribution (by sex) of hours spent studying per day. There are mass points in the distribution at 0, 0.5 and 1.0 hours. These three points account for 72 and 85 per cent of the girls’ and boys’ distributions, respectively. This limited variability in our data makes it difficult to uncover impact. Source: Survey data Figure 9.1 Distribution of time spent studying by sex 35 30 25 20 15 10 5 0 0 .02 .05 .07 .08 .1 .17 .25 .33 .42 .5 .5 .58 .67 .75 .83 1 1.02 1.17 1.25 1.33 1.5 2 2.5 3 Children report studying one hour after school each day with no meaningful difference between boys and girls. Girls do about two hours of household chores per week, slightly more than boys who report doing 1.4 hours. Disaggregating data by characteristics of the primary caregiver does not reveal any systematic differences in the proportion of these children engaged in these tasks or the amount of time they spend doing them. These descriptive findings have implications for our dose-response estimates. Because so few children work outside the household, we will focus these solely on studying and household chores. Boys Girls 72 The South African Child Support Grant Impact Assessment Table 9.2 Dose-response estimates of impact on study time Child studied in last week Age at first receipt of CSG Hours studying per day Predicted outcome Standard error Statistical significance Predicted outcome Standard error Statistical significance 0 0.88 0.02 *** 0.89 0.05 *** 1 0.83 0.02 *** 0.81 0.03 *** 2 0.82 0.02 *** 0.80 0.05 *** 3 0.85 0.03 *** 0.83 0.06 *** 4 0.88 0.02 *** 0.86 0.06 *** 5 0.90 0.02 *** 0.86 0.05 *** 6 0.90 0.02 *** 0.81 0.04 *** 7 0.88 0.03 *** 0.75 0.06 *** Difference between predicted impacts at receipt at age zero and six 0.016 -0.074 Source: Calculated from household survey. *, significant at the 10% level; **, significant at the 5% level; *** significant at the 1% level. Sample size is 1091. Table 9.3 Dose-response estimates of impact on housework Child does housework Age at first receipt of CSG Hours spent doing housework per week Predicted outcome Standard error Statistical significance Predicted outcome Standard error Statistical significance 0 0.53 0.03 *** 1.76 0.25 *** 1 0.54 0.02 *** 1.85 0.17 *** 2 0.56 0.03 *** 1.97 0.22 *** 3 0.59 0.03 *** 2.02 0.27 *** 4 0.60 0.03 *** 1.92 0.26 *** 5 0.58 0.03 *** 1.67 0.22 *** 6 0.55 0.03 *** 1.38 0.22 *** 7 0.51 0.05 *** 1.12 0.33 *** Difference between predicted impacts at receipt at age zero and six 0.012 -0.384 Source: Calculated from household survey. *, significant at the 10% level; **, significant at the 5% level; *** significant at the 1% level. Sample size is 1091. Chapter 9 The Impact of the CSG on Time Allocation and Labour Supply of 10-Year-Old Children 73 Table 9.4 Dose-response estimates of impact on study time and housework, children in households with no electricity Hours studying per day Age at first receipt of CSG Hours spent doing housework per week Predicted outcome Standard error Statistical significance Predicted outcome Standard error Statistical significance 0 0.923 0.103 *** 1.720 0.290 *** 1 0.869 0.080 *** 2.128 0.377 *** 2 0.796 0.068 *** 2.324 0.406 *** 3 0.719 0.059 *** 2.279 0.360 *** 4 0.661 0.052 *** 2.063 0.346 *** 5 0.633 0.049 *** 1.804 0.389 *** 6 0.636 0.051 *** 1.616 0.461 *** 7 0.657 0.058 *** 1.548 0.586 *** Difference between predicted impacts at receipt at age zero and six -0.286 0.086 *** -0.104 Source: Calculated from household survey. *, significant at the 10% level; **, significant at the 5% level; *** significant at the 1% level. Sample size is 288. 9.4 Summary This chapter has assessed the impact of the Child Support Grant on time allocation and labour supply of 10-year-old children. At this age, we find few children working for pay outside the household. There is no statistically significant impact of early CSG enrolment on time spent studying or time spent doing housework. There are no statistically significant impacts when we disaggregate by sex or by maternal education. For children in households with no electricity, late enrolment in the CSG reduces the amount of time spent studying, but the magnitude of this impact is small. Limited variability in the data may be a reason why it is not possible to uncover an impact. 74 The South African Child Support Grant Impact Assessment © UNICEF Part 5 Receipt of the CSG and impacts on adolescents Chapter 10 Adolescent and Household Receipt of the Child Support Grant 75 Chapter 10 Adolescent and Household Receipt of the Child Support Grant Table 10.2 Realised samples of adolescents 10.1 Samples of adolescents In the overall evaluation strategy defined in advance of the evaluation, the plan was to sample a distribution of 15, 16, and 17-year-old adolescents that would allow for credible identification of CSG programme impacts. The basic idea was to compare adolescents of a similar age – those just above (non-beneficiaries) and just below (beneficiaries) the age eligibility cut-off for CSG receipt. The plan included oversampling 16-year-olds, some who were expected to already be CSG beneficiaries, and others who would be eligible but not yet receiving the CSG. Table 10.1 below shows the target sample sizes for each of the three adolescent age groups as presented in the overall evaluation strategy document. Table 10.1 Target sample sizes by adolescent age groups Age at time of sampling Beneficiaries Nonbeneficiaries Total 15 408 0 408 16 612 408 1,020 17 0 612 612 1,020 1,020 2,040 Total The data that were collected consist of 1,726 adolescents (85% of the target sample size of 2,040 adolescents), including 876 in the target group of 15- to 16-year-old CSG beneficiaries (85.9% of the 1,020 targeted) and 850 in the target group of 16- to 17-year-old non-beneficiaries (83.3% of the intended sub-sample), as confirmed in the household questionnaire.130 The realised distribution of adolescents, shown in Table 10.2, suggests that the sampling strategy worked effectively in the field to generate the desired sample composition. 130. Among the respondents to the questionnaire administered to households of adolescents, 97% were the child’s main caregiver; 23 cases had a brother or sister respond; 11 cases had an aunt or uncle respond, and others included grandparents, in-laws, cousins, mother/father, or in three cases, the sampled adolescent him/ herself. Belongs to the list of: Frequency Per cent 15/16 year-old beneficiaries 876 50.75 16/17 year-old non-beneficiaries 850 49.25 1,726 100.00 Total Consistent with this original study design, we anticipated conducting an analysis that would compare adolescents who are current CSG beneficiaries with the sampled adolescents who are not receiving the grant. Household and adolescent responses to questions about household and adolescent receipt of the CSG indicated, however, that many of the sampled adolescents who are not currently receiving the CSG have received it in the past. A simple cross-tabulation showed that 410 (48%) of the 850 16- to 17-year-old non-beneficiaries (not currently receiving the CSG) likely received the CSG sometime in the past. Furthermore, in many households where the adolescents are not currently receiving the CSG, there is current receipt of the CSG for one or more other household members. Figure 10.1 shows the number of child support grants received by the households in the adolescent sample, irrespective of whether the grant is being received for the adolescent. This chart shows that there are only about 16 per cent of households with adolescents in our sample that were not receiving the CSG at the time that they completed the questionnaire. The South African Child Support Grant Impact Assessment Figure 10.1 CSG Receipt in households with adolescents Percent of households 76 Figure 10.2 Adolescent CSG Receipt by age at CSG start .25 100 .2 80 .15 60 .1 40 .05 20 0 21,9 25 78,1 75 30,1 43 21,3 78,7 69,9 57 0 0 2 4 6 8 10 12 Number of Child Support Grants currently received in household In addition, the questionnaire also asked for the year when the CSG was first received for the sampled adolescent. With this additional information, it was possible to examine the age at which adolescents started receiving the CSG, even if the household was not currently receiving the CSG for the adolescent. Figure 10.2 shows the age at which adolescents first began receiving the CSG, stratified by whether the household is currently receiving the CSG for the adolescent, and only including households (n = 1,113) where there has been some CSG receipt for the adolescent (i.e., currently or in the past). Under 3 years 3-4 years 5-9 years 10-13 years 14+ years HH received CSG for adolescent in past HH where adolescent is current CSG beneficiary There is an important pattern in the above figure on CSG receipt that warrants further discussion, because it has implications for the estimation of CSG impacts and the evaluation findings. Specifically, this graph suggests that youth who first obtained access to the CSG at an early age (four years or younger), or more recently at age 14 years or older, are significantly more likely to be in households that are currently receiving the CSG for the adolescent (at the time of the questionnaire). Most notable is the comparatively low proportion of adolescents who first began receiving the CSG at age 10–13 years and are in households currently receiving the CSG for them (i.e., 57%, vs 78% for those starting at the youngest or the oldest ages). Not surprisingly, the age at which adolescents first began receiving the CSG is, in general, correlated with the length of time the CSG is received, but this exception creates a nonlinearity in the measure of CSG receipt (the treatment in this evaluation) that influences the empirical strategy for estimation and the observed CSG impacts. Chapter 10 Adolescent and Household Receipt of the Child Support Grant 77 10.2 Definition of adolescent treatment and comparison groups 3 = Current CSG in the household but the household has never received the CSG for the adolescent As suggested by the results presented in Part 4 (on the impacts of the CSG on young children), it would be problematic to ignore the early CSG receipt for some adolescents (in the past) and substantial current CSG receipt in the households of adolescents in comparing adolescents in households that are currently receiving the CSG for them with those adolescents who are not current beneficiaries. In exploring how to characterise CSG receipt, taking into account receipt of the CSG in the household (even if not for the adolescent) and receipt of the CSG for the adolescent, either earlier or currently, the following five groups of adolescents were constructed: 4 = No current CSG in the household but the household received the CSG for the adolescent in the past 0 = No current CSG in the household and the household never received CSG for adolescent 1 = Current CSG in the household and the household is currently receiving the CSG for the adolescent 2 = Current CSG in the household; not currently receiving it for the adolescent, but the household received the CSG in the past for the adolescent Figure 10.3 below shows the distribution of these five groups in our sample of adolescents. Group 0 is the only group with no reported CSG receipt, either in the household or for the adolescent ever; this group constitutes slightly more than 10 per cent of the adolescent sample. The only other group with no current CSG receipt in the household (regardless of for whom the grant is received) is group 4, which includes about five per cent of adolescents who reside in households without the CSG but who received the CSG sometime in the past. Approximately half of all of the adolescents are in households that are receiving the CSG and specifically for the adolescent (group 1). Another 19 per cent (group 2) are in households currently receiving the CSG, but where the household no longer receives the grant specifically for the adolescent (but did so in the past). The final group (3) makes up about 15 per cent of the sample and consists of those in households that are receiving the CSG, but the grant was never received specifically for the adolescent. Figure 10.3 CSG receipt by household and for adolescent 5,2% 10,7% 14,9% Group 0. No current CSG in HH but received for adolescent in past Group 1. Current CSG in HH but never received for adolescent Group 2. Current CSG in HH; not for adolescent but received in past 18,7% 50,6% Group 3. Current CSG in HH & receiving for adolescent Group 4. No current CSG in HH; never received for adolescent The retrospective questions about household characteristics at the time of application for the CSG that were asked of households in the young children sample were not asked of the households with adolescents. Therefore, we have limited ability to assess and account for these characteristics that likely influenced the early take-up of the CSG 78 The South African Child Support Grant Impact Assessment among households with adolescents who started receiving the CSG at an earlier age. One variable that is available and could be viewed as relatively stable and influential in access to the CSG is the household head’s education level. This characteristic was assessed for the five subgroups of adolescents described above, and it was found that those in group 0, with no reported CSG receipt either in the household or for the adolescent ever, were clearly more advantaged by this measure. Approximately 22 per cent of heads of households in group zero have an education level of 12 years or higher, compared to approximately 8–8.5 per cent of heads of households for adolescents in groups 1, 2 and 3 (where the CSG is currently received by the household). For group 4, which includes households currently without the CSG but who received the CSG sometime in the past for the adolescent, about 13.5 per cent of household heads have 12 or more years of education. In summary, it does appear that the adolescents in households without current CSG receipt may be living in more advantaged households. This creates a selection problem – those adolescents in households without the CSG likely differ in ways for which we have limited ability to adjust for with the available data. ˜˜ Adolescent learned of the CSG from informal sources Still, empirical models developed to assess what affects enrolment in the CSG among adolescents suggested a number of important predictors of access to the CSG. The following measures were included in a multinomial logit model that was estimated to predict adolescent/household selection into these five treatment subgroups: ˜˜ Province = Western Cape Adolescent characteristics and awareness/knowledge of CSG availability: ˜˜ Adolescent age ˜˜ Adolescent is male ˜˜ Adolescent is not African ˜˜ Adolescent is not aware of the CSG ˜˜ Adolescent encouraged someone in the household to apply for the CSG ˜˜ Adolescent learned of the CSG from formal sources (e.g., SASSA/DSD, school teachers, social workers, hospital, churches, NGOs, road show, radio) (friends, neighbours, acquaintances) ˜˜ Adolescent knows the current eligible age for the CSG Household characteristics and awareness/knowledge of CSG availability and application effort: ˜˜ Mother applied for the CSG ˜˜ Household head education: grade K–5, 6–8, 9–11, 12 years or higher ˜˜ Household head disabled ˜˜ Someone in the household reapplied following change in eligibility ˜˜ Household respondent knows the current eligible age for the CSG ˜˜ Household respondent learned of the CSG from formal sources ˜˜ Age of household head ˜˜ Household head is female ˜˜ Province = Gauteng ˜˜ Province = Eastern Cape ˜˜ Province = Limpopo ˜˜ Omitted province = KwaZulu-Natal The base (omitted) treatment group in the analysis was group 1, which includes the adolescents (approximately 50%) in households that are receiving the CSG specifically for the adolescent. Table 10.3 on the following page summarises the results of this analysis, which simultaneously but separately predicts the probabilities of being in the other four treatment groups (as shown in Figure 10.3) relative to being in a household that is currently receiving the CSG specifically for the adolescent. The results are presented only for statistically significant predictors (at a 95% confidence level) and in the form of the percentage change in odds of being in that particular treatment group (relative to the group of households currently receiving the CSG for the adolescent) for a given characteristic or measure. Chapter 10 Adolescent and Household Receipt of the Child Support Grant 79 Examining the results in Table 10.3, the importance of adolescent age is apparent (and expected). As the adolescent’s age increases by one year, he/she has approximately 1,100%–1,700% higher odds of being in one of these other treatment groups where the household is not currently receiving the CSG specifically for the adolescent; the odds are highest for the two groups (0 and 3) where the household has never received the CSG for the adolescent. In addition, the major (reported) reason that the grant stopped in households with adolescents that formerly received the CSG but are no longer receiving it is because the adolescent became too old (84% for group 2 and 78% for group 4). If the adolescent is not aware of the availability of the CSG, he/she is 156%–333% more likely to be in one of these other treatment groups where the household is not currently receiving the CSG specifically for the adolescent. It also appears that if the adolescent has learned about the CSG from informal sources (friends, neighbours, acquaintances), he/she is 111–141% more likely to be in groups 2, 3 or 4, where the CSG is not specifically received for the adolescent, but may have been received for him or her in the past and/or currently for other household members. Adolescents who learned about the CSG from formal sources were 140% more likely to be in households not currently receiving the CSG for the adolescent (but who received it for the adolescent in the past) than to be in households with current CSG receipt for the adolescent. And among the last adolescent characteristics shown, adolescents with correct knowledge about the current CSG eligible age had 114% higher odds of being in households that were not receiving the CSG and had never received it for the adolescent (possibly reflecting these households’ higher education and other unmeasured socioeconomic factors). Among the household characteristics, if the mother applied for the CSG, adolescents had 84% lower odds of being in households that were not receiving the CSG and had never received it for the adolescent, and 82% lower odds of being in households that were currently receiving the CSG but had never received it for the adolescent (versus being in households with current CSG receipt for the adolescent). Correspondingly, if the mother is the head of household, the odds of the household not currently receiving the CSG and having never received it for the adolescent are 48% lower. The multinomial logit estimation also confirms that households with more highly educated heads (12 or more years of schooling) had significantly higher odds (201%) of no current CSG receipt and no receipt ever for the adolescent (likely reflecting socioeconomic advantages). Clearly, household re-applications for the CSG following policy changes affecting eligibility were very important in predicting who was in these four treatment groups relative to being in a household that is currently receiving the CSG specifically for the adolescent. The odds of being in one of these other treatment groups where the household is not currently receiving the CSG specifically for the adolescent are 76% to 100% lower if the household reapplies for the CSG. In addition, if someone in the household learns about the CSG from formal sources, the household has 91% (group 0) to 94% (group 3) lower odds of never having received the CSG for the adolescent. Finally, there were several province indicators that were significantly related to household and adolescent CSG receipt. Adolescents in Gauteng, which has the lowest poverty rate of South Africa’s provinces, are significantly more likely (165%) to be in households where the CSG was never received, consistent with the other (limited) evidence that these are more advantaged households. Correspondingly, adolescents residing in Western Cape have significantly lower odds (48%) of being in households with current CSG receipt in the household and past receipt for the adolescent, and those in the very poor province of Limpopo have significantly higher odds (98%) of being in this group (relative to being in households with current receipt specifically for the adolescent). This latter finding may raise a concern about why there is not more current receipt of the CSG specifically for the adolescents (i.e., in their teenage years) in the poorest of South Africa’s provinces. In summary, there are a number of factors that similarly influence selection into any one of these four treatment groups where the household is not currently receiving the CSG for the adolescent. At the same time, there are also some important differences (such as the influence of the household head’s education). We also see that the factors influencing selection into groups 0 and 3 (where the CSG 80 The South African Child Support Grant Impact Assessment Table 10.3 Predictors of CSG receipt by the household and for the adolescent, current and past Predictors of household and adolescent CSG receipt, current and past* (reported in percentage change in odds for statistically significant predictors from multinomial logit estimation) 0 = no HH CSG, never for adolescent 2 = current HH CSG, for adolescent in past 3 = current HH CSG, never for adolescent 4 = no HH CSG, received in past for adolescent Adolescent age (measured in years) 1,649% 1,565% 1,725% 1,124% Adolescent is not aware of the CSG 200% 166% 333% 156% Adolescent characteristics, awareness/knowledge of CSG Adolescent learned of the CSG from formal sources 140% Adolescent learned of the CSG from informal sources Adolescent knows the current eligible age for the CSG 124% 111% 141% 114% Household characteristics, awareness/knowledge/application effort Mother applied for the CSG -84% Household head education: Grade 12 or higher 201% Someone in the household reapplied following change in eligibility -100% Household respondent learned of the CSG from formal sources -91% Age of household head -82% -76% -99% -80% -94% -2% Household head is female -48% Province = Gauteng† 165% Province = Western Cape -48% Province = Limpopo 98% *Base (reference) group 1: Households currently receiving CSG for adolescent †Omitted province = KZN; Eastern Cape not statistically significant has never been received for the adolescent) appear more similar to one another, as do the predictors for groups 2 and 4 (in which the CSG was received for the adolescent in the past). The predicted probabilities from this multinomial logit model estimating selection into the five treatment subgroups are shown graphically in Figure 10.4. Here one can visually see the similarities in the distributions of predicted probabilities for groups 0 and 3, where the CSG has never been received for the adolescent, but households in group 3 are receiving the CSG for another family member. However, there are clearly issues of limited overlap for other potential comparisons across these subgroups. Notably, there are no adolescents other than those in group 1 (in households currently receiving the benefit for the adolescent) that have a predicted probability higher than 0.8, and the predicted probabilities for group 4 have a relatively narrow range of lower predicted probabilities (i.e., below 0.3). Chapter 10 Adolescent and Household Receipt of the Child Support Grant 81 Figure 10.4 Predicting selection into five CSG receipt subgroups Probability of selection into treatment subgroup 1 Comparison 1 (greatest contrast in statuses): group 0 (no current CSG in household; household never received CSG for adolescent) versus group 1(currently receiving CSG specifically for adolescent) Comparison 2 (closest match between groups): group 0 versus group 3 (households that have never received CSG for adolescent, but group 3 households receive CSG for another household member) .8 .6 .4 .2 0 Pr(Group=0) Pr(Group=1) Pr(Group=2) Pr(Group=3) Pr(Group=4) Treatment subgroup Together, these findings on household and adolescent receipt of the CSG suggest several different strategies for estimating CSG impacts for adolescents. One option is to follow the approach used in estimating CSG impacts for young children by using the age at which CSG receipt began as the treatment measure in generalised propensity score (GPS) models of the various outcomes. This analysis, however, is necessarily limited to adolescents in households with some CSG receipt for the adolescent, which excludes subgroups (0 and 4) for which no age at first receipt of the CSG is recorded. It is also possible that for some adolescent outcomes, it may be equally or more important that CSG receipt occurs at the time of adolescence, suggesting the importance of comparing adolescents with and without the CSG in their adolescent years (or at the time the questionnaire was administered). The five subgroups described above were used to set up the following comparisons between different beneficiary and non-beneficiary statuses (of adolescents and their households): Comparison 3: group 2 versus group 4 (households that received the CSG for adolescent in the past, but only group 2 households currently receive CSG for another household member) Comparison 4: groups 1, 2 and 3 households are combined as all currently receive the CSG (irrespective of current or past receipt specifically for the adolescent) versus groups 0 and 4 combined, which have no current CSG receipt (irrespective of past receipt) Propensity score matching (PSM) analyses were conducted separately for each of these different comparisons of adolescent households to estimate the impacts of the various possible combinations (or contrasts) of adolescent and household CSG receipt on adolescent outcomes. Importantly, a common support is imposed to exclude poor matches between adolescents in the different groups (or combinations of groups) that are compared. Even then, it is not possible to fully balance the characteristics between the different adolescents compared in some of these analyses. Some of the analyses are also conducted separately for male and female adolescents. 82 The South African Child Support Grant Impact Assessment Chapter 11 The Impact of the CSG on Adolescent Schooling Outcomes 11.1 Introduction Prior to estimating propensity score and generalised propensity models of adolescent schooling outcomes, we examined associations between CSG receipt and measures of schooling outcomes descriptively. Both the household respondents and the adolescents (separately) answered questions about their enrolment and attendance at school, and grade repetition and educational attainment were also measured in the questionnaires. Figure 11.1 Agreement between adolescents and their families on school enrolment 1000 30 800 852 600 400 222 200 11.2 School enrolment The household respondents report that 95 per cent of adolescents in the sample are currently enrolled in school. However, some adolescent reports of school enrolment differed from the report of the household respondent. Among the 1,504 adolescents who completed the behavioural survey, 391 (26%) did not answer the question asking if they are currently enrolled in school. Of the 1,113 who responded, a little over 20 per cent answered differently than the household respondent; seven of these 229 indicated that they are enrolled in school when the household respondent says that they are not enrolled, but most reported not being in school, in contrast to the household respondent replying that they were enrolled (see Figure 11.1). 0 7 Adolescent and household responses agree Adolescent and household responses differ Not currently enrolled in school Currently enrolled in school Where adolescent and household responses differ, adolescents are more likely to report not being enrolled in school. Performing cross-tabulations using adolescent reports of enrolment for the 1,113 who responded, no statistically significant differences were found across the five treatment subgroups (of household and adolescent CSG receipt) in enrolment rates (see Figure 11.2). Holding age constant (i.e., comparing only the 16-year-olds across this measure), there were still no statistically significant differences in enrolment rates, although group 4 stands out with a higher proportion enrolled (see Figure 11.2). Chapter 11 The Impact of the CSG on Adolescent Schooling Outcomes 83 Figure 11.2 School enrolment rates by treatment group 60 500 59,7 53,3 50 48,6 20 48,6 42,7 40 42 43,6 452 400 43,8 300 35,8 30 Figure 11.3 Adolescent and household reports of adolescent absences from school 200 25 144 100 10 105 50 0 Group 0 Group 1 Group 2 Group 3 Group 4 0 0 days 49 14 1-2 days 3-5 days 2 37 6-10 days 0 8 11-20 days 19 0 20+ days Reached grade 10 by age 16 Adolescent and household responses differ Repeated a grade Adolescent and household responses agree 11.3 Absences from school Turning to another schooling measure, the number of days absent from school was reported by both the household survey respondent and the adolescent in the behavioural questionnaire. These continuous variables were recoded into categories indicating: 0 days absent, (1) 1–2 days absent, (2) 3–5 days absent, (3) 6–10 days absent, (4) 11–20 days absent and (5) more than 20 days absent in a period of eight weeks. There were 1,587 (of 1,726) household responses to this question, whereas only 916 of the adolescents answered this question in the behavioural survey. When compared, the patterns across household and adolescent responses on absences are similar to those of enrolments; that is, the adolescents self-report more days absent from school (43% report one or more days absent) than the household survey respondents (who report one or more days for just 18% of adolescents). Again, looking descriptively, a chi-square test suggests that there are no statistically significant differences in the number of days absent across the five treatment subgroups of adolescent and household CSG receipt (see Figure 11.4). However, group 4 (where there is no current CSG receipt by the household but it was received for the adolescent in the past) looks different from the others, with adolescents in these households reporting more absences. 84 The South African Child Support Grant Impact Assessment Figure 11.4 Number of school absences by treatment group 80 70 60 50 40 30 20 Figure 11.5 Adolescent grade repetition and attainment by treatment subgroup 10 0 in their household) are clearly different from the others, with a higher rate of grade attainment (nearly 60% reaching Grade 10 by age 16) and considerably lower grade repetition (25%); this likely reflects, as indicated before, that they are a more advantaged group of adolescents. Adolescents in group 1, in which households are receiving the CSG specifically for the adolescent, also have a lower rate of grade repetition (36%) than that of adolescents in groups 2, 3 and 4, although there is little difference in their grade attainment at age 16 (compared to the other groups). Group 0 Group 1 Group 2 Group 3 Group 4 60 >20 days

3-5 days

10-20 days

1-2 days

6-10 days

0 days

More than a third (37%) of adolescents in our sample
repeated a grade at some time in their education. Descriptive
statistics showed, however, no statistically significant differences in grade repetition by the age at first start of receipt of
the CSG or the length of time they were receiving the CSG
(if beneficiaries). Similarly, cross-tabulations show no statistically significant relationship between grade attainment (at
age 16) and the age at which adolescents started to receive
the CSG. Not surprisingly, though, there is a very strong
relationship between grade repetition and grade attainment,
where adolescents are significantly more likely to reach
grades 10, 11 or 12 by age 16 if they have not repeated a
grade.
We do also see some statistically significant differences (confirmed by chi-square tests) in rates of grade repetition and
grade attainment across the five treatment groups (see Figure
11.5). Adolescents in group 0 (who have never had the CSG

53,3
48,6

48,6
42,7

43,6

43,8

35,8

30
20

11.4 Grade repetition and grade
attainment

59,7

10
0

Group 0

Group 1

Group 2

Group 3

Group 4

Reached grade 10 by age 16
Repeated a grade

Having examined these patterns in schooling outcomes
descriptively, we acknowledge that they are merely suggestive, as they do not adjust for the observed and unobserved
selective differences that are likely present among these
groups of adolescents (as seen in the estimation in Chapter
10). Thus, we turn now to look at the results of matching
analyses that were used to better understand if there are
potential causal linkages between CSG receipt and adolescents’ schooling outcomes.

Chapter 11 The Impact of the CSG on Adolescent Schooling Outcomes 85

11.5 Results of the estimation of CSG
impacts on schooling outcomes
In estimating CSG impacts on adolescents, generalised propensity score (GPS) matching methods were used for the
interval treatment measure of age at which the CSG was
first received (ranging from 0–16 years). When the treatment measure was binary (i.e., in comparing different groups
of adolescents based on their current and past household and
adolescent CSG receipt), propensity score matching (PSM)
methods were used with alternative matching algorithms
such as radius and nearest neighbour matching. In the
analysis of all of the adolescent outcomes, estimation with
both interval and binary outcome measures of CSG receipt
was performed. For brevity, the detailed findings are only
presented for outcomes where statistically significant relationships between CSG receipt and the outcomes are found.
As expected from the descriptive analyses, which showed
no significant associations between age at first CSG receipt
and education outcomes, we did not find any statistically
significant relationships between age at first CSG receipt
and these four education outcomes – enrolment in school,
absences from school, grade repetition and grade attainment
– in the GPS matching models (which exclude adolescents
in households that never received the CSG). However, comparing adolescents in the treatment subgroups that account
for CSG receipt in the household as well as for the adolescent, the PSM models indicated statistically significant
impacts of CSG receipt on adolescent absences from school,
although not for school enrolment, grade repetition or grade
attainment.

Table 11.1 presents the findings from comparisons between
treatment subgroups that indicated statistically significant
impacts of CSG receipt on adolescent days absent from
school. The analyses were performed for all adolescents as
well as separately for males and females. The findings from
the first comparison appear to suggest that what is most
important in reducing adolescent absences is current receipt
of the CSG in the household, regardless of whether it is
ever received for the adolescent. This is qualified, however,
by the finding that these impacts are larger for males and
are not statistically significant in the females-only analyses.
Specifically, adolescents in households currently receiving
the CSG (but that never received it for the adolescent) were
absent 2.2 fewer days than adolescents in households that do
not (and never did) receive the CSG. In addition, males were
absent nearly four days fewer if they were in a household currently receiving the CSG (even though the grant was never
received for the adolescent), compared to adolescent males
in households with no CSG receipt ever. (Note that the
standard error on this latter estimate is larger, likely due to
the smaller sample size in this analysis, although the t-ratio
indicates statistical significance at about 90% confidence).
The results of the second comparison presented in Table
11.1 show that adolescents in households currently receiving the CSG are absent approximately 2.3 days fewer than
those in households not receiving the CSG, irrespective of
whether the CSG was ever received for the adolescent in the
past or for whom the grant is currently received. Again, this
effect is much stronger for males; males in households currently receiving the CSG were absent approximately seven
days fewer than males in households not receiving the CSG.

The South African Child Support Grant Impact Assessment

Table 11.1 PSM estimation – comparison of subgroups
No. of obs.*

Ave. days absent from school
Treatment

T-C difference
(impact)

Comparison

Std. Error

T-ratio

Group 3 vs 0: current versus no CSG in household, never received for adolescent
All adolescents

197

1.23

3.44

-2.22

1.05

-2.11

Males

1.45

5.29

-3.85

2.36

-1.63

Females

106

1.12

1.10

0.03

0.64

0.04

Groups 1, 2, 3 versus 0 & 4: current versus no CSG in household, irrespective of past receipt, who receives the CSG
All adolescents

861

1.92

4.18

-2.26

1.42

-1.60

Males

387

2.29

9.34

-7.05

3.35

-2.10

Females†

398

1.72

2.54

-0.82

0.70

-1.08

* On the common support
†83 cases were excluded (not on common support)

In sum, the impact of receipt of the CSG by household
members at the time of adolescence, even if not specifically for the adolescent, appears to reduce male adolescent
absences from school, although we do not find statistically
significant impacts of CSG receipt on other adolescent
schooling outcomes.

Chapter 12 The Impact of the CSG on Adolescent Work 87

Chapter 12 The Impact of the CSG
on Adolescent Work
12.1 Introduction
Both the household respondent and the adolescent (in a
separate survey) were asked about adolescent work inside
and outside of the home. All adolescents reported doing
some work in the home, so there is little variation in this
adolescent activity to explore. However, there were substantial differences in reports of work outside the home between
the household respondent and adolescent responses in the
behavioural survey. Specifically, the household respondents
suggested that only two per cent of adolescents work outside
the home, while 18.5 per cent of 1,355 adolescents who
answered this question indicated that they worked outside
the home (see Figure 12.1 below). In the analyses of the
impacts of the CSG on adolescent work, adolescent rather
than household responses are used.

Figure 12.1 Reports of adolescent work outside the home
1200

1000

Investigation of the types of work adolescents are engaged
in showed that 59 per cent of 943 adolescents who reported
some work inside or outside the home did not receive any
money in exchange for their labour. The most common type
of work (reported by 11.5% of the adolescents) is cleaning
and washing, and approximately half of adolescents perform
this type of work inside the home (without pay), while
another half do this work outside the home (or both inside
and outside). Female adolescents accounted for approximately 60 per cent of adolescents that are doing cleaning
and washing.
Gardening and cutting is the next most common type of
work among adolescents, performed by 59 adolescents (6%)
in the sub-sample of those working, with 76 per cent performing this work outside the home, mostly by males. Other
types of work include: child care (performed by about 5% of
adolescents, with more than two-thirds of this work done in
the home and 60% by females); hairdressing (by 3%, mostly
females) and car washing (by 3%, mostly males), with more
than three-quarters of these types of work performed outside
the home; and other smaller categories of work activities
(e.g., ploughing, street vending, others).

1087

800

12.2 Adolescent behaviours associated
with work outside the home

600
400
237
200
0

13
Adolescent works outside home

Adolescent does not work outside home

Adolescent and household responses differ
Adolescent and household responses agree
Where adolescent and household responses differ, adolescents are more likely to
report working outside the home.

Household respondents were asked about the times of the
day and days of the week when adolescents worked inside
and outside of the home. However, few household respondents reported adolescent work outside of the home, and so
these responses yielded little useful information. Of interest
in this investigation is whether adolescent work interferes
with their schooling or increases their exposure to risky
behaviours. These associations were examined descriptively
with cross-tabulations and chi-square analyses (see Figure
12.2).
Among the approximately 18 per cent of adolescents working outside the home, chi-square tests suggested that they
are absent for a significantly great number of days from
school and that grade attainment is significantly lower (by

The South African Child Support Grant Impact Assessment

about a quarter of a grade) for this group. In addition, those
working outside the home are significantly more likely to
have ever used alcohol (31% versus 21%) and drugs (approx.
30% versus 20%), to have participated in criminal activities
(22% versus 11%) and to be a member of a gang (11% versus 4%). And although a higher proportion of adolescents
working outside the home (21%) have had sex than those
not working outside the home (17%), this difference was
not statistically significant. There were also no statistically
significant differences in the proportion of adolescents
reporting a pregnancy (7%) by whether or not they worked
outside the home.

Figure 12.2 Adolescent engagement in risky behaviours by
work outside the home

7,2
% ever pregnant

% had sex

3,8
% in gang

% in criminal activity

% consumed drugs

% consumed alcohol

Highest grade attained

1,1
0,68

9,1
9,4

7,1

11,5

10,6

19,6

21,1

22,5

29,5

30,8

Days absent from school

Adolescent does not work outside home
Adolescent works outside home

A cross-tabulation of adolescent reports of work outside the
home with the age at which adolescents started receiving
the CSG (for adolescents with some CSG receipt) suggested
that those who started receiving the CSG at the infant/

pre-school age were less likely to be working outside the
home (13%) than those who did not receive the CSG until
they were 14 years or older (21%), although these differences
were not statistically significant. There were also no statistically significant differences in the proportion of adolescents
working outside the home when compared across the five
treatment subgroups, with the proportions ranging from
15.3 per cent to 19.8 per cent.

12.3 Results of the estimation of CSG
impacts on adolescent work outside
the household
In estimating the impacts of the CSG on adolescent work
outside the home, PSM was used with all possible treatment
subgroup comparisons, and GPS matching was used with
the measure of the age at which CSG receipt for the adolescent began. The analyses were also performed separately
for males and females. Although the estimates in the PSM
analyses were not statistically significant, the patterns of
effects between age at first CSG receipt and adolescent work
outside the home are statistically significant and suggest that
receipt of the CSG at an earlier age reduces outside work,
particularly for females. The basic regression model for the
GPS estimation is: Y(works outside home) = T + GPS +
T*GPS, where the generalised propensity score is estimated
in a first-stage equation including the predictors described
above in section 10.2. Table 12.1 shows the estimated doseresponse effects and their bootstrapped standard errors at
each age of first CSG receipt for all adolescents and for
females and males separately.
The results of the GPS analysis for all adolescents and for
females and males separately are presented graphically in
Figures 12.3, 12.4 and 12.5.

Chapter 12 The Impact of the CSG on Adolescent Work 89

Table 12.1 Dose-response effect results from GPS models of adolescent work outside the home
All adolescents (n = 874)
Age at first receipt
of CSG

Adolescent females (n = 472)

Adolescent males (n = 402)

Dose-response effect

Bootstrapped
standard errors

Dose-response effect

Bootstrapped
standard errors

Dose-response effect

Bootstrapped
standard errors

0.143

0.039

0.074

0.030

0.229

0.080

0.150

0.033

0.085

0.028

0.231

0.067

0.157

0.027

0.099

0.025

0.232

0.054

0.166

0.024

0.114

0.024

0.233

0.044

0.175

0.023

0.131

0.024

0.235

0.038

0.184

0.022

0.146

0.025

0.235

0.034

0.192

0.021

0.157

0.024

0.236

0.032

0.199

0.020

0.163

0.021

0.238

0.029

0.204

0.020

0.164

0.021

0.240

0.028

0.207

0.022

0.162

0.025

0.242

0.030

0.207

0.021

0.158

0.029

0.246

0.032

0.207

0.021

0.156

0.031

0.250

0.033

0.206

0.019

0.157

0.031

0.255

0.033

0.205

0.019

0.162

0.031

0.260

0.034

0.205

0.024

0.171

0.034

0.265

0.041

0.206

0.033

0.183

0.042

0.270

0.053

0.208

0.043

0.197

0.056

0.275

0.068

The South African Child Support Grant Impact Assessment

Figure 12.3 Probability adolescent works outside
household

Figure 12.5 Probability male adolescent works outside
household
.4

Confidence bounds at .95 % level
Dose response function = Probability of a positive outcome
Regression command = logit

Adolescent works outside HH

Confidence Bounds at .95 % level
Dose response function = Probability of a positive outcome
Regression command = logit

0
0

Age started receiving CSG

Figure 12.4 Probability female adolescent works outside
household
.3

Confidence bounds at .95 % level
Dose response function = Probability of a positive outcome
Regression command = logit

10
Age started receiving CSG

Dose response
Upper bound
Low bound

Age started receiving CSG

Dose response
Upper bound
Low bound

Adolescent works outside HH

The dose-response effect estimates shown in Table 12.1
clearly show that the probability of working outside the
home is considerably higher for males, even at early ages
of CSG receipt. The dose-response curve for males (Figure
12.5) is correspondingly much flatter than that for females.
Figures 12.3 (for all adolescents) and 12.4 (for females)
show that the slope of the dose-response curve is steeper
from age 0–7 years (of first CSG receipt), and then it levels
off. For females, there is another slight shift upward in the
slope at around age 13 years, although confidence intervals
are larger at both ends of the age range (where the fraction
of the sample represented is smaller). In summary, connecting children with the grant earlier in their childhood (age
0–7 years) reduces the likelihood that they will work outside
the home (as reported when they are adolescents), and there
appears to be a particularly important protective effect of the
CSG for females who receive the grant in early childhood.

Chapter 13 The Impact of the CSG on Adolescent Risky Behaviours 91

Chapter 13 The Impact of the CSG
on Adolescent Risky Behaviours
13.1 Introduction

and gang membership is an indicator variable for ‘never in
a gang.’

Past studies have found impacts of the CSG on school
attendance, nutrition and child hunger, child labour (measured in hours children spent collecting water and fuel) and
labour force participation in the household (see a discussion
of these studies in Samson et al., 2011). However, this evaluation presents the first opportunity to look in greater detail
at the potential impacts of the CSG on adolescent risky
behaviours. The qualitative component of this evaluation
offered the first, in-depth look at the risks that adolescents
in South Africa face, and through the administration of a
confidential survey of adolescents in the CSG impact evaluation, we also have new data to quantitatively analyse the
scope and implications of these risks to adolescents and the
potential of the CSG to ameliorate them.

We first explored the relationship between receipt of the CSG
and these risky behaviours in descriptive statistics (cross-tabulations with chi-square tests). The results of these analyses
showed statistically significant associations between CSG
receipt (as measured by the five treatment subgroups) and
sexual activity, pregnancy and alcohol use, and for males only,
with gang membership. For each of these outcome measures,
adolescents who were in households currently receiving the
CSG for the adolescent were less likely to engage in these
risky behaviours. Statistically significant associations were not
observed in the descriptive analyses, however, between age at
first receipt of the CSG and these risky behaviours.

In the survey that adolescents self-administered, they were
asked to identify the main problems that teenagers face in
their community. The results are reported in Table 13.1 and
show quite clearly that sexual activity is a major issue, with
nearly two-thirds (65%) of adolescents identifying pregnancy as a main problem. This is followed by alcohol or drug
use, which accounts for another nearly 18 per cent of the
responses.
Table 13.2 (excerpted from Table 20 of the preparatory
qualitative research report, August 10, 2010) confirms that
these potential behavioural risks are widespread for South
African adolescents.
Six main risky behaviours – sexual activity (and number
of sex partners), pregnancy, alcohol use, drug use, criminal
activity and gang membership – are examined in empirical
analyses of CSG impacts. These risky behaviours are measured primarily as binary – 1/0 – outcome variables. Sexual
activity is measured as an indicator that the adolescent ‘never
had sex’ (i.e., sexual intercourse); the number of sex partners
is an interval measure, and pregnancy is an indicator of ‘ever
pregnant.’ Similarly, alcohol use is measured as ‘never drank
alcohol,’ and drug use as ‘never used drugs.’ And finally,
criminal activity is a binary measure of ‘no criminal activity,’

Table 13.1 Adolescent reports of main problems teenagers
face in community
Problems teenagers report that they face in
the community

Frequency

Per cent

Pregnancy

854

56.78

Alcohol or drug use

264

17.55

Gang activity

2.06

Rape or sexual abuse

1.00

Crime and violence

1.60

Harassment

0.20

Prostitution

0.13

Peer pressure

1.46

Lack of money

2.39

Problems with family members

0.93

HIV/AIDS

0.33

Sexually transmitted infections

0.07

Caring for sick household members

0.00

Lack of knowledge on risks

0.33

Dating older men/women

0.40

Children dropping out of school early

1.66

Schools are very bad quality

1.06

The South African Child Support Grant Impact Assessment

•

Meyerton (P)

•

Seshego (P)

•

Groothoek (R)

•

Moletije (R)

•

Umlazi (U)

•

• (W)

•

Merrivale (R)

•

• (W)

•

Port Elizabeth (U)

•

Engcobo (R)

•

Umtata (P)

•

KwaZulu-Natal Izingolweni (P)

Eastern Cape

TOTAL

•

•
5

Prostitution

•

Gangs

Involved with
teachers

•

Power determined
by money

Lenasia (U)

Date older men or
women for money
(W/M)

•

Peer pressure

•

Lack of guidance/
discipline

•

Crime

•

Dating older women

Shoshanguve (U)

Sexual abuse

Limpopo

Dating older men

Gauteng

Drugs and alcohol

Province

Locality
U = Urban
P = Peri-urban
R = Rural

HIV/AIDS and STIs

Table 13.2 Risky behaviours among South African adolescents as identified in the qualitative component of the CSG impact
evaluation

Pregnancy

•

Table 13.3 PSM estimation of impact of CSG on adolescent sexual activity (outcome: never had sex)
All adolescents

Common
support N

Treated

Comparison

T-C difference
(impact)

Std. Error

T-ratio

Group 1 versus 0

767

369

0.887

0.717

0.170*

0.098

1.73

Group 3 versus 0

299

284

0.771

0.644

0.127

0.059

2.15

Group 2 versus 4

295

281

0.842

0.678

0.164

0.066

2.49

Groups 1, 2, 3
versus 0, 4

1231

1195

0.854

0.694

0.160

0.060

2.66

Group 1 versus 0

273

127

0.927

0.949

-0.022*

0.115

-0.19

Group 3 versus 0

167

157

0.753

0.728

0.025

0.085

0.29

Group 2 versus 4

157

134

0.857

0.684

0.174

0.095

1.83

Groups 1, 2, 3
versus 0, 4

675

641

0.871

0.714

0.157

0.072

2.18

Females only

*After-matching balancing tests reject that an acceptable level of balance was attained.

Chapter 13 The Impact of the CSG on Adolescent Risky Behaviours 93

13.2 Results of the estimation of CSG
impacts on adolescent sexual activity
and pregnancy
Approximately 17.5 per cent of the adolescents in this sample
reported having sexual intercourse. Of those who indicated
the frequency of their sexual activity, less than three per
cent had sex more than three times in a month, and 17 per
cent of these had more than one sex partner. Examining the
impact of the CSG on adolescent sexual activity, descriptive
analyses showed strong associations between CSG receipt
and sexual activity, and this relationship was also confirmed
in the PSM and GPS analyses of CSG impacts. Table 13.3
presents the results of PSM models comparing alternative
treatment subgroups for all adolescents and for females only.
As described in Chapter 10, four comparisons were made
among the alternative treatment subgroups, and as expected,
the comparison between group 1 versus 0 is the most tenuous. Specifically, there is minimal overlap (common support)
between adolescents in households that never received the
CSG (group 0, a more advantaged group) and those in
households currently receiving the CSG specifically for the
adolescent (group 1); even after nearly 400 cases are excluded
(more than half of the sample), the balance in covariates
between these two groups is still poor. Thus, although the
results for all adolescents in this comparison look similar to
those for the other groups, they should be discounted.
The results of the comparison of group 3 (households
receiving the CSG for another household member but never
for the adolescent) and group 0 (household never received
the CSG) suggest that even if the CSG is not received for
the adolescent, there is a statistically significant relationship
between current CSG receipt and adolescents abstaining
from sex – that is, those in households with the CSG are
nearly 13 percentage points less likely to have had sex. In
general, these subgroups have lower rates of abstention
from sex, but the CSG still appears to offer some protection
against this risky behaviour when there is at least one grant
in the household; for females, this result is weaker and not
statistically significant.

The average rates of adolescent sexual activity are similar for
the ‘treated’ adolescents in the other two comparisons in Table
13.2, and sexual activity is also at comparable levels in the two
comparison subgroups, so the differences (or estimated CSG
impacts) are likewise close in size in these analyses. The third
comparison (group 2 versus 4) is between households that all
received the CSG for adolescent in the past, but only group 2
households currently receive the CSG (and do so for another
household member). The final comparison defines treatment
as all households currently receiving the CSG (irrespective of
current or past receipt specifically for the adolescent) versus
the two groups (0 and 4) that have no current CSG receipt
(irrespective of past receipt). Across these comparisons, for
all adolescents and for females, it again appears that what is
important is current receipt of the CSG in the household (at
the time of adolescence) to reduce the likelihood that adolescents have sexual intercourse; specifically, adolescents in
households receiving the CSG are about 16 percentage points
more likely to be abstaining from sex.
GPS models were also estimated to explore the relationship
between age at first receipt of the CSG and sexual activity (abstinence from sexual intercourse), and the results are
presented separately for all adolescents and for females in
Figures 13.1 and 13.2 and in Table 13.4. From either eyeballing the graphs or looking at the table of dose-response
effects (Table 13.4), it is clear that adolescent females are
overall more likely to refrain from sexual activity versus
males, and the probability of reporting that they never
had sex is higher when they began receiving the CSG at a
younger age (see Figure 13.2 and Table 13.4). At the same
time, it does appear that the probability of reporting that
they never had sex is also higher for all adolescents, including females, if they began receiving the CSG at age 11 years
or older. This result seems consistent with the PSM findings
that suggest that receipt of the CSG in the household at the
time of adolescence provides important protection against
adolescent engagement in sexual activity.

The South African Child Support Grant Impact Assessment

Table 13.4 Dose-response results from GPS model of adolescent and female adolescent sexual activity (never had sex)
Treatment

Adolescent females (n = 446)

All adolescents (n = 820)
Dose-response effect

Bootstrapped standard errors

Dose-response effect

Bootstrapped standard errors

0.866

0.039

0.920

0.063

0.872

0.030

0.918

0.050

0.876

0.023

0.915

0.038

0.879

0.019

0.911

0.028

0.878

0.018

0.905

0.022

0.874

0.018

0.897

0.020

0.866

0.018

0.887

0.019

0.856

0.018

0.876

0.019

0.844

0.018

0.867

0.020

0.836

0.020

0.861

0.024

0.832

0.021

0.859

0.028

0.835

0.021

0.862

0.029

0.844

0.019

0.869

0.028

0.857

0.020

0.878

0.028

0.871

0.024

0.886

0.031

0.885

0.031

0.894

0.037

0.896

0.038

0.900

0.044

Age at first receipt of CSG

The ‘trough’ in the predicted probabilities of ‘never had sex’
that is seen around age 10 years in the above graphs may
relate to the patterns of CSG receipt for the adolescent
that were discussed in Chapter 10 and shown in Figure
10.2. More specifically, we observed lower proportions of
current CSG receipt in the households of adolescents who
first began receiving the CSG at age 10–13 years. Again,
this seems consistent with the explanation that current CSG
receipt in the household is important to realising the benefit
of the CSG in deterring this risky behaviour in adolescence.
In addition, we saw a similar pattern in the GPS estimation of the impact of the CSG on adolescents’ number of
sexual partners (see Figure 13.3). As the age at first receipt
of CSG increases, the number of sexual partners likewise

increases, but then there is a downturn in the number of
sexual partners for adolescents who start receiving the CSG
in adolescence (ages 14 or older).
One might reasonably expect adolescent sexual activity and
sexual partnering to be associated with the risk of (females)
becoming pregnant. As shown in Table 13.1, adolescents
identified pregnancy as the main problem in their communities. PSM analyses with the comparisons of different
treatment subgroups did not show any statistically significant impacts of the CSG on female adolescent pregnancy.
However, the GPS analysis of the relationship between age
at first receipt of the CSG and pregnancy among adolescents
suggested a pattern of effects that might be unlikely to be
discerned in the PSM estimation.

Chapter 13 The Impact of the CSG on Adolescent Risky Behaviours 95

Figure 13.1 Probability that adolescents never had sexual
intercourse

Figure 13.2 Probability female adolescents never had sex

1.05

Confidence Bounds at .95 % level
Dose response function = Probability of a positive outcome
Regression command = logit

1
.95
Never had sex

Never had sex

.85

Confidence Bounds at .95 % level
Dose response function = Probability of a positive outcome
Regression command = logit

.6
0

10
Age at first CSG receipt

Dose response
Upper bound
Low bound

131. A closer look at the data suggests that the drop-off in current receipt of the CSG
begins with those first receiving the grant at age eight years.

Dose response
Upper bound
Low bound

Figure 13.3 Adolescent number of sex partners

.6
Average number of sex partners

Figure 13.4 shows graphically the results of the GPS analysis, and Table 13.5 presents the dose-response effects and the
bootstrapped standard errors. The pattern of effects is similar
to what was observed for sexual activity and sexual partnering, in that the probability of ever being pregnant is lower at
younger ages of first CSG receipt (up to age five years) and
then it falls again for those who begin receiving the CSG at
age 11 years and later. In fact, the estimated probability of
ever being pregnant is lowest for those who began receiving
the CSG at ages 14–16 years, and the bootstrapped standard
errors on these estimates are also the smallest. These results
appear consistent with the emerging explanation that it is
particularly important to protect against these risks (including pregnancy) with receipt of the CSG at the time that
adolescent risks are presenting or intensifying in the teenage
years. Again, Figure 10.2 shows lower proportions of current
CSG receipt in the household for adolescents who began
receiving the grant at ages 5–9 or 10–13 years.131 However,
given that only 344 female adolescents are included in this
analysis, some caution should be used in generalising from
these results.

10
Age at first CSG receipt

Confidence Bounds at .95 % level
Dose response function = Linear prediction

0
0

10
Age started receiving CSG

Dose response
Upper bound
Low bound

The South African Child Support Grant Impact Assessment

Table 13.5 Dose-response results from GPS model of
female adolescent pregnancy
Treatment

Figure 13.4 Probability ever pregnant for female
adolescents

Adolescent females (n = 344)
Dose-response effect

Bootstrapped standard
errors

.15

0.050

0.043

0.047

0.033

0.046

0.026

0.046

0.021

0.049

0.020

0.055

0.022

0.064

0.024

0.075

0.024

0.086

0.023

0.094

0.022

0.094

0.023

0.088

0.025

0.075

0.022

0.061

0.018

0.047

0.015

0.037

0.015

0.030

0.016

Age at first receipt of
CSG

13.3 Results of the estimation of CSG
impacts on adolescent alcohol and
drug use
For the risky behaviours of adolescent alcohol and drug use,
the patterns of estimated effects are very similar to those
discussed above for adolescent sexual activity and pregnancy.
Rates of alcohol and drug use are higher among males than
females. Approximately 79 per cent of females in the adolescent sample compared to 74 per cent of males have never
used alcohol, while 79 per cent of females compared to 70
per cent of males have never used drugs. In addition, the role
of the CSG in ameliorating these risky behaviours appears
to be stronger for female adolescents than for males, with

Has been pregnant

.05

Confidence bounds at .95 % level
Dose response function = Probability of a positive outcome
Regression command = logit

-.05

Age at first CSG receipt

Dose response
Upper bound
Low bound

the same interesting patterns observed above, where there
is variation in teenagers’ engagement in these behaviours
according to the age at which the CSG was first received
(which is also correlated with current receipt of the CSG in
the household).
No statistically significant average effects of CSG receipt
on adolescent alcohol and drug use were found in the PSM
model estimations that made comparisons between the
various treatment subgroups, either for all adolescents or in
separate analyses for males and females. However, the GPS
models that estimate the relationship between age at first
receipt of the CSG and alcohol and drug use in adolescence
again suggest that the effects of the CSG are varied depending on the timing of first receipt of the grant. Figures 13.5,
13.6 and 13.7 present the GPS results graphically for adolescent alcohol use (i.e., ‘never used alcohol’), including for
females and males separately, and Table 13.6 presents the
dose-response estimates for age at first CSG receipt (0–16
years) for the outcome ‘never used alcohol’. The graph for
females (Figure 13.5) shows the most marked relationship
between early receipt of the CSG and abstinence from
alcohol; the earlier CSG receipt occurs (and particularly

Chapter 13 The Impact of the CSG on Adolescent Risky Behaviours 97

The results of the GPS analyses for the outcome ‘never used
drugs’ are shown in Figures 13.8, 13.9 and 13.10 for adolescents, adolescent females and adolescent males, respectively,
and in Table 13.7. For all adolescents combined, the results
suggest a steady decrease in the probability that an adolescent has never used drugs as the age at first CSG receipt
increases. However, the patterns for males and females
again differ. As for other risky behaviours, early receipt of
the CSG by females appears to provide stronger protection
against drug use in adolescence than it does for males.

Figure 13.5 Probability of no adolescent alcohol use

5
10
Age started receiving CSG

Dose response
Upper bound
Low bound

Figure 13.7 Probability of no male adolescent alcohol use

Never used alcohol

Confidence bounds at .95 % level
Dose response function = Probability of a positive outcome
Regression command = logit

.85
No alcohol use

Figure 13.6 Probability of no female adolescent alcohol use

No alcohol use

before age five years), the more important it appears to be
in averting female adolescent alcohol use. In addition, the
results also suggest that CSG receipt that starts in adolescence likewise forestalls alcohol use among teenagers, and
this relationship is observed for all adolescents.

.7
Confidence Bounds at .95 % level
Dose response function = Probability of a positive outcome
Regression command = logit

.75
.6
.7
0

10
Age started receiving CSG

Dose response
Upper bound
Low bound

10
Age at first CSG receipt

Dose response
Upper bound
Low bound

The South African Child Support Grant Impact Assessment

Table 13.6 Dose-response results from GPS models of adolescent alcohol use
Age at first receipt
of CSG

All adolescents (n = 866)

Adolescent females (n = 460)

Adolescent males (n = 406)

Dose-response effect

Bootstrapped
standard errors

Dose-response effect

Bootstrapped
standard errors

Dose-response effect

Bootstrapped
standard errors

0.813

0.049

0.926

0.040

0.734

0.079

0.813

0.039

0.911

0.038

0.747

0.062

0.811

0.031

0.890

0.034

0.761

0.047

0.808

0.025

0.862

0.031

0.774

0.037

0.802

0.023

0.829

0.029

0.783

0.032

0.794

0.023

0.793

0.028

0.789

0.031

0.784

0.023

0.761

0.029

0.788

0.029

0.773

0.022

0.740

0.029

0.783

0.028

0.763

0.020

0.733

0.028

0.774

0.027

0.756

0.020

0.739

0.030

0.763

0.029

0.753

0.021

0.756

0.034

0.753

0.032

0.755

0.021

0.776

0.036

0.747

0.034

0.761

0.021

0.795

0.037

0.745

0.036

0.770

0.022

0.809

0.038

0.749

0.041

0.780

0.026

0.817

0.042

0.757

0.050

0.790

0.034

0.820

0.051

0.766

0.062

0.798

0.042

0.818

0.062

0.777

0.074

Table 13.7 Dose-response results from GPS models of adolescent drug use
Age at first receipt
of CSG

All adolescents (n = 944)

Adolescent females (n = 504)

Adolescent males (n = 440)

Dose-response effect

Bootstrapped
standard errors

Dose-response effect

Bootstrapped
standard errors

Dose-response effect

Bootstrapped
standard errors

0.788

0.047

0.878

0.052

0.668

0.087

0.787

0.037

0.863

0.044

0.693

0.069

0.786

0.029

0.846

0.036

0.719

0.052

0.786

0.024

0.827

0.030

0.743

0.039

0.786

0.023

0.810

0.028

0.762

0.032

0.785

0.024

0.797

0.029

0.772

0.030

0.782

0.024

0.791

0.029

0.772

0.031

0.778

0.022

0.792

0.028

0.760

0.032

0.771

0.020

0.799

0.026

0.739

0.034

0.763

0.021

0.808

0.026

0.710

0.036

0.754

0.024

0.816

0.026

0.679

0.038

0.743

0.027

0.820

0.026

0.653

0.040

0.733

0.027

0.818

0.026

0.635

0.041

0.724

0.029

0.810

0.029

0.628

0.045

0.716

0.034

0.798

0.038

0.630

0.052

0.708

0.045

0.782

0.054

0.639

0.063

0.701

0.057

0.764

0.072

0.650

0.076

Chapter 13 The Impact of the CSG on Adolescent Risky Behaviours 99

Figure 13.8 Probability of no drug use ever by adolescents

Confidence Bounds at .95 % level
Dose response function = Probability of a positive outcome
Regression command = logit

Confidence bounds at .95 % level
Dose response function = Probability of a positive outcome
Regression command = logit

Never used drugs

Figure 13.10 Probability of no drug use ever by male
adolescents

.6
.6

Age started receiving CSG

Dose response
Upper bound
Low bound

Figure 13.9 Probability of no drug use ever by female
adolescents
1

Confidence bounds at .95 % level
Dose response function = Probability of a positive outcome
Regression command = logit

13.4 Results of the estimation of CSG
impacts on criminal activity and gang
membership

Never used drugs

.6
0

5
10
Age started receiving CSG

Dose response
Upper bound
Low bound

Among adolescents, 14 per cent reported engaging in some
criminal activity (16% of males and 12% of females), and
approximately five per cent reported being in a gang (7.5%
of males and 4% of females). The most common types of
criminal activities reported were stealing and housebreaking
(by more than a third of those who responded to the question asking about the type of criminal activity they engaged
in).
In PSM analyses examining the average treatment effects,
we did not find statistically significant relationships between
CSG receipt and criminal activity or gang membership. We
also explored the relationship between age at first receipt of
the CSG and criminal activity and gang membership. Table
13.8, which shows the probability that adolescents and adolescent males refrain from criminal activity, reveals a gradual
decline in the probability that adolescents do not commit
crimes as age at first CSG receipt increases. This can also

The South African Child Support Grant Impact Assessment

be seen graphically in Figures 13.11 and 13.12, although
the dose-response curves suggest a relatively flatter relationship compared to that between CSG receipt and other risky
behaviours.

Figure 13.11 Probability of no criminal activity, adolescents
.95

Confidence bounds at .95 % level
Dose response function = Probability of a positive outcome
Regression command = logit

Table 13.8 Dose-response results from GPS models of
adolescent criminal activity
Age at first
receipt of
CSG

All adolescents (n = 773)

No criminal activity

.85

Adolescent males (n = 440)

.75

Doseresponse
effect

Bootstrapped
standard
errors

Doseresponse
effect

Bootstrapped
standard
errors

0.877

0.037

0.871

0.068

0.871

0.031

0.863

0.057

0.865

0.026

0.854

0.046

0.859

0.023

0.846

0.037

0.854

0.022

0.839

0.031

0.851

0.022

0.834

0.030

0.851

0.021

0.831

0.031

0.852

0.019

0.829

0.031

0.855

0.018

0.828

0.031

0.858

0.019

0.827

0.033

0.859

0.020

0.825

0.035

0.858

0.021

0.820

0.035

0.855

0.021

0.812

0.033

0.849

0.023

0.802

0.035

0.842

0.029

0.789

0.044

0.833

0.037

0.775

0.061

0.825

0.047

0.761

0.083

5
10
Age started receiving CSG

Dose response
Upper bound
Low bound

Figure 13.12 Probability of no criminal activity, adolescent
males

Confidence Bounds at .95 % level
Dose response function = Probability of a positive outcome
Regression command = logit

.9
No criminal activity

100

Age started receiving CSG

Dose response
Upper bound
Low bound

Chapter 13 The Impact of the CSG on Adolescent Risky Behaviours 101

Of the adolescents who offered a reason for joining a gang,
approximately half reported that it was for money (most
to buy basic necessities, but also to buy expensive things).
Others joined a gang to ‘fit in’, to participate in gang activity, or to get money for drugs. There was little relationship
observed, however, between CSG receipt and gang membership. In the PSM estimation, most estimated differences in
gang membership between treatment subgroups and their
comparisons (impacts) were close to zero. An exception
was the comparison between adolescents who had received
the CSG in the past and for which there was current CSG
receipt in the household, with adolescents who had received
the CSG in the past but there was currently no CSG in
the household; those in households currently receiving the
CSG had lower gang membership by six percentage points,
although the result was not statistically significant. In the
GPS estimation, the coefficient on the treatment measure
(T, age at CSG receipt) was likewise close to zero (0.002)
and the confidence bounds for the dose-response effects
were relatively wide. Keeping in mind that only five per cent
of adolescents reported participating in gangs, we conclude
from this analysis that the CSG likely has little impact on
adolescent gang membership.

early teenage/adolescent years) appeared to be at greater risk
of poorer outcomes (schooling, work and risky behaviours),
which we infer might be related to the fact that this group
of youth is also less likely to be in households with current
CSG receipt at the time of adolescence. This observed pattern of access to the CSG may be a policy implementation
artefact, possibly relating to changes in the age of eligibility
for the CSG over time, where some youth were not reached
at an early age and also did not stay connected with the CSG
through their teenage years. Further exploration of this pattern of CSG access showed that adolescents in Limpopo
and Eastern Cape, followed by KwaZulu-Natal (the three
poorest provinces), had significantly lower rates of current
CSG receipt for adolescents who first began receiving the
CSG between age 10 and 13 years. One important policy
implication might be that greater efforts should be made to
ensure continuous access to the CSG by households with
eligible children through adolescence, so that the potential
benefits of the CSG may be fully realised.

In general, the results of the analysis of CSG impacts on
adolescents suggest the importance of both early receipt of
the CSG by children and receipt of the CSG in the household at the time of adolescence. Early CSG receipt appears
important in protecting adolescents against (or reducing
their engagement in) the risky behaviours of sexual intercourse, alcohol use, drug use and criminal activity, and in
reducing the number of sexual partners and early pregnancy.
In addition, the findings suggested that household receipt
of the CSG in the adolescents’ teenage years is important
in reducing absences from school (particularly for males) as
well as engagement in the above risky behaviours.

Finally, it is important to reiterate some of the limitations of
this analysis. First, some of the fieldworker notes from the
administration of the questionnaires suggested that some
respondents became tense in sections (such as the disclosure
of work and earnings) due to fear that their responses might
jeopardise continued receipt of the CSG or that the data collection was part of increased enforcement efforts by SASSA.
This might explain some of the discrepancies observed
between household respondent and adolescent responses
to several questions on the survey (such as the frequency
of work outside the home by adolescents). A total of 1,504
(87%) of the 1,726 adolescents completed the confidential
adolescent survey, which is a high response rate, but not
all adolescents provided responses to all questions. A large
majority of the non-responses were expected in logical skip
patterns on the questionnaire, but there were also other missing values for observations that were handled in the analysis
as if the nonresponses were at random. This is reflected in
differing numbers of observations included in the analyses of
the impacts of the CSG for the various outcomes.

Adolescents who first began receiving the CSG in the middle age range of childhood (not in the pre-school years or

For all of the propensity score matching (PSM) and generalise propensity score (GPS) analyses, after-matching

13.5 Concluding notes for analysis of
CSG impacts on adolescents

102

The South African Child Support Grant Impact Assessment

balancing tests were performed to assess statistical equivalence across adolescents in the treated and comparison states.
As noted above, balance was not always achieved for some
of the analyses, and there was no treatment subgroup that
compared well with the group of adolescents who lived in
households that never received the CSG (i.e., the apparently
more advantaged group of adolescents). Indeed, in comparisons between the subgroup of adolescents who were in
households currently receiving the CSG for them and those
in households that never received the CSG (the greatest
contrast between treatment and comparison adolescents),
half or more of the cases fell off the common support (i.e.,
were not comparable and were excluded from the analysis),

leading us to disregard these results. The GPS analyses with
age at first receipt of the CSG as the treatment measure
fared better in satisfying the balancing tests, although there
was one covariate in particular that sometimes did not balance for one or more of the treatment intervals: the province
indicator Limpopo. It was expected that balance would be
better for the age at first CSG receipt treatment measure, as
all of the adolescents included in this analysis received the
CSG at some time in their lives. The fact that patterns of
CSG impacts related to the age and timing of CSG receipt
appear to be consistent across a range of adolescent schooling, work and behavioural outcomes also bolsters confidence
in these study results.

Chapter 13 The Impact of the CSG on Adolescent Risky Behaviours 103

104

The South African Child Support Grant Impact Assessment

Part 6

Conclusions

Chapter 14 Conclusions 105

Chapter 14 Conclusions
Returning to the key questions raised in Chapter 1 of this
report, the study finds clear evidence of the positive impact
of the Child Support Grant on the lives of South African
children, adolescents and their households. In terms of how
early enrolment in the Child Support Grant programme
(in the first two years of a child’s life, compared to enrolment after age 4) affects the well-being and development of
children, the study finds positive anthropometric nutritional
impacts as well as attributable improvements in areas of
health and schooling. In terms of the impact of the extension of the Child Support Grant to adolescent children, the
evidence documents attributable impacts in terms of reduced
incidence of risky behaviour and improvements in a range of
developmental outcomes. In terms of the determinants of
Child Support Grant receipt, this research makes a number
of findings to both support the rigor of the impact analysis
as well as provide insight into policy improvements that can
expand access. The following sections summarise the key
findings in these areas from the preceding chapters.

Access to the Child Support Grant
Receipt of the CSG varies over different age groups. Take-up
rates peak for children seven to 10 years in age, while infants
have relatively low take-up rates. Caregivers face a number
of challenges to enrol their children from birth, including
delays in becoming aware of the Child Support Grant,
problems in securing the necessary documents to successfully apply, difficulties in navigating the application process,
and others.
In addition, youth in newly-eligible age groups have
relatively low take-up rates. This finding helps explain why
adolescents are relatively less likely to receive the CSG
when compared to younger children. Receipt of the CSG is
correlated with multiple household re-applications as well
as household knowledge of the CSG from formal sources.
Generally, relatively poorer and/or less educated households
are more likely to have received the CSG. In Limpopo,
however, adolescents who first began receiving the CSG
between age 10 and 13 years have significantly lower odds

of continuing to receive a CSG at age 15 or older, an unexpected result which is a subject of future research.

The impact of the Child Support Grant
on outcomes in early life
Early life receipt of the CSG (in the first two years of life)
increases the likelihood that a child’s growth is monitored132
and improves height-for-age scores for children whose
mothers have more than eight grades of schooling.133 Since
children’s cognitive development depends on receiving
appropriate nutrition in the first few years of life, this result
provides important evidence of the Child Support Grant’s
role as an investment in human capabilities – a critical determinant of multi-dimensional poverty reduction. This also
suggests that a mother’s education complements the Child
Support Grant in strengthening important impacts.
For the full sample, there is no evidence that early receipt of
the CSG affects attendance at crèches or nursery schools.
There is some evidence that early receipt increases the duration of attendance by girls and reduces it for children whose
mothers have eight or more grades of schooling but these
impacts are not precisely measured.

Impact of the Child Support Grant
on schooling and cognitive skills of
children
Analysis of grade attainment, scores on mathematical ability tests and scores on reading and vocabulary tests provides
evidence of the impact of the Child Support Grant on
schooling outcomes of children 10 years old at the time
of the survey. Children who were enrolled in the CSG at
birth completed significantly more grades of schooling than
children who were enrolled at age six, and achieved higher

132. The improvement is 7.7 percentage points, statistically significant at the 10 per
cent level.
133. The improvement in the height-for-age z-score is 0.19 standard deviations, a
large impact significant at the five per cent level.

106

The South African Child Support Grant Impact Assessment

scores on a math test.134 Impacts for girls were particularly
significant, with early receipt of the CSG increasing girls’
grade attainment by a quarter of a grade compared to
those receiving the grant only at age 6. The impact largely
resulted from early receipt of the CSG reducing delays in
girls entering school by 27 per cent, with girls enrolling early
obtaining higher scores on math and reading tests. For children whose mothers have less than eight grades of schooling,
the impacts were even greater. Early enrolment in the CSG
raises grade attainment by 10.2 per cent (0.38 grades). The
CSG appears to play a compensatory role for children with
less educated mothers, narrowing the schooling gap between
children whose mothers have less education and those who
have more. In these ways the Child Support Grant promotes
human capital development, improves gender outcomes and
helps to reduce the historical legacy of inequality.

The impact of the Child Support Grant
on time allocation and labour supply
of children
Analysis of the time allocation and labour supply of 10-yearold children provides evidence of the Child Support Grant’s
impact on the amount of time spent studying, doing chores
or working outside the household. The study finds few
10-year-old children working for pay outside the household.
The timing of CSG enrolment has no statistically significant impact on time spent studying or doing housework.
However, for children in households with no electricity,
early enrolment in the CSG increases the amount of time
spent studying, but the magnitude of this impact is small.

Variation in receipt of the Child
Support Grant among adolescents
The impact of the Child Support Grant
on children’s health
Analysis of current illness- and health-related expenditures
provides evidence of the impact of the Child Support Grant
on child health. Early enrolment in the CSG reduced the
likelihood of illness (as measured by a 15 day period prior
to the survey), with the effect particularly stronger for boys.
Boys enrolled at birth had a 21 per cent likelihood of being
ill, compared to a 30 per cent likelihood for boys enrolled
later.135 Children enrolled at birth whose mothers have
eight or more grades of schooling have a significantly lower
likelihood of being ill, relative to otherwise comparable children enrolled at age six,136 again suggesting that a mother’s
education further complements the Child Support Grant in
strengthening other important impacts, and that these positive impacts are fairly persistent.137

134. The mean increase in grades of schooling was 0.14, and the increase in the mean
math test score was 6.0 per cent.
135. This result was significant at the 10% level.
136. The improvement in this indicator was 8.5 percentage points.
137. At least from birth to age 10, the age threshold used for this part of the study.

A significant pattern identified in the survey data played
an important role in the evaluation of impacts of the Child
Support Grant on adolescents. Adolescents who first started
receiving the Child Support Grant at an early age (four years
or younger), or more recently at age 14 years or older, are significantly more likely to be in households that are currently
receiving the CSG for the adolescent (at the time of the
survey). On the other hand, a comparatively low proportion
of adolescents who first began receiving the CSG between
the ages of 10 and 13 are in households currently receiving
the grant for them, particularly in the province of Limpopo,
which has one of the highest poverty rates in South Africa.
This study finds that important predictors of successful Child
Support Grant receipt by an adolescent’s caregiver include
(1) application for the grant by the adolescent’s biological
mother, (2) the mother of the adolescent being the head of
the household, (3) adolescent awareness of the availability
of the CSG programme, (4) lower educational attainment
for the household head, (5) persistent re-application for the
CSG in the face of initial rejection.

Chapter 14 Conclusions 107

The impact of the Child Support
Grant on schooling outcomes of
adolescents

(1) reduced sexual activity and a fewer number of sexual
partners, particularly when the adolescent also received
the grant in early childhood;

Analysis of adolescent absences from school provides some
evidence of the impact of the Child Support Grant on
schooling outcomes for adolescents. Receipt of the CSG by
the household reduces adolescent absences from school, particularly for male adolescents, even when the household does
not receive the grant specifically for the adolescent. The study
does not find statistically significant impacts of Child Support
Grant receipt on other adolescent schooling outcomes.

(2) reduced pregnancy, again particularly when the adolescent also received the grant in early childhood;
(3) reduced alcohol and drug use, particularly for females,
and with the effect strengthened by early childhood
receipt of the CSG.

General conclusions
The impact of the Child Support Grant
on work inside and outside the home
The households in the sample reported fairly similar
responses about the degree to which adolescents worked
inside the home. However, adolescents and their caregivers
reported very different patterns of work outside the home:
the household respondents indicated that only two per cent
of the sampled adolescents work outside the home, while 18.5
per cent of 1,355 adolescents who answered this question
indicated that they worked outside the home. Early receipt
of the Child Support Grant (in the first seven years of life)
reduces the likelihood that they will grow up into adolescents
who will work outside the home (as reported in the adolescent
survey). Additionally, there appears to be a particularly important impact in terms of reduced work outside of the home for
females who received the grant in early childhood.

The results of this study identify the positive developmental
impact of the Child Support Grant in promoting nutritional,
educational and health outcomes. Early receipt significantly
strengthens a number of these important impacts, providing
an investment in people that reduces multiple dimension
indicators of poverty, promotes better gender outcomes and
reduces inequality. The study also finds that adolescents
receiving the Child Support Grant are more likely to have
some positive educational outcomes, are somewhat less likely
to experience child labour, and are significantly less likely to
engage in behaviours that put their health and well-being at
serious risk. These results convey several key messages:
(1) The Child Support Grant generates positive developmental impact that multiplies its benefits in terms of
directly reducing poverty and vulnerability;
(2) Early enrolment in the Child Support Grant programme
substantially strengthens impacts. Promoting continuous
access to the CSG for eligible children through adolescence would help to maximise the potential benefits of
the grant;
(3) Receipt of the grant by adolescents generates a range
of positive impacts, not least of which is the reduction
in risky behaviours, which in the context of high HIV
prevalence, generates a particularly protective impact.

108

The South African Child Support Grant Impact Assessment

Part 7

Bibliography and
Appendices

Bibliography 109

Bibliography
Agüero, J., Carter, M., & Woolard, I. (September 2007). The
Impact of Unconditional Cash Transfers on Nutrition: the
South African Child Support Grant. Brasilia: International
Poverty Centre Working Paper #39.
Bia, M., & Mattei, A. (2008). A Stata Package for the
Estimation of the Dose-Response through Adjustment
for the Generalized Propensity Score. Stata Journal, 8 (3),
354-373.
Black, R., Allen, L., & Bhutta, Z. (2008, January 17). Lancet.
Budlender, D., & Woolard, I. (2006). The Impact of the South
African Child Support and Old Age Grant on Children’s
schooling and work. Geneva: International Labour Office.
Carneiro, P., & Rodrigues, M. (2009). Evaluating the
Effect of Maternal Time on Child Development Using the
Generalized Propensity Score. University College London,
Mimeo, Department of Economics.
Child Support Grant. (2012). Retrieved 04 20, 2012, from
South Africa Government Sources: http://www.services.
gov.za/services/content/Home/ServicesForPeople/
Socialbenefits/childsupportgrant/en_ZA
(2012). Child Support Grant Impact Evaluation. Data Report,
Reform Development Consulting, Johannesburg.
de Brauw, A., & Hoddinott, J. (2011). Must conditional cash
transfer programs be conditioned to be effective? The
impact of conditioning transfers on school enrollment
in Mexico. Journal of Development Economics, 96 (2),
359–370.
Delany, A., Ismail, Z., Graham, L., & Ramkisson, Y.
( June 2008). Review of the Child Support Grant: Uses,
Implementation and Obstacles. United Nations Children’s
Fund.

(2007). Demographic and Health Survey 2003. Department of
Health, Medical Research Council, OrcMacro. Pertoria:
Department of Health.
(2009). Early Grade Mathematics Assessment (EGMA):
A Conceptual Framework Based on Mathematics Skills
Development in Children. RTI International. Report
prepared for USAID.
Engle, P., Fernald, L., Alderman, H., Behrman, J., O’Gara,
C., Yousafzai, A., et al. (2011). Strategies for reducing
inequalities and improving developmental outcomes
for young children in low-income and middle-income
countries. The Lancet.
(2012). Fact sheet no 2 of 2012: Summary of social grants
distribution in South Africa. Pretoria: South African Social
Security Agency.
Frölich, M. (2004). Finite-Sample Properties of PropensityScore Matching and Weighting Estimators. Review of
Economics and Statistics, 86 (1), 77–90.
Gilligan, O. D., & Hoddinott, J. (2007). Is There Persistence
in the Impact of Emergency Food Aid? Evidence on
Consumption, Food Security, and Assets in Rural
Ethiopia. American Journal of Agricultural Economics, 89
(2), 225–242.
Gilligan, O. D., Hoddinott, J., & Taffesse, A. S. (2009). The
Impact of Ethiopia’s Productive Safety Net Programme
and its Linkages. Journal of Development Studies, 45 (10),
1684–1706.
Hall, K. (2010, July). Income and Social Grants – Child Support
Grants. Retrieved 04 20, 2012, from Statistics on children
in South Africa: http://www.childrencount.ci.org.za/
indicator.php?id=2&indicator=10

110

The South African Child Support Grant Impact Assessment

Heckman, J. J., Ichimura, H., & Todd, P. E. ((1997).
Matching As An Econometric Evaluation Estimator:
Evidence from Evaluating a Job Training Programme.
Review of Economic Studies, 64, 605–654.
Heckman, J., Ichimura, H., & Todd, P. (1998). Matching as
an Econometric Evaluation Estimator. Review of Economic
Studies, 65, 261–294.
Heckman, J., Ichimura, H., Smith, A., & Todd, P. E. (1998).
Characterizing selection bias using experimental data.
Econometrica, 66 (5), 1017–1098.
Hirano, K., & Imbens, G. W. (2005). The Propensity Score
with Continuous Treatments. In A. Gelman, & X.-L.
Meng (Eds.), Applied Bayesian Modeling and Causal
Inference from Incomplete-Data Perspectives: An Essential
Journey with Donald Rubin’s Statistical Family. Chichester,
UK: John Wiley & Sons, Ltd.
Hoddinott, J., Maluccio, J., Behrman, J., Martorell, R.,
Melgar, P., Quisumbing, A. R., et al. (2011). The
consequences of early childhood growth failure over the
life course.
Imbens, G. W., & Wooldridge, J. M. (2009). Recent
Developments in the Econometrics of Program
Evaluation”. Journal of Economic Literature, 47 (1), 5–86.
(2012). International Data Base: South Africa 2012 Population
Pyramid. U.S. Census Bureau, International Data Base.
Kluve, J., Schneider, H., Uhlendorff, A., & Zhao, Z. (2007).
Evaluating Continuous Training Programs Using the
Generalized Propensity Score. Ruhr-Universitat Bochum,
Germany, Department of Economics.
Kruger, J. (1998). From single parents to poor children:
refocusing South Africa’s transfers to poor households with
children. in ISSA’s 2nd International Research Conference
on Social Security.

Leuven, E., & Sianesi, B. (2011, 12 13). PSMATCH2:
Stata module to perform full Mahalanobis and propensity
score matching, common support graphing, and covariate
imbalance testing. Retrieved from http://EconPapers.
repec.org/RePEc:boc:bocode:s432001
Makiwane, M., & Udjo, E. (2006). Is the Child Support
Grant associated with an increase in teenage fertility in South
Africa? Evidence from national survey and administrative
data. Human Sciences Research Council, South Africa.
Maluccio, J. A., & Flores, R. (2005). Impact Evaluation of
a Conditional Cash Transfer Program: The Nicaraguan Red
de Protección Social. Washington, DC: International Food
Policy Research Institute.
Martorell, R. (1999). The nature of child malnutrition and
its long-term implications. Food and Nutrition Bulletin,
20, 288–292.
McEwen, H., Kannemeyer, C., & Woolard, I. ( July 2009).
Social Assistance Grants: Analysis of the NIDS Wave 1
Dataset. Discussion Paper no. 10.
(2009). Promoting Pro-Poor Growth Employment and Social
Protection. Organisation for Economic Cooperation and
Development.
(2009). Promoting Pro-Poor Growth: Social Protection.
Organisation for Economic Co-operation and
Development (OECD).
Rawlings, L. B., & Rubio, G. M. (2005). Evaluating the
Impact of Conditional Cash Transfer Programs. World Bank
Research Observer.
(1996). Report of the Lund Committee on Child and Family
Support. South African Government Information.

Bibliography 111

Riemenschneider, N. (2011). Child Support Grant Impact
Evaluation, 7 challenges: Determining the weights. Draft,
Oxford Policy Management.
Rosa, S., Leatt, A., & Hall, K. (2005). Does the means justify
the end? Targeting the Child Support Grant. Children’s
Institute, University of Cape Town.
Rosenbaum, P. R., & Rubin, D. B. (1983). The central role
of the propensity score in observational studies for causal
effects. Biometrika, 70 (1), 41–55.
Samson, M. J. (2002). HIV/AIDS and Poverty in Households
with Children Suffering from Malnutrition: The Role
of Social Security in Mount Frere. (F. l. Booysen, & S.
Dike, Eds.) The South African Journal of Economics, 70 (7),
1148–1172.
Samson, M., Heinrich, C., Williams, M., Kaniki, S.,
Muzondo, T., Mac Quene, K., et al. (2008). Quantitative
Analysis of the Impact of the Child Support Grant. United
Nations Children’s Fund.
Samson, M., Renaud, B., Miller, E., McTague, E., de
Neubourg, E., Deonauth, T., et al. (Forthcoming).
Feasibility study on the universal provision of the Child
Support Grant in South Africa. Economic Policy Research
Institute. United Nations Children’s Fund, South Africa.
Samson, M., van Niekerk, I., & Mac Quene, K. (2006).
Designing and Implementing Social Transfer Programmes.
Cape Town: EPRI Press.
Samson, M., van Niekerk, I., & Mac Quene, K. (2010).
Rooting social transfers within an evidence-based
approach. In M. Samson, I. van Niekerk, & K. Mac Quene,
Designing and Implementing Social Transfer Programmes
(pp. 29–50). Economic Policy Research Institute Press.

Samson, M., Ndlebe, A, MacQuene, K., van Niekerk, I.,
Ghandi, V., Harigya, T. et al. (2004). The Social and Economic
Impact of South Africa’s Social Security System. Economic
Policy Research Institute. Cape Town: Commissioned by
the Department of Social Development.
Smith, J. A., & Todd, P. E. (2005). Does matching overcome
LaLonde’s critique of nonexperimental estimators?
Journal of Econometrics, 125 (1–2), 305–353.
Smith, J. A., & Todd, P. E. (2001). Reconciling Conflicting
Evidence on the Performance of Propensity-Score
Matching Methods. The American Economic Review, 91
(2), 112–118.
Williams, M. J. (2007). The Social and Economic Impacts of
South Africa’s Child Support Grant. (can be found on www.
epri.org.za as Working Paper #39).

112

The South African Child Support Grant Impact Assessment

Appendix 5.1 Estimating Impact
using Propensity Scores
All matching methods measure programme impact as the average difference in outcomes for treated households minus a
weighted average of outcomes for comparison households where the weights are a function of observables X,
? ATT =

?
1 ? 1
w ( X , i, j ) Y j0 ? , (1)
? ?Yi – ?
n i?T ?
j?C
?

The difference between alternative matching methods centres on to their approach to estimating the weights, w(X, i, j). We
rely on two forms of matching to identify programme impacts, propensity score matching (PSM) and nearest neighbour
matching (NNM). Their estimates of impact can be improved by measuring outcomes for treatment and comparison groups
before and after the programme begins. This makes it possible to construct ‘difference-in-differences’ (DID) estimates of
programme impact, defined as the average change in the outcome in the treatment group, T, minus the average change in
the outcome in the comparison group, C, defined by the evaluation,
ATT
? DID
= ( y1T – y0T )- ( y1C – y0C ) . (2)

The main strength of DID estimates of treatment effects is that they remove the effect of any unobserved variables that
represent persistent (time-invariant) differences between the treatment and comparison group. This helps to control for the
fixed component of various contextual differences between treatment and comparison groups, including depth of markets,
agro-climatic conditions, and any persistent differences in infrastructure development. To take another example, if nonbeneficiary households have stronger average

managerial ability than beneficiaries that is reflected in their level of food
security, the effect of this ability difference on measures of programme impact on food security is removed, or ‘subtracted
out,’ when outcomes are expressed as change in food security. As a result, DID estimates can lead to a substantial reduction
in selection bias of estimated programme impact. Below we describe how propensity score matching constructs a counterfactual comparison group for the evaluation

problem, following Heckman, Ichimura, and Todd (1997) and Smith and Todd
(2001, 2005).
0

Let Yi be the outcome of the ith household if it is a beneficiary of the programme and let Yi be that household’s outcome if it does not receive the programme. The

impact of the programme is given by ? = Yi1 – Yi 0 . However, only Y1 or
Y0 is realised for each household. Let D indicate whether the household receives the programme or ‘treatment’ or receives
it early: D = 1 if the household receives the programme or receives it early; D = 0 otherwise. The evaluation problem is to
estimate the average impact of the programme on those that receive it:
1

ATT

= E ( ? | X , D = 1) = E ( Y – Y | X , D = 1) = E ( Y | X , D = 1) – E ( Y | X , D = 1) ,
1

(3)

where X is a vector of control variables and subscripts have been dropped. This measure of programme impact is generally referred to as the ‘average impact of the

treatment on the treated.’ In expression (1), E( Y0 | X, D = 1) is not observed.
Propensity score matching provides one method for estimating this counterfactual outcome for participants138. Let
P(X) = Pr(D = 1 | X ) be the probability of participating in the programme. Propensity score matching constructs a statistical
comparison group by matching observations on beneficiary households to observations on comparison households with
similar values of P(X). This requires two assumptions:

E (Y 0 | X , D = 1 ) = E (Y 0 | X , D = 0 ) , (4)

138. Rosenbaum & Rubin, 1983.

Appendix 5.1 Estimating Impact using Propensity Scores 113

and

0 < P(X) < 1, for all X. (5) The first assumption, known as ‘conditional mean independence,’ requires that after controlling for X, mean outcomes for nonparticipants are identical to outcomes of participants if they had not received the programme. Expression (A9) assures valid matches by assuming that P(X) is well-defined for all values of X. Covariate matching methods estimate E(Y0 | X, D = 1) by E(Y0 | X, D = 0) using mean outcomes of comparison households matched with beneficiaries directly on the X variables. This procedure is complicated for large X, which is known as the ‘curse of dimensionality.’ Propensity score matching overcomes this problem. Rosenbaum and Rubin show that if outcomes are independent of programme participation after conditioning on X, then outcomes are independent of programme participation after conditioning only on P(X). If (A8) and (A9) hold, propensity score matching provides a valid method for estimating E(Y0 | X, D = 1) and obtaining unbiased estimates of (3). Although it is not possible to test the assumptions in (4) and (5) on nonexperimental data, Heckman, Ichimura, and Todd (1997, 1998) and Heckman et al. (1998) use experimental data to identify the conditions under which propensity score matching provides reliable, low-bias estimates of programme impact. These include that (1) the same data source is used for participants and nonparticipants, (2) participants and nonparticipants have access to the same markets, and (3) the data include meaningful X variables capable of identifying programme participation and outcomes. The survey data used for this analysis were collected expressly for the purpose of this evaluation, making it possible to ensure that each of these conditions is met. Condition (1) is satisfied for all estimates presented. We satisfy condition (2) by restricting matching to households within the same region in many specifications and by adding urban level dummy variables to the X matrix to ensure that unobserved local effects are captured when forming the match. Also, as discussed the data set provides detailed information on maternal and household characteristics believed to be associated with eligibility in the CSG. 114 The South African Child Support Grant Impact Assessment Appendix 5.2 Estimating DoseResponse Functions 1 Let Yi be the outcome of the ith household if it is a beneficiary of an intervention such as the CSG and let Yi 0 be that household’s outcome if it does not receive the programme. The impact of the programme is given by ? = Yi 1 - Yi 0 . However, we only observe the household, and therefore Yi in one state, the household either gets or does not get the programme (or gets it early versus late). Let D indicate whether the household receives CSG transfers (the ‘treatment’): D = 1 if the household receives the programme; D = 0 otherwise. Accordingly, the evaluation problem is to estimate the average impact of the programme on those that receive it: ? ATT = E (? | X , D = 1) = E (Y 1 -Y 0 | X , D = 1) = E (Y 1 | X , D = 1) E (Y 0 | X , D = 1) , (1) where X is a vector of child, maternal and household characteristics that serve as control variables and subscripts have been dropped. This measure of programme impact is generally referred to as the ‘average impact of the treatment on the treated.’ We observe values for the expression E ( Y 1 | X, D = 1) in our data. That is, for households who receive CSG benefits (or receive them early), we do observe outcomes Y 1 given their characteristics, X. The problem we face is that E(Y 0 | X, D = 1) – conditional on X, the outcome values that a CSG child (D = 1) would have received if it had not received programme benefits or received them late, (Y 0 ), is not observed. One way of addressing this problem would be to match households that were similar – that is, they have comparable X’s. While this might be feasible if there were only one or two relevant household characteristics, it is infeasible when the number of elements in X is large (the ‘curse of dimensionality’). As noted in Appendix 5.1, Rosenbaum and Rubin’s (1983) contribution was to show that matching can be made on the basis of the probability (or propensity) to participate in a programme, given the set of characteristics X. Let P(X) be the probability of receiving the CSG at a given age. Using this notation, P(X) = Pr(D = 1 | X ). Propensity score matching constructs a statistical comparison group by matching observations on beneficiary households to observations on non-beneficiaries with similar values of P(X). This requires that: E ( Y 0 |X,D = 1) = E ( Y 0 |X,D = 0), (2) and 0

Order from us and get better grades. We are the service you have been looking for.

Order essay

Type of paper needed:

Academic level:

Deadline:

Pages:

1650 words

Total price: $0.00