Desktop version

Home arrow Management arrow Improving Access and Quality of Public Services in Latin America: To Govern and To Serve

Accountability

Among the accountability variables available in PISA, we found that after controlling for individual characteristics, grade, and school characteristics, there is a positive association between publishing the results and performance in all evaluated areas (Table 5.12).

Estimates indicate that publications of results are significantly positively related to student school progress. This result is valid for the full sample and for the lowest quartile subsample. The estimated association is higher for the lowest quartile, with a substantial estimated marginal effect (0.32, valuated at the means of the rest of the covariates). We also find a statistically significant positive relationship between the use of achievement data to compare the school to district or national performance and school progress. The effect is statistically significant at the mean of the full sample (Table 5.13).

Interestingly, we found a positive association between parents’ influence over staffing and budgeting and school progress, significant in the full sample. In the rest of the accountability variables we did not find conclusive results, except that monitoring teachers in class by the principal seems to be negatively associated with educational outcomes. This would contradict the existing literature on this issue, but it is relative as it is difficult to judge what type of monitoring is taking place in the schools. This variable may reflect reverse causality, that is, the principal monitors less qualified teachers (Table 5.14).

The issue of school and teacher accountability was highly contentious in both countries. While in Chile principals and teachers were acquainted with the notion of accountability and all of the interviewees embraced it as a desirable aspect of any “healthy” education system, in Uruguay the notion was not part of the discourse of actors interviewed and there was a tendency to associate it with the economic transparency in school and not with responsibility of stakeholders in educational achievements. While in Chile the criticism was centered on the ineffectiveness of the existing accountability mechanisms in place, in Uruguay there is very limTable 5.12 Relationship between tests scores and accountability for the full sample, after controlling for individual characteristics and school inputs

Full sample

Science

Maths

Reading

Science

Maths

Reading

Science

Maths

Reading

Achievement data are posted publicly

9.724

11.31*

8.197

5.476

7.269

3.661

7.915*

10.33**

6.492*

Achievement data are used in decisions about

instructional resource allocation to the school

-2.591

-0.483

2.642

-1.455

0.453

3.931

-5.003

-3.660

-0.705

Assessments are used to compare the school to district or national performance

-8.492*

-11.48**

-10.77**

-9.757**

-12.78***

-12.07***

-6.288

-9.562**

-9.338**

(continued)

Full sample

Science

Maths

Reading

Science

Maths

Reading

Science

Maths

Reading

Grade

achievement data are provided to parents in comparison to same grade in other schools

1.898

0.539

1.553

3.850

2.223

3.728

3.318

3.456

2.406

Parental

achievement

pressure

35.19***

40.32***

41.02***

30.17***

35.40***

35.73***

11.30**

12.83**

15.92***

Monitoring of teacher lessons by principal

-10.21*

-15.63***

-12.52**

-7.205

-12.75**

-9.329**

-9.395*

-13.97***

-11.04***

Monitoring of teacher lessons by external inspectors

-10.31*

-6.979

-12.55**

-6.022

-2.854

-7.986*

-7.322

-4.531

-8.316*

Individual

controls

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Grade

No

No

No

Yes

Yes

Yes

Yes

Yes

Yes

School

controls

No

No

No

No

No

No

Yes

Yes

Yes

Table 5.12 (continued)

Full sample

Science

Maths

Reading

Science

Maths

Reading

Science

Maths

Reading

Private-Public

No

No

No

No

No

No

Yes

Yes

Yes

Private

subsidized

No

No

No

No

No

No

Yes

Yes

Yes

Country FE

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Source: Elaborated by authors

Full sample

First quartile SES

Second quartile SES

Third quartile SES

Fourth quartile SES

Corf.

Marginal

Effect

Coef.

Marginal

Effect

Coef.

Marginal

Effect

Coef.

Marginal

Effect

Coef.

Marginal

Effect

Achievement data are posted publicly

0.203**

0.061

0.887***

0.324

0.118

0.041

0.0552

0.016

0.130

0.026

Achievement data are used in decisions about

instructional resource allocation to the school

-0.120

-0.037

0.0527

0.021

-0.215

-0.074

-0.197

-0.055

-0.299

-0.054

Assessments are used to compare the school to district or national performance

0.150**

0.046

0.151

0.059

0.298**

0.104

0.0426

0.012

-0.125

-0.026

Full sample

First quartile SES

Second quartile SES

Third quartile SES

Fourth quartile SES

Corf.

Marginal

Effect

Coef.

Marginal

Effect

Coef.

Marginal

Effect

Coef.

Marginal

Effect

Coef.

Marginal

Effect

Grade

achievement data are provided to parents in comparison to same grade in other schools

-0.148*

-0.047

-0.0768

-0.030

-0.112

-0.040

-0.232

-0.070

-0.0222

-0.005

Parental

achievement

pressure

0.0631

0.019

0.174

0.067

0.500*

0.151

0.00978

0.003

0.397**

0.082

Monitoring of teacher lessons by principal

-0.226***

-0.068

-0.0822

-0.032

-0.364**

-0.122

-0.312**

-0.085

-0.130

-0.025

Monitoring of teacher lessons by external inspectors

-0.142*

-0.045

-0.395*

-0.156

-0.0716

-0.025

-0.113

-0.033

-0.267*

-0.056

Individual

controls

Yes

Yes

Yes

Yes

Yes

Grade

Yes

-

Yes

-

Yes

-

Yes

-

Yes

-

School

controls

Yes

-

Yes

-

Yes

-

Yes

-

Yes

-

Private-public

Yes

-

Yes

-

Yes

-

Yes

-

Yes

-

(continued)

Table 5.13 (continued)

Full sample

First quartile SES

Second quartile SES

Third quartile SES

Fourth quartile SES

Corf.

Marginal

Effect

Coef.

Marginal

Effect

Coef.

Marginal

Effect

Coef.

Marginal

Effect

Coef.

Marginal

Effect

Private

subsidized

Yes

-

Yes

-

Yes

-

Yes

-

Yes

-

Country FE

Yes

-

Yes

-

Yes

-

Yes

-

Yes

-

Obs.

10,206

~

1654

~

2531

~

3039

~

2977

~

Pseudo R2

0.0983

-

0.1707

-

0.1465

-

0.0698

-

0.0528

-

Source: Elaborated by authors

Full sample

First quartile SES

Second quartile SES

Third quartile SES

Fourth quartile SES

Corf.

Marginal

effect

Coef.

Marginal

effect

Coef.

Marginal

effect

Coef.

Marginal

effect

Coef.

Marginal

effect

Parents influence in staffing and budgeting

0.148**

0.045

-0.229

-0.090

-0.0285

-0.010

0.0932

0.027

0.107

0.021

Parents influence in instructional content and assessment practices

0.190

0.056

-0.000704

0.000

-0.220

-0.079

0.280

0.073

-0.106

-0.023

Teachers influence in staffing and budgeting

0.0309

0.009

0.246

0.095

-0.180

-0.065

-0.108

-0.032

0.0357

0.007

Teachers influence in instructional content and assessment practices

-0.0610

-0.019

-0.0406

-0.016

0.291

0.104

-0.170*

-0.049

-0.286**

-0.055

(continued)

Full sample

First quartile SES

Second quartile SES

Third quartile SES

Fourth quartile SES

Corf.

Marginal

effect

Coef.

Marginal

effect

Coef.

Marginal

effect

Coef.

Marginal

effect

Coef.

Marginal

effect

Student

-0.0867

-0.028

0.271

0.104

0.428**

0.135

-0.166

-0.051

-0.115

-0.025

influence in staffing and budgeting Student

-0.0313

-0.010

0.176

0.069

0.0256

0.009

0.0493

0.014

0.0252

0.005

influence in

instructional

content and

assessment

practices

Individual

Yes

Yes

Yes

Yes

Yes

controls

Grade

Yes

Yes

Yes

Yes

Yes

School

Yes

-

Yes

-

Yes

-

Yes

-

Yes

-

controls

Private-public

Yes

Yes

Yes

Yes

Yes

Private

Yes

-

Yes

-

Yes

-

Yes

-

Yes

-

subsidized Country FE

Yes

Yes

Yes

Yes

Yes

Obs.

10801

-

1761

-

2742

-

3125

-

3168

-

Pseudo R2

0.088

-

0.147

-

0.1414

-

0.0651

-

0.0477

-

Source: Elaborated by authors ited discussion around accountability based on student achievement and teacher performance. This tendency would confirm the idea, expressed in the theoretical framework, that the accountability scheme in Chile is based on setting the responsibility over student results on teachers and schools, while the Uruguayan model reflects an idea of social accountability in which students’ results are explained by linkage to social factors.

Accountability? First of all, it is a concept that is unknown in Uruguay, on the part of teachers and even heads of schools. It is assumed that those that have to be accountable are the government authorities; in Uruguay we never hold the school center accountable. (Uruguay/School head/public/big/good result)

When asked about SIMCE and its effects, teachers and principals in Chile were very critical of its limitations and its lack of acknowledgement of contextual variables that intervene in the results. Some concern was expressed regarding the homogenizing parameters against which very different schools were measured. Paradoxically, the discourse that predominated in private schools was that the SIMCE was a very limited measurement of student achievement and that it was taken as one of several indicators to measure their educational success; emphasis was placed on educating critical thinkers, not so much on academic achievement. This could, of course, be because private schools perform considerably better than the rest.

The SIMCE is not the only indicator we use, obviously. In a sense, we are ‘guilty’ of not preparing it like the other schools do; their educational project is preparing them for the test.(Chile/head of school/private/good SIMCE/small)

SIMCE results had a paramount part in the school culture in public and private subsidized schools. Teachers expressed the pressure they felt from parents, schools, and municipal authorities when achievement levels were reported, but, at the same time, they expressed feeling impotent over actually affecting the results through instruction. Additionally, some teachers were quite cynical about SIMCE as an end in itself, rather than a means for detecting areas of improvement. Several of the interviewees referred to the “prepare for the exam” culture that has been in existence since the implementation of SIMCE:

Teachers teach for the SIMCE, they ask certain students to stay at home that day____(Chile/teacher/private school/good results)

The qualitative evidence showed that there is a completely different comprehension of the notion of teacher accountability over results in both national cases. In Chile there is an overall consensus on the part of actors that evaluation of the different stakeholders is not only desirable but also necessary. No one opposed the idea that teachers and principals should be held responsible for educational outcomes but, similar to the pattern found regarding SIMCE, there was a generalized discontent with the way teaching evaluation is carried out. This would not be applicable to the private system in which teachers are subject to a more complex method of evaluation. Additionally, there were several criticisms of the leeway teachers had for manipulating the system of evaluation to their own advantage.

I completely agree with the teacher evaluation. But I have to say, it has errors, for example, there are teachers that don’t do it; they hire other

teacher to do it for them. So it’s not very effective____The class that you

film is a small percentage of the evaluation, and since they know they will be observed, teachers prepare wonderful classes, they even rehearse them....

So, it doesn’t really measure what it seeks to measure. What it does measure is that teachers are acquainted with theories of good teaching; that they are capable of teaching; but it doesn’t measure what teachers do daily, it doesn’t measure if they apply what they know in their everyday teaching. (Chile/ teacher/public/bad SIMCE/big)

The rich amount of evidence collected in Chile regarding accountability contrasts with the virtual lack of reference to this issue in the case of Uruguay. This is due partially to the fact that there is no regular system of assessment of student achievements at the secondary school level. The measurements that are used are those of PISA but at an aggregated level to avoid comparison among schools. With regard to teacher evaluation, the predominant model does not link teacher performance to student results. This is the underlying premise of the social accountability described above.

I think that the idea of being accountable is something that difficult to accept, it’s one of those concepts that has been rejected by teachers; concepts like management, accountability, etc., have always been seen an intromission of economists and business in education.. In school we are always talking about evaluation and assessment; but it’s difficult to accept for our work to be evaluates. (Uruguay/principal/private/good result)

 
Source
< Prev   CONTENTS   Source   Next >

Related topics