(PDF) Inconsistencies in Reported Employment Characteristics ...ftp.iza.org/dp3908.pdf · Regione Veneto Ugo Trivellato Universtiy of Padova, CESifo and IZA Discussion Paper No. 3908 December - DOKUMEN.TIPS (2023)

IZA DP No. 3908

Inconsistencies in Reported EmploymentCharacteristics amongEmployed Stayers

Francesca BassiAlessandra PadoanUgo Trivellato

DI

SC

US

SI

ON

PA

PE

R S

ER

IE

S

Forschungsinstitutzur Zukunft der ArbeitInstitute for theStudyof Labor

December 2008

Inconsistencies in Reported Employment Characteristics

among Employed Stayers

Francesca Bassi University of Padova

Alessandra Padoan

Regione Veneto

Ugo Trivellato Universtiy of Padova,

CESifo and IZA

Discussion Paper No. 3908 December 2008

IZA

P.O. Box 7240 53072 Bonn

Germany

Phone: +49-228-3894-0 Fax: +49-228-3894-180

E-mail: [emailprotected]

Any opinions expressed here are those of the author(s) and notthose of IZA. Research published in this series may include viewson policy, but the institute itself takes no institutional policypositions. The Institute for the Study of Labor (IZA) in Bonn is alocal and virtual international research center and a place ofcommunication between science, politics and business. IZA is anindependent nonprofit organization supported by Deutsche Post WorldNet. The center is associated with the University of Bonn andoffers a stimulating research environment through its internationalnetwork, workshops and conferences, data service, project support,research visits and doctoral program. IZA engages in (i) originaland internationally competitive research in all fields of laboreconomics, (ii) development of policy concepts, and (iii)dissemination of research results and concepts to the interestedpublic. IZA Discussion Papers often represent preliminary work andare circulated to encourage discussion. Citation of such a papershould account for its provisional character. A revised version maybe available directly from the author.

mailto:[emailprotected]

IZA Discussion Paper No. 3908 December 2008

ABSTRACT

Inconsistencies in Reported Employment Characteristics amongEmployed Stayers

The paper deals with measurement error, and its potentiallydistorting role, in information on industry and professional statuscollected by labour force surveys. The focus of our analyses is oninconsistent information on these employment characteristicsresulting from yearly transition matrices for workers who werecontinuously employed over the year and who did not change job. Asa case-study we use yearly panel data for the period from April1993 to April 2003 collected by the Italian Quarterly Labour ForceSurvey. The analysis goes through four steps: (i) descriptiveindicators of (dis)agreement; (ii) testing whether the consistencyof repeated information significantly increases when the number ofcategories is collapsed; (iii) examination of the pattern ofinconsistencies among response categories by means of Goodman’squasi-independence model; (iv) comparisons of alternativeclassifications. Results document sizable measurement error, whichis only moderately reduced by more aggregated classifications. Theysuggest that even cross-section estimates of employment by industryand/or professional status are affected by non-random measurementerror. JEL Classification: C12, C13, C83, J21 Keywords: industry,professional status, measurement errors, survey data Correspondingauthor: Ugo Trivellato Faculty of Statistical Science University ofPadova Via Cesare Battisti 241 35121 Padova Italy E-mail:[emailprotected]

mailto:[emailprotected]

1

1. Introduction and summary* In recent years, labour markets inindustrialised countries have shown quite a high degree ofmobility. Extensive literature on the micro-dynamics of the labourmarket focuses on job-to-job flows (Steven and Haltiwanger, 1999;Fallick and Fleischman, 2004; Shimer, 2005, among many others).Only a few papers explore the kind of job changes that workersexperienced. The complexity of job mobility also demands analysingthese changes while changing employer (Neal, 1999). The literatureon job matching suggests that a significant number of workers whoswitch job also change employment characteristics, mainly industryand occupation (Miller, 1984; McCall, 1990).

Some studies show that job characteristics, particularlyindustry and occupation, collected in surveys are affected bymeasurement error. The effect of these errors is to exaggerate theoccurrence of changes in such characteristics, at least wheninformation is obtained at two points in time with independentinterviews (Bound, Brown and Mathiowetz, 2001). Sala and Lynn(2006) compare estimates of changes in industry and occupationobtained in two survey waves 17 months apart but with differentinterview techniques: traditional independent interviewing vs.dependent interviewing. They show that dependent interviewingresults in lower levels of observed changes and that this shrinkagerepresents a reduction in measurement error, since the effect isparticularly pronounced among respondents who do not change jobbetween waves. Other studies demonstrate that, in general, industryis reported more accurately than occupation and that, notsurprisingly, the agreement rate between employees’ and employers’reports, classified according to a single-digit coding scheme, ishigher than that resulting when reports are categorised accordingto the more detailed three-digit classification (Mellow and Sider,1983; Mathiowetz, 1992).

In this paper we deal with measurement error, and itspotentially distorting role, in information on industry andprofessional status1. As a case-study we consider two-wave panelsone year apart collected by the Italian Quarterly Labour ForceSurvey (QLFS) in the period from April 1993 to April 2003.

In Italy, information on job characteristics can be obtainedfrom various sources. As a prominent example, WHIP (Work HistoriesItalian Panel), a panel built from a sample of micro-data from theadministrative archives of the Italian social security agency(INPS),

* Research for this paper was supported by grants 2003139334 and2005131989 from the Italian Ministry for Education, University andResearch, for years 2004-05 and 2006-07, under the PRIN Programme.Individual anonymised data from the Italian Labour Force Surveywere kindly provided by Istat (Italian statistical agency), under aresearch agreement with the Department of Statistics, University ofPadova. An earlier version of this paper was presented at theEuropean Conference on Quality in Official Statistics 2008, Rome,July 8-11, 2008. We are grateful to Guido Masarotto and AdrianoPaggiaro for insightful suggestions on preliminary drafts, and toconference participants for useful comments. The usual disclaimerapplies. 1 As we will show in Section 2.1, the classification usedby the Italian Quarterly Labour Force Survey questionnaire tocollect information about occupation is a collapsed mixture of twoclassifications: by “occupation”, as defined by the InternationalStandard Classification of Occupation, and by “status ofemployment”, as defined by the International Classification ofStatus of Employment (see ILO, 1993 and 2008, respectively).Following Eurostat (2000), from now on we will call it“professional status”.

2

provides longitudinal information on work histories of employeesin the private non-agricultural sector. Using data for the period1985-1996 and a three-digit ATECO81 classification2, Leombruni andQuaranta (2002) show that 53% of job changes coincide with industrychanges. Mobility among occupations, measured according to a4-category classification3, is lower: 7% of blue-collar workers andexecutives change occupation while changing job; the percentagerises to 17% for white-collar workers. Updated estimates from thesame authors (Leombruni and Quaranta, 2005) document thepersistence of these patterns.

Another fundamental source for the analysis of short-termdynamics and persistence in the Italian labour market is the QLFS.It has the distinctive advantage of referring to a sample of theresident non-institutional population. Thus, it collectsinformation on job characteristics of (almost) all the employed.The survey is cross-sectional with a 2-2-2 rotating design, whichyields two-wave panels one quarter and one year apart (see, e.g.,Trivellato, 1997). The focus of our analyses is on inconsistentinformation on employment characteristics – industry andprofessional status – resulting from yearly transition matrices forworkers who reported that they were continuously employed over theyear and did not change job.

First, we compute and comment upon some usual indicators ofdisagreement. We find clear evidence that there is sizablemeasurement error in both industry and professional status, andthat industry is reported more accurately than professional status.We then expand our analysis in three directions: (i) we testwhether the consistency of repeated information, provided byemployment stayers, significantly increases when the number ofcategories is collapsed; (ii) we explore the pattern ofinconsistencies among response categories using Goodman’s (1968)quasi-independence model; (iii) we compare the appropriateness ofalternative classifications jointly by professional status andindustry.

As regards the detail of variable classification forcross-section estimates (admittedly less demanding than estimatesfrom two-wave panel data), Istat – the Italian statistical agency –provides the following indications. For professional status, areliable classification reduces to a binary one: Employee andSelf-employed. For industry, Istat asserts as dependable aclassification in 12 categories. Based on the hierarchical Kappacoefficient, for each of the two variables, industry andprofessional status, we test if reducing the number of categoriessignificantly increases the consistency of information reported intwo interviews one year apart. Evidence from these analysessupports the first indication by Istat, but casts severe doubts onthe second. Significant results in terms of measurement errorreduction are obtained for a 6- or 5-category classification ofindustry.

We further explore the patterns of inconsistencies amongcategories of variables by testing several specifications ofGoodman’s quasi-independence model, which is almost alwaysrejected. 2 ATECO81 is the old Italian version of NACE(Nomenclature statistique des Activités économiques dans laCommunauté Européenne). 3 The 4-category classification foroccupation consists of Executives, White-collars, Blue-collars andApprentices.

3

Lastly, we consider and compare alternative 4-categoryclassifications obtained by collapsing professional status andindustry into a single variable. The standard classification labelsrespondents as Self-employed, Employee in agriculture, Employee inindustrial sector, and Employee in services. As an alternative,another 4-category classification was recently used by Trivellatoet al. (2005) in their study of worker turnover, with the fourcategories given by Self-employed, Employee in agriculture,Employee in industrial sector and private services, and Employee inPublic Administration and social services. Interestingly enough,the latter classification turns out to be almost uniformly betterthan the former, standard one.

The paper proceeds as follows. Section 2 contains a briefdescription of the data and presents the methods and design of theanalyses. Section 3 reports the main results. Section 4concludes.

2. Data and methods 2.1. Data

As already mentioned, QLFS is a quarterly survey with a 2-2-2rotating design4. It collects information about labour marketparticipation on a sample of respondents from the residentnon-institutional population. For all persons who declarethemselves as employed or report that they worked at least one hourduring the reference week, the questionnaire includes a series ofquestions on employment characteristics: working hours,professional status, industry, place of work, number of personsworking at the local unit, type of contract, and date in whichperson started working for the current employer or in the currentactivity.

Professional status is identified by means of a closed-formquestion with 11 categories, 6 for employees (Manager, Executive,Clerk, Workman, Apprentice, Outworker) and 5 for self-employed(Entrepreneur, Professional, Own-account worker, Member of aproducers’ cooperative, Contributing family worker). Information onindustry is collected with an open-ended question and coded by theinterviewer according to the ATECO20025 classification. As alreadynoted, Istat (2003) warns that these two variables may not betotally reliable, if used at their maximum degree of detail. Itsuggests using the binary classification, Employee/Self-employed,for professional status, and a 12-category classification,corresponding to the two-digit ATECO2002, for industry – the 12categories are listed in Table 4, column 1.

4 The description of the survey given here applies until 2003.In 2004, the survey was substantially redesigned. The main changeregards the timing of interviews: while up to 2003 QLFS interviewstook place only in one week – usually the second – of each quarter,with the new Continuous LFS interviews spread out over all weeks ofthe quarter. Other notable changes regard the largely newquestionnaire and the mode of interview: from paper and pencil toCAPI-CATI (Istat, 2004). 5 ATECO2002 is identical to NACE Rev. 1.1at four-digit level.

4

In this paper we use 10 two-wave yearly panels, from April 1993to April 2003. Among workers who were continuously employed duringthe year – yearly employment stayers - we consider only those whodid not change job, for a total of 263,584 sample units, around25,000 for each panel. We adopted a quite strict criterion foridentifying these workers: they are those respondents who, in bothone-year-apart interviews, were classified as employed and reportedthe same date when answering the question: “When did you startworking with the current employer or in the currentself-employment?”6.

Following de Angelini and Giraldo (2003), we considerinconsistencies in job characteristics reported one year apart asdue to measurement errors affecting these variables, and assumethat no or negligible errors are made in reporting dates. We alsoassume that, among yearly employment stayers who did not changeemployer or current self-employment, genuine levels of change inindustry and/or professional status were likely to be very low(Sala and Lynn, 2006). Thus, operationally for those workers allobserved change in industry and/or professional status isattributed to measurement error (Mathiowetz and McGonagle,2000).

As regards professional status, measurement error is likely tobe due to the detailed classification offered to respondents; forindustry, mainly to the nature of the open-ended question used tocollect information. Minor changes in the wording used byrespondents to describe their branch of activity or in recodinginformation by the interviewer might have led to a differentindustry classification, when in fact no change occurred.

2.2. Methods and design of analyses

Our study consists of analysing inconsistencies in industry andprofessional status reported in two independent interviews one yearapart by workers who were (reasonably assumed to be) continuouslyemployed and did not change job. The analysis develops alongseveral lines.

First, the usual descriptive indicators to assessinconsistencies are computed and compared across the two variablesand over time.

Transition matrices among job characteristics declared one yearapart provide the basic information for quantifyinginconsistencies. As an example, Tables 1 and 2 report thetransition matrices by industry and professional status,respectively, of the April 1993-April 1994 panel. The frequencieson the main diagonal refer to consistent responses, while thoseoutside the main diagonal point to inconsistencies.

A simple indicator of disagreement is the percentage offrequencies outside the main diagonal (P), or gross difference rateas defined by Hansen, Hurwitz and Pritzker (1964) in their seminalpaper.

6 The criterion is conservative: in accordance with the aim ofour work, it is meant to minimise the risk of including falsepositives. Thus, we decided to eliminate (i) records with missingdata regarding dates, and (ii) records with dates which differed inthe two interviews, although they were consistent with the factthat the worker was continuously employed during the year and didnot change job.

5

Table 1: Transition matrix by industry, April 1993 to April1994

1994 1993 Agri-colture

Min-ing

Manu-factur-ing

Con-struc-tion

Whole-sale

Ac-co-mod.

Trans-porta-tion

Finan-cial act.s

Profes- sional act.s

Public.admin.

Educa-tion

Other public act.s

Agricolture 1,624 4 26 8 17 4 2 1 3 30 7 4 Mining 1 270 32 22 140 2 4 2 8 1 1 Manufacturing 16 28 5,315 87 179 15 41 11 43 29 37 36Construction 12 12 90 1,721 28 0 16 6 27 17 12 21 Wholesale 24 6171 55 3,648 40 31 6 22 19 28 34 Accomodation 2 1 6 5 26 705 10 0 06 15 14 Transportation 11 2 47 22 31 4 1,306 10 4 44 11 21Financial act.s 2 5 17 8 21 2 10 777 12 12 11 10 Profess. act.s 2 239 39 32 3 9 23 814 22 22 65 Public admin. 23 11 37 22 25 10 48 1312 2,098 164 27 Educ.& helth 7 2 25 10 28 13 10 10 11 108 3,18838 Other public & social act.s

9 5 25 14 39 19 25 8 36 40 58 938

Table 2: Transition matrix by professional status, April 1993 toApril 1994

1994 1993 Man-ager

Execu-tive

Clerk Work-man

Appren-tice

Out-worker

Entre-preneur

Profes-sional

Own-account

Coop’s member

Contr. family

Manager 243 89 39 14 0 0 6 7 4 0 0 Executive 65 684 218 11 0 0 011 4 0 1 Clerk 55 284 6,790 465 0 2 13 31 44 8 12 Workman 15 16 5577,882 25 11 8 6 138 22 35 Apprentice 0 0 11 79 98 1 0 1 2 0 1Outworker 0 0 4 18 0 29 0 1 5 0 0 Entrepreneur 4 0 6 4 0 0 238 21123 6 10 Professional 6 13 36 6 0 1 7 641 93 2 3 Own-account w. 4 440 118 2 6 102 89 4,353 74 102 Coop’s member 0 0 6 14 0 0 8 3 54115 7 Contr. family w. 0 3 36 36 5 1 12 7 117 14 767

6

Cohen’s (1960) Kappa coefficient, )1/()( eeo pppK −−= , where pois the observed proportion of agreement and pe is the proportion ofagreement expected under independence of answers, is awell-established index of agreement. It is based on the comparisonbetween observed counts on the main diagonal of the matrix with thecorresponding expected cell counts under the model of independence.It ranges from –1 (total disagreement) to 1 (perfect agreement).Zero is obtained when agreement is totally due to chance7 8.

Second, we want to ascertain if reducing the number of thecategories in which the variables are coded significantly increasesthe consistency of information reported in two subsequentinterviews. We test that by using the hierarchical Kappacoefficient (Cohen, 1968).

The hierarchical Kappa coefficient provides a framework forinvestigating if patterns of disagreement pertain primarily tointerchanges among similar response categories, as opposed tosubstantively important misclassification. Partial credit ispermitted for a certain type of disagreement, which impliesassigning a set of weights to some specific matrix cells. Forexample, the weights can be chosen so that the associated Kappameasures indicate the increments in agreement which result bysuccessively

combining relevant categories of the response variable. In thiscase: ��= =

=I

i

I

iiiiio pwp

1 1''' and

��= =

=I

i

I

iiiiie ppwp

1 1''..' , where wii’ is the weight assigned to the ii’ cell,pii’ is the proportion

7 Other indicators suggested by the literature, given atransition matrix with absolute frequencies, are (see, e.g., Hauserand Massagli, 1983): (i) The net difference rate

. . ..[( ) / ] 100

i i ie X X X= − × , where X.i is the ith

marginal column sum, Xi. is the ith marginal row sum, and X.. isthe total count. This expression is simply the difference betweeninterviews in the marginal proportions in the same categories. Itranges from –100 to 100, 0 is obtained when marginal proportionsfor a category are exactly the same. Its limitation is that theremay appear to be significant differences in marginal proportions inseveral categories as a result of a smaller number of netclassification differences. (ii) The index of inconsistency:

( ) ( ) ( )[ ]{ } 100/../2 .......... ×−+−−+= XXXXXXXXXXIiiiiiiiii , where Xii is the ith diagonal entry (for a criticalreview of this index, see Biemer, 2004). This is the ratio ofobserved discrepancies (off-diagonal counts) in a given category tothose discrepancies expected under simple independence. It rangesfrom 0 – no inconsistencies – to 100 – complete randomness amonganswers. Its major defect is that the model of independence almostnever fits repeated measurements. We calculated the net differencerate and the index of inconsistency on our transition matrices, andobtained results fully consistent with those revealed by indexes Pand K, shown in Table 3. 8 Cohen’s Kappa is especially appropriatein the medical sciences, where studies are often designed to assessthe agreement between different raters or different diagnosticinstruments. If the two readings are from two different raters, Kaccounts for rater bias; if the two readings are from replicatedmeasurements, an intraclass correlation coefficient is appropriate,since we may assume no bias (Barnhart and Williamson, 2002). In ourcase, we aim at measuring agreement between responses obtained onthe same sample in interviews one year apart: the interviewer isnot usually the same; in addition, the respondent might alsochange, since proxy respondents were allowed, and indeed frequent(see, e.g., Gandolfo and Gennari, 2000, who document that, in theperiod April 1998 to January 1999, the rate of proxy respondents inthe QLFS was on average slightly over 40%). For these reasons, weprefer Cohen’s Kappa to the intraclass correlation coefficient.

7

observed in cell ii’ and pi.p.i’ is the proportion expectedunder the model of independence (Koch et al., 1977).

The problem is formalised as a simple test of hypothesis. Let1K̂ and 2K̂ be two hierarchical Kappa coefficients, estimated withtwo different sets of weights so that the

second one implies a less disaggregated classification, thehypothesis test 12 ˆˆ: KKH o = vs.

121ˆˆ: KKH > allows one to verify if aggregating categoriesimproves significantly

agreement (Landis and Koch, 1977). In our study, the weightswii’ are chosen so that they imply aggregation among categoriesidentifying similar employment (industry or professional status)characteristics. The weights have value 1 for cells in which thereis perfect agreement (on the main diagonal) and for cells outsidethe main diagonal linking similar categories – whose observedfrequencies are considered as agreements, value 0 for all othercells.

Finally, the patterns of inconsistencies among responsecategories, at various levels of disaggregation, are explored byestimating log-linear models of quasi-independence.

Log-linear models can be usefully applied in order to detectinconsistencies in contingency tables (Hagenaars, 1990). Inparticular, the model of quasi-independence is used to evaluate if,leaving aside the main diagonal cells, the remaining cells showparticular systematic patterns of association or whether there isindependence on this (truncated) table. In the model ofquasi-independence, the entries on the diagonal cells of atransition matrix are blocked, and the model of independence isspecified for the off-diagonal cells (Goodman, 1968). Theexpression of the log-linear model of quasi-independence for an IxItable is the following:

ijjiijF µµµµ +++=log ,

where ijF is the expected frequency in the generic cell of thetwo-way contingency table;

µ is the grand mean; iµ are row effects with Ii ≤≤1 ; jµ arecolumn effects with Ij ≤≤1 , and ijµ are interaction effects fordiagonal cells, 0=ijµ if ji ≠ .

In our case, assuming quasi-independence implies that errors inreporting industry or professional status are independent in twointerviews one year apart. Rejecting the model implies thatinconsistencies do not occur randomly: rather, there are systematicpatterns of associations among response categories. Afterestimating the model, a close inspection of residual frequenciesmay give information on the sizes of associations.

3. Main results 3.1. Descriptive evidence

For the 10 two-wave yearly panels – from April 1993 to April2003 – of yearly employment stayers who did not change job, asdescribed in Section 2.1, Table 3 contains the values of thedescriptive statistics of inconsistency: the percentage offrequencies

8

Table 3: Measures of inconsistencies with reference to industryand professional status

Industry Professional status Panels P * K** P * K**

93-94 11.8 0.8672 14.0 0.8132 94-95 10.6 0.8785 13.1 0.825595-96 10.9 0.8750 13.1 0.8265 96-97 9.5 0.8915 12.5 0.8349 97-989.7 0.8896 12.9 0.8297 98-99 10.6 0.8787 12.9 0.8301 99-00 10.90.8758 13.1 0.8276 00-01 10.9 0.8754 13.7 0.8195 01-02 10.3 0.882212.5 0.8346 02-03 9.7 0.8892 12.4 0.8355

* P is the percentage of frequencies outside the main diagonal.** K is the Cohen’s Kappa coefficient.

outside the main diagonal (P) and Cohen’s Kappa (K), withreference to industry classified into 12 categories, as recommendedby Istat, and to professional status classified into 11 categories,as in the questionnaire.

It is worth noting that industry is reported with fewerinconsistencies than professional status, according to bothindices9. Another interesting piece of evidence is that there is nosignificant trend in the indices of inconsistencies: the effect ofmeasurement error in the survey is fairly constant over thedecade.

Looking at the indices calculated for the various categories ofthe two variables – not reported here for the sake of space10 – itemerges that the most consistent categories of professional statusare Clerk and Workman among employees, and Professional andOwn-account worker among self-employed. In industry, Agriculture,Mining and raw material extraction, Professional and supportservice activities, and Other public, social and personal serviceactivities are reported with the least inconsistencies.

In order to interpret the values of Cohen’s Kappa, two scalesare mainly used in empirical studies. The scale proposed by Fleiss(1981) defines as marginal agreement values of the coefficientwhich are lower than 0.40, good agreement values between 0.41 and0.75, and excellent agreement values over 0.75. Landis and Koch(1977) propose to consider as slight agreement values lower than0.20, fair agreement those between 0.21 and 0.40, moderateagreement values between 0.41 and 0.60, and substantial agreementin

9 As already mentioned, this result can also be found in otherstudies. However, in our case study it is slightly puzzling, sinceprofessional status was reported by answering a closed-formquestion, whereas industry was asked by means of an open-endedquestion and answers were afterwards coded by the interviewer.Literature on measurement errors in surveys, specifically inreporting job characteristics, shows that inconsistencies over timeare more likely when information is collected with open-endedquestions (Mathiowetz and McGonagle, 2000). 10 Available from theauthors on request.

9

cases of values between 0.61 and 0.80. A value greater than 0.80denotes almost perfect agreement.

As shown in Table 3, for both variables – industry andprofessional status – the values of Cohen’s Kappa are quite high,resulting in almost perfect agreement according to both scales. Allcoefficients are also statistically different from zero. However,the peculiarity of our case should be taken into account. Indeed,in the absence of measurement error we expect no inconsistencies atall between information reported in two subsequent interviews.First evidence of non-negligible inconsistencies in our data is thepercentage of frequencies outside the main diagonal of thematrices, around 10% for industry and 12-14% for professionalstatus. In addition, under the hypothesis of no inconsistencies,one should expect a Kappa coefficiente equal to or very close to 1,which does not seem to be the case11. Overall, we interpret theseresults as indicative of sizable measurement errors.

We proceed therefore to verify if aggregating categories wouldimprove agreement, i.e., significantly diminish the percentage ofinconsistent information.

3.2. A strategy for testing a sequence of less disaggregatedclassifications

As already explained, the procedure based on hierarchical Kappacoefficients is appropriate for assessing the pattern of agreementamong two or more classifications of some categorical responsevariables. A sequence of hierarchical Kappa coefficients refers toprogressively less stringent, usually nested, definitions ofagreement. The values of the coefficient obtained yield largervalues for corresponding broader views of agreement. Since Kappacoefficients have an approximate multivariate normal distributionfor large samples, it is possible to test the significance ofsuccessive differences by means of Wald statistics.

Hierarchical Kappas are formulated using sets of criterionweights. In our application, the first set of weights defines theagreement as the occurrence of the same response category in bothinterviews. The other sets of weights correspond to more aggregatedclassifications, which also consider as agreement the occurrence,in two consecutive interviews, of responses which are different butbelong to similar categories. Similar categories are treated asequivalent, yielding to a less stringent definition of consistency.The aggregation process, described in Tables 4 and 6, was basedboth on judgment and on evidence on the distribution and size ofinconsistencies across categories documented by the descriptiveanalyses.

For industry, the first set of weights is given by the12-category classification recommended by Istat. The second setresults in a 6-category classification, as shown in column 2 ofTable 4. The third set of weights corresponds to a 5-categoryclassification, derived from the previous one as shown in column 3of Table 4. The last set of weights

11 Note that it would be quite complicated to build atest-statistic to ascertain the hypothesis that the empirical Kappacoefficients were significantly different from 1, since it wouldimply testing a parameter value at the boundary of the parameterspace.

10

Table 4: Scheme of categories aggregation process: industry

12 categories 6 categories 5 categories 3 categories AgricultureAgriculture Agriculture Agriculture Mining and raw materialextraction Manufacturing

Manufacturing and mining

Manufacturing and mining

Construction Construction Construction

Industrial sector

Wholesale and retail trade Wholesale and retail trade Wholesaleand retail trade Accommodation and food services Transportation andcommunications Financial and real estate activities Professionaland support service activities

Services

Public Administration, defence and compulsory social securityEducation, health and other services Other public, social andpersonal service activities

Public Administration

Other activities

Services

Table 5: Hierarchical Kappa coefficients and Wald tests:industry

Kappa coefficients Wald tests Panels

12 categories

6 categories

5 categories

3 categories

6 vs. 12 5 vs. 6 3 vs. 5

93-94 0.8672 0.8833 0.8940 0.9020 217.75*** 96.26*** 26.59***94-95 0.8785 0.8939 0.9037 0.9113 174.26*** 71.25*** 21.41** 95-960.8750 0.8899 0.8989 0.9005 193.51*** 71.79*** 1.17 96-97 0.89150.9044 0.9104 0.9159 165.64*** 38.77*** 14.39 97-98 0.8896 0.89820.9044 0.9082 81.08*** 34.85*** 6.19 98-99 0.8787 0.8894 0.89640.9008 107.82*** 40.62*** 7.65 99-00 0.8758 0.8880 0.8931 0.8944132.95*** 21.90** 0.65 00-01 0.8754 0.8865 0.8883 0.8903 109.25***2.91 1.39 01-02 0.8822 0.8910 0.8926 0.8943 80.50*** 2.47 1.0802-03 0.8892 0.9008 0.9046 0.9044 131.86*** 14.05 0.02

* Significant at �=0.1, ** significant at �=0.05, ***significant at �=0.01.

11

implies the usual 3-category classification: Agriculture,Industrial sector, and Services (column 4 of Table 4).

Table 5 contains the values of the Kappa coefficients calculatedusing the four sets of classifications by industry on the 10panels, and the results of Wald tests performed on the differencesbetween each hierarchical Kappa coefficient and the onecorresponding to a more aggregated classification12. Switching from12 to 6 categories significantly improves agreement among responsesin all 10 panels; reducing further categories to 5 significantlyimproves agreement in 7 out of 10 panels; no significant increaseis obtained when reducing answers to the usual 3-categoryclassification. From these results, it appears that the twoclassifications that minimise inconsistencies in information onindustry collected in the QLFS are those with 6 or 5 categories.This evidence challenges the Istat’s recommendation to use the12-category classification.

For professional status, the first set of weights consists ofthe 11-category classification used in the questionnaire. Thesecond set results in a 6-category classification obtained byaggregation, as in column 2 of Table 6. The last set of weightscorresponds to the binary classification recommended by Istat:Employee and Self-employed (column 3 of Table 6).

Table 7 is similar to Table 5, and contains the Kappacoefficient values calculated using the three sets ofclassifications by professional status on the 10 panels, as well asthe results of the tests performed on the differences betweenhierarchical Kappa coefficients. Switching from 11 to 6 categoriessignificantly improves agreement among responses in all

Table 6: Scheme of categories aggregation process: professionalstatus

11 categories 6 categories 2 categories Manager ExecutiveClerk

White-collar

Workman Apprentice

Blue-collar

Outworker Outworker

Employee

Entrepreneur Professional Own-account worker

Self-employed

Member of a producers’ cooperative

Member of a producers’ cooperative

Contributing family worker Contributing family worker

Self-employed

12 Since Kappa coefficients have an approximated multivariatenormal distribution for large samples, chi-square tests for linearhypotheses about them can be carried out with Wald statistics.

12

Table 7: Hierarchical Kappa coefficients and Wald tests:professional status

Kappa coefficients Wald tests Panels

11 categories 6 categories 2 categories 6 vs. 11 2 vs. 6 93-940.8132 0.8709 0.9317 1,035.71*** 621.99*** 94-95 0.8255 0.88030.9371 816.87*** 486.58*** 95-96 0.8265 0.8804 0.9361 931.28***560.05*** 96-97 0.8349 0.8904 0.9402 965.87*** 487.31*** 97-980.8297 0.8850 0.9406 927.84*** 552.70*** 98-99 0.8301 0.8863 0.9398922.88*** 512.05*** 99-00 0.8276 0.8816 0.9388 874.81*** 565.95***00-01 0.8195 0.8764 0.9317 893.42*** 481.62*** 01-02 0.8346 0.89110.9413 922.73*** 466.68*** 02-03 0.8355 0.8894 0.9432 893.83***527.38***

*** Significant at �=0.01.

panels13; reducing categories to 2 further increases agreement.The Istat’s recommendation to classify professional status with adichotomous variable, Employee/Self-employed, is neatlyconfirmed.

3.3. Does the quasi-independence model hold?

In the case of industry, the log-linear model ofquasi-independence was estimated so to reproduce the 12-, 6-, 5-and 3-category classifications. Table 8 lists the results of modelfitting (Pearson X2 and log-likelihood ratio L2 statistics andassociated p-values and BIC index) for the 10 panels. Thehypothesis of quasi-independence is always rejected, for all panelsand at all levels of aggregation, indicating that one-year-apartresponses show non-random association even at the maximum level ofaggregation (3 categories).

Although the model of quasi-independence does not fit, the BICindex gives some interesting results. It reaches its minimum valuewith the 6-category classification in 7 out of 10 panels,indicating that this is the level of aggregation which fits thedata best, and with the 5-category classification in the remainingthree panels. This evidence confirms the conclusions reached byapplying the hierarchical Kappa procedure.

Estimated residuals of a quasi-independence model measureassociation exceeding that expected under random behaviour. Thegreater their value, the higher the difference between the observedassociation and that expected under the null hypothesis ofrandomness. Specifically, positive estimated residuals indicatethat the model underestimates association between the twocategories involved; negative estimated residuals indicateoverestimation. Inspecting estimated residuals of thequasi-independence model implying 3 categories, i.e., testingquasi-independence among inconsistencies which

13 In Section 2.1 we assumed that, among continuously employedworkers who did not change job, genuine levels of change inprofessional status (and/or industry) were likely to be negligible.In any case, with the 6-category classification, true transitionsamong professional statuses for those workers are definitelyimplausible.

13

Table 8: Goodness-of-fit statistics for quasi-independencemodel: industry

Panels Number of categories

X2* L2* Degrees of freedom

BIC index

93-94 12 1,310.95 1,180.35 109 74.87 6 651.81 596.52 89 -306.125 350.64 308.96 65 -350.27 3 202.72 190.25 47 -286.42 94-95 121,117.23 993.54 109 -93.73 6 557.06 514.57 89 -373.20 5 234.53219.63 65 -428.75 3 111.99 108.30 47 -360.52 95-96 12 1,073.58965.40 109 -138.19 6 520.16 501.48 89 -399.61 5 253.15 240.34 65-417.77 3 145.81 145.19 47 -330.67 96-97 12 1,038.38 896.49 109-206.29 6 411.06 393.80 89 -506.63 5 220.96 212.57 65 -445.05 3117.53 119.77 47 -355.74 97-98 12 776.94 692.80 109 -407.74 6359.91 365.29 89 -533.31 5 240.31 229.31 65 -426.97 3 111.58 111.9547 -362.59 98-99 12 990.77 887.54 109 -210.85 6 511.34 489.06 89-407.79 5 277.61 273.25 65 -381.76 3 138.15 142.21 47 -331.40 99-0012 1,087.13 997.56 109 -102.56 6 542.33 524.01 89 -374.25 5 284.80273.11 65 -382.92 3 183.87 174.50 47 -299.86 00-01 12 976.55 861.55109 -234.43 6 360.33 351.77 89 -543.10 5 204.88 197.68 65 -455.88 3125.39 131.36 47 -351.22 01-02 12 949.90 841.43 109 -258.07 6470.26 426.90 89 -470.86 5 233.38 219.82 65 -435.85 3 120.53 121.4547 -352.65 02-03 12 952.57 873.38 109 -228.43 6 434.87 428.31 89-471.33 5 244.36 235.23 65 -421.81 3 113.92 115.33 47 -359.76

* All p-values are lower than 0.0001.

14

Table 9: Goodness-of-fit statistics for quasi-independencemodel: professional status

Panels Number of categories

X2* L2* Degrees of freedom

BIC index

93-94 11 3,941.76 3,452.42 89 2,549.77 6 1,716.59 1,300.78 75540.12 2 252.24 240.00 39 -155.53 94-95 11 3,248.49 2,765.17 891,877.40 6 1,332.34 981.83 75 233.71 2 180.09 177.74 39 -211.2895-96 11 3,816.64 3,182.43 89 2,281.33 6 1,532.71 1,143.73 75384.38 2 210.07 207.18 39 -187.68 96-97 11 3,496.75 3,051.29 892,150.86 6 1,346.41 1,047.90 75 289.11 2 202.00 209.37 39 -185.2097-98 11 3,502.93 3,035.06 89 2,136.46 6 1,229.64 947.31 75 190.062 129.44 129.67 39 -264.10 98-99 11 3,525.77 3,077.31 89 2,180.46 61,433.62 1,047.19 75 291.42 2 201.73 218.96 39 -174.04 99-00 113,469.02 3,103.43 89 2,205.16 6 1,496.51 1,134.56 75 377.59 2198.24 208.87 39 -184.75 00-01 11 3,509.98 2,986.26 89 2,091.38 61,280.38 958.92 75 204.81 2 172.38 180.34 39 -211.80 01-02 113,454.76 3,029.94 89 2,132.18 6 1,168.10 890.99 75 134.45 2 169.99175.73 39 -217.67 02-03 11 3,439.02 3,047.52 89 2,147.87 6 1,324.17990.89 75 25.55 2 186.34 161.28 39 -232.94

* All p-values are lower than 0.0001.

are due to response categories aggregated in different sectors(Agriculture, Industrial sector, Services) in the two occasions, wenote that it underestimates the association between the followingcouples of categories: Agriculture and Public Administration,defence and compulsory social security; Manufacturing and Wholesaleand retail trade; Construction and Professional and support serviceactivities. Instead, the model overestimates association betweenManufacturing and Public Administration, defence and compulsorysocial security; Construction and Wholesale and retail trade; andProfessional and support service activities and Agriculture. It ishard to attribute these results only to inconsistencies inresponses given one year apart; it appears more convincing toascribe them to non- random measurement error affecting responsesin each wave of the survey.

15

In the case of professional status, the log-linear model ofquasi-independence was estimated so to reproduce the 11-, 6- and2-category classifications. Table 9 is similar to Table 8, andlists the results of model fitting for the classifications ofprofessional status for the 10 panels. Also for this variable, thehypothesis of quasi-independence is always rejected, for all panelsand at all levels of aggregation, indicating that one-year-apartresponses show non-random association even at the maximum level ofaggregation (2 categories).

The BIC index always reaches its minimum value with the2-category classification. This indicates that this is the level ofaggregation which fits the data best, and again confirms theconclusions reached by applying the hierarchical Kappaprocedure.

Inspecting estimated residuals of the quasi-independence modelimplying 2 categories, i.e., testing quasi-independence amonginconsistencies which are due to response categories classified asself-employed on one occasion and as employee on the other, we notethat it underestimates the association between the following pairsof categories: Manager and Entrepreneur, Executive andProfessional, Clerk and Professional. In this case, patterns ofassociation appear more reasonable and easier to interpret than inthe case of industry. The results show that, in fact, residualassociation tends to concentrate among the highest classes of bothemployees and self-employed. Usually those in high positions, evenif they are employees, enjoy more flexible working conditions, sothat they sometimes confuse themselves with self-employed workers.Another sensible explanation is that in Italy, as in other Europeancountries, the standard dichotomising of workers intoEmployee/Self-employed has recently become too rigid, and is unableto cope with the growth of non-standard forms of employment (see,e.g., Burchell, Deakin and Honey, 1999).

3.4. Testing a set of different classifications jointly byprofessional status and industry

It is of some interest to evaluate what happens in terms ofinconsistencies of one-year-apart responses, when we specify jointclassifications by professional status and industry, starting fromone with 13 classes: Self-employed on the one hand, and employeesdivided by the 12-category classification of industry on theother.

The aggregation process for the joint classifications isdocumented in Table 10. It moves from the 13-categoryclassification just mentioned to 7-, 6- and 4-categoryclassifications, respectively, obtained considering Self-employedworkers systematically in one category and aggregating employees byindustry in 6, 5 and 3 categories, consistently with the strategyfor collapsing classes previously used for that variable. Inaddition to this 4-category classification, denoted classification(a) (Self-employed, Employee in agriculture, Employee in theindustrial sector, and Employee in services), we add an alternative4-category classification recently introduced, on heuristicgrounds, by Trivellato et al. (2005), which we denoteclassification (b) (with the four categories given bySelf-employed, Employee in agriculture, Employee in industrialsector and private services, and Employee in public administrationand social services). Simply stated, the motivation for thisalternative classification is the following: the distinctionbetween

16

Table 10: Scheme of categories aggregation process: jointclassification by occupational status and industry

13 categories 7 categories 6 categories 4 categories:classification (a)

4 categories: classification (b)

Self-employed Self-employed Self-employed Self-employedSelf-employed Employee in: Employee in: Employee in: Employee in:Employee in: Agriculture Agriculture Agriculture AgricultureAgriculture Mining and raw material extraction Manufacturing

Manufacturing and mining

Manufacturing and mining

Construction Construction Construction

Industrial sector

Wholesale and retail trade

Wholesale and retail trade

Wholesale and retail trade

Accommodation and food services Transportation andcommunications Financial and real estate activities Professionaland support service activities

Services

Industrial sector and private services

Public Administration, defence and compulsory social securityEducation, health and other services Other public, social andpersonal service activities

Public Administration

Other activities

Services

Public Administration and social services

employees in (basically) non-market services on the one hand andin the industrial sector and private services on the other isperceived by respondents as more clear than the traditional onewhich contrasts employees in the industrial sector vs. employees inservices. The main reason is possibly the extensive process ofoutsourcing in the industrial sector, which has made thetraditional distinction blurred.

Table 11 lists the values of hierarchical Kappa coefficientscalculated with these five sets of weights on the 10 panels, andthe results of the tests performed on their

17

Table 11: Hierarchical Kappa coefficients and Wald tests: jointclassifications by professional status and industry

Kappa coefficients Wald tests Panels 13

categories 7

categories 6

categories 4

categories: classif. (a)

4 categories: classif. (b)

7 vs. 13 6 vs. 7 4 class. (a) vs. 6

93-94 0.8713 0.8874 0.8972 0.9066 0.9083 242.54*** 108.54***88.87*** 94-95 0.8866 0.9020 0.9116 0.9205 0.9195 198.08***93.26*** 73.29*** 95-96 0.8822 0.8977 0.9058 0.9123 0.9170231.98*** 84.40*** 50.84*** 96-97 0.8961 0.9096 0.9148 0.92130.9263 198.07*** 43.66*** 53.38*** 97-98 0.8957 0.9046 0.91210.9189 0.9224 103.14*** 69.91*** 54.33*** 98-99 0.8881 0.90030.9075 0.9143 0.9221 156.82*** 63.94*** 51.69*** 99-00 0.88340.8969 0.9036 0.9116 0.9178 183.26*** 55.30*** 63.87*** 00-010.8814 0.8942 0.8989 0.9058 0.9162 160.76*** 28.69 45.10*** 01-020.8905 0.9014 0.9061 0.9125 0.9234 137.23*** 31.17*** 43.51***02-03 0.8945 0.9074 0.9132 0.9181 0.9255 181.95*** 14.05***29.19***

* Significant at �=0.1, ** significant at �=0.05, ***significant at �=0.01.

differences. Switching from 13 to 7 categories significantlyimproves agreement among responses in all panels; reducingcategories to 6 and then to 4-classification (a), furthersignificantly increases agreement.

Lastly, we compare the alternative 4-category classification (b)with the standard 4-category classification (a). Also in the caseof classification (b) switching from 13 to 4 categoriessignificantly increases agreement among responses in all panels. Inaddition, the 4-category classification (b) has a higher (andstatistically significant) level of agreement in 9 (8) out of 10panels. The overall �2, with 10 degrees of freedom, is 145.05, witha p-value close to zero, and definitely confirms that the latter4-category classification is superior to the former one.

4. Conclusions

The focus of this paper is on inconsistencies in jobcharacteristics reported in one-year-apart independent waves of theQLFS by workers continuously employed and who did not change job.Transition matrices by professional status (collected with 11categories) and industry (recoded into 12 categories) show asignificant percentage of frequencies outside the maindiagonal.

Aggregating categories improves agreement, as the application ofthe hierarchical Kappa procedure clearly demonstrates. Forprofessional status the best level of aggregation is the binaryone: Employee/Self-employed. For industry two classificationsminimise inconsistencies: with 5 classes (Agriculture,Manufacturing and mining, Construction, Wholesale and trade, Otheractivities) and 6 classes (with a split of Other activities inServices and Public Administration), respectively. In the case of ajoint classification by professional status and industry, the bestlevel of aggregation is given by the 4-category classificationrecently advocated by Trivellato et. al. (2005), whichdistinguishes Self-

18

employed, Employee in agriculture, Employee in industrial sectorand private services, and Employee in public administration andsocial services.

Inspection of estimated residuals of the log-linear model ofquasi-independence – which does not fit the data even at themaximum level of aggregation – suggests that even cross-sectioninformation is affected by non-random measurement error, since notall residual association can be explained by inconsistencies amongresponses perceived as similar by respondents.

Abundant literature (Mathiowetz and McGonagle, 2000; Sala andLynn, 2006; Lynn, Jäckle, Jenkins and Sala, 2006, among others)shows that dependent interviewing results in lower levels ofobserved changes in job characteristics collected in two subsequentinterviews, compared with changes observed with independentinterviewing. It also reveals that this reduction in observedchanges coincides with reduction of measurement error, since it isparticularly pronounced among workers who do not change job.

The partly disappointing results of our analyses on 1993-2003two-wave panel data from the QLFS, coupled with evidence from theliterature on dependent interviewing, highlight the importance ofinnovations with the new Continuous LFS, in operation in Italy from2004. It introduced a definitely better questionnaire and aCAPI-CAPI mode of interview from the start, and is progressivelyextending dependent interviewing to several sections of thequestionnaire, including the one on job characteristics.

Finally, our results bear some significance from the perspectiveof cross-country studies of job mobility (see, e.g, the recentreport to the European Commission by Andersen et al., 2008). Itwould be improper to extend inferences from a case-study limited tojust one country. However, our results about inconsistencies inreported employment characteristics suggest that indicators of jobmobility, especially of occupational mobility, based on harmonizedhousehold surveys – such as the European Labour Force Survey, theEuropean Community Household Panel and the European UnionStatistics on Income and Living Conditions – might be affected bysizable measurement error, possibly varying across countries.Explicit consideration of such measurement error and its effectsseems to be essential in order to get reliable (though eventuallyless detailed) indicators of job mobility.

References Andersen T., J. Henrik Haahr, M. Eggert Hansen and M.Holm-Pedersen (2008), Job mobility in

the European Union: Optimising its social and economic benefits.Final report, Report prepared under contract to the EuropeanCommission, Directorate General for Employment, Social Affairs andEqual Opportunities, Aarhus, Danish Technological Institute[http://ec.europa.eu/social/BlobServlet?docId=514&langId=en].

de Angelini A. and A. Giraldo (2003), La mobilità dei lavoratorinel Veneto. Confronto fra misure su dati RTFL e su dati NETLABOR,PRIN Research Project “Dinamiche e persistenze nel mercato dellavoro italiano ed effetti di politiche”, Working Paper n. 61,Padova, University of Padova, Statistics Department.

19

Barnhart H.X. and J.M. Williamson (2002), “Weightedleast-squares approach for comparing correlated Kappa”, Biometrics,58: 1012-1019.

Biemer P.P. (2004), “The twelfth Morris Hansen Lecture simpleresponse variance: Then and now”, Journal of Official Statistics,20: 417-439.

Bound J., C. Brown and N. Mathiowetz (2001), “Measurement errorin survey data”, in J.J. Heckman and E. Leamer (eds.), Handbook ofEconometrics. Volume 5, Amsterdam: Elsevier Science.

Burchell B., S. Deakin and S. Honey (1999), The employmentstatus of individuals in non-standard employment, EMAR PublicationsNo. 6, London: Department of Trade and Industry.

Cohen J. (1960), “A coefficient of agreement for nominaltables”, Educational and Psychological Measurement, 20: 37-46.

Cohen J. (1968), “Weighted kappa: Nominal scale agreement withprovision for scale disagreement or partial credit”, PsychologicalBulletin, 70: 213-220.

Eurostat (2000), “Commission Regulation (EC) No 1575/2000 of 19July 2000 implementing Council Regulation (EC) No 577/98 on theorganisation of a labour force sample survey in the Communityconcerning the codification to be used for data transmission from2001 onwards”, Official Journal of the European Community, 43(L181), 20.7.2000: 16-34.

Fallick B. and C.A. Fleischman (2004), Employer-to-employerflows in the U.S. labour market: The complete picture of grossworker flows, Finance and Economics Discussion Series 2004-34,Washington D.C.: Board of Governors of the Federal Reserve System(U.S.).

Fleiss J.L. (1981), Statistical methods for rates andproportions, New York: Wiley.

Gandolfo M. and P. Gennari (2000), “Il lavoro sommerso, èpossibile rilevarlo? In Società Italiana di Statistica”, Atti dellaXL Riunione Scientifica. Firenze 26-28 April 2000, University ofFlorence, Statistics Department.

Goodman L.A. (1968), “The analysis of cross-classified data:independence, quasi-independence, and interaction in contingencytables”, Journal of the American Statistical Association, 63:1019-1131.

Hagenaars J.A. (1990), Categorical longitudinal data, NewburyPark: Sage.

Hansen M., W.N. Hurwitz and L. Pritzker (1964), “The estimationand interpretation of gross differences and the simple responsevariance”, in C.R. Rao (ed.), Contributions to Statistics(presented to C. Mahalanobis on the occasion of his 70th birthday),Calcutta: Statistical Publishing Society.

Hauser R.M. and M.P. Massagli (1983), “Some models of agreementand disagreement in repeated measurements of occupation”,Demography, 20: 449-460.

Koch G.G., J.R. Landis, J.L. Freeman, D.H. Freeman and R.G.Lehnen (1977), “A general methodology for the analysis ofexperiments with repeated measurements of categorical data”,Biometrics, 33: 133-158.

ILO (1993), Resolution concerning the InternationalClassification of Status in Employment (ICSE), Adopted by theFifteenth International Conference of Labour Statisticians (28January 1993), Geneva[http://www.ilo.org/wcmsp5/groups/public/---dgreports/---integration/---stat/documents/normativeinstrument/wcms_087562.pdf].

ILO (2007), Resolution concerning updating the InternationalStandard Classification of Occupation, adopted by the TripartiteMeeting of Experts on Labour Statistics on Updating theInternational Standard Classification of Occupation (ISCO), Geneva,7 December 2007[http://www.ilo.org/public/english/bureau/stat/isco/docs/resol08.pdf].

20

Istat (2003), Rilevazione trimestrale sulle forze di lavoro.Tracciato record del file standard longitudinale (Aprile2003-Aprile 2002), Roma (mimeo).

Istat (2004), La nuova rilevazione sulle forze di lavoro -Contenuti, metodologie, organizzazione, Roma.

Landis J.R. and G.G. Koch (1977), “The measurement of observeragreement for categorical data”, Biometrics, 33: 159-174.

Leombruni R. and R. Quaranta (2002), “Mobilità dei lavoratori inItalia, 1985-1996”, in B. Contini (ed.), Osservatorio sullamobilità del lavoro in Italia, Bologna: Il Mulino.

Leombruni R. and R. Quaranta (2005), “Eppure si muoveva già. Unabreve storia della mobilità dei lavoratori in Italia”, in B.Contini and U. Trivellato (eds.), Eppur si muove. Dinamiche epersistenze nel mercato del lavoro italiano, Bologna: IlMulino.

Lynn P., A. Jäckle, S.P. Jenkins and E. Sala (2006), “Theeffects of dependent interviewing on responses to questions onincome sources”, Journal of Official Statistics, 22: 357-384.

Mathiowetz N. (1992), “Errors in reports of occupation”, PublicOpinion Quarterly, 56: 352-355.

Mathiowetz N. and A. McGonagle (2000), “An assessment of thecurrent state of dependent interviewing in household surveys”,Journal of Official Statistics, 16: 401-418.

Mellow W. and H. Sider (1983), “Accuracy of response in labormarket surveys: Evidence and implications”, Journal of LaborEconomics, 1: 331-344.

McCall B.P. (1990), “Occupational matching: A test of sorts”,Journal of Political Economy, 89: 45-69.

Miller R.A. (1984), “Job matching and occupation choice”,Journal of Political Economy, 92: 1086-1120.

Neal D. (1999), “The complexity of job mobility among youngmen”, Journal of Labour Economics, 17: 237-261.

Sala E. and P. Lynn (2006), “Measuring change in employmentcharacteristics: The effects of dependent interviewing”,International Journal of Public Opinion Research, 18, 500-509.

Shimer R. (2005), “The cyclicality of hires, separations, andjob-to-job transitions”, Federal Reserve Bank of St Louis Review,87: 493-597.

Steven D.J. and J. Haltiwanger (1999), “Measuring gross workerand job flows,” in J. Haltiwanger, M. Manser and R. Topel (eds.),Labor Statistics Measurement Issues, Chicago: University of ChicagoPress.

Trivellato U. (1997), “Le misure della partecipazione al lavoronel quadro comunitario”, in L. Frey (ed.), Le informazioni sullavoro in Italia: significato e limiti delle informazioniprovenienti da indagini sulle famiglie, Quaderni di Economia delLavoro n. 59, Milano: Franco Angeli.

Trivellato U., A. Paggiaro, R, Leombruni and S. Rosati (2005),“La dinamica recente della mobilità dei lavoratori, 1998-2003”, inB. Contini and U. Trivellato (eds.), Eppur si muove. Dinamiche epersistenze nel mercato del lavoro italiano, Bologna: IlMulino.