RAND > Labor & Population > Conferences/Seminars > Workshop

Labor and PopulationReturn to Labor and Population Homepage AddThis Social Bookmark Button

RAND/UCLA/WHO Workshop On Health Status Measurement In Social Surveys

Santa Monica, October, 1999

Duncan Thomas and Elizabeth Frankenberg





 
INTRODUCTION AND OVERVIEW


The idea for this workshop emerged out of a series of discussions between Dean Jamison, Ritu Sadana, Elizabeth Frankenberg and Duncan Thomas. Broadly speaking, it was intended to provide a forum for a dialogue on the meaning and interpretation of health indicators collected in social surveys. In recent years, that has been a dramatic change in the technology available to measure health status in a field setting and there has, concomitantly, been increasing interest in the collection of biomarkers. Several studies have implemented new methods for measuring health while others have tested new or revised instruments that collect health status indicators in interviews. At the same time, there have been many important advances in our understanding of

  • the meaning of these health indicators

  • correlations among the health indicators

  • correlations between health status early in life and health status later in life

  • correlations between health and other life course outcomes including productivity.

 Recently, there has been nothing short of an explosion in the extent of interest among social scientists in the links between health and various dimensions of socio-economic status and the variation in this relationship over the life course. The overarching goal of the workshop was to bring some of these many threads of current enquiry together drawing on international experts who span the health and medical sciences, social sciences and survey research. The workshop was attended by participants from the United States, Europe, Mexico and several South and Southeast Asian countries.The workshop began with a discussion by Ritu Sadana of some of the key issues in the measurement of health currently facing the World Health Organization, WHO. First, there is a need to improve the conceptual basis used to measure health. This will likely involve the use of both qualitative and quantitative methods as WHO attempts to develop a set of standardized protocols and terminologies that can be applied cross-nationally. It will involve coming up with useful definitions of the boundaries of health, specification of health profiles and identifying a core set of domains of health. From a practical point of view, of particular interest is the use of self-reported measures of health and performance. These measures are widely used in developed countries and have been extensively validated. There are real questions about their validity in lower income countries and on-going work by WHO is attempting to provide some new insights into this issue. Nonetheless, the future is bright.

There are good reasons to anticipate substantial increases in the breadth and quality of data collected on health and socio-economic status (SES) in integrated surveys in the near future. There are also good reasons to anticipate important improvements in the extent and quality of the analyses of those data as they become more widely used.Jennifer Madan then described some of the issues that National Center for Health Statistics (NCHS) faces as it undertakes its responsibilities of collecting, analyzing and disseminating information on health status in the United States. In contrast with many agencies and many researchers, NCHS tends to take health apart rather than try to put it together into a single index. NCHS takes as a given that health should be defined extremely broadly and includes biomedical indicators, risk factors and behaviors, social factors and the mediating influence of SES and demographic characteristics. The National Health and Nutrition Examination Survey (NHANES) and Health Interview Surveys (HIS) are the centerpiece of NCHS survey efforts although they are only one element of a much broader portfolio of health data collection.There can be no doubt that NCHS stands head and shoulders above health data collection agencies in almost any other country. There is an increasing awareness at NCHS of the need to co-ordinate data collection efforts within agencies of the US Government and, particularly, to link survey data with administrative records. Multiple initiatives along these lines are currently underway at NCHS. There are, however, many obstacles -- confidentiality taking center stage in this regard.

There is also a push at NCHS to develop a greater longitudinal capacity and discussions are underway to consider undertaking surveys that more closely resemble the British Cohort Studies. This is a reflection of a greater interest among the research and policy constituencies to better understand how health flows evolve -- that is, in understanding health transitions -- and to develop more insights into the causal pathways that culminate in the stock of health of an individual, community or population at any point in time
 

.HEALTH OVER THE LIFE COURSE: MEASUREMENT AND INTERPRETATION


These remarks set the stage for the first panel of the workshop. In recent years, there has been a marked increase in our understanding of the links between early life experiences and health and well-being later in life. Important work along these lines is being conducted by David Barker and his associates who have placed particular emphasis on intra-uterine health and, more generally, the inter-generational transmission of health from mother to child.This innovative line of research was introduced by Barbara Cohn. Barker's hypothesis argues that insults during early life -- be it prenatal or in early infancy -- programs the body so that years later, during adulthood, health problems emerge. These problems include chronic adulthood diseases such as coronary heart disease, strokes and hypertension as well as reproductive, neurological, cognitive and behavioral functioning. For example, Barker argues that if a baby born with low birthweight may be indicative of the foetus being starved of critical nutrients of oxygen which resulted in stunting development of some parts of the body, relative to others.

Depending on the timing and extent of the intake deficiency, this will result in some organs being less well developed than others. For example, reduced body mass growth is often associated with less developed livers which is associated with higher incidence of coronary heart disease in adulthood.Cohn pointed out that Barker's work is not uncontroversial and that several critics have argued that the work has failed to take into account other confounding factors -- including a fuller conception of maternal health and the role of the environment. She described two important cohort studies in the United States that have the potential to provide further insights into the links between foetal and infant experience and later life outcomes. They are the Child Health and Development Studies (CHDS) and the Collaborative Perinatal Project (CPP). Michael Wadsworth had prepared remarks but, because he had been taken ill, they were presented by Duncan Thomas. Wadsworth introduced the British Cohort Studies which include 3 national longitudinal studies conducted in Britain which share a common design. Each cohort has been surveyed since birth with periodic interviews and assessments of health and cognitive functioning. The first cohort included a sample of all children born in the first week of March, 1946. It has been followed up 22 times with the most recent interviews being at ages 43 (1989) and 51 (1997).

A second cohort followed up all births during the first week of March, 1958, and a third cohort was begun in 1979. Plans are underway for a fourth birth cohort which will parallel the Education and Child Longitudinal Study-Birth Cohort (ECLS-B) which will begin in 2001 in the United States. He also mentioned the Avon longitudinal study of 10,000 children and the Barker follow-up studies. Wadsworth presented a broad overview of some of the results from these and other cohort studies. He began with the effects of early life factors on physical health. He showed that elevated blood pressure at mid-life is associated with poor nutrition and growth in childhood. Risk of later health problems is greatest among low birthweight babies who subsequently are overweight in childhood: a high ponderal index (weight/length 2) is protective. Childhood micro-nutrient intake affects physical development which, in turn, impacts health in adulthood. Elevated levels of Vitamin C intakes in childhood are associated with better lung development and lowers the probability of pulmonary disease (COPD) in adulthood. He proceeded to argue that early life factors influence mental health and cognition drawing on evidence regarding breastfeeding and IQ, emotional disturbance during childhood and physical growth, and the inter-generational transmission of emotional vulnerability. While there was widespread agreement that early life experiences probably do have consequences later in life, there was less agreement about the precise mechanisms. A central concern that emerged in the discussion is the need to parse out the role of phenotype and genotype influences. Demonstrating a correlation between, say, birthweight or breastfeeding and later health or cognitive functioning does not provide the information needed to assess the causal pathways that link poor health early in life with poor health later in life.

There are many confounding factors and an understanding of those factors is currently missing. That said, there was also general agreement that this is an extremely exciting line of inquiryfrom the point of view of those in the biomedical and social sciences and, perhaps especially, for those groups whose work transcends the boundary of both domains. Workshop participants agreed that investments in the collection and analysis of high quality long-term panel surveys that contain comprehensive information on both health and socio-economic status will surely yield very high pay offs.
 

INTERVIEW BASED ASSESSMENTS


The workshop discussed some of the interview-based methods of collecting information on health status. Bob Wallace plays a key role in the design of the Health and Retirement Study (HRS) and the Assets and Health Dynamics of the Oldest Old (AHEAD). The former is a random sample of households with one member born between 1931 and 1941; the latter sampled households with a respondent born prior to 1923. He described the scientific issues that were part of the original motivation of the surveys and discussed how those issues had been modified as the survey progressed. He discussed how issues that had arisen had been resolved and how new issues had emerged during the course of the survey. HRS and AHEAD are done on the telephone and so rely on self-reported health indicators. Wallace's discussion of their limitations included concerns about the validity of illness reports in an environment in which there are important limits to personal knowledge about current health conditions. There are special challenges that are particularly germane when working with older respondents revolving around cognitive functioning, the complexity of health states and illnesses because of co-morbidities, poly-therapies and decreased functioning with age of many of the body systems. Several of these factors are likely to be associated with a respondent's willingness to participate in a study of this nature: participation tends to decline with age, reduced cognitive functioning and elevated health problems. This choice-based sample poses special challenges for research that seeks to understand the behaviors of the population. It was noted that the fact that respondents enter these surveys only after they have reached age 50 means that many of the life events that will have affected their health will have already occurred prior to the first interview. Whereas Wallace is very skeptical that attempts to reconstruct key events in an individual's health life history, others in the audience felt that remained an open question and one that should be tested empirically.

This seems right particularly in view of the life course approach to examining health outcomes discussed in the previous session of the workshop. Jane Gentleman followed up with a discussion of the Health Interview Survey (HIS). It is a very large scale annual survey that serves both as a key source of information on prevalence of health problems and also as a sampling frame for more focussed surveys such as the Medical Expenditures Panel Survey (MEPS). The multiple demands on HIS has dictated a number of design compromises. For example, it is important to include a large number of households to build the MEPS sampling frame; that precludes interviewing more than one adult and one child in each household.The discussion moved on to the construction and interpretation of health status scales. John Ware describes health status as being characterized by a series of concentric circles. Biological functioning lies at the center; the next layer involves physical and mental health; the next layer is concerned with social and role participation; finally, there is health-related quality of life. When measuring health, one needs to be clear about which of these levels one is interested in. Moreover, he notes that it is important to distinguish between objective functioning, distress and well-being and a respondent's articulation of their perception of their own health.That said, Ware advocates rigorous standardization of health measurement to provide comparability of indicators across time and space using a common metric or ruler. He described the International Quality of Life Assessment (IQOLA) Project which seeks to translate and adapt the Medical Outcomes Study SF-36 so that those question can be validated, normed and documented for use in international comparisons of health. SF-36 and SF-12 translations and adaptions have been successful in 46 countries. The distinction between physical and mental health plays a key role in the SF-36 based scales. Ware noted that there appear to be different correlations among the individual items among Japanese respondents relative to respondents in the United States and Europe. These differences indicate that the physical and mental health do not the mean the same thing to a Japanese and American person. Ware concluded with some observations on some of the exciting opportunities that the internet affords.
 

ACTIVITIES OF DAILY LIVING (ADLs)


These discussions have all raised, either implicitly or explicitly, the central issue of how one should interpret self-reported health status indicators. As an example, consider the first question in SF-36. "In general, would you say you health as (1) excellent, (2) very good (3) good (4) poor or (5) very poor?" Does this question mean the same thing to all respondents? Would the answer from someone who is at the peak of good health in his or her twenties be comparable with the response from someone one who is slowing down in his or her sixties or seventies? The question does not provide an explicit comparison group. Should you compare your health now with health at some other time and, if so, what other time? Should it be the recent past, the distant past or even the future? Or should you compare yourself with someone else in the population? If so, who is that person? The average health of someone in your age group, in your socio-economic group, in your circle of friends or simply your perception of the average of the population in general. Clearly, measures of general health status (GHS) and morbidities capture not only underlying health status but also knowledge about health status and that knowledge tends to vary in systematic ways with SES. In a relatively homogenous population where variation in knowledge is relatively minor, this will be a second order problem.

However, validation of these sorts of questions in international settings suggests that in many low income settings, the issue is not negligible. It is, for example, not uncommon to find that health, as measured by GHS, is worse among the better off in a society relative to the poorest when health. A similar finding has been reported for morbidities, especially those that are based on diagnoses which are often known only after some interaction with a health care professional. Use of health care is positively correlated with income and SES in most setting and this correlation is particularly strong in low income countries with health care access is not universal. Thus, it is not surprising that knowledge of ailments is greater among higher income groups.Activities of daily living (ADLs) and Instrumental ADLs (IADLs) are thought to ameliorate some of these shortcomings.

John Strauss explained that ADLs ask about difficulties with specific tasks that respondents are likely to undertake on a daily basis -- such as difficulty walking a certain distance, climbing stairs, dressing. While there are likely to be differences in perceptions about what is meant by "difficulty", there is less scope for heterogeneity in interpreting "walking 100 metres". Strauss observed that the correlations between difficulties with many ADLs and indicators of SES tend to indicate the better off are in better health although he noted that does not mean ADLs do a good job capturing underlying health -- rather they do not appear to have as grave problems in interpretation as GHS and morbidity information. Nonetheless, they do suffer from two serious shortcomings as they are currently conceived in most surveys. First, the ADLs that have been extensively used tend to do a poor job of discriminating among reasonably healthy people unless they are fairly old; few prime age adults have difficulty dressing or bathing themselves. Strauss encouraged thinking about identifying activities that might help yield information about health of these people. Second, as John Ware point out, ADLs are context sensitive. It makes no sense to ask a respondent whether he or she has difficulty climbing stairs if stairs are not part of the person's normal routine. It is important, therefore, to design the questions so that they are relevant for the respondent.It was also noted that direct observation might provide useful information that complements responses to ADL questions.

For example, tests of mobility such as time to walk a distance or time to stand from a sitting position, may yield insights into the links between this array of indicators, underlying health status and SES -- and how those links vary across age, gender and ethnicity. Raynard Kington noted that both ADLs and mobility tests are included in NHANES; they are also included in the MHSS and there is one mobility test in IFLS. Duncan Thomas noted that even timed tests should not be interpreted without caution; they will not only capture health status (or difficulties with certain activities) but they are likely to pick up enthusiasm and exuberance as well.
 

NUTRITION
Nutrition plays a central role in discussions of health in low income populations, where malnutrition rates are high; it is also of importance in higher income societies, where obesity and excess fat intake pose serious public health threats. Jean Pierre Habicht started out by saying that he would focus on the former. He discussed how nutrition might be defined -- at the molecular level, the intra-cellular level, the cellular level (wherein the storage of nutrients will reflect nutritional status) and the extra-cellular level (which is often taken to be nutritional status) and encompasses the transport and excretion of nutrients and metabolites.Turning to the determinants of nutritional outcomes, Habicht highlights the importance of diet and illnesses and notes that malnutrition and illness are likely to work synergistically to impair growth and also increase the severity of disease. He proceeded to discuss measurement issues and argues there are two distinct domains over which "nutrition" might be measured: intakes of nutrients (accurate measurement of which involves serious difficulties in anything but a very controlled setting) and nutritional status (which can be assessed in a variety of ways with each indicator reflecting different aspects of nutrition).In a survey setting, attempts to measure nutrition using diet are unlikely to be very valuable unless one adopts an extremely intrusive and very expensive design that has intakes of respondents being measured at the individual level. Even in the best of designs, there will be serious problems associated with leakage from waste, recall error in diet recall designs, periodicity effects (since diets are not the same from day to day) and in the quality of foods consumed. All of these errors in measurement are likely to be systematically related to SES -- the better off tend to waste more food, they tend to have greater variation in their diets and they tend to eat higher quality foods.
Habicht argues that the vast majority of the information that is likely to be gleaned from intake studies can be obtained much more efficiently by asking about consumption patterns. He suggests first assessing which key nutrients are of interest for the scientific question at hand. He then suggests asking questions of respondents about their own consumption of specific foods. For example, Vitamin A is found in carrots and sweet potatoes; he recommends asking about frequency of consumption of those foods and then drawing inferences about Vitamin A intake on the basis of those answers. Similarly, iron is absorbed from meat and green leafy vegetables -- but rice tends to hinder absorption, particularly from non-animal sources. He recommends asking about consumption of those items. More generally, he notes that several micro-nutrients are consumed at levels in excess of need and that knowledge of frequency of consumption of specific foods will yield the key information needed to draw this level of distinction within a population. Habicht suggested that the critical micro-nutrients for which deficiencies have serious consequences in low income settings are probably iron (associated with anemia which is associated with elevated levels of morbidity and fatigue), iodine (associated with goiter and cretinism), vitamin A (associated with blindness and reduced immunity to infection), niacin (associated with pellagra), thiamin (associated with beri-beri), vitamin C (associated with scurvy). Recent scientific research has indicated that zinc and vitamin B (folates) have emerged as important micro-nutrients in a variety of studies. He cautioned, however, that the vast majority of the literatures suggests that an individual is seldom debilitated because of an inadequacy of only one nutrient. Rather, research has shown that there are very complex interactions among nutrients which need to be better understood if we are to interpret associations appropriately. In comparison with difficulties in measurement of diet, measurement of nutritional status is relatively straightforward. Anthropometry has proved to be a very powerful tool for nutrition, the biological sciences and the social sciences. Typical measures are height (to capture retarded growth and stunting), weight given height (to capture thinness or wasting), weight for age (a combination of both longer run growth and shorter run fluctuations in growth). These indicators, in combination with birthweight and birth length, leg length, waist to hip ratio, fat fold and head circumference shed considerable light on individual-specific exposures to nutritional stresses over the life course. Habicht concluded with an example of how easy it is to mis-interpret relationships between nutritional status and other outcomes. Focussing on the association between weight gain during pregnancy and birthweight, he demonstrated the difficulties associated with drawing inferences about the causal links between maternal nutrition and child health when there is incomplete information on prior conditions. The example highlights the fact that nutritional status is a stock and that, in many instances, it is the flow (change in nutritional status) that is a key determining factor. With this cautionary tale, Habicht concluded that nutrition is a very powerful, if subtle and complex, force in the lives of many who are poor.

CLINICALS & SPECIMENS
Recent innovations in technology and understanding of the links between biochemistry and functional health status has led to an explosion in the collection, use and interpretation of clinical data and specimens in population-based studies of health and well-being. As an introduction, Theresa Seeman discussed results from one of the projects that has pioneered this line of research. She presented data from the MacArthur Project on Successful Aging. The data are from a longitudinal community-based study that was started in 1998 and included respondents age 70-79 at that time who were living in East Boston, MA, New Haven, CT, and Durham, NC. The respondents were in the top 1/3 of their peers in terms of cognitive and physical functioning at baseline. The question she was trying to answer concerned the effect of allostatic load on cognitive and physical functioning. Allostatic load is the cumulative physiologic toll exacted on the body over time by efforts to adapt to life experiences. In this study allostatic load was based on ten parameters related to the functioning of various regulatory systems, including systolic and diastolic blood pressure, waist to hip ratio, total cholesterol, glycosolated hemoglobin, urinary cortisol, urinary norepinephrine, and urinary epinephrine. Allostatic load was quantified in terms of the number of indicators for which the respondent was in the highest risk quartile. The results suggest that higher allostatic loads are associated with poorer functioning and predict future declines in functional status. As a concept, allostatic load is potentially extremely powerful. Seeman pointed out the difficulties associated with developing empirical measures that capture the underlying theoretical construct. With this in mind, she presented the results which suggest that there is little evidence of important synergies among the different indicators. That is, people with, say, four conditions were not more than twice as worse off in terms of functional health than people with only two conditions. Instead, the relationships appear to be remarkably linear. Jim Smith echoed the concerns regarding measurement of allostatic load and wondered whether the way allostatic load was measured in this study captures the cumulative dimension of health insults or the synergistic dimension that the various risk factors might pose in combination.Seeman concluded with a discussion of some of the practical issues associated with the collection of specimens. First, she noted that the assays are very expensive ($200 per test excluding the equipment and manpower costs). Second, the specimens have to be kept at very low temperatures; otherwise they deteriorate very quickly. In a study that spans many years, this imposes very substantial logistic problems. Third, she suggested that many tests can be conducted reasonably effectively but the researcher needs to be aware of the fact that some tests increase refusal rates (she cited a 24 hour urine sample) and other tests need to be repeated several times in order to minimize the effect of measurement error (she cited salivary cortisol for which she recommended at least 8-9 samples). As a practical matter, researchers interested in these sorts of tests need to be sure the logistical infrastructure is in place; thoroughly pre-test each assessment to determine whether it involves such a burden on the respondent that it results in increased refusals; ensure that those tests that are included are repeated sufficient times to reduce the noise and thus obtain a useful signal and, above all, perhaps, she emphasized the importance of sufficient resources.Picking up the theme of what is practical, Raynard Kington, described components of the NHANES (National Health and Nutrition Examination Survey). Kington is the director of NHANES which is undergoing considerable change. Traditionally NHANES has been a cross-section survey that was conducted over two or three years in the 1960s, again in the 1970s and again in the 1980/90s. NHANES has moved to a schedule whereby data are continuously collected with each 3 year window being representative of the US population. This will yield, for example, 3-year national estimates for the 1999-2001 period.NHANES is a very impressive operation. It is also very expensive. The survey collects some information on socio-demographic characteristics of participants but the center-piece of the survey is an extremely comprehensive clinical evaluation of each respondent. The evaluation is done in a mobile clinic which is outfitted with state of the art equipment. The mobile clinics are driven to the study site and the examinations are conducted by doctors and nurses who travel with the survey teams. The health and nutrition examination component of the survey is very long (around 4 hours). The breadth and depth of the survey is breathtaking -- it is surely one of the richest surveys in existence. NHANES III collected data from 40,000 people and examined 30,000 with an oversample of children, older people, blacks, Mexican Americans. The components of the study that Kington singled out as having provided particularly relevant results are: blood cholesterol (decreased over time), overweight prevalence (risen in recent years), serum cotinine (which reflects exposure to nicotine and provides insights into the effects of smoking reduction efforts), and nutrient intake data which have been used to develop new Daily Reference Intakes for folate and related B vitamins. A small number of countries have attempted to replicate the NHANES strategy. Greg Pappas talked about one example: the 1994 National Health Survey of Pakistan. One of the goals of the survey was to produce national estimates of diseases and risk factors. The survey was nationally representative and collected data on people of all ages in 80 primary sampling units in Pakistan. Data collection took 3 years. The survey covered nutrition, maternal and child health, infectious diseases, chronic diseases and disabilities, and health care and hygiene. Interviews were conducted in the home and were followed with physical health assessments conducted at a fixed site within the primary sampling unit. Physical health measures included an exam by a physician, anthropometry, vision testing, and blood and urine tests. Overall, response rates in the Pakistan survey were very high. Non-response rates were highest for males 15-44 (around 21% in urban areas, around 16% in rural areas), lowest for children under 15 and adults over 64 (2-9%), and low for women (5-10%). This evidence calls into question the notion that getting respondents to visit a fixed clinic automatically results in low response rates and, perhaps even more importantly, calls into question the argument that with fixed sites, the sample of respondents is not random. Moreover, the evidence described by Pappas suggests the data are of high quality. He argued, therefore, that there can be little argument NHANES-type surveys are feasible outside the United States. Pappas also talked about the cost and suggested that the surveys are also cost-effective. He noted that one of the key features of the survey was the work that had been done to disseminate the results to planners and policymakers in Pakistan. There was some discussion about placing the data in the public domain although there are no plans for that activity at this time.Considerable attention was paid to the equipment used for blood work in the Pakistan survey. A Boerringer Reflowtron was used to test venous blood for cholesterol and creatinine. It is affordable (~$300) but requires a technician and is somewhat delicate. The Pakistanis built a sleeve around it that could be filled with ice or with warm water to keep the machine within a certain temperature range. Because of its sensitivity, Pappas thought it would not be feasible to take the machine to households and so use of this equipment requires setting up a fixed site that respondents visit. It was noted, however, that it may be feasible in some locations to draw the venous blood, place it on paper and test it in a lab which is set up in each primary sampling unit; this would mean the respondent does not need to visit the lab but the health worker would need to return to the lab as soon as the measures are completed. The conference group concluded that there is much to be gained from pushing the frontier of current socio-economic surveys to include at least some of the physical measurements conducted in NHANES and the National Health Survey of Pakistan. It was also suggested that surveys like NHANES might be more useful if greater time in the survey was allocated to collection of socio-demographic and economic characteristics of respondents.

HEALTH EXPENDITURES AND HEALTH PRICES
Along with measuring the benefits of investments in health -- as manifest in health outcomes -- it is important to also measure the costs of those investments. Measurement of expenditures on health care and even the price of health care is notoriously difficult. It has, however, been shown that prices do matter -- both in the United States and elsewhere -- and that it would be imprudent to ignore this side of the equation. Moreover, in many models, we are interested in the effect of health on outcomes such as wealth accumulation, labor market productivity, cognitive functioning etc. In these models, we will need instruments that predict health status but have no direct effect on the outcome of interest; prices are one set of candidates.Dana Goldman laid out the issues from the perspective of research conducted in a developed country setting. He noted that it is not obvious how to attribute health care expenditures in an environment in which the costs are shared by consumers, insurers and the government. At the most simple level, the costs of care should be the sum of out-of-pocket expenses paid by the consumer plus insurer- covered expenses. How should the costs of care for an uninsured person who receives care in an emergency room but spends none of his or her own resources be calculated? (Should the cost be the marginal or average cost of the care to the provider?) How should expenditures on insurance premiums be attributed? How should expenditures on new technologies and the development of those technologies be priced? Putting aside these conceptual issues, measurement of what is paid is not straightforward. In survey data, questions are often badly worded and there are issues revolving around the appropriate recall period as we trade off accuracy and a long enough time span to catch sufficient catastrophic expenditures. Moreover, there is evidence that the extent of telescoping is likely to be related to the salience of the events being recalled and so we might expect those people who had larger expenditures are more likely to telescope those events into the recall period and thus bias upward estimates of health care expenses during the reference period. An alternative approach is to rely on administrative data such as billing claims. These data, however, often exclude subsidies such as care provided at government or employer clinics. Moreover, in the United States, billings and actual costs of care are often not closely linked which makes these data of limited value. Bottom line, Goldman argued, is that health care prices and expenditures on health care are virtually impossible to collect.Goldman continued with the observation that even if one could collect good data on prices of care of health care expenditures, it is not clear they would be useful as instruments for health status. This is because the elasticity of demand for health care is extremely low and prices likely only affect the very poorest. This argument met with some skepticism; some in the audience argued that there is abundant evidence in the United States and elsewhere that prices do matter for health care use and health outcomes. Contrasting evidence from HRS/AHEAD and MEPS (Medical Expenditure Panel Survey), Mike Hurd took issue with Goldman's claim that measurement of health prices and health care expenditures is a fool's errand. HRS/AHEAD collects information on health service use and costs for the two years prior to the survey. HRS collects these data with two sets of questions:

  • Itemized expenses:overnight hospital stays, nursing home stays, doctor visits, outpatient surgery, dentist visits, home health care, special facilities or services

  • Total out-of-pocket expenses for three categories:

    • overnight stays (hospital and nursing homes)

    • health center/hospital visits and outpatient services

    • home health care and special facilities or services


(using brackets for "don't know" or refusals in each case to reduce the incidence of missing values). MEPS has a long battery of questions that are thought to provide the "gold" standard in this literature. The astonishing result is the fact that the abbreviated set of questions in HRS yield estimates of expenditures that are remarkably close to those in MEPS. This suggests that even relatively simple questions do a very good job of capturing the bulk of individual health expenditures. Moreover, Hurd pointed out that the correlation between health expenditures, use of health care and health status and follow the kinds of patterns one might expect -- both for total expenditures (including insurance) and for out-of-pocket expenditures. The patterns reported in the RAND Health Insurance Experiment are largely reflected in the HRS. Contrasts between the United States and low income societies were brought out by Elizabeth Frankenberg . Drawing on her experience collecting data on health expenditures and prices at both the household level and at the facility level in Indonesia she noted that the "price" of health care encompasses substantially more than the pecuniary costs associated with care but also involve the costs of getting to health care providers and the lost income associated with seeking out care. In low income settings, health insurance remains relatively rare. Thus, health plans (or health insurance coverage) is not the primary determinant of care seeking behaviors. Indeed, it is typical to observe people using multiple care options either sequentially or possibly even simultaneously. Moreover, it is often difficult to classify providers into well-defined sub-groups -- many providers take on multiple roles. In addition, providers of traditional care are an important part of the health care delivery system in many countries, and they are often excluded from surveys. Probing all sources of care is critical for any survey interested in learning about use of health care in low income settings. Collection of information on health care expenditures is complicated by two facts. First, payment schemes are often complicated and can involve payments over extended periods of time, can involve payments that are conditional on outcomes (such as being cured) or subsidies for particular types of care during a limited time interval. In Indonesia, for example, in some areas it is possible to obtain a health card which provides free health care for a period of time; the mechanism whereby those cards are allocated is not entirely clear and so it is key that surveys collect information on these sorts of institutions in order to interpret reported health expenditures. Probing for expenditures on different types of care is key: spending on traditional care is seldom perceived as expenditure on health care per se. Second, part of the cost of health care is the cost of getting that care. In many low income settings, the travel costs and time costs of getting to a health center are large -- and dominate the costs of getting care once at the health center. Waiting times are often long at public health facilities -- and they should be included in the (expected) time costs associated with getting care. Moreover, the majority of the population in most low income countries do not have jobs that include benefits such as sick leave. For these people, time spent at a health center is time away from earning income. These non-pecuniary costs are individual-idiosyncratic and depend on the distance from the respondents home (or place of work) to the health facility and the value of that person's time. While computation of those costs is far from straightforward, ignoring them is likely to yield results that provide little by way of insights into the determinants of health-seeking behaviors. Note also that these costs also likely affect the choice of type of health care and whether or not someone will seek health care at all in systematic ways. A related but more subtle issue revolves around measurement of the quality of health services. In principle, the price of health care should depend on the quality of that care; the best studies of the effects of prices on use of care (and health outcomes) will adjust prices for quality. This is not easy. It is, however, potentially critically important given the tremendous heterogeneity in quality of care that is commonplace in most developing countries. Frankenberg described the methods used in IFLS2 to collect information on health expenditures, prices and perceptions of quality of health services at the individual and household level. She described how IFLS interviewed community informants to obtain information on the availability and quality of health services in the community. She briefly described how health facilities were sampled (randomly from a listing of all facilities that are in the catchment area for respondents in the enumeration area based on the lists provided by household and community informants) and the information that was gathered at each of the health facilities that was visited. This included prices of services, availability of services and a broad array of characteristics that are likely to be indicative of quality. An important element of health status is nutrition. This suggests that the relative price of foods will influence health outcomes. Frankenberg noted that food prices are collected at both the household and community level in IFLS -- drawing on up to 6 different informants at the community level. She also pointed out that data on prices are typically readily available from central statistical agencies and those data might productively be matched with household survey data. It is abundantly clear that collection of "prices" of health care services is very difficult -- whether it be in a low income or higher income environment. Frankenberg proceeded to highlight one of the advantages of collecting longitudinal data on household and facilities: it may be feasible to measure changes in "prices" better than measuring levels of prices. For example, during the hiatus between the 1993 and 1997 waves of the IFLS, there was a very large increase in investments in a program of village midwives who are intended to serve the needs of reproductive-age women and their children. Frankenberg suggested that contrasting changes in health care use and health outcomes of reproductive age women between 1993 and 1997 in areas where a village midwife was introduced with women in areas where no village midwife existed could provide some insights into the effect of the associated change in the "price" of health care. Moreover, to control for other changes that are occurring (such as changes in investments in health infrastructure more generally), one could contrast this difference with the difference in health status of older women (who would not be the focus of the village midwife) and also of men. Similarly, with the economic crisis in Indonesia, the price of public health services has increased dramatically between 1997 and 1998 while the quality of services have declined. In the private sector, prices increased less and there is little evidence of quality changes. These provide arguably exogenous changes in the "price" of health services and they can potentially serve to identify the effects of prices on health care use as well as health outcomes.In order to exploit these sorts of "shocks" -- which are all too common nowadays -- we need to have on-going large scale surveys in place prior to the policy change (or economic shock) as well as follow-on surveys after the policy change (or economic shock). The surveys need to collect a very broad array of indicators of health service availability, prices and quality at the facility and community level along with matched household surveys that collect information on use of health care, health status, health expenditures and socio-demographic characteristics of respondents. This is not easy -- but without a concerted attempt to build this sort of capacity, researchers and policy makers will continue to handicap themselves.
 

QUO VADIS?


While we cannot possibly do justice to the richness of new ideas that emerged, the range of points of views expressed and the quality of the discussion that took place during the workshop, we have sought to provide an overview of some of the main themes that emerged. Wrapping up this phase of the workshop, James P. Smith opened the bidding by identifying the key unresolved issues before us. Much has been learnt in recent years and there has been a spectacular increase in the quantity and quality of scientific data on health and socio economic status in the last decade. However, Smith argued, much remains to be learned and we will need to seriously re-think our strategies for the collection of data if we are going to make progress in some key dimensions in which our understanding of the processes underlying health outcomes is seriously lacking. It has been argued that health is a stock which reflects the cumulative effect of life events; there are important feedbacks between life events and health outcomes and between health status and life choices. If this is true, then the evolution of a person's health depends critically on its past trajectory. Knowing something about a person's health at a single point of time cannot possibly provide us with the sort of information that we need to reliably predict the future trajectory. Smith called, therefore, for increased investments in longitudinal surveys; echoing Jennifer Madan's introductory comments about discussions within NCHS about the feasibility of conducting cohort studies along the lines adopted in Britain, he applauded the work of Barker, Wadsworth and Marmot. He suggested that longitudinal surveys should not be the monopoly of higher income countries but that those which are started now in low income settings would likely have a tremendous pay off as it is in those countries that there is spectacular change on-going. We cannot wait 40 years for these studies to yield the kind of cohort data that is available in the United Kingdom. Smith thus also called for experimenting with the collection of retrospective data on health. He suggested that the survey research literature on recall error indicates that it is possible to collect high quality information on salient life time events: these might include hospitalizations, major illness during childhood or as an adult, the health and survivorship of siblings and parents. Incorporating this sort of information into cross-section surveys could substantially enhance their value in developing models of health transitions and understanding behaviors of individuals. Many are skeptical that this sort of information can be collected. Smith argued that skepticism is largely based on ignorance: there have been very few careful attempts to collect this sort of data and even fewer evaluations of the data. The question should not be can we collect this information? Rather it should be what information can we collect retrospectively? And how should that information be interpreted? It would be naive to think that prior health status will be measured accurately -- but will measures that are correlated with prior health status be measured sufficiently accurately to be useful for analysis and policy making?Smith noted that there has been a very substantial increase in the health content in many household-based socio-economic surveys. This is reflected in the HRS/AHEAD, several European surveys and in many of the surveys conducted in developing countries. It is clear from the workshop that there are forces in place that will likely result in this trend continuing. In contrast, however, there has been rather little movement towards collecting indicators of socio-economic status in health surveys. He suggested that in those surveys, the marginal product of yet another health measure is likely to be much, much smaller than the marginal product of measures of resources such as income and wealth. The links between health and SES are surely at the center of much of the debate in health care -- and health surveys are not being left out in the cold because they simply do not have adequate reliable information on SES. Smith called for bringing people from outside the health discipline to the table when designing health surveys. Kington responded by noting that NHANES has taken that criticism seriously and is currently piloting an SES module which is estimated to take about 20 minutes. The workshop moved on to discuss the specifics of several surveys conducted in low income settings in an effort to provide a sense of the practical lessons learned from the fielding of those surveys. Greg Pappas and Omar Kahn described the Pakistan National Health Survey; Kelly Hallman described the work conducted by IFPRI in Bangladesh; Randall Kuhn provided an overview of the Matlab Health and Socioeconomic Survey and Elizabeth Frankenberg described the Indonesian Family Life Survey. Kathleen Beegle and Bondan Sikoki reviewed some of the ideas that have been suggested for the next wave of the Indonesian survey and they were given extremely valuable feedback by the experts in the workshop. Dean Jamison ended the workshop and called for the development of an informal working group that would provide a forum to share both ideas and experience with regard to the collection and analysis of survey data that would subsequently inform research and policy making. The goals of this forum would be three-fold:

  • To facilitate enhancing the quality of research in the field by bringing together people from disparate fields; these people would not ordinarily interact with each other and yet, given the complexity of the issues at hand, there would be considerable benefit to everyone involved.

  • To encourage new and creative thinking that spans disciplinary and locational boundaries and foster an environment that supports risk-taking in research studies. This would involve taking calculated risks in both the collection of new, innovative survey data and in the analysis of those data. He challenged the workshop participants to think about how the richness of survey data could be brought to bear on the issues that are germane in the macroeconomic growth literature.

  • To help disseminate the work that is on-going and the lessons that have been learned both in terms of feasibility and desirability of fieldwork strategies and in terms of scientific results that are of substantive interest to the research and policy-making forum.


RAND/UCLA/WHO WORKSHOP
on
HEALTH STATUS MEASUREMENT IN SOCIAL SURVEYS
AGENDA

RAND, Santa Monica
October 16-18, 1999

PART I: HEALTH STATUS MEASUREMENT AND INTERPRETATION, SATURDAY OCT 16

INTRODUCTION AND OVERVIEW
09.00-09.30
Duncan Thomas - Introduction and welcome
Ritu Sadana, WHO - Goals of the workshop
Jennifer Madans, NCHS - NCHS approach to collection of health indicators

HEALTH OVER THE LIFE COURSE: MEASUREMENT AND INTERPRETATION
09.30-11.00
Chair: Elizabeth Frankenberg
Barbara Cohn - Evidence from US
Michael Wadsworth - Evidence from Europe
General discussion

INTERVIEW BASED ASSESSMENTS
11.00-13.15
Chair: Elizabeth Frankenberg
Bob Wallace - Health status in the HRS
Jane Gentleman - Health in the HIS
John Ware - Use of health scales
General discussion

ADLs
14.15-15.15
Chair:John Strauss - ADLs and physical assessments
General discussion

NUTRITION
15.30-17.30
Jean-Pierre Habicht - Nutrition and health
General discussion
18.30
Reception at the home of Kin Bing Wu and Dean Jamison

SUNDAY OCT 17
CLINICALS & SPECIMENS
9.00-12.30
Chair: Duncan Thomas
Theresa Seeman - Experience in US: MaCarthur project
Raynard Kington - Experience in US: NHANES
Greg Pappas - Prospects, problems and prerequisites in LDCs
General discussion

HEALTH EXPENDITURES AND HEALTH PRICES
13.30-15.00
Chair: Ritu Sadana
Dana Goldman - Experience with NMCES and Admin. data
Mike Hurd - Experience in HRS
Elizabeth Frankenberg - Experience in LDCs
Jim Smith - Critical unresolved issues
General discussion
 

PART II: MEASUREMENT OF HEALTH: RECENT AND PLANNED SURVEYS IN LOW INCOME POPULATIONS
 

The second part of the workshop will focus on practical issues that are germane to surveys in low income populations. A goal is to bring together the information that has been laid out during the prior day and half along with recent field experience with health measurement and use that knowledge to inform the next generation of health surveys in LDCs.

LESSONS FROM THE FIELD: feasibility, costs and benefits of different approaches
15.30-17.30
Chair: John Strauss
Elizabeth Frankenberg - experience from Indonesia
Kelly Hallman Bangladesh
Omar Khan (by speaker phone) Pakistan
Randall Kuhn Bangladesh
General discussion

MONDAY 18 OCT
LOOKING INTO THE FUTURE
09.00-12.00
Review of what has been learned from the workshop with specific reference to forthcoming surveys in low income populations. Informal discussion about the big questions that need to be addressed and the approaches that should be considered in surveys currently being planned in Bangladesh, Indonesia, Malaysia and Mexico. Financial support for the workshop was provided by the World Health Organization, the National Institute on Aging (through the Aging Center at RAND) and the National Institute Child and Human Development (through the Population Research Center and the Center for Research on the Family and Development Center at RAND).


RAND/UCLA/WHO Workshop
on
Health Status Measurement in Social Surveys

Participants

  • Victoria Beard, RAND

  • Kathleen Beegle, RAND

  • Alok Bhargava, University of Houston

  • Jay Bhattacharya, RAND

  • Virginia Cain, National Institutes of Health

  • Barbara Cohn, Public Health Institute, Berkeley, California

  • Eileen Crimmins, University of Southern California

  • Julie DaVanzo, RAND

  • Elizabeth Frankenberg, RAND

  • Jane Gentleman, National Center for Health Statistics/CDC

  • Dana Goldman, RAND

  • Jean Pierre Habicht, Cornell Unviersity

  • Kelly Hallman, International Food Policy Research Institute

  • Paula Hamilton, RAND

  • Michael Hurd, RAND

  • Dean Jamison, UCLA and World Health Organisation

  • Omar A. Khan, Johns Hopkins University

  • Raynard Kington, National Center for Health Statistics/CDC

  • Randall Kuhn, RAND

  • David Kurth, RAND

  • Jennifer Madans, National Center for Health Statistics/CDC

  • Joyce Mann, RAND

  • Gregory Pappas, Office of the Assistant Secretary for Public Health and Science

  • Endang Pudjani, Lembaga Demografi

  • Iip Rifai, RAND

  • Luis Rubalcava, Centro de Investigacion y Docencia Economicas (CIDE), Mexico City

  • Ritu Sadana, World Health Organization

  • Muda Saputra, (Ciput) Lembaga Demografi

  • Sondi Sararaks, Public Health Institute, Kuala Lumpur, Malaysia

  • Narayan Sastry, RAND

  • Teresa Seeman, School of Medicine, UCLA

  • Bondan Sikoki, RAND and University of Port Harcourt, Nigeria

  • James P. Smith, RAND

  • John Strauss, Michigan State University

  • Paramita Sudharto, Ministry of Health, Jakarta, Indonesia

  • Sukria Sumantri, (Cecep) Lembaga Demografi

  • Wayan Suriastini, Lembaga Demografi

  • Graciela Teruel, Universidad Iberoamericana, Mexico City

  • Duncan Thomas, RAND and UCLA

  • Michael Wadsworth, Univesity College, London

  • Robert Wallace, University of Iowa

  • John Ware, New England Medical Center

  • David Weir, University of Chicago

  • Robert Willis, University of Michigan

  • Kin Bing Wu, The World Bank

  • Peter Yau, UCLA

  • Julie Zissimopoulos, UCLA

RAND Home Stay Informed Search RAND Publications View Cart