New Orleans One Year After Katrina: Obtaining a Representative Sample and Conducting a House-to-House Survey

Transcription

1 New Orleans One Year After Katrina: Obtaining a Representative Sample and Conducting a House-to-House Survey Mollyann Brodie, PhD Vice President, Public Opinion & Media Research, Kaiser Family Foundation DC-AAPOR Seminar: Surveying Under Adverse Conditions October 9, 2007

2 Giving Voice to the People of New Orleans Sept. 2005: Survey of evacuees in the Houston shelters June 2006: National survey of Americans attitudes toward the recovery effort Sept. Nov. 2006: Face-to-face survey of New Orleans area residents

3 Lake Pontchartrain Jefferson Parish Orleans Parish St. Bernard Parish Jefferson Parish Plaquemines Parish

4 ONE YEAR LATER

5 Challenges we faced Objective: To obtain a representative sample of people currently residing in New Orleans, and be able to talk specifically about key neighborhoods to assist in rebuilding planning efforts Barriers: Lack of any reliable demographic information about who was actually living where Conventional wisdom about influx of Latinos post-katrina Temporary housing situations (FEMA trailer parks, tent cities, hotels, etc.) Lack of money Lack of trained interviewers in the New Orleans area Needed neighborhood-level population estimates in order to properly draw and weight the sample

6 What we knew at the time Rapidly changing population as people returned to the area Few existing estimates of who was there post-katrina Post office Delivery Sequence File (DSF) was not updated before we went in the field, so we used pre- Katrina DSF combined with estimates of housing destruction by neighborhood to determine how many segments to select in each neighborhood

7 Methodology Four Parish Area: Orleans, Jefferson, Plaquemines, St. Bernard Face-to-face interviews with 1,504 randomly selected adults Stratified area probability sample: 456 randomly selected Census-defined areas Visited and documented condition of 17,000 housing locations within those areas Random selection of households and individuals, resulting in 1,504 interviews Oversample in Orleans Parish; results weighted so that total reflects expected population in each parish and neighborhood Interviews in English and Spanish Interviewing Sept. 12 thru Nov. 13, 2006 Margin of sampling error +/-4 overall, +/-5 for results based on Orleans or Jefferson Parishes

8

9 Pre-Katrina DSF Addresses Street Address City State ZIP Code ZIP+4 Current DSF TH ST NEW ORLEANS LA TH ST NEW ORLEANS LA TH ST NEW ORLEANS LA TH ST NEW ORLEANS LA TH ST NEW ORLEANS LA /2 18TH ST NEW ORLEANS LA TH ST NEW ORLEANS LA TH ST NEW ORLEANS LA TH ST NEW ORLEANS LA TH ST NEW ORLEANS LA TH ST NEW ORLEANS LA X TH ST NEW ORLEANS LA /2 18TH ST NEW ORLEANS LA TH ST NEW ORLEANS LA TH ST NEW ORLEANS LA TH ST NEW ORLEANS LA TH ST NEW ORLEANS LA TH ST NEW ORLEANS LA TH ST NEW ORLEANS LA TH ST NEW ORLEANS LA TH ST NEW ORLEANS LA TH ST NEW ORLEANS LA TH ST NEW ORLEANS LA TH ST NEW ORLEANS LA TH ST NEW ORLEANS LA TH ST NEW ORLEANS LA TH ST NEW ORLEANS LA TH ST NEW ORLEANS LA TH ST NEW ORLEANS LA TH ST NEW ORLEANS LA TH ST NEW ORLEANS LA TH ST NEW ORLEANS LA TH ST NEW ORLEANS LA TH ST NEW ORLEANS LA TH ST NEW ORLEANS LA TH ST NEW ORLEANS LA FLEUR DE LIS DR NEW ORLEANS LA PONTCHARTRAIN BLVD NEW ORLEANS LA PONTCHARTRAIN BLVD NEW ORLEANS LA PONTCHARTRAIN BLVD NEW ORLEANS LA PONTCHARTRAIN BLVD NEW ORLEANS LA PONTCHARTRAIN BLVD NEW ORLEANS LA X

10 Fieldworker-assessed destruction status Completely destroyed: apparent utter destruction (i.e. house not standing, no roof remaining) Seriously damaged: apparent but not catastrophic damage (i.e. boarded up windows, seriously damaged roof) Not seriously damaged: any housing unit not fitting in above categories (Note: likely to be an overestimate, since many apparently livable households were vacant and interviewers could not determine conditions indoors)

11 Housing Destruction by Parish Orleans Jefferson St. Bernard Plaquemines % completely destroyed 6% 1% 25% 6% % seriously damaged 27% 3% 38% 13% % not seriously damaged 54% 77% 32% 77% % undetermined/ no access 13% 19% 4% 4% Total households observed or visited 11,332 4,

12 Housing Destruction in Orleans Parish Orleans Parish Total (11,332 units) Percent completely destroyed or seriously damaged Completely destroyed or seriously damaged Lower 9 th Ward (317 units) 97% Undetermined/ No access 13% 33% New Orleans East (1,336 units) 65% Garden District (1,715 units) 22% 54% Uptown (1,813 units) 22% Not seriously damaged Algiers (1,598 units) 9% Source: The Kaiser Post-Katrina Baseline Survey of the New Orleans Area (conducted September 12 November 13, 2006)

13 Vacancy Rates in Orleans Parish Among households in Orleans Parish classified as not seriously damaged where interviews were attempted: 52% were occupied 1,577 1,469 48% were vacant

14

15

16 Weighting Original plan was to weight using only the information we found in the field to determine occupancy rates by neighborhood. Problem with the approach is that it is less reliable in smaller neighborhoods where we only visited a few blocks. In October 2006, an updated DSF file became available from USPS, so it was possible to use DSF to create independent estimates of occupied housing units for each segment. HOWEVER, we were concerned that DSF might overestimate occupied households in a place like New Orleans, where people might be receiving mail but not living in a given household. FURTHER, we were concerned that the DSF overstatement might vary by neighborhood, and this variation could be dependent on the level of post-katrina destruction. SO, we used the results of our fieldwork to create an adjustment factor for each neighborhood by matching entire blocks from our sample to the DSF file and counting how many occupied households our fieldworkers found compared with how many DSF said were occupied.

17 Uptown STBernard Plaquemines Noeast Midcity Marigny Lower 9th Lakeview Jefferson Gentilly Garden Dist French Qtr EnglishTurn BWCooper Audubon Algiers th Ward Grand Total Stratum Adjustment Factor Raw Diff Fieldwork count: Occupied HHs October 2006 DSF: Active Residences Stratum DSF adjustments by neighborhood Uptown STBernard Plaquemines Noeast Midcity Marigny Lower 9th Lakeview Jefferson Gentilly Garden Dist French Qtr EnglishTurn BWCooper Audubon Algiers th Ward Grand Total Stratum Adjustment Factor Raw Diff Fieldwork count: Occupied HHs October 2006 DSF: Active Residences Stratum Uptown STBernard Plaquemines Noeast Midcity Marigny Lower 9th Lakeview Jefferson Gentilly Garden Dist French Qtr EnglishTurn BWCooper Audubon Algiers th Ward Grand Total Stratum Adjustment Factor Raw Diff Fieldwork count: Occupied HHs October 2006 DSF: Active Residences Stratum Uptown STBernard Plaquemines Noeast Midcity Marigny Lower 9th Lakeview Jefferson Gentilly Garden Dist French Qtr EnglishTurn BWCooper Audubon Algiers th Ward Grand Total Stratum Adjustment Factor Raw Diff Fieldwork count: Occupied HHs October 2006 DSF: Active Residences Stratum

18 Using secondary information (DSF) reduced the variance of estimates made using only fieldwork information Selection stratum DSF-adjusted estimate Estimate using fieldwork observations only Lakeview neighborhood Total 2,113 2, , SD Mean CV

19 Population information that became available after we went in the field In June 2006, Census (ACS) released 2005 ACS Special Product for the Gulf Coast Area, containing population and demographic estimates for Sept.-Dec for areas affected by the hurricanes, including the New Orleans MSA and Jefferson Parish, but no parish- or neighborhood-level estimates for Orleans In January 2007, Louisiana Health & Population Survey released results of household mail surveys conducted June- October 2006, including population estimates by Parish, but not by neighborhood In April 2007, the Census released updated estimates of population numbers at the Parish level as of July 2006

20 How our estimates compared KFF LHPS Census Sample frame date October 2006 June 2006 July 2006 Orleans 220, , ,388 Jefferson 455, , ,361 Plaquemines 16,298 17,860 22,512 St. Bernard 18,278 25,296 15,514 4-Parish Area 711, , ,775

21 When did people come back? 28% 19% 18% 1% 7% 3% 4% 2% 2% 2% 2% 2% 1% 1% 5% Never left Aug- 05 Sep- 05 Oct- 05 Nov- 05 Dec- 05 Jan- 06 Feb- 06 Mar- 06 Apr- 06 May- 06 Jun- 06 Jul- 06 Aug- 06 Sep- 06 Source: The Kaiser Post-Katrina Baseline Survey of the New Orleans Area (conducted September 12 November 13, 2006)

22 What does this mean for other survey projects in New Orleans? As population becomes more stable, further surveys should be easier Better estimates exist now, including updated demographics from Census (ACS) for 2006 Possible to do phone surveys now and still get a good sample? Need to include cell phones?

23 Cell phone and land line coverage Cell phone only Cell phone + land line Land line only No cell phone or land line Total 4- parish area 21% 62% 14% 3% Orleans 27% 52% 15% 3% Jefferson 14% 67% 15% 3% Source: The Kaiser Post-Katrina Baseline Survey of the New Orleans Area (conducted September 12 November 13, 2006)

24 Brookings/Census Report on 2006 New Orleans population: Key Findings One year after the storms, the city of New Orleans black population declined by 57 percent, while its white population decreased by 36 percent (yet the city remained a majority minority community). New Orleans 2006 post-storm population was smaller, older, more educated, less poor, with fewer renters, and fewer households with children than was recorded in Census Compared with stayers in the city of New Orleans, out-migrants were younger, poorer, more likely to be black, and more likely to have children. One year after the storm, black New Orleanians were most likely to have moved to the Houston metro area, whereas whites mostly moved elsewhere in the New Orleans metropolitan area.

25 Influx of Latinos? Orleans Jefferson Census 2000 KFF 2006 Diff Census 2000 KFF 2006 Diff Adults by race White non-hispanic 31% 39% +8% 69% 62% -7% Black non-hispanic 62% 53% -9% 20% 19% -1% Hispanic 3% 5% +2% 7% 13% +6% Other 4% 3% -1% 4% 6% +2%

26 Recent Brookings/Census publication One aspect of these estimates that may appear surprising is the relatively small Hispanic population that appears in the Census Bureau s 2006 estimates for New Orleans city and its metropolitan areas This is the case despite the fact that both [pre- and post-katrina] data sources included group quarters populations as well as household populations living in permanent residences. However, any group quarters that were established after the ACS sampling frame was established were not included in the survey. To the extent that Latino workers moved to New Orleans after the storm and lived in hastily arranged housing, it is very likely that the transitory nature of temporary working conditions of primarily Hispanic construction and service workers has eluded traditional estimation and survey techniques.

27 What does this mean for surveys in areas with rapidly changing populations? Try to use the best available information at the time Hybrid approaches to sampling and weighting can help