New Orleans One Year After Katrina: Obtaining a Representative Sample and Conducting a House-to-House Survey Mollyann Brodie, PhD Vice President, Public Opinion & Media Research, Kaiser Family Foundation DC-AAPOR Seminar: Surveying Under Adverse Conditions October 9, 2007
Giving Voice to the People of New Orleans Sept. 2005: Survey of evacuees in the Houston shelters June 2006: National survey of Americans attitudes toward the recovery effort Sept. Nov. 2006: Face-to-face survey of New Orleans area residents
Lake Pontchartrain Jefferson Parish Orleans Parish St. Bernard Parish Jefferson Parish Plaquemines Parish
ONE YEAR LATER
Challenges we faced Objective: To obtain a representative sample of people currently residing in New Orleans, and be able to talk specifically about key neighborhoods to assist in rebuilding planning efforts Barriers: Lack of any reliable demographic information about who was actually living where Conventional wisdom about influx of Latinos post-katrina Temporary housing situations (FEMA trailer parks, tent cities, hotels, etc.) Lack of money Lack of trained interviewers in the New Orleans area Needed neighborhood-level population estimates in order to properly draw and weight the sample
What we knew at the time Rapidly changing population as people returned to the area Few existing estimates of who was there post-katrina Post office Delivery Sequence File (DSF) was not updated before we went in the field, so we used pre- Katrina DSF combined with estimates of housing destruction by neighborhood to determine how many segments to select in each neighborhood
Methodology Four Parish Area: Orleans, Jefferson, Plaquemines, St. Bernard Face-to-face interviews with 1,504 randomly selected adults Stratified area probability sample: 456 randomly selected Census-defined areas Visited and documented condition of 17,000 housing locations within those areas Random selection of households and individuals, resulting in 1,504 interviews Oversample in Orleans Parish; results weighted so that total reflects expected population in each parish and neighborhood Interviews in English and Spanish Interviewing Sept. 12 thru Nov. 13, 2006 Margin of sampling error +/-4 overall, +/-5 for results based on Orleans or Jefferson Parishes
Pre-Katrina DSF Addresses Street Address City State ZIP Code ZIP+4 Current DSF 117 18TH ST NEW ORLEANS LA 70124 1225 119 18TH ST NEW ORLEANS LA 70124 1225 127 18TH ST NEW ORLEANS LA 70124 1225 129 18TH ST NEW ORLEANS LA 70124 1225 137 18TH ST NEW ORLEANS LA 70124 1225 137 1/2 18TH ST NEW ORLEANS LA 70124 1225 141 18TH ST NEW ORLEANS LA 70124 1225 143 18TH ST NEW ORLEANS LA 70124 1225 145 18TH ST NEW ORLEANS LA 70124 1225 149 18TH ST NEW ORLEANS LA 70124 1225 205 18TH ST NEW ORLEANS LA 70124 1227 X 211 18TH ST NEW ORLEANS LA 70124 1227 211 1/2 18TH ST NEW ORLEANS LA 70124 1227 217 18TH ST NEW ORLEANS LA 70124 1227 223 18TH ST NEW ORLEANS LA 70124 1227 229 18TH ST NEW ORLEANS LA 70124 1227 231 18TH ST NEW ORLEANS LA 70124 1227 237 18TH ST NEW ORLEANS LA 70124 1227 245 18TH ST NEW ORLEANS LA 70124 1227 120 20TH ST NEW ORLEANS LA 70124 1234 124 20TH ST NEW ORLEANS LA 70124 1234 130 20TH ST NEW ORLEANS LA 70124 1234 138 20TH ST NEW ORLEANS LA 70124 1234 144 20TH ST NEW ORLEANS LA 70124 1234 150 20TH ST NEW ORLEANS LA 70124 1234 220 20TH ST NEW ORLEANS LA 70124 1236 228 20TH ST NEW ORLEANS LA 70124 1236 230 20TH ST NEW ORLEANS LA 70124 1236 232 20TH ST NEW ORLEANS LA 70124 1236 234 20TH ST NEW ORLEANS LA 70124 1236 236 20TH ST NEW ORLEANS LA 70124 1236 238 20TH ST NEW ORLEANS LA 70124 1236 240 20TH ST NEW ORLEANS LA 70124 1236 242 20TH ST NEW ORLEANS LA 70124 1236 246 20TH ST NEW ORLEANS LA 70124 1236 256 20TH ST NEW ORLEANS LA 70124 1236 6165 FLEUR DE LIS DR NEW ORLEANS LA 70124 1250 6176 PONTCHARTRAIN BLVD NEW ORLEANS LA 70124 2043 6178 PONTCHARTRAIN BLVD NEW ORLEANS LA 70124 2043 6180 PONTCHARTRAIN BLVD NEW ORLEANS LA 70124 2043 6182 PONTCHARTRAIN BLVD NEW ORLEANS LA 70124 2043 6184 PONTCHARTRAIN BLVD NEW ORLEANS LA 70124 2043 X
Fieldworker-assessed destruction status Completely destroyed: apparent utter destruction (i.e. house not standing, no roof remaining) Seriously damaged: apparent but not catastrophic damage (i.e. boarded up windows, seriously damaged roof) Not seriously damaged: any housing unit not fitting in above categories (Note: likely to be an overestimate, since many apparently livable households were vacant and interviewers could not determine conditions indoors)
Housing Destruction by Parish Orleans Jefferson St. Bernard Plaquemines % completely destroyed 6% 1% 25% 6% % seriously damaged 27% 3% 38% 13% % not seriously damaged 54% 77% 32% 77% % undetermined/ no access 13% 19% 4% 4% Total households observed or visited 11,332 4,518 811 240
Housing Destruction in Orleans Parish Orleans Parish Total (11,332 units) Percent completely destroyed or seriously damaged Completely destroyed or seriously damaged Lower 9 th Ward (317 units) 97% Undetermined/ No access 13% 33% New Orleans East (1,336 units) 65% Garden District (1,715 units) 22% 54% Uptown (1,813 units) 22% Not seriously damaged Algiers (1,598 units) 9% Source: The Kaiser Post-Katrina Baseline Survey of the New Orleans Area (conducted September 12 November 13, 2006)
Vacancy Rates in Orleans Parish Among households in Orleans Parish classified as not seriously damaged where interviews were attempted: 52% were occupied 1,577 1,469 48% were vacant
Weighting Original plan was to weight using only the information we found in the field to determine occupancy rates by neighborhood. Problem with the approach is that it is less reliable in smaller neighborhoods where we only visited a few blocks. In October 2006, an updated DSF file became available from USPS, so it was possible to use DSF to create independent estimates of occupied housing units for each segment. HOWEVER, we were concerned that DSF might overestimate occupied households in a place like New Orleans, where people might be receiving mail but not living in a given household. FURTHER, we were concerned that the DSF overstatement might vary by neighborhood, and this variation could be dependent on the level of post-katrina destruction. SO, we used the results of our fieldwork to create an adjustment factor for each neighborhood by matching entire blocks from our sample to the DSF file and counting how many occupied households our fieldworkers found compared with how many DSF said were occupied.
0.86 0.86-85 542 627 Uptown 0.96 0.96-6 143 149 STBernard 0.83 0.83-20 98 118 Plaquemines 1.03 4 142 138 Noeast -19 162 181 Midcity 1.15 31 244 213 Marigny 0.18-33 7 40 Lower 9th 0.06-51 3 54 Lakeview 0.95 0.95-70 1323 1393 Jefferson 0.49-42 41 83 Gentilly -52 473 525 Garden Dist 1.06 3 52 49 French Qtr 0.54-6 7 13 EnglishTurn --- 13 13 0 BWCooper 1.08 11 156 145 Audubon 0.91 0.91-54 517 571 Algiers 0.93 0.93-12 148 160 7th Ward 0.91-388 4071 4459 Grand Total Stratum Adjustment Factor Raw Diff Fieldwork count: Occupied HHs October 2006 DSF: Active Residences Stratum DSF adjustments by neighborhood 0.86 0.86-85 542 627 Uptown 0.96 0.96-6 143 149 STBernard 0.83 0.83-20 98 118 Plaquemines 1.03 4 142 138 Noeast -19 162 181 Midcity 1.15 31 244 213 Marigny 0.18-33 7 40 Lower 9th 0.06-51 3 54 Lakeview 0.95 0.95-70 1323 1393 Jefferson 0.49-42 41 83 Gentilly -52 473 525 Garden Dist 1.06 3 52 49 French Qtr 0.54-6 7 13 EnglishTurn --- 13 13 0 BWCooper 1.08 11 156 145 Audubon 0.91 0.91-54 517 571 Algiers 0.93 0.93-12 148 160 7th Ward 0.91-388 4071 4459 Grand Total Stratum Adjustment Factor Raw Diff Fieldwork count: Occupied HHs October 2006 DSF: Active Residences Stratum 0.86 0.86-85 542 627 Uptown 0.96 0.96-6 143 149 STBernard 0.83 0.83-20 98 118 Plaquemines 1.03 4 142 138 Noeast -19 162 181 Midcity 1.15 31 244 213 Marigny 0.18-33 7 40 Lower 9th 0.06-51 3 54 Lakeview 0.95 0.95-70 1323 1393 Jefferson 0.49-42 41 83 Gentilly -52 473 525 Garden Dist 1.06 3 52 49 French Qtr 0.54-6 7 13 EnglishTurn --- 13 13 0 BWCooper 1.08 11 156 145 Audubon 0.91 0.91-54 517 571 Algiers 0.93 0.93-12 148 160 7th Ward 0.91-388 4071 4459 Grand Total Stratum Adjustment Factor Raw Diff Fieldwork count: Occupied HHs October 2006 DSF: Active Residences Stratum 0.86 0.86-85 542 627 Uptown 0.96 0.96-6 143 149 STBernard 0.83 0.83-20 98 118 Plaquemines 1.03 4 142 138 Noeast -19 162 181 Midcity 1.15 31 244 213 Marigny 0.18-33 7 40 Lower 9th 0.06-51 3 54 Lakeview 0.95 0.95-70 1323 1393 Jefferson 0.49-42 41 83 Gentilly -52 473 525 Garden Dist 1.06 3 52 49 French Qtr 0.54-6 7 13 EnglishTurn --- 13 13 0 BWCooper 1.08 11 156 145 Audubon 0.91 0.91-54 517 571 Algiers 0.93 0.93-12 148 160 7th Ward 0.91-388 4071 4459 Grand Total Stratum Adjustment Factor Raw Diff Fieldwork count: Occupied HHs October 2006 DSF: Active Residences Stratum
Using secondary information (DSF) reduced the variance of estimates made using only fieldwork information Selection stratum DSF-adjusted estimate Estimate using fieldwork observations only Lakeview neighborhood Total 2,113 2,015 801 703 1,162 802 137 0 803 142 62 804 257 137 805 872 654 SD 620 679 Mean 791 851 CV 0.784 0.798
Population information that became available after we went in the field In June 2006, Census (ACS) released 2005 ACS Special Product for the Gulf Coast Area, containing population and demographic estimates for Sept.-Dec. 2005 for areas affected by the hurricanes, including the New Orleans MSA and Jefferson Parish, but no parish- or neighborhood-level estimates for Orleans In January 2007, Louisiana Health & Population Survey released results of household mail surveys conducted June- October 2006, including population estimates by Parish, but not by neighborhood In April 2007, the Census released updated estimates of population numbers at the Parish level as of July 2006
How our estimates compared KFF LHPS Census Sample frame date October 2006 June 2006 July 2006 Orleans 220,831 191,139 223,388 Jefferson 455,941 434,666 431,361 Plaquemines 16,298 17,860 22,512 St. Bernard 18,278 25,296 15,514 4-Parish Area 711,348 668,961 692,775
When did people come back? 28% 19% 18% 1% 7% 3% 4% 2% 2% 2% 2% 2% 1% 1% 5% Never left Aug- 05 Sep- 05 Oct- 05 Nov- 05 Dec- 05 Jan- 06 Feb- 06 Mar- 06 Apr- 06 May- 06 Jun- 06 Jul- 06 Aug- 06 Sep- 06 Source: The Kaiser Post-Katrina Baseline Survey of the New Orleans Area (conducted September 12 November 13, 2006)
What does this mean for other survey projects in New Orleans? As population becomes more stable, further surveys should be easier Better estimates exist now, including updated demographics from Census (ACS) for 2006 Possible to do phone surveys now and still get a good sample? Need to include cell phones?
Cell phone and land line coverage Cell phone only Cell phone + land line Land line only No cell phone or land line Total 4- parish area 21% 62% 14% 3% Orleans 27% 52% 15% 3% Jefferson 14% 67% 15% 3% Source: The Kaiser Post-Katrina Baseline Survey of the New Orleans Area (conducted September 12 November 13, 2006)
Brookings/Census Report on 2006 New Orleans population: Key Findings One year after the storms, the city of New Orleans black population declined by 57 percent, while its white population decreased by 36 percent (yet the city remained a majority minority community). New Orleans 2006 post-storm population was smaller, older, more educated, less poor, with fewer renters, and fewer households with children than was recorded in Census 2000. Compared with stayers in the city of New Orleans, out-migrants were younger, poorer, more likely to be black, and more likely to have children. One year after the storm, black New Orleanians were most likely to have moved to the Houston metro area, whereas whites mostly moved elsewhere in the New Orleans metropolitan area.
Influx of Latinos? Orleans Jefferson Census 2000 KFF 2006 Diff Census 2000 KFF 2006 Diff Adults by race White non-hispanic 31% 39% +8% 69% 62% -7% Black non-hispanic 62% 53% -9% 20% 19% -1% Hispanic 3% 5% +2% 7% 13% +6% Other 4% 3% -1% 4% 6% +2%
Recent Brookings/Census publication One aspect of these estimates that may appear surprising is the relatively small Hispanic population that appears in the Census Bureau s 2006 estimates for New Orleans city and its metropolitan areas This is the case despite the fact that both [pre- and post-katrina] data sources included group quarters populations as well as household populations living in permanent residences. However, any group quarters that were established after the ACS sampling frame was established were not included in the survey. To the extent that Latino workers moved to New Orleans after the storm and lived in hastily arranged housing, it is very likely that the transitory nature of temporary working conditions of primarily Hispanic construction and service workers has eluded traditional estimation and survey techniques.
What does this mean for surveys in areas with rapidly changing populations? Try to use the best available information at the time Hybrid approaches to sampling and weighting can help