The results of an apparently invalid flood-mitigation equity formula could be used to steer billions of dollars in future flood-mitigation funding. Multiple data quality and collection issues may compound errors and the formula itself sometimes renders inconsistent, counter-intuitive results.

In the 6/28/22 Harris County Commissioners Court meeting, the Community Flood Resilience Task Force presented its first annual report. The report contains a lengthy discussion of a flood-mitigation equity formula developed by several Task Force members to “objectively” compare the “equity” of flood-mitigation investments (project costs). See the appendix starting on Page XVI and ending on Page XX.

The attempt to create objective comparisons between investments in different areas is well intentioned. However, I fear the proposed formula will create the appearance of objectivity while skewing data and producing misleading results. Here’s why.

The Formula

The formula is…

Flood Mitigation Benefits Index = Total Cost to Date/(Population Density X Risk)

…where…

• Population Density is the number of people per square mile, calculated at the US Census Tract level.
• Flood Risk is the current annual chance of inundation. For instance, a 1% chance = 1. Or a 10% chance = 10, etc.
• Total Cost to Date shows cumulative dollars spent on flood-risk-reduction projects (construction only, adjusted for inflation) over the longest time period for which records are available, calculated at the US Census Tract level.

The report claims that a higher index means people have received more investment and therefore have less flood risk (i.e., more benefit). Conversely, a low index indicates less investment, more risk and less benefit.

The focus on census tracts is designed to make the data more granular than watersheds. Flood risk estimates will be averaged across the census tract and updated after MAAPnext data becomes available.

Here are several issues I have articulated to the Task Force.

Data Collection and Quality Issues

1. Calculating only construction costs excludes other capital improvement costs such as engineering, design and right-of-way acquisition. Since 2000, construction costs have comprised only 40% of capital improvement costs. See below. And those costs don’t even reflect maintenance and repairs, which are crucial in reducing flooding.

2. According to the report, costs factored into the formula will include those from City of Houston projects and Harris County Flood Control projects. But they don’t include other municipalities’. There are at least 33 other cities in the County. The formula will reflect street-flooding risk, but not all spending to reduce that risk.

3. Likewise, it’s not clear whether the risk reflects pre- or post-mitigation spending, or both. Every time I ask about that, I get silence not an answer. Flood Control has spent more than $1.5 billion on flood mitigation since Harvey, while simultaneously developing new flood maps. Will the numerator of the formula sometimes reflect that investment but not the denominator?

4. Readily available digital spending data goes back only to 2000. But the Task Force committee chairman insists on getting data going back to the start of the Flood Control District – in 1937. If those records still exist, they will radically skew historical comparisons between watersheds, many of which were farms or forests until much more recently.

5. Flood risk depends on more than just mitigation investments. It’s a shifting target that has changed multiple times since 1937 as our understanding of rainfall probabilities has improved, and as different jurisdictions recognize that risk at different times. Flood risk also depends on upstream growth. That has been exponential. In the 2020 Census, Harris County had 4.7 million people. But in the 1930 Census, Harris County had only 359,328 – one thirteenth of today’s population, and presumably one thirteenth of the census tracts. So, attributing all change in risk to investment is fallacious.

6. Many of the census tracts have changed since the 1930s. Census tract boundaries are only “relatively” permanent. They often change based on Census results. For instance, when a census tract’s internal population grows over 8,000 persons, it may split into two or more smaller census tracts. Also, census tract boundaries may cross watershed boundaries. Major thoroughfares usually define census tract boundaries, not the direction of flowing water.

7. HCFCD has said they do not collect spending data by census tract. They calculate how much it costs to remove structures from the floodplain. So census tract data will have to be estimated manually – something that makes data-quality experts nervous.

8. Many neighborhoods outside Beltway 8 didn’t exist back in the 1930s. Beltway 8 didn’t even exist then. Nor did Lake Houston; the City began impounding water only in 1954.

9. The formula – designed to reduce flood damage – doesn’t measure flood damage.

10. So much data in this study won’t be directly comparable that I worry the authors won’t be able to highlight areas worthy of future investment. Final results will include compounded error on multiple levels. It doesn’t compare apples to apples; it compares apples to oranges, bananas, blueberries, cherries, strawberries, coconuts, Monty Python’s elderberries and more.

Questionable Validity

In fairness, I’m sure the final report, when it becomes available, would disclose these problems in an appendix or footnotes. But how many people dig into those? And who will “peer review” this study?

I have worked with market research my entire career and know the painstaking extents to which researchers go to ensure validity of their studies.

Validity has to do with accuracy. Are you really measuring what you purport to measure? For instance, is flood risk influenced ONLY by mitigation investment? Or is it ALSO influenced by other factors, such as:

• Amount of insufficiently mitigated development upstream?
• Proximity of homes to streams and the difficulty of obtaining rights-of-way in such situations?
• Elevation of homes above street level?
• Building codes in effect at the time developments were built?
• Type and volume of street drainage?
• Lack of maintenance?
• Upstream drainage regulations?

The answer is a resounding YES to all those questions and more.

Good research studies typically measure the impact of one variable on another variable. For instance, in Harris County, what was the death rate last year among adults over 50 who contracted Covid among vaccinated and unvaccinated groups? Researchers carefully match the two groups being studied for factors such as randomness of subject recruitment, age, living situation, and history of other diseases. There is only one variable: vaccination. That way, they can tell whether the death rate varies among vaccinated people.

But the Flood Mitigation Benefit Index wasn’t designed with that kind of rigor. For example:

While purporting to compare ‘benefits’ of flood-mitigation to different areas, it doesn’t even employ pre- or post-measurements.

Further reducing comparability of results during the period studied:

• Census tracts changed.
• Population density changed.
• Building codes changed.
• Channels filled up with sediment, but maintenance won’t be measured.
• AND the data does not measure street-flooding mitigation investments in almost HALF the county.

Because the flood-equity formula doesn’t control for such factors, we won’t know what caused variation in the results.

Formula Produces Inconsistent Results

The flood-equity formula does not even yield results that vary intuitively. For instance, when you hold population density and flood risk constant, but increase investment, the benefit goes up.

• Example A: If Density = 5000, Risk = 10 and Investment = $100,000, then Benefit Index = 2
• Example B: If Density = 5000, Risk = 10, and Investment = $1 million, then Benefit Index = 20

So, spending more money to get the same results increases benefits? Shouldn’t it be the opposite?

That’s both depressing and confusing. You spend 10X the money; flood risk remains the same; and the “benefit” increases!!!???

You would think spending less money to achieve identical results would be more beneficial. It certainly is for taxpayers.

Although the Task Force won’t admit it, the formula is really trying to prove “historical disinvestment,” a claim tossed around frequently in Task-Force and Commissioners Court meetings without data to back it up.

But if the goal is to protect the most people from future flooding, why not just invest in projects where the highest risk remains for the greatest number of people? Both of those are simple, unambiguous direct measurements. But those might not produce the results that the authors of this formula hope to get.

I believe we should look forward, not back, with our flood-mitigation dollars. We can’t change the past…whatever it was. We can only affect the future by what we do today.

Posted by Bob Rehak on 7/3/2022

1769 Days since Hurricane Harvey

The thoughts expressed in this post represent opinions on matters of public concern and safety. They are protected by the First Amendment of the US Constitution and the Anti-SLAPP Statute of the Great State of Texas.