First in a three-part series about ChatGPT and Flood Knowledge
The flood of articles about ChatGPT recently prompted me to wonder whether the artificial intelligence application could accurately answer questions about flooding. So I set up an account, started asking questions, and checked the answers with experts. The responses revealed patterns of strengths and weaknesses.
- The more general the questions, the better the answers.
- The more specific the questions, the more it missed.
- In one test, it seriously missed an answer and gave misleading advice.
- In another, it omitted an important component of the answer.
- Sometimes I got no responses at all.
Despite ChatGPT’s considerable strengths, my test raised a red “caution flag” on this newly minted intellectual booster rocket. In my opinion, the biggest danger in using it is this.
- Use it, but also use it with caution.
- Understand the limitations.
- Drill down into topics and cross check the answers with other sources.
Test Run and Validation
I first asked ChatGPT, “What are the most effective techniques for flood control?” I sent the results to an industry leading hydrologist who specializes in flood control and the leaders of two leading conservation groups in the Houston area.
I asked them to grade the answer on a scale of A through F and to answer two simple questions. Was the ChatGPT response accurate? And did it omit anything important?
ChatGPT Response to Test Question
Below are my question and the ChatGPT response reproduced verbatim:
Q: “What are the most effective techniques for flood control?”
A: “Flood control involves a combination of structural and non-structural techniques to minimize the impact of flooding. Here are some of the most effective techniques used for flood control:
- Dams and Reservoirs: Constructing dams and reservoirs helps regulate the flow of water in rivers and store excess water during heavy rainfall. This reduces downstream flood risk by controlling the release of water.
- Levees and Flood Walls: Building levees and flood walls along riverbanks and coastlines can provide physical barriers to prevent floodwaters from reaching populated areas. These structures are designed to withstand the pressure of water and redirect it away from vulnerable regions.
- Channelization: Modifying and straightening river channels can increase their capacity and improve the flow of water, reducing the risk of overflow. This technique involves reshaping riverbanks, removing obstacles, and dredging to deepen the channel.
- Floodplain Zoning: Implementing land-use planning measures such as floodplain zoning helps prevent development in high-risk areas. By restricting construction and promoting open spaces or green infrastructure, floodwater can be absorbed or diverted, reducing damage to buildings and infrastructure.
- Stormwater Management: Effective stormwater management systems capture and store rainwater, reducing the volume of water entering rivers and drainage systems during storms. Techniques include the use of retention ponds, green roofs, permeable pavements, and sustainable drainage systems (SUDS).
- Wetland Restoration: Wetlands act as natural buffers against flooding, absorbing and storing excess water. Restoring and preserving wetlands can help regulate water flow, reduce flood peaks, and provide habitats for wildlife.
- River Embankments and Terracing: Strengthening riverbanks through embankments and terracing helps contain floodwaters within the river channel. These techniques can reduce erosion and prevent water from spreading onto surrounding floodplains.
- Urban Drainage Systems: Well-designed urban drainage systems, including storm sewers, culverts, and retention basins, efficiently collect and transport stormwater away from urban areas. Proper maintenance of these systems is crucial to prevent blockages and ensure effective flood control.
- Early Warning Systems: Implementing advanced monitoring and early warning systems allows for timely dissemination of flood alerts to at-risk communities. This gives people more time to evacuate or take necessary precautions to protect life and property.
- Public Awareness and Education: Raising public awareness about flood risks, safety procedures, and emergency response plans can significantly reduce the impact of floods. Educating communities on flood preparedness, including evacuation routes and the importance of insurance coverage, helps people respond effectively during flood events.
It’s worth noting that the most effective flood control strategies often involve a combination of these techniques, tailored to the specific characteristics and challenges of the area at risk. Additionally, climate change adaptation measures and sustainable water management practices play a crucial role in mitigating the long-term effects of flooding.”
Evaluation of Response by Experts
Hydrologist/flood expert: “That response is pretty good. It really depends on the audience. I would give it a B+ because it is thorough, but very general. Obviously, it doesn’t take into account the location, development, topography, etc., so that is why it can’t ever give an “A+” answer.”
Environmentalist #1: “I would give this an A. It’s basic, but it’s pretty comprehensive. Especially items 5,6,7, which are not always considered as common strategies employed for flood mitigation.”
Environmentalist #2: “I would give it an A- or B+. It hit many of the green strategies. But it did not seem to talk about preservation of natural ecosystems, like forests, floodplains, beaches, and grasslands. Nor did it discuss the use of hybrid solutions that incorporate engineered features with natural materials that mimic the functioning of natural ecosystems. I would also add another item to the list, ‘Use of native vegetation in residential and commercial areas to increase infiltration.'”
I concluded ChatGPT is a research tool like any other.
It’s definitely faster than search engines that would require you to explore dozens of links in order to compile a list like the one above.
It even composes well-formed summaries/answers for you in simple, easy-to-understand language.
It feels as though you are conversing with a knowledgeable, patient teacher.
It’s a great jumping-off point to start intellectual exploration.
If this were an “Intro to…” or “Principles of…” course, I would give it an A- or B+. It gives good, comprehensive, well-formed answers. The grammar is impressive for something computer generated.
But that last strength can inspire a dangerous overconfidence that will become more apparent in the next two posts.
Also, it never quantified “most effective,” but in fairness, the last paragraph of the response (“worth noting”) explains why that would be problematic.
Down the Rabbit Holes
Armed with my expert’s evaluations, I started down some rabbit holes that revealed more strengths, weaknesses, limitations and an outright error on different aspects of flood control.
But more on those in the next two posts. One involves construction best management practices. Another will focus on the capabilities of a government agency, the Texas General Land Office.
Posted by Bob Rehak on 5/27/2023
2097 days since Hurricane Harvey