It should come as no surprise that political leaders
enjoy quoting the positive ratings from surveys of the communities that they
serve – a sort of badge of honor for their job performance. Ex-Dallas City Manager A.C. Gonzales is no
exception, making reference to a recent citizen satisfaction survey of 1,512
Dallas City residents that showed an overall community that, with some
exceptions, appeared quite happy with City services. Mayor Mike Rawlings has also referenced these
positive ratings from these surveys as well. At the national level, Republican
nominee Donald Trump recently pointed to positive student satisfaction ratings to
counter allegations of fraud in lawsuits against Trump University. Indeed, positive ratings are like candy to
politicians, whether deserved or not.
But how much faith can we place in these satisfaction
ratings? In a recent column by Dallas Morning News columnist Robert Wilonsky, he
noted the apparent paradox of the City’s continuing high ratings given the multitude
of problems that are left unresolved, such as potholes, loose dogs mauling
citizens in poor neighborhoods, contracting irregularities, deteriorating air
quality, traffic congestion, and a host of other issues. Wilonsky also pointed out that the survey vendor’s
report curiously omitted information about the ages represented by the study
respondents. Indeed, the report tells us
nothing about the satisfaction levels across racial-ethnic groups, income
groups, age groups or other key demographics – information that would provide
more insight on how well the study sample mirrored Dallas’ diverse population.
The City of Dallas is now 41 percent Latino, 24 percent black, 3 percent Asian,
and 29 percent white – a diverse community of residents that are entitled to
have their voices heard in surveys sponsored by their tax dollars.
While City leaders have no problem embracing citizen
satisfaction ratings, we should be cautious about embracing the results of
satisfaction surveys, especially those that consistently show their sponsors in
a positive light. In the case of the City of Dallas, there is reason to believe
that these satisfaction ratings could be inflated and a self-serving exercise
for City leaders:
- Past community surveys for the City have shown a pattern of under-representing certain racial-ethnic groups, age groups, non-English speakers, and the lower income – groups who are more likely to have negative experiences and opinions of City services. Loose dogs and potholes, for example, are more common in poor neighborhoods. To what extent would the positive ratings diminish if the voices of such residents were properly represented in the survey?
- Of course, the survey vendor’s quality of work may be spectacular, making it easier to eliminate the competition. However, the most recent City satisfaction report omitted standard demographic information about the 1,512 city residents that completed the survey. One has no idea if the survey respondents accurately reflected the diversity of this community by race, ethnicity, gender or age. This is information that is considered standard in industry research reports --- information that is commonly used to judge the scientific credibility of the survey findings. Why have City staff allowed the omission of this important information from its report?
- Given the positive ratings that the City continues to enjoy from these surveys, it is not surprising that the survey company that conducts these surveys has enjoyed a preferred vendor status for many years. While the survey contract is bid competitively, the same out-of-state vendor has been successful in obtaining the contract year after year even though there are various local vendors that are equally qualified to conduct the work. Are City leaders and staff concerned that a different vendor would change the positive ratings that they enjoy?
Community satisfaction
ratings provide one measure of the City’s performance in serving a community,
but provide an incomplete picture of its actual performance since key groups
are often omitted or under-represented in such studies. The fascination of City
leaders with these positive ratings and comparisons to other U.S. cities
creates the false impression that everything in Dallas is just peachy. A guided tour of City neighborhoods tells
quite a different story.
Clearly, the next City
Manager for Dallas, as well as the next Mayor, will have a long list of
City-related needs that will require their immediate attention. If the results
of citizen satisfaction surveys continue to be used by City leaders and staff
as a benchmark of their annual or periodic performance, some changes will be
needed to inspire more confidence in the ratings provided by this survey. First, it is absolutely essential that the
public is provided access to a detailed methodology that describes the steps
used to conduct the study, including the extent of support in languages other
than English. This is important because
many studies confirm that over half of Latino and Asian adults prefer to
communicate in their native language, a fact that improves comprehension and survey
participation. Second, the report must
provide a detailed demographic profile of the survey respondents – a standard
requirement in all research industry studies – and perhaps the only evidence
that the random selection of City households resulted in a fair and unbiased
representation of the City’s diverse community.
Lastly, to remove the appearance of favoritism in the vendor selection
process, City staff should be required to justify the continued selection of
one vendor over several years despite the availability of various equally
qualified survey vendors.
