Transparency troubles at Recovery.gov
When the American Recovery and Reconstruction Act passed back in February, its backers promised an unparalleled level of disclosure about where the money is going. While I applaud the sentiment, the release of stimulus data in the last month has been anything but smooth.
It’s a classic case of over-promising and under-delivering.
First, it was the job numbers. Now comes the phantom Congressional districts.
Since much of the direct stimulus aid goes to state capitals, any attempt to analyze stimulus spending by Congressional district alone would skew the figures. In the chart below, about half of the stimulus awards in Oklahoma — $1.3 billion — is going to the 5th Congressional District, which includes Oklahoma City.
Source: Recovery.gov
After our story came out today, one reader e-mailed and said recipients could have entered their state House or Senate districts by mistake instead of the Congressional districts. For me, more troubling than the so-called “fake” Congressional districts is the fact that a good chunk of the data don’t include any Congressional district at all (the “Null” field above).
Still, in the wake of all the negative press about the Congressional districts, the officials behind Recovery.gov said they have updated the data on the site that erroneously placed stimulus awards in nonexistent districts.
If a recipient reported an incorrect or invalid congressional district, the code “ZZ” appears in the “Congressional District” field as a placeholder. The recipient will change the report with the correct congressional district during the next reporting period, beginning January 1, 2010.
Also, the federal watchdogs in charge of stimulus spending were grilled on Capitol Hill this morning. In a related report, the Government Accountability Office had some interesting things to say about how stimulus transparency has fared so far. What I found interesting is its investigators started doing error analysis on the same day the data was released to the general public.
GAO performed an initial set of basic analyses on the final recipient report data that first became available at www.recovery.gov on October 30, 2009; reviewed documents; interviewed relevant state and federal officials; and conducted fieldwork in selected states, focusing on a sample of highway and education projects.
While much of the blame for erroneous or missing data is being placed at the recipients who filled out the Web forms on FederalReporting.gov, many are asking if there shouldn’t be some type of validation for the data before it’s released to the public.
The guidance released back in the summer by the Office of Management and Budget lays out who is responsible for data quality in stimulus reporting.
Data quality is an important responsibility of key stakeholders identified in the Recovery Act. Prime recipients, as owners of the data submitted, have the principal responsibility for the quality of the information submitted. Sub-recipients delegated to report on behalf of prime recipients share in this responsibility. Agencies funding Recovery Act projects and activities provide a layer of oversight that augments recipient data quality. Oversight authorities including the OMB, the Recovery Board, and Federal agency Inspectors General also have roles to play in data quality. The general public and non-governmental entities interested in “good government” can help with data quality, as well, by highlighting problems for correction.
To be fair, more than 100,000 reports on stimulus spending have been filed so far. Some data entry errors or miscategorizations were bound to happen when you have that many people filling in Web forms. Here’s what the GAO said it found:
Our review also identified a number of cases in which other anomalies suggest a need for review: discrepancies between award amounts and the amounts reported as received, implausible amounts, or misidentification of awarding agencies. While these occurred in a relatively small number of cases, they indicate the need for further data quality efforts.
Officials hope that future stimulus reports will contain fewer errors as recipients become more comfortable filling out the forms and the requirements are refined. In the meantime, Recovery.gov could make an easy fix by allowing users of the site to flag data that plainly looks wrong. It could function much like the “star ratings” systems on other sites, where users could “grade” their view of the data accuracy in a report.
For more on Recovery.gov data quality, check out the Sunlight Foundation and ProPublica. The White House also published a lengthy retort to critics earlier this week on its blog.
–Paul
UPDATE, 6:10 p.m., 11/19/09: From the prepared remarks of the Earl Devaney, chairman of the Recovery Accountability and Transparency Board:
These mistakes do not surprise me, however, and in a serendipitious way, they are not unequivocally bad. In reality, this data should serve in the long run as evidence of what transparency can achieve.
In the past, this data would have been scrubbed from top to bottom before its release, and the agencies would never have released the information until it was perfect. You — and the American public — are now seeing what agencies have seen, internally, in the past. And what we are all seeing, at least following this first reporting period, is not particularly pretty.
This raw-form, unsanitized data may cause embarrassment for some agencies and recipients, but my expectation is that any embarrassment suffered will encourage self-correcting behavior and lead to more accurate reporting in the future.
Did Cash for Clunkers help?
There’s some healthy debate out there as to whether the government’s Cash for Clunkers program really helped spur auto sales and boosted the economy last summer.
My story about the Oklahoma rebates is here, but some data I requested from the Oklahoma Tax Commission came in too late for my deadline. I wanted to know whether the Cash for Clunkers rebates had much effect on the number of new vehicles titled by the commission. Here’s the month-by-month breakdown for January to October for 2008 and 2009:
Source: Oklahoma Tax Commission
I’m wary of drawing too many conclusions, but it looks like the number of new vehicles titled in Oklahoma did get a boost in the late summer months this year. It’s clear that new vehicle sales were pretty slow in the early part of this year. Overall, more than 8,700 vehicles in Oklahoma qualified for Cash for Clunkers rebates. Those sales were spread over the three months of the program, which ended in late August and spilled over into early September.
Paul Taylor, the chief economist for the National Automobile Dealers Association, said the Cash for Clunkers program probably helped the economy in the third quarter as states received extra tax revenue and showrooms stayed busy. It may also have some spillover effects into the fourth quarter as auto manufacturers, which had slashed production in the wake of sluggish sales, ramped up production on assembly lines to replace inventory.
The Clunkers program lit up the market. Auto showrooms went from almost empty to overflowing. It’s hard to imagine how anyone who takes an objective look at the Cash for Clunkers program can reach any conclusion other than it gave a dramatic boost to retail sales and manufacturing output,” Taylor said.
The results on the environmental front are a little more mixed. Sure, some true clunkers and gas guzzlers were taken off the road and crushed in salvage yards. But many of the new vehicles bought using the rebates were trucks, so it’s done little to change consumer habits. And gas prices are down from their record highs in 2008, so that tiny economical car doesn’t look as attractive as it once did when gasoline was topping $4/gallon.
–Paul
–UPDATE: If you want to download the Oklahoma data yourself, just go here. It’s a pretty large Excel file of about 5MB.
–UPDATE: I just stumbled across this Daily Show clip from last night. Apparently, the demolition-derby constituency isn’t very happy with the whole Cash for Clunkers program.
| The Daily Show With Jon Stewart | Mon – Thurs 11p / 10c | |||
| Crash for Clunkers | ||||
|
||||
Technology and the OK Supreme Court
A recent ruling by the Oklahoma Supreme Court illustrates a common complaint of government agencies and bureaucracies, namely that they haven’t kept up with the expansion of technology.
The court ruled earlier this month that any requests for bulk electronic court case data–basically large downloads or exports of court record information–was now off limits. (Read my story here; the Tulsa World’s story is here. For more on the idea of bulk data downloads from government, check out Web guru Tim O’Reilly’s blog.)
Over the last few decades, Oklahoma county district court clerks have moved their case management systems from paper files to electronic formats. But there are two separate systems, the state-run Oklahoma State Courts Network and the privately run On Demand Court Records. They’ve each evolved over time in response to the needs of district court clerks across the state. There are 13 district courts–including the state’s two largest counties–covered by OSCN; 64 district courts have signed up with ODCR, which is operated by KellPro Inc. of Duncan.
The Oklahoma Supreme Court ruling followed an open records request earlier this year by an Edmond-based firm, INAD Data Services LLC. The company requested electronic copies of all court case information for state district courts and the Oklahoma Workers’ Compensation Court.
The request appeared to have caught Supreme Court administrators off guard. They responded to INAD’s attorney with a letter in late July. The letter said some court information was confidential under state and federal law. It also said the Supreme Court would be asking the Attorney General’s office for advice. The Supreme Court Administrative Office’s general counsel, Debra Charles, said:
If I am satisfied that all or a portion of the records on OSCN.net can be released in bulk, you should anticipate paying a reasonable fee to search and reproduce all of the public records that can be appropriately segregated for public view. Please understand that, if the electronic records can be reasonably segregated, the cost of a system-wide search of this nature will likely be significant.
We’re still trying to figure out what happened in between that letter and the Supreme Court’s ruling Oct. 8 to forbid release of bulk records. But Oklahoma Chief Justice James Edmondson (the brother of Attorney General Drew Edmondson), sent a letter to INAD’s attorney and state Rep. Mike Reynolds, R-Oklahoma City, dated Oct. 5. The letter, which was effectively a denial of the open records request, stated:
… Copies of all information, documents and electronic court records would encompass cases dating back to 1984 in Oklahoma and Tulsa counties. During the intervening 25 years, the other 75 counties have moved to digital record keeping. This literally amounts to millions of pages.
In summary, everything you have requested can be readily accessed through oscn.net or is available on a case-by-case basis at any court clerk’s office in each of the courthouses in Oklahoma. We direct you to oscn.net for full and complete open access to court information.
Later that week, justices issued their ruling.
Justice Edmondson said Tuesday that the ruling was intended to cover only commercial requesters of bulk court data. Other requests, such as those made by noncommercial entities or the media, would be decided on a case-by-case basis, he said. But that’s not explicitly stated in the ruling.
Furthermore, Edmondson said the court signed a contract with KellPro to convert ODCR court case data in preparation for a single, unified Web site of district court information. The contract authorizes the court to spend up to $1.15 million this fiscal year for the data conversion.
Tim Keller, the founder of KellPro, said the contract covers only the data conversion. He expects the Supreme Court will put the work of the unified system out to a competitive bid once the data conversion is finished. In the meantime, KellPro is selling expanded access to court information to members of the Oklahoma Bar Association.
Since hearing about the Supreme Court’s ruling late Monday, I’ve had several conversations with attorney Doug Wilson of Stillwater, who specializes in electronic government information and data. Wilson said the ruling could raise constitutional issues, including one that forbids the state from granting a preference to one company over another.
To play devil’s advocate, I can see why KellPro would like to protect its system of court information. After all, the company’s employees spent time, energy and money pursuing case management system contracts with district court clerks across the state. For a company to come after the fact and request bulk information from their systems doesn’t seem fair.
However, the information isn’t KellPro’s in the first place. They developed the system and software, but the “ownership” stake of the records themselves lies with the people of Oklahoma, whose tax dollars fund the state’s legal system.
It’s interesting to note that this isn’t the first time the Supreme Court has gotten involved in Web access to court documents. In response to privacy concerns, the court issued a ruling last year that would have stopped online access to court filings across the state. They rescinded that ruling after complaints from the public, the media and open-records advocates.
–Paul
About those stimulus job numbers so far
New figures were released this morning on Recovery.org about the estimated jobs saved and/or created from stimulus spending so far. The numbers are coming from the first-round of information reported by contractors earlier this month.
In Oklahoma, the results are underwhelming to say the least. According to the site, Oklahoma companies have signed 120 stimulus-related contracts so far for $92.3 million. And the jobs created or saved? Just 202.
Nationwide, about 30,300 jobs have been created or saved so far, according to data collected so far. That’s not much considering the economy needs to be creating about 100,000 jobs each month just to keep up with population growth.
One White House economist, Jared Bernstein, said it’s still too early to say whether the stimulus is working as intended. But he pointed to “private estimates” as proof that many more jobs are being created.
“It is too soon to draw any global conclusions from this partial and preliminary data, as it reports on just $16 billion of the $339 billion in Recovery Act efforts before September 30th, but the early indications are quite positive. The direct count by Recovery Act recipients of jobs created or saved from this small percentage of the Recovery Act exceeds our projections. All signs — from private estimates to this fragmentary data — point to the conclusion that the Recovery Act did indeed create or save about 1 million jobs in its first seven months, a much needed lift in a very difficult period for our economy. We look forward to the much larger, comprehensive report due on October 30th.”
Just last month, the president’s Council of Economic Advisers put out its estimates of stimulus-related job creation in the first-quarter. Here’s the relevant table:
Source: The White House
Buried deep in the report, the council says it used three methodologies to estimate job impacts by state.
None of these three approaches does a perfect job of measuring the geographic distribution of employment effects, and each has advantages and disadvantages relative to the others. Thus, to obtain a reasonable estimate of state-level job impacts, we use a simple average of the three approaches.
Of course, simply because their populations are larger, we estimate that larger states have seen larger jobs impacts. Similarly, because their employment is more cyclically sensitive, industrial states are estimated to have had larger employment effects relative to their populations. Finally, both because of their industrial composition and because state fiscal relief and aid to those directly impacted have been larger in states hit harder by the recession, we estimate that states with higher unemployment rates at the time of passage have seen larger employment effects of the ARRA relative to their populations.
The Washington Post has a good wrap-up of the expectations created, and the reality of reporting job figures, here:
… Others say the reports being released this month will underscore the challenge of trying to quantify the jobs being created. Initial recipients of the stimulus money, and any government or company that they pass it on to, must report how they use the funds and how many jobs they create. But the reporting requirements do not apply to additional levels of contractors who receive the money.
My advice is to treat those early job numbers as estimates and best-guesses, at least until we get more information later this month and in the months to come.
To find out who’s getting stimulus contracts so far, just check out Recovery.gov. Here’s a list of the Oklahoma contracts signed as of earlier this month, either by Oklahoma companies or for work to be done in Oklahoma. (We also have a link to the state government’s stimulus site on our Right to Know page, which includes other databases of local interest.)
–Paul
Finding state and local budget details
The recession and lower tax revenues are crimping state and local budgets, so it’s good to see several policy groups are making it easier for Oklahomans to find information about their government spending and taxation.
The Oklahoma Policy Institute has just released its Online Budget Guide, a detailed look at state and local budgets in Oklahoma.
I’ve spent a little time checking out the site, and it’s certainly comprehensive. There’s a wealth of information, and the Oklahoma Policy Institute folks say they are committed to keeping the facts and figures timely and relevant. They plan to later add information about federal stimulus spending in Oklahoma and possibly details of state and local bond debt.
As Matt Guillory, executive director of the institute, puts it:
“We’ve designed the Guide to be a resource for anyone interested or affected by government finance in Oklahoma. Those just getting interested will find it to be a clear and simple overview, but it will also serve as a great reference tool for legislators, advocates, members of the media, teachers and others with greater experience in budget issues.”
The nice thing about the guide is that it also includes information on all types of money flowing into state and local coffers, not just sales, income or property taxes. For example, it has information on federal pass-through money, user charges, utilities and trust revenues.
On the expenditure side, the guide has a look at where the money goes and how effective that spending is.
“This is a look at not only what we’re spending, but what we’re getting in return,” said Paul Shinn, the institute’s consultant and the primary author of the guide.
The Policy Challenges section gets into the institute’s bread-and-butter advocacy for tax fairness in Oklahoma, as well as some long-term fiscal challenges that lawmakers will have to deal with in the future.
“We don’t pretend to have solutions, but we offer options,” Shinn said.
If you want the Cliff’s Notes version of the budget guide, you can download the highlights here.
For more on state spending, check out the state’s Open Books site. The link is also on our Right to Know page.
–Paul
Dueling health insurance stats
Figuring out who has health insurance coverage and who doesn’t is an ongoing challenge for policymakers.
Do you count people who went without coverage for a week or a month as being uninsured? What about the ranks of the long-term uninsured? How many of them might qualify for government programs or subsidies but just haven’t signed up?
It’s not an easy task, which is why today’s story on the numbers of uninsured might shed some light on the issue. The Census Bureau’s American Community Survey, a wide-ranging sample of 3 million households each year, asked a health insurance coverage question for the first time last year. For a look at how the question was asked, check out page 8 of the survey form.
Our analysis of the Census’ ACS estimates showed that about 22 percent of Oklahomans under 65 went without health insurance coverage when they were surveyed last year. That put Oklahoma at No. 5 in the nation for the percentage of residents younger than 65 without health insurance.
Of course, it didn’t help matters that the Census just two weeks ago put out another survey that had information on the uninsured from its long-running Current Population Survey. That survey showed an estimated 15.9 percent of Oklahomans of all ages were without health insurance in the CPS’ 2-year average from 2007 to 2008. (For more discussion of that survey data, see the Oklahoma Policy Institute posting here, and the thoughts from our editorial page here. The journal Health Affairs also has a more detailed discussion of the CPS health insurance estimates here.)
David Blatt, policy director for the Oklahoma Policy Institute, said that no matter which Census survey policymakers use, “I think the true numbers lie somewhere between those estimates. Whatever the number, we have a heck of a lot of Oklahomans without health insurance coverage.”
Researchers are hopeful that the bigger survey sample of the American Community Survey will provide more accurate figures in the future. Here’s how the University of Minnesota’s State Health Access Data Assistance Center, or SHADAC, puts it:
The ACS is a great development for health services researchers, but as with all surveys, it will have its problems. In summary, the greatest advantage is that the ACS will be a regular source of health insurance coverage for local areas. The timely releases will fill a significant information void. The biggest limitation is format of the health insurance item is and the ability of respondents to recognize what type of health insurance coverage they have. Some error is always expected in survey research, and we have yet to see how it will compare to other surveys.
In the meantime, here’s a look at some of the latest uninsured estimates from the Census’ American Community Survey for the Oklahoma counties and Congressional districts covered under the latest 2008 survey data:
Finally, NPR has a nice set of interactive maps using the same data:
–Paul
Oklahoma stimulus funding update
The latest stimulus figures were released this morning by the state. Here’s a quick glance at which agency has gotten funding through Aug. 31.
Source: Oklahoma Office of State Finance
–Paul
Obama’s school speech visualized
President Obama’s school speech has been big news for the last week or so, and today’s the day. The White House released the text of Obama’s speech, so I ran it through Wordle to get a text cloud.
–Paul
What does the Internet know about you?
We volunteer more and more information to the Internet, but have you stopped to think about what can be gleaned about you, your lifestyle and family from blog posts, social media sites and friends’ pages?
I came across this site, Personas, earlier this week. It’s an online art “installation” from an exhibit at the Massachusetts Institute of Technology. You plug in your name, and it performs some complex, behind-the-scenes calculations using algorithms and data mining techniques to see what it can find out about you on the Internet. Think of it as a vanity Google search on speed.
Here’s what it said it found out about me from the Internet, split into broad categories:
Of course, as the installation notes, it’s meant as a critique of information gathering and data mining.
Personas attempts to demonstrate this process. It does not reveal where its data comes from, nor does it allow you to weight the inputs. The model designer chose how to build the pre-determined categories and underlying statistical techniques to reflect her world view and a priori knowledge. Uncanny insights and inaccuracies are a part of the intended experience, inviting you to reflect on the larger social consequences of an empirically-driven world.
Still, I was curious where it got so much of my information about movies. I’m a big movie fan, but I haven’t written much online about movies. Then I remembered my dormant MySpace page, which I hadn’t logged into in more than two years. That profile had some information on my favorite movies.
With Google bots and spiders and all manner of information trawling, nothing is ever truly gone from the Web. But I decided to close my MySpace account anyway.
–Paul
Should OKC withhold name of disciplined employee?
My colleague Bryan Dean has been following the fallout of a reorganization of Oklahoma City departments, which has apparently unearthed some questions about a federal grant program.
The city recently moved its Weed and Seed Program, a federally funded urban crime and quality of life initiative, from the now defunct neighborhood services department to the police department. (Read Bryan’s stories here and here.)
The City of Oklahoma City has put two employees on leave after an investigation into Weed and Seed grant funds. Officials claim they accidentally gave the newspaper the name of the first employee, and they’re refusing to give the name of the second employee disciplined.
Meantime, the city also refused to release the date of birth of the first employee, Ed Martin. The newspaper is seeking his date of birth so it can find out more about Martin’s background. But with nothing more than a name, verifying public records we already have makes it virtually impossible to find out which of the seven Ed Martins in Oklahoma City is a city employee.
In a rather bizarre twist of logic, the city attorney said releasing Martin’s date of birth would be an invasion of privacy because it’s listed on his drivers’ license.
Before we go any further, let me ask you the last time you were asked to show your drivers license to a stranger? I do it an average of 3 or 4 times a week at a variety of retail establishments. I hardly consider my date of birth to be super-secret, private information.
Legally, city attorneys do have a point with certain drivers license information being closed to the public. But as Oklahoma State University journalism associate professor Joey Senat points out, that federal law governs the state Department of Public Safety, which issues drivers licenses. It does not govern the disclosure of city personnel actions:
The federal statute doesn’t explicitly list date of birth among the personal information on a driver’s license that should not be disclosed.
‘Personal information’ means information that identifies an individual, including an individual’s photograph, social security number, driver identification number, name, address (but not the 5-digit zip code), telephone number, and medical or disability information.
The fight over public employee records and dates of birth also has been brewing in Texas, where the Legislature has been trying to close access. My friend and former coworker, Ryan McNeill, has more at the Dallas Morning News Investigates blog.
Further afield in Washington state, former Seattle Mayor Greg Nickels asked that state’s attorney general to make a ruling on closing access to city employee records and dates of birth. After researching the issue, Attorney General Rob McKenna said there was no basis for such a decision:
I have looked carefully into this matter and have concluded that public employees’ dates of birth should not be statutorily or categorically exempt from disclosure, for three main reasons. First, dates of birth are already widely available on the Internet and elsewhere, including state voter registration records that are publicly available. Thus, for practical purposes, there is simply no privacy interest left to protect. Second, dates of birth are an important tool to help keep government accountable. Finally, a more targeted and effective way to fight identity theft would be to allow consumers to freeze access to their credit histories to prevent identity thieves from opening credit accounts in their names.
One of the most cited reason for closing public records, identity theft, is largely a red herring. When asked, our local police couldn’t name an instance of identity theft from public records. Most identity theft comes from old-fashioned thefts, lost wallets or a wayward relative with easy access to credit cards or mail, according to the latest research report by Javelin:
Despite the hefty blame – largely perpetuated by the media – placed on the Internet and cyber-crime, online identity theft methods (phishing, hacking and malware) only accounted for 11% of fraud cases in 2008. The truth is, most known cases of fraud occur through traditional methods, when a criminal has direct, physical access to the victim’s information. These instances include stolen and lost wallets, checkbooks, or credit cards, or even through the simple act of a criminal surreptitiously eavesdropping into your conversation as you make a purchase.
For more on the issue of public records, privacy and ID theft, check out my previous blog posts here and here.
–Paul
