The data: checking and management
What is the Charity Commission’s data cleaning methodology?
The Charity Commission runs a number of checks on the information entered by charities into the Annual Return, before the data is released to CAF. These checks include automated and manual checks.
Manual checks must be run on charities which have submitted their Annual Return by paper rather than online; and are also conducted for issues such as whether a charity has classified themselves into the correct cause categories. Where any errors are suspected, the Charity Commission resolves the issue by speaking to charities, and any necessary changes to financial data must then be signed off by their trustees.
Automated checks are run on numerical information, and there are 37 automated checks on Part B of the Annual Return alone. These checks generate prompts while completing the form online if certain logic rules are not met; and some of these take the form of ‘fatal error’ messages which mean that a correction must be made, and an example trigger for this is where a voluntary income figure entered is lower than the combined value of legacies and endowments. The other type of prompts are ‘warning’ messages which require that a charity confirms that the number that they have entered is correct, and an example trigger for this is where a charity says that it has more than 100,000 employees.
What is CAF’s data manipulation methodology?
When CAF receives the information from the Charity Commission it is imported into an Access database using the DTS (Data Transformation Service) functionality of SQL server 2000 (a relational database management system). The Research team is then able to interrogate the database to create tables and charts of the data, and these are used in the CAF research publications that are available on this website. The Research team also gives the data from the Charity Commission to the contractor that created and maintains the search tool, for them to use the latest data on the website.