What I work on

Software projects

Current

• 2015 SpenDB, a light-weight web tool for analysing and visualizing government financial data.
• 2014-2015 grano, a social network analysis toolkit for news applications.
• 2012-2014 dataset, easy-to-use Python library for data processing using relational databases.
• 2012-2015 nomenklatura, an entity management tool that provides easy ways to integrate, clean and query datasets.
• 20102-15 OffenerHaushalt, visualization of scraped German budget data on the federal, state and local levels.

Previous

• 2014 OpenInterests, a database that combines information about lobbying and influence in the EU with data about financial transactions.
• 2011-2013 OffenesParlament, legislative activity tracking system for both chambers of the German parliament. Features plenary speeches, legislative drafts and personal profiles.
• 2013 twindle, client toolkit for the Twitter streaming API, storing and analysing statuses.
• 2013 databin, a pastebin site for tabular data, enables easy sharing of small data snippets.
• 2011-2013 ReGENESIS, data extraction from the repository of the German statistical office, data cube reconstruction.
• 2010-2013 OpenSpending. Data warehousing system to support the analysis and visualization of budgetary and spending data from across the globe. Set up budget sites for Cameroon, Slovakia, Nepal, Bosnia and Minas Gerais (Brasil). Contributed to Where Does My Money Go? and the data.gov.uk spend browser.
• 2011-2013 messytables, extracting structured data from badly-formatted tabular data.
• 2012-2013 LobbyFacts API, an API to provide clean and current data on lobby activity in the European Union, cooperation with Corporate Europe, LobbyControl and Friends of the Earth Europe.
• 2011 WebStore, RESTful, SQL-based data store and API generator service.
• 2011 YourTopia, contributor, voting on a custom index for human development criteria. Third price in the World Bank's Apps for Development competition.
• 2011 DataHub, light-weight, experimental, activity-centric clone of CKAN.
• 2010-2011 CKAN, worked on customizations for PublicData.eu, IATI Registry, Helsinki Regional Infoshare, the Data Hub and various community instances, including OffeneDaten.de in Germany.
• 2009-2011 Adhocracy, collaborative policy drafting platform for distributed groups. Deployments for the Commission Internet and Digital Society of the German Parliament, the City of Munich and various other organisations.

Datasets

Many of my projects require that I keep up-to-date copies of various public datasets. These are often scraped, or additional information had been added to facilitate analysis. Below is a collection of resources I'm re-publishing because they are significantly enhanced compared to their source form.

European Union

• Tenders Electronic Daily, the EU's common procurement portal, scraped tenders and contract awards notices from their web site and released data as CSV. For more info, see OpenTED.
• EC Financial Transparency System, the Commission's direct expenditure (data source, scripts). This version has been geo-coded and reconciled with OpenCorporates.
• EC/EP Expert Groups, data processing script to transform data about expert groups from XML into a relational database.
• EC/EP Transparency Register, the API provides up-to-date CSV and JSON exports of a cleaned version of the XML data released by the secretariat. An archive of the register is also being kept here.
• EU budget, released upon request as XML data, processing scripts will convert this to CSV (2005-2010).

For context, also see the briefing on the finance data sources of the European Union.

Germany

• Documents, transcripts and other data from the German Bundestag and Bundesrat, scraped on a nightly basis and significantly extended for use in OffenesParlament.
• German federal budget, scraped from the Ministry of Finance's web site - some of which is now gone. Please note the comments on data quality included in the scraper repository.
• Granular results for the 2009 federal election, voting district level results data from a CD sold by StaBa, unfortunately doesn't include geographic boundary data.
• Research funding of eight German ministries, scraped from the Förderkatalog portal.
• Director's Dealings, German banking oversight dataset on stock transactions of officers of publicly traded companies.
• Signatories of parliamentary petitions, as released by the previous version of the Bundestag's petition system. A historic database for research is available upon request.
• Administrative boundaries (Kreise), web-ready SVG vector maps based on the official BKG data.

United Kingdom

• Central Government Monthly Expenditures (in cooperation with data.gov.uk), aggregating detailed transactional expenditure data from all central government authorities into a single dataset.
• Whole of Government Accounts (WGA), attempts to decipher the audited accounts section of the UK's COINS database, which involves a particularly nasty Excel sheet.

Community projects

• 2012-2013 StadtLandCode, with Julia, an incubator for civic applications in Germany. We eventually helped to bootstrap Bürger Baut Stadt, Teikei and Argufactum.
• 2012-2013 OKFNLabs, community of data wranglers and series of hack days on EU lobbying and EU procurement.
• 2011-2012 Apps für Deutschland, co-organizing an open data competition under the patronage of the German Federal Ministry of the Interior.
• 2010-2011 Berlin Open Data Day (D2B1), community/public local open data interest group.
• 2010-2011 Munich Open Government Day (MOGDy), open innovation platform, consulting the city of Munich on open data policies.

Training and materials

• 2012-2013 World Bank Institute Data Bootcamps, series of multi-day data literacy workshops for journalists, activists and coders in developing nations. Data trainer in Nairobi, Kenya, Grahamstown, South Africa, Chisinau, Moldova and Kathmandu, Nepal.
• 2011-2013 School of Data Journalism, series of training events with the OKFN and European Journalism Centre. Contributed to workshops in Warsaw, Utrecht, Perugia, Vienna and Konjic (BiH).
• 2012 Spending Data Handbook, book sprint to create guidance for civil society organisations in their use and advocacy of government financial data.
• 2011-2012 Data Journalism Handbook, contributed chapters on data acquistion and produced a web version of the book.
• 2011-2012 Open Data FAQ, frequently asked questions on the progress of open data initiatives in Germany and the relevant political and technical background.
• 2010-2011 DataPatterns, a collection of design patterns for data processing. Deprecated in favor of the School of Data Handbook.