So, How Do We Put Elizabeth Warren’s Calendar Online?

One of the most popular features on this site is an interactive copy of Elizabeth Warren’s calendar, which includes an RSS feed and the raw calendar data. We think it is already the richest, most detailed leadership calendar that a .gov web site has offered to date, and we hope to make it even better. Today, we want to share a bit of the work that goes into creating and updating this tool.

The calendar was a high priority for the web site launch. The question wasn’t if we would provide Professor Warren’s full calendar, but how. We figured the effort would be smooth sailing from there.

We were wrong. We learned very quickly why interactive leadership calendars are rare: they’re hard. Getting this calendar from Professor Warren’s computer to the web takes a lot of planning, technical work, and ongoing maintenance.

The calendar Professor Warren uses on a day-to-day basis is maintained in a desktop application. Fortunately, that application lets you save a calendar as an .ics file, a standard calendar format that many applications can interpret. (A key component of openness is using open, cross-compatible data formats when possible.) At the end of every month, we save a copy of that month’s events as an .ics file. That lets us work with the calendar without altering the original.

The next step is the redaction process under the Freedom of Information Act. Any given month contains several events with information that may be personal or pre-decisional. For example, if Professor Warren interviews a job candidate or has dinner at someone’s home, we must remove the candidate’s name or the address of the home. Such redactions are noted on the calendar as “REDACTED”. Clicking on the event will give you more information about the nature of the deleted information.

We save the redactions back to the source file, a step that is much harder than it sounds. This way, the modified file can be moved to an internal testing, or “staging,” version of our web site. This test is the most labor-intensive task of the whole process. We triple-check that the information we’re about to put online contains no personal information. We can’t be certain unless we check the raw source file; combing through that metadata is quite a chore. To give you an idea of what we’re looking through, the image below is an example of what that data looks like.

Going through this process just once was enough to give us a new appreciation for open government projects. This is painstaking work, and it took us weeks to figure out the process. Each cycle has taught us lessons that streamline the workflow, cutting the process down and increasing our efficiency.

Yesterday, we added February’s calendar to the December and January calendars we posted earlier. We have also back-filled the calendar with October’s and November’s events. We hope the calendar can serve as a starting point for more open government initiatives as we continue building this consumer bureau.