GPO's FDsys: 1 billion items served and ready for more

 

Connecting state and local government leaders

Government Printing Office's FDsys Web portal has served up more than a billion documents, videos, audio recordings and images since 2009 and plans for updated search engine and improved support for mobile devices.

On March 26, a visitor at the Government Printing Office’s Federal Digital System (FDsys)  downloaded a Federal Register notice from the Securities and Exchange Commission, which was the one billionth object retrieved through the site.

Introduced in 2009, FDsys has become a workhorse portal for public access to more than 1 million official documents, videos, audio recordings and images from all three branches of federal government. Almost half of the 1 billion retrievals came during the last 13 months. The most commonly viewed item is the Patient Protection and Affordable Care Act, with more than 14 million views since it became law in 2010. The system also hosts historical documents, such as the 1964 Warren Commission Report on the assassination of President Kennedy, which has been viewed more than 20,000 times.

FDsys was developed under a disciplined program management process using in-house expertise and infrastructure, designed for high availability and to support high demand volumes. The resulting system is a platform for managing government resources in a variety of formats.

“It’s more complex than just a Web server,” said FDsys manager Lisa LaPlant. Official documents are digitally signed, assuring the user that the documents are genuine and content has not been altered, and the system provides content management and a preservation repository for archival material as well as a site for online public access. It has demonstrated its ability to reliably deliver content during sudden spikes in demand, such as during the release of the federal budget, which now is distributed primarily in a digital format.

But, “We are not resting on our laurels,” said GPO’s Chief Technology Officer Ric Davis. A major refresh of FDsys now is in the planning stages, which will include an updated search engine and improved support for mobile devices.

FDsys uses Extensible Markup Language (XML) and an ISO standard format for archival information to enable searching across multiple collections, a feature not available in the original GPO website, GPO Access.

“It was really a flat store of files with a search engine on top of it,” LaPlant said of the old Access. “We needed something to better manage and preserve.”

GPO Access was mandated in 1993 to provide public access to GPO products and went online in 1994. It was developed in the infancy of online government, and “it served us very well for 15 years,” Davis said. “But it did have some limitations.” It used a search engine developed in-house, and the site’s only function was to serve up documents over the Internet. By 2000 it was clear that it was in need of an update, and a decision was made to develop a complete replacement.

Planning for FDsys began in 2005 and it was launched in January of 2009, running alongside GPO Access for a year before the older site was retired in 2010.

“To us it seemed like blinding speed,” Davis said of the four-year development process.

The development team used the agile development process of iterative and incremental software development, a cycle that includes planning, development, integration, testing and deployment.

“The part that was really critical was developing requirements,” Davis said. The development team consulted with all stakeholders in the project, including Federal Depository librarians, congressional staff and other agencies that would use the system. The goal was not new technology, but serving the stakeholders. “First of all, you want to do no harm,” he said. The idea was to keep users happy while adding new features. Success required having a clear vision to define the framework.

“There was a lot of involvement from subject matter experts” who produce and use the documents in developing requirements, LaPlant said.

Because GPO must ensure that its digital material will remain available in the future, through both FDsys and the 1,200 libraries in the Federal Depository Library Program, the framework uses the Open Archival Information System, a reference model from the International Organization for Standardization that specifies the concepts for a digital preservation system.

“We knew that was something that would fit our needs,” LaPlant said. A basic element of the model is the Archival Information Package in which both primary material and its metadata are stored as a single object. Managing data and metadata together allows information to be easily searched and to be transferred as one package.

Metadata is provided in XML, which allows documents to be encoded in a format that is human and machine readable, making it ideal for documents being searched for and accessed through the Internet. XML has become the go-to format for information in digital government initiatives, part of the Obama administration’s policy of leveraging the Web and virtual environments for delivery of government products and services.

In 2012 GPO began replacing its 30-year-old composition engine called Microcomp with XML Professional Publisher (XPP) to enable the direct XML formatting of documents for both electronic and print publication. This eliminated the step of transforming documents for publication in XML.

The decision was made to develop and host FDsys in-house because GPO staff already was familiar with the agency’s systems, the material being produced and the needs of FDsys. With the shift at GPO away from primarily ink-on-paper to electronic publishing, “we had the technical capability to handle it ourselves,” Davis said, although, “I wouldn’t say finding the staff was easy.”

GPO uses a commercial content delivery network for reliable response and to accommodate bursts in demand, but hosts FDsys on its premises, as well as a backup site for continuity of operations and disaster recovery.

The agency is evaluating cloud-based technology for FDsys as part of its upcoming major refresh, along with a new an open source search engine, Solr, which promises fault tolerant performance on a large scale.

A new user interface for mobile devices also is in the works. The FDsys site now is accessible through mobile browsers, “but you have to do a lot of pinching and scrolling,” LaPlant said. A mobile interface has always been on the FDsys roadmap, and with the rapid adoption of the devices, “we saw it was time to do it.”

With FDsys established as a scalable and reliable, GPO also is offering it to other agencies as a platform for archiving and disseminating material. It already is hosting such diverse materials as historic Treasury Department documents, audio recordings of radio traffic from Air Force One following the JFK assassination, and President’ Nixon’s Watergate grand jury testimony. GPO wants to expand these services, possibly to include online sites branded for client agencies.

“It depends on the needs of the client agency,” Davis said.

X
This website uses cookies to enhance user experience and to analyze performance and traffic on our website. We also share information about your use of our site with our social media, advertising and analytics partners. Learn More / Do Not Sell My Personal Information
Accept Cookies
X
Cookie Preferences Cookie List

Do Not Sell My Personal Information

When you visit our website, we store cookies on your browser to collect information. The information collected might relate to you, your preferences or your device, and is mostly used to make the site work as you expect it to and to provide a more personalized web experience. However, you can choose not to allow certain types of cookies, which may impact your experience of the site and the services we are able to offer. Click on the different category headings to find out more and change our default settings according to your preference. You cannot opt-out of our First Party Strictly Necessary Cookies as they are deployed in order to ensure the proper functioning of our website (such as prompting the cookie banner and remembering your settings, to log into your account, to redirect you when you log out, etc.). For more information about the First and Third Party Cookies used please follow this link.

Allow All Cookies

Manage Consent Preferences

Strictly Necessary Cookies - Always Active

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Sale of Personal Data, Targeting & Social Media Cookies

Under the California Consumer Privacy Act, you have the right to opt-out of the sale of your personal information to third parties. These cookies collect information for analytics and to personalize your experience with targeted ads. You may exercise your right to opt out of the sale of personal information by using this toggle switch. If you opt out we will not be able to offer you personalised ads and will not hand over your personal information to any third parties. Additionally, you may contact our legal department for further clarification about your rights as a California consumer by using this Exercise My Rights link

If you have enabled privacy controls on your browser (such as a plugin), we have to take that as a valid request to opt-out. Therefore we would not be able to track your activity through the web. This may affect our ability to personalize ads according to your preferences.

Targeting cookies may be set through our site by our advertising partners. They may be used by those companies to build a profile of your interests and show you relevant adverts on other sites. They do not store directly personal information, but are based on uniquely identifying your browser and internet device. If you do not allow these cookies, you will experience less targeted advertising.

Social media cookies are set by a range of social media services that we have added to the site to enable you to share our content with your friends and networks. They are capable of tracking your browser across other sites and building up a profile of your interests. This may impact the content and messages you see on other websites you visit. If you do not allow these cookies you may not be able to use or see these sharing tools.

If you want to opt out of all of our lead reports and lists, please submit a privacy request at our Do Not Sell page.

Save Settings
Cookie Preferences Cookie List

Cookie List

A cookie is a small piece of data (text file) that a website – when visited by a user – asks your browser to store on your device in order to remember information about you, such as your language preference or login information. Those cookies are set by us and called first-party cookies. We also use third-party cookies – which are cookies from a domain different than the domain of the website you are visiting – for our advertising and marketing efforts. More specifically, we use cookies and other tracking technologies for the following purposes:

Strictly Necessary Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Functional Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Performance Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Sale of Personal Data

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.

Social Media Cookies

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.

Targeting Cookies

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.