Fatcat is versioned, publicly-editable catalog of research publications: journal articles, conference proceedings, pre-prints, blog posts, and so forth. The goal is to improve the state of preservation and access to these works by providing a manifest of full-text content versions and locations.
This service does not directly contain full-text content itself, but provides basic access for human and machine readers through links to copies in web archives, repositories, and the public web.
Significantly more context and background information can be found in The Guide.
Feedback and queries can be directed to email@example.com.
Goals and Features
A few things set Fatcat apart from similar indexing and discovery services:
- inclusion of archival, file-level metadata (hashes) in addition to URLs, which allows automated verification ("do I have the right copy"), reveals content-drift over time, and enables efficient distribution of content through the ecosystem
- native support for "post-PDF" digital media, including archival web captures and datasets, as well as content stored on the distributed web
- data model that captures the work/edition distinction, grouping pre-print, post-review, published, re-published, and updated versions of a work together
- public editing interface, allowing metadata corrections and improvements from individuals and bots in addition to automated imports from authoritative sources
- focus on providing a stable API and corpus (making integration with diverse user-facing applications simple), while enabling full replication and mirroring of the corpus to reduce the risks of centralized control
This service aspires to be a piece of sustainable, long-term, non-profit, free-software, collaborative, open digital infrastructure. It is primarily designed to support the archival and dissemination roles of scholarly communication. It may also support the registration role (establishing precedence and authorship), but explicitly does not aid with certification of content, and is not intended to be used for evaluation of individuals, institutions, or venues. This service is "universal", not currated, and happily includes retracted and "predatory" content).
Sources of MetadataThe source of all bibliographic information is recorded in edit history metadata, which allows the provenance of all records to be reconstructed. A few major sources are worth highlighting here:
- Release metadata from Crossref, via their public REST API
- Release metadata and linked full-text content from NIH Pubmed and arXiv.org
- Release metadata and linked public domain full-text content the JSTOR Early Journal Content collection
- Creator names and de-duplication from ORCID, via their annual public data releases
- Journal title metadata from DOAJ, ISSN ROAD, and SHERPA/RoMEO
- Full-text URL lists from CORE, Unpaywall, Semantic Scholar, CiteseerX, and Microsoft Academic Graph.
- The Guide lists more major sources
Support and Acknowledgments
Fatcat is a project of the Internet Archive, a US-based non-profit digital library, well known for its Wayback Machine web archive and Open Library book digitization and lending service. All Fatcat databases and services run on Internet Archive servers in California, and a copy of most full-text content is stored in the Archive's collections and/or web archives.
Development of Fatcat and related web harvesting, indexing, and preservation efforts at the Archive have been partially funded (for the 2018-2019 period) by a generous grant from the Mellon Foundation ("Long-tail Open Access Journal Preservation"). Fatcat supports this work by both tracking which open access works in known archives and providing minimum-viable indexing and access mechanisms for long-tail works which otherwise would lack them.
The service would not technically be possible without hundreds of Free Software components and the efforts of their individual and organizational maintainers, more than can be listed here (please see the source code for full lists). A few major components include the PostgreSQL database, Elasticsearch search engine, Flask python web framework, Rust programming language, Diesel database library, Swagger/OpenAPI code generators, Kafka distributed log, Ansible configuration management tool, and Ubuntu GNU/Linux operating system distribution.
The front-page photo of a large feline with a cup of coffee is by Quinn Kampschroer, under a CC-0 license. The name "Fatcat" can be interpreted as short for "large catalog", as the service aspires to be a complete catalog of the digital scholarly record.
A list of technical contributors, including volunteers, is maintained in the
source code repository (
CONTRIBUTORS.md). Thanks everybody!