Internet Archive - Competitive Research Tools

Internet Archive

The Internet Archive is a 501(c)(3) non-profit that was founded to build an Internet library. Its purposes include offering permanent access for researchers, historians, scholars, people with disabilities, and the general public to historical collections that exist in digital format. Founded in 1996 and located in the Presidio of San Francisco, the Archive has been receiving data donations from Alexa Internet and others. In late 1999, the organization started to grow to include more well-rounded collections. Now the Internet Archive includes texts, audio, moving images, and software as well as archived web pages in our collections, and is working to provide specialized services relating to training, education, or adaptive reading or information access needs of blind or other persons with disabilities.
Why the Archive is Building an 'Internet Library'
Libraries exist to preserve society's cultural artifacts and to provide access to them. If libraries are to continue to foster education and scholarship in this era of digital technology, it's essential for them to extend those functions into the digital world.

Many early movies were recycled to recover the silver in the film. The Library of Alexandria - an ancient center of learning containing a copy of every book in the world - was eventually burned to the ground. Even now, at the turn of the 21st century, no comprehensive archives of television or radio programs exist.

But without cultural artifacts, civilization has no memory and no mechanism to learn from its successes and failures. And paradoxically, with the explosion of the Internet, we live in what Danny Hillis has referred to as our "digital dark age."

The Internet Archive is working to prevent the Internet - a new medium with major historical significance - and other "born-digital" materials from disappearing into the past. Collaborating with institutions including the Library of Congress and the Smithsonian, we are working to preserve a record for generations to come.

Open and free access to literature and other writings has long been considered essential to education and to the maintenance of an open society. Public and philanthropic enterprises have supported it through the ages.

The Internet Archive is opening its collections to researchers, historians, and scholars. The Archive has no vested interest in the discoveries of the users of its collections, nor is it a grant-making organization.

At present, the size of our Web collection is such that using it requires programming skills. However, we are hopeful about the development of tools and methods that will give the general public easy and meaningful access to our collective history. In addition to developing our own collections, we are working to promote the formation of other Internet libraries in the United States and elsewhere.

Find out

How to make a Monetary Donation to the Archvive
About our announcement and discussion lists on Internet libraries and movie archives
as well as our user forums
Future Libraries - How People Envision Using Internet Libraries
From ephemera to artifact: Internet libraries can change the content of the Internet from ephemera to enduring artifacts of our political and cultural lives.

"I believe historians need every possible piece of paper and archived byte of digital data they can muster. The Smithsonian Institution sees the value, and has affiliated with the Archive to preserve the 1996 campaign Web sites, official and unofficial."

Dan Gillmor, computing editor, San Jose Mercury News, 1 September 1996

Protecting our right to know: Most states have pre-Internet sunshine laws that require public access to government documents. Yet while the Internet has generally increased public access to information, states have just begun to amend those laws to reflect today's Internet environment. According to Bill Chamberlin, director of the Marion Brechner Citizen Access Project at the University of Florida's College of Journalism and Communications, such laws are being enacted "piecemeal, one state at a time," and cover information that varies widely in nature - everything from "all public records" to specialized information such as education reports and the licensing status of medical practitioners. In the meantime, while public officials are posting more information on the Internet than their state legislatures require, there's little regulatory control over exactly what is posted, when it's taken off, or how often it's updated. This leaves a gap that online libraries can help to fill.

Exercising our "right to remember": Without paper libraries, it would be hard to exercise our "right to remember" our political history or hold government accountable. With much of the public's business now moving from paper to digital media, Internet libraries are certain to become essential in maintaining that right. Imagine, for instance, how news coverage of an election campaign might suffer if journalists had only limited access to previous statements that candidates had made in the media.

"The Internet Archive is a service so essential that its founding is bound to be looked back on with the fondness and respect that people now have for the public libraries seeded by Andrew Carnegie a century ago.... Digitized information, especially on the Internet, has such rapid turnover these days that total loss is the norm. Civilization is developing severe amnesia as a result; indeed it may have become too amnesiac already to notice the problem properly. The Internet Archive is the beginning of a cure - the beginning of complete, detailed, accessible, searchable memory for society, and not just scholars this time, but everyone."

Stewart Brand, president, The Long Now Foundation

Establishing Internet centers internationally: What is a country without a memory of its cultural heritage? Internet libraries are the place to preserve the aspect of a country's heritage that exists on the Internet.

Tracing the way our language changes: During the late 19th century, James Murray, a professor at Oxford University, built the first edition of the Oxford English Dictionary by sending copies of selected books to "men of letters" who volunteered to search them for the first occurrences of words and to trace the migration of their various meanings. Internet libraries could allow linguists to automate much of this extremely labor-intensive process.

Tracking the Web's evolution: Historians, sociologists, and journalists could use Internet libraries to hold up a mirror to society. For example, they might ask when different ethnic groups or special interests or certain businesses became a presence on the Internet.

"We don't know where this Internet is going, and once we get there it will be very instructive to look back."

Donald Heath, president of the Internet Society in Reston, Virginia

Reviving dead links: A few services - such as UC Berkeley's Digital Library Project, the Online Computer Library Center, and Alexa Internet are starting to offer access to archived versions of Web pages when those pages have been removed from the Web. This means that if you get a "404 - Page Not Found" error, you'll still be able to find a version of the page.

Understanding the economy: Economists could use Archive data such as link structures - what and how many links a site contains - to investigate how the Web affects commerce.

Finding out what the Web tells us about ourselves: Researchers could use data on links and traffic to better understand human behavior and communication.

"Researchers could use the Archive's Web snapshots in combination with usage statistics to compare how people in different countries use the Web over long periods of time.... Political scientists and sociologists could use the data to study how public opinion gets formed. For example, suppose a device for increasing privacy became available: Would it change usage patterns?"

Bernardo Huberman, Xerox Palo Alto Research Center

"The Internet Archive has created a kind of test tube that allows a broad range of researchers to analyze the Web in ways that have never been possible before. What makes this type of research unique is that it often requires the fusion of traditional tools and techniques with new methods, and it results in the development of new theories, techniques, and metrics."

James Pitkow, Xerox Palo Alto Research Center

Looking back: With a "way-back machine" - a device that displayed the Web as it looked on a given date - historians and others would literally have a window on the past.