|
Poster:
|
Tomboy |
Date:
|
February 10, 2012 04:16:12pm |
|
Forum:
|
web
|
Subject:
|
Archived site no longer showing up |
The website
http://www.roadmaps.org/ has been archived with all the captures from 1998 onward on the Internet Archive until recently, and says "Page cannot be crawled or displayed due to robots.txt." Does this mean they will reappear again soon (I hope)?
This post was modified by Tomboy on 2012-02-11 00:10:34
This post was modified by Tomboy on 2012-02-11 00:16:12
|
Poster:
|
Goldsnich |
Date:
|
March 10, 2012 02:48:01pm |
|
Forum:
|
web
|
Subject:
|
Re: Archived site no longer showing up |
Seems like your site available already. How many days had passed since it become available again in archive? I have same issue now. But i know what happened - I disabled all robots in robots.txt, then same error as yours appeared, so I open robots.txt for indexing and now waiting when site come visible in archive again. Weird inconvenience, why i just not able to see previous snapshots when robots.txt blocking crawler ?
TVNews
OpenLibrary