Is Google Wrongly Blocking Your Web Site?
Security Notice Regarding Brief Hacking of AIDSvideos.org Home Page
and Subsequent Google / Firefox Warning Message
Google Wrongly Blocked AIDSvideos.org for Five Months After
Brief Site Hacking Incident
Last updated 22 December 2008Update: After I filed a seventh review request, posted this protest notice, and posted a complaint about this situation in the Google Blog, Google finally removed AIDSvideos.org from its malware site blocking list after five months of requests. If you are a webmaster whose web site is safe but is being wrongly blocked by Google, I recommend that you do the following:
- Post a protest notice about the problem on your web site immediately; patience is not
rewarded in this situation
- File a review request through Google Webmaster Tools
- If your review request is rejected, post your own complaint
about this situation in the thread on the Google Blog; right now,
that seems to be the only route to escalate to a human being. Be aware
that email messages to Google employees that contain a link to a
"suspected attack site" may not get through to their @google.com
account. (See below.)
- Google should provide a way in Google Webmaster Tools for webmasters whose review requests have been rejected to escalate their review request to a human being. Filing repeated review requests over and over again to the same automated process is a waste of time for the webmaster and does not resolve the problem. Google should feel free to charge a reasonable fee of some kind for processing these human requests; this would defray Google's costs for human review of the requests, incent webmasters to carefully review their web site before escalating to a human, and help protect Google from spam review requests.
- I assume given its business model and gross margin targets that Google does not want to be in the business of manually reviewing web site security for profit. Therefore, Google could identify a set of independent, third party security review firms whose web site security verdicts it trusts, enable webmasters whose review request was rejected by Google Webmaster Tools to submit an "escalation" review request to those firms for a fee determined by each firm, and enable those firms to submit a "certified safe" verdict back to Google, which Google would then use as a final binding input into its web site security assessment. This would of course create competition among the firms on price that would help to keep down the cost to webmasters of this kind of service.
- Google could also have its webmaster tools automatically scan
for situations where firms rendered a "certified safe" verdict for a
site but the tools detected a genuine threat still in existence. Such
situations could be automatically escalated to a Google employee who
manages the "suspected attack site" program. This would help to detect
any situations where a security review firm was being lax in its
reviewing or was in fact a bad actor who was knowingly rendering false
verdicts for profit.
- Google should make sure that its email security system does not block emails to its own employees from webmasters who are in this situation. I learned today that at least one email message that I sent this month requesting help to a Google employee at their @google.com email address appears not to have been delivered. Perhaps the email was just wrongly flagged by ordinary spam filtering. However, Gmail has begun flagging emails that include the URLs of "suspected attack sites" with a warning message, and it would not surprise me if Google's security administrators have chosen to block or redirect emails that include links to "suspected attack sites" in a well-intentioned attempt to protect their own network and employees from actual attack. If this is the case, Google could enhance that blocking program to instead automatically edit the email message, insert a warning notice, and automatically edit the links so they don't function when clicked. This would make it easier for "innocent victim" webmasters to get through to a Google employee when they need to.
- Google should carefully review the code by which Google Webmaster Tools determines whether a site poses a threat. The Google Webmaster Tools report for AIDSvideos.org claimed that AIDSvideos.org was still hosting an exploit as of September, 2008. This appears to be an erroneous conclusion because I cleaned the site in late July. Obviously, if Google Webmaster Tools is incorrectly flagging innocent site characteristics as harmful, that greatly increases the risk that it will wrongly flag innocent sites as a threat.
- Ideally, Google should also enhance its system to look up the
whois record for a site that its about to block and send an email
notice to the email address of the site administrator informing them of
the detected exploit. This might cause site administrators to notice
(and resolve) the problem sooner, therefore shortening the duration of
any real exploit.
-- Eric Krock
On July 4th, 2008, a hacker managed to replace the AIDSvideos.org page with one that was identical but included a JavaScript redirect instruction at the bottom of the home page. They appear to have done this by using sophisticated password cracking software to break the site's FTP password. The password in use at that time was not stupid (e.g. "password," the administrator or site or account name, or a word you could find in a dictionary) and it used letters and numbers, but it was not particularly long, so a long-term attack could have broken it. We apologize for the inconvenience. The lesson here is to use very long and complex FTP passwords (or better yet, S/FTP) and to change them periodically.
This problem came to the attention of the site administrator in late July when Firefox began blocking the site as part of its integration of Google's malware site detection and flagging system. The site was promptly analyzed and repaired. Every character of every HTML file on the site was manually inspected for evidence of tampering. All pages were either restored from original backups (as in the case of Word.doc binary files that could be hard to manually review) or from a copy that had been manually inspected character by character and reparied as needed (as in the case of the home page). All means of accessing the site were reviewed and FTP accounts previously provided to other not-for-profits were disabled. The hosting service provider was notified and confirmed that the binaries of the web servers and operating systems were fully up to date including all patches. The password in use was GREATLY strengthened. Recommendations on Google's "best practices" list for site administrators have been followed. AIDSvideos.org has been entirely free of any hostile or foreign code or links since late July, 2008.
The AIDSvideos.org site administrator then filed SIX requests to Google to have AIDSvideos.org removed from its malware blocking list. All six were denied without explanation or justification.
The first three times, the automated system reported that the site still posed a threat and said "Here is an example URL of a page that poses a threat:" but provided no URL, indicating a poorly-written review system that wasn't even internally consistent in the presentation of its results.
On the next three review requests, the messaging had been modified to eliminate any reference to a particular URL, but the system still claimed without explanation or justification that the site might pose a threat to visitors. Google's search engine still claimed that "This site may harm your computer." This claim by Google was false. The web site AIDSvideos.org is very simple. It has about 170 static HTML, .doc, and .txt files. It uses simple SSI includes of static header files but does not use CGI scripts of any kind. There is no way that the site could have been posting a threat to users during this period. By falsely claiming that it might, Google was misleading and obstructing Internet users who were attempting to access potentially lifesaving HIV/AIDS prevention education messages.
In an attempt to get some human attention on the problem from Google, the site administrator also repeatedly emailed friends at Google. This approach too yielded no results.
Recently, the Open Media Network, to which AIDSvideos.org previously linked for hosting of high quality videos, ceased operations. The administrator wonders whether these broken links were wrongly flagged as a potential risk. All those links were replaced by links to archive.org, and a seventh review request was submitted.
Google (and by extension Firefox) blocked the site and wrongly claimed that the site posed a threat to visitors for five months. This caused significant problems for AIDSvideos.org in our efforts to provide scientifically and medically accurate HIV/AIDS prevention education messages to the world. One leading not-for-profit eliminated a link to the web site until it is removed from Google's list. Prospective donors and volunteers have asked about the situation. People at workplaces may have been completely unable to access the site during this period, and anyone using Firefox was (wrongly) warned on every page access that the site may pose a threat, interfering with their attempts to access the site's content. Repeatedly filing review requests and emailing contacts at Google seeking help consumed valuable time that would have been better spent on editing and publishing new prevention education videos. The site's brand was harmed.
AIDSvideos.org respectfully requests that Google improve the timeliness and accuracy of its responses to site review requests going forward. With great power comes great reponsibility. Google has every right to try to warn Internet users about web sites that genuinely pose a threat, but it has a responsibility to not continue to falsely flag as a threat innocent web sites that have been hacked but have recovered, repeatedly requested review and removal, and are now trying to continue doing good work in a hostile, dangerous world.