Search Logger
Posts from: Vanessa Fox, Google Engineering

Author Archive

Trouble with verification

8:11 pm - November 30, 2005 in Inside Google Sitemaps
Some of you have had one of the following responses when trying to verify:



"Our system has experienced a temporary problem."



or,



" The system is currently busy. Please try again in a few minutes."



However, when you do try again later, you continue to get one of these messages. This is a known issue and we are working to resolve it as quickly as possible. Thanks for your patience.
 

New version of Sitemap Generator

11:25 am - December 7, 2005 in Inside Google Sitemaps
We recently uploaded a new version (v1.4) of the the Sitemap Generator tool.



This version has the same features as the last one, but fixes a subtle bug in writing GZip compressed Sitemap files. The old version stored more path information than it needed to when it created GZip files, and this was a point of concern for some webmasters.



The bug was found, and the bugfix suggested, by members of the Sitemaps community. Thanks for bringing it to our attention.
 

Lowercase verification filenames

8:15 pm - December 12, 2005 in Inside Google Sitemaps
We've been looking into ways to enhance our verification process and noticed that some members of our Group have been unable to upload case-sensitive filenames to their webservers. To help them, we now check for a lowercase verification filename if we don't find the case-sensitive one. So now you can upload a verification file with the lowercase filename. (If you have already verified, you don't need to do anything new.)



Thanks to our Groups members for their input -- and keep that feedback coming!
 

More query stats; verification enhancements

8:20 pm - December 13, 2005 in Inside Google Sitemaps
We've made a few improvements that we wanted to let you know about.



Expanded query stats

We've expanded the number of query stats that we show you (both top search queries and top search query clicks). Note that the exact number of stats you see depends on how often your site comes up in search results. A large site that has been around for a while will generally have more stats than a new, smaller site.



Verification enhancements

We've also been working on the verification process. Last month, we posted to the Group that we were looking for your input on how to improve this process. Yesterday, we told you that we added support for lowercase verification filenames. Now, we've added another improvement. Some of you have received one of the following messages when you try to verify:



Our system has experienced a temporary problem.



Or,



The system is currently busy. Please try again in a few minutes.



In the past, you've had to manually try again later. Now, instead of seeing one of those messages, you'll see this message:



The system is currently busy. We will process this verification as soon as possible. Please check back later for an updated status.



We'll add your verification request to a queue and try it for you later, so that you don't have to keep trying manually.



If you receive another error that might be due to a temporary issue (for instance, if we can't reach your server due to a DNS timeout), we'll show you the error, but we'll also add your verification request to the queue.
  • While your request is in the queue, you'll see "PENDING" as the status in the Verify tab.
  • Once we've processed the request successfully, we will display your site as verified and you can access your stats.
  • If we aren't able to successfully process the request, you'll see "NOT VERIFIED" as the status in the Verify tab. If you see this, check that your verification file is in the correct location, is named exactly as we ask, that your robots.txt file doesn't block access, and that your server is up and responding to requests. Then, try your verification request again.

 

Verifying a site located in a subdirectory

7:51 pm - December 19, 2005 in Inside Google Sitemaps
If your site is located in a subdirectory, rather than at the root level of a domain (for instance, at http://www.example.com/site/), you are given two choices when you verify. You can either verify at the subdirectory level or at the root level.





If you verify at the root level (http://www.example.com/), we can show you a greater variety of statistics.



If you are unable to upload a file at the root level, you can still view error information for the subdirectory at which you've verified, as well as information about your Sitemaps. Verification does not affect your Sitemap submission or the crawling or indexing of your site.



If you verify at the subdirectory level, some statistics are not available. Instead, some stats pages provide a link for verifying at the root level.







If you have root-level access, simply click the verify link, place the requested verification file in the root directory and then click Verify.



Some questions you may have about verifying at the subdirectory level:



I verified successfully at http://www.example.com/site/. But when I access my stats, I see a message asking me to verify again. Why is it asking me this since I already verified?

That message is providing a link so you can verify at the root level. Rest assured that your site is still verified at the subdirectory level and you can access all information that we have available at this time for subdirectories (index stats, Sitemap details, and site errors).



I verified successfully at http://www.example.com/site/. But I can still see the Verify tab. When I access that tab and click "Verify", I get a message that Google couldn't find my verification file and that my site isn't verified. The original verification file is still present at http://www.example.com/site/, so why am I getting this message?

If you are successfully verified at the subdirectory level, the Verify tab will still appear so that you can later go back and verify at the root level. When you are already verified at the subdirectory level, we look for the verification file in the root of the domain when you try to verify again. You'll see an error message if we don't find it there.



I can't verify at the root. Will this affect my listings in Google?

No. Verification at the root lets you see a greater variety of statistics for your site. It does not affect how we use your Sitemap, how we crawl your site, your PageRank, or any other factor.



A previous version let me verify at the subdirectory level. Why did you change it in this version?

You can still verify at the subdirectory level as you could before and see everything you could before (everything listed under the Sitemaps tab and Errors tab). We've added the option of verifying at the root, which lets you see root-level stats.
 

www vs non-www versions of a site

4:47 pm - December 21, 2005 in Inside Google Sitemaps
Two URLs to a site -- one that is prefaced with www and one that is not (for instance, http://www.example.com/ and http://example.com/) -- often point to the same location on a server. But depending on the server configuration, they may point to different locations, so search engines can't assume they are the same.

This post provides tips for viewing stats if the www and non-www version of your site point to the same location.

If you have added your site without prefacing the domain with www (for instance, http://example.com/), and the www version of this domain points to the same location, try adding the www version of the domain (for instance, http://www.example.com/) to your account. You may see a wider variety of stats for the www version of the domain.

You can add a site by:
  1. Clicking the Add tab.
  2. Scrolling to the "If you don't have a Sitemap" area and entering the site URL.
  3. Clicking Add Site.
Similarly, if you have added the www version of the domain and the non-www version points to the same location, you can add the non- www version to see stats for that version.

If the verification file still exists in the root of your site and both versions of the domain point to the same location, you can verify the second version simply by accessing the Verify tab and clicking the Verify button.

Note that having both versions of the site's URL listed in your account won't affect the indexing of your site as long as you have submitted a Sitemap for only one version - the version you want to be indexed. Don't submit a Sitemap for both versions if the location and content are the same.

If both domains point to the same location and you have pages indexed under both versions, see our Google Help Center for more information on consolidating the listings under one domain.

We hear requests for help with this often, so we'll be looking at ways to improve this issue in the coming months.
 

More language support

1:20 pm - January 10, 2006 in Inside Google Sitemaps
We've added support for four more languages and a Sitemaps Google Group for each one:



Danish

Finnish

Norwegian

Swedish



As with the other languages we support, if you already use Google in one of these languages, you should see the user interface and documentation automatically. Otherwise, you can click the Preferences link from the Google home page and choose the language from the interface list.
 

Answers to Friday questions

7:58 pm - January 13, 2006 in Inside Google Sitemaps
Today, we wanted to review a few things that we get a lot of questions about.



I tried to verify and I got a message that said "pending verification". Why?

If we can't process the verification request right away, we'll add it to a queue and process it as soon as possible. While your request is in this queue, the status will show as pending. Once we successfully process your request, your site will show as verified. If the request is unsuccessful, you'll see a status of "not verified". You can read more about reasons the request might not be successful or take a look at our documentation.



I submitted my Sitemap and it now as a status of "Timed out downloading robots.txt". I don't have a robots.txt file on my site. What happened?

Any time we access your site, we first check to see if you have a robots.txt file that restricts our access. If we get a timeout from your server when we do this check, you'll see this message. This is probably a temporary problem. Once your server is accessible again, we will be able to see that you don't have a robots.txt file, and will then access your Sitemap. If you continue to see this error, make sure that your webserver is up and responding to requests.



I submitted my sitemap at http://www.example.com/mysite/ and verified it. But now you are asking me to verify at http://www.example.com/. I can't verify at that location because I don't have access. Why are you asking me to do this?

Some stats we can only show if you verify at the root level. If you don't have access to this location, we will show you all the information we are able to (such as errors we had crawling your site). Verification doesn't impact Sitemap submission or indexing in any way, and there is no penalty if you can't verify at the root level.



I've submitted my Sitemap, verified my site, and you've downloaded my Sitemap, but many of my stats show "Data is not available at the time time". Why?

We show you stats about what we know about your site. If we haven't yet crawled and indexed much of your site, we may not have many stats to show you. As we crawl more of your site and learn more about it, you'll start to see more stats.
 

More about changing domain names

12:27 pm - January 27, 2006 in Inside Google Sitemaps
Recently, someone asked me about moving from one domain to another. He had read that Google recommends using a 301 redirect to let Googlebot know about the move, but he wasn't sure if he should do that. He wondered if Googlebot would follow the 301 to the new site, see that it contained the same content as the pages already indexed from the old site, and think it was duplicate content (and therefore not index it). He wondered if a 302 redirect would be a better option.

I told him that a 301 redirect was exactly what he should do. A 302 redirect tells Googlebot that the move is temporary and that Google should continue to index the old domain. A 301 redirect tells Googlebot that the move is permanent and that Google should start indexing the new domain instead. Googlebot won't see the new site as duplicate content, but as moved content. And that's exactly what someone who is changing domains wants.

He also wondered how long it would take for the new site to show up in Google search results. He thought that a new site could take longer to index than new pages of an existing site. I told him that if he noticed that it took a while for a new site to be indexed, it was generally because it took Googlebot a while to learn about the new site. Googlebot learns about new pages to crawl by following links from other pages and from Sitemaps. If Googlebot already knows about a site, it generally finds out about new pages on that site quickly, since the site links to the new pages.

I told him that by using a 301 to redirect Googlebot from the old domain to the new one and by submitting a Sitemap for the new domain, Googlebot could much more quickly learn about the new domain than it might otherwise. He could also let other sites that link to him know about the domain change so they could update their links.

The crawling and indexing processes are completely automated, so I couldn't tell him exactly when the domain would start showing up in results. But letting Googlebot know about the site (using a 301 redirect and a Sitemap) is an important first step in that process.

You can find out more about submitting a Sitemap in our documentation and you can find out more about how to use a 301 redirect by doing a Google search for [301 redirect].
 

Giving others access to Sitemaps account information

12:54 pm - February 1, 2006 in Inside Google Sitemaps
Once you add a site or Sitemap to your account and verify ownership, we show you statistics and errors about the site. Now someone else on your team also wants to view information about it. Or perhaps you set up a Sitemap for a client site and now they wants to control the Sitemap submission and view site information.



Anyone who wants to view site information can simply create a Sitemaps Account, add the site (or Sitemap), and verify ownership. The site isn't penalized in any way if it is added to multiple accounts.



Some questions you may have:



I have Sitemaps for multiple clients in my account. I don't want one client to see everything that's in my account. How do I prevent this?

The client should create their own Sitemaps account and verify site ownership. The client won't see anything that's in your account (even though a site or Sitemap may be listed in both accounts).



I've already verified site ownership and the verification file is still on my server. Does the client also have to verify site ownership?

Yes, each account holder must verify site ownership separately. We ask for a different verification file for each account.



What happens if I submit a Sitemap and then someone else in my company submits that same Sitemap using a different Sitemaps account? Do you see that as duplicate entries and penalize the site?

No, we don't penalize the site in any way. No matter the number of accounts that list the Sitemap, we see it as one Sitemap and process it accordingly.



I want the client to be able to see all the stats for this site right away. I don't want them to have to wait for stats to populate. Can I transfer my account information to the client?

Stats and errors are computed for a site, not for an account. There's no need to transfer your account information because anyone who adds a site and verifies site ownership will see all information that we available to show for that site right away.



But I don't want to list the site in my account anymore. Are you sure I can't just transfer the information to the other person's account?

If you don't want the site or Sitemap in your account any longer, you can simply delete it from your account. This won't penalize the site in any way.
 
 
 
 
 
 
It's All About Search | © clsc.net |
2012.05.1823:32
Tech used here: Valid HTML - Valid CSS - Valid RSS - JavaScript - PHP - Smarty - MySQL - and a partridge in a pear tree.