Ning Developer Network

I noticed in my google webmasters tool that the below urls are showing up as URL restricted by robots.txt. Once I saw that I looked at my robots.txt URL at http://www.pickuptara.com/robots.txt and found the following:

User-agent: *
Disallow: /xn/ningbar.php/
Disallow: /xn/atom/
Disallow: /xn/rest/
Disallow: /xn/css/
Disallow: /xn/loader/
Disallow: /main/search/search/
Disallow: /main/authorization/

Are these restrictions what I want for an ideal setup to have www.pickuptara.com indexed properly by Google and other search engines? I am most concerned with the last two URLs below being skipped, specifically http://www.pickuptara.com/profiles

Thoughts? Should I update my robotos.txt or is this how it should be ideally?

URL
http://www.pickuptara.com/main/authorization/privacyPolicy?previous...
http://www.pickuptara.com/main/authorization/privacyPolicy?previous...
http://www.pickuptara.com/main/authorization/privacyPolicy?previous...
http://www.pickuptara.com/main/authorization/privacyPolicy?previous...
http://www.pickuptara.com/main/authorization/privacyPolicy?previous...
http://www.pickuptara.com/main/authorization/signIn?target=http%3A%...
http://www.pickuptara.com/main/authorization/signIn?target=http%3A%...
http://www.pickuptara.com/main/authorization/signIn?target=http%3A%...
http://www.pickuptara.com/main/authorization/signIn?target=http%3A%...
http://www.pickuptara.com/main/authorization/signUp?target=http%3A%...
http://www.pickuptara.com/main/authorization/signUp?target=http%3A%...
http://www.pickuptara.com/main/authorization/signUp?target=http%3A%...
http://www.pickuptara.com/main/authorization/signUp?target=http%3A%...
http://www.pickuptara.com/main/authorization/signUp?target=http%3A%...
http://www.pickuptara.com/main/authorization/signUp?target=http%3A%...
http://www.pickuptara.com/main/authorization/signUp?target=http%3A%...
http://www.pickuptara.com/main/authorization/signUp?target=http%3A%...
http://www.pickuptara.com/main/authorization/termsOfService?previou...
http://www.pickuptara.com/main/authorization/termsOfService?previou...
http://www.pickuptara.com/main/authorization/termsOfService?previou...
http://www.pickuptara.com/main/authorization/termsOfService?previou...
http://www.pickuptara.com/main/authorization/termsOfService?previou...
http://www.pickuptara.com/profiles
http://www.pickuptara.com/video/video/chooseUploader

Tags: robots, robots.txt

Share Twitter

Reply to This

Replies to This Discussion

Ditto, I am also concerned. I am getting lots of "errors" on this too, blogs, profiles, you name it... What is the official Ning response to this?

Reply to This

why has no one responded to this?

Reply to This

Reply to This

RSS

© 2010   Created by Ning Developer Admin

Badges  |  Report an Issue  |  Privacy  |  Terms of Service