.Google.com's Gary Illyes shows Googlebot gathers links rather than "adhering to" all of them, negating formal documents.Googlebot gathers links as opposed to following all of them in real-time.Gary Illyes discussed this info in Google's 'Explore Off The Document' podcast.Google's documents opposes the real crawler habits.