Uploaded image for project: 'AdMax'
  1. AdMax
  2. ADMAX-2853

ADR v3 - Spider crawling extraneous domains

          Details

          • Type: Bug
          • Status: Closed
          • Priority: Major
          • Resolution: Fixed
          • Affects Version/s: None
          • Fix Version/s: Cloud Spider 3.03
          • Component/s: Cloud Spider
          • Labels:
            None

            Description

            This crawl did not have any subdomain option on and even if it did the URLs seems from a complete different root domain:

            The spider I ran for Viking River Cruises on 10-3 returned an ADR that contained URLs for not only http://www.vikingrivercruises.com but also:

            • http://www.vikingrivercruises.com.au

            • https://www.vikingrivercruises.com

            • https://www.vikingrivercruises.com.au

            This made a lot of the reporting incorrect, especially when it came to duplicate content issues (title tags, H1 tags, meta description, meta keywords). I am currently manually removing these URLs from the report but wanted to alert you to this bug.

            ID's

            Spider: S-1235

            ADR: R-1564

              Attachments

                Activity

                  People

                  • Assignee:
                    antony Antony Rajiv (Inactive)
                    Reporter:
                    jshih Jeff Shih (Inactive)
                  • Votes:
                    0 Vote for this issue
                    Watchers:
                    0 Start watching this issue

                    Dates

                    • Created:
                      Updated:
                      Resolved: