Uploaded image for project: 'AdMax'
  1. AdMax
  2. ADMAX-2771

Cloud Spider: ADR generattion remains in_progress for long time when the crawl is aborted from not_started state

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: Cloud Spider 3.02
    • Component/s: Cloud Spider
    • Labels:
      None
    • Environment:

      Test Env: (192.168.70.86)

      Services - Trunk Maven Build - 862

      AWS - Cloud Spider Apps - Trunk Build - 7

      UI - AdMax-Trunk Build - 79 (client-pc)

      Description

      1. Log into AdMax application and navigate to SEO-> Spider module

      2. Select an Account to relate the AWS costs and enter a valid URL to spider

      3. Select all the reports for generation and click "Submit" button

      4. In the View Crawl Reports, stop the crawl before it gets started

      5. Found that the crawl gets aborted without crawling any pages and the Pixel and CSS reports failed without any pages as expected (S-566)

      Observed that the ADR (R-1973) remains remains in_progress for a long time until we stop it thro db. In the Msglistener log file observed the below error

      ----------------

      Wed 2011/08/17 12:51:30.993| |Thread-253|StreamGobbler|STDERR: 11/08/17 12:51:30 INFO amazonaws.request: Received error response: Status Code: 404, AWS Request ID: A0AA695F1824D918, AWS Error Code: NoSuchKey, AWS Error Message: The specified key does not exist., S3 Extended Request ID: bapTuTwZBldLS97Jnulabp74oYFLzm407IGh/C2nUHxrjtRL5Mz4yS71uIDzfH20

      Wed 2011/08/17 12:51:30.994| |Thread-254|StreamGobbler|STDOUT: #### ERROR: Download initiation failed. Status Code: 404, AWS Request ID: A0AA695F1824D918, AWS Error Code: NoSuchKey, AWS Error Message: The specified key does not exist., S3 Extended Request ID: bapTuTwZBldLS97Jnulabp74oYFLzm407IGh/C2nUHxrjtRL5Mz4yS71uIDzfH20 (attached log file)

      -----------------

      Also observed the same issue with ADR's R-1964, R-1965

      Report1: ec2-72-44-42-6.compute-1.amazonaws.com

        Attachments

          Activity

            People

            • Assignee:
              saravanan.t Saravanan (Inactive)
              Reporter:
              saravanan.t Saravanan (Inactive)
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: