Uploaded image for project: 'AdMax'
  1. AdMax
  2. ADMAX-3123

Cloud Spider: Clean up script not deleting log files older than 10 days

          Details

          • Type: Bug
          • Status: Resolved
          • Priority: Major
          • Resolution: Fixed
          • Affects Version/s: None
          • Fix Version/s: Sustaining
          • Component/s: Cloud Spider
          • Labels:
            None
          • Sprint:
            Sprint 2

            Description

            The clean up script on node "/nutch/search/scripts/clean-up.sh" is not able to delete log files under /mnt directory on spider node. Due to this files remain on node for long time and ultimately disk space gets filled up failing crawls due to disk space issues.

            This appears to be permission issue for the script. Log files under /mnt directory are created under root and the script is executed as nutch user which can't delete these file created by root. Running script as sudo in MSgListener may solve the issue.

            Another alternate approach is to edit script itself.

              Attachments

                Activity

                  People

                  • Assignee:
                    jshih Jeff Shih (Inactive)
                    Reporter:
                    abhiram Abhiram Bhagwat
                  • Votes:
                    0 Vote for this issue
                    Watchers:
                    1 Start watching this issue

                    Dates

                    • Created:
                      Updated:
                      Resolved: