Uploaded image for project: 'AdMaxLocal'
  1. AdMaxLocal
  2. AML-4380

Convert std to eta script returns some 0 byte eta files

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: Giraffe
    • Fix Version/s: Giraffe
    • Component/s: Other / Misc.
    • Labels:
      None
    • Environment:

      File: convert_std_ads_to_eta.sh

    • Sprint:
      Sprint 7

      Description

      The convert script is unable to convert few std ad text files to eta files and produces 0 byte files.
      All files were converted from unix2dos.
      Convert script was run at one time on 1939 std ad files.
      Around 100 files were empty though their respective std ad files had proper value.
      Log for the file that has 0 byte size:

      Input file: /backup/amlops-QA4/us_taxonomy/business_taxonomy/sem/ads//ads-smartphones-39-10291-0.csv loading into 'standardAdCsvLoader'.
      Started converting 1552 out of 1939 standard ads files into ETA format.
      Output file: /backup/amlops-QA4/us_taxonomy/business_taxonomy/sem/ads_giraffee/eta/eta-smartphones-39-10291-0.csv
      /backup/amlops-QA4/us_taxonomy/business_taxonomy/sem/ads_giraffee/eta/eta-smartphones-39-10291-0.csv is not UTF-8 encoded (found )
      Convert to UTF-8
      Successfully converted 1552 standard ads files into ETA format.

      Log for files that got converted successfully:

      Input file: /backup/amlops-QA4/us_taxonomy/business_taxonomy/sem/ads//ads-boutique_hotels-1-12774-0.csv loading into 'standardAdCsvLoader'.
      Started converting 235 out of 1939 standard ads files into ETA format.
      Output file: /backup/amlops-QA4/us_taxonomy/business_taxonomy/sem/ads_giraffee/eta/eta-boutique_hotels-1-12774-0.csv
      /backup/amlops-QA4/us_taxonomy/business_taxonomy/sem/ads_giraffee/eta/eta-boutique_hotels-1-12774-0.csv is properly encoded
      Successfully converted 235 standard ads files into ETA format.

      Re-ran the conversion script with few files that got generated with 0 byte size

      ./convert_std_ads_to_eta.sh -h aml-qa4-mysql1 -u spike -p tar63t -i /backup/amlops-QA4/us_taxonomy/business_taxonomy/sem/ads_0/ -o /backup/amlops-QA4/us_taxonomy/business_taxonomy/sem/ads_0/eta/
      /backup/amlops-QA4/us_taxonomy/business_taxonomy/sem/ads_0/
      /backup/amlops-QA4/us_taxonomy/business_taxonomy/sem/ads_0//ads-smartphones-50-10291-0.csv /backup/amlops-QA4/us_taxonomy/business_taxonomy/sem/ads_0//ads-specialty_teeshirts_online-174-12544-0.csv /backup/amlops-QA4/us_taxonomy/business_taxonomy/sem/ads_0//ads-sports_promoters-7-12616-0.csv /backup/amlops-QA4/us_taxonomy/business_taxonomy/sem/ads_0//ads-texas_rangers_tickets-173-12231-0.csv
      Using /backup/amlops-QA4/us_taxonomy/business_taxonomy/sem/ads_0/ as input directory.
      /backup/amlops-QA4/us_taxonomy/business_taxonomy/sem/ads_0/eta/
      Using /backup/amlops-QA4/us_taxonomy/business_taxonomy/sem/ads_0/eta/ directory.
      Using /backup/amlops-QA4/us_taxonomy/business_taxonomy/sem/ads_0/eta/ as output directory.
      Input file: /backup/amlops-QA4/us_taxonomy/business_taxonomy/sem/ads_0//ads-smartphones-50-10291-0.csv loading into 'standardAdCsvLoader'.
      Started converting 1 out of 4 standard ads files into ETA format.
      Output file: /backup/amlops-QA4/us_taxonomy/business_taxonomy/sem/ads_0/eta/eta-smartphones-50-10291-0.csv
      /backup/amlops-QA4/us_taxonomy/business_taxonomy/sem/ads_0/eta/eta-smartphones-50-10291-0.csv is properly encoded
      Successfully converted 1 standard ads files into ETA format.
      Input file: /backup/amlops-QA4/us_taxonomy/business_taxonomy/sem/ads_0//ads-specialty_teeshirts_online-174-12544-0.csv loading into 'standardAdCsvLoader'.
      Started converting 2 out of 4 standard ads files into ETA format.
      Output file: /backup/amlops-QA4/us_taxonomy/business_taxonomy/sem/ads_0/eta/eta-specialty_teeshirts_online-174-12544-0.csv
      /backup/amlops-QA4/us_taxonomy/business_taxonomy/sem/ads_0/eta/eta-specialty_teeshirts_online-174-12544-0.csv is properly encoded
      Successfully converted 2 standard ads files into ETA format.
      Input file: /backup/amlops-QA4/us_taxonomy/business_taxonomy/sem/ads_0//ads-sports_promoters-7-12616-0.csv loading into 'standardAdCsvLoader'.
      Started converting 3 out of 4 standard ads files into ETA format.
      Output file: /backup/amlops-QA4/us_taxonomy/business_taxonomy/sem/ads_0/eta/eta-sports_promoters-7-12616-0.csv
      /backup/amlops-QA4/us_taxonomy/business_taxonomy/sem/ads_0/eta/eta-sports_promoters-7-12616-0.csv is properly encoded
      Successfully converted 3 standard ads files into ETA format.
      Input file: /backup/amlops-QA4/us_taxonomy/business_taxonomy/sem/ads_0//ads-texas_rangers_tickets-173-12231-0.csv loading into 'standardAdCsvLoader'.
      Started converting 4 out of 4 standard ads files into ETA format.
      Output file: /backup/amlops-QA4/us_taxonomy/business_taxonomy/sem/ads_0/eta/eta-texas_rangers_tickets-173-12231-0.csv
      /backup/amlops-QA4/us_taxonomy/business_taxonomy/sem/ads_0/eta/eta-texas_rangers_tickets-173-12231-0.csv is properly encoded
      Successfully converted 4 standard ads files into ETA format.

        Attachments

          Activity

            People

            • Assignee:
              varshini.padmanabhan Varshini Padmanabhan (Inactive)
              Reporter:
              varshini.padmanabhan Varshini Padmanabhan (Inactive)
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: