[TSAApp@aml-qa7-core1 bidmgr]$ ./sedatacheck.sh -d3 -U202 -T2016-05-11 -E2016-05-12 ------- Loaded DatabaseAccounts properties from file:/usr/local/tsa/bidmgr-distribution-2.9.1-dev-t20160510-0933/libs/deployment-bidmgr-shaded-jar-2.9.1.jar!/core-jdb-datasource.properties ------- {datasource-user=spike, datasource-password=tar63t} 2016-05-12 14:20:19.886 (3) [main]: Debug Level set to 3 2016-05-12 14:20:19.888 (3) [main]: Licensing environment detected, loading configuration. 2016-05-12 14:20:19.889 (3) [main]: Loaded property: distribution-service-url=http://aml-qa7-front1:8081/distribution 2016-05-12 14:20:19.889 (3) [main]: Loaded property: distribution-service-username=readwrite 2016-05-12 14:20:19.889 (3) [main]: Loaded property: distribution-service-password=readwrite 2016-05-12 14:20:19.889 (3) [main]: Loaded property: sapi-auth-url=http://aml-qa7-front1:8080/sapi/rest/security/tokens 2016-05-12 14:20:19.890 (3) [main]: Loaded property: sapi-merchant-service-url=http://aml-qa7-front1:8080/sapi/rest/merchants 2016-05-12 14:20:19.890 (3) [main]: Loaded property: sapi-reseller-service-url=http://aml-qa7-front1:8080/sapi/rest/resellers 2016-05-12 14:20:19.890 (3) [main]: Loaded property: sapi-username=admin+tsa@thesearchagency.com 2016-05-12 14:20:19.890 (3) [main]: Loaded property: sapi-password=admin 2016-05-12 14:20:19.890 (3) [main]: Loaded property: sapi-realm=thesearchagency 2016-05-12 14:20:19.890 (3) [main]: Loaded property: tsa.google.mcc.developerToken=ojCnjZaM6RD1h0yt_DOJZg 2016-05-12 14:20:19.890 (3) [main]: Loaded property: tsa.google.mcc.isReseller=true 2016-05-12 14:20:19.891 (3) [main]: Loaded property: tsa.google.mcc.userAgent=AdMaxLocal AdWords Software TSA QA 2016-05-12 14:20:19.891 (3) [main]: Loaded property: tsa.timezone=UTC 2016-05-12 14:20:20.006 (3) [main]: DatabasePool with a limit of 7 created 2016-05-12 14:20:20.387 (3) [main]: JDBC Driver Version: 5.1 (com.mysql.jdbc.Driver) 2016-05-12 14:20:20.421 (3) [main]: Debug Level set to 3 2016-05-12 14:20:20.421 (3) [main]: =============== Starting Search Engine Data Checker ================ 2016-05-12 14:20:20.421 (3) [main]: ---------------- Distribution Dispatcher for Search Engine Data Checker starting 2016-05-12 14:20:20.469 (3) [P1T1]: Starting Distribution 178 2016-05-12 14:20:20.469 (3) [P1T1]: ----------------Search Engine Data Checker - Distribution Worker starting 2016-05-12 14:20:20.495 (3) [P1T1]: JDBC Driver Version: 5.1 (com.mysql.jdbc.Driver) 2016-05-12 14:20:20.506 (3) [P2T1]: Starting User (202) "1f2e1716-7124-4ec6-94b6-792abf8c0673_1" 2016-05-12 14:20:20.549 (3) [P2T1]: JDBC Driver Version: 5.1 (com.mysql.jdbc.Driver) 2016-05-12 14:20:22 INFO [P2T1] com.thesearchagency.searchengines.yahoojapan.YahooCredentials:232 - Decrpting YJ Credentials For aSearchEngineUserId: 202 2016-05-12 14:20:23 INFO [P2T1] org.apache.cxf.wsdl.service.factory.ReflectionServiceFactoryBean:434 - Creating Service {http://ss.yahooapis.jp/V5}ReportDefinitionService from class jp.yahooapis.ss.V5.ReportDefinitionService.ReportDefinitionServiceInterface 2016-05-12 14:20:23 INFO [P2T1] org.apache.cxf.wsdl.service.factory.ReflectionServiceFactoryBean:434 - Creating Service {http://ss.yahooapis.jp/V5}ReportDefinitionService from class jp.yahooapis.ss.V5.ReportDefinitionService.ReportDefinitionServiceInterface 2016-05-12 14:20:24 INFO [P2T1] org.apache.cxf.services.ReportDefinitionService.ReportDefinitionServiceInterfacePort.ReportDefinitionServiceInterface:253 - Outbound Message --------------------------- ID: 1 Address: https://colo03.ss.yahooapis.jp/services/V5.3/ReportDefinitionService Encoding: UTF-8 Http-Method: POST Content-Type: text/xml Headers: {Accept=[*/*], SOAPAction=[""]} Payload: 1327-7818-1693-55525567-7860-7473-8517MtFuj1586827yTMCt-rZDE-h0Ldu-w24fTSATrialYahoo55ADD586827586827586827_DATA_CHECK_20160511CAMPAIGNCUSTOM_DATE2016051120160512IMPRESSIONSGREATER_THAN0DAY+CAMPAIGNIDCAMPAIGNIDCAMPAIGNNAMEIMPRESSIONSCLICKSCOSTXMLUTF-8ONENONETIMENO -------------------------------------- 2016-05-12 14:20:26 INFO [P2T1] org.apache.cxf.services.ReportDefinitionService.ReportDefinitionServiceInterfacePort.ReportDefinitionServiceInterface:253 - Inbound Message ---------------------------- ID: 1 Response-Code: 200 Encoding: UTF-8 Content-Type: text/xml; charset=utf-8 Headers: {Age=[0], Cache-Control=[private], connection=[keep-alive], Content-Length=[1662], content-type=[text/xml; charset=utf-8], Date=[Thu, 12 May 2016 14:09:48 GMT], Server=[ATS], Vary=[Accept-Encoding]} Payload: ReportDefinitionService-1-10.3096ReportDefinitionReturnValueADDtrue331381322586827586827_DATA_CHECK_20160511CAMPAIGNCUSTOM_DATE2016051120160512IMPRESSIONSGREATER_THAN0DAY+CAMPAIGNIDCAMPAIGNIDCAMPAIGNNAMEIMPRESSIONSCLICKSCOSTXMLUTF-8ONENONETIMENO -------------------------------------- 2016-05-12 14:20:26 INFO [P2T1] org.apache.cxf.wsdl.service.factory.ReflectionServiceFactoryBean:434 - Creating Service {http://ss.yahooapis.jp/V5}ReportService from class jp.yahooapis.ss.V5.ReportService.ReportServiceInterface 2016-05-12 14:20:26 INFO [P2T1] org.apache.cxf.wsdl.service.factory.ReflectionServiceFactoryBean:434 - Creating Service {http://ss.yahooapis.jp/V5}ReportService from class jp.yahooapis.ss.V5.ReportService.ReportServiceInterface 2016-05-12 14:20:26 INFO [P2T1] org.apache.cxf.services.ReportService.ReportServiceInterfacePort.ReportServiceInterface:253 - Outbound Message --------------------------- ID: 2 Address: https://colo03.ss.yahooapis.jp/services/V5.3/ReportService Encoding: UTF-8 Http-Method: POST Content-Type: text/xml Headers: {Accept=[*/*], SOAPAction=[""]} Payload: 1327-7818-1693-55525567-7860-7473-8517MtFuj1586827yTMCt-rZDE-h0Ldu-w24fTSATrialYahoo55ADD586827331381322 -------------------------------------- 2016-05-12 14:20:27 INFO [P2T1] org.apache.cxf.services.ReportService.ReportServiceInterfacePort.ReportServiceInterface:253 - Inbound Message ---------------------------- ID: 2 Response-Code: 200 Encoding: UTF-8 Content-Type: text/xml; charset=utf-8 Headers: {Age=[2], Cache-Control=[private], connection=[keep-alive], Content-Length=[1126], content-type=[text/xml; charset=utf-8], Date=[Thu, 12 May 2016 14:09:49 GMT], Server=[ATS], Vary=[Accept-Encoding]} Payload: ReportService-1-10.4337ReportReturnValueADDtrue58682750246435433138132220160512230950CUSTOM_DATE2016051120160512IN_PROGRESS -------------------------------------- 2016-05-12 14:20:27 INFO [P2T1] org.apache.cxf.services.ReportService.ReportServiceInterfacePort.ReportServiceInterface:253 - Outbound Message --------------------------- ID: 3 Address: https://colo03.ss.yahooapis.jp/services/V5.3/ReportService Encoding: UTF-8 Http-Method: POST Content-Type: text/xml Headers: {Accept=[*/*], SOAPAction=[""]} Payload: 1327-7818-1693-55525567-7860-7473-8517MtFuj1586827yTMCt-rZDE-h0Ldu-w24fTSATrialYahoo5558682750246435411 -------------------------------------- 2016-05-12 14:20:28 INFO [P2T1] org.apache.cxf.services.ReportService.ReportServiceInterfacePort.ReportServiceInterface:253 - Inbound Message ---------------------------- ID: 3 Response-Code: 200 Encoding: UTF-8 Content-Type: text/xml; charset=utf-8 Headers: {Age=[0], Cache-Control=[private], connection=[keep-alive], Content-Length=[1182], content-type=[text/xml; charset=utf-8], Date=[Thu, 12 May 2016 14:09:50 GMT], Server=[ATS], Vary=[Accept-Encoding]} Payload: ReportService-1-10.11771ReportPagetrue586827502464354331381322586827_DATA_CHECK_2016051120160512230950CUSTOM_DATE2016051120160512IN_PROGRESS -------------------------------------- 2016-05-12 14:20:48 INFO [P2T1] org.apache.cxf.services.ReportService.ReportServiceInterfacePort.ReportServiceInterface:253 - Outbound Message --------------------------- ID: 4 Address: https://colo03.ss.yahooapis.jp/services/V5.3/ReportService Encoding: UTF-8 Http-Method: POST Content-Type: text/xml Headers: {Accept=[*/*], SOAPAction=[""]} Payload: 1327-7818-1693-55525567-7860-7473-8517MtFuj1586827yTMCt-rZDE-h0Ldu-w24fTSATrialYahoo5558682750246435411 -------------------------------------- 2016-05-12 14:20:48 INFO [P2T1] org.apache.cxf.services.ReportService.ReportServiceInterfacePort.ReportServiceInterface:253 - Inbound Message ---------------------------- ID: 4 Response-Code: 200 Encoding: UTF-8 Content-Type: text/xml; charset=utf-8 Headers: {Age=[0], Cache-Control=[private], connection=[keep-alive], Content-Length=[1421], content-type=[text/xml; charset=utf-8], Date=[Thu, 12 May 2016 14:10:11 GMT], Server=[ATS], Vary=[Accept-Encoding]} Payload: ReportService-1-10.0931ReportPagetrue586827502464354331381322586827_DATA_CHECK_201605112016051223095020160512230955CUSTOM_DATE2016051120160512COMPLETEDhttps://colo03.ss.yahooapis.jp/report/V5.3/download/6Y8u1Z0GWxEN0MGEcRB1.FryAepNod.lQvEONO08qp0nDo_1hnRYg1reZgHIB2GeppS6_1Yi2UvsJx56rZfpzkt2Gj26r17FhG_ZTw-- -------------------------------------- 2016-05-12 14:20:49 INFO [P2T1] org.apache.cxf.services.ReportService.ReportServiceInterfacePort.ReportServiceInterface:253 - Outbound Message --------------------------- ID: 5 Address: https://colo03.ss.yahooapis.jp/services/V5.3/ReportService Encoding: UTF-8 Http-Method: POST Content-Type: text/xml Headers: {Accept=[*/*], SOAPAction=[""]} Payload: 1327-7818-1693-55525567-7860-7473-8517MtFuj1586827yTMCt-rZDE-h0Ldu-w24fTSATrialYahoo55REMOVE586827502464354 -------------------------------------- 2016-05-12 14:20:50 INFO [P2T1] org.apache.cxf.services.ReportService.ReportServiceInterfacePort.ReportServiceInterface:253 - Inbound Message ---------------------------- ID: 5 Response-Code: 200 Encoding: UTF-8 Content-Type: text/xml; charset=utf-8 Headers: {Age=[0], Cache-Control=[private], connection=[keep-alive], Content-Length=[814], content-type=[text/xml; charset=utf-8], Date=[Thu, 12 May 2016 14:10:12 GMT], Server=[ATS], Vary=[Accept-Encoding]} Payload: ReportService-1-10.1013ReportReturnValueREMOVEtrue502464354 -------------------------------------- 2016-05-12 14:20:50.054 (3) [P2T1]: Parsing /var/local/tsa/sedata/yahoojapan/tsa-check-report-daily-2016-05-11-2016-05-12-U202.xml.zip into staging database 2016-05-12 14:20:50.116 (3) [P2T1]: JDBC Driver Version: 5.1 (com.mysql.jdbc.Driver) 2016-05-12 14:20:50.121 (3) [P2T1]: DatabasePool with a limit of 7 created 2016-05-12 14:20:50.144 (3) [P2T1]: JDBC Driver Version: 5.1 (com.mysql.jdbc.Driver) 2016-05-12 14:20:50.158 (3) [P2T1]: Deleting existing records where ((`date`>="2016-05-11") and (`date`<"2016-05-12") and (`campaignid` in ("1193273","1193274","1193275","1193276","1193282","1193283","1193310","1193311","1193312","1193313","1193319","1193321","1193323","1193324","1193343","1193344","1193345","1193346","1193360","1193361","1193362","1193363","1193364","1193365","1193366","1193367","1193729","1193730","1193731","1193732","1193809","1193810","1226134","1226135","853664","853665"))) 2016-05-12 14:20:50.161 (3) [P2T1]: Inserting new data into `staging`.`yahooJDailyCheck` 2016-05-12 14:20:50.200 (3) [P2T1]: Finished User (202) "1f2e1716-7124-4ec6-94b6-792abf8c0673_1" 2016-05-12 14:20:50.201 (3) [P1T1]: ------------Search Engine Data Checker - Distribution Worker done 2016-05-12 14:20:50.201 (3) [main]: ------------ Distribution Dispatcher for Search Engine Data Checker done 2016-05-12 14:20:50.201 (3) [main]: =============== Completed Search Engine Data Checker ================ 2016-05-12 14:20:50.203 (3) [main]: Search Engine Data Checker: 29.780s 2016-05-12 14:20:50.203 (3) [main]: by key: 2016-05-12 14:20:50.203 (3) [main]: db: 0.021s 2016-05-12 14:20:50.203 (3) [main]: by thread: 2016-05-12 14:20:50.203 (3) [main]: P2T1: 0.012s 2016-05-12 14:20:50.203 (3) [main]: P1T1: 0.001s 2016-05-12 14:20:50.203 (3) [main]: main: 0.008s 2016-05-12 14:20:50.203 (3) [main]: other: 29.759s