Details
-
Type:
Improvement
-
Status: Closed
-
Priority:
Major
-
Resolution: Fixed
-
Affects Version/s: None
-
Fix Version/s: Fuji
-
Component/s: None
-
Labels:None
-
Environment:
aml13-core1 (y!j staging)
-
Sprint:Sprint 10
Description
When a report download fails because the /var/local/tsa/sedata/yahoojapan does not exist, a FileNotFound exception is thrown. First improve the retry logic to catch and retry IOExceptions, then Improve the retry logic to create the directory and try again.
2014-10-28 02:14:59,458 [P1T1] DEBUG c.t.s.c.y.impl.YahooJapanSSClient - Report '/var/local/tsa/sedata/yahoojapan/tsa-yj-report-admax-2014-10-26-U5.xml.zip' is not ready for download. Waiting 20 seconds.
2014-10-28 02:15:19,459 [P1T1] DEBUG c.t.s.c.y.i.s.ReportServiceClient - ReportServiceClient - Getting download URL for report job ID 222355489
2014-10-28 02:15:19,473 [P1T1] DEBUG c.t.s.c.y.util.SoapClientHandler - REQUEST XML: <?xml version="1.0" encoding="UTF-8" standalone="no"?>
<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
<SOAP-ENV:Header xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/">
<RequestHeader xmlns="http://ss.yahooapis.jp/V5">
<license>1327-7818-1693-5552</license>
<apiAccountId>5567-7860-7473-8517</apiAccountId>
<apiAccountPassword>MtFuj1</apiAccountPassword>
<accountId>616432</accountId>
<onBehalfOfAccountId>883gS-BYve-RUfBV-c4Nb</onBehalfOfAccountId>
<onBehalfOfPassword>TSATrialYahoo55</onBehalfOfPassword>
</RequestHeader>
</SOAP-ENV:Header>
<soap:Body>
<getDownloadUrl xmlns="http://ss.yahooapis.jp/V5">
<selector>
<accountId>616432</accountId>
<reportJobIds>222355489</reportJobIds>
<paging>
<startIndex>1</startIndex>
<numberResults>1</numberResults>
</paging>
</selector>
</getDownloadUrl>
</soap:Body>
</soap:Envelope>
2014-10-28 02:15:19,476 [P1T1] INFO o.a.c.s.R.R.ReportServiceInterface - Outbound Message
---------------------------
ID: 4
Address: https://colo05.ss.yahooapis.jp/services/V5.0/ReportService
Encoding: UTF-8
Http-Method: POST
Content-Type: text/xml
Headers: {Accept=[*/*], SOAPAction=[""]}
Payload: <soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/"><SOAP-ENV:Header xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/"><RequestHeader xmlns="http://ss.yahooapis.jp/V5"><license>1327-7818-1693-5552</license><apiAccountId>5567-7860-7473-8517</apiAccountId><apiAccountPassword>MtFuj1</apiAccountPassword><accountId>616432</accountId><onBehalfOfAccountId>883gS-BYve-RUfBV-c4Nb</onBehalfOfAccountId><onBehalfOfPassword>TSATrialYahoo55</onBehalfOfPassword></RequestHeader></SOAP-ENV:Header><soap:Body><getDownloadUrl xmlns="http://ss.yahooapis.jp/V5"><selector><accountId>616432</accountId><reportJobIds>222355489</reportJobIds><paging><startIndex>1</startIndex><numberResults>1</numberResults></paging></selector></getDownloadUrl></soap:Body></soap:Envelope>
--------------------------------------
2014-10-28 02:15:20,406 [P1T1] INFO o.a.c.s.R.R.ReportServiceInterface - Inbound Message
----------------------------
ID: 4
Response-Code: 200
Encoding: UTF-8
Content-Type: text/xml; charset=utf-8
Headers: {Cache-Control=[private], connection=[close], Content-Length=[1414], content-type=[text/xml; charset=utf-8], Date=[Tue, 28 Oct 2014 02:15:20 GMT], P3P=[policyref="http://privacy.yahoo.co.jp/w3c/p3p_jp.xml", CP="CAO DSP COR CUR ADM DEV TAI PSA PSD IVAi IVDi CONi TELo OTPi OUR DELi SAMi OTRi UNRi PUBi IND PHY ONL UNI PUR FIN COM NAV INT DEM CNT STA POL HEA PRE GOV"], Vary=[Accept-Encoding]}
Payload: <?xml version="1.0" encoding="UTF-8"?>
<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ns1="http://ss.yahooapis.jp/V5"><SOAP-ENV:Header><ns1:ResponseHeader><ns1:service>ReportService</ns1:service><ns1:remainingQuota>4841</ns1:remainingQuota><ns1:quotaUsedForThisRequest>1</ns1:quotaUsedForThisRequest><ns1:timeTakenMillis>0.1183</ns1:timeTakenMillis></ns1:ResponseHeader></SOAP-ENV:Header><SOAP-ENV:Body><ns1:getDownloadUrlResponse><ns1:rval><ns1:totalNumEntries>1</ns1:totalNumEntries><ns1:Page.Type>ReportPage</ns1:Page.Type><ns1:values><ns1:operationSucceeded>true</ns1:operationSucceeded><ns1:reportDownloadUrl><ns1:accountId>616432</ns1:accountId><ns1:reportJobId>222355489</ns1:reportJobId><ns1:reportId>155267990</ns1:reportId><ns1:reportName>616432_ADMAX_20141026</ns1:reportName><ns1:requestTime>20141028111458</ns1:requestTime><ns1:completeTime>20141028111501</ns1:completeTime><ns1:dateRangeType>CUSTOM_DATE</ns1:dateRangeType><ns1:dateRange><ns1:startDate>20141026</ns1:startDate><ns1:endDate>20141026</ns1:endDate></ns1:dateRange><ns1:status>COMPLETED</ns1:status><ns1:downloadUrl>https://colo05.ss.yahooapis.jp/report/V5.0/download/d7EB1bP9KWuqFUW_l89oVnZOaYCpitETv7TyF0qxSZvguVqVO_PVailRTtlfZ94ixTqkaqa4eUYU4kmVianXXDG4nujg0kIXyg8-</ns1:downloadUrl></ns1:reportDownloadUrl></ns1:values></ns1:rval></ns1:getDownloadUrlResponse></SOAP-ENV:Body></SOAP-ENV:Envelope>
--------------------------------------
2014-10-28 02:15:20,421 [P1T1] DEBUG c.t.s.c.y.util.SoapClientHandler - RESPONSE XML: <?xml version="1.0" encoding="UTF-8" standalone="no"?>
<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ns1="http://ss.yahooapis.jp/V5">
<SOAP-ENV:Header>
<ns1:ResponseHeader>
<ns1:service>ReportService</ns1:service>
<ns1:remainingQuota>4841</ns1:remainingQuota>
<ns1:quotaUsedForThisRequest>1</ns1:quotaUsedForThisRequest>
<ns1:timeTakenMillis>0.1183</ns1:timeTakenMillis>
</ns1:ResponseHeader>
</SOAP-ENV:Header>
<SOAP-ENV:Body>
<ns1:getDownloadUrlResponse>
<ns1:rval>
<ns1:totalNumEntries>1</ns1:totalNumEntries>
<ns1:Page.Type>ReportPage</ns1:Page.Type>
<ns1:values>
<ns1:operationSucceeded>true</ns1:operationSucceeded>
<ns1:reportDownloadUrl>
<ns1:accountId>616432</ns1:accountId>
<ns1:reportJobId>222355489</ns1:reportJobId>
<ns1:reportId>155267990</ns1:reportId>
<ns1:reportName>616432_ADMAX_20141026</ns1:reportName>
<ns1:requestTime>20141028111458</ns1:requestTime>
<ns1:completeTime>20141028111501</ns1:completeTime>
<ns1:dateRangeType>CUSTOM_DATE</ns1:dateRangeType>
<ns1:dateRange>
<ns1:startDate>20141026</ns1:startDate>
<ns1:endDate>20141026</ns1:endDate>
</ns1:dateRange>
<ns1:status>COMPLETED</ns1:status>
<ns1:downloadUrl>https://colo05.ss.yahooapis.jp/report/V5.0/download/d7EB1bP9KWuqFUW_l89oVnZOaYCpitETv7TyF0qxSZvguVqVO_PVailRTtlfZ94ixTqkaqa4eUYU4kmVianXXDG4nujg0kIXyg8-</ns1:downloadUrl>
</ns1:reportDownloadUrl>
</ns1:values>
</ns1:rval>
</ns1:getDownloadUrlResponse>
</SOAP-ENV:Body>
</SOAP-ENV:Envelope>
2014-10-28 02:15:21,836 [P1T1] ERROR c.t.s.c.y.impl.YahooJapanSSClient - Error downloading report /var/local/tsa/sedata/yahoojapan/tsa-yj-report-admax-2014-10-26-U5.xml.zip
java.io.FileNotFoundException: /var/local/tsa/sedata/yahoojapan/tsa-yj-report-admax-2014-10-26-U5.xml.zip (No such file or directory)
at java.io.FileOutputStream.open(Native Method) ~[na:1.6.0_27]
at java.io.FileOutputStream.<init>(FileOutputStream.java:194) ~[na:1.6.0_27]
at java.io.FileOutputStream.<init>(FileOutputStream.java:145) ~[na:1.6.0_27]
at com.thesearchagency.services.client.yahoojapan.impl.YahooJapanSSClient.downloadReport(YahooJapanSSClient.java:623) [deployment-bidmgr-shaded-jar-2.7.0-YAHOO-SNAPSHOT.jar:na]
at com.thesearchagency.services.client.yahoojapan.impl.YahooJapanSSClient.getReport(YahooJapanSSClient.java:481) [deployment-bidmgr-shaded-jar-2.7.0-YAHOO-SNAPSHOT.jar:na]
at com.thesearchagency.searchengines.yahoojapan.YahooJapanSearchEngine.downloadReport(YahooJapanSearchEngine.java:687) [deployment-bidmgr-shaded-jar-2.7.0-YAHOO-SNAPSHOT.jar:na]
at com.thesearchagency.searchengines.yahoojapan.reporting.YahooJapanReportSEUserWorker.call(YahooJapanReportSEUserWorker.java:92) [deployment-bidmgr-shaded-jar-2.7.0-YAHOO-SNAPSHOT.jar:na]
at com.thesearchagency.searchengines.yahoojapan.reporting.YahooJapanReportSEUserWorker.call(YahooJapanReportSEUserWorker.java:27) [deployment-bidmgr-shaded-jar-2.7.0-YAHOO-SNAPSHOT.jar:na]
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) [na:1.6.0_27]
at java.util.concurrent.FutureTask.run(FutureTask.java:138) [na:1.6.0_27]
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) [na:1.6.0_27]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) [na:1.6.0_27]
at java.lang.Thread.run(Thread.java:662) [na:1.6.0_27]
2014-10-28 02:15:21,837 [P1T1] DEBUG c.t.s.c.y.i.s.ReportServiceClient - ReportServiceClient - removing report jobs [222355489] for account 616432
