Running: ./v2pixel_playback.sh -a 541 -d 4
./v2pixel_playback.sh: line 11: ifconfig: command not found
2011-05-04 05:56:55.094 (3) [main]: Debug Level set to 4
2011-05-04 05:56:55.120 (3) [main]: No licensing config file found, assuming TSA production environment
2011-05-04 05:56:55.124 (2) [main]: Using DB user/pass: spike / tar63t
2011-05-04 05:56:55.133 (2) [main]: Event Batch Dispatch Interval Set To: 900
2011-05-04 05:56:55.134 (2) [main]: Event Batch Dispatch Limit Set To: 10
2011-05-04 05:56:55.134 (2) [main]: Event Generation: Enabled
2011-05-04 05:56:55.134 (2) [main]: Account Dispatcher Thread Pool Size Set To: 8
2011-05-04 05:56:55.137 (2) [main]: Using Account IDs: (541)
2011-05-04 05:56:55.158 (2) [main]: Account List Operator:  in 
2011-05-04 05:56:55.398 (2) [main]: Pixel Log Directory: /usr/local/share/tsa/logs/spike-v2
2011-05-04 05:56:55.766 (4) [main]: no account, or instance for databaseInstance, skipping lookup...
2011-05-04 05:56:55.769 (3) [main]: ----------------Pixel Log Playback starting
2011-05-04 05:57:02.046 (4) [main]: opened [jdbc:mysql://tsacommon-01-write/tsacommon?useUnicode=true&characterEncoding=UTF8&zeroDateTimeBehavior=convertToNull&dontTrackOpenResources=true&jdbcCompliantTruncation=false]
2011-05-04 05:57:02.394 (4) [main]: spike: SQL->[select * from `tsacommon`.`accounts` where ((`tsacommon`.`accounts`.`id` in (541))) order by `tsacommon`.`accounts`.`tier` asc,`tsacommon`.`accounts`.`priority` asc,`tsacommon`.`accounts`.`id` desc]
2011-05-04 05:57:02.927 (4) [main]: time=0.656
2011-05-04 05:57:02.931 (4) [main]: Query executed in 0.656s
2011-05-04 05:57:03.295 (4) [P1T1]: no account, or instance for databaseInstance, skipping lookup...
2011-05-04 05:57:07.124 (4) [P1T1]: opened [jdbc:mysql://tsacommon-01-write/tsacommon?useUnicode=true&characterEncoding=UTF8&zeroDateTimeBehavior=convertToNull&dontTrackOpenResources=true&jdbcCompliantTruncation=false]
2011-05-04 05:57:07.279 (4) [P1T1]: spike: SQL->[select * from `tsacommon`.`accounts` where (`id`=541) limit 1]
2011-05-04 05:57:07.292 (4) [P1T1]: time=0.012
2011-05-04 05:57:07.292 (4) [P1T1]: Query executed in 0.012s
2011-05-04 05:57:07.296 (4) [P1T1]:   Record Loaded
2011-05-04 05:57:07.296 (4) [P1T1]: DataCache: put key acct-st-tracker 541
2011-05-04 05:57:07.312 (4) [P1T1]: spike: SQL->[select * from `tsacommon`.`databaseInstances` where ((`type`="mysql") and (`name`="acct-st-tracker-1")) limit 1]
2011-05-04 05:57:07.324 (4) [P1T1]: time=0.012
2011-05-04 05:57:07.324 (4) [P1T1]: Query executed in 0.012s
2011-05-04 05:57:07.326 (4) [P1T1]:   Record Loaded
2011-05-04 05:57:07.328 (4) [P1T1]: DataCache: put key acct-st-tracker-1
2011-05-04 05:57:07.332 (4) [P1T1]: closed [jdbc:mysql://tsacommon-01-write/tsacommon?useUnicode=true&characterEncoding=UTF8&zeroDateTimeBehavior=convertToNull&dontTrackOpenResources=true&jdbcCompliantTruncation=false]
2011-05-04 05:57:07.344 (3) [P1T1]: DatabasePool with a limit of 5 created
2011-05-04 05:57:07.351 (4) [ConnectionCloser]: Starting DatabasePool ConnectionCloserThread
2011-05-04 05:57:07.699 (4) [P1T1]: opened [jdbc:mysql://acctdb-01-write:3306/st-tracker?useUnicode=true&characterEncoding=UTF8&zeroDateTimeBehavior=convertToNull&dontTrackOpenResources=true&jdbcCompliantTruncation=false]
2011-05-04 05:57:07.700 (4) [P1T1]: Created new Object
2011-05-04 05:57:07.702 (4) [P1T1]: spike: SQL->[select @@version]
2011-05-04 05:57:07.708 (4) [P1T1]: time=0.0080
2011-05-04 05:57:07.708 (4) [P1T1]: Query executed in 0.008s
2011-05-04 05:57:07.711 (4) [P1T1]: test query succeeded, returned "5.1.49-rel11.3"
2011-05-04 05:57:07.712 (4) [P1T1]: no account, or instance for databaseInstance, skipping lookup...
2011-05-04 05:57:07.712 (3) [P1T1]: DatabasePool with a limit of 5 created
2011-05-04 05:57:08.916 (4) [P1T1]: opened [jdbc:mysql://tsacommon-01-write/tsacommon?useUnicode=true&characterEncoding=UTF8&zeroDateTimeBehavior=convertToNull&dontTrackOpenResources=true&jdbcCompliantTruncation=false]
2011-05-04 05:57:08.919 (4) [P1T1]: Created new Object
2011-05-04 05:57:08.920 (4) [P1T1]: spike: SQL->[select @@version]
2011-05-04 05:57:08.928 (4) [P1T1]: time=0.0090
2011-05-04 05:57:08.928 (4) [P1T1]: Query executed in 0.009s
2011-05-04 05:57:08.931 (4) [P1T1]: test query succeeded, returned "5.0.30-enterprise-gpl"
2011-05-04 05:57:09.132 (4) [P1T1]: spike: SQL->[select * from `tsacommon`.`sites` where ((`accountID`=541)) limit 1]
2011-05-04 05:57:09.144 (4) [P1T1]: time=0.012
2011-05-04 05:57:09.147 (4) [P1T1]: Query executed in 0.012s
2011-05-04 05:57:09.152 (4) [P1T1]:   Record Loaded
2011-05-04 05:57:09.152 (2) [P1T1]: Site Initialized For accountID: 541 / siteID: 560
2011-05-04 05:57:09.263 (3) [P1T1]: Using TimeZone: America/Los_Angeles
2011-05-04 05:57:09.544 (4) [P1T1]: spike: SQL->[select * from `settings` where ((((`settings`.`scope`="sites")) and ((`settings`.`scopeId`=560))))]
2011-05-04 05:57:09.553 (4) [P1T1]: time=0.0090
2011-05-04 05:57:09.555 (4) [P1T1]: Query executed in 0.009s
2011-05-04 05:57:09.560 (4) [P1T1]: spike: SQL->[select * from `settingDefaults` where (`settingDefaults`.`scope`="sites")]
2011-05-04 05:57:09.567 (4) [P1T1]: time=0.0070
2011-05-04 05:57:09.568 (4) [P1T1]: Query executed in 0.007s
2011-05-04 05:57:09.572 (4) [P1T1]: DataCache: put key sites 560
2011-05-04 05:57:09.572 (3) [P1T1]: Schema 'waypointFilteringDisabled' doesn't exist in scope sites
2011-05-04 05:57:09.843 (4) [P1T1]: spike: SQL->[select `id`,`field`,`value` from `spike_v2`.`waypointFilter` where `enabled` = 1 and `siteID` in (0, 560) order by `siteID` desc]
2011-05-04 05:57:09.911 (4) [P1T1]: time=0.271
2011-05-04 05:57:09.919 (4) [P1T1]: Query executed in 0.271s
2011-05-04 05:57:09.948 (3) [P1T1]: Loaded 3 User Agent Filters
2011-05-04 05:57:09.949 (3) [P1T1]: Loaded 32 Remote Address (IP) Filters
2011-05-04 05:57:09.984 (4) [P1T1]: spike: SQL->[select `id`,`field`,`value` from `spike_v2`.`waypointFilter` where `enabled` = 1 and `siteID` in (0, 560) order by `siteID` desc]
2011-05-04 05:57:09.996 (4) [P1T1]: time=0.012
2011-05-04 05:57:09.996 (4) [P1T1]: Query executed in 0.012s
2011-05-04 05:57:10.183 (3) [P1T1]: Loaded 3 User Agent Filters
2011-05-04 05:57:10.184 (3) [P1T1]: Loaded 32 Remote Address (IP) Filters
2011-05-04 05:57:10.196 (4) [P1T1]: spike: SQL->[select * from `st-tracker`.`waypoints` where siteID = 560]
2011-05-04 05:57:10.205 (4) [P1T1]: time=0.0090
2011-05-04 05:57:10.207 (4) [P1T1]: Query executed in 0.009s
2011-05-04 05:57:10.511 (2) [P1T1]: Successfully Opened File: /usr/local/share/tsa/logs/spike-v2/working/2011-05-04-1300-560.tsv
2011-05-04 05:57:10.631 (3) [P1T1]: Temp Processing File: /tmp/pixel_log62058csv
2011-05-04 05:57:10.643 (3) [P1T1]: Temp Processing File: /tmp/pixel_product_log62059csv
2011-05-04 05:57:10.840 (2) [P1T1]: Playback Started For File [2011-05-04-1300-560.tsv] For Account: TSA - Martian Tourism [541]
2011-05-04 05:57:10.860 (4) [P1T1]: spike: SQL->[select `spike_v2`.`timeOK`(560,1304514551,439334)]
2011-05-04 05:57:10.892 (4) [P1T1]: time=0.032
2011-05-04 05:57:10.893 (4) [P1T1]: Query executed in 0.032s
2011-05-04 05:57:10.921 (4) [P1T1]: spike: SQL->[select * from `settings` where ((`scopeId`=541) and (`scope`="accounts") and (`schema`="altKeyParam")) limit 1]
2011-05-04 05:57:10.931 (4) [P1T1]: time=0.01
2011-05-04 05:57:10.931 (4) [P1T1]: Query executed in 0.010s
2011-05-04 05:57:10.932 (4) [P1T1]:   Record Not Found
2011-05-04 05:57:10.939 (4) [P1T1]: spike: SQL->[select * from `settingDefaults` where ((`scope`="accounts") and (`schema`="altKeyParam")) limit 1]
2011-05-04 05:57:10.945 (4) [P1T1]: time=0.0060
2011-05-04 05:57:10.945 (4) [P1T1]: Query executed in 0.006s
2011-05-04 05:57:10.948 (4) [P1T1]:   Record Loaded
2011-05-04 05:57:10.951 (4) [P1T1]: spike: SQL->[select * from `settings` where ((`scopeId`=541) and (`scope`="accounts") and (`schema`="creativeAltKeyParam")) limit 1]
2011-05-04 05:57:10.959 (4) [P1T1]: time=0.0080
2011-05-04 05:57:10.960 (4) [P1T1]: Query executed in 0.008s
2011-05-04 05:57:10.961 (4) [P1T1]:   Record Not Found
2011-05-04 05:57:10.963 (4) [P1T1]: spike: SQL->[select * from `settingDefaults` where ((`scope`="accounts") and (`schema`="creativeAltKeyParam")) limit 1]
2011-05-04 05:57:10.971 (4) [P1T1]: time=0.0080
2011-05-04 05:57:10.973 (4) [P1T1]: Query executed in 0.008s
2011-05-04 05:57:10.975 (4) [P1T1]:   Record Loaded
2011-05-04 05:57:10.987 (3) [P1T1]: Initializing Search Engine IP Referers Map
2011-05-04 05:57:11.148 (4) [P1T1]: spike: SQL->[select * from `searchEngineReferers` where (`searchEngineReferers`.`ipAddresses`!="") order by `searchEngineReferers`.`priority`]
2011-05-04 05:57:11.159 (4) [P1T1]: time=0.011
2011-05-04 05:57:11.161 (4) [P1T1]: Query executed in 0.011s
2011-05-04 05:57:11.325 (2) [P1T1]: Preprocess Complete of 2 Lines, Filtered: 0 Lines, Errors: 0 Lines, Elapsed Time: 0.681s
2011-05-04 05:57:11.327 (4) [P1T1]: spike: SQL->[set SESSION net_write_timeout=3600, SESSION net_read_timeout=3600]
2011-05-04 05:57:11.335 (4) [P1T1]: time=0.0080
2011-05-04 05:57:11.533 (4) [P1T1]: Query executed in 0.008s
2011-05-04 05:57:11.535 (4) [P1T1]: spike: SQL->[create temporary table `tmp`.`waypointLog`(   `rowNum` int not null primary key,   `unixTimestamp` INTEGER default 0,   `microseconds` int(6) not null default 0,   `siteID` smallint not null,   `waypointID` int not null,   `visitorKey` varchar(128),   `visitorKeyHash` integer unsigned not null,   `sessionKey` varchar(128),   `sessionKeyHash` integer unsigned not null,   `sourceKey` varchar(255),   `creativeKey` varchar(255),   `baseUltimateRefererURL` varchar(1024),   `userAgent` varchar(255),   `cpc` double,   `distributionID` int,   `campaignID` int default null,   `internalURLPatternMatch` tinyint,   `isOrganic` tinyint,   `keyword` varchar(255) default null,   `sourceID` int null,   `searchEngineListingID` int not null default 0,   `userAgentHash` integer unsigned not null,   `userAgentID` int not null default 0,   `refererURL` varchar(1024) default NULL,   `ultimateRefererURL` varchar(1024) default NULL,  `remoteAddr` varchar(45) default NULL,  `params` varchar(255) default NULL,  `debounced` tinyint(1) not null default '0',   `backRef` varchar(128) default null,   `matchType` varchar(16) default null,   `destination` varchar(255) default null,   `tsaTime` BIGINT default null,   `duplicate` tinyint(1) not null default '0',   `recordedOutOfOrder` tinyint(1) not null default '0',   `parentUnixTimestamp` INTEGER default null,   `parentMicroseconds` int default null,   `parentRowNum` int default null,   key visitorKey (visitorKey, sessionKey),   key userAgentID(userAgentID),   key `baseUltimateRefererURL` (`baseUltimateRefererURL`(255)),   key `debounced`(`debounced`) ) ENGINE=MyISAM DEFAULT CHARSET=utf8]
2011-05-04 05:57:11.617 (4) [P1T1]: time=0.082
2011-05-04 05:57:11.619 (4) [P1T1]: Query executed in 0.082s
2011-05-04 05:57:11.621 (4) [P1T1]: spike: SQL->[create temporary table `tmp`.`conversionLog`(   `rowNum` int not null,   `unixTimestamp` INTEGER default 0,   `microseconds` int(6) not null default 0,   `siteID` smallint not null,   `convNum` int(11) NOT NULL,   `name` varchar(255) CHARACTER SET latin1 NOT NULL,   `count` double NOT NULL default '0',   `value` double NOT NULL default '0',   `cogs` double NOT NULL default '0',   `debounced` tinyint(1) not null default '0',   `productID` integer default null,   INDEX `productID` (`productID`),   INDEX `rowNum`(`rowNum`) ) ENGINE=MyISAM DEFAULT CHARSET=utf8]
2011-05-04 05:57:11.631 (4) [P1T1]: time=0.01
2011-05-04 05:57:11.633 (4) [P1T1]: Query executed in 0.010s
2011-05-04 05:57:11.635 (4) [P1T1]: spike: SQL->[load data local infile '/tmp/pixel_log62058csv' into table `tmp`.`waypointLog` CHARACTER SET UTF8 fields terminated by ',' optionally enclosed by '"' escaped by ''  (`rowNum`, `unixTimestamp`, `microseconds`, `siteID`, `waypointID`,   `visitorKey`, `visitorKeyHash`, `sessionKey`, `sessionKeyHash`, `sourceKey`, `creativeKey`,   `baseUltimateRefererURL`, `userAgent`, `userAgentHash`, @cpc, `distributionID`,   @campaignID, `internalURLPatternMatch`,   `keyword`,   `isOrganic`, @sourceID,   `refererURL`,   `ultimateRefererURL`,   `remoteAddr`,   `params`,   `backRef`,   @matchType,   `destination`,   @tsaTime,  `recordedOutOfOrder` )  set `sourceID`=if (@sourceID > 0, @sourceID, null),  `matchType`=if(@matchType = '', null, @matchType),  `tsaTime`=if(@tsaTime = '', null, @tsaTime),  `campaignID`=if(@campaignID = '', null, @campaignID),  `cpc`=if(@cpc='', null, @cpc)]
2011-05-04 05:57:11.707 (4) [P1T1]: time=0.074
2011-05-04 05:57:11.709 (4) [P1T1]: Query executed in 0.074s
2011-05-04 05:57:11.711 (4) [P1T1]: spike: SQL->[update `tmp`.`waypointLog` set  `keyword` = if (`keyword` = '', null, `keyword`),  `backRef` = if (`backRef` = '', null, `backRef`),  `destination` = if (`destination` = '', null, `destination`)]
2011-05-04 05:57:11.723 (4) [P1T1]: time=0.012
2011-05-04 05:57:11.724 (4) [P1T1]: Query executed in 0.012s
2011-05-04 05:57:11.725 (2) [P1T1]: loadStagingData completed: 0.92s
2011-05-04 05:57:11.727 (4) [P1T1]: spike: SQL->[load data local infile '/tmp/pixel_product_log62059csv' into table `tmp`.`conversionLog` CHARACTER SET UTF8 fields terminated by ',' optionally enclosed by '"' escaped by ''  (`rowNum`, `unixTimestamp`, `microseconds`, `siteID`, `convNum`,  `name`,  `count`, `value`, `cogs`) ]
2011-05-04 05:57:11.739 (4) [P1T1]: time=0.014
2011-05-04 05:57:11.741 (4) [P1T1]: Query executed in 0.014s
2011-05-04 05:57:11.857 (2) [P1T1]: loadWaypointConversions completed: 0.132s
2011-05-04 05:57:11.863 (4) [P1T1]: spike: SQL->[update `tmp`.`waypointLog` wpl  join `spike_v2`.`waypointLog` wp on (  wp.siteID = wpl.siteID and  wp.unixTimestamp = wpl.unixTimestamp and  wp.microseconds = wpl.microseconds and  wp.rowNum = wpl.rowNum)  set wpl.duplicate = 1]
2011-05-04 05:57:12.085 (4) [P1T1]: time=0.222
2011-05-04 05:57:12.087 (4) [P1T1]: Query executed in 0.222s
2011-05-04 05:57:12.089 (4) [P1T1]: spike: SQL->[select count(rowNum) from `tmp`.`waypointLog` where duplicate = 1]
2011-05-04 05:57:12.097 (4) [P1T1]: time=0.0080
2011-05-04 05:57:12.099 (4) [P1T1]: Query executed in 0.008s
2011-05-04 05:57:12.101 (3) [P1T1]: DuplicateWaypointFilter::getCount completed: 0.12s
2011-05-04 05:57:12.101 (3) [P1T1]: DuplicateWaypointFilter::filter Found 0 duplicates, completed: 0.238s
2011-05-04 05:57:12.103 (2) [P1T1]: filterDuplicateWaypoints completed: 0.244s
2011-05-04 05:57:12.109 (4) [P1T1]: spike: SQL->[update `tmp`.`conversionLog` cst  join `st-tracker`.products p on (p.productCode = cst.name and p.accountID = 541)  set cst.productID = p.id  where cst.productID is null ]
2011-05-04 05:57:12.117 (4) [P1T1]: time=0.0080
2011-05-04 05:57:12.119 (4) [P1T1]: Query executed in 0.008s
2011-05-04 05:57:12.121 (3) [P1T1]: Conversions::update completed: 0.12s
2011-05-04 05:57:12.123 (4) [P1T1]: spike: SQL->[select * from `settings` where ((`scopeId`=560) and (`scope`="sites") and (`schema`="countConversions")) limit 1]
2011-05-04 05:57:12.131 (4) [P1T1]: time=0.0080
2011-05-04 05:57:12.133 (4) [P1T1]: Query executed in 0.008s
2011-05-04 05:57:12.133 (4) [P1T1]:   Record Not Found
2011-05-04 05:57:12.135 (4) [P1T1]: spike: SQL->[select * from `settingDefaults` where ((`scope`="sites") and (`schema`="countConversions")) limit 1]
2011-05-04 05:57:12.143 (4) [P1T1]: time=0.0060
2011-05-04 05:57:12.143 (4) [P1T1]: Query executed in 0.006s
2011-05-04 05:57:12.143 (4) [P1T1]:   Record Not Found
2011-05-04 05:57:12.145 (3) [P1T1]: Conversions::doCountConversions completed: 0.24s
2011-05-04 05:57:12.145 (4) [P1T1]: spike: SQL->[insert ignore into `st-tracker`.products  (`productCode`, `accountID`, `siteID`, `countConversions`)  select name, 541,  siteID, 'false' from `tmp`.`conversionLog` where productID is null and  name <> 'TSAOther' and  name <> 'TSACounted' and  name <> 'TSAUncounted']
2011-05-04 05:57:12.153 (4) [P1T1]: time=0.0080
2011-05-04 05:57:12.155 (4) [P1T1]: Query executed in 0.008s
2011-05-04 05:57:12.163 (4) [P1T1]: spike: SQL->[select * from `products` where ((`accountID`="541") and (`productCode`="TSAOther")) limit 1]
2011-05-04 05:57:12.173 (4) [P1T1]: time=0.01
2011-05-04 05:57:12.175 (4) [P1T1]: Query executed in 0.010s
2011-05-04 05:57:12.205 (4) [P1T1]:   Record Loaded
2011-05-04 05:57:12.207 (3) [P1T1]: Conversions::needCatchAllProductMade completed (false): 0.52s
2011-05-04 05:57:12.209 (4) [P1T1]: spike: SQL->[ select name  from `tmp`.`conversionLog` where productID is null and name = 'TSACounted' limit 1]
2011-05-04 05:57:12.217 (4) [P1T1]: time=0.0080
2011-05-04 05:57:12.217 (4) [P1T1]: Query executed in 0.008s
2011-05-04 05:57:12.219 (3) [P1T1]: Conversions::needTSACountedProductMade completed (false): 0.10s
2011-05-04 05:57:12.221 (4) [P1T1]: spike: SQL->[ select name  from `tmp`.`conversionLog` where productID is null and name = 'TSAUncounted' limit 1]
2011-05-04 05:57:12.227 (4) [P1T1]: time=0.0080
2011-05-04 05:57:12.229 (4) [P1T1]: Query executed in 0.008s
2011-05-04 05:57:12.229 (3) [P1T1]: Conversions::needTSAUncountedProductMade completed (false): 0.10s
2011-05-04 05:57:12.229 (3) [P1T1]: Conversions::createMissing completed: 0.108s
2011-05-04 05:57:12.231 (4) [P1T1]: spike: SQL->[update `tmp`.`conversionLog` cst  join `st-tracker`.products p on (p.productCode = cst.name and p.accountID = 541)  set cst.productID = p.id  where cst.productID is null ]
2011-05-04 05:57:12.239 (4) [P1T1]: time=0.0080
2011-05-04 05:57:12.241 (4) [P1T1]: Query executed in 0.008s
2011-05-04 05:57:12.241 (3) [P1T1]: Conversions::update completed: 0.10s
2011-05-04 05:57:12.243 (2) [P1T1]: resolveConversions completed: 0.140s
2011-05-04 05:57:12.255 (4) [P1T1]: spike: SQL->[create temporary table `tmp`.waypointSources (rowNum int, sourceID int, primary key(rowNum,sourceID)) ENGINE=MyISAM DEFAULT CHARSET=utf8 as  select wpl.rowNum, s.id as sourceID  from `tmp`.`waypointLog` wpl  join `st-tracker`.sources s use index (`playback`)  on (s.accountID=541 and s.altkey=wpl.sourcekey)  where wpl.isOrganic = 0 and wpl.sourceID is null and wpl.sourceKey <> '' and wpl.sourceKey is not null  and if (wpl.matchType is null, 1, (wpl.matchType = s.searchEngineMatchType))  and if (wpl.distributionID is null or wpl.distributionID <> 0, 1, (wpl.distributionID = s.distributionID))  group by rowNum,s.id  having max( if(s.altkey=wpl.sourceKey,32,0) + if(s.keyword=wpl.sourcekey,16,0) +  if(s.searchEngineStatus='ok',8,0) + if(s.keyword=wpl.keyword,4,0) +  if(s.searchEngineAccountID>0,2,0)+if(s.siteID=wpl.siteID,1,0) ) order by null ]
2011-05-04 05:57:12.271 (4) [P1T1]: time=0.016
2011-05-04 05:57:12.273 (4) [P1T1]: Query executed in 0.016s
2011-05-04 05:57:12.273 (3) [P1T1]: WaypointSources::resolveSources create waypointSources complete: 0.18s
2011-05-04 05:57:12.273 (4) [P1T1]: spike: SQL->[start transaction]
2011-05-04 05:57:12.479 (4) [P1T1]: time=0.206
2011-05-04 05:57:12.479 (4) [P1T1]: Query executed in 0.206s
2011-05-04 05:57:12.481 (4) [P1T1]: spike: SQL->[update `tmp`.`waypointLog` wpl  join `tmp`.waypointSources wps on (wpl.rowNum=wps.rowNum)  join `st-tracker`.sources s on (s.id = wps.sourceID)  set wpl.sourceID=wps.sourceID, wpl.cpc = if (wpl.cpc is null, s.cost, wpl.cpc)]
2011-05-04 05:57:12.609 (4) [P1T1]: time=0.128
2011-05-04 05:57:12.611 (4) [P1T1]: Query executed in 0.128s
2011-05-04 05:57:12.611 (4) [P1T1]: spike: SQL->[commit]
2011-05-04 05:57:12.617 (4) [P1T1]: time=0.0040
2011-05-04 05:57:12.617 (4) [P1T1]: Query executed in 0.004s
2011-05-04 05:57:12.617 (3) [P1T1]: WaypointSources::resolveSources update waypointLog complete: 0.344s
2011-05-04 05:57:12.625 (4) [P1T1]: spike: SQL->[select @@max_error_count]
2011-05-04 05:57:12.633 (4) [P1T1]: time=0.0080
2011-05-04 05:57:12.633 (4) [P1T1]: Query executed in 0.008s
2011-05-04 05:57:12.635 (4) [P1T1]: spike: SQL->[set SESSION max_error_count=0]
2011-05-04 05:57:12.641 (4) [P1T1]: time=0.0060
2011-05-04 05:57:12.641 (4) [P1T1]: Query executed in 0.006s
2011-05-04 05:57:12.641 (4) [P1T1]: spike: SQL->[insert ignore into `st-tracker`.sources(`accountID`, `altKey`, `siteID`, `distributionID`, `campaignID`, `description`, `type`, `keyword`, `landingURL`, `searchEngineMatchType`, `lastChecked`) select 541,  wpl.sourceKey,  wpl.siteID,  if (wpl.distributionID is not null and wpl.distributionID != 0, wpl.distributionID , 34), 13,  if ((wpl.keyword is not null and wpl.keyword <> ''), concat('Keyword "', wpl.keyword, '"'), concat('Unknown Tag "', wpl.sourceKey, '"')),  if ((wpl.keyword is not null and wpl.keyword <> ''), 1, 2),  wpl.keyword,  '' as landingURL,  if (wpl.matchType is null, 'unknown', wpl.matchType) as searchEngineMatchType, NOW()  from `tmp`.`waypointLog` wpl  where wpl.sourceID is null and wpl.sourceKey is not null and wpl.sourceKey <> '' and wpl.isOrganic = 0  group by wpl.sourceKey, wpl.siteID]
2011-05-04 05:57:12.653 (4) [P1T1]: time=0.012
2011-05-04 05:57:12.653 (4) [P1T1]: Query executed in 0.012s
2011-05-04 05:57:12.655 (4) [P1T1]: spike: SQL->[set SESSION max_error_count=64]
2011-05-04 05:57:12.661 (4) [P1T1]: time=0.0080
2011-05-04 05:57:12.661 (4) [P1T1]: Query executed in 0.008s
2011-05-04 05:57:12.661 (3) [P1T1]: WaypointSources::createRefcdSources insert into st-tracker.sources complete: 0.44s
2011-05-04 05:57:12.663 (4) [P1T1]: spike: SQL->[create temporary table `tmp`.waypointRefcdSources (rowNum int, sourceID int, primary key(rowNum,sourceID)) ENGINE=MyISAM DEFAULT CHARSET=utf8 as  select wpl.rowNum, s.id as sourceID  from `tmp`.`waypointLog` wpl  join `st-tracker`.sources s use index (`playback`)  on (s.accountID=541 and s.altkey=wpl.sourcekey)  where wpl.sourceKey is not null and wpl.sourceKey <> '' and wpl.sourceID is null and wpl.isOrganic = 0  group by rowNum,s.id  having max( if(s.altkey=wpl.sourceKey,32,0) + if(s.keyword=wpl.sourcekey,16,0) +  if(s.searchEngineStatus='ok',8,0) + if(s.keyword=wpl.keyword,4,0) +  if(s.searchEngineAccountID>0,2,0)+if(s.siteID=wpl.siteID,1,0) ) order by null ]
2011-05-04 05:57:12.675 (4) [P1T1]: time=0.012
2011-05-04 05:57:12.675 (4) [P1T1]: Query executed in 0.012s
2011-05-04 05:57:12.677 (3) [P1T1]: WaypointSources::createRefcdSources create waypointRefcdSources complete: 0.16s
2011-05-04 05:57:12.677 (4) [P1T1]: spike: SQL->[start transaction]
2011-05-04 05:57:12.893 (4) [P1T1]: time=0.214
2011-05-04 05:57:12.893 (4) [P1T1]: Query executed in 0.214s
2011-05-04 05:57:12.893 (4) [P1T1]: spike: SQL->[update `tmp`.`waypointLog` wpl  join `tmp`.waypointRefcdSources wps on (wpl.rowNum=wps.rowNum)  set wpl.sourceID=wps.sourceID]
2011-05-04 05:57:12.901 (4) [P1T1]: time=0.0080
2011-05-04 05:57:12.903 (4) [P1T1]: Query executed in 0.008s
2011-05-04 05:57:12.903 (4) [P1T1]: spike: SQL->[commit]
2011-05-04 05:57:12.909 (4) [P1T1]: time=0.0040
2011-05-04 05:57:12.909 (4) [P1T1]: Query executed in 0.004s
2011-05-04 05:57:12.909 (3) [P1T1]: WaypointSources::createRefcdSources update waypointLog complete: 0.232s
2011-05-04 05:57:12.911 (3) [P1T1]: WaypointSources::createRefcdSources complete: 0.294s
2011-05-04 05:57:12.911 (4) [P1T1]: spike: SQL->[start transaction]
2011-05-04 05:57:13.039 (4) [P1T1]: time=0.126
2011-05-04 05:57:13.039 (4) [P1T1]: Query executed in 0.126s
2011-05-04 05:57:13.039 (4) [P1T1]: spike: SQL->[update `tmp`.`waypointLog` wpl  join `st-tracker`.sources s on (  s.accountID = 541 and  s.siteID = wpl.siteID and  s.type = 2 and  s.sourceURL = wpl.baseUltimateRefererURL )  set wpl.sourceID = s.id  where wpl.sourceID is null and wpl.isOrganic = 0 and  wpl.internalURLPatternMatch = 0 and wpl.baseUltimateRefererURL <> '' and wpl.sourceKey = '']
2011-05-04 05:57:13.053 (4) [P1T1]: time=0.014
2011-05-04 05:57:13.053 (4) [P1T1]: Query executed in 0.014s
2011-05-04 05:57:13.055 (4) [P1T1]: spike: SQL->[commit]
2011-05-04 05:57:13.061 (4) [P1T1]: time=0.0060
2011-05-04 05:57:13.061 (4) [P1T1]: Query executed in 0.006s
2011-05-04 05:57:13.061 (3) [P1T1]: WaypointSources::resolveLinkSources update waypointLog(1) complete: 0.150s
2011-05-04 05:57:13.063 (4) [P1T1]: spike: SQL->[select @@max_error_count]
2011-05-04 05:57:13.069 (4) [P1T1]: time=0.0060
2011-05-04 05:57:13.071 (4) [P1T1]: Query executed in 0.006s
2011-05-04 05:57:13.071 (4) [P1T1]: spike: SQL->[set SESSION max_error_count=0]
2011-05-04 05:57:13.647 (4) [P1T1]: time=0.57
2011-05-04 05:57:13.649 (4) [P1T1]: Query executed in 0.570s
2011-05-04 05:57:13.649 (4) [P1T1]: spike: SQL->[insert ignore into `st-tracker`.sources  (`accountID`, `siteID`, `distributionID`, `campaignID`, `description`, `sourceURL`, `type`, `landingURL`, `lastChecked`)  select 541, wpl.siteID, 126, 15,  wpl.baseUltimateRefererURL,  wpl.baseUltimateRefererURL, 2, '', NOW()  from `tmp`.`waypointLog` wpl  where wpl.sourceID is null and wpl.isOrganic = 0 and  wpl.internalURLPatternMatch = 0 and wpl.baseUltimateRefererURL <> '' and wpl.sourceKey = ''  group by wpl.baseUltimateRefererURL, wpl.siteID]
2011-05-04 05:57:13.661 (4) [P1T1]: time=0.012
2011-05-04 05:57:13.661 (4) [P1T1]: Query executed in 0.012s
2011-05-04 05:57:13.663 (4) [P1T1]: spike: SQL->[set SESSION max_error_count=64]
2011-05-04 05:57:13.667 (4) [P1T1]: time=0.0040
2011-05-04 05:57:13.669 (4) [P1T1]: Query executed in 0.004s
2011-05-04 05:57:13.669 (3) [P1T1]: WaypointSources::createLinkSources complete: 0.608s
2011-05-04 05:57:13.669 (4) [P1T1]: spike: SQL->[start transaction]
2011-05-04 05:57:13.675 (4) [P1T1]: time=0.0060
2011-05-04 05:57:13.675 (4) [P1T1]: Query executed in 0.006s
2011-05-04 05:57:13.675 (4) [P1T1]: spike: SQL->[update `tmp`.`waypointLog` wpl  join `st-tracker`.sources s on (  s.accountID = 541 and  s.siteID = wpl.siteID and  s.type = 2 and  s.sourceURL = wpl.baseUltimateRefererURL )  set wpl.sourceID = s.id  where wpl.sourceID is null and wpl.isOrganic = 0 and  wpl.internalURLPatternMatch = 0 and wpl.baseUltimateRefererURL <> '' and wpl.sourceKey = '']
2011-05-04 05:57:13.686 (4) [P1T1]: time=0.01
2011-05-04 05:57:13.687 (4) [P1T1]: Query executed in 0.010s
2011-05-04 05:57:13.687 (4) [P1T1]: spike: SQL->[commit]
2011-05-04 05:57:13.693 (4) [P1T1]: time=0.0060
2011-05-04 05:57:13.694 (4) [P1T1]: Query executed in 0.006s
2011-05-04 05:57:13.695 (3) [P1T1]: WaypointSources::resolveLinkSources update waypointLog(2) complete: 0.634s
2011-05-04 05:57:13.695 (3) [P1T1]: WaypointSources::resolveLinkSources complete: 0.784s
2011-05-04 05:57:13.695 (3) [P1T1]: WaypointSources::resolveSources complete: 1.440s
2011-05-04 05:57:13.695 (2) [P1T1]: resolveWaypointSources completed: 1.452s
2011-05-04 05:57:13.701 (4) [P1T1]: spike: SQL->[update `tmp`.`waypointLog` wpl join `spike_v2`.`userAgent` ua on (ua.hash = wpl.userAgentHash and ua.description = wpl.userAgent)  set wpl.userAgentID = ua.id where wpl.userAgentID = 0]
2011-05-04 05:57:13.711 (4) [P1T1]: time=0.01
2011-05-04 05:57:13.711 (4) [P1T1]: Query executed in 0.010s
2011-05-04 05:57:13.713 (3) [P1T1]: UserAgent::makeUserAgentTable: 0.14s
2011-05-04 05:57:13.713 (4) [P1T1]: spike: SQL->[select SQL_NO_CACHE count(wpl.rowNum) as cnt from `tmp`.`waypointLog` wpl where wpl.userAgentId = 0]
2011-05-04 05:57:13.721 (4) [P1T1]: time=0.0060
2011-05-04 05:57:13.721 (4) [P1T1]: Query executed in 0.006s
2011-05-04 05:57:13.723 (4) [P1T1]: spike: SQL->[update `tmp`.`waypointLog` wpl join `spike_v2`.`userAgent` ua on (ua.hash = wpl.userAgentHash and ua.description = wpl.userAgent)  set wpl.userAgentID = ua.id where wpl.userAgentID = 0]
2011-05-04 05:57:13.730 (4) [P1T1]: time=0.0060
2011-05-04 05:57:13.731 (4) [P1T1]: Query executed in 0.006s
2011-05-04 05:57:13.731 (3) [P1T1]: UserAgent::assignNewUserAgents: 0.8s
2011-05-04 05:57:13.731 (2) [P1T1]: resolveUserAgents completed: 0.36s
2011-05-04 05:57:13.735 (4) [P1T1]: spike: SQL->[update `tmp`.`waypointLog` wpl join tsacommon.searchEngineAccounts sea on (sea.accountID=541)  join `st-tracker`.searchEngineListings sel on (sel.altKey = wpl.creativeKey and sel.searchEngineAccountID = sea.id)  set wpl.searchEngineListingID = sel.id]
2011-05-04 05:57:13.747 (4) [P1T1]: time=0.01
2011-05-04 05:57:13.747 (4) [P1T1]: Query executed in 0.010s
2011-05-04 05:57:13.747 (2) [P1T1]: resolveCreatives completed: 0.16s
2011-05-04 05:57:13.762 (3) [P1T1]: Temp Processing File: /tmp/pixel_log_sources62060csv
2011-05-04 05:57:13.763 (3) [P1T1]: Temp Processing File: /tmp/pixel_log_debounce62061csv
2011-05-04 05:57:13.765 (4) [P1T1]: spike: SQL->[create temporary table `tmp`.`firstWaypointLog` (rowNum int not null primary key) ENGINE=MyISAM DEFAULT CHARSET=utf8 as select min(rowNum) as rowNum from `tmp`.`waypointLog` group by visitorKey, sessionKey order by null]
2011-05-04 05:57:13.779 (4) [P1T1]: time=0.012
2011-05-04 05:57:13.779 (4) [P1T1]: Query executed in 0.012s
2011-05-04 05:57:13.779 (3) [P1T1]: SessionSources::makeFirstWaypointLog: 0.14s
2011-05-04 05:57:13.781 (4) [P1T1]: spike: SQL->[create temporary table `tmp`.pastHits (id integer auto_increment primary key,  wplRowNum integer not null,  unixTimestamp INTEGER,  microseconds integer,  rowNum integer,  key wplRowNum(wplRowNum))  ENGINE=MyISAM DEFAULT CHARSET=utf8 as  select wpl.rowNum as wplRowNum,  wp.unixTimestamp as unixTimestamp,  wp.microseconds as microseconds,  wp.rowNum as rowNum  from `tmp`.`waypointLog` wpl  join `tmp`.`firstWaypointLog` fwp on (fwp.rowNum = wpl.rowNum)  join `spike_v2`.`waypointLog` wp use index (`previousSession`)  on (wp.siteID = wpl.siteID and wp.visitorKeyHash = wpl.visitorKeyHash and wp.sessionKeyHash = wpl.sessionKeyHash and  wp.unixTimestamp > (wpl.unixTimestamp - 172800) and wp.unixTimestamp <= wpl.unixTimestamp and  wp.visitorKey = wpl.visitorKey and wp.sessionKey = wpl.sessionKey and wp.isNewSession = 1)  order by wp.unixTimestamp, wp.microseconds, wp.rowNum]
2011-05-04 05:57:13.803 (4) [P1T1]: time=0.022
2011-05-04 05:57:13.803 (4) [P1T1]: Query executed in 0.022s
2011-05-04 05:57:13.803 (4) [P1T1]: spike: SQL->[create temporary table `tmp`.lastPastHits as select max(id) as pastHitsID from `tmp`.pastHits group by wplRowNum order by null]
2011-05-04 05:57:13.815 (4) [P1T1]: time=0.012
2011-05-04 05:57:13.815 (4) [P1T1]: Query executed in 0.012s
2011-05-04 05:57:13.815 (4) [P1T1]: spike: SQL->[create temporary table `tmp`.`sessionSource` (rowNum int not null primary key,  isNew tinyint(1) not null default 0,  sourceID int default null,  parentUnixTimestamp INTEGER default null,  parentMicroseconds int(6) default null,  parentRowNum int default null  ) ENGINE=MyISAM DEFAULT CHARSET=utf8 as  select wpl.rowNum,  if (wpl.internalURLPatternMatch = 0, 1, 0) as isNew,  if (wpl.internalURLPatternMatch = 1, wp.unixTimestamp, null) as parentUnixTimestamp,  if (wpl.internalURLPatternMatch = 1, wp.microseconds, null) as parentMicroseconds,  if (wpl.internalURLPatternMatch = 1, wp.rowNum, null) as parentRowNum  from `tmp`.`waypointLog` wpl  join `tmp`.pastHits wp on (wp.wplRowNum = wpl.rowNum)  join `tmp`.lastPastHits lpht on (lpht.pastHitsID = wp.id)]
2011-05-04 05:57:13.829 (4) [P1T1]: time=0.014
2011-05-04 05:57:13.831 (4) [P1T1]: Query executed in 0.014s
2011-05-04 05:57:13.831 (3) [P1T1]: SessionSources::makeSessionSourceTable: 0.52s
2011-05-04 05:57:13.831 (4) [P1T1]: spike: SQL->[select @@max_error_count]
2011-05-04 05:57:13.837 (4) [P1T1]: time=0.0060
2011-05-04 05:57:13.839 (4) [P1T1]: Query executed in 0.006s
2011-05-04 05:57:13.839 (4) [P1T1]: spike: SQL->[set SESSION max_error_count=0]
2011-05-04 05:57:13.845 (4) [P1T1]: time=0.0060
2011-05-04 05:57:13.845 (4) [P1T1]: Query executed in 0.006s
2011-05-04 05:57:13.847 (4) [P1T1]: spike: SQL->[drop temporary table if exists `tmp`.`sessionSourceRowsSnapshot`]
2011-05-04 05:57:13.851 (4) [P1T1]: time=0.0040
2011-05-04 05:57:13.855 (4) [P1T1]: Query executed in 0.004s
2011-05-04 05:57:13.855 (4) [P1T1]: spike: SQL->[set SESSION max_error_count=64]
2011-05-04 05:57:13.859 (4) [P1T1]: time=0.0040
2011-05-04 05:57:13.861 (4) [P1T1]: Query executed in 0.004s
2011-05-04 05:57:13.861 (4) [P1T1]: spike: SQL->[create temporary table `tmp`.`sessionSourceRowsSnapshot` (rowNum int not null primary key) ENGINE=MyISAM DEFAULT CHARSET=utf8 as  select rowNum from `tmp`.`sessionSource`]
2011-05-04 05:57:13.882 (4) [P1T1]: time=0.02
2011-05-04 05:57:13.883 (4) [P1T1]: Query executed in 0.020s
2011-05-04 05:57:13.883 (3) [P1T1]: SessionSources::makeSessionSourceRowsSnapshot: 0.52s
2011-05-04 05:57:13.885 (4) [P1T1]: spike: SQL->[insert into `tmp`.`sessionSource` (rowNum, isNew, sourceID)  select wpl.rowNum,  1 as isNew,  s.id as sourceID  from `tmp`.`waypointLog` wpl  join `tmp`.`firstWaypointLog` fwp on (fwp.rowNum = wpl.rowNum)  left join `tmp`.`sessionSourceRowsSnapshot` tss on (tss.rowNum = fwp.rowNum)  join `st-tracker`.sources s on  (s.accountID = 541 and  s.siteID = wpl.siteID and  s.distributionID = 127 and  s.campaignID = 16 and  s.type = 2 and  s.description = wpl.baseUltimateRefererURL and  s.sourceURL = wpl.baseUltimateRefererURL and  s.landingURL = '')  where (tss.rowNum is null and wpl.internalURLPatternMatch = 1) ]
2011-05-04 05:57:13.953 (4) [P1T1]: time=0.064
2011-05-04 05:57:13.957 (4) [P1T1]: Query executed in 0.064s
2011-05-04 05:57:13.965 (3) [P1T1]: SessionSources::resolveInternalLinkSources insert pre-existing internal sources complete: 0.82s
2011-05-04 05:57:13.969 (4) [P1T1]: spike: SQL->[select @@max_error_count]
2011-05-04 05:57:14.019 (4) [P1T1]: time=0.048
2011-05-04 05:57:14.019 (4) [P1T1]: Query executed in 0.048s
2011-05-04 05:57:14.019 (4) [P1T1]: spike: SQL->[set SESSION max_error_count=0]
2011-05-04 05:57:14.025 (4) [P1T1]: time=0.0060
2011-05-04 05:57:14.025 (4) [P1T1]: Query executed in 0.006s
2011-05-04 05:57:14.027 (4) [P1T1]: spike: SQL->[drop temporary table if exists `tmp`.`sessionSourceRowsSnapshot`]
2011-05-04 05:57:14.031 (4) [P1T1]: time=0.0060
2011-05-04 05:57:14.033 (4) [P1T1]: Query executed in 0.006s
2011-05-04 05:57:14.033 (4) [P1T1]: spike: SQL->[set SESSION max_error_count=64]
2011-05-04 05:57:14.049 (4) [P1T1]: time=0.012
2011-05-04 05:57:14.057 (4) [P1T1]: Query executed in 0.012s
2011-05-04 05:57:14.061 (4) [P1T1]: spike: SQL->[create temporary table `tmp`.`sessionSourceRowsSnapshot` (rowNum int not null primary key) ENGINE=MyISAM DEFAULT CHARSET=utf8 as  select rowNum from `tmp`.`sessionSource`]
2011-05-04 05:57:14.117 (4) [P1T1]: time=0.056
2011-05-04 05:57:14.118 (4) [P1T1]: Query executed in 0.056s
2011-05-04 05:57:14.119 (3) [P1T1]: SessionSources::makeSessionSourceRowsSnapshot: 0.150s
2011-05-04 05:57:14.119 (4) [P1T1]: spike: SQL->[select @@max_error_count]
2011-05-04 05:57:14.127 (4) [P1T1]: time=0.0080
2011-05-04 05:57:14.127 (4) [P1T1]: Query executed in 0.008s
2011-05-04 05:57:14.127 (4) [P1T1]: spike: SQL->[set SESSION max_error_count=0]
2011-05-04 05:57:14.133 (4) [P1T1]: time=0.0060
2011-05-04 05:57:14.134 (4) [P1T1]: Query executed in 0.006s
2011-05-04 05:57:14.135 (4) [P1T1]: spike: SQL->[insert ignore into `st-tracker`.sources  (`accountID`, `siteID`, `distributionID`, `campaignID`, `description`, `sourceURL`, `type`, `landingURL`, `lastChecked`)  select 541 as `accountID`,  wpl.siteID as `siteID`, 127 as `distributionID`, 16 as `campaignID`,  wpl.baseUltimateRefererURL as `description`,  wpl.baseUltimateRefererURL as `sourceURL`, 2 as `type`, '' as `landingURL`,  NOW() as `lastChecked`  from `tmp`.`waypointLog` wpl  join `tmp`.`firstWaypointLog` fwp on (fwp.rowNum = wpl.rowNum)  left join `tmp`.`sessionSourceRowsSnapshot` tss on (tss.rowNum = fwp.rowNum)  where (tss.rowNum is null and wpl.internalURLPatternMatch = 1) group by wpl.baseUltimateRefererURL]
2011-05-04 05:57:14.142 (4) [P1T1]: time=0.0080
2011-05-04 05:57:14.143 (4) [P1T1]: Query executed in 0.008s
2011-05-04 05:57:14.143 (4) [P1T1]: spike: SQL->[set SESSION max_error_count=64]
2011-05-04 05:57:14.149 (4) [P1T1]: time=0.0060
2011-05-04 05:57:14.151 (4) [P1T1]: Query executed in 0.006s
2011-05-04 05:57:14.151 (3) [P1T1]: SessionSources::createInternalLinkSources complete: 0.32s
2011-05-04 05:57:14.151 (4) [P1T1]: spike: SQL->[insert into `tmp`.`sessionSource` (rowNum, isNew, sourceID)  select wpl.rowNum,  1 as isNew,  s.id as sourceID  from `tmp`.`waypointLog` wpl  join `tmp`.`firstWaypointLog` fwp on (fwp.rowNum = wpl.rowNum)  left join `tmp`.`sessionSourceRowsSnapshot` tss on (tss.rowNum = fwp.rowNum)  join `st-tracker`.sources s on  (s.accountID = 541 and  s.siteID = wpl.siteID and  s.distributionID = 127 and  s.campaignID = 16 and  s.type = 2 and  s.description = wpl.baseUltimateRefererURL and  s.sourceURL = wpl.baseUltimateRefererURL and  s.landingURL = '')  where (tss.rowNum is null and wpl.internalURLPatternMatch = 1) ]
2011-05-04 05:57:14.159 (4) [P1T1]: time=0.0080
2011-05-04 05:57:14.161 (4) [P1T1]: Query executed in 0.008s
2011-05-04 05:57:14.161 (3) [P1T1]: SessionSources::resolveInternalLinkSources insert new internal sources complete: 0.10s
2011-05-04 05:57:14.161 (3) [P1T1]: SessionSources::resolveInternalLinkSources complete: 0.330s
2011-05-04 05:57:14.161 (4) [P1T1]: spike: SQL->[select @@max_error_count]
2011-05-04 05:57:14.169 (4) [P1T1]: time=0.0080
2011-05-04 05:57:14.169 (4) [P1T1]: Query executed in 0.008s
2011-05-04 05:57:14.169 (4) [P1T1]: spike: SQL->[set SESSION max_error_count=0]
2011-05-04 05:57:14.175 (4) [P1T1]: time=0.0060
2011-05-04 05:57:14.175 (4) [P1T1]: Query executed in 0.006s
2011-05-04 05:57:14.175 (4) [P1T1]: spike: SQL->[drop temporary table if exists `tmp`.`sessionSourceRowsSnapshot`]
2011-05-04 05:57:14.183 (4) [P1T1]: time=0.0080
2011-05-04 05:57:14.185 (4) [P1T1]: Query executed in 0.008s
2011-05-04 05:57:14.185 (4) [P1T1]: spike: SQL->[set SESSION max_error_count=64]
2011-05-04 05:57:14.191 (4) [P1T1]: time=0.0060
2011-05-04 05:57:14.191 (4) [P1T1]: Query executed in 0.006s
2011-05-04 05:57:14.191 (4) [P1T1]: spike: SQL->[create temporary table `tmp`.`sessionSourceRowsSnapshot` (rowNum int not null primary key) ENGINE=MyISAM DEFAULT CHARSET=utf8 as  select rowNum from `tmp`.`sessionSource`]
2011-05-04 05:57:14.201 (4) [P1T1]: time=0.01
2011-05-04 05:57:14.203 (4) [P1T1]: Query executed in 0.010s
2011-05-04 05:57:14.203 (3) [P1T1]: SessionSources::makeSessionSourceRowsSnapshot: 0.42s
2011-05-04 05:57:14.203 (4) [P1T1]: spike: SQL->[insert into `tmp`.`sessionSource` (rowNum, isNew, sourceID)  select wpl.rowNum,  1 as isNew,  null as sourceID  from `tmp`.`waypointLog` wpl  join `tmp`.`firstWaypointLog` fwp on (fwp.rowNum = wpl.rowNum)  left join `tmp`.`sessionSourceRowsSnapshot` tss on (tss.rowNum = fwp.rowNum)  where (tss.rowNum is null)]
2011-05-04 05:57:14.211 (4) [P1T1]: time=0.0080
2011-05-04 05:57:14.211 (4) [P1T1]: Query executed in 0.008s
2011-05-04 05:57:14.211 (3) [P1T1]: SessionSources::updateRemainingFirstSessions update waypointLog.sourceID: 0.8s
2011-05-04 05:57:14.213 (4) [P1T1]: spike: SQL->[create temporary table `tmp`.sortedWaypointLog  (rowNum int) ENGINE=MyISAM DEFAULT CHARSET=utf8 as  SELECT wpl.rowNum FROM `tmp`.`waypointLog` wpl order by wpl.visitorKey asc, wpl.sessionKey, wpl.rowNum asc]
2011-05-04 05:57:14.225 (4) [P1T1]: time=0.011
2011-05-04 05:57:14.225 (4) [P1T1]: Query executed in 0.011s
2011-05-04 05:57:14.226 (3) [P1T1]: SessionSources::remainingHits create sortedWaypointLog: 0.13s
2011-05-04 05:57:14.227 (4) [P1T1]: spike: SQL->[create temporary table `tmp`.remainingHits  (visitorKey varchar(128) not null, sessionKey varchar(128) not null, key `sessionKey`(`sessionKey`)) ENGINE=MyISAM DEFAULT CHARSET=utf8 as  SELECT wpl.visitorKey, wpl.sessionKey FROM `tmp`.`waypointLog` wpl  left join `tmp`.`sessionSource` tss on (tss.rowNum = wpl.rowNum and tss.isNew = 1)  where tss.rowNum is null group by wpl.visitorKey, wpl.sessionKey]
2011-05-04 05:57:14.241 (4) [P1T1]: time=0.014
2011-05-04 05:57:14.243 (4) [P1T1]: Query executed in 0.014s
2011-05-04 05:57:14.243 (3) [P1T1]: SessionSources::remainingHits create remainingHits: 0.16s
2011-05-04 05:57:14.243 (4) [P1T1]: spike: SQL->[select SQL_NO_CACHE  wpl.siteID,  wpl.unixTimestamp,  wpl.microseconds,  wpl.rowNum,  tss.parentUnixTimestamp,  tss.parentMicroseconds,  tss.parentRowNum,  if (tss.sourceID is null, wpl.sourceID, tss.sourceID) as sourceID,  wpl.searchEngineListingID as searchEngineListingID,  wpl.keyword,  wpl.internalURLPatternMatch,  wpl.visitorKey, wpl.sessionKey, wpl.visitorKeyHash, wpl.sessionKeyHash,  wpl.waypointID,  tss.isNew  from `tmp`.sortedWaypointLog swh  join `tmp`.`waypointLog` wpl on (wpl.rowNum = swh.rowNum)  join `tmp`.remainingHits rh on (wpl.visitorKey = rh.visitorKey and wpl.sessionKey = rh.sessionKey)  left join `tmp`.`sessionSource` tss on (tss.rowNum = wpl.rowNum) ]
2011-05-04 05:57:14.269 (4) [P1T1]: time=0.026
2011-05-04 05:57:14.271 (4) [P1T1]: Query executed in 0.026s
2011-05-04 05:57:14.271 (3) [P1T1]: SessionSources::remainingHits select sorted remaining rows: 0.28s
2011-05-04 05:57:14.271 (3) [P1T1]: Done Preprocessing Source Sessions: 0.524s
2011-05-04 05:57:14.301 (3) [P1T1]: Batch Session Assignments Created: 0.28s
2011-05-04 05:57:14.301 (3) [P1T1]: SessionSources::loadSessionSourceData Loading 1 Rows
2011-05-04 05:57:14.303 (4) [P1T1]: spike: SQL->[alter table `tmp`.`sessionSource` disable keys]
2011-05-04 05:57:14.349 (4) [P1T1]: time=0.046
2011-05-04 05:57:14.350 (4) [P1T1]: Query executed in 0.046s
2011-05-04 05:57:14.351 (4) [P1T1]: spike: SQL->[load data local infile '/tmp/pixel_log_sources62060csv' replace into table `tmp`.`sessionSource` fields terminated by ',' optionally enclosed by '"' escaped by ''  (`rowNum`, `isNew`, @sourceID, @parentUnixTimestamp, @parentMicroseconds, @parentRowNum)  set `sourceID` = if (@sourceID is null or @sourceID = '', null, @sourceID), `parentUnixTimestamp` = if (@parentUnixTimestamp is null or @parentUnixTimestamp = '', null, @parentUnixTimestamp), `parentMicroseconds` = if (@parentMicroseconds is null or @parentMicroseconds = '', null, @parentMicroseconds), `parentRowNum` = if (@parentRowNum is null or @parentRowNum = '', null, @parentRowNum)]
2011-05-04 05:57:14.383 (4) [P1T1]: time=0.03
2011-05-04 05:57:14.383 (4) [P1T1]: Query executed in 0.030s
2011-05-04 05:57:14.383 (4) [P1T1]: spike: SQL->[alter table `tmp`.`sessionSource` enable keys]
2011-05-04 05:57:14.392 (4) [P1T1]: time=0.0080
2011-05-04 05:57:14.393 (4) [P1T1]: Query executed in 0.008s
2011-05-04 05:57:14.395 (3) [P1T1]: SessionSources::loadSessionSourceData Complete: 0.94s
2011-05-04 05:57:14.395 (3) [P1T1]: SessionSources::loadDebounceData Loading 0 Rows
2011-05-04 05:57:14.395 (4) [P1T1]: spike: SQL->[create temporary table `tmp`.`debounce` (rowNum int not null primary key)]
2011-05-04 05:57:14.403 (4) [P1T1]: time=0.0080
2011-05-04 05:57:14.403 (4) [P1T1]: Query executed in 0.008s
2011-05-04 05:57:14.405 (4) [P1T1]: spike: SQL->[load data local infile '/tmp/pixel_log_debounce62061csv' into table `tmp`.`debounce` fields terminated by ',' optionally enclosed by '"' escaped by '' (`rowNum`)]
2011-05-04 05:57:14.417 (4) [P1T1]: time=0.01
2011-05-04 05:57:14.417 (4) [P1T1]: Query executed in 0.010s
2011-05-04 05:57:14.419 (4) [P1T1]: spike: SQL->[update `tmp`.`waypointLog` wpl  join `tmp`.`debounce` d on (wpl.rowNum = d.rowNum)  set wpl.debounced = 1]
2011-05-04 05:57:14.437 (4) [P1T1]: time=0.018
2011-05-04 05:57:14.439 (4) [P1T1]: Query executed in 0.018s
2011-05-04 05:57:14.439 (4) [P1T1]: spike: SQL->[update `tmp`.`conversionLog` wpc  join `tmp`.`debounce` d on (wpc.rowNum = d.rowNum)  set wpc.debounced = 1]
2011-05-04 05:57:14.455 (4) [P1T1]: time=0.015
2011-05-04 05:57:14.455 (4) [P1T1]: Query executed in 0.015s
2011-05-04 05:57:14.455 (3) [P1T1]: SessionSources::loadDebounceData Complete: 0.60s
2011-05-04 05:57:14.455 (2) [P1T1]: Debounce Count: 0
2011-05-04 05:57:14.465 (3) [P1T1]: Debounce Check History Count: 0 / Debounce Check History Time: 0.0s
2011-05-04 05:57:14.466 (2) [P1T1]: resolveSessions completed: 0.719s
2011-05-04 05:57:14.466 (3) [P1T1]: Starting commitToWaypointLog
2011-05-04 05:57:14.467 (4) [P1T1]: spike: SQL->[START TRANSACTION]
2011-05-04 05:57:14.482 (4) [P1T1]: time=0.014
2011-05-04 05:57:14.483 (4) [P1T1]: Query executed in 0.014s
2011-05-04 05:57:14.483 (3) [P1T1]: Start Transaction for `spike_v2` complete: 0.17s
2011-05-04 05:57:14.483 (4) [P1T1]: spike: SQL->[insert into `spike_v2`.`waypointLog` ( `rowNum`,  `unixTimestamp`,  `microseconds`,  `siteID`,  `waypointID`,  `visitorKey`,  `visitorKeyHash`,  `sessionKey`,  `sessionKeyHash`,  `userAgentID`,  `cpc`,  `distributionID`,  `campaignID`,  `keyword`,  `sourceID`,  `searchEngineListingID`,  `isNewSession`,  `refererURL`,  `ultimateRefererURL`,  `remoteAddr`,  `params`,  `sourceKey`,  `creativeKey`,  `baseUltimateRefererURL`,  `internalURLPatternMatch`,  `isOrganic`,  `backRef`,  `destination`,  `tsaTime`,  `recordedOutOfOrder`,  `parentUnixTimestamp`,  `parentMicroseconds`,  `parentRowNum`)  select  wpl.`rowNum`,  `unixTimestamp`,  `microseconds`,  wpl.siteID,  wpl.waypointID,  `visitorKey`,  `visitorKeyHash`,  `sessionKey`,  `sessionKeyHash`,  `userAgentID`,  if (`cpc` is null, 0.00, `cpc`) as `cpc`,  if (s.id is null, wpl.distributionID, s.distributionID) as distributionID,  if (s.id is null, wpl.campaignID, s.campaignID) as campaignID,  wpl.keyword as keyword,  if (tss.sourceID is null, wpl.sourceID, tss.sourceID) as sourceID,  wpl.searchEngineListingID as searchEngineListingID,  if (tss.isNew, 1, 0) as isNewSession,  `refererURL`,  `ultimateRefererURL`,  `remoteAddr`,  `params`,  `sourceKey`,  `creativeKey`,  `baseUltimateRefererURL`,  `internalURLPatternMatch`,  wpl.isOrganic as isOrganic,  `backRef`, `destination`,  `tsaTime`,  `recordedOutOfOrder`,  if (tss.parentUnixTimestamp is null, wpl.parentUnixTimestamp, tss.parentUnixTimestamp) as parentUnixTimestamp,  if (tss.parentMicroseconds is null, wpl.parentMicroseconds, tss.parentMicroseconds) as parentMicroseconds,  if (tss.parentRowNum is null, wpl.parentRowNum, tss.parentRowNum) as parentRowNum from `tmp`.`waypointLog` wpl  left join `tmp`.`sessionSource` tss on (tss.rowNum = wpl.rowNum)  left join `st-tracker`.sources s on (s.id = if (tss.sourceID is null, wpl.sourceID, tss.sourceID))  where wpl.debounced = 0]
2011-05-04 05:57:14.509 (4) [P1T1]: time=0.026
2011-05-04 05:57:14.511 (4) [P1T1]: Query executed in 0.026s
2011-05-04 05:57:14.511 (3) [P1T1]: copy to `spike_v2`.`waypointLog` complete: 0.28s
2011-05-04 05:57:14.511 (4) [P1T1]: spike: SQL->[insert into `spike_v2`.`waypointConversionLog` ( `siteID`,  `unixTimestamp`,  `microseconds`,  `rowNum`,  `convNum`,  `name`,  `count`,  `value`,  `cogs`,  `productID`)  select  `siteID`,  `unixTimestamp`,  `microseconds`,  wpc.`rowNum`,  `convNum`,  `name`,  `count`,  `value`,  `cogs`,  `productID`  from `tmp`.`conversionLog` wpc  where wpc.debounced = 0]
2011-05-04 05:57:14.529 (4) [P1T1]: time=0.018
2011-05-04 05:57:14.530 (4) [P1T1]: Query executed in 0.018s
2011-05-04 05:57:14.531 (3) [P1T1]: commitToWaypointConversionLog completed: 0.20s
2011-05-04 05:57:14.531 (4) [P1T1]: spike: SQL->[COMMIT]
2011-05-04 05:57:14.548 (4) [P1T1]: time=0.016
2011-05-04 05:57:14.549 (4) [P1T1]: Query executed in 0.016s
2011-05-04 05:57:14.550 (3) [P1T1]: commit to `spike_v2` complete: 0.19s
2011-05-04 05:57:14.551 (2) [P1T1]: commitToWaypointLog completed: 0.85s
2011-05-04 05:57:14.551 (0) [P1T1]: Processed 2 Log Lines / 2 Hits Imported / Total Time: 3.908s (1,842 lines/hour, 1,842 hits/hour)
2011-05-04 05:57:14.557 (4) [P1T1]: spike: SQL->[call `spike_v2`.`logPlaybackStatus`(560, '2011-05-04-1300-560.tsv', 'completed', 1304513830, 1304513834, 3908, 2, 0, 0, 0, 0, 0, 0, 2)]
2011-05-04 05:57:14.566 (4) [P1T1]: time=0.0090
2011-05-04 05:57:14.567 (4) [P1T1]: Query executed in 0.009s
2011-05-04 05:57:14.577 (4) [P1T1]: spike: SQL->[select * from `pixelPlaybackStatus` where (`siteID`=560) limit 1]
2011-05-04 05:57:14.585 (4) [P1T1]: time=0.0080
2011-05-04 05:57:14.586 (4) [P1T1]: Query executed in 0.008s
2011-05-04 05:57:14.586 (4) [P1T1]:   Record Not Found
2011-05-04 05:57:14.587 (1) [P1T1]: Error: Unable To Create PixelPlaybackStatus Row For SiteID: 560
2011-05-04 05:57:14.589 (4) [P1T1]: Moved Pixel Log File: [2011-05-04-1300-560.tsv] To: [/usr/local/share/tsa/logs/spike-v2/done/560/2011-05-04-1300-560.tsv]
2011-05-04 05:57:14.591 (4) [P1T1]: spike: SQL->[select @@max_error_count]
2011-05-04 05:57:14.595 (4) [P1T1]: time=0.0040
2011-05-04 05:57:14.597 (4) [P1T1]: Query executed in 0.004s
2011-05-04 05:57:14.598 (4) [P1T1]: spike: SQL->[set SESSION max_error_count=0]
2011-05-04 05:57:14.605 (4) [P1T1]: time=0.0050
2011-05-04 05:57:14.605 (4) [P1T1]: Query executed in 0.005s
2011-05-04 05:57:14.606 (4) [P1T1]: spike: SQL->[select group_concat(concat('`tmp`', '.', '`', TABLE_NAME, '`')) from `information_schema`.`TEMPORARY_TABLES` where SESSION_ID = connection_id() and TABLE_SCHEMA = 'tmp']
2011-05-04 05:57:14.615 (4) [P1T1]: time=0.0090
2011-05-04 05:57:14.615 (4) [P1T1]: Query executed in 0.009s
2011-05-04 05:57:14.617 (4) [P1T1]: spike: SQL->[drop temporary table `tmp`.`debounce`,`tmp`.`sessionSource`,`tmp`.`remainingHits`,`tmp`.`sortedWaypointLog`,`tmp`.`sessionSourceRowsSnapshot`,`tmp`.`lastPastHits`,`tmp`.`pastHits`,`tmp`.`firstWaypointLog`,`tmp`.`waypointRefcdSources`,`tmp`.`waypointSources`,`tmp`.`conversionLog`,`tmp`.`waypointLog`]
2011-05-04 05:57:14.627 (4) [P1T1]: time=0.01
2011-05-04 05:57:14.629 (4) [P1T1]: Query executed in 0.010s
2011-05-04 05:57:14.629 (4) [P1T1]: spike: SQL->[set SESSION max_error_count=64]
2011-05-04 05:57:14.635 (4) [P1T1]: time=0.0060
2011-05-04 05:57:14.635 (4) [P1T1]: Query executed in 0.006s
2011-05-04 05:57:14.635 (4) [P1T1]: spike: SQL->[select @@max_error_count]
2011-05-04 05:57:14.653 (4) [P1T1]: time=0.016
2011-05-04 05:57:14.653 (4) [P1T1]: Query executed in 0.016s
2011-05-04 05:57:14.654 (4) [P1T1]: spike: SQL->[set SESSION max_error_count=0]
2011-05-04 05:57:14.665 (4) [P1T1]: time=0.011
2011-05-04 05:57:14.668 (4) [P1T1]: Query executed in 0.011s
2011-05-04 05:57:14.670 (4) [P1T1]: spike: SQL->[select group_concat(concat('`tmp`', '.', '`', TABLE_NAME, '`')) from `information_schema`.`TEMPORARY_TABLES` where SESSION_ID = connection_id() and TABLE_SCHEMA = 'tmp']
2011-05-04 05:57:14.679 (4) [P1T1]: time=0.0080
2011-05-04 05:57:14.679 (4) [P1T1]: Query executed in 0.008s
2011-05-04 05:57:14.679 (4) [P1T1]: spike: SQL->[set SESSION max_error_count=64]
2011-05-04 05:57:14.685 (4) [P1T1]: time=0.0060
2011-05-04 05:57:14.685 (4) [P1T1]: Query executed in 0.006s
2011-05-04 05:57:14.686 (4) [P1T1]: Returning Object to Pool
2011-05-04 05:57:14.687 (4) [P1T1]: Returning Object to Pool
2011-05-04 05:57:15.087 (4) [main]: closed [jdbc:mysql://tsacommon-01-write/tsacommon?useUnicode=true&characterEncoding=UTF8&zeroDateTimeBehavior=convertToNull&dontTrackOpenResources=true&jdbcCompliantTruncation=false]
2011-05-04 05:57:15.087 (3) [main]: ------------Pixel Log Playback done
2011-05-04 05:57:15.091 (3) [Thread-0]: shutting down. reached point of no return ...
shutting down. reached point of no return ...