Details
-
Type: Bug
-
Status: Closed
-
Priority: Major
-
Resolution: Fixed
-
Affects Version/s: Daily Data Availability Phase 2
-
Fix Version/s: None
-
Component/s: Data Summarization
-
Labels:None
Description
WarehouseSummarizer runs low gran for 10 hours currently on production, starting at 4:30 Pacific. When it hits 14:30 (10 hours has passed) it summarizes all data, even data without 'success' as data availability status.
With the advent of time zone support, this will not be sufficient.
As a work-around:
The time zones we have to consider are:
America/Los_Angeles, America/New_York, and Europe/London
The difference between our earliest (Europe/London) and latest (America/Los_Angeles) time zones is 8 hours. So I propose the following (note the AccountLowGranDispatcher has been updated to use account time):
We start wh lowgran at 4:30 Europe/London time (16:30 America/Los_Angeles) and end at 14:30 America/Los_Angeles, as it does now. That will cover everything we need and give wh lowgran 2 hours to go through all its exit code (running everything that doesn't have 'success' as DA status). Jeph, you said it generally takes only 15 minutes to complete this.
Of course, this won't work if we add more accounts in time zones earlier than America/Los_Angeles or later than Europe/London, but it will work for now and won't require us pushing back the deployment date.
But the app itself needs to be timezone-aware. Some notes from Mike S.:
this process should still only run for a limited time - to avoid
crash problems like event listener did.
And will check every hour for accounts in timezone that should have finished 10
hours earlier (or whatever time we pick).