This is honestly a known issue, but also a bit forgotten.
The issue itself is written down, but we mostly work with urgent things and very seldom actually look in our database of todo-tasks.
There are at least two reaons behind this.
-
The job starts once per week. If it gets interrupted, by for example site maintenance, it does not restart until next week, and then from the beginning again.
-
We are building the GPXs for so many users that it's hard to actually manage them all in a week. Last time I checked it took 3-4 days. Ie, twice as many members and it would be impossible to do it in a week. We might have reached this.
There are solutions planned for this, and I will try to at least counter the first one today.
The solution to the first one is quite simple, restart the job every hour and only build for those that has at least 6 days old files. Then there will be a new file about every 7 days, sometimes a bit faster, sometimes a bit slower.
For the second one we do have two ideas in mind. First off, we do plan to upgrade our database servers. However, the server with all the log text is not of highest priority. It's a huge amounts of data and we are unsure if it's worth it. The alternative solution that we will probably go for first, is to make this opt-in. We do think that most members don't use the daily created GPX files, and we therefore will create a setting to enable this. Hoping that we can reduce the numbers of GPX files to build with 90%.
UPDATE: Task number one has been written and is currently under testing.
UPDATE 2: Another patch has been applied that will make the procedure a bit faster. I will now start looking into making this opt-in as well.
UPDATE 3: Instead of making it opt-in, I managed to optimize the code to run slightly more than twice as fast. We should therefore manage 5-10 more members before this is a real problem. By then we might have another database infrastructure, and therefore I will postpone any other actions. Currently running the job to create full GPX files for all members. Done about 5% in 2-3 hours. But from now on it will go twice as fast.
