Why is the queue for refreshing geocaching data unusually long?

Question

For quite some time now Project-GC has been posting the following message on the top of the Dashboard on a daily basis:

"The queue for refreshing geocache data is unusually long. The servers are working on catching up. Meanwhile our data might not be as fresh as normal."

Would it be possible for Project-GC to provide it's paying members and contributors an overview of what the technical issues are that is causing these on-going issues?  Are the issues related to aging server technology? outdated code? API issues with GroundSpeak? Other?

It would also be helpful to understand what steps Project-GC is taking to address these technical issues as people like me, may be reluctant in continuing to making cash donations to Project-GC on a yearly basis (above an beyond the annual membership fee).

I think this is a great site and I have been very complementary over the years to staff at Project-GC and I hope this question will be received well for the benefit of all paying members.

Answer 1

Besides the FAQ entry pinkunicorn referred to I can add a few other facts.

• The queue is not outdated due to outdated hardware or to little of it.
• The code is outdated in many ways, but it's not related to this. :)
• It's related to the API in the matter that it's rate limited. This isn't an issue under normal circumstances.
• We are not doing much about it since this is not normal, it will soon be over.
• It's not likely that you actually are affected by this, at least not related to finds you have made. Data that are being updated are done so in an order of magic priority, though it's unpredictable as well. Data on your hides are more likely to have a higher latency, if those finding them don't use Project-GC regularly.
• API issues does have something to do with it. Somewhat. I'll get to that.

We discovered a bug (or unexpected behaviour) in the API Project-GC uses to retrieve data a few months ago.  It was probably more of a misunderstanding than a bug, anyway, adjustments to the API was made.

This forced us to fetch ALL data about geocaches and their logs from Geocaching.com. Normally we fetch incremental changes instead (added, deleted, updated logs). This caused a huge spike in our queue back then.

It also meant that most data was updated in a shorter time span than usual. Geocaches that hasn't been refreshed for a long time will get updated again after a certain time. This happen to be a lot of geocaches right now since most were update "at the same time" a while back. The bump this time is much smaller than last time and with time it will be even more spread out again. I would expect a new bump in a few months, though I would guess the alert will only exist for a week or two.

For the current warning, I expect it to be gone sometime this week. My guess is within 50 hours.

Bottom line is that there is no real issue here. Almost all data is updated as you would expect. You are raising a question about a warning, not an issue in itself. At least you are not mentioning that actually miss any data.

I hope my tone doesn't sound harsh, it's not meant to be. English isn't my primarily language and I have a fairly hard time explaining this in my native language, even harder in English then.

UPDATE: A screenshot that shows the queue over time, and the effect of the bumps, http://storage1.static.itmages.com/i/17/0516/h_1494960851_3831008_80a30c870c.png

Comments

Ganja, thank you very much for the detailed response.  Your tone was not harsh and I certainly understand the language differential.  I very much appreciate a better understanding of what is causing the data refresh issues.  I know you and your team work hard to provide a quality product to your members and I look forward to seeing the results of the bug that was detected and fixed with the API.

Keep up the great work and thanks for the timely response.

Regards,
RPStew

---

Added a screenshot of the queue, which shows the effect of the bumps:
http://storage1.static.itmages.com/i/17/0516/h_1494960851_3831008_80a30c870c.png

Answer 2

As if by a cosmic coincidence, an entry was added to the FAQ earlier today: http://project-gc.com/Home/FAQ#429247161 explaining at least some of the inner workings.