Keep the new UK polygon data?

Question

This week we have updated polygon data for the United Kingdom. This included updating data for Scotland which had old and very broken data before, and for Wales.

It also ended up with some changes for England. We have used the data from OpenStreetMap as we normally do. For all other countries we use the data defined as administrative borders. Their data is described here, http://wiki.openstreetmap.org/wiki/Tag:boundary%3Dadministrative.

The OSM data can be viewed online here, https://osm.wno-edv-service.de/boundaries/

We know many argue for using ceremonial borders for UK. Our tools doesn't allow that today and it would be quite a lot of work. Our old data didn't use them either, so there is no difference for this case. All other countries use the administrative borders, so it's a way more consistent with the rest of the world.

Considering we are using Metropolitan and non-metropolitan, the data we have now should be as correct as OSM has it. The future changes should then be minor, and only change if OSM changes their data (not live, it's still a manual task).

The question is, should we keep this data, or use the old data? Remember, using the old data will ruin the data of Scotland again.

We also just fixed a bug with the maps in Profile stats. If they rendered weirdly for you, hold down shift or ctrl while refreshing the page and it should fix itself.

Feel free to leave a comment about your thoughts.

Comments

Will this eventually lead to solving my older question? http://project-gc.com/qa/?qa=8922/why-cant-i-show-regions-on-the-uk-map

---

No, it actually won't. The definition of regions that GS uses does not exist in OSM at all. In fact, we can't really find any information about them at all.

Since we can't get the data, we can't render the border for them correctly, and then, we can't render the map.

I believe that UK is the only country with plenty of active geocachers that is affected by this.

---

How do you get the information for the clickable 'region' maps, which show separately when you click on the name of each region?

---

I don't think I understand the question. For the "Map Regions" page, we need polygon data for each region, ie thousands/millions of lat/lon.

Since we want to use the same regions as Geocaching.com, we need to find their definition. That doesn't exist in OSM (for all we know).

We therefore can't draw such map. The region data on the geocaches are from Geocaching.com, assigned by its users.

---

I'm meaning the maps of regions as shown here: http://i.imgur.com/xW1BlBp.jpg

As you can see, clicking each region name brings up a map of that region. Those region maps must come from somewhere. Whether that means it's possible to make a larger map of the regions, I don't know...

---

The geocaches has been assigned to counties, by polygon definitions we have. We then "guess" which region a county belongs to by statistics. Ie, if county X has 90% of the geocaches in region Y, it's likely to be inside that one.

The map you linked isn't a drawing of the region in itself, but all the counties inside it, filled with white or green.

IF the "sum" of a couple of counties actually equals the region definitions, we could in theory join the county polygons into regions. There are two issues here.
* We can't be sure the borders for the regions actually are the same. A county could exist over regions, especially since we don't know where the region definitions are from.
* Joining several polygons into one works very well if the share borders completely. That often isn't the case and there are then gaps in the middle of them. An example could be that a river between two counties. In those cases the county polygon is most likely defined wrong, but it's a fairly common case. The join region polygon will then look quite crap.

To be honest, we haven't tried this for UK. Since we don't do it anywhere else, we don't really have the tools to investigate it and see the end result. In short, it's quite a lot of work, and we don't know if the end result will be good enough to keep.

I hope I explained it thorough enough.

---

That's fine, I thought those maps might just be a manual thing. Thanks for explaining it.

---

They are automatically generated. That's why they are quite ugly. We have some ideas how to make them better though, but it requires quite a lot of work, and will most likely bring us plenty of issues if we start doing it.

---

I'm hoping that I can persuade the OSM guys to permit a tagging of "UK geocaching regions" in some manner so that you'd have boundary objects with region tags. The plan is these would be based on grouping together "UK geocaching counties" of ceremonial counties. Its a steep learning curve of what's possible and learning how to edit the OSM data safely, but its an interesting project to work on.

---

A follow-up vote, regarding Ceremonial: http://project-gc.com/qa/?qa=9832/uk-polygons-ceremonial-or-administrative

Answer 1

Given that the old data was inconsistent, I'm of the opinion that the new data should be kept. I'd prefer ceremonial counties, but that isn't what'd being asked here. Whilst the new data has a lot more 'counties', at least it's consistent now, whereas the old version wasn't.

Comments

This sums up my view exactly, especially the large number of counties.  But, if this provides a consistent and firm basis on which to move forward and develop it gets my vote

Answer 2

I'm working with OSM data guys to hopefully get a longer term solution with ceremonial counties of England as well as tiding up some anomilies in small islands off the Scottish Mainland. This data is a big step forward, once people realise the map being white where they have cached was a temporary bug, the bulk of the "OMG you broke it" wailing and gnashing of teeth should subside.

Keep this fix and I'll help you work towards a solution that works longer term too.

Comments

The 7 minor islands :
Beàrnaraigh Beag
Beàrnaraigh Mòr
Burray
Eilean na Gobhail
Lamb Holm
Sanday
South Ronaldsay

that were showing in OSM as counties have now been fixed on OSM and are no longer inaccurately admin_level 6. Isle of Lewis is under investigation, it too should be fixed soon. That will tidy up the erroneous 8 "counties" in Northern Scotland.

---

I'll try to look at the site we use in the beginning of next week to see the changes have applied. I am not sure it's in real-time. If I forget, reply with a comment to the poll and I will get an email about it.

---

https://osm.wno-edv-service.de/boundaries/ has been updated with the 7 minor islands and the larger Isle of Lewis correctly tagged as islands and not counties. I believe this is the service you use?

---

I've noted on https://osm.wno-edv-service.de/boundaries/ that a lot of the "counties" in England have admin_level=8 rather than 6. I'm guessing this is the root of a lot of the issues, although not using them would have left holes.

I've gone through all the counties in that database and come up with a mapping of county to Ceremonial county (as per https://en.wikipedia.org/wiki/Ceremonial_counties_of_England). It might take ages to get a proper long term fix at OSM so I was thinking there is a much quicker interim solution that would really please the end users.

After the initial import you could implement a script that simply remapped the erroneous counties to the ceremonial counties. The import is effectively creating them at present, they just have small anomiles. So for instance after import search and replace all Derby to be Derbyshire. Hey presto its fixed.

Might it be possible to work with you to develop such a simple script such that the import would do a replace and allocate the erroneous counties to the correct ones people are clamering for.

Later I can also work on generating a Regions data file that is extracted from the OSM data that should match the Groundspeak Regions using the counties as building blocks.

Answer 3

I have now completed work on generating the Ceremonial counties from the OSM data. It's done in such a way that I can easily generate fresh polygons should you wish to refresh them in future.

Using Groundspeaks own map at https://wiki.groundspeak.com/download/attachments/11173980/ukfullmap.jpg?version=1&modificationDate=1405415769333&api=v2 to determine "Regions". You will note the polygons my process has generated perfectly matches Groundspeaks map.

The data I have generated will give you polygons for both UK counties and finally having matching polygons for Groundspeaks UK Regions. What format is the best to provide you with the data? I'm assuming shape files (.shp)? Also what is best way to provide the data to you? (email? upload to a sharing site eg: dropbox?)

Once you have loaded the data into your system then your users will finally have proper UK regions and counties for entire country, can't wait.

Comments

Sorry for taking so long to answer. Would you mind creating a support issue in our support system so that we can discuss details? It's easier to keep track there.

(support-button, "Project-GC Support". Or, send an email to info@.

Our plan is that we will look at the data, then we will make another poll here in the QA forum, to investigate if we should switch or not.

---

Support ticket created. Is there some means there to upload files? Or do I just await feedback for instructions on how to upload?

Re: Poll I'd recommend showing users a screenshot of new proposed counties vs current possibly also using Groundspeak map. A picture's worth a thousand words :)

---

I have seen the ticket, and assigned it to myself. I will try my best to get back to you tomorrow. Right now it's 7 pm here, and soon dinner time.

Screenshots in the poll is a good suggestion, I will keep that in mind, and feel free to complain if I forget it.