The Next Generation IP Geolocation service

In our company blog post, IP Geolocation Demystified, we outlined how IP Geolocation works, the methods used by service providers and the pitfalls of existing technologies in this space.

The main summary of the article was as follows:

So how do IP Geolocation service providers operate?

The entry level IP geolocation providers are likely to use less data sources, largely using WhoIs data only, which limits their decision scope to a much fewer options. This makes the process easier and maybe faster, but as a trade-off, it is much less accurate.

The more advanced IP Geolocation providers presumably work hard to organise and improve their results by delegating many of the final decisions to a human personal, in addition to some low-level automated process. Unfortunately manual work does not guarantee better results, as humans are also prone to errors, and definitely makes the process slower. As a result, we often see commercial IP geolocation databases updated on a monthly basis only, or weekly as the best.

The IP address space is a very dynamic area. Millions of IP addresses changing hands or are reallocated continuously, every hour. Therefore monthly or weekly updates are certainly not suitable for most IP geolocation applications.

In summary, none of the currently existing methods is sufficiently accurate. Even though a combination of methods allows for more precise estimation of IP location, this does not solve the problem of accuracy on a global scale. Moreover, the lack of a fully automated and deterministic methodology prevents existing IP geolocation databases from being updated frequently enough to cope with the highly dynamic nature of the internet IP address space.

So how do we overcome these limitations? We, at BigDataCloud, have carried out extensive research to develop a fully scientific and automated approach to geolocate using IP addresses.

Imagine getting an accurate Geolocation profile of your customer in mere milliseconds as opposed to seconds. This opens the doors to actual real-time Geolocation throughout your customer’s user experience. There is no longer any need to send requests server-side or for caching/pre-empting content to get around slow, unresponsive and inaccurate Geolocation services.

With the current trend in online connectivity, the internet-of-things and the integration of internet-connected commerce and marketing, it is absolutely vital that you can make immediate, qualitative decisions based on your customer’s location and demographics regardless of the media they are using.

How BigDataCloud’s IP Geolocation differs from existing services?

We at BigDataCloud have seen the processes and methods used by existing IP Geolocation providers to gather their data and to provide you with their best-guess of Geolocation results. In most cases, these guesses are inaccurate, outdated and prone to errors, as discussed in the previous article. Learning from the mistakes these companies have made in the past and are continuing to make, we took a step back to get a clearer picture and re-think the strategies and technologies used. In doing so we have come up with a much improved, streamlined and more accurate solution than is currently available.

The below outlines some of the technologies we have utilised and implemented.

The technology

The Internet is a packet-switched, hierarchical network similar to conventional parcel postage networks around the world. When a device connects to the internet and wishes to communicate with another device (e.g. a website), it sends out a data packet towards the destination address. If the source is directly connected to the destination, the packet goes straight there. In nearly all situations, this is not the case, and the packet needs to be routed using network router devices that ensure global connectivity.

This routing process is almost identical to a conventional parcel postage service. For example, if a parcel is sent from Norwood, South Australia to Denver, Colorado US, the parcel is first accepted by the local post office (presumably in Norwood SA).

The parcel is then examined, and since the target destination/address is not within the post office's delivery area, the parcel is routed to an upper hub — such as the bigger regional postage hub of Adelaide, South Australia (the capital of this state). Such large hubs (especially if they have access to an international airport) can use other predefined strategies such as: “What shall we do if the parcel is going to the US, or even more specifically, to Colorado US? If there is no appropriate and direct international route defined from that hub, the parcel will be further transited to another hub in the same country (perhaps Sydney, the capital of NSW) that does have the necessary, defined international route. From there it is redirected to a hub on US soil, presumably a large one but not necessarily even in the Colorado state area. It is again redirected closer until it reaches the closest post office servicing the final destination/address. Only then, the parcel is finally delivered to the recipient.

In this scenario, it is clearly understood that:

• Every postage hub/post office that delivers parcels to recipients has its predefined service coverage area.
• The last post office in a delivery chain is essentially the closest one to the destination.
• We can assume with high accuracy that if a person accepts a parcel delivery in Denver Colorado, he/she lives or works in that area.

The very same logic can be applied to the Internet. Countries or States are represented on the Internet as Autonomous Systems (AS). Routers are taking part in the delivery chain exactly as postage hubs/post offices do. Some are responsible for serving customers at their destination (Access or Aggregation routers), while others redirect traffic within the same AS (Core routers), and some handle inbound and outbound extra AS traffic (Edge or Border routers). Others can do it all (Multifunctional routers). Autonomous Systems (AS) are the same as Countries/States in the parcel delivery network and are interconnected using some predefined but dynamic and flexible policies — powered by the Border Router Gateway (BGP) protocol.

Using the postage example as an analogy, we can determine with a high level of confidence a geographic location for every IP address, if:

• We manage to identify and classify all the public-facing network routers on the Internet;
• We determine what is the actual service area for each of these;
• We know which is the last serving router for each of the remainder of the IP addresses on the internet.
• Finally, knowing the closest router interface and its expected service area, we can estimate the location-likelihood for every reachable IP address in the world.

About BigDataCloud’s Patented Tech

1. We non-intrusively and non-abusively discover the entire routable IP address space, revealing all the public ‘postage’ routes worldwide
2. We note and further classify all the world router interfaces detected
3. Then, by using a comparatively small amount of IP addresses with known Geolocations (field evidence data), we estimate the actual service area for every router interface detected
4. Finally, knowing the closest router interface and its expected service area, we can estimate the location likelihood for every reachable IP address in the world

As a result, the main benefits of BigDataCloud’s Next Generation IP Geolocation technology include:

• Superior geolocation accuracy at a global scale
• Superior data granulation — no more blocks up to a single IP address geolocation resolution!
• Superior data update rate — the industry’s first factual daily updated system
• More valuable data — revealing the operational area in addition to a conventional location point estimation
• Perfectly suitable for mobile networks' geolocation

To see an example of our Geolocation services in action, check out our What is My IP? page, where you can see first-hand the speed and quality of data our APIs offer.

To find out more about our IP Geolocation APIs, including live demos, data objects and integration instructions, head over to the IP Geolocation APIs page.

Originally published at https://www.bigdatacloud.com.