Changes to GeoLite2 license and distribution require changes to how you download databases

[2shortplanks]

The URL used to directly download GeoLite2 databases will no longer function after December 30th 2019.

Due to upcoming data privacy regulations, MaxMind are making significant changes to how users access free MaxMind GeoLite2 databases starting December 30, 2019. The databases will continue to be available without charge and for redistribution. However, you will be required to create an account and use a license key to download the databases, and agree to a new EULA that addresses applicable data privacy regulations.

Learn more on the MaxMind blog: https://blog.maxmind.com/2019/12/18/significant-changes-to-accessing-and-using-geolite2-databases/.

Mark Fowler
MaxMind

[jahshuah]

The script scripts/fetch_geo.py could be updated with the following (or something like it):

import tarfile
import requests
import shutil
import ConfigParser

from pyasn import mrtx, __version__

maxmind_conf = ConfigParser.RawConfigParser()
maxmind_conf.read('../maxmind.conf')

LICENSE_KEY = maxmind_conf.get('license', 'license_key')

APP_BASE = "/tmp"

if __name__ == '__main__':
    url = "https://download.maxmind.com/app/geoip_download?edition_id=GeoLite2-City&license_key={0}&suffix=tar.gz".format(LICENSE_KEY)
    response = requests.get(url)
    path = '%s/GeoLite2-City.tar.gz' % (APP_BASE)
    open(path, 'wb').write(response.content)
    tar = tarfile.open(path)
    files = tar.getmembers()
    tar.extractall(path='/tmp/')
    tar.close()

    for file in files:
        if not file.name.endswith('.mmdb'):
            continue
        shutil.move('/tmp/%s' % file.name, '/tmp/GeoLite2-City.mmdb')

Where maxmind.conf looks like this:

[license]
# Change to match that of your own license key
license_key = MY_LICENSE_KEY

I'm not sure where the optimal placement of the config file for that would be, but I suppose the root of the project wouldn't be too bad a place. Especially considering that the instructions define /opt as the location for the netinfo directory. Setting the perms on the file to chmod 0600 maxmind.conf would limit visibility of the contents to the owner only.

These suggestions follow those listed in the link in @2shortplanks comment as well as in this document: https://dev.maxmind.com/geoip/geoipupdate/#Direct_Downloads. If all are agreed, I'd be willing to make a PR.

[jahshuah]

The task function would also have to be updated. Just noticed that.