Exploring the Internet Using Public Discovery
Objectives
-
Visualize the structure of a portion of the internet
-
Have fun while learning the Google Maps Javascript API
-
Use Python for
ETL
-
Experiment with public geocoding services
Methods
-
Traceroute to both random and selected IP addresses
-
Python programs to parse traceroute output, geocoding, and fetching whois info
-
MySQL database in a simple backend generating node and edge lists
-
Javascript to parse backend responses and drive Google Maps API
Challenges
-
Concisely depicting multiple IP addresses at a common location
-
Obtaining reasonably accurate location of IP addresses (geocoding)
-
Finding traceable addresses in less-technological areas of the world; even their government and university sites are frequently hosted in "hi-tech" countries
Findings
-
The Google Maps API is powerful and convenient. Performance is good with thousands of markers and lines.
-
Many open-source enhancements are available for Google Maps. I experimented with one called
Overlapping Marker Spiderfier
before settling on the current solution of consolidating them on the backend before sending to the UI.
-
Python is a handy language for ETL, but it takes a while to break the habit of typing semicolons and curly braces.
-
Sharing interesting features with others is facilitated by a URI specifying a map view (location and zoom). These links are available when viewing the details of a map marker.
-
Routes to nearby countries with which the U.S. has experienced tension (e.g., Cuba and Venezuela) seem to be via intermediate countries, greatly increasing path length.
See Also
This alternate view
depicts the same information in a purely logical directed graph.
Here is a map
presenting a schematic view of undersea cable routes. Unsurprisingly, my map shows a very strong correlation with undersea cable routes for transcontinental connections.