X

Exploring the Internet Using Public Discovery

Objectives

• Visualize the structure of a portion of the internet
• Have fun while learning the Google Maps Javascript API
• Use Python for ETL
• Experiment with public geocoding services

Methods

• Traceroute to both random and selected IP addresses
• Python programs to parse traceroute output, geocoding, and fetching whois info
• MySQL database in a simple backend generating node and edge lists
• Javascript to parse backend responses and drive Google Maps API

Challenges

• Concisely depicting multiple IP addresses at a common location
• Obtaining reasonably accurate location of IP addresses (geocoding)
• Finding traceable addresses in less-technological areas of the world; even their government and university sites are frequently hosted in "hi-tech" countries

Findings

• The Google Maps API is powerful and convenient. Performance is good with thousands of markers and lines.
• Many open-source enhancements are available for Google Maps. I experimented with one called Overlapping Marker Spiderfier before settling on the current solution of consolidating them on the backend before sending to the UI.
• Python is a handy language for ETL, but it takes a while to break the habit of typing semicolons and curly braces.
• Sharing interesting features with others is facilitated by a URI specifying a map view (location and zoom). These links are available when viewing the details of a map marker.
• Routes to nearby countries with which the U.S. has experienced tension (e.g., Cuba and Venezuela) seem to be via intermediate countries, greatly increasing path length.

See Also

This alternate view depicts the same information in a purely logical directed graph.

Here is a map presenting a schematic view of undersea cable routes. Unsurprisingly, my map shows a very strong correlation with undersea cable routes for transcontinental connections.