Data routinely gathered in Web logs – IP address, cookie ID, operating system, browser type, user-agent strings – can threaten online privacy because they can be used to identify the activity of individual machines, according to Microsoft researchers.
At the same time, analysis of such data when anonymised can help detect malicious activity and so improve overall Internet security, the research said.
The researchers found that 62% of the time, HTTP user-agent information alone can accurately tag a host. Combine that same information with the IP address, and the accuracy jumps to 80.6%. When the user-agent information was combined with just the IP prefix the accuracy was still 79.3%.
The highest accuracy came when more than one user ID was linked to a single host, as would be the case in a family that shares a single computer. In such cases, multiple IDs would accurately represent that one host computer. The accuracy rate was 92.8%.
The analysis of this seemingly benign information was based on a month – August 2010 – of anonymised Hotmail and Bing data on hundreds of millions of users. The researchers said they tried to find out whether a single piece of log data can uniquely reveal a particular host.
They found that even anonymised data can leak information. For example, replacing an IP address with its IP prefix still yields enough information that when combined with other commonly logged factors can be revealing.
“”[C]oarse grained IP prefixes achieve similar host-tracking accuracy to that of precise IP address information when they are combined with hashed [user-agent] strings,” they said.
They looked at data gathered from application-layer events directed at Web servers within the Hotmail and Bing networks.
From Hotmail, they gleaned coarse data about the OS and browser types, source IP address, time of login and anonymized user IDs. From Bing, they gathered anonymized HTTP user-agent strings, source IP addresses of queries, times of queries, anonymised cookie IDs issued by Bing and creation dates of the cookies.
The researchers set out to detail how much identifying information gets revealed by common identifiers. They weren’t trying to discover specific individuals’ activities, but to understand the patterns of aggregated activities and explore their implications.
The researchers say their use of the data falls within Microsoft’s privacy policies and as part of that policy the data can’t be made available to outside researchers.
They found that service providers can recognise 88% of devices that receive a cookie, clear the cookie, then return to the site, if they examine other identifying factors they gathered during the initial connection. Even if they use private browsing mode, which is designed to protect user identity, they can still be identified, the researchers said.
“Our analysis suggests that users who do not wish to be tracked should do much more than clear cookies,” they said, and note that in some circumstances clearing cookies can help identify a particular host.
“Uncommon behaviors such as clearing cookies for each request may instead distinguish a host from others who do not do so,” they added.
