"I love Bots vs Browsers - Public Bots and User Agents Database and Commentary"

2022-09-07 04:00:48

Bots vs Browsers - Public Bot / User Agent Database & CommentaryHomeArchivesAbout User AgentsUser Agent Test TrackRecent AdditionsContributorsDonateTop Searches Bots vs Browsers - database of 6,004,978 user agents and growingThe Latest:MAY 192021Goodbye Internet ExplorerFor more than 25 years Internet Explorer has helped web users view the web at large, and now Microsoft has announced that they are finally ending support for the legacy browser. At several points in IE's lifespan it carried the distinguished title as the world's most prominently used web browser. IE competed head-to-head with other notable browsers included Netscape, Firefox, and eventually Google Chrome before Microsoft signalled with the 2015 launch of Windows 10 that it would eventually sunset Internet Explorer.Microsoft's Edge browser is the heir apparent for Windows users to continue on with, as Microsoft has used Edge in recent years to help phase out Internet Explorer by promoting Edge usage with Windows users over Internet Explorer.Edge is marketted as a more secure and modern browser than IE and could be considered a direct competitor to Google's Chrome web browser, as well as Firefox. Microsoft Edge also offers "IE Mode", which allows users to access legacy Internet Explorer based websites with a similar experience to what was available in IE. This feature is likely the reason Microsoft was finally able to say goodbye to IE by providing an alternative for legacy users and not risk impacting user experience.Internet Explorer will officially be out-of-support on June 15, 2022. You can read more about the announcement here on the Microsoft blog.In the 15 years that Bots vs Browsers has been studying user agents, we've cataloged nearly 1 million variations of Internet Explorer in the wild. IE was one of the few browsing options available back in 2006 when we got started, and was a big part of our journey in the early days - RIP MSIE!MAR 172016User Agent Debuts for Brave and Servo BrowsersAs far as new browsers go, Microsoft's Edge browser release was the only major news last year.However, over the last few weeks, two other browsers-in-the-making have caught our attention in the news.Thanks to our users, we also have sample user agents for both that we are happy to share.A Brave new BrowserBrave is an ambitious new browser that intends to revolutionize users' online experience by making browsing faster and safer.To be specific, this would include blocking ads and trackers that are involved with gathering user information.The basic idea is that Brave would insert their own non-intrusive ads into the blocked ads' slots, and publishers would be paid via a new revenue share model that is described in more detail here.The company was founded by Brendan Eich, who is best known as the creator of JavaScript.Read more about Brave by visiting their official site at Brave.comand check out our new Brave user agentsServo, the Parallel Browser Engine ProjectMozilla is building a new browser engine called Servo using their own programming language Rust.Using their own words, "Servo is a modern, performant browser engine designed to be appropriate for applications including embedded use".Servo has received a great deal of support in the industry beyond just Mozilla and individual contributors - Samsung has also contributed heavily to the efforts.Here are some sample Servo user agentsJAN 20201610 Years Old, and Big Changes for Bots vs Browsers!To celebrate the site's 10th year in operation, we're working hard to make sure Bots vs Browsers becomes more useful than ever.We recently released a responsive mobile version of the site to accomodate our users on smaller screens.We also implemented additional features for IP address details, including new geographical info and ASN details.In addition, we have changed our user agent categories menu structure to handle extensive new classifications.As our categories and classifications have grown organically over the years, some need to be broken down into smaller categories.Older categories need to be set aside for users doing historical research, and more relevant newer categories need to be promoted into higher prominenence for users that are interested in more cutting-edge research.To accomodate this, we have built a new classification structure and built new pattern-matching sequences to place user agents into their proper categories.This will be an ongoing process, and as our site grows, we have built a new menu structure to hold large numbers of categories and ease navigation around the site.We will also be addressing our search indexing, as the database has grown considerably since our first version of the site.We did not anticipate searching over 1 million user agents when we first released, so it's time for an update!At the 10 year mark, Bots vs Browsers now has 1,453,272 user agents, and we have identified 24,128 bots out of the mix.JAN 62016Internet Explorer : Out with the old, in with the new!Just in time for the new year, Microsoft has announced that they will be pulling the plug on Internet Explorer versions 8, 9, and 10 next Tuesday (January 12).The final patch for these browser versions will be released, and there will be no more security updates or technical support after that.As with any legacy version retirement, it's never that simple.Microsoft has several exclusions for older versions of IE on certain operating systems and under certain conditions.If you are curious if your system or users will be affected, then you should read the details on Microsoft's full announcement here.This move will push users to upgrade to either Internet Explorer 11 or to Microsoft's new Edge browser, built for Windows 10.For those of you with nostaligic interest, don't worry - we have user agent classifications for all those old versions of IE below:Internet Explorer 8Internet Explorer 9Internet Explorer 10DEC 112015Microsoft Edge browser extensions on the way!Microsoft released an "accidental" preview site for the upcoming Edge browser extensions.The release was a preview for a developer site which announced Edge browser extensions support.Indications from the preview site are that developers will have the chance to write extensions ahead of the general release, and Windows 10 testers would have the ability to try the extenstions out ahead of time as well.Microsoft had originally planned to release extensions for the Edge browser in a patch release of the Windows 10 operating system, but delays caused the extensions release to be moved to early 2016.It is likely that the extensions would leverage the Windows Store, which is integrated with Windows 10.The Edge browser was released with Windows 10 initially, and is not available for previous versions of Windows.From the screenshot of the extensions site, it appears that extensions for Reddit and Pinterest are available for download.Details of the extensions preview site from Microsoft can be found on Twitter here.More information about Edge browsers is available on our Edge browser user agent strings page.DEC 132013Are the Bots beating the Browsers???After years of observing Bots vs Browsers by analyzing user agent logs, the struggle appears to have swung heavily in favor of the bots in 2013. As late as 2012, the browsers (humans) were still within 1% of the majority of website traffic, but 2013 saw robot traffic increase with a vengeance.According to an article published by Incapsula, robots are now responsible for 61.5% of all website traffic.The study last year measured much closer, at 51% of all website traffic coming from bots.This pushes human traffic down from 2012 at 49% down to 38.5% in 2013.As far as the breakdown of what type of robots are responsible for all of this traffic, here is a quick list:31% Search Engines (i.e. the "good" bots)5% Scrapers4.5% Hacking tools0.5% Spammers20.5% Other Impersonators (basically all other non-humans)You can read more about the findings here.FEB 32013Tablets, Slates, and Touch-enabled Devices - Part 1 : The iPadNew form factors for computers have changed the way we consume the web, and along the way we have left behind a trail of breadcrumbs in our user agents that we can now look back on to see this progression.This week we'll focus on Apple's contribution to this space with the release of the iPad.Here are some popular variations of the iPad user agent across all versions.Remember that time when Apple decided the tablets of the time sucked and we needed something more practical?In January of 2010, Apple announced the 1st generation iPad, which was released in April of 2010.Not only did this mark the birth of the iPad, but also the birth of a slimmer, more portable and stripped-down personal computing experience.Here are some variations of the first generation iPad user agent.In March 2011, Apple went on to improve in the first generation iPad.The second generation iPad was announced March 2, 2011, and released on March 11, 2011.With the second iPad came double the memory of the original device, from 256 MB to 512 MB.Second generation iPads also introduced a 0.7 MP camera, added a gyroscope sensor to the device, and upgraded the iOS version 5.1.1 to 6.1.Here are some second generation iPad user agents that we have seen over the years.Almost exactly one year after the iPad 2, Apple announced the iPad 3.The third generation iPad was announced March 7, 2012, and released on March 16, 2012.Once again the new iPad 3 doubled the memory of the device, this time from 512 MB to 1 GB.The display of the iPad 3 sported the new Retina Display technology that Apple has become known for.Other upgrades included Bluetooth 4.0 (from version 2.1 on previous iPads), and major upgrades to the camera to bring it up to 5 megapixel and 1080p HD video recording.Third generation iPad user agents began showing up, and we were listening.In the interests of breaking tradition, Apple announced the iPad 4 only 8 months after the iPad 3's release.On October 23, 2012 Apple announced the new iPad 4, which would be released to consumers on November 2, 2012.While many criticized the iPad 4 of being too similar to the iPad 3, and accused Apple of releasing a new iPad only to combat the release of the Microsoft Surface RT, there were some areas that were upgraded.The fourth generation iPad improved the front-facing camera from .3 megapixel to 1.2 megapixel, and changed the device's processor from a 1 GHz dual-core ARM processor to a 1.4 GHz dual-core Apple Swift processor.Most importantly, the iPad 4 added a new storage size of 128 GB - all previous iPad versions were available with 3 storage sizes: 16, 32, or 64 GB.iPad 4 user agents have been appearing ever since, and the rest is history.In the same announcement as the iPad 4, Apple announced an iPad Mini as well.As far as user agents go, the iPad Mini is a bit of a mystery.Apple does not identify an iPad Mini separately from other iPads, so developers trying to target this model had to use other means to detect the devices' capabilities.We hope this post is helpful and informative to you, and we enjoyed revisiting the history of the iPad user agent. Thanks for stopping by!Our growing total of user agents is now at 1,021,791, with 20,089 bots included in the mix.JAN 1520137 Years Old, One Million User Agents!Wow - 7 years old. What started out as a goofy idea of narrating user agents in our web traffic logs has grown so much in 7 years.Not to mention that four our seventh birthday, we found our one-millionth user agent to put us into seven-digits!We are now sitting at over 1 million user agents, of which 20,000+ are identified as bots.The most notable change in the last seven years has to be the nature of devices that we see from our sites and contributing sites.In the early days, most user agents were either Internet Explorer, Netscape, Opera, or some form of Mozilla, and everything else was pretty much bots or proxies.Now looking over our categories, mobile and non-PC user agents seem to be the most diverse and rapid changing.In our first two or three years we never saw advertising, spamming, or script injection attempts in user agents.Since the Script Injections category appeared, it has grown rapidly, and the number of user agents containing spam has also been a challenge to weed out from useful data.We've come a long way from the days where we just wanted to know if you were Netscape Gold 3 or Netscape Navigator 4, and if your browser was capable of handling that new-fangled concept of client-side JavaScript.As always, thanks to our site contributors who take the time to send us their data to mine through, and to our readers - we hope that you continue to find our site useful in the coming years!Over the next few weeks and months, we'll focus on posting about different areas that have changed in the User Agents realm over the past few years.Look for posts to focus on different user agents from phones, tablet and touch enabled devices, gaming consoles, and last but not least, bots!At seven years of operation, we now have 1,000,047 user agents and 20,014 bots tracked to date!JAN 122010Happy 4th Birthday, Bots vs Browsers!Today we turn 4 years old!Thanks to everyone who has contributed to the site, and to our readers - you are the reason that we are here in the first place!Here is the latest news from our user agent community:Google AppEngine user agents have been on the rise, so we have started to tag them in their own category.related...Macrothela bot variation Rassler has been steadily advancing its bot.We have seen versions 0.17 through 0.24 in the last few months.In addition, many new flavors of the Macrothela bot have been spotted, including atlantic, mars, martin, and zeger.related...In hacker news, we have seen some new SQL injection attempts - The first is an attempt access database schema information from an unsecure site - SELECT * FROM information_schema.tables WHERE information_schema=database().The second one tries to UNION the numbers 1 through 5 to an existing SELECT statement - related...Here are two other non-SQL related script injection attempts against PHP sites - this one tries to include a remote txt file,and this one is intended to execute the injected PHP code on the server.related...We also saw 3 new Sheen web crawlers this month - SheenBot-1.0.0, 1.0.1, and 1.0.2.related...In our four years of operation, we have added 421,718 user agents and 4,030 bots to our community database!JUL 122009300,000 User Agents!This was a big month for the site - we finally broke 300,000 user agents!To celebrate, we also added some features to our IP Directory - here are the details:We recently added a new feature to our IP Directory.We now provide best-guess geographical information about each IP address including country, city, region, and even Latitude and longitude.Just think - now you can find out the hometown of all your favorite bots!We saw our Rassler collection grow again this month - we had seen versions 0.6 through 0.13, and this month we addedrassler/0.14,rassler/0.15 andrassler/0.16.related...We also saw a surprise visit from the O'Reilly Link Checker (links from our published works) crawler.Maybe we are referenced in one of their published works - hopefully in a positive light!related...The SaladSpoon/ShopSalad 1.0 bot dropped in as well.This bot crawls for the online shopping aggregator site known as ShopSalad.com.related...New Nutch versions were rampant as usual, with these 11 debuting this month:volverine/Nutch-0.9cancho/Nutch-1.0PR Crawler/Nutch-1.0nutch.us/Nutch-1.0Webcrawler/Nutch-1.0DataPatrol/Nutch-1.0blackcrawl/Nutch-0.9SHC/Nutch-1.0nutch-solr-integration/Nutch-1.0Martin/Nutch-1.0related...Another crafty script injection attempt - this attack was geared towards SQL injection, not cross site scripting.This is the part of the post where I get on my soapbox and tell you that you should protect any and all sites you work on by adding protections from the web application layer all the way down to proper database permissions.related...Here are the failed attempts we saw this month:'; DROP TABLE `member`--'; DROP TABLE `member`-- ">'>Yacybot was extremely busy this month, dropping 23 new versions of their bots in our logs.We won't bother listing them all, just follow this link to see our Yacybot collection.This installment brings us to 316,779 user agents and 3,483 total bots.More...USER AGENT CATEGORIESBOTSAll Bots (1,471,229)Googlebot (6,390)Bing (MSN Bot) (125)Yahoo! Bots (1,093)Baidu Search Bots (2,015)Ask Jeeves / Teoma (120)Ichiro Bots (27)Nutch Bots (664)Yacy Bots (526)Sitemap related (16)Google Code AppEngine (1,024)BROWSERSMicrosoft Edge (618)Brave (31)Servo (10)Internet Explorer 11 (398)Samsung Galaxy Tablets (1,028)Android Tablets (598)Apple iPad (6,765)Apple iPad 1 (119)Apple iPad 2 (134)Apple iPad 3 (134)Apple iPad 4 (134)Apple iTunes (70)Apple iPod (1,847)Windows 8 Touch Devices (291)Windows 8 / IE 10 (557)Amazon Kindle (46)Amazon Kindle Fire (13)Alexa Toolbar (7,382)MOBILE BROWSERSAndroid (3,348,047)Nexus One (160)HTC (4,931)Apple iPhone (1,074,488)Blackberry (4,373)Windows Mobile 10 (216)Windows Phone 8 (394)Windows Phone 7 (161)Nokia Phones (8,440)Nokia / Microsoft Lumia (537)OLD SCHOOLWindows CE (1,614)Windows Mobile (677)Netscape 3 (10)Netscape 4 (20)Netscape 5 (20)Netscape 6 (24)Netscape 7 (405)AOL 3 (5)AOL 4 (50)AOL 5 (87)AOL 6 (133)AOL 7 (442)AOL 8 (914)AOL 9 (24,448)AOL 10 (469)Internet Explorer 3 (200)Internet Explorer 4 (739)Internet Explorer 5 (3,560)Internet Explorer 6 (163,612)Internet Explorer 7 (257,268)Internet Explorer 8 (248,308)Internet Explorer 9 (5,025)Internet Explorer 10 (2,435)Zune (12,948)OTHERTwitter Related (142)Facebook Related (83)Google Earth Related (88)Microsoft XBox (334)Playstation 4 (54)Nintendo WII (97)Nintendo DS (96)Amazon FireTV (6)Roku (67)RSS Related (133)Proxy User Agents (2,699)Sony Playstation (406)Playstation 3 (81)Playstation Vita (33)PlayStation Portable (32)Unidentified (375,723)SPAM / HACKSScript Injections (336)SQL Injection (7,699)PHP Injection (1,993)XSS Attacks (64)Link Injection (250,866)User Agent SearchClick here to Search our User AgentsTop Searches: [view all]"iPhone""Pcore""W3C""Touch""Android""Silk""iPod""mobile""Blackberry""script""motorola""google earth""spyware""facebook""psycheclone""iTunes""Tablet""Spider""google image""mediapartners google""grub client""web products""spam""linux""bot""media center""robot""alawar""Red Hat"