Spiders and Robots Service

IAB/ABC International Spiders & Bots List

The IAB/ABC International Spiders & Robots list helps filter known, non-human activity that can significantly inflate ad impression and site traffic counts. Effective use of this data can lead to a more transparent and accurate measurement for you and your clients.

Implementation of this list is recommended in the UK by our ‘Web Traffic Reporting Standards’ and is a requirement in the US for the IAB‘s ‘Ad Impression Measurement Guidelines‘ & the MRC‘s ‘Invalid Traffic Detection and Filtration Guidelines‘.

What’s included?
  • Best practice guidelines
  • Technical support
  • Industry leading spider & bot detection and filtering:
The IAB/ABC International Spiders & Robots List
  • To help exclude known robotic traffic, the IAB/ABC International Spiders & Bots List contains a list of User-Agent strings previously found to have made material levels of robotic activity.

The IAB/ABC International list of valid browsers
  • Media Owners may wish to use a more rigorous robot exclusion process by adopting the dual-pass approach recommended by the IAB. This approach first checks that the User-Agent matches the include list and then excludes it if it matches the exclude list.

The ABC IP Address Exclusion List

  • ABC also publishes a list of IP addresses that have previously been found to make material levels of invalid robotic activity. These IP addresses typically represent activity from external site monitoring tools, and from certain robots which use generic User-Agents.


Suzy Saker1

Jo Reid

Senior Account Manager

Contact me to discuss any of these services 

Icon tel pink This email address is being protected from spambots. You need JavaScript enabled to view it. Linkedin icon


Media Audits

ABC releases data for the UK media industry to use when trading advertising. Our audits can cover your whole brand.

Third Party Auditing & Consultancy

ABC are a leading industry-owned auditor for media products and services, with specialist skills in digital ad trading. Our process and bespoke audits cover the following areas.

Our logo stands for quality and trust in media, empowering our industry to trade with confidence.

ABC Logo