## Reference robots.txt

User-agent: *
Disallow: /web
Disallow: /Web
Disallow: /WEB
Disallow: /browse
Disallow: /article/
Disallow: /slideshow/
Disallow: /fragment
Disallow: /_clk
Disallow: /slp
Disallow: /log/
Disallow: *ad=
Disallow: *aq=
Disallow: /wp-admin/
Allow: /wp-admin/admin-ajax.php

User-agent: SemrushBot-SA
Disallow: /

User-agent: SemrushBot
Disallow: /

User-agent: Mediapartners-Google
Disallow:

User-agent: Adsbot-Google
Disallow:

User-agent: AdsBot-Google-Mobile
Disallow:

User-agent: Teoma
Disallow:

User-agent: netseer
Disallow:

User-agent: AdIdxBot
Disallow:

User-agent: proximic
Disallow:

User-agent: grapeshot
Disallow:

User-agent: GPTBot
Disallow: /

Sitemap: https://www.reference.com/sitemap.xml