Allow Chatgpt Bot | Community
Skip to main content
bjoern__koth
bjoern__koth
New Participant
March 11, 2025
Solved

Allow Chatgpt Bot

Hi all,

 

does anyone know whether ChatGPT and other AI crawlers are currently already included in the IAB bot list?

 

We're currently having the IAB bot list feature enabled, and are capturing the user agent of incoming traffic, and as far as I can see there are not "openai" user agents captured

https://platform.openai.com/docs/bots/

 

Just wondering since our business stakeholders would be interested in getting the bot crawler traffic through, and the WAF is already opened up to let that kind of traffic through.

Is there actually a way to unblock a certain browser in the Bot rules that may be included in the bot list?

FarazHusain
FarazHusainAccepted solution
Employee
March 11, 2025

Hey @bjoern__koth As of now, there is no public information confirming that OpenAI's crawlers, such as GPTBot or OAI-SearchBot, are included in the IAB/ABC list, so difficult to comment. 

 

Regarding unblocking, I doubt that's possible using bot rules, not sure if VISTA could help here. 

View reply

10 replies

kayzee
New Participant
May 15, 2025

I would echo bjoern's post.

Knowing the content consumption by GenAI-platforms is highly valuable to evaluate since there is a behavior shift from search engines towards chat platforms.

Especially when companies like OpenAI explicitly declare their useragents.

We are able to see those useragent in server logs but I dont see them in AA too.

Additionally, I couldn't find any into if the IAB lot list contains chatgpt-like useragents/.

Translate
kayzee
New Participant
June 25, 2025

Just wondering if anyone has figured this out.

 

While tracking referral traffic from AI chatbot sites is fairly straightforward, I'm still trying to determine how to track AI bots that fetch information directly from websites to generate responses.

 

OpenAI has shared more details about ChatGPT bots here:
https://platform.openai.com/docs/bots

 

It would be great to hear Adobe’s point of view on this.

Translate
Jennifer_Dungan
Jennifer_Dungan
Level 2
March 11, 2025

I also am not sure about whether the IAB included AI Bots... but I am pretty sure that there is no "bot whitelist".

 

That said, how deep do you need the bot information? Workspace does have some limited data from Bots, including the Bot Name and the Page that the Bot hit...

 

So, you should be able to get a simple report without opening up traffic collection to those bots....

Translate
    FarazHusain
    FarazHusainAccepted solution
    Employee
    March 11, 2025

    Hey @bjoern__koth As of now, there is no public information confirming that OpenAI's crawlers, such as GPTBot or OAI-SearchBot, are included in the IAB/ABC list, so difficult to comment. 

     

    Regarding unblocking, I doubt that's possible using bot rules, not sure if VISTA could help here. 

    Translate
      bjoern__koth
      bjoern__kothAuthor
      New Participant
      March 11, 2025

      Hmm ok, so a bit of a black box. Fair enough, I cannot see any user agent containing "openai", also no Bot Name over the past 90 days.

      So, maybe it's already on the IAB list

       

      Translate
      Cheers from Switzerland!
        Jennifer_Dungan
        Jennifer_Dungan
        Level 2
        June 26, 2025

        Hi @bjoern__koth,

         

        If you are using the IAB list, and openai is on the list, I would expect to see it in the table you made...

         

        On the other hand, if my site,I look at referrers for "openai" I see regular PVs and Occurrences:

         

        So I would say that it doesn't look like they are yet in the IAB list.

        Translate
        Terms & ConditionsAccessibility statement

        Loading footer...