Crawler Page
Website owners can use robots.txt to control whether Jina AI URL reading requests sent with RisphaiBot may read pages and use their content in AI responses.
Last updated: October 14, 2026
Website Owner Controls (robots.txt)
For page-reading requests through Jina AI, Risphai uses RisphaiBot as the robots identifier. Website owners can add the following rules to restrict that retrieval path.
User-agent: RisphaiBot Disallow: /
You can also disallow specific paths instead of the entire site.
This only applies to Jina AI URL reading. It does not affect Exa AI search or Exa AI page retrieval, and it does not control whether a page appears in web search results shown in the Risphai UI.
RisphaiBot rules apply to that page-reading path. For copyright, unlawful content, or access concerns involving Risphai, contact support@risphai.com with the URLs, your authority to make the request, and an explanation. For removal from an independent provider's index, contact that provider as well.