First published: October 25, 2021
Last updated: October 16, 2023
IndexNow is an easy way for websites owners to instantly inform search engines about latest content changes on their website. In a nut shell, IndexNow is a simple ping so that search engines know that a URL and its content has been added, updated, or deleted, allowing search engines to quickly reflect this change in their search results.
You are aware that it can take days to weeks for search engines to discover that the content has changed, as search engines don’t crawl every URL often. With IndexNow, search engines know immediately the "URLs that have changed, helping them prioritize crawl for these URLs and thereby limiting organic crawling to discover new content."
Search Engines that are IndexNow-enabled
IndexNow is offered under the terms of the Attribution-ShareAlike Creative Commons License and has support from Microsoft Bing, Yandex.
So the pioneer IndexNow-enable search engines are Bing and Yandex.
The search engines have endpoints to submit URLs are as follows:
• Microsoft Bing
https://www.bing.com/indexnow?url=url-changed&key=your-key
• Yandex
https://yandex.com/indexnow?url=url-changed&key=your-key
IndexNow will commence by November, 2021
Documentation
Submitting One URL
To submit a URL using an HTTP request (replace with the URL provided by the search engine), issue your request to the following URL:
https://<searchengine>/indexnow?url=url-changed&key=your-key
• url-changed is a URL of your website which has been added, updated, or deleted. URL must be URL-escaped and encoded and please make sure that your URLs follow the RFC-3986 standard for URIs.
• Your-key should have a minimum of 8 and a
of 128 hexadecimal characters. The key can contain only the following characters: lowercase characters (a-z), uppercase characters (A-Z), numbers (0-9), and dashes (-).
For example, if you want to notify search engines that https://www.example.org/product.html has been updated, and you want to use this key c59ff45687024cfb83c1cc6abbb0515c
https://<searchengine>/indexnow?
url=https://www.example.org/product.html&
key=c59ff45687024cfb83c1cc6abbb0515c
You can issue the HTTP request using your browser, wget, curl, or any other mechanism of your choosing. A successful request will return an HTTP 200 response code; if you receive a different response, verify that you don’t submit too often, that the key and URL are valid and resubmit the request. The HTTP 200 response code only indicates that the search engine has received your URL.
How to Summit URLs
To submit a set of URLs using an HTTP request issue your POST JSON request to the URL provided by Search Engines. Replace by the host name of the search engine.
POST /IndexNow HTTP/1.1
Content-Type: application/json; charset=utf-8
Host: <searchengine>
{
"host": "www.example.org",
"key": "c59ff45687024cfb83c1cc6abbb0515c",
"urlList": [
"https://www.example.org/url1",
"https://www.example.org/folder/url2",
"https://www.example.org/url3"
]
}
You can submit up to 10,000 URLs per post, mixing http and https URLs if needed.
You can issue the HTTP request using wget, curl, or another mechanism of your choosing. A successful request will return an HTTP 200 response code; if you receive a different response, you should verify your request and if everything looks fine, resubmit your request. The HTTP 200 response code only indicates that the search engine has received your set of URLs.
The recommended way is to automate submission of URLs as soon as the content is added, updated, or deleted up to some limit; see best practices for user generated content in the Frequently Asked Questions.
How to Verify Ownership Via the Key
You need to prove your ownership of the host for which URLs are being submitted. To do this, you need to host at least one text file within the host. Once you summit your URLs to search engines, search engines crawl the key file to verify your ownership and use the key to index your URLs until you change the key. Third party should not know the key and the location of the file key, but you and search engines.
How to Verify Ownership
IndexNow offers two methods to verify Ownership of Your Site
To submit URLs, you must "prove" ownership of the host for which URLs are being submitted by hosting at least one text file within the host. Once you submit your URLs to search engines, search engines will crawl the key file to verify ownership and use the key until you change the key. Only you and the search engines should know the key and your file key location.
Option 1
Hosting a text key file at the root directory of your host.
You must host a UTF-8 encoded text key file {your-key}.txt listing the key in the file at the root directory of your website.
For instance for the previous examples, you will need to host your UTF-8 key file at https://www.example.org/c59ff45687024cfb83c1cc6abbb0515c.txt and this file must contain the key c59ff45687024cfb83c1cc6abbb0515c
Option 2
Hosting a text key file within your host.
You can also host one to many UTF-8 encoded text key files in other locations within the same host and you must tell search engines the location of this text key file in each IndexNow notification by specifying the location using the keyLocation variable.
If you submit an URL, specify the key file location as key Location URLs parameter value.
https://<searchengine>/indexnow?
url=http://www.example.org/product.html&
key=c59ff45687024cfb83c1cc6abbb0515c&
keyLocation=http://www.example.org/myIndexNowKey63638.txt
If you submit a set of URLs, specify the key file location as key Location variable in the JSON content.
POST /IndexNow HTTP/1.1
Content-Type: application/json; charset=utf-8
Host: <searchengine>
{
"host": "www.example.org",
"key": "c59ff45687024cfb83c1cc6abbb0515c",
"keyLocation": "https://www.example.org/myIndexNowKey63638.txt",
"urlList": [
"https://www.example.org/url1",
"https://www.example.org/folder/url2",
"https://www.example.org/url3"
]
}
In this option 2, the location of a key file determines the set of URLs that can be included with this key. A key file located at http://example.org/catalog/key12457EDd.txt can include any URLs starting with http://example.org/catalog/ but cannot include URLs starting with http://example.org/help/.
• http://example.com/catalog/show?item=23
• http://example.com/catalog/show?item=233&user=3453
URLs not considered valid in http://example.com/catalog/sitemap.xml include:
• http://example.com/image/show?item=23
• http://example.com/image/show?item=233&user=3453
• https://example.com/catalog/page1.php
URLs that are not considered valid in option 2 may not be considered for indexing. It is strongly recommended that you use Option 1 and place your file key at the root directory of your web server.
NOTE: IndexNow syas: “Using IndexNow ensures that search engines are aware of your website changes. Using IndexNow does not guarantee that web pages will be crawled or indexed by search engines. It may take time for the change to reflect in search engines.”