Sitemap (Sitemap.xml) is special file in .xml format, stored in the root directory of the server. This is a file with information about the pages of the site that need to be indexed. Usually a sitemap is created for Yandex and Google in order to notify search robots about the pages that need to be indexed. The sitemap also checks how often updates occur, and which web documents are indexed most importantly.
Video from Yandex Webmaster:
Sitemap.xml is compiled taking into account a special syntax understandable to search engines, which will list all pages to be indexed, indicating their degree of importance, date latest update and approximate refresh rate.
There are two main files that any web project should have - robots.txt and sitemap.xml. If your project does not have them or they are not filled in correctly, then with a high degree of probability you will greatly harm your resource and do not allow it to open up to one hundred percent.
Does the sitemap.xml file affect website promotion
If you do not have a sitemap, this does not mean that search engines will not index the resource. Search robots often do a good job of crawling sites and including them in the search. But sometimes failures can occur, due to which sometimes it is not possible to find all web documents. The main reasons are:
- Sections of the site that can only be reached by making a long chain of transitions;
- Dynamic URLs.
Creating a sitemap.xml helps solve this problem.
Sitemap.xml affects SEO indirectly: it makes it easier and faster to index pages.
Sitemap in HTML format
Sitemaps are divided into 2 main types or formats: sitemap html and sitemap xml file. HTML sitemap is a page of the site, which lists the links. Usually these are links to the most important sections and pages of the site. HTML sitemap is more for people rather than robots and helps to quickly navigate the main sections of the site. For a sitemap in the form of an html page, there are serious restrictions on the number of links on one page. If there are too many links on the page, not all links may be indexed, or the sitemap page may be excluded from the search for an excessive number of links, even internal links.
In order for the html sitemap to be correctly indexed and adequately perceived by visitors, you should not place more than 100 links on the page. This is more than enough to place on the page all the sections and subsections that did not fit into the main menu.
Usually, HTML sitemap file has a tree structure, where the expanded sections and subsections are indicated. Unnecessarily cumbersome HTML sitemaps are often decorated with graphic elements, CSS-styles and supplemented with Java-script. However, the html sitemap does not have of great importance for search engines.
An HTML sitemap is not a complete sitemap. What if the site has hundreds, thousands, tens of thousands of pages? To do this, you need to place links to all pages in the sitemap in xml format.
Sitemap sitemap.txt
Another way to design a sitemap as a file can be a sitemap in txt format:
1. http://site.ru/ 2. http://site.ru/page/ 3. http://site.ru/page1/
Everything is simple. The sitemap.txt file lists all the necessary links line by line. Sitemap in txt format is a "lazy option". A similar sitemap xml limit of 50,000 links works here. However, the TXT sitemap lacks the ability to specify a date. last change and page priority.
Sitemap XML Sitemap
An XML sitemap is an xml file of the form sitemap.xml, which is usually located at the root of the site. An xml sitemap has many advantages over an html sitemap:
- Sitemap xml is a special sitemap format, which is determined by all popular search engines, such as Google and Yandex.
- You can specify up to 50,000 links in xml sitemap.
- In the sitemap xml, you can specify the relative priority and frequency of page updates.
The content of the sitemap is only a recommendation for the search robot. For example, if you set the site page to an annual update frequency, search robots will still go more often. And if you set the page refresh rate to hourly, this does not mean that robots will index the page every hour.
How to create the correct sitemap.xml
Consider how to make the correct xml-map. The following requirements must be met here:
- The file size should be no more than 10 MB;
- The map must contain no more than 50,000 links. In cases where there are more links, you can create several maps and include them in the main xml map;
- The sitemap address should be written in robots.txt;
- Also upload the sitemap to Yandex and Google (how to add a file is described below);
- Search engines must have access to the map. It is necessary to use special tags that let search engines know that this is a map, and not something else;
- The sitemap must be UTF-8 encoded.
The content of the sitemap.xml file looks like:
Where the following required tags are used:
- parent tag, it contains all url-addresses; - a tag that contains information about a specific url-address; https://bar812.ru/en/- in this tag, the url is indicated directly.
- this tag contains the date of the last modification of the page; - the tag is used to indicate how often the page changes: always, hourly, daily, weekly, monthly, yearly, never; Indicates the priority of a particular page, relative to other pages on the site, from 0.1 - low priority, to 1 - high priority.
Changefreq indicates how often the page changes:
- Hourly - update hourly;
- Always - always updated;
- Weekly - updated once a week;
- Daily - updates occur daily;
- Monthly - updates occur once a month;
- Yearly - once a year;
- Never - does not update (it is better not to use such a value).
Priority tells search engines how important a page is compared to others. The priority can be set from 0.1 (low) to 1 (high).
The sitemap.xml file must contain an indication of the XML language namespace:
Xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"
If the sitemap file contains more than 50 thousand links or the sitemap.xml size exceeds 10 mb, it is recommended to split the sitemap into several files. In this case, in the sitemap, you must specify several links to different map files.
The tags already familiar to us are used here.
- parent tag, which contains the addresses of all sitemaps; - a tag that contains the parameters for each sitemap.
How to create Sitemap.xml
Create a sitemap an important process in which it is necessary to clearly indicate which pages of the site should be indexed and how best to index them. Depending on what type of site map we are talking about, different how to create a sitemap. There is no point in discussing how to create an html sitemap separately. Consider how to make a map map in xml file format. There are several basic ways to create a sitemap, but they all have in common where the sitemap is located and how the sitemap is determined by search engines.
As already written above - sitemap file is placed in the root of the site. Search engines are able to find the sitemap file on their own. But there are several ways to directly link to the sitemap file(s) for faster discovery by search engines. The easiest way to specify the location of the sitemap file is to directly specify a link or several links to sitemap files in Yandex and Google webmaster tools. There you can check sitemap, spend analysis of the site map for correctness, matching which pages from the sitemap are found by the search engine and how many of them are indexed.
The second way to tell search engines the location of the sitemap file is sitemap directive robots.txt file.
Sitemap: http://site.ru/sitemap.xml
You can specify several sitemap files in robots.txt, after which it will be automatically added to webmaster tools. We looked at how to find a sitemap, now let's move on to how to create a sitemap.
Basic ways to create a sitemap
- Sitemap generation site management system, if the CMS has a similar built-in capability.
- Download sitemap from outside online service. There are many online sitemap generators with different capabilities and limitations. Probably one of the most famous online generators sitemaps is Sitemap Generator . It has a fairly large functionality and will allow you to generate a sitemap for 1500 pages for free, which is quite a lot. There is also xml-sitemaps.com, which has the ability to configure sitemap parameters, but has a limit of 500 links in the sitemap.xml file.
- Download Sitemap Generator. Such generator programs are usually paid, but with their help you can regularly generate an xml sitemap for one or more sites. Here are a couple of examples of such generator programs: SiteMap XML Dynamic SiteMap Generator, WonderWebWare SiteMap Generator.
- Automatic creation sitemap sitemap in Joomla (Jumla), WordPress (Wordpress), Bitrix (Bitrix), ModX.
- Creating a Sitemap Manually.
WordPress Sitemap
You can create a sitemap for WordPress using the Google XML Sitemaps plugin. For it, you can make a lot of settings that will allow you to exclude some of the materials on your site, and you can also set the expected refresh rate. In addition to creating a map, Google plugin XML Sitemaps notifies many search engines when new content is published on your blog, inviting them to quickly index it.
You can set the path to the sitemap file yourself in the plugin settings and you can even give it a name different from the classic sitemap.xml.
Joomla Sitemap
You can create a sitemap for Joomla using the Xmap component.
Check Sitemap for broken links
In order not to deceive the search robot, the sitemap.xml must be configured without errors. Therefore, after each file update, you need to check the sitemap for broken links.
Go to Yandex Webmaster - section "Tools" - "Analysis of sitemap files".
Choose one of the file upload methods:
- copy text sitmepa.xml;
- submit the sitemap URL;
- upload xml file to service.
Checking sitemap.xml in Yandex Webmaster
Sitemap.xml analysis by PixelPlus
If errors are found (this happens often), the service will tell you which URLs give an incorrect response (removed, unnecessary, and so on).
Sitemap.xml is an important tool
A sitemap is one of the most important SEO tools for website promotion. It doesn't matter how the sitemap is created. What matters is which links will be listed in the sitemap and how often it will be updated. Sometimes, everything is uploaded to the sitemap, and even those links that are prohibited in robots.txt or non-canonical links. A sitemap is updated once a month or less. Such an attitude to the site map can not only make it useless, but even worse - confuse the search robot, which will negatively affect the indexing and site positions in the search.
Create a sitemap for your resource. But be careful and think carefully about what to upload to the sitemap and what not.
Create sitemap.xml
Page limit: 1200
Price: Is free!
An approach: Professional!
The main purpose of sitemap.xml is to tell search engines which pages need to be indexed first. Some do not know about this role of the sitemap, while others simply ignore it and do not attach any special importance to it. A file that has the more common name is considered important tool, one might say, the main one, in order to indicate to search engines which pages of the resource need to be indexed, although visitors to your project will not see the file itself.
Sitemap contains important information, which is embedded in it, for which special syntax is used. This lists pages for search engines, and also indicates what their degree of importance how often and when they were last updated. All this is incomprehensible and is not necessary for either visitors or the owner of the site. This is only for search engines.
Without this file, or if it is not filled out correctly, the site owner risks that his resource may be damaged or the site will not be able to perform to its full potential. Pages that are important may not be indexed for a long time, and this is reflected in the visit to the site, and, accordingly, in its rating. This can be confirmed by specialists who know the statistics of many projects whose creators neglected sitemap.xml. One revealing and interesting study uncovered the most popular reasons why projects are moving very slowly. If the sitemap, which has an xml extension, does not match desired format or just missing, is the reason for slow promotion, which is the second of ten reasons behind low-quality content. These facts can convince of the importance of the card more than mere words.
To be sure that the project is correctly indexed by search engines, creating a sitemap file is not enough. You can have it, but search engines may not see it. It is necessary that the search engines know about its existence. To do this, you need to create it correctly and choose exactly those pages that are considered the most important. For a correct search, you need to make certain settings that will allow you not to include all the material on the site in the search, so as not to slow down the robot.
This tool can help. In addition to helping to create a map, this tool will notify search engines about the appearance of new material, thereby making it clear that they need to be indexed as soon as possible. To hide the map from prying eyes, you can even change its name using the tool.
Where the sitemap.xml is located can be indicated to search engines in two ways. In the directive, without omitting anything, specify the full path to the file.
Which are needed for search robots. Someone will say that it is not needed, because all sections are already displayed. However, the need for such a page exists if the site contains fifty pages or more. For search engines and for users, it will serve as a guide to help you understand where this or that information is contained.
XML and HTML files
Since it is used not only for search robots, but also for users visiting the site, they usually make up two maps: in XML and HTML formats.
To create a Sitemap for search robots, an XML file is used. Thanks to it, robots enter new ones into their search base. In the absence of a map on a multi-page site, a large number of pages may not be indexed for a sometimes very long time.
An HTML file is used to create a sitemap for users. The importance of this map lies in the fact that its convenience directly determines whether the user will find the information of interest to him or not. Therefore, such a map is created for those Internet projects in which all sections and their subsections do not fit in the main menu.
How to Create an XML Sitemap
There are three ways to solve this problem:
Buying a sitemap generator.
Create a Sitemap using online services.
Hand writing a file.
To significantly save time, it is proposed to purchase generators. Therefore, if twenty to thirty dollars to buy a license is a small waste of money for a webmaster, then it won’t hurt to buy it, especially for a large Internet resource, since then you won’t need to create a site manually.
For a site containing several hundred pages, online services are recommended, where in order to create a Sitemap, you only need to specify the address of the Internet resource and download the result.
The surest option is to manually create a map. To do this, you need to know tags such as url, urlset, loc, lastmod, changefreg and priority. In this case, the first three tags are considered mandatory, and the last three can be dispensed with.
Creating a Sitemap in Joomla
To create a Sitemap on a site, Joomla and Wordpress have special add-ons, like most well-known administration systems, thanks to which a sitemap is created manually or automatically. For large Internet projects that constantly update materials, this add-on is very convenient.
In Joomla it's called Xmap, in Wordpress it's called Google XML Sitemaps.
Automatic sitemap generation
Free online servers help to automatically create a Sitemap if the site has no more than five hundred pages. Here's how easy it is to generate a sitemap:
Having entered one of these Internet resources, you need to find the item "Generate Sitemap", click on the "Create" button and create a Sitemap file automatically.
Find the “Site URL” and enter the address of the site for which the map is being created.
Perhaps the system will require the introduction verification code. It must also be entered and click "Start".
Upload the finished map to the site.
Manual way to create a map
This method is, on the one hand, the most difficult and time-consuming, but on the other hand, it is the surest method used in cases where other options are not suitable. So, for example, if there are many pages that do not need to be included in the sitemap, but they automatically get there, of course, the manual method will save the map from “overdose” of such pages. Another reason for choosing this method is poor site navigation.
To implement manual creation of a map, you must:
Collect pages to include in the map.
Insert all addresses into the third column in the excel file.
In the 1st and 2nd columns, insert both url and loc.
In the 4th and 5th columns, insert the closing url and loc.
Apply the "link" function to connect five columns.
In make a sitemap.xml.
Add both urlset and /urlset tags to this file.
Insert a connected column between them.
The resulting file must be checked. You can do this, for example, in Yandex, in the webmaster panel.
How to create a Sitemap for Yandex and Google
After the site is created, it is added to the site. For this purpose, the sitemap file should be named Sitemap.xml and added to the root directory. To quickly find it, Google and Yandex have special tools. They are called Webmaster Tools (in Google) and Yandex Webmaster (in Yandex).
Adding Sitemap to Google
Adding Sitemap to "Yandex"
Similarly, in "Yandex Webmaster" you must first log in. Then go to indexing / Sitemap files, specify the file path there and click the "Add" button.
Search robots today will take only those files that contain no more than fifty thousand urls.
If the map exceeds ten megabytes, it is better to split it into several files. Thanks to this, the server will not be overloaded.
To create a Sitemap xml correctly, with several files, you need to register them all in the index file using the sitemapindex, sitemap, loc and lastmod tags.
All pages must be written either with the "www" prefix or without using it.
The required file encoding is UTF8.
You also need to add an indication of the language namespace in the file.
How to create a sitemap for users
Since such a map is created for users, it should be as simple and visual as possible. Despite this, it is necessary to qualitatively convey all the information about the structure of the site used.
HTML maps generally have a familiar user structure, consisting of sections and subsections set off in a certain way, for example, css styles and graphic elements.
To create a Sitemap for a large Internet project, as with an XML map, splitting is also recommended here. In this case, it is carried out in the form of separate tabs, relieving the map from bulkiness.
Increase the functionality of the page will allow JavaScript language, which is allowed to be used in this map, since it is created not for search engine robots, but for users.
Order for sitemap file
It is desirable that the created file containing the Sitemap is always clean and tidy, especially if the site has a large number of pages. Since search engine robots scan sitemaps very quickly, there may simply not be enough time to view the entire file of a large Internet resource.
Therefore, if you get used to adding pages to the sitemap not at the bottom, but at the top of it, then, on the one hand, there is no doubt that the search robot will have time to look at the addresses of new pages, and on the other hand, it will be much easier to control all pages.
You need to register to create a sitemap.
There are two types of sitemaps: sitemap. xml and sitemap. html.
Xml sitemap (sitemap.xml)
XML sitemap is created for search robots. Using the xml map, you can specify how often this or that page should be indexed, how often it is updated, how important it is within your site. A sitemap captures all the links on a site.
In other words, the site structure becomes transparent to search robots. For "live" users xml map is useless.
If you only need an xml map, then it can be easily generated automatically using one of the many Internet resources. On most of them, creating a map for a site up to 100 pages is free. The generation process takes a certain time: the more pages the site for which you generate the site-map, the longer this process. It can take from a few minutes (for small sites) to an hour or more (for sites with hundreds of pages and numerous cross-links).
Necessary remember that for any changes that affect the structure of the site or the priority of pages, it is necessary to re-generate the sitemap.
In order for the search robot to find your sitemap.xml, you must specify the path to it in the robots.txt file.
Html sitemap (sitemap.html)
Html sitemap is created for visitors to your site, it is similar to the table of contents in a book. It often happens that with a complex or not very transparent structure, the visitor gets lost in navigating the site and does not reach the pages (he simply does not find them) that he would need and leaves the site unsatisfied. As a result, website conversions decrease.
If the html map exists and is made with high quality, the sections are correctly and capaciously titled, then any section, any page of the site of the desired subject will be easily found by the visitor of your site.
Sitemap creation online on our website
Using the proposed module you can create sitemap.xml and sitemap.html at the same time.
In this case, you create only an html site map, and an xml map is created automatically.
Changes that are made to the site structure after the sitemap has been created do not require a new generation of sitemap.xml, the xml map is corrected automatically after the html map is corrected. Thus, your site will always have a correct map for search engines (sitemap.xml) and a correct table of contents (sitemap.html).
Comparison of automatic sitemap generation and semi-automatic
- customizable appearance and css styles.
- Customizable tree system.
- Full customization under search engines.
- All entered data is saved.
- Correcting the finished sitemap will take no more than a minute.
- Intuitive interface.
- Automatic check page availability (eliminates errors and duplicates).
- The program reads the title of the page on its own.
- The html map automatically adds a page title to each link.
- To add to the site, just copy the code or download the file.
- Simultaneous creating html and xml sitemap.
File Sitemap- this is a file with information about the pages of the site, otherwise called site `s map. With a Sitemap file, you can tell the search engine which pages of your site to index, how often information on the pages is updated, and which pages are most important to index.
Here you can generate the required Google and Yandex XML code for the sitemap. And also create a visual site map online, which you can place on one of your pages.
You can read more about the XML format of a Sitemap file at sitemaps.org
Beginner Expert VIP Info XML Structure History FAQ
Sitemap technical details
Sitemap.xml file tags:
- changefreq- the most likely frequency of change of the content of the specified URL;
- lastmod- the actual date of the last update of the URL;
- loc- the URL itself;
- priority- the priority of the specified page in relation to other pages of the site;
- url- this tag serves as a parent for the specified first four tags;
- urlset- this tag combines tags of the first five types.
A feature of using the XML format is the need to replace all service characters with their safe representation. Below is a table of characters that occur in the URL and are subject to mandatory replacement:
- & - ampersand;
- " - single quotes;
- " - double quotes;
- > - sign "greater than";
- < - "less than" sign.
The sitemap has a 10 megabyte index file size limit. In order for this limitation not to affect the owners of small websites, it is possible to compress files that are intended for Sitemap using the open gz archive format (gzip archiver).
You can specify the location of the file Sitemap using file robots.txt. To do this, simply add to the file robots.txt next line:
Sitemap: http://DOMAIN.ru/sitemap.xml
The URL must be the full URL for the Sitemap file.
To provide search engines with new Sitemap use a line like:
Search engine/ping?sitemap=http://DOMEN.ru/sitemap.xml
For Google: http://www.google.com/webmasters/sitemaps/ping?sitemap=http://domain.ru/sitemap.xml
To check the correctness of the map of your site, you should first check it for correctness. The easiest way to do this is with one of the utilities located on the websites or xml.org . In this case, you will need the XML data schema of the sitemap index file. To indicate in XML file check the checkbox for the link to the verification scheme Add reference to XSD schema for validation". After that, when you open the Sitemap.xml file in the browser, if there are errors in the structure, you will receive a corresponding message.
Sitemap development history
For as long as the Internet can remember itself, webmasters have always tried to please search spiders - to help them collect detailed information about the structure of the site being promoted.
The first, most ancient and quite natural way was to create a special page on which all the links of the site were placed. Such pages were called and are called sitemaps. The principle of their work is simple: when a search robot visits this page, it will automatically follow the links published there and add them to the index.
Such a scheme was convenient for the time being. Later, when sites ceased to be exclusively static pages with a small number of pages, new problem associated with the inefficiency of the method of publishing the classic version of the site map, because there were not enough places for links, the pages turned out to be large in size, uncomfortable.
On some sites, the number of pages has exceeded 1,000, naturally, it has become impossible to place absolutely all links on the site map. The indexing index of such sites began to decline. The world needed a new effective solution to the problem that had arisen.
And the problem was solved. In June 2005, the company Google a new convenient format for describing new pages of the site was created to improve the quality of their indexing by search engines.
The standard is called Sitemaps 0.84. The principle was adopted text base XML data that allows you to compactly describe the site structure in a text file, including new and previously published site pages with the ability to specify additional META information. Effective Solution Sitemaps proposed Google became wildly popular.
The popularity also affected the opinion of other search engines to a new approach to creating sitemaps. So in November 2006, such search giants as msn and Yahoo, and in April and Ask.com. To celebrate the format Sitemaps received new version Sitemap 0.90 expanding functionality.
In May 2007, support for connecting the standard was officially announced. Sitemaps in file robots.txt search engines Ask, Google and Yahoo, i.e. find the sitemap Sitemap became easier: it is enough to indicate its location in this file, and for the source of information for Sitemap use RSS feeds or plain text files.
June 2, 2008, 3 years after the announcement of the protocol Google a message about the support of this protocol appeared on the Yandex blog for webmasters Yandex ohm.
How to make sure that your site always has a fresh map? For this, a free sitemap maintenance service has been created.
You can also purchase a ready-made universal