- What is the purpose of a robots.txt file? It instructs search engine crawlers on which pages to avoid. - How do **onical tags help with co** management? They prevent duplicate pages from being indexed, ensuring the **inal remains recognized. - What types of co** should typically be excluded from