J
jamseo
New Member
Hello,
I'm working on a website that has approx 70 pages indexed. When I crawl this website using Screaming Frog, I'm getting literally tens of thousands of pages coming up in the HTML section of the crawl. A lot of these pages are redirects from URLs containing .html to the non.html version of the URL. There's also a lot of URLs containing /CSS/asests/.... etc but the 'Content Type' of these URLs is text/html; charset= UTF 8.
I'm unable to share screenshots (apologies!) but does anyone know if I should be concerned about crawl budget, etc with this amount of URLs being crawled? Any advice appreciated on how best to handle.
Thank you!
I'm working on a website that has approx 70 pages indexed. When I crawl this website using Screaming Frog, I'm getting literally tens of thousands of pages coming up in the HTML section of the crawl. A lot of these pages are redirects from URLs containing .html to the non.html version of the URL. There's also a lot of URLs containing /CSS/asests/.... etc but the 'Content Type' of these URLs is text/html; charset= UTF 8.
I'm unable to share screenshots (apologies!) but does anyone know if I should be concerned about crawl budget, etc with this amount of URLs being crawled? Any advice appreciated on how best to handle.
Thank you!