Extract Internal and External Links
This addon included free with ScrapeBox is able to extract all the internal, external or both internal and external links from a list of webpages. This is ideal if you want a count of the amount of links on every page of your website, or you need to extract all the outbound links for reasons such as scanning them in the ScrapeBox Malware Filter addon to ensure you are not linking to any bad pages from your website. It can also be used for things such as collecting all the internal links on your website and then creating a .xml or .html sitemap which ScrapeBox can create in seconds.
Once installed, you can load a list of domains harvested from ScrapeBox or load domains from a file, you can set how many concurrent threads you wish to use. The addon is multi-threaded and can retrieve the Internal and External links for 100’s of domains per minute. Once the results are fetched, you can export the data to a plain text file for use in ScrapeBox, other software.
Google and other search engines generally recommend not creating more than 100 links per page for usability and not linking to bad neighborhoods, this addon is useful for doing a checkup of your sites pages, it may also assist in discovering hidden links in your sites source code which are not visible when viewing the page normally in a browser.