CrawlSpider breaks down the article. It collects all the keywords and keyphrases using NLP and scores each keywords. This is used to create a context for each page. This is then used during the link discovery phase where each candidate page is checked for relevant context and then it creating entries in the match table.
The NLP algorithm is designed to break down super long sentences into meaningful “just long enough” keyphrases. This is how it can find juicy anchor text suggestions. Check this video where it explains the process in some details https://www.crawlspider.com/docs/internal-link-builder-wordpress/link-discovery/
We have an NLP algorithm and our custom Matching algorithm. The algorithm is fine tuned and adjusted with default settings. We will use feedback from our customers to adjust these as time goes by.