Hi! In v0.7.7, robotstxt:::guess_domain("www.some-domain.com") returns www.some, also when prefixing http(s):// or suffixing /some/path/index.html. see: https://github.com/ropensci/robotstxt/pull/56
Hi! In v0.7.7, robotstxt:::guess_domain("www.some-domain.com") returns www.some, also when prefixing http(s):// or suffixing /some/path/index.html.
see: #56