A native Rust port of Google's robots.txt parser and matcher C++ library.
-
Updated
Feb 13, 2021 - Rust
#
google-robots-parser
Here is 1 public repository matching this topic...
Add this topic to your repo
To associate your repository with the google-robots-parser topic, visit your repo's landing page and select "manage topics."