Hacker News
new
|
past
|
comments
|
ask
|
show
|
jobs
|
submit
login
nihilum
on Dec 25, 2017
|
parent
|
context
|
favorite
| on:
Analyzing One Million Robots.txt Files
WTF with the casual racism, too, and the fact that no one in this thread mentioned that part?
Consider applying for YC's Summer 2026 batch! Applications are open till May 4
Guidelines
|
FAQ
|
Lists
|
API
|
Security
|
Legal
|
Apply to YC
|
Contact
Search:
