where are the robots

Challenge Description

Solution and Analysis

Go the web page first as given in the challenge description. There is a site which most webpages have named robots.txt which web crawlers read and check which pages of the site they are allowed to read, simply type in /robots.txt in the url to access it as shown

access the disallowed page to get the flag