User-agent: * Disallow: /files/ Disallow: /cgi-bin/ Disallow: /dev/ Disallow: /twurfl/ Disallow: /DADA/ Disallow: /json/ Disallow: /Mobi/