User-agent: Googlebot
User-agent: msnbot
User-agent: Bingbot
User-agent: Mediapartners-Google
User-agent: YandexBot
User-agent: YodaoBot
User-agent: YoudaoBot
User-agent: Yahoo! Slurp
User-agent: Yahoo! Slurp China
Disallow: /list/
Disallow: /api/
Disallow: /timegate/
Disallow: /timemap/
Disallow: /reconstruct/
Disallow: /memento/