Sottopop

Last Updated: November 9, 2018

Sottopop is UpContent's page requester and web crawler technology. Its purpose is to identify and download elements of web pages to afford UpContent the ability for these pages to be found by our users.

WEBMASTER INFORMATION

Sottopop Access Rates

Sottopop intelligently determines the number of requests to make to your site to ensure that requests are minimized, sporadic, and only made when new information is available to be gathered. If you feel that we are accessing your site too frequently, please submit to let us know. We will respond to all requests within two business days.

At the most, Sottopop will limit page requests to two per every five seconds per hostname - to minimize burden on your servers. Network conditions and other factors may cause Sottopop to occasionally exceed this rate, but all efforts are being made to ensure this is, indeed, rare. If Sottopop continues to request pages at a faster rate for more than two minutes, you should submit, and we will address immediately.

Sottopop will also limit the number of redirects we will follow to three oer initial URL. Following redirects may increase our request rate, temporarily.

Identifying Sottopop

Generally, you can identify Sottopop traffic by the User-Agent header in the HTTP requests that we send. This value is often logged in your server's access logs. Our User-Agent string will take the following form, where 1.0 can be any version number: Sottopop/1.0(+https://upcontent.com/robots).

You can also verify the identity of a particular server that claims to be Sottopop by doing a reverse DNS lookup on the IP address of that server.

Blocking Sottopop

Sottopop does not currently respect robots.txt files. This is a deficiency that we are aware of and which will be remedied in the next version of Sottopop. For future compatibility you may use the line User-agent: Sottopop in your robots.txt file now and it will take effect when we launch robots.txt support.

If you wish to prevent Sottopop from accessing content on your site immediately, please configure your server to respond to requests with our user agent with an HTTP 404 or 410 response. If you choose to do so, we would appreciate you contacting us using our contact form to let us know why you do not wish for your site to be accessed by Sottopop.