All posts by Szymon Nowicki

Story, Terms changes

Free of charge uptime monitoring

Leave a comment

From now on, you can test your site completely free of charge with AgentSlug.com.

We have just added 5000 points for each of our users and we will do it every month for everyone. 5000 checks is more than enough to monitor a single website during whole month with 10 minutes interval uptime monitoring check.  And from now on, it’s all completely free of charge.

It means that everyone gets additional 5000 checks without any fees, every month, no matter if you’re a paying customer or not.

We might change a little bit the refill policy to avoid some kind of abuse in the future. However, we can promise that everyone will be able to get a single website monitoring free of charge anyway.

Enjoy!

coding tips, Story

Freezing curl causing a lot of troubles

Leave a comment

Not-so-infinite loops

Our website monitoring mechanism is actually quite simple.  We use curl to make a connection with the website and then we grab some data to check if your site is up or down (actually, curl has more power and we use it in our new features we develop right now as well).

To make sure we are able to test your site in almost equal time intervals, we need to have a couple of testing agents working at the same time.

The job of these agents is to sit and wait for available tasks for some period of time. If there are any, agent reserves one of them and does the actual website uptime test.

However, since we started the Ukrainian and Russian government websites monitoring site (lots of downtimes), we found out that some of the scripts sometimes freeze for no good reason.

It wasn’t a serious problem, because it didn’t influence the uptime monitoring service reliability. However, the zombie agents were taking our precious server resources to do nothing.

Curl has two timeout options

And then, there comes the time when we decided to fix this issue. While debugging we found out that the solution is quite trivial.

As you probably know, we assumed that if your site is not responding in 10 seconds after our call, it means that the website is down. Nobody is waiting 10 seconds for first document request anyway. Even if your server works, the user nearby our agent’s server location might not be able to notice this.

These 10 seconds of timeout are set as a curl option (CURLOPT_CONNECTTIMEOUT). It means “if the server is not responding in X seconds, do not wait more and return connection error [http 0]”.

What we didn’t notice is the other curl option – CURLOPT_TIMEOUT and it was the cause of our problems. The option is more important in this case since it forbids curl to freeze. Connection timeout is related to the connection only. If connection is made quickly, the clock for the whole script is not ticking any more.

Summary (tl;dr)

If you plan to use a curl mechanism inside a do-while loop with iteration limit, remember to set a curlopt_timeout or your loop won’t die sometimes.

 

 

coding tips

Tutorial for custom maintenance mode in WordPress

Leave a comment

Maintenance mode in WordPress

Maintenance mode is the “be right back” state of your website.  You should use it any time you are changing the WordPress code or making anything else with a non-zero possibility to fail and affect a user experience.

You’ve probably noticed that WordPress is actually turning the built-in maintenance mode while upgrading the WP core, or adding new plugins.

In this article, you’ll learn how it works, how to turn the maintenance mode on manually, and how to make it look better.

How the built-in maintenance mode works

When you click the “upgrade WordPress” button,  Wordpress is inserting the .maintenance php file into the site’s root directory with $upgrading variable set to time().

Wordpress maintenance mode function

Every single time someone enters your site, WordPress is looking for this particular file and if it exists, WordPress is comparing the $upgrading variable with the time() value. If the difference is less than 600 seconds (10 minutes), the maintenance mode function is looking for a custom maintenance.php inside wp-content directory.

If it exists, WordPress loads that file and exits the compilation process.

If not, the maintenance function compiles the built-in maintenance landing page and ends with the compilation process. What’s important, WordPress handles the maintenance mode well with a proper http code sending to a user.

Find out why the proper http code is important.

How to make a custom maintenance mode landing page?

To make a custom maintenance mode landing page, you just need to make your own php script and save it into wp-content directory as maintenance.php file. Every time your site is in the maintenance mode, WordPress will look for this file and compile it, ignoring the built-in landing page.

To make it easier for you, we’ve made an example script. You can grab it from our GitHub repository and customise it any way you like.

How to turn on the maintenance mode manually?

If you would like to turn on the maintenance mode manually, without any plugin you just need to insert the .maintenance file into your WordPress main folder.

We made the file for you, extending it with an IP white list and secret key white list support. You can grab it from our GitHub repo as well as the maintenance.php example file.

 

General, Knowledge base

Common HTTP codes list with meanings

Leave a comment

HTTP? What a heck is that?

Long story short, HTTP is an abbreviation of “Hypertext Transfer Protocol”. This protocol defines how machines like your computer or mobile phone should talk to any other machine.

Of course, there are plenty of other protocols with different purposes. You’re probably familiar with some of them like FTP, or SSH.

The main and most common purpose of HTTP is to connect a client machine with a server machine to get or post some data.

Here,  we are focused mostly on HTTP protocols. Mostly because it’s the most common protocol which we deal with on AgentSlug.com while we monitor your sites.

You can find more about http protocol in “how machines talk to each other” article.

HTTP codes list

You can find full list of http codes on the Wikipedia.  If you’re familiar with the server maintenance, you probably already know them. If not, you should check the Wikipedia list anyway.

In this article I’d like to tell you more about most common http codes, and their meanings related to AgentSlug.com service (or any other uptime monitoring service as well). The article is meant to be a quick manual for those who are not familiar with technology, but need to know the most important things for their website.

HTTP 200 OK

Your server should respond to our request with this code when everything is fine. When server sends that code to a client with a document header, it means “everything is OK, you can find the document you requested below”.

However,  AgentSlug.com treats this kind of response as a downtime anyway in one case – when you set a keyword to be found in document body and it’s not there.

We thought about this case a lot and as a result we decided to make this that way.

As far as we know, sometimes websites doesn’t respond with proper http code when they are down. For example, there are lots of buggy WordPress themes which return http 200 OK even when a site is under the maintenance mode.  We can not change the world, but we can deal with it giving you a tool to detect the downtime anyway.

Besides the buggy maintenance mode, this feature is also considered a deface detection trigger.

The other downtime case with the http 200 code is when your site is hacked (cracked) and defaced.  Most often, the cracker is going to change the entire site to get the attention. If you set a must-have keyword to your test, there’s quite good chance to detect the problem and send you a notice right away.

HTTP 500 Internal Server Error

In the perfect world we wouldn’t see this code in any production environment.  It means that something really wrong happened while compiling a website code or there is some problem with your hosting.

If you see a downtime notice from AgentSlug.com with this error, you should check your website as soon as possible.

If you’re able to check error logs on your server, you should do that as well.

However, if you can’t find any bug on your website, and downtimes are still happening from time to time – you should consider changing your hosting provider.

HTTP 503 Service Unavailable

This code is a common maintenance mode server response. It means that your site is temporary unavailable, and all visitors (including Google spiders) should check it back in a couple of minutes.

It’s important to set that code when you’re working on production environment and your work can possibly affect user experience with the site.

For example, if you’re upgrading your WordPress, the site is turning a maintenance mode for the entire upgrading process with that code.  Any user coming to your WordPress site at that exact time would see the “come back later” notice.

Why is it so important?

If your “maintenance mode” sends HTTP 200 OK code, there is possibility that in that exact time some search crawlers are going to visit your site to make a copy to its index. I bet you don’t want to get your site indexed while it’s in the maintenance mode.Miniclip.com - maintenance mode with http 200 code indexed

HTTP 0, “not available”

It’s not a real http code. You won’t find it on the Wikipedia list, and your server is not responding with that code anytime.

However we use this code as a “no response” code.

If your server is not responding as quickly as any user is willing to wait (~10–15 seconds) it means your site is down. AgentSlug.com treats this connection as failed as well and sends you a downtime notice with this code in description.

HTTP 404 Not Found

This code simply means that given URL doesn’t match any resource on the server. If you see this code on your downtime message and you’re sure that the URL you set on your test is valid, you should check your site as soon as possible.

HTTP 301, 302, 303 – redirections

All of these codes mean that the resource matches some URL but it’s on another location right now.

Some of the developers use http 302 code as a temporary redirect for maintenance mode. It’s not a good practice, since the code means something else. It’s also not an evil practice, since search crawlers won’t probably download the site with this code.

However, the same developers often redirect user with that code to some /maintenance page which sends http 200 OK code which is actually evil as devil itself (see miniclip.com example above).

403 Forbidden

If you see this code on your downtime message you should check your site immediately. It means that resource behind the given URL is not meant to be shown to public/anonymous user.

Here are some possible causes of this kind of behavior.

  • someone changed your website/app directory permissions (“chmod”) on server and your app can’t be compiled/executed
  • someone/something deleted all your app files and your server can’t find any
  • the app server stopped like the AgentSlug.com and treats it as an abusing service (firewall sends us back home)

That’s all for now. As always, any comments are welcome below. Feel free to ask for explaining any other code you’d like to have explained.

Upgrades

Secondary email, quick settings link and more

Leave a comment

New features you’ll love

Last couple of weeks we’ve been tuning up the AgentSlug.com to make it even better, faster and more reliable. We’re also preparing ourselves to do a big pivot of the app by adding some cool functionalities.

Aside of planning and wireframing the big v 2.0, we’ve already made some new features which seem to be a must-have as soon as possible.

All features listed below are available since today.

Secondary emails

After the upgrade, every website test (monitoring) can be labeled with additional, secondary (alternative) email.

When site is down (or up), AgentSlug.com will send a notification to the secondary email. Even if you set “email notifications” off in your account settings. Secondary email attribute is fully independent.

User story

I’m monitoring (personally) dozen of websites within AgentSlug.com uptime monitoring service. Not all of them are important to me nor my client. Some are just triggered to grab some data from the world.

For example, after we had deployed Ukraine-Russia cyber war monitoring I got flooded with up/down notifications from monitored websites. There were so many emails, that I was forced to turn the email notifications off to my whole account to not get flooded to death.

Now, with a secondary email feature, I still have email notifications turned off as a global setting. However, I set a secondary email to every important website.

Turn off email notifications within notification itself

If you’re a hardcore AgentSlug.com user, there is some probability that you get a lot of messages from the service.

Sometimes you’re aware of a problem with your site and don’t want to get more messages anymore.

Now, you can turn them off by clicking a “turn off” link. The link is valid up to two days and works even if you’re logged out.

Turning off the email alerts

Email notification turned off alert

Just in case you forget that you turned off email alerts, we will send you an email to remind you about that.

It would be sent no more often than once a month and only if one of your sites is down. If none of your site is down, you won’t get any message. Ever.

 

Story

Next step

Leave a comment

Feedback is crucial

AgentSlug.com is getting more and more feedback from our first users.  Most of it is positive and that makes me happy, but what’s more important to me is that some of it is really constructive and learning.

As a developer who has decided to make a tool based on “scratching own itch” problem, I can’t be sure what are others’ needs and why they are actually using the app.  Feedback is crucial. Many thanks to everyone for their effort.

Next step

From the first deployment date we are tuning the uptime monitoring module to make it better and more optimized. However, we are about to go further than that.

AgentSlug.com is an uptime monitoring tool for now, but we want to build on it a complementary automatic webmaster tool focused on website monitoring in many dimensions:

  • full content monitoring
    • dead links detection
    • html validation
    • spell checker
    • WAI problems detection within the content (like no [alt] attributes on images)
  • anomalies detection – if your homepage suddenly grows or shrinks it might be hacked (defaced)
  • real users’ load time
  • resources load time testing

That’s just a shortlist of the ideas that we are fully agreed are important for every webmaster or website owner.

General

How machines talk to each other

Leave a comment

A guide for human beings – HTTP codes

AgentSlug.com is meant to be so simple that anyone can use it without any trouble.

However, there are some kind of technical terms you should actually know to understand what’s going on with your website.

This article is not for technical people, it simplifies many complicated issues.

How does it work and why it matters?

Imagine you are looking for some company document. You know it probably exists, but you don’t really know where exactly.  You just know its name.  Let’s say it’s a Holiday Funding Request Form*

First thing you do is you call your colleague who works here longer than you and knows everyone. He tells you that you should call a guy from HR department. The HR guy redirects you to Finances. You go there, and you are told to be back in couple of minutes, because they are doing some maintenance and can’t serve you now.

After a couple of minutes you go back and get the proper form.

Machines works almost the same way. URLs are something like names.  Unified names with whole addresses.

When you type an URL and hit the enter key, your browser sends a request to a DNS server. DNS is you friend who knows everyone or knows people who know people. DNS routes your browser to a proper server by resolving the domain name to a related IP number.  Sometimes the first DNS knows the name and knows the IP. Sometimes it has to ask other DNS .

Then your request is routed to a proper server by the IP address. If the server is up and responding, it should send some response to your browser.

When the URL matches any document on server, response should contain HTTP 200 “OK” code. It means “everything is fine, here’s what you are looking for”.

If the server is down for maintenance, it should send you an HTTP 503 response. Translation of this code to the human language is “sorry, we have some issue over here, come back later”.

If the server can’t find the document, but knows where you should go to find it, it will redirect you (your browser) by sending an HTTP 301 code which means “there’s nothing here, the resources were moved to another location, you should go somewhere else”.

So, why does it matter?

Machines communicate in a human-like way. They were made by humans and they are similar to humans in many dimensions.

scuba_maintenance_modeYour server should respond properly. Imagine you are asking the finance guy for a Holiday Form. He tells you “OK, here you go” and doesn’t give it to you. It would be weird, right?

The same story is when machines speak to each other.  We noticed that some websites in the maintenance mode, send HTTP 200 code with a nice “be right back” landing page. If one of the Google indexing crawlers came there during this pseudo-maintenance mode, it would identify the landing page as a regular page and switch the indexed page with the maintenance notice.

Learn more about http codes and their meanings.

* In many EU countries, companies can participate in employees’ vacation.

Story

How it started

Leave a comment

Ad hoc script

The AgentSlug.com idea went out after we were unable to use one of the biggest uptime monitoring service. I decided that I’d write my own simple ad hoc script, hooked to cron jobs. It worked quite well, and then, as you’ve probably guessed, I thought that we can extend it to a service with nice interface and some additional useful functionalities.

The very first script did just the uptime monitoring.  There was no configuration workflow. Everything was hard-coded directly on the server.

On that foundation, during the last half year, we’ve made a nice and simple uptime monitoring service.  Now we’re trying to make it even more simple.

Because uptime monitoring service seems not to be enough, we want to go further and extend the AgentSlug.com to a complementary website monitoring service. With content monitoring and other useful stuff. We’ve got some plans, but we are still gathering some feedback from our very first users.

Here is a request for you, dear visitor.  If you use or will use AgentSlug.com, don’t hesitate to contact us and tell what you think. Any opinion, any idea is really important for us and we’d love to hear it.