Introduction

This article is the first in a series focussed on Statistics, Logging and Graphing web application events and information

When creating a web application it is important to keep track of everything a user does. Some people may think that this is a little over the top but the more information we can gather within a web application the better. It also allows simple and easy moderation practices for those that are moderators within the application.

Pre-Planning

This is by far the most important stage in development. During this stage you are going to need to create proper data structures that in theory will not have to be change (although this is extremely rare.) This means that you are going to create the general database that will handle most of the current up to date information and another section that will be used for logging.

Within the main database we are going to have simple things like a user table, a user settings table, a user profile table and a user information table. On tables that hold general information it is important to add a time stamp column that holds the value of last modified. The user should have a column that tells us when the users profile was created. These values aren’t going to really help us when gathering information for a single user simply because the sample of data that we can compare it to is so small. Fortunately these simple values are excellent when we need to pull up values quickly in our system to display user information.

Single User Sample

The logging database is going to help us create a large sample of a single user. This is where we are going to constantly insert information and never delete or modify the records. In one of these tables we can store required values that will tell us when a user has had one of their comments deleted by a moderator. Within this table we are going to log the moderator’s id, the user’s id, the time stamp that this action was performed on. and a numeric value that reflects on the reason that the comment was removed. With this data we can then count the comments that are still alive and well and count the comments that were deleted. With this information we can divide the comments that are still alive and well by the comments that were deleted thus giving us a ratio that we can work with.

Note: It is a good idea to keep track of numbers like this as well and when the ratio was calculated. This will allow us to track user behaviour.

Multiple User Sample

Given that we could have a ratio of how well the user follows rules when it comes to commenting on things and taking part in discussion we filter out the users that are not contributing to the community/application. We can also sort users by this ratio and from there bury into their statistics, activities and logs.

Because we have logged all of the individual comments from this user we can closely examine the user interactions by graphing comments, removed comment, comments removed for x

Related posts:

1. Redundant Data Class in PHP {Theory}
2. Password Recovery {Theory}
3. Creating a Single Page login – Design View

Password Recovery is a must have in any web application and you as a software engineer need to make sure that you handle this process properly. There are two methods that I like to use but in this article I will only be using one of them. Before you start programming it is a good idea to go through some of the larger web applications to see what they are doing. You may want to modify your process.

Before we begin I should probably make sure that you know just what you are in for before trying this. You are going to need to have knowledge of a server side language, the ability to send mail on your server, store Cookies and Sessions, along with having a running database with access to user emails as a unique field in the database.

Stage 1

The password recovery system is going to first require the user to input their email and submit it for checking. On this processing step we will do the following.
1. Check that the email follows proper formatting and there are no bad characters.
2. Check the email against the database to make sure that the user does exist and they have an active account.
3. Check the password recovery table to make sure that the user has not had their password reset in the last 15 minutes.

Now that we have checked and bypassed anything that will put a large hold on the password recovery process we can move on to actually storing the information and content that is required to reset the users password.

Because we are going to be resetting the users password we need to make sure that the user supplied us with a proper email. Even if they did not we are going to show a success screen saying that an email has been sent. This way a bot that is entering in randomness to get emails will not be able to find them this way since everything will return true.

Note: Brute Force should be check for on all forum submissions and thus a lock out system should be added but that is a whole other article.

We now need to generate the items that are going to be stored. Because we will be storing information in a cookie and inside of a session. I know guys that store two keys but I just store a time stamp in the database, break it up into segments, add alpahnumeric characters and hash the information. This way I can keep a time stamp in the database and keep the information secretly stored in a session specific to the user.

Note: When storing time only use the generating function once and store the value into a variable so we have the same time when committing to the database.

Now that we have the information set into the appropriate variables we can store the information via insert to a table that holds the user id, key and time stamp. This way we can track our two keys and a time stamp of when they tried to reset their password. If we wish to lock the user out of the system for a certain amount of time we can simply change the key to null and check for this. If the (current time – date stored) < (15 * 60) and there is no key then we can just display an error message.

When everything is properly stored into the database we can send the user an email. This email will contain a link to the password recovery page along with a key which will be the one saved in the cookie. This way we can pass two keys back and know that the only way for this user to be the wrong user is if they have the email account as well.

Stage 2

This page is going to check for a key that is alphanumeric and of a certain length. With this key we are going to check to make sure it is the same as the cookie and that it is in the database. If the user is in the database we can pull the rest of the information including their email from the other table to make sure that we are able to regenerate one of their keys assuming that we used it.

Finally we need to make sure that the generated key matches the one that is stored inside of the session and the date is within the allotted time. From here we need to first delete the record in the database, generate a random password, hash it and store it into the system, and send them a copy of this new password via email.

During this last portion you need to handle the case that the page was accessed with improper values or missing values. eg: if the user does not have the cookie set or the time is out of range we need to handle the case and remove the key from the database so we know that that user won’t be able to try and reset their password for another 15 minutes. You can also bump up the time that is stored in the database depending on what was missing in case of a bad attempt which will allow you to easily lock a user out.

Related posts:

1. Redundant Data Class in PHP {Theory}
2. SQL – An introduction to Keys
3. Password Protecting Your Directory With .htaccess & .htpasswd

It is time for web developers and software engineers to make a new approach when checking the validity of URLs and emails that are provided by users. Icann has decided that it is going to allow new suffix’ and the ability to host a website on a domain that has no suffix ie: ‘http://codewithdesign/index.php’. With this change in URLs means that all regular expressions are going to have to not force a suffix or that last decimal as a root URL.

How Will The New Domains Affect Me?

The new domain scheme is going to change how your website validates proper emails and URLs which can be a rather large change on websites that do not have formatting called from one place. Because of this change you will have to overhaul your website/blog/app to support new URLs and emails.

How Should I Go About Making The Change

If you haven’t done so already it is a good idea to make sure that your formatting and checking is coming from one place and will be able to handle the errors accordingly. The first thing that is required is either a functions file or a class file that will support multiple formats of input. This way when something like this changes you only need to update the code once.

Ways To Check URLs And Emails

When working with a URL you can change your regular expression to just check for proper characters and the presence of ‘http://’ or ‘https://’, but there is a much more fun way to check for the new format as well. You should still be using a filter or a regular expression but make sure that your version of PHP is high enough if you are going to use a filter.

Check For An Existing Email

Because of the new URL scheme we are going to need to handle the case that the user is from a website such as ‘http://bearattacksarenotgood/’ This will require us to check for an existing email without the checking of a proper suffix in the email since a user can have the email ‘calebjonasson@bearattacksarenotgood’.

We can easily check for an existing email by using a function called ‘checkdnsrr’ but first we need to split the email name from the URL which will give us ‘calebjonasson’ and ‘bearattacksarenotgood’ which we can accomplish by using the list function will will break apart a string by finding a character.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

<?php
//load a variable with an email.
$emailAddress = 'calebjonasson@bearattacksarenotgood';
//split the email by the user name and domain name
list($userName, $domainName) = split('@', $emailAddress);
//use checkdnsrr to validate the domain names existence.
if(checkdnsrr($domainName, 'MX'))
{
    //return value on success.
    return true;
}else{
    //return value on failure.
    return false;
}
 
?>

Pinging A Server

It is possible to ping a server through PHP and it is also possible to just use a cURL to request the page information and check and the response headers. The first thing I will show you is how to send a request which will allow you to check for ping of the data server in question.

My prefered method of doing so is to install net ping onto the server. This is the best solution that I have found on the web to this day and is very simple to use. Here is a tutorial from Code Diesel.

Related posts:

1. Password Recovery {Theory}
2. Check Email Function PHP
3. .htaccess Handling Bad Server Requests

As a server side programmer you are going to be spending a lot of time working with data that is pulled from the database and in order to make sure that you do not have any loose ends it is a good idea to handle this situation with a class. This article is not going to cover how to code a data class nor is it going to help you create a specific data class. This article is here to show you how to make your data redundant and error proof.

Pulling all of your information

When pulling the information off the get go you are going to want to do one thing and one thing only: query all of the data that you can use in the class. The query is going to take place inside of a function and will most likely be used every time that you are in need of the data in the database. Once the query to obtain this information has taken place then you are going to either have results or no results and the easy way to check is simply by checking the number of rows returned.

Say we were trying to access user information for a user page. It would be a good idea to query and make sure that all of the information is available. In this query we are most likely going to be joining up to multiple tables because having all of the user information stored in one table would create a fair amount of overhead and would increase the time it took to go through the table. For this reason we break things apart.

An Example Of Database Tables

A well structured database table is going to be pulling from an auto incrementing ID. This auto incrementing id is going to be found on every table but since we can not just assume that if we insert a record into tbl_user and tbl_user_profile at the same time we are going to get ID’s that match properly. Because this is a flawed way of looking at the table set up we are going to use the auto incremental ID from the tbl_user and plug this into a new column on the other user tables. This way we can simply join the tbl_user.user_id to tbl_user_profile.user_nid. Now that we have a relationship in the database that will work and is pretty redundant in theory we are still going to have to deal with the situation of a table not being created properly upon user registration and thus we are lead to…

Enforcing Existing Tables

Remember back up at the top when I was talking about the query all function either working or not working? Well this function upon returning zero rows will tell us one of two things. (Assuming the SQL was written properly.) The first thing that it will tell us is that this user does not exist, and the second thing is that the user is missing a table. This could mean that something was deleted, or maybe the table was recovered in backup at a later date and there was a record missing. Either way we know that by checking the user table for the users existence we are able to confirm that we are dealing with missing data and this is where the checking functions come into place.

In a well written data handling class there will be functions that exist to pull results from each of the individual tables. These functions are excellent for checking that a user does exist in the following table and are a great way to quickly and easily pull sections of content based on the user which are handy when loading content on an interval via AJAX. But now we are getting a little off topic. Creating a function that can pull information and return a number as a status and creating another function that will insert defaults into the table is an excellent way to make sure that you do not lose data and that all of the data is being pulled properly.

Recap

When using the class you are most likely just going to need to get the information which means that you will be using a function that behaves like the queryAll that I was talking about. If the user does not exist then you can simply return false.

The next step is to create functions that will check individual tables based on that original tbl_user.user_id (or whatever yours is) starting with the initial tbl_user which the rest of the tables are based on. This will tell you if it was created in the first place or not.

If we have gotten far enough to know that the user does exist and the user simply doesn’t have a record in one of the tables then we can query each of the tables and find out where the record is missing. Now that we know the table we can simply insert blank data into the table and maybe send a notification to the user that they may need to update a certain part of their profile.

Knowing when we have an error in table creation

Through this data class it is just a matter of adding in an error log message to let the administrators tell if the application has a bug in it but this shouldn’t be a problem if you are handling proper inserts and updates through SQL and checking for an affected row upon creation.

Related posts:

1. Password Recovery {Theory}
2. Create Your Own PHP/AMF Data Service For Flex
3. Statistics – Simple Logging Design

The other day at work we ran into an issue where the server would return a 403 error page when retrieving page information from a cURL call. After searching around the web for a while thinking that we had a server permission issue on our hands it ended up just being a PHP problem.

In order to make a cURL request from your own server you must first make sure that the session has been destroyed prior to and cURL commands. This is because your server cannot have two pages that can access sessions up at the same time and the primary file that you are working from is going to lock the secondary file that you are trying to bring in.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

<?php
session_start();
 
//authentication code.
 
//destroy session first.
session_destroy();
 
//cURL code.
$ch_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, $fields_string);
$response = curl_exec($ch)
curl_close($ch);
 
//now we can start the session again
session_start();
 
//do some other stuff...

My only guess as to why PHP does this is to protect itself from breaking sessions with multiple page access which is a pretty good security issue to have in place. Better error reporting would have been nice though.

Related posts:

1. Stop Direct Page Access With PHP
2. PHP: Increasing a scripts runtime

When creating applications that are going to be used by hundreds of thousands; it is important to make sure that you have the proper facilities in place to handle curse words that are entered in by users. This can be done by checking an array of bad words.

The code is simply…

1
2
3
4
5

<?php
//foul language array
$this->badWords = array('word1', 'word2','word3');
//Now you just need to go through your string and make comparisons.
?>

Rather then posting the code directly onto the blog I would rather have a site that is safe for all readers and not be indexed with foul language and racial slurs and thus is why I am offering the array via a text file within a compressed zip.

Related posts:

1. Looping with an array
2. .htaccess Handling Bad Server Requests
3. Creating a Multi-Page Site From One Page Using GET

Pulling In Content

Php has a nice small feature that allows us to automatically load classes that we use in our web application. This will check the designated location’s files and pull in any classes that we reference inside of the application.

function __autoload($class) {
    include $class . '.php';
}
 
$car = new carObject();
$house = new houseObject();
$boat = new boatObject();

Pull Classes From A Path

It is nice to keep all application classes in the same directory. Here is a quick fix to the above code to import everything from the classes folder on the server.

function __autoload($class) {
    include '/class/'.$class . '.php';
}
 
$car = new carObject();
$house = new houseObject();
$boat = new boatObject();

This is a small quick fix that works because the include function works with a simple string.

Related posts:

1. Redundant Data Class in PHP {Theory}
2. Create Your Own PHP/AMF Data Service For Flex
3. Random Images Using A PHP Function: cj_random_image

Introduction

The Mysqli class is a great help to working towards proper security when sending and pulling information from the database. It can stop injection attacks in their tracks and will allow you to specify data types allowing you to not let anything bad slip into the system.

In this article I will be going through the step by step approach to creating a query using prepared statements through MySQL. We are going to follow best practices and use the object oriented style to achieve a connection and gather our results.

Connecting to the database

The first thing that we will be doing is connecting to the database. Traditionally this should be done using an outside class to prevent an access to the connection information. The connection string will attempt to connect to the database and open up a persistent connection.

1

$this->connection = new mysqli('localhost','username','password','database');

Creating the prepared statement

The next thing that is required is for us to store the prepared statement. This should be attempted inside of an if statement. This way we can handle any errors appropriately.

1
2
3
4
5
6
7
8
9
10

$sql = "SELECT id, name, year FROM tbl_books WHERE name = ? AND year = ? LIMIT ?";
 
//store the prepared statement into the $stmt variable.
//note that the $stmt is a common naming for prepared statements.
if($stmt = $this->connection->prepare($sql))
{
 
}else{
	die(trigger_error($this->connection->error));
}

Wild cards

You probably noticed that the sql code has something new inside of it. This is the wild card for the sql statement and a ‘?’ is used to display it. This will allow us to bind variables in place of these questions marks and specify their data type.

Bind parameters

Bind parameters is the next step that takes place when there are wild cards inside of the sql statement. These allow us to force a data type into the format of our choosing. There are 4 data types that we can use.

‘I’ Integer
‘s’ String
‘d’ Decimal
‘b’ Blob

1
2
3
4
5
6
7
8
9
10
11

$sql = "SELECT id, name, year FROM tbl_books WHERE name = ? AND year = ? LIMIT ?";
 
//store the prepared statement into the $stmt variable.
//note that the $stmt is a common naming for prepared statements.
if($stmt = $this->connection->prepare($sql))
{
	//Now bind the data to the types.
	$stmt->bind_param('sii', $bookName, $bookYear, $limit);
}else{
	die(trigger_error($this->connection->error));
}

The above function that we just used will take a minimum of two arguments, the first to define the variable types and everything after that is a variable. Notice that all of the type declarations are in the same string. A common error for this piece of code is that you have an invalid amount of arguments being passed. Another error commonly given is that you cannot have data placed directly in the string. You need to reference variables.

1
2

//This is not allowed:
$stmt->bind_param('sii',"the php book", 4, 20);

Execution

Once we have properly set up our bindings we can execute the prepared statement and try to get a result from the database. As you can see in the code below; this can fail and an error message will be stored in case it does. This is why we are going to have it placed inside of an if statement.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

$sql = "SELECT id, name, year FROM tbl_books WHERE name = ? AND year = ? LIMIT ?";
 
//store the prepared statement into the $stmt variable.
//note that the $stmt is a common naming for prepared statements.
if($stmt = $this->connection->prepare($sql))
{
	//Now bind the data to the types.
	$stmt->bind_param('sii', $bookName, $bookYear, $limit);
	//We will now execute the statement. This will throw an error if unsuccessful.
	if($stmt->execute())
	{
 
	}else{
		die(trigger_error($stmt->error));
	}
}else{
	die(trigger_error($this->connection->error));
}

Number of Rows

The next step in our quest for world domination is to store the results inside of the statement variable. This is going to let us make a call for the number of rows so we can handle the case of no rows being returned.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60

$sql = "SELECT id, name, year FROM tbl_books WHERE name = ? AND year = ? LIMIT ?";
 
//store the prepared statement into the $stmt variable.
//note that the $stmt is a common naming for prepared statements.
if($stmt = $this->connection->prepare($sql))
{
	//Now bind the data to the types.
	$stmt->bind_param('sii', $bookName, $bookYear, $limit);
	//We will now execute the statement. This will throw an error if unsuccessful.
	if($stmt->execute())
	{
		//check the number of rows.
		$stmt->store_result();
		if($stmt->num_rows >= 1)
		{
 
		}else{
			echo 'Nothing was returned from the database.';
		}
	}else{
		die(trigger_error($stmt->error));
	}
}else{
	die(trigger_error($this->connection->error));
}
<pre>
<h2>Binding Results to Variables</h2>
The last thing that is required is binding the results and looping through the results. As you can see we are actually creating variables with this function and storing the information into them. The variables created are gathered from the select part of the statement. This means that we need to define the variables in the same order as the sql statement.
<pre lang="PHP" line="1">
$sql = "SELECT id, name, year FROM tbl_books WHERE name = ? AND year = ? LIMIT ?";
 
//store the prepared statement into the $stmt variable.
//note that the $stmt is a common naming for prepared statements.
if($stmt = $this->connection->prepare($sql))
{
	//Now bind the data to the types.
	$stmt->bind_param('sii', $bookName, $bookYear, $limit);
	//We will now execute the statement. This will throw an error if unsuccessful.
	if($stmt->execute())
	{
		//check the number of rows.
		$stmt->store_result();
		if($stmt->num_rows >= 1)
		{
			//bind the results
			$stmt->bind_result($id, $name, $year);
			//loop through the results.
			while($stmt->fetch())
			{
				echo "$name: $year<br>";
			}
		}else{
			echo 'Nothing was returned from the database.';
		}
	}else{
		die(trigger_error($stmt->error));
	}
}else{
	die(trigger_error($this->connection->error));
}

It is now time to run your code and gather the results using the fetch function. If you only have to return one row from the database then you can just use the fetch function outside of the while loop and it will allow you to pull the results and use the variables in the same way you would in the while loop.

Related posts:

1. Searching With %?% In a Prepared Statement
2. Create Your Own PHP/AMF Data Service For Flex
3. SQL: One To Many Using Inner Join

A while back when working on a project I learned that when using the wild card: ‘%’ in SQL with a prepared statement you are going to get an error message. This is because the prepared statement class has an issue with a wild card surrounding wild cards. ‘%?%’.

Because of this a simple sql query such as this is going to have issues pulling data from the database.

1

$sql = "SELECT * FROM tbl_comments WHERE comment LIKE %?% LIMIT 20";

Rather than attempt to get the above code to work you are going to need to take a separate approach and store the wild cards in with the search variable. This only takes one extra line of code and your search will be back up and running in no time at all.

1
2

$comment = '%'.$comment.'%';
$sql = "SELECT * FROM tbl_comments WHERE comment LIKE ? LIMIT 20";

Related posts:

1. Getting Secured Using Mysqli
2. SQL: One To Many Using Inner Join
3. SQL: Using Unions in Queries

This is a quick and short tutorial on how to create your own data services for flex to read. Also, note that in this tutorial we will be focussing on what a PHP Developer would be doing and not what a Flex Developer would do, we are not going to go over setting up data services or Flex, we are just going to be writing out classes.

Classes

The class is a rather simple thing to write out when focussing on creating an object that flex will be able to easily read. The trick to creating proper services is knowing what flex can do with your functions and your class.

The first thing to know is that flex is not going to be passing any values into the class and the class is not going to behave in a similar fashion to flex. This means that everything you do needs to operate on a function level.

The Construct

The construct is going to allow you to store information into the classes variables such as anything else and thus it is a good place to initialize and store a connection.

There isn’t a standard class declaration line within Flex and thus you cannot simply pass arguments through the parameters of the class.

The construct should look like this:

1
2
3
4
5
6

public function __construct()
{
    session_start();
    $this->connection = new mysqli('localhost','root','password','databasename');
 
}

Function Error Returns

Remember that the front end development team is going to need to know how to handle every situation of data that they receive and they will need to handle any scenario. Because of this you are going to need to return a status and message within the object in case anything goes wrong. A common standard is to simply store a boolean of if it worked as the first value returned. If the boolean is a false value then we know that the second object in the array is going to be an error message or an error code letting the front end know what went wrong.

Function Arguments

When setting up your arguments in the parameters of a function you are going to need to break away from pre setting the value of the argument with:

1

public function getMeta($metaName, $metaType = "video"){}

Values that are pre set in PHP do not show up in flex as an optional parameter.

Returning the Object

When using PHP/AMF as a data service it is good likely that you will need to return some rather complex objects to the front end team. The best way to pass data and allow them to use it is to store the information within an multi-dimensional array.

This means that when you use something like a prepared statement, you are going to need to handle the data and not just throw it back in a PHP array. If you throw back an unhandled mysqli array the front end team may kill you, especially if you have joins within your connection. This will force them to recreate an object from a singular one dimensional array.

Example of a custom Data Service.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

public function pullVideoFromUser($userNid)
    {
        //Query the database and pull video information
        $sql = "SELECT t.name tour_name, t.tour_nid video_nid, tm.created tour_created, tm.modified tour_modified, tm.viewcount tour_viewcount
            FROM tbl_user_video ut
            INNER JOIN tbl_video t ON ut.video_nid = t.video_nid
            INNER JOIN tbl_video_meta tm ON tm.video_nid = tu.video_nid
            WHERE u.user_nid = ?";
 
        $videoReturn = array();
        if($stmt = $this->connect->prepare($sql))
        {
            $stmt->bind_param('i', $userNid);
            $stmt->execute();
            $stmt->store_results();
            if($stmt->num_rows >= 1)
            {
                $stmt->bind_results($key, $name, $nid, $created, $modified, $viewcount);
                while($stmt->fetch())
                {
                    //create an object holding the appropriate information
                    $videoReturn[] = array('status' => 1,'key' =>$key, 'name' => $name, 'nid' => $nid, 'created' => $created, 'modified' => $modified, 'viewcount' => $viewcount);
                }
            }else{
                //Store a status of false and return an error message within the object.
		$videoReturn[] = array('status' => 0, 'message' => 'There was an issue returning the video.')
	    }
        }else{
            //Store a status of false and return an error message within the object.
	    $videoReturn[] = array('status' => 0, 'message' => $this->connection->error);
	}
        //return the object
        return $videoReturn;
 
    }

Related posts:

1. Flex Data Service Debugging Software
2. Getting Secured Using Mysqli
3. Redundant Data Class in PHP {Theory}