An ASP Tutorial to create your own Database driven Search Engine

Reviewing the code
We will now very quickly look into the code and see how it's done. Although this page might seem a bit long, but if you carefully look into it, all it contains are simple VBScript function calls and nothing more than that.

This is first line of the page. All it does is to include 'editme.asp' file into this page. 'editme.asp' contains a single variable to store the relative location of the database. We will look into 'editme.asp' page later.

Tells ASP script interpreter to continue with the next statement and not to stop if it receives an error during script execution. Actually it consists of two parts :

By writing it we tell ASP script that we will receive the error and we will decide whether to stop the script from further execution ( by doing nothing ) or do some thing with it. On Error is an event which is fired when an error occurs in the execution of the script.

By writing this we decide that we want ASP script interpreter to continue with the execution of the script and don't stop if it receives an error. This discussion has got a bit long, but it is very important that you understand what this statement does since you will be seeing it quite often in the ASP scripts, so you should be at ease with it. Tip, when you are testing your ASP code knock this line off, this will let you see the error and make corrections, but when you are releasing your ASP code then write this statement on top of your ASP page so that your script users might not see the errors if produced.

We define a list of variables which we will be using later in our script. Note in ASP all the variables are of type VARIANT, which means they can be of any type e.g. string ( text ), int ( number ), object etc.

strURL receives the URL which will be entered in the Form 1 of 'addtodb.htm' page. Since Form 1 had only one field with name 'look_for', we receive it's value by using this field name in the Request.QueryString method.

Now this is the real part to retrieve the contents of the page specified in the Form 1's 'look_for' field by using HTTP method. Note that ASP uses server side scripting languages to generate dynamic content, it is not possible to retrieve the contents of any page on the web using scripting languages. Scripting languages simply don't have that power. So what do we do ? we use our own component to retrieve the content of that web page using HTTP protocol. This is where ASP is very handy. It lets us use COM components to do tasks which we cannot do with simple VBScript. So with this search engine I am including my own Free component which can by used to retrieve page contents into a variable using HTTP protocol. All you have to do to use it can be seen by looking at the above three lines. Yes that's right, it is very simple to use. Just create an instance of it and then use it's only method Get() to get the page contents in a variable. Note that the only argument to Get() is strURL variable which is the URL of the page we want to retrieve.

As you might be knowing that in order to use a COM component on your computer ( or server ), you have to register it using regsvr32 command. So in order to use this component go to the DOS prompt and then move to the location of the directory where you have kept the search engine files. Now type the following command :

And press enter. You will see a small window saying that component was successfully registered. That's it. Now you can use this component. Note, the ProgID of this component is Stardeveloper.GetURL.

Since above three paragraphs of ASP script are similar we will discuss them together. All they do is to check the availability of certain HTML meta tags within that page. If you are familiar with HTML then you must be knowing about title, description and keyword meta tags. Above code looks for the presence of these meta tags in the contents of that page ( which is present in the strFileContents variable ). Note that earlier we retrieved all the contents of that page in the strFileContents variable, so we use this variable when we look for meta tags.

Since we are looking for meta tags to index their contents, make sure that title, description and keywords tags are present in the <head></head> section of your page which you want to index in the following manner :

So our above ASP script indexed the contents of given web page and then retrieved the values of title, description and keywords tags in to title, description and keywords variables respectively.

After populating the three variables title, description and keywords, we now move forward to add them to the database. We begin by creating Connection object. Then we open the database. Note as we discussed earlier, 'editme.asp' contains the location of the database in the variable strDB. So after including 'editme.asp' page in our 'addtodb.asp' page we can easily use the value of strDB in our script.

We then check the database to see if this given URL has been added to the database before. We do it by selecting title, description and keywords fields from the 'all_pages' table where URL is the given URL. If the returning recordset is empty then it shows that the URL has not been added to the database. But if the returning recordset is not empty then it means that URL has been added to the database before.

	<!--#include file="editme.asp"-->

Reviewing the code :

<html>
<head>
	<style>
		body { font-family : Verdana; font-size : 8pt; }
	</style>
</head>
<body>

Above few lines are HTML tags with no ASP content.

	<%
		On Error Resume Next

Tells ASP script interpreter to continue with the next statement and not to stop if it receives an error during script execution. Actually it consists of two parts :

	On Error

	Resume Next

		Dim geturl, title, description, keywords, strURL
		Dim strDB, con, results

	' URL
		strURL = Request.QueryString("look_for")

	Set geturl = CreateObject("Stardeveloper.GetURL")
		strFileContents = geturl.Get(strURL)
	Set geturl = Nothing

	regsvr32 GetURL.dll

And press enter. You will see a small window saying that component was successfully registered. That's it. Now you can use this component. Note, the ProgID of this component is Stardeveloper.GetURL.

   ' Keywords
   key1 = InStr(1, strFileContents, "<meta name=""keywords"" content=""", 1)
   key1 = key1 + Len("<meta name=""keywords"" content=""")
   key2 = InStr(key1, strFileContents, """>", 1)

   keywords = "," & Trim(Mid(strFileContents, key1, (key2 - key1))) & ","
   keywords = Replace (keywords, "'", " ")

   ' Description
   desc1 = InStr(1, strFileContents, "<meta name=""description"" content=""", 1)
   desc1 = desc1 + Len("<meta name=""description"" content=""")
   desc2 = InStr(desc1, strFileContents, """>", 1)

   description = Trim(Mid(strFileContents, desc1, (desc2 - desc1)))
   description = Replace (description, "'", " ")

   ' Title
   tit1 = InStr(1, lcase(strFileContents), "<title>", 1)
   tit1 = tit1 + Len("<title>")
   tit2 = InStr(tit1, strFileContents, "</title>", 1)

   title = Trim(Mid(strFileContents, tit1, (tit2 - tit1)))
   title = Replace (title, "'", " ")

<title>Your Page Title Goes Here</title>
<meta name="description" content="Your page description goes here">
<meta name="keywords" content="keyword1, keyword2, keyword3, keyword4 ...">

So our above ASP script indexed the contents of given web page and then retrieved the values of title, description and keywords tags in to title, description and keywords variables respectively.

Reviewing the code :

	' Our Connection Object
	Set con = CreateObject("ADODB.Connection")
		con.Open strDB
	Set results = con.Execute("select title, description, keywords " & _
		"from all_pages where url = '" & strURL & "'")