Creating a site to query a database of tables

I have a small problem. I am working with some manual testers who are untrained in programming/database design. Our current process means that these manual testers need to insert data into our database at certain times whilst we build a GUI to facilitate this in the future.

In the interim, I would like to create a simple site. What I would like to do with the site is, simply, connect to our database, allow the manual tester to enter some keywords, and return any columns within tables that are close/related to the keywords provided. This would save a lot of time for our testers searching for colums in our (rather large) database.

How could I create a site like this? I think it could be useful for a lot of people, so I have decided to post the question up here to gather the thoughts of StackOverflow.

At the moment, I am thinking a simple PHP page with a textbox, which allows the user to enter some data, separated by commas. Explode the data based on commas, hold it in an array. Connect to my database, then use the Information Schema View to retrieve column information. My main problem is - what is the most effective way to use the Information Schema View to retrieve columns related to the keywords entered by the users ? How can I ensure the columns returned are the most suitable?

Any input here would be greatly appreciated. Thanks a lot.

Tl;dr is the bolded part, for busy people :)

I think you could achieve this with a simple form and some ajax calls using on key up. Here is a simple example in which the list will update each time the user enters a letter in the column name they are searching for.

Index.html

  <!DOCTYPE html>
<html lang="en">
  <head>
  <script type="text/javascript"> 
      $(document).ready(function() {

$("#faq_search_input").keyup(function()
{
var faq_search_input = $(this).val();
var dataString = 'keyword='+ faq_search_input;
if(faq_search_input.length>1)

{
$.ajax({
type: "GET",
url: "ajax-search.php",
data: dataString,
success: function(server_response)
{
 document.getElementById("searchresultdata").style.display = "block";
$('#searchresultdata').html(server_response).show();

}
});
}return false;
});
});

</script>


  </head>
  <body>
<div class="searchholder">
    <input  name="query" class="quicksearch" type="text" id="faq_search_input" />
        <div id="searchresultdata" class="searchresults" style="display:none;"> </div>
</div>
  </body>
</html>

next we need a script to carry out our search

ajax-search.php

    //you must define your database settings
define("DB_HOST", "FOO");
define("DB_USERNAME", "BAR");
define("DB_PASSWORD", "YOUR PASSWORD");
define("DB_NAME", "DATABASE NAME");
if(isset($_GET['keyword']))
    {
        $search = new mysqli(DB_HOST, DB_USERNAME, DB_PASSWORD, DB_NAME);
            if ($search->connect_errno)
            {
                echo "Failed to connect to MySQL: (" . $search->connect_errno . ") " . $search->connect_error;
                $search->close();
            }
                $keyword =  trim($_GET['keyword']) ;
                $query ="SELECT COLUMN_NAME FROM ".DB_NAME.".INFORMATION_SCHEMA.COLUMNS WHERE COLUMN_NAME LIKE '%".$keyword."%'";
                $values = $search->query($query);
                    if($values->num_rows != 0)
                    {
                        while($row = $values->fetch_assoc())
                        { 
                            echo $row['COLUMN_NAME']."<br>";
                        } 
                    }
                    else
                        {
                            echo 'No Results for :"'.$_GET['keyword'].'"';
                        }
    }

As the user types out a column name all of the column name like this will be returned and updated on the fly, without page reload. Hope this helps

You should do something like this:

Form:

<form action="search.php" method="post">
<textarea name="words"></textarea>
<input type="submit">
</form>

search.php

<?php

// You will need a DB user with enough permissions
$link = mysqli_connect($server,$user,$pass);
mysqli_select_db($link,$database_name);

print "<table>";

// Comma separated
$words = explode(",",$_POST['words']);

foreach ($words as $word)
{

 $sql = "SELECT COLUMN_NAME FROM ".$database_name.".INFORMATION_SCHEMA.COLUMNS
 WHERE COLUMN_NAME LIKE '%".$word."%'";

 $res = mysqli_query($link,$sql);

 while ($row = mysqli_fetch_assoc($res))
 {
  print "<tr><td>".$row['COLUMN_NAME']."</td></tr>";
 }

}

print "</table>";

?>

I can see why you are asking this interesting question. If the tester enters a list of keywords, and you use the information schema view to obtain a list of matching columns, then there is a danger that there will be a lot of false matches that could waste time or cause the tester to enter incorrect information into your system. You want to know how to determine which columns are the best matches to the tester's query. But you want to keep it simple because this is just a temporary workaround, it's not your main application.

The answer is to supplement search results using a reputation-based system. Here is a very simple one that should work well for your application.

First, create two simple tables to store rating information for the tables and columns in your database. Here is the starting structure.

TEST_SEARCH_TABLES:
  TABLE_ID
  TABLE_NAME
  RATING

TEST_SEARCH_COLUMNS:
   COLUMN_ID
   TABLE_ID
   COLUMN_NAME
   RATING

Populate TEST_SEARCH_TABLES with the name of every table in your database. Populate TEST_SEARCH_COLUMNS with the name of every column, and link it to the corresponding table. Initialize all the RATING columns to 1000.0 - you will be using the Elo Rating System to supplement your rankings because it is simple, easy to implement and it works great.

When the user enters a list of keywords, don't use Information Schema View. Instead, search the TEST_SEARCH_COLUMNS table for any columns that have any of those keywords. Assign each column a WEIGHT based on the number of hits. (For example, if the search is "customer,amount,income" then a column CUSTOMER_ID would have a weight of 1. A column CUSTOMER_INCOME would have a weight of 2, and CUSTOMER_INCOME_AMOUNT would have a weight of 3.) Calculate the WEIGHT of each table as the sum of the weights of its columns.

Now for each table and column returned by your search, multiply the WEIGHT times the RATING to determine the SEARCH VALUE. Give the tester a list of matching tables in descending order of search value. Within each table, also list the columns in descending order of their search value.

Every time a table or column appears in a search, use the Elo Rating System to give it a WIN against an opponent rated 1000.0. Every time a user selects a column to work with, give both that column and its table a win against an opponent rated 1500.0. In this way, the most useful and successful tables and columns will organically float to the top of your search lists over time.

A side benefit to this approach (using tables instead of information schema view) is that this approach is more extensible. As an enhancement, you could put DESCRIPTION and COMMENTS columns on the TEST_SEARCH_TABLES and TEST_SEARCH_COLUMNS tables, and also search those columns for keyword matches as well.

Here is another optional enhancement - you could put a (+) and (-) button next to each table and column and give it a win against a 2000-rated opponent if the user clicks (+) and a loss against a zero-rated opponent if the user clicks (-). That will allow your testers to vote for columns they find important and to vote against columns that are always getting in the way.

I'm not sure if I fully understood your issue

Take a look at this:

http://php.net/manual/en/function.mysql-list-tables.php

you can get all the tables on a database , store them in an array then filter them using your keywords

I think this could be done in the following steps without any PHP programming and even without need in any web-server.

Write SQL-script which makes everything to retrieve data you need.
Modify script to add columns to result set with simple html-formatting to make you result record like the following:
```
'<tr><td>', 'resultcolumn1', '</td><td>', 'resultcolumn2','</td></tr>'
```
Run this script using sqlcmd with output option to file. Give resulting file .html extension.
Place sqlcmd call inside cmd file. After calling sqlcmd call web browser with resulting html file name as parameter. This will display your results to tester.

So, your testers only run cmd file with some parameters and get html page with results. Of course you need to form correct html head and body tags, but this is not a problem.

Now about your main question about how you can be sure the columns returned are the most suitable. I think the most reliable from the most simple ways is to create thesaurus table which contains synonyms for your column names. (This could be done by testers themselves). So you can search your column names from Information Schema View using LIKE in INFORMATION_SCHEMA.COLUMNS as well as in thesaurus table.

Not sure if you want spend time on writing and supporting your solution. For php/mysql I would use http://www.phpmyadmin.net/home_page/index.php or if users can access db directly http://dev.mysql.com/downloads/gui-tools/5.0.html

Might take some time to tech them how to use it, but will save a lot of problems in a long run.

Another thing, you can create *.sql files that would populate db automatically.

query.sql

CREATE TABLE "example" (
    "id" INT NOT NULL AUTO_INCREMENT, 
    "name" VARCHAR(30), 
    "age" INT
);

INSERT INTO "example" VALUES
('1', 'a', 1),    
('2', 'b', 2);

than you can run it from command line: