Creating a site to query a database of tables

Question

I have a small problem. I am working with some manual testers who are untrained in programming/database design. Our current process means that these manual testers need to i

Answer 1

I can see why you are asking this interesting question. If the tester enters a list of keywords, and you use the information schema view to obtain a list of matching columns, then there is a danger that there will be a lot of false matches that could waste time or cause the tester to enter incorrect information into your system. You want to know how to determine which columns are the best matches to the tester's query. But you want to keep it simple because this is just a temporary workaround, it's not your main application.

The answer is to supplement search results using a reputation-based system. Here is a very simple one that should work well for your application.

First, create two simple tables to store rating information for the tables and columns in your database. Here is the starting structure.

TEST_SEARCH_TABLES:
  TABLE_ID
  TABLE_NAME
  RATING

TEST_SEARCH_COLUMNS:
   COLUMN_ID
   TABLE_ID
   COLUMN_NAME
   RATING

Populate TEST_SEARCH_TABLES with the name of every table in your database. Populate TEST_SEARCH_COLUMNS with the name of every column, and link it to the corresponding table. Initialize all the RATING columns to 1000.0 - you will be using the Elo Rating System to supplement your rankings because it is simple, easy to implement and it works great.

When the user enters a list of keywords, don't use Information Schema View. Instead, search the TEST_SEARCH_COLUMNS table for any columns that have any of those keywords. Assign each column a WEIGHT based on the number of hits. (For example, if the search is "customer,amount,income" then a column CUSTOMER_ID would have a weight of 1. A column CUSTOMER_INCOME would have a weight of 2, and CUSTOMER_INCOME_AMOUNT would have a weight of 3.) Calculate the WEIGHT of each table as the sum of the weights of its columns.

Now for each table and column returned by your search, multiply the WEIGHT times the RATING to determine the SEARCH VALUE. Give the tester a list of matching tables in descending order of search value. Within each table, also list the columns in descending order of their search value.

Every time a table or column appears in a search, use the Elo Rating System to give it a WIN against an opponent rated 1000.0. Every time a user selects a column to work with, give both that column and its table a win against an opponent rated 1500.0. In this way, the most useful and successful tables and columns will organically float to the top of your search lists over time.

A side benefit to this approach (using tables instead of information schema view) is that this approach is more extensible. As an enhancement, you could put DESCRIPTION and COMMENTS columns on the TEST_SEARCH_TABLES and TEST_SEARCH_COLUMNS tables, and also search those columns for keyword matches as well.

Here is another optional enhancement - you could put a (+) and (-) button next to each table and column and give it a win against a 2000-rated opponent if the user clicks (+) and a loss against a zero-rated opponent if the user clicks (-). That will allow your testers to vote for columns they find important and to vote against columns that are always getting in the way.