Best-practices for localizing a SQL Server (2005/2008) database

Question

Question

I\'m sure many of you have been faced by the challenge of localizing a database backend to an application. If you\'ve not then I\'d be pretty confident in

Answer 1

I think you can stick with XML which allows for a cleaner design. I would go further and take advantage of the xml:lang attribute which is designed for this usage :

<l10n>
  <text xml:lang="sv-SE">Detta är ett namn</text>
  <text xml:lang="en-EN">This is a name</text>
</l10n>

One step further, you could select the localized resource in your query via a XPath query (as suggested in the comments) to avoid any client side treatment. This would give something like this (untested) :

SELECT Name.value('(l10n/text[lang()="en"])[1]', 'NVARCHAR(MAX)')
  FROM Product
  WHERE Product.ID=10;

Note that this solution would be an elegant but less efficient solution than the separate table one. Which may be OK for some application.

Answer 2

Here is how I've done it. I don't use LINQ or SP for this one, because the query is too complex and is dynamically built and this is just a excerpt of the query.

I have a products table:

* id
* price
* stocklevel
* active
* name
* shortdescription
* longdescription

and a products_globalization table:

* id
* products_id
* name
* shortdescription
* longdescription

As you can see the products-table contains all the globalization-columns aswell. These columns contains the default-language (thus, being able to skip doing a join when requesting the default culture - BUT I'm not sure if this is worth the trouble, I mean the join between the two tables are index-based so... - give me some feedback on this one).

I prefer having a side-by-side table over a global-resourcetable because in certain situations you might need to do i.e. a database (MySQL) MATCH on a couple of columns, such as MATCH(name, shortdescription, longdescription) AGAINST ('Something here').

In a normal scenario some of the product translations might be missing but I still want to show all products (not just the ones who are translated). So it's not enough to do to a join, we actually need to do a left join based on the products-table.

Pseudo:

string query = "";
if(string.IsNullOrEmpty(culture)) {
   // No culture specified, no join needed.
   query = "SELECT p.price, p.name, p.shortdescription FROM products p WHERE p.price > ?Price";
} else {
   query = "SELECT p.price, case when pg.name is null then p.name else pg.name end as 'name', case when pg.shortdescription is null then p.shortdescription else pg.shortdescription end as 'shortdescription' FROM products p"
   + " LEFT JOIN products_globalization pg ON pg.products_id = p.id AND pg.culture = ?Culture"
   + " WHERE p.price > ?Price";
}

I would go with COALESCE instead of CASE ELSE but thats besides the point.

Well, thats my take on it. Feel free to critize my suggestion...

Kind regards, Richard

Answer 3

I can't see why you need multiple text tables. A single text table, with a "globally" unique text ID, should be sufficient. The table would have ID, language, text columns, and you would only ever get the text in the language that you need to present (or perhaps not retrieve the text at all). The join should be fairly efficient, since the combination of (ID, language) is the primary key.

Answer 4

That's one of the questions that are difficult to answer because there are so many "it depends" in the answer :-)

The answer depends on the amount of localized items in the database, on deployment scenarios, caching issues, access patterns and so on. If you can give us some data on how big the application is, how many concurrent users it will have and how it will be deployed, that would be very helpful.

In general terms I usually use one of two approaches:

1. Store the localized items near the executable (localized ressource dlls)
2. Store localized items in the DB and introduce a localeID column in tables that contain the localized items.

The advantage of the first method is the good VisualStudio support. The advantage of the second is the centralized deployment.

Answer 5

Another approach to consider: don't store content in the database, but keep "application" supporting database records and "content" as separate entities.

I used an approach similar to this when creating multiple themes for my ecommerce website. Several of the products have a manufacturer logo which also must match the website theme. Since there is no real database support for themes, I had a problem. The solution I came up with was to use a token in the database to identify the ClientID of the image, rather than storing the URL of the image (which would vary based on theme).

Following the same approach, you could change your database from storing the name and description of the product into storing a name token and a description token that would identify the resource (in an resx file or in the database using Rick Strahl's approach) that contains the content. The built-in functionality of .NET would then handle the switching of language rather than attempting to do it in the database (it is rarely a good idea to put business logic in the database). You could then use the token on the client to look up the correct resource.

Label1.Text = GetLocalResourceObject("TokenStoredInDatabase").ToString()

The disadvantage to this approach is obviously keeping the database tokens and the resource tokens in sync (because products could be added without any descriptions), but could potentially be done easier using a resourceprovider such as the one Rick Strahl created. This approach may not work if you have products that change frequently, but for some people it might.

The advantage is that you have a small amount of data to transfer to the client from the database, your content is cleanly separated from your database, and your database won't need to be more complex than it is now.

On a side note, if you are running an e-Commerce store and actually want to get your localized pages indexed, you have to deviate a little from the seemingly natural way that Microsoft created. There is clearly disagreement between a practical and logical design flow and what Google recommends for SEO. Indeed, some webmasters have complained that their pages weren't indexed by the search engines for anything but the "default" culture because the search engines only will index a single URL once even if it changes depending on the culture of the browser.

Fortunately, there is a simple approach to get around this: put links on the page to translate it into the other languages based on a querystring parameter. An example of this can be found (oops, they won't let me post another link!!) and if you check, each culture of the page has been indexed by both Google and Yahoo (although not by Bing). A more advanced approach may use URL rewriting in combination with some fancy regular expressions to make your single localized page look like it has multiple directories, but actually pass a querystring parameter to the page instead.

Answer 6

I see no advantage in using the XML-columns to store the localized values. Except maybe that you have all localized versions of one item "in one place" if that's worth something to you.

I would propose to use a cultureID-column in each table that has localizable items. That way you don't need any XML-handling at all. You already have your data in a relational schema so why introduce another layer of complexity when the relational schema is perfectly capable of handling the problem?

Let's say "sv-SE" has cultureID = 1 and "en-EN" has 2.

Then your query would be modified as

SELECT *
From Product
Where Product.ID = 10 AND Product.cultureID = 1

for a swedish client.

This solution I have seen frequently in localized databases. It scales well with both number of cultures and number of datarecords. It avoids XML-parsing and processing and is easy to implement.

And another point: The XML-solution gives you a flexibility you don't need: You could for example take the "sv-SE"-value from the "Name"-column and the "en-EN"-value from the "Description"-column. However, you don't need this since your client will request only one culture at a time. Flexibility usually has a cost. In this case it is that you need to parse all columns individually while with the cultureID solution you get the whole record with all the values right for the requested culture.