How to create a simple custom language colorization to VS Code

Question

I'm trying to create a simple colorization for log files, now that it's possible include custom languages in Code (I'm on 0.9.2). I have created a simple .tmLanguage file for colorizing the letter 'q', just for starting up, but have been unsuccessful.

My new language, log, is associated correctly with the file extension and I can also select it manually from inside Code, but no coloring takes places. I have a feeling it has to do with what "scope" I associate my pattern with, but I'm not sure. Is there a list of valid scope to choose from? Initially I thought I'd use something general, such as "comment" to get some color, but it doesn't seem to work.

Here's my .tmLanguage file:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
    <dict>
        <key>scopeName</key>
        <string>text.log</string>

        <key>fileTypes</key>
        <array>
            <string>log</string>
        </array>

        <key>name</key>
        <string>Log file</string>

        <key>patterns</key>
        <array>
            <dict>
                <key>match</key>
                <string>q</string>
                <key>name</key>
                <string>comment</string>
            </dict>
        </array>
    </dict>
</plist>

I'm probably misunderstanding something here, so any help is very appreciated :-)

Answer 1

You need to use regular expressions instead of static strings to describe the pattern:

<key>match</key>
<string>q</string>  <- This needs to be a regular expression
<key>name</key>
<string>comment</string>

I provide a more useful example for a log file highlighter. It colors numbers, hints, warnings and errors in different colors. The rules to identify these keywords and numbers are based on regular expression.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
    <dict>
        <key>scopeName</key>
        <string>text.log</string>

        <key>fileTypes</key>
        <array>
            <string>log</string>
        </array>

        <key>name</key>
        <string>Log file</string>

        <key>patterns</key>
        <array>
            <dict>
                <key>match</key>
                <string>\b(?i:(hint|info|information))\b</string>
                <key>name</key>

                <string>info-token</string>
            </dict>                
            <dict>
                <key>match</key>
                <string>\b(?i:(warning|warn))\b</string>
                <key>name</key>
                <string>warn-token</string>
            </dict>
            <dict>
                <key>match</key>
                <string>\b(?i:(Error|Failure|Fail))\b</string>
                <key>name</key>
                <string>error-token</string>
            </dict>
            <dict>
                <key>match</key>
                <string>\b((0(x|X)[0-9a-fA-F]*)|(([0-9]+\.?[0-9]*)|(\.[0-9]+))((e|E)(\+|-)?[0-9]+)?)(L|l|UL|ul|u|U|F|f|ll|LL|ull|ULL)?\b</string>
                <key>name</key>
                <string>constant.numeric</string>
            </dict>                                
        </array>
        <key>uuid</key>
        <string>FF0550E0-3A29-11E3-AA6E-0800200C9A77</string>
    </dict>
</plist>

The highlighter gives a result like this (using the default theme):

I didn't find an official documentation about the available tokens (like error-token, constant.numeric etc). But there is a file located in %VSCODE_INSTALLATION%\resources\app\out\vs\languages\markdown\common\tokens.css. It seems to list all available tokens etc. Use it as a reference when you create the .tmLanguage file.

But pay attention: Some themes are using only the basic tokens. And some other themes are using the same color for many different tokens. So you should test the highlighter frequently against the most common themes to see whether the result looks good or not.

You should definitely visit this page about Language Grammars to learn more.

Answer 2

We just released a language extension that brings colorization to the Output panel. Basically, it does the same thing as the approved answer on this thread, and adds the text/x-code-output mime type, which is used by the Output panel.

Get it free here: https://marketplace.visualstudio.com/items?itemName=IBM.output-colorizer

Source here: https://github.com/IBM-Bluemix/vscode-log-output-colorizer Please help contribute! Bugs, feature requests, contributions all welcome.

Here are some screenshots of it working:

Answer 3

As @Woshi said, you need regular expressions.

As for the scopes that generally work with most color themes, I'll link you to this answer. Keep in mind that for scope to be picked up, it needs to be in dictionary with key "name".