Fastest way to find strings in a file

Question

I have a log file that is not more than 10KB (File size can go up to 2 MB max) and I want to find if atleast one group of these strings occurs in the files. These strings w

Answer 1

Take a look at How to Read Text From a File. You might also want to take a look at the String.Contains() method.

Basically you will loop through all the files. For each file read line-by-line and see if any of the lines contains 1 of your special "Sections".

Answer 2

I would read it line by line and check the conditions. Once you have seen a group you can quit. This way you don't need to read the whole file into memory. Like this:

    public bool ContainsGroup(string file)
    {
        using (var reader = new StreamReader(file))
        {
            var hasAction = false;
            var hasInput = false;
            var hasResult = false;
            while (!reader.EndOfStream)
            {
                var line = reader.ReadLine();
                if (!hasAction)
                {
                    if (line.StartsWith("ACTION:"))
                        hasAction = true;
                }
                else if (!hasInput)
                {
                    if (line.StartsWith("INPUT:"))
                        hasInput = true;
                }
                else if (!hasResult)
                {
                    if (line.StartsWith("RESULT:"))
                        hasResult = true;
                }

                if (hasAction && hasInput && hasResult)
                    return true;
            }
            return false;
        }
    }

This code checks if there is a line starting with ACTION then one with INPUT and then one with RESULT. If the order of those is not important then you can omit the if () else if () checks. In case the line does not start with the strings replace StartsWith with Contains.

Answer 3

Here's one possible way to do it:

StreamReader sr;
string fileContents;

string[] logFiles = Directory.GetFiles(@"C:\Logs");

foreach (string file in logFiles)
{

    using (StreamReader sr = new StreamReader(file))
    {

        fileContents = sr.ReadAllText();

        if (fileContents.Contains("ACTION:") || fileContents.Contains("INPUT:") || fileContents.Contains("RESULT:"))
        {
             // Do what you need to here
        }

    }
}

You may need to do some variation based on your exact implementation needs - for example, what if the word spans two lines, does the line need to start with the word, etc.

Added

Alternate line-by-line check:

StreamReader sr;
string[] lines;

string[] logFiles = Directory.GetFiles(@"C:\Logs");

foreach (string file in logFiles)
{

    using (StreamReader sr = new StreamReader(file)
    {

        lines = sr.ReadAllLines();

        foreach (string line in lines)
        {        
            if (line.Contains("ACTION:") || line.Contains("INPUT:") || line.Contains("RESULT:"))
            {
                // Do what you need to here
            }
        }

    }
}

Answer 4

You don't have much of a choice with text files when it comes to efficiency. The easiest way would definitely be to loop through each line of data. When you grab a line in a string, split it on the spaces. Then match those words to your words until you find a match. Then do whatever you need.

I don't know how to do it in c# but in vb it would be something like...

Dim yourString as string
Dim words as string()
Do While objReader.Peek() <> -1
   yourString = objReader.ReadLine()
   words = yourString.split(" ")
   For Each word in words()
      If Myword = word Then
         do stuff
      End If
   Next
Loop

Hope that helps

Answer 5

This code sample searches for strings in a large text file. The words are contained in a HashSet. It writes the found lines in a temp file.

        if (File.Exists(@"temp.txt")) File.Delete(@"temp.txt");

        String line;
        String oldLine = "";
        using (var fs = File.OpenRead(largeFileName))
        using (var sr = new StreamReader(fs, Encoding.UTF8, true))
        {
            HashSet<String> hash = new HashSet<String>();
            hash.Add("house");
            using (var sw = new StreamWriter(@"temp.txt"))
            {
                while ((line = sr.ReadLine()) != null)
                {
                    foreach (String str in hash)
                    {
                        if (oldLine.Contains(str))
                        {
                            sw.WriteLine(oldLine); 
                            // write the next line as well (optional)
                            sw.WriteLine(line + "\r\n");                                    
                        }
                    }
                    oldLine = line;
                }
            }
        }