Directory.EnumerateFiles => UnauthorizedAccessException

Question

There is a nice new method in .NET 4.0 for getting files in a directory in a streaming way via enumeration.

The problem here is that if one wishes to enumerate all f

Answer 1

Ths issue with the above answer is that is does not take care of exception in sub directories. This would be a better way to handling those exceptions so you get ALL files from ALL subdirectories except those with threw an access exception:

    /// <summary>
    /// A safe way to get all the files in a directory and sub directory without crashing on UnauthorizedException or PathTooLongException
    /// </summary>
    /// <param name="rootPath">Starting directory</param>
    /// <param name="patternMatch">Filename pattern match</param>
    /// <param name="searchOption">Search subdirectories or only top level directory for files</param>
    /// <returns>List of files</returns>
    public static IEnumerable<string> GetDirectoryFiles(string rootPath, string patternMatch, SearchOption searchOption)
    {
        var foundFiles = Enumerable.Empty<string>();

        if (searchOption == SearchOption.AllDirectories)
        {
            try
            {
                IEnumerable<string> subDirs = Directory.EnumerateDirectories(rootPath);
                foreach (string dir in subDirs)
                {
                    foundFiles = foundFiles.Concat(GetDirectoryFiles(dir, patternMatch, searchOption)); // Add files in subdirectories recursively to the list
                }
            }
            catch (UnauthorizedAccessException) { }
            catch (PathTooLongException) {}
        }

        try
        {
            foundFiles = foundFiles.Concat(Directory.EnumerateFiles(rootPath, patternMatch)); // Add files from the current directory
        }
        catch (UnauthorizedAccessException) { }

        return foundFiles;
    }

Answer 2

I'm late, but I suggest using observable pattern instead:

public class FileUtil
{
  private static void FindFiles_(string path, string pattern,
    SearchOption option, IObserver<string> obs, CancellationToken token)
  {
    try
    {
      foreach (var file in Directory.EnumerateFiles(path, pattern,
        SearchOption.TopDirectoryOnly))
      {
        if (token.IsCancellationRequested) break;
        obs.OnNext(file);
      }

      if (option != SearchOption.AllDirectories) return;

      foreach (var dir in Directory.EnumerateDirectories(path, "*", 
        SearchOption.TopDirectoryOnly))
      {
        FindFiles_(dir, pattern, option, obs, token);
      }
    }
    catch (UnauthorizedAccessException) { }
    catch (PathTooLongException) { }
    catch (IOException) { }
    catch (Exception err) { obs.OnError(err); }
  }

  public static IObservable<string> GetFiles(string root, string pattern,
    SearchOption option)
  {
    return Observable.Create<string>(
      (obs, token) =>
        Task.Factory.StartNew(
          () =>
          {
            FindFiles_(root, pattern, option, obs, token);
            obs.OnCompleted();
          },
          token));
  }
}

Answer 3

I understand it's MoveNext that throws the exception.

I tried to write a method that safe-walks a sequence and tries to ignore MoveNext exceptions. However I'm not sure if MoveNext advances position when it throws an exception, so this might as well be infinite loop. It is also bad idea because we would rely on implementation details.

But it's just so much fun!

public static IEnumerable<T> SafeWalk<T> (this IEnumerable<T> source)
{
    var enumerator = source.GetEnumerator();
    bool? hasCurrent = null;

    do {
        try {
            hasCurrent = enumerator.MoveNext();
        } catch {
            hasCurrent = null; // we're not sure
        }

        if (hasCurrent ?? false) // if not sure, do not return value
            yield return enumerator.Current;

    } while (hasCurrent ?? true); // if not sure, continue walking
}

foreach (var file in Directory.EnumerateFiles("c:\\", "*", SearchOption.AllDirectories)
                              .SafeWalk())
{
    // ...
}

This will only work if the following conditions are true about framework's implementation of this iterator (see FileSystemEnumerableIterator<TSource> in Reflector for reference):

• MoveNext advances its position when it fails;
• When MoveNext fails on last element, subsequent calls will return false instead of throwing an exception;
• This behavior is consistent for different versions of .NET Framework;
• I haven't made any logic or syntax mistakes.

Even if it works, please, never use it in production!
But I really wonder if it does.

Answer 4

I made my own Implementation of a Class that works arround this, as previous answers didnt seem to do what i wanted to. This simply skips all Files and Folders that it can not access, and returns all files that it could access.

public static class SafeWalk
{
    public static IEnumerable<string> EnumerateFiles(string path, string searchPattern, SearchOption searchOpt)
    {
        if (searchOpt == SearchOption.TopDirectoryOnly)
        {
            return Directory.EnumerateFiles(path, searchPattern, SearchOption.TopDirectoryOnly);
        }

        List<string> folders = new List<string>() { path };
        int folCount = 1;
        List<string> files = new List<string>() { };

        for (int i = 0; i < folCount; i++)
        {
            try
            {
                foreach (var newDir in Directory.EnumerateDirectories(folders[i], "*", SearchOption.TopDirectoryOnly))
                {
                    folders.Add(newDir);
                    folCount++;
                    try
                    {

                        foreach (var file in Directory.EnumerateFiles(newDir, searchPattern))
                        {
                            files.Add(file);
                        }
                    } catch (UnauthorizedAccessException)
                    {
                        // Failed to read a File, skipping it.
                    }
                }
            }
            catch (UnauthorizedAccessException)
            {
                // Failed to read a Folder, skipping it.
                continue;
            }
        }
        return files;
    }
}

Usable exactly like the regular EnumerateFiles function, simply using SafeWalk.EnumerateFiles(...) instead of Dictionary.EnumerateFiles(...)

Answer 5

I Couldn't get the above to work, but here is my implementation, i've tested it on c:\users on a "Win7" box, because if has all these "nasty" dirs:

SafeWalk.EnumerateFiles(@"C:\users", "*.jpg", SearchOption.AllDirectories).Take(10)

Class:

public static class SafeWalk
{
    public static IEnumerable<string> EnumerateFiles(string path, string searchPattern, SearchOption searchOpt)
    {   
        try
        {
            var dirFiles = Enumerable.Empty<string>();
            if(searchOpt == SearchOption.AllDirectories)
            {
                dirFiles = Directory.EnumerateDirectories(path)
                                    .SelectMany(x => EnumerateFiles(x, searchPattern, searchOpt));
            }
            return dirFiles.Concat(Directory.EnumerateFiles(path, searchPattern));
        }
        catch(UnauthorizedAccessException ex)
        {
            return Enumerable.Empty<string>();
        }
    }
}