I have a sample server
var host = new WebHostBuilder()
.UseKestrel()
.UseContentRoot(Directory.GetCurrentDirectory())
Another approach is to copy those files into your unit test project
Then
As far as you maintain the same structure folder of the project that you are testing, it will find out the files and pass it to the dependency injection process.
XUnit starts the site from a different directory. We've had to work around it like this: https://github.com/aspnet/StaticFiles/blob/d692066b2bd711653150ad2cccc2268583355532/test/Microsoft.AspNetCore.StaticFiles.Tests/StaticFilesTestServer.cs#L20