benchmarking

I know that I have to use: rdtsc. The measured function is deterministic but the result is far from being repeatable (I get 5% oscillations from run to run). Possible causes are: context switching cache misses Do you know any other causes? How to eliminate them? TSCs (what rdtsc uses) are often not synchronized on multi-processor systems. It may help to set the CPU affinity in order to bind the process to a single CPU. You could also get timestamps from HPET timers if available, which aren't prone to the same problem. As for repeatability, those variances are true. You could disable caching,

I'm about to start doing some benchmarking/testing of our builds, and I'd like to drive the whole thing from a command line. I am aware of DevEnv but am not convinced it can do what I want. If I could have a single file built within a single project, I'd be happy. Can this be done? The magical incantation is as follows. Note that this has only been tested with VS 2010 - I have heard this is the first version of Visual Studio with this capability: The Incantation <msbuild> <project> <settings> <file> Where msbuild is a path to MSBuild.exe . Usually this should be set up for you by the VS2010

I'm a great fan of the relatively recent trend of using lambda expressions instead of strings for indicating properties in, for instance, ORM mapping. Strongly typed >>>> Stringly typed. To be clear, this is what I'm talking about: builder.Entity<WebserviceAccount>() .HasTableName( "webservice_accounts" ) .HasPrimaryKey( _ => _.Id ) .Property( _ => _.Id ).HasColumnName( "id" ) .Property( _ => _.Username ).HasColumnName( "Username" ).HasLength( 255 ) .Property( _ => _.Password ).HasColumnName( "Password" ).HasLength( 255 ) .Property( _ => _.Active ).HasColumnName( "Active" ); In some recent

I am doing performance tests for my master thesis and I'm getting very poor performance of Symfony2 simple application. It's simple app, one query and some math. Test results for command: ab -c10 -t60 http://sf2.cities.localhost/app.php Server Software: Apache/2.2.20 Server Hostname: sf2.cities.localhost Server Port: 80 Document Path: /app.php Document Length: 2035 bytes Concurrency Level: 10 Time taken for tests: 60.162 seconds Complete requests: 217 Failed requests: 68 (Connect: 0, Receive: 0, Length: 68, Exceptions: 0) Write errors: 0 Non-2xx responses: 68 Total transferred: 393876 bytes

I am about to embark in very detailed benchmarking of a set of complex functions in C. This is "science level" detail. I'm wondering, what would be the best way to do serious benchmarking? I was thinking about running them, say, 10 times each, averaging the timing results and give the standard dev, for instance, just using <time.h> . What would you guys do to obtain good benchmarks? Reporting an average and standard deviation gives a good description of a distribution when the distribution in question is approximately normal. However, this is rarely true of computational performance

I have been thinking about a scenario where one lets users (can be anyone, possibly with bad intentions) submit code which is run on a Linux PC (let's call it the benchmark node). The goal is to make a kind of automated benchmarking environment for single-threaded routines. Let's say that a website posts some code to a proxy. This proxy hands this code to the benchmark node, and the benchmark node only has an ethernet connection to the proxy, not internet itself. If one lets whatever user post C/asm code to be run on the benchmark node, what security challenges will one face? The following