Creating an online judge from scratch

Question

I\'ve participated in a few online programming contests and found the online judges used quite remarkable in functionality.

Coming to the point of the topic, my college

Answer 1

I'm really not sure what your question is about. It's not hard to write a design spec for a judge from scratch.

You run a thing with given input data and feed the output data to the test program written by the question's author (because there's not always unique answer). People do sandboxing by running it remotely on a clean machine.

Add: and please, no code analysis. You have two choices, either you make this code analysis available to contestants during the contest, or not.

• If you do: they spend last 5 minutes of their time to make sure there are no down points. The code grows much worse in the process.

• If you don't: you break the "rule of law": that people know the mechanism for giving points (also, that's why you always give them the first test in the text of the problem).

Update: Sorry, I didn't notice at first that you ask some specific questions. Sandboxing may be less important than you think -- in a good competition the code becomes publicly available, so the "hackers" would be really embarrassed. However, I think I saw a practice where you can't do the i/o, filesystem or any other interaction with the system directly (they write main() for you and it's always the same; you only write the algorithm part with given input/output streams). Your judge should run only what it itself compiled from the source.

Answer 2

Have a look at wikipedia. At the end it has great resources.

Answer 3

One search term might be for autograder.

For example:

http://www.users.muohio.edu/helmicmt/autograder/index.php
http://prisms.cs.umass.edu/mcorner/autograder

http://74.125.95.132/search?q=cache:VSuCE566d1oJ:www.cs.odu.edu/~gpd/msprojects/cpasupul.0/AutoGrader.ppt+autograder&cd=5&hl=en&ct=clnk&gl=us&client=firefox-a

Answer 4

I am not sure what an online judge is, but I assume it is a piece of software to evaluate programs for correctness.

I would use some build, test and analysis libraries for this. Examples would be Ant, JUnit, and Checkstyle.

You would take the code provided by the participant, and drop it into a file. Use the build tool to compile it.

• build fails: 0 points
• build succeeds with warnings 1 point
• build succeeds without warnings 2 points

Then run some tests, that verify the correctness of the solution.

• For each test passed: 1 point

Finally run some code analysis utility to judge the quality of the code.

• minus 1 point for each complaint of the utility

Of course you might want to shift the point values to your needs.