Prevent Lua infinite loop

Question

I use lua interfaces to get lua support in my C# program, the worker thread will freeze if the user submits code like this

while true do end

<

Answer 1

Sandboxing Lua

Setting hooks is not sufficient at all to prevent unintended waste of resources, let alone abuse- here's a simple example (the time is spent during string pattern matching- no debug hooks get called):

s=('a'):rep(20000):match('.-b')

The only reliable way to force time/memory constraints on a piece of Lua code is to run the Lua interpreter in a process of its own and to make your OS monitor that process.

The nice thing with Lua is that you won't need any complicated, OS-dependent permission setup for sandboxing: you just limit time and memory (reasonable; on windows there are Job Objects, Unix has ulimits- relevant: Linux resource limitation) and then keep things like os.execute, half the io-library and modules like luasocket away from the scripter (pretty easy).

Recovering from errors in sandboxed code

You can handle almost everything (except violation of time/memory limits) without trashing your Lua interpreter: Just wrap the execution of user-supplied code inside a pcall; if you call any Lua-API functions that might fail yourself, you need to wrap them inside a function that you can pcall, too (or set a Lua panic function and handle it from there).

---

[I didn't want people glancing at this thread to assume that debug.sethook is adequate for sandboxing, and stackoverflow would not let me comment (yet)]

Answer 2

I suggest handling it just like any other Lua error. In other words, threat the code above as if the user had just written

error("Script execution has been cancelled after stalling for too long")

(I'm assuming that you detect infinite loops by assuming that no script should take more than a fixed amount of time to run. Change the message appropiatedly if this is not the case)

The way you will have to handle this will depend on how you deal with Lua errors - but you will have to do it anyway, so chances are that most of the code is there already.

EDIT: It seems that Martin James' suggestion is your best option. You can use the debug.setHook() to run some lua code every 100 Lua instructions or so, and if too much time has passed on the same client code, throw the error. Details about this can be found on this mailing list, and code sample on the lua-project repo.

Answer 3

I have used two techniques to address this issue. The first is to call the DoFile or DoString method inside a separate Task. That way you can abort the thread. I do not know of any technique (say from within a hook) to exit the interpreter.

Another way would be to setup a separate Appdomain. Using this you can also set separate constraints (evidence and permission set) see CreateDomain for details.

Answer 4

Put the Lua loop inside a coroutine and at the end of every loop, just use Yield return null to wait for a frame (Allowing the worker thread to be executed).

Answer 5

What about setting line hook before running lua code? I don't know is it possible in C#, but it is possible in original C lua.

Is yor loop detector already run in different C# task?

• If yes, use C# variable with C# hook. In this case your loop detector set and hool check C# variable like terminateLua and throw lua error (it should be possible).
• If no, and you detect loop on lua side (also using hooks?), set lua variable (make it upvalue to prevent user tricks) and also throw error.

Important part is to don't reset this (either C# or lua) variable before you are really sure about finishing your task, because user can catch your error via pcall and try to handle it.