I have tried out and combined solutions from ethan-kulla and eatmeimadanish
The following code works:
my_func = '''
def function(a,b):
constant = {input_var}
return a*b + constant
'''
exec(my_func.format(input_var=5), globals())
# For Python 3.x, use globals() to change global context
function(2,2)