Learning how programming languages work

Question

I\'ve been programming for years (mainly Python), but I don\'t understand what happens behind the scenes when I compile or execute my code.

In the vein of a question I a

Answer 1

Code to execution in a nutshell

A program (code) is fed into the compiler (or interpretor).

Characters are used to form tokens (+ , identifiers, numbers) and their value is stored in some thing called a symbol table.

These tokens are put together to form statements: (int a = 6 + b * c;). Mostly in the form of a syntax tree:

                     =
                    / \
                   /   \ 
                  a     +
                       / \
                      /   \
                     6     *
                          / \
                         b   c

Within an interpretor the tree is executed directly.

With a compiler, the tree is finally translated into either intermediate code or assembler code.

You now have one or more "object files". These contain the assembler code without the precise jumps (because these values are not known yet especially if the targets are in other object files). The object files are linked together with a linker which fills in the blanks for the jumps (ans references). The output of the linker is a library (which can be linked too) or an executable file.

If you start the executable, the program data is copied into memory and there is some other link jugling to match the pointers with the correct memory locations. And then control is given to the first instruction.

Answer 2

This site has a great series of lectures on the Structure and Interpretation of Computer Programs, which is exactly the type of thing you are wanting to learn. The accompanying textbook is useful too, tho i havent personally read thru the whole thing. I think watching the lectures is pretty good, gets you about 60% of the way there.

Answer 3

Whoa, this is a huge question with tons of written books all about this. I really doubt you will get a decent answer in SO about this. You need to get to your local book store or pick up a few comp sci classes.

To give you a quick intro:

• Compiler: A program that converts written code into instructions that are natively understood by the processor.
• Interpreter: A program that reads written code and, on-the-fly, translates and gives corresponding processor-native instructions.
• Managed code: Code that runs in a virtual machine, e.g. to give cross-platform compatibility (Java).
• Virtual machine: A program that emulates the behavior, or rather the API, of a full-blown computer environment. Among other things, this gives some security advantages and cross platform compatibility.

Answer 4

If you want to know how one goes from source code to something that actually runs on a target machine, you should get a copy of the famous Red Dragon Book. I've used it for building parsers and lexical analyzers. While it dates back to 1986, and I'm sure there's been progress in the interim, as far as I can tell, it hasn't been surpassed as a text.

It appears that Addison-Wesley has done a reprint of its predecessor, the Green Dragon Book, and is passing it off as something recent, so be careful to get the genuine article.