Learning assembly [closed]

Answer 1

I recently took a computer systems class. One of the topics was Assembly as a tool to communicate with the hardware.

For me, the knowledge of assembly wouldn't have been complete without understanding the details of how computer systems work. Understanding that, brings in an new understanding of why assembly instructions on one processor architecture is great but is terrible on another architecture.

Given this, I'm inclined to recommend my class text book:

Computer Systems:A programmer's perspective.

_{(source: cmu.edu)}

It does cover x86 assembly but the book is much more broad than that. It covers processor pipe-lining and memory as a cache, the virtual memory system and much more. All of this can affect how assembly could be optimized for the given features.

Answer 2

Off topic I know, but since you are a Windows programmer I can't help but think that it may be a more appropriate and/or better use of your time to learn MSIL. No, it's not assembly, but it's probably more relevant in this .NET era.

Answer 3

The suggestion to use debug is a fun one, many neat tricks can be done with that. However, for a modern operating system, learning 16bit assembly may be slightly less useful. Consider, instead, using ntsd.exe. It's built into Windows XP (it was yanked in Server 2003 and above, unfortunately), which makes it a convenient tool to learn since it's so widely available.

That said, the original version in XP suffers from a number of bugs. If you really want to use it (or cdb, or windbg, which are essentially different interfaces with the same command syntax and debugging back-end), you should install the free windows debugging tools package.

The debugger.chm file included in that package is especially useful when trying to figure out the unusual syntax.

The great thing about ntsd is you can pop it up on any XP machine you're near and use it to assembly or disassemble. It makes a /great/ X86 assembly learning tool. For example (using cdb since it's inline in the dos prompt, it's otherwise identical):

(symbol errors skipped since they're irrelevant -- also, I hope this formatting works, this is my first post)

C:\Documents and Settings\User>cdb calc

Microsoft (R) Windows Debugger Version 6.10.0003.233 X86
Copyright (c) Microsoft Corporation. All rights reserved.

CommandLine: calc
Symbol search path is: *** Invalid ***
Executable search path is:
ModLoad: 01000000 0101f000   calc.exe
ModLoad: 7c900000 7c9b2000   ntdll.dll
ModLoad: 7c800000 7c8f6000   C:\WINDOWS\system32\kernel32.dll
ModLoad: 7c9c0000 7d1d7000   C:\WINDOWS\system32\SHELL32.dll
ModLoad: 77dd0000 77e6b000   C:\WINDOWS\system32\ADVAPI32.dll
ModLoad: 77e70000 77f02000   C:\WINDOWS\system32\RPCRT4.dll
ModLoad: 77fe0000 77ff1000   C:\WINDOWS\system32\Secur32.dll
ModLoad: 77f10000 77f59000   C:\WINDOWS\system32\GDI32.dll
ModLoad: 7e410000 7e4a1000   C:\WINDOWS\system32\USER32.dll
ModLoad: 77c10000 77c68000   C:\WINDOWS\system32\msvcrt.dll
ModLoad: 77f60000 77fd6000   C:\WINDOWS\system32\SHLWAPI.dll
(f2c.208): Break instruction exception - code 80000003 (first chance)
eax=001a1eb4 ebx=7ffd6000 ecx=00000007 edx=00000080 esi=001a1f48 edi=001a1eb4
eip=7c90120e esp=0007fb20 ebp=0007fc94 iopl=0         nv up ei pl nz na po nc
cs=001b  ss=0023  ds=0023  es=0023  fs=003b  gs=0000             efl=00000202
ntdll!DbgBreakPoint:
7c90120e cc              int     3
0:000> r eax
eax=001a1eb4
0:000> r eax=0
0:000> a eip
7c90120e add eax,0x100
7c901213
0:000> u eip
ntdll!DbgBreakPoint:
7c90120e 0500010000      add     eax,100h
7c901213 c3              ret
7c901214 8bff            mov     edi,edi
7c901216 8b442404        mov     eax,dword ptr [esp+4]
7c90121a cc              int     3
7c90121b c20400          ret     4
ntdll!NtCurrentTeb:
7c90121e 64a118000000    mov     eax,dword ptr fs:[00000018h]
7c901224 c3              ret
0:000> t
eax=00000100 ebx=7ffd6000 ecx=00000007 edx=00000080 esi=001a1f48 edi=001a1eb4
eip=7c901213 esp=0007fb20 ebp=0007fc94 iopl=0         nv up ei pl nz na pe nc
cs=001b  ss=0023  ds=0023  es=0023  fs=003b  gs=0000             efl=00000206
ntdll!DbgUserBreakPoint+0x1:
7c901213 c3              ret
0:000>`

Also -- while you're playing with IDA, make sure to check out the IDA Pro Book by Chris Eagle (unlinked since StackOverflow doesn't want to let me post more than two links for my first post). It's hands-down the best reference out there.

Answer 4

The assembly you would write by hand and the assembly generated by a compiler are often very different when viewed from a high level. Of course, the innards of the program will be very similar (there are only so many different ways to encode a = b + c, after all), but they're not the trouble when you're trying to reverse engineer something. The compiler will add a ton of boilerplate code to even simple executables: last time I compared, "Hello World" compiled by GCC was about 4kB, while if written by hand in assembly it's around 100 bytes. It's worse on Windows: last time I compared (admittedly, this was last century) the smallest "Hello World" I could get my Windows compiler of then-choice to generate was 52kB! Usually this boilerplate is only executed once, if at all, so it doesn't much affect program speed -- like I said above, the core of the program, the part where most execution time is spent, is usually pretty similar whether compiled or written by hand.

At the end of the day, this means that an expert assembly programmer and an expert disassembler are two different specialties. Commonly they're found in the same person, but they're really separate, and learning how to be an excellent assembly coder won't help you that much to learn reverse engineering.

What you want to do is grab the IA-32 and AMD64 (both are covered together) architecture manuals from Intel and AMD, and look through the early sections on instructions and opcodes. Maybe read a tutorial or two on assembly language, just to get the basics of assembly language down. Then grab a small sample program that you're interested in and disassemble it: step through its control flow and try to understand what it's doing. See if you can patch it to do something else. Then try again with another program, and repeat until you're comfortable enough to try to achieve a more useful goal. You might be interested in things like "crackmes", produced by the reverse engineering community, which are challenges for people interested in reverse engineering to try their hand at, and hopefully learn something along the way. They range in difficulty from basic (start here!) to impossible.

Above all, you just need to practice. As in many other disciplines, with reverse engineering, practice makes perfect... or at least better.

Answer 5

To actually reach your goal, you might consider starting with the IDE you are in. The generally is a disassembler window, so you can do single stepping through code. There is usually a view of some sort to let you see the registers and look into memory areas.

Examination of unoptimized c/c++ code will help build a link into the kind of code that the compiler generates for your sources. Some compilers have some sort of ASM reserved word which lets you insert machine instructions in your code.

My advice would be to play around with those sorts of tools for a while and get your feet wet, then step up? down? to straight assembler code on what ever platform you are running on.

There are a lot of great tools out there, but you might find it more fun, to avoid the steep learning curve at first.

Answer 6

You can check out xorpd x86 Assembly video course. (I wrote it). The course itself is paid, but the exercises are open sourced, on github. If you have some programming experience I think you should be able to work just with the exercises and understand everything.

Note that the code is for the Windows platform, and is written using the Fasm assembler. The course and the exercises do not contain any high level constructs, however you could use Fasm to create very complicated macros, if you ever wish to do so.