Assembly programming - WinAsm vs Visual Studio 2017

Question

I\'m here to ask you some stuff about VS2017.
In the past I had used WinAsm for MASM and I never got problems with it.

However, when I\'m trying to do something

Answer 1

How to build a x64/x86-project with a standalone x64/x86 assembly file

1) Start Visual Studio (Community) 2017 and choose FILE - New - Project.

2) In the next window choose Empty Project.

3) Make sure, that the project is highlighted in the Solution Explorer and and choose PROJECT - Build Customizations....

4) In the next window tick masm(.targets,.props) and click on OK.

5) Choose PROJECT - Add New Item from the menu.

6) In the next window choose C++File(.cpp) and - IMPORTANT! - give it a name with an .asm extension. Click on Add.

7) Now you can fill the file with content.

Source.asm:

EXTERN GetStdHandle : PROC
EXTERN WriteFile    : PROC
EXTERN ExitProcess  : PROC

.DATA?
    hFile           QWORD ?
    BytesWritten    DWORD ?

.DATA
    hello   BYTE 'Hello world!', 13, 10

.CODE
main PROC
    ; https://blogs.msdn.microsoft.com/oldnewthing/20160623-00/?p=93735
    sub rsp, 40                 ; Shadow space (4 * 8) & 1 parameter (8 bytes)
    ; https://docs.microsoft.com/en-us/cpp/build/stack-allocation
    and spl, -16                ; Align to 16

    ; https://msdn.microsoft.com/library/windows/desktop/ms683231.aspx
    mov ecx, -11                ; DWORD         nStdHandle = STD_OUTPUT_HANDLE
    call GetStdHandle           ; Call WinApi
    mov hFile, rax              ; Save returned handle

    ; https://msdn.microsoft.com/library/windows/desktop/aa365747.aspx
    mov rcx, hFile              ; HANDLE        hFile (here: Stdout)
    lea rdx, hello              ; LPCVOID       lpBuffer
    lea r9, BytesWritten        ; LPDWORD       lpNumberOfBytesWritten
    mov r8d, LENGTHOF hello     ; DWORD         nNumberOfBytesToWrite
    mov qword ptr [rsp+32], 0   ; LPOVERLAPPED  lpOverlapped = NULL
    call WriteFile              ; Call WinAPI

exit:
    ; https://msdn.microsoft.com/library/windows/desktop/ms682658.aspx
    xor ecx, ecx                ; Set RCX to null for return value
    call ExitProcess            ; Call WinAPI to exit
main ENDP

end

This is a 64-bit Console application that starts at the procedure main.

8) Change the Solution Platforms to x64

9) Choose PROJECT - Properties.

10) In the Properties window you have to complete two linker options:

• Entry Point: main
• SubSystem: Console (/SUBSYSTEM:CONSOLE)

Choose at the left side Configuration Properties - Linker - All Options , change both options at once and click OK.

11) Build and run the .exe with CTRL-F5. The application will be opened in a new window.

---

Now overwrite Source.asm with a 32-bit Console application:

.MODEL flat, stdcall

; https://docs.microsoft.com/en-us/cpp/assembler/masm/proto
GetStdHandle PROTO STDCALL,     ; https://docs.microsoft.com/en-us/windows/console/getstdhandle
    nStdHandle: SDWORD
WriteFile PROTO STDCALL,        ; https://docs.microsoft.com/en-us/windows/desktop/api/fileapi/nf-fileapi-writefile
    hFile: DWORD,                       ; output handle
    lpBuffer: PTR BYTE,                 ; pointer to buffer
    nNumberOfBytesToWrite: DWORD,       ; size of buffer
    lpNumberOfBytesWritten: PTR DWORD,  ; num bytes written
    lpOverlapped: PTR DWORD             ; ptr to asynchronous info
ExitProcess PROTO STDCALL,      ; https://docs.microsoft.com/en-us/windows/desktop/api/processthreadsapi/nf-processthreadsapi-exitprocess
    dwExitCode: DWORD                   ; return code

.DATA                   ; https://docs.microsoft.com/en-us/cpp/assembler/masm/dot-data
    Hallo db "Hello world!",13,10

.DATA?                  ; https://docs.microsoft.com/en-us/cpp/assembler/masm/dot-data-q
    lpNrOfChars dd ?

.CODE                   ; https://docs.microsoft.com/en-us/cpp/assembler/masm/dot-code
main PROC               ; docs.microsoft.com/en-us/cpp/assembler/masm/proc
    invoke GetStdHandle, -11            ; -> StdOut-Handle into EAX
    invoke WriteFile, eax, OFFSET Hallo, LENGTHOF Hallo, OFFSET lpNrOfChars, 0
    invoke ExitProcess, 0
main ENDP

END main                ; https://docs.microsoft.com/en-us/cpp/assembler/masm/end-masm

Change the Solution Platforms to x86 (No. 8 above) and complete the project properties with SubSystem: Console (/SUBSYSTEM:CONSOLE) (No. 10 above). You must not set the Entry point, because ml32 expects the entry point after the END directive (END main). Build and run the .exe with CTRL-F5.