Right now your algorithm answers question "which squares does the optimal path go through?" Your graph has a node for every square and an edge for every pair of adjacent squares.
Change it to "where does the optimal path crosses borders between squares?"
Your graph will change:
- Graph nodes: middle of every edge between adjacent squares + start + finish.
- Graph edges: in every square they connect every pair of square edges.
And now you can price differently connections of opposite square edges and connections of adjacent square edges. Giving bigger weight to the second will reduce number of zig-zags.