How to get Git log with short stat in one line?

问题

Following command outputs following lines of text on console

git log --pretty=format:"%h;%ai;%s" --shortstat
ed6e0ab;2014-01-07 16:32:39 +0530;Foo
 3 files changed, 14 insertions(+), 13 deletions(-)

cdfbb10;2014-01-07 14:59:48 +0530;Bar
 1 file changed, 21 insertions(+)

5fde3e1;2014-01-06 17:26:40 +0530;Merge Baz
772b277;2014-01-06 17:09:42 +0530;Qux
 7 files changed, 72 insertions(+), 7 deletions(-)

I'm interested in having above format to be displayed like this

ed6e0ab;2014-01-07 16:32:39 +0530;Foo;3;14;13
cdfbb10;2014-01-07 14:59:48 +0530;Bar;1;21;0
5fde3e1;2014-01-06 17:26:40 +0530;Merge Baz;0;0;0
772b277;2014-01-06 17:09:42 +0530;Qux;7;72;7

This will be consumed in some report which can parse semicolon separated values. The thing is the text "\n 3 files changed, 14 insertions(+), 13 deletions(-)" (new line included) gets converted to 3;14;13 (without new line) One possible corner case is text like "5fde3e1;2014-01-06 17:26:40 +0530;Merge Baz" which doesn't have such line. In that case I want ;0;0;0

Overall the goal is to analyze file change stats over a period of time. I read the git log documentation but couldn't find any format which will help me to render in this format. The best I came up was the above command mentioned.

So any command or shell script which can generate the expected format would be of great help.

Thanks!

回答1:

This is, unfortunately, impossible to achieve using only git log. One has to use other scripts to compensate for something most people aren't aware of: some commits don't have stats, even if they are not merges.

I have been working on a project that converts git log to JSON and to get it done I had to do what you need: get each commit, with stats, in one line. The project is called Gitlogg and you're welcome to tweak it to your needs: https://github.com/dreamyguy/gitlogg

Below is the relevant part of Gitlogg, that will get you close to what you'd like:

git log --all --no-merges --shortstat --reverse --pretty=format:'commits\tcommit_hash\t%H\tcommit_hash_abbreviated\t%h\ttree_hash\t%T\ttree_hash_abbreviated\t%t\tparent_hashes\t%P\tparent_hashes_abbreviated\t%p\tauthor_name\t%an\tauthor_name_mailmap\t%aN\tauthor_email\t%ae\tauthor_email_mailmap\t%aE\tauthor_date\t%ad\tauthor_date_RFC2822\t%aD\tauthor_date_relative\t%ar\tauthor_date_unix_timestamp\t%at\tauthor_date_iso_8601\t%ai\tauthor_date_iso_8601_strict\t%aI\tcommitter_name\t%cn\tcommitter_name_mailmap\t%cN\tcommitter_email\t%ce\tcommitter_email_mailmap\t%cE\tcommitter_date\t%cd\tcommitter_date_RFC2822\t%cD\tcommitter_date_relative\t%cr\tcommitter_date_unix_timestamp\t%ct\tcommitter_date_iso_8601\t%ci\tcommitter_date_iso_8601_strict\t%cI\tref_names\t%d\tref_names_no_wrapping\t%D\tencoding\t%e\tsubject\t%s\tsubject_sanitized\t%f\tcommit_notes\t%N\tstats\t' |
  sed '/^[ \t]*$/d' |               # remove all newlines/line-breaks, including those with empty spaces
  tr '\n' 'ò' |                     # convert newlines/line-breaks to a character, so we can manipulate it without much trouble
  tr '\r' 'ò' |                     # convert carriage returns to a character, so we can manipulate it without much trouble
  sed 's/tòcommits/tòòcommits/g' |  # because some commits have no stats, we have to create an extra line-break to make `paste -d ' ' - -` consistent
  tr 'ò' '\n' |                     # bring back all line-breaks
  sed '{
      N
      s/[)]\n\ncommits/)\
  commits/g
  }' |                              # some rogue mystical line-breaks need to go down to their knees and beg for mercy, which they're not getting
  paste -d ' ' - -                  # collapse lines so that the `shortstat` is merged with the rest of the commit data, on a single line

Note that I've used the tab character ( \t ) to separate fields as ; could have been used on the commit message.

Another important part of this script is that each line must begin with an unique string (in this case it's commits). That's because our script needs to know where the line begins. In fact, whatever comes after the git log command is there to compensate for the fact that some commits might not have stats.

But it strikes me that what you want to achieve is to have commits neatly outputted in a format you can reliably consume. Gitlogg is perfect for that! Some of its features are:

Parse the git log of multiple repositories into one JSON file.
Introduced repository key/value.
Introduced files changed, insertions and deletions keys/values.
Introduced impact key/value, which represents the cumulative changes for the commit (insertions - deletions).
Sanitise double quotes " by converting them to single quotes ' on all values that allow or are created by user input, like subject.
Nearly all the pretty=format: placeholders are available.
Easily include / exclude which keys/values will be parsed to JSON by commenting out/uncommenting the available ones.
Easy to read code that's thoroughly commented.
Script execution feedback on console.
Error handling (since path to repositories needs to be set correctly).

Success, the JSON was parsed and saved.

Error 001: path to repositories does not exist.

Error 002: path to repositories exists, but is empty.

回答2:

git log  --oneline --pretty="@%h"  --stat   |grep -v \| |  tr "\n" " "  |  tr "@" "\n"

This will show something like this:

a596f1e   1 file changed, 6 insertions(+), 3 deletions(-) 
4a9a4a1   1 file changed, 6 deletions(-) 
b8325fd   1 file changed, 65 insertions(+), 4 deletions(-) 
968ef81   1 file changed, 4 insertions(+), 5 deletions(-)

回答3:

git doesn't support stat info with plain --format, which is shame :( but it's easy to script it away, here's my quick and dirty solution, should be quite readable:

#!/bin/bash

format_log_entry ()
{
    read commit
    read date
    read summary
    local statnum=0
    local add=0
    local rem=0
    while true; do
        read statline
        if [ -z "$statline" ]; then break; fi
        ((statnum += 1))
        ((add += $(echo $statline | cut -d' ' -f1)))
        ((rem += $(echo $statline | cut -d' ' -f2)))
    done
    if [ -n "$commit" ]; then
        echo "$commit;$date;$summary;$statnum;$add;$rem"
    else
        exit 0
    fi
}

while true; do
    format_log_entry
done

I'm sure, that it can be scripted better, but hey - it's both quick AND dirty ;)

usage:

$ git log --pretty=format:"%h%n%ai%n%s" --numstat | ./script

Please note, that format, that you specified is not bulletproof. Semicolon can appear in commit summary, which will break number of fields in such line - you can either move summary to end of line or escape it somehow - how do you want to do it?

回答4:

This is one approach with awk.

awk 'BEGIN{FS="[,;]"; OFS=";"} /;/ {a=$0} /^ /{gsub(/[a-z(+-) ]/,"") gsub(",",";"); print a,$0}'

For the given input it returns:

ed6e0ab;2014-01-07 16:32:39 +0530;Foo;3;14;13
cdfbb10;2014-01-07 14:59:48 +0530;Bar;1;21
772b277;2014-01-06 17:09:42 +0530;Qux;7;72;7

Still not working for lines like 5fde3e1;2014-01-06 17:26:40 +0530;Merge Baz that do not have a 3 files changed, 14 insertions(+), 13 deletions(-) after it.

回答5:

Following up @user2461539 to parse it into columns. Works with more complex cols like "Subject" too. Hack away to choose your own suitable delimiters. Currently need to cut subject line as it'll truncate other columns when it overflows.

#!/bin/bash
# assumes "_Z_Z_Z_" and "_Y_Y_" "_X_X_" as unused characters 
# Truncate subject line sanitized (%f) or not (%s) to 79 %<(79,trunc)%f
echo commit,author_name,time_sec,subject,files_changed,lines_inserted,lines_deleted>../tensorflow_log.csv;
git log --oneline --pretty="_Z_Z_Z_%h_Y_Y_\"%an\"_Y_Y_%at_Y_Y_\"%<(79,trunc)%f\"_Y_Y__X_X_"  --stat    \
    | grep -v \| \
    | sed -E 's/@//g' \
    | sed -E 's/_Z_Z_Z_/@/g' \
    |  tr "\n" " "   \
    |  tr "@" "\n" |sed -E 's/,//g'  \
    | sed -E 's/_Y_Y_/, /g' \
    | sed -E 's/(changed [0-9].*\+\))/,\1,/'  \
    | sed -E 's/(changed [0-9]* deleti.*-\)) /,,\1/' \
    | sed -E 's/insertion.*\+\)//g' \
    | sed -E 's/deletion.*\-\)//g' \
    | sed -E 's/,changed/,/' \
    | sed -E 's/files? ,/,/g'  \
    | sed -E 's/_X_X_ $/,,/g'  \
    | sed -E 's/_X_X_//g'>>../tensorflow_log.csv

回答6:

I put something like this in my ~/.bashrc:

function git-lgs() {
   git --no-pager log --numstat --format=%ai "$1" | sed ':a;N;$!ba;s/\n\n/\t/g' | sed 's/\(\t[0-9]*\t*[0-9]*\).*/\1/'
}

Where git-lgs's argument is the filename for which you want to display the log.

回答7:

combining all answers above, here are my 2 cents in case anyone is looking:

echo "commit id,author,date,comment,changed files,lines added,lines deleted" > res.csv 
git log --since='last year'  --date=local --all --pretty="%x40%h%x2C%an%x2C%ad%x2C%x22%s%x22%x2C" --shortstat | tr "\n" " " | tr "@" "\n" >> res.csv
sed -i 's/ files changed//g' res.csv
sed -i 's/ file changed//g' res.csv
sed -i 's/ insertions(+)//g' res.csv
sed -i 's/ insertion(+)//g' res.csv
sed -i 's/ deletions(-)//g' res.csv
sed -i 's/ deletion(-)//g' res.csv

and either save it into git-logs-into-csv.sh file or just copy/paste into console.

I think it's relatively self-explaining but just in case: