[PATCH 00/13] Support Autodetect Projects as example hook

[Jacob Keller]

This patch series adds support for autodetection of the project, as an example
hook. The hook uses a perl tool called "git-find-base" I created which takes an
mbox input and uses the diff-index information to bind matching blobs. Doing
this allows finding the most recent commit that can apply the mbox cleanly.

In order to implement this, several changes are required as well as some bug fixes.

First, fixup the aiaiai-email scripts and aiaiai-gerrit scripts to use the
correct srcdir, which in turn fixes the switch_to_dash checks.

Next, fix a few bugs in the dispatcher. We can't process the config file
without including the aiaiai-email-sh-functions block, and we were passing the
old handle $mbox instead of $file to the dispatcher hook.

In accordance with request, change the CONFIGURATION to configuration.txt

A minor change, update config so that branch can't default. Possibly should
instead make path default as well, I am still on the fence as to whether this
is useful.

Update the dispatcher to check for internal errors on the hook script. Assume
same semantics as aiaiai-email-test-patchset, which only returns non-zero on
internal failure, not on general patch failure.

Now we start to get to the interesting features.

1) add new config variable "canonical_url" which should be used for displaying
   the url of the repository to the user. It does not get used internally for
   other purposes, but can be a nice sugar for displaying emails with a public
   facing url.

2) Add custom header "X-Aiaiai-Base-Commit" which supplies the base commit to
   test against instead of using the tip of the project. A hook could then
   determine the full base instead of just what the project is.

3) Add custom header "X-Aiaiai-Cancel-Email-Test-Patchset" which allows hook to
   stop the patch testing, if it determines that there is no value in running
   tests.

4) Update doc file explaining hook configuration and how a hook should use the
   config file.

Now, the *real* fun is with the next few patches. First add a perl script which takes mbox in stdin, as well as any git-log options.

It runs git-log along with --pretty format of "%T %H" which displays the tree
object id, and the commit object id. Doing this allows the caller to have full
control over how to limit what commits are checked.

Git patches contain the diff lines along with the index information, which can
be used to tell which blob object that diff chunk applies to.

git-find-base goes in reverse, by extracting filenames (equivalent of trees!)
and the blob information. Once done, it checks each commit and tests whether
the commit has matching blobs in its tree for every single path/blob combo from
the diff.

The first commit which does so is displayed to STDIN. This way, we can
automatically find which commit on a given branch is applicable. We don't need
to do any fancy subject or Change-id lines. This works even if the patch was
not based on the tip of the branch, since we can have the log include any
sequence of commits we think are valid to test against. In addition, even if
the branch has updated, it is possible to still find a valid commit, due to
looking at the blob object indexes. This actually enables us to find if the
patch will apply cleanly to the tip of the queue or not. Since the index blobs
almost certainly have to match for a clean application, this is good. It *is*
possible that the modified files would still apply cleanly to the tip of the
branch even if the blobs didn't match, but I think the requirement that all
submissions be based on the last known tip of the project is a fine
requirement. (In theory, some sort of testing could be done to see if the patch
would apply cleanly without matching blobs first, but that is a bit more complex.)

Next, we add a hook which uses the cfgfile to parse out projects, and attempt
to find-base on the projects in order (starting with the supplied project..
*if* one was supplied. It would no longer be required!).

If we find a valid project, we can then test against that project instead of
the supplied project, or instead of failing if no project was supplied.

Finally, I have the last carried patch which is my systemd scripts. I have
removed the queue script, since it would no longer be required in this new
model. In addition, I added a simple aiaiai-project-update script which just
calls git-remote-update on all projects in the cfgfile. These services make
controlling the aiaiai-email-dispatcher quite simple.

Jacob Keller (13):
  aiaiai: fixup srcdir in email and gerrit scripts
  dispatcher: add aiaiai-email-sh-functions to source list
  dispatcher: use $file instead of $mbox
  doc: rename CONFIGURATION to configuration.txt
  email: don't allow pcfg_branch to default
  dispatcher: check of hook returns error
  config: add canonical_url variable for future use
  email: add support for X-Aiaiai-Base-Commit
  email: add support for X-Aiaiai-Cancel-Email-Test-Patchset
  doc: add section explaining hook configuration variables
  add git-find-base utility for finding base commit from mbox
  example hook for aiaiai-email-dispatcher-helper
  systemd: add scripts for running the aiaiai processes

 doc/email/{CONFIGURATION => configuration.txt} |  36 +++-
 doc/email/example-aiaiai.cfg                   |  31 ++--
 email/aiaiai-email-autodetect-project          | 222 +++++++++++++++++++++++++
 email/aiaiai-email-dispatcher                  |   4 +-
 email/aiaiai-email-dispatcher-helper           |  24 ++-
 email/aiaiai-email-lda                         |   4 +-
 email/aiaiai-email-sh-functions                |  22 ++-
 email/aiaiai-email-test-patchset               |  24 ++-
 gerrit/aiaiai-jenkins-test-patchset            |   4 +-
 helpers/git-find-base                          | 131 +++++++++++++++
 systemd/aiaiai-project-update                  |  82 +++++++++
 systemd/aiaiai-project-update.service          |  10 ++
 systemd/aiaiai-project-update.timer            |  12 ++
 systemd/aiaiai.service                         |  11 ++
 systemd/email.conf                             |  13 ++
 systemd/send-mail-on-failure.sh                |   7 +
 systemd/send-mail-on-failure at .service          |   6 +
 17 files changed, 596 insertions(+), 47 deletions(-)
 rename doc/email/{CONFIGURATION => configuration.txt} (87%)
 create mode 100755 email/aiaiai-email-autodetect-project
 create mode 100755 helpers/git-find-base
 create mode 100755 systemd/aiaiai-project-update
 create mode 100644 systemd/aiaiai-project-update.service
 create mode 100644 systemd/aiaiai-project-update.timer
 create mode 100644 systemd/aiaiai.service
 create mode 100644 systemd/email.conf
 create mode 100755 systemd/send-mail-on-failure.sh
 create mode 100644 systemd/send-mail-on-failure at .service

-- 
1.8.3.1