[PATCH v2] test-patchset: allow proper support for randconfig

[Artem Bityutskiy]

On Mon, 2014-07-28 at 20:23 +0000, Keller, Jacob E wrote:
> On Tue, 2014-07-15 at 14:58 -0700, Jacob Keller wrote:
> > Make randconfig pre-generate a configuration so that the pre and post
> > patch series builds use the same configuration (vs using random configs
> > each time). In addition, append the random configuration whenever there
> > is a build diff, so that the user can see.
> > 
> > Signed-off-by: Jacob Keller <jacob.e.keller at intel.com>
> > ---
> 
> I did some live testing of this, there were a few issues, and one thing
> I am wondering how we could implement:
> 
> Is it possible to make it so we add the configuration as an attachment
> of the email? I don't know how we would manage this.. but that would be
> ideal as the information spat out by the system is completely pointless
> if i can't see the config. I tried printing it but apparently that
> doesn't get caught by the email.
> 
> Attachment would be the most ideal method of passing the random
> configuration.
> 
> Any initial thoughts on how to design a point for passing back
> attachments from the test-patchset portion into the email portion?

Well, quick thoughts are... The config is some kind of out-of-band data,
because aiaiai should handle it differently comparing to how it handle
patches. The difference is that the config should be applied before
doing both the patched and non-patched builds.

In other words, the config should be applied to the base commit.

And you may think about other scenarios, where the base commit is
broken, but you want to test against it anyway. So you apply a hotfix,
or a quirk, to the base commit before doing the patched and non-patched
tests.

So my point is that in theory this may be a general patch, not just a
config.

So from this point of view, I'd say that one way to go is to mark a
number of (first) patches in the series with a "base-tree" flag, and
then make aiaiai treat them a special way - apply them to the base tree,
and only test the other patches on top of them (the "base-tree" patches
would not be tested, they are assumed to just work fine).

How to label the patches? Not sure, a keyword in the subject, or a mail
header, or both. And probably "base-tree" is not the best term.

So, say, you have 3 patches to test against some config. You send a
series of 4 patches. The 1st is contains the config and it is marked as
"base-tree". Aiaiai will apply it to the base tree, and will test the
other 3 patches on top of that base tree.

Now, from the user perspective, it may be easier to just attach the
config file to the first patch or the cover-letter. Not sure. Although I
do not know how to do this with git-send-email, and for me defconfig as
a patch is more natural. This could be implemented as a special case,
where one of the early scripts translates the attachment to a patch and
pretends the series with this patch. But this may be difficult to do,
not sure.

Generally, aiaiai works with patches, so handling an '.config'
attachements is harder than handling a '.config-as-a-patch'... But it
may be simpler and nicer to just add support for config-as-attachement
without considering the more general picture.

-- 
Best Regards,
Artem Bityutskiy