[PATCH v2] test-patchset: allow proper support for randconfig

[Keller, Jacob E]

On Tue, 2014-07-29 at 10:41 +0300, Artem Bityutskiy wrote:
> On Mon, 2014-07-28 at 20:23 +0000, Keller, Jacob E wrote:
> > On Tue, 2014-07-15 at 14:58 -0700, Jacob Keller wrote:
> > > Make randconfig pre-generate a configuration so that the pre and post
> > > patch series builds use the same configuration (vs using random configs
> > > each time). In addition, append the random configuration whenever there
> > > is a build diff, so that the user can see.
> > > 
> > > Signed-off-by: Jacob Keller <jacob.e.keller at intel.com>
> > > ---
> > 
> > I did some live testing of this, there were a few issues, and one thing
> > I am wondering how we could implement:
> > 
> > Is it possible to make it so we add the configuration as an attachment
> > of the email? I don't know how we would manage this.. but that would be
> > ideal as the information spat out by the system is completely pointless
> > if i can't see the config. I tried printing it but apparently that
> > doesn't get caught by the email.
> > 
> > Attachment would be the most ideal method of passing the random
> > configuration.
> > 
> > Any initial thoughts on how to design a point for passing back
> > attachments from the test-patchset portion into the email portion?
> 
> Well, quick thoughts are... The config is some kind of out-of-band data,
> because aiaiai should handle it differently comparing to how it handle
> patches. The difference is that the config should be applied before
> doing both the patched and non-patched builds.
> 
> In other words, the config should be applied to the base commit.
> 
> And you may think about other scenarios, where the base commit is
> broken, but you want to test against it anyway. So you apply a hotfix,
> or a quirk, to the base commit before doing the patched and non-patched
> tests.
> 
> So my point is that in theory this may be a general patch, not just a
> config.
> 
> So from this point of view, I'd say that one way to go is to mark a
> number of (first) patches in the series with a "base-tree" flag, and
> then make aiaiai treat them a special way - apply them to the base tree,
> and only test the other patches on top of them (the "base-tree" patches
> would not be tested, they are assumed to just work fine).
> 
> How to label the patches? Not sure, a keyword in the subject, or a mail
> header, or both. And probably "base-tree" is not the best term.
> 
> So, say, you have 3 patches to test against some config. You send a
> series of 4 patches. The 1st is contains the config and it is marked as
> "base-tree". Aiaiai will apply it to the base tree, and will test the
> other 3 patches on top of that base tree.
> 
> Now, from the user perspective, it may be easier to just attach the
> config file to the first patch or the cover-letter. Not sure. Although I
> do not know how to do this with git-send-email, and for me defconfig as
> a patch is more natural. This could be implemented as a special case,
> where one of the early scripts translates the attachment to a patch and
> pretends the series with this patch. But this may be difficult to do,
> not sure.
> 
> Generally, aiaiai works with patches, so handling an '.config'
> attachements is harder than handling a '.config-as-a-patch'... But it
> may be simpler and nicer to just add support for config-as-attachement
> without considering the more general picture.
> 

I think you misunderstand. I want *aiaiai* to send the config it
generated via make randconfig back as part of its response in an
attachment so that it doesn't clutter the plainbody of the email
message.

I am unsure how to send this data out-of-band back to the
aiaiai-email-test-patchset.

I do like the idea of users sending their own configs, though I'm unsure
exactly how to handle that yet.

Thanks,
Jake