From vgoyal at redhat.com  Fri Jan  2 04:54:23 2015
From: vgoyal at redhat.com (Vivek Goyal)
Date: Fri, 2 Jan 2015 07:54:23 -0500
Subject: [PATCH] kexec, remove panic_on_warn kernel parameter from kdump
 situations
In-Reply-To: <1419951471-19765-1-git-send-email-prarit@redhat.com>
References: <1419951471-19765-1-git-send-email-prarit@redhat.com>
Message-ID: <20150102125423.GA18785@redhat.com>

On Tue, Dec 30, 2014 at 09:57:51AM -0500, Prarit Bhargava wrote:
> panic_on_warn kernel parameter will cause the kernel to panic when a
> WARN() is hit in the kernel.  This is not a good situation for the kdump
> kernel because then it would be possible for the kdump kernel to panic in
> a non-fatal WARN().
> 
> This patch removes panic_on_warn as a kernel parameter for the kdump
> kernel.
> 

I think modifying kexec-tools is not best place for this. It probably is better to take care of this in distribution specific scripts.

In the past we have learnt that it is best that kexec-tools does least
amount of manipulation with command line.

Thanks
Vivek

> Signed-off-by: Prarit Bhargava <prarit at redhat.com>
> Cc: Dave Young <dyoung at redhat.com>
> Cc: Vivek Goyal <vgoyal at redhat.com>
> Cc: WANG Chao <chaowang at redhat.com>
> ---
>  kexec/kexec.c |    4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
> 
> diff --git a/kexec/kexec.c b/kexec/kexec.c
> index b088916..323cafb 100644
> --- a/kexec/kexec.c
> +++ b/kexec/kexec.c
> @@ -1048,8 +1048,10 @@ char *get_command_line(void)
>  	line[strlen(line) - 1] = '\0';
>  
>  	remove_parameter(line, "BOOT_IMAGE");
> -	if (kexec_flags & KEXEC_ON_CRASH)
> +	if (kexec_flags & KEXEC_ON_CRASH) {
>  		remove_parameter(line, "crashkernel");
> +		remove_parameter(line, "panic_on_warn");
> +	}
>  
>  	return line;
>  }
> -- 
> 1.7.9.3


From prarit at redhat.com  Fri Jan  2 05:07:20 2015
From: prarit at redhat.com (Prarit Bhargava)
Date: Fri, 02 Jan 2015 08:07:20 -0500
Subject: [PATCH] kexec, remove panic_on_warn kernel parameter from kdump
 situations
In-Reply-To: <20150102125423.GA18785@redhat.com>
References: <1419951471-19765-1-git-send-email-prarit@redhat.com>
 <20150102125423.GA18785@redhat.com>
Message-ID: <54A69808.4010401@redhat.com>



On 01/02/2015 07:54 AM, Vivek Goyal wrote:
> On Tue, Dec 30, 2014 at 09:57:51AM -0500, Prarit Bhargava wrote:
>> panic_on_warn kernel parameter will cause the kernel to panic when a
>> WARN() is hit in the kernel.  This is not a good situation for the kdump
>> kernel because then it would be possible for the kdump kernel to panic in
>> a non-fatal WARN().
>>
>> This patch removes panic_on_warn as a kernel parameter for the kdump
>> kernel.
>>
> 
> I think modifying kexec-tools is not best place for this. It probably is better to take care of this in distribution specific scripts.
> 
> In the past we have learnt that it is best that kexec-tools does least
> amount of manipulation with command line.

Well .. here's the question to think about: what does adding panic_on_warn to
the kdump kernel get you?  AFAICT, nothing.

If panic_on_warn is specified, the only thing that will happen is that kdump
will fail (which is always bad IMO).  There is no real difference in the stack
trace between the WARN() and panic situations so there is no information loss.

So I disagree -- we should never specify panic_on_warn on kdump kernel.

P.

> 
> Thanks
> Vivek
> 
>> Signed-off-by: Prarit Bhargava <prarit at redhat.com>
>> Cc: Dave Young <dyoung at redhat.com>
>> Cc: Vivek Goyal <vgoyal at redhat.com>
>> Cc: WANG Chao <chaowang at redhat.com>
>> ---
>>  kexec/kexec.c |    4 +++-
>>  1 file changed, 3 insertions(+), 1 deletion(-)
>>
>> diff --git a/kexec/kexec.c b/kexec/kexec.c
>> index b088916..323cafb 100644
>> --- a/kexec/kexec.c
>> +++ b/kexec/kexec.c
>> @@ -1048,8 +1048,10 @@ char *get_command_line(void)
>>  	line[strlen(line) - 1] = '\0';
>>  
>>  	remove_parameter(line, "BOOT_IMAGE");
>> -	if (kexec_flags & KEXEC_ON_CRASH)
>> +	if (kexec_flags & KEXEC_ON_CRASH) {
>>  		remove_parameter(line, "crashkernel");
>> +		remove_parameter(line, "panic_on_warn");
>> +	}
>>  
>>  	return line;
>>  }
>> -- 
>> 1.7.9.3


From vgoyal at redhat.com  Fri Jan  2 05:17:09 2015
From: vgoyal at redhat.com (Vivek Goyal)
Date: Fri, 2 Jan 2015 08:17:09 -0500
Subject: [PATCH] kexec, remove panic_on_warn kernel parameter from kdump
 situations
In-Reply-To: <54A69808.4010401@redhat.com>
References: <1419951471-19765-1-git-send-email-prarit@redhat.com>
 <20150102125423.GA18785@redhat.com> <54A69808.4010401@redhat.com>
Message-ID: <20150102131709.GB18785@redhat.com>

On Fri, Jan 02, 2015 at 08:07:20AM -0500, Prarit Bhargava wrote:
> 
> 
> On 01/02/2015 07:54 AM, Vivek Goyal wrote:
> > On Tue, Dec 30, 2014 at 09:57:51AM -0500, Prarit Bhargava wrote:
> >> panic_on_warn kernel parameter will cause the kernel to panic when a
> >> WARN() is hit in the kernel.  This is not a good situation for the kdump
> >> kernel because then it would be possible for the kdump kernel to panic in
> >> a non-fatal WARN().
> >>
> >> This patch removes panic_on_warn as a kernel parameter for the kdump
> >> kernel.
> >>
> > 
> > I think modifying kexec-tools is not best place for this. It probably is better to take care of this in distribution specific scripts.
> > 
> > In the past we have learnt that it is best that kexec-tools does least
> > amount of manipulation with command line.
> 
> Well .. here's the question to think about: what does adding panic_on_warn to
> the kdump kernel get you?  AFAICT, nothing.

Let us consider a hypothetical situation. What if we have some buggy code
which will corrupt file system in certain situation and we detect that
situation and throw a warning. 

In that case as a work around specifying panic_on_warn in kdump kernel
will make sense as we don't want to make further progress if we hit
the warning as it has potential to corrupt fs.

Again this is hypothetical but it can happen. So panic_on_warn might
still be useful in kdump kernel for some corner debugging cases.

That's why I think we should do it in distribution specific scripts
and that too only if user did not specify panic_on_warn for second
kernel explicitly.

> 
> If panic_on_warn is specified, the only thing that will happen is that kdump
> will fail (which is always bad IMO).  There is no real difference in the stack
> trace between the WARN() and panic situations so there is no information loss.
> 
> So I disagree -- we should never specify panic_on_warn on kdump kernel.

I am saying that do it in distribution specific scripts and not in 
kexec-tools.

Thanks
Vivek


From vgoyal at redhat.com  Fri Jan  2 05:42:40 2015
From: vgoyal at redhat.com (Vivek Goyal)
Date: Fri, 2 Jan 2015 08:42:40 -0500
Subject: [PATCH v2] kdump, vmcoreinfo: report actual value of phys_base
In-Reply-To: <20141215151120.b44112413e15a8c390d39147@linux-foundation.org>
References: <20141113.113011.630216167285231824.d.hatayama@jp.fujitsu.com>
 <20141215151120.b44112413e15a8c390d39147@linux-foundation.org>
Message-ID: <20150102134240.GD18785@redhat.com>

On Mon, Dec 15, 2014 at 03:11:20PM -0800, Andrew Morton wrote:
> 
> (cc trimmed a bit)
> 
> On Thu, 13 Nov 2014 11:30:11 +0900 (JST) HATAYAMA Daisuke <d.hatayama at jp.fujitsu.com> wrote:
> 
> > Currently, VMCOREINFO note information reports the virtual address of
> > phys_base that is assigned to symbol phys_base. But this doesn't make
> > sense because to refer to phys_base, it's necessary to get the value
> > of phys_base itself we are now about to refer to.
> 
> Folks, could we please get a bit of reviewing, acking or nacking for
> this one?

To me this patch is more like a hack or a quick fix to get dump filtering
working when dump is generated using virsh-dump like tools. I feel there
should be discussion on what's the proper way to do this. I don't have
very specific ideas though at this point of time.

Thanks
Vivek


From kuleshovmail at gmail.com  Fri Jan  2 09:56:19 2015
From: kuleshovmail at gmail.com (Alexander Kuleshov)
Date: Fri,  2 Jan 2015 23:56:19 +0600
Subject: [PATCH] kexec: Fix a typo in comment
Message-ID: <1420221379-4553-1-git-send-email-kuleshovmail@gmail.com>

Signed-off-by: Alexander Kuleshov <kuleshovmail at gmail.com>
---
 kernel/kexec.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/kernel/kexec.c b/kernel/kexec.c
index 9a8a01a..75a8b7e 100644
--- a/kernel/kexec.c
+++ b/kernel/kexec.c
@@ -444,7 +444,7 @@ arch_kexec_apply_relocations(const Elf_Ehdr *ehdr, Elf_Shdr *sechdrs,
 }
 
 /*
- * Free up memory used by kernel, initrd, and comand line. This is temporary
+ * Free up memory used by kernel, initrd, and command line. This is temporary
  * memory allocation which is not needed any more after these buffers have
  * been loaded into separate segments and have been copied elsewhere.
  */
-- 
2.2.1.202.g44ae4ee.dirty



From ebiederm at xmission.com  Fri Jan  2 10:48:51 2015
From: ebiederm at xmission.com (Eric W. Biederman)
Date: Fri, 02 Jan 2015 12:48:51 -0600
Subject: [PATCH] kexec: Fix a typo in comment
In-Reply-To: <1420221379-4553-1-git-send-email-kuleshovmail@gmail.com>
 (Alexander Kuleshov's message of "Fri, 2 Jan 2015 23:56:19 +0600")
References: <1420221379-4553-1-git-send-email-kuleshovmail@gmail.com>
Message-ID: <87vbkpvx0s.fsf@x220.int.ebiederm.org>

Alexander Kuleshov <kuleshovmail at gmail.com> writes:

> Signed-off-by: Alexander Kuleshov <kuleshovmail at gmail.com>
Acked-by: "Eric W. Biederman" <ebiederm at xmission.com>

> ---
>  kernel/kexec.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/kernel/kexec.c b/kernel/kexec.c
> index 9a8a01a..75a8b7e 100644
> --- a/kernel/kexec.c
> +++ b/kernel/kexec.c
> @@ -444,7 +444,7 @@ arch_kexec_apply_relocations(const Elf_Ehdr *ehdr, Elf_Shdr *sechdrs,
>  }
>  
>  /*
> - * Free up memory used by kernel, initrd, and comand line. This is temporary
> + * Free up memory used by kernel, initrd, and command line. This is temporary
>   * memory allocation which is not needed any more after these buffers have
>   * been loaded into separate segments and have been copied elsewhere.
>   */


From vgoyal at redhat.com  Fri Jan  2 10:59:07 2015
From: vgoyal at redhat.com (Vivek Goyal)
Date: Fri, 2 Jan 2015 13:59:07 -0500
Subject: [PATCH] kexec: Fix a typo in comment
In-Reply-To: <87vbkpvx0s.fsf@x220.int.ebiederm.org>
References: <1420221379-4553-1-git-send-email-kuleshovmail@gmail.com>
 <87vbkpvx0s.fsf@x220.int.ebiederm.org>
Message-ID: <20150102185907.GB3733@redhat.com>

On Fri, Jan 02, 2015 at 12:48:51PM -0600, Eric W. Biederman wrote:
> Alexander Kuleshov <kuleshovmail at gmail.com> writes:
> 
> > Signed-off-by: Alexander Kuleshov <kuleshovmail at gmail.com>
> Acked-by: "Eric W. Biederman" <ebiederm at xmission.com>

[ CC akpm ]

Simple fix.

Acked-by: Vivek Goyal <vgoyal at redhat.com>

Thanks
Vivek

> 
> > ---
> >  kernel/kexec.c | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/kernel/kexec.c b/kernel/kexec.c
> > index 9a8a01a..75a8b7e 100644
> > --- a/kernel/kexec.c
> > +++ b/kernel/kexec.c
> > @@ -444,7 +444,7 @@ arch_kexec_apply_relocations(const Elf_Ehdr *ehdr, Elf_Shdr *sechdrs,
> >  }
> >  
> >  /*
> > - * Free up memory used by kernel, initrd, and comand line. This is temporary
> > + * Free up memory used by kernel, initrd, and command line. This is temporary
> >   * memory allocation which is not needed any more after these buffers have
> >   * been loaded into separate segments and have been copied elsewhere.
> >   */


From horms at verge.net.au  Sun Jan  4 17:49:47 2015
From: horms at verge.net.au (Simon Horman)
Date: Mon, 5 Jan 2015 10:49:47 +0900
Subject: [PATCH] kexec, remove panic_on_warn kernel parameter from kdump
 situations
In-Reply-To: <20150102131709.GB18785@redhat.com>
References: <1419951471-19765-1-git-send-email-prarit@redhat.com>
 <20150102125423.GA18785@redhat.com> <54A69808.4010401@redhat.com>
 <20150102131709.GB18785@redhat.com>
Message-ID: <20150105014947.GC10874@verge.net.au>

On Fri, Jan 02, 2015 at 08:17:09AM -0500, Vivek Goyal wrote:
> On Fri, Jan 02, 2015 at 08:07:20AM -0500, Prarit Bhargava wrote:
> > 
> > 
> > On 01/02/2015 07:54 AM, Vivek Goyal wrote:
> > > On Tue, Dec 30, 2014 at 09:57:51AM -0500, Prarit Bhargava wrote:
> > >> panic_on_warn kernel parameter will cause the kernel to panic when a
> > >> WARN() is hit in the kernel.  This is not a good situation for the kdump
> > >> kernel because then it would be possible for the kdump kernel to panic in
> > >> a non-fatal WARN().
> > >>
> > >> This patch removes panic_on_warn as a kernel parameter for the kdump
> > >> kernel.
> > >>
> > > 
> > > I think modifying kexec-tools is not best place for this. It probably is better to take care of this in distribution specific scripts.
> > > 
> > > In the past we have learnt that it is best that kexec-tools does least
> > > amount of manipulation with command line.
> > 
> > Well .. here's the question to think about: what does adding panic_on_warn to
> > the kdump kernel get you?  AFAICT, nothing.
> 
> Let us consider a hypothetical situation. What if we have some buggy code
> which will corrupt file system in certain situation and we detect that
> situation and throw a warning. 
> 
> In that case as a work around specifying panic_on_warn in kdump kernel
> will make sense as we don't want to make further progress if we hit
> the warning as it has potential to corrupt fs.
> 
> Again this is hypothetical but it can happen. So panic_on_warn might
> still be useful in kdump kernel for some corner debugging cases.
> 
> That's why I think we should do it in distribution specific scripts
> and that too only if user did not specify panic_on_warn for second
> kernel explicitly.
> 
> > 
> > If panic_on_warn is specified, the only thing that will happen is that kdump
> > will fail (which is always bad IMO).  There is no real difference in the stack
> > trace between the WARN() and panic situations so there is no information loss.
> > 
> > So I disagree -- we should never specify panic_on_warn on kdump kernel.
> 
> I am saying that do it in distribution specific scripts and not in 
> kexec-tools.

I have reverted this patch until some consensus is reached.


From ptesarik at suse.cz  Mon Jan  5 00:49:11 2015
From: ptesarik at suse.cz (Petr Tesarik)
Date: Mon, 5 Jan 2015 09:49:11 +0100
Subject: [PATCH v2] kdump, vmcoreinfo: report actual value of phys_base
In-Reply-To: <20150102134240.GD18785@redhat.com>
References: <20141113.113011.630216167285231824.d.hatayama@jp.fujitsu.com>
 <20141215151120.b44112413e15a8c390d39147@linux-foundation.org>
 <20150102134240.GD18785@redhat.com>
Message-ID: <20150105094911.40508832@hananiah.suse.cz>

On Fri, 2 Jan 2015 08:42:40 -0500
Vivek Goyal <vgoyal at redhat.com> wrote:

> On Mon, Dec 15, 2014 at 03:11:20PM -0800, Andrew Morton wrote:
> > 
> > (cc trimmed a bit)
> > 
> > On Thu, 13 Nov 2014 11:30:11 +0900 (JST) HATAYAMA Daisuke <d.hatayama at jp.fujitsu.com> wrote:
> > 
> > > Currently, VMCOREINFO note information reports the virtual address of
> > > phys_base that is assigned to symbol phys_base. But this doesn't make
> > > sense because to refer to phys_base, it's necessary to get the value
> > > of phys_base itself we are now about to refer to.
> > 
> > Folks, could we please get a bit of reviewing, acking or nacking for
> > this one?
> 
> To me this patch is more like a hack or a quick fix to get dump filtering
> working when dump is generated using virsh-dump like tools. I feel there
> should be discussion on what's the proper way to do this. I don't have
> very specific ideas though at this point of time.

For one thing, the virtual address of phys_base is useless. This is
nicely demonstrated by the fact that it is not used by any kernel dump
processing project I'm aware of (crash, makedumpfile, libkdumpfile).

On the other hand, either the _value_ of phys_base or its _physical_
location in RAM would be useful. If it was available, I would
immediately start using it in libkdumpfile to determine kernel physical
base. In fact, writing a similar patch was already on my TODO list when
Daisuke sent his.

Petr T


From dyoung at redhat.com  Mon Jan  5 17:44:05 2015
From: dyoung at redhat.com (Dave Young)
Date: Tue, 6 Jan 2015 09:44:05 +0800
Subject: [PATCH] kexec, remove panic_on_warn kernel parameter from kdump
 situations
In-Reply-To: <20150102131709.GB18785@redhat.com>
References: <1419951471-19765-1-git-send-email-prarit@redhat.com>
 <20150102125423.GA18785@redhat.com> <54A69808.4010401@redhat.com>
 <20150102131709.GB18785@redhat.com>
Message-ID: <20150106014405.GA2113@darkstar.redhat.com>

On 01/02/15 at 08:17am, Vivek Goyal wrote:
> On Fri, Jan 02, 2015 at 08:07:20AM -0500, Prarit Bhargava wrote:
> > 
> > 
> > On 01/02/2015 07:54 AM, Vivek Goyal wrote:
> > > On Tue, Dec 30, 2014 at 09:57:51AM -0500, Prarit Bhargava wrote:
> > >> panic_on_warn kernel parameter will cause the kernel to panic when a
> > >> WARN() is hit in the kernel.  This is not a good situation for the kdump
> > >> kernel because then it would be possible for the kdump kernel to panic in
> > >> a non-fatal WARN().
> > >>
> > >> This patch removes panic_on_warn as a kernel parameter for the kdump
> > >> kernel.
> > >>
> > > 
> > > I think modifying kexec-tools is not best place for this. It probably is better to take care of this in distribution specific scripts.
> > > 
> > > In the past we have learnt that it is best that kexec-tools does least
> > > amount of manipulation with command line.
> > 
> > Well .. here's the question to think about: what does adding panic_on_warn to
> > the kdump kernel get you?  AFAICT, nothing.
> 
> Let us consider a hypothetical situation. What if we have some buggy code
> which will corrupt file system in certain situation and we detect that
> situation and throw a warning. 
> 
> In that case as a work around specifying panic_on_warn in kdump kernel
> will make sense as we don't want to make further progress if we hit
> the warning as it has potential to corrupt fs.
> 
> Again this is hypothetical but it can happen. So panic_on_warn might
> still be useful in kdump kernel for some corner debugging cases.
> 
> That's why I think we should do it in distribution specific scripts
> and that too only if user did not specify panic_on_warn for second
> kernel explicitly.

Thinking of user who use upstream kexec-tools instead of distribution toolset,
In case kexec --reuse-cmdline, it will copy /proc/cmdline, but user will have
no way to remove part of them.

I do want to insist on removing 'panic_on_warn' in upstream kexec-tools, but
we should give user an option to remove it. Something like:

kexec --reuse-cmdline --remove-params="panic_on_warn" will be good.

Thanks
Dave


From vgoyal at redhat.com  Mon Jan  5 17:54:12 2015
From: vgoyal at redhat.com (Vivek Goyal)
Date: Mon, 5 Jan 2015 20:54:12 -0500
Subject: [PATCH] kexec, remove panic_on_warn kernel parameter from kdump
 situations
In-Reply-To: <20150106014405.GA2113@darkstar.redhat.com>
References: <1419951471-19765-1-git-send-email-prarit@redhat.com>
 <20150102125423.GA18785@redhat.com> <54A69808.4010401@redhat.com>
 <20150102131709.GB18785@redhat.com>
 <20150106014405.GA2113@darkstar.redhat.com>
Message-ID: <20150106015412.GB20058@redhat.com>

On Tue, Jan 06, 2015 at 09:44:05AM +0800, Dave Young wrote:
> On 01/02/15 at 08:17am, Vivek Goyal wrote:
> > On Fri, Jan 02, 2015 at 08:07:20AM -0500, Prarit Bhargava wrote:
> > > 
> > > 
> > > On 01/02/2015 07:54 AM, Vivek Goyal wrote:
> > > > On Tue, Dec 30, 2014 at 09:57:51AM -0500, Prarit Bhargava wrote:
> > > >> panic_on_warn kernel parameter will cause the kernel to panic when a
> > > >> WARN() is hit in the kernel.  This is not a good situation for the kdump
> > > >> kernel because then it would be possible for the kdump kernel to panic in
> > > >> a non-fatal WARN().
> > > >>
> > > >> This patch removes panic_on_warn as a kernel parameter for the kdump
> > > >> kernel.
> > > >>
> > > > 
> > > > I think modifying kexec-tools is not best place for this. It probably is better to take care of this in distribution specific scripts.
> > > > 
> > > > In the past we have learnt that it is best that kexec-tools does least
> > > > amount of manipulation with command line.
> > > 
> > > Well .. here's the question to think about: what does adding panic_on_warn to
> > > the kdump kernel get you?  AFAICT, nothing.
> > 
> > Let us consider a hypothetical situation. What if we have some buggy code
> > which will corrupt file system in certain situation and we detect that
> > situation and throw a warning. 
> > 
> > In that case as a work around specifying panic_on_warn in kdump kernel
> > will make sense as we don't want to make further progress if we hit
> > the warning as it has potential to corrupt fs.
> > 
> > Again this is hypothetical but it can happen. So panic_on_warn might
> > still be useful in kdump kernel for some corner debugging cases.
> > 
> > That's why I think we should do it in distribution specific scripts
> > and that too only if user did not specify panic_on_warn for second
> > kernel explicitly.
> 
> Thinking of user who use upstream kexec-tools instead of distribution toolset,
> In case kexec --reuse-cmdline, it will copy /proc/cmdline, but user will have
> no way to remove part of them.
> 
> I do want to insist on removing 'panic_on_warn' in upstream kexec-tools, but
> we should give user an option to remove it. Something like:
> 
> kexec --reuse-cmdline --remove-params="panic_on_warn" will be good.

If user is using --reuse-commandline at the same time does not want some
of the parameters from command line, then don't use --reuse-commandline.

This is overenginnering. First provide an option to reuse the commandline
and provide another option to selectively remove some parameters from that
commandline.

What's wrong with existing parameters of --command-line. This just allows
user to specify whatever command line is suitable.

So, no, we should not provide --remove-params. If existing command line
does not work for new kenrel, then user should not use
--reuse-commandline option.

Thanks
Vivek


From indou.takao at jp.fujitsu.com  Mon Jan  5 16:18:46 2015
From: indou.takao at jp.fujitsu.com (Takao Indoh)
Date: Tue, 6 Jan 2015 09:18:46 +0900
Subject: [PATCH 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel
In-Reply-To: <54A0C761.5060604@hp.com>
References: <1419239729-26529-1-git-send-email-zhen-hual@hp.com>
 <549CEE8F.2090209@jp.fujitsu.com> <549D0459.4080202@hp.com>
 <549D0DEE.8000301@jp.fujitsu.com> <54A0C761.5060604@hp.com>
Message-ID: <54AB29E6.6070403@jp.fujitsu.com>

On 2014/12/29 12:15, Li, ZhenHua wrote:
> Hi Takao Indoh,
> 
> Happy New Year, and thank you very much for you help.  The flush is quite

Happy new year!

>   a problem,  as there are several places the flush function should be called,
> I think the flush should be placed in functions like __iommu_update_old_*.
> Created a small patch for this, it is attached.
> 
> 
> 
> As I cannot reproduce your problems on my system, so could you please try
> these steps?
> 1. Apply the latest patchset, including 9/10 and 10/10, and then apply the
> attached patch_for_flush.patch.  And then test the kernel.

No inter-remap fault, but there is still DMAR fault message.

> 
> 2.  If 1 does not fix the DMAR fault  problems, then it might be caused by
> 7/10, so please *unpatch* it from the kernel (others and the  attached one
> should be patched), and then test the kernel.

DMAR fault still occurs. I'll dig iommu driver code to find out the
reason.

Thanks,
Takao Indoh

> 
> Regards
> Zhenhua
> 
> On 12/26/2014 03:27 PM, Takao Indoh wrote:
>> On 2014/12/26 15:46, Li, ZhenHua wrote:
>>> Hi Takao Indoh,
>>>
>>> Thank you very much for your testing. I will add your update in next
>>> version.
>>> Also I think a flush for __iommu_update_old_root_entry is also necessary.
>>>
>>> Currently I have no idea about your fault, does it happen before or
>>> during its loading? Could you send me your full kernel log as an
>>> attachment?
>> Sure, see attached file.
>>
>> I removed 9/10 and 10/10 patches from my kernel to avoid panic problem I
>> reported in previous mail, and then tested kdump. So please ignore
>> intr-remap fault message in log file. Also please ignore stack trace
>> starting with the following message, it's a problem of my box.
>>
>>    Flags mismatch irq 0. 00000080 (i801_smbus) vs. 00015a00 (timer)
>>
>> Thanks,
>> Takao Indoh
>>
>>> Regards and Merry Christmas.
>>> Zhenhua
>>>
>>> On 12/26/2014 01:13 PM, Takao Indoh wrote:
>>>> Hi Zhen-Hua,
>>>>
>>>> I tested your patch and found two problems.
>>>>
>>>> [1]
>>>> Kenel panic occurs during 2nd kernel boot.
>>>>
>>>> ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
>>>> Kernel panic - not syncing: timer doesn't work through Interrupt-remapped IO-APIC
>>>> CPU: 0 PID: 1 Comm: swapper/0 Not tainted 3.18.0 #25
>>>> Hardware name: FUJITSU-SV PRIMERGY BX920 S2/D3030, BIOS 080015 Rev.3D81.3030 02/10/2012
>>>>    0000000000000002 ffff880036167d08 ffffffff815b1c6a 0000000000000000
>>>>    ffffffff817f7670 ffff880036167d88 ffffffff815b19f1 0000000000000008
>>>>    ffff880036167d98 ffff880036167d38 ffffffff810a5d2f ffff880036167d98
>>>> Call Trace:
>>>>    [<ffffffff815b1c6a>] dump_stack+0x48/0x5e
>>>>    [<ffffffff815b19f1>] panic+0xbb/0x1fa
>>>>    [<ffffffff810a5d2f>] ? vprintk_default+0x1f/0x30
>>>>    [<ffffffff814c6a6c>] panic_if_irq_remap+0x1c/0x20
>>>>    [<ffffffff81b53985>] check_timer+0x1e7/0x5ed
>>>>    [<ffffffff8129bd9d>] ? radix_tree_lookup+0xd/0x10
>>>>    [<ffffffff81b5413b>] setup_IO_APIC+0x261/0x292
>>>>    [<ffffffff81b50302>] native_smp_prepare_cpus+0x214/0x25d
>>>>    [<ffffffff81b41c65>] kernel_init_freeable+0x1dc/0x28c
>>>>    [<ffffffff815aaf00>] ? rest_init+0x80/0x80
>>>>    [<ffffffff815aaf0e>] kernel_init+0xe/0xf0
>>>>    [<ffffffff815b5d2c>] ret_from_fork+0x7c/0xb0
>>>>    [<ffffffff815aaf00>] ? rest_init+0x80/0x80
>>>> ---[ end Kernel panic - not syncing: timer doesn't work through Interrupt-remapped IO-APIC
>>>>
>>>>
>>>> This panic seems to be related to unflushed cache. I confirmed this
>>>> problem was fixed by the following patch.
>>>>
>>>> --- a/drivers/iommu/intel_irq_remapping.c
>>>> +++ b/drivers/iommu/intel_irq_remapping.c
>>>> @@ -200,8 +200,13 @@ static int modify_irte(int irq, struct irte *irte_modified)
>>>>    	set_64bit(&irte->high, irte_modified->high);
>>>>    
>>>>    #ifdef CONFIG_CRASH_DUMP
>>>> -	if (is_kdump_kernel())
>>>> +	if (is_kdump_kernel()) {
>>>>    		__iommu_update_old_irte(iommu, index);
>>>> +		__iommu_flush_cache(iommu,
>>>> +			iommu->ir_table->base_old_virt +
>>>> +			index * sizeof(struct irte),
>>>> +			sizeof(struct irte));
>>>> +	}
>>>>    #endif
>>>>    	__iommu_flush_cache(iommu, irte, sizeof(*irte));
>>>>    
>>>>
>>>> [2]
>>>> Some DMAR error messages are still found in 2nd kernel boot.
>>>>
>>>> dmar: DRHD: handling fault status reg 2
>>>> dmar: DMAR:[DMA Write] Request device [01:00.0] fault addr ffded000
>>>> DMAR:[fault reason 01] Present bit in root entry is clear
>>>>
>>>> I confiremd your commit 1a2262 was already applied. Any idea?
>>>>
>>>> Thanks,
>>>> Takao Indoh
>>>>
>>>>
>>>> On 2014/12/22 18:15, Li, Zhen-Hua wrote:
>>>>> This patchset is an update of Bill Sumner's patchset, implements a fix for:
>>>>> If a kernel boots with intel_iommu=on on a system that supports intel vt-d,
>>>>> when a panic happens, the kdump kernel will boot with these faults:
>>>>>
>>>>>        dmar: DRHD: handling fault status reg 102
>>>>>        dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr fff80000
>>>>>        DMAR:[fault reason 01] Present bit in root entry is clear
>>>>>
>>>>>        dmar: DRHD: handling fault status reg 2
>>>>>        dmar: INTR-REMAP: Request device [[61:00.0] fault index 42
>>>>>        INTR-REMAP:[fault reason 34] Present field in the IRTE entry is clear
>>>>>
>>>>> On some system, the interrupt remapping fault will also happen even if the
>>>>> intel_iommu is not set to on, because the interrupt remapping will be enabled
>>>>> when x2apic is needed by the system.
>>>>>
>>>>> The cause of the DMA fault is described in Bill's original version, and the
>>>>> INTR-Remap fault is caused by a similar reason. In short, the initialization
>>>>> of vt-d drivers causes the in-flight DMA and interrupt requests get wrong
>>>>> response.
>>>>>
>>>>> To fix this problem, we modifies the behaviors of the intel vt-d in the
>>>>> crashdump kernel:
>>>>>
>>>>> For DMA Remapping:
>>>>> 1. To accept the vt-d hardware in an active state,
>>>>> 2. Do not disable and re-enable the translation, keep it enabled.
>>>>> 3. Use the old root entry table, do not rewrite the RTA register.
>>>>> 4. Malloc and use new context entry table and page table, copy data from the
>>>>>       old ones that used by the old kernel.
>>>>> 5. to use different portions of the iova address ranges for the device drivers
>>>>>       in the crashdump kernel than the iova ranges that were in-use at the time
>>>>>       of the panic.
>>>>> 6. After device driver is loaded, when it issues the first dma_map command,
>>>>>       free the dmar_domain structure for this device, and generate a new one, so
>>>>>       that the device can be assigned a new and empty page table.
>>>>> 7. When a new context entry table is generated, we also save its address to
>>>>>       the old root entry table.
>>>>>
>>>>> For Interrupt Remapping:
>>>>> 1. To accept the vt-d hardware in an active state,
>>>>> 2. Do not disable and re-enable the interrupt remapping, keep it enabled.
>>>>> 3. Use the old interrupt remapping table, do not rewrite the IRTA register.
>>>>> 4. When ioapic entry is setup, the interrupt remapping table is changed, and
>>>>>       the updated data will be stored to the old interrupt remapping table.
>>>>>
>>>>> Advantages of this approach:
>>>>> 1. All manipulation of the IO-device is done by the Linux device-driver
>>>>>       for that device.
>>>>> 2. This approach behaves in a manner very similar to operation without an
>>>>>       active iommu.
>>>>> 3. Any activity between the IO-device and its RMRR areas is handled by the
>>>>>       device-driver in the same manner as during a non-kdump boot.
>>>>> 4. If an IO-device has no driver in the kdump kernel, it is simply left alone.
>>>>>       This supports the practice of creating a special kdump kernel without
>>>>>       drivers for any devices that are not required for taking a crashdump.
>>>>> 5. Minimal code-changes among the existing mainline intel vt-d code.
>>>>>
>>>>> Summary of changes in this patch set:
>>>>> 1. Added some useful function for root entry table in code intel-iommu.c
>>>>> 2. Added new members to struct root_entry and struct irte;
>>>>> 3. Functions to load old root entry table to iommu->root_entry from the memory
>>>>>       of old kernel.
>>>>> 4. Functions to malloc new context entry table and page table and copy the data
>>>>>       from the old ones to the malloced new ones.
>>>>> 5. Functions to enable support for DMA remapping in kdump kernel.
>>>>> 6. Functions to load old irte data from the old kernel to the kdump kernel.
>>>>> 7. Some code changes that support other behaviours that have been listed.
>>>>> 8. In the new functions, use physical address as "unsigned long" type, not
>>>>>       pointers.
>>>>>
>>>>> Original version by Bill Sumner:
>>>>>        https://lkml.org/lkml/2014/1/10/518
>>>>>        https://lkml.org/lkml/2014/4/15/716
>>>>>        https://lkml.org/lkml/2014/4/24/836
>>>>>
>>>>> Zhenhua's last of Bill's patchset:
>>>>>        https://lkml.org/lkml/2014/10/21/134
>>>>>        https://lkml.org/lkml/2014/12/15/121
>>>>>
>>>>> Changed in this version:
>>>>> 1. Do not disable and re-enable traslation and interrupt remapping.
>>>>> 2. Use old root entry table.
>>>>> 3. Use old interrupt remapping table.
>>>>> 4. Use "unsigned long" as physical address.
>>>>> 5. Use intel_unmap to unmap the old dma;
>>>>>
>>>>> This patchset should be applied with this one together:
>>>>>        https://lkml.org/lkml/2014/11/5/43
>>>>>        x86/iommu: fix incorrect bit operations in setting values
>>>>>
>>>>> Bill Sumner (5):
>>>>>      iommu/vt-d: Update iommu_attach_domain() and its callers
>>>>>      iommu/vt-d: Items required for kdump
>>>>>      iommu/vt-d: data types and functions used for kdump
>>>>>      iommu/vt-d: Add domain-id functions
>>>>>      iommu/vt-d: enable kdump support in iommu module
>>>>>
>>>>> Li, Zhen-Hua (10):
>>>>>      iommu/vt-d: Update iommu_attach_domain() and its callers
>>>>>      iommu/vt-d: Items required for kdump
>>>>>      iommu/vt-d: Add domain-id functions
>>>>>      iommu/vt-d: functions to copy data from old mem
>>>>>      iommu/vt-d: Add functions to load and save old re
>>>>>      iommu/vt-d: datatypes and functions used for kdump
>>>>>      iommu/vt-d: enable kdump support in iommu module
>>>>>      iommu/vtd: assign new page table for dma_map
>>>>>      iommu/vt-d: Copy functions for irte
>>>>>      iommu/vt-d: Use old irte in kdump kernel
>>>>>
>>>>>     drivers/iommu/intel-iommu.c         | 1050 +++++++++++++++++++++++++++++++++--
>>>>>     drivers/iommu/intel_irq_remapping.c |   99 +++-
>>>>>     include/linux/intel-iommu.h         |   18 +
>>>>>     3 files changed, 1123 insertions(+), 44 deletions(-)
>>>>>
>>>
>>>
> 




From zhen-hual at hp.com  Mon Jan  5 18:04:33 2015
From: zhen-hual at hp.com (Li, ZhenHua)
Date: Tue, 06 Jan 2015 10:04:33 +0800
Subject: [PATCH 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel
In-Reply-To: <54AB29E6.6070403@jp.fujitsu.com>
References: <1419239729-26529-1-git-send-email-zhen-hual@hp.com>
 <549CEE8F.2090209@jp.fujitsu.com> <549D0459.4080202@hp.com>
 <549D0DEE.8000301@jp.fujitsu.com> <54A0C761.5060604@hp.com>
 <54AB29E6.6070403@jp.fujitsu.com>
Message-ID: <54AB42B1.2010001@hp.com>

Thank you very much for your help.

I have found there are several places need flush, and I will send a new
version
of this patchset with the flush functions.

Regards
Zhenhua

On 01/06/2015 08:18 AM, Takao Indoh wrote:
> On 2014/12/29 12:15, Li, ZhenHua wrote:
>> Hi Takao Indoh,
>>
>> Happy New Year, and thank you very much for you help.  The flush is quite
> Happy new year!
>
>>   a problem,  as there are several places the flush function should be called,
>> I think the flush should be placed in functions like __iommu_update_old_*.
>> Created a small patch for this, it is attached.
>>
>>
>>
>> As I cannot reproduce your problems on my system, so could you please try
>> these steps?
>> 1. Apply the latest patchset, including 9/10 and 10/10, and then apply the
>> attached patch_for_flush.patch.  And then test the kernel.
> No inter-remap fault, but there is still DMAR fault message.
>
>> 2.  If 1 does not fix the DMAR fault  problems, then it might be caused by
>> 7/10, so please *unpatch* it from the kernel (others and the  attached one
>> should be patched), and then test the kernel.
> DMAR fault still occurs. I'll dig iommu driver code to find out the
> reason.
>
> Thanks,
> Takao Indoh
>
>> Regards
>> Zhenhua
>>
>> On 12/26/2014 03:27 PM, Takao Indoh wrote:
>>> On 2014/12/26 15:46, Li, ZhenHua wrote:
>>>> Hi Takao Indoh,
>>>>
>>>> Thank you very much for your testing. I will add your update in next
>>>> version.
>>>> Also I think a flush for __iommu_update_old_root_entry is also necessary.
>>>>
>>>> Currently I have no idea about your fault, does it happen before or
>>>> during its loading? Could you send me your full kernel log as an
>>>> attachment?
>>> Sure, see attached file.
>>>
>>> I removed 9/10 and 10/10 patches from my kernel to avoid panic problem I
>>> reported in previous mail, and then tested kdump. So please ignore
>>> intr-remap fault message in log file. Also please ignore stack trace
>>> starting with the following message, it's a problem of my box.
>>>
>>>    Flags mismatch irq 0. 00000080 (i801_smbus) vs. 00015a00 (timer)
>>>
>>> Thanks,
>>> Takao Indoh
>>>
>>>> Regards and Merry Christmas.
>>>> Zhenhua
>>>>
>>>> On 12/26/2014 01:13 PM, Takao Indoh wrote:
>>>>> Hi Zhen-Hua,
>>>>>
>>>>> I tested your patch and found two problems.
>>>>>
>>>>> [1]
>>>>> Kenel panic occurs during 2nd kernel boot.
>>>>>
>>>>> ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
>>>>> Kernel panic - not syncing: timer doesn't work through Interrupt-remapped IO-APIC
>>>>> CPU: 0 PID: 1 Comm: swapper/0 Not tainted 3.18.0 #25
>>>>> Hardware name: FUJITSU-SV PRIMERGY BX920 S2/D3030, BIOS 080015 Rev.3D81.3030 02/10/2012
>>>>>    0000000000000002 ffff880036167d08 ffffffff815b1c6a 0000000000000000
>>>>>    ffffffff817f7670 ffff880036167d88 ffffffff815b19f1 0000000000000008
>>>>>    ffff880036167d98 ffff880036167d38 ffffffff810a5d2f ffff880036167d98
>>>>> Call Trace:
>>>>>    [<ffffffff815b1c6a>] dump_stack+0x48/0x5e
>>>>>    [<ffffffff815b19f1>] panic+0xbb/0x1fa
>>>>>    [<ffffffff810a5d2f>] ? vprintk_default+0x1f/0x30
>>>>>    [<ffffffff814c6a6c>] panic_if_irq_remap+0x1c/0x20
>>>>>    [<ffffffff81b53985>] check_timer+0x1e7/0x5ed
>>>>>    [<ffffffff8129bd9d>] ? radix_tree_lookup+0xd/0x10
>>>>>    [<ffffffff81b5413b>] setup_IO_APIC+0x261/0x292
>>>>>    [<ffffffff81b50302>] native_smp_prepare_cpus+0x214/0x25d
>>>>>    [<ffffffff81b41c65>] kernel_init_freeable+0x1dc/0x28c
>>>>>    [<ffffffff815aaf00>] ? rest_init+0x80/0x80
>>>>>    [<ffffffff815aaf0e>] kernel_init+0xe/0xf0
>>>>>    [<ffffffff815b5d2c>] ret_from_fork+0x7c/0xb0
>>>>>    [<ffffffff815aaf00>] ? rest_init+0x80/0x80
>>>>> ---[ end Kernel panic - not syncing: timer doesn't work through Interrupt-remapped IO-APIC
>>>>>
>>>>>
>>>>> This panic seems to be related to unflushed cache. I confirmed this
>>>>> problem was fixed by the following patch.
>>>>>
>>>>> --- a/drivers/iommu/intel_irq_remapping.c
>>>>> +++ b/drivers/iommu/intel_irq_remapping.c
>>>>> @@ -200,8 +200,13 @@ static int modify_irte(int irq, struct irte *irte_modified)
>>>>>    	set_64bit(&irte->high, irte_modified->high);
>>>>>    
>>>>>    #ifdef CONFIG_CRASH_DUMP
>>>>> -	if (is_kdump_kernel())
>>>>> +	if (is_kdump_kernel()) {
>>>>>    		__iommu_update_old_irte(iommu, index);
>>>>> +		__iommu_flush_cache(iommu,
>>>>> +			iommu->ir_table->base_old_virt +
>>>>> +			index * sizeof(struct irte),
>>>>> +			sizeof(struct irte));
>>>>> +	}
>>>>>    #endif
>>>>>    	__iommu_flush_cache(iommu, irte, sizeof(*irte));
>>>>>    
>>>>>
>>>>> [2]
>>>>> Some DMAR error messages are still found in 2nd kernel boot.
>>>>>
>>>>> dmar: DRHD: handling fault status reg 2
>>>>> dmar: DMAR:[DMA Write] Request device [01:00.0] fault addr ffded000
>>>>> DMAR:[fault reason 01] Present bit in root entry is clear
>>>>>
>>>>> I confiremd your commit 1a2262 was already applied. Any idea?
>>>>>
>>>>> Thanks,
>>>>> Takao Indoh
>>>>>
>>>>>
>>>>> On 2014/12/22 18:15, Li, Zhen-Hua wrote:
>>>>>> This patchset is an update of Bill Sumner's patchset, implements a fix for:
>>>>>> If a kernel boots with intel_iommu=on on a system that supports intel vt-d,
>>>>>> when a panic happens, the kdump kernel will boot with these faults:
>>>>>>
>>>>>>        dmar: DRHD: handling fault status reg 102
>>>>>>        dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr fff80000
>>>>>>        DMAR:[fault reason 01] Present bit in root entry is clear
>>>>>>
>>>>>>        dmar: DRHD: handling fault status reg 2
>>>>>>        dmar: INTR-REMAP: Request device [[61:00.0] fault index 42
>>>>>>        INTR-REMAP:[fault reason 34] Present field in the IRTE entry is clear
>>>>>>
>>>>>> On some system, the interrupt remapping fault will also happen even if the
>>>>>> intel_iommu is not set to on, because the interrupt remapping will be enabled
>>>>>> when x2apic is needed by the system.
>>>>>>
>>>>>> The cause of the DMA fault is described in Bill's original version, and the
>>>>>> INTR-Remap fault is caused by a similar reason. In short, the initialization
>>>>>> of vt-d drivers causes the in-flight DMA and interrupt requests get wrong
>>>>>> response.
>>>>>>
>>>>>> To fix this problem, we modifies the behaviors of the intel vt-d in the
>>>>>> crashdump kernel:
>>>>>>
>>>>>> For DMA Remapping:
>>>>>> 1. To accept the vt-d hardware in an active state,
>>>>>> 2. Do not disable and re-enable the translation, keep it enabled.
>>>>>> 3. Use the old root entry table, do not rewrite the RTA register.
>>>>>> 4. Malloc and use new context entry table and page table, copy data from the
>>>>>>       old ones that used by the old kernel.
>>>>>> 5. to use different portions of the iova address ranges for the device drivers
>>>>>>       in the crashdump kernel than the iova ranges that were in-use at the time
>>>>>>       of the panic.
>>>>>> 6. After device driver is loaded, when it issues the first dma_map command,
>>>>>>       free the dmar_domain structure for this device, and generate a new one, so
>>>>>>       that the device can be assigned a new and empty page table.
>>>>>> 7. When a new context entry table is generated, we also save its address to
>>>>>>       the old root entry table.
>>>>>>
>>>>>> For Interrupt Remapping:
>>>>>> 1. To accept the vt-d hardware in an active state,
>>>>>> 2. Do not disable and re-enable the interrupt remapping, keep it enabled.
>>>>>> 3. Use the old interrupt remapping table, do not rewrite the IRTA register.
>>>>>> 4. When ioapic entry is setup, the interrupt remapping table is changed, and
>>>>>>       the updated data will be stored to the old interrupt remapping table.
>>>>>>
>>>>>> Advantages of this approach:
>>>>>> 1. All manipulation of the IO-device is done by the Linux device-driver
>>>>>>       for that device.
>>>>>> 2. This approach behaves in a manner very similar to operation without an
>>>>>>       active iommu.
>>>>>> 3. Any activity between the IO-device and its RMRR areas is handled by the
>>>>>>       device-driver in the same manner as during a non-kdump boot.
>>>>>> 4. If an IO-device has no driver in the kdump kernel, it is simply left alone.
>>>>>>       This supports the practice of creating a special kdump kernel without
>>>>>>       drivers for any devices that are not required for taking a crashdump.
>>>>>> 5. Minimal code-changes among the existing mainline intel vt-d code.
>>>>>>
>>>>>> Summary of changes in this patch set:
>>>>>> 1. Added some useful function for root entry table in code intel-iommu.c
>>>>>> 2. Added new members to struct root_entry and struct irte;
>>>>>> 3. Functions to load old root entry table to iommu->root_entry from the memory
>>>>>>       of old kernel.
>>>>>> 4. Functions to malloc new context entry table and page table and copy the data
>>>>>>       from the old ones to the malloced new ones.
>>>>>> 5. Functions to enable support for DMA remapping in kdump kernel.
>>>>>> 6. Functions to load old irte data from the old kernel to the kdump kernel.
>>>>>> 7. Some code changes that support other behaviours that have been listed.
>>>>>> 8. In the new functions, use physical address as "unsigned long" type, not
>>>>>>       pointers.
>>>>>>
>>>>>> Original version by Bill Sumner:
>>>>>>        https://lkml.org/lkml/2014/1/10/518
>>>>>>        https://lkml.org/lkml/2014/4/15/716
>>>>>>        https://lkml.org/lkml/2014/4/24/836
>>>>>>
>>>>>> Zhenhua's last of Bill's patchset:
>>>>>>        https://lkml.org/lkml/2014/10/21/134
>>>>>>        https://lkml.org/lkml/2014/12/15/121
>>>>>>
>>>>>> Changed in this version:
>>>>>> 1. Do not disable and re-enable traslation and interrupt remapping.
>>>>>> 2. Use old root entry table.
>>>>>> 3. Use old interrupt remapping table.
>>>>>> 4. Use "unsigned long" as physical address.
>>>>>> 5. Use intel_unmap to unmap the old dma;
>>>>>>
>>>>>> This patchset should be applied with this one together:
>>>>>>        https://lkml.org/lkml/2014/11/5/43
>>>>>>        x86/iommu: fix incorrect bit operations in setting values
>>>>>>
>>>>>> Bill Sumner (5):
>>>>>>      iommu/vt-d: Update iommu_attach_domain() and its callers
>>>>>>      iommu/vt-d: Items required for kdump
>>>>>>      iommu/vt-d: data types and functions used for kdump
>>>>>>      iommu/vt-d: Add domain-id functions
>>>>>>      iommu/vt-d: enable kdump support in iommu module
>>>>>>
>>>>>> Li, Zhen-Hua (10):
>>>>>>      iommu/vt-d: Update iommu_attach_domain() and its callers
>>>>>>      iommu/vt-d: Items required for kdump
>>>>>>      iommu/vt-d: Add domain-id functions
>>>>>>      iommu/vt-d: functions to copy data from old mem
>>>>>>      iommu/vt-d: Add functions to load and save old re
>>>>>>      iommu/vt-d: datatypes and functions used for kdump
>>>>>>      iommu/vt-d: enable kdump support in iommu module
>>>>>>      iommu/vtd: assign new page table for dma_map
>>>>>>      iommu/vt-d: Copy functions for irte
>>>>>>      iommu/vt-d: Use old irte in kdump kernel
>>>>>>
>>>>>>     drivers/iommu/intel-iommu.c         | 1050 +++++++++++++++++++++++++++++++++--
>>>>>>     drivers/iommu/intel_irq_remapping.c |   99 +++-
>>>>>>     include/linux/intel-iommu.h         |   18 +
>>>>>>     3 files changed, 1123 insertions(+), 44 deletions(-)
>>>>>>
>>>>
>



From liu.song11 at zte.com.cn  Mon Jan  5 22:44:14 2015
From: liu.song11 at zte.com.cn (liu.song11 at zte.com.cn)
Date: Tue, 6 Jan 2015 14:44:14 +0800
Subject: some questions about elf_ppc64_load
Message-ID: <OF9C6CA8A8.9D7923CD-ON48257DC5.0023F61F-48257DC5.0024E800@zte.com.cn>


hi all?

I use kexec-tools-2.0.8 for ppc64, in fuction elf_ppc64_load prompt "Can't use ramdisk with device tree blob input".

I don't know why here ramdisk can't together with dtb,and need some help.

I also have some questions, when I just ./kexe -l vmlinux then ./kexec -e, will jump to second kernel sucessfully,

but without useful dtb, then device can't init correct. But, when I input ./kexec -l vmlinux --dtb=my.dtb , then

./kexec -e, the kernel couldn't work and crash in head_64.S, this problem also make me confused.

Thanks
Regards
--------------------------------------------------------
ZTE Information Security Notice: The information contained in this mail (and any attachment transmitted herewith) is privileged and confidential and is intended for the exclusive use of the addressee(s).  If you are not an intended recipient, any disclosure, reproduction, distribution or other dissemination or use of the information contained is strictly prohibited.  If you have received this mail in error, please delete it and notify us immediately.

From zhen-hual at hp.com  Mon Jan  5 23:05:46 2015
From: zhen-hual at hp.com (Li, ZhenHua)
Date: Tue, 06 Jan 2015 15:05:46 +0800
Subject: [PATCH 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel
In-Reply-To: <20150106063718.GA2454@dhcp-16-105.nay.redhat.com>
References: <1419239729-26529-1-git-send-email-zhen-hual@hp.com>
 <20150106063718.GA2454@dhcp-16-105.nay.redhat.com>
Message-ID: <54AB894A.3070601@hp.com>

According to Takao Indoh's testing, seems adding flush after loading old 
irte and updating old irte does not fix the dmar faults.

According to Takao Indoh's log and your log, the faults happens while 
and after driver is loaded.  Maybe I am using incorrect code in 08/10.


On 01/06/2015 02:37 PM, Baoquan He wrote:
> Hi Zhenhua,
>
> I just tested your patchset based on 3.19.0-rc2+, and found several dmar
> fault and intr-remap fault, it seems not the same as Takao's, please
> check the attachment.
>
> Thanks
> Baoquan
>



From dyoung at redhat.com  Tue Jan  6 00:05:00 2015
From: dyoung at redhat.com (Dave Young)
Date: Tue, 6 Jan 2015 16:05:00 +0800
Subject: [PATCH] kexec, remove panic_on_warn kernel parameter from kdump
 situations
In-Reply-To: <20150106015412.GB20058@redhat.com>
References: <1419951471-19765-1-git-send-email-prarit@redhat.com>
 <20150102125423.GA18785@redhat.com> <54A69808.4010401@redhat.com>
 <20150102131709.GB18785@redhat.com>
 <20150106014405.GA2113@darkstar.redhat.com>
 <20150106015412.GB20058@redhat.com>
Message-ID: <20150106080500.GD2113@darkstar.redhat.com>

On 01/05/15 at 08:54pm, Vivek Goyal wrote:
> On Tue, Jan 06, 2015 at 09:44:05AM +0800, Dave Young wrote:
> > On 01/02/15 at 08:17am, Vivek Goyal wrote:
> > > On Fri, Jan 02, 2015 at 08:07:20AM -0500, Prarit Bhargava wrote:
> > > > 
> > > > 
> > > > On 01/02/2015 07:54 AM, Vivek Goyal wrote:
> > > > > On Tue, Dec 30, 2014 at 09:57:51AM -0500, Prarit Bhargava wrote:
> > > > >> panic_on_warn kernel parameter will cause the kernel to panic when a
> > > > >> WARN() is hit in the kernel.  This is not a good situation for the kdump
> > > > >> kernel because then it would be possible for the kdump kernel to panic in
> > > > >> a non-fatal WARN().
> > > > >>
> > > > >> This patch removes panic_on_warn as a kernel parameter for the kdump
> > > > >> kernel.
> > > > >>
> > > > > 
> > > > > I think modifying kexec-tools is not best place for this. It probably is better to take care of this in distribution specific scripts.
> > > > > 
> > > > > In the past we have learnt that it is best that kexec-tools does least
> > > > > amount of manipulation with command line.
> > > > 
> > > > Well .. here's the question to think about: what does adding panic_on_warn to
> > > > the kdump kernel get you?  AFAICT, nothing.
> > > 
> > > Let us consider a hypothetical situation. What if we have some buggy code
> > > which will corrupt file system in certain situation and we detect that
> > > situation and throw a warning. 
> > > 
> > > In that case as a work around specifying panic_on_warn in kdump kernel
> > > will make sense as we don't want to make further progress if we hit
> > > the warning as it has potential to corrupt fs.
> > > 
> > > Again this is hypothetical but it can happen. So panic_on_warn might
> > > still be useful in kdump kernel for some corner debugging cases.
> > > 
> > > That's why I think we should do it in distribution specific scripts
> > > and that too only if user did not specify panic_on_warn for second
> > > kernel explicitly.
> > 
> > Thinking of user who use upstream kexec-tools instead of distribution toolset,
> > In case kexec --reuse-cmdline, it will copy /proc/cmdline, but user will have
> > no way to remove part of them.
> > 
> > I do want to insist on removing 'panic_on_warn' in upstream kexec-tools, but
> > we should give user an option to remove it. Something like:
> > 
> > kexec --reuse-cmdline --remove-params="panic_on_warn" will be good.
> 
> If user is using --reuse-commandline at the same time does not want some
> of the parameters from command line, then don't use --reuse-commandline.
> 
> This is overenginnering. First provide an option to reuse the commandline
> and provide another option to selectively remove some parameters from that
> commandline.
> 
> What's wrong with existing parameters of --command-line. This just allows
> user to specify whatever command line is suitable.
> 
> So, no, we should not provide --remove-params. If existing command line
> does not work for new kenrel, then user should not use
> --reuse-commandline option.

Hmm, ok. So hope one who is use panic_on_warn in 1st kernel know what he is doing
and do not simply copy the 1st kernel cmdline for 2nd kernel.


From office7mail at gmail.com  Mon Jan  5 23:50:08 2015
From: office7mail at gmail.com (Melvin)
Date: Tue, 06 Jan 2015 02:50:08 -0500
Subject: Can we talk on skype brief?
Message-ID: <201501060757.t067rfan015500@fairis.com.ec>

Hello,

I have an important business i want to discuss with you. Can we talk on skype brief? My skype id is: mil1960111mail

Regards,
Mikey Melvin Kurchenko
Tel: +706 284 0687

-- 
Este mensaje ha sido analizado por MailScanner
en busca de virus y otros contenidos peligrosos,
y se considera que est? limpio.



From ldufour at linux.vnet.ibm.com  Tue Jan  6 00:37:03 2015
From: ldufour at linux.vnet.ibm.com (Laurent Dufour)
Date: Tue, 06 Jan 2015 09:37:03 +0100
Subject: some questions about elf_ppc64_load
In-Reply-To: <OF9C6CA8A8.9D7923CD-ON48257DC5.0023F61F-48257DC5.0024E800@zte.com.cn>
References: <OF9C6CA8A8.9D7923CD-ON48257DC5.0023F61F-48257DC5.0024E800@zte.com.cn>
Message-ID: <54AB9EAF.2020004@linux.vnet.ibm.com>

On 06/01/2015 07:44, liu.song11 at zte.com.cn wrote:
> 
> hi all?
> 
> I use kexec-tools-2.0.8 for ppc64, in fuction elf_ppc64_load prompt "Can't use ramdisk with device tree blob input".
> 
> I don't know why here ramdisk can't together with dtb,and need some help.

The device tree is also specifying initrd (see kexec/fs2dt.c putnode),
so there is 2 initrd specify in that case. The code is a bit rough here,
but I guess this is because there is not yet a valid use case.

> I also have some questions, when I just ./kexe -l vmlinux then ./kexec -e, will jump to second kernel sucessfully,
> 
> but without useful dtb, then device can't init correct. But, when I input ./kexec -l vmlinux --dtb=my.dtb , then
> 
> ./kexec -e, the kernel couldn't work and crash in head_64.S, this problem also make me confused.

It's hard to state with no more details. I guess the device tree blob
you're using is not valid, or contain bad data.

Cheers,
Laurent.

> Thanks
> Regards
> --------------------------------------------------------
> ZTE Information Security Notice: The information contained in this mail (and any attachment transmitted herewith) is privileged and confidential and is intended for the exclusive use of the addressee(s).  If you are not an intended recipient, any disclosure, reproduction, distribution or other dissemination or use of the information contained is strictly prohibited.  If you have received this mail in error, please delete it and notify us immediately.
> _______________________________________________
> kexec mailing list
> kexec at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/kexec
> 



From kumagai-atsushi at mxc.nes.nec.co.jp  Tue Jan  6 00:36:33 2015
From: kumagai-atsushi at mxc.nes.nec.co.jp (Atsushi Kumagai)
Date: Tue, 6 Jan 2015 08:36:33 +0000
Subject: [PATCH 1/2] use raw i/o and root device to use less memory
In-Reply-To: <20141231193447.GA30752@sgi.com>
References: <20141209220711.GB18652@sgi.com>
 <0910DD04CBD6DE4193FCF86B9C00BE9701D77F76@BPXM01GP.gisp.nec.co.jp>
 <20141211154447.GA25032@sgi.com>
 <0910DD04CBD6DE4193FCF86B9C00BE9701D78D4A@BPXM01GP.gisp.nec.co.jp>
 <20141231193447.GA30752@sgi.com>
Message-ID: <0910DD04CBD6DE4193FCF86B9C00BE9701D7D5A1@BPXM01GP.gisp.nec.co.jp>

Hello Cliff,

>Hi Mr. Kumagai,
>
>On Mon, Dec 15, 2014 at 02:33:58AM +0000, Atsushi Kumagai wrote:
>> >On Thu, Dec 11, 2014 at 06:34:32AM +0000, Atsushi Kumagai wrote:
>> >> Hello Cliff,
>> >>
>> >> >From: Cliff Wickman <cpw at sgi.com>
>> >> >
>> >> >This patch adds a -j to makedumpfile. With this option it uses direct i/o on the dump
>> >> >file and the bitmap file, thus enabling makedumpfile to run mode in a fairly small
>> >> >crashkernel area without using cyclic mode. It can dump system with many terabytes of
>> >> >memory using crashkernel=450M.
>> >>
>> >> First, let's separate the problems that you have.
>> >> (Actually you did it in previous patches.)
>> >>
>> >>   1. The cyclic mode is slow.
>> >>     -> You try to avoid this by using a disk for the bitmap.
>> >>
>> >>   2. Page cache uses up the memory for crash kernel.
>> >>     -> You try to avoid this by using direct i/o.
>> >>
>> >> >Without direct i/o the crash kernel will use kernel page cache for the writes.  This
>> >> >will use up a great deal of the crash kernel's alloted memory.
>> >>
>> >> This is the second problem.
>> >> Actually we faced a OOM caused by page cache (probably):
>> >>
>> >>   http://lists.infradead.org/pipermail/kexec/2014-April/011639.html
>> >>
>> >> so direct i/o may be helpful for such small crashkernel environments.
>> >>
>> >> >The -j option will also implicitly avoid cyclic mode.  Cyclic mode is slower, and
>> >> >is not needed if we use direct i/o.
>> >>
>> >> This is the first problem.
>> >> Direct i/o doesn't enable the non-cyclic mode, using a disk does it.
>> >> Anyway, I still think it's enough to change TMPDIR to a disk if you want
>> >> to choose --non-cyclic. I haven't gotten the reason why you change the
>> >> code yet.
>> >
>> >As you noticed in the patch, -j not only causes direct i/o. It
>> >also causes the bitmaps to use disk instead of tmpfs.
>> >
>> >I made the -j option turn off cyclic mode because I'm trading slower
>> >disk i/o for a smaller crashkernel.  And if the crashkernel stays
>> >small (even as it builds huge bitmaps) we don't need to use cyclic mode,
>> >So we save the extra pass through the page structures.
>>
>> The -j option looks a set of configuration designed to fit YOUR case.
>> Direct i/o and non-cyclic mode and using a disk are individual setting,
>> they should be implemented as separated options, then we can discuss
>> the usefulness of each feature.
>>
>> Moreover, the cyclic mode can be turned off with the --non-cyclic option,
>> please don't add an additional option.
>>
>> >Cyclic mode works nicely. And the extra pass is hardly noticeable -- until
>> >you get into terabytes of memory.
>>
>> Further, I want to get rid of the non-cyclic mode as I said before:
>>
>>   http://lists.infradead.org/pipermail/kexec/2013-September/009489.html
>>
>> Now the disadvantage of the cyclic mode is that it has to take a
>> two-pass approach for filtering, and you want to avoid it, right ?
>> The penalty time of two-pass filtering was about 20sec/1TB in the
>> case below:
>>
>>   http://lists.infradead.org/pipermail/kexec/2013-March/008300.html
>>
>> Sure, it would be better if we can remove this extra filtering time.
>>
>> If the number of cycle is one, there is no need to take the two-pass
>> filtering because the whole of bitmap can be stored on the memory.
>> The current code always does the two-pass filtering regardless of the
>> number of cycle, but it can be fixed.
>>
>> So the remain issue is the space to store the bitmap. I think we have
>> to use a disk as you did, but it shouldn't depend on the non-cyclic
>> mode, it should be designed for the cyclic mode.
>> So I propose a new option to use a backing-file for the bitmap.
>> If we specify a file path, the file is used as bitmap file by mmap().
>> This idea can be implemented easily, but it's just a rough idea.
>>
>> Anyway, I think "using a disk for one-pass filtering" may be useful but
>> isn't a general feature because there will be no disk if network dump
>> (-F option) is used, so we should make a specific option to declare to
>> use a disk.
>>
>> To sum it up, my idea is:
>>
>>    1. Get rid of the non-cyclic mode.
>>    2. Fix the redundant page scanning for 1 cycle case.
>>    3. Introduce the new option to specify the backing-file for bitmap.
>>
>> I think this also can solve your problem, can't it ?
>
>I have been experimenting with a patch to implement your suggestions, as
>they would indeed solve my problem.  (That problem being the need for
>a very large crashkernel space on a big machine.)  It's not a showstopper,
>but a nuisance.
>
>The redundant page scanning can be fixed rather easily by determining up
>front that there will only be one cycle.  Then at the end of the first
>page scan the bitmap can be preserved for use by the dump writing process.
>
>But using a backing file for the bitmap is not so easy.  The difficulty lies
>in writing the bitmap and dump pages in piecemeal fashion (in cyclic mode)
>but using well-formed i/o as required when the dump file is opened for
>direct i/o.
>
>In my patch you will see the machinations needed in write_kdump_pages() to
>make this work in noncyclic mode.  It is all the more complicated in
>cyclic mode.
>(The dump file design includes a non-wellformed boundary between the
> page descriptors and the page data.)
>That's why I chose the easy way out and simply forced noncyclic mode when
>direct i/o was desired.
>
>I know, I know, the easy way out is not necessarily justification for not
>doing it a better way.  But I think you will find that cyclic writing of
>the dump is not trivial when it has to be done on block boundaries.
>
>But as the current patch stands, this small change makes a big difference.
>The use of direct i/o and the avoidance of tmpfs not only save a lot of
>crashkernel memory space. It also makes prediction of needed crashkernel
>memory size much easier.

To be honest, I'm not sure that disabling the page cache is meaningful.
The page cache is basically reclaimable, I guess it shouldn't be a problem.
Actually I said I found a OOM issue which seems be related to page cache,
but the free memory space of the machine was little in the first place,
I guess page cache isn't an essential cause.

Anyway, I'm working to implement the non-cyclic mode as 1-cycle case now. 
I'll post some patches later, I hope you will rebase your patches on them.
(At least the one adding -e option is great, I'd like to merge it.)

The code of the current non-cyclic mode is almost the same as the cyclic
mode's, I still think they should be removed.


Thanks
Atsushi Kumagai

>> By the way, the non-cyclic is faster than the cyclic mode in your
>> environments ? Certainly the non-cyclic mode can avoid two-pass
>> filtering but it needs disk i/o to read and write the bitmap file
>> instead. I'm concerned that the disadvantage of the non-cyclic mode
>> defeats the advantage of it.
>> I think it's better to confirm it before we start actual work,
>> otherwise it will be wasted effort.
>>
>> >> >Direct i/o is of course a bit slower, but not significantly slower when used in this
>> >> >almost-entirely sequential fashion.
>> >>
>> >> If you have a performance comparison between direct i/o and normal
>> >> file i/o, I'm curious to see it.
>> >
>> >Dumping a 2TB system
>> >- using the -j patch so that bitmaps and dump are using disk
>> >- everything being equal except the opening of files with or without O_DIRECT
>> >
>> >* using the -e patch so that we're not dumping page structures for
>> >  non-dumped pages:          dump size 570M (compressed)
>> >Page cached I/O
>> >  200 seconds   (writing dump file: 100 sec)
>> >Direct I/O
>> >  223 seconds   (writing dump file: 103 sec)
>> >
>> >* not using the -e patch:    dump size 3625M (compressed)
>> >Page cached I/O
>> >  620 seconds   (writing dump file: 525 sec)
>> >Direct I/O
>> >  700 seconds   (writing dump file: 590 sec)
>>
>> Thanks, the benefit of -e option is pretty obvious, and now
>> this feature is optional and detectable by the flag in the header,
>> the basic idea sounds good to me. However, I hope also this feature
>> will be designed for cyclic mode on the same reason as I said below.
>>
>>
>> Thanks
>> Atsushi Kumagai
>>
>> >-Cliff
>> >
>> >> >---
>> >> > makedumpfile.c |  417 ++++++++++++++++++++++++++++++++++++++++++++++-----------
>> >> > makedumpfile.h |    6
>> >> > print_info.c   |    5
>> >> > 3 files changed, 347 insertions(+), 81 deletions(-)
>> >> >
>> >> >Index: makedumpfile-1.5.7/makedumpfile.h
>> >> >===================================================================
>> >> >--- makedumpfile-1.5.7.orig/makedumpfile.h
>> >> >+++ makedumpfile-1.5.7/makedumpfile.h
>> >> >@@ -18,6 +18,7 @@
>> >> >
>> >> > #include <stdio.h>
>> >> > #include <stdlib.h>
>> >> >+#define __USE_GNU
>> >> > #include <fcntl.h>
>> >> > #include <gelf.h>
>> >> > #include <sys/stat.h>
>> >> >@@ -222,6 +223,7 @@ isAnon(unsigned long mapping)
>> >> > #define FILENAME_BITMAP		"kdump_bitmapXXXXXX"
>> >> > #define FILENAME_STDOUT		"STDOUT"
>> >> > #define MAP_REGION		(4096*1024)
>> >> >+#define DIRECT_ALIGN		(512)
>> >> >
>> >> > /*
>> >> >  * Minimam vmcore has 2 ProgramHeaderTables(PT_NOTE and PT_LOAD).
>> >> >@@ -892,7 +894,8 @@ struct dump_bitmap {
>> >> > 	int		fd;
>> >> > 	int		no_block;
>> >> > 	char		*file_name;
>> >> >-	char		buf[BUFSIZE_BITMAP];
>> >> >+	char		*buf;
>> >> >+	char		*buf_malloced;
>> >> > 	off_t		offset;
>> >> > };
>> >> >
>> >> >@@ -900,6 +903,7 @@ struct cache_data {
>> >> > 	int	fd;
>> >> > 	char	*file_name;
>> >> > 	char	*buf;
>> >> >+	char    *buf_malloced;
>> >> > 	size_t	buf_size;
>> >> > 	size_t	cache_size;
>> >> > 	off_t	offset;
>> >> >Index: makedumpfile-1.5.7/print_info.c
>> >> >===================================================================
>> >> >--- makedumpfile-1.5.7.orig/print_info.c
>> >> >+++ makedumpfile-1.5.7/print_info.c
>> >> >@@ -58,7 +58,7 @@ print_usage(void)
>> >> > 	MSG("\n");
>> >> > 	MSG("Usage:\n");
>> >> > 	MSG("  Creating DUMPFILE:\n");
>> >> >-	MSG("  # makedumpfile    [-c|-l|-p|-E] [-d DL] [-x VMLINUX|-i VMCOREINFO] VMCORE\n");
>> >> >+	MSG("  # makedumpfile    [-c|-l|-p|-E] [-d DL] [-j] [-x VMLINUX|-i VMCOREINFO] VMCORE\n");
>> >> > 	MSG("    DUMPFILE\n");
>> >> > 	MSG("\n");
>> >> > 	MSG("  Creating DUMPFILE with filtered kernel data specified through filter config\n");
>> >> >@@ -108,6 +108,9 @@ print_usage(void)
>> >> > 	MSG("      -E option, because the ELF format does not support compressed data.\n");
>> >> > 	MSG("      THIS IS ONLY FOR THE CRASH UTILITY.\n");
>> >> > 	MSG("\n");
>> >> >+	MSG("  [-j]:\n");
>> >> >+	MSG("      Use raw (O_DIRECT) i/o on dump and bitmap files to avoid expanding kernel pagecache.\n");
>> >> >+	MSG("\n");
>> >> > 	MSG("  [-d DL]:\n");
>> >> > 	MSG("      Specify the type of unnecessary page for analysis.\n");
>> >> > 	MSG("      Pages of the specified type are not copied to DUMPFILE. The page type\n");
>> >> >Index: makedumpfile-1.5.7/makedumpfile.c
>> >> >===================================================================
>> >> >--- makedumpfile-1.5.7.orig/makedumpfile.c
>> >> >+++ makedumpfile-1.5.7/makedumpfile.c
>> >> >@@ -79,8 +79,11 @@ mdf_pfn_t pfn_free;
>> >> > mdf_pfn_t pfn_hwpoison;
>> >> >
>> >> > mdf_pfn_t num_dumped;
>> >> >+long blocksize;
>> >> >
>> >> > int retcd = FAILED;	/* return code */
>> >> >+// jflag is rawio on the dumpfile and bitmap file
>> >> >+int jflag = 0;
>> >> >
>> >> > #define INITIALIZE_LONG_TABLE(table, value) \
>> >> > do { \
>> >> >@@ -966,10 +969,17 @@ int
>> >> > open_dump_file(void)
>> >> > {
>> >> > 	int fd;
>> >> >-	int open_flags = O_RDWR|O_CREAT|O_TRUNC;
>> >> >+	int open_flags;
>> >> >
>> >> >+	if (jflag)
>> >> >+		open_flags = O_RDWR|O_CREAT|O_TRUNC|O_DIRECT;
>> >> >+	else
>> >> >+		open_flags = O_RDWR|O_CREAT|O_TRUNC;
>> >> >+
>> >> >+#if 0
>> >> > 	if (!info->flag_force)
>> >> > 		open_flags |= O_EXCL;
>> >> >+#endif
>> >> >
>> >> > 	if (info->flag_flatten) {
>> >> > 		fd = STDOUT_FILENO;
>> >> >@@ -1005,12 +1015,40 @@ check_dump_file(const char *path)
>> >> > int
>> >> > open_dump_bitmap(void)
>> >> > {
>> >> >-	int i, fd;
>> >> >-	char *tmpname;
>> >> >-
>> >> >-	tmpname = getenv("TMPDIR");
>> >> >-	if (!tmpname)
>> >> >-		tmpname = "/tmp";
>> >> >+	int i, fd, flags;
>> >> >+	char *tmpname, *cp;
>> >> >+	char prefix[100];
>> >> >+	int len;
>> >> >+
>> >> >+	/* -j: saving memory by doing direct i/o, so also avoid /tmp for the bit map files
>> >> >+	 *     because /tmp is using tmpfs */
>> >> >+	if (!jflag) {
>> >> >+		tmpname = getenv("TMPDIR");
>> >> >+		if (!tmpname)
>> >> >+			tmpname = "/tmp";
>> >> >+	} else {
>> >> >+		/* for the crash kernel environment use the prefix of
>> >> >+ 		   the dump name   e.g. /mnt//var/.... */
>> >> >+		if (!strchr(info->name_dumpfile,'v')) {
>> >> >+			printf("no /var found in name_dumpfile %s\n",
>> >> >+			info->name_dumpfile);
>> >> >+			exit(1);
>> >> >+		} else {
>> >> >+			cp = strchr(info->name_dumpfile,'v');
>> >> >+			if (strncmp(cp-1, "/var", 4)) {
>> >> >+				printf("no /var found in name_dumpfile %s\n",
>> >> >+					info->name_dumpfile);
>> >> >+				exit(1);
>> >> >+			}
>> >> >+		}
>> >> >+		len = cp - info->name_dumpfile - 1;
>> >> >+		strncpy(prefix, info->name_dumpfile, len);
>> >> >+		if (*(prefix + len - 1) == '/')
>> >> >+			len -= 1;
>> >> >+		*(prefix + len) = '\0';
>> >> >+		tmpname = prefix;
>> >> >+		strcat(tmpname, "/");
>> >> >+ 	}
>> >> >
>> >> > 	if ((info->name_bitmap = (char *)malloc(sizeof(FILENAME_BITMAP) +
>> >> > 						strlen(tmpname) + 1)) == NULL) {
>> >> >@@ -1019,9 +1057,12 @@ open_dump_bitmap(void)
>> >> > 		return FALSE;
>> >> > 	}
>> >> > 	strcpy(info->name_bitmap, tmpname);
>> >> >-	strcat(info->name_bitmap, "/");
>> >> > 	strcat(info->name_bitmap, FILENAME_BITMAP);
>> >> >-	if ((fd = mkstemp(info->name_bitmap)) < 0) {
>> >> >+	if (jflag)
>> >> >+		flags = O_RDWR|O_CREAT|O_TRUNC|O_DIRECT;
>> >> >+	else
>> >> >+		flags = O_RDWR|O_CREAT|O_TRUNC;
>> >> >+	if ((fd = open(info->name_bitmap, flags)) < 0) {
>> >> > 		ERRMSG("Can't open the bitmap file(%s). %s\n",
>> >> > 		    info->name_bitmap, strerror(errno));
>> >> > 		return FALSE;
>> >> >@@ -2985,6 +3026,7 @@ initialize_bitmap_memory(void)
>> >> > 	struct dump_bitmap *bmp;
>> >> > 	off_t bitmap_offset;
>> >> > 	off_t bitmap_len, max_sect_len;
>> >> >+	char *cp;
>> >> > 	mdf_pfn_t pfn;
>> >> > 	int i, j;
>> >> > 	long block_size;
>> >> >@@ -3006,7 +3048,14 @@ initialize_bitmap_memory(void)
>> >> > 	bmp->fd        = info->fd_memory;
>> >> > 	bmp->file_name = info->name_memory;
>> >> > 	bmp->no_block  = -1;
>> >> >-	memset(bmp->buf, 0, BUFSIZE_BITMAP);
>> >> >+	if ((cp = malloc(blocksize + DIRECT_ALIGN)) == NULL) {
>> >> >+		ERRMSG("Can't allocate memory for the bitmap buffer. %s\n",
>> >> >+		    strerror(errno));
>> >> >+		exit(1);
>> >> >+	}
>> >> >+	bmp->buf_malloced = cp;
>> >> >+	bmp->buf = cp - ((unsigned long)cp % DIRECT_ALIGN) + DIRECT_ALIGN;
>> >> >+	memset(bmp->buf, 0, blocksize);
>> >> > 	bmp->offset = bitmap_offset + bitmap_len / 2;
>> >> > 	info->bitmap_memory = bmp;
>> >> >
>> >> >@@ -3018,6 +3067,7 @@ initialize_bitmap_memory(void)
>> >> > 	if (info->valid_pages == NULL) {
>> >> > 		ERRMSG("Can't allocate memory for the valid_pages. %s\n",
>> >> > 		    strerror(errno));
>> >> >+		free(bmp->buf_malloced);
>> >> > 		free(bmp);
>> >> > 		return FALSE;
>> >> > 	}
>> >> >@@ -3318,9 +3368,18 @@ out:
>> >> > void
>> >> > initialize_bitmap(struct dump_bitmap *bitmap)
>> >> > {
>> >> >+	char *cp;
>> >> >+
>> >> > 	bitmap->fd        = info->fd_bitmap;
>> >> > 	bitmap->file_name = info->name_bitmap;
>> >> > 	bitmap->no_block  = -1;
>> >> >+	if ((cp = malloc(blocksize + DIRECT_ALIGN)) == NULL) {
>> >> >+		ERRMSG("Can't allocate memory for the bitmap buffer. %s\n",
>> >> >+		    strerror(errno));
>> >> >+		exit(1);
>> >> >+	}
>> >> >+	bitmap->buf_malloced = cp;
>> >> >+	bitmap->buf = cp - ((unsigned long)cp % DIRECT_ALIGN) + DIRECT_ALIGN;
>> >> > 	memset(bitmap->buf, 0, BUFSIZE_BITMAP);
>> >> > }
>> >> >
>> >> >@@ -3385,9 +3444,9 @@ set_bitmap(struct dump_bitmap *bitmap, m
>> >> > 	byte = (pfn%PFN_BUFBITMAP)>>3;
>> >> > 	bit  = (pfn%PFN_BUFBITMAP) & 7;
>> >> > 	if (val)
>> >> >-		bitmap->buf[byte] |= 1<<bit;
>> >> >+		*(bitmap->buf + byte) |= 1<<bit;
>> >> > 	else
>> >> >-		bitmap->buf[byte] &= ~(1<<bit);
>> >> >+		*(bitmap->buf + byte) &= ~(1<<bit);
>> >> >
>> >> > 	return TRUE;
>> >> > }
>> >> >@@ -3570,6 +3629,29 @@ read_cache(struct cache_data *cd)
>> >> > 	return TRUE;
>> >> > }
>> >> >
>> >> >+void
>> >> >+fill_to_offset(struct cache_data *cd, int blocksize)
>> >> >+{
>> >> >+	off_t current;
>> >> >+	long num_blocks;
>> >> >+	long i;
>> >> >+
>> >> >+	current = lseek(cd->fd, 0, SEEK_CUR);
>> >> >+	if ((cd->offset - current) % blocksize) {
>> >> >+		printf("ERROR: fill area is %#lx\n", cd->offset - current);
>> >> >+		exit(1);
>> >> >+	}
>> >> >+	if (cd->cache_size < blocksize) {
>> >> >+		printf("ERROR: cache buf is only %ld\n", cd->cache_size);
>> >> >+		exit(1);
>> >> >+	}
>> >> >+	num_blocks = (cd->offset - current) / blocksize;
>> >> >+	for (i = 0; i < num_blocks; i++) {
>> >> >+		write(cd->fd, cd->buf, blocksize);
>> >> >+	}
>> >> >+	return;
>> >> >+}
>> >> >+
>> >> > int
>> >> > is_bigendian(void)
>> >> > {
>> >> >@@ -3639,6 +3721,14 @@ write_buffer(int fd, off_t offset, void
>> >> > int
>> >> > write_cache(struct cache_data *cd, void *buf, size_t size)
>> >> > {
>> >> >+	/* sanity check; do not overflow this buffer */
>> >> >+	/* (it is of cd->cache_size + info->page_size) */
>> >> >+	if (size > ((cd->cache_size - cd->buf_size) + info->page_size)) {
>> >> >+		fprintf(stderr, "write_cache buffer overflow! size %#lx\n",
>> >> >+			size);
>> >> >+		exit(1);
>> >> >+	}
>> >> >+
>> >> > 	memcpy(cd->buf + cd->buf_size, buf, size);
>> >> > 	cd->buf_size += size;
>> >> >
>> >> >@@ -3651,6 +3741,8 @@ write_cache(struct cache_data *cd, void
>> >> >
>> >> > 	cd->buf_size -= cd->cache_size;
>> >> > 	memcpy(cd->buf, cd->buf + cd->cache_size, cd->buf_size);
>> >> >+	if (cd->buf_size)
>> >> >+		memcpy(cd->buf, cd->buf + cd->cache_size, cd->buf_size);
>> >> > 	cd->offset += cd->cache_size;
>> >> > 	return TRUE;
>> >> > }
>> >> >@@ -3682,6 +3774,21 @@ write_cache_zero(struct cache_data *cd,
>> >> > 	return write_cache_bufsz(cd);
>> >> > }
>> >> >
>> >> >+/* flush the full cache to the file */
>> >> >+int
>> >> >+write_cache_flush(struct cache_data *cd)
>> >> >+{
>> >> >+	if (cd->buf_size == 0)
>> >> >+		return TRUE;
>> >> >+	if (cd->buf_size < cd->cache_size) {
>> >> >+		memset(cd->buf + cd->buf_size, 0, cd->cache_size - cd->buf_size);
>> >> >+	}
>> >> >+	cd->buf_size = cd->cache_size;
>> >> >+	if (!write_cache_bufsz(cd))
>> >> >+		return FALSE;
>> >> >+	return TRUE;
>> >> >+}
>> >> >+
>> >> > int
>> >> > read_buf_from_stdin(void *buf, int buf_size)
>> >> > {
>> >> >@@ -4414,11 +4521,19 @@ create_1st_bitmap(void)
>> >> > {
>> >> > 	int i;
>> >> > 	unsigned int num_pt_loads = get_num_pt_loads();
>> >> >- 	char buf[info->page_size];
>> >> >+ 	char *buf;
>> >> > 	mdf_pfn_t pfn, pfn_start, pfn_end, pfn_bitmap1;
>> >> > 	unsigned long long phys_start, phys_end;
>> >> > 	struct timeval tv_start;
>> >> > 	off_t offset_page;
>> >> >+	char *cp;
>> >> >+
>> >> >+	if ((cp = malloc(blocksize + DIRECT_ALIGN)) == NULL) {
>> >> >+		ERRMSG("Can't allocate memory for the bitmap buffer. %s\n",
>> >> >+		    strerror(errno));
>> >> >+		exit(1);
>> >> >+	}
>> >> >+	buf = cp - ((unsigned long)cp % DIRECT_ALIGN) + DIRECT_ALIGN;
>> >> >
>> >> > 	if (info->flag_refiltering)
>> >> > 		return copy_1st_bitmap_from_memory();
>> >> >@@ -4429,7 +4544,7 @@ create_1st_bitmap(void)
>> >> > 	/*
>> >> > 	 * At first, clear all the bits on the 1st-bitmap.
>> >> > 	 */
>> >> >-	memset(buf, 0, sizeof(buf));
>> >> >+	memset(buf, 0, blocksize);
>> >> >
>> >> > 	if (lseek(info->bitmap1->fd, info->bitmap1->offset, SEEK_SET) < 0) {
>> >> > 		ERRMSG("Can't seek the bitmap(%s). %s\n",
>> >> >@@ -4975,9 +5090,17 @@ int
>> >> > copy_bitmap(void)
>> >> > {
>> >> > 	off_t offset;
>> >> >-	unsigned char buf[info->page_size];
>> >> >+	unsigned char *buf;
>> >> >+	unsigned char *cp;
>> >> >  	const off_t failed = (off_t)-1;
>> >> >
>> >> >+	if ((cp = malloc(blocksize + DIRECT_ALIGN)) == NULL) {
>> >> >+		ERRMSG("Can't allocate memory for the bitmap buffer. %s\n",
>> >> >+		    strerror(errno));
>> >> >+		exit(1);
>> >> >+	}
>> >> >+	buf = cp - ((unsigned long)cp % DIRECT_ALIGN) + DIRECT_ALIGN;
>> >> >+
>> >> > 	offset = 0;
>> >> > 	while (offset < (info->len_bitmap / 2)) {
>> >> > 		if (lseek(info->bitmap1->fd, info->bitmap1->offset + offset,
>> >> >@@ -4986,7 +5109,7 @@ copy_bitmap(void)
>> >> > 			    info->name_bitmap, strerror(errno));
>> >> > 			return FALSE;
>> >> > 		}
>> >> >-		if (read(info->bitmap1->fd, buf, sizeof(buf)) != sizeof(buf)) {
>> >> >+		if (read(info->bitmap1->fd, buf, blocksize) != blocksize) {
>> >> > 			ERRMSG("Can't read the dump memory(%s). %s\n",
>> >> > 			    info->name_memory, strerror(errno));
>> >> > 			return FALSE;
>> >> >@@ -4997,12 +5120,12 @@ copy_bitmap(void)
>> >> > 			    info->name_bitmap, strerror(errno));
>> >> > 			return FALSE;
>> >> > 		}
>> >> >-		if (write(info->bitmap2->fd, buf, sizeof(buf)) != sizeof(buf)) {
>> >> >+		if (write(info->bitmap2->fd, buf, blocksize) != blocksize) {
>> >> > 			ERRMSG("Can't write the bitmap(%s). %s\n",
>> >> > 		    	info->name_bitmap, strerror(errno));
>> >> > 			return FALSE;
>> >> > 		}
>> >> >-		offset += sizeof(buf);
>> >> >+		offset += blocksize;
>> >> > 	}
>> >> >
>> >> > 	return TRUE;
>> >> >@@ -5160,6 +5283,8 @@ void
>> >> > free_bitmap1_buffer(void)
>> >> > {
>> >> > 	if (info->bitmap1) {
>> >> >+		if (info->bitmap1->buf_malloced)
>> >> >+			free(info->bitmap1->buf_malloced);
>> >> > 		free(info->bitmap1);
>> >> > 		info->bitmap1 = NULL;
>> >> > 	}
>> >> >@@ -5169,6 +5294,8 @@ void
>> >> > free_bitmap2_buffer(void)
>> >> > {
>> >> > 	if (info->bitmap2) {
>> >> >+		if (info->bitmap2->buf_malloced)
>> >> >+			free(info->bitmap2->buf_malloced);
>> >> > 		free(info->bitmap2);
>> >> > 		info->bitmap2 = NULL;
>> >> > 	}
>> >> >@@ -5287,25 +5414,31 @@ get_loads_dumpfile(void)
>> >> > int
>> >> > prepare_cache_data(struct cache_data *cd)
>> >> > {
>> >> >+	char *cp;
>> >> >+
>> >> > 	cd->fd         = info->fd_dumpfile;
>> >> > 	cd->file_name  = info->name_dumpfile;
>> >> > 	cd->cache_size = info->page_size << info->block_order;
>> >> > 	cd->buf_size   = 0;
>> >> > 	cd->buf        = NULL;
>> >> >
>> >> >-	if ((cd->buf = malloc(cd->cache_size + info->page_size)) == NULL) {
>> >> >+	if ((cp = malloc(cd->cache_size + info->page_size + DIRECT_ALIGN)) == NULL) {
>> >> > 		ERRMSG("Can't allocate memory for the data buffer. %s\n",
>> >> > 		    strerror(errno));
>> >> > 		return FALSE;
>> >> > 	}
>> >> >+	cd->buf_malloced = cp;
>> >> >+	cd->buf = cp - ((unsigned long)cp % DIRECT_ALIGN) + DIRECT_ALIGN;
>> >> > 	return TRUE;
>> >> > }
>> >> >
>> >> > void
>> >> > free_cache_data(struct cache_data *cd)
>> >> > {
>> >> >-	free(cd->buf);
>> >> >+	if (cd->buf_malloced)
>> >> >+		free(cd->buf_malloced);
>> >> > 	cd->buf = NULL;
>> >> >+	cd->buf_malloced = NULL;
>> >> > }
>> >> >
>> >> > int
>> >> >@@ -5554,19 +5687,21 @@ out:
>> >> > }
>> >> >
>> >> > int
>> >> >-write_kdump_header(void)
>> >> >+write_kdump_header(struct cache_data *cd)
>> >> > {
>> >> > 	int ret = FALSE;
>> >> > 	size_t size;
>> >> > 	off_t offset_note, offset_vmcoreinfo;
>> >> >-	unsigned long size_note, size_vmcoreinfo;
>> >> >+	unsigned long size_note, size_vmcoreinfo, remaining_size_note;
>> >> >+	unsigned long write_size, room;
>> >> > 	struct disk_dump_header *dh = info->dump_header;
>> >> > 	struct kdump_sub_header kh;
>> >> >-	char *buf = NULL;
>> >> >+	char *buf = NULL, *cp;
>> >> >
>> >> > 	if (info->flag_elf_dumpfile)
>> >> > 		return FALSE;
>> >> >
>> >> >+	/* uses reads of /proc/vmcore */
>> >> > 	get_pt_note(&offset_note, &size_note);
>> >> >
>> >> > 	/*
>> >> >@@ -5583,6 +5718,7 @@ write_kdump_header(void)
>> >> > 	dh->bitmap_blocks  = divideup(info->len_bitmap, dh->block_size);
>> >> > 	memcpy(&dh->timestamp, &info->timestamp, sizeof(dh->timestamp));
>> >> > 	memcpy(&dh->utsname, &info->system_utsname, sizeof(dh->utsname));
>> >> >+	blocksize = dh->block_size;
>> >> > 	if (info->flag_compress & DUMP_DH_COMPRESSED_ZLIB)
>> >> > 		dh->status |= DUMP_DH_COMPRESSED_ZLIB;
>> >> > #ifdef USELZO
>> >> >@@ -5595,7 +5731,7 @@ write_kdump_header(void)
>> >> > #endif
>> >> >
>> >> > 	size = sizeof(struct disk_dump_header);
>> >> >-	if (!write_buffer(info->fd_dumpfile, 0, dh, size, info->name_dumpfile))
>> >> >+	if (!write_cache(cd, dh, size))
>> >> > 		return FALSE;
>> >> >
>> >> > 	/*
>> >> >@@ -5651,9 +5787,21 @@ write_kdump_header(void)
>> >> > 				goto out;
>> >> > 		}
>> >> >
>> >> >-		if (!write_buffer(info->fd_dumpfile, kh.offset_note, buf,
>> >> >-		    kh.size_note, info->name_dumpfile))
>> >> >-			goto out;
>> >> >+		/* the note may be huge, so do this in a loop to not
>> >> >+		   overflow the cache */
>> >> >+		remaining_size_note = kh.size_note;
>> >> >+		cp = buf;
>> >> >+		do {
>> >> >+			room = cd->cache_size - cd->buf_size;
>> >> >+			if (remaining_size_note > room)
>> >> >+				write_size = room;
>> >> >+			else
>> >> >+				write_size = remaining_size_note;
>> >> >+			if (!write_cache(cd, cp, write_size))
>> >> >+				goto out;
>> >> >+			remaining_size_note -= write_size;
>> >> >+			cp += write_size;
>> >> >+		} while (remaining_size_note);
>> >> >
>> >> > 		if (has_vmcoreinfo()) {
>> >> > 			get_vmcoreinfo(&offset_vmcoreinfo, &size_vmcoreinfo);
>> >> >@@ -5669,8 +5817,7 @@ write_kdump_header(void)
>> >> > 			kh.size_vmcoreinfo = size_vmcoreinfo;
>> >> > 		}
>> >> > 	}
>> >> >-	if (!write_buffer(info->fd_dumpfile, dh->block_size, &kh,
>> >> >-	    size, info->name_dumpfile))
>> >> >+	if (!write_cache(cd, &kh, size))
>> >> > 		goto out;
>> >> >
>> >> > 	info->sub_header = kh;
>> >> >@@ -6267,13 +6414,15 @@ write_elf_pages_cyclic(struct cache_data
>> >> > }
>> >> >
>> >> > int
>> >> >-write_kdump_pages(struct cache_data *cd_header, struct cache_data *cd_page)
>> >> >+write_kdump_pages(struct cache_data *cd_descs, struct cache_data *cd_page)
>> >> > {
>> >> > 	mdf_pfn_t pfn, per, num_dumpable;
>> >> > 	mdf_pfn_t start_pfn, end_pfn;
>> >> > 	unsigned long size_out;
>> >> >+	long prefix;
>> >> > 	struct page_desc pd, pd_zero;
>> >> > 	off_t offset_data = 0;
>> >> >+	off_t initial_offset_data;
>> >> > 	struct disk_dump_header *dh = info->dump_header;
>> >> > 	unsigned char buf[info->page_size], *buf_out = NULL;
>> >> > 	unsigned long len_buf_out;
>> >> >@@ -6281,8 +6430,12 @@ write_kdump_pages(struct cache_data *cd_
>> >> > 	struct timeval tv_start;
>> >> > 	const off_t failed = (off_t)-1;
>> >> > 	unsigned long len_buf_out_zlib, len_buf_out_lzo, len_buf_out_snappy;
>> >> >+	int saved_bytes = 0;
>> >> >+	int cpysize;
>> >> >+	char *save_block1, *save_block_cur, *save_block2;
>> >> >
>> >> > 	int ret = FALSE;
>> >> >+	int status;
>> >> >
>> >> > 	if (info->flag_elf_dumpfile)
>> >> > 		return FALSE;
>> >> >@@ -6324,13 +6477,42 @@ write_kdump_pages(struct cache_data *cd_
>> >> > 	per = per ? per : 1;
>> >> >
>> >> > 	/*
>> >> >-	 * Calculate the offset of the page data.
>> >> >+	 * Calculate the offset of the page_desc's and page data.
>> >> > 	 */
>> >> >-	cd_header->offset
>> >> >+	cd_descs->offset
>> >> > 	    = (DISKDUMP_HEADER_BLOCKS + dh->sub_hdr_size + dh->bitmap_blocks)
>> >> > 		* dh->block_size;
>> >> >-	cd_page->offset = cd_header->offset + sizeof(page_desc_t)*num_dumpable;
>> >> >-	offset_data  = cd_page->offset;
>> >> >+
>> >> >+	/* this is already a pagesize multiple, so well-formed for i/o */
>> >> >+
>> >> >+	cd_page->offset = cd_descs->offset + (sizeof(page_desc_t) * num_dumpable);
>> >> >+	offset_data = cd_page->offset;
>> >> >+
>> >> >+	/* for i/o, round this page data offset down to a block boundary */
>> >> >+	prefix = cd_page->offset % blocksize;
>> >> >+	cd_page->offset -= prefix;
>> >> >+	initial_offset_data = cd_page->offset;
>> >> >+	cd_page->buf_size = prefix;
>> >> >+	memset(cd_page->buf, 0, prefix);
>> >> >+
>> >> >+	fill_to_offset(cd_descs, blocksize);
>> >> >+
>> >> >+	if ((save_block1 = malloc(blocksize * 2)) == NULL) {
>> >> >+		ERRMSG("Can't allocate memory for save block. %s\n",
>> >> >+		       strerror(errno));
>> >> >+		goto out;
>> >> >+	}
>> >> >+	/* put on block address boundary for well-rounded i/o */
>> >> >+	save_block1 += (blocksize - (unsigned long)save_block1 % blocksize);
>> >> >+	save_block_cur = save_block1 + prefix;
>> >> >+	saved_bytes += prefix;
>> >> >+	if ((save_block2 = malloc(blocksize + DIRECT_ALIGN)) == NULL) {
>> >> >+		ERRMSG("Can't allocate memory for save block2. %s\n",
>> >> >+		       strerror(errno));
>> >> >+		goto out;
>> >> >+	}
>> >> >+	/* put on block address boundary for well-rounded i/o */
>> >> >+	save_block2 += (DIRECT_ALIGN - (unsigned long)save_block2 % DIRECT_ALIGN);
>> >> >
>> >> > 	/*
>> >> > 	 * Set a fileoffset of Physical Address 0x0.
>> >> >@@ -6354,6 +6536,14 @@ write_kdump_pages(struct cache_data *cd_
>> >> > 		memset(buf, 0, pd_zero.size);
>> >> > 		if (!write_cache(cd_page, buf, pd_zero.size))
>> >> > 			goto out;
>> >> >+
>> >> >+		cpysize = pd_zero.size;
>> >> >+		if ((saved_bytes + cpysize) > blocksize)
>> >> >+			cpysize = blocksize - saved_bytes;
>> >> >+		memcpy(save_block_cur, buf, cpysize);
>> >> >+		saved_bytes += cpysize;
>> >> >+		save_block_cur += cpysize;
>> >> >+
>> >> > 		offset_data  += pd_zero.size;
>> >> > 	}
>> >> > 	if (info->flag_split) {
>> >> >@@ -6387,7 +6577,7 @@ write_kdump_pages(struct cache_data *cd_
>> >> > 		 */
>> >> > 		if ((info->dump_level & DL_EXCLUDE_ZERO)
>> >> > 		    && is_zero_page(buf, info->page_size)) {
>> >> >-			if (!write_cache(cd_header, &pd_zero, sizeof(page_desc_t)))
>> >> >+			if (!write_cache(cd_descs, &pd_zero, sizeof(page_desc_t)))
>> >> > 				goto out;
>> >> > 			pfn_zero++;
>> >> > 			continue;
>> >> >@@ -6435,25 +6625,68 @@ write_kdump_pages(struct cache_data *cd_
>> >> > 		/*
>> >> > 		 * Write the page header.
>> >> > 		 */
>> >> >-		if (!write_cache(cd_header, &pd, sizeof(page_desc_t)))
>> >> >+		if (!write_cache(cd_descs, &pd, sizeof(page_desc_t)))
>> >> > 			goto out;
>> >> >
>> >> > 		/*
>> >> > 		 * Write the page data.
>> >> > 		 */
>> >> >+		/* kludge: save the partial block where page desc's and data overlap */
>> >> >+		/* (this is the second part of the full block (save_block) where
>> >> >+		    they overlap) */
>> >> >+		if (saved_bytes < blocksize) {
>> >> >+			memcpy(save_block_cur, buf, pd.size);
>> >> >+			saved_bytes += pd.size;
>> >> >+			save_block_cur += pd.size;
>> >> >+		}
>> >> > 		if (!write_cache(cd_page, pd.flags ? buf_out : buf, pd.size))
>> >> > 			goto out;
>> >> > 	}
>> >> >
>> >> > 	/*
>> >> >-	 * Write the remainder.
>> >> >+	 * Write the remainder (well-formed blocks)
>> >> > 	 */
>> >> >-	if (!write_cache_bufsz(cd_page))
>> >> >-		goto out;
>> >> >-	if (!write_cache_bufsz(cd_header))
>> >> >+	/* adjust the cd_descs to write out only full blocks beyond the
>> >> >+	   data in the buffer */
>> >> >+	if (cd_descs->buf_size % blocksize) {
>> >> >+		cd_descs->buf_size +=
>> >> >+			(blocksize - (cd_descs->buf_size % blocksize));
>> >> >+		cd_descs->cache_size = cd_descs->buf_size;
>> >> >+	}
>> >> >+	if (!write_cache_flush(cd_descs))
>> >> > 		goto out;
>> >> >
>> >> > 	/*
>> >> >+	 * kludge: the page data will overwrite the last block of the page_desc's,
>> >> >+	 * so re-construct a block from:
>> >> >+	 *   the last block of the page_desc's (length 'prefix') (will read into
>> >> >+	 *   save_block2) and the end (4096-prefix) of the page data we saved in
>> >> >+	 *   save_block1.
>> >> >+	 */
>> >> >+	if (!write_cache_flush(cd_page))
>> >> >+ 		goto out;
>> >> >+
>> >> >+	if (lseek(cd_page->fd, initial_offset_data, SEEK_SET) == failed) {
>> >> >+		printf("kludge: seek to %#lx, fd %d failed errno %d\n",
>> >> >+			initial_offset_data, cd_page->fd, errno);
>> >> >+		exit(1);
>> >> >+	}
>> >> >+	if (read(cd_page->fd, save_block2, blocksize) != blocksize) {
>> >> >+		printf("kludge: read block2 failed\n");
>> >> >+		exit(1);
>> >> >+	}
>> >> >+	/* combine the overlapping parts into save_block1 */
>> >> >+	memcpy(save_block1, save_block2, prefix);
>> >> >+
>> >> >+	if (lseek(cd_page->fd, initial_offset_data, SEEK_SET) == failed) {
>> >> >+		printf("kludge: seek to %#lx, fd %d failed errno %d\n",
>> >> >+			initial_offset_data, cd_page->fd, errno);
>> >> >+		exit(1);
>> >> >+	}
>> >> >+	status = write(cd_page->fd, save_block1, blocksize);
>> >> >+	/* end of kludged block */
>> >> >+
>> >> >+	/*
>> >> > 	 * print [100 %]
>> >> > 	 */
>> >> > 	print_progress(PROGRESS_COPY, num_dumpable, num_dumpable);
>> >> >@@ -6462,8 +6695,6 @@ write_kdump_pages(struct cache_data *cd_
>> >> >
>> >> > 	ret = TRUE;
>> >> > out:
>> >> >-	if (buf_out != NULL)
>> >> >-		free(buf_out);
>> >> > #ifdef USELZO
>> >> > 	if (wrkmem != NULL)
>> >> > 		free(wrkmem);
>> >> >@@ -6863,51 +7094,47 @@ write_kdump_eraseinfo(struct cache_data
>> >> > }
>> >> >
>> >> > int
>> >> >-write_kdump_bitmap(void)
>> >> >+write_kdump_bitmap(struct cache_data *cd)
>> >> > {
>> >> > 	struct cache_data bm;
>> >> > 	long long buf_size;
>> >> >-	off_t offset;
>> >> >+	long write_size;
>> >> >
>> >> > 	int ret = FALSE;
>> >> >
>> >> > 	if (info->flag_elf_dumpfile)
>> >> > 		return FALSE;
>> >> >
>> >> >+	/* set up to read bit map file in big blocks from the start */
>> >> > 	bm.fd        = info->fd_bitmap;
>> >> > 	bm.file_name = info->name_bitmap;
>> >> > 	bm.offset    = 0;
>> >> > 	bm.buf       = NULL;
>> >> >-
>> >> >-	if ((bm.buf = calloc(1, BUFSIZE_BITMAP)) == NULL) {
>> >> >-		ERRMSG("Can't allocate memory for dump bitmap buffer. %s\n",
>> >> >-		    strerror(errno));
>> >> >-		goto out;
>> >> >+	bm.cache_size = cd->cache_size;
>> >> >+	bm.buf = cd->buf; /* use the bitmap cd */
>> >> >+	/* using the dumpfile cd_bitmap buffer and fd */
>> >> >+	if (lseek(cd->fd, info->offset_bitmap1, SEEK_SET) < 0) {
>> >> >+		ERRMSG("Can't seek the dump file(%s). %s\n",
>> >> >+		       info->name_memory, strerror(errno));
>> >> >+		return FALSE;
>> >> > 	}
>> >> >-	offset = info->offset_bitmap1;
>> >> > 	buf_size = info->len_bitmap;
>> >> >-
>> >> > 	while (buf_size > 0) {
>> >> >-		if (buf_size >= BUFSIZE_BITMAP)
>> >> >-			bm.cache_size = BUFSIZE_BITMAP;
>> >> >-		else
>> >> >-			bm.cache_size = buf_size;
>> >> >-
>> >> > 		if(!read_cache(&bm))
>> >> > 			goto out;
>> >> >-
>> >> >-		if (!write_buffer(info->fd_dumpfile, offset,
>> >> >-		    bm.buf, bm.cache_size, info->name_dumpfile))
>> >> >-			goto out;
>> >> >-
>> >> >-		offset += bm.cache_size;
>> >> >-		buf_size -= BUFSIZE_BITMAP;
>> >> >+		write_size = cd->cache_size;
>> >> >+		if (buf_size < cd->cache_size) {
>> >> >+			write_size = buf_size;
>> >> >+		}
>> >> >+		if (write(cd->fd, cd->buf, write_size) != write_size) {
>> >> >+			ERRMSG("Can't write a destination file. %s\n",
>> >> >+				strerror(errno));
>> >> >+			exit(1);
>> >> >+		}
>> >> >+		buf_size -= bm.cache_size;
>> >> > 	}
>> >> > 	ret = TRUE;
>> >> > out:
>> >> >-	if (bm.buf != NULL)
>> >> >-		free(bm.buf);
>> >> >-
>> >> > 	return ret;
>> >> > }
>> >> >
>> >> >@@ -7992,7 +8219,7 @@ int
>> >> > writeout_dumpfile(void)
>> >> > {
>> >> > 	int ret = FALSE;
>> >> >-	struct cache_data cd_header, cd_page;
>> >> >+	struct cache_data cd_header, cd_page_descs, cd_page, cd_bitmap;
>> >> >
>> >> > 	info->flag_nospace = FALSE;
>> >> >
>> >> >@@ -8005,11 +8232,20 @@ writeout_dumpfile(void)
>> >> > 	}
>> >> > 	if (!prepare_cache_data(&cd_header))
>> >> > 		return FALSE;
>> >> >+	cd_header.offset = 0;
>> >> >
>> >> > 	if (!prepare_cache_data(&cd_page)) {
>> >> > 		free_cache_data(&cd_header);
>> >> > 		return FALSE;
>> >> > 	}
>> >> >+	if (!prepare_cache_data(&cd_page_descs)) {
>> >> >+		free_cache_data(&cd_header);
>> >> >+		free_cache_data(&cd_page);
>> >> >+		return FALSE;
>> >> >+	}
>> >> >+	if (!prepare_cache_data(&cd_bitmap))
>> >> >+		return FALSE;
>> >> >+
>> >> > 	if (info->flag_elf_dumpfile) {
>> >> > 		if (!write_elf_header(&cd_header))
>> >> > 			goto out;
>> >> >@@ -8023,22 +8259,36 @@ writeout_dumpfile(void)
>> >> > 		if (!write_elf_eraseinfo(&cd_header))
>> >> > 			goto out;
>> >> > 	} else if (info->flag_cyclic) {
>> >> >-		if (!write_kdump_header())
>> >> >+		if (!write_kdump_header(&cd_header))
>> >> > 			goto out;
>> >> > 		if (!write_kdump_pages_and_bitmap_cyclic(&cd_header, &cd_page))
>> >> > 			goto out;
>> >> > 		if (!write_kdump_eraseinfo(&cd_page))
>> >> > 			goto out;
>> >> > 	} else {
>> >> >-		if (!write_kdump_header())
>> >> >-			goto out;
>> >> >-		if (!write_kdump_pages(&cd_header, &cd_page))
>> >> >-			goto out;
>> >> >-		if (!write_kdump_eraseinfo(&cd_page))
>> >> >-			goto out;
>> >> >-		if (!write_kdump_bitmap())
>> >> >-			goto out;
>> >> >-	}
>> >> >+		/*
>> >> >+		 * Use cd_header for the caching operation up to the bit map.
>> >> >+		 * Use cd_bitmap for 1-block (4096) operations on the bit map.
>> >> >+		 * (it fits between the file header and page_desc's, both of
>> >> >+		 *  which end and start on block boundaries)
>> >> >+		 * Then use cd_page_descs and cd_page for page headers and
>> >> >+		 * data (and eraseinfo).
>> >> >+		 * Then back to cd_header to fill in the bitmap.
>> >> >+		 */
>> >> >+
>> >> >+		if (!write_kdump_header(&cd_header))
>> >> >+			goto out;
>> >> >+		write_cache_flush(&cd_header);
>> >> >+
>> >> >+		if (!write_kdump_pages(&cd_page_descs, &cd_page))
>> >> >+ 			goto out;
>> >> >+ 		if (!write_kdump_eraseinfo(&cd_page))
>> >> >+ 			goto out;
>> >> >+
>> >> >+		cd_bitmap.offset = info->offset_bitmap1;
>> >> >+		if (!write_kdump_bitmap(&cd_bitmap))
>> >> >+ 			goto out;
>> >> >+ 	}
>> >> > 	if (info->flag_flatten) {
>> >> > 		if (!write_end_flat_header())
>> >> > 			goto out;
>> >> >@@ -8198,11 +8448,17 @@ create_dumpfile(void)
>> >> > 		if (!get_elf_info(info->fd_memory, info->name_memory))
>> >> > 			return FALSE;
>> >> > 	}
>> >> >+	blocksize = info->page_size;
>> >> >+	if (!blocksize)
>> >> >+		blocksize = sysconf(_SC_PAGE_SIZE);
>> >> > 	if (!initial())
>> >> > 		return FALSE;
>> >> >
>> >> > 	print_vtop();
>> >> >
>> >> >+	if (jflag)
>> >> >+		PROGRESS_MSG("Using O_DIRECT i/o for dump and bitmap.\n");
>> >> >+
>> >> > 	num_retry = 0;
>> >> > retry:
>> >> > 	if (info->flag_refiltering) {
>> >> >@@ -9285,7 +9541,6 @@ int show_mem_usage(void)
>> >> > 		return FALSE;
>> >> > 	}
>> >> >
>> >> >-
>> >> > 	if (!info->flag_cyclic)
>> >> > 		info->flag_cyclic = TRUE;
>> >> >
>> >> >@@ -9379,7 +9634,7 @@ main(int argc, char *argv[])
>> >> >
>> >> > 	info->block_order = DEFAULT_ORDER;
>> >> > 	message_level = DEFAULT_MSG_LEVEL;
>> >> >-	while ((opt = getopt_long(argc, argv, "b:cDd:EFfg:hi:lpRvXx:", longopts,
>> >> >+	while ((opt = getopt_long(argc, argv, "b:cDd:EFfg:hi:jlpRvXx:", longopts,
>> >> > 	    NULL)) != -1) {
>> >> > 		switch (opt) {
>> >> > 		case OPT_BLOCK_ORDER:
>> >> >@@ -9423,6 +9678,10 @@ main(int argc, char *argv[])
>> >> > 			info->flag_read_vmcoreinfo = 1;
>> >> > 			info->name_vmcoreinfo = optarg;
>> >> > 			break;
>> >> >+		case 'j':
>> >> >+			jflag = 1;
>> >> >+			info->flag_cyclic = FALSE; // saving memory to avoid cyclic
>> >> >+			break;
>> >> > 		case OPT_DISKSET:
>> >> > 			if (!sadump_add_diskset_info(optarg))
>> >> > 				goto out;
>> >> >
>> >> >_______________________________________________
>> >> >kexec mailing list
>> >> >kexec at lists.infradead.org
>> >> >http://lists.infradead.org/mailman/listinfo/kexec
>> >
>> >--
>> >Cliff Wickman
>> >SGI
>> >cpw at sgi.com
>> >(651)683-7524 vnet 207524
>> >(651)482-9347 home
>
>--
>Cliff Wickman
>SGI
>cpw at sgi.com
>(651)683-7524 vnet 207524
>(651)482-9347 home


From liu.song11 at zte.com.cn  Tue Jan  6 01:20:27 2015
From: liu.song11 at zte.com.cn (liu.song11 at zte.com.cn)
Date: Tue, 6 Jan 2015 17:20:27 +0800
Subject: some questions about elf_ppc64_load
In-Reply-To: <54AB9EAF.2020004@linux.vnet.ibm.com>
References: <OF9C6CA8A8.9D7923CD-ON48257DC5.0023F61F-48257DC5.0024E800@zte.com.cn>
 <54AB9EAF.2020004@linux.vnet.ibm.com>
Message-ID: <OF89302667.7C36AC27-ON48257DC5.002FF943-48257DC5.00333569@zte.com.cn>



Laurent Dufour <ldufour at linux.vnet.ibm.com> wrote on 2015-01-06 16:37:03:

> From: Laurent Dufour <ldufour at linux.vnet.ibm.com>
> To: liu.song11 at zte.com.cn,
> Cc: kexec at lists.infradead.org
> Date: 2015-01-06 16:37
> Subject: Re: some questions about elf_ppc64_load
>
> On 06/01/2015 07:44, liu.song11 at zte.com.cn wrote:
> >
> > hi all?
> >
> > I use kexec-tools-2.0.8 for ppc64, in fuction elf_ppc64_load prompt "Can't use ramdisk with device tree blob input".
> >
> > I don't know why here ramdisk can't together with dtb,and need some help.
>
> The device tree is also specifying initrd (see kexec/fs2dt.c putnode),
> so there is 2 initrd specify in that case. The code is a bit rough here,
> but I guess this is because there is not yet a valid use case.
hi Cheers,
In "elf_ppc64_load", if we haven't assign a special dtb by --dtb=<filename>, then here will use "create_flatten_tree" to
create one. However, My board need assign a dtb, so I use --dtb=my.dtb to assign it. The dtb file is valid which I use it
to start kernel under uboot. For now, I just confuse is why the "create_flatten_tree" create dtb could work, but I assigned
dtb couldn't, my.dtb is 40KB and smaller than create dtb's 76KB. The two ways both use add_buffer with param "buf_end = -1",
so will add segment for the top of the valid mem range, and my.dtb is smaller which less chance of overlap some data in memory.


>
> > I also have some questions, when I just ./kexe -l vmlinux then ./kexec -e, will jump to second kernel sucessfully,
> >
> > but without useful dtb, then device can't init correct. But, when I input ./kexec -l vmlinux --dtb=my.dtb , then
> >
> > ./kexec -e, the kernel couldn't work and crash in head_64.S, this problem also make me confused.
>
> It's hard to state with no more details. I guess the device tree blob
> you're using is not valid, or contain bad data.
>
> Cheers,
> Laurent.
>
> > Thanks
> > Regards
> > --------------------------------------------------------
> > ZTE Information Security Notice: The information contained in this mail (and any attachment transmitted herewith) is privileged and confidential and is intended for the exclusive use of the addressee(s).  If you are not an intended recipient, any disclosure, reproduction, distribution or other dissemination or use of the information contained is strictly prohibited.  If you have received this mail in error, please delete it and notify us immediately.
> > _______________________________________________
> > kexec mailing list
> > kexec at lists.infradead.org
> > http://lists.infradead.org/mailman/listinfo/kexec
> >
>
--------------------------------------------------------
ZTE Information Security Notice: The information contained in this mail (and any attachment transmitted herewith) is privileged and confidential and is intended for the exclusive use of the addressee(s).  If you are not an intended recipient, any disclosure, reproduction, distribution or other dissemination or use of the information contained is strictly prohibited.  If you have received this mail in error, please delete it and notify us immediately.

From ldufour at linux.vnet.ibm.com  Tue Jan  6 02:00:07 2015
From: ldufour at linux.vnet.ibm.com (Laurent Dufour)
Date: Tue, 06 Jan 2015 11:00:07 +0100
Subject: some questions about elf_ppc64_load
In-Reply-To: <OF89302667.7C36AC27-ON48257DC5.002FF943-48257DC5.00333569@zte.com.cn>
References: <OF9C6CA8A8.9D7923CD-ON48257DC5.0023F61F-48257DC5.0024E800@zte.com.cn>
 <54AB9EAF.2020004@linux.vnet.ibm.com>
 <OF89302667.7C36AC27-ON48257DC5.002FF943-48257DC5.00333569@zte.com.cn>
Message-ID: <54ABB227.7010203@linux.vnet.ibm.com>

On 06/01/2015 10:20, liu.song11 at zte.com.cn wrote:
> 
> 
> Laurent Dufour <ldufour at linux.vnet.ibm.com> wrote on 2015-01-06 16:37:03:
> 
>> From: Laurent Dufour <ldufour at linux.vnet.ibm.com>
>> To: liu.song11 at zte.com.cn,
>> Cc: kexec at lists.infradead.org
>> Date: 2015-01-06 16:37
>> Subject: Re: some questions about elf_ppc64_load
>>
>> On 06/01/2015 07:44, liu.song11 at zte.com.cn wrote:
>>>
>>> hi all?
>>>
>>> I use kexec-tools-2.0.8 for ppc64, in fuction elf_ppc64_load prompt "Can't use ramdisk with device tree blob input".
>>>
>>> I don't know why here ramdisk can't together with dtb,and need some help.
>>
>> The device tree is also specifying initrd (see kexec/fs2dt.c putnode),
>> so there is 2 initrd specify in that case. The code is a bit rough here,
>> but I guess this is because there is not yet a valid use case.
> hi Cheers,
> In "elf_ppc64_load", if we haven't assign a special dtb by --dtb=<filename>, then here will use "create_flatten_tree" to
> create one. However, My board need assign a dtb, so I use --dtb=my.dtb to assign it. The dtb file is valid which I use it
> to start kernel under uboot. For now, I just confuse is why the "create_flatten_tree" create dtb could work, but I assigned
> dtb couldn't, my.dtb is 40KB and smaller than create dtb's 76KB. The two ways both use add_buffer with param "buf_end = -1",
> so will add segment for the top of the valid mem range, and my.dtb is smaller which less chance of overlap some data in memory.

The service create_flatten_tree create a DTB from the one inherited at
the boot time of the currently running kernel (/proc/device-tree). I
guess this device tree is valid since the current kernel is running ;)

You may try to see what the device tree passed to your kexed kernel is,
may be there is something wrong there. Also looking at what going wrong
when the kexed kernel is panicing may be helpful.

Cheers,
Laurent.

> 
>>
>>> I also have some questions, when I just ./kexe -l vmlinux then ./kexec -e, will jump to second kernel sucessfully,
>>>
>>> but without useful dtb, then device can't init correct. But, when I input ./kexec -l vmlinux --dtb=my.dtb , then
>>>
>>> ./kexec -e, the kernel couldn't work and crash in head_64.S, this problem also make me confused.
>>
>> It's hard to state with no more details. I guess the device tree blob
>> you're using is not valid, or contain bad data.
>>
>> Cheers,
>> Laurent.
>>
>>> Thanks
>>> Regards
>>> --------------------------------------------------------
>>> ZTE Information Security Notice: The information contained in this mail (and any attachment transmitted herewith) is privileged and confidential and is intended for the exclusive use of the addressee(s).  If you are not an intended recipient, any disclosure, reproduction, distribution or other dissemination or use of the information contained is strictly prohibited.  If you have received this mail in error, please delete it and notify us immediately.
>>> _______________________________________________
>>> kexec mailing list
>>> kexec at lists.infradead.org
>>> http://lists.infradead.org/mailman/listinfo/kexec
>>>
>>
> --------------------------------------------------------
> ZTE Information Security Notice: The information contained in this mail (and any attachment transmitted herewith) is privileged and confidential and is intended for the exclusive use of the addressee(s).  If you are not an intended recipient, any disclosure, reproduction, distribution or other dissemination or use of the information contained is strictly prohibited.  If you have received this mail in error, please delete it and notify us immediately.
> 



From bhe at redhat.com  Tue Jan  6 04:46:44 2015
From: bhe at redhat.com (Baoquan He)
Date: Tue, 6 Jan 2015 20:46:44 +0800
Subject: [PATCH] kexec, remove panic_on_warn kernel parameter from kdump
 situations
In-Reply-To: <20150106080500.GD2113@darkstar.redhat.com>
References: <1419951471-19765-1-git-send-email-prarit@redhat.com>
 <20150102125423.GA18785@redhat.com> <54A69808.4010401@redhat.com>
 <20150102131709.GB18785@redhat.com>
 <20150106014405.GA2113@darkstar.redhat.com>
 <20150106015412.GB20058@redhat.com>
 <20150106080500.GD2113@darkstar.redhat.com>
Message-ID: <20150106124644.GB5965@dhcp-16-105.nay.redhat.com>

On 01/06/15 at 04:05pm, Dave Young wrote:
> On 01/05/15 at 08:54pm, Vivek Goyal wrote:
> > On Tue, Jan 06, 2015 at 09:44:05AM +0800, Dave Young wrote:
> > > On 01/02/15 at 08:17am, Vivek Goyal wrote:
> > > > On Fri, Jan 02, 2015 at 08:07:20AM -0500, Prarit Bhargava wrote:
> > > > > 
> > > > > 
> > > > > On 01/02/2015 07:54 AM, Vivek Goyal wrote:
> > > > > > On Tue, Dec 30, 2014 at 09:57:51AM -0500, Prarit Bhargava wrote:
> > > > > >> panic_on_warn kernel parameter will cause the kernel to panic when a
> > > > > >> WARN() is hit in the kernel.  This is not a good situation for the kdump
> > > > > >> kernel because then it would be possible for the kdump kernel to panic in
> > > > > >> a non-fatal WARN().
> > > > > >>
> > > > > >> This patch removes panic_on_warn as a kernel parameter for the kdump
> > > > > >> kernel.
> > > > > >>
> > > > > > 
> > > > > > I think modifying kexec-tools is not best place for this. It probably is better to take care of this in distribution specific scripts.
> > > > > > 
> > > > > > In the past we have learnt that it is best that kexec-tools does least
> > > > > > amount of manipulation with command line.
> > > > > 
> > > > > Well .. here's the question to think about: what does adding panic_on_warn to
> > > > > the kdump kernel get you?  AFAICT, nothing.
> > > > 
> > > > Let us consider a hypothetical situation. What if we have some buggy code
> > > > which will corrupt file system in certain situation and we detect that
> > > > situation and throw a warning. 
> > > > 
> > > > In that case as a work around specifying panic_on_warn in kdump kernel
> > > > will make sense as we don't want to make further progress if we hit
> > > > the warning as it has potential to corrupt fs.
> > > > 
> > > > Again this is hypothetical but it can happen. So panic_on_warn might
> > > > still be useful in kdump kernel for some corner debugging cases.
> > > > 
> > > > That's why I think we should do it in distribution specific scripts
> > > > and that too only if user did not specify panic_on_warn for second
> > > > kernel explicitly.
> > > 
> > > Thinking of user who use upstream kexec-tools instead of distribution toolset,
> > > In case kexec --reuse-cmdline, it will copy /proc/cmdline, but user will have
> > > no way to remove part of them.
> > > 
> > > I do want to insist on removing 'panic_on_warn' in upstream kexec-tools, but
> > > we should give user an option to remove it. Something like:
> > > 
> > > kexec --reuse-cmdline --remove-params="panic_on_warn" will be good.
> > 
> > If user is using --reuse-commandline at the same time does not want some
> > of the parameters from command line, then don't use --reuse-commandline.
> > 
> > This is overenginnering. First provide an option to reuse the commandline
> > and provide another option to selectively remove some parameters from that
> > commandline.
> > 
> > What's wrong with existing parameters of --command-line. This just allows
> > user to specify whatever command line is suitable.
> > 
> > So, no, we should not provide --remove-params. If existing command line
> > does not work for new kenrel, then user should not use
> > --reuse-commandline option.
> 
> Hmm, ok. So hope one who is use panic_on_warn in 1st kernel know what he is doing
> and do not simply copy the 1st kernel cmdline for 2nd kernel.

I am fine with which postion it should be cared in some extent.
This is truly a problem we need consider. If one distribution used
doesn't handle it, and user using latest upstream kernel will be
surprised by this.


> 
> _______________________________________________
> kexec mailing list
> kexec at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/kexec


From vgoyal at redhat.com  Tue Jan  6 05:06:01 2015
From: vgoyal at redhat.com (Vivek Goyal)
Date: Tue, 6 Jan 2015 08:06:01 -0500
Subject: [PATCH] kexec, remove panic_on_warn kernel parameter from kdump
 situations
In-Reply-To: <20150106080500.GD2113@darkstar.redhat.com>
References: <1419951471-19765-1-git-send-email-prarit@redhat.com>
 <20150102125423.GA18785@redhat.com> <54A69808.4010401@redhat.com>
 <20150102131709.GB18785@redhat.com>
 <20150106014405.GA2113@darkstar.redhat.com>
 <20150106015412.GB20058@redhat.com>
 <20150106080500.GD2113@darkstar.redhat.com>
Message-ID: <20150106130601.GE20433@redhat.com>

On Tue, Jan 06, 2015 at 04:05:00PM +0800, Dave Young wrote:
> On 01/05/15 at 08:54pm, Vivek Goyal wrote:
> > On Tue, Jan 06, 2015 at 09:44:05AM +0800, Dave Young wrote:
> > > On 01/02/15 at 08:17am, Vivek Goyal wrote:
> > > > On Fri, Jan 02, 2015 at 08:07:20AM -0500, Prarit Bhargava wrote:
> > > > > 
> > > > > 
> > > > > On 01/02/2015 07:54 AM, Vivek Goyal wrote:
> > > > > > On Tue, Dec 30, 2014 at 09:57:51AM -0500, Prarit Bhargava wrote:
> > > > > >> panic_on_warn kernel parameter will cause the kernel to panic when a
> > > > > >> WARN() is hit in the kernel.  This is not a good situation for the kdump
> > > > > >> kernel because then it would be possible for the kdump kernel to panic in
> > > > > >> a non-fatal WARN().
> > > > > >>
> > > > > >> This patch removes panic_on_warn as a kernel parameter for the kdump
> > > > > >> kernel.
> > > > > >>
> > > > > > 
> > > > > > I think modifying kexec-tools is not best place for this. It probably is better to take care of this in distribution specific scripts.
> > > > > > 
> > > > > > In the past we have learnt that it is best that kexec-tools does least
> > > > > > amount of manipulation with command line.
> > > > > 
> > > > > Well .. here's the question to think about: what does adding panic_on_warn to
> > > > > the kdump kernel get you?  AFAICT, nothing.
> > > > 
> > > > Let us consider a hypothetical situation. What if we have some buggy code
> > > > which will corrupt file system in certain situation and we detect that
> > > > situation and throw a warning. 
> > > > 
> > > > In that case as a work around specifying panic_on_warn in kdump kernel
> > > > will make sense as we don't want to make further progress if we hit
> > > > the warning as it has potential to corrupt fs.
> > > > 
> > > > Again this is hypothetical but it can happen. So panic_on_warn might
> > > > still be useful in kdump kernel for some corner debugging cases.
> > > > 
> > > > That's why I think we should do it in distribution specific scripts
> > > > and that too only if user did not specify panic_on_warn for second
> > > > kernel explicitly.
> > > 
> > > Thinking of user who use upstream kexec-tools instead of distribution toolset,
> > > In case kexec --reuse-cmdline, it will copy /proc/cmdline, but user will have
> > > no way to remove part of them.
> > > 
> > > I do want to insist on removing 'panic_on_warn' in upstream kexec-tools, but
> > > we should give user an option to remove it. Something like:
> > > 
> > > kexec --reuse-cmdline --remove-params="panic_on_warn" will be good.
> > 
> > If user is using --reuse-commandline at the same time does not want some
> > of the parameters from command line, then don't use --reuse-commandline.
> > 
> > This is overenginnering. First provide an option to reuse the commandline
> > and provide another option to selectively remove some parameters from that
> > commandline.
> > 
> > What's wrong with existing parameters of --command-line. This just allows
> > user to specify whatever command line is suitable.
> > 
> > So, no, we should not provide --remove-params. If existing command line
> > does not work for new kenrel, then user should not use
> > --reuse-commandline option.
> 
> Hmm, ok. So hope one who is use panic_on_warn in 1st kernel know what he is doing
> and do not simply copy the 1st kernel cmdline for 2nd kernel.

--reuse-commandline will work only for kexec case and not kdump case. And
in case of kexec, it is fine to use panic_on_warn in kexeced kernel. So 
there is no need to worry here.

Thanks
Vivek



From vgoyal at redhat.com  Tue Jan  6 05:07:35 2015
From: vgoyal at redhat.com (Vivek Goyal)
Date: Tue, 6 Jan 2015 08:07:35 -0500
Subject: [PATCH] kexec, remove panic_on_warn kernel parameter from kdump
 situations
In-Reply-To: <20150106124644.GB5965@dhcp-16-105.nay.redhat.com>
References: <1419951471-19765-1-git-send-email-prarit@redhat.com>
 <20150102125423.GA18785@redhat.com> <54A69808.4010401@redhat.com>
 <20150102131709.GB18785@redhat.com>
 <20150106014405.GA2113@darkstar.redhat.com>
 <20150106015412.GB20058@redhat.com>
 <20150106080500.GD2113@darkstar.redhat.com>
 <20150106124644.GB5965@dhcp-16-105.nay.redhat.com>
Message-ID: <20150106130735.GF20433@redhat.com>

On Tue, Jan 06, 2015 at 08:46:44PM +0800, Baoquan He wrote:
> On 01/06/15 at 04:05pm, Dave Young wrote:
> > On 01/05/15 at 08:54pm, Vivek Goyal wrote:
> > > On Tue, Jan 06, 2015 at 09:44:05AM +0800, Dave Young wrote:
> > > > On 01/02/15 at 08:17am, Vivek Goyal wrote:
> > > > > On Fri, Jan 02, 2015 at 08:07:20AM -0500, Prarit Bhargava wrote:
> > > > > > 
> > > > > > 
> > > > > > On 01/02/2015 07:54 AM, Vivek Goyal wrote:
> > > > > > > On Tue, Dec 30, 2014 at 09:57:51AM -0500, Prarit Bhargava wrote:
> > > > > > >> panic_on_warn kernel parameter will cause the kernel to panic when a
> > > > > > >> WARN() is hit in the kernel.  This is not a good situation for the kdump
> > > > > > >> kernel because then it would be possible for the kdump kernel to panic in
> > > > > > >> a non-fatal WARN().
> > > > > > >>
> > > > > > >> This patch removes panic_on_warn as a kernel parameter for the kdump
> > > > > > >> kernel.
> > > > > > >>
> > > > > > > 
> > > > > > > I think modifying kexec-tools is not best place for this. It probably is better to take care of this in distribution specific scripts.
> > > > > > > 
> > > > > > > In the past we have learnt that it is best that kexec-tools does least
> > > > > > > amount of manipulation with command line.
> > > > > > 
> > > > > > Well .. here's the question to think about: what does adding panic_on_warn to
> > > > > > the kdump kernel get you?  AFAICT, nothing.
> > > > > 
> > > > > Let us consider a hypothetical situation. What if we have some buggy code
> > > > > which will corrupt file system in certain situation and we detect that
> > > > > situation and throw a warning. 
> > > > > 
> > > > > In that case as a work around specifying panic_on_warn in kdump kernel
> > > > > will make sense as we don't want to make further progress if we hit
> > > > > the warning as it has potential to corrupt fs.
> > > > > 
> > > > > Again this is hypothetical but it can happen. So panic_on_warn might
> > > > > still be useful in kdump kernel for some corner debugging cases.
> > > > > 
> > > > > That's why I think we should do it in distribution specific scripts
> > > > > and that too only if user did not specify panic_on_warn for second
> > > > > kernel explicitly.
> > > > 
> > > > Thinking of user who use upstream kexec-tools instead of distribution toolset,
> > > > In case kexec --reuse-cmdline, it will copy /proc/cmdline, but user will have
> > > > no way to remove part of them.
> > > > 
> > > > I do want to insist on removing 'panic_on_warn' in upstream kexec-tools, but
> > > > we should give user an option to remove it. Something like:
> > > > 
> > > > kexec --reuse-cmdline --remove-params="panic_on_warn" will be good.
> > > 
> > > If user is using --reuse-commandline at the same time does not want some
> > > of the parameters from command line, then don't use --reuse-commandline.
> > > 
> > > This is overenginnering. First provide an option to reuse the commandline
> > > and provide another option to selectively remove some parameters from that
> > > commandline.
> > > 
> > > What's wrong with existing parameters of --command-line. This just allows
> > > user to specify whatever command line is suitable.
> > > 
> > > So, no, we should not provide --remove-params. If existing command line
> > > does not work for new kenrel, then user should not use
> > > --reuse-commandline option.
> > 
> > Hmm, ok. So hope one who is use panic_on_warn in 1st kernel know what he is doing
> > and do not simply copy the 1st kernel cmdline for 2nd kernel.
> 
> I am fine with which postion it should be cared in some extent.
> This is truly a problem we need consider. If one distribution used
> doesn't handle it, and user using latest upstream kernel will be
> surprised by this.

This is true for all kernel parameters. One distribution might decide
to use some kenrel parameter by default during installation and other
might not. Is kexec-tools supposed to keep track of all kernel parameters?
It is just not possible.

That's why it is up to distributions to figure out what parameters work
for them and modify their scripts accordingly.

Thanks
Vivek


From bhe at redhat.com  Mon Jan  5 22:37:18 2015
From: bhe at redhat.com (Baoquan He)
Date: Tue, 6 Jan 2015 14:37:18 +0800
Subject: [PATCH 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel
In-Reply-To: <1419239729-26529-1-git-send-email-zhen-hual@hp.com>
References: <1419239729-26529-1-git-send-email-zhen-hual@hp.com>
Message-ID: <20150106063718.GA2454@dhcp-16-105.nay.redhat.com>

Hi Zhenhua,

I just tested your patchset based on 3.19.0-rc2+, and found several dmar
fault and intr-remap fault, it seems not the same as Takao's, please
check the attachment.

Thanks
Baoquan

-------------- next part --------------

[root at dhcp-16-105 ~]# kdumpctl restart
kexec: failed to unloaded kdump kernel
Stopping kdump: [FAILED]
+ /sbin/kexec -p '--command-line=BOOT_IMAGE=/vmlinuz-3.19.0-rc2+ root=UUID=f170152e-de83-46ee-9546-8ccd53f9753b ro rd.md=0 rd.lvm=0 rd.dm=0 vconsole.keymap=us rd.luk+
+ set +x
kexec: loaded kdump kernel
Starting kdump: [OK]
[root at dhcp-16-105 ~]# echo c >/proc/sysrq-trigger 
[  722.444549] SysRq : Trigger a crash
[  722.448133] BUG: unable to handle kernel NULL pointer dereference at           (null)
[  722.456044] IP: [<ffffffff8145a3d6>] sysrq_handle_crash+0x16/0x20
[  722.462179] PGD 404720067 PUD 41afd6067 PMD 0 
[  722.466719] Oops: 0002 [#1] SMP 
[  722.469999] Modules linked in: fuse ipt_MASQUERADE nf_nat_masquerade_ipv4 xt_CHECKSUM tun ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 cfg80211 xt_conntrack ebtable_nc
[  722.570165] CPU: 0 PID: 1979 Comm: bash Not tainted 3.19.0-rc2+ #93
[  722.576453] Hardware name: Hewlett-Packard HP Z420 Workstation/1589, BIOS J61 v01.02 03/09/2012
[  722.585176] task: ffff88041d6a9b60 ti: ffff88040882c000 task.ti: ffff88040882c000
[  722.592699] RIP: 0010:[<ffffffff8145a3d6>]  [<ffffffff8145a3d6>] sysrq_handle_crash+0x16/0x20
[  722.601292] RSP: 0018:ffff88040882fe58  EFLAGS: 00010246
[  722.606618] RAX: 000000000000000f RBX: ffffffff81cafdc0 RCX: 0000000000000000
[  722.613776] RDX: 0000000000000000 RSI: ffff88042fc0e6f8 RDI: 0000000000000063
[  722.620932] RBP: ffff88040882fe58 R08: 00000000000000c2 R09: ffffffff81eec85c
[  722.628098] R10: 000000000000040f R11: 000000000000040e R12: 0000000000000063
[  722.635249] R13: 0000000000000000 R14: 0000000000000007 R15: 0000000000000000
[  722.642415] FS:  00007f0cca07b740(0000) GS:ffff88042fc00000(0000) knlGS:0000000000000000
[  722.650529] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  722.656294] CR2: 0000000000000000 CR3: 00000004095e4000 CR4: 00000000000407f0
[  722.663444] Stack:
[  722.665471]  ffff88040882fe88 ffffffff8145abc7 0000000000000002 00007f0cca092000
[  722.672960]  0000000000000002 ffff88040882ff48 ffff88040882fea8 ffffffff8145b073
[  722.680440]  0000000000000001 ffff8803ee48b300 ffff88040882fed8 ffffffff8126ebcd
[  722.687927] Call Trace:
[  722.690390]  [<ffffffff8145abc7>] __handle_sysrq+0x107/0x170
[  722.696079]  [<ffffffff8145b073>] write_sysrq_trigger+0x33/0x40
[  722.702029]  [<ffffffff8126ebcd>] proc_reg_write+0x3d/0x80
[  722.707536]  [<ffffffff81205a47>] vfs_write+0xb7/0x1f0
[  722.712697]  [<ffffffff810227ec>] ? do_audit_syscall_entry+0x6c/0x70
[  722.719084]  [<ffffffff812066c5>] SyS_write+0x55/0xd0
[  722.724159]  [<ffffffff81747069>] system_call_fastpath+0x12/0x17
[  722.730176] Code: c1 f7 ff ff eb db 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 66 66 66 66 90 55 c7 05 b4 e9 a8 00 01 00 00 00 48 89 e5 0f ae f8 <c6> 04 25 00 0 
[  722.750274] RIP  [<ffffffff8145a3d6>] sysrq_handle_crash+0x16/0x20
[  722.756509]  RSP <ffff88040882fe58>
[  722.760023] CR2: 0000000000000000
h
 [    0.000000] ERROR: earlyprintk= earlyser already used
[    0.000000] debug: ignoring loglevel setting.
[    0.000000] NX (Execute Disable) protection: active
[    0.000000] SMBIOS 2.7 present.
[    0.000000] DMI: Hewlett-Packard HP Z420 Workstation/1589, BIOS J61 v01.02 03/09/2012
[    0.000000] e820: update [mem 0x00000000-0x00000fff] usable ==> reserved
[    0.000000] e820: remove [mem 0x000a0000-0x000fffff] usable
[    0.000000] e820: last_pfn = 0x38000 max_arch_pfn = 0x400000000
[    0.000000] MTRR default type: write-back
[    0.000000] MTRR fixed ranges enabled:
[    0.000000]   00000-9FFFF write-back
[    0.000000]   A0000-BFFFF uncachable
[    0.000000]   C0000-DBFFF write-protect
[    0.000000]   DC000-E9FFF uncachable
[    0.000000]   EA000-FFFFF write-protect
[    0.000000] MTRR variable ranges enabled:
[    0.000000]   0 base 0000CC000000 mask 3FFFFC000000 uncachable
[    0.000000]   1 base 0000D0000000 mask 3FFFF0000000 uncachable
[    0.000000]   2 base 0000E0000000 mask 3FFFE0000000 uncachable
[    0.000000]   3 base 000430000000 mask 3FFFF0000000 uncachable
[    0.000000]   4 disabled
[    0.000000]   5 disabled
[    0.000000]   6 disabled
[    0.000000]   7 disabled
[    0.000000]   8 disabled
[    0.000000]   9 disabled
[    0.000000] PAT configuration [0-7]: WB  WC  UC- UC  WB  WC  UC- UC  
[    0.000000] x2apic enabled by BIOS, switching to x2apic ops
[    0.000000] found SMP MP-table at [mem 0x000f4bc0-0x000f4bcf] mapped at [ffff8800000f4bc0]
[    0.000000] Base memory trampoline at [ffff880000090000] 90000 size 24576
[    0.000000] Using GB pages for direct mapping
[    0.000000] init_memory_mapping: [mem 0x00000000-0x000fffff]
[    0.000000]  [mem 0x00000000-0x000fffff] page 4k
[    0.000000] BRK [0x36ffa000, 0x36ffafff] PGTABLE
[    0.000000] BRK [0x36ffb000, 0x36ffbfff] PGTABLE
[    0.000000] BRK [0x36ffc000, 0x36ffcfff] PGTABLE
[    0.000000] init_memory_mapping: [mem 0x37c00000-0x37dfffff]
[    0.000000]  [mem 0x37c00000-0x37dfffff] page 2M
[    0.000000] init_memory_mapping: [mem 0x34000000-0x37bfffff]
[    0.000000]  [mem 0x34000000-0x37bfffff] page 2M
[    0.000000] init_memory_mapping: [mem 0x28000000-0x33ffffff]
[    0.000000]  [mem 0x28000000-0x33ffffff] page 2M
[    0.000000] init_memory_mapping: [mem 0x37e00000-0x37f65fff]
[    0.000000]  [mem 0x37e00000-0x37f65fff] page 4k
[    0.000000] BRK [0x36ffd000, 0x36ffdfff] PGTABLE
[    0.000000] RAMDISK: [mem 0x32c65000-0x35ffffff]
[    0.000000] ACPI: Early table checksum verification disabled
[    0.000000] ACPI: RSDP 0x00000000000F9810 000024 (v02 HPQOEM)
[    0.000000] ACPI: XSDT 0x00000000CBA28078 00006C (v01 HPQOEM SLIC-WKS 01072009 AMI  00010013)
[    0.000000] ACPI: FACP 0x00000000CBA304C8 0000F4 (v04 HPQOEM SLIC-WKS 01072009 AMI  00010013)
[    0.000000] ACPI: DSDT 0x00000000CBA28170 008352 (v02 HPQOEM SLIC-WKS 00000102 INTL 20051117)
[    0.000000] ACPI: FACS 0x00000000CBB5BF80 000040
[    0.000000] ACPI: APIC 0x00000000CBA305C0 00007E (v03 HPQOEM SLIC-WKS 01072009 AMI  00010013)
[    0.000000] ACPI: MCFG 0x00000000CBA30640 00003C (v01 HPQOEM OEMMCFG. 01072009 MSFT 00000097)
[    0.000000] ACPI: HPET 0x00000000CBA30680 000038 (v01 HPQOEM SLIC-WKS 01072009 AMI. 00000004)
[    0.000000] ACPI: ASF! 0x00000000CBA306B8 0000A0 (v32 INTEL   HCG     00000001 TFSM 000F4240)
[    0.000000] ACPI: SSDT 0x00000000CBA30758 0058DA (v01 COMPAQ WMI      00000001 MSFT 03000001)
[    0.000000] ACPI: SLIC 0x00000000CBA36038 000176 (v01 HPQOEM SLIC-WKS 00000001      00000000)
[    0.000000] ACPI: SSDT 0x00000000CBA361B0 06E284 (v02 INTEL  CpuPm    00004000 INTL 20051117)
[    0.000000] ACPI: DMAR 0x00000000CBAA4438 0000A0 (v01 A M I  OEMDMAR  00000001 INTL 00000001)
[    0.000000] ACPI: Local APIC address 0xfee00000
[    0.000000] Setting APIC routing to cluster x2apic.
[    0.000000] NUMA turned off
[    0.000000] Faking a node at [mem 0x0000000000000000-0x0000000037ffffff]
[    0.000000] NODE_DATA(0) allocated [mem 0x37f52000-0x37f65fff]
[    0.000000]  [ffffea0000000000-ffffea0000dfffff] PMD -> [ffff880037000000-ffff8800375fffff] on node 0
[    0.000000] Zone ranges:
[    0.000000]   DMA      [mem 0x00001000-0x00ffffff]
[    0.000000]   DMA32    [mem 0x01000000-0x37ffffff]
[    0.000000]   Normal   empty
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x00001000-0x00095fff]
[    0.000000]   node   0: [mem 0x28000000-0x37f65fff]
[    0.000000] Initmem setup node 0 [mem 0x00001000-0x37f65fff]
[    0.000000] On node 0 totalpages: 65531
[    0.000000]   DMA zone: 3 pages used for memmap
[    0.000000]   DMA zone: 21 pages reserved
[    0.000000]   DMA zone: 149 pages, LIFO batch:0
[    0.000000]   DMA32 zone: 1022 pages used for memmap
[    0.000000]   DMA32 zone: 65382 pages, LIFO batch:15
[    0.000000] ACPI: PM-Timer IO Port: 0x408
[    0.000000] ACPI: Local APIC address 0xfee00000
[    0.000000] ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x01] lapic_id[0x02] enabled)
[    0.000000] ACPI: NR_CPUS/possible_cpus limit of 1 reached.  Processor 1/0x2 ignored.
[    0.000000] ACPI: LAPIC (acpi_id[0x02] lapic_id[0x04] enabled)
[    0.000000] ACPI: NR_CPUS/possible_cpus limit of 1 reached.  Processor 2/0x4 ignored.
[    0.000000] ACPI: LAPIC (acpi_id[0x03] lapic_id[0x06] enabled)
[    0.000000] ACPI: NR_CPUS/possible_cpus limit of 1 reached.  Processor 3/0x6 ignored.
[    0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] high edge lint[0x1])
[    0.000000] ACPI: IOAPIC (id[0x00] address[0xfec00000] gsi_base[0])
[    0.000000] IOAPIC[0]: apic_id 0, version 32, address 0xfec00000, GSI 0-23
[    0.000000] ACPI: IOAPIC (id[0x02] address[0xfec01000] gsi_base[24])
[    0.000000] IOAPIC[1]: apic_id 2, version 32, address 0xfec01000, GSI 24-47
[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
[    0.000000] ACPI: IRQ0 used by override.
[    0.000000] ACPI: IRQ9 used by override.
[    0.000000] Using ACPI (MADT) for SMP configuration information
[    0.000000] ACPI: HPET id: 0x8086a701 base: 0xfed00000
[    0.000000] smpboot: 4 Processors exceeds NR_CPUS limit of 1
[    0.000000] smpboot: Allowing 1 CPUs, 0 hotplug CPUs
[    0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff]
[    0.000000] PM: Registered nosave memory: [mem 0x00096000-0x00096fff]
[    0.000000] PM: Registered nosave memory: [mem 0x00097000-0x0009ffff]
[    0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000dffff]
[    0.000000] PM: Registered nosave memory: [mem 0x000e0000-0x000fffff]
[    0.000000] PM: Registered nosave memory: [mem 0x00100000-0x27ffffff]
[    0.000000] PM: Registered nosave memory: [mem 0x37f66000-0x37ffffff]
[    0.000000] e820: [mem 0x38000000-0xcb74ffff] available for PCI devices
[    0.000000] Booting paravirtualized kernel on bare hardware
[    0.000000] setup_percpu: NR_CPUS:8 nr_cpumask_bits:8 nr_cpu_ids:1 nr_node_ids:1
[    0.000000] PERCPU: Embedded 31 pages/cpu @ffff880037c00000 s87168 r8192 d31616 u2097152
[    0.000000] pcpu-alloc: s87168 r8192 d31616 u2097152 alloc=1*2097152
[    0.000000] pcpu-alloc: [0] 0 
[    0.000000] Built 1 zonelists in Node order, mobility grouping on.  Total pages: 64485
[    0.000000] Policy zone: DMA32
[    0.000000] Kernel command line: BOOT_IMAGE=/vmlinuz-3.19.0-rc2+ root=UUID=f170152e-de83-46ee-9546-8ccd53f9753b ro rd.md=0 rd.lvm=0 rd.dm=0 vconsole.keymap=us rd.K
[    0.000000] Intel-IOMMU: enabled
[    0.000000] Misrouted IRQ fixup and polling support enabled
[    0.000000] This may significantly impact system performance
[    0.000000] Disabling memory control group subsystem
[    0.000000] PID hash table entries: 1024 (order: 1, 8192 bytes)
[    0.000000] xsave: enabled xstate_bv 0x7, cntxt size 0x340 using standard form
[    0.000000] Memory: 186236K/262124K available (7472K kernel code, 1128K rwdata, 3196K rodata, 1464K init, 1436K bss, 75888K reserved, 0K cma-reserved)
[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
[    0.000000] Hierarchical RCU implementation.
[    0.000000]  RCU restricting CPUs from NR_CPUS=8 to nr_cpu_ids=1.
[    0.000000] RCU: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1
[    0.000000] NR_IRQS:4352 nr_irqs:256 16
[    0.000000] Spurious LAPIC timer interrupt on cpu 0
[    0.000000] Console: colour VGA+ 80x25
[    0.000000] console [ttyS0] enabled
[    0.000000] console [ttyS0] enabled
[    0.000000] bootconsole [earlyser0] disabled
[    0.000000] bootconsole [earlyser0] disabled
[    0.000000] hpet clockevent registered
[    0.000000] tsc: Fast TSC calibration using PIT
[    0.000000] tsc: Detected 2793.395 MHz processor
[    0.000062] Calibrating delay loop (skipped), value calculated using timer frequency.. 5586.79 BogoMIPS (lpj=2793395)
[    0.010727] pid_max: default: 32768 minimum: 301
[    0.015366] ACPI: Core revision 20141107
[    0.072637] ACPI: All ACPI Tables successfully acquired
[    0.077975] Security Framework initialized
[    0.082099] SELinux:  Initializing.
[    0.085612] SELinux:  Starting in permissive mode
[    0.090415] Dentry cache hash table entries: 32768 (order: 6, 262144 bytes)
[    0.097498] Inode-cache hash table entries: 16384 (order: 5, 131072 bytes)
[    0.104440] Mount-cache hash table entries: 512 (order: 0, 4096 bytes)
[    0.110984] Mountpoint-cache hash table entries: 512 (order: 0, 4096 bytes)
[    0.118250] Initializing cgroup subsys memory
[    0.122628] Initializing cgroup subsys devices
[    0.127089] Initializing cgroup subsys freezer
[    0.131551] Initializing cgroup subsys net_cls
[    0.136009] Initializing cgroup subsys blkio
[    0.140285] Initializing cgroup subsys perf_event
[    0.144995] Initializing cgroup subsys hugetlb
[    0.149495] CPU: Physical Processor ID: 0
[    0.153520] CPU: Processor Core ID: 0
[    0.157211] Last level iTLB entries: 4KB 512, 2MB 8, 4MB 8
[    0.157211] Last level dTLB entries: 4KB 512, 2MB 32, 4MB 32, 1GB 0
[    0.180149] Freeing SMP alternatives memory: 28K (ffffffff81e8a000 - ffffffff81e91000)
[    0.190750] ftrace: allocating 27288 entries in 107 pages
[    0.221595] dmar: Host address width 46
[    0.225443] dmar: DRHD base: 0x000000dfffc000 flags: 0x1
[    0.230791] dmar: IOMMU 0: reg_base_addr dfffc000 ver 1:0 cap d2078c106f0462 ecap f020fe
[    0.238903] dmar: RMRR base: 0x000000cba11000 end: 0x000000cba27fff
[    0.245193] dmar: ATSR flags: 0x0
[    0.248623] IOAPIC id 0 under DRHD base  0xdfffc000 IOMMU 0
[    0.254211] IOAPIC id 2 under DRHD base  0xdfffc000 IOMMU 0
[    0.259804] HPET id 0 under DRHD base 0xdfffc000
[    0.264440] Queued invalidation will be enabled to support x2apic and Intr-remapping.
[    0.272697] Enabled IRQ remapping in x2apic mode
[    0.277920] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
[    0.293984] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-1603 0 @ 2.80GHz (fam: 06, model: 2d, stepping: 07)
[    0.303410] TSC deadline timer enabled
[    0.307209] Performance Events: PEBS fmt1+, 16-deep LBR, SandyBridge events, full-width counters, Broken BIOS detected, complain to your hardware vendor.
[    0.321134] [Firmware Bug]: the BIOS has corrupted hw-PMU resources (MSR 38d is b0)
[    0.328812] Intel PMU driver.
[    0.331801] ... version:                3
[    0.335826] ... bit width:              48
[    0.339936] ... generic registers:      8
[    0.343965] ... value mask:             0000ffffffffffff
[    0.349298] ... max period:             0000ffffffffffff
[    0.354629] ... fixed-purpose events:   3
[    0.358657] ... event mask:             00000007000000ff
[    0.365034] x86: Booted up 1 node, 1 CPUs
[    0.369065] smpboot: Total of 1 processors activated (5586.79 BogoMIPS)
[    0.375727] NMI watchdog: enabled on all CPUs, permanently consumes one hw-PMU counter.
[    0.386634] devtmpfs: initialized
[    0.394369] PM: Registering ACPI NVS region [mem 0xcb750000-0xcb7dafff] (569344 bytes)
[    0.402333] PM: Registering ACPI NVS region [mem 0xcbaad000-0xcbaaefff] (8192 bytes)
[    0.410103] PM: Registering ACPI NVS region [mem 0xcbabb000-0xcbacdfff] (77824 bytes)
[    0.417956] PM: Registering ACPI NVS region [mem 0xcbb56000-0xcbb5dfff] (32768 bytes)
[    0.425811] PM: Registering ACPI NVS region [mem 0xcbb71000-0xcbffffff] (4780032 bytes)
[    0.434188] atomic64_test: passed for x86-64 platform with CX8 and with SSE
[    0.441185] pinctrl core: initialized pinctrl subsystem
[    0.446481] RTC time:  6:11:52, date: 01/06/15
[    0.451115] NET: Registered protocol family 16
[    0.455937] cpuidle: using governor menu
[    0.460114] ACPI: bus type PCI registered
[    0.464147] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5
[    0.470735] PCI: MMCONFIG for domain 0000 [bus 00-ff] at [mem 0xe0000000-0xefffffff] (base 0xe0000000)
[    0.480070] PCI: MMCONFIG at [mem 0xe0000000-0xefffffff] reserved in E820
[    0.487473] PCI: Using configuration type 1 for base access
[    0.495477] ACPI: Added _OSI(Module Device)
[    0.499682] ACPI: Added _OSI(Processor Device)
[    0.504143] ACPI: Added _OSI(3.0 _SCP Extensions)
[    0.508863] ACPI: Added _OSI(Processor Aggregator Device)
[    0.526781] ACPI: Executed 1 blocks of module-level executable AML code
[    0.649090] ACPI: Interpreter enabled
[    0.652776] ACPI Exception: AE_NOT_FOUND, While evaluating Sleep State [\_S1_] (20141107/hwxface-580)
[    0.662072] ACPI Exception: AE_NOT_FOUND, While evaluating Sleep State [\_S2_] (20141107/hwxface-580)
[    0.671386] ACPI: (supports S0 S3 S4 S5)
[    0.675328] ACPI: Using IOAPIC for interrupt routing
[    0.680361] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug
[    0.690950] [Firmware Bug]: ACPI: BIOS _OSI(Linux) query ignored
[    0.712237] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-7f])
[    0.718448] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI]
[    0.726873] acpi PNP0A08:00: _OSC: platform does not support [PCIeCapability]
[    0.734139] acpi PNP0A08:00: _OSC: not requesting control; platform does not support [PCIeCapability]
[    0.743387] acpi PNP0A08:00: _OSC: OS requested [PCIeHotplug PME AER PCIeCapability]
[    0.751155] acpi PNP0A08:00: _OSC: platform willing to grant [PCIeHotplug PME AER]
[    0.758747] acpi PNP0A08:00: _OSC failed (AE_SUPPORT); disabling ASPM
[    0.766045] PCI host bridge to bus 0000:00
[    0.770164] pci_bus 0000:00: root bus resource [bus 00-7f]
[    0.775675] pci_bus 0000:00: root bus resource [io  0x0000-0x03af]
[    0.781875] pci_bus 0000:00: root bus resource [io  0x03e0-0x0cf7]
[    0.788077] pci_bus 0000:00: root bus resource [io  0x03b0-0x03df]
[    0.794277] pci_bus 0000:00: root bus resource [io  0x0d00-0xffff]
[    0.800479] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff]
[    0.807376] pci_bus 0000:00: root bus resource [mem 0x000c0000-0x000dffff]
[    0.814271] pci_bus 0000:00: root bus resource [mem 0xd4000000-0xdfffffff]
[    0.821170] pci_bus 0000:00: root bus resource [mem 0x3c0000000000-0x3c007fffffff]
[    0.828779] pci 0000:00:00.0: [8086:3c00] type 00 class 0x060000
[    0.834900] pci 0000:00:00.0: PME# supported from D0 D3hot D3cold
[    0.841139] pci 0000:00:01.0: [8086:3c02] type 01 class 0x060400
[    0.847265] pci 0000:00:01.0: PME# supported from D0 D3hot D3cold
[    0.853447] pci 0000:00:01.0: System wakeup disabled by ACPI
[    0.859199] pci 0000:00:02.0: [8086:3c04] type 01 class 0x060400
[    0.865321] pci 0000:00:02.0: PME# supported from D0 D3hot D3cold
[    0.871492] pci 0000:00:02.0: System wakeup disabled by ACPI
[    0.877244] pci 0000:00:03.0: [8086:3c08] type 01 class 0x060400
[    0.883370] pci 0000:00:03.0: PME# supported from D0 D3hot D3cold
[    0.889541] pci 0000:00:03.0: System wakeup disabled by ACPI
[    0.895288] pci 0000:00:05.0: [8086:3c28] type 00 class 0x088000
[    0.901487] pci 0000:00:05.2: [8086:3c2a] type 00 class 0x088000
[    0.907671] pci 0000:00:05.4: [8086:3c2c] type 00 class 0x080020
[    0.913717] pci 0000:00:05.4: reg 0x10: [mem 0xd7346000-0xd7346fff]
[    0.920174] pci 0000:00:11.0: [8086:1d3e] type 01 class 0x060400
[    0.926327] pci 0000:00:11.0: PME# supported from D0 D3hot D3cold
[    0.932554] pci 0000:00:16.0: [8086:1d3a] type 00 class 0x078000
[    0.938604] pci 0000:00:16.0: reg 0x10: [mem 0xd7345000-0xd734500f 64bit]
[    0.945506] pci 0000:00:16.0: PME# supported from D0 D3hot D3cold
[    0.951731] pci 0000:00:16.2: [8086:1d3c] type 00 class 0x010185
[    0.957778] pci 0000:00:16.2: reg 0x10: [io  0xf0b0-0xf0b7]
[    0.963382] pci 0000:00:16.2: reg 0x14: [io  0xf0a0-0xf0a3]
[    0.968983] pci 0000:00:16.2: reg 0x18: [io  0xf090-0xf097]
[    0.974588] pci 0000:00:16.2: reg 0x1c: [io  0xf080-0xf083]
[    0.980191] pci 0000:00:16.2: reg 0x20: [io  0xf070-0xf07f]
[    0.985943] pci 0000:00:16.3: [8086:1d3d] type 00 class 0x070002
[    0.991997] pci 0000:00:16.3: reg 0x10: [io  0xf060-0xf067]
[    0.997605] pci 0000:00:16.3: reg 0x14: [mem 0xd7344000-0xd7344fff]
[    1.004085] pci 0000:00:19.0: [8086:1502] type 00 class 0x020000
[    1.010136] pci 0000:00:19.0: reg 0x10: [mem 0xd7300000-0xd731ffff]
[    1.016435] pci 0000:00:19.0: reg 0x14: [mem 0xd7349000-0xd7349fff]
[    1.022731] pci 0000:00:19.0: reg 0x18: [io  0xf040-0xf05f]
[    1.028404] pci 0000:00:19.0: PME# supported from D0 D3hot D3cold
[    1.034570] pci 0000:00:19.0: System wakeup disabled by ACPI
[    1.040316] pci 0000:00:1a.0: [8086:1d2d] type 00 class 0x0c0320
[    1.046371] pci 0000:00:1a.0: reg 0x10: [mem 0xd734b000-0xd734b3ff]
[    1.052772] pci 0000:00:1a.0: PME# supported from D0 D3hot D3cold
[    1.058943] pci 0000:00:1a.0: System wakeup disabled by ACPI
[    1.064683] pci 0000:00:1b.0: [8086:1d20] type 00 class 0x040300
[    1.070728] pci 0000:00:1b.0: reg 0x10: [mem 0xd7340000-0xd7343fff 64bit]
[    1.077630] pci 0000:00:1b.0: PME# supported from D0 D3hot D3cold
[    1.083856] pci 0000:00:1c.0: [8086:1d12] type 01 class 0x060400
[    1.089991] pci 0000:00:1c.0: PME# supported from D0 D3hot D3cold
[    1.096138] pci 0000:00:1c.0: Enabling MPC IRBNCE
[    1.100860] pci 0000:00:1c.0: Intel PCH root port ACS workaround enabled
[    1.107622] pci 0000:00:1c.0: System wakeup disabled by ACPI
[    1.113376] pci 0000:00:1c.5: [8086:1d18] type 01 class 0x060400
[    1.119547] pci 0000:00:1c.5: PME# supported from D0 D3hot D3cold
[    1.125692] pci 0000:00:1c.5: Enabling MPC IRBNCE
[    1.130417] pci 0000:00:1c.5: Intel PCH root port ACS workaround enabled
[    1.137181] pci 0000:00:1c.5: System wakeup disabled by ACPI
[    1.142923] pci 0000:00:1c.6: [8086:1d14] type 01 class 0x060400
[    1.149052] pci 0000:00:1c.6: PME# supported from D0 D3hot D3cold
[    1.155194] pci 0000:00:1c.6: Enabling MPC IRBNCE
[    1.159917] pci 0000:00:1c.6: Intel PCH root port ACS workaround enabled
[    1.166684] pci 0000:00:1c.6: System wakeup disabled by ACPI
[    1.172422] pci 0000:00:1c.7: [8086:1d16] type 01 class 0x060400
[    1.178553] pci 0000:00:1c.7: PME# supported from D0 D3hot D3cold
[    1.184695] pci 0000:00:1c.7: Enabling MPC IRBNCE
[    1.189419] pci 0000:00:1c.7: Intel PCH root port ACS workaround enabled
[    1.196185] pci 0000:00:1c.7: System wakeup disabled by ACPI
[    1.201926] pci 0000:00:1d.0: [8086:1d26] type 00 class 0x0c0320
[    1.207974] pci 0000:00:1d.0: reg 0x10: [mem 0xd734a000-0xd734a3ff]
[    1.214374] pci 0000:00:1d.0: PME# supported from D0 D3hot D3cold
[    1.220544] pci 0000:00:1d.0: System wakeup disabled by ACPI
[    1.226281] pci 0000:00:1e.0: [8086:244e] type 01 class 0x060401
[    1.232419] pci 0000:00:1e.0: System wakeup disabled by ACPI
[    1.238160] pci 0000:00:1f.0: [8086:1d41] type 00 class 0x060100
[    1.244409] pci 0000:00:1f.2: [8086:2826] type 00 class 0x010400
[    1.250461] pci 0000:00:1f.2: reg 0x10: [io  0xf0f0-0xf0f7]
[    1.256058] pci 0000:00:1f.2: reg 0x14: [io  0xf0e0-0xf0e3]
[    1.261661] pci 0000:00:1f.2: reg 0x18: [io  0xf0d0-0xf0d7]
[    1.267263] pci 0000:00:1f.2: reg 0x1c: [io  0xf0c0-0xf0c3]
[    1.272863] pci 0000:00:1f.2: reg 0x20: [io  0xf020-0xf03f]
[    1.278464] pci 0000:00:1f.2: reg 0x24: [mem 0xd7348000-0xd73487ff]
[    1.284815] pci 0000:00:1f.2: PME# supported from D3hot
[    1.290161] pci 0000:00:1f.3: [8086:1d22] type 00 class 0x0c0500
[    1.296210] pci 0000:00:1f.3: reg 0x10: [mem 0xd7347000-0xd73470ff 64bit]
[    1.303042] pci 0000:00:1f.3: reg 0x20: [io  0xf000-0xf01f]
[    1.308845] pci 0000:00:01.0: PCI bridge to [bus 03]
[    1.313940] pci 0000:05:00.0: [10de:10d8] type 00 class 0x030000
[    1.319981] pci 0000:05:00.0: reg 0x10: [mem 0xd6000000-0xd6ffffff]
[    1.326276] pci 0000:05:00.0: reg 0x14: [mem 0xd8000000-0xdbffffff 64bit pref]
[    1.333527] pci 0000:05:00.0: reg 0x1c: [mem 0xdc000000-0xddffffff 64bit pref]
[    1.340777] pci 0000:05:00.0: reg 0x24: [io  0xd000-0xd07f]
[    1.346378] pci 0000:05:00.0: reg 0x30: [mem 0xd7000000-0xd707ffff pref]
[    1.353232] pci 0000:05:00.1: [10de:0be3] type 00 class 0x040300
[    1.359273] pci 0000:05:00.1: reg 0x10: [mem 0xd7080000-0xd7083fff]
[    1.367597] pci 0000:00:02.0: PCI bridge to [bus 05]
[    1.372586] pci 0000:00:02.0:   bridge window [io  0xd000-0xdfff]
[    1.378700] pci 0000:00:02.0:   bridge window [mem 0xd6000000-0xd70fffff]
[    1.385512] pci 0000:00:02.0:   bridge window [mem 0xd8000000-0xddffffff 64bit pref]
[    1.393372] pci 0000:00:03.0: PCI bridge to [bus 04]
[    1.398480] pci 0000:02:00.0: [8086:1d6b] type 00 class 0x010700
[    1.404538] pci 0000:02:00.0: reg 0x10: [mem 0xde87c000-0xde87ffff 64bit pref]
[    1.411797] pci 0000:02:00.0: reg 0x18: [mem 0xde400000-0xde7fffff 64bit pref]
[    1.419053] pci 0000:02:00.0: reg 0x20: [io  0xe000-0xe0ff]
[    1.424782] pci 0000:02:00.0: reg 0x164: [mem 0xde800000-0xde803fff 64bit pref]
[    1.432235] pci 0000:00:11.0: PCI bridge to [bus 02]
[    1.437224] pci 0000:00:11.0:   bridge window [io  0xe000-0xefff]
[    1.443344] pci 0000:00:11.0:   bridge window [mem 0xde400000-0xde8fffff 64bit pref]
[    1.451199] pci 0000:00:1c.0: PCI bridge to [bus 01]
[    1.456306] pci 0000:00:1c.5: PCI bridge to [bus 06]
[    1.461393] pci 0000:00:1c.6: PCI bridge to [bus 07]
[    1.466494] pci 0000:08:00.0: [104c:8241] type 00 class 0x0c0330
[    1.472555] pci 0000:08:00.0: reg 0x10: [mem 0xd7200000-0xd720ffff 64bit]
[    1.479389] pci 0000:08:00.0: reg 0x18: [mem 0xd7210000-0xd7211fff 64bit]
[    1.486330] pci 0000:08:00.0: supports D1 D2
[    1.490615] pci 0000:08:00.0: PME# supported from D0 D1 D2 D3hot D3cold
[    1.499278] pci 0000:00:1c.7: PCI bridge to [bus 08]
[    1.504269] pci 0000:00:1c.7:   bridge window [mem 0xd7200000-0xd72fffff]
[    1.511177] pci 0000:09:04.0: [1317:0985] type 00 class 0x020000
[    1.517226] pci 0000:09:04.0: reg 0x10: [io  0xc000-0xc0ff]
[    1.522826] pci 0000:09:04.0: reg 0x14: [mem 0xd7121000-0xd71213ff]
[    1.529160] pci 0000:09:04.0: reg 0x30: [mem 0xd7100000-0xd711ffff pref]
[    1.535916] pci 0000:09:04.0: supports D1 D2
[    1.540202] pci 0000:09:04.0: PME# supported from D0 D1 D2 D3hot D3cold
[    1.546916] pci 0000:09:05.0: [11c1:5811] type 00 class 0x0c0010
[    1.552964] pci 0000:09:05.0: reg 0x10: [mem 0xd7120000-0xd7120fff]
[    1.559342] pci 0000:09:05.0: supports D1 D2
[    1.563634] pci 0000:09:05.0: PME# supported from D0 D1 D2 D3hot
[    1.569759] pci 0000:00:1e.0: PCI bridge to [bus 09] (subtractive decode)
[    1.576572] pci 0000:00:1e.0:   bridge window [io  0xc000-0xcfff]
[    1.582686] pci 0000:00:1e.0:   bridge window [mem 0xd7100000-0xd71fffff]
[    1.589499] pci 0000:00:1e.0:   bridge window [io  0x0000-0x03af] (subtractive decode)
[    1.597445] pci 0000:00:1e.0:   bridge window [io  0x03e0-0x0cf7] (subtractive decode)
[    1.605386] pci 0000:00:1e.0:   bridge window [io  0x03b0-0x03df] (subtractive decode)
[    1.613328] pci 0000:00:1e.0:   bridge window [io  0x0d00-0xffff] (subtractive decode)
[    1.621269] pci 0000:00:1e.0:   bridge window [mem 0x000a0000-0x000bffff] (subtractive decode)
[    1.629904] pci 0000:00:1e.0:   bridge window [mem 0x000c0000-0x000dffff] (subtractive decode)
[    1.638535] pci 0000:00:1e.0:   bridge window [mem 0xd4000000-0xdfffffff] (subtractive decode)
[    1.647159] pci 0000:00:1e.0:   bridge window [mem 0x3c0000000000-0x3c007fffffff] (subtractive decode)
[    1.657334] ACPI: PCI Root Bridge [PCI1] (domain 0000 [bus 80-ff])
[    1.663550] acpi PNP0A08:01: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI]
[    1.671964] acpi PNP0A08:01: _OSC: platform does not support [PCIeCapability]
[    1.679206] acpi PNP0A08:01: _OSC: not requesting control; platform does not support [PCIeCapability]
[    1.688435] acpi PNP0A08:01: _OSC: OS requested [PCIeHotplug PME AER PCIeCapability]
[    1.696190] acpi PNP0A08:01: _OSC: platform willing to grant [PCIeHotplug PME AER]
[    1.703765] acpi PNP0A08:01: _OSC failed (AE_SUPPORT); disabling ASPM
[    1.710589] PCI host bridge to bus 0000:80
[    1.714693] pci_bus 0000:80: root bus resource [bus 80-ff]
[    1.720184] pci_bus 0000:80: root bus resource [io  0x0000-0x03af]
[    1.726370] pci_bus 0000:80: root bus resource [io  0x03e0-0x0cf7]
[    1.732558] pci_bus 0000:80: root bus resource [mem 0x000c0000-0x000dffff]
[    1.739590] ACPI: PCI Interrupt Link [LNKA] (IRQs 3 4 5 6 7 10 *11 12 14 15), disabled.
[    1.747920] ACPI: PCI Interrupt Link [LNKB] (IRQs 3 4 5 6 7 *10 11 12 14 15), disabled.
[    1.756271] ACPI: PCI Interrupt Link [LNKC] (IRQs 3 4 5 6 10 *11 12 14 15), disabled.
[    1.764428] ACPI: PCI Interrupt Link [LNKD] (IRQs *3 4 5 6 10 11 12 14 15), disabled.
[    1.772590] ACPI: PCI Interrupt Link [LNKE] (IRQs 3 4 *5 6 7 10 11 12 14 15), disabled.
[    1.780940] ACPI: PCI Interrupt Link [LNKF] (IRQs 3 4 5 6 7 10 11 12 14 15) *0, disabled.
[    1.789484] ACPI: PCI Interrupt Link [LNKG] (IRQs 3 4 *5 6 7 10 11 12 14 15), disabled.
[    1.797828] ACPI: PCI Interrupt Link [LNKH] (IRQs 3 4 5 6 7 *10 11 12 14 15), disabled.
[    1.806426] ACPI: NR_CPUS/possible_cpus limit of 1 reached.  Processor 4/0x2 ignored.
[    1.814284] ACPI: Unable to map lapic to logical cpu number
[    1.820027] ACPI: NR_CPUS/possible_cpus limit of 1 reached.  Processor 5/0x4 ignored.
[    1.827878] ACPI: Unable to map lapic to logical cpu number
[    1.833616] ACPI: NR_CPUS/possible_cpus limit of 1 reached.  Processor 6/0x6 ignored.
[    1.841472] ACPI: Unable to map lapic to logical cpu number
[    1.847715] ACPI: Enabled 2 GPEs in block 00 to 3F
[    1.852759] vgaarb: setting as boot device: PCI:0000:05:00.0
[    1.858441] vgaarb: device added: PCI:0000:05:00.0,decodes=io+mem,owns=io+mem,locks=none
[    1.866561] vgaarb: loaded
[    1.869281] vgaarb: bridge control possible 0000:05:00.0
[    1.874738] SCSI subsystem initialized
[    1.878566] libata version 3.00 loaded.
[    1.882473] ACPI: bus type USB registered
[    1.886518] usbcore: registered new interface driver usbfs
[    1.892024] usbcore: registered new interface driver hub
[    1.897361] usbcore: registered new device driver usb
[    1.902574] PCI: Using ACPI for IRQ routing
[    1.914574] PCI: Discovered peer bus ff
[    1.918422] PCI: root bus ff: using default resources
[    1.923479] PCI: Probing PCI hardware (bus ff)
[    1.927987] PCI host bridge to bus 0000:ff
[    1.932104] pci_bus 0000:ff: root bus resource [io  0x0000-0xffff]
[    1.938304] pci_bus 0000:ff: root bus resource [mem 0x00000000-0x3fffffffffff]
[    1.945537] pci_bus 0000:ff: No busn resource found for root bus, will use [bus ff-ff]
[    1.953461] pci_bus 0000:ff: busn_res: can not insert [bus ff] under domain [bus 00-ff] (conflicts with (null) [bus 80-ff])
[    1.964603] pci 0000:ff:08.0: [8086:3c80] type 00 class 0x088000
[    1.970705] pci 0000:ff:08.3: [8086:3c83] type 00 class 0x088000
[    1.976829] pci 0000:ff:08.4: [8086:3c84] type 00 class 0x088000
[    1.982965] pci 0000:ff:09.0: [8086:3c90] type 00 class 0x088000
[    1.989075] pci 0000:ff:09.3: [8086:3c93] type 00 class 0x088000
[    1.995209] pci 0000:ff:09.4: [8086:3c94] type 00 class 0x088000
[    2.001344] pci 0000:ff:0a.0: [8086:3cc0] type 00 class 0x088000
[    2.007448] pci 0000:ff:0a.1: [8086:3cc1] type 00 class 0x088000
[    2.013551] pci 0000:ff:0a.2: [8086:3cc2] type 00 class 0x088000
[    2.019658] pci 0000:ff:0a.3: [8086:3cd0] type 00 class 0x088000
[    2.025767] pci 0000:ff:0b.0: [8086:3ce0] type 00 class 0x088000
[    2.031872] pci 0000:ff:0b.3: [8086:3ce3] type 00 class 0x088000
[    2.037981] pci 0000:ff:0c.0: [8086:3ce8] type 00 class 0x088000
[    2.044083] pci 0000:ff:0c.1: [8086:3ce8] type 00 class 0x088000
[    2.050172] pci 0000:ff:0c.6: [8086:3cf4] type 00 class 0x088000
[    2.056260] pci 0000:ff:0c.7: [8086:3cf6] type 00 class 0x088000
[    2.062358] pci 0000:ff:0d.0: [8086:3ce8] type 00 class 0x088000
[    2.068462] pci 0000:ff:0d.1: [8086:3ce8] type 00 class 0x088000
[    2.074564] pci 0000:ff:0d.6: [8086:3cf5] type 00 class 0x088000
[    2.080665] pci 0000:ff:0e.0: [8086:3ca0] type 00 class 0x088000
[    2.086768] pci 0000:ff:0e.1: [8086:3c46] type 00 class 0x110100
[    2.092886] pci 0000:ff:0f.0: [8086:3ca8] type 00 class 0x088000
[    2.099006] pci 0000:ff:0f.1: [8086:3c71] type 00 class 0x088000
[    2.105129] pci 0000:ff:0f.2: [8086:3caa] type 00 class 0x088000
[    2.111254] pci 0000:ff:0f.3: [8086:3cab] type 00 class 0x088000
[    2.117380] pci 0000:ff:0f.4: [8086:3cac] type 00 class 0x088000
[    2.123500] pci 0000:ff:0f.5: [8086:3cad] type 00 class 0x088000
[    2.129626] pci 0000:ff:0f.6: [8086:3cae] type 00 class 0x088000
[    2.135737] pci 0000:ff:10.0: [8086:3cb0] type 00 class 0x088000
[    2.141870] pci 0000:ff:10.1: [8086:3cb1] type 00 class 0x088000
[    2.147999] pci 0000:ff:10.2: [8086:3cb2] type 00 class 0x088000
[    2.154122] pci 0000:ff:10.3: [8086:3cb3] type 00 class 0x088000
[    2.160249] pci 0000:ff:10.4: [8086:3cb4] type 00 class 0x088000
[    2.166375] pci 0000:ff:10.5: [8086:3cb5] type 00 class 0x088000
[    2.172499] pci 0000:ff:10.6: [8086:3cb6] type 00 class 0x088000
[    2.178619] pci 0000:ff:10.7: [8086:3cb7] type 00 class 0x088000
[    2.184744] pci 0000:ff:11.0: [8086:3cb8] type 00 class 0x088000
[    2.190864] pci 0000:ff:13.0: [8086:3ce4] type 00 class 0x088000
[    2.196972] pci 0000:ff:13.1: [8086:3c43] type 00 class 0x110100
[    2.203081] pci 0000:ff:13.4: [8086:3ce6] type 00 class 0x110100
[    2.209184] pci 0000:ff:13.5: [8086:3c44] type 00 class 0x110100
[    2.215295] pci 0000:ff:13.6: [8086:3c45] type 00 class 0x088000
[    2.221411] pci_bus 0000:ff: busn_res: [bus ff] end is updated to ff
[    2.227789] pci_bus 0000:ff: busn_res: can not insert [bus ff] under domain [bus 00-ff] (conflicts with (null) [bus 80-ff])
[    2.238954] PCI: pci_cache_line_size set to 64 bytes
[    2.244095] e820: reserve RAM buffer [mem 0x00096400-0x0009ffff]
[    2.250127] e820: reserve RAM buffer [mem 0x37f66000-0x37ffffff]
[    2.256334] NetLabel: Initializing
[    2.259758] NetLabel:  domain hash size = 128
[    2.264128] NetLabel:  protocols = UNLABELED CIPSOv4
[    2.269138] NetLabel:  unlabeled traffic allowed by default
[    2.274959] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0, 0, 0, 0, 0, 0
[    2.281330] hpet0: 8 comparators, 64-bit 14.318180 MHz counter
[    2.289218] Switched to clocksource hpet
[    2.302446] pnp: PnP ACPI init
[    2.305719] system 00:00: [mem 0xfc000000-0xfcffffff window] has been reserved
[    2.312986] system 00:00: [mem 0xfd000000-0xfdffffff window] has been reserved
[    2.320245] system 00:00: [mem 0xfe000000-0xfeafffff window] has been reserved
[    2.327500] system 00:00: [mem 0xfeb00000-0xfebfffff window] has been reserved
[    2.334752] system 00:00: [mem 0xfed00400-0xfed3ffff window] could not be reserved
[    2.342345] system 00:00: [mem 0xfed45000-0xfedfffff window] has been reserved
[    2.349591] system 00:00: [mem 0xdffff000-0xdfffffff window] has been reserved
[    2.356839] system 00:00: Plug and Play ACPI device, IDs PNP0c01 (active)
[    2.363885] system 00:01: [io  0x0620-0x063f] has been reserved
[    2.369840] system 00:01: [io  0x0610-0x061f] has been reserved
[    2.375783] system 00:01: Plug and Play ACPI device, IDs PNP0c02 (active)
[    2.382667] pnp 00:02: Plug and Play ACPI device, IDs PNP0303 PNP030b (active)
[    2.389992] pnp 00:03: Plug and Play ACPI device, IDs PNP0f03 PNP0f13 (active)
[    2.397276] pnp 00:04: Plug and Play ACPI device, IDs PNP0b00 (active)
[    2.403920] system 00:05: [io  0x04d0-0x04d1] has been reserved
[    2.409851] system 00:05: Plug and Play ACPI device, IDs PNP0c02 (active)
[    2.416906] pnp 00:06: [dma 0 disabled]
[    2.420786] pnp 00:06: Plug and Play ACPI device, IDs PNP0501 (active)
[    2.427663] system 00:07: [io  0x0400-0x0453] could not be reserved
[    2.433947] system 00:07: [io  0x0458-0x047f] has been reserved
[    2.439900] system 00:07: [io  0x1180-0x119f] has been reserved
[    2.445847] system 00:07: [io  0x0500-0x057f] has been reserved
[    2.451790] system 00:07: [mem 0xfed1c000-0xfed1ffff] has been reserved
[    2.458434] system 00:07: [mem 0xfec00000-0xfecfffff] could not be reserved
[    2.465418] system 00:07: [mem 0xfed08000-0xfed08fff] has been reserved
[    2.472056] system 00:07: [mem 0xff000000-0xffffffff] has been reserved
[    2.478697] system 00:07: Plug and Play ACPI device, IDs PNP0c01 (active)
[    2.485629] system 00:08: [io  0x0454-0x0457] has been reserved
[    2.491577] system 00:08: Plug and Play ACPI device, IDs INT3f0d PNP0c02 (active)
[    2.499647] system 00:09: [mem 0xfed40000-0xfed44fff] has been reserved
[    2.506269] system 00:09: Plug and Play ACPI device, IDs PNP0c02 (active)
[    2.513051] pnp: PnP ACPI: found 10 devices
[    2.524317] pci 0000:00:01.0: PCI bridge to [bus 03]
[    2.529309] pci 0000:00:02.0: PCI bridge to [bus 05]
[    2.534268] pci 0000:00:02.0:   bridge window [io  0xd000-0xdfff]
[    2.540368] pci 0000:00:02.0:   bridge window [mem 0xd6000000-0xd70fffff]
[    2.547170] pci 0000:00:02.0:   bridge window [mem 0xd8000000-0xddffffff 64bit pref]
[    2.554928] pci 0000:00:03.0: PCI bridge to [bus 04]
[    2.559916] pci 0000:00:11.0: PCI bridge to [bus 02]
[    2.564886] pci 0000:00:11.0:   bridge window [io  0xe000-0xefff]
[    2.570992] pci 0000:00:11.0:   bridge window [mem 0xde400000-0xde8fffff 64bit pref]
[    2.578746] pci 0000:00:1c.0: PCI bridge to [bus 01]
[    2.583731] pci 0000:00:1c.5: PCI bridge to [bus 06]
[    2.588723] pci 0000:00:1c.6: PCI bridge to [bus 07]
[    2.593707] pci 0000:00:1c.7: PCI bridge to [bus 08]
[    2.598681] pci 0000:00:1c.7:   bridge window [mem 0xd7200000-0xd72fffff]
[    2.605489] pci 0000:00:1e.0: PCI bridge to [bus 09]
[    2.610476] pci 0000:00:1e.0:   bridge window [io  0xc000-0xcfff]
[    2.616594] pci 0000:00:1e.0:   bridge window [mem 0xd7100000-0xd71fffff]
[    2.623420] pci_bus 0000:00: resource 4 [io  0x0000-0x03af]
[    2.629020] pci_bus 0000:00: resource 5 [io  0x03e0-0x0cf7]
[    2.634612] pci_bus 0000:00: resource 6 [io  0x03b0-0x03df]
[    2.640206] pci_bus 0000:00: resource 7 [io  0x0d00-0xffff]
[    2.645793] pci_bus 0000:00: resource 8 [mem 0x000a0000-0x000bffff]
[    2.652065] pci_bus 0000:00: resource 9 [mem 0x000c0000-0x000dffff]
[    2.658337] pci_bus 0000:00: resource 10 [mem 0xd4000000-0xdfffffff]
[    2.664697] pci_bus 0000:00: resource 11 [mem 0x3c0000000000-0x3c007fffffff]
[    2.671752] pci_bus 0000:05: resource 0 [io  0xd000-0xdfff]
[    2.677338] pci_bus 0000:05: resource 1 [mem 0xd6000000-0xd70fffff]
[    2.683607] pci_bus 0000:05: resource 2 [mem 0xd8000000-0xddffffff 64bit pref]
[    2.690849] pci_bus 0000:02: resource 0 [io  0xe000-0xefff]
[    2.696441] pci_bus 0000:02: resource 2 [mem 0xde400000-0xde8fffff 64bit pref]
[    2.703691] pci_bus 0000:08: resource 1 [mem 0xd7200000-0xd72fffff]
[    2.709979] pci_bus 0000:09: resource 0 [io  0xc000-0xcfff]
[    2.715573] pci_bus 0000:09: resource 1 [mem 0xd7100000-0xd71fffff]
[    2.721853] pci_bus 0000:09: resource 4 [io  0x0000-0x03af]
[    2.727439] pci_bus 0000:09: resource 5 [io  0x03e0-0x0cf7]
[    2.733031] pci_bus 0000:09: resource 6 [io  0x03b0-0x03df]
[    2.738625] pci_bus 0000:09: resource 7 [io  0x0d00-0xffff]
[    2.744220] pci_bus 0000:09: resource 8 [mem 0x000a0000-0x000bffff]
[    2.750510] pci_bus 0000:09: resource 9 [mem 0x000c0000-0x000dffff]
[    2.756800] pci_bus 0000:09: resource 10 [mem 0xd4000000-0xdfffffff]
[    2.763178] pci_bus 0000:09: resource 11 [mem 0x3c0000000000-0x3c007fffffff]
[    2.770249] pci_bus 0000:80: resource 4 [io  0x0000-0x03af]
[    2.775851] pci_bus 0000:80: resource 5 [io  0x03e0-0x0cf7]
[    2.781454] pci_bus 0000:80: resource 6 [mem 0x000c0000-0x000dffff]
[    2.787752] pci_bus 0000:ff: resource 4 [io  0x0000-0xffff]
[    2.793347] pci_bus 0000:ff: resource 5 [mem 0x00000000-0x3fffffffffff]
[    2.800034] NET: Registered protocol family 2
[    2.804672] TCP established hash table entries: 2048 (order: 2, 16384 bytes)
[    2.811770] TCP bind hash table entries: 2048 (order: 3, 32768 bytes)
[    2.818230] TCP: Hash tables configured (established 2048 bind 2048)
[    2.824610] TCP: reno registered
[    2.827857] UDP hash table entries: 256 (order: 1, 8192 bytes)
[    2.833708] UDP-Lite hash table entries: 256 (order: 1, 8192 bytes)
[    2.840046] NET: Registered protocol family 1
[    2.845566] pci 0000:05:00.0: Video device with shadowed ROM
[    2.851666] PCI: CLS 64 bytes, default 64
[    2.855768] Unpacking initramfs...
[    4.447503] Freeing initrd memory: 52844K (ffff880032c65000 - ffff880036000000)
[    4.454872] IOMMU Skip disabling iommu hardware translations
[    4.460761] IOMMU Copying translate tables from panicked kernel
[    4.468304] IOMMU: root_cache:0xffff880031960000 phys:0x0003eeae1000
[    4.474686] IOMMU:0 Domain ids from panicked kernel:
[    4.479673] DID did:4(0x0004)
[    4.482657] DID did:10(0x000a)
[    4.485720] DID did:8(0x0008)
[    4.488692] DID did:6(0x0006)
[    4.491671] DID did:3(0x0003)
[    4.494651] DID did:2(0x0002)
[    4.497629] DID did:7(0x0007)
[    4.500605] DID did:1(0x0001)
[    4.503583] DID did:9(0x0009)
[    4.506558] DID did:0(0x0000)
[    4.509536] DID did:11(0x000b)
[    4.512599] DID did:5(0x0005)
[    4.515576] ----------------------------------------
[    4.520546] IOMMU: dmar0 using Queued invalidation
[    4.525355] PCI-DMA: Intel(R) Virtualization Technology for Directed I/O
[    4.534191] RAPL PMU detected, hw unit 2^-16 Joules, API unit is 2^-32 Joules, 3 fixed counters 163840 ms ovfl timer
[    4.545218] AVX version of gcm_enc/dec engaged.
[    4.549767] AES CTR mode by8 optimization enabled
[    4.557005] alg: No test for __gcm-aes-aesni (__driver-gcm-aes-aesni)
[    4.563932] futex hash table entries: 256 (order: 2, 16384 bytes)
[    4.570069] Initialise system trusted keyring
[    4.574510] audit: initializing netlink subsys (disabled)
[    4.579954] audit: type=2000 audit(1420524713.791:1): initialized
[    4.587459] HugeTLB registered 2 MB page size, pre-allocated 0 pages
[    4.596339] zpool: loaded
[    4.598985] zbud: loaded
[    4.601812] VFS: Disk quotas dquot_6.5.2
[    4.605822] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[    4.613374] Key type big_key registered
[    4.617230] SELinux:  Registering netfilter hooks
[    4.622975] alg: No test for stdrng (krng)
[    4.627095] NET: Registered protocol family 38
[    4.631581] Key type asymmetric registered
[    4.635704] Asymmetric key parser 'x509' registered
[    4.640662] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 252)
[    4.648115] io scheduler noop registered
[    4.652063] io scheduler deadline registered
[    4.656388] io scheduler cfq registered (default)
[    4.662588] pci_hotplug: PCI Hot Plug PCI Core version: 0.5
[    4.668203] pciehp: PCI Express Hot Plug Controller Driver version: 0.4
[    4.674879] intel_idle: MWAIT substates: 0x21120
[    4.679506] intel_idle: v0.4 model 0x2D
[    4.683350] intel_idle: lapic_timer_reliable_states 0xffffffff
[    4.689379] input: Power Button as /devices/LNXSYSTM:00/LNXSYBUS:00/PNP0C0C:00/input/input0
[    4.697749] ACPI: Power Button [PWRB]
[    4.701491] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input1
[    4.708894] ACPI: Power Button [PWRF]
[    4.714402] GHES: HEST is not enabled!
[    4.718275] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
[    4.745225] serial 00:06: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A
[    4.774137] serial 0000:00:16.3: ttyS1 at I/O 0xf060 (irq = 17, base_baud = 115200) is a 16550A
[    4.783298] Non-volatile memory driver v1.3
[    4.787509] Linux agpgart interface v0.103
[    4.791901] ahci 0000:00:1f.2: version 3.0
[    4.807019] ahci 0000:00:1f.2: AHCI 0001.0300 32 slots 6 ports 6 Gbps 0x5 impl RAID mode
[    4.815128] ahci 0000:00:1f.2: flags: 64bit ncq sntf led clo pio ems sxs apst 
[    4.822491] dmar: DRHD: handling fault status reg 2
[    4.827365] dmar: DMAR:[DMA Write] Request device [00:1f.2] fault addr fffe0000 
[    4.827365] DMAR:[fault reason 10] non-zero reserved fields in RTP
[    4.840966] dmar: DRHD: handling fault status reg 102
[    4.846011] dmar: DMAR:[DMA Write] Request device [00:1f.2] fault addr fffc0000 
[    4.846011] DMAR:[fault reason 10] non-zero reserved fields in RTP
[    4.859567] dmar: DMAR:[DMA Write] Request device [00:1e.0] fault addr fff38000 
[    4.859567] DMAR:[fault reason 10] non-zero reserved fields in RTP
[    4.873136] dmar: DRHD: handling fault status reg 300
[    4.879013] scsi host0: ahci
[    4.882095] scsi host1: ahci
[    4.885105] scsi host2: ahci
[    4.888113] scsi host3: ahci
[    4.891111] scsi host4: ahci
[    4.894115] scsi host5: ahci
[    4.897073] ata1: SATA max UDMA/133 abar m2048 at 0xd7348000 port 0xd7348100 irq 27
[    4.904480] ata2: DUMMY
[    4.906934] ata3: SATA max UDMA/133 abar m2048 at 0xd7348000 port 0xd7348200 irq 27
[    4.914327] ata4: DUMMY
[    4.916784] ata5: DUMMY
[    4.919239] ata6: DUMMY
[    4.922035] libphy: Fixed MDIO Bus: probed
[    4.926316] xhci_hcd 0000:08:00.0: xHCI Host Controller
[    4.931617] xhci_hcd 0000:08:00.0: new USB bus registered, assigned bus number 1
[    4.939457] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002
[    4.946256] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    4.953493] usb usb1: Product: xHCI Host Controller
[    4.958387] usb usb1: Manufacturer: Linux 3.19.0-rc2+ xhci-hcd
[    4.964232] usb usb1: SerialNumber: 0000:08:00.0
[    4.969025] hub 1-0:1.0: USB hub found
[    4.972805] hub 1-0:1.0: 4 ports detected
[    4.977029] xhci_hcd 0000:08:00.0: xHCI Host Controller
[    4.982320] xhci_hcd 0000:08:00.0: new USB bus registered, assigned bus number 2
[    4.989730] dmar: DRHD: handling fault status reg 302
[    4.994791] dmar: DMAR:[DMA Read] Request device [08:00.0] fault addr ffffc000 
[    4.994791] DMAR:[fault reason 10] non-zero reserved fields in RTP
[    5.008314] usb usb2: New USB device found, idVendor=1d6b, idProduct=0003
[    5.015116] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    5.022345] usb usb2: Product: xHCI Host Controller
[    5.027240] usb usb2: Manufacturer: Linux 3.19.0-rc2+ xhci-hcd
[    5.033080] usb usb2: SerialNumber: 0000:08:00.0
[    5.037852] hub 2-0:1.0: USB hub found
[    5.041632] hub 2-0:1.0: 4 ports detected
[    5.045862] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[    5.052422] ehci-pci: EHCI PCI platform driver
[    5.057035] ehci-pci 0000:00:1a.0: EHCI Host Controller
[    5.062338] ehci-pci 0000:00:1a.0: new USB bus registered, assigned bus number 3
[    5.069755] ehci-pci 0000:00:1a.0: debug port 2
[    5.078282] ehci-pci 0000:00:1a.0: cache line size of 64 is not supported
[    5.085109] ehci-pci 0000:00:1a.0: irq 16, io mem 0xd734b000
[    5.096342] ehci-pci 0000:00:1a.0: USB 2.0 started, EHCI 1.00
[    5.102099] do_IRQ: 0.35 No irq handler for vector (irq -1)
[    5.107750] usb usb3: New USB device found, idVendor=1d6b, idProduct=0002
[    5.114549] usb usb3: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    5.121783] usb usb3: Product: EHCI Host Controller
[    5.126669] usb usb3: Manufacturer: Linux 3.19.0-rc2+ ehci_hcd
[    5.132511] usb usb3: SerialNumber: 0000:00:1a.0
[    5.137332] hub 3-0:1.0: USB hub found
[    5.141108] hub 3-0:1.0: 3 ports detected
[    5.145588] ehci-pci 0000:00:1d.0: EHCI Host Controller
[    5.150912] ehci-pci 0000:00:1d.0: new USB bus registered, assigned bus number 4
[    5.158328] ehci-pci 0000:00:1d.0: debug port 2
[    5.166876] ehci-pci 0000:00:1d.0: cache line size of 64 is not supported
[    5.173700] ehci-pci 0000:00:1d.0: irq 23, io mem 0xd734a000
[    5.184439] ehci-pci 0000:00:1d.0: USB 2.0 started, EHCI 1.00
[    5.190210] dmar: DRHD: handling fault status reg 402
[    5.195273] dmar: INTR-REMAP: Request device [[00:1f.7] fault index 20
[    5.195273] INTR-REMAP:[fault reason 38] Blocked an interrupt request due to source-id verification failure
[    5.211603] usb usb4: New USB device found, idVendor=1d6b, idProduct=0002
[    5.218402] usb usb4: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    5.225636] usb usb4: Product: EHCI Host Controller
[    5.230521] usb usb4: Manufacturer: Linux 3.19.0-rc2+ ehci_hcd
[    5.236362] usb usb4: SerialNumber: 0000:00:1d.0
[    5.240996] dmar: DRHD: handling fault status reg 502
[    5.246052] dmar: DMAR:[DMA Write] Request device [00:1f.2] fault addr fffe0000 
[    5.246052] DMAR:[fault reason 10] non-zero reserved fields in RTP
[    5.259633] ata1: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
[    5.265834] dmar: DRHD: handling fault status reg 602
[    5.270878] dmar: DMAR:[DMA Read] Request device [00:1f.2] fault addr fffe0000 
[    5.270878] DMAR:[fault reason 10] non-zero reserved fields in RTP
[    5.284582] hub 4-0:1.0: USB hub found
[    5.288359] dmar: DRHD: handling fault status reg 702
[    5.293417] dmar: DMAR:[DMA Write] Request device [00:1f.2] fault addr fffc0000 
[    5.293417] DMAR:[fault reason 10] non-zero reserved fields in RTP
[    5.306996] ata3: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
[    5.313193] dmar: DRHD: handling fault status reg 2
[    5.318075] dmar: DMAR:[DMA Read] Request device [00:1f.2] fault addr fffc0000 
[    5.318075] DMAR:[fault reason 10] non-zero reserved fields in RTP
[    5.331577] hub 4-0:1.0: 3 ports detected
[    5.335807] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
[    5.342010] ohci-pci: OHCI PCI platform driver
[    5.346487] uhci_hcd: USB Universal Host Controller Interface driver
[    5.352936] usbcore: registered new interface driver usbserial
[    5.358789] usbcore: registered new interface driver usbserial_generic
[    5.365340] usbserial: USB Serial support registered for generic
[    5.371407] i8042: PNP: PS/2 Controller [PNP0303:PS2K,PNP0f03:PS2M] at 0x60,0x64 irq 1,12
[    5.381157] ata1.00: failed to IDENTIFY (I/O error, err_mask=0x100)
[    5.388819] serio: i8042 KBD port at 0x60,0x64 irq 1
[    5.393812] serio: i8042 AUX port at 0x60,0x64 irq 12
[    5.398996] mousedev: PS/2 mouse device common for all mice
[    5.404895] rtc_cmos 00:04: RTC can wake from S4
[    5.409704] rtc_cmos 00:04: rtc core: registered rtc_cmos as rtc0
[    5.415870] rtc_cmos 00:04: alarms up to one month, y3k, 114 bytes nvram, hpet irqs
[    5.423648] device-mapper: uevent: version 1.0.3
[    5.428783] device-mapper: ioctl: 4.29.0-ioctl (2014-10-28) initialised: dm-devel at redhat.com
[    5.437325] Intel P-state driver initializing.
[    5.443561] hidraw: raw HID events driver (C) Jiri Kosina
[    5.449184] usbcore: registered new interface driver usbhid
[    5.454784] usbhid: USB HID core driver
[    5.458944] drop_monitor: Initializing network drop monitor service
[    5.465401] ip_tables: (C) 2000-2006 Netfilter Core Team
[    5.470755] TCP: cubic registered
[    5.474075] Initializing XFRM netlink socket
[    5.478847] NET: Registered protocol family 10
[    5.483542] mip6: Mobile IPv6
[    5.486536] NET: Registered protocol family 17
[    5.491000] mce: Unable to init device /dev/mcelog (rc: -5)
[    5.496744] usb 3-1: new high-speed USB device number 2 using ehci-pci
[    5.503460] Loading compiled-in X.509 certificates
[    5.508276] dmar: DRHD: handling fault status reg 102
[    5.513331] dmar: DMAR:[DMA Read] Request device [00:1a.0] fault addr fffff000 
[    5.513331] DMAR:[fault reason 10] non-zero reserved fields in RTP
[    5.528003] Loaded X.509 cert 'Magrathea: Glacier signing key: d196e1366cc7c91295775c1e77c548314c3ad291'
[    5.537492] tsc: Refined TSC clocksource calibration: 2793.268 MHz
[    5.543697] registered taskstats version 1
[    5.548416]   Magic number: 15:667:164
[    5.552271] rtc_cmos 00:04: setting system clock to 2015-01-06 06:11:57 UTC (1420524717)
[    5.560455] PM: Hibernation image not present or could not be loaded.
[    5.566997] ehci-pci 0000:00:1a.0: fatal error
[    5.571447] ehci-pci 0000:00:1a.0: HC died; cleaning up
[    5.649981] usb 4-1: new high-speed USB device number 2 using ehci-pci
[    5.656537] dmar: DRHD: handling fault status reg 202
[    5.661590] dmar: DMAR:[DMA Read] Request device [00:1d.0] fault addr fffff000 
[    5.661590] DMAR:[fault reason 10] non-zero reserved fields in RTP
[    5.853075] dmar: DRHD: handling fault status reg 302
[    5.858152] dmar: DMAR:[DMA Write] Request device [00:1e.0] fault addr fffff000 
[    5.858152] DMAR:[fault reason 10] non-zero reserved fields in RTP
[    6.172770] dmar: DRHD: handling fault status reg 402
[    6.177833] dmar: DMAR:[DMA Write] Request device [00:1e.0] fault addr fffff000 
[    6.177833] DMAR:[fault reason 10] non-zero reserved fields in RTP
[    6.548988] Switched to clocksource tsc
[   10.271066] ata3.00: failed to IDENTIFY (I/O error, err_mask=0x100)
[   10.621432] dmar: DRHD: handling fault status reg 502
[   10.626494] dmar: DMAR:[DMA Write] Request device [00:1f.2] fault addr fffe0000 
[   10.626494] DMAR:[fault reason 10] non-zero reserved fields in RTP
[   10.640039] ata1: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
[   10.646238] dmar: DRHD: handling fault status reg 602
[   10.651293] dmar: DMAR:[DMA Read] Request device [00:1f.2] fault addr fffe0000 
[   10.651293] DMAR:[fault reason 10] non-zero reserved fields in RTP
[   10.664768] dmar: DRHD: handling fault status reg 702
[   10.669809] dmar: DMAR:[DMA Write] Request device [00:1f.2] fault addr fffc0000 
[   10.669809] DMAR:[fault reason 10] non-zero reserved fields in RTP
[   10.683382] ata3: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
[   10.689577] dmar: DRHD: handling fault status reg 2
[   10.694458] dmar: DMAR:[DMA Read] Request device [00:1f.2] fault addr fffc0000 
[   10.694458] DMAR:[fault reason 10] non-zero reserved fields in RTP
[   15.713048] ata3.00: qc timeout (cmd 0xa1)
[   15.717176] ata3.00: failed to IDENTIFY (I/O error, err_mask=0x4)
[   15.723299] ata3: limiting SATA link speed to 1.5 Gbps
[   15.728474] ata1.00: qc timeout (cmd 0xec)
[   15.732595] ata1.00: failed to IDENTIFY (I/O error, err_mask=0x4)
[   15.738722] ata1: limiting SATA link speed to 3.0 Gbps
[   16.046415] dmar: DRHD: handling fault status reg 102
[   16.051488] dmar: DMAR:[DMA Write] Request device [00:1f.2] fault addr fffc0000 
[   16.051488] DMAR:[fault reason 10] non-zero reserved fields in RTP
[   16.065103] ata3: SATA link up 1.5 Gbps (SStatus 113 SControl 310)
[   16.071323] dmar: DRHD: handling fault status reg 202
[   16.076394] dmar: DMAR:[DMA Read] Request device [00:1f.2] fault addr fffc0000 
[   16.076394] DMAR:[fault reason 10] non-zero reserved fields in RTP
[   16.089913] dmar: DRHD: handling fault status reg 302
[   16.094983] dmar: DMAR:[DMA Write] Request device [00:1f.2] fault addr fffe0000 
[   16.094983] DMAR:[fault reason 10] non-zero reserved fields in RTP
[   16.108590] ata1: SATA link up 3.0 Gbps (SStatus 123 SControl 320)
[   16.114794] dmar: DRHD: handling fault status reg 402
[   16.119857] dmar: DMAR:[DMA Read] Request device [00:1f.2] fault addr fffe0000 
[   16.119857] DMAR:[fault reason 10] non-zero reserved fields in RTP
[   20.834724] ehci-pci 0000:00:1d.0: port 1 reset error -110
[   21.697680] ehci-pci 0000:00:1d.0: port 1 reset error -110
[   22.509576] ehci-pci 0000:00:1d.0: port 1 reset error -110
[   23.321463] ehci-pci 0000:00:1d.0: port 1 reset error -110
[   24.133359] ehci-pci 0000:00:1d.0: port 1 reset error -110
[   24.742006] usb usb4-port1: Cannot enable. Maybe the USB cable is bad?
[   24.801096] ehci-pci 0000:00:1d.0: port 1 reset error -110
[   25.664047] ehci-pci 0000:00:1d.0: port 1 reset error -110
[   26.143538] ata3.00: qc timeout (cmd 0xa1)
[   26.147661] ata3.00: failed to IDENTIFY (I/O error, err_mask=0x4)
[   26.153799] ata1.00: qc timeout (cmd 0xec)
[   26.157930] ata1.00: failed to IDENTIFY (I/O error, err_mask=0x4)
[   26.466898] dmar: DRHD: handling fault status reg 502
[   26.471972] dmar: DMAR:[DMA Write] Request device [00:1f.2] fault addr fffc0000 
[   26.471972] DMAR:[fault reason 10] non-zero reserved fields in RTP
[   26.485588] ata3: SATA link up 1.5 Gbps (SStatus 113 SControl 310)
[   26.493772] ehci-pci 0000:00:1d.0: port 1 reset error -110
[   26.499297] dmar: DRHD: handling fault status reg 602
[   26.504366] dmar: DMAR:[DMA Write] Request device [00:1f.2] fault addr fffe0000 
[   26.504366] DMAR:[fault reason 10] non-zero reserved fields in RTP
[   26.517974] ata1: SATA link up 3.0 Gbps (SStatus 123 SControl 320)
[   26.524591] Freeing unused kernel memory: 1464K (ffffffff81d1c000 - ffffffff81e8a000)
[   26.532455] Write protecting the kernel read-only data: 12288k
[   26.538662] Freeing unused kernel memory: 708K (ffff88003674f000 - ffff880036800000)
[   26.546676] Freeing unused kernel memory: 900K (ffff880036b1f000 - ffff880036c00000)
[   26.556078] ehci-pci 0000:00:1d.0: fatal error
[   26.560540] ehci-pci 0000:00:1d.0: HC died; cleaning up
[   26.567014] systemd[1]: systemd 208 running in system mode. (+PAM +LIBWRAP +AUDIT +SELINUX +IMA +SYSVINIT +LIBCRYPTSETUP +GCRYPT +ACL +XZ)
[   26.579620] systemd[1]: Running in initial RAM disk.

Welcome to Fedora 20 (Heisenbug) dracut-038-14.git20140724.fc22 (Initramfs)!

[   26.593159] systemd[1]: Set hostname to <dhcp-16-105.nay.redhat.com>.
[   26.600193] random: systemd urandom read with 6 bits of entropy available
[   26.631984] systemd[1]: Expecting device dev-disk-by\x2duuid-3429ef24\x2d72c1\x2d435f\x2da55b\x2d15132ef30516.device...
         Expecting device dev-disk-by\x2duuid-3429ef24\x2d72c...30516.device...
[   26.651165] systemd[1]: Expecting device dev-disk-by\x2duuid-06476532\x2d3675\x2d40ce\x2d98dd\x2df0b6564e5455.device...
         Expecting device dev-disk-by\x2duuid-06476532\x2d367...e5455.device...
[   26.670181] systemd[1]: Expecting device dev-disk-by\x2duuid-f170152e\x2dde83\x2d46ee\x2d9546\x2d8ccd53f9753b.device...
         Expecting device dev-disk-by\x2duuid-f170152e\x2dde8...9753b.device...
[   26.689199] systemd[1]: Starting -.slice.
[  OK  ] Created slice -.slice.
[   26.698214] systemd[1]: Created slice -.slice.
[   26.702693] systemd[1]: Starting System Slice.
[  OK  ] Created slice System Slice.
[   26.713228] systemd[1]: Created slice System Slice.
[   26.718146] systemd[1]: Starting Slices.
[  OK  ] Reached target Slices.
[   26.726239] systemd[1]: Reached target Slices.
[   26.730717] systemd[1]: Starting Timers.
[  OK  ] Reached target Timers.
[   26.739285] systemd[1]: Reached target Timers.
[   26.743773] systemd[1]: Starting Dispatch Password Requests to Console Directory Watch.
[   26.751844] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.
[   26.759798] systemd[1]: Starting Paths.
[  OK  ] Reached target Paths.
[   26.768283] systemd[1]: Reached target Paths.
[   26.772672] systemd[1]: Starting Journal Socket.
[  OK  ] Listening on Journal Socket.
[   26.782304] systemd[1]: Listening on Journal Socket.
[   26.787342] systemd[1]: Started dracut ask for additional cmdline parameters.
[   26.794597] systemd[1]: Starting dracut cmdline hook...
         Starting dracut cmdline hook...
[   26.804560] systemd[1]: Starting Apply Kernel Variables...
         Starting Apply Kernel Variables...
[   26.818484] systemd[1]: Starting Journal Service...
         Starting Journal Service...
[  OK  ] Started Journal Service.
[   26.836324] systemd[1]: Started Journal Service.
         Starting Create list of required static device nodes...rrent kernel...
[  OK  ] Listening on udev Kernel Socket.
[  OK  ] Listening on udev Control Socket.
[  OK  ] Reached target Sockets.
[  OK  ] Reached target Swap.
[  OK  ] Reached target Local File Systems.
[  OK  ] Started Apply Kernel Variables.
[  OK  ] Started Create list of required static device nodes ...current kernel.
         Starting Create static device nodes in /dev...
[  OK  ] Started Create static device nodes in /dev.
[  OK  ] Started dracut cmdline hook.
         Starting dracut pre-udev hook...
[  OK  ] Started dracut pre-udev hook.
         Starting udev Kernel Device Manager...
[   26.961900] systemd-udevd[209]: starting version 208
[  OK  ] Started udev Kernel Device Manager.
         Starting udev Coldplug all Devices...
[  OK  ] Started udev Coldplug all Devices.
         Starting dracut initqueue hook...
[  OK  ] Reached target System Initialization.
[  OK  ] Reached target Basic System.
         Mounting Configuration File System...
[  OK  ] Mounted Configuration File System.
[   27.047418] wmi: Mapper loaded
[   27.145121] [drm] Initialized drm 1.1.0 20060810
[   27.304098] isci: Intel(R) C600 SAS Controller Driver - version 1.2.0
[   27.322844] systemd-udevd invoked oom-killer: gfp_mask=0x280da, order=0, oom_score_adj=-1000
[   27.331312] systemd-udevd cpuset=/ mems_allowed=0
[   27.336086] CPU: 0 PID: 209 Comm: systemd-udevd Not tainted 3.19.0-rc2+ #93
[   27.343072] Hardware name: Hewlett-Packard HP Z420 Workstation/1589, BIOS J61 v01.02 03/09/2012
[   27.351798]  0000000000000000 0000000061818f1a ffff880035c0f9f8 ffffffff8173f728
[   27.359300]  0000000000000000 ffff880035c3db40 ffff880035c0fa98 ffffffff8173c406
[   27.366776]  0000000000001c02 0000000035c0fb58 0000000000001c02 0000000000000292
[   27.374271] Call Trace:
[   27.376744]  [<ffffffff8173f728>] dump_stack+0x45/0x57
[   27.381913]  [<ffffffff8173c406>] dump_header+0x89/0x20a
[   27.387239]  [<ffffffff8118fb80>] oom_kill_process+0x240/0x390
[   27.393075]  [<ffffffff811903f2>] out_of_memory+0x502/0x540
[   27.398657]  [<ffffffff81195cc8>] __alloc_pages_nodemask+0x868/0xa30
[   27.405018]  [<ffffffff811df9d7>] alloc_pages_vma+0x97/0x160
[   27.410686]  [<ffffffff811be1d6>] handle_mm_fault+0xcf6/0x1120
[   27.416524]  [<ffffffff8105d32c>] __do_page_fault+0x1dc/0x5b0
[   27.422279]  [<ffffffff8124b3d8>] ? ep_poll+0x128/0x380
[   27.427510]  [<ffffffff810cefc0>] ? pick_next_task_fair+0x610/0x870
[   27.433785]  [<ffffffff81612891>] ? __sys_recvmsg+0x51/0x90
[   27.439360]  [<ffffffff8105d722>] do_page_fault+0x22/0x30
[   27.444769]  [<ffffffff81749028>] page_fault+0x28/0x30
[   27.449911] Mem-Info:
[   27.452194] Node 0 DMA per-cpu:
[   27.455368] CPU    0: hi:    0, btch:   1 usd:   0
[   27.460164] Node 0 DMA32 per-cpu:
[   27.463505] CPU    0: hi:   90, btch:  15 usd:  27
[   27.468308] active_anon:800 inactive_anon:1025 isolated_anon:0
[   27.468308]  active_file:16 inactive_file:36 isolated_file:0
[   27.468308]  unevictable:39595 dirty:0 writeback:0 unstable:0
[   27.468308]  free:613 slab_reclaimable:2165 slab_unreclaimable:3423
[   27.468308]  mapped:1596 shmem:1036 pagetables:140 bounce:0
[   27.468308]  free_cma:0
[   27.499745] Node 0 DMA free:512kB min:4kB low:4kB high:4kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB s
[   27.537985] lowmem_reserve[]: 0 233 233 233
[   27.542288] Node 0 DMA32 free:1940kB min:1944kB low:2428kB high:2916kB active_anon:3200kB inactive_anon:4100kB active_file:64kB inactive_file:144kB unevictable:15s
[   27.585391] lowmem_reserve[]: 0 0 0 0
[   27.589164] Node 0 DMA: 0*4kB 0*8kB 0*16kB 0*32kB 2*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 512kB
[   27.601030] Node 0 DMA32: 1*4kB (R) 0*8kB 1*16kB (R) 0*32kB 0*64kB 1*128kB (R) 1*256kB (R) 1*512kB (R) 1*1024kB (R) 0*2048kB 0*4096kB = 1940kB
[   27.614272] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
[   27.622733] 40680 total pagecache pages
[   27.626588] 0 pages in swap cache
[   27.629916] Swap cache stats: add 0, delete 0, find 0/0
[   27.635160] Free swap  = 0kB
[   27.638054] Total swap = 0kB
[   27.640949] 65531 pages RAM
[   27.643758] 0 pages HighMem/MovableOnly
[   27.647612] 4986 pages reserved
[   27.650767] 0 pages hwpoisoned
[   27.653840] [ pid ]   uid  tgid total_vm      rss nr_ptes swapents oom_score_adj name
[   27.661699] [  126]     0   126     9732      841      23        0             0 systemd-journal
[   27.670513] [  209]     0   209     8857      920      20        0         -1000 systemd-udevd
[   27.679147] [  213]     0   213    28751      617      19        0             0 systemd-udevd
[   27.687780] [  214]     0   214     9769      616      19        0             0 systemd-udevd
[   27.696416] [  216]     0   216     3010      751      14        0             0 dracut-initqueu
[   27.705225] [  225]     0   225     8160      652      19        0             0 udevadm
[   27.713341] Out of memory: Kill process 126 (systemd-journal) score 13 or sacrifice child
[   27.721539] Killed process 126 (systemd-journal) total-vm:38928kB, anon-rss:292kB, file-rss:3072kB
[   27.731134] systemd[1]: systemd-journald.service holdoff time over, scheduling restart.
[   27.740866] systemd[1]: Stopping Journal Service...
[   27.745800] isci 0000:02:00.0: driver configured for rev: 5 silicon
         Stoppin[   27.752726] systemd-udevd invoked oom-killer: gfp_mask=0x280da, order=0, oom_score_adj=-1000
g Journal Servic[   27.762130] systemd-udevd cpuset=e...
/ mems_allowed=0
[   27.768777] CPU: 0 PID: 209 Comm: systemd-udevd Not tainted 3.19.0-rc2+ #93
[   27.775760] Hardware name: Hewlett-Packard HP Z420 Workstation/1589, BIOS J61 v01.02 03/09/2012
[   27.784481]  0000000000000000 0000000061818f1a ffff880035c0f9f8 ffffffff8173f728
[   27.791998]  0000000000000000 ffff880035c3db40 ffff880035c0fa98 ffffffff8173c406
[   27.799479]  0000000000000000 0000000035c0fb58 0000000000000000 0000000000000292
[   27.806963] Call Trace:
[   27.809419]  [<ffffffff8173f728>] dump_stack+0x45/0x57
[   27.814575]  [<ffffffff8173c406>] dump_header+0x89/0x20a
[   27.819907]  [<ffffffff8118fb80>] oom_kill_process+0x240/0x390
[   27.825756]  [<ffffffff811903f2>] out_of_memory+0x502/0x540
[   27.831351]  [<ffffffff81195cc8>] __alloc_pages_nodemask+0x868/0xa30
[   27.837722]  [<ffffffff811df9d7>] alloc_pages_vma+0x97/0x160
[   27.843399]  [<ffffffff811be1d6>] handle_mm_fault+0xcf6/0x1120
[   27.849253]  [<ffffffff8105d32c>] __do_page_fault+0x1dc/0x5b0
[   27.855020]  [<ffffffff8124b3d8>] ? ep_poll+0x128/0x380
[   27.860263]  [<ffffffff810cefc0>] ? pick_next_task_fair+0x610/0x870
[   27.866549]  [<ffffffff81612891>] ? __sys_recvmsg+0x51/0x90
[   27.872139]  [<ffffffff8105d722>] do_page_fault+0x22/0x30
[   27.877559]  [<ffffffff81749028>] page_fault+0x28/0x30
[   27.882714] Mem-Info:
[   27.884999] Node 0 DMA per-cpu:
[   27.888166] CPU    0: hi:    0, btch:   1 usd:   0
[   27.892975] Node 0 DMA32 per-cpu:
[   27.896327] CPU    0: hi:   90, btch:  15 usd:  41
[   27.901137] active_anon:758 inactive_anon:1027 isolated_anon:0
[   27.901137]  active_file:3 inactive_file:2 isolated_file:0
[   27.901137]  unevictable:39644 dirty:0 writeback:0 unstable:0
[   27.901137]  free:610 slab_reclaimable:2087 slab_unreclaimable:3423
[   27.901137]  mapped:1377 shmem:1037 pagetables:124 bounce:0
[   27.901137]  free_cma:0
[   27.932472] Node 0 DMA free:512kB min:4kB low:4kB high:4kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB s
[   27.970798] lowmem_reserve[]: 0 233 233 233
[   27.975102] Node 0 DMA32 free:1928kB min:1944kB low:2428kB high:2916kB active_anon:3032kB inactive_anon:4108kB active_file:12kB inactive_file:8kB unevictable:1585s
[   28.017660] lowmem_reserve[]: 0 0 0 0
[   28.021443] Node 0 DMA: 0*4kB 0*8kB 0*16kB 0*32kB 2*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 512kB
[   28.033320] Node 0 DMA32: 0*4kB 1*8kB (R) 0*16kB 0*32kB 0*64kB 1*128kB (R) 1*256kB (R) 1*512kB (R) 1*1024kB (R) 0*2048kB 0*4096kB = 1928kB
[   28.046165] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
[   28.054633] 40684 total pagecache pages
[   28.058481] 0 pages in swap cache
[   28.061812] Swap cache stats: add 0, delete 0, find 0/0
[   28.067061] Free swap  = 0kB
[   28.069954] Total swap = 0kB
[   28.072853] 65531 pages RAM
[   28.075660] 0 pages HighMem/MovableOnly
[   28.079517] 4986 pages reserved
[   28.082671] 0 pages hwpoisoned
[   28.085743] [ pid ]   uid  tgid total_vm      rss nr_ptes swapents oom_score_adj name
[   28.093602] [  209]     0   209     8857      920      20        0         -1000 systemd-udevd
[   28.102233] [  213]     0   213    28751      617      19        0             0 systemd-udevd
[   28.110871] [  214]     0   214     9769      616      19        0             0 systemd-udevd
[   28.119508] [  216]     0   216     3010      751      14        0             0 dracut-initqueu
[   28.128320] [  225]     0   225     8160      652      19        0             0 udevadm
[   28.136434] [  226]     0   226     3257      241      10        0             0 systemd-cgroups
[   28.145245] Out of memory: Kill process 216 (dracut-initqueu) score 12 or sacrifice child
[   28.153444] Killed process 225 (udevadm) total-vm:32640kB, anon-rss:256kB, file-rss:2352kB
[   28.520126] dracut-initqueu invoked oom-killer: gfp_mask=0x2000d0, order=2, oom_score_adj=0
[   28.528504] dracut-initqueu cpuset=/ mems_allowed=0
[   28.533442] CPU: 0 PID: 216 Comm: dracut-initqueu Not tainted 3.19.0-rc2+ #93
[   28.540600] Hardware name: Hewlett-Packard HP Z420 Workstation/1589, BIOS J61 v01.02 03/09/2012
[   28.549324]  0000000000000000 000000000a9a4636 ffff880035bcba80 ffffffff8173f728
[   28.556830]  0000000000000000 ffff880035c3ed80 ffff880035bcbb20 ffffffff8173c406
[   28.564320]  0000000000000005 0000000035bcbbe0 0000000000000005 0000000000000282
[   28.571807] Call Trace:
[   28.574267]  [<ffffffff8173f728>] dump_stack+0x45/0x57
[   28.579422]  [<ffffffff8173c406>] dump_header+0x89/0x20a
[   28.584754]  [<ffffffff8118fb80>] oom_kill_process+0x240/0x390
[   28.590609]  [<ffffffff811903f2>] out_of_memory+0x502/0x540
[   28.596203]  [<ffffffff81195cc8>] __alloc_pages_nodemask+0x868/0xa30
[   28.602577]  [<ffffffff81195efd>] alloc_kmem_pages_node+0x6d/0x130
[   28.608786]  [<ffffffff81092753>] ? copy_process.part.26+0x113/0x1b70
[   28.615250]  [<ffffffff81092773>] copy_process.part.26+0x133/0x1b70
[   28.621540]  [<ffffffff8105d352>] ? __do_page_fault+0x202/0x5b0
[   28.627488]  [<ffffffff8130a77f>] ? selinux_inode_permission+0xaf/0x170
[   28.634125]  [<ffffffff81094371>] do_fork+0xe1/0x360
[   28.639116]  [<ffffffff810a3db6>] ? __set_current_blocked+0x36/0x50
[   28.645400]  [<ffffffff81094676>] SyS_clone+0x16/0x20
[   28.650472]  [<ffffffff817473a9>] stub_clone+0x69/0x90
[   28.655633]  [<ffffffff81747069>] ? system_call_fastpath+0x12/0x17
[   28.661827] Mem-Info:
[   28.664115] Node 0 DMA per-cpu:
[   28.667298] CPU    0: hi:    0, btch:   1 usd:   0
[   28.672106] Node 0 DMA32 per-cpu:
[   28.675449] CPU    0: hi:   90, btch:  15 usd:   0
[   28.680259] active_anon:697 inactive_anon:1027 isolated_anon:0
[   28.680259]  active_file:0 inactive_file:0 isolated_file:0
[   28.680259]  unevictable:39647 dirty:0 writeback:0 unstable:0
[   28.680259]  free:618 slab_reclaimable:2083 slab_unreclaimable:3424
[   28.680259]  mapped:1313 shmem:1037 pagetables:108 bounce:0
[   28.680259]  free_cma:0
[   28.711600] Node 0 DMA free:512kB min:4kB low:4kB high:4kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB s
[   28.749918] lowmem_reserve[]: 0 233 233 233
[   28.754205] Node 0 DMA32 free:1960kB min:1944kB low:2428kB high:2916kB active_anon:2788kB inactive_anon:4108kB active_file:0kB inactive_file:0kB unevictable:15858s
[   28.796590] lowmem_reserve[]: 0 0 0 0
[   28.800368] Node 0 DMA: 0*4kB 0*8kB 0*16kB 0*32kB 2*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 512kB
[   28.812229] Node 0 DMA32: 0*4kB 1*8kB (R) 0*16kB 1*32kB (R) 0*64kB 1*128kB (R) 1*256kB (R) 1*512kB (R) 1*1024kB (R) 0*2048kB 0*4096kB = 1960kB
[   28.825466] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
[   28.833931] 40684 total pagecache pages
[   28.837781] 0 pages in swap cache
[   28.841115] Swap cache stats: add 0, delete 0, find 0/0
[   28.846359] Free swap  = 0kB
[   28.849254] Total swap = 0kB
[   28.852151] 65531 pages RAM
[   28.854959] 0 pages HighMem/MovableOnly
[   28.858813] 4986 pages reserved
[   28.861971] 0 pages hwpoisoned
[   28.865039] [ pid ]   uid  tgid total_vm      rss nr_ptes swapents oom_score_adj name
[   28.872893] [  209]     0   209     8857      920      20        0         -1000 systemd-udevd
[   28.881532] [  213]     0   213    28751      617      19        0             0 systemd-udevd
[   28.890168] [  214]     0   214     9769      616      19        0             0 systemd-udevd
[   28.898802] [  216]     0   216     3010      751      14        0             0 dracut-initqueu
[   28.907614] [  226]     0   226     3257      241      10        0             0 systemd-cgroups
[   28.916423] Out of memory: Kill process 216 (dracut-initqueu) score 12 or sacrifice child
[   28.924627] Killed process 216 (dracut-initqueu) total-vm:12040kB, anon-rss:564kB, file-rss:2440kB
[   28.934627] systemd-udevd invoked oom-killer: gfp_mask=0x2d2, order=0, oom_score_adj=0
[   28.942570] systemd-udevd cpuset=/ mems_allowed=0
[   28.947339] CPU: 0 PID: 213 Comm: systemd-udevd Not tainted 3.19.0-rc2+ #93
[   28.954321] Hardware name: Hewlett-Packard HP Z420 Workstation/1589, BIOS J61 v01.02 03/09/2012
[   28.963045]  0000000000000000 0000000040da54f4 ffff880035cdfa38 ffffffff8173f728
[   28.970550]  0000000000000000 ffff880035c3ada0 ffff880035cdfad8 ffffffff8173c406
[   28.978065]  0000000000000000 0000000035cdfb98 0000000000000000 0000000000000292
[   28.985534] Call Trace:
[   28.987984]  [<ffffffff8173f728>] dump_stack+0x45/0x57
[   28.993129]  [<ffffffff8173c406>] dump_header+0x89/0x20a
[   28.998451]  [<ffffffff8101de03>] ? check_tsc_unstable+0x3/0x10
[   29.004372]  [<ffffffff8118fb80>] oom_kill_process+0x240/0x390
[   29.010211]  [<ffffffff811903f2>] out_of_memory+0x502/0x540
[   29.015791]  [<ffffffff81195cc8>] __alloc_pages_nodemask+0x868/0xa30
[   29.022156]  [<ffffffff811dd161>] alloc_pages_current+0x91/0x100
[   29.028163]  [<ffffffff811cce52>] __vmalloc_node_range+0x1a2/0x280
[   29.034352]  [<ffffffff81110085>] ? copy_module_from_fd.isra.48+0xe5/0x180
[   29.041229]  [<ffffffff811cd19b>] vmalloc+0x4b/0x50
[   29.046110]  [<ffffffff81110085>] ? copy_module_from_fd.isra.48+0xe5/0x180
[   29.052998]  [<ffffffff81110085>] copy_module_from_fd.isra.48+0xe5/0x180
[   29.059714]  [<ffffffff811140df>] SyS_finit_module+0x6f/0xd0
[   29.065397]  [<ffffffff81747069>] system_call_fastpath+0x12/0x17
[   29.071421] Mem-Info:
[   29.073701] Node 0 DMA per-cpu:
[   29.076884] CPU    0: hi:    0, btch:   1 usd:   0
[   29.081696] Node 0 DMA32 per-cpu:
[   29.085044] CPU    0: hi:   90, btch:  15 usd:  63
[   29.089849] active_anon:582 inactive_anon:1027 isolated_anon:0
[   29.089849]  active_file:0 inactive_file:0 isolated_file:0
[   29.089849]  unevictable:39647 dirty:0 writeback:0 unstable:0
[   29.089849]  free:614 slab_reclaimable:2083 slab_unreclaimable:3424
[   29.089849]  mapped:1089 shmem:1037 pagetables:108 bounce:0
[   29.089849]  free_cma:0
[   29.121124] Node 0 DMA free:512kB min:4kB low:4kB high:4kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB s
[   29.159363] lowmem_reserve[]: 0 233 233 233
[   29.163652] Node 0 DMA32 free:1944kB min:1944kB low:2428kB high:2916kB active_anon:2328kB inactive_anon:4108kB active_file:0kB inactive_file:0kB unevictable:15858s
[   29.205958] lowmem_reserve[]: 0 0 0 0
[   29.209727] Node 0 DMA: 0*4kB 0*8kB 0*16kB 0*32kB 2*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 512kB
[   29.221567] Node 0 DMA32: 20*4kB (MR) 1*8kB (R) 0*16kB 0*32kB 1*64kB (R) 0*128kB 1*256kB (R) 1*512kB (R) 1*1024kB (R) 0*2048kB 0*4096kB = 1944kB
[   29.234944] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
[   29.243391] 40684 total pagecache pages
[   29.247229] 0 pages in swap cache
[   29.250552] Swap cache stats: add 0, delete 0, find 0/0
[   29.255786] Free swap  = 0kB
[   29.258674] Total swap = 0kB
[   29.261563] 65531 pages RAM
[   29.264364] 0 pages HighMem/MovableOnly
[   29.268208] 4986 pages reserved
[   29.271355] 0 pages hwpoisoned
[   29.274418] [ pid ]   uid  tgid total_vm      rss nr_ptes swapents oom_score_adj name
[   29.282259] [  209]     0   209     8857      920      20        0         -1000 systemd-udevd
[   29.290873] [  213]     0   213    28751      617      19        0             0 systemd-udevd
[   29.299512] [  214]     0   214     9769      616      19        0             0 systemd-udevd
[   29.308149] [  226]     0   226     3257      241      10        0             0 systemd-cgroups
[   29.316962] [  231]     0   231     3257      235      12        0             0 systemd-cgroups
[   29.325773] Out of memory: Kill process 213 (systemd-udevd) score 10 or sacrifice child
[   29.333807] Killed process 213 (systemd-udevd) total-vm:115004kB, anon-rss:380kB, file-rss:2088kB
[  OK  ] Stopped Journa[   29.347845] isci 0000:02:00.0: OEM SAS parameters (version: 1.3) loaded (firmware)
l Service.
[   29.358130] systemd[1]: Starting Journal Service...
         Starting Journal Servic[   29.367713] vmalloc: allocation failure, allocated 49369088 of 82141184 bytes
e...
[   29.375098] systemd-udevd: page allocation failure: order:0, mode:0xd2
[   29.382169] CPU: 0 PID: 213 Comm: systemd-udevd Not tainted 3.19.0-rc2+ #93
[   29.389151] Hardware name: Hewlett-Packard HP Z420 Workstation/1589, BIOS J61 v01.02 03/09/2012
[   29.397874]  0000000000000000 0000000040da54f4 ffff880035cdfcf8 ffffffff8173f728
[   29.405367]  0000000000000000 00000000000000d2 ffff880035cdfd88 ffffffff81191ef9
[   29.412884]  ffffffff81a42ed0 ffff880035cdfd18 0000000000000018 ffff880035cdfd98
[   29.420383] Call Trace:
[   29.422842]  [<ffffffff8173f728>] dump_stack+0x45/0x57
[   29.428000]  [<ffffffff81191ef9>] warn_alloc_failed+0xf9/0x160
[   29.433858]  [<ffffffff811dd161>] ? alloc_pages_current+0x91/0x100
[   29.440054]  [<ffffffff811ccef8>] __vmalloc_node_range+0x248/0x280
[   29.446256]  [<ffffffff81110085>] ? copy_module_from_fd.isra.48+0xe5/0x180
[   29.453155]  [<ffffffff811cd19b>] vmalloc+0x4b/0x50
[   29.458054]  [<ffffffff81110085>] ? copy_module_from_fd.isra.48+0xe5/0x180
[   29.464950]  [<ffffffff81110085>] copy_module_from_fd.isra.48+0xe5/0x180
[   29.471674]  [<ffffffff811140df>] SyS_finit_module+0x6f/0xd0
[   29.477356]  [<ffffffff81747069>] system_call_fastpath+0x12/0x17
[   29.483380] Mem-Info:
[   29.485667] Node 0 DMA per-cpu:
[   29.488834] CPU    0: hi:    0, btch:   1 usd:   0
[   29.493642] Node 0 DMA32 per-cpu:
[   29.496982] CPU    0: hi:   90, btch:  15 usd:  53
[   29.501785] active_anon:582 inactive_anon:1027 isolated_anon:0
[   29.501785]  active_file:0 inactive_file:0 isolated_file:0
[   29.501785]  unevictable:39647 dirty:0 writeback:0 unstable:0
[   29.501785]  free:0 slab_reclaimable:1835 slab_unreclaimable:3404
[   29.501785]  mapped:1089 shmem:1037 pagetables:108 bounce:0
[   29.501785]  free_cma:0
[   29.532945] Node 0 DMA free:0kB min:4kB low:4kB high:4kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB iss
[   29.571086] lowmem_reserve[]: 0 233 233 233
[   29.575389] Node 0 DMA32 free:0kB min:1944kB low:2428kB high:2916kB active_anon:2328kB inactive_anon:4108kB active_file:0kB inactive_file:0kB unevictable:158588kBs
[   29.617544] lowmem_reserve[]: 0 0 0 0
[   29.621324] Node 0 DMA: 0*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 0kB
[   29.631920] Node 0 DMA32: 0*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 0kB
[   29.642688] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
[   29.651152] 40684 total pagecache pages
[   29.655003] 0 pages in swap cache
[   29.658334] Swap cache stats: add 0, delete 0, find 0/0
[   29.663583] Free swap  = 0kB
[   29.666476] Total swap = 0kB
[   29.669373] 65531 pages RAM
[   29.672182] 0 pages HighMem/MovableOnly
[   29.676039] 4986 pages reserved
[   29.679194] 0 pages hwpoisoned
[  OK  ] Started Journa[   29.686692] systemd-journald[232]: File /run/log/journal/5504ea9d96a745d3bfa88caececd2740/system.journal corrupted or uncleanly shut down, .
l Service.
[   29.703496] systemd[1]: Started Journal Service.
[  OK  ] Started dracut initqueue hook.
[  OK  ] Reached target Initrd Root File System.
         Starting Reload Configu[   29.726403] isci 0000:02:00.0: SCU controller 0: phy 3-0 cables: {short, short, short, short}
ration from the Real Root...
[  OK  ] Reached target Remote File Systems (Pre).
[  OK  ] Reached target Remote File Systems.
[   29.760493] scsi host7: ata_generic
[   29.771845] scsi host6: isci
[   29.780869] scsi host8: ata_generic
[   29.791001] ata7: PATA max UDMA/100 cmd 0xf0b0 ctl 0xf0a0 bmdma 0xf070 irq 18
[   29.798170] ata8: PATA max UDMA/100 cmd 0xf090 ctl 0xf080 bmdma 0xf078 irq 18
[  OK  ] Started Reload Configuration f[   29.817718] dmar: DRHD: handling fault status reg 702
[   29.823891] dmar: INTR-REMAP: Request device [[02:00.0] fault index 22
[   29.823891] INTR-REMAP:[fault reason 38] Blocked an interrupt request due to source-id verification failure
rom the Real Root.
[  OK  ] Reached target Initrd File Systems.
[  OK  ] Reached target Initrd Default Target.
[   30.249111] pps_core: LinuxPPS API ver. 1 registered
[   30.254100] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti at linux.it>
[   30.288618] PTP clock support registered
[   30.349920] e1000e: Intel(R) PRO/1000 Network Driver - 2.3.2-k
[   30.355764] e1000e: Copyright(c) 1999 - 2014 Intel Corporation.
[   30.383553] e1000e 0000:00:19.0: Interrupt Throttling Rate (ints/sec) set to dynamic conservative mode
[   30.394670] tulip: Linux Tulip driver version 1.1.15 (Feb 27, 2007)
[   30.401642] tulip: tulip_init_one: Enabled WOL support for AN983B
[   30.407754] dmar: DRHD: handling fault status reg 2
[   30.412634] dmar: DMAR:[DMA Write] Request device [00:1e.0] fault addr 0 
[   30.412634] DMAR:[fault reason 05] PTE Write access is not set
[   30.426642] tulip 0000:09:04.0 (unnamed net_device) (uninitialized): tulip_stop_rxtx() failed (CSR5 0xfc0dc0d2 CSR6 0xff970111)
[   30.438566] tulip0:  MII transceiver #1 config 1000 status 786d advertising 05e1
[   30.462753] net eth0: ADMtek Comet rev 17 at MMIO 0xd7121000, 00:b0:c0:06:70:90, IRQ 20
[   30.548322] firewire_ohci 0000:09:05.0: added OHCI v1.0 device as card 0, 8 IR + 8 IT contexts, quirks 0x0
[   30.689459] e1000e 0000:00:19.0 eth1: registered PHC clock
[   30.694975] e1000e 0000:00:19.0 eth1: (PCI Express:2.5GT/s:Width x1) 80:c1:6e:f8:9f:92
[   30.702925] e1000e 0000:00:19.0 eth1: Intel(R) PRO/1000 Network Connection
[   30.711299] e1000e 0000:00:19.0 eth1: MAC: 10, PHY: 11, PBA No: 0100FF-0FF
[   30.822844] e1000e 0000:00:19.0 em1: renamed from eth1
[   30.828497] tulip 0000:09:04.0 p6p1: renamed from eth0
[   30.833799] systemd-udevd[233]: renamed network interface eth1 to em1
[   30.840351] systemd-udevd[214]: renamed network interface eth0 to p6p1
[   30.850352] iTCO_vendor_support: vendor-support=0
[   30.860111] iTCO_wdt: Intel TCO WatchDog Timer Driver v1.11
[   30.865760] iTCO_wdt: unable to reset NO_REBOOT flag, device disabled by hardware/BIOS
[   31.068066] firewire_core 0000:09:05.0: created device fw0: GUID 0060b000008cec98, S400

From zhen-hual at hp.com  Tue Jan  6 20:04:24 2015
From: zhen-hual at hp.com (Li, Zhen-Hua)
Date: Wed,  7 Jan 2015 12:04:24 +0800
Subject: [PATCH v7 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel
Message-ID: <1420603474-4235-1-git-send-email-zhen-hual@hp.com>

This patchset is an update of Bill Sumner's patchset, implements a fix for:
If a kernel boots with intel_iommu=on on a system that supports intel vt-d, 
when a panic happens, the kdump kernel will boot with these faults:

    dmar: DRHD: handling fault status reg 102
    dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr fff80000
    DMAR:[fault reason 01] Present bit in root entry is clear

    dmar: DRHD: handling fault status reg 2
    dmar: INTR-REMAP: Request device [[61:00.0] fault index 42
    INTR-REMAP:[fault reason 34] Present field in the IRTE entry is clear

On some system, the interrupt remapping fault will also happen even if the 
intel_iommu is not set to on, because the interrupt remapping will be enabled 
when x2apic is needed by the system.

The cause of the DMA fault is described in Bill's original version, and the 
INTR-Remap fault is caused by a similar reason. In short, the initialization 
of vt-d drivers causes the in-flight DMA and interrupt requests get wrong 
response.

To fix this problem, we modifies the behaviors of the intel vt-d in the 
crashdump kernel:

For DMA Remapping:
1. To accept the vt-d hardware in an active state,
2. Do not disable and re-enable the translation, keep it enabled.
3. Use the old root entry table, do not rewrite the RTA register.
4. Malloc and use new context entry table and page table, copy data from the 
   old ones that used by the old kernel.
5. to use different portions of the iova address ranges for the device drivers
   in the crashdump kernel than the iova ranges that were in-use at the time
   of the panic.  
6. After device driver is loaded, when it issues the first dma_map command, 
   free the dmar_domain structure for this device, and generate a new one, so 
   that the device can be assigned a new and empty page table. 
7. When a new context entry table is generated, we also save its address to 
   the old root entry table.

For Interrupt Remapping:
1. To accept the vt-d hardware in an active state,
2. Do not disable and re-enable the interrupt remapping, keep it enabled.
3. Use the old interrupt remapping table, do not rewrite the IRTA register.
4. When ioapic entry is setup, the interrupt remapping table is changed, and 
   the updated data will be stored to the old interrupt remapping table.

Advantages of this approach:
1. All manipulation of the IO-device is done by the Linux device-driver
   for that device.
2. This approach behaves in a manner very similar to operation without an
   active iommu.
3. Any activity between the IO-device and its RMRR areas is handled by the
   device-driver in the same manner as during a non-kdump boot.
4. If an IO-device has no driver in the kdump kernel, it is simply left alone.
   This supports the practice of creating a special kdump kernel without
   drivers for any devices that are not required for taking a crashdump. 
5. Minimal code-changes among the existing mainline intel vt-d code.

Summary of changes in this patch set:
1. Added some useful function for root entry table in code intel-iommu.c
2. Added new members to struct root_entry and struct irte;
3. Functions to load old root entry table to iommu->root_entry from the memory 
   of old kernel.
4. Functions to malloc new context entry table and page table and copy the data
   from the old ones to the malloced new ones.
5. Functions to enable support for DMA remapping in kdump kernel.
6. Functions to load old irte data from the old kernel to the kdump kernel.
7. Some code changes that support other behaviours that have been listed.
8. In the new functions, use physical address as "unsigned long" type, not 
   pointers.

Original version by Bill Sumner:
    https://lkml.org/lkml/2014/1/10/518
    https://lkml.org/lkml/2014/4/15/716
    https://lkml.org/lkml/2014/4/24/836

Zhenhua's updates:
    https://lkml.org/lkml/2014/10/21/134
    https://lkml.org/lkml/2014/12/15/121
    https://lkml.org/lkml/2014/12/22/53

Changelog[v7]:
    1. Use __iommu_flush_cache to flush the data to hardware.

Changelog[v6]:
    1. Use "unsigned long" as type of physical address.
    2. Use new function unmap_device_dma to unmap the old dma.
    3. Some small incorrect bits order for aw shift.

Changelog[v5]:
    1. Do not disable and re-enable traslation and interrupt remapping. 
    2. Use old root entry table.
    3. Use old interrupt remapping table.
    4. New functions to copy data from old kernel, and save to old kernel mem.
    5. New functions to save updated root entry table and irte table.
    6. Use intel_unmap to unmap the old dma;
    7. Allocate new pages while driver is being loaded.

Changelog[v4]:
    1. Cut off the patches that move some defines and functions to new files.
    2. Reduce the numbers of patches to five, make it more easier to read.
    3. Changed the name of functions, make them consistent with current context
       get/set functions.
    4. Add change to function __iommu_attach_domain.

Changelog[v3]:
    1. Commented-out "#define DEBUG 1" to eliminate debug messages.
    2. Updated the comments about changes in each version.
    3. Fixed: one-line added to Copy-Translations patch to initialize the iovad
          struct as recommended by Baoquan He [bhe at redhat.com]
          init_iova_domain(&domain->iovad, DMA_32BIT_PFN);

Changelog[v2]:
    The following series implements a fix for:
    A kdump problem about DMA that has been discussed for a long time. That is,
    when a kernel panics and boots into the kdump kernel, DMA started by the
    panicked kernel is not stopped before the kdump kernel is booted and the
    kdump kernel disables the IOMMU while this DMA continues.  This causes the
    IOMMU to stop translating the DMA addresses as IOVAs and begin to treat
    them as physical memory addresses -- which causes the DMA to either:
        (1) generate DMAR errors or 
        (2) generate PCI SERR errors or 
        (3) transfer data to or from incorrect areas of memory. Often this 
            causes the dump to fail.

Changelog[v1]:
    The original version.

Changed in this version:
1. Do not disable and re-enable traslation and interrupt remapping. 
2. Use old root entry table.
3. Use old interrupt remapping table.
4. Use "unsigned long" as physical address.
5. Use intel_unmap to unmap the old dma;

Baoquan He <bhe at redhat.com> helps testing this patchset.

  iommu/vt-d: Update iommu_attach_domain() and its callers
  iommu/vt-d: Items required for kdump
  iommu/vt-d: Add domain-id functions
  iommu/vt-d: functions to copy data from old mem
  iommu/vt-d: Add functions to load and save old re
  iommu/vt-d: datatypes and functions used for kdump
  iommu/vt-d: enable kdump support in iommu module
  iommu/vt-d: assign new page table for dma_map
  iommu/vt-d: Copy functions for irte
  iommu/vt-d: Use old irte in kdump kernel

Signed-off-by: Bill Sumner <billsumnerlinux at gmail.com>
Signed-off-by: Li, Zhen-Hua <zhen-hual at hp.com>
Signed-off-by: Takao Indoh <indou.takao at jp.fujitsu.com>
Tested-by: Baoquan He <bhe at redhat.com>
---
 drivers/iommu/intel-iommu.c         | 1050 +++++++++++++++++++++++++++++++++--
 drivers/iommu/intel_irq_remapping.c |  104 +++-
 include/linux/intel-iommu.h         |   18 +
 3 files changed, 1130 insertions(+), 42 deletions(-)

-- 
2.0.0-rc0



From zhen-hual at hp.com  Tue Jan  6 20:04:26 2015
From: zhen-hual at hp.com (Li, Zhen-Hua)
Date: Wed,  7 Jan 2015 12:04:26 +0800
Subject: [PATCH v7 02/10] iommu/vt-d: Items required for kdump
In-Reply-To: <1420603474-4235-1-git-send-email-zhen-hual@hp.com>
References: <1420603474-4235-1-git-send-email-zhen-hual@hp.com>
Message-ID: <1420603474-4235-3-git-send-email-zhen-hual@hp.com>

Add structure type domain_values_entry used for kdump;
Add context entry functions needed for kdump.

Bill Sumner:
    Original version;

Li, Zhenhua:
    Changed the name of new functions, make them consistent with current
    context get/set functions.

Signed-off-by: Bill Sumner <billsumnerlinux at gmail.com>
Signed-off-by: Li, Zhen-Hua <zhen-hual at hp.com>
---
 drivers/iommu/intel-iommu.c | 70 +++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 70 insertions(+)

diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 2dc6250..5ce2850 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -40,6 +40,7 @@
 #include <linux/pci-ats.h>
 #include <linux/memblock.h>
 #include <linux/dma-contiguous.h>
+#include <linux/crash_dump.h>
 #include <asm/irq_remapping.h>
 #include <asm/cacheflush.h>
 #include <asm/iommu.h>
@@ -208,6 +209,12 @@ get_context_addr_from_root(struct root_entry *root)
 		NULL);
 }
 
+static inline unsigned long
+get_context_phys_from_root(struct root_entry *root)
+{
+	return  root_present(root) ? (root->val & VTD_PAGE_MASK) : 0;
+}
+
 /*
  * low 64 bits:
  * 0: present
@@ -228,6 +235,32 @@ static inline bool context_present(struct context_entry *context)
 {
 	return (context->lo & 1);
 }
+
+static inline int context_fault_enable(struct context_entry *c)
+{
+	return((c->lo >> 1) & 0x1);
+}
+
+static inline int context_translation_type(struct context_entry *c)
+{
+	return((c->lo >> 2) & 0x3);
+}
+
+static inline u64 context_address_root(struct context_entry *c)
+{
+	return((c->lo >> VTD_PAGE_SHIFT));
+}
+
+static inline int context_address_width(struct context_entry *c)
+{
+	return((c->hi >> 0) & 0x7);
+}
+
+static inline int context_domain_id(struct context_entry *c)
+{
+	return((c->hi >> 8) & 0xffff);
+}
+
 static inline void context_set_present(struct context_entry *context)
 {
 	context->lo |= 1;
@@ -313,6 +346,43 @@ static inline int first_pte_in_page(struct dma_pte *pte)
 	return !((unsigned long)pte & ~VTD_PAGE_MASK);
 }
 
+
+#ifdef CONFIG_CRASH_DUMP
+
+/*
+ * Fix Crashdump failure caused by leftover DMA through a hardware IOMMU
+ *
+ * Fixes the crashdump kernel to deal with an active iommu and legacy
+ * DMA from the (old) panicked kernel in a manner similar to how legacy
+ * DMA is handled when no hardware iommu was in use by the old kernel --
+ * allow the legacy DMA to continue into its current buffers.
+ *
+ * In the crashdump kernel, this code:
+ * 1. skips disabling the IOMMU's translating of IO Virtual Addresses (IOVA).
+ * 2. Do not re-enable IOMMU's translating.
+ * 3. In kdump kernel, use the old root entry table.
+ * 4. Leaves the current translations in-place so that legacy DMA will
+ *    continue to use its current buffers.
+ * 5. Allocates to the device drivers in the crashdump kernel
+ *    portions of the iova address ranges that are different
+ *    from the iova address ranges that were being used by the old kernel
+ *    at the time of the panic.
+ *
+ */
+
+struct domain_values_entry {
+	struct list_head link;		/* link entries into a list */
+	struct iova_domain iovad;	/* iova's that belong to this domain */
+	struct dma_pte	*pgd;		/* virtual address */
+	int    did;			/* domain id */
+	int    gaw;			/* max guest address width */
+	int    iommu_superpage;		/* Level of superpages supported:
+					   0 == 4KiB (no superpages), 1 == 2MiB,
+					   2 == 1GiB, 3 == 512GiB, 4 == 1TiB */
+};
+
+#endif /* CONFIG_CRASH_DUMP */
+
 /*
  * This domain is a statically identity mapping domain.
  *	1. This domain creats a static 1:1 mapping to all usable memory.
-- 
2.0.0-rc0



From zhen-hual at hp.com  Tue Jan  6 20:04:25 2015
From: zhen-hual at hp.com (Li, Zhen-Hua)
Date: Wed,  7 Jan 2015 12:04:25 +0800
Subject: [PATCH v7 01/10] iommu/vt-d: Update iommu_attach_domain() and its
 callers
In-Reply-To: <1420603474-4235-1-git-send-email-zhen-hual@hp.com>
References: <1420603474-4235-1-git-send-email-zhen-hual@hp.com>
Message-ID: <1420603474-4235-2-git-send-email-zhen-hual@hp.com>

Allow specification of the domain-id for the new domain.
This patch only adds the 'did' parameter to iommu_attach_domain()
and modifies all of its callers to specify the default value of -1
which says "no did specified, allocate a new one".

This is no functional change from current behaviour -- just enables
a functional change to be made in a later patch.

Bill Sumner:
    Original version.

Li, Zhenhua:
    Minor change, add change to function __iommu_attach_domain.

Signed-off-by: Bill Sumner <billsumnerlinux at gmail.com>
Signed-off-by: Li, Zhen-Hua <zhen-hual at hp.com>
---
 drivers/iommu/intel-iommu.c | 34 ++++++++++++++++++++--------------
 1 file changed, 20 insertions(+), 14 deletions(-)

diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 1232336..2dc6250 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -1534,31 +1534,36 @@ static struct dmar_domain *alloc_domain(int flags)
 }
 
 static int __iommu_attach_domain(struct dmar_domain *domain,
-				 struct intel_iommu *iommu)
+				 struct intel_iommu *iommu,
+				 int domain_number)
 {
 	int num;
 	unsigned long ndomains;
 
 	ndomains = cap_ndoms(iommu->cap);
-	num = find_first_zero_bit(iommu->domain_ids, ndomains);
-	if (num < ndomains) {
-		set_bit(num, iommu->domain_ids);
-		iommu->domains[num] = domain;
-	} else {
-		num = -ENOSPC;
-	}
+	if (domain_number < 0) {
+		num = find_first_zero_bit(iommu->domain_ids, ndomains);
+		if (num < ndomains) {
+			set_bit(num, iommu->domain_ids);
+			iommu->domains[num] = domain;
+		} else {
+			num = -ENOSPC;
+		}
+	} else
+		num = domain_number;
 
 	return num;
 }
 
 static int iommu_attach_domain(struct dmar_domain *domain,
-			       struct intel_iommu *iommu)
+			       struct intel_iommu *iommu,
+			       int domain_number)
 {
 	int num;
 	unsigned long flags;
 
 	spin_lock_irqsave(&iommu->lock, flags);
-	num = __iommu_attach_domain(domain, iommu);
+	num = __iommu_attach_domain(domain, iommu, domain_number);
 	spin_unlock_irqrestore(&iommu->lock, flags);
 	if (num < 0)
 		pr_err("IOMMU: no free domain ids\n");
@@ -1577,7 +1582,7 @@ static int iommu_attach_vm_domain(struct dmar_domain *domain,
 		if (iommu->domains[num] == domain)
 			return num;
 
-	return __iommu_attach_domain(domain, iommu);
+	return __iommu_attach_domain(domain, iommu, -1);
 }
 
 static void iommu_detach_domain(struct dmar_domain *domain,
@@ -2231,6 +2236,7 @@ static struct dmar_domain *get_domain_for_dev(struct device *dev, int gaw)
 	u16 dma_alias;
 	unsigned long flags;
 	u8 bus, devfn;
+	int did = -1;   /* Default to "no domain_id supplied" */
 
 	domain = find_domain(dev);
 	if (domain)
@@ -2264,7 +2270,7 @@ static struct dmar_domain *get_domain_for_dev(struct device *dev, int gaw)
 	domain = alloc_domain(0);
 	if (!domain)
 		return NULL;
-	domain->id = iommu_attach_domain(domain, iommu);
+	domain->id = iommu_attach_domain(domain, iommu, did);
 	if (domain->id < 0) {
 		free_domain_mem(domain);
 		return NULL;
@@ -2442,7 +2448,7 @@ static int __init si_domain_init(int hw)
 		return -EFAULT;
 
 	for_each_active_iommu(iommu, drhd) {
-		ret = iommu_attach_domain(si_domain, iommu);
+		ret = iommu_attach_domain(si_domain, iommu, -1);
 		if (ret < 0) {
 			domain_exit(si_domain);
 			return -EFAULT;
@@ -3866,7 +3872,7 @@ static int intel_iommu_add(struct dmar_drhd_unit *dmaru)
 	iommu_enable_translation(iommu);
 
 	if (si_domain) {
-		ret = iommu_attach_domain(si_domain, iommu);
+		ret = iommu_attach_domain(si_domain, iommu, -1);
 		if (ret < 0 || si_domain->id != ret)
 			goto disable_iommu;
 		domain_attach_iommu(si_domain, iommu);
-- 
2.0.0-rc0



From zhen-hual at hp.com  Tue Jan  6 20:04:27 2015
From: zhen-hual at hp.com (Li, Zhen-Hua)
Date: Wed,  7 Jan 2015 12:04:27 +0800
Subject: [PATCH v7 03/10] iommu/vt-d: Add domain-id functions
In-Reply-To: <1420603474-4235-1-git-send-email-zhen-hual@hp.com>
References: <1420603474-4235-1-git-send-email-zhen-hual@hp.com>
Message-ID: <1420603474-4235-4-git-send-email-zhen-hual@hp.com>

Interfaces for when a new domain in the crashdump kernel needs some
values from the panicked kernel's context entries.

Signed-off-by: Bill Sumner <billsumnerlinux at gmail.com>
Signed-off-by: Li, Zhen-Hua <zhen-hual at hp.com>
---
 drivers/iommu/intel-iommu.c | 62 +++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 62 insertions(+)

diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 5ce2850..c0bebd6 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -381,6 +381,13 @@ struct domain_values_entry {
 					   2 == 1GiB, 3 == 512GiB, 4 == 1TiB */
 };
 
+static struct domain_values_entry *intel_iommu_did_to_domain_values_entry(
+		int did, struct intel_iommu *iommu);
+
+static int intel_iommu_get_dids_from_old_kernel(struct intel_iommu *iommu);
+
+static int device_to_domain_id(struct intel_iommu *iommu, u8 bus, u8 devfn);
+
 #endif /* CONFIG_CRASH_DUMP */
 
 /*
@@ -4832,3 +4839,58 @@ static void __init check_tylersburg_isoch(void)
 	printk(KERN_WARNING "DMAR: Recommended TLB entries for ISOCH unit is 16; your BIOS set %d\n",
 	       vtisochctrl);
 }
+
+#ifdef CONFIG_CRASH_DUMP
+
+/*
+ * Interfaces for when a new domain in the crashdump kernel needs some
+ * values from the panicked kernel's context entries
+ *
+ */
+static struct domain_values_entry *intel_iommu_did_to_domain_values_entry(
+		int did, struct intel_iommu *iommu)
+{
+	struct domain_values_entry *dve;	/* iterator */
+
+	list_for_each_entry(dve, &domain_values_list[iommu->seq_id], link)
+		if (dve->did == did)
+			return dve;
+	return NULL;
+}
+
+/* Mark domain-id's from old kernel as in-use on this iommu so that a new
+ * domain-id is allocated in the case where there is a device in the new kernel
+ * that was not in the old kernel -- and therefore a new domain-id is needed.
+ */
+static int intel_iommu_get_dids_from_old_kernel(struct intel_iommu *iommu)
+{
+	struct domain_values_entry *dve;	/* iterator */
+
+	pr_info("IOMMU:%d Domain ids from panicked kernel:\n", iommu->seq_id);
+
+	list_for_each_entry(dve, &domain_values_list[iommu->seq_id], link) {
+		set_bit(dve->did, iommu->domain_ids);
+		pr_info("DID did:%d(0x%4.4x)\n", dve->did, dve->did);
+	}
+
+	pr_info("----------------------------------------\n");
+	return 0;
+}
+
+static int device_to_domain_id(struct intel_iommu *iommu, u8 bus, u8 devfn)
+{
+	int did = -1;			/* domain-id returned */
+	struct root_entry *root;
+	struct context_entry *context;
+	unsigned long flags;
+
+	spin_lock_irqsave(&iommu->lock, flags);
+	root = &iommu->root_entry[bus];
+	context = get_context_addr_from_root(root);
+	if (context && context_present(context+devfn))
+		did = context_domain_id(context+devfn);
+	spin_unlock_irqrestore(&iommu->lock, flags);
+	return did;
+}
+
+#endif /* CONFIG_CRASH_DUMP */
-- 
2.0.0-rc0



From zhen-hual at hp.com  Tue Jan  6 20:04:28 2015
From: zhen-hual at hp.com (Li, Zhen-Hua)
Date: Wed,  7 Jan 2015 12:04:28 +0800
Subject: [PATCH v7 04/10] iommu/vt-d: functions to copy data from old mem
In-Reply-To: <1420603474-4235-1-git-send-email-zhen-hual@hp.com>
References: <1420603474-4235-1-git-send-email-zhen-hual@hp.com>
Message-ID: <1420603474-4235-5-git-send-email-zhen-hual@hp.com>

Add some functions to copy the data from old kernel.
These functions are used to copy context tables and page tables.

To avoid calling iounmap between spin_lock_irqsave and spin_unlock_irqrestore,
use a link here, store the pointers , and then use iounmap to free them in
another place.

Li, Zhen-hua:
    The functions and logics.

Takao Indoh:
    Check if pfn is ram:
        if (page_is_ram(pfn))

Signed-off-by: Li, Zhen-Hua <zhen-hual at hp.com>
Signed-off-by: Takao Indoh <indou.takao at jp.fujitsu.com>
---
 drivers/iommu/intel-iommu.c | 97 +++++++++++++++++++++++++++++++++++++++++++++
 include/linux/intel-iommu.h |  9 +++++
 2 files changed, 106 insertions(+)

diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index c0bebd6..8a7ad72 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -388,6 +388,13 @@ static int intel_iommu_get_dids_from_old_kernel(struct intel_iommu *iommu);
 
 static int device_to_domain_id(struct intel_iommu *iommu, u8 bus, u8 devfn);
 
+struct iommu_remapped_entry {
+	struct list_head list;
+	void __iomem *mem;
+};
+static LIST_HEAD(__iommu_remapped_mem);
+static DEFINE_MUTEX(__iommu_mem_list_lock);
+
 #endif /* CONFIG_CRASH_DUMP */
 
 /*
@@ -4843,6 +4850,96 @@ static void __init check_tylersburg_isoch(void)
 #ifdef CONFIG_CRASH_DUMP
 
 /*
+ * Copy memory from a physically-addressed area into a virtually-addressed area
+ */
+int __iommu_load_from_oldmem(void *to, unsigned long from, unsigned long size)
+{
+	unsigned long pfn;		/* Page Frame Number */
+	size_t csize = (size_t)size;	/* Num(bytes to copy) */
+	unsigned long offset;		/* Lower 12 bits of to */
+	void __iomem *virt_mem;
+	struct iommu_remapped_entry *mapped;
+
+	pfn = from >> VTD_PAGE_SHIFT;
+	offset = from & (~VTD_PAGE_MASK);
+
+	if (page_is_ram(pfn)) {
+		memcpy(to, pfn_to_kaddr(pfn) + offset, csize);
+	} else{
+
+		mapped = kzalloc(sizeof(struct iommu_remapped_entry),
+				GFP_KERNEL);
+		if (!mapped)
+			return -ENOMEM;
+
+		virt_mem = ioremap_cache((unsigned long)from, size);
+		if (!virt_mem) {
+			kfree(mapped);
+			return -ENOMEM;
+		}
+		memcpy(to, virt_mem, size);
+
+		mutex_lock(&__iommu_mem_list_lock);
+		mapped->mem = virt_mem;
+		list_add_tail(&mapped->list, &__iommu_remapped_mem);
+		mutex_unlock(&__iommu_mem_list_lock);
+	}
+	return size;
+}
+
+/*
+ * Copy memory from a virtually-addressed area into a physically-addressed area
+ */
+int __iommu_save_to_oldmem(unsigned long to, void *from, unsigned long size)
+{
+	unsigned long pfn;		/* Page Frame Number */
+	size_t csize = (size_t)size;	/* Num(bytes to copy) */
+	unsigned long offset;		/* Lower 12 bits of to */
+	void __iomem *virt_mem;
+	struct iommu_remapped_entry *mapped;
+
+	pfn = to >> VTD_PAGE_SHIFT;
+	offset = to & (~VTD_PAGE_MASK);
+
+	if (page_is_ram(pfn)) {
+		memcpy(pfn_to_kaddr(pfn) + offset, from, csize);
+	} else{
+		mapped = kzalloc(sizeof(struct iommu_remapped_entry),
+				GFP_KERNEL);
+		if (!mapped)
+			return -ENOMEM;
+
+		virt_mem = ioremap_cache((unsigned long)to, size);
+		if (!virt_mem) {
+			kfree(mapped);
+			return -ENOMEM;
+		}
+		memcpy(virt_mem, from, size);
+		mutex_lock(&__iommu_mem_list_lock);
+		mapped->mem = virt_mem;
+		list_add_tail(&mapped->list, &__iommu_remapped_mem);
+		mutex_unlock(&__iommu_mem_list_lock);
+	}
+	return size;
+}
+
+/*
+ * Free the mapped memory for ioremap;
+ */
+int __iommu_free_mapped_mem(void)
+{
+	struct iommu_remapped_entry *mem_entry, *tmp;
+
+	mutex_lock(&__iommu_mem_list_lock);
+	list_for_each_entry_safe(mem_entry, tmp, &__iommu_remapped_mem, list) {
+		iounmap(mem_entry->mem);
+		list_del(&mem_entry->list);
+		kfree(mem_entry);
+	}
+	mutex_unlock(&__iommu_mem_list_lock);
+	return 0;
+}
+/*
  * Interfaces for when a new domain in the crashdump kernel needs some
  * values from the panicked kernel's context entries
  *
diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h
index a65208a..8ffa523 100644
--- a/include/linux/intel-iommu.h
+++ b/include/linux/intel-iommu.h
@@ -26,6 +26,7 @@
 #include <linux/iova.h>
 #include <linux/io.h>
 #include <linux/dma_remapping.h>
+#include <linux/crash_dump.h>
 #include <asm/cacheflush.h>
 #include <asm/iommu.h>
 
@@ -368,4 +369,12 @@ extern int dmar_ir_support(void);
 
 extern const struct attribute_group *intel_iommu_groups[];
 
+#ifdef CONFIG_CRASH_DUMP
+extern int __iommu_load_from_oldmem(void *to, unsigned long from,
+					unsigned long size);
+extern int __iommu_save_to_oldmem(unsigned long to, void *from,
+					unsigned long size);
+extern int __iommu_free_mapped_mem(void);
+#endif /* CONFIG_CRASH_DUMP */
+
 #endif
-- 
2.0.0-rc0



From zhen-hual at hp.com  Tue Jan  6 20:04:29 2015
From: zhen-hual at hp.com (Li, Zhen-Hua)
Date: Wed,  7 Jan 2015 12:04:29 +0800
Subject: [PATCH v7 05/10] iommu/vt-d: Add functions to load and save old re
In-Reply-To: <1420603474-4235-1-git-send-email-zhen-hual@hp.com>
References: <1420603474-4235-1-git-send-email-zhen-hual@hp.com>
Message-ID: <1420603474-4235-6-git-send-email-zhen-hual@hp.com>

Add functions to load root entry table from old kernel, and to save updated
root entry table.
Add two member in struct intel_iommu, to store the RTA in old kernel, and
the mapped virt address of it.

We use the old RTA in dump kernel, and when the iommu->root_entry is used as
a cache in kdump kernel, its phys address will not be save to RTA register,
but when its data is changed, we will save the new data to old root entry table.

Li, Zhen-hua:
    The functions and logics.

Takao Indoh:
    Add __iommu_flush_cache.

Signed-off-by: Li, Zhen-Hua <zhen-hual at hp.com>
Signed-off-by: Takao Indoh <indou.takao at jp.fujitsu.com>
---
 drivers/iommu/intel-iommu.c | 53 +++++++++++++++++++++++++++++++++++++++++++++
 include/linux/intel-iommu.h |  5 +++++
 2 files changed, 58 insertions(+)

diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 8a7ad72..f3059b8 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -388,6 +388,10 @@ static int intel_iommu_get_dids_from_old_kernel(struct intel_iommu *iommu);
 
 static int device_to_domain_id(struct intel_iommu *iommu, u8 bus, u8 devfn);
 
+static void __iommu_load_old_root_entry(struct intel_iommu *iommu);
+
+static void __iommu_update_old_root_entry(struct intel_iommu *iommu, int index);
+
 struct iommu_remapped_entry {
 	struct list_head list;
 	void __iomem *mem;
@@ -4990,4 +4994,53 @@ static int device_to_domain_id(struct intel_iommu *iommu, u8 bus, u8 devfn)
 	return did;
 }
 
+/*
+ * Load the old root entry table to new root entry table.
+ */
+static void __iommu_load_old_root_entry(struct intel_iommu *iommu)
+{
+	if ((!iommu)
+		|| (!iommu->root_entry)
+		|| (!iommu->root_entry_old_virt)
+		|| (!iommu->root_entry_old_phys))
+		return;
+	memcpy(iommu->root_entry, iommu->root_entry_old_virt, PAGE_SIZE);
+
+	__iommu_flush_cache(iommu, iommu->root_entry, PAGE_SIZE);
+}
+
+/*
+ * When the data in new root entry table is changed, this function
+ * must be called to save the updated data to old root entry table.
+ */
+static void __iommu_update_old_root_entry(struct intel_iommu *iommu, int index)
+{
+	u8 start;
+	unsigned long size;
+	void __iomem *to;
+	void *from;
+
+	if ((!iommu)
+		|| (!iommu->root_entry)
+		|| (!iommu->root_entry_old_virt)
+		|| (!iommu->root_entry_old_phys))
+		return;
+
+	if (index < -1 || index >= ROOT_ENTRY_NR)
+		return;
+
+	if (index == -1) {
+		start = 0;
+		size = ROOT_ENTRY_NR * sizeof(struct root_entry);
+	} else {
+		start = index * sizeof(struct root_entry);
+		size = sizeof(struct root_entry);
+	}
+	to = iommu->root_entry_old_virt;
+	from = iommu->root_entry;
+	memcpy(to + start, from + start, size);
+
+	__iommu_flush_cache(iommu, to + start, size);
+}
+
 #endif /* CONFIG_CRASH_DUMP */
diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h
index 8ffa523..8e29b97 100644
--- a/include/linux/intel-iommu.h
+++ b/include/linux/intel-iommu.h
@@ -329,6 +329,11 @@ struct intel_iommu {
 	spinlock_t	lock; /* protect context, domain ids */
 	struct root_entry *root_entry; /* virtual address */
 
+#ifdef CONFIG_CRASH_DUMP
+	void __iomem *root_entry_old_virt; /* mapped from old root entry */
+	unsigned long root_entry_old_phys; /* root entry in old kernel */
+#endif
+
 	struct iommu_flush flush;
 #endif
 	struct q_inval  *qi;            /* Queued invalidation info */
-- 
2.0.0-rc0



From zhen-hual at hp.com  Tue Jan  6 20:04:30 2015
From: zhen-hual at hp.com (Li, Zhen-Hua)
Date: Wed,  7 Jan 2015 12:04:30 +0800
Subject: [PATCH v7 06/10] iommu/vt-d: datatypes and functions used for kdump
In-Reply-To: <1420603474-4235-1-git-send-email-zhen-hual@hp.com>
References: <1420603474-4235-1-git-send-email-zhen-hual@hp.com>
Message-ID: <1420603474-4235-7-git-send-email-zhen-hual@hp.com>

Populate it with support functions to copy iommu translation tables from
from the panicked kernel into the kdump kernel in the event of a crash.

Functions:
    malloc new context table and copy old context table to the new one.
    malloc new page table and copy old page table to the new one.

Bill Sumner:
    Original version, the creation of the data types and functions.

Li, Zhenhua:
    Minor change:
    Update the usage of context_get_* and context_put*, use context_*
    and context_set_* for replacement.
    Update the name of the function that copies root entry table.
    Use new function to copy old context entry tables and page tables.
    Use "unsigned long" for physical address.
    Change incorrect aw_shift[4] and a few comments in copy_context_entry().

Signed-off-by: Bill Sumner <billsumnerlinux at gmail.com>
Signed-off-by: Li, Zhen-Hua <zhen-hual at hp.com>
---
 drivers/iommu/intel-iommu.c | 543 ++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 543 insertions(+)

diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index f3059b8..d2c19a0 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -399,6 +399,62 @@ struct iommu_remapped_entry {
 static LIST_HEAD(__iommu_remapped_mem);
 static DEFINE_MUTEX(__iommu_mem_list_lock);
 
+/* ========================================================================
+ * Copy iommu translation tables from old kernel into new  kernel.
+ * Entry to this set of functions is: intel_iommu_load_translation_tables()
+ * ------------------------------------------------------------------------
+ */
+
+/*
+ * Lists of domain_values_entry to hold domain values found during the copy.
+ * One list for each iommu in g_number_of_iommus.
+ */
+static struct list_head *domain_values_list;
+
+
+#define RET_BADCOPY -1	/* Return-code: Cannot copy translate tables */
+
+/*
+ * Struct copy_page_addr_parms is used to allow copy_page_addr()
+ * to accumulate values across multiple calls and returns.
+ */
+struct copy_page_addr_parms {
+	u32 first;	/* flag: first-time  */
+	u32 last;	/* flag: last-time */
+	u32 bus;	/* last bus number we saw */
+	u32 devfn;	/* last devfn we saw */
+	u32 shift;	/* last shift we saw */
+	u64 pte;	/* Page Table Entry */
+	u64 next_addr;	/* next-expected page_addr */
+
+	u64 page_addr;	/* page_addr accumulating size */
+	u64 page_size;	/* page_size accumulated */
+
+	struct domain_values_entry *dve;	/* to accumulate iova ranges */
+};
+
+enum returns_from_copy_context_entry {
+RET_CCE_NOT_PRESENT = 1,
+RET_CCE_NEW_PAGE_TABLES,
+RET_CCE_PASS_THROUGH_1,
+RET_CCE_PASS_THROUGH_2,
+RET_CCE_RESERVED_VALUE,
+RET_CCE_PREVIOUS_DID
+};
+
+static int copy_context_entry(struct intel_iommu *iommu, u32 bus, u32 devfn,
+			      void *ppap, struct context_entry *ce);
+
+static int copy_context_entry_table(struct intel_iommu *iommu,
+				    u32 bus, void *ppap,
+				    unsigned long *context_new_p,
+				    unsigned long context_old_phys);
+
+static int copy_root_entry_table(struct intel_iommu *iommu, void *ppap);
+
+static int intel_iommu_load_translation_tables(struct dmar_drhd_unit *drhd,
+		int g_num_of_iommus);
+
 #endif /* CONFIG_CRASH_DUMP */
 
 /*
@@ -5043,4 +5099,491 @@ static void __iommu_update_old_root_entry(struct intel_iommu *iommu, int index)
 	__iommu_flush_cache(iommu, to + start, size);
 }
 
+/*
+ * constant for initializing instances of copy_page_addr_parms properly.
+ */
+static struct copy_page_addr_parms copy_page_addr_parms_init = {1, 0};
+
+
+
+/*
+ * Lowest-level function in the 'Copy Page Tables' set
+ * Called once for each page_addr present in an iommu page-address table.
+ *
+ * Because of the depth-first traversal of the page-tables by the
+ * higher-level functions that call 'copy_page_addr', all pages
+ * of a domain will be presented in ascending order of IO Virtual Address.
+ *
+ * This function accumulates each contiguous range of these IOVAs and
+ * reserves it within the proper domain in the crashdump kernel when a
+ * non-contiguous range is detected, as determined by any of the following:
+ * 1. a change in the bus or device owning the presented page
+ * 2. a change in the page-size of the presented page (parameter shift)
+ * 3. a change in the page-table entry of the presented page
+ * 4. a presented IOVA that does not match the expected next-page address
+ * 5. the 'last' flag is set, indicating that all IOVAs have been seen.
+ */
+static int copy_page_addr(u64 page_addr, u32 shift, u32 bus, u32 devfn,
+				u64 pte, struct domain_values_entry *dve,
+				void *parms)
+{
+	struct copy_page_addr_parms *ppap = parms;
+
+	u64 page_size = ((u64)1 << shift);	/* page_size */
+	u64 pfn_lo;				/* For reserving IOVA range */
+	u64 pfn_hi;				/* For reserving IOVA range */
+	struct iova *iova_p;			/* For reserving IOVA range */
+
+	if (!ppap) {
+		pr_err("ERROR: ppap is NULL: 0x%3.3x(%3.3d) DevFn: 0x%3.3x(%3.3d) Page: 0x%16.16llx Size: 0x%16.16llx(%lld)\n",
+			bus, bus, devfn, devfn,  page_addr,
+			page_size, page_size);
+		return 0;
+	}
+
+	/* If (only extending current addr range) */
+	if (ppap->first     == 0      &&
+	    ppap->last      == 0      &&
+	    ppap->bus       == bus    &&
+	    ppap->devfn     == devfn  &&
+	    ppap->shift     == shift  &&
+	    (ppap->pte & ~VTD_PAGE_MASK) == (pte & ~VTD_PAGE_MASK) &&
+	    ppap->next_addr == page_addr) {
+
+		/* Update page size and next-expected address */
+		ppap->next_addr += page_size;
+		ppap->page_size += page_size;
+		return 0;
+	}
+
+	if (!ppap->first) {
+		/* Close-out the accumulated IOVA address range */
+
+		if (!ppap->dve) {
+			pr_err("%s ERROR: ppap->dve is NULL -- needed to reserve range for B:D:F=%2.2x:%2.2x:%1.1x\n",
+				__func__,
+				ppap->bus, ppap->devfn >> 3, ppap->devfn & 0x7);
+			return RET_BADCOPY;
+		}
+		pfn_lo = IOVA_PFN(ppap->page_addr);
+		pfn_hi = IOVA_PFN(ppap->page_addr + ppap->page_size);
+		iova_p = reserve_iova(&ppap->dve->iovad, pfn_lo, pfn_hi);
+	}
+
+	/* Prepare for a new IOVA address range */
+	ppap->first     = 0;		/* Not first-time anymore */
+	ppap->bus       = bus;
+	ppap->devfn     = devfn;
+	ppap->shift     = shift;
+	ppap->pte       = pte;
+	ppap->next_addr = page_addr + page_size; /* Next-expected page_addr */
+
+	ppap->page_addr = page_addr;	/* Addr(new page) */
+	ppap->page_size = page_size;	/* Size(new page) */
+
+	ppap->dve	= dve;	/* adr(device_values_entry for new range) */
+
+	return 0;
+}
+
+/*
+ * Recursive function to copy the tree of page tables (max 6 recursions)
+ * Parameter 'shift' controls the recursion
+ */
+static int copy_page_table(unsigned long *dma_pte_new_p,
+			   unsigned long dma_pte_phys,
+			   u32 shift, u64 page_addr,
+			   struct intel_iommu *iommu,
+			   u32 bus, u32 devfn,
+			   struct domain_values_entry *dve, void *ppap)
+{
+	int ret;			/* Integer return code */
+	struct dma_pte *p;		/* Virtual adr(each entry) iterator */
+	struct dma_pte *pgt_new_virt;	/* Adr(dma_pte in new kernel) */
+	unsigned long dma_pte_next;	/* Adr(next table down)  */
+	u64 u;				/* index(each entry in page_table) */
+
+
+	/* If (already done all levels -- problem) */
+	if (shift < 12) {
+		pr_err("ERROR %s shift < 12 %lx\n", __func__, dma_pte_phys);
+		pr_err("shift %d, page_addr %16.16llu bus %3.3u devfn %3.3u\n",
+			shift, page_addr, bus, devfn);
+		return RET_BADCOPY;
+	}
+
+	/* allocate a page table in the new kernel
+	 * copy contents from old kernel
+	 * then update each entry in the table in the new kernel
+	 */
+
+	pgt_new_virt = (struct dma_pte *)alloc_pgtable_page(iommu->node);
+	if (!pgt_new_virt)
+		return -ENOMEM;
+
+	ret = __iommu_load_from_oldmem(pgt_new_virt,
+					dma_pte_phys,
+					VTD_PAGE_SIZE);
+
+	if (ret <= 0)
+		return ret;
+
+	for (u = 0, p = pgt_new_virt; u < 512; u++, p++) {
+
+		if (((p->val & DMA_PTE_READ) == 0) &&
+		    ((p->val & DMA_PTE_WRITE) == 0))
+			continue;
+
+		if (dma_pte_superpage(p) || (shift == 12)) {
+
+			ret = copy_page_addr(page_addr | (u << shift),
+				shift, bus, devfn, p->val, dve, ppap);
+			if (ret)
+				return ret;
+			continue;
+		}
+
+		ret = copy_page_table(&dma_pte_next,
+				(p->val & VTD_PAGE_MASK),
+				shift-9, page_addr | (u << shift),
+				iommu, bus, devfn, dve, ppap);
+		if (ret)
+			return ret;
+
+		p->val &= ~VTD_PAGE_MASK;	/* Clear old and set new pgd */
+		p->val |= ((u64)dma_pte_next & VTD_PAGE_MASK);
+	}
+
+	*dma_pte_new_p = virt_to_phys(pgt_new_virt);
+
+	return 0;
+}
+
+
+/*
+ * Called once for each context_entry found in a copied context_entry_table
+ * Each context_entry represents one PCIe device handled by the IOMMU.
+ *
+ * The 'domain_values_list' contains one 'domain_values_entry' for each
+ * unique domain-id found while copying the context entries for each iommu.
+ *
+ * The Intel-iommu spec. requires that every context_entry that contains
+ * the same domain-id point to the same set of page translation tables.
+ * The hardware uses this to improve the use of its translation cache.
+ * In order to insure that the copied translate tables abide by this
+ * requirement, this function keeps a list of domain-ids (dids) that
+ * have already been seen for this iommu. This function checks each entry
+ * already on the list for a domain-id that matches the domain-id in this
+ * context_entry.  If found, this function places the address of the previous
+ * context's tree of page translation tables into this context_entry.
+ * If a matching previous entry is not found, a new 'domain_values_entry'
+ * structure is created for the domain-id in this context_entry and
+ * copy_page_table is called to duplicate its tree of page tables.
+ */
+static int copy_context_entry(struct intel_iommu *iommu, u32 bus, u32 devfn,
+			      void *ppap, struct context_entry *ce)
+{
+	int ret = 0;			/* Integer Return Code */
+	u32 shift = 0;			/* bits to shift page_addr  */
+	u64 page_addr = 0;		/* Address of translated page */
+	unsigned long pgt_old_phys;	/* Adr(page_table in the old kernel) */
+	unsigned long pgt_new_phys;	/* Adr(page_table in the new kernel) */
+	u8  t;				/* Translation-type from context */
+	u8  aw;				/* Address-width from context */
+	u32 aw_shift[8] = {
+		12+9+9,		/* [000b] 30-bit AGAW (2-level page table) */
+		12+9+9+9,	/* [001b] 39-bit AGAW (3-level page table) */
+		12+9+9+9+9,	/* [010b] 48-bit AGAW (4-level page table) */
+		12+9+9+9+9+9,	/* [011b] 57-bit AGAW (5-level page table) */
+		12+9+9+9+9+9+7,	/* [100b] 64-bit AGAW (6-level page table) */
+		0,		/* [101b] Reserved */
+		0,		/* [110b] Reserved */
+		0,		/* [111b] Reserved */
+	};
+
+	struct domain_values_entry *dve = NULL;
+
+	if (!context_present(ce)) {	/* If (context not present) */
+		ret = RET_CCE_NOT_PRESENT;		/* Skip it */
+		goto exit;
+	}
+
+	t = context_translation_type(ce);
+	/* If we have seen this domain-id before on this iommu,
+	 * give this context the same page-tables and we are done.
+	 */
+	list_for_each_entry(dve, &domain_values_list[iommu->seq_id], link) {
+		if (dve->did == (int) context_domain_id(ce)) {
+			switch (t) {
+			case 0:	/* page tables */
+			case 1:	/* page tables */
+				context_set_address_root(ce,
+						virt_to_phys(dve->pgd));
+				ret = RET_CCE_PREVIOUS_DID;
+				break;
+
+			case 2:	/* Pass through */
+				if (dve->pgd == NULL)
+					ret =  RET_CCE_PASS_THROUGH_2;
+				else
+					ret = RET_BADCOPY;
+				break;
+
+			default: /* Bad value of 't'*/
+				ret = RET_BADCOPY;
+				break;
+			}
+			goto exit;
+		}
+	}
+
+	/* Since we now know that this is a new domain-id for this iommu,
+	 * create a new entry, add it to the list, and handle its
+	 * page tables.
+	 */
+
+	dve = kcalloc(1, sizeof(struct domain_values_entry), GFP_KERNEL);
+	if (!dve) {
+		ret = -ENOMEM;
+		goto exit;
+	}
+
+	dve->did = (int) context_domain_id(ce);
+	dve->gaw = (int) agaw_to_width(context_address_width(ce));
+	dve->pgd = NULL;
+	init_iova_domain(&dve->iovad, DMA_32BIT_PFN);
+
+	list_add(&dve->link, &domain_values_list[iommu->seq_id]);
+
+
+	if (t == 0 || t == 1) {		/* If (context has page tables) */
+		aw = context_address_width(ce);
+		shift = aw_shift[aw];
+
+		pgt_old_phys = context_address_root(ce) << VTD_PAGE_SHIFT;
+
+		ret = copy_page_table(&pgt_new_phys, pgt_old_phys,
+			shift-9, page_addr, iommu, bus, devfn, dve, ppap);
+
+		__iommu_flush_cache(iommu, phys_to_virt(pgt_new_phys),
+				VTD_PAGE_SIZE);
+
+		if (ret)		/* if (problem) bail out */
+			goto exit;
+
+		context_set_address_root(ce, pgt_new_phys);
+		dve->pgd = phys_to_virt(pgt_new_phys);
+		ret = RET_CCE_NEW_PAGE_TABLES;
+		goto exit;
+	}
+
+	if (t == 2) {		/* If (Identity mapped pass-through) */
+		ret = RET_CCE_PASS_THROUGH_1;	/* REVISIT: Skip for now */
+		goto exit;
+	}
+
+	ret = RET_CCE_RESERVED_VALUE;	/* Else ce->t is a Reserved value */
+	/* Note fall-through */
+
+exit:	/* all returns come through here to insure good clean-up */
+	return ret;
+}
+
+
+/*
+ * Called once for each context_entry_table found in the root_entry_table
+ */
+static int copy_context_entry_table(struct intel_iommu *iommu,
+				    u32 bus, void *ppap,
+				    unsigned long *context_new_p,
+				    unsigned long context_old_phys)
+{
+	int ret = 0;				/* Integer return code */
+	struct context_entry *ce;		/* Iterator */
+	unsigned long context_new_phys;		/* adr(table in new kernel) */
+	struct context_entry *context_new_virt;	/* adr(table in new kernel) */
+	u32 devfn = 0;				/* PCI Device & function */
+
+	/* allocate a context-entry table in the new kernel
+	 * copy contents from old kernel
+	 * then update each entry in the table in the new kernel
+	 */
+	context_new_virt =
+		(struct context_entry *)alloc_pgtable_page(iommu->node);
+	if (!context_new_virt)
+		return -ENOMEM;
+
+	context_new_phys = virt_to_phys(context_new_virt);
+
+	__iommu_load_from_oldmem(context_new_virt,
+					context_old_phys,
+					VTD_PAGE_SIZE);
+
+	for (devfn = 0, ce = context_new_virt; devfn < 256; devfn++, ce++) {
+
+		if (!context_present(ce))	/* If (context not present) */
+			continue;		/* Skip it */
+
+		ret = copy_context_entry(iommu, bus, devfn, ppap, ce);
+		if (ret < 0)		/* if (problem) */
+			return RET_BADCOPY;
+
+		switch (ret) {
+		case RET_CCE_NOT_PRESENT:
+			continue;
+		case RET_CCE_NEW_PAGE_TABLES:
+			continue;
+		case RET_CCE_PASS_THROUGH_1:
+			continue;
+		case RET_CCE_PASS_THROUGH_2:
+			continue;
+		case RET_CCE_RESERVED_VALUE:
+			return RET_BADCOPY;
+		case RET_CCE_PREVIOUS_DID:
+			continue;
+		default:
+			return RET_BADCOPY;
+		};
+	}
+
+	*context_new_p = context_new_phys;
+	return 0;
+}
+
+
+/*
+ * Highest-level function in the 'copy translation tables' set of functions
+ */
+static int copy_root_entry_table(struct intel_iommu *iommu, void *ppap)
+{
+	int ret = 0;				/* Integer return code */
+	u32 bus;				/* Index: root-entry-table */
+	struct root_entry  *re;			/* Virt(iterator: new table) */
+	unsigned long context_old_phys;	/* Phys(context table entry) */
+	unsigned long context_new_phys;	/* Phys(new context_entry) */
+
+	/*
+	 * allocate a root-entry table in the new kernel
+	 * copy contents from old kernel
+	 * then update each entry in the table in the new kernel
+	 */
+
+	if (!iommu->root_entry_old_phys)
+		return -ENOMEM;
+
+	for (bus = 0, re = iommu->root_entry; bus < 256; bus += 1, re += 1) {
+		if (!root_present(re))
+			continue;
+
+		context_old_phys = get_context_phys_from_root(re);
+
+		if (!context_old_phys)
+			continue;
+
+		context_new_phys = 0;
+		ret = copy_context_entry_table(iommu, bus, ppap,
+						&context_new_phys,
+						context_old_phys);
+		__iommu_flush_cache(iommu,
+				phys_to_virt(context_new_phys),
+				VTD_PAGE_SIZE);
+
+		if (ret)
+			return ret;
+
+		set_root_value(re, context_new_phys);
+	}
+
+	return 0;
+}
+/*
+ * Interface to the "copy translation tables" set of functions
+ * from mainline code.
+ */
+static int intel_iommu_load_translation_tables(struct dmar_drhd_unit *drhd,
+		int g_num_of_iommus)
+{
+	struct intel_iommu *iommu;	/* Virt(iommu hardware registers) */
+	unsigned long long q;		/* quadword scratch */
+	int ret = 0;			/* Integer return code */
+	int i = 0;			/* Loop index */
+	unsigned long flags;
+
+	/* Structure so copy_page_addr() can accumulate things
+	 * over multiple calls and returns
+	 */
+	struct copy_page_addr_parms ppa_parms = copy_page_addr_parms_init;
+	struct copy_page_addr_parms *ppap = &ppa_parms;
+
+
+	iommu = drhd->iommu;
+	q = dmar_readq(iommu->reg + DMAR_RTADDR_REG);
+	if (!q)
+		return -1;
+
+	/* If (list needs initializing) do it here */
+	if (!domain_values_list) {
+		domain_values_list =
+			 kcalloc(g_num_of_iommus, sizeof(struct list_head),
+					GFP_KERNEL);
+
+		if (!domain_values_list) {
+			pr_err("Allocation failed for domain_values_list array\n");
+			return -ENOMEM;
+		}
+		for (i = 0; i < g_num_of_iommus; i++)
+			INIT_LIST_HEAD(&domain_values_list[i]);
+	}
+
+	spin_lock_irqsave(&iommu->lock, flags);
+
+	/* Load the root-entry table from the old kernel
+	 * foreach context_entry_table in root_entry
+	 *    foreach context_entry in context_entry_table
+	 *       foreach level-1 page_table_entry in context_entry
+	 *          foreach level-2 page_table_entry in level 1 page_table_entry
+	 *             Above pattern continues up to 6 levels of page tables
+	 *                Sanity-check the entry
+	 *                Process the bus, devfn, page_address, page_size
+	 */
+	if (!iommu->root_entry) {
+		iommu->root_entry =
+			(struct root_entry *)alloc_pgtable_page(iommu->node);
+		if (!iommu->root_entry) {
+			spin_unlock_irqrestore(&iommu->lock, flags);
+			return -ENOMEM;
+		}
+	}
+
+	iommu->root_entry_old_phys = q & VTD_PAGE_MASK;
+	if (!iommu->root_entry_old_phys) {
+		pr_err("Could not read old root entry address.");
+		return -1;
+	}
+
+	iommu->root_entry_old_virt = ioremap_cache(iommu->root_entry_old_phys,
+						VTD_PAGE_SIZE);
+	if (!iommu->root_entry_old_virt) {
+		pr_err("Could not map the old root entry.");
+		return -ENOMEM;
+	}
+
+	__iommu_load_old_root_entry(iommu);
+	ret = copy_root_entry_table(iommu, ppap);
+	__iommu_flush_cache(iommu, iommu->root_entry, PAGE_SIZE);
+	__iommu_update_old_root_entry(iommu, -1);
+
+	spin_unlock_irqrestore(&iommu->lock, flags);
+
+	__iommu_free_mapped_mem();
+
+	if (ret)
+		return ret;
+
+	ppa_parms.last = 1;
+	copy_page_addr(0, 0, 0, 0, 0, NULL, ppap);
+
+	return 0;
+}
+
 #endif /* CONFIG_CRASH_DUMP */
-- 
2.0.0-rc0



From zhen-hual at hp.com  Tue Jan  6 20:04:31 2015
From: zhen-hual at hp.com (Li, Zhen-Hua)
Date: Wed,  7 Jan 2015 12:04:31 +0800
Subject: [PATCH v7 07/10] iommu/vt-d: enable kdump support in iommu module
In-Reply-To: <1420603474-4235-1-git-send-email-zhen-hual@hp.com>
References: <1420603474-4235-1-git-send-email-zhen-hual@hp.com>
Message-ID: <1420603474-4235-8-git-send-email-zhen-hual@hp.com>

Modify the operation of the following functions when called during crash dump:
    device_to_domain_id
    get_domain_for_dev
    init_dmars
    intel_iommu_init

Bill Sumner:
    Original version.

Zhenhua:
    Minor change,
    The name of new calling functions.
    Do not disable and re-enable TE in kdump kernel.

Signed-off-by: Bill Sumner <billsumnerlinux at gmail.com>
Signed-off-by: Li, Zhen-Hua <zhen-hual at hp.com>
---
 drivers/iommu/intel-iommu.c | 135 +++++++++++++++++++++++++++++++++++++++-----
 1 file changed, 120 insertions(+), 15 deletions(-)

diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index d2c19a0..8807710 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -907,6 +907,11 @@ static struct context_entry * device_to_context_entry(struct intel_iommu *iommu,
 		set_root_value(root, phy_addr);
 		set_root_present(root);
 		__iommu_flush_cache(iommu, root, sizeof(*root));
+
+#ifdef CONFIG_CRASH_DUMP
+		if (is_kdump_kernel())
+			__iommu_update_old_root_entry(iommu, bus);
+#endif
 	}
 	spin_unlock_irqrestore(&iommu->lock, flags);
 	return &context[devfn];
@@ -958,7 +963,8 @@ static void free_context_table(struct intel_iommu *iommu)
 
 	spin_lock_irqsave(&iommu->lock, flags);
 	if (!iommu->root_entry) {
-		goto out;
+		spin_unlock_irqrestore(&iommu->lock, flags);
+		return;
 	}
 	for (i = 0; i < ROOT_ENTRY_NR; i++) {
 		root = &iommu->root_entry[i];
@@ -966,10 +972,23 @@ static void free_context_table(struct intel_iommu *iommu)
 		if (context)
 			free_pgtable_page(context);
 	}
+
+#ifdef CONFIG_CRASH_DUMP
+	if (is_kdump_kernel()) {
+		iommu->root_entry_old_phys = 0;
+		root = iommu->root_entry_old_virt;
+		iommu->root_entry_old_virt = NULL;
+	}
+#endif
 	free_pgtable_page(iommu->root_entry);
 	iommu->root_entry = NULL;
-out:
+
 	spin_unlock_irqrestore(&iommu->lock, flags);
+
+#ifdef CONFIG_CRASH_DUMP
+	if (is_kdump_kernel())
+		iounmap(root);
+#endif
 }
 
 static struct dma_pte *pfn_to_dma_pte(struct dmar_domain *domain,
@@ -2381,6 +2400,9 @@ static struct dmar_domain *get_domain_for_dev(struct device *dev, int gaw)
 	unsigned long flags;
 	u8 bus, devfn;
 	int did = -1;   /* Default to "no domain_id supplied" */
+#ifdef CONFIG_CRASH_DUMP
+	struct domain_values_entry *dve = NULL;
+#endif /* CONFIG_CRASH_DUMP */
 
 	domain = find_domain(dev);
 	if (domain)
@@ -2414,6 +2436,24 @@ static struct dmar_domain *get_domain_for_dev(struct device *dev, int gaw)
 	domain = alloc_domain(0);
 	if (!domain)
 		return NULL;
+
+#ifdef CONFIG_CRASH_DUMP
+	if (is_kdump_kernel()) {
+		/*
+		 * if this device had a did in the old kernel
+		 * use its values instead of generating new ones
+		 */
+		did = device_to_domain_id(iommu, bus, devfn);
+		if (did > 0 || (did == 0 && !cap_caching_mode(iommu->cap)))
+			dve = intel_iommu_did_to_domain_values_entry(did,
+								iommu);
+		if (dve)
+			gaw = dve->gaw;
+		else
+			did = -1;
+	}
+#endif /* CONFIG_CRASH_DUMP */
+
 	domain->id = iommu_attach_domain(domain, iommu, did);
 	if (domain->id < 0) {
 		free_domain_mem(domain);
@@ -2425,6 +2465,18 @@ static struct dmar_domain *get_domain_for_dev(struct device *dev, int gaw)
 		return NULL;
 	}
 
+#ifdef CONFIG_CRASH_DUMP
+	if (is_kdump_kernel() && dve) {
+
+		if (domain->pgd)
+			free_pgtable_page(domain->pgd);
+
+		domain->pgd = dve->pgd;
+
+		copy_reserved_iova(&dve->iovad, &domain->iovad);
+	}
+#endif /* CONFIG_CRASH_DUMP */
+
 	/* register PCI DMA alias device */
 	if (dev_is_pci(dev)) {
 		tmp = dmar_insert_dev_info(iommu, PCI_BUS_NUM(dma_alias),
@@ -2948,14 +3000,35 @@ static int __init init_dmars(void)
 		if (ret)
 			goto free_iommu;
 
-		/*
-		 * TBD:
-		 * we could share the same root & context tables
-		 * among all IOMMU's. Need to Split it later.
-		 */
-		ret = iommu_alloc_root_entry(iommu);
-		if (ret)
-			goto free_iommu;
+#ifdef CONFIG_CRASH_DUMP
+		if (is_kdump_kernel()) {
+			pr_info("IOMMU Copying translate tables from panicked kernel\n");
+			ret = intel_iommu_load_translation_tables(drhd,
+					g_num_of_iommus);
+			if (ret) {
+				pr_err("IOMMU: Copy translate tables failed\n");
+
+				/* Best to stop trying */
+				goto free_iommu;
+			}
+			pr_info("IOMMU: root_cache:0x%12.12llx phys:0x%12.12llx\n",
+				(u64)iommu->root_entry,
+				(u64)iommu->root_entry_old_phys);
+			intel_iommu_get_dids_from_old_kernel(iommu);
+		} else {
+#endif /* CONFIG_CRASH_DUMP */
+			/*
+			 * TBD:
+			 * we could share the same root & context tables
+			 * among all IOMMU's. Need to Split it later.
+			 */
+			ret = iommu_alloc_root_entry(iommu);
+			if (ret)
+				goto free_iommu;
+#ifdef CONFIG_CRASH_DUMP
+		}
+#endif
+
 		if (!ecap_pass_through(iommu->ecap))
 			hw_pass_through = 0;
 	}
@@ -2972,6 +3045,16 @@ static int __init init_dmars(void)
 
 	check_tylersburg_isoch();
 
+#ifdef CONFIG_CRASH_DUMP
+	/*
+	 * In the crashdump kernel: Skip setting-up new domains for
+	 * si, rmrr, and the isa bus on the expectation that these
+	 * translations were copied from the old kernel.
+	 */
+	if (is_kdump_kernel())
+		goto skip_new_domains_for_si_rmrr_isa;
+#endif /* CONFIG_CRASH_DUMP */
+
 	/*
 	 * If pass through is not set or not enabled, setup context entries for
 	 * identity mappings for rmrr, gfx, and isa and may fall back to static
@@ -3012,6 +3095,10 @@ static int __init init_dmars(void)
 
 	iommu_prepare_isa();
 
+#ifdef CONFIG_CRASH_DUMP
+skip_new_domains_for_si_rmrr_isa:;
+#endif /* CONFIG_CRASH_DUMP */
+
 	/*
 	 * for each drhd
 	 *   enable fault log
@@ -3040,7 +3127,15 @@ static int __init init_dmars(void)
 
 		iommu->flush.flush_context(iommu, 0, 0, 0, DMA_CCMD_GLOBAL_INVL);
 		iommu->flush.flush_iotlb(iommu, 0, 0, 0, DMA_TLB_GLOBAL_FLUSH);
-		iommu_enable_translation(iommu);
+
+#ifdef CONFIG_CRASH_DUMP
+		if (is_kdump_kernel()) {
+			if (!(iommu->gcmd & DMA_GCMD_TE))
+				iommu_enable_translation(iommu);
+		} else
+#endif
+			iommu_enable_translation(iommu);
+
 		iommu_disable_protect_mem_regions(iommu);
 	}
 
@@ -4351,12 +4446,22 @@ int __init intel_iommu_init(void)
 		goto out_free_dmar;
 	}
 
+#ifdef CONFIG_CRASH_DUMP
 	/*
-	 * Disable translation if already enabled prior to OS handover.
+	 * If (This is the crash kernel)
+	 *    Set: copy iommu translate tables from old kernel
+	 *    Skip disabling the iommu hardware translations
 	 */
-	for_each_active_iommu(iommu, drhd)
-		if (iommu->gcmd & DMA_GCMD_TE)
-			iommu_disable_translation(iommu);
+	if (is_kdump_kernel()) {
+		pr_info("IOMMU Skip disabling iommu hardware translations\n");
+	} else
+#endif /* CONFIG_CRASH_DUMP */
+		/*
+		 * Disable translation if already enabled prior to OS handover.
+		 */
+		for_each_active_iommu(iommu, drhd)
+			if (iommu->gcmd & DMA_GCMD_TE)
+				iommu_disable_translation(iommu);
 
 	if (dmar_dev_scope_init() < 0) {
 		if (force_on)
-- 
2.0.0-rc0



From zhen-hual at hp.com  Tue Jan  6 20:04:32 2015
From: zhen-hual at hp.com (Li, Zhen-Hua)
Date: Wed,  7 Jan 2015 12:04:32 +0800
Subject: [PATCH v7 08/10] iommu/vt-d: assign new page table for dma_map
In-Reply-To: <1420603474-4235-1-git-send-email-zhen-hual@hp.com>
References: <1420603474-4235-1-git-send-email-zhen-hual@hp.com>
Message-ID: <1420603474-4235-9-git-send-email-zhen-hual@hp.com>

When a device driver issues the first dma_map command for a
device, we assign a new and empty page-table, thus removing all
mappings from the old kernel for the device.

Signed-off-by: Li, Zhen-Hua <zhen-hual at hp.com>
---
 drivers/iommu/intel-iommu.c | 56 ++++++++++++++++++++++++++++++++++++++-------
 1 file changed, 48 insertions(+), 8 deletions(-)

diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 8807710..57ae08b 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -44,6 +44,7 @@
 #include <asm/irq_remapping.h>
 #include <asm/cacheflush.h>
 #include <asm/iommu.h>
+#include <linux/dma-mapping.h>
 
 #include "irq_remapping.h"
 
@@ -455,6 +456,8 @@ static int copy_root_entry_table(struct intel_iommu *iommu, void *ppap);
 static int intel_iommu_load_translation_tables(struct dmar_drhd_unit *drhd,
 		int g_num_of_iommus);
 
+static void unmap_device_dma(struct dmar_domain *domain, struct device *dev);
+
 #endif /* CONFIG_CRASH_DUMP */
 
 /*
@@ -3196,14 +3199,30 @@ static struct dmar_domain *__get_valid_domain_for_dev(struct device *dev)
 		return NULL;
 	}
 
-	/* make sure context mapping is ok */
-	if (unlikely(!domain_context_mapped(dev))) {
-		ret = domain_context_mapping(domain, dev, CONTEXT_TT_MULTI_LEVEL);
-		if (ret) {
-			printk(KERN_ERR "Domain context map for %s failed",
-			       dev_name(dev));
-			return NULL;
-		}
+	/* if in kdump kernel, we need to unmap the mapped dma pages,
+	 * detach this device first.
+	 */
+	if (likely(domain_context_mapped(dev))) {
+#ifdef CONFIG_CRASH_DUMP
+		if (is_kdump_kernel()) {
+			unmap_device_dma(domain, dev);
+			domain = get_domain_for_dev(dev,
+				DEFAULT_DOMAIN_ADDRESS_WIDTH);
+			if (!domain) {
+				pr_err("Allocating domain for %s failed",
+				       dev_name(dev));
+				return NULL;
+			}
+		} else
+#endif
+			return domain;
+	}
+
+	ret = domain_context_mapping(domain, dev, CONTEXT_TT_MULTI_LEVEL);
+	if (ret) {
+		pr_err("Domain context map for %s failed",
+		       dev_name(dev));
+		return NULL;
 	}
 
 	return domain;
@@ -5691,4 +5710,25 @@ static int intel_iommu_load_translation_tables(struct dmar_drhd_unit *drhd,
 	return 0;
 }
 
+static void unmap_device_dma(struct dmar_domain *domain, struct device *dev)
+{
+	struct intel_iommu *iommu;
+	struct context_entry *ce;
+	struct iova *iova;
+	u8 bus, devfn;
+	phys_addr_t phys_addr;
+	dma_addr_t dev_addr;
+
+	iommu = device_to_iommu(dev, &bus, &devfn);
+	ce = device_to_context_entry(iommu, bus, devfn);
+	phys_addr = context_address_root(ce) << VTD_PAGE_SHIFT;
+	dev_addr = phys_to_dma(dev, phys_addr);
+
+	iova = find_iova(&domain->iovad, IOVA_PFN(dev_addr));
+	if (iova)
+		intel_unmap(dev, dev_addr);
+
+	domain_remove_one_dev_info(domain, dev);
+}
+
 #endif /* CONFIG_CRASH_DUMP */
-- 
2.0.0-rc0



From zhen-hual at hp.com  Tue Jan  6 20:04:33 2015
From: zhen-hual at hp.com (Li, Zhen-Hua)
Date: Wed,  7 Jan 2015 12:04:33 +0800
Subject: [PATCH v7 09/10] iommu/vt-d: Copy functions for irte
In-Reply-To: <1420603474-4235-1-git-send-email-zhen-hual@hp.com>
References: <1420603474-4235-1-git-send-email-zhen-hual@hp.com>
Message-ID: <1420603474-4235-10-git-send-email-zhen-hual@hp.com>

Functions to copy the irte data from the old kernel into the kdump kernel.

Signed-off-by: Li, Zhen-Hua <zhen-hual at hp.com>
---
 drivers/iommu/intel_irq_remapping.c | 62 +++++++++++++++++++++++++++++++++++++
 include/linux/intel-iommu.h         |  4 +++
 2 files changed, 66 insertions(+)

diff --git a/drivers/iommu/intel_irq_remapping.c b/drivers/iommu/intel_irq_remapping.c
index a55b207..d37fd62 100644
--- a/drivers/iommu/intel_irq_remapping.c
+++ b/drivers/iommu/intel_irq_remapping.c
@@ -8,6 +8,7 @@
 #include <linux/irq.h>
 #include <linux/intel-iommu.h>
 #include <linux/acpi.h>
+#include <linux/crash_dump.h>
 #include <asm/io_apic.h>
 #include <asm/smp.h>
 #include <asm/cpu.h>
@@ -17,6 +18,11 @@
 
 #include "irq_remapping.h"
 
+#ifdef CONFIG_CRASH_DUMP
+static int __iommu_load_old_irte(struct intel_iommu *iommu);
+static int __iommu_update_old_irte(struct intel_iommu *iommu, int index);
+#endif /* CONFIG_CRASH_DUMP */
+
 struct ioapic_scope {
 	struct intel_iommu *iommu;
 	unsigned int id;
@@ -1296,3 +1302,59 @@ int dmar_ir_hotplug(struct dmar_drhd_unit *dmaru, bool insert)
 
 	return ret;
 }
+
+#ifdef CONFIG_CRASH_DUMP
+
+static int __iommu_load_old_irte(struct intel_iommu *iommu)
+{
+	if ((!iommu)
+		|| (!iommu->ir_table)
+		|| (!iommu->ir_table->base)
+		|| (!iommu->ir_table->base_old_phys)
+		|| (!iommu->ir_table->base_old_virt))
+		return -1;
+
+	memcpy(iommu->ir_table->base,
+		iommu->ir_table->base_old_virt,
+		INTR_REMAP_TABLE_ENTRIES*sizeof(struct irte));
+
+	__iommu_flush_cache(iommu, iommu->ir_table->base,
+		INTR_REMAP_TABLE_ENTRIES*sizeof(struct irte));
+
+	return 0;
+}
+
+static int __iommu_update_old_irte(struct intel_iommu *iommu, int index)
+{
+	int start;
+	unsigned long size;
+	void __iomem *to;
+	void *from;
+
+	if ((!iommu)
+		|| (!iommu->ir_table)
+		|| (!iommu->ir_table->base)
+		|| (!iommu->ir_table->base_old_phys)
+		|| (!iommu->ir_table->base_old_virt))
+		return -1;
+
+	if (index < -1 || index >= INTR_REMAP_TABLE_ENTRIES)
+		return -1;
+
+	if (index == -1) {
+		start = 0;
+		size = INTR_REMAP_TABLE_ENTRIES * sizeof(struct irte);
+	} else {
+		start = index * sizeof(struct irte);
+		size = sizeof(struct irte);
+	}
+
+	to = iommu->ir_table->base_old_virt;
+	from = iommu->ir_table->base;
+	memcpy(to + start, from + start, size);
+
+	__iommu_flush_cache(iommu, to + start, size);
+
+	return 0;
+}
+#endif /* CONFIG_CRASH_DUMP */
diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h
index 8e29b97..76c6ea5 100644
--- a/include/linux/intel-iommu.h
+++ b/include/linux/intel-iommu.h
@@ -290,6 +290,10 @@ struct q_inval {
 struct ir_table {
 	struct irte *base;
 	unsigned long *bitmap;
+#ifdef CONFIG_CRASH_DUMP
+	void __iomem *base_old_virt;
+	unsigned long base_old_phys;
+#endif
 };
 #endif
 
-- 
2.0.0-rc0



From zhen-hual at hp.com  Tue Jan  6 20:04:34 2015
From: zhen-hual at hp.com (Li, Zhen-Hua)
Date: Wed,  7 Jan 2015 12:04:34 +0800
Subject: [PATCH v7 10/10] iommu/vt-d: Use old irte in kdump kernel
In-Reply-To: <1420603474-4235-1-git-send-email-zhen-hual@hp.com>
References: <1420603474-4235-1-git-send-email-zhen-hual@hp.com>
Message-ID: <1420603474-4235-11-git-send-email-zhen-hual@hp.com>

Fix the intr-remapping fault.

[1.594890] dmar: DRHD: handling fault status reg 2
[1.594894] dmar: INTR-REMAP: Request device [[41:00.0] fault index 4d
[1.594894] INTR-REMAP:[fault reason 34] Present field in the IRTE entry
is clear

Use old irte in kdump kernel, do not disable and re-enable interrupt
remapping.

Signed-off-by: Li, Zhen-Hua <zhen-hual at hp.com>
---
 drivers/iommu/intel_irq_remapping.c | 42 ++++++++++++++++++++++++++++++++-----
 1 file changed, 37 insertions(+), 5 deletions(-)

diff --git a/drivers/iommu/intel_irq_remapping.c b/drivers/iommu/intel_irq_remapping.c
index d37fd62..58356cb 100644
--- a/drivers/iommu/intel_irq_remapping.c
+++ b/drivers/iommu/intel_irq_remapping.c
@@ -198,6 +198,11 @@ static int modify_irte(int irq, struct irte *irte_modified)
 
 	set_64bit(&irte->low, irte_modified->low);
 	set_64bit(&irte->high, irte_modified->high);
+
+#ifdef CONFIG_CRASH_DUMP
+	if (is_kdump_kernel())
+		__iommu_update_old_irte(iommu, index);
+#endif
 	__iommu_flush_cache(iommu, irte, sizeof(*irte));
 
 	rc = qi_flush_iec(iommu, index, 0);
@@ -259,6 +264,11 @@ static int clear_entries(struct irq_2_iommu *irq_iommu)
 	bitmap_release_region(iommu->ir_table->bitmap, index,
 			      irq_iommu->irte_mask);
 
+#ifdef CONFIG_CRASH_DUMP
+	if (is_kdump_kernel())
+		__iommu_update_old_irte(iommu, -1);
+#endif
+
 	return qi_flush_iec(iommu, index, irq_iommu->irte_mask);
 }
 
@@ -640,11 +650,20 @@ static int __init intel_enable_irq_remapping(void)
 		 */
 		dmar_fault(-1, iommu);
 
-		/*
-		 * Disable intr remapping and queued invalidation, if already
-		 * enabled prior to OS handover.
-		 */
-		iommu_disable_irq_remapping(iommu);
+#ifdef CONFIG_CRASH_DUMP
+		if (is_kdump_kernel()) {
+			/* Do notdisable irq and then re-enable again. */
+		} else {
+#endif
+			/*
+			 * Disable intr remapping and queued invalidation,
+			 * if already enabled prior to OS handover.
+			 */
+			iommu_disable_irq_remapping(iommu);
+
+#ifdef CONFIG_CRASH_DUMP
+		}
+#endif
 
 		dmar_disable_qi(iommu);
 	}
@@ -687,7 +706,20 @@ static int __init intel_enable_irq_remapping(void)
 		if (intel_setup_irq_remapping(iommu))
 			goto error;
 
+#ifdef CONFIG_CRASH_DUMP
+	if (is_kdump_kernel()) {
+		unsigned long long q;
+
+		q = dmar_readq(iommu->reg + DMAR_IRTA_REG);
+		iommu->ir_table->base_old_phys = q & VTD_PAGE_MASK;
+		iommu->ir_table->base_old_virt = ioremap_cache(
+				iommu->ir_table->base_old_phys,
+				INTR_REMAP_TABLE_ENTRIES*sizeof(struct irte));
+		__iommu_load_old_irte(iommu);
+	} else
+#endif
 		iommu_set_irq_remapping(iommu, eim);
+
 		setup = 1;
 	}
 
-- 
2.0.0-rc0



From zhen-hual at hp.com  Tue Jan  6 20:11:21 2015
From: zhen-hual at hp.com (Li, ZhenHua)
Date: Wed, 07 Jan 2015 12:11:21 +0800
Subject: [PATCH v7 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel
In-Reply-To: <1420603474-4235-1-git-send-email-zhen-hual@hp.com>
References: <1420603474-4235-1-git-send-email-zhen-hual@hp.com>
Message-ID: <54ACB1E9.9070904@hp.com>

Many thanks to Takao Indoh and Baoquan He, for your testing on more
different systems.

The calling of flush functions are added to this version.

The usage of __iommu_flush_cache function :
1. Fixes a dump on Takao's system.
2. Reduces the count of faults on Baoquan's system.

Regards
Zhenhua

On 01/07/2015 12:04 PM, Li, Zhen-Hua wrote:
> This patchset is an update of Bill Sumner's patchset, implements a fix for:
> If a kernel boots with intel_iommu=on on a system that supports intel vt-d,
> when a panic happens, the kdump kernel will boot with these faults:
>
>      dmar: DRHD: handling fault status reg 102
>      dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr fff80000
>      DMAR:[fault reason 01] Present bit in root entry is clear
>
>      dmar: DRHD: handling fault status reg 2
>      dmar: INTR-REMAP: Request device [[61:00.0] fault index 42
>      INTR-REMAP:[fault reason 34] Present field in the IRTE entry is clear
>
> On some system, the interrupt remapping fault will also happen even if the
> intel_iommu is not set to on, because the interrupt remapping will be enabled
> when x2apic is needed by the system.
>
> The cause of the DMA fault is described in Bill's original version, and the
> INTR-Remap fault is caused by a similar reason. In short, the initialization
> of vt-d drivers causes the in-flight DMA and interrupt requests get wrong
> response.
>
> To fix this problem, we modifies the behaviors of the intel vt-d in the
> crashdump kernel:
>
> For DMA Remapping:
> 1. To accept the vt-d hardware in an active state,
> 2. Do not disable and re-enable the translation, keep it enabled.
> 3. Use the old root entry table, do not rewrite the RTA register.
> 4. Malloc and use new context entry table and page table, copy data from the
>     old ones that used by the old kernel.
> 5. to use different portions of the iova address ranges for the device drivers
>     in the crashdump kernel than the iova ranges that were in-use at the time
>     of the panic.
> 6. After device driver is loaded, when it issues the first dma_map command,
>     free the dmar_domain structure for this device, and generate a new one, so
>     that the device can be assigned a new and empty page table.
> 7. When a new context entry table is generated, we also save its address to
>     the old root entry table.
>
> For Interrupt Remapping:
> 1. To accept the vt-d hardware in an active state,
> 2. Do not disable and re-enable the interrupt remapping, keep it enabled.
> 3. Use the old interrupt remapping table, do not rewrite the IRTA register.
> 4. When ioapic entry is setup, the interrupt remapping table is changed, and
>     the updated data will be stored to the old interrupt remapping table.
>
> Advantages of this approach:
> 1. All manipulation of the IO-device is done by the Linux device-driver
>     for that device.
> 2. This approach behaves in a manner very similar to operation without an
>     active iommu.
> 3. Any activity between the IO-device and its RMRR areas is handled by the
>     device-driver in the same manner as during a non-kdump boot.
> 4. If an IO-device has no driver in the kdump kernel, it is simply left alone.
>     This supports the practice of creating a special kdump kernel without
>     drivers for any devices that are not required for taking a crashdump.
> 5. Minimal code-changes among the existing mainline intel vt-d code.
>
> Summary of changes in this patch set:
> 1. Added some useful function for root entry table in code intel-iommu.c
> 2. Added new members to struct root_entry and struct irte;
> 3. Functions to load old root entry table to iommu->root_entry from the memory
>     of old kernel.
> 4. Functions to malloc new context entry table and page table and copy the data
>     from the old ones to the malloced new ones.
> 5. Functions to enable support for DMA remapping in kdump kernel.
> 6. Functions to load old irte data from the old kernel to the kdump kernel.
> 7. Some code changes that support other behaviours that have been listed.
> 8. In the new functions, use physical address as "unsigned long" type, not
>     pointers.
>
> Original version by Bill Sumner:
>      https://lkml.org/lkml/2014/1/10/518
>      https://lkml.org/lkml/2014/4/15/716
>      https://lkml.org/lkml/2014/4/24/836
>
> Zhenhua's updates:
>      https://lkml.org/lkml/2014/10/21/134
>      https://lkml.org/lkml/2014/12/15/121
>      https://lkml.org/lkml/2014/12/22/53
>
> Changelog[v7]:
>      1. Use __iommu_flush_cache to flush the data to hardware.
>
> Changelog[v6]:
>      1. Use "unsigned long" as type of physical address.
>      2. Use new function unmap_device_dma to unmap the old dma.
>      3. Some small incorrect bits order for aw shift.
>
> Changelog[v5]:
>      1. Do not disable and re-enable traslation and interrupt remapping.
>      2. Use old root entry table.
>      3. Use old interrupt remapping table.
>      4. New functions to copy data from old kernel, and save to old kernel mem.
>      5. New functions to save updated root entry table and irte table.
>      6. Use intel_unmap to unmap the old dma;
>      7. Allocate new pages while driver is being loaded.
>
> Changelog[v4]:
>      1. Cut off the patches that move some defines and functions to new files.
>      2. Reduce the numbers of patches to five, make it more easier to read.
>      3. Changed the name of functions, make them consistent with current context
>         get/set functions.
>      4. Add change to function __iommu_attach_domain.
>
> Changelog[v3]:
>      1. Commented-out "#define DEBUG 1" to eliminate debug messages.
>      2. Updated the comments about changes in each version.
>      3. Fixed: one-line added to Copy-Translations patch to initialize the iovad
>            struct as recommended by Baoquan He [bhe at redhat.com]
>            init_iova_domain(&domain->iovad, DMA_32BIT_PFN);
>
> Changelog[v2]:
>      The following series implements a fix for:
>      A kdump problem about DMA that has been discussed for a long time. That is,
>      when a kernel panics and boots into the kdump kernel, DMA started by the
>      panicked kernel is not stopped before the kdump kernel is booted and the
>      kdump kernel disables the IOMMU while this DMA continues.  This causes the
>      IOMMU to stop translating the DMA addresses as IOVAs and begin to treat
>      them as physical memory addresses -- which causes the DMA to either:
>          (1) generate DMAR errors or
>          (2) generate PCI SERR errors or
>          (3) transfer data to or from incorrect areas of memory. Often this
>              causes the dump to fail.
>
> Changelog[v1]:
>      The original version.
>
> Changed in this version:
> 1. Do not disable and re-enable traslation and interrupt remapping.
> 2. Use old root entry table.
> 3. Use old interrupt remapping table.
> 4. Use "unsigned long" as physical address.
> 5. Use intel_unmap to unmap the old dma;
>
> Baoquan He <bhe at redhat.com> helps testing this patchset.
>
>    iommu/vt-d: Update iommu_attach_domain() and its callers
>    iommu/vt-d: Items required for kdump
>    iommu/vt-d: Add domain-id functions
>    iommu/vt-d: functions to copy data from old mem
>    iommu/vt-d: Add functions to load and save old re
>    iommu/vt-d: datatypes and functions used for kdump
>    iommu/vt-d: enable kdump support in iommu module
>    iommu/vt-d: assign new page table for dma_map
>    iommu/vt-d: Copy functions for irte
>    iommu/vt-d: Use old irte in kdump kernel
>
> Signed-off-by: Bill Sumner <billsumnerlinux at gmail.com>
> Signed-off-by: Li, Zhen-Hua <zhen-hual at hp.com>
> Signed-off-by: Takao Indoh <indou.takao at jp.fujitsu.com>
> Tested-by: Baoquan He <bhe at redhat.com>
> ---
>   drivers/iommu/intel-iommu.c         | 1050 +++++++++++++++++++++++++++++++++--
>   drivers/iommu/intel_irq_remapping.c |  104 +++-
>   include/linux/intel-iommu.h         |   18 +
>   3 files changed, 1130 insertions(+), 42 deletions(-)
>



From bhe at redhat.com  Tue Jan  6 21:02:47 2015
From: bhe at redhat.com (Baoquan He)
Date: Wed, 7 Jan 2015 13:02:47 +0800
Subject: [PATCH v7 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel
In-Reply-To: <54ACB1E9.9070904@hp.com>
References: <1420603474-4235-1-git-send-email-zhen-hual@hp.com>
 <54ACB1E9.9070904@hp.com>
Message-ID: <20150107050247.GA16050@dhcp-16-116.nay.redhat.com>

On 01/07/15 at 12:11pm, Li, ZhenHua wrote:
> Many thanks to Takao Indoh and Baoquan He, for your testing on more
> different systems.
> 
> The calling of flush functions are added to this version.
> 
> The usage of __iommu_flush_cache function :
> 1. Fixes a dump on Takao's system.
> 2. Reduces the count of faults on Baoquan's system.

I am testing the version you sent to me yesterday afternoon. Is that
different with this patchset? I found your patchset man reserve a big
contiguous memory region under 896M, this will cause the crashkernel
reservation failed when I set crashkernel=320M. The reason I increase
the crashkerenl reservation to 320M is 256M is not enough and cause OOM
when that patchset is tested.

I am checking what happened.


Thanks
Baoquan

> 
> Regards
> Zhenhua
> 
> On 01/07/2015 12:04 PM, Li, Zhen-Hua wrote:
> >This patchset is an update of Bill Sumner's patchset, implements a fix for:
> >If a kernel boots with intel_iommu=on on a system that supports intel vt-d,
> >when a panic happens, the kdump kernel will boot with these faults:
> >
> >     dmar: DRHD: handling fault status reg 102
> >     dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr fff80000
> >     DMAR:[fault reason 01] Present bit in root entry is clear
> >
> >     dmar: DRHD: handling fault status reg 2
> >     dmar: INTR-REMAP: Request device [[61:00.0] fault index 42
> >     INTR-REMAP:[fault reason 34] Present field in the IRTE entry is clear
> >
> >On some system, the interrupt remapping fault will also happen even if the
> >intel_iommu is not set to on, because the interrupt remapping will be enabled
> >when x2apic is needed by the system.
> >
> >The cause of the DMA fault is described in Bill's original version, and the
> >INTR-Remap fault is caused by a similar reason. In short, the initialization
> >of vt-d drivers causes the in-flight DMA and interrupt requests get wrong
> >response.
> >
> >To fix this problem, we modifies the behaviors of the intel vt-d in the
> >crashdump kernel:
> >
> >For DMA Remapping:
> >1. To accept the vt-d hardware in an active state,
> >2. Do not disable and re-enable the translation, keep it enabled.
> >3. Use the old root entry table, do not rewrite the RTA register.
> >4. Malloc and use new context entry table and page table, copy data from the
> >    old ones that used by the old kernel.
> >5. to use different portions of the iova address ranges for the device drivers
> >    in the crashdump kernel than the iova ranges that were in-use at the time
> >    of the panic.
> >6. After device driver is loaded, when it issues the first dma_map command,
> >    free the dmar_domain structure for this device, and generate a new one, so
> >    that the device can be assigned a new and empty page table.
> >7. When a new context entry table is generated, we also save its address to
> >    the old root entry table.
> >
> >For Interrupt Remapping:
> >1. To accept the vt-d hardware in an active state,
> >2. Do not disable and re-enable the interrupt remapping, keep it enabled.
> >3. Use the old interrupt remapping table, do not rewrite the IRTA register.
> >4. When ioapic entry is setup, the interrupt remapping table is changed, and
> >    the updated data will be stored to the old interrupt remapping table.
> >
> >Advantages of this approach:
> >1. All manipulation of the IO-device is done by the Linux device-driver
> >    for that device.
> >2. This approach behaves in a manner very similar to operation without an
> >    active iommu.
> >3. Any activity between the IO-device and its RMRR areas is handled by the
> >    device-driver in the same manner as during a non-kdump boot.
> >4. If an IO-device has no driver in the kdump kernel, it is simply left alone.
> >    This supports the practice of creating a special kdump kernel without
> >    drivers for any devices that are not required for taking a crashdump.
> >5. Minimal code-changes among the existing mainline intel vt-d code.
> >
> >Summary of changes in this patch set:
> >1. Added some useful function for root entry table in code intel-iommu.c
> >2. Added new members to struct root_entry and struct irte;
> >3. Functions to load old root entry table to iommu->root_entry from the memory
> >    of old kernel.
> >4. Functions to malloc new context entry table and page table and copy the data
> >    from the old ones to the malloced new ones.
> >5. Functions to enable support for DMA remapping in kdump kernel.
> >6. Functions to load old irte data from the old kernel to the kdump kernel.
> >7. Some code changes that support other behaviours that have been listed.
> >8. In the new functions, use physical address as "unsigned long" type, not
> >    pointers.
> >
> >Original version by Bill Sumner:
> >     https://lkml.org/lkml/2014/1/10/518
> >     https://lkml.org/lkml/2014/4/15/716
> >     https://lkml.org/lkml/2014/4/24/836
> >
> >Zhenhua's updates:
> >     https://lkml.org/lkml/2014/10/21/134
> >     https://lkml.org/lkml/2014/12/15/121
> >     https://lkml.org/lkml/2014/12/22/53
> >
> >Changelog[v7]:
> >     1. Use __iommu_flush_cache to flush the data to hardware.
> >
> >Changelog[v6]:
> >     1. Use "unsigned long" as type of physical address.
> >     2. Use new function unmap_device_dma to unmap the old dma.
> >     3. Some small incorrect bits order for aw shift.
> >
> >Changelog[v5]:
> >     1. Do not disable and re-enable traslation and interrupt remapping.
> >     2. Use old root entry table.
> >     3. Use old interrupt remapping table.
> >     4. New functions to copy data from old kernel, and save to old kernel mem.
> >     5. New functions to save updated root entry table and irte table.
> >     6. Use intel_unmap to unmap the old dma;
> >     7. Allocate new pages while driver is being loaded.
> >
> >Changelog[v4]:
> >     1. Cut off the patches that move some defines and functions to new files.
> >     2. Reduce the numbers of patches to five, make it more easier to read.
> >     3. Changed the name of functions, make them consistent with current context
> >        get/set functions.
> >     4. Add change to function __iommu_attach_domain.
> >
> >Changelog[v3]:
> >     1. Commented-out "#define DEBUG 1" to eliminate debug messages.
> >     2. Updated the comments about changes in each version.
> >     3. Fixed: one-line added to Copy-Translations patch to initialize the iovad
> >           struct as recommended by Baoquan He [bhe at redhat.com]
> >           init_iova_domain(&domain->iovad, DMA_32BIT_PFN);
> >
> >Changelog[v2]:
> >     The following series implements a fix for:
> >     A kdump problem about DMA that has been discussed for a long time. That is,
> >     when a kernel panics and boots into the kdump kernel, DMA started by the
> >     panicked kernel is not stopped before the kdump kernel is booted and the
> >     kdump kernel disables the IOMMU while this DMA continues.  This causes the
> >     IOMMU to stop translating the DMA addresses as IOVAs and begin to treat
> >     them as physical memory addresses -- which causes the DMA to either:
> >         (1) generate DMAR errors or
> >         (2) generate PCI SERR errors or
> >         (3) transfer data to or from incorrect areas of memory. Often this
> >             causes the dump to fail.
> >
> >Changelog[v1]:
> >     The original version.
> >
> >Changed in this version:
> >1. Do not disable and re-enable traslation and interrupt remapping.
> >2. Use old root entry table.
> >3. Use old interrupt remapping table.
> >4. Use "unsigned long" as physical address.
> >5. Use intel_unmap to unmap the old dma;
> >
> >Baoquan He <bhe at redhat.com> helps testing this patchset.
> >
> >   iommu/vt-d: Update iommu_attach_domain() and its callers
> >   iommu/vt-d: Items required for kdump
> >   iommu/vt-d: Add domain-id functions
> >   iommu/vt-d: functions to copy data from old mem
> >   iommu/vt-d: Add functions to load and save old re
> >   iommu/vt-d: datatypes and functions used for kdump
> >   iommu/vt-d: enable kdump support in iommu module
> >   iommu/vt-d: assign new page table for dma_map
> >   iommu/vt-d: Copy functions for irte
> >   iommu/vt-d: Use old irte in kdump kernel
> >
> >Signed-off-by: Bill Sumner <billsumnerlinux at gmail.com>
> >Signed-off-by: Li, Zhen-Hua <zhen-hual at hp.com>
> >Signed-off-by: Takao Indoh <indou.takao at jp.fujitsu.com>
> >Tested-by: Baoquan He <bhe at redhat.com>
> >---
> >  drivers/iommu/intel-iommu.c         | 1050 +++++++++++++++++++++++++++++++++--
> >  drivers/iommu/intel_irq_remapping.c |  104 +++-
> >  include/linux/intel-iommu.h         |   18 +
> >  3 files changed, 1130 insertions(+), 42 deletions(-)
> >
> 


From zhen-hual at hp.com  Tue Jan  6 21:25:51 2015
From: zhen-hual at hp.com (Li, ZhenHua)
Date: Wed, 07 Jan 2015 13:25:51 +0800
Subject: [PATCH v7 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel
In-Reply-To: <20150107050247.GA16050@dhcp-16-116.nay.redhat.com>
References: <1420603474-4235-1-git-send-email-zhen-hual@hp.com>
 <54ACB1E9.9070904@hp.com> <20150107050247.GA16050@dhcp-16-116.nay.redhat.com>
Message-ID: <54ACC35F.1070007@hp.com>

It is same as the last one I send to you yesterday.

The continuous memory that needed for data in this patchset:
RE: PAGE_SIZE, 4096 Bytes;
IRTE: 65536 * 16 ; 1M Bytes;

It should use same memory as the old versions of this patchset. The 
changes for the last version do not need more memory.

Regards
Zhenhua

On 01/07/2015 01:02 PM, Baoquan He wrote:
> On 01/07/15 at 12:11pm, Li, ZhenHua wrote:
>> Many thanks to Takao Indoh and Baoquan He, for your testing on more
>> different systems.
>>
>> The calling of flush functions are added to this version.
>>
>> The usage of __iommu_flush_cache function :
>> 1. Fixes a dump on Takao's system.
>> 2. Reduces the count of faults on Baoquan's system.
>
> I am testing the version you sent to me yesterday afternoon. Is that
> different with this patchset? I found your patchset man reserve a big
> contiguous memory region under 896M, this will cause the crashkernel
> reservation failed when I set crashkernel=320M. The reason I increase
> the crashkerenl reservation to 320M is 256M is not enough and cause OOM
> when that patchset is tested.
>
> I am checking what happened.
>
>
> Thanks
> Baoquan
>
>>
>> Regards
>> Zhenhua
>>
>> On 01/07/2015 12:04 PM, Li, Zhen-Hua wrote:
>>> This patchset is an update of Bill Sumner's patchset, implements a fix for:
>>> If a kernel boots with intel_iommu=on on a system that supports intel vt-d,
>>> when a panic happens, the kdump kernel will boot with these faults:
>>>
>>>      dmar: DRHD: handling fault status reg 102
>>>      dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr fff80000
>>>      DMAR:[fault reason 01] Present bit in root entry is clear
>>>
>>>      dmar: DRHD: handling fault status reg 2
>>>      dmar: INTR-REMAP: Request device [[61:00.0] fault index 42
>>>      INTR-REMAP:[fault reason 34] Present field in the IRTE entry is clear
>>>
>>> On some system, the interrupt remapping fault will also happen even if the
>>> intel_iommu is not set to on, because the interrupt remapping will be enabled
>>> when x2apic is needed by the system.
>>>
>>> The cause of the DMA fault is described in Bill's original version, and the
>>> INTR-Remap fault is caused by a similar reason. In short, the initialization
>>> of vt-d drivers causes the in-flight DMA and interrupt requests get wrong
>>> response.
>>>
>>> To fix this problem, we modifies the behaviors of the intel vt-d in the
>>> crashdump kernel:
>>>
>>> For DMA Remapping:
>>> 1. To accept the vt-d hardware in an active state,
>>> 2. Do not disable and re-enable the translation, keep it enabled.
>>> 3. Use the old root entry table, do not rewrite the RTA register.
>>> 4. Malloc and use new context entry table and page table, copy data from the
>>>     old ones that used by the old kernel.
>>> 5. to use different portions of the iova address ranges for the device drivers
>>>     in the crashdump kernel than the iova ranges that were in-use at the time
>>>     of the panic.
>>> 6. After device driver is loaded, when it issues the first dma_map command,
>>>     free the dmar_domain structure for this device, and generate a new one, so
>>>     that the device can be assigned a new and empty page table.
>>> 7. When a new context entry table is generated, we also save its address to
>>>     the old root entry table.
>>>
>>> For Interrupt Remapping:
>>> 1. To accept the vt-d hardware in an active state,
>>> 2. Do not disable and re-enable the interrupt remapping, keep it enabled.
>>> 3. Use the old interrupt remapping table, do not rewrite the IRTA register.
>>> 4. When ioapic entry is setup, the interrupt remapping table is changed, and
>>>     the updated data will be stored to the old interrupt remapping table.
>>>
>>> Advantages of this approach:
>>> 1. All manipulation of the IO-device is done by the Linux device-driver
>>>     for that device.
>>> 2. This approach behaves in a manner very similar to operation without an
>>>     active iommu.
>>> 3. Any activity between the IO-device and its RMRR areas is handled by the
>>>     device-driver in the same manner as during a non-kdump boot.
>>> 4. If an IO-device has no driver in the kdump kernel, it is simply left alone.
>>>     This supports the practice of creating a special kdump kernel without
>>>     drivers for any devices that are not required for taking a crashdump.
>>> 5. Minimal code-changes among the existing mainline intel vt-d code.
>>>
>>> Summary of changes in this patch set:
>>> 1. Added some useful function for root entry table in code intel-iommu.c
>>> 2. Added new members to struct root_entry and struct irte;
>>> 3. Functions to load old root entry table to iommu->root_entry from the memory
>>>     of old kernel.
>>> 4. Functions to malloc new context entry table and page table and copy the data
>>>     from the old ones to the malloced new ones.
>>> 5. Functions to enable support for DMA remapping in kdump kernel.
>>> 6. Functions to load old irte data from the old kernel to the kdump kernel.
>>> 7. Some code changes that support other behaviours that have been listed.
>>> 8. In the new functions, use physical address as "unsigned long" type, not
>>>     pointers.
>>>
>>> Original version by Bill Sumner:
>>>      https://lkml.org/lkml/2014/1/10/518
>>>      https://lkml.org/lkml/2014/4/15/716
>>>      https://lkml.org/lkml/2014/4/24/836
>>>
>>> Zhenhua's updates:
>>>      https://lkml.org/lkml/2014/10/21/134
>>>      https://lkml.org/lkml/2014/12/15/121
>>>      https://lkml.org/lkml/2014/12/22/53
>>>
>>> Changelog[v7]:
>>>      1. Use __iommu_flush_cache to flush the data to hardware.
>>>
>>> Changelog[v6]:
>>>      1. Use "unsigned long" as type of physical address.
>>>      2. Use new function unmap_device_dma to unmap the old dma.
>>>      3. Some small incorrect bits order for aw shift.
>>>
>>> Changelog[v5]:
>>>      1. Do not disable and re-enable traslation and interrupt remapping.
>>>      2. Use old root entry table.
>>>      3. Use old interrupt remapping table.
>>>      4. New functions to copy data from old kernel, and save to old kernel mem.
>>>      5. New functions to save updated root entry table and irte table.
>>>      6. Use intel_unmap to unmap the old dma;
>>>      7. Allocate new pages while driver is being loaded.
>>>
>>> Changelog[v4]:
>>>      1. Cut off the patches that move some defines and functions to new files.
>>>      2. Reduce the numbers of patches to five, make it more easier to read.
>>>      3. Changed the name of functions, make them consistent with current context
>>>         get/set functions.
>>>      4. Add change to function __iommu_attach_domain.
>>>
>>> Changelog[v3]:
>>>      1. Commented-out "#define DEBUG 1" to eliminate debug messages.
>>>      2. Updated the comments about changes in each version.
>>>      3. Fixed: one-line added to Copy-Translations patch to initialize the iovad
>>>            struct as recommended by Baoquan He [bhe at redhat.com]
>>>            init_iova_domain(&domain->iovad, DMA_32BIT_PFN);
>>>
>>> Changelog[v2]:
>>>      The following series implements a fix for:
>>>      A kdump problem about DMA that has been discussed for a long time. That is,
>>>      when a kernel panics and boots into the kdump kernel, DMA started by the
>>>      panicked kernel is not stopped before the kdump kernel is booted and the
>>>      kdump kernel disables the IOMMU while this DMA continues.  This causes the
>>>      IOMMU to stop translating the DMA addresses as IOVAs and begin to treat
>>>      them as physical memory addresses -- which causes the DMA to either:
>>>          (1) generate DMAR errors or
>>>          (2) generate PCI SERR errors or
>>>          (3) transfer data to or from incorrect areas of memory. Often this
>>>              causes the dump to fail.
>>>
>>> Changelog[v1]:
>>>      The original version.
>>>
>>> Changed in this version:
>>> 1. Do not disable and re-enable traslation and interrupt remapping.
>>> 2. Use old root entry table.
>>> 3. Use old interrupt remapping table.
>>> 4. Use "unsigned long" as physical address.
>>> 5. Use intel_unmap to unmap the old dma;
>>>
>>> Baoquan He <bhe at redhat.com> helps testing this patchset.
>>>
>>>    iommu/vt-d: Update iommu_attach_domain() and its callers
>>>    iommu/vt-d: Items required for kdump
>>>    iommu/vt-d: Add domain-id functions
>>>    iommu/vt-d: functions to copy data from old mem
>>>    iommu/vt-d: Add functions to load and save old re
>>>    iommu/vt-d: datatypes and functions used for kdump
>>>    iommu/vt-d: enable kdump support in iommu module
>>>    iommu/vt-d: assign new page table for dma_map
>>>    iommu/vt-d: Copy functions for irte
>>>    iommu/vt-d: Use old irte in kdump kernel
>>>
>>> Signed-off-by: Bill Sumner <billsumnerlinux at gmail.com>
>>> Signed-off-by: Li, Zhen-Hua <zhen-hual at hp.com>
>>> Signed-off-by: Takao Indoh <indou.takao at jp.fujitsu.com>
>>> Tested-by: Baoquan He <bhe at redhat.com>
>>> ---
>>>   drivers/iommu/intel-iommu.c         | 1050 +++++++++++++++++++++++++++++++++--
>>>   drivers/iommu/intel_irq_remapping.c |  104 +++-
>>>   include/linux/intel-iommu.h         |   18 +
>>>   3 files changed, 1130 insertions(+), 42 deletions(-)
>>>
>>



From bhe at redhat.com  Wed Jan  7 00:28:13 2015
From: bhe at redhat.com (Baoquan He)
Date: Wed, 7 Jan 2015 16:28:13 +0800
Subject: [PATCH v7 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel
In-Reply-To: <54ACC35F.1070007@hp.com>
References: <1420603474-4235-1-git-send-email-zhen-hual@hp.com>
 <54ACB1E9.9070904@hp.com>
 <20150107050247.GA16050@dhcp-16-116.nay.redhat.com>
 <54ACC35F.1070007@hp.com>
Message-ID: <20150107082813.GB16050@dhcp-16-116.nay.redhat.com>

On 01/07/15 at 01:25pm, Li, ZhenHua wrote:
> It is same as the last one I send to you yesterday.
> 
> The continuous memory that needed for data in this patchset:
> RE: PAGE_SIZE, 4096 Bytes;
> IRTE: 65536 * 16 ; 1M Bytes;
> 
> It should use same memory as the old versions of this patchset. The
> changes for the last version do not need more memory.

Hi Zhenhua,

It was my mistake because I didn't strip the debug info of modules, then
initramfs is bloated very big. Just now I tested the latest version, it
works well and dump is successful. No dmar fault and intr-remap fault
seen any more, good job!

Thanks
Baoquan


> 
> Regards
> Zhenhua
> 
> On 01/07/2015 01:02 PM, Baoquan He wrote:
> >On 01/07/15 at 12:11pm, Li, ZhenHua wrote:
> >>Many thanks to Takao Indoh and Baoquan He, for your testing on more
> >>different systems.
> >>
> >>The calling of flush functions are added to this version.
> >>
> >>The usage of __iommu_flush_cache function :
> >>1. Fixes a dump on Takao's system.
> >>2. Reduces the count of faults on Baoquan's system.
> >
> >I am testing the version you sent to me yesterday afternoon. Is that
> >different with this patchset? I found your patchset man reserve a big
> >contiguous memory region under 896M, this will cause the crashkernel
> >reservation failed when I set crashkernel=320M. The reason I increase
> >the crashkerenl reservation to 320M is 256M is not enough and cause OOM
> >when that patchset is tested.
> >
> >I am checking what happened.
> >
> >
> >Thanks
> >Baoquan
> >
> >>
> >>Regards
> >>Zhenhua
> >>
> >>On 01/07/2015 12:04 PM, Li, Zhen-Hua wrote:
> >>>This patchset is an update of Bill Sumner's patchset, implements a fix for:
> >>>If a kernel boots with intel_iommu=on on a system that supports intel vt-d,
> >>>when a panic happens, the kdump kernel will boot with these faults:
> >>>
> >>>     dmar: DRHD: handling fault status reg 102
> >>>     dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr fff80000
> >>>     DMAR:[fault reason 01] Present bit in root entry is clear
> >>>
> >>>     dmar: DRHD: handling fault status reg 2
> >>>     dmar: INTR-REMAP: Request device [[61:00.0] fault index 42
> >>>     INTR-REMAP:[fault reason 34] Present field in the IRTE entry is clear
> >>>
> >>>On some system, the interrupt remapping fault will also happen even if the
> >>>intel_iommu is not set to on, because the interrupt remapping will be enabled
> >>>when x2apic is needed by the system.
> >>>
> >>>The cause of the DMA fault is described in Bill's original version, and the
> >>>INTR-Remap fault is caused by a similar reason. In short, the initialization
> >>>of vt-d drivers causes the in-flight DMA and interrupt requests get wrong
> >>>response.
> >>>
> >>>To fix this problem, we modifies the behaviors of the intel vt-d in the
> >>>crashdump kernel:
> >>>
> >>>For DMA Remapping:
> >>>1. To accept the vt-d hardware in an active state,
> >>>2. Do not disable and re-enable the translation, keep it enabled.
> >>>3. Use the old root entry table, do not rewrite the RTA register.
> >>>4. Malloc and use new context entry table and page table, copy data from the
> >>>    old ones that used by the old kernel.
> >>>5. to use different portions of the iova address ranges for the device drivers
> >>>    in the crashdump kernel than the iova ranges that were in-use at the time
> >>>    of the panic.
> >>>6. After device driver is loaded, when it issues the first dma_map command,
> >>>    free the dmar_domain structure for this device, and generate a new one, so
> >>>    that the device can be assigned a new and empty page table.
> >>>7. When a new context entry table is generated, we also save its address to
> >>>    the old root entry table.
> >>>
> >>>For Interrupt Remapping:
> >>>1. To accept the vt-d hardware in an active state,
> >>>2. Do not disable and re-enable the interrupt remapping, keep it enabled.
> >>>3. Use the old interrupt remapping table, do not rewrite the IRTA register.
> >>>4. When ioapic entry is setup, the interrupt remapping table is changed, and
> >>>    the updated data will be stored to the old interrupt remapping table.
> >>>
> >>>Advantages of this approach:
> >>>1. All manipulation of the IO-device is done by the Linux device-driver
> >>>    for that device.
> >>>2. This approach behaves in a manner very similar to operation without an
> >>>    active iommu.
> >>>3. Any activity between the IO-device and its RMRR areas is handled by the
> >>>    device-driver in the same manner as during a non-kdump boot.
> >>>4. If an IO-device has no driver in the kdump kernel, it is simply left alone.
> >>>    This supports the practice of creating a special kdump kernel without
> >>>    drivers for any devices that are not required for taking a crashdump.
> >>>5. Minimal code-changes among the existing mainline intel vt-d code.
> >>>
> >>>Summary of changes in this patch set:
> >>>1. Added some useful function for root entry table in code intel-iommu.c
> >>>2. Added new members to struct root_entry and struct irte;
> >>>3. Functions to load old root entry table to iommu->root_entry from the memory
> >>>    of old kernel.
> >>>4. Functions to malloc new context entry table and page table and copy the data
> >>>    from the old ones to the malloced new ones.
> >>>5. Functions to enable support for DMA remapping in kdump kernel.
> >>>6. Functions to load old irte data from the old kernel to the kdump kernel.
> >>>7. Some code changes that support other behaviours that have been listed.
> >>>8. In the new functions, use physical address as "unsigned long" type, not
> >>>    pointers.
> >>>
> >>>Original version by Bill Sumner:
> >>>     https://lkml.org/lkml/2014/1/10/518
> >>>     https://lkml.org/lkml/2014/4/15/716
> >>>     https://lkml.org/lkml/2014/4/24/836
> >>>
> >>>Zhenhua's updates:
> >>>     https://lkml.org/lkml/2014/10/21/134
> >>>     https://lkml.org/lkml/2014/12/15/121
> >>>     https://lkml.org/lkml/2014/12/22/53
> >>>
> >>>Changelog[v7]:
> >>>     1. Use __iommu_flush_cache to flush the data to hardware.
> >>>
> >>>Changelog[v6]:
> >>>     1. Use "unsigned long" as type of physical address.
> >>>     2. Use new function unmap_device_dma to unmap the old dma.
> >>>     3. Some small incorrect bits order for aw shift.
> >>>
> >>>Changelog[v5]:
> >>>     1. Do not disable and re-enable traslation and interrupt remapping.
> >>>     2. Use old root entry table.
> >>>     3. Use old interrupt remapping table.
> >>>     4. New functions to copy data from old kernel, and save to old kernel mem.
> >>>     5. New functions to save updated root entry table and irte table.
> >>>     6. Use intel_unmap to unmap the old dma;
> >>>     7. Allocate new pages while driver is being loaded.
> >>>
> >>>Changelog[v4]:
> >>>     1. Cut off the patches that move some defines and functions to new files.
> >>>     2. Reduce the numbers of patches to five, make it more easier to read.
> >>>     3. Changed the name of functions, make them consistent with current context
> >>>        get/set functions.
> >>>     4. Add change to function __iommu_attach_domain.
> >>>
> >>>Changelog[v3]:
> >>>     1. Commented-out "#define DEBUG 1" to eliminate debug messages.
> >>>     2. Updated the comments about changes in each version.
> >>>     3. Fixed: one-line added to Copy-Translations patch to initialize the iovad
> >>>           struct as recommended by Baoquan He [bhe at redhat.com]
> >>>           init_iova_domain(&domain->iovad, DMA_32BIT_PFN);
> >>>
> >>>Changelog[v2]:
> >>>     The following series implements a fix for:
> >>>     A kdump problem about DMA that has been discussed for a long time. That is,
> >>>     when a kernel panics and boots into the kdump kernel, DMA started by the
> >>>     panicked kernel is not stopped before the kdump kernel is booted and the
> >>>     kdump kernel disables the IOMMU while this DMA continues.  This causes the
> >>>     IOMMU to stop translating the DMA addresses as IOVAs and begin to treat
> >>>     them as physical memory addresses -- which causes the DMA to either:
> >>>         (1) generate DMAR errors or
> >>>         (2) generate PCI SERR errors or
> >>>         (3) transfer data to or from incorrect areas of memory. Often this
> >>>             causes the dump to fail.
> >>>
> >>>Changelog[v1]:
> >>>     The original version.
> >>>
> >>>Changed in this version:
> >>>1. Do not disable and re-enable traslation and interrupt remapping.
> >>>2. Use old root entry table.
> >>>3. Use old interrupt remapping table.
> >>>4. Use "unsigned long" as physical address.
> >>>5. Use intel_unmap to unmap the old dma;
> >>>
> >>>Baoquan He <bhe at redhat.com> helps testing this patchset.
> >>>
> >>>   iommu/vt-d: Update iommu_attach_domain() and its callers
> >>>   iommu/vt-d: Items required for kdump
> >>>   iommu/vt-d: Add domain-id functions
> >>>   iommu/vt-d: functions to copy data from old mem
> >>>   iommu/vt-d: Add functions to load and save old re
> >>>   iommu/vt-d: datatypes and functions used for kdump
> >>>   iommu/vt-d: enable kdump support in iommu module
> >>>   iommu/vt-d: assign new page table for dma_map
> >>>   iommu/vt-d: Copy functions for irte
> >>>   iommu/vt-d: Use old irte in kdump kernel
> >>>
> >>>Signed-off-by: Bill Sumner <billsumnerlinux at gmail.com>
> >>>Signed-off-by: Li, Zhen-Hua <zhen-hual at hp.com>
> >>>Signed-off-by: Takao Indoh <indou.takao at jp.fujitsu.com>
> >>>Tested-by: Baoquan He <bhe at redhat.com>
> >>>---
> >>>  drivers/iommu/intel-iommu.c         | 1050 +++++++++++++++++++++++++++++++++--
> >>>  drivers/iommu/intel_irq_remapping.c |  104 +++-
> >>>  include/linux/intel-iommu.h         |   18 +
> >>>  3 files changed, 1130 insertions(+), 42 deletions(-)
> >>>
> >>
> 


From zhen-hual at hp.com  Wed Jan  7 00:52:28 2015
From: zhen-hual at hp.com (Li, ZhenHua)
Date: Wed, 07 Jan 2015 16:52:28 +0800
Subject: [PATCH v7 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel
In-Reply-To: <20150107082813.GB16050@dhcp-16-116.nay.redhat.com>
References: <1420603474-4235-1-git-send-email-zhen-hual@hp.com>
 <54ACB1E9.9070904@hp.com> <20150107050247.GA16050@dhcp-16-116.nay.redhat.com>
 <54ACC35F.1070007@hp.com> <20150107082813.GB16050@dhcp-16-116.nay.redhat.com>
Message-ID: <54ACF3CC.7090004@hp.com>

Well, that's quite good news.
Looking forward Takao's testing on his system.

Regards
Zhenhua
On 01/07/2015 04:28 PM, Baoquan He wrote:
> On 01/07/15 at 01:25pm, Li, ZhenHua wrote:
>> It is same as the last one I send to you yesterday.
>>
>> The continuous memory that needed for data in this patchset:
>> RE: PAGE_SIZE, 4096 Bytes;
>> IRTE: 65536 * 16 ; 1M Bytes;
>>
>> It should use same memory as the old versions of this patchset. The
>> changes for the last version do not need more memory.
>
> Hi Zhenhua,
>
> It was my mistake because I didn't strip the debug info of modules, then
> initramfs is bloated very big. Just now I tested the latest version, it
> works well and dump is successful. No dmar fault and intr-remap fault
> seen any more, good job!
>
> Thanks
> Baoquan
>
>
>>
>> Regards
>> Zhenhua
>>
>> On 01/07/2015 01:02 PM, Baoquan He wrote:
>>> On 01/07/15 at 12:11pm, Li, ZhenHua wrote:
>>>> Many thanks to Takao Indoh and Baoquan He, for your testing on more
>>>> different systems.
>>>>
>>>> The calling of flush functions are added to this version.
>>>>
>>>> The usage of __iommu_flush_cache function :
>>>> 1. Fixes a dump on Takao's system.
>>>> 2. Reduces the count of faults on Baoquan's system.
>>>
>>> I am testing the version you sent to me yesterday afternoon. Is that
>>> different with this patchset? I found your patchset man reserve a big
>>> contiguous memory region under 896M, this will cause the crashkernel
>>> reservation failed when I set crashkernel=320M. The reason I increase
>>> the crashkerenl reservation to 320M is 256M is not enough and cause OOM
>>> when that patchset is tested.
>>>
>>> I am checking what happened.
>>>
>>>
>>> Thanks
>>> Baoquan
>>>
>>>>
>>>> Regards
>>>> Zhenhua
>>>>
>>>> On 01/07/2015 12:04 PM, Li, Zhen-Hua wrote:
>>>>> This patchset is an update of Bill Sumner's patchset, implements a fix for:
>>>>> If a kernel boots with intel_iommu=on on a system that supports intel vt-d,
>>>>> when a panic happens, the kdump kernel will boot with these faults:
>>>>>
>>>>>      dmar: DRHD: handling fault status reg 102
>>>>>      dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr fff80000
>>>>>      DMAR:[fault reason 01] Present bit in root entry is clear
>>>>>
>>>>>      dmar: DRHD: handling fault status reg 2
>>>>>      dmar: INTR-REMAP: Request device [[61:00.0] fault index 42
>>>>>      INTR-REMAP:[fault reason 34] Present field in the IRTE entry is clear
>>>>>
>>>>> On some system, the interrupt remapping fault will also happen even if the
>>>>> intel_iommu is not set to on, because the interrupt remapping will be enabled
>>>>> when x2apic is needed by the system.
>>>>>
>>>>> The cause of the DMA fault is described in Bill's original version, and the
>>>>> INTR-Remap fault is caused by a similar reason. In short, the initialization
>>>>> of vt-d drivers causes the in-flight DMA and interrupt requests get wrong
>>>>> response.
>>>>>
>>>>> To fix this problem, we modifies the behaviors of the intel vt-d in the
>>>>> crashdump kernel:
>>>>>
>>>>> For DMA Remapping:
>>>>> 1. To accept the vt-d hardware in an active state,
>>>>> 2. Do not disable and re-enable the translation, keep it enabled.
>>>>> 3. Use the old root entry table, do not rewrite the RTA register.
>>>>> 4. Malloc and use new context entry table and page table, copy data from the
>>>>>     old ones that used by the old kernel.
>>>>> 5. to use different portions of the iova address ranges for the device drivers
>>>>>     in the crashdump kernel than the iova ranges that were in-use at the time
>>>>>     of the panic.
>>>>> 6. After device driver is loaded, when it issues the first dma_map command,
>>>>>     free the dmar_domain structure for this device, and generate a new one, so
>>>>>     that the device can be assigned a new and empty page table.
>>>>> 7. When a new context entry table is generated, we also save its address to
>>>>>     the old root entry table.
>>>>>
>>>>> For Interrupt Remapping:
>>>>> 1. To accept the vt-d hardware in an active state,
>>>>> 2. Do not disable and re-enable the interrupt remapping, keep it enabled.
>>>>> 3. Use the old interrupt remapping table, do not rewrite the IRTA register.
>>>>> 4. When ioapic entry is setup, the interrupt remapping table is changed, and
>>>>>     the updated data will be stored to the old interrupt remapping table.
>>>>>
>>>>> Advantages of this approach:
>>>>> 1. All manipulation of the IO-device is done by the Linux device-driver
>>>>>     for that device.
>>>>> 2. This approach behaves in a manner very similar to operation without an
>>>>>     active iommu.
>>>>> 3. Any activity between the IO-device and its RMRR areas is handled by the
>>>>>     device-driver in the same manner as during a non-kdump boot.
>>>>> 4. If an IO-device has no driver in the kdump kernel, it is simply left alone.
>>>>>     This supports the practice of creating a special kdump kernel without
>>>>>     drivers for any devices that are not required for taking a crashdump.
>>>>> 5. Minimal code-changes among the existing mainline intel vt-d code.
>>>>>
>>>>> Summary of changes in this patch set:
>>>>> 1. Added some useful function for root entry table in code intel-iommu.c
>>>>> 2. Added new members to struct root_entry and struct irte;
>>>>> 3. Functions to load old root entry table to iommu->root_entry from the memory
>>>>>     of old kernel.
>>>>> 4. Functions to malloc new context entry table and page table and copy the data
>>>>>     from the old ones to the malloced new ones.
>>>>> 5. Functions to enable support for DMA remapping in kdump kernel.
>>>>> 6. Functions to load old irte data from the old kernel to the kdump kernel.
>>>>> 7. Some code changes that support other behaviours that have been listed.
>>>>> 8. In the new functions, use physical address as "unsigned long" type, not
>>>>>     pointers.
>>>>>
>>>>> Original version by Bill Sumner:
>>>>>      https://lkml.org/lkml/2014/1/10/518
>>>>>      https://lkml.org/lkml/2014/4/15/716
>>>>>      https://lkml.org/lkml/2014/4/24/836
>>>>>
>>>>> Zhenhua's updates:
>>>>>      https://lkml.org/lkml/2014/10/21/134
>>>>>      https://lkml.org/lkml/2014/12/15/121
>>>>>      https://lkml.org/lkml/2014/12/22/53
>>>>>
>>>>> Changelog[v7]:
>>>>>      1. Use __iommu_flush_cache to flush the data to hardware.
>>>>>
>>>>> Changelog[v6]:
>>>>>      1. Use "unsigned long" as type of physical address.
>>>>>      2. Use new function unmap_device_dma to unmap the old dma.
>>>>>      3. Some small incorrect bits order for aw shift.
>>>>>
>>>>> Changelog[v5]:
>>>>>      1. Do not disable and re-enable traslation and interrupt remapping.
>>>>>      2. Use old root entry table.
>>>>>      3. Use old interrupt remapping table.
>>>>>      4. New functions to copy data from old kernel, and save to old kernel mem.
>>>>>      5. New functions to save updated root entry table and irte table.
>>>>>      6. Use intel_unmap to unmap the old dma;
>>>>>      7. Allocate new pages while driver is being loaded.
>>>>>
>>>>> Changelog[v4]:
>>>>>      1. Cut off the patches that move some defines and functions to new files.
>>>>>      2. Reduce the numbers of patches to five, make it more easier to read.
>>>>>      3. Changed the name of functions, make them consistent with current context
>>>>>         get/set functions.
>>>>>      4. Add change to function __iommu_attach_domain.
>>>>>
>>>>> Changelog[v3]:
>>>>>      1. Commented-out "#define DEBUG 1" to eliminate debug messages.
>>>>>      2. Updated the comments about changes in each version.
>>>>>      3. Fixed: one-line added to Copy-Translations patch to initialize the iovad
>>>>>            struct as recommended by Baoquan He [bhe at redhat.com]
>>>>>            init_iova_domain(&domain->iovad, DMA_32BIT_PFN);
>>>>>
>>>>> Changelog[v2]:
>>>>>      The following series implements a fix for:
>>>>>      A kdump problem about DMA that has been discussed for a long time. That is,
>>>>>      when a kernel panics and boots into the kdump kernel, DMA started by the
>>>>>      panicked kernel is not stopped before the kdump kernel is booted and the
>>>>>      kdump kernel disables the IOMMU while this DMA continues.  This causes the
>>>>>      IOMMU to stop translating the DMA addresses as IOVAs and begin to treat
>>>>>      them as physical memory addresses -- which causes the DMA to either:
>>>>>          (1) generate DMAR errors or
>>>>>          (2) generate PCI SERR errors or
>>>>>          (3) transfer data to or from incorrect areas of memory. Often this
>>>>>              causes the dump to fail.
>>>>>
>>>>> Changelog[v1]:
>>>>>      The original version.
>>>>>
>>>>> Changed in this version:
>>>>> 1. Do not disable and re-enable traslation and interrupt remapping.
>>>>> 2. Use old root entry table.
>>>>> 3. Use old interrupt remapping table.
>>>>> 4. Use "unsigned long" as physical address.
>>>>> 5. Use intel_unmap to unmap the old dma;
>>>>>
>>>>> Baoquan He <bhe at redhat.com> helps testing this patchset.
>>>>>
>>>>>    iommu/vt-d: Update iommu_attach_domain() and its callers
>>>>>    iommu/vt-d: Items required for kdump
>>>>>    iommu/vt-d: Add domain-id functions
>>>>>    iommu/vt-d: functions to copy data from old mem
>>>>>    iommu/vt-d: Add functions to load and save old re
>>>>>    iommu/vt-d: datatypes and functions used for kdump
>>>>>    iommu/vt-d: enable kdump support in iommu module
>>>>>    iommu/vt-d: assign new page table for dma_map
>>>>>    iommu/vt-d: Copy functions for irte
>>>>>    iommu/vt-d: Use old irte in kdump kernel
>>>>>
>>>>> Signed-off-by: Bill Sumner <billsumnerlinux at gmail.com>
>>>>> Signed-off-by: Li, Zhen-Hua <zhen-hual at hp.com>
>>>>> Signed-off-by: Takao Indoh <indou.takao at jp.fujitsu.com>
>>>>> Tested-by: Baoquan He <bhe at redhat.com>
>>>>> ---
>>>>>   drivers/iommu/intel-iommu.c         | 1050 +++++++++++++++++++++++++++++++++--
>>>>>   drivers/iommu/intel_irq_remapping.c |  104 +++-
>>>>>   include/linux/intel-iommu.h         |   18 +
>>>>>   3 files changed, 1130 insertions(+), 42 deletions(-)
>>>>>
>>>>
>>



From mtk.manpages at gmail.com  Wed Jan  7 13:17:56 2015
From: mtk.manpages at gmail.com (Michael Kerrisk (man-pages))
Date: Wed, 07 Jan 2015 22:17:56 +0100
Subject: Edited kexec_load(2) [kexec_file_load()] man page for review
In-Reply-To: <20141111213037.GA31445@redhat.com>
References: <545FBDDD.9060801@gmail.com> <20141111213037.GA31445@redhat.com>
Message-ID: <54ADA284.30502@gmail.com>

Hi Vivek,

Thanks for your comments, and my apologies for a the long delayed follow-up.

On 11/11/2014 10:30 PM, Vivek Goyal wrote:
> On Sun, Nov 09, 2014 at 08:17:49PM +0100, Michael Kerrisk (man-pages) wrote:
>> Hello Vivek (and all),
>>
>> Thanks for the kexec_file_load() patch [for the kexec_load(2) man page]
>> that you quite some time ago sent. I have merged it and done some
>> substantial editing as well. Could you please take a look at the 
>> draft below, and check that the kexec_file_load() material is okay.
>> Please could you especially pay attention to the pieces marked
>> "FIXME(kexec_file_load)", since those are pieces about which i
>> had questions or doubts.
>>
> 
> Hi Michael,
> 
> Thanks for editing this man page. I have some thoughts inline.
> 
> [..]
>> .B #include <linux/kexec.h>
>>
>> .BI "long kexec_load(unsigned long " entry ", unsigned long " nr_segments ","
>> .BI "                struct kexec_segment *" segments \
>> ", unsigned long " flags ");"
>>
>> .\" FIXME(kexec_file_load):
>> .\"     Why are the return types of kexec_load() and kexec_file_load()
>> .\"     different?
>> .BI "int kexec_file_load(int " kernel_fd ", int " initrd_fd ","
> 
> I think this is ignorance on my part. It probably should be "long" as
> SYSCALL_DEFINE() seems to expand to.
> 
> asmlinkage long SyS##name(__MAP(x,__SC_LONG,__VA_ARGS__));

Okay -- I've changed to 'long' in the man page.

>> .br
>> .BI "                    unsigned long " cmdline_len  \
>> ", const char *" cmdline ","
>> .BI "                    unsigned long " flags ");"
>>
>> .fi
>> .IR Note :
>> There are no glibc wrappers for these system calls; see NOTES.
>> .SH DESCRIPTION
>> The
>> .BR kexec_load ()
>> system call loads a new kernel that can be executed later by
>> .BR reboot (2).
>> .PP
>> The
>> .I flags
>> argument is a bit mask that controls the operation of the call.
>> The following values can be specified in
>> .IR flags :
>> .TP
>> .BR KEXEC_ON_CRASH " (since Linux 2.6.13)"
>> Execute the new kernel automatically on a system crash.
>> .\" FIXME Explain in more detail how KEXEC_ON_CRASH is actually used

I wasn't expecting that you would respond to the FIXMEs that were 
not labeled "kexec_file_load", but I was hoping you might ;-). Thanks!
I have a few additional questions to your nice notes.

> Upon boot first kernel reserves a chunk of contiguous memory (if
> crashkernel=<> command line paramter is passed). This memory is
> is used to load the crash kernel (Kernel which will be booted into
> if first kernel crashes).

Can I just confirm: is it in all cases only possible to use kexec_load() 
and kexec_file_load() if the kernel was booted with the 'crashkernel'
parameter set?

> Location of this reserved memory is exported to user space through
> /proc/iomem file. 

Is that export via an entry labeled "Crash kernel" in the 
/proc/iomem file?

> User space can parse it and prepare list of segments
> specifying this reserved memory as destination.

I'm not quite clear on "specifying this reserved memory as destination".
Is that done by specifying the address in the kexec_segment.mem fields?

> Once kernel sees the flag KEXEC_ON_CRASH, it makes sure that all the
> segments are destined for reserved memory otherwise kernel load operation
> fails.

Could you point me to where this checking is done? Also, what is the
error (errno) that occurs when the load operation fails? (I think the
answers to these questions are "at the start of kimage_alloc_init()"
and "EADDRNOTAVAIL", but I'd like to confirm.)

> [..]
>> struct kexec_segment {
>>     void   *buf;        /* Buffer in user space */
>>     size_t  bufsz;      /* Buffer length in user space */
>>     void   *mem;        /* Physical address of kernel */
>>     size_t  memsz;      /* Physical address length */
>> };
>> .fi
>> .in
>> .PP
>> .\" FIXME Explain the details of how the kernel image defined by segments
>> .\" is copied from the calling process into previously reserved memory.
> 
> Kernel image defined by segments is copied into kernel either in regular
> memory 

Could you clarify what you mean by "regular memory"?

> or in reserved memory (if KEXEC_ON_CRASH is set). Kernel first
> copies list of segments in kernel memory and then goes does various
> sanity checks on the segments. If everything looks line, kernel copies
> segment data to kernel memory.
> 
> In case of normal kexec, segment data is loaded in any available memory
> and segment data is moved to final destination at the kexec reboot time.

By "moved to final destination", do you mean "moved from user space to the
final kernel-space destination"?

> In case of kexec on panic (KEXEC_ON_CRASH flag set), segment data is
> directly loaded to reserved memory and after crash kexec simply jumps

By "directly", I assume you mean "at the time of the kexec_laod() call",
right?

> to starting point.
> 
> [..]
>> .\" FIXME(kexec_file_load):
>> .\" Is the following rationale accurate? Does it need expanding?
>> The
>> .BR kexec_file_load ()
>> .\" See also http://lwn.net/Articles/603116/
>> system call was added to provide support for systems
>> where "kexec" loading should be restricted to
>> only kernels that are signed.
> 
> Yes, this rationale looks good.

Okay -- thanks.

>> The
>> .BR kexec_load ()
>> system call is available only if the kernel was configured with
>> .BR CONFIG_KEXEC .
>> The
>> .BR kexec_file_load ()
>> system call is available only if the kernel was configured with
>> .BR CONFIG_KEXEC_FILE .
>> .\" FIXME(kexec_file_load):
>> .\"     Does kexec_file_load() need any other CONFIG_* options to be defined?
> 
> Yes, it requires some other config options too.
> 
> depends on KEXEC
> depends on X86_64
> depends on CRYPTO=y
> depends on CRYPTO_SHA256=y
> 
> CONFIG_KEXEC_VERIFY_SIG=y
> CONFIG_KEXEC_BZIMAGE_VERIFY_SIG=y
> CONFIG_SIGNED_PE_FILE_VERIFICATION=y
> CONFIG_PKCS7_MESSAGE_PARSER=y
> CONFIG_X509_CERTIFICATE_PARSER=y
> CONFIG_ASYMMETRIC_PUBLIC_KEY_SUBTYPE=y
> 
> So dependency list seems pretty long. Not sure how many of these should
> we specify in man page.

On reflection, since they're dependencies of CONFIG_KEXEC_FILE, perhaps
it's not necessary to add any of the others.

Cheers,

Michael


-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/


From indou.takao at jp.fujitsu.com  Wed Jan  7 17:00:18 2015
From: indou.takao at jp.fujitsu.com (Takao Indoh)
Date: Thu, 8 Jan 2015 10:00:18 +0900
Subject: [PATCH v7 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel
In-Reply-To: <54ACF3CC.7090004@hp.com>
References: <1420603474-4235-1-git-send-email-zhen-hual@hp.com>
 <54ACB1E9.9070904@hp.com> <20150107050247.GA16050@dhcp-16-116.nay.redhat.com>
 <54ACC35F.1070007@hp.com> <20150107082813.GB16050@dhcp-16-116.nay.redhat.com>
 <54ACF3CC.7090004@hp.com>
Message-ID: <54ADD6A2.5020807@jp.fujitsu.com>

On 2015/01/07 17:52, Li, ZhenHua wrote:
> Well, that's quite good news.
> Looking forward Takao's testing on his system.

Unfortunately DMAR fault still occurs with this patch...
I attach console log.

Thanks,
Takao Indoh

> 
> Regards
> Zhenhua
> On 01/07/2015 04:28 PM, Baoquan He wrote:
>> On 01/07/15 at 01:25pm, Li, ZhenHua wrote:
>>> It is same as the last one I send to you yesterday.
>>>
>>> The continuous memory that needed for data in this patchset:
>>> RE: PAGE_SIZE, 4096 Bytes;
>>> IRTE: 65536 * 16 ; 1M Bytes;
>>>
>>> It should use same memory as the old versions of this patchset. The
>>> changes for the last version do not need more memory.
>>
>> Hi Zhenhua,
>>
>> It was my mistake because I didn't strip the debug info of modules, then
>> initramfs is bloated very big. Just now I tested the latest version, it
>> works well and dump is successful. No dmar fault and intr-remap fault
>> seen any more, good job!
>>
>> Thanks
>> Baoquan
>>
>>
>>>
>>> Regards
>>> Zhenhua
>>>
>>> On 01/07/2015 01:02 PM, Baoquan He wrote:
>>>> On 01/07/15 at 12:11pm, Li, ZhenHua wrote:
>>>>> Many thanks to Takao Indoh and Baoquan He, for your testing on more
>>>>> different systems.
>>>>>
>>>>> The calling of flush functions are added to this version.
>>>>>
>>>>> The usage of __iommu_flush_cache function :
>>>>> 1. Fixes a dump on Takao's system.
>>>>> 2. Reduces the count of faults on Baoquan's system.
>>>>
>>>> I am testing the version you sent to me yesterday afternoon. Is that
>>>> different with this patchset? I found your patchset man reserve a big
>>>> contiguous memory region under 896M, this will cause the crashkernel
>>>> reservation failed when I set crashkernel=320M. The reason I increase
>>>> the crashkerenl reservation to 320M is 256M is not enough and cause OOM
>>>> when that patchset is tested.
>>>>
>>>> I am checking what happened.
>>>>
>>>>
>>>> Thanks
>>>> Baoquan
>>>>
>>>>>
>>>>> Regards
>>>>> Zhenhua
>>>>>
>>>>> On 01/07/2015 12:04 PM, Li, Zhen-Hua wrote:
>>>>>> This patchset is an update of Bill Sumner's patchset, implements a fix for:
>>>>>> If a kernel boots with intel_iommu=on on a system that supports intel vt-d,
>>>>>> when a panic happens, the kdump kernel will boot with these faults:
>>>>>>
>>>>>>      dmar: DRHD: handling fault status reg 102
>>>>>>      dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr fff80000
>>>>>>      DMAR:[fault reason 01] Present bit in root entry is clear
>>>>>>
>>>>>>      dmar: DRHD: handling fault status reg 2
>>>>>>      dmar: INTR-REMAP: Request device [[61:00.0] fault index 42
>>>>>>      INTR-REMAP:[fault reason 34] Present field in the IRTE entry is clear
>>>>>>
>>>>>> On some system, the interrupt remapping fault will also happen even if the
>>>>>> intel_iommu is not set to on, because the interrupt remapping will be enabled
>>>>>> when x2apic is needed by the system.
>>>>>>
>>>>>> The cause of the DMA fault is described in Bill's original version, and the
>>>>>> INTR-Remap fault is caused by a similar reason. In short, the initialization
>>>>>> of vt-d drivers causes the in-flight DMA and interrupt requests get wrong
>>>>>> response.
>>>>>>
>>>>>> To fix this problem, we modifies the behaviors of the intel vt-d in the
>>>>>> crashdump kernel:
>>>>>>
>>>>>> For DMA Remapping:
>>>>>> 1. To accept the vt-d hardware in an active state,
>>>>>> 2. Do not disable and re-enable the translation, keep it enabled.
>>>>>> 3. Use the old root entry table, do not rewrite the RTA register.
>>>>>> 4. Malloc and use new context entry table and page table, copy data from the
>>>>>>     old ones that used by the old kernel.
>>>>>> 5. to use different portions of the iova address ranges for the device drivers
>>>>>>     in the crashdump kernel than the iova ranges that were in-use at the time
>>>>>>     of the panic.
>>>>>> 6. After device driver is loaded, when it issues the first dma_map command,
>>>>>>     free the dmar_domain structure for this device, and generate a new one, so
>>>>>>     that the device can be assigned a new and empty page table.
>>>>>> 7. When a new context entry table is generated, we also save its address to
>>>>>>     the old root entry table.
>>>>>>
>>>>>> For Interrupt Remapping:
>>>>>> 1. To accept the vt-d hardware in an active state,
>>>>>> 2. Do not disable and re-enable the interrupt remapping, keep it enabled.
>>>>>> 3. Use the old interrupt remapping table, do not rewrite the IRTA register.
>>>>>> 4. When ioapic entry is setup, the interrupt remapping table is changed, and
>>>>>>     the updated data will be stored to the old interrupt remapping table.
>>>>>>
>>>>>> Advantages of this approach:
>>>>>> 1. All manipulation of the IO-device is done by the Linux device-driver
>>>>>>     for that device.
>>>>>> 2. This approach behaves in a manner very similar to operation without an
>>>>>>     active iommu.
>>>>>> 3. Any activity between the IO-device and its RMRR areas is handled by the
>>>>>>     device-driver in the same manner as during a non-kdump boot.
>>>>>> 4. If an IO-device has no driver in the kdump kernel, it is simply left alone.
>>>>>>     This supports the practice of creating a special kdump kernel without
>>>>>>     drivers for any devices that are not required for taking a crashdump.
>>>>>> 5. Minimal code-changes among the existing mainline intel vt-d code.
>>>>>>
>>>>>> Summary of changes in this patch set:
>>>>>> 1. Added some useful function for root entry table in code intel-iommu.c
>>>>>> 2. Added new members to struct root_entry and struct irte;
>>>>>> 3. Functions to load old root entry table to iommu->root_entry from the memory
>>>>>>     of old kernel.
>>>>>> 4. Functions to malloc new context entry table and page table and copy the data
>>>>>>     from the old ones to the malloced new ones.
>>>>>> 5. Functions to enable support for DMA remapping in kdump kernel.
>>>>>> 6. Functions to load old irte data from the old kernel to the kdump kernel.
>>>>>> 7. Some code changes that support other behaviours that have been listed.
>>>>>> 8. In the new functions, use physical address as "unsigned long" type, not
>>>>>>     pointers.
>>>>>>
>>>>>> Original version by Bill Sumner:
>>>>>>      https://lkml.org/lkml/2014/1/10/518
>>>>>>      https://lkml.org/lkml/2014/4/15/716
>>>>>>      https://lkml.org/lkml/2014/4/24/836
>>>>>>
>>>>>> Zhenhua's updates:
>>>>>>      https://lkml.org/lkml/2014/10/21/134
>>>>>>      https://lkml.org/lkml/2014/12/15/121
>>>>>>      https://lkml.org/lkml/2014/12/22/53
>>>>>>
>>>>>> Changelog[v7]:
>>>>>>      1. Use __iommu_flush_cache to flush the data to hardware.
>>>>>>
>>>>>> Changelog[v6]:
>>>>>>      1. Use "unsigned long" as type of physical address.
>>>>>>      2. Use new function unmap_device_dma to unmap the old dma.
>>>>>>      3. Some small incorrect bits order for aw shift.
>>>>>>
>>>>>> Changelog[v5]:
>>>>>>      1. Do not disable and re-enable traslation and interrupt remapping.
>>>>>>      2. Use old root entry table.
>>>>>>      3. Use old interrupt remapping table.
>>>>>>      4. New functions to copy data from old kernel, and save to old kernel mem.
>>>>>>      5. New functions to save updated root entry table and irte table.
>>>>>>      6. Use intel_unmap to unmap the old dma;
>>>>>>      7. Allocate new pages while driver is being loaded.
>>>>>>
>>>>>> Changelog[v4]:
>>>>>>      1. Cut off the patches that move some defines and functions to new files.
>>>>>>      2. Reduce the numbers of patches to five, make it more easier to read.
>>>>>>      3. Changed the name of functions, make them consistent with current context
>>>>>>         get/set functions.
>>>>>>      4. Add change to function __iommu_attach_domain.
>>>>>>
>>>>>> Changelog[v3]:
>>>>>>      1. Commented-out "#define DEBUG 1" to eliminate debug messages.
>>>>>>      2. Updated the comments about changes in each version.
>>>>>>      3. Fixed: one-line added to Copy-Translations patch to initialize the iovad
>>>>>>            struct as recommended by Baoquan He [bhe at redhat.com]
>>>>>>            init_iova_domain(&domain->iovad, DMA_32BIT_PFN);
>>>>>>
>>>>>> Changelog[v2]:
>>>>>>      The following series implements a fix for:
>>>>>>      A kdump problem about DMA that has been discussed for a long time. That is,
>>>>>>      when a kernel panics and boots into the kdump kernel, DMA started by the
>>>>>>      panicked kernel is not stopped before the kdump kernel is booted and the
>>>>>>      kdump kernel disables the IOMMU while this DMA continues.  This causes the
>>>>>>      IOMMU to stop translating the DMA addresses as IOVAs and begin to treat
>>>>>>      them as physical memory addresses -- which causes the DMA to either:
>>>>>>          (1) generate DMAR errors or
>>>>>>          (2) generate PCI SERR errors or
>>>>>>          (3) transfer data to or from incorrect areas of memory. Often this
>>>>>>              causes the dump to fail.
>>>>>>
>>>>>> Changelog[v1]:
>>>>>>      The original version.
>>>>>>
>>>>>> Changed in this version:
>>>>>> 1. Do not disable and re-enable traslation and interrupt remapping.
>>>>>> 2. Use old root entry table.
>>>>>> 3. Use old interrupt remapping table.
>>>>>> 4. Use "unsigned long" as physical address.
>>>>>> 5. Use intel_unmap to unmap the old dma;
>>>>>>
>>>>>> Baoquan He <bhe at redhat.com> helps testing this patchset.
>>>>>>
>>>>>>    iommu/vt-d: Update iommu_attach_domain() and its callers
>>>>>>    iommu/vt-d: Items required for kdump
>>>>>>    iommu/vt-d: Add domain-id functions
>>>>>>    iommu/vt-d: functions to copy data from old mem
>>>>>>    iommu/vt-d: Add functions to load and save old re
>>>>>>    iommu/vt-d: datatypes and functions used for kdump
>>>>>>    iommu/vt-d: enable kdump support in iommu module
>>>>>>    iommu/vt-d: assign new page table for dma_map
>>>>>>    iommu/vt-d: Copy functions for irte
>>>>>>    iommu/vt-d: Use old irte in kdump kernel
>>>>>>
>>>>>> Signed-off-by: Bill Sumner <billsumnerlinux at gmail.com>
>>>>>> Signed-off-by: Li, Zhen-Hua <zhen-hual at hp.com>
>>>>>> Signed-off-by: Takao Indoh <indou.takao at jp.fujitsu.com>
>>>>>> Tested-by: Baoquan He <bhe at redhat.com>
>>>>>> ---
>>>>>>   drivers/iommu/intel-iommu.c         | 1050 +++++++++++++++++++++++++++++++++--
>>>>>>   drivers/iommu/intel_irq_remapping.c |  104 +++-
>>>>>>   include/linux/intel-iommu.h         |   18 +
>>>>>>   3 files changed, 1130 insertions(+), 42 deletions(-)
>>>>>>
>>>>>
>>>
> 
> 
> 

-------------- next part --------------
SysRq : Trigger a crash
BUG: unable to handle kernel NULL pointer dereference at           (null)
IP: [<ffffffff8135ed76>] sysrq_handle_crash+0x16/0x20
PGD 239d95067 PUD 239b0a067 PMD 0 
Oops: 0002 [#1] SMP 
Modules linked in: ebtable_nat ebtables xt_CHECKSUM iptable_mangle bridge autofs4 8021q garp stp llc cpufreq_ondemand ipt_REJECT nf_reject_ipv4 nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables ip6t_REJECT nf_reject_ipv6 nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables ipv6 vhost_net macvtap macvlan vhost tun kvm_intel kvm uinput iTCO_wdt iTCO_vendor_support microcode serio_raw pcspkr tpm_infineon ipmi_si ipmi_msghandler i2c_i801 lpc_ich mfd_core ioatdma i7core_edac edac_core sg igb dca i2c_algo_bit ptp pps_core acpi_cpufreq ext4 jbd2 mbcache mptsas mptscsih mptbase scsi_transport_sas lpfc scsi_transport_fc megaraid_sas dm_mirror dm_region_hash dm_log dm_mod
CPU: 4 PID: 3000 Comm: bash Not tainted 3.19.0-rc3 #36
Hardware name: FUJITSU-SV PRIMERGY BX920 S2/D3030, BIOS 080015 Rev.3D81.3030 02/10/2012
task: ffff88023931e250 ti: ffff880239b64000 task.ti: ffff880239b64000
RIP: 0010:[<ffffffff8135ed76>]  [<ffffffff8135ed76>] sysrq_handle_crash+0x16/0x20
RSP: 0018:ffff880239b67e88  EFLAGS: 00010296
RAX: 000000000000000f RBX: 0000000000000063 RCX: 0000000000000000
RDX: ffff88023fc0ea38 RSI: ffff88023fc0d238 RDI: 0000000000000063
RBP: ffff880239b67e88 R08: 0000000000019cd8 R09: 00000000000004c4
R10: 0000000000000003 R11: 00000000000004c3 R12: ffffffff81ac4c00
R13: 0000000000000008 R14: 0000000000000000 R15: 0000000000000001
FS:  00007f01234f3700(0000) GS:ffff88023fc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000000000 CR3: 00000002356e6000 CR4: 00000000000007e0
Stack:
 ffff880239b67eb8 ffffffff8135f6f1 0000000000000002 fffffffffffffffb
 00007f0123504000 ffff880239b67f50 ffff880239b67ed8 ffffffff8135f78d
 ffff880239b67ef8 ffff88023960ddc0 ffff880239b67ef8 ffffffff81201403
Call Trace:
 [<ffffffff8135f6f1>] __handle_sysrq+0x121/0x180
 [<ffffffff8135f78d>] write_sysrq_trigger+0x3d/0x40
 [<ffffffff81201403>] proc_reg_write+0x43/0x70
 [<ffffffff8119dbbe>] vfs_write+0xce/0x180
 [<ffffffff8119e326>] SyS_write+0x56/0xd0
 [<ffffffff810eca3c>] ? __audit_syscall_entry+0xac/0x110
 [<ffffffff815b6152>] system_call_fastpath+0x12/0x17
Code: f1 6b 25 00 31 c0 eb ae 90 90 90 90 90 90 90 90 90 90 90 90 90 55 48 89 e5 66 66 66 66 90 c7 05 ed 17 a0 00 01 00 00 00 0f ae f8 <c6> 04 25 00 00 00 00 01 c9 c3 55 48 89 e5 66 66 66 66 90 8d 47 
RIP  [<ffffffff8135ed76>] sysrq_handle_crash+0x16/0x20
 RSP <ffff880239b67e88>
CR2: 0000000000000000
Initializing cgroup subsys cpuset
Initializing cgroup subsys cpu
Initializing cgroup subsys cpuacct
Linux version 3.19.0-rc3 (root at indowsXP) (gcc version 4.4.7 20120313 (Red Hat 4.4.7-4) (GCC) ) #36 SMP Wed Jan 7 18:33:24 JST 2015
Command line: ro root=UUID=bfc88f62-f080-492f-a9f5-17f0c7c6215a rd_NO_LUKS console=ttyS0,115200n8 rd_NO_MD KEYBOARDTYPE=pc KEYTABLE=jp106 LANG=ja_JP.UTF-8 rd_NO_LVM rd_NO_DM intel_iommu=on irqpoll nr_cpus=1 reset_devices cgroup_disable=memory mce=off  memmap=exactmap memmap=627K at 4K memmap=130425K at 770675K elfcorehdr=901100K memmap=4K$0K memmap=9K$631K memmap=100K$924K memmap=8K$3137080K memmap=52K#3137088K memmap=204K#3137140K memmap=64K$3137344K memmap=8268K$3137460K memmap=262144K$3670016K memmap=4K$4175872K memmap=2048K$4192256K
e820: BIOS-provided physical RAM map:
BIOS-e820: [mem 0x0000000000000100-0x000000000009dbff] usable
BIOS-e820: [mem 0x000000000009dc00-0x000000000009ffff] reserved
BIOS-e820: [mem 0x00000000000e7000-0x00000000000fffff] reserved
BIOS-e820: [mem 0x0000000000100000-0x00000000bf77ffff] usable
BIOS-e820: [mem 0x00000000bf78e000-0x00000000bf78ffff] reserved
BIOS-e820: [mem 0x00000000bf790000-0x00000000bf79cfff] ACPI data
BIOS-e820: [mem 0x00000000bf79d000-0x00000000bf7cffff] ACPI NVS
BIOS-e820: [mem 0x00000000bf7d0000-0x00000000bf7dffff] reserved
BIOS-e820: [mem 0x00000000bf7ed000-0x00000000bfffffff] reserved
BIOS-e820: [mem 0x00000000e0000000-0x00000000efffffff] reserved
BIOS-e820: [mem 0x00000000fee00000-0x00000000fee00fff] reserved
BIOS-e820: [mem 0x00000000ffe00000-0x00000000ffffffff] reserved
BIOS-e820: [mem 0x0000000100000000-0x000000023fffffff] usable
e820: last_pfn = 0x240000 max_arch_pfn = 0x400000000
NX (Execute Disable) protection: active
e820: user-defined physical RAM map:
user: [mem 0x0000000000000000-0x0000000000000fff] reserved
user: [mem 0x0000000000001000-0x000000000009dbff] usable
user: [mem 0x000000000009dc00-0x000000000009ffff] reserved
user: [mem 0x00000000000e7000-0x00000000000fffff] reserved
user: [mem 0x000000002f09cc00-0x0000000036ffafff] usable
user: [mem 0x00000000bf78e000-0x00000000bf78ffff] reserved
user: [mem 0x00000000bf790000-0x00000000bf7cffff] ACPI data
user: [mem 0x00000000bf7d0000-0x00000000bf7dffff] reserved
user: [mem 0x00000000bf7ed000-0x00000000bfffffff] reserved
user: [mem 0x00000000e0000000-0x00000000efffffff] reserved
user: [mem 0x00000000fee00000-0x00000000fee00fff] reserved
user: [mem 0x00000000ffe00000-0x00000000ffffffff] reserved
SMBIOS 2.5 present.
AGP: No AGP bridge found
e820: last_pfn = 0x36ffb max_arch_pfn = 0x400000000
PAT configuration [0-7]: WB  WC  UC- UC  WB  WC  UC- UC  
total RAM covered: 8184M
Found optimal setting for mtrr clean up
 gran_size: 64K 	chunk_size: 16M 	num_reg: 5  	lose cover RAM: 0G
found SMP MP-table at [mem 0x000ff780-0x000ff78f] mapped at [ffff8800000ff780]
Using GB pages for direct mapping
init_memory_mapping: [mem 0x00000000-0x000fffff]
init_memory_mapping: [mem 0x36800000-0x369fffff]
init_memory_mapping: [mem 0x34000000-0x367fffff]
init_memory_mapping: [mem 0x2f09d000-0x33ffffff]
init_memory_mapping: [mem 0x36a00000-0x36ffafff]
RAMDISK: [mem 0x36ab5000-0x36feefff]
ACPI: Early table checksum verification disabled
ACPI: RSDP 0x00000000000FAA50 000024 (v02 ACPIAM)
ACPI: XSDT 0x00000000BF790100 000094 (v01 AAA             20120210 MSFT 00000097)
ACPI: FACP 0x00000000BF790290 0000F4 (v04 021012 FACP1717 20120210 MSFT 00000097)
ACPI BIOS Warning (bug): 32/64X length mismatch in FADT/Gpe0Block: 128/64 (20141107/tbfadt-618)
ACPI: DSDT 0x00000000BF7906F0 00712B (v02 TU1O   TU1O3D81 00003D81 INTL 20051117)
ACPI: FACS 0x00000000BF79D000 000040
ACPI: APIC 0x00000000BF790390 00011E (v02 021012 APIC1717 20120210 MSFT 00000097)
ACPI: SPCR 0x00000000BF7904B0 000050 (v01 021012 SPCR1717 20120210 MSFT 00000097)
ACPI: MCFG 0x00000000BF790500 00003C (v01 021012 OEMMCFG  20120210 MSFT 00000097)
ACPI: SLIT 0x00000000BF790540 000030 (v01 021012 OEMSLIT  20120210 MSFT 00000097)
ACPI: SLIC 0x00000000BF790570 000176 (v01 AAA             20120210 MSFT 00000097)
ACPI: SRAT 0x00000000BF79A6F0 0001D0 (v02 021012 OEMSRAT  00000001 INTL 00000001)
ACPI: HPET 0x00000000BF79A8C0 000038 (v01 021012 OEMHPET  20120210 MSFT 00000097)
ACPI: DMAR 0x00000000BF79D0E0 000140 (v01 AMI    OEMDMAR  00000001 MSFT 00000097)
ACPI: SSDT 0x00000000BF79F9A0 000363 (v01 DpgPmm CpuPm    00000012 INTL 20051117)
ACPI: EINJ 0x00000000BF79A900 000130 (v01 AMIER  AMI_EINJ 20120210 MSFT 00000097)
ACPI: BERT 0x00000000BF79AA90 000030 (v01 AMIER  AMI_BERT 20120210 MSFT 00000097)
ACPI: ERST 0x00000000BF79AAC0 0001B0 (v01 AMIER  AMI_ERST 20120210 MSFT 00000097)
ACPI: HEST 0x00000000BF79AC70 0000A8 (v01 AMIER  ABC_HEST 20120210 MSFT 00000097)
SRAT: PXM 0 -> APIC 0x00 -> Node 0
SRAT: PXM 0 -> APIC 0x02 -> Node 0
SRAT: PXM 0 -> APIC 0x12 -> Node 0
SRAT: PXM 0 -> APIC 0x14 -> Node 0
SRAT: PXM 0 -> APIC 0x01 -> Node 0
SRAT: PXM 0 -> APIC 0x03 -> Node 0
SRAT: PXM 0 -> APIC 0x13 -> Node 0
SRAT: PXM 0 -> APIC 0x15 -> Node 0
SRAT: PXM 1 -> APIC 0x20 -> Node 1
SRAT: PXM 1 -> APIC 0x22 -> Node 1
SRAT: PXM 1 -> APIC 0x32 -> Node 1
SRAT: PXM 1 -> APIC 0x34 -> Node 1
SRAT: PXM 1 -> APIC 0x21 -> Node 1
SRAT: PXM 1 -> APIC 0x23 -> Node 1
SRAT: PXM 1 -> APIC 0x33 -> Node 1
SRAT: PXM 1 -> APIC 0x35 -> Node 1
SRAT: Node 0 PXM 0 [mem 0x00000000-0x0009ffff]
SRAT: Node 0 PXM 0 [mem 0x00100000-0xbfffffff]
SRAT: Node 0 PXM 0 [mem 0x100000000-0x13fffffff]
SRAT: Node 1 PXM 1 [mem 0x140000000-0x23fffffff]
NUMA: Node 0 [mem 0x00000000-0x0009ffff] + [mem 0x00100000-0x36ffafff] -> [mem 0x00000000-0x36ffafff]
NODE_DATA(0) allocated [mem 0x36a8f000-0x36ab4fff]
Zone ranges:
  DMA      [mem 0x00001000-0x00ffffff]
  DMA32    [mem 0x01000000-0x36ffafff]
  Normal   empty
Movable zone start for each node
Early memory node ranges
  node   0: [mem 0x00001000-0x0009cfff]
  node   0: [mem 0x2f09d000-0x36ffafff]
Initmem setup node 0 [mem 0x00001000-0x36ffafff]
ACPI: PM-Timer IO Port: 0x808
ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
ACPI: NR_CPUS/possible_cpus limit of 1 almost reached. Keeping one slot for boot cpu.  Processor 0/0x0 ignored.
ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] enabled)
ACPI: NR_CPUS/possible_cpus limit of 1 almost reached. Keeping one slot for boot cpu.  Processor 1/0x2 ignored.
ACPI: LAPIC (acpi_id[0x03] lapic_id[0x12] enabled)
ACPI: NR_CPUS/possible_cpus limit of 1 almost reached. Keeping one slot for boot cpu.  Processor 2/0x12 ignored.
ACPI: LAPIC (acpi_id[0x04] lapic_id[0x14] enabled)
ACPI: NR_CPUS/possible_cpus limit of 1 almost reached. Keeping one slot for boot cpu.  Processor 3/0x14 ignored.
ACPI: LAPIC (acpi_id[0x05] lapic_id[0x20] enabled)
ACPI: LAPIC (acpi_id[0x06] lapic_id[0x22] enabled)
ACPI: NR_CPUS/possible_cpus limit of 1 reached.  Processor 5/0x22 ignored.
ACPI: LAPIC (acpi_id[0x07] lapic_id[0x32] enabled)
ACPI: NR_CPUS/possible_cpus limit of 1 reached.  Processor 6/0x32 ignored.
ACPI: LAPIC (acpi_id[0x08] lapic_id[0x34] enabled)
ACPI: NR_CPUS/possible_cpus limit of 1 reached.  Processor 7/0x34 ignored.
ACPI: LAPIC (acpi_id[0x09] lapic_id[0x01] enabled)
ACPI: NR_CPUS/possible_cpus limit of 1 reached.  Processor 8/0x1 ignored.
ACPI: LAPIC (acpi_id[0x0a] lapic_id[0x03] enabled)
ACPI: NR_CPUS/possible_cpus limit of 1 reached.  Processor 9/0x3 ignored.
ACPI: LAPIC (acpi_id[0x0b] lapic_id[0x13] enabled)
ACPI: NR_CPUS/possible_cpus limit of 1 reached.  Processor 10/0x13 ignored.
ACPI: LAPIC (acpi_id[0x0c] lapic_id[0x15] enabled)
ACPI: NR_CPUS/possible_cpus limit of 1 reached.  Processor 11/0x15 ignored.
ACPI: LAPIC (acpi_id[0x0d] lapic_id[0x21] enabled)
ACPI: NR_CPUS/possible_cpus limit of 1 reached.  Processor 12/0x21 ignored.
ACPI: LAPIC (acpi_id[0x0e] lapic_id[0x23] enabled)
ACPI: NR_CPUS/possible_cpus limit of 1 reached.  Processor 13/0x23 ignored.
ACPI: LAPIC (acpi_id[0x0f] lapic_id[0x33] enabled)
ACPI: NR_CPUS/possible_cpus limit of 1 reached.  Processor 14/0x33 ignored.
ACPI: LAPIC (acpi_id[0x10] lapic_id[0x35] enabled)
ACPI: NR_CPUS/possible_cpus limit of 1 reached.  Processor 15/0x35 ignored.
ACPI: LAPIC (acpi_id[0x11] lapic_id[0x90] disabled)
ACPI: LAPIC (acpi_id[0x12] lapic_id[0x91] disabled)
ACPI: LAPIC (acpi_id[0x13] lapic_id[0x92] disabled)
ACPI: LAPIC (acpi_id[0x14] lapic_id[0x93] disabled)
ACPI: LAPIC (acpi_id[0x15] lapic_id[0x94] disabled)
ACPI: LAPIC (acpi_id[0x16] lapic_id[0x95] disabled)
ACPI: LAPIC (acpi_id[0x17] lapic_id[0x96] disabled)
ACPI: LAPIC (acpi_id[0x18] lapic_id[0x97] disabled)
ACPI: LAPIC_NMI (acpi_id[0xff] high dfl lint[0x1])
ACPI: IOAPIC (id[0x06] address[0xfec00000] gsi_base[0])
IOAPIC[0]: apic_id 6, version 32, address 0xfec00000, GSI 0-23
ACPI: IOAPIC (id[0x07] address[0xfec8a000] gsi_base[24])
IOAPIC[1]: apic_id 7, version 32, address 0xfec8a000, GSI 24-47
ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
Using ACPI (MADT) for SMP configuration information
ACPI: HPET id: 0x8086a301 base: 0xfed00000
smpboot: 24 Processors exceeds NR_CPUS limit of 1
smpboot: Allowing 1 CPUs, 0 hotplug CPUs
PM: Registered nosave memory: [mem 0x00000000-0x00000fff]
PM: Registered nosave memory: [mem 0x0009d000-0x0009dfff]
PM: Registered nosave memory: [mem 0x0009e000-0x0009ffff]
PM: Registered nosave memory: [mem 0x000a0000-0x000e6fff]
PM: Registered nosave memory: [mem 0x000e7000-0x000fffff]
PM: Registered nosave memory: [mem 0x00100000-0x2f09cfff]
e820: [mem 0x36ffb000-0xbf78dfff] available for PCI devices
setup_percpu: NR_CPUS:8192 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:2
PERCPU: Embedded 29 pages/cpu @ffff880036600000 s80640 r8192 d29952 u2097152
Built 1 zonelists in Node order, mobility grouping on.  Total pages: 32291
Policy zone: DMA32
Kernel command line: ro root=UUID=bfc88f62-f080-492f-a9f5-17f0c7c6215a rd_NO_LUKS console=ttyS0,115200n8 rd_NO_MD KEYBOARDTYPE=pc KEYTABLE=jp106 LANG=ja_JP.UTF-8 rd_NO_LVM rd_NO_DM intel_iommu=on irqpoll nr_cpus=1 reset_devices cgroup_disable=memory mce=off  memmap=exactmap memmap=627K at 4K memmap=130425K at 770675K elfcorehdr=901100K memmap=4K$0K memmap=9K$631K memmap=100K$924K memmap=8K$3137080K memmap=52K#3137088K memmap=204K#3137140K memmap=64K$3137344K memmap=8268K$3137460K memmap=262144K$3670016K memmap=4K$4175872K memmap=2048K$4192256K
Intel-IOMMU: enabled
Misrouted IRQ fixup and polling support enabled
This may significantly impact system performance
PID hash table entries: 512 (order: 0, 4096 bytes)
AGP: Checking aperture...
AGP: No AGP bridge found
Memory: 101568K/131048K available (5867K kernel code, 1199K rwdata, 2848K rodata, 1748K init, 2104K bss, 29480K reserved, 0K cma-reserved)
Hierarchical RCU implementation.
	RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=1.
RCU: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1
NR_IRQS:524544 nr_irqs:256 16
Spurious LAPIC timer interrupt on cpu 0
Console: colour dummy device 80x25
console [ttyS0] enabled
tsc: Fast TSC calibration using PIT
tsc: Detected 2133.414 MHz processor
Calibrating delay loop (skipped), value calculated using timer frequency.. 4266.82 BogoMIPS (lpj=2133414)
pid_max: default: 32768 minimum: 301
ACPI: Core revision 20141107
ACPI: All ACPI Tables successfully acquired
Security Framework initialized
SELinux:  Initializing.
Dentry cache hash table entries: 16384 (order: 5, 131072 bytes)
Inode-cache hash table entries: 8192 (order: 4, 65536 bytes)
Mount-cache hash table entries: 512 (order: 0, 4096 bytes)
Mountpoint-cache hash table entries: 512 (order: 0, 4096 bytes)
Initializing cgroup subsys devices
Initializing cgroup subsys freezer
Initializing cgroup subsys net_cls
Initializing cgroup subsys blkio
Initializing cgroup subsys perf_event
CPU: Physical Processor ID: 1
CPU: Processor Core ID: 0
Last level iTLB entries: 4KB 512, 2MB 7, 4MB 7
Last level dTLB entries: 4KB 512, 2MB 32, 4MB 32, 1GB 0
Freeing SMP alternatives memory: 20K (ffffffff81ce2000 - ffffffff81ce7000)
ftrace: allocating 23906 entries in 94 pages
dmar: Host address width 40
dmar: DRHD base: 0x000000fbffe000 flags: 0x1
dmar: IOMMU 0: reg_base_addr fbffe000 ver 1:0 cap c90780106f0462 ecap f0207e
dmar: RMRR base: 0x000000000e7000 end: 0x000000000e9fff
dmar: RMRR base: 0x000000bf7ed000 end: 0x000000bf7fffff
dmar: ATSR flags: 0x0
IOAPIC id 6 under DRHD base  0xfbffe000 IOMMU 0
IOAPIC id 7 under DRHD base  0xfbffe000 IOMMU 0
Enabled IRQ remapping in xapic mode
..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
smpboot: CPU0: Intel(R) Xeon(R) CPU           L5630  @ 2.13GHz (fam: 06, model: 2c, stepping: 02)
Performance Events: PEBS fmt1+, 16-deep LBR, Westmere events, Broken BIOS detected, complain to your hardware vendor.
[Firmware Bug]: the BIOS has corrupted hw-PMU resources (MSR 38d is b0)
Intel PMU driver.
perf_event_intel: CPUID marked event: 'bus cycles' unavailable
... version:                3
... bit width:              48
... generic registers:      4
... value mask:             0000ffffffffffff
... max period:             000000007fffffff
... fixed-purpose events:   3
... event mask:             000000070000000f
x86: Booted up 1 node, 1 CPUs
smpboot: Total of 1 processors activated (4266.82 BogoMIPS)
NMI watchdog: enabled on all CPUs, permanently consumes one hw-PMU counter.
devtmpfs: initialized
NET: Registered protocol family 16
cpuidle: using governor ladder
cpuidle: using governor menu
ACPI FADT declares the system doesn't support PCIe ASPM, so disable it
ACPI: bus type PCI registered
acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5
PCI: MMCONFIG for domain 0000 [bus 00-ff] at [mem 0xe0000000-0xefffffff] (base 0xe0000000)
PCI: MMCONFIG at [mem 0xe0000000-0xefffffff] reserved in E820
PCI: Using configuration type 1 for base access
ACPI: Added _OSI(Module Device)
ACPI: Added _OSI(Processor Device)
ACPI: Added _OSI(3.0 _SCP Extensions)
ACPI: Added _OSI(Processor Aggregator Device)
ACPI: Executed 1 blocks of module-level executable AML code
ACPI: Interpreter enabled
ACPI Exception: AE_NOT_FOUND, While evaluating Sleep State [\_S2_] (20141107/hwxface-580)
ACPI Exception: AE_NOT_FOUND, While evaluating Sleep State [\_S3_] (20141107/hwxface-580)
ACPI: (supports S0 S1 S4 S5)
ACPI: Using IOAPIC for interrupt routing
HEST: Table parsing has been initialized.
PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug
ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])
acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI]
acpi PNP0A08:00: _OSC failed (AE_NOT_FOUND); disabling ASPM
acpi PNP0A08:00: ignoring host bridge window [mem 0x000d0000-0x000dffff] (conflicts with Adapter ROM [mem 0x000ce800-0x000d4fff])
PCI host bridge to bus 0000:00
pci_bus 0000:00: root bus resource [bus 00-ff]
pci_bus 0000:00: root bus resource [io  0x0000-0x0cf7]
pci_bus 0000:00: root bus resource [io  0x0d00-0xffff]
pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff]
pci_bus 0000:00: root bus resource [mem 0xc0000000-0xdfffffff]
pci_bus 0000:00: root bus resource [mem 0xf0000000-0xfed8ffff]
pci 0000:00:01.0: System wakeup disabled by ACPI
pci 0000:00:02.0: System wakeup disabled by ACPI
pci 0000:00:03.0: System wakeup disabled by ACPI
pci 0000:00:07.0: System wakeup disabled by ACPI
pci 0000:00:08.0: System wakeup disabled by ACPI
pci 0000:00:09.0: System wakeup disabled by ACPI
pci 0000:00:0a.0: System wakeup disabled by ACPI
pci 0000:00:1a.0: System wakeup disabled by ACPI
pci 0000:00:1a.1: System wakeup disabled by ACPI
pci 0000:00:1a.2: System wakeup disabled by ACPI
pci 0000:00:1a.7: System wakeup disabled by ACPI
pci 0000:00:1c.0: System wakeup disabled by ACPI
pci 0000:00:1d.0: System wakeup disabled by ACPI
pci 0000:00:1d.1: System wakeup disabled by ACPI
pci 0000:00:1d.2: System wakeup disabled by ACPI
pci 0000:00:1d.7: System wakeup disabled by ACPI
pci 0000:00:1e.0: System wakeup disabled by ACPI
pci 0000:01:00.0: System wakeup disabled by ACPI
pci 0000:01:00.1: System wakeup disabled by ACPI
pci 0000:00:01.0: PCI bridge to [bus 01]
pci 0000:02:00.0: System wakeup disabled by ACPI
pci 0000:02:00.1: System wakeup disabled by ACPI
pci 0000:00:02.0: PCI bridge to [bus 02]
pci 0000:00:03.0: PCI bridge to [bus 03]
pci 0000:00:07.0: PCI bridge to [bus 10]
pci 0000:00:08.0: PCI bridge to [bus 20]
pci 0000:00:09.0: PCI bridge to [bus 30]
pci 0000:00:0a.0: PCI bridge to [bus 40]
pci 0000:00:1c.0: PCI bridge to [bus 50]
pci 0000:00:1c.4: PCI bridge to [bus 60]
pci 0000:00:1e.0: PCI bridge to [bus 61] (subtractive decode)
ACPI: PCI Interrupt Link [LNKA] (IRQs 3 4 6 7 *10 11 12 14 15), disabled.
ACPI: PCI Interrupt Link [LNKB] (IRQs *5), disabled.
ACPI: PCI Interrupt Link [LNKC] (IRQs 3 4 6 7 10 *11 12 14 15), disabled.
ACPI: PCI Interrupt Link [LNKD] (IRQs 3 4 6 7 10 11 12 14 *15), disabled.
ACPI: PCI Interrupt Link [LNKE] (IRQs 3 4 6 7 10 11 12 14 15) *0, disabled.
ACPI: PCI Interrupt Link [LNKF] (IRQs 3 4 6 7 10 11 12 *14 15), disabled.
ACPI: PCI Interrupt Link [LNKG] (IRQs 3 4 6 7 10 11 12 14 15) *0, disabled.
ACPI: PCI Interrupt Link [LNKH] (IRQs *3 4 6 7 10 11 12 14 15), disabled.
ACPI: Enabled 2 GPEs in block 00 to 3F
vgaarb: setting as boot device: PCI:0000:60:00.0
vgaarb: device added: PCI:0000:60:00.0,decodes=io+mem,owns=io+mem,locks=none
vgaarb: loaded
vgaarb: bridge control possible 0000:60:00.0
SCSI subsystem initialized
ACPI: bus type USB registered
usbcore: registered new interface driver usbfs
usbcore: registered new interface driver hub
usbcore: registered new device driver usb
PCI: Using ACPI for IRQ routing
PCI: Discovered peer bus fe
PCI host bridge to bus 0000:fe
pci_bus 0000:fe: root bus resource [io  0x0000-0xffff]
pci_bus 0000:fe: root bus resource [mem 0x00000000-0xffffffffff]
pci_bus 0000:fe: No busn resource found for root bus, will use [bus fe-ff]
PCI: Discovered peer bus ff
PCI host bridge to bus 0000:ff
pci_bus 0000:ff: root bus resource [io  0x0000-0xffff]
pci_bus 0000:ff: root bus resource [mem 0x00000000-0xffffffffff]
pci_bus 0000:ff: No busn resource found for root bus, will use [bus ff-ff]
NetLabel: Initializing
NetLabel:  domain hash size = 128
NetLabel:  protocols = UNLABELED CIPSOv4
NetLabel:  unlabeled traffic allowed by default
hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0, 0
hpet0: 4 comparators, 64-bit 14.318180 MHz counter
Switched to clocksource hpet
pnp: PnP ACPI init
system 00:00: [mem 0xfbf00000-0xfbffffff] could not be reserved
system 00:00: [mem 0xfc000000-0xfcffffff] has been reserved
system 00:00: [mem 0xfd000000-0xfdffffff] has been reserved
system 00:00: [mem 0xfe000000-0xfebfffff] has been reserved
system 00:00: [mem 0xfec8a000-0xfec8afff] could not be reserved
system 00:00: [mem 0xfed10000-0xfed10fff] has been reserved
system 00:04: [io  0x0ca0-0x0ca1] has been reserved
system 00:04: [io  0x0ca4-0x0ca7] has been reserved
system 00:04: [io  0x0caa-0x0caf] has been reserved
system 00:04: [io  0x04d0-0x04d1] has been reserved
system 00:04: [io  0x0800-0x087f] has been reserved
system 00:04: [io  0x0500-0x057f] has been reserved
system 00:04: [mem 0xfed1c000-0xfed1ffff] has been reserved
system 00:04: [mem 0xfed20000-0xfed3ffff] has been reserved
system 00:04: [mem 0xfed40000-0xfed8ffff] has been reserved
system 00:05: [mem 0xfec00000-0xfec00fff] could not be reserved
system 00:05: [mem 0xfee00000-0xfee00fff] has been reserved
system 00:07: [mem 0xe0000000-0xefffffff] has been reserved
system 00:08: [mem 0x00000000-0x0009ffff] could not be reserved
system 00:08: [mem 0x000e0000-0x000fffff] could not be reserved
system 00:08: [mem 0x00100000-0xbfffffff] could not be reserved
system 00:08: [mem 0xfed90000-0xffffffff] could not be reserved
pnp: PnP ACPI: found 9 devices
pci 0000:00:01.0: PCI bridge to [bus 01]
pci 0000:00:01.0:   bridge window [io  0xa000-0xafff]
pci 0000:00:01.0:   bridge window [mem 0xfa800000-0xfa8fffff]
pci 0000:00:02.0: PCI bridge to [bus 02]
pci 0000:00:02.0:   bridge window [io  0xb000-0xbfff]
pci 0000:00:02.0:   bridge window [mem 0xfa900000-0xfa9fffff]
pci 0000:00:03.0: PCI bridge to [bus 03]
pci 0000:00:03.0:   bridge window [io  0xc000-0xcfff]
pci 0000:00:03.0:   bridge window [mem 0xfaa00000-0xfaafffff]
pci 0000:00:03.0:   bridge window [mem 0xc0000000-0xc01fffff 64bit pref]
pci 0000:00:07.0: PCI bridge to [bus 10]
pci 0000:00:08.0: PCI bridge to [bus 20]
pci 0000:00:09.0: PCI bridge to [bus 30]
pci 0000:00:09.0:   bridge window [io  0xd000-0xdfff]
pci 0000:00:09.0:   bridge window [mem 0xfab00000-0xfabfffff]
pci 0000:00:0a.0: PCI bridge to [bus 40]
pci 0000:00:1c.0: PCI bridge to [bus 50]
pci 0000:00:1c.0:   bridge window [io  0xe000-0xefff]
pci 0000:00:1c.0:   bridge window [mem 0xfac00000-0xfaffffff]
pci 0000:00:1c.0:   bridge window [mem 0xc0200000-0xc03fffff 64bit pref]
pci 0000:60:00.0: BAR 6: assigned [mem 0xfb800000-0xfb80ffff pref]
pci 0000:00:1c.4: PCI bridge to [bus 60]
pci 0000:00:1c.4:   bridge window [io  0x1000-0x1fff]
pci 0000:00:1c.4:   bridge window [mem 0xfb000000-0xfbefffff]
pci 0000:00:1c.4:   bridge window [mem 0xf9000000-0xf9ffffff 64bit pref]
pci 0000:00:1e.0: PCI bridge to [bus 61]
NET: Registered protocol family 2
TCP established hash table entries: 1024 (order: 1, 8192 bytes)
TCP bind hash table entries: 1024 (order: 2, 16384 bytes)
TCP: Hash tables configured (established 1024 bind 1024)
TCP: reno registered
UDP hash table entries: 256 (order: 1, 8192 bytes)
UDP-Lite hash table entries: 256 (order: 1, 8192 bytes)
NET: Registered protocol family 1
pci 0000:01:00.0: Disabling L0s
pci 0000:01:00.0: can't disable ASPM; OS doesn't have ASPM control
pci 0000:01:00.1: Disabling L0s
pci 0000:01:00.1: can't disable ASPM; OS doesn't have ASPM control
pci 0000:02:00.0: Disabling L0s
pci 0000:02:00.0: can't disable ASPM; OS doesn't have ASPM control
pci 0000:02:00.1: Disabling L0s
pci 0000:02:00.1: can't disable ASPM; OS doesn't have ASPM control
Trying to unpack rootfs image as initramfs...
Freeing initrd memory: 5352K (ffff880036ab5000 - ffff880036fef000)
IOMMU Skip disabling iommu hardware translations
IOMMU Copying translate tables from panicked kernel
IOMMU: root_cache:0xffff880033ed8000 phys:0x0000bac2f000
IOMMU:0 Domain ids from panicked kernel:
DID did:12(0x000c)
DID did:11(0x000b)
DID did:10(0x000a)
DID did:9(0x0009)
DID did:24(0x0018)
DID did:23(0x0017)
DID did:22(0x0016)
DID did:21(0x0015)
DID did:8(0x0008)
DID did:7(0x0007)
DID did:6(0x0006)
DID did:5(0x0005)
DID did:4(0x0004)
DID did:3(0x0003)
DID did:2(0x0002)
DID did:1(0x0001)
DID did:0(0x0000)
DID did:20(0x0014)
DID did:19(0x0013)
DID did:18(0x0012)
DID did:17(0x0011)
DID did:16(0x0010)
DID did:15(0x000f)
DID did:14(0x000e)
DID did:13(0x000d)
----------------------------------------
IOMMU: dmar0 using Queued invalidation
PCI-DMA: Intel(R) Virtualization Technology for Directed I/O
futex hash table entries: 256 (order: 2, 16384 bytes)
audit: initializing netlink subsys (disabled)
audit: type=2000 audit(1420624212.480:1): initialized
HugeTLB registered 2 MB page size, pre-allocated 0 pages
VFS: Disk quotas dquot_6.5.2
VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
bounce: pool size: 64 pages
Block layer SCSI generic (bsg) driver version 0.4 loaded (major 252)
dmar: DRHD: handling fault status reg 2
dmar: DMAR:[DMA Write] Request device [01:00.0] fault addr ffdf2000 
DMAR:[fault reason 05] PTE Write access is not set
io scheduler noop registered
io scheduler deadline registered
io scheduler cfq registered (default)
pci_hotplug: PCI Hot Plug PCI Core version: 0.5
pciehp: PCI Express Hot Plug Controller Driver version: 0.4
input: Power Button as /devices/LNXSYSTM:00/LNXSYBUS:00/PNP0C0C:00/input/input0
ACPI: Power Button [PWRB]
input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input1
ACPI: Power Button [PWRF]
APEI: Can not request [mem 0xbf7b3d3a-0xbf7b3d3b] for APEI ERST registers
[Firmware Warn]: GHES: Poll interval is 0 for generic hardware error source: 1, disabled.
GHES: APEI firmware first mode is enabled by WHEA _OSC.
Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
serial 00:02: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A
Non-volatile memory driver v1.3
Linux agpgart interface v0.103
brd: module loaded
loop: module loaded
libphy: Fixed MDIO Bus: probed
ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
ehci-pci: EHCI PCI platform driver
ehci-pci 0000:00:1a.7: EHCI Host Controller
ehci-pci 0000:00:1a.7: new USB bus registered, assigned bus number 1
ehci-pci 0000:00:1a.7: debug port 1
ehci-pci 0000:00:1a.7: irq 18, io mem 0xfa7de000
ehci-pci 0000:00:1a.7: USB 2.0 started, EHCI 1.00
usb usb1: New USB device found, idVendor=1d6b, idProduct=0002
usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
usb usb1: Product: EHCI Host Controller
dmar: DRHD: handling fault status reg 102
dmar: DMAR:[DMA Write] Request device [01:00.0] fault addr ffce8000 
DMAR:[fault reason 05] PTE Write access is not set
usb usb1: Manufacturer: Linux 3.19.0-rc3 ehci_hcd
usb usb1: SerialNumber: 0000:00:1a.7
hub 1-0:1.0: USB hub found
hub 1-0:1.0: 6 ports detected
ehci-pci 0000:00:1d.7: EHCI Host Controller
ehci-pci 0000:00:1d.7: new USB bus registered, assigned bus number 2
ehci-pci 0000:00:1d.7: debug port 1
ehci-pci 0000:00:1d.7: irq 23, io mem 0xfa7dc000
ehci-pci 0000:00:1d.7: USB 2.0 started, EHCI 1.00
usb usb2: New USB device found, idVendor=1d6b, idProduct=0002
usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
usb usb2: Product: EHCI Host Controller
usb usb2: Manufacturer: Linux 3.19.0-rc3 ehci_hcd
usb usb2: SerialNumber: 0000:00:1d.7
hub 2-0:1.0: USB hub found
hub 2-0:1.0: 6 ports detected
ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
ohci-pci: OHCI PCI platform driver
uhci_hcd: USB Universal Host Controller Interface driver
uhci_hcd 0000:00:1a.0: UHCI Host Controller
uhci_hcd 0000:00:1a.0: new USB bus registered, assigned bus number 3
uhci_hcd 0000:00:1a.0: detected 2 ports
uhci_hcd 0000:00:1a.0: irq 16, io base 0x00009c00
usb usb3: New USB device found, idVendor=1d6b, idProduct=0001
usb usb3: New USB device strings: Mfr=3, Product=2, SerialNumber=1
usb usb3: Product: UHCI Host Controller
usb usb3: Manufacturer: Linux 3.19.0-rc3 uhci_hcd
usb usb3: SerialNumber: 0000:00:1a.0
hub 3-0:1.0: USB hub found
hub 3-0:1.0: 2 ports detected
uhci_hcd 0000:00:1a.1: UHCI Host Controller
uhci_hcd 0000:00:1a.1: new USB bus registered, assigned bus number 4
uhci_hcd 0000:00:1a.1: detected 2 ports
uhci_hcd 0000:00:1a.1: irq 21, io base 0x00009880
usb usb4: New USB device found, idVendor=1d6b, idProduct=0001
usb usb4: New USB device strings: Mfr=3, Product=2, SerialNumber=1
usb usb4: Product: UHCI Host Controller
usb usb4: Manufacturer: Linux 3.19.0-rc3 uhci_hcd
usb usb4: SerialNumber: 0000:00:1a.1
hub 4-0:1.0: USB hub found
hub 4-0:1.0: 2 ports detected
uhci_hcd 0000:00:1a.2: UHCI Host Controller
uhci_hcd 0000:00:1a.2: new USB bus registered, assigned bus number 5
uhci_hcd 0000:00:1a.2: detected 2 ports
uhci_hcd 0000:00:1a.2: irq 19, io base 0x00009800
usb usb5: New USB device found, idVendor=1d6b, idProduct=0001
usb usb5: New USB device strings: Mfr=3, Product=2, SerialNumber=1
usb usb5: Product: UHCI Host Controller
usb usb5: Manufacturer: Linux 3.19.0-rc3 uhci_hcd
usb usb5: SerialNumber: 0000:00:1a.2
hub 5-0:1.0: USB hub found
hub 5-0:1.0: 2 ports detected
uhci_hcd 0000:00:1d.0: UHCI Host Controller
uhci_hcd 0000:00:1d.0: new USB bus registered, assigned bus number 6
uhci_hcd 0000:00:1d.0: detected 2 ports
uhci_hcd 0000:00:1d.0: irq 23, io base 0x00009480
usb usb6: New USB device found, idVendor=1d6b, idProduct=0001
dmar: DRHD: handling fault status reg 202
dmar: DMAR:[DMA Write] Request device [01:00.0] fault addr ffef3000 
DMAR:[fault reason 05] PTE Write access is not set
usb usb6: New USB device strings: Mfr=3, Product=2, SerialNumber=1
usb usb6: Product: UHCI Host Controller
usb usb6: Manufacturer: Linux 3.19.0-rc3 uhci_hcd
usb usb6: SerialNumber: 0000:00:1d.0
hub 6-0:1.0: USB hub found
hub 6-0:1.0: 2 ports detected
uhci_hcd 0000:00:1d.1: UHCI Host Controller
usb 1-2: new high-speed USB device number 2 using ehci-pci
uhci_hcd 0000:00:1d.1: new USB bus registered, assigned bus number 7
uhci_hcd 0000:00:1d.1: detected 2 ports
uhci_hcd 0000:00:1d.1: irq 19, io base 0x00009400
usb usb7: New USB device found, idVendor=1d6b, idProduct=0001
usb usb7: New USB device strings: Mfr=3, Product=2, SerialNumber=1
usb usb7: Product: UHCI Host Controller
usb usb7: Manufacturer: Linux 3.19.0-rc3 uhci_hcd
usb usb7: SerialNumber: 0000:00:1d.1
hub 7-0:1.0: USB hub found
hub 7-0:1.0: 2 ports detected
uhci_hcd 0000:00:1d.2: UHCI Host Controller
uhci_hcd 0000:00:1d.2: new USB bus registered, assigned bus number 8
uhci_hcd 0000:00:1d.2: detected 2 ports
uhci_hcd 0000:00:1d.2: irq 18, io base 0x00009080
usb usb8: New USB device found, idVendor=1d6b, idProduct=0001
usb usb8: New USB device strings: Mfr=3, Product=2, SerialNumber=1
usb usb8: Product: UHCI Host Controller
usb usb8: Manufacturer: Linux 3.19.0-rc3 uhci_hcd
usb usb8: SerialNumber: 0000:00:1d.2
hub 8-0:1.0: USB hub found
hub 8-0:1.0: 2 ports detected
i8042: PNP: No PS/2 controller found. Probing ports directly.
dmar: DRHD: handling fault status reg 302
dmar: DMAR:[DMA Write] Request device [01:00.0] fault addr ffce7000 
DMAR:[fault reason 05] PTE Write access is not set
i8042: Failed to disable AUX port, but continuing anyway... Is this a SiS?
i8042: If AUX port is really absent please use the 'i8042.noaux' option
dmar: DRHD: handling fault status reg 402
dmar: DMAR:[DMA Write] Request device [01:00.0] fault addr ffce5000 
DMAR:[fault reason 05] PTE Write access is not set
sched: RT throttling activated
serio: i8042 KBD port at 0x60,0x64 irq 1
mousedev: PS/2 mouse device common for all mice
rtc_cmos 00:01: RTC can wake from S4
rtc_cmos 00:01: rtc core: registered rtc_cmos as rtc0
rtc_cmos 00:01: alarms up to one month, y3k, 114 bytes nvram, hpet irqs
hidraw: raw HID events driver (C) Jiri Kosina
usb 1-2: New USB device found, idVendor=04b4, idProduct=6560
usb 1-2: New USB device strings: Mfr=0, Product=0, SerialNumber=0
usbcore: registered new interface driver usbhid
usbhid: USB HID core driver
drop_monitor: Initializing network drop monitor service
hub 1-2:1.0: USB hub found
TCP: cubic registered
hub 1-2:1.0: 4 ports detected
Initializing XFRM netlink socket
NET: Registered protocol family 17
mce: Unable to init device /dev/mcelog (rc: -5)
registered taskstats version 1
ima: No TPM chip found, activating TPM-bypass!
rtc_cmos 00:01: setting system clock to 2015-01-07 09:50:20 UTC (1420624220)
Freeing unused kernel memory: 1748K (ffffffff81b2d000 - ffffffff81ce2000)
Write protecting the kernel read-only data: 10240k
Freeing unused kernel memory: 264K (ffff8800305be000 - ffff880030600000)
Freeing unused kernel memory: 1248K (ffff8800308c8000 - ffff880030a00000)
Mounting proc filesystem
Mounting sysfs filesystem
Creating /dev
Creating initial device nodes
setfont: KDFONTOP: Inappropriate ioctl for dedevice-mapper: uevent: version 1.0.3
vice
Free memordevice-mapper: ioctl: 4.29.0-ioctl (2014-10-28) initialised: dm-devel at redhat.com
y/Total memory (free %): 76920 / 110200 ( 69.8004 )
Loading dm-mod.ko module
Loading dm-log.koip_tables: (C) 2000-2006 Netfilter Core Team
 module
Loading dm-region-hash.nf_conntrack version 0.5.0 (860 buckets, 3440 max)
ko module
Loadiip6_tables: (C) 2000-2006 Netfilter Core Team
ng dm-mirror.ko module
Loading dm-zero.ko module
Loading dm-buNET: Registered protocol family 10
fio.ko module
Loading dm-snapshot.ko module
Lotun: Universal TUN/TAP device driver, 1.6
ading cpufreq_ontun: (C) 1999-2004 Max Krasnyansky <maxk at qualcomm.com>
demand.ko module
Loading nf_reject_ipv4.ko module
Loading nf_defrag_ipv4.ko moiTCO_vendor_support: vendor-support=0
dule
Loading ip_tables.ko modulinput: PC Speaker as /devices/platform/pcspkr/input/input3
e
Loading nf_conntrack.ko modulipmi message handler version 39.2
e
Loading ip6_tables.ko module
genirq: Flags mismatch irq 0. 00000080 (i801_smbus) vs. 00015a00 (timer)
CPU: 0 PID: 145 Comm: insmod Not tainted 3.19.0-rc3 #36
Hardware name: FUJITSU-SV PRIMERGY BX920 S2/D3030, BIOS 080015 Rev.3D81.3030 02/10/2012
 ffff8800335d7940 ffff880033503a48 ffffffff815b200a 0000000000015a00
 ffff88003603d600 ffff880033503aa8 ffffffff810a8ccc ffff880033503a98
 0000000000000246 ffff880033503aa8 ffff880033ba5c00 ffff88003603d600
Call Trace:
 [<ffffffff815b200a>] dump_stack+0x48/0x5e
 [<ffffffff810a8ccc>] __setup_irq+0x39c/0x4f0
 [<ffffffff810a9640>] request_threaded_irq+0x100/0x170
 [<ffffffffa0195c30>] ? dmi_check_onboard_devices+0x1a0/0x1a0 [i2c_i801]
 [<ffffffffa0195964>] i801_probe+0x4b4/0x5e0 [i2c_i801]
 [<ffffffff815b4296>] ? mutex_lock+0x16/0x40
 [<ffffffff812d16bc>] local_pci_probe+0x4c/0xb0
 [<ffffffff812d17a9>] pci_call_probe+0x89/0xb0
 [<ffffffff812d15fe>] ? pci_match_device+0xde/0x110
 [<ffffffff812d1a89>] pci_device_probe+0x79/0xa0
 [<ffffffff8139af02>] ? driver_sysfs_add+0x82/0xb0
 [<ffffffff8139b181>] really_probe+0x81/0x350
 [<ffffffff8139b497>] driver_probe_device+0x47/0xa0
 [<ffffffff8139b59b>] __driver_attach+0xab/0xb0
 [<ffffffff8139b4f0>] ? driver_probe_device+0xa0/0xa0
 [<ffffffff8139b4f0>] ? driver_probe_device+0xa0/0xa0
 [<ffffffff81399374>] bus_for_each_dev+0x94/0xb0
 [<ffffffff8139ae2e>] driver_attach+0x1e/0x20
 [<ffffffff8139a890>] bus_add_driver+0x1b0/0x250
 [<ffffffff8139bc34>] driver_register+0x64/0xf0
 [<ffffffff812d1b7c>] __pci_register_driver+0x4c/0x50
 [<ffffffffa019a0a5>] i2c_i801_init+0xa5/0x1000 [i2c_i801]
 [<ffffffffa019a000>] ? 0xffffffffa019a000
 [<ffffffff81000287>] do_one_initcall+0xb7/0x1d0
 [<ffffffff81175352>] ? __vunmap+0xc2/0x110
 [<ffffffff810cf600>] do_init_module+0x30/0x1a0
 [<ffffffff810d1dee>] load_module+0x4ee/0x620
 [<ffffffff810ceeb0>] ? mod_sysfs_teardown+0x150/0x150
 [<ffffffff81175bf5>] ? __vmalloc_node+0x35/0x40
 [<ffffffff810ce7f0>] ? module_sect_show+0x30/0x30
 [<ffffffff810d20a4>] SyS_init_module+0x94/0xc0
 [<ffffffff815b6152>] system_call_fastpath+0x12/0x17

Loading ipv6.koi801_smbus 0000:00:1f.3: Failed to allocate irq 0: -16
 module
Loadingi801_smbus 0000:00:1f.3: SMBus using polling
 macvlan.ko module
Loading vhosEDAC MC: Ver: 3.0.0
t.ko module
Loading tun.ko modudca service started, version 1.12.1
le
Loading kvm.ko module
Loadipps_core: LinuxPPS API ver. 1 registered
ng uinput.ko modpps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti at linux.it>
ule
Loading iTCO_vendor_support.ko module
Loading serio_raw.koFusion MPT base driver 3.04.20
 module
LoadingCopyright (c) 1999-2008 LSI Corporation
 pcspkr.ko module
Loading ipmi_msghandler.ko module
Loading tpm_infineon.ko momegasas: 06.805.06.01-rc1
dule
Loading i2megasas: 0x1000:0x0079:0x1734:0x1176: c-i801.ko modulebus 3:slot 0:func 0

Loading mfd-comegaraid_sas 0000:03:00.0: Can't allocate Firmware crash dump DMA buffer
re.ko module
Lomegasas: Waiting for FW to come to ready state
ading edac_core.ko module
Loading sg.ko module
Loading dca.ko module
Loading i2c-algo-bit.ko module
Loading pps_core.ko modudmar: DRHD: handling fault status reg 502
dmar: DMAR:[DMA Write] Request device [03:00.0] fault addr ffdfe000 
DMAR:[fault reason 05] PTE Write access is not set
le
Loading acpiusb 5-1: new full-speed USB device number 2 using uhci_hcd
-cpufreq.ko modumegasas: FW now in Ready state
le
insmod: can'megaraid_sas 0000:03:00.0: [scsi0]: FW supports<0> MSIX vector,Online CPUs: <1>,Current MSIX <1>
t insert '/lib/modules/3.19.0-rc3/acpi-cpufreq.ko': No such device
Loading jbd2.ko module
Loading mbcache.ko module
Loading mptbase.ko module
Loading scsi_transport_sas.ko module
Loading scsi_transport_fc.ko module
Loading megaraid_sas.ko module
megasas_init_mfi: fw_support_ieee=0
megasas: INIT adapter done
megaraid_sas 0000:03:00.0: Controller type: MR,Memory size is: 512MB
scsi host0: LSI SAS based MegaRAID driver
usb 5-1: New USB device found, idVendor=0000, idProduct=0000
usb 5-1: New USB device strings: Mfr=1, Product=2, SerialNumber=3
usb 5-1: Product: iRMC USB Device
usb 5-1: Manufacturer: Fujitsu
usb 5-1: SerialNumber: 401B794770E1CC
scsi 0:2:0:0: Direct-Access     LSI      RAID 5/6 SAS 6G  2.13 PQ: 0 ANSI: 5
scsi 0:2:1:0: Direct-Access     LSI      RAID 5/6 SAS 6G  2.13 PQ: 0 ANSI: 5
scsi 0:2:2:0: Direct-Access     LSI      RAID 5/6 SAS 6G  2.13 PQ: 0 ANSI: 5
scsi 0:2:3:0: Direct-Access     LSI      RAID 5/6 SAS 6G  2.13 PQ: 0 ANSI: 5
scsi 0:2:4:0: Direct-Access     LSI      RAID 5/6 SAS 6G  2.13 PQ: 0 ANSI: 5
scsi 0:2:5:0: Direct-Access     LSI      RAID 5/6 SAS 6G  2.13 PQ: 0 ANSI: 5
scsi 0:2:6:0: Direct-Access     LSI      RAID 5/6 SAS 6G  2.13 PQ: 0 ANSI: 5
input: Fujitsu iRMC USB Device as /devices/pci0000:00/0000:00:1a.2/usb5/5-1/5-1:1.0/0003:0000:0000.0001/input/input4
scsi 0:2:7:0: Direct-Access     LSI      RAID 5/6 SAS 6G  2.13 PQ: 0 ANSI: 5
sd 0:2:0:0: [sda] 285671424 512-byte logical blocks: (146 GB/136 GiB)
sd 0:2:0:0: [sda] 4096-byte physical blocks
sd 0:2:0:0: Attached scsi generic sg0 type 0
sd 0:2:0:0: [sda] Write Protect is off
sd 0:2:1:0: [sdb] 285671424 512-byte logical blocks: (146 GB/136 GiB)
sd 0:2:1:0: [sdb] 4096-byte physical blocks
sd 0:2:1:0: Attached scsi generic sg1 type 0
sd 0:2:0:0: [sda] Write cache: enabled, read cache: disabled, doesn't support DPO or FUA
hid-generic 0003:0000:0000.0001: input,hidraw0: USB HID v1.11 Keyboard [Fujitsu iRMC USB Device] on usb-0000:00:1a.2-1/input0
sd 0:2:1:0: [sdb] Write Protect is off
sd 0:2:2:0: [sdc] 285671424 512-byte logical blocks: (146 GB/136 GiB)
sd 0:2:2:0: [sdc] 4096-byte physical blocks
sd 0:2:2:0: Attached scsi generic sg2 type 0
sd 0:2:1:0: [sdb] Write cache: enabled, read cache: disabled, doesn't support DPO or FUA
sd 0:2:2:0: [sdc] Write Protect is off
sd 0:2:3:0: [sdd] 285671424 512-byte logical blocks: (146 GB/136 GiB)
sd 0:2:3:0: [sdd] 4096-byte physical blocks
sd 0:2:3:0: Attached scsi generic sg3 type 0
sd 0:2:2:0: [sdc] Write cache: enabled, read cache: disabled, doesn't support DPO or FUA
sd 0:2:3:0: [sdd] Write Protect is off
 sda: sda1
sd 0:2:4:0: [sde] 285671424 512-byte logical blocks: (146 GB/136 GiB)
sd 0:2:4:0: [sde] 4096-byte physical blocks
sd 0:2:4:0: Attached scsi generic sg4 type 0
sd 0:2:0:0: [sda] Attached SCSI disk
Switched to clocksource tsc
input: Fujitsu iRMC USB Device as /devices/pci0000:00/0000:00:1a.2/usb5/5-1/5-1:1.1/0003:0000:0000.0002/input/input5
sd 0:2:3:0: [sdd] Write cache: enabled, read cache: disabled, doesn't support DPO or FUA
sd 0:2:4:0: [sde] Write Protect is off
sd 0:2:5:0: [sdf] 285671424 512-byte logical blocks: (146 GB/136 GiB)
sd 0:2:5:0: [sdf] 4096-byte physical blocks
 sdb: sdb1
sd 0:2:5:0: Attached scsi generic sg5 type 0
sd 0:2:1:0: [sdb] Attached SCSI disk
sd 0:2:4:0: [sde] Write cache: enabled, read cache: disabled, doesn't support DPO or FUA
 sdc: sdc1 sdc2
sd 0:2:6:0: [sdg] 285671424 512-byte logical blocks: (146 GB/136 GiB)
sd 0:2:6:0: [sdg] 4096-byte physical blocks
sd 0:2:6:0: Attached scsi generic sg6 type 0
sd 0:2:2:0: [sdc] Attached SCSI disk
sd 0:2:5:0: [sdf] Write Protect is off
hid-generic 0003:0000:0000.0002: input,hidraw1: USB HID v1.11 Mouse [Fujitsu iRMC USB Device] on usb-0000:00:1a.2-1/input1
sd 0:2:7:0: [sdh] 285671424 512-byte logical blocks: (146 GB/136 GiB)
sd 0:2:7:0: [sdh] 4096-byte physical blocks
 sdd: sdd1 sdd2
sd 0:2:7:0: Attached scsi generic sg7 type 0
sd 0:2:5:0: [sdf] Write cache: enabled, read cache: disabled, doesn't support DPO or FUA
sd 0:2:3:0: [sdd] Attached SCSI disk
sd 0:2:6:0: [sdg] Write Protect is off
sd 0:2:7:0: [sdh] Write Protect is off
sd 0:2:6:0: [sdg] Write cache: enabled, read cache: disabled, doesn't support DPO or FUA
sd 0:2:7:0: [sdh] Write cache: enabled, read cache: disabled, doesn't support DPO or FUA
 sde: sde1 sde2 sde3 sde4 < sde5 sde6 sde7 sde8 >
 sdf: sdf1 sdf2
sd 0:2:5:0: [sdf] Attached SCSI disk
sd 0:2:4:0: [sde] Attached SCSI disk
 sdg: unknown partition table
 sdh: unknown partition table
sd 0:2:6:0: [sdg] Attached SCSI disk
sd 0:2:7:0: [sdh] Attached SCSI disk
Loading ipt_REJECT.ko module
Loading nf_conntrack_ipv4.ko module
Loading iptable_filter.ko modkvm: VM_EXIT_LOAD_IA32_PERF_GLOBAL_CTRL does not work properly. Using workaround
ule
Loading nf_iTCO_wdt: Intel TCO WatchDog Timer Driver v1.11
reject_ipv6.ko mIPMI System Interface driver.
odule
Loading nipmi_si: probing via ACPI
f_defrag_ipv6.koipmi_si 00:06: [io  0x0ca2-0x0ca3] regsize 1 spacing 1 irq 0
 module
Loadingipmi_si: Adding ACPI-specified kcs state machine xt_state.ko mod
ule
Loading ip6ipmi_si: probing via SMBIOS
table_filter.ko ipmi_si: SMBIOS: io 0xca2 regsize 1 spacing 1 irq 0
ipmi_si: Adding SMBIOS-specified kcs state machinemodule
Loading  duplicate interface
macvtap.ko modulipmi_si: Trying ACPI-specified kcs state machine at i/o address 0xca2, slave address 0x0, irq 0
e
Loading kvm-intel.ko module
Loading iTCO_wdt.ko module
Loading ipmi_si.ko module
ipmi_si 00:06: Found new BMC (man_id: 0x002880, prod_id: 0x0276, dev_id: 0x32)
ipmi_si 00:06: IPMI kcs interface initialized
Loading lpc_ich.iTCO_wdt: Found a ICH10 TCO device (Version=2, TCOBASE=0x0860)
ko module
iTCO_wdt: initialized. heartbeat=30 sec (nowayout=0)
ACPI Warning: SystemIO range 0x0000000000000828-0x000000000000082f conflicts with OpRegion 0x0000000000000800-0x000000000000084f (\PMRG) (20141107/utaddress-258)
ACPI: If an ACPI driver is available for this device, you should use it instead of the native driver
ACPI Warning: SystemIO range 0x0000000000000500-0x000000000000052f conflicts with OpRegion 0x0000000000000500-0x000000000000051b (\_SI_.BLNK) (20141107/utaddress-258)
ACPI: If an ACPI driver is available for this device, you should use it instead of the native driver
lpc_ich: Resource conflict(s) found affecting gpio_ich
Loading ioatdma.ioatdma: Intel(R) QuickData Technology Driver 4.00
ko module
Loading i7core_edac.ko module
EDAC MC1: Giving out device to module i7core_edac.c controller i7 core #1: DEV 0000:fe:03.0 (POLLED)
EDAC PCI0: Giving out device to module i7core_edac controller EDAC PCI controller: DEV 0000:fe:03.0 (POLLED)
EDAC MC0: Giving out device to module i7core_edac.c controller i7 core #0: DEV 0000:ff:03.0 (POLLED)
EDAC PCI1: Giving out device to module i7core_edac controller EDAC PCI controller: DEV 0000:ff:03.0 (POLLED)
EDAC i7core: Driver loaded, 2 memory controller(s) found.
Loading ptp.ko mPTP clock support registered
odule
Loading ext4.ko module
Loading mptscsih.ko module
Loading lpfc.ko module
Emulex LightPulse Fibre Channel SCSI driver 10.4.8000.0.
Copyright(c) 2004-2014 Emulex.  All rights reserved.
scsi host1: Emulex LPe12000 PCIe Fibre Channel Adapter  on PCI bus 30 device 00 irq 40
scsi host2: Emulex LPe12000 PCIe Fibre Channel Adapter  on PCI bus 30 device 01 irq 43
Loading ip6t_REJECT.ko module
Loading nf_conntrack_ipv6.ko moduigb: Intel(R) Gigabit Ethernet Network Driver - version 5.2.15-k
le
Loading vhosigb: Copyright (c) 2007-2014 Intel Corporation.
t_net.ko module
Loading igb.ko module
igb 0000:01:00.0: DCA enabled
igb 0000:01:00.0: Intel(R) Gigabit Ethernet Network Connection
igb 0000:01:00.0: eth0: (PCIe:2.5Gb/s:Width x2) c8:0a:a9:9d:fa:52
igb 0000:01:00.0: eth0: PBA No: Unknown
igb 0000:01:00.0: Using MSI-X interrupts. 1 rx queue(s), 1 tx queue(s)
igb 0000:01:00.1: DCA enabled
igb 0000:01:00.1: Intel(R) Gigabit Ethernet Network Connection
igb 0000:01:00.1: eth1: (PCIe:2.5Gb/s:Width x2) c8:0a:a9:9d:fa:53
igb 0000:01:00.1: eth1: PBA No: Unknown
igb 0000:01:00.1: Using MSI-X interrupts. 1 rx queue(s), 1 tx queue(s)
igb 0000:02:00.0: DCA enabled
igb 0000:02:00.0: Intel(R) Gigabit Ethernet Network Connection
igb 0000:02:00.0: eth2: (PCIe:2.5Gb/s:Width x2) c8:0a:a9:9d:fa:54
igb 0000:02:00.0: eth2: PBA No: Unknown
igb 0000:02:00.0: Using MSI-X interrupts. 1 rx queue(s), 1 tx queue(s)
igb 0000:02:00.1: DCA enabled
igb 0000:02:00.1: Intel(R) Gigabit Ethernet Network Connection
igb 0000:02:00.1: eth3: (PCIe:2.5Gb/s:Width x2) c8:0a:a9:9d:fa:55
igb 0000:02:00.1: eth3: PBA No: Unknown
igb 0000:02:00.1: Using MSI-X interrupts. 1 rx queue(s), 1 tx queue(s)
Loading mptsas.kFusion MPT SAS Host driver 3.04.20
o module
mptbase: ioc0: Initiating bringup
ioc0: LSISAS1064E B3: Capabilities={Initiator}
scsi host3: ioc0: LSISAS1064E B3, FwRev=011e0000h, Ports=1, MaxQ=277, IRQ=16
mptsas: ioc0: attaching sata device: fw_channel 0, fw_id 0, phy 0, sas_addr 0x1221000000000000
scsi 3:0:0:0: Direct-Access     ATA      ST9160511NS      FTD4 PQ: 0 ANSI: 5
sd 3:0:0:0: Attached scsi generic sg8 type 0
sd 3:0:0:0: [sdi] 312581808 512-byte logical blocks: (160 GB/149 GiB)
mptsas: ioc0: attaching sata device: fw_channel 0, fw_id 1, phy 1, sas_addr 0x1221000001000000
scsi 3:0:1:0: Direct-Access     ATA      ST9160511NS      FTD4 PQ: 0 ANSI: 5
sd 3:0:1:0: Attached scsi generic sg9 type 0
sd 3:0:1:0: [sdj] 312581808 512-byte logical blocks: (160 GB/149 GiB)
sd 3:0:0:0: [sdi] Write Protect is off
sd 3:0:0:0: [sdi] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
sd 3:0:1:0: [sdj] Write Protect is off
sd 3:0:1:0: [sdj] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
 sdi: sdi1 sdi2 sdi3
random: nonblocking pool is initialized
 sdj: sdj1 sdj2 sdj3 sdj4 < sdj5 sdj6 >
sd 3:0:0:0: [sdi] Attached SCSI disk
sd 3:0:1:0: [sdj] Attached SCSI disk
Waiting for required block device discovery
Waiting for device with scsi_ids: 360030057014aa330182f86352027e114 ...
Creating block device loop0
Creating block device loop1
Creating block device loop2
Creating block device loop3
Creating block device loop4
Creating block device loop5
Creating block device loop6
Creating block device loop7
Creating block device ram0
Creating block device ram1
Creating block device ram10
Creating block device ram11
Creating block device ram12
Creating block device ram13
Creating block device ram14
Creating block device ram15
Creating block device ram2
Creating block device ram3
Creating block device ram4
Creating block device ram5
Creating block device ram6
Creating block device ram7
Creating block device ram8
Creating block device ram9
Creating block d sda: sda1
evice sda
Creating block device sdb
 sdb: sdb1
Creating block d sdc: sdc1 sdc2
evice sdc
Creating block d sdd: sdd1 sdd2
evice sdd
Creating block device sde
 sde: sde1 sde2 sde3 sde4 < sde5 sde6 sde7 sde8 >
Creating block d sdf: sdf1 sdf2
evice sdf
Creating block d sdg: unknown partition table
evice sdg
Creating block d sdh: unknown partition table
evice sdh
Creating block device sdi
 sdi: sdi1 sdi2 sdi3
Creating block device sdj
 sdj: sdj1 sdj2 sdj3 sdj4 < sdj5 sdj6 >
Found device with scsi_ids: 360030057014aa330182f86352027e114
Creating Remain Block Devices
Saving to the local filesystem LABEL=/dump2
e2fsck 1.41.12 (17-May-2010)
/dump2: clean, 35/8929280 files, 929518/35708672 blocks
EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: 
Free memory/Total memory (free %): 61812 / 110200 ( 56.0907 )
Saving vmcore-dmesg.txt
Missing the struct log size export
Saving vmcore-dmesg.txt failed

Excluding unnecessary pages        : [  0 %] 
Excluding unnecessary pages        : [100 %] 
Excluding unnecessary pages        : [ 12 %] 
Excluding unnecessary pages        : [100 %] 
Copying data                       : [  2 %] 
Copying data                       : [ 12 %] 
Copying data                       : [ 19 %] 
Copying data                       : [ 26 %] 
Copying data                       : [ 36 %] 
Copying data                       : [ 47 %] 
Copying data                       : [ 57 %] 
Copying data                       : [ 68 %] 
Copying data                       : [ 75 %] 
Copying data                       : [ 86 %] 
Copying data                       : [ 95 %] 
Copying data                       : [100 %] 
[0JSaving core complete
kvm: exiting hardware virtualization
sd 3:0:1:0: [sdj] Synchronizing SCSI cache
sd 3:0:0:0: [sdi] Synchronizing SCSI cache
sd 0:2:7:0: [sdh] Synchronizing SCSI cache
sd 0:2:6:0: [sdg] Synchronizing SCSI cache
sd 0:2:5:0: [sdf] Synchronizing SCSI cache
sd 0:2:4:0: [sde] Synchronizing SCSI cache
sd 0:2:3:0: [sdd] Synchronizing SCSI cache
sd 0:2:2:0: [sdc] Synchronizing SCSI cache
sd 0:2:1:0: [sdb] Synchronizing SCSI cache
sd 0:2:0:0: [sda] Synchronizing SCSI cache
reboot: Restarting system
reboot: machine restart

From zhen-hual at hp.com  Wed Jan  7 19:11:36 2015
From: zhen-hual at hp.com (Li, Zhen-Hua)
Date: Thu, 8 Jan 2015 03:11:36 +0000
Subject: [PATCH v7 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel
In-Reply-To: <54ADD6A2.5020807@jp.fujitsu.com>
References: <1420603474-4235-1-git-send-email-zhen-hual@hp.com>
 <54ACB1E9.9070904@hp.com>
 <20150107050247.GA16050@dhcp-16-116.nay.redhat.com>
 <54ACC35F.1070007@hp.com>
 <20150107082813.GB16050@dhcp-16-116.nay.redhat.com>
 <54ACF3CC.7090004@hp.com> <54ADD6A2.5020807@jp.fujitsu.com>
Message-ID: <1267BB84A242B24E81E0C0C8955CFFC061AD4A8E@G9W0343.americas.hpqcorp.net>

In your log, it seems something incorrect while copying pages.

Your last DMAR fault is:
DMAR:[fault reason 01] Present bit in root entry is clear

But this time, it is:
DMAR:[fault reason 05] PTE Write access is not set


So I think this line I added to this version , it works.
function intel_iommu_load_translation_tables, line:
    __iommu_flush_cache(iommu, iommu->root_entry, PAGE_SIZE);


I checked the code, found I missed one flush in function copy_page_table.
How do you think we add one flush after this lines:

			ret = copy_page_table(&dma_pte_next, 
					(p->val & VTD_PAGE_MASK),
					shift-9, page_addr | (u << shift),
					iommu, bus, devfn, dve, ppap);

+			__iommu_flush_cache(iommu, phys_to_virt(dma_pte_next),
+				VTD_PAGE_SIZE);

If this does not work, I have no ideas currently, need to dig the code more.

Regards
Zhenhua

-----Original Message-----
From: Takao Indoh [mailto:indou.takao at jp.fujitsu.com] 
Sent: Thursday, January 08, 2015 9:00 AM
To: Li, Zhen-Hua; bhe at redhat.com
Cc: dwmw2 at infradead.org; joro at 8bytes.org; vgoyal at redhat.com; dyoung at redhat.com; iommu at lists.linux-foundation.org; linux-kernel at vger.kernel.org; linux-pci at vger.kernel.org; kexec at lists.infradead.org; alex.williamson at redhat.com; ddutile at redhat.com; ishii.hironobu at jp.fujitsu.com; bhelgaas at google.com; Hatch, Douglas B (HPS Linux PM); Hoemann, Jerry; Vaden, Tom (HP Server OS Architecture); Zhang, Li (Zoe at HPservers-Core-OE-PSC); Mitchell, Lisa (MCLinux in Fort Collins); billsumnerlinux at gmail.com; Wright, Randy (HP Servers Linux)
Subject: Re: [PATCH v7 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel

On 2015/01/07 17:52, Li, ZhenHua wrote:
> Well, that's quite good news.
> Looking forward Takao's testing on his system.

Unfortunately DMAR fault still occurs with this patch...
I attach console log.

Thanks,
Takao Indoh

> 
> Regards
> Zhenhua
> On 01/07/2015 04:28 PM, Baoquan He wrote:
>> On 01/07/15 at 01:25pm, Li, ZhenHua wrote:
>>> It is same as the last one I send to you yesterday.
>>>
>>> The continuous memory that needed for data in this patchset:
>>> RE: PAGE_SIZE, 4096 Bytes;
>>> IRTE: 65536 * 16 ; 1M Bytes;
>>>
>>> It should use same memory as the old versions of this patchset. The 
>>> changes for the last version do not need more memory.
>>
>> Hi Zhenhua,
>>
>> It was my mistake because I didn't strip the debug info of modules, 
>> then initramfs is bloated very big. Just now I tested the latest 
>> version, it works well and dump is successful. No dmar fault and 
>> intr-remap fault seen any more, good job!
>>
>> Thanks
>> Baoquan
>>
>>
>>>
>>> Regards
>>> Zhenhua
>>>
>>> On 01/07/2015 01:02 PM, Baoquan He wrote:
>>>> On 01/07/15 at 12:11pm, Li, ZhenHua wrote:
>>>>> Many thanks to Takao Indoh and Baoquan He, for your testing on 
>>>>> more different systems.
>>>>>
>>>>> The calling of flush functions are added to this version.
>>>>>
>>>>> The usage of __iommu_flush_cache function :
>>>>> 1. Fixes a dump on Takao's system.
>>>>> 2. Reduces the count of faults on Baoquan's system.
>>>>
>>>> I am testing the version you sent to me yesterday afternoon. Is 
>>>> that different with this patchset? I found your patchset man 
>>>> reserve a big contiguous memory region under 896M, this will cause 
>>>> the crashkernel reservation failed when I set crashkernel=320M. The 
>>>> reason I increase the crashkerenl reservation to 320M is 256M is 
>>>> not enough and cause OOM when that patchset is tested.
>>>>
>>>> I am checking what happened.
>>>>
>>>>
>>>> Thanks
>>>> Baoquan
>>>>
>>>>>
>>>>> Regards
>>>>> Zhenhua
>>>>>
>>>>> On 01/07/2015 12:04 PM, Li, Zhen-Hua wrote:
>>>>>> This patchset is an update of Bill Sumner's patchset, implements a fix for:
>>>>>> If a kernel boots with intel_iommu=on on a system that supports 
>>>>>> intel vt-d, when a panic happens, the kdump kernel will boot with these faults:
>>>>>>
>>>>>>      dmar: DRHD: handling fault status reg 102
>>>>>>      dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr fff80000
>>>>>>      DMAR:[fault reason 01] Present bit in root entry is clear
>>>>>>
>>>>>>      dmar: DRHD: handling fault status reg 2
>>>>>>      dmar: INTR-REMAP: Request device [[61:00.0] fault index 42
>>>>>>      INTR-REMAP:[fault reason 34] Present field in the IRTE entry 
>>>>>> is clear
>>>>>>
>>>>>> On some system, the interrupt remapping fault will also happen 
>>>>>> even if the intel_iommu is not set to on, because the interrupt 
>>>>>> remapping will be enabled when x2apic is needed by the system.
>>>>>>
>>>>>> The cause of the DMA fault is described in Bill's original 
>>>>>> version, and the INTR-Remap fault is caused by a similar reason. 
>>>>>> In short, the initialization of vt-d drivers causes the in-flight 
>>>>>> DMA and interrupt requests get wrong response.
>>>>>>
>>>>>> To fix this problem, we modifies the behaviors of the intel vt-d 
>>>>>> in the crashdump kernel:
>>>>>>
>>>>>> For DMA Remapping:
>>>>>> 1. To accept the vt-d hardware in an active state, 2. Do not 
>>>>>> disable and re-enable the translation, keep it enabled.
>>>>>> 3. Use the old root entry table, do not rewrite the RTA register.
>>>>>> 4. Malloc and use new context entry table and page table, copy data from the
>>>>>>     old ones that used by the old kernel.
>>>>>> 5. to use different portions of the iova address ranges for the device drivers
>>>>>>     in the crashdump kernel than the iova ranges that were in-use at the time
>>>>>>     of the panic.
>>>>>> 6. After device driver is loaded, when it issues the first dma_map command,
>>>>>>     free the dmar_domain structure for this device, and generate a new one, so
>>>>>>     that the device can be assigned a new and empty page table.
>>>>>> 7. When a new context entry table is generated, we also save its address to
>>>>>>     the old root entry table.
>>>>>>
>>>>>> For Interrupt Remapping:
>>>>>> 1. To accept the vt-d hardware in an active state, 2. Do not 
>>>>>> disable and re-enable the interrupt remapping, keep it enabled.
>>>>>> 3. Use the old interrupt remapping table, do not rewrite the IRTA register.
>>>>>> 4. When ioapic entry is setup, the interrupt remapping table is changed, and
>>>>>>     the updated data will be stored to the old interrupt remapping table.
>>>>>>
>>>>>> Advantages of this approach:
>>>>>> 1. All manipulation of the IO-device is done by the Linux device-driver
>>>>>>     for that device.
>>>>>> 2. This approach behaves in a manner very similar to operation without an
>>>>>>     active iommu.
>>>>>> 3. Any activity between the IO-device and its RMRR areas is handled by the
>>>>>>     device-driver in the same manner as during a non-kdump boot.
>>>>>> 4. If an IO-device has no driver in the kdump kernel, it is simply left alone.
>>>>>>     This supports the practice of creating a special kdump kernel without
>>>>>>     drivers for any devices that are not required for taking a crashdump.
>>>>>> 5. Minimal code-changes among the existing mainline intel vt-d code.
>>>>>>
>>>>>> Summary of changes in this patch set:
>>>>>> 1. Added some useful function for root entry table in code 
>>>>>> intel-iommu.c 2. Added new members to struct root_entry and 
>>>>>> struct irte; 3. Functions to load old root entry table to iommu->root_entry from the memory
>>>>>>     of old kernel.
>>>>>> 4. Functions to malloc new context entry table and page table and copy the data
>>>>>>     from the old ones to the malloced new ones.
>>>>>> 5. Functions to enable support for DMA remapping in kdump kernel.
>>>>>> 6. Functions to load old irte data from the old kernel to the kdump kernel.
>>>>>> 7. Some code changes that support other behaviours that have been listed.
>>>>>> 8. In the new functions, use physical address as "unsigned long" type, not
>>>>>>     pointers.
>>>>>>
>>>>>> Original version by Bill Sumner:
>>>>>>      https://lkml.org/lkml/2014/1/10/518
>>>>>>      https://lkml.org/lkml/2014/4/15/716
>>>>>>      https://lkml.org/lkml/2014/4/24/836
>>>>>>
>>>>>> Zhenhua's updates:
>>>>>>      https://lkml.org/lkml/2014/10/21/134
>>>>>>      https://lkml.org/lkml/2014/12/15/121
>>>>>>      https://lkml.org/lkml/2014/12/22/53
>>>>>>
>>>>>> Changelog[v7]:
>>>>>>      1. Use __iommu_flush_cache to flush the data to hardware.
>>>>>>
>>>>>> Changelog[v6]:
>>>>>>      1. Use "unsigned long" as type of physical address.
>>>>>>      2. Use new function unmap_device_dma to unmap the old dma.
>>>>>>      3. Some small incorrect bits order for aw shift.
>>>>>>
>>>>>> Changelog[v5]:
>>>>>>      1. Do not disable and re-enable traslation and interrupt remapping.
>>>>>>      2. Use old root entry table.
>>>>>>      3. Use old interrupt remapping table.
>>>>>>      4. New functions to copy data from old kernel, and save to old kernel mem.
>>>>>>      5. New functions to save updated root entry table and irte table.
>>>>>>      6. Use intel_unmap to unmap the old dma;
>>>>>>      7. Allocate new pages while driver is being loaded.
>>>>>>
>>>>>> Changelog[v4]:
>>>>>>      1. Cut off the patches that move some defines and functions to new files.
>>>>>>      2. Reduce the numbers of patches to five, make it more easier to read.
>>>>>>      3. Changed the name of functions, make them consistent with current context
>>>>>>         get/set functions.
>>>>>>      4. Add change to function __iommu_attach_domain.
>>>>>>
>>>>>> Changelog[v3]:
>>>>>>      1. Commented-out "#define DEBUG 1" to eliminate debug messages.
>>>>>>      2. Updated the comments about changes in each version.
>>>>>>      3. Fixed: one-line added to Copy-Translations patch to initialize the iovad
>>>>>>            struct as recommended by Baoquan He [bhe at redhat.com]
>>>>>>            init_iova_domain(&domain->iovad, DMA_32BIT_PFN);
>>>>>>
>>>>>> Changelog[v2]:
>>>>>>      The following series implements a fix for:
>>>>>>      A kdump problem about DMA that has been discussed for a long time. That is,
>>>>>>      when a kernel panics and boots into the kdump kernel, DMA started by the
>>>>>>      panicked kernel is not stopped before the kdump kernel is booted and the
>>>>>>      kdump kernel disables the IOMMU while this DMA continues.  This causes the
>>>>>>      IOMMU to stop translating the DMA addresses as IOVAs and begin to treat
>>>>>>      them as physical memory addresses -- which causes the DMA to either:
>>>>>>          (1) generate DMAR errors or
>>>>>>          (2) generate PCI SERR errors or
>>>>>>          (3) transfer data to or from incorrect areas of memory. Often this
>>>>>>              causes the dump to fail.
>>>>>>
>>>>>> Changelog[v1]:
>>>>>>      The original version.
>>>>>>
>>>>>> Changed in this version:
>>>>>> 1. Do not disable and re-enable traslation and interrupt remapping.
>>>>>> 2. Use old root entry table.
>>>>>> 3. Use old interrupt remapping table.
>>>>>> 4. Use "unsigned long" as physical address.
>>>>>> 5. Use intel_unmap to unmap the old dma;
>>>>>>
>>>>>> Baoquan He <bhe at redhat.com> helps testing this patchset.
>>>>>>
>>>>>>    iommu/vt-d: Update iommu_attach_domain() and its callers
>>>>>>    iommu/vt-d: Items required for kdump
>>>>>>    iommu/vt-d: Add domain-id functions
>>>>>>    iommu/vt-d: functions to copy data from old mem
>>>>>>    iommu/vt-d: Add functions to load and save old re
>>>>>>    iommu/vt-d: datatypes and functions used for kdump
>>>>>>    iommu/vt-d: enable kdump support in iommu module
>>>>>>    iommu/vt-d: assign new page table for dma_map
>>>>>>    iommu/vt-d: Copy functions for irte
>>>>>>    iommu/vt-d: Use old irte in kdump kernel
>>>>>>
>>>>>> Signed-off-by: Bill Sumner <billsumnerlinux at gmail.com>
>>>>>> Signed-off-by: Li, Zhen-Hua <zhen-hual at hp.com>
>>>>>> Signed-off-by: Takao Indoh <indou.takao at jp.fujitsu.com>
>>>>>> Tested-by: Baoquan He <bhe at redhat.com>
>>>>>> ---
>>>>>>   drivers/iommu/intel-iommu.c         | 1050 +++++++++++++++++++++++++++++++++--
>>>>>>   drivers/iommu/intel_irq_remapping.c |  104 +++-
>>>>>>   include/linux/intel-iommu.h         |   18 +
>>>>>>   3 files changed, 1130 insertions(+), 42 deletions(-)
>>>>>>
>>>>>
>>>
> 
> 
> 



From liu.song11 at zte.com.cn  Wed Jan  7 20:21:44 2015
From: liu.song11 at zte.com.cn (liu.song11 at zte.com.cn)
Date: Thu, 8 Jan 2015 12:21:44 +0800
Subject: some questions about elf_ppc64_load
In-Reply-To: <54ABB227.7010203@linux.vnet.ibm.com>
References: <OF9C6CA8A8.9D7923CD-ON48257DC5.0023F61F-48257DC5.0024E800@zte.com.cn>
 <54AB9EAF.2020004@linux.vnet.ibm.com>
 <OF89302667.7C36AC27-ON48257DC5.002FF943-48257DC5.00333569@zte.com.cn>
 <54ABB227.7010203@linux.vnet.ibm.com>
Message-ID: <OF45591076.2B9B091A-ON48257DC7.00179FF6-48257DC7.0017DC56@zte.com.cn>



Laurent Dufour <ldufour at linux.vnet.ibm.com> wrote on 2015-01-06 18:00:07:

> From: Laurent Dufour <ldufour at linux.vnet.ibm.com>
> To: liu.song11 at zte.com.cn,
> Cc: kexec at lists.infradead.org
> Date: 2015-01-06 18:00
> Subject: Re: some questions about elf_ppc64_load
>
> On 06/01/2015 10:20, liu.song11 at zte.com.cn wrote:
> >
> >
> > Laurent Dufour <ldufour at linux.vnet.ibm.com> wrote on 2015-01-06 16:37:03:
> >
> >> From: Laurent Dufour <ldufour at linux.vnet.ibm.com>
> >> To: liu.song11 at zte.com.cn,
> >> Cc: kexec at lists.infradead.org
> >> Date: 2015-01-06 16:37
> >> Subject: Re: some questions about elf_ppc64_load
> >>
> >> On 06/01/2015 07:44, liu.song11 at zte.com.cn wrote:
> >>>
> >>> hi all?
> >>>
> >>> I use kexec-tools-2.0.8 for ppc64, in fuction elf_ppc64_load prompt "Can't use ramdisk with device tree blob input".
> >>>
> >>> I don't know why here ramdisk can't together with dtb,and need some help.
> >>
> >> The device tree is also specifying initrd (see kexec/fs2dt.c putnode),
> >> so there is 2 initrd specify in that case. The code is a bit rough here,
> >> but I guess this is because there is not yet a valid use case.
> > hi Cheers,
> > In "elf_ppc64_load", if we haven't assign a special dtb by --dtb=<filename>, then here will use "create_flatten_tree" to
> > create one. However, My board need assign a dtb, so I use --dtb=my.dtb to assign it. The dtb file is valid which I use it
> > to start kernel under uboot. For now, I just confuse is why the "create_flatten_tree" create dtb could work, but I assigned
> > dtb couldn't, my.dtb is 40KB and smaller than create dtb's 76KB. The two ways both use add_buffer with param "buf_end = -1",
> > so will add segment for the top of the valid mem range, and my.dtb is smaller which less chance of overlap some data in memory.
>
> The service create_flatten_tree create a DTB from the one inherited at
> the boot time of the currently running kernel (/proc/device-tree). I
> guess this device tree is valid since the current kernel is running ;)
>
> You may try to see what the device tree passed to your kexed kernel is,
> may be there is something wrong there. Also looking at what going wrong
> when the kexed kernel is panicing may be helpful.
>
> Cheers,
> Laurent.
>
Thanks Laurent, the dtb which I used is invalid for kexec a kernel, this dtb just used in u-boot and it would be processed by u-boot,
then it could fit to kernel's require.

Thanks
Regards

> >
> >>
> >>> I also have some questions, when I just ./kexe -l vmlinux then ./kexec -e, will jump to second kernel sucessfully,
> >>>
> >>> but without useful dtb, then device can't init correct. But, when I input ./kexec -l vmlinux --dtb=my.dtb , then
> >>>
> >>> ./kexec -e, the kernel couldn't work and crash in head_64.S, this problem also make me confused.
> >>
> >> It's hard to state with no more details. I guess the device tree blob
> >> you're using is not valid, or contain bad data.
> >>
> >> Cheers,
> >> Laurent.
> >>
> >>> Thanks
> >>> Regards
> >>> --------------------------------------------------------
> >>> ZTE Information Security Notice: The information contained in this mail (and any attachment transmitted herewith) is privileged and confidential and is intended for the exclusive use of the addressee(s).  If you are not an intended recipient, any disclosure, reproduction, distribution or other dissemination or use of the information contained is strictly prohibited.  If you have received this mail in error, please delete it and notify us immediately.
> >>> _______________________________________________
> >>> kexec mailing list
> >>> kexec at lists.infradead.org
> >>> http://lists.infradead.org/mailman/listinfo/kexec
> >>>
> >>
> > --------------------------------------------------------
> > ZTE Information Security Notice: The information contained in this mail (and any attachment transmitted herewith) is privileged and confidential and is intended for the exclusive use of the addressee(s).  If you are not an intended recipient, any disclosure, reproduction, distribution or other dissemination or use of the information contained is strictly prohibited.  If you have received this mail in error, please delete it and notify us immediately.
> >
>
--------------------------------------------------------
ZTE Information Security Notice: The information contained in this mail (and any attachment transmitted herewith) is privileged and confidential and is intended for the exclusive use of the addressee(s).  If you are not an intended recipient, any disclosure, reproduction, distribution or other dissemination or use of the information contained is strictly prohibited.  If you have received this mail in error, please delete it and notify us immediately.

From anurup.m at huawei.com  Wed Jan  7 22:35:48 2015
From: anurup.m at huawei.com (Anurup M)
Date: Thu, 8 Jan 2015 12:05:48 +0530
Subject: Kdump on ARM64
Message-ID: <54AE2544.2010801@huawei.com>

Hi Akashi, Geoff,

	I tried to check kdump on arm64 on Foundation model. The/proc/vmcore is generated.
But I couldnot check if it is proper. Is there a makedumpfile binary for arm64? and also crash tool.
I used the master branch of https://git.linaro.org/people/geoff.levand/linux-kexec.git to verify.
 	Please provide some information about the current status or support for kdump on arm64?
and the ways to verify or start development on Foundation model.

Thanks,
Anurup M


From takahiro.akashi at linaro.org  Thu Jan  8 00:10:34 2015
From: takahiro.akashi at linaro.org (AKASHI Takahiro)
Date: Thu, 08 Jan 2015 17:10:34 +0900
Subject: Kdump on ARM64
In-Reply-To: <54AE2544.2010801@huawei.com>
References: <54AE2544.2010801@huawei.com>
Message-ID: <54AE3B7A.4070301@linaro.org>

Hi Anurup,

On 01/08/2015 03:35 PM, Anurup M wrote:
> Hi Akashi, Geoff,
>
> 	I tried to check kdump on arm64 on Foundation model. The/proc/vmcore is generated.
> But I couldnot check if it is proper. Is there a makedumpfile binary for arm64? and also crash tool.
> I used the master branch of https://git.linaro.org/people/geoff.levand/linux-kexec.git to verify.
>   	Please provide some information about the current status or support for kdump on arm64?
> and the ways to verify or start development on Foundation model.

The story is that I used to work on kdump, but handed it off
to Geoff at some time. Since then, he merged it into his repo
but I don't think that he has tested (verified) it yet.
So please take it as a study work (or prototype), although I'm
pretty sure that it works well on model except some issues.

Regarding /proc/vmcore, if you can retrieve it to your host machine,
you can simply verify the contents as follows:
   $ aarch64-linux-gnu-gdb vmlinux core(=vmcore)
   (gdb) info threads
   (gdb) bt
    ...

I also have a trivial patch for crash tool to cross-build it, but
arm64 support (by RedHat folk) is already there.

Thanks,
-Takahiro AKASHI

> Thanks,
> Anurup M
>


From bhe at redhat.com  Thu Jan  8 18:09:10 2015
From: bhe at redhat.com (Baoquan He)
Date: Fri, 9 Jan 2015 10:09:10 +0800
Subject: [resend Patch v3 1/2] kaslr: check if kernel location is changed
In-Reply-To: <1412060896-1902-1-git-send-email-bhe@redhat.com>
References: <1412060896-1902-1-git-send-email-bhe@redhat.com>
Message-ID: <20150109020910.GA22338@dhcp-16-116.nay.redhat.com>

Hi hpa,

Ping.

Do you have further plan or idea on this issue? Or could you
please merge this patch to fix reported bug for now?

Thanks
Baoquan

On 09/30/14 at 03:08pm, Baoquan He wrote:
> Function handle_relocations() is used to do the relocations handling
> for i686 and kaslr of x86_64. For 32 bit the relocation handling is
> mandotary to perform. For x86_64 only when kaslr is enabled and a
> random kernel location is chosen successfully the relocation handling
> shound be done. However previous implementation only compared the
> kernel loading address and LOAD_PHYSICAL_ADDR where kernel were
> compiled to run at. This would casue system to be exceptional in
> few conditions like when delta between load address and compiled
> address is bigger than what 32bit signed relocations can handle.
> Also there will be limitations that delta can't be too big otherwise
> kernel text virtual addresses will overflow in module address space.
> 
> So in this patch check if kernel location is changed after
> choose_kernel_location() when x86_64. If and only if in x86_64
> and kernel location is changed, we say a kaslr random kernel
> location is chosen, then the relocation handling is needed.
> 
> Signed-off-by: Baoquan He <bhe at redhat.com>
> Acked-by: Vivek Goyal <vgoyal at redhat.com>
> Acked-by: Kees Cook <keescook at chromium.org>
> Tested-by: Thomas D. <whissi at whissi.de>
> Cc: stable at vger.kernel.org
> ---
>  arch/x86/boot/compressed/misc.c | 26 ++++++++++++++++++++++----
>  1 file changed, 22 insertions(+), 4 deletions(-)
> 
> diff --git a/arch/x86/boot/compressed/misc.c b/arch/x86/boot/compressed/misc.c
> index 57ab74d..3bb2a17 100644
> --- a/arch/x86/boot/compressed/misc.c
> +++ b/arch/x86/boot/compressed/misc.c
> @@ -230,8 +230,9 @@ static void error(char *x)
>  		asm("hlt");
>  }
>  
> -#if CONFIG_X86_NEED_RELOCS
> -static void handle_relocations(void *output, unsigned long output_len)
> +#ifdef CONFIG_X86_NEED_RELOCS
> +static void handle_relocations(void *output_orig, void *output,
> +			       unsigned long output_len)
>  {
>  	int *reloc;
>  	unsigned long delta, map, ptr;
> @@ -239,6 +240,20 @@ static void handle_relocations(void *output, unsigned long output_len)
>  	unsigned long max_addr = min_addr + output_len;
>  
>  	/*
> +	* 32bit always requires relocations to be performed. For x86_64,
> +	* relocations need to be performed only if kaslr has chosen a
> +	* different load address then kernel was originally loaded at.
> +	*
> +	* If we are here, either kaslr is not configured in or kaslr is disabled
> +	* or kaslr has chosen not to change the load location of kernel. Don't
> +	* perform any relocations.
> +	*/
> +#if CONFIG_X86_64
> +	if (output_orig == output)
> +		return;
> +#endif
> +
> +	/*
>  	 * Calculate the delta between where vmlinux was linked to load
>  	 * and where it was actually loaded.
>  	 */
> @@ -299,7 +314,8 @@ static void handle_relocations(void *output, unsigned long output_len)
>  #endif
>  }
>  #else
> -static inline void handle_relocations(void *output, unsigned long output_len)
> +static inline void handle_relocations(void *output_orig, void *output,
> +				      unsigned long output_len)
>  { }
>  #endif
>  
> @@ -360,6 +376,8 @@ asmlinkage __visible void *decompress_kernel(void *rmode, memptr heap,
>  				  unsigned char *output,
>  				  unsigned long output_len)
>  {
> +	unsigned char *output_orig = output;
> +
>  	real_mode = rmode;
>  
>  	sanitize_boot_params(real_mode);
> @@ -402,7 +420,7 @@ asmlinkage __visible void *decompress_kernel(void *rmode, memptr heap,
>  	debug_putstr("\nDecompressing Linux... ");
>  	decompress(input_data, input_len, NULL, NULL, output, NULL, error);
>  	parse_elf(output);
> -	handle_relocations(output, output_len);
> +	handle_relocations(output_orig, output, output_len);
>  	debug_putstr("done.\nBooting the kernel.\n");
>  	return output;
>  }
> -- 
> 1.8.5.3
> 


From tonyj at suse.de  Thu Jan  8 22:27:03 2015
From: tonyj at suse.de (Tony Jones)
Date: Thu, 08 Jan 2015 22:27:03 -0800
Subject: [PATCH] kexec/x86: fix crashkernel reserved mem detection
In-Reply-To: <1419514701-9397-1-git-send-email-wolchkov.yuri@gmail.com>
References: <1419514701-9397-1-git-send-email-wolchkov.yuri@gmail.com>
Message-ID: <54AF74B7.4000807@suse.de>

On 12/25/2014 05:38 AM, Yuri Volchkov wrote:
> Function crashkernel_mem_callback had wrong parameters type
> So crash_reserved_mem got completely insane values
> As a result kexec was unable to allocate segments for crashkernel
> For instance, I got this message:
> 	"Could not find a free area of memory of 0x9f000 bytes..."
> 
> Signed-off-by: Yuri Volchkov <wolchkov.yuri at gmail.com>

See the patch attached to https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=771671
which also fixes kexec/arch/sh/crashdump-sh.c and kexec/arch/sh/kexec-sh.c

Definitely a problem.  Without the x86 fix we seem same failure on i586

tony

> ---
>  kexec/arch/i386/crashdump-x86.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/kexec/arch/i386/crashdump-x86.c b/kexec/arch/i386/crashdump-x86.c
> index 77bdad5..82bf239 100644
> --- a/kexec/arch/i386/crashdump-x86.c
> +++ b/kexec/arch/i386/crashdump-x86.c
> @@ -1041,8 +1041,8 @@ int get_max_crash_kernel_limit(uint64_t *start, uint64_t *end)
>  
>  static int crashkernel_mem_callback(void *UNUSED(data), int nr,
>                                            char *UNUSED(str),
> -                                          unsigned long base,
> -                                          unsigned long length)
> +                                          unsigned long long base,
> +                                          unsigned long long length)
>  {
>  	if (nr >= CRASH_RESERVED_MEM_NR)
>  		return 1;
> 



From bhe at redhat.com  Fri Jan  9 00:59:27 2015
From: bhe at redhat.com (Baoquan He)
Date: Fri, 9 Jan 2015 16:59:27 +0800
Subject: [PATCH] kexec/x86: fix crashkernel reserved mem detection
In-Reply-To: <54AF74B7.4000807@suse.de>
References: <1419514701-9397-1-git-send-email-wolchkov.yuri@gmail.com>
 <54AF74B7.4000807@suse.de>
Message-ID: <20150109085927.GA8288@dhcp-16-105.nay.redhat.com>

On 01/08/15 at 10:27pm, Tony Jones wrote:
> On 12/25/2014 05:38 AM, Yuri Volchkov wrote:
> > Function crashkernel_mem_callback had wrong parameters type
> > So crash_reserved_mem got completely insane values
> > As a result kexec was unable to allocate segments for crashkernel
> > For instance, I got this message:
> > 	"Could not find a free area of memory of 0x9f000 bytes..."
> > 
> > Signed-off-by: Yuri Volchkov <wolchkov.yuri at gmail.com>
> 
> See the patch attached to https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=771671
> which also fixes kexec/arch/sh/crashdump-sh.c and kexec/arch/sh/kexec-sh.c
> 
> Definitely a problem.  Without the x86 fix we seem same failure on i586

Hi Tony,

commit 4362bfa make that change, but skip the crashkernel_mem_callback.
It need be fixed.

I am wondering how the insane values happened on i386, and it didn't
happen on x86_64.

Thanks
Baoquan

> 
> tony
> 
> > ---
> >  kexec/arch/i386/crashdump-x86.c | 4 ++--
> >  1 file changed, 2 insertions(+), 2 deletions(-)
> > 
> > diff --git a/kexec/arch/i386/crashdump-x86.c b/kexec/arch/i386/crashdump-x86.c
> > index 77bdad5..82bf239 100644
> > --- a/kexec/arch/i386/crashdump-x86.c
> > +++ b/kexec/arch/i386/crashdump-x86.c
> > @@ -1041,8 +1041,8 @@ int get_max_crash_kernel_limit(uint64_t *start, uint64_t *end)
> >  
> >  static int crashkernel_mem_callback(void *UNUSED(data), int nr,
> >                                            char *UNUSED(str),
> > -                                          unsigned long base,
> > -                                          unsigned long length)
> > +                                          unsigned long long base,
> > +                                          unsigned long long length)
> >  {
> >  	if (nr >= CRASH_RESERVED_MEM_NR)
> >  		return 1;
> > 
> 
> 
> _______________________________________________
> kexec mailing list
> kexec at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/kexec


From barr24 at gmail.com  Fri Jan  9 11:35:43 2015
From: barr24 at gmail.com (BARR ALHARIRI)
Date: Fri, 09 Jan 2015 21:35:43 +0200
Subject: =?UTF-8?Q?=D9=84=D8=B3=D9=84=D8=A7=D9=85=20=D8=B9=D9=84=D9=8A?=
 =?UTF-8?Q?=D9=83=20=D9=88=D8=B1=D8=AD=D9=85=D8=A9=20=D8=A7=D9=84=D9=84?=
 =?UTF-8?Q?=D9=87?=
Message-ID: <f0ed65529e701a1e020c13d2075373fb@ulib.sk>



-- 
????? ?????
????? ???  ??? ?????? ????? ??????? ?? ????? ?????? ???? ????? ??? ????? 
???? ?????? ??? ???? ?? ????? ???????? ???? ???????? ??? ????? ????????? 
???????? ???
???? ?????? ??? ??? ???????? : barralhariri24 at gmail.com
????? ???????



From vgoyal at redhat.com  Mon Jan 12 14:16:34 2015
From: vgoyal at redhat.com (Vivek Goyal)
Date: Mon, 12 Jan 2015 17:16:34 -0500
Subject: Edited kexec_load(2) [kexec_file_load()] man page for review
In-Reply-To: <54ADA284.30502@gmail.com>
References: <545FBDDD.9060801@gmail.com> <20141111213037.GA31445@redhat.com>
 <54ADA284.30502@gmail.com>
Message-ID: <20150112221634.GD16162@redhat.com>

On Wed, Jan 07, 2015 at 10:17:56PM +0100, Michael Kerrisk (man-pages) wrote:

[..]
> >> .BR KEXEC_ON_CRASH " (since Linux 2.6.13)"
> >> Execute the new kernel automatically on a system crash.
> >> .\" FIXME Explain in more detail how KEXEC_ON_CRASH is actually used
> 
> I wasn't expecting that you would respond to the FIXMEs that were 
> not labeled "kexec_file_load", but I was hoping you might ;-). Thanks!
> I have a few additional questions to your nice notes.
> 
> > Upon boot first kernel reserves a chunk of contiguous memory (if
> > crashkernel=<> command line paramter is passed). This memory is
> > is used to load the crash kernel (Kernel which will be booted into
> > if first kernel crashes).
> 

Hi Michael,

> Can I just confirm: is it in all cases only possible to use kexec_load() 
> and kexec_file_load() if the kernel was booted with the 'crashkernel'
> parameter set?

As of now, only kexec_load() and kexec_file_load() system calls can
make use of memory reserved by crashkernel=<> kernel parameter. And
this is used only if we are trying to load a crash kernel (KEXEC_ON_CRASH
flag specified).

> 
> > Location of this reserved memory is exported to user space through
> > /proc/iomem file. 
> 
> Is that export via an entry labeled "Crash kernel" in the 
> /proc/iomem file?

Yes.

> 
> > User space can parse it and prepare list of segments
> > specifying this reserved memory as destination.
> 
> I'm not quite clear on "specifying this reserved memory as destination".
> Is that done by specifying the address in the kexec_segment.mem fields?

You are absolutely right. User space can specify in kexec_segment.mem
field the memory location where it expecting a particular segment to
be loaded by kernel.

> 
> > Once kernel sees the flag KEXEC_ON_CRASH, it makes sure that all the
> > segments are destined for reserved memory otherwise kernel load operation
> > fails.
> 
> Could you point me to where this checking is done? Also, what is the
> error (errno) that occurs when the load operation fails? (I think the
> answers to these questions are "at the start of kimage_alloc_init()"
> and "EADDRNOTAVAIL", but I'd like to confirm.)

This checking happens in sanity_check_segment_list() which is called
by kimage_alloc_init().

And yes, error code returned is -EADDRNOTAVAIL.

> 
> > [..]
> >> struct kexec_segment {
> >>     void   *buf;        /* Buffer in user space */
> >>     size_t  bufsz;      /* Buffer length in user space */
> >>     void   *mem;        /* Physical address of kernel */
> >>     size_t  memsz;      /* Physical address length */
> >> };
> >> .fi
> >> .in
> >> .PP
> >> .\" FIXME Explain the details of how the kernel image defined by segments
> >> .\" is copied from the calling process into previously reserved memory.
> > 
> > Kernel image defined by segments is copied into kernel either in regular
> > memory 
> 
> Could you clarify what you mean by "regular memory"?

I meant memory which is not reserved memory.

> 
> > or in reserved memory (if KEXEC_ON_CRASH is set). Kernel first
> > copies list of segments in kernel memory and then goes does various
> > sanity checks on the segments. If everything looks line, kernel copies
> > segment data to kernel memory.
> > 
> > In case of normal kexec, segment data is loaded in any available memory
> > and segment data is moved to final destination at the kexec reboot time.
> 
> By "moved to final destination", do you mean "moved from user space to the
> final kernel-space destination"?

No. Segment data moves from user space to kernel space once kexec_load()
call finishes successfully. But when user does reboot (kexec -e), at that
time kernel moves that segment data to its final location. Kernel could
not place the segment at its final location during kexec_load() time as
that memory is already in use by running kernel. But once we are about
to reboot to new kernel, we can overwrite the old kernel's memory.

> 
> > In case of kexec on panic (KEXEC_ON_CRASH flag set), segment data is
> > directly loaded to reserved memory and after crash kexec simply jumps
> 
> By "directly", I assume you mean "at the time of the kexec_laod() call",
> right?

Yes.

Thanks
Vivek



From zhen-hual at hp.com  Mon Jan 12 17:28:53 2015
From: zhen-hual at hp.com (Li, ZhenHua)
Date: Tue, 13 Jan 2015 09:28:53 +0800
Subject: [PATCH v8 01/10] iommu/vt-d: Update iommu_attach_domain() and
 its callers
In-Reply-To: <20150112151855.GB6343@8bytes.org>
References: <1421046388-27925-1-git-send-email-zhen-hual@hp.com>
 <1421046388-27925-2-git-send-email-zhen-hual@hp.com>
 <20150112151855.GB6343@8bytes.org>
Message-ID: <54B474D5.8090500@hp.com>

On 01/12/2015 11:18 PM, Joerg Roedel wrote:
> On Mon, Jan 12, 2015 at 03:06:19PM +0800, Li, Zhen-Hua wrote:
>> Allow specification of the domain-id for the new domain.
>> This patch only adds the 'did' parameter to iommu_attach_domain()
>> and modifies all of its callers to specify the default value of -1
>> which says "no did specified, allocate a new one".
>
> I think its better to keep the old iommu_attach_domain() interface in
> place and introduce a new function (like iommu_attach_domain_with_id()
> or something) which has the additional parameter. Then you can rewrite
> iommu_attach_domain():
>
> 	iommu_attach_domai(...)
> 	{
> 		return iommu_attach_domain_with_id(..., -1);
> 	}
>
> This way you don't have to update all the callers of
> iommu_attach_domain() and the interface is more readable.
>
>
> 	Joerg
>

That's a good way. I will do this in next version.

Thanks
Zhenhua




From zhen-hual at hp.com  Sun Jan 11 23:06:18 2015
From: zhen-hual at hp.com (Li, Zhen-Hua)
Date: Mon, 12 Jan 2015 15:06:18 +0800
Subject: [PATCH v8 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel
Message-ID: <1421046388-27925-1-git-send-email-zhen-hual@hp.com>

This patchset is an update of Bill Sumner's patchset, implements a fix for:
If a kernel boots with intel_iommu=on on a system that supports intel vt-d, 
when a panic happens, the kdump kernel will boot with these faults:

    dmar: DRHD: handling fault status reg 102
    dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr fff80000
    DMAR:[fault reason 01] Present bit in root entry is clear

    dmar: DRHD: handling fault status reg 2
    dmar: INTR-REMAP: Request device [[61:00.0] fault index 42
    INTR-REMAP:[fault reason 34] Present field in the IRTE entry is clear

On some system, the interrupt remapping fault will also happen even if the 
intel_iommu is not set to on, because the interrupt remapping will be enabled 
when x2apic is needed by the system.

The cause of the DMA fault is described in Bill's original version, and the 
INTR-Remap fault is caused by a similar reason. In short, the initialization 
of vt-d drivers causes the in-flight DMA and interrupt requests get wrong 
response.

To fix this problem, we modifies the behaviors of the intel vt-d in the 
crashdump kernel:

For DMA Remapping:
1. To accept the vt-d hardware in an active state,
2. Do not disable and re-enable the translation, keep it enabled.
3. Use the old root entry table, do not rewrite the RTA register.
4. Malloc and use new context entry table and page table, copy data from the 
   old ones that used by the old kernel.
5. to use different portions of the iova address ranges for the device drivers
   in the crashdump kernel than the iova ranges that were in-use at the time
   of the panic.  
6. After device driver is loaded, when it issues the first dma_map command, 
   free the dmar_domain structure for this device, and generate a new one, so 
   that the device can be assigned a new and empty page table. 
7. When a new context entry table is generated, we also save its address to 
   the old root entry table.

For Interrupt Remapping:
1. To accept the vt-d hardware in an active state,
2. Do not disable and re-enable the interrupt remapping, keep it enabled.
3. Use the old interrupt remapping table, do not rewrite the IRTA register.
4. When ioapic entry is setup, the interrupt remapping table is changed, and 
   the updated data will be stored to the old interrupt remapping table.

Advantages of this approach:
1. All manipulation of the IO-device is done by the Linux device-driver
   for that device.
2. This approach behaves in a manner very similar to operation without an
   active iommu.
3. Any activity between the IO-device and its RMRR areas is handled by the
   device-driver in the same manner as during a non-kdump boot.
4. If an IO-device has no driver in the kdump kernel, it is simply left alone.
   This supports the practice of creating a special kdump kernel without
   drivers for any devices that are not required for taking a crashdump. 
5. Minimal code-changes among the existing mainline intel vt-d code.

Summary of changes in this patch set:
1. Added some useful function for root entry table in code intel-iommu.c
2. Added new members to struct root_entry and struct irte;
3. Functions to load old root entry table to iommu->root_entry from the memory 
   of old kernel.
4. Functions to malloc new context entry table and page table and copy the data
   from the old ones to the malloced new ones.
5. Functions to enable support for DMA remapping in kdump kernel.
6. Functions to load old irte data from the old kernel to the kdump kernel.
7. Some code changes that support other behaviours that have been listed.
8. In the new functions, use physical address as "unsigned long" type, not 
   pointers.

Original version by Bill Sumner:
    https://lkml.org/lkml/2014/1/10/518
    https://lkml.org/lkml/2014/4/15/716
    https://lkml.org/lkml/2014/4/24/836

Zhenhua's updates:
    https://lkml.org/lkml/2014/10/21/134
    https://lkml.org/lkml/2014/12/15/121
    https://lkml.org/lkml/2014/12/22/53
    https://lkml.org/lkml/2015/1/6/1166

Changelog[v8]:
    1. Add a missing __iommu_flush_cache in function copy_page_table.

Changelog[v7]:
    1. Use __iommu_flush_cache to flush the data to hardware.

Changelog[v6]:
    1. Use "unsigned long" as type of physical address.
    2. Use new function unmap_device_dma to unmap the old dma.
    3. Some small incorrect bits order for aw shift.

Changelog[v5]:
    1. Do not disable and re-enable traslation and interrupt remapping. 
    2. Use old root entry table.
    3. Use old interrupt remapping table.
    4. New functions to copy data from old kernel, and save to old kernel mem.
    5. New functions to save updated root entry table and irte table.
    6. Use intel_unmap to unmap the old dma;
    7. Allocate new pages while driver is being loaded.

Changelog[v4]:
    1. Cut off the patches that move some defines and functions to new files.
    2. Reduce the numbers of patches to five, make it more easier to read.
    3. Changed the name of functions, make them consistent with current context
       get/set functions.
    4. Add change to function __iommu_attach_domain.

Changelog[v3]:
    1. Commented-out "#define DEBUG 1" to eliminate debug messages.
    2. Updated the comments about changes in each version.
    3. Fixed: one-line added to Copy-Translations patch to initialize the iovad
          struct as recommended by Baoquan He [bhe at redhat.com]
          init_iova_domain(&domain->iovad, DMA_32BIT_PFN);

Changelog[v2]:
    The following series implements a fix for:
    A kdump problem about DMA that has been discussed for a long time. That is,
    when a kernel panics and boots into the kdump kernel, DMA started by the
    panicked kernel is not stopped before the kdump kernel is booted and the
    kdump kernel disables the IOMMU while this DMA continues.  This causes the
    IOMMU to stop translating the DMA addresses as IOVAs and begin to treat
    them as physical memory addresses -- which causes the DMA to either:
        (1) generate DMAR errors or 
        (2) generate PCI SERR errors or 
        (3) transfer data to or from incorrect areas of memory. Often this 
            causes the dump to fail.

Changelog[v1]:
    The original version.

Changed in this version:
1. Do not disable and re-enable traslation and interrupt remapping. 
2. Use old root entry table.
3. Use old interrupt remapping table.
4. Use "unsigned long" as physical address.
5. Use intel_unmap to unmap the old dma;

Baoquan He <bhe at redhat.com> helps testing this patchset.
Takao Indoh <indou.takao at jp.fujitsu.com> gives valuable suggestions.

  iommu/vt-d: Update iommu_attach_domain() and its callers
  iommu/vt-d: Items required for kdump
  iommu/vt-d: Add domain-id functions
  iommu/vt-d: functions to copy data from old mem
  iommu/vt-d: Add functions to load and save old re
  iommu/vt-d: datatypes and functions used for kdump
  iommu/vt-d: enable kdump support in iommu module
  iommu/vt-d: assign new page table for dma_map
  iommu/vt-d: Copy functions for irte
  iommu/vt-d: Use old irte in kdump kernel

Signed-off-by: Bill Sumner <billsumnerlinux at gmail.com>
Signed-off-by: Li, Zhen-Hua <zhen-hual at hp.com>
Signed-off-by: Takao Indoh <indou.takao at jp.fujitsu.com>
Tested-by: Baoquan He <bhe at redhat.com>
---
 drivers/iommu/intel-iommu.c         | 1054 +++++++++++++++++++++++++++++++++--
 drivers/iommu/intel_irq_remapping.c |  104 +++-
 include/linux/intel-iommu.h         |   18 +
 3 files changed, 1134 insertions(+), 42 deletions(-)

-- 
2.0.0-rc0



From zhen-hual at hp.com  Sun Jan 11 23:06:19 2015
From: zhen-hual at hp.com (Li, Zhen-Hua)
Date: Mon, 12 Jan 2015 15:06:19 +0800
Subject: [PATCH v8 01/10] iommu/vt-d: Update iommu_attach_domain() and its
 callers
In-Reply-To: <1421046388-27925-1-git-send-email-zhen-hual@hp.com>
References: <1421046388-27925-1-git-send-email-zhen-hual@hp.com>
Message-ID: <1421046388-27925-2-git-send-email-zhen-hual@hp.com>

Allow specification of the domain-id for the new domain.
This patch only adds the 'did' parameter to iommu_attach_domain()
and modifies all of its callers to specify the default value of -1
which says "no did specified, allocate a new one".

This is no functional change from current behaviour -- just enables
a functional change to be made in a later patch.

Bill Sumner:
    Original version.

Li, Zhenhua:
    Minor change, add change to function __iommu_attach_domain.

Signed-off-by: Bill Sumner <billsumnerlinux at gmail.com>
Signed-off-by: Li, Zhen-Hua <zhen-hual at hp.com>
---
 drivers/iommu/intel-iommu.c | 34 ++++++++++++++++++++--------------
 1 file changed, 20 insertions(+), 14 deletions(-)

diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 40dfbc0..8d5c400 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -1534,31 +1534,36 @@ static struct dmar_domain *alloc_domain(int flags)
 }
 
 static int __iommu_attach_domain(struct dmar_domain *domain,
-				 struct intel_iommu *iommu)
+				 struct intel_iommu *iommu,
+				 int domain_number)
 {
 	int num;
 	unsigned long ndomains;
 
 	ndomains = cap_ndoms(iommu->cap);
-	num = find_first_zero_bit(iommu->domain_ids, ndomains);
-	if (num < ndomains) {
-		set_bit(num, iommu->domain_ids);
-		iommu->domains[num] = domain;
-	} else {
-		num = -ENOSPC;
-	}
+	if (domain_number < 0) {
+		num = find_first_zero_bit(iommu->domain_ids, ndomains);
+		if (num < ndomains) {
+			set_bit(num, iommu->domain_ids);
+			iommu->domains[num] = domain;
+		} else {
+			num = -ENOSPC;
+		}
+	} else
+		num = domain_number;
 
 	return num;
 }
 
 static int iommu_attach_domain(struct dmar_domain *domain,
-			       struct intel_iommu *iommu)
+			       struct intel_iommu *iommu,
+			       int domain_number)
 {
 	int num;
 	unsigned long flags;
 
 	spin_lock_irqsave(&iommu->lock, flags);
-	num = __iommu_attach_domain(domain, iommu);
+	num = __iommu_attach_domain(domain, iommu, domain_number);
 	spin_unlock_irqrestore(&iommu->lock, flags);
 	if (num < 0)
 		pr_err("IOMMU: no free domain ids\n");
@@ -1577,7 +1582,7 @@ static int iommu_attach_vm_domain(struct dmar_domain *domain,
 		if (iommu->domains[num] == domain)
 			return num;
 
-	return __iommu_attach_domain(domain, iommu);
+	return __iommu_attach_domain(domain, iommu, -1);
 }
 
 static void iommu_detach_domain(struct dmar_domain *domain,
@@ -2231,6 +2236,7 @@ static struct dmar_domain *get_domain_for_dev(struct device *dev, int gaw)
 	u16 dma_alias;
 	unsigned long flags;
 	u8 bus, devfn;
+	int did = -1;   /* Default to "no domain_id supplied" */
 
 	domain = find_domain(dev);
 	if (domain)
@@ -2264,7 +2270,7 @@ static struct dmar_domain *get_domain_for_dev(struct device *dev, int gaw)
 	domain = alloc_domain(0);
 	if (!domain)
 		return NULL;
-	domain->id = iommu_attach_domain(domain, iommu);
+	domain->id = iommu_attach_domain(domain, iommu, did);
 	if (domain->id < 0) {
 		free_domain_mem(domain);
 		return NULL;
@@ -2442,7 +2448,7 @@ static int __init si_domain_init(int hw)
 		return -EFAULT;
 
 	for_each_active_iommu(iommu, drhd) {
-		ret = iommu_attach_domain(si_domain, iommu);
+		ret = iommu_attach_domain(si_domain, iommu, -1);
 		if (ret < 0) {
 			domain_exit(si_domain);
 			return -EFAULT;
@@ -3866,7 +3872,7 @@ static int intel_iommu_add(struct dmar_drhd_unit *dmaru)
 	iommu_enable_translation(iommu);
 
 	if (si_domain) {
-		ret = iommu_attach_domain(si_domain, iommu);
+		ret = iommu_attach_domain(si_domain, iommu, -1);
 		if (ret < 0 || si_domain->id != ret)
 			goto disable_iommu;
 		domain_attach_iommu(si_domain, iommu);
-- 
2.0.0-rc0



From zhen-hual at hp.com  Sun Jan 11 23:06:20 2015
From: zhen-hual at hp.com (Li, Zhen-Hua)
Date: Mon, 12 Jan 2015 15:06:20 +0800
Subject: [PATCH v8 02/10] iommu/vt-d: Items required for kdump
In-Reply-To: <1421046388-27925-1-git-send-email-zhen-hual@hp.com>
References: <1421046388-27925-1-git-send-email-zhen-hual@hp.com>
Message-ID: <1421046388-27925-3-git-send-email-zhen-hual@hp.com>

Add structure type domain_values_entry used for kdump;
Add context entry functions needed for kdump.

Bill Sumner:
    Original version;

Li, Zhenhua:
    Changed the name of new functions, make them consistent with current
    context get/set functions.

Signed-off-by: Bill Sumner <billsumnerlinux at gmail.com>
Signed-off-by: Li, Zhen-Hua <zhen-hual at hp.com>
---
 drivers/iommu/intel-iommu.c | 70 +++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 70 insertions(+)

diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 8d5c400..a71de3f 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -40,6 +40,7 @@
 #include <linux/pci-ats.h>
 #include <linux/memblock.h>
 #include <linux/dma-contiguous.h>
+#include <linux/crash_dump.h>
 #include <asm/irq_remapping.h>
 #include <asm/cacheflush.h>
 #include <asm/iommu.h>
@@ -208,6 +209,12 @@ get_context_addr_from_root(struct root_entry *root)
 		NULL);
 }
 
+static inline unsigned long
+get_context_phys_from_root(struct root_entry *root)
+{
+	return  root_present(root) ? (root->val & VTD_PAGE_MASK) : 0;
+}
+
 /*
  * low 64 bits:
  * 0: present
@@ -228,6 +235,32 @@ static inline bool context_present(struct context_entry *context)
 {
 	return (context->lo & 1);
 }
+
+static inline int context_fault_enable(struct context_entry *c)
+{
+	return((c->lo >> 1) & 0x1);
+}
+
+static inline int context_translation_type(struct context_entry *c)
+{
+	return((c->lo >> 2) & 0x3);
+}
+
+static inline u64 context_address_root(struct context_entry *c)
+{
+	return((c->lo >> VTD_PAGE_SHIFT));
+}
+
+static inline int context_address_width(struct context_entry *c)
+{
+	return((c->hi >> 0) & 0x7);
+}
+
+static inline int context_domain_id(struct context_entry *c)
+{
+	return((c->hi >> 8) & 0xffff);
+}
+
 static inline void context_set_present(struct context_entry *context)
 {
 	context->lo |= 1;
@@ -313,6 +346,43 @@ static inline int first_pte_in_page(struct dma_pte *pte)
 	return !((unsigned long)pte & ~VTD_PAGE_MASK);
 }
 
+
+#ifdef CONFIG_CRASH_DUMP
+
+/*
+ * Fix Crashdump failure caused by leftover DMA through a hardware IOMMU
+ *
+ * Fixes the crashdump kernel to deal with an active iommu and legacy
+ * DMA from the (old) panicked kernel in a manner similar to how legacy
+ * DMA is handled when no hardware iommu was in use by the old kernel --
+ * allow the legacy DMA to continue into its current buffers.
+ *
+ * In the crashdump kernel, this code:
+ * 1. skips disabling the IOMMU's translating of IO Virtual Addresses (IOVA).
+ * 2. Do not re-enable IOMMU's translating.
+ * 3. In kdump kernel, use the old root entry table.
+ * 4. Leaves the current translations in-place so that legacy DMA will
+ *    continue to use its current buffers.
+ * 5. Allocates to the device drivers in the crashdump kernel
+ *    portions of the iova address ranges that are different
+ *    from the iova address ranges that were being used by the old kernel
+ *    at the time of the panic.
+ *
+ */
+
+struct domain_values_entry {
+	struct list_head link;		/* link entries into a list */
+	struct iova_domain iovad;	/* iova's that belong to this domain */
+	struct dma_pte	*pgd;		/* virtual address */
+	int    did;			/* domain id */
+	int    gaw;			/* max guest address width */
+	int    iommu_superpage;		/* Level of superpages supported:
+					   0 == 4KiB (no superpages), 1 == 2MiB,
+					   2 == 1GiB, 3 == 512GiB, 4 == 1TiB */
+};
+
+#endif /* CONFIG_CRASH_DUMP */
+
 /*
  * This domain is a statically identity mapping domain.
  *	1. This domain creats a static 1:1 mapping to all usable memory.
-- 
2.0.0-rc0



From zhen-hual at hp.com  Sun Jan 11 23:06:21 2015
From: zhen-hual at hp.com (Li, Zhen-Hua)
Date: Mon, 12 Jan 2015 15:06:21 +0800
Subject: [PATCH v8 03/10] iommu/vt-d: Add domain-id functions
In-Reply-To: <1421046388-27925-1-git-send-email-zhen-hual@hp.com>
References: <1421046388-27925-1-git-send-email-zhen-hual@hp.com>
Message-ID: <1421046388-27925-4-git-send-email-zhen-hual@hp.com>

Interfaces for when a new domain in the crashdump kernel needs some
values from the panicked kernel's context entries.

Signed-off-by: Bill Sumner <billsumnerlinux at gmail.com>
Signed-off-by: Li, Zhen-Hua <zhen-hual at hp.com>
---
 drivers/iommu/intel-iommu.c | 62 +++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 62 insertions(+)

diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index a71de3f..c594b2c 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -381,6 +381,13 @@ struct domain_values_entry {
 					   2 == 1GiB, 3 == 512GiB, 4 == 1TiB */
 };
 
+static struct domain_values_entry *intel_iommu_did_to_domain_values_entry(
+		int did, struct intel_iommu *iommu);
+
+static int intel_iommu_get_dids_from_old_kernel(struct intel_iommu *iommu);
+
+static int device_to_domain_id(struct intel_iommu *iommu, u8 bus, u8 devfn);
+
 #endif /* CONFIG_CRASH_DUMP */
 
 /*
@@ -4828,3 +4835,58 @@ static void __init check_tylersburg_isoch(void)
 	printk(KERN_WARNING "DMAR: Recommended TLB entries for ISOCH unit is 16; your BIOS set %d\n",
 	       vtisochctrl);
 }
+
+#ifdef CONFIG_CRASH_DUMP
+
+/*
+ * Interfaces for when a new domain in the crashdump kernel needs some
+ * values from the panicked kernel's context entries
+ *
+ */
+static struct domain_values_entry *intel_iommu_did_to_domain_values_entry(
+		int did, struct intel_iommu *iommu)
+{
+	struct domain_values_entry *dve;	/* iterator */
+
+	list_for_each_entry(dve, &domain_values_list[iommu->seq_id], link)
+		if (dve->did == did)
+			return dve;
+	return NULL;
+}
+
+/* Mark domain-id's from old kernel as in-use on this iommu so that a new
+ * domain-id is allocated in the case where there is a device in the new kernel
+ * that was not in the old kernel -- and therefore a new domain-id is needed.
+ */
+static int intel_iommu_get_dids_from_old_kernel(struct intel_iommu *iommu)
+{
+	struct domain_values_entry *dve;	/* iterator */
+
+	pr_info("IOMMU:%d Domain ids from panicked kernel:\n", iommu->seq_id);
+
+	list_for_each_entry(dve, &domain_values_list[iommu->seq_id], link) {
+		set_bit(dve->did, iommu->domain_ids);
+		pr_info("DID did:%d(0x%4.4x)\n", dve->did, dve->did);
+	}
+
+	pr_info("----------------------------------------\n");
+	return 0;
+}
+
+static int device_to_domain_id(struct intel_iommu *iommu, u8 bus, u8 devfn)
+{
+	int did = -1;			/* domain-id returned */
+	struct root_entry *root;
+	struct context_entry *context;
+	unsigned long flags;
+
+	spin_lock_irqsave(&iommu->lock, flags);
+	root = &iommu->root_entry[bus];
+	context = get_context_addr_from_root(root);
+	if (context && context_present(context+devfn))
+		did = context_domain_id(context+devfn);
+	spin_unlock_irqrestore(&iommu->lock, flags);
+	return did;
+}
+
+#endif /* CONFIG_CRASH_DUMP */
-- 
2.0.0-rc0



From zhen-hual at hp.com  Sun Jan 11 23:06:22 2015
From: zhen-hual at hp.com (Li, Zhen-Hua)
Date: Mon, 12 Jan 2015 15:06:22 +0800
Subject: [PATCH v8 04/10] iommu/vt-d: functions to copy data from old mem
In-Reply-To: <1421046388-27925-1-git-send-email-zhen-hual@hp.com>
References: <1421046388-27925-1-git-send-email-zhen-hual@hp.com>
Message-ID: <1421046388-27925-5-git-send-email-zhen-hual@hp.com>

Add some functions to copy the data from old kernel.
These functions are used to copy context tables and page tables.

To avoid calling iounmap between spin_lock_irqsave and spin_unlock_irqrestore,
use a link here, store the pointers , and then use iounmap to free them in
another place.

Li, Zhen-hua:
    The functions and logics.

Takao Indoh:
    Check if pfn is ram:
        if (page_is_ram(pfn))

Signed-off-by: Li, Zhen-Hua <zhen-hual at hp.com>
Signed-off-by: Takao Indoh <indou.takao at jp.fujitsu.com>
---
 drivers/iommu/intel-iommu.c | 97 +++++++++++++++++++++++++++++++++++++++++++++
 include/linux/intel-iommu.h |  9 +++++
 2 files changed, 106 insertions(+)

diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index c594b2c..2335831 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -388,6 +388,13 @@ static int intel_iommu_get_dids_from_old_kernel(struct intel_iommu *iommu);
 
 static int device_to_domain_id(struct intel_iommu *iommu, u8 bus, u8 devfn);
 
+struct iommu_remapped_entry {
+	struct list_head list;
+	void __iomem *mem;
+};
+static LIST_HEAD(__iommu_remapped_mem);
+static DEFINE_MUTEX(__iommu_mem_list_lock);
+
 #endif /* CONFIG_CRASH_DUMP */
 
 /*
@@ -4839,6 +4846,96 @@ static void __init check_tylersburg_isoch(void)
 #ifdef CONFIG_CRASH_DUMP
 
 /*
+ * Copy memory from a physically-addressed area into a virtually-addressed area
+ */
+int __iommu_load_from_oldmem(void *to, unsigned long from, unsigned long size)
+{
+	unsigned long pfn;		/* Page Frame Number */
+	size_t csize = (size_t)size;	/* Num(bytes to copy) */
+	unsigned long offset;		/* Lower 12 bits of to */
+	void __iomem *virt_mem;
+	struct iommu_remapped_entry *mapped;
+
+	pfn = from >> VTD_PAGE_SHIFT;
+	offset = from & (~VTD_PAGE_MASK);
+
+	if (page_is_ram(pfn)) {
+		memcpy(to, pfn_to_kaddr(pfn) + offset, csize);
+	} else{
+
+		mapped = kzalloc(sizeof(struct iommu_remapped_entry),
+				GFP_KERNEL);
+		if (!mapped)
+			return -ENOMEM;
+
+		virt_mem = ioremap_cache((unsigned long)from, size);
+		if (!virt_mem) {
+			kfree(mapped);
+			return -ENOMEM;
+		}
+		memcpy(to, virt_mem, size);
+
+		mutex_lock(&__iommu_mem_list_lock);
+		mapped->mem = virt_mem;
+		list_add_tail(&mapped->list, &__iommu_remapped_mem);
+		mutex_unlock(&__iommu_mem_list_lock);
+	}
+	return size;
+}
+
+/*
+ * Copy memory from a virtually-addressed area into a physically-addressed area
+ */
+int __iommu_save_to_oldmem(unsigned long to, void *from, unsigned long size)
+{
+	unsigned long pfn;		/* Page Frame Number */
+	size_t csize = (size_t)size;	/* Num(bytes to copy) */
+	unsigned long offset;		/* Lower 12 bits of to */
+	void __iomem *virt_mem;
+	struct iommu_remapped_entry *mapped;
+
+	pfn = to >> VTD_PAGE_SHIFT;
+	offset = to & (~VTD_PAGE_MASK);
+
+	if (page_is_ram(pfn)) {
+		memcpy(pfn_to_kaddr(pfn) + offset, from, csize);
+	} else{
+		mapped = kzalloc(sizeof(struct iommu_remapped_entry),
+				GFP_KERNEL);
+		if (!mapped)
+			return -ENOMEM;
+
+		virt_mem = ioremap_cache((unsigned long)to, size);
+		if (!virt_mem) {
+			kfree(mapped);
+			return -ENOMEM;
+		}
+		memcpy(virt_mem, from, size);
+		mutex_lock(&__iommu_mem_list_lock);
+		mapped->mem = virt_mem;
+		list_add_tail(&mapped->list, &__iommu_remapped_mem);
+		mutex_unlock(&__iommu_mem_list_lock);
+	}
+	return size;
+}
+
+/*
+ * Free the mapped memory for ioremap;
+ */
+int __iommu_free_mapped_mem(void)
+{
+	struct iommu_remapped_entry *mem_entry, *tmp;
+
+	mutex_lock(&__iommu_mem_list_lock);
+	list_for_each_entry_safe(mem_entry, tmp, &__iommu_remapped_mem, list) {
+		iounmap(mem_entry->mem);
+		list_del(&mem_entry->list);
+		kfree(mem_entry);
+	}
+	mutex_unlock(&__iommu_mem_list_lock);
+	return 0;
+}
+/*
  * Interfaces for when a new domain in the crashdump kernel needs some
  * values from the panicked kernel's context entries
  *
diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h
index a65208a..8ffa523 100644
--- a/include/linux/intel-iommu.h
+++ b/include/linux/intel-iommu.h
@@ -26,6 +26,7 @@
 #include <linux/iova.h>
 #include <linux/io.h>
 #include <linux/dma_remapping.h>
+#include <linux/crash_dump.h>
 #include <asm/cacheflush.h>
 #include <asm/iommu.h>
 
@@ -368,4 +369,12 @@ extern int dmar_ir_support(void);
 
 extern const struct attribute_group *intel_iommu_groups[];
 
+#ifdef CONFIG_CRASH_DUMP
+extern int __iommu_load_from_oldmem(void *to, unsigned long from,
+					unsigned long size);
+extern int __iommu_save_to_oldmem(unsigned long to, void *from,
+					unsigned long size);
+extern int __iommu_free_mapped_mem(void);
+#endif /* CONFIG_CRASH_DUMP */
+
 #endif
-- 
2.0.0-rc0



From zhen-hual at hp.com  Sun Jan 11 23:06:23 2015
From: zhen-hual at hp.com (Li, Zhen-Hua)
Date: Mon, 12 Jan 2015 15:06:23 +0800
Subject: [PATCH v8 05/10] iommu/vt-d: Add functions to load and save old re
In-Reply-To: <1421046388-27925-1-git-send-email-zhen-hual@hp.com>
References: <1421046388-27925-1-git-send-email-zhen-hual@hp.com>
Message-ID: <1421046388-27925-6-git-send-email-zhen-hual@hp.com>

Add functions to load root entry table from old kernel, and to save updated
root entry table.
Add two member in struct intel_iommu, to store the RTA in old kernel, and
the mapped virt address of it.

We use the old RTA in dump kernel, and when the iommu->root_entry is used as
a cache in kdump kernel, its phys address will not be save to RTA register,
but when its data is changed, we will save the new data to old root entry table.

Li, Zhen-hua:
    The functions and logics.

Takao Indoh:
    Add __iommu_flush_cache.

Signed-off-by: Li, Zhen-Hua <zhen-hual at hp.com>
Signed-off-by: Takao Indoh <indou.takao at jp.fujitsu.com>
---
 drivers/iommu/intel-iommu.c | 53 +++++++++++++++++++++++++++++++++++++++++++++
 include/linux/intel-iommu.h |  5 +++++
 2 files changed, 58 insertions(+)

diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 2335831..5f11f43 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -388,6 +388,10 @@ static int intel_iommu_get_dids_from_old_kernel(struct intel_iommu *iommu);
 
 static int device_to_domain_id(struct intel_iommu *iommu, u8 bus, u8 devfn);
 
+static void __iommu_load_old_root_entry(struct intel_iommu *iommu);
+
+static void __iommu_update_old_root_entry(struct intel_iommu *iommu, int index);
+
 struct iommu_remapped_entry {
 	struct list_head list;
 	void __iomem *mem;
@@ -4986,4 +4990,53 @@ static int device_to_domain_id(struct intel_iommu *iommu, u8 bus, u8 devfn)
 	return did;
 }
 
+/*
+ * Load the old root entry table to new root entry table.
+ */
+static void __iommu_load_old_root_entry(struct intel_iommu *iommu)
+{
+	if ((!iommu)
+		|| (!iommu->root_entry)
+		|| (!iommu->root_entry_old_virt)
+		|| (!iommu->root_entry_old_phys))
+		return;
+	memcpy(iommu->root_entry, iommu->root_entry_old_virt, PAGE_SIZE);
+
+	__iommu_flush_cache(iommu, iommu->root_entry, PAGE_SIZE);
+}
+
+/*
+ * When the data in new root entry table is changed, this function
+ * must be called to save the updated data to old root entry table.
+ */
+static void __iommu_update_old_root_entry(struct intel_iommu *iommu, int index)
+{
+	u8 start;
+	unsigned long size;
+	void __iomem *to;
+	void *from;
+
+	if ((!iommu)
+		|| (!iommu->root_entry)
+		|| (!iommu->root_entry_old_virt)
+		|| (!iommu->root_entry_old_phys))
+		return;
+
+	if (index < -1 || index >= ROOT_ENTRY_NR)
+		return;
+
+	if (index == -1) {
+		start = 0;
+		size = ROOT_ENTRY_NR * sizeof(struct root_entry);
+	} else {
+		start = index * sizeof(struct root_entry);
+		size = sizeof(struct root_entry);
+	}
+	to = iommu->root_entry_old_virt;
+	from = iommu->root_entry;
+	memcpy(to + start, from + start, size);
+
+	__iommu_flush_cache(iommu, to + start, size);
+}
+
 #endif /* CONFIG_CRASH_DUMP */
diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h
index 8ffa523..8e29b97 100644
--- a/include/linux/intel-iommu.h
+++ b/include/linux/intel-iommu.h
@@ -329,6 +329,11 @@ struct intel_iommu {
 	spinlock_t	lock; /* protect context, domain ids */
 	struct root_entry *root_entry; /* virtual address */
 
+#ifdef CONFIG_CRASH_DUMP
+	void __iomem *root_entry_old_virt; /* mapped from old root entry */
+	unsigned long root_entry_old_phys; /* root entry in old kernel */
+#endif
+
 	struct iommu_flush flush;
 #endif
 	struct q_inval  *qi;            /* Queued invalidation info */
-- 
2.0.0-rc0



From zhen-hual at hp.com  Sun Jan 11 23:06:24 2015
From: zhen-hual at hp.com (Li, Zhen-Hua)
Date: Mon, 12 Jan 2015 15:06:24 +0800
Subject: [PATCH v8 06/10] iommu/vt-d: datatypes and functions used for kdump
In-Reply-To: <1421046388-27925-1-git-send-email-zhen-hual@hp.com>
References: <1421046388-27925-1-git-send-email-zhen-hual@hp.com>
Message-ID: <1421046388-27925-7-git-send-email-zhen-hual@hp.com>

Populate it with support functions to copy iommu translation tables from
from the panicked kernel into the kdump kernel in the event of a crash.

Functions:
    malloc new context table and copy old context table to the new one.
    malloc new page table and copy old page table to the new one.

Bill Sumner:
    Original version, the creation of the data types and functions.

Li, Zhenhua:
    Minor change:
    Update the usage of context_get_* and context_put*, use context_*
    and context_set_* for replacement.
    Update the name of the function that copies root entry table.
    Use new function to copy old context entry tables and page tables.
    Use "unsigned long" for physical address.
    Change incorrect aw_shift[4] and a few comments in copy_context_entry().

Signed-off-by: Bill Sumner <billsumnerlinux at gmail.com>
Signed-off-by: Li, Zhen-Hua <zhen-hual at hp.com>
---
 drivers/iommu/intel-iommu.c | 547 ++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 547 insertions(+)

diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 5f11f43..277b294 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -399,6 +399,62 @@ struct iommu_remapped_entry {
 static LIST_HEAD(__iommu_remapped_mem);
 static DEFINE_MUTEX(__iommu_mem_list_lock);
 
+/* ========================================================================
+ * Copy iommu translation tables from old kernel into new  kernel.
+ * Entry to this set of functions is: intel_iommu_load_translation_tables()
+ * ------------------------------------------------------------------------
+ */
+
+/*
+ * Lists of domain_values_entry to hold domain values found during the copy.
+ * One list for each iommu in g_number_of_iommus.
+ */
+static struct list_head *domain_values_list;
+
+
+#define RET_BADCOPY -1	/* Return-code: Cannot copy translate tables */
+
+/*
+ * Struct copy_page_addr_parms is used to allow copy_page_addr()
+ * to accumulate values across multiple calls and returns.
+ */
+struct copy_page_addr_parms {
+	u32 first;	/* flag: first-time  */
+	u32 last;	/* flag: last-time */
+	u32 bus;	/* last bus number we saw */
+	u32 devfn;	/* last devfn we saw */
+	u32 shift;	/* last shift we saw */
+	u64 pte;	/* Page Table Entry */
+	u64 next_addr;	/* next-expected page_addr */
+
+	u64 page_addr;	/* page_addr accumulating size */
+	u64 page_size;	/* page_size accumulated */
+
+	struct domain_values_entry *dve;	/* to accumulate iova ranges */
+};
+
+enum returns_from_copy_context_entry {
+RET_CCE_NOT_PRESENT = 1,
+RET_CCE_NEW_PAGE_TABLES,
+RET_CCE_PASS_THROUGH_1,
+RET_CCE_PASS_THROUGH_2,
+RET_CCE_RESERVED_VALUE,
+RET_CCE_PREVIOUS_DID
+};
+
+static int copy_context_entry(struct intel_iommu *iommu, u32 bus, u32 devfn,
+			      void *ppap, struct context_entry *ce);
+
+static int copy_context_entry_table(struct intel_iommu *iommu,
+				    u32 bus, void *ppap,
+				    unsigned long *context_new_p,
+				    unsigned long context_old_phys);
+
+static int copy_root_entry_table(struct intel_iommu *iommu, void *ppap);
+
+static int intel_iommu_load_translation_tables(struct dmar_drhd_unit *drhd,
+		int g_num_of_iommus);
+
 #endif /* CONFIG_CRASH_DUMP */
 
 /*
@@ -5039,4 +5095,495 @@ static void __iommu_update_old_root_entry(struct intel_iommu *iommu, int index)
 	__iommu_flush_cache(iommu, to + start, size);
 }
 
+/*
+ * constant for initializing instances of copy_page_addr_parms properly.
+ */
+static struct copy_page_addr_parms copy_page_addr_parms_init = {1, 0};
+
+
+
+/*
+ * Lowest-level function in the 'Copy Page Tables' set
+ * Called once for each page_addr present in an iommu page-address table.
+ *
+ * Because of the depth-first traversal of the page-tables by the
+ * higher-level functions that call 'copy_page_addr', all pages
+ * of a domain will be presented in ascending order of IO Virtual Address.
+ *
+ * This function accumulates each contiguous range of these IOVAs and
+ * reserves it within the proper domain in the crashdump kernel when a
+ * non-contiguous range is detected, as determined by any of the following:
+ * 1. a change in the bus or device owning the presented page
+ * 2. a change in the page-size of the presented page (parameter shift)
+ * 3. a change in the page-table entry of the presented page
+ * 4. a presented IOVA that does not match the expected next-page address
+ * 5. the 'last' flag is set, indicating that all IOVAs have been seen.
+ */
+static int copy_page_addr(u64 page_addr, u32 shift, u32 bus, u32 devfn,
+				u64 pte, struct domain_values_entry *dve,
+				void *parms)
+{
+	struct copy_page_addr_parms *ppap = parms;
+
+	u64 page_size = ((u64)1 << shift);	/* page_size */
+	u64 pfn_lo;				/* For reserving IOVA range */
+	u64 pfn_hi;				/* For reserving IOVA range */
+	struct iova *iova_p;			/* For reserving IOVA range */
+
+	if (!ppap) {
+		pr_err("ERROR: ppap is NULL: 0x%3.3x(%3.3d) DevFn: 0x%3.3x(%3.3d) Page: 0x%16.16llx Size: 0x%16.16llx(%lld)\n",
+			bus, bus, devfn, devfn,  page_addr,
+			page_size, page_size);
+		return 0;
+	}
+
+	/* If (only extending current addr range) */
+	if (ppap->first     == 0      &&
+	    ppap->last      == 0      &&
+	    ppap->bus       == bus    &&
+	    ppap->devfn     == devfn  &&
+	    ppap->shift     == shift  &&
+	    (ppap->pte & ~VTD_PAGE_MASK) == (pte & ~VTD_PAGE_MASK) &&
+	    ppap->next_addr == page_addr) {
+
+		/* Update page size and next-expected address */
+		ppap->next_addr += page_size;
+		ppap->page_size += page_size;
+		return 0;
+	}
+
+	if (!ppap->first) {
+		/* Close-out the accumulated IOVA address range */
+
+		if (!ppap->dve) {
+			pr_err("%s ERROR: ppap->dve is NULL -- needed to reserve range for B:D:F=%2.2x:%2.2x:%1.1x\n",
+				__func__,
+				ppap->bus, ppap->devfn >> 3, ppap->devfn & 0x7);
+			return RET_BADCOPY;
+		}
+		pfn_lo = IOVA_PFN(ppap->page_addr);
+		pfn_hi = IOVA_PFN(ppap->page_addr + ppap->page_size);
+		iova_p = reserve_iova(&ppap->dve->iovad, pfn_lo, pfn_hi);
+	}
+
+	/* Prepare for a new IOVA address range */
+	ppap->first     = 0;		/* Not first-time anymore */
+	ppap->bus       = bus;
+	ppap->devfn     = devfn;
+	ppap->shift     = shift;
+	ppap->pte       = pte;
+	ppap->next_addr = page_addr + page_size; /* Next-expected page_addr */
+
+	ppap->page_addr = page_addr;	/* Addr(new page) */
+	ppap->page_size = page_size;	/* Size(new page) */
+
+	ppap->dve	= dve;	/* adr(device_values_entry for new range) */
+
+	return 0;
+}
+
+/*
+ * Recursive function to copy the tree of page tables (max 6 recursions)
+ * Parameter 'shift' controls the recursion
+ */
+static int copy_page_table(unsigned long *dma_pte_new_p,
+			   unsigned long dma_pte_phys,
+			   u32 shift, u64 page_addr,
+			   struct intel_iommu *iommu,
+			   u32 bus, u32 devfn,
+			   struct domain_values_entry *dve, void *ppap)
+{
+	int ret;			/* Integer return code */
+	struct dma_pte *p;		/* Virtual adr(each entry) iterator */
+	struct dma_pte *pgt_new_virt;	/* Adr(dma_pte in new kernel) */
+	unsigned long dma_pte_next;	/* Adr(next table down)  */
+	u64 u;				/* index(each entry in page_table) */
+
+
+	/* If (already done all levels -- problem) */
+	if (shift < 12) {
+		pr_err("ERROR %s shift < 12 %lx\n", __func__, dma_pte_phys);
+		pr_err("shift %d, page_addr %16.16llu bus %3.3u devfn %3.3u\n",
+			shift, page_addr, bus, devfn);
+		return RET_BADCOPY;
+	}
+
+	/* allocate a page table in the new kernel
+	 * copy contents from old kernel
+	 * then update each entry in the table in the new kernel
+	 */
+
+	pgt_new_virt = (struct dma_pte *)alloc_pgtable_page(iommu->node);
+	if (!pgt_new_virt)
+		return -ENOMEM;
+
+	ret = __iommu_load_from_oldmem(pgt_new_virt,
+					dma_pte_phys,
+					VTD_PAGE_SIZE);
+
+	if (ret <= 0)
+		return ret;
+
+	for (u = 0, p = pgt_new_virt; u < 512; u++, p++) {
+
+		if (((p->val & DMA_PTE_READ) == 0) &&
+		    ((p->val & DMA_PTE_WRITE) == 0))
+			continue;
+
+		if (dma_pte_superpage(p) || (shift == 12)) {
+
+			ret = copy_page_addr(page_addr | (u << shift),
+				shift, bus, devfn, p->val, dve, ppap);
+			if (ret)
+				return ret;
+			continue;
+		}
+
+		ret = copy_page_table(&dma_pte_next,
+				(p->val & VTD_PAGE_MASK),
+				shift-9, page_addr | (u << shift),
+				iommu, bus, devfn, dve, ppap);
+
+		__iommu_flush_cache(iommu, phys_to_virt(dma_pte_next),
+				VTD_PAGE_SIZE);
+
+		if (ret)
+			return ret;
+
+		p->val &= ~VTD_PAGE_MASK;	/* Clear old and set new pgd */
+		p->val |= ((u64)dma_pte_next & VTD_PAGE_MASK);
+	}
+
+	*dma_pte_new_p = virt_to_phys(pgt_new_virt);
+
+	return 0;
+}
+
+
+/*
+ * Called once for each context_entry found in a copied context_entry_table
+ * Each context_entry represents one PCIe device handled by the IOMMU.
+ *
+ * The 'domain_values_list' contains one 'domain_values_entry' for each
+ * unique domain-id found while copying the context entries for each iommu.
+ *
+ * The Intel-iommu spec. requires that every context_entry that contains
+ * the same domain-id point to the same set of page translation tables.
+ * The hardware uses this to improve the use of its translation cache.
+ * In order to insure that the copied translate tables abide by this
+ * requirement, this function keeps a list of domain-ids (dids) that
+ * have already been seen for this iommu. This function checks each entry
+ * already on the list for a domain-id that matches the domain-id in this
+ * context_entry.  If found, this function places the address of the previous
+ * context's tree of page translation tables into this context_entry.
+ * If a matching previous entry is not found, a new 'domain_values_entry'
+ * structure is created for the domain-id in this context_entry and
+ * copy_page_table is called to duplicate its tree of page tables.
+ */
+static int copy_context_entry(struct intel_iommu *iommu, u32 bus, u32 devfn,
+			      void *ppap, struct context_entry *ce)
+{
+	int ret = 0;			/* Integer Return Code */
+	u32 shift = 0;			/* bits to shift page_addr  */
+	u64 page_addr = 0;		/* Address of translated page */
+	unsigned long pgt_old_phys;	/* Adr(page_table in the old kernel) */
+	unsigned long pgt_new_phys;	/* Adr(page_table in the new kernel) */
+	u8  t;				/* Translation-type from context */
+	u8  aw;				/* Address-width from context */
+	u32 aw_shift[8] = {
+		12+9+9,		/* [000b] 30-bit AGAW (2-level page table) */
+		12+9+9+9,	/* [001b] 39-bit AGAW (3-level page table) */
+		12+9+9+9+9,	/* [010b] 48-bit AGAW (4-level page table) */
+		12+9+9+9+9+9,	/* [011b] 57-bit AGAW (5-level page table) */
+		12+9+9+9+9+9+7,	/* [100b] 64-bit AGAW (6-level page table) */
+		0,		/* [101b] Reserved */
+		0,		/* [110b] Reserved */
+		0,		/* [111b] Reserved */
+	};
+
+	struct domain_values_entry *dve = NULL;
+
+	if (!context_present(ce)) {	/* If (context not present) */
+		ret = RET_CCE_NOT_PRESENT;		/* Skip it */
+		goto exit;
+	}
+
+	t = context_translation_type(ce);
+	/* If we have seen this domain-id before on this iommu,
+	 * give this context the same page-tables and we are done.
+	 */
+	list_for_each_entry(dve, &domain_values_list[iommu->seq_id], link) {
+		if (dve->did == (int) context_domain_id(ce)) {
+			switch (t) {
+			case 0:	/* page tables */
+			case 1:	/* page tables */
+				context_set_address_root(ce,
+						virt_to_phys(dve->pgd));
+				ret = RET_CCE_PREVIOUS_DID;
+				break;
+
+			case 2:	/* Pass through */
+				if (dve->pgd == NULL)
+					ret =  RET_CCE_PASS_THROUGH_2;
+				else
+					ret = RET_BADCOPY;
+				break;
+
+			default: /* Bad value of 't'*/
+				ret = RET_BADCOPY;
+				break;
+			}
+			goto exit;
+		}
+	}
+
+	/* Since we now know that this is a new domain-id for this iommu,
+	 * create a new entry, add it to the list, and handle its
+	 * page tables.
+	 */
+
+	dve = kcalloc(1, sizeof(struct domain_values_entry), GFP_KERNEL);
+	if (!dve) {
+		ret = -ENOMEM;
+		goto exit;
+	}
+
+	dve->did = (int) context_domain_id(ce);
+	dve->gaw = (int) agaw_to_width(context_address_width(ce));
+	dve->pgd = NULL;
+	init_iova_domain(&dve->iovad, DMA_32BIT_PFN);
+
+	list_add(&dve->link, &domain_values_list[iommu->seq_id]);
+
+
+	if (t == 0 || t == 1) {		/* If (context has page tables) */
+		aw = context_address_width(ce);
+		shift = aw_shift[aw];
+
+		pgt_old_phys = context_address_root(ce) << VTD_PAGE_SHIFT;
+
+		ret = copy_page_table(&pgt_new_phys, pgt_old_phys,
+			shift-9, page_addr, iommu, bus, devfn, dve, ppap);
+
+		__iommu_flush_cache(iommu, phys_to_virt(pgt_new_phys),
+				VTD_PAGE_SIZE);
+
+		if (ret)		/* if (problem) bail out */
+			goto exit;
+
+		context_set_address_root(ce, pgt_new_phys);
+		dve->pgd = phys_to_virt(pgt_new_phys);
+		ret = RET_CCE_NEW_PAGE_TABLES;
+		goto exit;
+	}
+
+	if (t == 2) {		/* If (Identity mapped pass-through) */
+		ret = RET_CCE_PASS_THROUGH_1;	/* REVISIT: Skip for now */
+		goto exit;
+	}
+
+	ret = RET_CCE_RESERVED_VALUE;	/* Else ce->t is a Reserved value */
+	/* Note fall-through */
+
+exit:	/* all returns come through here to insure good clean-up */
+	return ret;
+}
+
+
+/*
+ * Called once for each context_entry_table found in the root_entry_table
+ */
+static int copy_context_entry_table(struct intel_iommu *iommu,
+				    u32 bus, void *ppap,
+				    unsigned long *context_new_p,
+				    unsigned long context_old_phys)
+{
+	int ret = 0;				/* Integer return code */
+	struct context_entry *ce;		/* Iterator */
+	unsigned long context_new_phys;		/* adr(table in new kernel) */
+	struct context_entry *context_new_virt;	/* adr(table in new kernel) */
+	u32 devfn = 0;				/* PCI Device & function */
+
+	/* allocate a context-entry table in the new kernel
+	 * copy contents from old kernel
+	 * then update each entry in the table in the new kernel
+	 */
+	context_new_virt =
+		(struct context_entry *)alloc_pgtable_page(iommu->node);
+	if (!context_new_virt)
+		return -ENOMEM;
+
+	context_new_phys = virt_to_phys(context_new_virt);
+
+	__iommu_load_from_oldmem(context_new_virt,
+					context_old_phys,
+					VTD_PAGE_SIZE);
+
+	for (devfn = 0, ce = context_new_virt; devfn < 256; devfn++, ce++) {
+
+		if (!context_present(ce))	/* If (context not present) */
+			continue;		/* Skip it */
+
+		ret = copy_context_entry(iommu, bus, devfn, ppap, ce);
+		if (ret < 0)		/* if (problem) */
+			return RET_BADCOPY;
+
+		switch (ret) {
+		case RET_CCE_NOT_PRESENT:
+			continue;
+		case RET_CCE_NEW_PAGE_TABLES:
+			continue;
+		case RET_CCE_PASS_THROUGH_1:
+			continue;
+		case RET_CCE_PASS_THROUGH_2:
+			continue;
+		case RET_CCE_RESERVED_VALUE:
+			return RET_BADCOPY;
+		case RET_CCE_PREVIOUS_DID:
+			continue;
+		default:
+			return RET_BADCOPY;
+		};
+	}
+
+	*context_new_p = context_new_phys;
+	return 0;
+}
+
+
+/*
+ * Highest-level function in the 'copy translation tables' set of functions
+ */
+static int copy_root_entry_table(struct intel_iommu *iommu, void *ppap)
+{
+	int ret = 0;				/* Integer return code */
+	u32 bus;				/* Index: root-entry-table */
+	struct root_entry  *re;			/* Virt(iterator: new table) */
+	unsigned long context_old_phys;	/* Phys(context table entry) */
+	unsigned long context_new_phys;	/* Phys(new context_entry) */
+
+	/*
+	 * allocate a root-entry table in the new kernel
+	 * copy contents from old kernel
+	 * then update each entry in the table in the new kernel
+	 */
+
+	if (!iommu->root_entry_old_phys)
+		return -ENOMEM;
+
+	for (bus = 0, re = iommu->root_entry; bus < 256; bus += 1, re += 1) {
+		if (!root_present(re))
+			continue;
+
+		context_old_phys = get_context_phys_from_root(re);
+
+		if (!context_old_phys)
+			continue;
+
+		context_new_phys = 0;
+		ret = copy_context_entry_table(iommu, bus, ppap,
+						&context_new_phys,
+						context_old_phys);
+		__iommu_flush_cache(iommu,
+				phys_to_virt(context_new_phys),
+				VTD_PAGE_SIZE);
+
+		if (ret)
+			return ret;
+
+		set_root_value(re, context_new_phys);
+	}
+
+	return 0;
+}
+/*
+ * Interface to the "copy translation tables" set of functions
+ * from mainline code.
+ */
+static int intel_iommu_load_translation_tables(struct dmar_drhd_unit *drhd,
+		int g_num_of_iommus)
+{
+	struct intel_iommu *iommu;	/* Virt(iommu hardware registers) */
+	unsigned long long q;		/* quadword scratch */
+	int ret = 0;			/* Integer return code */
+	int i = 0;			/* Loop index */
+	unsigned long flags;
+
+	/* Structure so copy_page_addr() can accumulate things
+	 * over multiple calls and returns
+	 */
+	struct copy_page_addr_parms ppa_parms = copy_page_addr_parms_init;
+	struct copy_page_addr_parms *ppap = &ppa_parms;
+
+
+	iommu = drhd->iommu;
+	q = dmar_readq(iommu->reg + DMAR_RTADDR_REG);
+	if (!q)
+		return -1;
+
+	/* If (list needs initializing) do it here */
+	if (!domain_values_list) {
+		domain_values_list =
+			 kcalloc(g_num_of_iommus, sizeof(struct list_head),
+					GFP_KERNEL);
+
+		if (!domain_values_list) {
+			pr_err("Allocation failed for domain_values_list array\n");
+			return -ENOMEM;
+		}
+		for (i = 0; i < g_num_of_iommus; i++)
+			INIT_LIST_HEAD(&domain_values_list[i]);
+	}
+
+	spin_lock_irqsave(&iommu->lock, flags);
+
+	/* Load the root-entry table from the old kernel
+	 * foreach context_entry_table in root_entry
+	 *    foreach context_entry in context_entry_table
+	 *       foreach level-1 page_table_entry in context_entry
+	 *          foreach level-2 page_table_entry in level 1 page_table_entry
+	 *             Above pattern continues up to 6 levels of page tables
+	 *                Sanity-check the entry
+	 *                Process the bus, devfn, page_address, page_size
+	 */
+	if (!iommu->root_entry) {
+		iommu->root_entry =
+			(struct root_entry *)alloc_pgtable_page(iommu->node);
+		if (!iommu->root_entry) {
+			spin_unlock_irqrestore(&iommu->lock, flags);
+			return -ENOMEM;
+		}
+	}
+
+	iommu->root_entry_old_phys = q & VTD_PAGE_MASK;
+	if (!iommu->root_entry_old_phys) {
+		pr_err("Could not read old root entry address.");
+		return -1;
+	}
+
+	iommu->root_entry_old_virt = ioremap_cache(iommu->root_entry_old_phys,
+						VTD_PAGE_SIZE);
+	if (!iommu->root_entry_old_virt) {
+		pr_err("Could not map the old root entry.");
+		return -ENOMEM;
+	}
+
+	__iommu_load_old_root_entry(iommu);
+	ret = copy_root_entry_table(iommu, ppap);
+	__iommu_flush_cache(iommu, iommu->root_entry, PAGE_SIZE);
+	__iommu_update_old_root_entry(iommu, -1);
+
+	spin_unlock_irqrestore(&iommu->lock, flags);
+
+	__iommu_free_mapped_mem();
+
+	if (ret)
+		return ret;
+
+	ppa_parms.last = 1;
+	copy_page_addr(0, 0, 0, 0, 0, NULL, ppap);
+
+	return 0;
+}
+
 #endif /* CONFIG_CRASH_DUMP */
-- 
2.0.0-rc0



From zhen-hual at hp.com  Sun Jan 11 23:06:25 2015
From: zhen-hual at hp.com (Li, Zhen-Hua)
Date: Mon, 12 Jan 2015 15:06:25 +0800
Subject: [PATCH v8 07/10] iommu/vt-d: enable kdump support in iommu module
In-Reply-To: <1421046388-27925-1-git-send-email-zhen-hual@hp.com>
References: <1421046388-27925-1-git-send-email-zhen-hual@hp.com>
Message-ID: <1421046388-27925-8-git-send-email-zhen-hual@hp.com>

Modify the operation of the following functions when called during crash dump:
    device_to_domain_id
    get_domain_for_dev
    init_dmars
    intel_iommu_init

Bill Sumner:
    Original version.

Zhenhua:
    Minor change,
    The name of new calling functions.
    Do not disable and re-enable TE in kdump kernel.

Signed-off-by: Bill Sumner <billsumnerlinux at gmail.com>
Signed-off-by: Li, Zhen-Hua <zhen-hual at hp.com>
---
 drivers/iommu/intel-iommu.c | 135 +++++++++++++++++++++++++++++++++++++++-----
 1 file changed, 120 insertions(+), 15 deletions(-)

diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 277b294..324c504 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -907,6 +907,11 @@ static struct context_entry * device_to_context_entry(struct intel_iommu *iommu,
 		set_root_value(root, phy_addr);
 		set_root_present(root);
 		__iommu_flush_cache(iommu, root, sizeof(*root));
+
+#ifdef CONFIG_CRASH_DUMP
+		if (is_kdump_kernel())
+			__iommu_update_old_root_entry(iommu, bus);
+#endif
 	}
 	spin_unlock_irqrestore(&iommu->lock, flags);
 	return &context[devfn];
@@ -958,7 +963,8 @@ static void free_context_table(struct intel_iommu *iommu)
 
 	spin_lock_irqsave(&iommu->lock, flags);
 	if (!iommu->root_entry) {
-		goto out;
+		spin_unlock_irqrestore(&iommu->lock, flags);
+		return;
 	}
 	for (i = 0; i < ROOT_ENTRY_NR; i++) {
 		root = &iommu->root_entry[i];
@@ -966,10 +972,23 @@ static void free_context_table(struct intel_iommu *iommu)
 		if (context)
 			free_pgtable_page(context);
 	}
+
+#ifdef CONFIG_CRASH_DUMP
+	if (is_kdump_kernel()) {
+		iommu->root_entry_old_phys = 0;
+		root = iommu->root_entry_old_virt;
+		iommu->root_entry_old_virt = NULL;
+	}
+#endif
 	free_pgtable_page(iommu->root_entry);
 	iommu->root_entry = NULL;
-out:
+
 	spin_unlock_irqrestore(&iommu->lock, flags);
+
+#ifdef CONFIG_CRASH_DUMP
+	if (is_kdump_kernel())
+		iounmap(root);
+#endif
 }
 
 static struct dma_pte *pfn_to_dma_pte(struct dmar_domain *domain,
@@ -2381,6 +2400,9 @@ static struct dmar_domain *get_domain_for_dev(struct device *dev, int gaw)
 	unsigned long flags;
 	u8 bus, devfn;
 	int did = -1;   /* Default to "no domain_id supplied" */
+#ifdef CONFIG_CRASH_DUMP
+	struct domain_values_entry *dve = NULL;
+#endif /* CONFIG_CRASH_DUMP */
 
 	domain = find_domain(dev);
 	if (domain)
@@ -2414,6 +2436,24 @@ static struct dmar_domain *get_domain_for_dev(struct device *dev, int gaw)
 	domain = alloc_domain(0);
 	if (!domain)
 		return NULL;
+
+#ifdef CONFIG_CRASH_DUMP
+	if (is_kdump_kernel()) {
+		/*
+		 * if this device had a did in the old kernel
+		 * use its values instead of generating new ones
+		 */
+		did = device_to_domain_id(iommu, bus, devfn);
+		if (did > 0 || (did == 0 && !cap_caching_mode(iommu->cap)))
+			dve = intel_iommu_did_to_domain_values_entry(did,
+								iommu);
+		if (dve)
+			gaw = dve->gaw;
+		else
+			did = -1;
+	}
+#endif /* CONFIG_CRASH_DUMP */
+
 	domain->id = iommu_attach_domain(domain, iommu, did);
 	if (domain->id < 0) {
 		free_domain_mem(domain);
@@ -2425,6 +2465,18 @@ static struct dmar_domain *get_domain_for_dev(struct device *dev, int gaw)
 		return NULL;
 	}
 
+#ifdef CONFIG_CRASH_DUMP
+	if (is_kdump_kernel() && dve) {
+
+		if (domain->pgd)
+			free_pgtable_page(domain->pgd);
+
+		domain->pgd = dve->pgd;
+
+		copy_reserved_iova(&dve->iovad, &domain->iovad);
+	}
+#endif /* CONFIG_CRASH_DUMP */
+
 	/* register PCI DMA alias device */
 	if (dev_is_pci(dev)) {
 		tmp = dmar_insert_dev_info(iommu, PCI_BUS_NUM(dma_alias),
@@ -2948,14 +3000,35 @@ static int __init init_dmars(void)
 		if (ret)
 			goto free_iommu;
 
-		/*
-		 * TBD:
-		 * we could share the same root & context tables
-		 * among all IOMMU's. Need to Split it later.
-		 */
-		ret = iommu_alloc_root_entry(iommu);
-		if (ret)
-			goto free_iommu;
+#ifdef CONFIG_CRASH_DUMP
+		if (is_kdump_kernel()) {
+			pr_info("IOMMU Copying translate tables from panicked kernel\n");
+			ret = intel_iommu_load_translation_tables(drhd,
+					g_num_of_iommus);
+			if (ret) {
+				pr_err("IOMMU: Copy translate tables failed\n");
+
+				/* Best to stop trying */
+				goto free_iommu;
+			}
+			pr_info("IOMMU: root_cache:0x%12.12llx phys:0x%12.12llx\n",
+				(u64)iommu->root_entry,
+				(u64)iommu->root_entry_old_phys);
+			intel_iommu_get_dids_from_old_kernel(iommu);
+		} else {
+#endif /* CONFIG_CRASH_DUMP */
+			/*
+			 * TBD:
+			 * we could share the same root & context tables
+			 * among all IOMMU's. Need to Split it later.
+			 */
+			ret = iommu_alloc_root_entry(iommu);
+			if (ret)
+				goto free_iommu;
+#ifdef CONFIG_CRASH_DUMP
+		}
+#endif
+
 		if (!ecap_pass_through(iommu->ecap))
 			hw_pass_through = 0;
 	}
@@ -2972,6 +3045,16 @@ static int __init init_dmars(void)
 
 	check_tylersburg_isoch();
 
+#ifdef CONFIG_CRASH_DUMP
+	/*
+	 * In the crashdump kernel: Skip setting-up new domains for
+	 * si, rmrr, and the isa bus on the expectation that these
+	 * translations were copied from the old kernel.
+	 */
+	if (is_kdump_kernel())
+		goto skip_new_domains_for_si_rmrr_isa;
+#endif /* CONFIG_CRASH_DUMP */
+
 	/*
 	 * If pass through is not set or not enabled, setup context entries for
 	 * identity mappings for rmrr, gfx, and isa and may fall back to static
@@ -3012,6 +3095,10 @@ static int __init init_dmars(void)
 
 	iommu_prepare_isa();
 
+#ifdef CONFIG_CRASH_DUMP
+skip_new_domains_for_si_rmrr_isa:;
+#endif /* CONFIG_CRASH_DUMP */
+
 	/*
 	 * for each drhd
 	 *   enable fault log
@@ -3040,7 +3127,15 @@ static int __init init_dmars(void)
 
 		iommu->flush.flush_context(iommu, 0, 0, 0, DMA_CCMD_GLOBAL_INVL);
 		iommu->flush.flush_iotlb(iommu, 0, 0, 0, DMA_TLB_GLOBAL_FLUSH);
-		iommu_enable_translation(iommu);
+
+#ifdef CONFIG_CRASH_DUMP
+		if (is_kdump_kernel()) {
+			if (!(iommu->gcmd & DMA_GCMD_TE))
+				iommu_enable_translation(iommu);
+		} else
+#endif
+			iommu_enable_translation(iommu);
+
 		iommu_disable_protect_mem_regions(iommu);
 	}
 
@@ -4343,12 +4438,22 @@ int __init intel_iommu_init(void)
 		goto out_free_dmar;
 	}
 
+#ifdef CONFIG_CRASH_DUMP
 	/*
-	 * Disable translation if already enabled prior to OS handover.
+	 * If (This is the crash kernel)
+	 *    Set: copy iommu translate tables from old kernel
+	 *    Skip disabling the iommu hardware translations
 	 */
-	for_each_active_iommu(iommu, drhd)
-		if (iommu->gcmd & DMA_GCMD_TE)
-			iommu_disable_translation(iommu);
+	if (is_kdump_kernel()) {
+		pr_info("IOMMU Skip disabling iommu hardware translations\n");
+	} else
+#endif /* CONFIG_CRASH_DUMP */
+		/*
+		 * Disable translation if already enabled prior to OS handover.
+		 */
+		for_each_active_iommu(iommu, drhd)
+			if (iommu->gcmd & DMA_GCMD_TE)
+				iommu_disable_translation(iommu);
 
 	if (dmar_dev_scope_init() < 0) {
 		if (force_on)
-- 
2.0.0-rc0



From zhen-hual at hp.com  Sun Jan 11 23:06:26 2015
From: zhen-hual at hp.com (Li, Zhen-Hua)
Date: Mon, 12 Jan 2015 15:06:26 +0800
Subject: [PATCH v8 08/10] iommu/vt-d: assign new page table for dma_map
In-Reply-To: <1421046388-27925-1-git-send-email-zhen-hual@hp.com>
References: <1421046388-27925-1-git-send-email-zhen-hual@hp.com>
Message-ID: <1421046388-27925-9-git-send-email-zhen-hual@hp.com>

When a device driver issues the first dma_map command for a
device, we assign a new and empty page-table, thus removing all
mappings from the old kernel for the device.

Signed-off-by: Li, Zhen-Hua <zhen-hual at hp.com>
---
 drivers/iommu/intel-iommu.c | 56 ++++++++++++++++++++++++++++++++++++++-------
 1 file changed, 48 insertions(+), 8 deletions(-)

diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 324c504..ccbad3f 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -44,6 +44,7 @@
 #include <asm/irq_remapping.h>
 #include <asm/cacheflush.h>
 #include <asm/iommu.h>
+#include <linux/dma-mapping.h>
 
 #include "irq_remapping.h"
 
@@ -455,6 +456,8 @@ static int copy_root_entry_table(struct intel_iommu *iommu, void *ppap);
 static int intel_iommu_load_translation_tables(struct dmar_drhd_unit *drhd,
 		int g_num_of_iommus);
 
+static void unmap_device_dma(struct dmar_domain *domain, struct device *dev);
+
 #endif /* CONFIG_CRASH_DUMP */
 
 /*
@@ -3196,14 +3199,30 @@ static struct dmar_domain *__get_valid_domain_for_dev(struct device *dev)
 		return NULL;
 	}
 
-	/* make sure context mapping is ok */
-	if (unlikely(!domain_context_mapped(dev))) {
-		ret = domain_context_mapping(domain, dev, CONTEXT_TT_MULTI_LEVEL);
-		if (ret) {
-			printk(KERN_ERR "Domain context map for %s failed",
-			       dev_name(dev));
-			return NULL;
-		}
+	/* if in kdump kernel, we need to unmap the mapped dma pages,
+	 * detach this device first.
+	 */
+	if (likely(domain_context_mapped(dev))) {
+#ifdef CONFIG_CRASH_DUMP
+		if (is_kdump_kernel()) {
+			unmap_device_dma(domain, dev);
+			domain = get_domain_for_dev(dev,
+				DEFAULT_DOMAIN_ADDRESS_WIDTH);
+			if (!domain) {
+				pr_err("Allocating domain for %s failed",
+				       dev_name(dev));
+				return NULL;
+			}
+		} else
+#endif
+			return domain;
+	}
+
+	ret = domain_context_mapping(domain, dev, CONTEXT_TT_MULTI_LEVEL);
+	if (ret) {
+		pr_err("Domain context map for %s failed",
+		       dev_name(dev));
+		return NULL;
 	}
 
 	return domain;
@@ -5691,4 +5710,25 @@ static int intel_iommu_load_translation_tables(struct dmar_drhd_unit *drhd,
 	return 0;
 }
 
+static void unmap_device_dma(struct dmar_domain *domain, struct device *dev)
+{
+	struct intel_iommu *iommu;
+	struct context_entry *ce;
+	struct iova *iova;
+	u8 bus, devfn;
+	phys_addr_t phys_addr;
+	dma_addr_t dev_addr;
+
+	iommu = device_to_iommu(dev, &bus, &devfn);
+	ce = device_to_context_entry(iommu, bus, devfn);
+	phys_addr = context_address_root(ce) << VTD_PAGE_SHIFT;
+	dev_addr = phys_to_dma(dev, phys_addr);
+
+	iova = find_iova(&domain->iovad, IOVA_PFN(dev_addr));
+	if (iova)
+		intel_unmap(dev, dev_addr);
+
+	domain_remove_one_dev_info(domain, dev);
+}
+
 #endif /* CONFIG_CRASH_DUMP */
-- 
2.0.0-rc0



From zhen-hual at hp.com  Sun Jan 11 23:06:27 2015
From: zhen-hual at hp.com (Li, Zhen-Hua)
Date: Mon, 12 Jan 2015 15:06:27 +0800
Subject: [PATCH v8 09/10] iommu/vt-d: Copy functions for irte
In-Reply-To: <1421046388-27925-1-git-send-email-zhen-hual@hp.com>
References: <1421046388-27925-1-git-send-email-zhen-hual@hp.com>
Message-ID: <1421046388-27925-10-git-send-email-zhen-hual@hp.com>

Functions to copy the irte data from the old kernel into the kdump kernel.

Signed-off-by: Li, Zhen-Hua <zhen-hual at hp.com>
---
 drivers/iommu/intel_irq_remapping.c | 62 +++++++++++++++++++++++++++++++++++++
 include/linux/intel-iommu.h         |  4 +++
 2 files changed, 66 insertions(+)

diff --git a/drivers/iommu/intel_irq_remapping.c b/drivers/iommu/intel_irq_remapping.c
index a55b207..d37fd62 100644
--- a/drivers/iommu/intel_irq_remapping.c
+++ b/drivers/iommu/intel_irq_remapping.c
@@ -8,6 +8,7 @@
 #include <linux/irq.h>
 #include <linux/intel-iommu.h>
 #include <linux/acpi.h>
+#include <linux/crash_dump.h>
 #include <asm/io_apic.h>
 #include <asm/smp.h>
 #include <asm/cpu.h>
@@ -17,6 +18,11 @@
 
 #include "irq_remapping.h"
 
+#ifdef CONFIG_CRASH_DUMP
+static int __iommu_load_old_irte(struct intel_iommu *iommu);
+static int __iommu_update_old_irte(struct intel_iommu *iommu, int index);
+#endif /* CONFIG_CRASH_DUMP */
+
 struct ioapic_scope {
 	struct intel_iommu *iommu;
 	unsigned int id;
@@ -1296,3 +1302,59 @@ int dmar_ir_hotplug(struct dmar_drhd_unit *dmaru, bool insert)
 
 	return ret;
 }
+
+#ifdef CONFIG_CRASH_DUMP
+
+static int __iommu_load_old_irte(struct intel_iommu *iommu)
+{
+	if ((!iommu)
+		|| (!iommu->ir_table)
+		|| (!iommu->ir_table->base)
+		|| (!iommu->ir_table->base_old_phys)
+		|| (!iommu->ir_table->base_old_virt))
+		return -1;
+
+	memcpy(iommu->ir_table->base,
+		iommu->ir_table->base_old_virt,
+		INTR_REMAP_TABLE_ENTRIES*sizeof(struct irte));
+
+	__iommu_flush_cache(iommu, iommu->ir_table->base,
+		INTR_REMAP_TABLE_ENTRIES*sizeof(struct irte));
+
+	return 0;
+}
+
+static int __iommu_update_old_irte(struct intel_iommu *iommu, int index)
+{
+	int start;
+	unsigned long size;
+	void __iomem *to;
+	void *from;
+
+	if ((!iommu)
+		|| (!iommu->ir_table)
+		|| (!iommu->ir_table->base)
+		|| (!iommu->ir_table->base_old_phys)
+		|| (!iommu->ir_table->base_old_virt))
+		return -1;
+
+	if (index < -1 || index >= INTR_REMAP_TABLE_ENTRIES)
+		return -1;
+
+	if (index == -1) {
+		start = 0;
+		size = INTR_REMAP_TABLE_ENTRIES * sizeof(struct irte);
+	} else {
+		start = index * sizeof(struct irte);
+		size = sizeof(struct irte);
+	}
+
+	to = iommu->ir_table->base_old_virt;
+	from = iommu->ir_table->base;
+	memcpy(to + start, from + start, size);
+
+	__iommu_flush_cache(iommu, to + start, size);
+
+	return 0;
+}
+#endif /* CONFIG_CRASH_DUMP */
diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h
index 8e29b97..76c6ea5 100644
--- a/include/linux/intel-iommu.h
+++ b/include/linux/intel-iommu.h
@@ -290,6 +290,10 @@ struct q_inval {
 struct ir_table {
 	struct irte *base;
 	unsigned long *bitmap;
+#ifdef CONFIG_CRASH_DUMP
+	void __iomem *base_old_virt;
+	unsigned long base_old_phys;
+#endif
 };
 #endif
 
-- 
2.0.0-rc0



From zhen-hual at hp.com  Sun Jan 11 23:06:28 2015
From: zhen-hual at hp.com (Li, Zhen-Hua)
Date: Mon, 12 Jan 2015 15:06:28 +0800
Subject: [PATCH v8 10/10] iommu/vt-d: Use old irte in kdump kernel
In-Reply-To: <1421046388-27925-1-git-send-email-zhen-hual@hp.com>
References: <1421046388-27925-1-git-send-email-zhen-hual@hp.com>
Message-ID: <1421046388-27925-11-git-send-email-zhen-hual@hp.com>

Fix the intr-remapping fault.

[1.594890] dmar: DRHD: handling fault status reg 2
[1.594894] dmar: INTR-REMAP: Request device [[41:00.0] fault index 4d
[1.594894] INTR-REMAP:[fault reason 34] Present field in the IRTE entry
is clear

Use old irte in kdump kernel, do not disable and re-enable interrupt
remapping.

Signed-off-by: Li, Zhen-Hua <zhen-hual at hp.com>
---
 drivers/iommu/intel_irq_remapping.c | 42 ++++++++++++++++++++++++++++++++-----
 1 file changed, 37 insertions(+), 5 deletions(-)

diff --git a/drivers/iommu/intel_irq_remapping.c b/drivers/iommu/intel_irq_remapping.c
index d37fd62..58356cb 100644
--- a/drivers/iommu/intel_irq_remapping.c
+++ b/drivers/iommu/intel_irq_remapping.c
@@ -198,6 +198,11 @@ static int modify_irte(int irq, struct irte *irte_modified)
 
 	set_64bit(&irte->low, irte_modified->low);
 	set_64bit(&irte->high, irte_modified->high);
+
+#ifdef CONFIG_CRASH_DUMP
+	if (is_kdump_kernel())
+		__iommu_update_old_irte(iommu, index);
+#endif
 	__iommu_flush_cache(iommu, irte, sizeof(*irte));
 
 	rc = qi_flush_iec(iommu, index, 0);
@@ -259,6 +264,11 @@ static int clear_entries(struct irq_2_iommu *irq_iommu)
 	bitmap_release_region(iommu->ir_table->bitmap, index,
 			      irq_iommu->irte_mask);
 
+#ifdef CONFIG_CRASH_DUMP
+	if (is_kdump_kernel())
+		__iommu_update_old_irte(iommu, -1);
+#endif
+
 	return qi_flush_iec(iommu, index, irq_iommu->irte_mask);
 }
 
@@ -640,11 +650,20 @@ static int __init intel_enable_irq_remapping(void)
 		 */
 		dmar_fault(-1, iommu);
 
-		/*
-		 * Disable intr remapping and queued invalidation, if already
-		 * enabled prior to OS handover.
-		 */
-		iommu_disable_irq_remapping(iommu);
+#ifdef CONFIG_CRASH_DUMP
+		if (is_kdump_kernel()) {
+			/* Do notdisable irq and then re-enable again. */
+		} else {
+#endif
+			/*
+			 * Disable intr remapping and queued invalidation,
+			 * if already enabled prior to OS handover.
+			 */
+			iommu_disable_irq_remapping(iommu);
+
+#ifdef CONFIG_CRASH_DUMP
+		}
+#endif
 
 		dmar_disable_qi(iommu);
 	}
@@ -687,7 +706,20 @@ static int __init intel_enable_irq_remapping(void)
 		if (intel_setup_irq_remapping(iommu))
 			goto error;
 
+#ifdef CONFIG_CRASH_DUMP
+	if (is_kdump_kernel()) {
+		unsigned long long q;
+
+		q = dmar_readq(iommu->reg + DMAR_IRTA_REG);
+		iommu->ir_table->base_old_phys = q & VTD_PAGE_MASK;
+		iommu->ir_table->base_old_virt = ioremap_cache(
+				iommu->ir_table->base_old_phys,
+				INTR_REMAP_TABLE_ENTRIES*sizeof(struct irte));
+		__iommu_load_old_irte(iommu);
+	} else
+#endif
 		iommu_set_irq_remapping(iommu, eim);
+
 		setup = 1;
 	}
 
-- 
2.0.0-rc0



From zhen-hual at hp.com  Mon Jan 12 00:00:43 2015
From: zhen-hual at hp.com (Li, ZhenHua)
Date: Mon, 12 Jan 2015 16:00:43 +0800
Subject: [PATCH v8 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel
In-Reply-To: <1421046388-27925-1-git-send-email-zhen-hual@hp.com>
References: <1421046388-27925-1-git-send-email-zhen-hual@hp.com>
Message-ID: <54B37F2B.9050708@hp.com>

Comparing to v7, this version adds only a few lines code:

In function copy_page_table,

+		__iommu_flush_cache(iommu, phys_to_virt(dma_pte_next),
+				VTD_PAGE_SIZE);


On 01/12/2015 03:06 PM, Li, Zhen-Hua wrote:
> This patchset is an update of Bill Sumner's patchset, implements a fix for:
> If a kernel boots with intel_iommu=on on a system that supports intel vt-d,
> when a panic happens, the kdump kernel will boot with these faults:
>
>      dmar: DRHD: handling fault status reg 102
>      dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr fff80000
>      DMAR:[fault reason 01] Present bit in root entry is clear
>
>      dmar: DRHD: handling fault status reg 2
>      dmar: INTR-REMAP: Request device [[61:00.0] fault index 42
>      INTR-REMAP:[fault reason 34] Present field in the IRTE entry is clear
>
> On some system, the interrupt remapping fault will also happen even if the
> intel_iommu is not set to on, because the interrupt remapping will be enabled
> when x2apic is needed by the system.
>
> The cause of the DMA fault is described in Bill's original version, and the
> INTR-Remap fault is caused by a similar reason. In short, the initialization
> of vt-d drivers causes the in-flight DMA and interrupt requests get wrong
> response.
>
> To fix this problem, we modifies the behaviors of the intel vt-d in the
> crashdump kernel:
>
> For DMA Remapping:
> 1. To accept the vt-d hardware in an active state,
> 2. Do not disable and re-enable the translation, keep it enabled.
> 3. Use the old root entry table, do not rewrite the RTA register.
> 4. Malloc and use new context entry table and page table, copy data from the
>     old ones that used by the old kernel.
> 5. to use different portions of the iova address ranges for the device drivers
>     in the crashdump kernel than the iova ranges that were in-use at the time
>     of the panic.
> 6. After device driver is loaded, when it issues the first dma_map command,
>     free the dmar_domain structure for this device, and generate a new one, so
>     that the device can be assigned a new and empty page table.
> 7. When a new context entry table is generated, we also save its address to
>     the old root entry table.
>
> For Interrupt Remapping:
> 1. To accept the vt-d hardware in an active state,
> 2. Do not disable and re-enable the interrupt remapping, keep it enabled.
> 3. Use the old interrupt remapping table, do not rewrite the IRTA register.
> 4. When ioapic entry is setup, the interrupt remapping table is changed, and
>     the updated data will be stored to the old interrupt remapping table.
>
> Advantages of this approach:
> 1. All manipulation of the IO-device is done by the Linux device-driver
>     for that device.
> 2. This approach behaves in a manner very similar to operation without an
>     active iommu.
> 3. Any activity between the IO-device and its RMRR areas is handled by the
>     device-driver in the same manner as during a non-kdump boot.
> 4. If an IO-device has no driver in the kdump kernel, it is simply left alone.
>     This supports the practice of creating a special kdump kernel without
>     drivers for any devices that are not required for taking a crashdump.
> 5. Minimal code-changes among the existing mainline intel vt-d code.
>
> Summary of changes in this patch set:
> 1. Added some useful function for root entry table in code intel-iommu.c
> 2. Added new members to struct root_entry and struct irte;
> 3. Functions to load old root entry table to iommu->root_entry from the memory
>     of old kernel.
> 4. Functions to malloc new context entry table and page table and copy the data
>     from the old ones to the malloced new ones.
> 5. Functions to enable support for DMA remapping in kdump kernel.
> 6. Functions to load old irte data from the old kernel to the kdump kernel.
> 7. Some code changes that support other behaviours that have been listed.
> 8. In the new functions, use physical address as "unsigned long" type, not
>     pointers.
>
> Original version by Bill Sumner:
>      https://lkml.org/lkml/2014/1/10/518
>      https://lkml.org/lkml/2014/4/15/716
>      https://lkml.org/lkml/2014/4/24/836
>
> Zhenhua's updates:
>      https://lkml.org/lkml/2014/10/21/134
>      https://lkml.org/lkml/2014/12/15/121
>      https://lkml.org/lkml/2014/12/22/53
>      https://lkml.org/lkml/2015/1/6/1166
>
> Changelog[v8]:
>      1. Add a missing __iommu_flush_cache in function copy_page_table.
>
> Changelog[v7]:
>      1. Use __iommu_flush_cache to flush the data to hardware.
>
> Changelog[v6]:
>      1. Use "unsigned long" as type of physical address.
>      2. Use new function unmap_device_dma to unmap the old dma.
>      3. Some small incorrect bits order for aw shift.
>
> Changelog[v5]:
>      1. Do not disable and re-enable traslation and interrupt remapping.
>      2. Use old root entry table.
>      3. Use old interrupt remapping table.
>      4. New functions to copy data from old kernel, and save to old kernel mem.
>      5. New functions to save updated root entry table and irte table.
>      6. Use intel_unmap to unmap the old dma;
>      7. Allocate new pages while driver is being loaded.
>
> Changelog[v4]:
>      1. Cut off the patches that move some defines and functions to new files.
>      2. Reduce the numbers of patches to five, make it more easier to read.
>      3. Changed the name of functions, make them consistent with current context
>         get/set functions.
>      4. Add change to function __iommu_attach_domain.
>
> Changelog[v3]:
>      1. Commented-out "#define DEBUG 1" to eliminate debug messages.
>      2. Updated the comments about changes in each version.
>      3. Fixed: one-line added to Copy-Translations patch to initialize the iovad
>            struct as recommended by Baoquan He [bhe at redhat.com]
>            init_iova_domain(&domain->iovad, DMA_32BIT_PFN);
>
> Changelog[v2]:
>      The following series implements a fix for:
>      A kdump problem about DMA that has been discussed for a long time. That is,
>      when a kernel panics and boots into the kdump kernel, DMA started by the
>      panicked kernel is not stopped before the kdump kernel is booted and the
>      kdump kernel disables the IOMMU while this DMA continues.  This causes the
>      IOMMU to stop translating the DMA addresses as IOVAs and begin to treat
>      them as physical memory addresses -- which causes the DMA to either:
>          (1) generate DMAR errors or
>          (2) generate PCI SERR errors or
>          (3) transfer data to or from incorrect areas of memory. Often this
>              causes the dump to fail.
>
> Changelog[v1]:
>      The original version.
>
> Changed in this version:
> 1. Do not disable and re-enable traslation and interrupt remapping.
> 2. Use old root entry table.
> 3. Use old interrupt remapping table.
> 4. Use "unsigned long" as physical address.
> 5. Use intel_unmap to unmap the old dma;
>
> Baoquan He <bhe at redhat.com> helps testing this patchset.
> Takao Indoh <indou.takao at jp.fujitsu.com> gives valuable suggestions.
>
>    iommu/vt-d: Update iommu_attach_domain() and its callers
>    iommu/vt-d: Items required for kdump
>    iommu/vt-d: Add domain-id functions
>    iommu/vt-d: functions to copy data from old mem
>    iommu/vt-d: Add functions to load and save old re
>    iommu/vt-d: datatypes and functions used for kdump
>    iommu/vt-d: enable kdump support in iommu module
>    iommu/vt-d: assign new page table for dma_map
>    iommu/vt-d: Copy functions for irte
>    iommu/vt-d: Use old irte in kdump kernel
>
> Signed-off-by: Bill Sumner <billsumnerlinux at gmail.com>
> Signed-off-by: Li, Zhen-Hua <zhen-hual at hp.com>
> Signed-off-by: Takao Indoh <indou.takao at jp.fujitsu.com>
> Tested-by: Baoquan He <bhe at redhat.com>
> ---
>   drivers/iommu/intel-iommu.c         | 1054 +++++++++++++++++++++++++++++++++--
>   drivers/iommu/intel_irq_remapping.c |  104 +++-
>   include/linux/intel-iommu.h         |   18 +
>   3 files changed, 1134 insertions(+), 42 deletions(-)
>



From bhe at redhat.com  Mon Jan 12 01:07:39 2015
From: bhe at redhat.com (Baoquan He)
Date: Mon, 12 Jan 2015 17:07:39 +0800
Subject: [PATCH v8 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel
In-Reply-To: <54B37F2B.9050708@hp.com>
References: <1421046388-27925-1-git-send-email-zhen-hual@hp.com>
 <54B37F2B.9050708@hp.com>
Message-ID: <20150112090739.GA1539@dhcp-16-116.nay.redhat.com>

On 01/12/15 at 04:00pm, Li, ZhenHua wrote:
> Comparing to v7, this version adds only a few lines code:
> 
> In function copy_page_table,
> 
> +		__iommu_flush_cache(iommu, phys_to_virt(dma_pte_next),
> +				VTD_PAGE_SIZE);

So this adding fixs the reported dmar fault on Takao's system, right?

> 
> 
> On 01/12/2015 03:06 PM, Li, Zhen-Hua wrote:
> >This patchset is an update of Bill Sumner's patchset, implements a fix for:
> >If a kernel boots with intel_iommu=on on a system that supports intel vt-d,
> >when a panic happens, the kdump kernel will boot with these faults:
> >
> >     dmar: DRHD: handling fault status reg 102
> >     dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr fff80000
> >     DMAR:[fault reason 01] Present bit in root entry is clear
> >
> >     dmar: DRHD: handling fault status reg 2
> >     dmar: INTR-REMAP: Request device [[61:00.0] fault index 42
> >     INTR-REMAP:[fault reason 34] Present field in the IRTE entry is clear
> >
> >On some system, the interrupt remapping fault will also happen even if the
> >intel_iommu is not set to on, because the interrupt remapping will be enabled
> >when x2apic is needed by the system.
> >
> >The cause of the DMA fault is described in Bill's original version, and the
> >INTR-Remap fault is caused by a similar reason. In short, the initialization
> >of vt-d drivers causes the in-flight DMA and interrupt requests get wrong
> >response.
> >
> >To fix this problem, we modifies the behaviors of the intel vt-d in the
> >crashdump kernel:
> >
> >For DMA Remapping:
> >1. To accept the vt-d hardware in an active state,
> >2. Do not disable and re-enable the translation, keep it enabled.
> >3. Use the old root entry table, do not rewrite the RTA register.
> >4. Malloc and use new context entry table and page table, copy data from the
> >    old ones that used by the old kernel.
> >5. to use different portions of the iova address ranges for the device drivers
> >    in the crashdump kernel than the iova ranges that were in-use at the time
> >    of the panic.
> >6. After device driver is loaded, when it issues the first dma_map command,
> >    free the dmar_domain structure for this device, and generate a new one, so
> >    that the device can be assigned a new and empty page table.
> >7. When a new context entry table is generated, we also save its address to
> >    the old root entry table.
> >
> >For Interrupt Remapping:
> >1. To accept the vt-d hardware in an active state,
> >2. Do not disable and re-enable the interrupt remapping, keep it enabled.
> >3. Use the old interrupt remapping table, do not rewrite the IRTA register.
> >4. When ioapic entry is setup, the interrupt remapping table is changed, and
> >    the updated data will be stored to the old interrupt remapping table.
> >
> >Advantages of this approach:
> >1. All manipulation of the IO-device is done by the Linux device-driver
> >    for that device.
> >2. This approach behaves in a manner very similar to operation without an
> >    active iommu.
> >3. Any activity between the IO-device and its RMRR areas is handled by the
> >    device-driver in the same manner as during a non-kdump boot.
> >4. If an IO-device has no driver in the kdump kernel, it is simply left alone.
> >    This supports the practice of creating a special kdump kernel without
> >    drivers for any devices that are not required for taking a crashdump.
> >5. Minimal code-changes among the existing mainline intel vt-d code.
> >
> >Summary of changes in this patch set:
> >1. Added some useful function for root entry table in code intel-iommu.c
> >2. Added new members to struct root_entry and struct irte;
> >3. Functions to load old root entry table to iommu->root_entry from the memory
> >    of old kernel.
> >4. Functions to malloc new context entry table and page table and copy the data
> >    from the old ones to the malloced new ones.
> >5. Functions to enable support for DMA remapping in kdump kernel.
> >6. Functions to load old irte data from the old kernel to the kdump kernel.
> >7. Some code changes that support other behaviours that have been listed.
> >8. In the new functions, use physical address as "unsigned long" type, not
> >    pointers.
> >
> >Original version by Bill Sumner:
> >     https://lkml.org/lkml/2014/1/10/518
> >     https://lkml.org/lkml/2014/4/15/716
> >     https://lkml.org/lkml/2014/4/24/836
> >
> >Zhenhua's updates:
> >     https://lkml.org/lkml/2014/10/21/134
> >     https://lkml.org/lkml/2014/12/15/121
> >     https://lkml.org/lkml/2014/12/22/53
> >     https://lkml.org/lkml/2015/1/6/1166
> >
> >Changelog[v8]:
> >     1. Add a missing __iommu_flush_cache in function copy_page_table.
> >
> >Changelog[v7]:
> >     1. Use __iommu_flush_cache to flush the data to hardware.
> >
> >Changelog[v6]:
> >     1. Use "unsigned long" as type of physical address.
> >     2. Use new function unmap_device_dma to unmap the old dma.
> >     3. Some small incorrect bits order for aw shift.
> >
> >Changelog[v5]:
> >     1. Do not disable and re-enable traslation and interrupt remapping.
> >     2. Use old root entry table.
> >     3. Use old interrupt remapping table.
> >     4. New functions to copy data from old kernel, and save to old kernel mem.
> >     5. New functions to save updated root entry table and irte table.
> >     6. Use intel_unmap to unmap the old dma;
> >     7. Allocate new pages while driver is being loaded.
> >
> >Changelog[v4]:
> >     1. Cut off the patches that move some defines and functions to new files.
> >     2. Reduce the numbers of patches to five, make it more easier to read.
> >     3. Changed the name of functions, make them consistent with current context
> >        get/set functions.
> >     4. Add change to function __iommu_attach_domain.
> >
> >Changelog[v3]:
> >     1. Commented-out "#define DEBUG 1" to eliminate debug messages.
> >     2. Updated the comments about changes in each version.
> >     3. Fixed: one-line added to Copy-Translations patch to initialize the iovad
> >           struct as recommended by Baoquan He [bhe at redhat.com]
> >           init_iova_domain(&domain->iovad, DMA_32BIT_PFN);
> >
> >Changelog[v2]:
> >     The following series implements a fix for:
> >     A kdump problem about DMA that has been discussed for a long time. That is,
> >     when a kernel panics and boots into the kdump kernel, DMA started by the
> >     panicked kernel is not stopped before the kdump kernel is booted and the
> >     kdump kernel disables the IOMMU while this DMA continues.  This causes the
> >     IOMMU to stop translating the DMA addresses as IOVAs and begin to treat
> >     them as physical memory addresses -- which causes the DMA to either:
> >         (1) generate DMAR errors or
> >         (2) generate PCI SERR errors or
> >         (3) transfer data to or from incorrect areas of memory. Often this
> >             causes the dump to fail.
> >
> >Changelog[v1]:
> >     The original version.
> >
> >Changed in this version:
> >1. Do not disable and re-enable traslation and interrupt remapping.
> >2. Use old root entry table.
> >3. Use old interrupt remapping table.
> >4. Use "unsigned long" as physical address.
> >5. Use intel_unmap to unmap the old dma;
> >
> >Baoquan He <bhe at redhat.com> helps testing this patchset.
> >Takao Indoh <indou.takao at jp.fujitsu.com> gives valuable suggestions.
> >
> >   iommu/vt-d: Update iommu_attach_domain() and its callers
> >   iommu/vt-d: Items required for kdump
> >   iommu/vt-d: Add domain-id functions
> >   iommu/vt-d: functions to copy data from old mem
> >   iommu/vt-d: Add functions to load and save old re
> >   iommu/vt-d: datatypes and functions used for kdump
> >   iommu/vt-d: enable kdump support in iommu module
> >   iommu/vt-d: assign new page table for dma_map
> >   iommu/vt-d: Copy functions for irte
> >   iommu/vt-d: Use old irte in kdump kernel
> >
> >Signed-off-by: Bill Sumner <billsumnerlinux at gmail.com>
> >Signed-off-by: Li, Zhen-Hua <zhen-hual at hp.com>
> >Signed-off-by: Takao Indoh <indou.takao at jp.fujitsu.com>
> >Tested-by: Baoquan He <bhe at redhat.com>
> >---
> >  drivers/iommu/intel-iommu.c         | 1054 +++++++++++++++++++++++++++++++++--
> >  drivers/iommu/intel_irq_remapping.c |  104 +++-
> >  include/linux/intel-iommu.h         |   18 +
> >  3 files changed, 1134 insertions(+), 42 deletions(-)
> >
> 


From zhen-hual at hp.com  Mon Jan 12 01:28:18 2015
From: zhen-hual at hp.com (Li, ZhenHua)
Date: Mon, 12 Jan 2015 17:28:18 +0800
Subject: [PATCH v8 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel
In-Reply-To: <20150112090739.GA1539@dhcp-16-116.nay.redhat.com>
References: <1421046388-27925-1-git-send-email-zhen-hual@hp.com>
 <54B37F2B.9050708@hp.com> <20150112090739.GA1539@dhcp-16-116.nay.redhat.com>
Message-ID: <54B393B2.9090103@hp.com>

On 01/12/2015 05:07 PM, Baoquan He wrote:
> On 01/12/15 at 04:00pm, Li, ZhenHua wrote:
>> Comparing to v7, this version adds only a few lines code:
>>
>> In function copy_page_table,
>>
>> +		__iommu_flush_cache(iommu, phys_to_virt(dma_pte_next),
>> +				VTD_PAGE_SIZE);
>
> So this adding fixs the reported dmar fault on Takao's system, right?
I am not sure whether it can fix the dmar fault on Takao's system, but
I hope it can fix.

>
>>
>>
>> On 01/12/2015 03:06 PM, Li, Zhen-Hua wrote:
>>> This patchset is an update of Bill Sumner's patchset, implements a fix for:
>>> If a kernel boots with intel_iommu=on on a system that supports intel vt-d,
>>> when a panic happens, the kdump kernel will boot with these faults:
>>>
>>>      dmar: DRHD: handling fault status reg 102
>>>      dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr fff80000
>>>      DMAR:[fault reason 01] Present bit in root entry is clear
>>>
>>>      dmar: DRHD: handling fault status reg 2
>>>      dmar: INTR-REMAP: Request device [[61:00.0] fault index 42
>>>      INTR-REMAP:[fault reason 34] Present field in the IRTE entry is clear
>>>
>>> On some system, the interrupt remapping fault will also happen even if the
>>> intel_iommu is not set to on, because the interrupt remapping will be enabled
>>> when x2apic is needed by the system.
>>>
>>> The cause of the DMA fault is described in Bill's original version, and the
>>> INTR-Remap fault is caused by a similar reason. In short, the initialization
>>> of vt-d drivers causes the in-flight DMA and interrupt requests get wrong
>>> response.
>>>
>>> To fix this problem, we modifies the behaviors of the intel vt-d in the
>>> crashdump kernel:
>>>
>>> For DMA Remapping:
>>> 1. To accept the vt-d hardware in an active state,
>>> 2. Do not disable and re-enable the translation, keep it enabled.
>>> 3. Use the old root entry table, do not rewrite the RTA register.
>>> 4. Malloc and use new context entry table and page table, copy data from the
>>>     old ones that used by the old kernel.
>>> 5. to use different portions of the iova address ranges for the device drivers
>>>     in the crashdump kernel than the iova ranges that were in-use at the time
>>>     of the panic.
>>> 6. After device driver is loaded, when it issues the first dma_map command,
>>>     free the dmar_domain structure for this device, and generate a new one, so
>>>     that the device can be assigned a new and empty page table.
>>> 7. When a new context entry table is generated, we also save its address to
>>>     the old root entry table.
>>>
>>> For Interrupt Remapping:
>>> 1. To accept the vt-d hardware in an active state,
>>> 2. Do not disable and re-enable the interrupt remapping, keep it enabled.
>>> 3. Use the old interrupt remapping table, do not rewrite the IRTA register.
>>> 4. When ioapic entry is setup, the interrupt remapping table is changed, and
>>>     the updated data will be stored to the old interrupt remapping table.
>>>
>>> Advantages of this approach:
>>> 1. All manipulation of the IO-device is done by the Linux device-driver
>>>     for that device.
>>> 2. This approach behaves in a manner very similar to operation without an
>>>     active iommu.
>>> 3. Any activity between the IO-device and its RMRR areas is handled by the
>>>     device-driver in the same manner as during a non-kdump boot.
>>> 4. If an IO-device has no driver in the kdump kernel, it is simply left alone.
>>>     This supports the practice of creating a special kdump kernel without
>>>     drivers for any devices that are not required for taking a crashdump.
>>> 5. Minimal code-changes among the existing mainline intel vt-d code.
>>>
>>> Summary of changes in this patch set:
>>> 1. Added some useful function for root entry table in code intel-iommu.c
>>> 2. Added new members to struct root_entry and struct irte;
>>> 3. Functions to load old root entry table to iommu->root_entry from the memory
>>>     of old kernel.
>>> 4. Functions to malloc new context entry table and page table and copy the data
>>>     from the old ones to the malloced new ones.
>>> 5. Functions to enable support for DMA remapping in kdump kernel.
>>> 6. Functions to load old irte data from the old kernel to the kdump kernel.
>>> 7. Some code changes that support other behaviours that have been listed.
>>> 8. In the new functions, use physical address as "unsigned long" type, not
>>>     pointers.
>>>
>>> Original version by Bill Sumner:
>>>      https://lkml.org/lkml/2014/1/10/518
>>>      https://lkml.org/lkml/2014/4/15/716
>>>      https://lkml.org/lkml/2014/4/24/836
>>>
>>> Zhenhua's updates:
>>>      https://lkml.org/lkml/2014/10/21/134
>>>      https://lkml.org/lkml/2014/12/15/121
>>>      https://lkml.org/lkml/2014/12/22/53
>>>      https://lkml.org/lkml/2015/1/6/1166
>>>
>>> Changelog[v8]:
>>>      1. Add a missing __iommu_flush_cache in function copy_page_table.
>>>
>>> Changelog[v7]:
>>>      1. Use __iommu_flush_cache to flush the data to hardware.
>>>
>>> Changelog[v6]:
>>>      1. Use "unsigned long" as type of physical address.
>>>      2. Use new function unmap_device_dma to unmap the old dma.
>>>      3. Some small incorrect bits order for aw shift.
>>>
>>> Changelog[v5]:
>>>      1. Do not disable and re-enable traslation and interrupt remapping.
>>>      2. Use old root entry table.
>>>      3. Use old interrupt remapping table.
>>>      4. New functions to copy data from old kernel, and save to old kernel mem.
>>>      5. New functions to save updated root entry table and irte table.
>>>      6. Use intel_unmap to unmap the old dma;
>>>      7. Allocate new pages while driver is being loaded.
>>>
>>> Changelog[v4]:
>>>      1. Cut off the patches that move some defines and functions to new files.
>>>      2. Reduce the numbers of patches to five, make it more easier to read.
>>>      3. Changed the name of functions, make them consistent with current context
>>>         get/set functions.
>>>      4. Add change to function __iommu_attach_domain.
>>>
>>> Changelog[v3]:
>>>      1. Commented-out "#define DEBUG 1" to eliminate debug messages.
>>>      2. Updated the comments about changes in each version.
>>>      3. Fixed: one-line added to Copy-Translations patch to initialize the iovad
>>>            struct as recommended by Baoquan He [bhe at redhat.com]
>>>            init_iova_domain(&domain->iovad, DMA_32BIT_PFN);
>>>
>>> Changelog[v2]:
>>>      The following series implements a fix for:
>>>      A kdump problem about DMA that has been discussed for a long time. That is,
>>>      when a kernel panics and boots into the kdump kernel, DMA started by the
>>>      panicked kernel is not stopped before the kdump kernel is booted and the
>>>      kdump kernel disables the IOMMU while this DMA continues.  This causes the
>>>      IOMMU to stop translating the DMA addresses as IOVAs and begin to treat
>>>      them as physical memory addresses -- which causes the DMA to either:
>>>          (1) generate DMAR errors or
>>>          (2) generate PCI SERR errors or
>>>          (3) transfer data to or from incorrect areas of memory. Often this
>>>              causes the dump to fail.
>>>
>>> Changelog[v1]:
>>>      The original version.
>>>
>>> Changed in this version:
>>> 1. Do not disable and re-enable traslation and interrupt remapping.
>>> 2. Use old root entry table.
>>> 3. Use old interrupt remapping table.
>>> 4. Use "unsigned long" as physical address.
>>> 5. Use intel_unmap to unmap the old dma;
>>>
>>> Baoquan He <bhe at redhat.com> helps testing this patchset.
>>> Takao Indoh <indou.takao at jp.fujitsu.com> gives valuable suggestions.
>>>
>>>    iommu/vt-d: Update iommu_attach_domain() and its callers
>>>    iommu/vt-d: Items required for kdump
>>>    iommu/vt-d: Add domain-id functions
>>>    iommu/vt-d: functions to copy data from old mem
>>>    iommu/vt-d: Add functions to load and save old re
>>>    iommu/vt-d: datatypes and functions used for kdump
>>>    iommu/vt-d: enable kdump support in iommu module
>>>    iommu/vt-d: assign new page table for dma_map
>>>    iommu/vt-d: Copy functions for irte
>>>    iommu/vt-d: Use old irte in kdump kernel
>>>
>>> Signed-off-by: Bill Sumner <billsumnerlinux at gmail.com>
>>> Signed-off-by: Li, Zhen-Hua <zhen-hual at hp.com>
>>> Signed-off-by: Takao Indoh <indou.takao at jp.fujitsu.com>
>>> Tested-by: Baoquan He <bhe at redhat.com>
>>> ---
>>>   drivers/iommu/intel-iommu.c         | 1054 +++++++++++++++++++++++++++++++++--
>>>   drivers/iommu/intel_irq_remapping.c |  104 +++-
>>>   include/linux/intel-iommu.h         |   18 +
>>>   3 files changed, 1134 insertions(+), 42 deletions(-)
>>>
>>



From joro at 8bytes.org  Mon Jan 12 07:18:55 2015
From: joro at 8bytes.org (Joerg Roedel)
Date: Mon, 12 Jan 2015 16:18:55 +0100
Subject: [PATCH v8 01/10] iommu/vt-d: Update iommu_attach_domain() and
 its callers
In-Reply-To: <1421046388-27925-2-git-send-email-zhen-hual@hp.com>
References: <1421046388-27925-1-git-send-email-zhen-hual@hp.com>
 <1421046388-27925-2-git-send-email-zhen-hual@hp.com>
Message-ID: <20150112151855.GB6343@8bytes.org>

On Mon, Jan 12, 2015 at 03:06:19PM +0800, Li, Zhen-Hua wrote:
> Allow specification of the domain-id for the new domain.
> This patch only adds the 'did' parameter to iommu_attach_domain()
> and modifies all of its callers to specify the default value of -1
> which says "no did specified, allocate a new one".

I think its better to keep the old iommu_attach_domain() interface in
place and introduce a new function (like iommu_attach_domain_with_id()
or something) which has the additional parameter. Then you can rewrite
iommu_attach_domain():

	iommu_attach_domai(...)
	{
		return iommu_attach_domain_with_id(..., -1);
	}

This way you don't have to update all the callers of
iommu_attach_domain() and the interface is more readable.


	Joerg



From joro at 8bytes.org  Mon Jan 12 07:22:08 2015
From: joro at 8bytes.org (Joerg Roedel)
Date: Mon, 12 Jan 2015 16:22:08 +0100
Subject: [PATCH v8 02/10] iommu/vt-d: Items required for kdump
In-Reply-To: <1421046388-27925-3-git-send-email-zhen-hual@hp.com>
References: <1421046388-27925-1-git-send-email-zhen-hual@hp.com>
 <1421046388-27925-3-git-send-email-zhen-hual@hp.com>
Message-ID: <20150112152207.GC6343@8bytes.org>

On Mon, Jan 12, 2015 at 03:06:20PM +0800, Li, Zhen-Hua wrote:
> +
> +#ifdef CONFIG_CRASH_DUMP
> +
> +/*
> + * Fix Crashdump failure caused by leftover DMA through a hardware IOMMU
> + *
> + * Fixes the crashdump kernel to deal with an active iommu and legacy
> + * DMA from the (old) panicked kernel in a manner similar to how legacy
> + * DMA is handled when no hardware iommu was in use by the old kernel --
> + * allow the legacy DMA to continue into its current buffers.
> + *
> + * In the crashdump kernel, this code:
> + * 1. skips disabling the IOMMU's translating of IO Virtual Addresses (IOVA).
> + * 2. Do not re-enable IOMMU's translating.
> + * 3. In kdump kernel, use the old root entry table.
> + * 4. Leaves the current translations in-place so that legacy DMA will
> + *    continue to use its current buffers.
> + * 5. Allocates to the device drivers in the crashdump kernel
> + *    portions of the iova address ranges that are different
> + *    from the iova address ranges that were being used by the old kernel
> + *    at the time of the panic.
> + *
> + */

It looks like you are still copying the io-page-tables from the old
kernel into the kdump kernel, is that right? With the approach that was
proposed you only need to copy over the context entries 1-1. They are
still pointing to the page-tables in the old kernels memory (which is
just fine).

The root-entry of the old kernel is also re-used, and when the kdump
kernel starts to use a device, its context entry is updated to point to
a newly allocated page-table.


	Joerg



From joro at 8bytes.org  Mon Jan 12 08:06:46 2015
From: joro at 8bytes.org (Joerg Roedel)
Date: Mon, 12 Jan 2015 17:06:46 +0100
Subject: [PATCH v8 02/10] iommu/vt-d: Items required for kdump
In-Reply-To: <20150112152919.GA16162@redhat.com>
References: <1421046388-27925-1-git-send-email-zhen-hual@hp.com>
 <1421046388-27925-3-git-send-email-zhen-hual@hp.com>
 <20150112152207.GC6343@8bytes.org>
 <20150112152919.GA16162@redhat.com>
Message-ID: <20150112160646.GF6343@8bytes.org>

On Mon, Jan 12, 2015 at 10:29:19AM -0500, Vivek Goyal wrote:
> Kdump has the notion of backup region. Where certain parts of old kernels
> memory can be moved to a different location (first 640K on x86 as of now)
> and new kernel can make use of this memory now.
> 
> So we will have to just make sure that no parts of this old page table
> fall into backup region.

Uuh, looks like the 'iommu-with-kdump-issue' isn't complicated enough
yet ;)
Sadly, your above statement is true for all hardware-accessible data
structures in IOMMU code. I think about how we can solve this, is there
an easy way to allocate memory that is not in any backup region?

Thanks,

	Joerg



From joro at 8bytes.org  Mon Jan 12 08:48:03 2015
From: joro at 8bytes.org (Joerg Roedel)
Date: Mon, 12 Jan 2015 17:48:03 +0100
Subject: [PATCH v8 02/10] iommu/vt-d: Items required for kdump
In-Reply-To: <20150112161538.GB16162@redhat.com>
References: <1421046388-27925-1-git-send-email-zhen-hual@hp.com>
 <1421046388-27925-3-git-send-email-zhen-hual@hp.com>
 <20150112152207.GC6343@8bytes.org>
 <20150112152919.GA16162@redhat.com>
 <20150112160646.GF6343@8bytes.org>
 <20150112161538.GB16162@redhat.com>
Message-ID: <20150112164803.GG6343@8bytes.org>

On Mon, Jan 12, 2015 at 11:15:38AM -0500, Vivek Goyal wrote:
> On Mon, Jan 12, 2015 at 05:06:46PM +0100, Joerg Roedel wrote:
> > On Mon, Jan 12, 2015 at 10:29:19AM -0500, Vivek Goyal wrote:
> > > Kdump has the notion of backup region. Where certain parts of old kernels
> > > memory can be moved to a different location (first 640K on x86 as of now)
> > > and new kernel can make use of this memory now.
> > > 
> > > So we will have to just make sure that no parts of this old page table
> > > fall into backup region.
> > 
> > Uuh, looks like the 'iommu-with-kdump-issue' isn't complicated enough
> > yet ;)
> > Sadly, your above statement is true for all hardware-accessible data
> > structures in IOMMU code. I think about how we can solve this, is there
> > an easy way to allocate memory that is not in any backup region?
> 
> Hmm..., there does not seem to be any easy way to do this. In fact, as of
> now, kernel does not even know where is backup region. All these details are
> managed by user space completely (except for new kexec_file_load() syscall).
> 
> That means we are left with ugly options now.
> 
> - Define per arch kexec backup regions in kernel and export it to user
>   space and let kexec-tools make use of that deinition (instead of
>   defining its own). That way memory allocation code in kernel can look
>   at this backup area and skip it for certain allocations.

Yes, that makes sense. In fact, I think all allocations for DMA memory
need to take this into account to avoid potentially serious data
corruption.
If any memory for a disk superblock gets allocated in backup memory and
a kdump happens, the new kernel might zero out that area and the disk
controler then writes the zeroes to disk instead of the superblock.


	Joerg



From vgoyal at redhat.com  Mon Jan 12 07:29:19 2015
From: vgoyal at redhat.com (Vivek Goyal)
Date: Mon, 12 Jan 2015 10:29:19 -0500
Subject: [PATCH v8 02/10] iommu/vt-d: Items required for kdump
In-Reply-To: <20150112152207.GC6343@8bytes.org>
References: <1421046388-27925-1-git-send-email-zhen-hual@hp.com>
 <1421046388-27925-3-git-send-email-zhen-hual@hp.com>
 <20150112152207.GC6343@8bytes.org>
Message-ID: <20150112152919.GA16162@redhat.com>

On Mon, Jan 12, 2015 at 04:22:08PM +0100, Joerg Roedel wrote:
> On Mon, Jan 12, 2015 at 03:06:20PM +0800, Li, Zhen-Hua wrote:
> > +
> > +#ifdef CONFIG_CRASH_DUMP
> > +
> > +/*
> > + * Fix Crashdump failure caused by leftover DMA through a hardware IOMMU
> > + *
> > + * Fixes the crashdump kernel to deal with an active iommu and legacy
> > + * DMA from the (old) panicked kernel in a manner similar to how legacy
> > + * DMA is handled when no hardware iommu was in use by the old kernel --
> > + * allow the legacy DMA to continue into its current buffers.
> > + *
> > + * In the crashdump kernel, this code:
> > + * 1. skips disabling the IOMMU's translating of IO Virtual Addresses (IOVA).
> > + * 2. Do not re-enable IOMMU's translating.
> > + * 3. In kdump kernel, use the old root entry table.
> > + * 4. Leaves the current translations in-place so that legacy DMA will
> > + *    continue to use its current buffers.
> > + * 5. Allocates to the device drivers in the crashdump kernel
> > + *    portions of the iova address ranges that are different
> > + *    from the iova address ranges that were being used by the old kernel
> > + *    at the time of the panic.
> > + *
> > + */
> 
> It looks like you are still copying the io-page-tables from the old
> kernel into the kdump kernel, is that right? With the approach that was
> proposed you only need to copy over the context entries 1-1. They are
> still pointing to the page-tables in the old kernels memory (which is
> just fine).

Kdump has the notion of backup region. Where certain parts of old kernels
memory can be moved to a different location (first 640K on x86 as of now)
and new kernel can make use of this memory now.

So we will have to just make sure that no parts of this old page table
fall into backup region.

Thanks
Vivek


From vgoyal at redhat.com  Mon Jan 12 08:15:38 2015
From: vgoyal at redhat.com (Vivek Goyal)
Date: Mon, 12 Jan 2015 11:15:38 -0500
Subject: [PATCH v8 02/10] iommu/vt-d: Items required for kdump
In-Reply-To: <20150112160646.GF6343@8bytes.org>
References: <1421046388-27925-1-git-send-email-zhen-hual@hp.com>
 <1421046388-27925-3-git-send-email-zhen-hual@hp.com>
 <20150112152207.GC6343@8bytes.org>
 <20150112152919.GA16162@redhat.com>
 <20150112160646.GF6343@8bytes.org>
Message-ID: <20150112161538.GB16162@redhat.com>

On Mon, Jan 12, 2015 at 05:06:46PM +0100, Joerg Roedel wrote:
> On Mon, Jan 12, 2015 at 10:29:19AM -0500, Vivek Goyal wrote:
> > Kdump has the notion of backup region. Where certain parts of old kernels
> > memory can be moved to a different location (first 640K on x86 as of now)
> > and new kernel can make use of this memory now.
> > 
> > So we will have to just make sure that no parts of this old page table
> > fall into backup region.
> 
> Uuh, looks like the 'iommu-with-kdump-issue' isn't complicated enough
> yet ;)
> Sadly, your above statement is true for all hardware-accessible data
> structures in IOMMU code. I think about how we can solve this, is there
> an easy way to allocate memory that is not in any backup region?

Hmm..., there does not seem to be any easy way to do this. In fact, as of
now, kernel does not even know where is backup region. All these details are
managed by user space completely (except for new kexec_file_load() syscall).

That means we are left with ugly options now.

- Define per arch kexec backup regions in kernel and export it to user
  space and let kexec-tools make use of that deinition (instead of
  defining its own). That way memory allocation code in kernel can look
  at this backup area and skip it for certain allocations.

Thanks
Vivek


From anurup.m at huawei.com  Tue Jan 13 03:23:38 2015
From: anurup.m at huawei.com (Anurup M)
Date: Tue, 13 Jan 2015 16:53:38 +0530
Subject: Kdump on ARM64
In-Reply-To: <54AE3B7A.4070301@linaro.org>
References: <54AE2544.2010801@huawei.com> <54AE3B7A.4070301@linaro.org>
Message-ID: <54B5003A.1050508@huawei.com>


Hi Akashi,

	Thanks for the reply.

Hi  Akashi, Geoff,

	In the code I could find assert(arm64_mem.memstart) in the function virt_to_phys. This address can be 0x0 when the
  segment start address is 0x0. so can we use this assert?

	The code flow is elf_arm64_load->load_crashdump_segments->add_buffer_phys_virt->add_segment_phys_virt->virt_to_phys
  and this call is also before the call to load_elf_exec_in_crashmem which init's arm64_mem.memstart to crash_reserved_mem.start.
  So arm64_mem.memstart is also not set before the call to virt_to_phys.

	Please share your comments.

Thanks,
Anurup


On 1/8/2015 1:40 PM, AKASHI Takahiro wrote:
> Hi Anurup,
> 
> On 01/08/2015 03:35 PM, Anurup M wrote:
>> Hi Akashi, Geoff,
>>
>>     I tried to check kdump on arm64 on Foundation model. The/proc/vmcore is generated.
>> But I couldnot check if it is proper. Is there a makedumpfile binary for arm64? and also crash tool.
>> I used the master branch of https://git.linaro.org/people/geoff.levand/linux-kexec.git to verify.
>>       Please provide some information about the current status or support for kdump on arm64?
>> and the ways to verify or start development on Foundation model.
> 
> The story is that I used to work on kdump, but handed it off
> to Geoff at some time. Since then, he merged it into his repo
> but I don't think that he has tested (verified) it yet.
> So please take it as a study work (or prototype), although I'm
> pretty sure that it works well on model except some issues.
> 
> Regarding /proc/vmcore, if you can retrieve it to your host machine,
> you can simply verify the contents as follows:
>   $ aarch64-linux-gnu-gdb vmlinux core(=vmcore)
>   (gdb) info threads
>   (gdb) bt
>    ...
> 
> I also have a trivial patch for crash tool to cross-build it, but
> arm64 support (by RedHat folk) is already there.
> 
> Thanks,
> -Takahiro AKASHI
> 
>> Thanks,
>> Anurup M
>>
> 


From zhen-hual at hp.com  Tue Jan 13 00:12:29 2015
From: zhen-hual at hp.com (Li, ZhenHua)
Date: Tue, 13 Jan 2015 16:12:29 +0800
Subject: [PATCH v8 02/10] iommu/vt-d: Items required for kdump
In-Reply-To: <20150112152207.GC6343@8bytes.org>
References: <1421046388-27925-1-git-send-email-zhen-hual@hp.com>
 <1421046388-27925-3-git-send-email-zhen-hual@hp.com>
 <20150112152207.GC6343@8bytes.org>
Message-ID: <54B4D36D.8030204@hp.com>

On 01/12/2015 11:22 PM, Joerg Roedel wrote:
> On Mon, Jan 12, 2015 at 03:06:20PM +0800, Li, Zhen-Hua wrote:
>> +
>> +#ifdef CONFIG_CRASH_DUMP
>> +
>> +/*
>> + * Fix Crashdump failure caused by leftover DMA through a hardware IOMMU
>> + *
>> + * Fixes the crashdump kernel to deal with an active iommu and legacy
>> + * DMA from the (old) panicked kernel in a manner similar to how legacy
>> + * DMA is handled when no hardware iommu was in use by the old kernel --
>> + * allow the legacy DMA to continue into its current buffers.
>> + *
>> + * In the crashdump kernel, this code:
>> + * 1. skips disabling the IOMMU's translating of IO Virtual Addresses (IOVA).
>> + * 2. Do not re-enable IOMMU's translating.
>> + * 3. In kdump kernel, use the old root entry table.
>> + * 4. Leaves the current translations in-place so that legacy DMA will
>> + *    continue to use its current buffers.
>> + * 5. Allocates to the device drivers in the crashdump kernel
>> + *    portions of the iova address ranges that are different
>> + *    from the iova address ranges that were being used by the old kernel
>> + *    at the time of the panic.
>> + *
>> + */
>
> It looks like you are still copying the io-page-tables from the old
> kernel into the kdump kernel, is that right? With the approach that was
> proposed you only need to copy over the context entries 1-1. They are
> still pointing to the page-tables in the old kernels memory (which is
> just fine).
>
> The root-entry of the old kernel is also re-used, and when the kdump
> kernel starts to use a device, its context entry is updated to point to
> a newly allocated page-table.
>

Current status:
1. Use old root entry table,
2. Copy old context entry tables.
3. Copy old page tables.
4. Allocate new page table when device driver is loading.

I remember the progress we discussed was this. Checking the old mails,
founding we did not clearly discuss about the old page tables, only
agrred with allocating new page tables when device driver is loading.

To me, both of these two options are fine, copying old page tables, or
re-use the old ones. My debugging shows both of them works fine.

If we use the old page tables, the patchset will use less code.
Functions copy_page_addr, copy_page_table are no longer needed, and
functions copy_context_entry and copy_context_entry_table will be
reduced to several lines. The patchset will be much simpler.

Zhenhua




From bhe at redhat.com  Tue Jan 13 03:41:08 2015
From: bhe at redhat.com (Baoquan He)
Date: Tue, 13 Jan 2015 19:41:08 +0800
Subject: [PATCH v8 02/10] iommu/vt-d: Items required for kdump
In-Reply-To: <20150112152919.GA16162@redhat.com>
References: <1421046388-27925-1-git-send-email-zhen-hual@hp.com>
 <1421046388-27925-3-git-send-email-zhen-hual@hp.com>
 <20150112152207.GC6343@8bytes.org>
 <20150112152919.GA16162@redhat.com>
Message-ID: <20150113114108.GB24543@dhcp-16-105.nay.redhat.com>

On 01/12/15 at 10:29am, Vivek Goyal wrote:
> On Mon, Jan 12, 2015 at 04:22:08PM +0100, Joerg Roedel wrote:
> > It looks like you are still copying the io-page-tables from the old
> > kernel into the kdump kernel, is that right? With the approach that was
> > proposed you only need to copy over the context entries 1-1. They are
> > still pointing to the page-tables in the old kernels memory (which is
> > just fine).
> 
> Kdump has the notion of backup region. Where certain parts of old kernels
> memory can be moved to a different location (first 640K on x86 as of now)
> and new kernel can make use of this memory now.

Hi Vivek,

About backup region I am a bit confusing. Just say x86, we usually copy
it to a backup region. And this first 640K will be used as a usable
memory region in 2nd kernel since its content has been copied to backup
region. And that backup region is taken from crashkernel reserved
memory and not passed to 2nd kernel as usable memory region.

Here did you mean the old page table could fall into first 640K memory
region or that reserved backup region?  Since as I understand the backup
region is taken from crashkernel memory which is not used by 1st kernel
process.

Thanks
Baoquan

> 
> So we will have to just make sure that no parts of this old page table
> fall into backup region.
> 
> Thanks
> Vivek


From takahiro.akashi at linaro.org  Tue Jan 13 18:10:56 2015
From: takahiro.akashi at linaro.org (AKASHI Takahiro)
Date: Wed, 14 Jan 2015 11:10:56 +0900
Subject: Kdump on ARM64
In-Reply-To: <54B5003A.1050508@huawei.com>
References: <54AE2544.2010801@huawei.com> <54AE3B7A.4070301@linaro.org>
 <54B5003A.1050508@huawei.com>
Message-ID: <54B5D030.20809@linaro.org>

On 01/13/2015 08:23 PM, Anurup M wrote:
>
> Hi Akashi,
>
> 	Thanks for the reply.
>
> Hi  Akashi, Geoff,
>
> 	In the code I could find assert(arm64_mem.memstart) in the function virt_to_phys. This address can be 0x0 when the
>    segment start address is 0x0. so can we use this assert?

Probably you're right.

> 	The code flow is elf_arm64_load->load_crashdump_segments->add_buffer_phys_virt->add_segment_phys_virt->virt_to_phys
>    and this call is also before the call to load_elf_exec_in_crashmem which init's arm64_mem.memstart to crash_reserved_mem.start.
>    So arm64_mem.memstart is also not set before the call to virt_to_phys.

I need refresh my memory about the flow, but in this particular case,
the last argument, phys, of add_buffer_phys_virt() is 0 and virt_to_phys()
will not be called.

Do you have any other evidence that the assertion above is broken?

Thanks,
-Takahiro AKASHI


> 	Please share your comments.
>
> Thanks,
> Anurup
>
>
> On 1/8/2015 1:40 PM, AKASHI Takahiro wrote:
>> Hi Anurup,
>>
>> On 01/08/2015 03:35 PM, Anurup M wrote:
>>> Hi Akashi, Geoff,
>>>
>>>      I tried to check kdump on arm64 on Foundation model. The/proc/vmcore is generated.
>>> But I couldnot check if it is proper. Is there a makedumpfile binary for arm64? and also crash tool.
>>> I used the master branch of https://git.linaro.org/people/geoff.levand/linux-kexec.git to verify.
>>>        Please provide some information about the current status or support for kdump on arm64?
>>> and the ways to verify or start development on Foundation model.
>>
>> The story is that I used to work on kdump, but handed it off
>> to Geoff at some time. Since then, he merged it into his repo
>> but I don't think that he has tested (verified) it yet.
>> So please take it as a study work (or prototype), although I'm
>> pretty sure that it works well on model except some issues.
>>
>> Regarding /proc/vmcore, if you can retrieve it to your host machine,
>> you can simply verify the contents as follows:
>>    $ aarch64-linux-gnu-gdb vmlinux core(=vmcore)
>>    (gdb) info threads
>>    (gdb) bt
>>     ...
>>
>> I also have a trivial patch for crash tool to cross-build it, but
>> arm64 support (by RedHat folk) is already there.
>>
>> Thanks,
>> -Takahiro AKASHI
>>
>>> Thanks,
>>> Anurup M
>>>
>>


From bhe at redhat.com  Tue Jan 13 21:31:57 2015
From: bhe at redhat.com (Baoquan He)
Date: Wed, 14 Jan 2015 13:31:57 +0800
Subject: [PATCH] kexec/x86: fix crashkernel reserved mem detection
In-Reply-To: <20150109085927.GA8288@dhcp-16-105.nay.redhat.com>
References: <1419514701-9397-1-git-send-email-wolchkov.yuri@gmail.com>
 <54AF74B7.4000807@suse.de>
 <20150109085927.GA8288@dhcp-16-105.nay.redhat.com>
Message-ID: <20150114053157.GC24543@dhcp-16-105.nay.redhat.com>

On 01/09/15 at 04:59pm, Baoquan He wrote:
> On 01/08/15 at 10:27pm, Tony Jones wrote:
> > On 12/25/2014 05:38 AM, Yuri Volchkov wrote:
> > > Function crashkernel_mem_callback had wrong parameters type
> > > So crash_reserved_mem got completely insane values
> > > As a result kexec was unable to allocate segments for crashkernel
> > > For instance, I got this message:
> > > 	"Could not find a free area of memory of 0x9f000 bytes..."
> > > 
> > > Signed-off-by: Yuri Volchkov <wolchkov.yuri at gmail.com>
> > 
> > See the patch attached to https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=771671
> > which also fixes kexec/arch/sh/crashdump-sh.c and kexec/arch/sh/kexec-sh.c
> > 
> > Definitely a problem.  Without the x86 fix we seem same failure on i586
> 
> Hi Tony,
> 
> commit 4362bfa make that change, but skip the crashkernel_mem_callback.
> It need be fixed.
> 
> I am wondering how the insane values happened on i386, and it didn't
> happen on x86_64.

Never mind, have got it.



From bhe at redhat.com  Tue Jan 13 22:24:42 2015
From: bhe at redhat.com (Baoquan He)
Date: Wed, 14 Jan 2015 14:24:42 +0800
Subject: [PATCH] kexec/x86: fix crashkernel reserved mem detection
In-Reply-To: <1419514701-9397-1-git-send-email-wolchkov.yuri@gmail.com>
References: <1419514701-9397-1-git-send-email-wolchkov.yuri@gmail.com>
Message-ID: <20150114062442.GD24543@dhcp-16-105.nay.redhat.com>

On 12/25/14 at 10:38pm, Yuri Volchkov wrote:
> Function crashkernel_mem_callback had wrong parameters type
> So crash_reserved_mem got completely insane values
> As a result kexec was unable to allocate segments for crashkernel
> For instance, I got this message:
> 	"Could not find a free area of memory of 0x9f000 bytes..."
> 
> Signed-off-by: Yuri Volchkov <wolchkov.yuri at gmail.com>
> ---
>  kexec/arch/i386/crashdump-x86.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/kexec/arch/i386/crashdump-x86.c b/kexec/arch/i386/crashdump-x86.c
> index 77bdad5..82bf239 100644
> --- a/kexec/arch/i386/crashdump-x86.c
> +++ b/kexec/arch/i386/crashdump-x86.c
> @@ -1041,8 +1041,8 @@ int get_max_crash_kernel_limit(uint64_t *start, uint64_t *end)
>  

crash_sh_memory_range_callback in kexec/arch/sh/crashdump-sh.c also has
this problem, please consider it.

>  static int crashkernel_mem_callback(void *UNUSED(data), int nr,
>                                            char *UNUSED(str),
> -                                          unsigned long base,
> -                                          unsigned long length)
> +                                          unsigned long long base,
> +                                          unsigned long long length)
>  {
>  	if (nr >= CRASH_RESERVED_MEM_NR)
>  		return 1;
> -- 
> 2.2.1
> 
> 
> _______________________________________________
> kexec mailing list
> kexec at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/kexec


From anurup.m at huawei.com  Wed Jan 14 04:03:33 2015
From: anurup.m at huawei.com (Anurup M)
Date: Wed, 14 Jan 2015 17:33:33 +0530
Subject: Kdump on ARM64
In-Reply-To: <54B5D030.20809@linaro.org>
References: <54AE2544.2010801@huawei.com> <54AE3B7A.4070301@linaro.org>
 <54B5003A.1050508@huawei.com> <54B5D030.20809@linaro.org>
Message-ID: <54B65B15.7020704@huawei.com>



On 1/14/2015 7:40 AM, AKASHI Takahiro wrote:
> On 01/13/2015 08:23 PM, Anurup M wrote:
>>
>> Hi Akashi,
>>
>>     Thanks for the reply.
>>
>> Hi  Akashi, Geoff,
>>
>>     In the code I could find assert(arm64_mem.memstart) in the function virt_to_phys. This address can be 0x0 when the
>>    segment start address is 0x0. so can we use this assert?
> 
> Probably you're right.
> 
>>     The code flow is elf_arm64_load->load_crashdump_segments->add_buffer_phys_virt->add_segment_phys_virt->virt_to_phys
>>    and this call is also before the call to load_elf_exec_in_crashmem which init's arm64_mem.memstart to crash_reserved_mem.start.
>>    So arm64_mem.memstart is also not set before the call to virt_to_phys.
> 
> I need refresh my memory about the flow, but in this particular case,
> the last argument, phys, of add_buffer_phys_virt() is 0 and virt_to_phys()
> will not be called.
> 

Yes, You are right. The code flow which I gave was wrong. It is observed  when I don't input the -p option (only kexec). The error flow is
elf_arm64_load->elf_exec_load->add_segment->add_buffer_phys_virt->add_segment_phys_virt->virt_to_phys

Thanks,
Anurup

> Do you have any other evidence that the assertion above is broken?
> 
> Thanks,
> -Takahiro AKASHI
> 
> 
>>     Please share your comments.
>>
>> Thanks,
>> Anurup
>>
>>
>> On 1/8/2015 1:40 PM, AKASHI Takahiro wrote:
>>> Hi Anurup,
>>>
>>> On 01/08/2015 03:35 PM, Anurup M wrote:
>>>> Hi Akashi, Geoff,
>>>>
>>>>      I tried to check kdump on arm64 on Foundation model. The/proc/vmcore is generated.
>>>> But I couldnot check if it is proper. Is there a makedumpfile binary for arm64? and also crash tool.
>>>> I used the master branch of https://git.linaro.org/people/geoff.levand/linux-kexec.git to verify.
>>>>        Please provide some information about the current status or support for kdump on arm64?
>>>> and the ways to verify or start development on Foundation model.
>>>
>>> The story is that I used to work on kdump, but handed it off
>>> to Geoff at some time. Since then, he merged it into his repo
>>> but I don't think that he has tested (verified) it yet.
>>> So please take it as a study work (or prototype), although I'm
>>> pretty sure that it works well on model except some issues.
>>>
>>> Regarding /proc/vmcore, if you can retrieve it to your host machine,
>>> you can simply verify the contents as follows:
>>>    $ aarch64-linux-gnu-gdb vmlinux core(=vmcore)
>>>    (gdb) info threads
>>>    (gdb) bt
>>>     ...
>>>
>>> I also have a trivial patch for crash tool to cross-build it, but
>>> arm64 support (by RedHat folk) is already there.
>>>
>>> Thanks,
>>> -Takahiro AKASHI
>>>
>>>> Thanks,
>>>> Anurup M
>>>>
>>>
> .
> 


From ao108-help at archofficegroup.com  Wed Jan 14 04:47:16 2015
From: ao108-help at archofficegroup.com (ao108-help at archofficegroup.com)
Date: 14 Jan 2015 12:47:16 -0000
Subject: ezmlm warning
Message-ID: <1421239636.31018.ezmlm-warn@archofficegroup.com>

Bonjour ! Je suis le programme ezmlm. Je m'occupe de la liste
de diffusion ao108 at archofficegroup.com.


Un certain nombre de messages provenant de la liste de diffusion
<#l> n'ont pas pu vous etre remis correctement. En attachement, vous trouverez
une copie du premier message de retour a l'envoyeur que j'ai recu.

Si le message que vous lisez actuellement ne parvient pas non plus a
destination, un dernier message vous sera envoye. Si celui-ci echoue aussi,
votre adresse sera malheureusement retiree de la liste ao108.



J'ai conserve une liste des messages de la liste de diffusion ao108 qui
n'ont temporairement pas pu etre delivres a votre adresse.

Voici les numeros des messages en question :

   10

--- Ci-dessous se trouve une copie du message problematique qui m'est revenu :

Return-Path: <bounce-bounce at mx1.ovh.net>
Received: from b0.ovh.net (HELO queue) (213.186.33.50)
	by b0.ovh.net with SMTP; 9 Dec 2014 10:15:00 +0200
Received: from 2.mo65.mail-out.ovh.net (HELO mo65.mail-out.ovh.net) (178.33.255.144)
  by mx1.ovh.net with SMTP; 9 Dec 2014 10:14:54 +0200
Received: by mo65.mail-out.ovh.net (Postfix)
	id C5929FF87FE; Tue,  9 Dec 2014 09:14:54 +0100 (CET)
Date: Tue,  9 Dec 2014 09:14:54 +0100 (CET)
From: MAILER-DAEMON at mo65.mail-out.ovh.net (Mail Delivery System)
Subject: Undelivered Mail Returned to Sender
To: ao108-return-10-kexec=lists.infradead.org at archofficegroup.com
Auto-Submitted: auto-replied
MIME-Version: 1.0
Content-Type: multipart/report; report-type=delivery-status;
	boundary="767E0FF83E2.1418112894/mo65.mail-out.ovh.net"
Content-Transfer-Encoding: 8bit
Message-Id: <20141209081454.C5929FF87FE at mo65.mail-out.ovh.net>
X-Ovh-Tracer-Id: 12645544805227626612
X-Ovh-Remote: 178.33.255.144 (2.mo65.mail-out.ovh.net)
X-Ovh-Local: 213.186.33.29 (mx1.ovh.net)
X-OVH-SPAMSTATE: OK
X-OVH-SPAMSCORE: 0
X-OVH-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrfeejiedrfeehucetufdoteggodetrfcurfhrohhfihhlvgemucfqggfjnecuuegrihhlohhuthemuceftddtnecu

This is a MIME-encapsulated message.

--767E0FF83E2.1418112894/mo65.mail-out.ovh.net
Content-Description: Notification
Content-Type: text/plain; charset=us-ascii

This is the mail system at host mo65.mail-out.ovh.net.

I'm sorry to have to inform you that your message could not
be delivered to one or more recipients. It's attached below.

For further assistance, please send mail to postmaster.

If you do so, please include this problem report. You can
delete your own text from the attached returned message.

                   The mail system

<kexec at lists.infradead.org>: host bombadil.infradead.org[198.137.202.9] said:
    550-Mailing lists do not accept HTML mail. See 550
    http://david.woodhou.se/email.html (in reply to end of DATA command)

--767E0FF83E2.1418112894/mo65.mail-out.ovh.net
Content-Description: Delivery report
Content-Type: message/delivery-status

Reporting-MTA: dns; mo65.mail-out.ovh.net
X-Postfix-Queue-ID: 767E0FF83E2
X-Postfix-Sender: rfc822; ao108-return-10-kexec=lists.infradead.org at archofficegroup.com
Arrival-Date: Tue,  9 Dec 2014 09:14:27 +0100 (CET)

Final-Recipient: rfc822; kexec at lists.infradead.org
Original-Recipient: rfc822;kexec at lists.infradead.org
Action: failed
Status: 5.0.0
Remote-MTA: dns; bombadil.infradead.org
Diagnostic-Code: smtp; 550-Mailing lists do not accept HTML mail. See 550
    http://david.woodhou.se/email.html

--767E0FF83E2.1418112894/mo65.mail-out.ovh.net
Content-Description: Undelivered Message Headers
Content-Type: text/rfc822-headers
Content-Transfer-Encoding: 8bit

Return-Path: <ao108-return-10-kexec=lists.infradead.org at archofficegroup.com>
Received: from mail95.ha.ovh.net (gw6.ovh.net [213.251.189.206])
	by mo65.mail-out.ovh.net (Postfix) with SMTP id 767E0FF83E2
	for <kexec at lists.infradead.org>; Tue,  9 Dec 2014 09:14:27 +0100 (CET)
Received: from b0.ovh.net (HELO queueml) (213.186.33.50)
	by b0.ovh.net with SMTP; 9 Dec 2014 10:06:40 +0200
Mailing-List: Pour toute requete administrative, contactez ao108-help at archofficegroup.com; Liste geree par ezmlm
Precedence: bulk
X-No-Archive: yes
List-Post: <mailto:ao108 at archofficegroup.com>
List-Help: <mailto:ao108-help at archofficegroup.com>
List-Unsubscribe: <mailto:ao108-unsubscribe at archofficegroup.com>
List-Subscribe: <mailto:ao108-subscribe at archofficegroup.com>
Reply-To: info at archofficegroup.com
Delivered-To: mailing list ao108 at archofficegroup.com
Delivered-To: moderator for ao108 at archofficegroup.com
Received: from b0.ovh.net (HELO queue) (213.186.33.50)
	by b0.ovh.net with SMTP; 9 Dec 2014 09:55:16 +0200
Received: from localhost (HELO mail95.ha.ovh.net) (127.0.0.1)
  by localhost with SMTP; 9 Dec 2014 09:55:16 +0200
Received: from b0.ovh.net (HELO queueout) (213.186.33.50)
	by b0.ovh.net with SMTP; 9 Dec 2014 09:55:15 +0200
Received: from 230.140.0.109.rev.sfr.net (HELO ?192.168.2.141?) (info at archofficegroup.com@109.0.140.230)
  by ns0.ovh.net with SMTP; 9 Dec 2014 09:55:06 +0200
Message-ID: <5486AAD8.3050006 at archofficegroup.com>
Date: Tue, 09 Dec 2014 08:55:04 +0100
From: Arch'Office innove <info at archofficegroup.com>
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; fr; rv:1.9.1.10) Gecko/20100512 Thunderbird/3.0.5
MIME-Version: 1.0
To: ao108 at archofficegroup.com
Content-Type: multipart/alternative;
 boundary="------------050903020006080602050604"
X-Ovh-Tracer-Id: 12311152532950184046
X-Ovh-Remote: 109.0.140.230 (230.140.0.109.rev.sfr.net)
X-Ovh-Local: 213.186.33.20 (ns0.ovh.net)
X-OVH-SPAMSTATE: OK
X-OVH-SPAMSCORE: 0
X-OVH-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrfeejiedrfeehucetufdoteggodetrfcurfhrohhfihhlvgemucfqggfjnecuuegrihhlohhuthemuceftddtnecu
Subject: [ao108] Solutions acoustiques pour espaces bureaux
X-VR-SPAMSTATE: OK
X-VR-SPAMSCORE: 0
X-VR-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrfeejiedrfeehucetufdoteggodetrfcurfhrohhfihhlvgemucfqggfjnecuuegrihhlohhuthemuceftddtnecu

--767E0FF83E2.1418112894/mo65.mail-out.ovh.net--


From bhe at redhat.com  Wed Jan 14 19:28:58 2015
From: bhe at redhat.com (Baoquan He)
Date: Thu, 15 Jan 2015 11:28:58 +0800
Subject: [PATCH v8 06/10] iommu/vt-d: datatypes and functions used for
 kdump
In-Reply-To: <1421046388-27925-7-git-send-email-zhen-hual@hp.com>
References: <1421046388-27925-1-git-send-email-zhen-hual@hp.com>
 <1421046388-27925-7-git-send-email-zhen-hual@hp.com>
Message-ID: <20150115032858.GA2461@dhcp-16-105.nay.redhat.com>

On 01/12/15 at 03:06pm, Li, Zhen-Hua wrote:
> +/*
> + * Interface to the "copy translation tables" set of functions
> + * from mainline code.
> + */
> +static int intel_iommu_load_translation_tables(struct dmar_drhd_unit *drhd,
> +		int g_num_of_iommus)

The argument g_num_of_iommus is the same as the global variable, it's better
to rename it as num_of_iommus. And even it can be removed since you can
just use the global variable g_num_of_iommus in this function.

Argument drhd can be intel_iommu because no other member variable in
drhd is needed.

> +{
> +	struct intel_iommu *iommu;	/* Virt(iommu hardware registers) */
> +	unsigned long long q;		/* quadword scratch */
> +	int ret = 0;			/* Integer return code */
> +	int i = 0;			/* Loop index */
> +	unsigned long flags;
> +
> +	/* Structure so copy_page_addr() can accumulate things
> +	 * over multiple calls and returns
> +	 */
> +	struct copy_page_addr_parms ppa_parms = copy_page_addr_parms_init;
> +	struct copy_page_addr_parms *ppap = &ppa_parms;
> +
> +
> +	iommu = drhd->iommu;
> +	q = dmar_readq(iommu->reg + DMAR_RTADDR_REG);
> +	if (!q)
> +		return -1;
> +
> +	/* If (list needs initializing) do it here */

This initializing should not be here, because it's not only for this
drhd. It should be done in init_dmars().

> +	if (!domain_values_list) {
> +		domain_values_list =
> +			 kcalloc(g_num_of_iommus, sizeof(struct list_head),
> +					GFP_KERNEL);
> +
> +		if (!domain_values_list) {
> +			pr_err("Allocation failed for domain_values_list array\n");
> +			return -ENOMEM;
> +		}
> +		for (i = 0; i < g_num_of_iommus; i++)
> +			INIT_LIST_HEAD(&domain_values_list[i]);
> +	}
> +
> +	spin_lock_irqsave(&iommu->lock, flags);
> +
> +	/* Load the root-entry table from the old kernel
> +	 * foreach context_entry_table in root_entry
> +	 *    foreach context_entry in context_entry_table
> +	 *       foreach level-1 page_table_entry in context_entry
> +	 *          foreach level-2 page_table_entry in level 1 page_table_entry
> +	 *             Above pattern continues up to 6 levels of page tables
> +	 *                Sanity-check the entry
> +	 *                Process the bus, devfn, page_address, page_size
> +	 */
> +	if (!iommu->root_entry) {
> +		iommu->root_entry =
> +			(struct root_entry *)alloc_pgtable_page(iommu->node);
> +		if (!iommu->root_entry) {
> +			spin_unlock_irqrestore(&iommu->lock, flags);
> +			return -ENOMEM;
> +		}
> +	}
> +
> +	iommu->root_entry_old_phys = q & VTD_PAGE_MASK;
> +	if (!iommu->root_entry_old_phys) {
> +		pr_err("Could not read old root entry address.");
> +		return -1;
> +	}
> +
> +	iommu->root_entry_old_virt = ioremap_cache(iommu->root_entry_old_phys,
> +						VTD_PAGE_SIZE);
> +	if (!iommu->root_entry_old_virt) {
> +		pr_err("Could not map the old root entry.");
> +		return -ENOMEM;
> +	}
> +
> +	__iommu_load_old_root_entry(iommu);
> +	ret = copy_root_entry_table(iommu, ppap);
> +	__iommu_flush_cache(iommu, iommu->root_entry, PAGE_SIZE);
> +	__iommu_update_old_root_entry(iommu, -1);
> +
> +	spin_unlock_irqrestore(&iommu->lock, flags);
> +
> +	__iommu_free_mapped_mem();
> +
> +	if (ret)
> +		return ret;
> +
> +	ppa_parms.last = 1;
> +	copy_page_addr(0, 0, 0, 0, 0, NULL, ppap);
> +
> +	return 0;
> +}
> +
>  #endif /* CONFIG_CRASH_DUMP */
> -- 
> 2.0.0-rc0
> 
> 
> _______________________________________________
> kexec mailing list
> kexec at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/kexec


From zhen-hual at hp.com  Wed Jan 14 21:45:19 2015
From: zhen-hual at hp.com (Li, ZhenHua)
Date: Thu, 15 Jan 2015 13:45:19 +0800
Subject: [PATCH v8 06/10] iommu/vt-d: datatypes and functions used for
 kdump
In-Reply-To: <20150115032858.GA2461@dhcp-16-105.nay.redhat.com>
References: <1421046388-27925-1-git-send-email-zhen-hual@hp.com>
 <1421046388-27925-7-git-send-email-zhen-hual@hp.com>
 <20150115032858.GA2461@dhcp-16-105.nay.redhat.com>
Message-ID: <54B753EF.30609@hp.com>

Hi Baoquan,
Thank you very much for your review. But according to the latest
discussion, the page tables will not be copied from old kernel. We keep
using the old page tables before driver is loaded. So there are many
changes in next version;

See my comments.

On 01/15/2015 11:28 AM, Baoquan He wrote:
> On 01/12/15 at 03:06pm, Li, Zhen-Hua wrote:
>> +/*
>> + * Interface to the "copy translation tables" set of functions
>> + * from mainline code.
>> + */
>> +static int intel_iommu_load_translation_tables(struct dmar_drhd_unit *drhd,
>> +		int g_num_of_iommus)
>
> The argument g_num_of_iommus is the same as the global variable, it's better
> to rename it as num_of_iommus. And even it can be removed since you can
> just use the global variable g_num_of_iommus in this function.
>
> Argument drhd can be intel_iommu because no other member variable in
> drhd is needed.

This function is no longer used. So forget the parameters.

>
>> +{
>> +	struct intel_iommu *iommu;	/* Virt(iommu hardware registers) */
>> +	unsigned long long q;		/* quadword scratch */
>> +	int ret = 0;			/* Integer return code */
>> +	int i = 0;			/* Loop index */
>> +	unsigned long flags;
>> +
>> +	/* Structure so copy_page_addr() can accumulate things
>> +	 * over multiple calls and returns
>> +	 */
>> +	struct copy_page_addr_parms ppa_parms = copy_page_addr_parms_init;
>> +	struct copy_page_addr_parms *ppap = &ppa_parms;
>> +
>> +
>> +	iommu = drhd->iommu;
>> +	q = dmar_readq(iommu->reg + DMAR_RTADDR_REG);
>> +	if (!q)
>> +		return -1;
>> +
>> +	/* If (list needs initializing) do it here */
>
> This initializing should not be here, because it's not only for this
> drhd. It should be done in init_dmars().
>
Yes you are right. Though the variable domain_values_list will not be
used in next version, I think I need to check if there are any other
similar problems.

>> +	if (!domain_values_list) {
>> +		domain_values_list =
>> +			 kcalloc(g_num_of_iommus, sizeof(struct list_head),
>> +					GFP_KERNEL);
>> +
>> +		if (!domain_values_list) {
>> +			pr_err("Allocation failed for domain_values_list array\n");
>> +			return -ENOMEM;
>> +		}
>> +		for (i = 0; i < g_num_of_iommus; i++)
>> +			INIT_LIST_HEAD(&domain_values_list[i]);
>> +	}
>> +
>> +	spin_lock_irqsave(&iommu->lock, flags);
>> +
>> +	/* Load the root-entry table from the old kernel
>> +	 * foreach context_entry_table in root_entry
>> +	 *    foreach context_entry in context_entry_table
>> +	 *       foreach level-1 page_table_entry in context_entry
>> +	 *          foreach level-2 page_table_entry in level 1 page_table_entry
>> +	 *             Above pattern continues up to 6 levels of page tables
>> +	 *                Sanity-check the entry
>> +	 *                Process the bus, devfn, page_address, page_size
>> +	 */
>> +	if (!iommu->root_entry) {
>> +		iommu->root_entry =
>> +			(struct root_entry *)alloc_pgtable_page(iommu->node);
>> +		if (!iommu->root_entry) {
>> +			spin_unlock_irqrestore(&iommu->lock, flags);
>> +			return -ENOMEM;
>> +		}
>> +	}
>> +
>> +	iommu->root_entry_old_phys = q & VTD_PAGE_MASK;
>> +	if (!iommu->root_entry_old_phys) {
>> +		pr_err("Could not read old root entry address.");
>> +		return -1;
>> +	}
>> +
>> +	iommu->root_entry_old_virt = ioremap_cache(iommu->root_entry_old_phys,
>> +						VTD_PAGE_SIZE);
>> +	if (!iommu->root_entry_old_virt) {
>> +		pr_err("Could not map the old root entry.");
>> +		return -ENOMEM;
>> +	}
>> +
>> +	__iommu_load_old_root_entry(iommu);
>> +	ret = copy_root_entry_table(iommu, ppap);
>> +	__iommu_flush_cache(iommu, iommu->root_entry, PAGE_SIZE);
>> +	__iommu_update_old_root_entry(iommu, -1);
>> +
>> +	spin_unlock_irqrestore(&iommu->lock, flags);
>> +
>> +	__iommu_free_mapped_mem();
>> +
>> +	if (ret)
>> +		return ret;
>> +
>> +	ppa_parms.last = 1;
>> +	copy_page_addr(0, 0, 0, 0, 0, NULL, ppap);
>> +
>> +	return 0;
>> +}
>> +
>>   #endif /* CONFIG_CRASH_DUMP */
>> --
>> 2.0.0-rc0
>>
>>
>> _______________________________________________
>> kexec mailing list
>> kexec at lists.infradead.org
>> http://lists.infradead.org/mailman/listinfo/kexec



From bhe at redhat.com  Wed Jan 14 23:01:37 2015
From: bhe at redhat.com (Baoquan He)
Date: Thu, 15 Jan 2015 15:01:37 +0800
Subject: [PATCH v8 06/10] iommu/vt-d: datatypes and functions used for
 kdump
In-Reply-To: <54B753EF.30609@hp.com>
References: <1421046388-27925-1-git-send-email-zhen-hual@hp.com>
 <1421046388-27925-7-git-send-email-zhen-hual@hp.com>
 <20150115032858.GA2461@dhcp-16-105.nay.redhat.com>
 <54B753EF.30609@hp.com>
Message-ID: <20150115070137.GB2461@dhcp-16-105.nay.redhat.com>

On 01/15/15 at 01:45pm, Li, ZhenHua wrote:
> Hi Baoquan,
> Thank you very much for your review. But according to the latest
> discussion, the page tables will not be copied from old kernel. We keep
> using the old page tables before driver is loaded. So there are many
> changes in next version;

Oh, yes. So please ignore this comment.

> 
> See my comments.
> 
> On 01/15/2015 11:28 AM, Baoquan He wrote:
> >On 01/12/15 at 03:06pm, Li, Zhen-Hua wrote:
> >>+/*
> >>+ * Interface to the "copy translation tables" set of functions
> >>+ * from mainline code.
> >>+ */
> >>+static int intel_iommu_load_translation_tables(struct dmar_drhd_unit *drhd,
> >>+		int g_num_of_iommus)
> >
> >The argument g_num_of_iommus is the same as the global variable, it's better
> >to rename it as num_of_iommus. And even it can be removed since you can
> >just use the global variable g_num_of_iommus in this function.
> >
> >Argument drhd can be intel_iommu because no other member variable in
> >drhd is needed.
> 
> This function is no longer used. So forget the parameters.
> 
> >
> >>+{
> >>+	struct intel_iommu *iommu;	/* Virt(iommu hardware registers) */
> >>+	unsigned long long q;		/* quadword scratch */
> >>+	int ret = 0;			/* Integer return code */
> >>+	int i = 0;			/* Loop index */
> >>+	unsigned long flags;
> >>+
> >>+	/* Structure so copy_page_addr() can accumulate things
> >>+	 * over multiple calls and returns
> >>+	 */
> >>+	struct copy_page_addr_parms ppa_parms = copy_page_addr_parms_init;
> >>+	struct copy_page_addr_parms *ppap = &ppa_parms;
> >>+
> >>+
> >>+	iommu = drhd->iommu;
> >>+	q = dmar_readq(iommu->reg + DMAR_RTADDR_REG);
> >>+	if (!q)
> >>+		return -1;
> >>+
> >>+	/* If (list needs initializing) do it here */
> >
> >This initializing should not be here, because it's not only for this
> >drhd. It should be done in init_dmars().
> >
> Yes you are right. Though the variable domain_values_list will not be
> used in next version, I think I need to check if there are any other
> similar problems.
> 
> >>+	if (!domain_values_list) {
> >>+		domain_values_list =
> >>+			 kcalloc(g_num_of_iommus, sizeof(struct list_head),
> >>+					GFP_KERNEL);
> >>+
> >>+		if (!domain_values_list) {
> >>+			pr_err("Allocation failed for domain_values_list array\n");
> >>+			return -ENOMEM;
> >>+		}
> >>+		for (i = 0; i < g_num_of_iommus; i++)
> >>+			INIT_LIST_HEAD(&domain_values_list[i]);
> >>+	}
> >>+
> >>+	spin_lock_irqsave(&iommu->lock, flags);
> >>+
> >>+	/* Load the root-entry table from the old kernel
> >>+	 * foreach context_entry_table in root_entry
> >>+	 *    foreach context_entry in context_entry_table
> >>+	 *       foreach level-1 page_table_entry in context_entry
> >>+	 *          foreach level-2 page_table_entry in level 1 page_table_entry
> >>+	 *             Above pattern continues up to 6 levels of page tables
> >>+	 *                Sanity-check the entry
> >>+	 *                Process the bus, devfn, page_address, page_size
> >>+	 */
> >>+	if (!iommu->root_entry) {
> >>+		iommu->root_entry =
> >>+			(struct root_entry *)alloc_pgtable_page(iommu->node);
> >>+		if (!iommu->root_entry) {
> >>+			spin_unlock_irqrestore(&iommu->lock, flags);
> >>+			return -ENOMEM;
> >>+		}
> >>+	}
> >>+
> >>+	iommu->root_entry_old_phys = q & VTD_PAGE_MASK;
> >>+	if (!iommu->root_entry_old_phys) {
> >>+		pr_err("Could not read old root entry address.");
> >>+		return -1;
> >>+	}
> >>+
> >>+	iommu->root_entry_old_virt = ioremap_cache(iommu->root_entry_old_phys,
> >>+						VTD_PAGE_SIZE);
> >>+	if (!iommu->root_entry_old_virt) {
> >>+		pr_err("Could not map the old root entry.");
> >>+		return -ENOMEM;
> >>+	}
> >>+
> >>+	__iommu_load_old_root_entry(iommu);
> >>+	ret = copy_root_entry_table(iommu, ppap);
> >>+	__iommu_flush_cache(iommu, iommu->root_entry, PAGE_SIZE);
> >>+	__iommu_update_old_root_entry(iommu, -1);
> >>+
> >>+	spin_unlock_irqrestore(&iommu->lock, flags);
> >>+
> >>+	__iommu_free_mapped_mem();
> >>+
> >>+	if (ret)
> >>+		return ret;
> >>+
> >>+	ppa_parms.last = 1;
> >>+	copy_page_addr(0, 0, 0, 0, 0, NULL, ppap);
> >>+
> >>+	return 0;
> >>+}
> >>+
> >>  #endif /* CONFIG_CRASH_DUMP */
> >>--
> >>2.0.0-rc0
> >>
> >>
> >>_______________________________________________
> >>kexec mailing list
> >>kexec at lists.infradead.org
> >>http://lists.infradead.org/mailman/listinfo/kexec
> 


From mtk.manpages at gmail.com  Fri Jan 16 05:30:25 2015
From: mtk.manpages at gmail.com (Michael Kerrisk (man-pages))
Date: Fri, 16 Jan 2015 14:30:25 +0100
Subject: Edited kexec_load(2) [kexec_file_load()] man page for review
In-Reply-To: <20150112221634.GD16162@redhat.com>
References: <545FBDDD.9060801@gmail.com> <20141111213037.GA31445@redhat.com>
 <54ADA284.30502@gmail.com> <20150112221634.GD16162@redhat.com>
Message-ID: <54B91271.3000600@gmail.com>

Hello Vivek,

Thanks for your comments! I've added some further text to
the page based on those comments. See some follow-up 
questions below.

On 01/12/2015 11:16 PM, Vivek Goyal wrote:
> On Wed, Jan 07, 2015 at 10:17:56PM +0100, Michael Kerrisk (man-pages) wrote:
> 
> [..]
>>>> .BR KEXEC_ON_CRASH " (since Linux 2.6.13)"
>>>> Execute the new kernel automatically on a system crash.
>>>> .\" FIXME Explain in more detail how KEXEC_ON_CRASH is actually used
>>
>> I wasn't expecting that you would respond to the FIXMEs that were 
>> not labeled "kexec_file_load", but I was hoping you might ;-). Thanks!
>> I have a few additional questions to your nice notes.
>>
>>> Upon boot first kernel reserves a chunk of contiguous memory (if
>>> crashkernel=<> command line paramter is passed). This memory is
>>> is used to load the crash kernel (Kernel which will be booted into
>>> if first kernel crashes).
>>
> 
> Hi Michael,
> 
>> Can I just confirm: is it in all cases only possible to use kexec_load() 
>> and kexec_file_load() if the kernel was booted with the 'crashkernel'
>> parameter set?
> 
> As of now, only kexec_load() and kexec_file_load() system calls can
> make use of memory reserved by crashkernel=<> kernel parameter. And
> this is used only if we are trying to load a crash kernel (KEXEC_ON_CRASH
> flag specified).

Okay.

>>> Location of this reserved memory is exported to user space through
>>> /proc/iomem file. 
>>
>> Is that export via an entry labeled "Crash kernel" in the 
>> /proc/iomem file?
> 
> Yes.

Okay -- thanks.

>>> User space can parse it and prepare list of segments
>>> specifying this reserved memory as destination.
>>
>> I'm not quite clear on "specifying this reserved memory as destination".
>> Is that done by specifying the address in the kexec_segment.mem fields?
> 
> You are absolutely right. User space can specify in kexec_segment.mem
> field the memory location where it expecting a particular segment to
> be loaded by kernel.
> 
>>
>>> Once kernel sees the flag KEXEC_ON_CRASH, it makes sure that all the
>>> segments are destined for reserved memory otherwise kernel load operation
>>> fails.
>>
>> Could you point me to where this checking is done? Also, what is the
>> error (errno) that occurs when the load operation fails? (I think the
>> answers to these questions are "at the start of kimage_alloc_init()"
>> and "EADDRNOTAVAIL", but I'd like to confirm.)
> 
> This checking happens in sanity_check_segment_list() which is called
> by kimage_alloc_init().
> 
> And yes, error code returned is -EADDRNOTAVAIL.

Thanks. I added EADDRNOTAVAIL to the ERRORS.

>>> [..]
>>>> struct kexec_segment {
>>>>     void   *buf;        /* Buffer in user space */
>>>>     size_t  bufsz;      /* Buffer length in user space */
>>>>     void   *mem;        /* Physical address of kernel */
>>>>     size_t  memsz;      /* Physical address length */
>>>> };
>>>> .fi
>>>> .in
>>>> .PP
>>>> .\" FIXME Explain the details of how the kernel image defined by segments
>>>> .\" is copied from the calling process into previously reserved memory.
>>>
>>> Kernel image defined by segments is copied into kernel either in regular
>>> memory 
>>
>> Could you clarify what you mean by "regular memory"?
> 
> I meant memory which is not reserved memory.

Okay.

>>> or in reserved memory (if KEXEC_ON_CRASH is set). Kernel first
>>> copies list of segments in kernel memory and then goes does various
>>> sanity checks on the segments. If everything looks line, kernel copies
>>> segment data to kernel memory.
>>>
>>> In case of normal kexec, segment data is loaded in any available memory
>>> and segment data is moved to final destination at the kexec reboot time.
>>
>> By "moved to final destination", do you mean "moved from user space to the
>> final kernel-space destination"?
> 
> No. Segment data moves from user space to kernel space once kexec_load()
> call finishes successfully. But when user does reboot (kexec -e), at that
> time kernel moves that segment data to its final location. Kernel could
> not place the segment at its final location during kexec_load() time as
> that memory is already in use by running kernel. But once we are about
> to reboot to new kernel, we can overwrite the old kernel's memory.

Got it.

>>> In case of kexec on panic (KEXEC_ON_CRASH flag set), segment data is
>>> directly loaded to reserved memory and after crash kexec simply jumps
>>
>> By "directly", I assume you mean "at the time of the kexec_laod() call",
>> right?
> 
> Yes.

Thanks.

So, returning to the kexeec_segment structure:

           struct kexec_segment {
               void   *buf;        /* Buffer in user space */
               size_t  bufsz;      /* Buffer length in user space */
               void   *mem;        /* Physical address of kernel */
               size_t  memsz;      /* Physical address length */
           };

Are the following statements correct:
* buf + bufsz identify a memory region in the caller's virtual 
  address space that is the source of the copy
* mem + memsz specify the target memory region of the copy
* mem is  physical memory address, as seen from kernel space
* the number of bytes copied from userspace is min(bufsz, memsz)
* if bufsz > memsz, then excess bytes in the user-space buffer 
  are ignored.
* if memsz > bufsz, then excess bytes in the target kernel buffer
  are filled with zeros.
?

Also, it seems to me that 'mem' need not be page aligned.
Is that correct? Should the man page say something about that?
(E.g., is it generally desirable that 'mem' should be page aligned?)

Likewise, 'memsz' doesn't need to be a page multiple, IIUC.
Should the man page say anything about this? For example, should 
it note that the initialized kernel segment will be of size:

     (mem % PAGE_SIZE + memsz) rounded up to the next multiple of PAGE_SIZE

And should it note that if 'mem' is not a multiple of the page size, then
the initial bytes (mem % PAGE_SIZE)) in the first page of the kernel segment 
will be zeros?

(Hopefully I have read kimage_load_normal_segment() correctly.)

And one further question. Other than the fact that they are used with 
different system calls, what is the difference between KEXEC_ON_CRASH 
and KEXEC_FILE_ON_CRASH?

Thanks,

Michael

-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/


From geoff at infradead.org  Fri Jan 16 16:23:34 2015
From: geoff at infradead.org (Geoff Levand)
Date: Sat, 17 Jan 2015 00:23:34 +0000
Subject: [PATCH 6/8] arm64/kexec: Add pr_devel output
In-Reply-To: <cover.1421449714.git.geoff@infradead.org>
References: <cover.1421449714.git.geoff@infradead.org>
Message-ID: <87d3faa4db6d996be10bcd69eba01f168c6422f3.1421449714.git.geoff@infradead.org>

To aid in debugging kexec problems or when adding new functionality to kexec add
a new routine kexec_image_info() and several inline pr_devel statements.

Signed-off-by: Geoff Levand <geoff at infradead.org>
---
 arch/arm64/kernel/machine_kexec.c | 54 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 54 insertions(+)

diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c
index b0e5d76..3d84759 100644
--- a/arch/arm64/kernel/machine_kexec.c
+++ b/arch/arm64/kernel/machine_kexec.c
@@ -35,6 +35,37 @@ static bool kexec_is_dtb(const void *dtb)
 }
 
 /**
+ * kexec_image_info - For debugging output.
+ */
+#define kexec_image_info(_i) _kexec_image_info(__func__, __LINE__, _i)
+static void _kexec_image_info(const char *func, int line,
+	const struct kimage *image)
+{
+	unsigned long i;
+
+#if !defined(DEBUG)
+	return;
+#endif
+	pr_devel("%s:%d:\n", func, line);
+	pr_devel("  kexec image info:\n");
+	pr_devel("    type:        %d\n", image->type);
+	pr_devel("    start:       %lx\n", image->start);
+	pr_devel("    head:        %lx\n", image->head);
+	pr_devel("    nr_segments: %lu\n", image->nr_segments);
+
+	for (i = 0; i < image->nr_segments; i++) {
+		pr_devel("      segment[%lu]: %016lx - %016lx, %lx bytes, %lu pages%s\n",
+			i,
+			image->segment[i].mem,
+			image->segment[i].mem + image->segment[i].memsz,
+			image->segment[i].memsz,
+			image->segment[i].memsz /  PAGE_SIZE,
+			(kexec_is_dtb(image->segment[i].buf) ?
+				", dtb segment" : ""));
+	}
+}
+
+/**
  * kexec_find_dtb_seg - Helper routine to find the dtb segment.
  */
 static const struct kexec_segment *kexec_find_dtb_seg(
@@ -67,6 +98,8 @@ int machine_kexec_prepare(struct kimage *image)
 	arm64_kexec_dtb_addr = dtb_seg ? dtb_seg->mem : 0;
 	arm64_kexec_kimage_start = image->start;
 
+	kexec_image_info(image);
+
 	return 0;
 }
 
@@ -121,6 +154,27 @@ void machine_kexec(struct kimage *image)
 	reboot_code_buffer_phys = page_to_phys(image->control_code_page);
 	reboot_code_buffer = phys_to_virt(reboot_code_buffer_phys);
 
+	kexec_image_info(image);
+
+	pr_devel("%s:%d: control_code_page:        %p\n", __func__, __LINE__,
+		image->control_code_page);
+	pr_devel("%s:%d: reboot_code_buffer_phys:  %pa\n", __func__, __LINE__,
+		&reboot_code_buffer_phys);
+	pr_devel("%s:%d: reboot_code_buffer:       %p\n", __func__, __LINE__,
+		reboot_code_buffer);
+	pr_devel("%s:%d: relocate_new_kernel:      %p\n", __func__, __LINE__,
+		relocate_new_kernel);
+	pr_devel("%s:%d: relocate_new_kernel_size: 0x%lx(%lu) bytes\n",
+		__func__, __LINE__, relocate_new_kernel_size,
+		relocate_new_kernel_size);
+
+	pr_devel("%s:%d: kexec_dtb_addr:           %lx\n", __func__, __LINE__,
+		arm64_kexec_dtb_addr);
+	pr_devel("%s:%d: kexec_kimage_head:        %lx\n", __func__, __LINE__,
+		arm64_kexec_kimage_head);
+	pr_devel("%s:%d: kexec_kimage_start:       %lx\n", __func__, __LINE__,
+		arm64_kexec_kimage_start);
+
 	/*
 	 * Copy relocate_new_kernel to the reboot_code_buffer for use
 	 * after the kernel is shut down.
-- 
2.1.0




From geoff at infradead.org  Fri Jan 16 16:23:34 2015
From: geoff at infradead.org (Geoff Levand)
Date: Sat, 17 Jan 2015 00:23:34 +0000
Subject: [PATCH 2/8] arm64: Convert hcalls to use ISS field
In-Reply-To: <cover.1421449714.git.geoff@infradead.org>
References: <cover.1421449714.git.geoff@infradead.org>
Message-ID: <6719b2520095926ae0e8ac888663b87f2098f565.1421449714.git.geoff@infradead.org>

To allow for additional hcalls to be defined and to make the arm64 hcall API
more consistent across exception vector routines, change the hcall implementations
to use the ISS field of the ESR_EL2 register to specify the hcall type.

The existing arm64 hcall implementations are limited in that they only allow
for two distinct hcalls; with the x0 register either zero, or not zero.  Also,
the API of the hyp-stub exception vector routines and the KVM exception vector
routines differ; hyp-stub uses a non-zero value in x0 to implement
__hyp_set_vectors, whereas KVM uses it to implement kvm_call_hyp.

Define three new preprocessor macros HVC_GET_VECTORS, HVC_SET_VECTORS and
HVC_CALL_HYP and to be used as hcall type specifiers and convert the
existing __hyp_get_vectors(), __hyp_set_vectors() and kvm_call_hyp() routines
to use these new macros when executing an HVC call.  Also change the
corresponding hyp-stub and KVM el1_sync exception vector routines to use these
new macros.

Signed-off-by: Geoff Levand <geoff at infradead.org>
---
 arch/arm64/include/asm/virt.h | 20 ++++++++++++++++++++
 arch/arm64/kernel/hyp-stub.S  | 34 ++++++++++++++++++++++------------
 arch/arm64/kvm/hyp.S          | 18 +++++++++++-------
 3 files changed, 53 insertions(+), 19 deletions(-)

diff --git a/arch/arm64/include/asm/virt.h b/arch/arm64/include/asm/virt.h
index 7a5df52..99c319c 100644
--- a/arch/arm64/include/asm/virt.h
+++ b/arch/arm64/include/asm/virt.h
@@ -21,6 +21,26 @@
 #define BOOT_CPU_MODE_EL1	(0xe11)
 #define BOOT_CPU_MODE_EL2	(0xe12)
 
+/*
+ * HVC_GET_VECTORS - Return the value of the vbar_el2 register.
+ */
+
+#define HVC_GET_VECTORS 1
+
+/*
+ * HVC_SET_VECTORS - Set the value of the vbar_el2 register.
+ *
+ * @x0: Physical address of the new vector table.
+ */
+
+#define HVC_SET_VECTORS 2
+
+/*
+ * HVC_CALL_HYP - Execute a hyp routine.
+ */
+
+#define HVC_CALL_HYP 3
+
 #ifndef __ASSEMBLY__
 
 /*
diff --git a/arch/arm64/kernel/hyp-stub.S b/arch/arm64/kernel/hyp-stub.S
index a272f33..e3db3fd 100644
--- a/arch/arm64/kernel/hyp-stub.S
+++ b/arch/arm64/kernel/hyp-stub.S
@@ -22,6 +22,7 @@
 #include <linux/irqchip/arm-gic-v3.h>
 
 #include <asm/assembler.h>
+#include <asm/kvm_arm.h>
 #include <asm/ptrace.h>
 #include <asm/virt.h>
 
@@ -53,14 +54,22 @@ ENDPROC(__hyp_stub_vectors)
 	.align 11
 
 el1_sync:
-	mrs	x1, esr_el2
-	lsr	x1, x1, #26
-	cmp	x1, #0x16
-	b.ne	2f				// Not an HVC trap
-	cbz	x0, 1f
-	msr	vbar_el2, x0			// Set vbar_el2
+	mrs	x18, esr_el2
+	lsr	x17, x18, #ESR_ELx_EC_SHIFT
+	and	x18, x18, #ESR_ELx_ISS_MASK
+
+	cmp     x17, #ESR_ELx_EC_HVC64
+	b.ne    2f				// Not an HVC trap
+
+	cmp	x18, #HVC_GET_VECTORS
+	b.ne	1f
+	mrs	x0, vbar_el2
 	b	2f
-1:	mrs	x0, vbar_el2			// Return vbar_el2
+
+1:	cmp	x18, #HVC_SET_VECTORS
+	b.ne	2f
+	msr	vbar_el2, x0
+
 2:	eret
 ENDPROC(el1_sync)
 
@@ -100,11 +109,12 @@ ENDPROC(\label)
  * initialisation entry point.
  */
 
-ENTRY(__hyp_get_vectors)
-	mov	x0, xzr
-	// fall through
 ENTRY(__hyp_set_vectors)
-	hvc	#0
+	hvc	#HVC_SET_VECTORS
 	ret
-ENDPROC(__hyp_get_vectors)
 ENDPROC(__hyp_set_vectors)
+
+ENTRY(__hyp_get_vectors)
+	hvc	#HVC_GET_VECTORS
+	ret
+ENDPROC(__hyp_get_vectors)
diff --git a/arch/arm64/kvm/hyp.S b/arch/arm64/kvm/hyp.S
index c0d8202..1916c89 100644
--- a/arch/arm64/kvm/hyp.S
+++ b/arch/arm64/kvm/hyp.S
@@ -27,6 +27,7 @@
 #include <asm/kvm_asm.h>
 #include <asm/kvm_mmu.h>
 #include <asm/memory.h>
+#include <asm/virt.h>
 
 #define CPU_GP_REG_OFFSET(x)	(CPU_GP_REGS + x)
 #define CPU_XREG_OFFSET(x)	CPU_GP_REG_OFFSET(CPU_USER_PT_REGS + 8*x)
@@ -1106,12 +1107,9 @@ __hyp_panic_str:
  * in Hyp mode (see init_hyp_mode in arch/arm/kvm/arm.c).  Return values are
  * passed in r0 and r1.
  *
- * A function pointer with a value of 0 has a special meaning, and is
- * used to implement __hyp_get_vectors in the same way as in
- * arch/arm64/kernel/hyp_stub.S.
  */
 ENTRY(kvm_call_hyp)
-	hvc	#0
+	hvc	#HVC_CALL_HYP
 	ret
 ENDPROC(kvm_call_hyp)
 
@@ -1142,6 +1140,7 @@ el1_sync:					// Guest trapped into EL2
 
 	mrs	x1, esr_el2
 	lsr	x2, x1, #ESR_ELx_EC_SHIFT
+	and	x0, x1, #ESR_ELx_ISS_MASK
 
 	cmp	x2, #ESR_ELx_EC_HVC64
 	b.ne	el1_trap
@@ -1150,15 +1149,19 @@ el1_sync:					// Guest trapped into EL2
 	cbnz	x3, el1_trap			// called HVC
 
 	/* Here, we're pretty sure the host called HVC. */
+	mov	x18, x0
 	pop	x2, x3
 	pop	x0, x1
 
-	/* Check for __hyp_get_vectors */
-	cbnz	x0, 1f
+	cmp	x18, #HVC_GET_VECTORS
+	b.ne	1f
 	mrs	x0, vbar_el2
 	b	2f
 
-1:	push	lr, xzr
+1:	cmp	x18, #HVC_CALL_HYP
+	b.ne	2f
+
+	push	lr, xzr
 
 	/*
 	 * Compute the function address in EL2, and shuffle the parameters.
@@ -1171,6 +1174,7 @@ el1_sync:					// Guest trapped into EL2
 	blr	lr
 
 	pop	lr, xzr
+
 2:	eret
 
 el1_trap:
-- 
2.1.0




From geoff at infradead.org  Fri Jan 16 16:23:34 2015
From: geoff at infradead.org (Geoff Levand)
Date: Sat, 17 Jan 2015 00:23:34 +0000
Subject: [PATCH 0/8] arm64 kexec kernel patches V7
In-Reply-To: <cover.1415926876.git.geoff@infradead.orgg>
References: <cover.1415926876.git.geoff@infradead.orgg>
Message-ID: <cover.1421449714.git.geoff@infradead.org>

Hi All,

This series adds the core support for kexec re-boots on arm64.  This v7 of the
series is mainly just a rebase to the latest arm64 for-next/core branch
(v3.19-rc4), and a few very minor changes requested for v6.

I have tested with the ARM VE fast model, the ARM Base model and the ARM
Foundation model with various kernel config options for both the first and
second stage kernels.

To load a second stage kernel and execute a kexec re-boot on arm64 my patches to
kexec-tools [2], which have not yet been merged upstream, are needed.

Patch 1 here moves proc-macros.S from arm64/mm to arm64/include/asm so that the
dcache_line_size macro it defines can be uesd by kexec's relocate kernel
routine.

Patches 2-4 rework the arm64 hcall mechanism to give the arm64 soft_restart()
routine the ability to switch exception levels from EL1 to EL2 for kernels that
were entered in EL2.

Patches 5-8 add the actual kexec support.

Please consider all patches for inclusion.

Note that the location of my development repositories has changed:

[1]  https://git.kernel.org/cgit/linux/kernel/git/geoff/linux-kexec.git
[2]  https://git.kernel.org/cgit/linux/kernel/git/geoff/kexec-tools.git

Several things are known to have problems on kexec re-boot:

spin-table
----------

PROBLEM: The spin-table enable method does not implement all the methods needed
for CPU hot-plug, so the first stage kernel cannot be shutdown properly.

WORK-AROUND: Upgrade to system firmware that provides PSCI enable method
support, OR build the first stage kernel with CONFIG_SMP=n, OR pass 'maxcpus=1'
on the first stage kernel command line.

FIX: Upgrade system firmware to provide PSCI enable method support or add
missing spin-table support to the kernel.

KVM
---

PROBLEM: KVM acquires hypervisor resources on startup, but does not free those
resources on shutdown, so the first stage kernel cannot be shutdown properly
when using kexec.

WORK-AROUND:  Build the first stage kernel with CONFIG_KVM=n.

FIX: Fix KVM to support soft_restart().  KVM needs to restore default exception
vectors, etc.

UEFI
----

PROBLEM: UEFI does not manage its runtime services virtual mappings in a way
that is compatible with a kexec re-boot, so the second stage kernel hangs on
boot-up.

WORK-AROUND:  Disable UEFI in firmware.

FIX: Ard Biesheuvel has done work to fix this.  Basic kexec re-boot has been
tested and works.  More comprehensive testing is needed.

/memreserve/
----------

PROBLEM: The use of device tree /memreserve/ entries is not compatible with
kexec re-boot.  The second stage kernel will use the reserved regions and the
system will become unstable.

WORK-AROUND: Pass a user specified DTB using the kexec --dtb option.

FIX: An interface to expose a binary device tree to user space has been
proposed.  User kexec utilities will need to be updated to add support for this
new interface.

ACPI
----

PROBLEM: The kernel for ACPI based systems does not export a device tree to the
standard user space location of 'proc/device-tree'.  Current applications
expect to access device tree information from this standard location.

WORK-AROUND: Disable ACPI in firmware, OR pass 'acpi=off' on the first stage
kernel command line, OR pass a user specified DTB using the kexec --dtb option.

FIX: FIX: An interface to expose a binary device tree to user space has been
proposed.  User kexec utilities will need to be updated to add support for this
new interface.

----------------------------------------------------------------
The following changes since commit 6083fe74b7bfffc2c7be8c711596608bda0cda6e:

  arm64: respect mem= for EFI (2015-01-16 16:21:58 +0000)

are available in the git repository at:

  git://git.kernel.org/pub/scm/linux/kernel/git/geoff/linux-kexec.git kexec-v7

for you to fetch changes up to 7db998d9533d5efab816edfad3d7010fc2f7e62c:

  arm64/kexec: Enable kexec in the arm64 defconfig (2015-01-16 14:55:28 -0800)

----------------------------------------------------------------
Geoff Levand (8):
      arm64: Move proc-macros.S to include/asm
      arm64: Convert hcalls to use ISS field
      arm64: Add new hcall HVC_CALL_FUNC
      arm64: Add EL2 switch to soft_restart
      arm64/kexec: Add core kexec support
      arm64/kexec: Add pr_devel output
      arm64/kexec: Add checks for KVM
      arm64/kexec: Enable kexec in the arm64 defconfig

 arch/arm64/Kconfig                           |   9 ++
 arch/arm64/configs/defconfig                 |   1 +
 arch/arm64/include/asm/kexec.h               |  47 ++++++
 arch/arm64/include/asm/proc-fns.h            |   4 +-
 arch/arm64/{mm => include/asm}/proc-macros.S |   0
 arch/arm64/include/asm/virt.h                |  33 ++++
 arch/arm64/kernel/Makefile                   |   1 +
 arch/arm64/kernel/hyp-stub.S                 |  45 ++++--
 arch/arm64/kernel/machine_kexec.c            | 219 +++++++++++++++++++++++++++
 arch/arm64/kernel/process.c                  |  10 +-
 arch/arm64/kernel/relocate_kernel.S          | 160 +++++++++++++++++++
 arch/arm64/kvm/hyp.S                         |  18 ++-
 arch/arm64/mm/cache.S                        |   3 +-
 arch/arm64/mm/proc.S                         |  50 ++++--
 include/uapi/linux/kexec.h                   |   1 +
 15 files changed, 563 insertions(+), 38 deletions(-)
 create mode 100644 arch/arm64/include/asm/kexec.h
 rename arch/arm64/{mm => include/asm}/proc-macros.S (100%)
 create mode 100644 arch/arm64/kernel/machine_kexec.c
 create mode 100644 arch/arm64/kernel/relocate_kernel.S

-- 
2.1.0



From geoff at infradead.org  Fri Jan 16 16:23:34 2015
From: geoff at infradead.org (Geoff Levand)
Date: Sat, 17 Jan 2015 00:23:34 +0000
Subject: [PATCH 7/8] arm64/kexec: Add checks for KVM
In-Reply-To: <cover.1421449714.git.geoff@infradead.org>
References: <cover.1421449714.git.geoff@infradead.org>
Message-ID: <a3353cf2ac47f49c20bbbb88405301cbaa359805.1421449714.git.geoff@infradead.org>

Add runtime checks that fail the arm64 kexec syscall for situations that would
result in system instability do to problems in the KVM kernel support.
These checks should be removed when the KVM problems are resolved fixed.

Signed-off-by: Geoff Levand <geoff at infradead.org>
---
 arch/arm64/kernel/machine_kexec.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c
index 3d84759..a36459d 100644
--- a/arch/arm64/kernel/machine_kexec.c
+++ b/arch/arm64/kernel/machine_kexec.c
@@ -16,6 +16,9 @@
 #include <asm/cacheflush.h>
 #include <asm/system_misc.h>
 
+/* TODO: Remove this include when KVM can support a kexec reboot. */
+#include <asm/virt.h>
+
 /* Global variables for the relocate_kernel routine. */
 extern const unsigned char relocate_new_kernel[];
 extern const unsigned long relocate_new_kernel_size;
@@ -100,6 +103,13 @@ int machine_kexec_prepare(struct kimage *image)
 
 	kexec_image_info(image);
 
+	/* TODO: Remove this message when KVM can support a kexec reboot. */
+	if (IS_ENABLED(CONFIG_KVM) && is_hyp_mode_available()) {
+		pr_err("%s: Your kernel is configured with KVM support (CONFIG_KVM=y) which currently does not allow for kexec re-boot.\n",
+			__func__);
+		return -ENOSYS;
+	}
+
 	return 0;
 }
 
-- 
2.1.0




From geoff at infradead.org  Fri Jan 16 16:23:34 2015
From: geoff at infradead.org (Geoff Levand)
Date: Sat, 17 Jan 2015 00:23:34 +0000
Subject: [PATCH 1/8] arm64: Move proc-macros.S to include/asm
In-Reply-To: <cover.1421449714.git.geoff@infradead.org>
References: <cover.1421449714.git.geoff@infradead.org>
Message-ID: <e81815fb841db3ea40ce8e2fd18cf021ec9c6cf7.1421449714.git.geoff@infradead.org>

To allow the assembler macros defined in proc-macros.S to be used outside
the mm code move the proc-macros.S file from arch/arm64/mm/ to
arch/arm64/include/asm/ and fix up any preprocessor includes to use the new
file location.

Signed-off-by: Geoff Levand <geoff at infradead.org>
---
 arch/arm64/include/asm/proc-macros.S | 54 ++++++++++++++++++++++++++++++++++++
 arch/arm64/mm/cache.S                |  3 +-
 arch/arm64/mm/proc-macros.S          | 54 ------------------------------------
 arch/arm64/mm/proc.S                 |  3 +-
 4 files changed, 56 insertions(+), 58 deletions(-)
 create mode 100644 arch/arm64/include/asm/proc-macros.S
 delete mode 100644 arch/arm64/mm/proc-macros.S

diff --git a/arch/arm64/include/asm/proc-macros.S b/arch/arm64/include/asm/proc-macros.S
new file mode 100644
index 0000000..005d29e
--- /dev/null
+++ b/arch/arm64/include/asm/proc-macros.S
@@ -0,0 +1,54 @@
+/*
+ * Based on arch/arm/mm/proc-macros.S
+ *
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <asm/asm-offsets.h>
+#include <asm/thread_info.h>
+
+/*
+ * vma_vm_mm - get mm pointer from vma pointer (vma->vm_mm)
+ */
+	.macro	vma_vm_mm, rd, rn
+	ldr	\rd, [\rn, #VMA_VM_MM]
+	.endm
+
+/*
+ * mmid - get context id from mm pointer (mm->context.id)
+ */
+	.macro	mmid, rd, rn
+	ldr	\rd, [\rn, #MM_CONTEXT_ID]
+	.endm
+
+/*
+ * dcache_line_size - get the minimum D-cache line size from the CTR register.
+ */
+	.macro	dcache_line_size, reg, tmp
+	mrs	\tmp, ctr_el0			// read CTR
+	ubfm	\tmp, \tmp, #16, #19		// cache line size encoding
+	mov	\reg, #4			// bytes per word
+	lsl	\reg, \reg, \tmp		// actual cache line size
+	.endm
+
+/*
+ * icache_line_size - get the minimum I-cache line size from the CTR register.
+ */
+	.macro	icache_line_size, reg, tmp
+	mrs	\tmp, ctr_el0			// read CTR
+	and	\tmp, \tmp, #0xf		// cache line size encoding
+	mov	\reg, #4			// bytes per word
+	lsl	\reg, \reg, \tmp		// actual cache line size
+	.endm
diff --git a/arch/arm64/mm/cache.S b/arch/arm64/mm/cache.S
index 2560e1e..4fcfffa 100644
--- a/arch/arm64/mm/cache.S
+++ b/arch/arm64/mm/cache.S
@@ -23,8 +23,7 @@
 #include <asm/assembler.h>
 #include <asm/cpufeature.h>
 #include <asm/alternative-asm.h>
-
-#include "proc-macros.S"
+#include <asm/proc-macros.S>
 
 /*
  *	__flush_dcache_all()
diff --git a/arch/arm64/mm/proc-macros.S b/arch/arm64/mm/proc-macros.S
deleted file mode 100644
index 005d29e..0000000
--- a/arch/arm64/mm/proc-macros.S
+++ /dev/null
@@ -1,54 +0,0 @@
-/*
- * Based on arch/arm/mm/proc-macros.S
- *
- * Copyright (C) 2012 ARM Ltd.
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program.  If not, see <http://www.gnu.org/licenses/>.
- */
-
-#include <asm/asm-offsets.h>
-#include <asm/thread_info.h>
-
-/*
- * vma_vm_mm - get mm pointer from vma pointer (vma->vm_mm)
- */
-	.macro	vma_vm_mm, rd, rn
-	ldr	\rd, [\rn, #VMA_VM_MM]
-	.endm
-
-/*
- * mmid - get context id from mm pointer (mm->context.id)
- */
-	.macro	mmid, rd, rn
-	ldr	\rd, [\rn, #MM_CONTEXT_ID]
-	.endm
-
-/*
- * dcache_line_size - get the minimum D-cache line size from the CTR register.
- */
-	.macro	dcache_line_size, reg, tmp
-	mrs	\tmp, ctr_el0			// read CTR
-	ubfm	\tmp, \tmp, #16, #19		// cache line size encoding
-	mov	\reg, #4			// bytes per word
-	lsl	\reg, \reg, \tmp		// actual cache line size
-	.endm
-
-/*
- * icache_line_size - get the minimum I-cache line size from the CTR register.
- */
-	.macro	icache_line_size, reg, tmp
-	mrs	\tmp, ctr_el0			// read CTR
-	and	\tmp, \tmp, #0xf		// cache line size encoding
-	mov	\reg, #4			// bytes per word
-	lsl	\reg, \reg, \tmp		// actual cache line size
-	.endm
diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S
index 4e778b1..c507e25 100644
--- a/arch/arm64/mm/proc.S
+++ b/arch/arm64/mm/proc.S
@@ -25,8 +25,7 @@
 #include <asm/hwcap.h>
 #include <asm/pgtable-hwdef.h>
 #include <asm/pgtable.h>
-
-#include "proc-macros.S"
+#include <asm/proc-macros.S>
 
 #ifdef CONFIG_ARM64_64K_PAGES
 #define TCR_TG_FLAGS	TCR_TG0_64K | TCR_TG1_64K
-- 
2.1.0




From geoff at infradead.org  Fri Jan 16 16:23:34 2015
From: geoff at infradead.org (Geoff Levand)
Date: Sat, 17 Jan 2015 00:23:34 +0000
Subject: [PATCH 4/8] arm64: Add EL2 switch to soft_restart
In-Reply-To: <cover.1421449714.git.geoff@infradead.org>
References: <cover.1421449714.git.geoff@infradead.org>
Message-ID: <bb44d4eee4fab15278e4a8829a0981878bc9acd7.1421449714.git.geoff@infradead.org>

When a CPU is reset it needs to be put into the exception level it had when it
entered the kernel.  Update cpu_reset() to accept an argument el2_switch which
signals cpu_reset() to enter the soft reset address at EL2.  If el2_switch is
not set the soft reset address will be entered at EL1.

Update cpu_soft_restart() and soft_restart() to pass the return of
is_hyp_mode_available() as the el2_switch value to cpu_reset().  Also update the
comments of cpu_reset(), cpu_soft_restart() and soft_restart() to reflect this
change.

Signed-off-by: Geoff Levand <geoff at infradead.org>
---
 arch/arm64/include/asm/proc-fns.h |  4 ++--
 arch/arm64/kernel/process.c       | 10 ++++++++-
 arch/arm64/mm/proc.S              | 47 +++++++++++++++++++++++++++++----------
 3 files changed, 46 insertions(+), 15 deletions(-)

diff --git a/arch/arm64/include/asm/proc-fns.h b/arch/arm64/include/asm/proc-fns.h
index 9a8fd84..339394d 100644
--- a/arch/arm64/include/asm/proc-fns.h
+++ b/arch/arm64/include/asm/proc-fns.h
@@ -32,8 +32,8 @@ extern void cpu_cache_off(void);
 extern void cpu_do_idle(void);
 extern void cpu_do_switch_mm(unsigned long pgd_phys, struct mm_struct *mm);
 extern void cpu_reset(unsigned long addr) __attribute__((noreturn));
-void cpu_soft_restart(phys_addr_t cpu_reset,
-		unsigned long addr) __attribute__((noreturn));
+void cpu_soft_restart(phys_addr_t cpu_reset, unsigned long el2_switch,
+		      unsigned long addr) __attribute__((noreturn));
 extern void cpu_do_suspend(struct cpu_suspend_ctx *ptr);
 extern u64 cpu_do_resume(phys_addr_t ptr, u64 idmap_ttbr);
 
diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c
index fde9923..371bbf1 100644
--- a/arch/arm64/kernel/process.c
+++ b/arch/arm64/kernel/process.c
@@ -50,6 +50,7 @@
 #include <asm/mmu_context.h>
 #include <asm/processor.h>
 #include <asm/stacktrace.h>
+#include <asm/virt.h>
 
 #ifdef CONFIG_CC_STACKPROTECTOR
 #include <linux/stackprotector.h>
@@ -60,7 +61,14 @@ EXPORT_SYMBOL(__stack_chk_guard);
 void soft_restart(unsigned long addr)
 {
 	setup_mm_for_reboot();
-	cpu_soft_restart(virt_to_phys(cpu_reset), addr);
+
+	/* TODO: Remove this conditional when KVM can support CPU restart. */
+	if (IS_ENABLED(CONFIG_KVM))
+		cpu_soft_restart(virt_to_phys(cpu_reset), 0, addr);
+	else
+		cpu_soft_restart(virt_to_phys(cpu_reset),
+				 is_hyp_mode_available(), addr);
+
 	/* Should never get here */
 	BUG();
 }
diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S
index c507e25..b767032 100644
--- a/arch/arm64/mm/proc.S
+++ b/arch/arm64/mm/proc.S
@@ -26,6 +26,7 @@
 #include <asm/pgtable-hwdef.h>
 #include <asm/pgtable.h>
 #include <asm/proc-macros.S>
+#include <asm/virt.h>
 
 #ifdef CONFIG_ARM64_64K_PAGES
 #define TCR_TG_FLAGS	TCR_TG0_64K | TCR_TG1_64K
@@ -58,27 +59,48 @@ ENTRY(cpu_cache_off)
 ENDPROC(cpu_cache_off)
 
 /*
- *	cpu_reset(loc)
+ * cpu_reset(el2_switch, loc) - Helper for cpu_soft_restart.
  *
- *	Perform a soft reset of the system.  Put the CPU into the same state
- *	as it would be if it had been reset, and branch to what would be the
- *	reset vector. It must be executed with the flat identity mapping.
+ * @cpu_reset: Physical address of the cpu_reset routine.
+ * @el2_switch: Flag to indicate a swich to EL2 is needed.
+ * @addr: Location to jump to for soft reset.
  *
- *	- loc   - location to jump to for soft reset
+ * Put the CPU into the same state as it would be if it had been reset, and
+ * branch to what would be the reset vector. It must be executed with the
+ * flat identity mapping.
  */
+
 	.align	5
+
 ENTRY(cpu_reset)
-	mrs	x1, sctlr_el1
-	bic	x1, x1, #1
-	msr	sctlr_el1, x1			// disable the MMU
+	mrs	x2, sctlr_el1
+	bic	x2, x2, #1
+	msr	sctlr_el1, x2			// disable the MMU
 	isb
-	ret	x0
+
+	cbz	x0, 1f				// el2_switch?
+	mov	x0, x1
+	mov	x1, xzr
+	mov	x2, xzr
+	mov	x3, xzr
+	hvc	#HVC_CALL_FUNC			// no return
+
+1:	ret	x1
 ENDPROC(cpu_reset)
 
+/*
+ * cpu_soft_restart(cpu_reset, el2_switch, addr) - Perform a cpu soft reset.
+ *
+ * @cpu_reset: Physical address of the cpu_reset routine.
+ * @el2_switch: Flag to indicate a swich to EL2 is needed, passed to cpu_reset.
+ * @addr: Location to jump to for soft reset, passed to cpu_reset.
+ *
+ */
+
 ENTRY(cpu_soft_restart)
-	/* Save address of cpu_reset() and reset address */
-	mov	x19, x0
-	mov	x20, x1
+	mov	x19, x0				// cpu_reset
+	mov	x20, x1				// el2_switch
+	mov	x21, x2				// addr
 
 	/* Turn D-cache off */
 	bl	cpu_cache_off
@@ -87,6 +109,7 @@ ENTRY(cpu_soft_restart)
 	bl	flush_cache_all
 
 	mov	x0, x20
+	mov	x1, x21
 	ret	x19
 ENDPROC(cpu_soft_restart)
 
-- 
2.1.0




From geoff at infradead.org  Fri Jan 16 16:23:34 2015
From: geoff at infradead.org (Geoff Levand)
Date: Sat, 17 Jan 2015 00:23:34 +0000
Subject: [PATCH 8/8] arm64/kexec: Enable kexec in the arm64 defconfig
In-Reply-To: <cover.1421449714.git.geoff@infradead.org>
References: <cover.1421449714.git.geoff@infradead.org>
Message-ID: <7db998d9533d5efab816edfad3d7010fc2f7e62c.1421449714.git.geoff@infradead.org>

Signed-off-by: Geoff Levand <geoff at infradead.org>
---
 arch/arm64/configs/defconfig | 1 +
 1 file changed, 1 insertion(+)

diff --git a/arch/arm64/configs/defconfig b/arch/arm64/configs/defconfig
index 5376d90..85285dc 100644
--- a/arch/arm64/configs/defconfig
+++ b/arch/arm64/configs/defconfig
@@ -42,6 +42,7 @@ CONFIG_PREEMPT=y
 CONFIG_KSM=y
 CONFIG_TRANSPARENT_HUGEPAGE=y
 CONFIG_CMA=y
+CONFIG_KEXEC=y
 CONFIG_CMDLINE="console=ttyAMA0"
 # CONFIG_CORE_DUMP_DEFAULT_ELF_HEADERS is not set
 CONFIG_COMPAT=y
-- 
2.1.0



From geoff at infradead.org  Fri Jan 16 16:23:34 2015
From: geoff at infradead.org (Geoff Levand)
Date: Sat, 17 Jan 2015 00:23:34 +0000
Subject: [PATCH 3/8] arm64: Add new hcall HVC_CALL_FUNC
In-Reply-To: <cover.1421449714.git.geoff@infradead.org>
References: <cover.1421449714.git.geoff@infradead.org>
Message-ID: <3f1795bc1dad1d0510a9d24a168ecd5d83666507.1421449714.git.geoff@infradead.org>

Add the new hcall HVC_CALL_FUNC that allows execution of a function at EL2.
During CPU reset the CPU must be brought to the exception level it had on
entry to the kernel.  The HVC_CALL_FUNC hcall will provide the mechanism
needed for this exception level switch.

Signed-off-by: Geoff Levand <geoff at infradead.org>
---
 arch/arm64/include/asm/virt.h | 13 +++++++++++++
 arch/arm64/kernel/hyp-stub.S  | 17 ++++++++++++++---
 2 files changed, 27 insertions(+), 3 deletions(-)

diff --git a/arch/arm64/include/asm/virt.h b/arch/arm64/include/asm/virt.h
index 99c319c..4f23a48 100644
--- a/arch/arm64/include/asm/virt.h
+++ b/arch/arm64/include/asm/virt.h
@@ -41,6 +41,19 @@
 
 #define HVC_CALL_HYP 3
 
+/*
+ * HVC_CALL_FUNC - Execute a function at EL2.
+ *
+ * @x0: Physical address of the function to be executed.
+ * @x1: Passed as the first argument to the function.
+ * @x2: Passed as the second argument to the function.
+ * @x3: Passed as the third argument to the function.
+ *
+ * The called function must preserve the contents of register x18.
+ */
+
+#define HVC_CALL_FUNC 4
+
 #ifndef __ASSEMBLY__
 
 /*
diff --git a/arch/arm64/kernel/hyp-stub.S b/arch/arm64/kernel/hyp-stub.S
index e3db3fd..b5d36e7 100644
--- a/arch/arm64/kernel/hyp-stub.S
+++ b/arch/arm64/kernel/hyp-stub.S
@@ -66,9 +66,20 @@ el1_sync:
 	mrs	x0, vbar_el2
 	b	2f
 
-1:	cmp	x18, #HVC_SET_VECTORS
-	b.ne	2f
-	msr	vbar_el2, x0
+1:	cmp     x18, #HVC_SET_VECTORS
+	b.ne    1f
+	msr     vbar_el2, x0
+	b       2f
+
+1:	cmp     x18, #HVC_CALL_FUNC
+	b.ne    2f
+	mov     x18, lr
+	mov     lr, x0
+	mov     x0, x1
+	mov     x1, x2
+	mov     x2, x3
+	blr     lr
+	mov     lr, x18
 
 2:	eret
 ENDPROC(el1_sync)
-- 
2.1.0




From geoff at infradead.org  Fri Jan 16 16:23:34 2015
From: geoff at infradead.org (Geoff Levand)
Date: Sat, 17 Jan 2015 00:23:34 +0000
Subject: [PATCH 5/8] arm64/kexec: Add core kexec support
In-Reply-To: <cover.1421449714.git.geoff@infradead.org>
References: <cover.1421449714.git.geoff@infradead.org>
Message-ID: <a5c232448c1d8fa1da73a9005fa9f3927c635578.1421449714.git.geoff@infradead.org>

Add three new files, kexec.h, machine_kexec.c and relocate_kernel.S to the
arm64 architecture that add support for the kexec re-boot mechanism
(CONFIG_KEXEC) on arm64 platforms.

Signed-off-by: Geoff Levand <geoff at infradead.org>
---
 arch/arm64/Kconfig                  |   9 ++
 arch/arm64/include/asm/kexec.h      |  47 +++++++++++
 arch/arm64/kernel/Makefile          |   1 +
 arch/arm64/kernel/machine_kexec.c   | 155 ++++++++++++++++++++++++++++++++++
 arch/arm64/kernel/relocate_kernel.S | 160 ++++++++++++++++++++++++++++++++++++
 include/uapi/linux/kexec.h          |   1 +
 6 files changed, 373 insertions(+)
 create mode 100644 arch/arm64/include/asm/kexec.h
 create mode 100644 arch/arm64/kernel/machine_kexec.c
 create mode 100644 arch/arm64/kernel/relocate_kernel.S

diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index b1f9a20..d9eb9cd 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -474,6 +474,15 @@ config SECCOMP
 	  and the task is only allowed to execute a few safe syscalls
 	  defined by each seccomp mode.
 
+config KEXEC
+	depends on (!SMP || PM_SLEEP_SMP)
+	bool "kexec system call"
+	---help---
+	  kexec is a system call that implements the ability to shutdown your
+	  current kernel, and to start another kernel.  It is like a reboot
+	  but it is independent of the system firmware.   And like a reboot
+	  you can start any kernel with it, not just Linux.
+
 config XEN_DOM0
 	def_bool y
 	depends on XEN
diff --git a/arch/arm64/include/asm/kexec.h b/arch/arm64/include/asm/kexec.h
new file mode 100644
index 0000000..e7bd7ab
--- /dev/null
+++ b/arch/arm64/include/asm/kexec.h
@@ -0,0 +1,47 @@
+/*
+ * kexec for arm64
+ *
+ * Copyright (C) Linaro.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#if !defined(_ARM64_KEXEC_H)
+#define _ARM64_KEXEC_H
+
+/* Maximum physical address we can use pages from */
+
+#define KEXEC_SOURCE_MEMORY_LIMIT (-1UL)
+
+/* Maximum address we can reach in physical address mode */
+
+#define KEXEC_DESTINATION_MEMORY_LIMIT (-1UL)
+
+/* Maximum address we can use for the control code buffer */
+
+#define KEXEC_CONTROL_MEMORY_LIMIT (-1UL)
+
+#define KEXEC_CONTROL_PAGE_SIZE	4096
+
+#define KEXEC_ARCH KEXEC_ARCH_ARM64
+
+#if !defined(__ASSEMBLY__)
+
+/**
+ * crash_setup_regs() - save registers for the panic kernel
+ *
+ * @newregs: registers are saved here
+ * @oldregs: registers to be saved (may be %NULL)
+ */
+
+static inline void crash_setup_regs(struct pt_regs *newregs,
+				    struct pt_regs *oldregs)
+{
+	/* Empty routine needed to avoid build errors. */
+}
+
+#endif /* !defined(__ASSEMBLY__) */
+
+#endif
diff --git a/arch/arm64/kernel/Makefile b/arch/arm64/kernel/Makefile
index 77d3d95..eff1625 100644
--- a/arch/arm64/kernel/Makefile
+++ b/arch/arm64/kernel/Makefile
@@ -34,6 +34,7 @@ arm64-obj-$(CONFIG_KGDB)		+= kgdb.o
 arm64-obj-$(CONFIG_EFI)			+= efi.o efi-stub.o efi-entry.o
 arm64-obj-$(CONFIG_PCI)			+= pci.o
 arm64-obj-$(CONFIG_ARMV8_DEPRECATED)	+= armv8_deprecated.o
+arm64-obj-$(CONFIG_KEXEC)		+= machine_kexec.o relocate_kernel.o
 
 obj-y					+= $(arm64-obj-y) vdso/
 obj-m					+= $(arm64-obj-m)
diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c
new file mode 100644
index 0000000..b0e5d76
--- /dev/null
+++ b/arch/arm64/kernel/machine_kexec.c
@@ -0,0 +1,155 @@
+/*
+ * kexec for arm64
+ *
+ * Copyright (C) Linaro.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include <linux/kexec.h>
+#include <linux/of_fdt.h>
+#include <linux/slab.h>
+#include <linux/uaccess.h>
+
+#include <asm/cacheflush.h>
+#include <asm/system_misc.h>
+
+/* Global variables for the relocate_kernel routine. */
+extern const unsigned char relocate_new_kernel[];
+extern const unsigned long relocate_new_kernel_size;
+extern unsigned long arm64_kexec_dtb_addr;
+extern unsigned long arm64_kexec_kimage_head;
+extern unsigned long arm64_kexec_kimage_start;
+
+/**
+ * kexec_is_dtb - Helper routine to check the device tree header signature.
+ */
+static bool kexec_is_dtb(const void *dtb)
+{
+	__be32 magic;
+
+	return get_user(magic, (__be32 *)dtb) ? false :
+		(be32_to_cpu(magic) == OF_DT_HEADER);
+}
+
+/**
+ * kexec_find_dtb_seg - Helper routine to find the dtb segment.
+ */
+static const struct kexec_segment *kexec_find_dtb_seg(
+	const struct kimage *image)
+{
+	int i;
+
+	for (i = 0; i < image->nr_segments; i++) {
+		if (kexec_is_dtb(image->segment[i].buf))
+			return &image->segment[i];
+	}
+
+	return NULL;
+}
+
+void machine_kexec_cleanup(struct kimage *image)
+{
+	/* Empty routine needed to avoid build errors. */
+}
+
+/**
+ * machine_kexec_prepare - Prepare for a kexec reboot.
+ *
+ * Called from the core kexec code when a kernel image is loaded.
+ */
+int machine_kexec_prepare(struct kimage *image)
+{
+	const struct kexec_segment *dtb_seg = kexec_find_dtb_seg(image);
+
+	arm64_kexec_dtb_addr = dtb_seg ? dtb_seg->mem : 0;
+	arm64_kexec_kimage_start = image->start;
+
+	return 0;
+}
+
+/**
+ * kexec_list_flush - Helper to flush the kimage list to PoC.
+ */
+static void kexec_list_flush(unsigned long kimage_head)
+{
+	void *dest;
+	unsigned long *entry;
+
+	for (entry = &kimage_head, dest = NULL; ; entry++) {
+		unsigned int flag = *entry &
+			(IND_DESTINATION | IND_INDIRECTION | IND_DONE |
+			IND_SOURCE);
+		void *addr = phys_to_virt(*entry & PAGE_MASK);
+
+		switch (flag) {
+		case IND_INDIRECTION:
+			entry = (unsigned long *)addr - 1;
+			__flush_dcache_area(addr, PAGE_SIZE);
+			break;
+		case IND_DESTINATION:
+			dest = addr;
+			break;
+		case IND_SOURCE:
+			__flush_dcache_area(addr, PAGE_SIZE);
+			dest += PAGE_SIZE;
+			break;
+		case IND_DONE:
+			return;
+		default:
+			BUG();
+		}
+	}
+}
+
+/**
+ * machine_kexec - Do the kexec reboot.
+ *
+ * Called from the core kexec code for a sys_reboot with LINUX_REBOOT_CMD_KEXEC.
+ */
+void machine_kexec(struct kimage *image)
+{
+	phys_addr_t reboot_code_buffer_phys;
+	void *reboot_code_buffer;
+
+	BUG_ON(num_online_cpus() > 1);
+
+	arm64_kexec_kimage_head = image->head;
+
+	reboot_code_buffer_phys = page_to_phys(image->control_code_page);
+	reboot_code_buffer = phys_to_virt(reboot_code_buffer_phys);
+
+	/*
+	 * Copy relocate_new_kernel to the reboot_code_buffer for use
+	 * after the kernel is shut down.
+	 */
+	memcpy(reboot_code_buffer, relocate_new_kernel,
+		relocate_new_kernel_size);
+
+	/* Flush the reboot_code_buffer in preparation for its execution. */
+	__flush_dcache_area(reboot_code_buffer, relocate_new_kernel_size);
+
+	/* Flush the kimage list. */
+	kexec_list_flush(image->head);
+
+	pr_info("Bye!\n");
+
+	/* Disable all DAIF exceptions. */
+	asm volatile ("msr daifset, #0xf" : : : "memory");
+
+	/*
+	 * soft_restart() will shutdown the MMU, disable data caches, then
+	 * transfer control to the reboot_code_buffer which contains a copy of
+	 * the relocate_new_kernel routine.  relocate_new_kernel will use
+	 * physical addressing to relocate the new kernel to its final position
+	 * and then will transfer control to the entry point of the new kernel.
+	 */
+	soft_restart(reboot_code_buffer_phys);
+}
+
+void machine_crash_shutdown(struct pt_regs *regs)
+{
+	/* Empty routine needed to avoid build errors. */
+}
diff --git a/arch/arm64/kernel/relocate_kernel.S b/arch/arm64/kernel/relocate_kernel.S
new file mode 100644
index 0000000..1c1514d
--- /dev/null
+++ b/arch/arm64/kernel/relocate_kernel.S
@@ -0,0 +1,160 @@
+/*
+ * kexec for arm64
+ *
+ * Copyright (C) Linaro.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include <asm/assembler.h>
+#include <asm/kexec.h>
+#include <asm/memory.h>
+#include <asm/page.h>
+#include <asm/proc-macros.S>
+
+/* The list entry flags. */
+
+#define IND_DESTINATION_BIT 0
+#define IND_INDIRECTION_BIT 1
+#define IND_DONE_BIT        2
+#define IND_SOURCE_BIT      3
+
+/*
+ * relocate_new_kernel - Put a 2nd stage kernel image in place and boot it.
+ *
+ * The memory that the old kernel occupies may be overwritten when coping the
+ * new image to its final location.  To assure that the relocate_new_kernel
+ * routine which does that copy is not overwritten all code and data needed
+ * by relocate_new_kernel must be between the symbols relocate_new_kernel and
+ * relocate_new_kernel_end.  The machine_kexec() routine will copy
+ * relocate_new_kernel to the kexec control_code_page, a special page which
+ * has been set up to be preserved during the copy operation.
+ */
+.globl relocate_new_kernel
+relocate_new_kernel:
+
+	/* Setup the list loop variables. */
+	ldr	x18, arm64_kexec_kimage_head	/* x18 = list entry */
+	dcache_line_size x17, x0		/* x17 = dcache line size */
+	mov	x16, xzr			/* x16 = segment start */
+	mov	x15, xzr			/* x15 = entry ptr */
+	mov	x14, xzr			/* x14 = copy dest */
+
+	/* Check if the new image needs relocation. */
+	cbz	x18, .Ldone
+	tbnz	x18, IND_DONE_BIT, .Ldone
+
+.Lloop:
+	and	x13, x18, PAGE_MASK		/* x13 = addr */
+
+	/* Test the entry flags. */
+.Ltest_source:
+	tbz	x18, IND_SOURCE_BIT, .Ltest_indirection
+
+	mov x20, x14				/*  x20 = copy dest */
+	mov x21, x13				/*  x21 = copy src */
+
+	/* Invalidate dest page to PoC. */
+	mov	x0, x20
+	add	x19, x0, #PAGE_SIZE
+	sub	x1, x17, #1
+	bic	x0, x0, x1
+1:	dc	ivac, x0
+	add	x0, x0, x17
+	cmp	x0, x19
+	b.lo	1b
+	dsb	sy
+
+	/* Copy page. */
+1:	ldp	x22, x23, [x21]
+	ldp	x24, x25, [x21, #16]
+	ldp	x26, x27, [x21, #32]
+	ldp	x28, x29, [x21, #48]
+	add	x21, x21, #64
+	stnp	x22, x23, [x20]
+	stnp	x24, x25, [x20, #16]
+	stnp	x26, x27, [x20, #32]
+	stnp	x28, x29, [x20, #48]
+	add	x20, x20, #64
+	tst	x21, #(PAGE_SIZE - 1)
+	b.ne	1b
+
+	/* dest += PAGE_SIZE */
+	add	x14, x14, PAGE_SIZE
+	b	.Lnext
+
+.Ltest_indirection:
+	tbz	x18, IND_INDIRECTION_BIT, .Ltest_destination
+
+	/* ptr = addr */
+	mov	x15, x13
+	b	.Lnext
+
+.Ltest_destination:
+	tbz	x18, IND_DESTINATION_BIT, .Lnext
+
+	mov	x16, x13
+
+	/* dest = addr */
+	mov	x14, x13
+
+.Lnext:
+	/* entry = *ptr++ */
+	ldr	x18, [x15], #8
+
+	/* while (!(entry & DONE)) */
+	tbz	x18, IND_DONE_BIT, .Lloop
+
+.Ldone:
+	dsb	sy
+	isb
+	ic	ialluis
+	dsb	sy
+	isb
+
+	/* Start new image. */
+	ldr	x4, arm64_kexec_kimage_start
+	ldr	x0, arm64_kexec_dtb_addr
+	mov	x1, xzr
+	mov	x2, xzr
+	mov	x3, xzr
+	br	x4
+
+.align 3	/* To keep the 64-bit values below naturally aligned. */
+
+/* The machine_kexec routines set these variables. */
+
+/*
+ * arm64_kexec_kimage_start - Copy of image->start, the entry point of the new
+ * image.
+ */
+.globl arm64_kexec_kimage_start
+arm64_kexec_kimage_start:
+	.quad	0x0
+
+/*
+ * arm64_kexec_dtb_addr - Physical address of a device tree.
+ */
+.globl arm64_kexec_dtb_addr
+arm64_kexec_dtb_addr:
+	.quad	0x0
+
+/*
+ * arm64_kexec_kimage_head - Copy of image->head, the list of kimage entries.
+ */
+.globl arm64_kexec_kimage_head
+arm64_kexec_kimage_head:
+	.quad	0x0
+
+.Lrelocate_new_kernel_end:
+
+/*
+ * relocate_new_kernel_size - Number of bytes to copy to the control_code_page.
+ */
+.globl relocate_new_kernel_size
+relocate_new_kernel_size:
+	.quad .Lrelocate_new_kernel_end - relocate_new_kernel
+
+.org	KEXEC_CONTROL_PAGE_SIZE
diff --git a/include/uapi/linux/kexec.h b/include/uapi/linux/kexec.h
index 6925f5b..04626b9 100644
--- a/include/uapi/linux/kexec.h
+++ b/include/uapi/linux/kexec.h
@@ -39,6 +39,7 @@
 #define KEXEC_ARCH_SH      (42 << 16)
 #define KEXEC_ARCH_MIPS_LE (10 << 16)
 #define KEXEC_ARCH_MIPS    ( 8 << 16)
+#define KEXEC_ARCH_ARM64   (183 << 16)
 
 /* The artificial cap on the number of segments passed to kexec_load. */
 #define KEXEC_SEGMENT_MAX 16
-- 
2.1.0




From horms at verge.net.au  Sun Jan 18 15:58:01 2015
From: horms at verge.net.au (Simon Horman)
Date: Mon, 19 Jan 2015 08:58:01 +0900
Subject: [ANNOUNCE] kexec-tools v2.0.9 preparation
Message-ID: <20150118235800.GB2306@verge.net.au>

Hi all,

I am planning to release kexec-tools v2.0.9 around the time that
the v3.19 kernel is released.

As I estimate that the latter occur within the next three or four weeks I
would like to ask interested parties to send any patches they would like
included in v2.0.9 within the next two weeks so that I can make an rc release.

For reference, the (short) list of patches queued up since v2.0.8 is
as follows:

Revert "kexec, remove panic_on_warn kernel parameter from kdump situations"
kexec, remove panic_on_warn kernel parameter from kdump situations
kexec/fs2dt: Use slurp_file_len to avoid partial read of files
kexec: remove some re zero-assignment
kexec/fs2dt: Fix sorting of device tree
kexec: distribute kexec/arch/m68k/bootinfo.h
kexec-tools 2.0.8.git


From bp at alien8.de  Mon Jan 19 11:26:16 2015
From: bp at alien8.de (Borislav Petkov)
Date: Mon, 19 Jan 2015 20:26:16 +0100
Subject: [PATCH 0/3 v2] Fix kdump failures with crashkernel=high
In-Reply-To: <1420555874-11506-1-git-send-email-joro@8bytes.org>
References: <1420555874-11506-1-git-send-email-joro@8bytes.org>
Message-ID: <20150119192616.GB4398@pd.tnic>

Adding some more people to CC.

On Tue, Jan 06, 2015 at 03:51:11PM +0100, Joerg Roedel wrote:
> v1->v2:
> 
> * Updated comments based on feedback from Konrad
> * Added Acked-bys
> * Rebased to v3.19-rc3
> 
> Hi,
> 
> here is a patch-set to fix failed kdump kernel boots when
> the systems was booted with crashkernel=X,high. On those
> systems the kernel allocates only 72MiB of low-memory for
> DMA buffers, which showed to be too low on some systems.
> 
> The problem is that 64MiB of the low-memory is allocated by
> swiotlb, leaving 8MB for the page-allocator. But swiotlb
> tries to allocate DMA memory from the page-allocator first,
> which fails pretty fast in the boot sequence, causing
> warnings. This patch-set removes these warnings.
> 
> But even the 64MiB for swiotlb are eaten up on some systems,
> so that the default of low-memory allocated for the
> crash-kernel is increase from 72MB to 256MB (only changing
> the defaults, can still be overwritten by crashkernel=X,low).
> 
> This number comes from experiments on the affected systems,
> 128MiB low-memory was still not enough there, thus I set the
> value to 256MiB to fix the issues.
> 
> Any feedback appreciated.
> 
> Thanks,
> 
> 	Joerg
> 
> Joerg Roedel (3):
>   swiotlb: Warn on allocation failure in swiotlb_alloc_coherent
>   x86, swiotlb: Try coherent allocations with __GFP_NOWARN
>   x86, crash: Allocate enough low-mem when crashkernel=high
> 
>  arch/x86/kernel/pci-swiotlb.c |  8 ++++++++
>  arch/x86/kernel/setup.c       |  5 ++++-
>  lib/swiotlb.c                 | 11 +++++++++--
>  3 files changed, 21 insertions(+), 3 deletions(-)
> 
> -- 
> 1.9.1
> 
> Joerg Roedel (3):
>   swiotlb: Warn on allocation failure in swiotlb_alloc_coherent
>   x86, swiotlb: Try coherent allocations with __GFP_NOWARN
>   x86, crash: Allocate enough low-mem when crashkernel=high
> 
>  arch/x86/kernel/pci-swiotlb.c |  8 ++++++++
>  arch/x86/kernel/setup.c       |  5 ++++-
>  lib/swiotlb.c                 | 11 +++++++++--
>  3 files changed, 21 insertions(+), 3 deletions(-)
> 
> -- 
> 1.9.1
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo at vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 
> 

-- 
Regards/Gruss,
    Boris.

ECO tip #101: Trim your mails when you reply.
--


From anurup.m at huawei.com  Thu Jan 22 02:40:46 2015
From: anurup.m at huawei.com (Anurup M)
Date: Thu, 22 Jan 2015 16:10:46 +0530
Subject: Fwd: [PATCH] arm64: Read reg property based on address-cells value.
In-Reply-To: <1421913506-18587-1-git-send-email-anurup.m@huawei.com>
References: <1421913506-18587-1-git-send-email-anurup.m@huawei.com>
Message-ID: <54C0D3AE.9040405@huawei.com>

Hi Geoff,

    I have faced an issue during dtb file matching for CPU reg property. The dtb file which
I use has 32 bit value in the reg property as the #address-cells is 1. The kexec-tools code
reads the reg as a 64bit values and reads a invalid value and hence the below comparison in
the function fixup_cpu_nodes fails.

449         result = read_cpu_info(&info_2, dtb_2);
-----------------------
463
464                         if (cp_1->hwid != cp_2->hwid)
465                                 continue;
466
467                         to_process--;
-----------------------
476         if (to_process) {
477                 fprintf(stderr, "kexec: %s:%d: Warning: Failed to process %u CPUs.\n",
478                         __func__, __LINE__, to_process);
479                 result = -EINVAL;
480                 goto on_exit;

I have written a patch to read the value based on the #address-cells. Please find the patch
below. Please share your comments.

Regards,
Anurup

 Signed-off-by: Anurup M <anurup.m at huawei.com>

---
 kexec/arch/arm64/kexec-arm64.c |   29 ++++++++++++++++++++++++++---
 1 file changed, 26 insertions(+), 3 deletions(-)

diff --git a/kexec/arch/arm64/kexec-arm64.c b/kexec/arch/arm64/kexec-arm64.c
index c1577be..e9c92ba 100644
--- a/kexec/arch/arm64/kexec-arm64.c
+++ b/kexec/arch/arm64/kexec-arm64.c
@@ -141,7 +141,7 @@ struct cpu_properties {
  */

 static int read_cpu_properties(struct cpu_properties *cp,
-	const struct dtb *dtb, int node_offset)
+	const struct dtb *dtb, int node_offset, int address_cells)
 {
 	int result;
 	const void *data;
@@ -158,7 +158,18 @@ static int read_cpu_properties(struct cpu_properties *cp,
 		return result;
 	}

-	cp->hwid = fdt64_to_cpu(*(uint64_t *)data);
+	/* Read the value based on the #address-cells */
+	if (1 == address_cells) {
+		cp->hwid = fdt32_to_cpu(*(uint32_t *)data);
+	}
+	else if (2 == address_cells) {
+		cp->hwid = fdt64_to_cpu(*(uint64_t *)data);
+	}
+	else {
+		fprintf(stderr, "kexec: %s:%d: Invalid #address-cells value: %d\n",
+                        __func__, __LINE__, address_cells);
+                return -1;
+	}

 	result = fdt_get_path(dtb->buf, node_offset, cp->node_path,
 		sizeof(cp->node_path));
@@ -361,6 +372,8 @@ static int read_cpu_info(struct cpu_info *info, const struct dtb *dtb)
 	int offset;
 	int result;
 	int depth;
+	const void *data;
+	int address_cells = -1;

 	offset = fdt_subnode_offset(dtb->buf, 0, "cpus");

@@ -370,6 +383,16 @@ static int read_cpu_info(struct cpu_info *info, const struct dtb *dtb)
 		return offset;
 	}

+	/* Read the #address-cells to read the CPU address from the reg property */
+	data = fdt_getprop(dtb->buf, offset, "#address-cells", &result);
+
+	if (!data) {
+		fprintf(stderr, "kexec: %s:%d: read #address-cells failed: %s\n",
+			__func__, __LINE__, fdt_strerror(result));
+		return result;
+	}
+       address_cells = fdt32_to_cpu(*(uint32_t *)data);
+
 	for (i = 0, depth = 0; ; i++) {

 		offset = fdt_next_node(dtb->buf, offset, &depth);
@@ -395,7 +418,7 @@ static int read_cpu_info(struct cpu_info *info, const struct dtb *dtb)
 			goto on_error;
 		}

-		result = read_cpu_properties(&info->cp[i], dtb, offset);
+		result = read_cpu_properties(&info->cp[i], dtb, offset, address_cells);

 		if (result)
 			goto on_error;
-- 
1.7.9.5


.






From bshands at exegy.com  Thu Jan 22 11:18:50 2015
From: bshands at exegy.com (Mr. Berkley Shands)
Date: Thu, 22 Jan 2015 13:18:50 -0600
Subject: Failure to crashdump under Centos-6.4 x86_64 with kernel 3.12.30
Message-ID: <54C14D1A.1020307@exegy.com>

My kernel crashdumps don't work with vanilla 3.12.30 on a 256GB x86_64
intel box.
Centos-6.4. With the standard makedumpfile, makedumpfile segfaults, and
no output is written to disk.
with makedumpfile 1.5.7, kexec-tools-2.0.8, and crash 7.0.2 I actually
get vmcore files,
but then crash rejects them with:

crash vmcore /boot/vmlinux-3.12.30

crash 7.0.2
...
GNU gdb (GDB) 7.6
...

crash: seek error: kernel virtual address: ffff88207fffcb60  type:
"pglist node_id"


the grub entry for this kernel is:
     kernel /boot/vmlinuz-3.12.30 ro root=LABEL=ROOT
numa_balancing=disable intel_idle.max_cstate=0 processor.max_cstate=0
mce=ignore_mce clocksource=tsc nohz=off nomodeset nodmraid rd_NO_LUKS
rd_NO_LVM rd_NO_MD rd_NO_DM LANG=en_US.UTF-8 SYSFONT=latarcyrheb-sun16
KEYBOARDTYPE=pc KEYTABLE=us crashkernel=256M at 256M  console=ttyS1,115200n8


CONFIG_CRASH_DUMP=y
CONFIG_KEXEC_JUMP=y
CONFIG_PHYSICAL_START=0x2000000
CONFIG_RELOCATABLE=y
CONFIG_PHYSICAL_ALIGN=0x1000000

A similar issue appears with a grub crashkernel=128M at 32M

Turning off all the makedumpfile exclusions (dump all pages) results in
a much larger
vmcore, but it still gets the same failures.

Where do I start looking for the problems here?
kernel 3.10.65 won't dump either, but an old 3.0.29 kernel dumps just fine.
Is this a problem with some .config setting?

Berkley

--

//E. F. Berkley Shands, MSc
Senior Software Architect/Engineer
Exegy Systems Engineering//

**Exegy Inc.**

349 Marshall Road, Suite 100

St. Louis , MO 63119

Direct: (314) 218-3600 X450

Cell: (314) 303-2546

Office: (314) 218-3600

Fax: (314) 218-3601

The Usual Disclaimer follows...


________________________________

This e-mail and any documents accompanying it may contain legally privileged and/or confidential information belonging to Exegy, Inc. Such information may be protected from disclosure by law. The information is intended for use by only the addressee. If you are not the intended recipient, you are hereby notified that any disclosure or use of the information is strictly prohibited. If you have received this e-mail in error, please immediately contact the sender by e-mail or phone regarding instructions for return or destruction and do not use or disclose the content to others.


From anderson at redhat.com  Thu Jan 22 12:50:24 2015
From: anderson at redhat.com (Dave Anderson)
Date: Thu, 22 Jan 2015 15:50:24 -0500 (EST)
Subject: Failure to crashdump under Centos-6.4 x86_64 with kernel 3.12.30
In-Reply-To: <mailman.5.1421956801.918.kexec@lists.infradead.org>
References: <mailman.5.1421956801.918.kexec@lists.infradead.org>
Message-ID: <1765458944.16558276.1421959824481.JavaMail.zimbra@redhat.com>



----- Original Message -----

> My kernel crashdumps don't work with vanilla 3.12.30 on a 256GB x86_64 intel box.
> Centos-6.4. With the standard makedumpfile, makedumpfile segfaults, and no output is written to disk.
> with makedumpfile 1.5.7, kexec-tools-2.0.8, and crash 7.0.2 I actually get vmcore files,
> but then crash rejects them with:
> 
> crash vmcore /boot/vmlinux-3.12.30
> 
> crash 7.0.2
> ...
> GNU gdb (GDB) 7.6
> ...
> 
> crash: seek error: kernel virtual address: ffff88207fffcb60  type: "pglist node_id"

With respect to the crash error, since it's a seek error, it means that the page for
physical address 0x207fffcb60 (~130GB) is not included in the dumpfile.  Are you getting any 
other error messages prior to that one?  Getting some debug output might show something
else of interest, for example by entering:

  $ crash -d4 vmcore /boot/vmlinux-3.12.30

Also, for a sanity check, what happens when you do this: 

  $ crash --minimal vmcore /boot/vmlinux-3.12.30
  ...
  crash> rd -a linux_banner 

It should show something like this (taken from the only 3.12 dumpfile I have):

  crash> rd -a linux_banner
  ffffffff818000a0:  Linux version 3.12.0-rc2+ (root at vm9-f19) (gcc version 4.8.1 
  ffffffff818000dc:  20130603 (Red Hat 4.8.1-1) (GCC) ) #2 SMP Wed Nov 6 13:25:13
  ffffffff81800118:   EST 2013
  crash> 

Dave 

 
> 
> the grub entry for this kernel is:
>      kernel /boot/vmlinuz-3.12.30 ro root=LABEL=ROOT
> numa_balancing=disable intel_idle.max_cstate=0 processor.max_cstate=0
> mce=ignore_mce clocksource=tsc nohz=off nomodeset nodmraid rd_NO_LUKS
> rd_NO_LVM rd_NO_MD rd_NO_DM LANG=en_US.UTF-8 SYSFONT=latarcyrheb-sun16
> KEYBOARDTYPE=pc KEYTABLE=us crashkernel=256M at 256M  console=ttyS1,115200n8
> 
> 
> CONFIG_CRASH_DUMP=y
> CONFIG_KEXEC_JUMP=y
> CONFIG_PHYSICAL_START=0x2000000
> CONFIG_RELOCATABLE=y
> CONFIG_PHYSICAL_ALIGN=0x1000000
> 
> A similar issue appears with a grub crashkernel=128M at 32M
> 
> Turning off all the makedumpfile exclusions (dump all pages) results in a much larger
> vmcore, but it still gets the same failures.
> 
> Where do I start looking for the problems here?
> kernel 3.10.65 won't dump either, but an old 3.0.29 kernel dumps just fine.
> Is this a problem with some .config setting?
> 
> Berkley


From geoff at infradead.org  Thu Jan 22 14:37:43 2015
From: geoff at infradead.org (Geoff Levand)
Date: Thu, 22 Jan 2015 14:37:43 -0800
Subject: Fwd: [PATCH] arm64: Read reg property based on address-cells
 value.
In-Reply-To: <54C0D3AE.9040405@huawei.com>
References: <1421913506-18587-1-git-send-email-anurup.m@huawei.com>
 <54C0D3AE.9040405@huawei.com>
Message-ID: <1421966263.21823.6.camel@infradead.org>

Hi Anurup,

On Thu, 2015-01-22 at 16:10 +0530, Anurup M wrote:
>     I have faced an issue during dtb file matching for CPU reg property. The dtb file which
> I use has 32 bit value in the reg property as the #address-cells is 1. The kexec-tools code
> reads the reg as a 64bit values and reads a invalid value and hence the below comparison in
> the function fixup_cpu_nodes fails.

Thanks for reporting. I pushed out a fix very similar to yours.  I only
tested on a dtb with address-cells=2 though.  Let me know if it doesn't
work for you.

 git://git.kernel.org/pub/scm/linux/kernel/git/geoff/kexec-tools.git

-Geoff





From dyoung at redhat.com  Sat Jan 24 05:26:37 2015
From: dyoung at redhat.com (Dave Young)
Date: Sat, 24 Jan 2015 21:26:37 +0800
Subject: [RFD] efi assisted kdump
Message-ID: <20150124132637.GB2139@darkstar.redhat.com>

Hi,

Kdump has several limitations currently such as kdump kernel reboot will bypass
device shutdown path so device drivers should reset during initialization.

* One of such problem we encounter now is the iommu
issue, 1st kernel's on fly DMA requests cause 2nd kernel hang. There's some effort
on this area, Zhenhua Li posted patches to resolve the intel iommu issue which is
under review. But that is just one case there's other possible problems in the future.

* There's no serial console on most machines in the market especially for desktop
machines and laptops. kms enabled kernel need drm layer driver for framebuffer
console, after kernel crashing we need a console to see the 2nd kernel output
ideally a serial console because we can not switch back to VGA mode.

ppc64 has a feature "firmware assisted kdump", see below documentation:
Documentation/powerpc/firmware-assisted-dump.txt

In case UEFI machines I wonder if we can do similar things, basic idea is doing
minimum thing based on original kdump process.

kernel reserve crashkernel memory during early boot
 -> user (kdump service) save necessary informations in some way so that second
    kernel boot can access ie. efi runtime variables or in reserved memory
    * crashkernel memory ranges infomation
    * collect infomations and save elf notes for 2nd kernel vmcore initialization
    -> crashing
       * call efi warm reboot
       -> 2nd kernel boot
          -> efi stub checking if this is a warm reboot and if this is a crashed boot
             -> if this is not a crashed boot then
                    do nothing more than original logic
                else
                    boot kernel with limited memory which was reserved in previous kernel
                    -> 2nd kernel boot
                       vmcore intializing

There's several things I need to ask for help from EFI gurus to see if it is doable:

* Make sure in case EFI warm reboot the memory of previous kernel can be retained.

* How to determine if kernel is boot with EFI warm reboot in stub

* What is the right way to pass informations from 1st kernel to 2nd kernel.
  Is it ok for saving something with efi variables? 

* Other possible problems what I missed

Thanks
Dave


From ptesarik at suse.cz  Sat Jan 24 08:03:44 2015
From: ptesarik at suse.cz (Petr Tesarik)
Date: Sat, 24 Jan 2015 17:03:44 +0100
Subject: [RFD] efi assisted kdump
In-Reply-To: <20150124132637.GB2139@darkstar.redhat.com>
References: <20150124132637.GB2139@darkstar.redhat.com>
Message-ID: <20150124170344.3b06a23e@hananiah.suse.cz>

On Sat, 24 Jan 2015 21:26:37 +0800
Dave Young <dyoung at redhat.com> wrote:

> Hi,
> 
> Kdump has several limitations currently such as kdump kernel reboot will bypass
> device shutdown path so device drivers should reset during initialization.
> 
>[...]
> 
> ppc64 has a feature "firmware assisted kdump", see below documentation:
> Documentation/powerpc/firmware-assisted-dump.txt

Hi Dave,

while I'm no expert on either UEFI or IBM POWER, I'd like to warn you
that fadump (firmware-assisted dump) on PPC is not quite optimal in its
current form. One of the things that have always irritated me are
excessive RAM requirements.

The problem is that there is only one reboot in fadump - after saving
the dump, the secondary kernel discards the saved area and continues
booting as usual. However, many kernel structuers must be already
allocated at that point, e.g. the memmap array(s), but they are sized
by the total RAM, not the limited amount available to the 2nd kernel.

This issue is not so bad on a typical IBM POWER system, because these
machines tend to be partitioned into many small LPARs, where each has
relatively little RAM. But on a machine with 6TB RAM, you'll have to
reserve approx. 84G merely for the memmap array on the secondary
kernel's boot, which looks just a bit wasteful...

Maybe the two approaches can be combined somehow, so that the
kdump system uses kexec() to transition into the production system, but
that only works if a normal kexec does not have any of the issues
you're trying to address.

Just my two cents,
Petr Tesarik


From vgoyal at redhat.com  Mon Jan 26 06:08:43 2015
From: vgoyal at redhat.com (Vivek Goyal)
Date: Mon, 26 Jan 2015 09:08:43 -0500
Subject: [RFD] efi assisted kdump
In-Reply-To: <20150124132637.GB2139@darkstar.redhat.com>
References: <20150124132637.GB2139@darkstar.redhat.com>
Message-ID: <20150126140843.GA25559@redhat.com>

On Sat, Jan 24, 2015 at 09:26:37PM +0800, Dave Young wrote:
> Hi,
> 
> Kdump has several limitations currently such as kdump kernel reboot will bypass
> device shutdown path so device drivers should reset during initialization.
> 
> * One of such problem we encounter now is the iommu
> issue, 1st kernel's on fly DMA requests cause 2nd kernel hang. There's some effort
> on this area, Zhenhua Li posted patches to resolve the intel iommu issue which is
> under review. But that is just one case there's other possible problems in the future.
> 
> * There's no serial console on most machines in the market especially for desktop
> machines and laptops. kms enabled kernel need drm layer driver for framebuffer
> console, after kernel crashing we need a console to see the 2nd kernel output
> ideally a serial console because we can not switch back to VGA mode.
> 
> ppc64 has a feature "firmware assisted kdump", see below documentation:
> Documentation/powerpc/firmware-assisted-dump.txt
> 
> In case UEFI machines I wonder if we can do similar things, basic idea is doing
> minimum thing based on original kdump process.
> 
> kernel reserve crashkernel memory during early boot
>  -> user (kdump service) save necessary informations in some way so that second
>     kernel boot can access ie. efi runtime variables or in reserved memory
>     * crashkernel memory ranges infomation
>     * collect infomations and save elf notes for 2nd kernel vmcore initialization

I think anything related to vmcore initilization can be in memory
somewhere and should not be a problem.

But we do have to think about anything which is passed in bootparams or
command line to second kenrnel with current mechanism, how will it be
passed to second kernel.

Thanks
Vivek


From matt at codeblueprint.co.uk  Mon Jan 26 08:39:41 2015
From: matt at codeblueprint.co.uk (Matt Fleming)
Date: Mon, 26 Jan 2015 16:39:41 +0000
Subject: [RFD] efi assisted kdump
In-Reply-To: <20150124132637.GB2139@darkstar.redhat.com>
References: <20150124132637.GB2139@darkstar.redhat.com>
Message-ID: <20150126163941.GD3320@codeblueprint.co.uk>

On Sat, 24 Jan, at 09:26:37PM, Dave Young wrote:
> 
> There's several things I need to ask for help from EFI gurus to see if it is doable:
> 
> * Make sure in case EFI warm reboot the memory of previous kernel can be retained.
 
The only way to reserve memory across a reboot is with EFI capsules.

> * How to determine if kernel is boot with EFI warm reboot in stub

The presence of a capsule would indicate whether we were invoked from a
crash handler.
 
> * What is the right way to pass informations from 1st kernel to 2nd kernel.
>   Is it ok for saving something with efi variables? 

EFI capsules would the be most natural mechanism, but you could
conceivably do it with EFI variables (there's an upper limit on the size
of variable data, though).

> * Other possible problems what I missed

Support for runtime EFI capsules was rather spotty last time I looked.
Things may have moved on, but it's definitely something to watch out
for.

There have been patches on linux-efi in recent months for making use of
EFI capsules,

  https://lkml.kernel.org/r/1412692886-25306-1-git-send-email-matt at console-pimps.org
  https://lkml.kernel.org/r/1414984030-13859-1-git-send-email-hock.leong.kweh at intel.com

I could take a look at resubmitting the first series above if that would
be useful for doing kexec across a reset.

-- 
Matt Fleming, Intel Open Source Technology Center


From catalin.marinas at arm.com  Mon Jan 26 09:44:14 2015
From: catalin.marinas at arm.com (Catalin Marinas)
Date: Mon, 26 Jan 2015 17:44:14 +0000
Subject: [PATCH 0/8] arm64 kexec kernel patches V7
In-Reply-To: <cover.1421449714.git.geoff@infradead.org>
References: <cover.1415926876.git.geoff@infradead.orgg>
 <cover.1421449714.git.geoff@infradead.org>
Message-ID: <20150126174414.GF14048@e104818-lin.cambridge.arm.com>

Hi Geoff,

On Sat, Jan 17, 2015 at 12:23:34AM +0000, Geoff Levand wrote:
> This series adds the core support for kexec re-boots on arm64.  This v7 of the
> series is mainly just a rebase to the latest arm64 for-next/core branch
> (v3.19-rc4), and a few very minor changes requested for v6.

I haven't looked at the series in detail before, so some of my comments
may have already been discussed.

> Several things are known to have problems on kexec re-boot:
> 
> spin-table

I think that's not too bad, for complete kexec support (SMP->SMP) we can
require some CPU unplug mechanism and PSCI is one of them.

> FIX: Upgrade system firmware to provide PSCI enable method support or add
> missing spin-table support to the kernel.

What's the missing spin-table support?

> ACPI
> ----
> 
> PROBLEM: The kernel for ACPI based systems does not export a device tree to the
> standard user space location of 'proc/device-tree'.  Current applications
> expect to access device tree information from this standard location.
> 
> WORK-AROUND: Disable ACPI in firmware, OR pass 'acpi=off' on the first stage
> kernel command line, OR pass a user specified DTB using the kexec --dtb option.
> 
> FIX: FIX: An interface to expose a binary device tree to user space has been
> proposed.  User kexec utilities will need to be updated to add support for this
> new interface.

So the fix here is to boot the second stage kernel with dtb, which means
that we mandate the existence of a DT file for any ACPI system. Are
there plans to make the kexec'ed kernel reuse the ACPI tables?

-- 
Catalin


From catalin.marinas at arm.com  Mon Jan 26 09:45:21 2015
From: catalin.marinas at arm.com (Catalin Marinas)
Date: Mon, 26 Jan 2015 17:45:21 +0000
Subject: [PATCH 1/8] arm64: Move proc-macros.S to include/asm
In-Reply-To: <e81815fb841db3ea40ce8e2fd18cf021ec9c6cf7.1421449714.git.geoff@infradead.org>
References: <cover.1421449714.git.geoff@infradead.org>
 <e81815fb841db3ea40ce8e2fd18cf021ec9c6cf7.1421449714.git.geoff@infradead.org>
Message-ID: <20150126174521.GG14048@e104818-lin.cambridge.arm.com>

On Sat, Jan 17, 2015 at 12:23:34AM +0000, Geoff Levand wrote:
> To allow the assembler macros defined in proc-macros.S to be used outside
> the mm code move the proc-macros.S file from arch/arm64/mm/ to
> arch/arm64/include/asm/ and fix up any preprocessor includes to use the new
> file location.
> 
> Signed-off-by: Geoff Levand <geoff at infradead.org>
> ---
>  arch/arm64/include/asm/proc-macros.S | 54 ++++++++++++++++++++++++++++++++++++
>  arch/arm64/mm/cache.S                |  3 +-
>  arch/arm64/mm/proc-macros.S          | 54 ------------------------------------
>  arch/arm64/mm/proc.S                 |  3 +-

Actually, I would just merge proc-macros.S into assembler.h. Not wirth
keeping the former just for a few macros.

-- 
Catalin


From catalin.marinas at arm.com  Mon Jan 26 10:26:21 2015
From: catalin.marinas at arm.com (Catalin Marinas)
Date: Mon, 26 Jan 2015 18:26:21 +0000
Subject: [PATCH 2/8] arm64: Convert hcalls to use ISS field
In-Reply-To: <6719b2520095926ae0e8ac888663b87f2098f565.1421449714.git.geoff@infradead.org>
References: <cover.1421449714.git.geoff@infradead.org>
 <6719b2520095926ae0e8ac888663b87f2098f565.1421449714.git.geoff@infradead.org>
Message-ID: <20150126182621.GH14048@e104818-lin.cambridge.arm.com>

On Sat, Jan 17, 2015 at 12:23:34AM +0000, Geoff Levand wrote:
> To allow for additional hcalls to be defined and to make the arm64 hcall API
> more consistent across exception vector routines, change the hcall implementations
> to use the ISS field of the ESR_EL2 register to specify the hcall type.
> 
> The existing arm64 hcall implementations are limited in that they only allow
> for two distinct hcalls; with the x0 register either zero, or not zero.  Also,
> the API of the hyp-stub exception vector routines and the KVM exception vector
> routines differ; hyp-stub uses a non-zero value in x0 to implement
> __hyp_set_vectors, whereas KVM uses it to implement kvm_call_hyp.
> 
> Define three new preprocessor macros HVC_GET_VECTORS, HVC_SET_VECTORS and
> HVC_CALL_HYP and to be used as hcall type specifiers and convert the
> existing __hyp_get_vectors(), __hyp_set_vectors() and kvm_call_hyp() routines
> to use these new macros when executing an HVC call.  Also change the
> corresponding hyp-stub and KVM el1_sync exception vector routines to use these
> new macros.
> 
> Signed-off-by: Geoff Levand <geoff at infradead.org>

Using the #imm value for HVC to separate what gets called looks fine to
me. However, I'd like to see a review from Marc/Christoffer on this
patch.

Some comments below:

> diff --git a/arch/arm64/include/asm/virt.h b/arch/arm64/include/asm/virt.h
> index 7a5df52..99c319c 100644
> --- a/arch/arm64/include/asm/virt.h
> +++ b/arch/arm64/include/asm/virt.h
> @@ -21,6 +21,26 @@
>  #define BOOT_CPU_MODE_EL1	(0xe11)
>  #define BOOT_CPU_MODE_EL2	(0xe12)
>  
> +/*
> + * HVC_GET_VECTORS - Return the value of the vbar_el2 register.
> + */
> +
> +#define HVC_GET_VECTORS 1
> +
> +/*
> + * HVC_SET_VECTORS - Set the value of the vbar_el2 register.
> + *
> + * @x0: Physical address of the new vector table.
> + */
> +
> +#define HVC_SET_VECTORS 2
> +
> +/*
> + * HVC_CALL_HYP - Execute a hyp routine.
> + */
> +
> +#define HVC_CALL_HYP 3

I think you can ignore this case (make it the default), just define it
as 0 as that's the normal use-case after initialisation and avoid
checking it explicitly.

>  /*
> diff --git a/arch/arm64/kernel/hyp-stub.S b/arch/arm64/kernel/hyp-stub.S
> index a272f33..e3db3fd 100644
> --- a/arch/arm64/kernel/hyp-stub.S
> +++ b/arch/arm64/kernel/hyp-stub.S
> @@ -22,6 +22,7 @@
>  #include <linux/irqchip/arm-gic-v3.h>
>  
>  #include <asm/assembler.h>
> +#include <asm/kvm_arm.h>
>  #include <asm/ptrace.h>
>  #include <asm/virt.h>
>  
> @@ -53,14 +54,22 @@ ENDPROC(__hyp_stub_vectors)
>  	.align 11
>  
>  el1_sync:
> -	mrs	x1, esr_el2
> -	lsr	x1, x1, #26
> -	cmp	x1, #0x16
> -	b.ne	2f				// Not an HVC trap
> -	cbz	x0, 1f
> -	msr	vbar_el2, x0			// Set vbar_el2
> +	mrs	x18, esr_el2
> +	lsr	x17, x18, #ESR_ELx_EC_SHIFT
> +	and	x18, x18, #ESR_ELx_ISS_MASK
> +
> +	cmp     x17, #ESR_ELx_EC_HVC64
> +	b.ne    2f				// Not an HVC trap
> +
> +	cmp	x18, #HVC_GET_VECTORS
> +	b.ne	1f
> +	mrs	x0, vbar_el2
>  	b	2f
> -1:	mrs	x0, vbar_el2			// Return vbar_el2
> +
> +1:	cmp	x18, #HVC_SET_VECTORS
> +	b.ne	2f
> +	msr	vbar_el2, x0
> +
>  2:	eret
>  ENDPROC(el1_sync)

You seem to be using x17 and x18 here freely. Do you have any guarantees
that the caller saved/restored those registers? I guess you assume they
are temporary registers and the caller first branches to a function
(like __kvm_hyp_call) and expects them to be corrupted. But I'm not sure
that's always the case. Take for example the __invoke_psci_fn_hvc where
the function is in C (we should change this for other reasons).

> diff --git a/arch/arm64/kvm/hyp.S b/arch/arm64/kvm/hyp.S
> index c0d8202..1916c89 100644
> --- a/arch/arm64/kvm/hyp.S
> +++ b/arch/arm64/kvm/hyp.S
> @@ -27,6 +27,7 @@
>  #include <asm/kvm_asm.h>
>  #include <asm/kvm_mmu.h>
>  #include <asm/memory.h>
> +#include <asm/virt.h>
>  
>  #define CPU_GP_REG_OFFSET(x)	(CPU_GP_REGS + x)
>  #define CPU_XREG_OFFSET(x)	CPU_GP_REG_OFFSET(CPU_USER_PT_REGS + 8*x)
> @@ -1106,12 +1107,9 @@ __hyp_panic_str:
>   * in Hyp mode (see init_hyp_mode in arch/arm/kvm/arm.c).  Return values are
>   * passed in r0 and r1.
>   *
> - * A function pointer with a value of 0 has a special meaning, and is
> - * used to implement __hyp_get_vectors in the same way as in
> - * arch/arm64/kernel/hyp_stub.S.
>   */
>  ENTRY(kvm_call_hyp)
> -	hvc	#0
> +	hvc	#HVC_CALL_HYP
>  	ret
>  ENDPROC(kvm_call_hyp)
>  
> @@ -1142,6 +1140,7 @@ el1_sync:					// Guest trapped into EL2
>  
>  	mrs	x1, esr_el2
>  	lsr	x2, x1, #ESR_ELx_EC_SHIFT
> +	and	x0, x1, #ESR_ELx_ISS_MASK
>  
>  	cmp	x2, #ESR_ELx_EC_HVC64
>  	b.ne	el1_trap
> @@ -1150,15 +1149,19 @@ el1_sync:					// Guest trapped into EL2
>  	cbnz	x3, el1_trap			// called HVC
>  
>  	/* Here, we're pretty sure the host called HVC. */
> +	mov	x18, x0

Same comment here about corrupting x18. If it is safe, maybe add some
comments in the calling place.

>  	pop	x2, x3
>  	pop	x0, x1
>  
> -	/* Check for __hyp_get_vectors */
> -	cbnz	x0, 1f
> +	cmp	x18, #HVC_GET_VECTORS
> +	b.ne	1f
>  	mrs	x0, vbar_el2
>  	b	2f
>  
> -1:	push	lr, xzr
> +1:	cmp	x18, #HVC_CALL_HYP
> +	b.ne	2f
> +
> +	push	lr, xzr

At this point, we expect either HVC_GET_VECTORS or HVC_CALL_HYP. I think
you can simply assume HVC_CALL_HYP as default and ignore the additional
cmp.

-- 
Catalin


From grant.likely at linaro.org  Mon Jan 26 10:37:24 2015
From: grant.likely at linaro.org (Grant Likely)
Date: Mon, 26 Jan 2015 18:37:24 +0000
Subject: [PATCH 0/8] arm64 kexec kernel patches V7
In-Reply-To: <20150126174414.GF14048@e104818-lin.cambridge.arm.com>
References: <cover.1415926876.git.geoff@infradead.orgg>
 <cover.1421449714.git.geoff@infradead.org>
 <20150126174414.GF14048@e104818-lin.cambridge.arm.com>
Message-ID: <CACxGe6uyko0ZfMBPHphJ1H2XO_L+ROKPYfsk4sjumKndX9kbng@mail.gmail.com>

On Mon, Jan 26, 2015 at 5:44 PM, Catalin Marinas
<catalin.marinas at arm.com> wrote:
> Hi Geoff,
>
> On Sat, Jan 17, 2015 at 12:23:34AM +0000, Geoff Levand wrote:
>> This series adds the core support for kexec re-boots on arm64.  This v7 of the
>> series is mainly just a rebase to the latest arm64 for-next/core branch
>> (v3.19-rc4), and a few very minor changes requested for v6.
>
> I haven't looked at the series in detail before, so some of my comments
> may have already been discussed.
>
>> Several things are known to have problems on kexec re-boot:
>>
>> spin-table
>
> I think that's not too bad, for complete kexec support (SMP->SMP) we can
> require some CPU unplug mechanism and PSCI is one of them.
>
>> FIX: Upgrade system firmware to provide PSCI enable method support or add
>> missing spin-table support to the kernel.
>
> What's the missing spin-table support?
>
>> ACPI
>> ----
>>
>> PROBLEM: The kernel for ACPI based systems does not export a device tree to the
>> standard user space location of 'proc/device-tree'.  Current applications
>> expect to access device tree information from this standard location.
>>
>> WORK-AROUND: Disable ACPI in firmware, OR pass 'acpi=off' on the first stage
>> kernel command line, OR pass a user specified DTB using the kexec --dtb option.
>>
>> FIX: FIX: An interface to expose a binary device tree to user space has been
>> proposed.  User kexec utilities will need to be updated to add support for this
>> new interface.

The new interface is merged into mainline. /sys/firmware/fdt

> So the fix here is to boot the second stage kernel with dtb, which means
> that we mandate the existence of a DT file for any ACPI system. Are
> there plans to make the kexec'ed kernel reuse the ACPI tables?

Yes, the kexec'ed kernel will reuse the ACPI tables, and any other
data passed by UEFI. The DT we're talking about here is the DT
generated by the kernel's UEFI stub, and the kexec tools want access
to it so they can find the UEFI system table pointer.

g.


From mark.rutland at arm.com  Mon Jan 26 10:55:25 2015
From: mark.rutland at arm.com (Mark Rutland)
Date: Mon, 26 Jan 2015 18:55:25 +0000
Subject: [PATCH 0/8] arm64 kexec kernel patches V7
In-Reply-To: <20150126174414.GF14048@e104818-lin.cambridge.arm.com>
References: <cover.1415926876.git.geoff@infradead.orgg>
 <cover.1421449714.git.geoff@infradead.org>
 <20150126174414.GF14048@e104818-lin.cambridge.arm.com>
Message-ID: <20150126185524.GN23313@leverpostej>

On Mon, Jan 26, 2015 at 05:44:14PM +0000, Catalin Marinas wrote:
> Hi Geoff,
> 
> On Sat, Jan 17, 2015 at 12:23:34AM +0000, Geoff Levand wrote:
> > This series adds the core support for kexec re-boots on arm64.  This v7 of the
> > series is mainly just a rebase to the latest arm64 for-next/core branch
> > (v3.19-rc4), and a few very minor changes requested for v6.
> 
> I haven't looked at the series in detail before, so some of my comments
> may have already been discussed.
> 
> > Several things are known to have problems on kexec re-boot:
> > 
> > spin-table
> 
> I think that's not too bad, for complete kexec support (SMP->SMP) we can
> require some CPU unplug mechanism and PSCI is one of them.
> 
> > FIX: Upgrade system firmware to provide PSCI enable method support or add
> > missing spin-table support to the kernel.
> 
> What's the missing spin-table support?

As you mention above, a mechanism for returning the CPUs to FW. There is
no spec for doing this with spin-table, which would have to be written
and vetted.

Mark.


From mark.rutland at arm.com  Mon Jan 26 11:02:23 2015
From: mark.rutland at arm.com (Mark Rutland)
Date: Mon, 26 Jan 2015 19:02:23 +0000
Subject: [PATCH 4/8] arm64: Add EL2 switch to soft_restart
In-Reply-To: <bb44d4eee4fab15278e4a8829a0981878bc9acd7.1421449714.git.geoff@infradead.org>
References: <cover.1421449714.git.geoff@infradead.org>
 <bb44d4eee4fab15278e4a8829a0981878bc9acd7.1421449714.git.geoff@infradead.org>
Message-ID: <20150126190223.GO23313@leverpostej>

On Sat, Jan 17, 2015 at 12:23:34AM +0000, Geoff Levand wrote:
> When a CPU is reset it needs to be put into the exception level it had when it
> entered the kernel.  Update cpu_reset() to accept an argument el2_switch which
> signals cpu_reset() to enter the soft reset address at EL2.  If el2_switch is
> not set the soft reset address will be entered at EL1.
> 
> Update cpu_soft_restart() and soft_restart() to pass the return of
> is_hyp_mode_available() as the el2_switch value to cpu_reset().  Also update the
> comments of cpu_reset(), cpu_soft_restart() and soft_restart() to reflect this
> change.
> 
> Signed-off-by: Geoff Levand <geoff at infradead.org>
> ---
>  arch/arm64/include/asm/proc-fns.h |  4 ++--
>  arch/arm64/kernel/process.c       | 10 ++++++++-
>  arch/arm64/mm/proc.S              | 47 +++++++++++++++++++++++++++++----------
>  3 files changed, 46 insertions(+), 15 deletions(-)
> 
> diff --git a/arch/arm64/include/asm/proc-fns.h b/arch/arm64/include/asm/proc-fns.h
> index 9a8fd84..339394d 100644
> --- a/arch/arm64/include/asm/proc-fns.h
> +++ b/arch/arm64/include/asm/proc-fns.h
> @@ -32,8 +32,8 @@ extern void cpu_cache_off(void);
>  extern void cpu_do_idle(void);
>  extern void cpu_do_switch_mm(unsigned long pgd_phys, struct mm_struct *mm);
>  extern void cpu_reset(unsigned long addr) __attribute__((noreturn));
> -void cpu_soft_restart(phys_addr_t cpu_reset,
> -		unsigned long addr) __attribute__((noreturn));
> +void cpu_soft_restart(phys_addr_t cpu_reset, unsigned long el2_switch,
> +		      unsigned long addr) __attribute__((noreturn));
>  extern void cpu_do_suspend(struct cpu_suspend_ctx *ptr);
>  extern u64 cpu_do_resume(phys_addr_t ptr, u64 idmap_ttbr);
>  
> diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c
> index fde9923..371bbf1 100644
> --- a/arch/arm64/kernel/process.c
> +++ b/arch/arm64/kernel/process.c
> @@ -50,6 +50,7 @@
>  #include <asm/mmu_context.h>
>  #include <asm/processor.h>
>  #include <asm/stacktrace.h>
> +#include <asm/virt.h>
>  
>  #ifdef CONFIG_CC_STACKPROTECTOR
>  #include <linux/stackprotector.h>
> @@ -60,7 +61,14 @@ EXPORT_SYMBOL(__stack_chk_guard);
>  void soft_restart(unsigned long addr)
>  {
>  	setup_mm_for_reboot();
> -	cpu_soft_restart(virt_to_phys(cpu_reset), addr);
> +
> +	/* TODO: Remove this conditional when KVM can support CPU restart. */
> +	if (IS_ENABLED(CONFIG_KVM))
> +		cpu_soft_restart(virt_to_phys(cpu_reset), 0, addr);

If we haven't torn down KVM, doesn't that mean that KVM is active at EL2
(with MMU and caches on) at this point?

If that's the case then we cannot possibly try to call kexec(), because
we cannot touch the memory used by the page tables for those EL2
mappings. Things will explode if we do.

Mark.


From mark.rutland at arm.com  Mon Jan 26 11:16:20 2015
From: mark.rutland at arm.com (Mark Rutland)
Date: Mon, 26 Jan 2015 19:16:20 +0000
Subject: [PATCH 5/8] arm64/kexec: Add core kexec support
In-Reply-To: <a5c232448c1d8fa1da73a9005fa9f3927c635578.1421449714.git.geoff@infradead.org>
References: <cover.1421449714.git.geoff@infradead.org>
 <a5c232448c1d8fa1da73a9005fa9f3927c635578.1421449714.git.geoff@infradead.org>
Message-ID: <20150126191620.GP23313@leverpostej>

On Sat, Jan 17, 2015 at 12:23:34AM +0000, Geoff Levand wrote:
> Add three new files, kexec.h, machine_kexec.c and relocate_kernel.S to the
> arm64 architecture that add support for the kexec re-boot mechanism
> (CONFIG_KEXEC) on arm64 platforms.
> 
> Signed-off-by: Geoff Levand <geoff at infradead.org>
> ---
>  arch/arm64/Kconfig                  |   9 ++
>  arch/arm64/include/asm/kexec.h      |  47 +++++++++++
>  arch/arm64/kernel/Makefile          |   1 +
>  arch/arm64/kernel/machine_kexec.c   | 155 ++++++++++++++++++++++++++++++++++
>  arch/arm64/kernel/relocate_kernel.S | 160 ++++++++++++++++++++++++++++++++++++
>  include/uapi/linux/kexec.h          |   1 +
>  6 files changed, 373 insertions(+)
>  create mode 100644 arch/arm64/include/asm/kexec.h
>  create mode 100644 arch/arm64/kernel/machine_kexec.c
>  create mode 100644 arch/arm64/kernel/relocate_kernel.S
> 
> diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
> index b1f9a20..d9eb9cd 100644
> --- a/arch/arm64/Kconfig
> +++ b/arch/arm64/Kconfig
> @@ -474,6 +474,15 @@ config SECCOMP
>           and the task is only allowed to execute a few safe syscalls
>           defined by each seccomp mode.
> 
> +config KEXEC
> +       depends on (!SMP || PM_SLEEP_SMP)
> +       bool "kexec system call"
> +       ---help---
> +         kexec is a system call that implements the ability to shutdown your
> +         current kernel, and to start another kernel.  It is like a reboot
> +         but it is independent of the system firmware.   And like a reboot
> +         you can start any kernel with it, not just Linux.
> +

[...]

> +/**
> + * kexec_is_dtb - Helper routine to check the device tree header signature.
> + */
> +static bool kexec_is_dtb(const void *dtb)
> +{
> +       __be32 magic;
> +
> +       return get_user(magic, (__be32 *)dtb) ? false :
> +               (be32_to_cpu(magic) == OF_DT_HEADER);
> +}
> +
> +/**
> + * kexec_find_dtb_seg - Helper routine to find the dtb segment.
> + */
> +static const struct kexec_segment *kexec_find_dtb_seg(
> +       const struct kimage *image)
> +{
> +       int i;
> +
> +       for (i = 0; i < image->nr_segments; i++) {
> +               if (kexec_is_dtb(image->segment[i].buf))
> +                       return &image->segment[i];
> +       }
> +
> +       return NULL;
> +}

As mentioned before, _please_ move the dtb handling to the
userspace-provided purgatory. It would be far better to get userspace to
handle setting up the dtb pointer explicitly. That avoids fragility
w.r.t. policy here as userspace will get exactly what it asked for,
nothing more, nothing less.

The fact that this is done on 32-bit arm does not mean that we must do
it here.

[...]

> +       /* Start new image. */
> +       ldr     x4, arm64_kexec_kimage_start
> +       ldr     x0, arm64_kexec_dtb_addr
> +       mov     x1, xzr
> +       mov     x2, xzr
> +       mov     x3, xzr
> +       br      x4

Likewise, this should be part of the userspace-provided purgatory code.
If we're staying true to "like a reboot you can start any kernel with
it, not just Linux", we shouldn't embed the Linux boot protocol here.

Mark.


From mark.rutland at arm.com  Mon Jan 26 11:19:42 2015
From: mark.rutland at arm.com (Mark Rutland)
Date: Mon, 26 Jan 2015 19:19:42 +0000
Subject: [PATCH 7/8] arm64/kexec: Add checks for KVM
In-Reply-To: <a3353cf2ac47f49c20bbbb88405301cbaa359805.1421449714.git.geoff@infradead.org>
References: <cover.1421449714.git.geoff@infradead.org>
 <a3353cf2ac47f49c20bbbb88405301cbaa359805.1421449714.git.geoff@infradead.org>
Message-ID: <20150126191942.GQ23313@leverpostej>

On Sat, Jan 17, 2015 at 12:23:34AM +0000, Geoff Levand wrote:
> Add runtime checks that fail the arm64 kexec syscall for situations that would
> result in system instability do to problems in the KVM kernel support.
> These checks should be removed when the KVM problems are resolved fixed.
> 
> Signed-off-by: Geoff Levand <geoff at infradead.org>
> ---
>  arch/arm64/kernel/machine_kexec.c | 10 ++++++++++
>  1 file changed, 10 insertions(+)
> 
> diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c
> index 3d84759..a36459d 100644
> --- a/arch/arm64/kernel/machine_kexec.c
> +++ b/arch/arm64/kernel/machine_kexec.c
> @@ -16,6 +16,9 @@
>  #include <asm/cacheflush.h>
>  #include <asm/system_misc.h>
>  
> +/* TODO: Remove this include when KVM can support a kexec reboot. */
> +#include <asm/virt.h>
> +
>  /* Global variables for the relocate_kernel routine. */
>  extern const unsigned char relocate_new_kernel[];
>  extern const unsigned long relocate_new_kernel_size;
> @@ -100,6 +103,13 @@ int machine_kexec_prepare(struct kimage *image)
>  
>  	kexec_image_info(image);
>  
> +	/* TODO: Remove this message when KVM can support a kexec reboot. */
> +	if (IS_ENABLED(CONFIG_KVM) && is_hyp_mode_available()) {
> +		pr_err("%s: Your kernel is configured with KVM support (CONFIG_KVM=y) which currently does not allow for kexec re-boot.\n",
> +			__func__);
> +		return -ENOSYS;
> +	}

If you really don't want to implement KVM teardown, surely this should
be at the start of the series, so we don't have a point in the middle
where things may explode in this case?

Mark.


From christoffer.dall at linaro.org  Mon Jan 26 12:39:26 2015
From: christoffer.dall at linaro.org (Christoffer Dall)
Date: Mon, 26 Jan 2015 21:39:26 +0100
Subject: [PATCH 7/8] arm64/kexec: Add checks for KVM
In-Reply-To: <20150126191942.GQ23313@leverpostej>
References: <cover.1421449714.git.geoff@infradead.org>
 <a3353cf2ac47f49c20bbbb88405301cbaa359805.1421449714.git.geoff@infradead.org>
 <20150126191942.GQ23313@leverpostej>
Message-ID: <CAMJs5B9DdKfdUw0_7+Q4okiZ0rWfgwi3vo1W=54+Ug0rcy8hHA@mail.gmail.com>

On Mon, Jan 26, 2015 at 8:19 PM, Mark Rutland <mark.rutland at arm.com> wrote:
> On Sat, Jan 17, 2015 at 12:23:34AM +0000, Geoff Levand wrote:
>> Add runtime checks that fail the arm64 kexec syscall for situations that would
>> result in system instability do to problems in the KVM kernel support.
>> These checks should be removed when the KVM problems are resolved fixed.
>>
>> Signed-off-by: Geoff Levand <geoff at infradead.org>
>> ---
>>  arch/arm64/kernel/machine_kexec.c | 10 ++++++++++
>>  1 file changed, 10 insertions(+)
>>
>> diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c
>> index 3d84759..a36459d 100644
>> --- a/arch/arm64/kernel/machine_kexec.c
>> +++ b/arch/arm64/kernel/machine_kexec.c
>> @@ -16,6 +16,9 @@
>>  #include <asm/cacheflush.h>
>>  #include <asm/system_misc.h>
>>
>> +/* TODO: Remove this include when KVM can support a kexec reboot. */
>> +#include <asm/virt.h>
>> +
>>  /* Global variables for the relocate_kernel routine. */
>>  extern const unsigned char relocate_new_kernel[];
>>  extern const unsigned long relocate_new_kernel_size;
>> @@ -100,6 +103,13 @@ int machine_kexec_prepare(struct kimage *image)
>>
>>       kexec_image_info(image);
>>
>> +     /* TODO: Remove this message when KVM can support a kexec reboot. */
>> +     if (IS_ENABLED(CONFIG_KVM) && is_hyp_mode_available()) {
>> +             pr_err("%s: Your kernel is configured with KVM support (CONFIG_KVM=y) which currently does not allow for kexec re-boot.\n",
>> +                     __func__);
>> +             return -ENOSYS;
>> +     }
>
> If you really don't want to implement KVM teardown, surely this should
> be at the start of the series, so we don't have a point in the middle
> where things may explode in this case?
>
So this caters to support systems that don't support KVM (don't boot
in EL2) but is configured with both KVM and KEXEC?

Why not just make the kexec config dependent on !KVM ?

-Christoffer


From geoff at infradead.org  Mon Jan 26 12:57:19 2015
From: geoff at infradead.org (Geoff Levand)
Date: Mon, 26 Jan 2015 12:57:19 -0800
Subject: [PATCH 0/8] arm64 kexec kernel patches V7
In-Reply-To: <20150126174414.GF14048@e104818-lin.cambridge.arm.com>
References: <cover.1415926876.git.geoff@infradead.orgg>
 <cover.1421449714.git.geoff@infradead.org>
 <20150126174414.GF14048@e104818-lin.cambridge.arm.com>
Message-ID: <1422305839.21823.11.camel@infradead.org>

Hi Catalin,


On Mon, 2015-01-26 at 17:44 +0000, Catalin Marinas wrote:
> On Sat, Jan 17, 2015 at 12:23:34AM +0000, Geoff Levand wrote:
> > This series adds the core support for kexec re-boots on arm64.  This v7 of the
> > series is mainly just a rebase to the latest arm64 for-next/core branch
> > (v3.19-rc4), and a few very minor changes requested for v6.
> 
> I haven't looked at the series in detail before, so some of my comments
> may have already been discussed.
> 
> > Several things are known to have problems on kexec re-boot:
> > 
> > spin-table
> 
> I think that's not too bad, for complete kexec support (SMP->SMP) we can
> require some CPU unplug mechanism and PSCI is one of them.
> 
> > FIX: Upgrade system firmware to provide PSCI enable method support or add
> > missing spin-table support to the kernel.
> 
> What's the missing spin-table support?

I had a working spin-table implementation, but I have not kept that
up to date.  It is somewhat complicated, and it requires a mechanism
to return the secondary CPUs to the spin table code.  As Mark
mentioned, that mechanism would need to be decided on.  I don't plan
to put any more work into spin-table support.

Just for anyone interested, my old spin-table support patches are in
this branch:

https://git.kernel.org/cgit/linux/kernel/git/geoff/linux-kexec.git/log/?h=spin-table

> 
> > ACPI
> > ----
> > 
> > PROBLEM: The kernel for ACPI based systems does not export a device tree to the
> > standard user space location of 'proc/device-tree'.  Current applications
> > expect to access device tree information from this standard location.
> > 
> > WORK-AROUND: Disable ACPI in firmware, OR pass 'acpi=off' on the first stage
> > kernel command line, OR pass a user specified DTB using the kexec --dtb option.
> > 
> > FIX: FIX: An interface to expose a binary device tree to user space has been
> > proposed.  User kexec utilities will need to be updated to add support for this
> > new interface.
> 
> So the fix here is to boot the second stage kernel with dtb, which means
> that we mandate the existence of a DT file for any ACPI system. Are
> there plans to make the kexec'ed kernel reuse the ACPI tables?

As Grant mentioned, the dtb the UEFI stub creates and passes to the first
stage kernel is what is passed to the second stage kernel.  The second
stage kernel uses that to setup its ACPI support.

-Geoff





From geoff at infradead.org  Mon Jan 26 12:58:25 2015
From: geoff at infradead.org (Geoff Levand)
Date: Mon, 26 Jan 2015 12:58:25 -0800
Subject: [PATCH 7/8] arm64/kexec: Add checks for KVM
In-Reply-To: <CAMJs5B9DdKfdUw0_7+Q4okiZ0rWfgwi3vo1W=54+Ug0rcy8hHA@mail.gmail.com>
References: <cover.1421449714.git.geoff@infradead.org>
 <a3353cf2ac47f49c20bbbb88405301cbaa359805.1421449714.git.geoff@infradead.org>
 <20150126191942.GQ23313@leverpostej>
 <CAMJs5B9DdKfdUw0_7+Q4okiZ0rWfgwi3vo1W=54+Ug0rcy8hHA@mail.gmail.com>
Message-ID: <1422305905.21823.12.camel@infradead.org>

On Mon, 2015-01-26 at 21:39 +0100, Christoffer Dall wrote:
> On Mon, Jan 26, 2015 at 8:19 PM, Mark Rutland <mark.rutland at arm.com> wrote:
> > On Sat, Jan 17, 2015 at 12:23:34AM +0000, Geoff Levand wrote:
> >> Add runtime checks that fail the arm64 kexec syscall for situations that would
> >> result in system instability do to problems in the KVM kernel support.
> >> These checks should be removed when the KVM problems are resolved fixed.
> >>
> >> Signed-off-by: Geoff Levand <geoff at infradead.org>
> >> ---
> >>  arch/arm64/kernel/machine_kexec.c | 10 ++++++++++
> >>  1 file changed, 10 insertions(+)
> >>
> >> diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c
> >> index 3d84759..a36459d 100644
> >> --- a/arch/arm64/kernel/machine_kexec.c
> >> +++ b/arch/arm64/kernel/machine_kexec.c
> >> @@ -16,6 +16,9 @@
> >>  #include <asm/cacheflush.h>
> >>  #include <asm/system_misc.h>
> >>
> >> +/* TODO: Remove this include when KVM can support a kexec reboot. */
> >> +#include <asm/virt.h>
> >> +
> >>  /* Global variables for the relocate_kernel routine. */
> >>  extern const unsigned char relocate_new_kernel[];
> >>  extern const unsigned long relocate_new_kernel_size;
> >> @@ -100,6 +103,13 @@ int machine_kexec_prepare(struct kimage *image)
> >>
> >>       kexec_image_info(image);
> >>
> >> +     /* TODO: Remove this message when KVM can support a kexec reboot. */
> >> +     if (IS_ENABLED(CONFIG_KVM) && is_hyp_mode_available()) {
> >> +             pr_err("%s: Your kernel is configured with KVM support (CONFIG_KVM=y) which currently does not allow for kexec re-boot.\n",
> >> +                     __func__);
> >> +             return -ENOSYS;
> >> +     }
> >
> > If you really don't want to implement KVM teardown, surely this should
> > be at the start of the series, so we don't have a point in the middle
> > where things may explode in this case?
> >
> So this caters to support systems that don't support KVM (don't boot
> in EL2) but is configured with both KVM and KEXEC?
> 
> Why not just make the kexec config dependent on !KVM ?

Sure, that would work.  I put it this way so we can get build testing of
kexec since the arm64 defconfig has KVM set.

-Geoff




From geoff at infradead.org  Mon Jan 26 13:00:27 2015
From: geoff at infradead.org (Geoff Levand)
Date: Mon, 26 Jan 2015 13:00:27 -0800
Subject: [PATCH 7/8] arm64/kexec: Add checks for KVM
In-Reply-To: <20150126191942.GQ23313@leverpostej>
References: <cover.1421449714.git.geoff@infradead.org>
 <a3353cf2ac47f49c20bbbb88405301cbaa359805.1421449714.git.geoff@infradead.org>
 <20150126191942.GQ23313@leverpostej>
Message-ID: <1422306027.21823.14.camel@infradead.org>

Hi Mark,

On Mon, 2015-01-26 at 19:19 +0000, Mark Rutland wrote:
> On Sat, Jan 17, 2015 at 12:23:34AM +0000, Geoff Levand wrote:
> > Add runtime checks that fail the arm64 kexec syscall for situations that would
> > result in system instability do to problems in the KVM kernel support.
> > These checks should be removed when the KVM problems are resolved fixed.
> > 
> > Signed-off-by: Geoff Levand <geoff at infradead.org>
> > ---
> >  arch/arm64/kernel/machine_kexec.c | 10 ++++++++++
> >  1 file changed, 10 insertions(+)
> > 
> > diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c
> > index 3d84759..a36459d 100644
> > --- a/arch/arm64/kernel/machine_kexec.c
> > +++ b/arch/arm64/kernel/machine_kexec.c
> > @@ -16,6 +16,9 @@
> >  #include <asm/cacheflush.h>
> >  #include <asm/system_misc.h>
> >  
> > +/* TODO: Remove this include when KVM can support a kexec reboot. */
> > +#include <asm/virt.h>
> > +
> >  /* Global variables for the relocate_kernel routine. */
> >  extern const unsigned char relocate_new_kernel[];
> >  extern const unsigned long relocate_new_kernel_size;
> > @@ -100,6 +103,13 @@ int machine_kexec_prepare(struct kimage *image)
> >  
> >  	kexec_image_info(image);
> >  
> > +	/* TODO: Remove this message when KVM can support a kexec reboot. */
> > +	if (IS_ENABLED(CONFIG_KVM) && is_hyp_mode_available()) {
> > +		pr_err("%s: Your kernel is configured with KVM support (CONFIG_KVM=y) which currently does not allow for kexec re-boot.\n",
> > +			__func__);
> > +		return -ENOSYS;
> > +	}
> 
> If you really don't want to implement KVM teardown, surely this should
> be at the start of the series, so we don't have a point in the middle
> where things may explode in this case?

Yes, you're right.  I'm hoping we can get the KVM fix done soon so this
is not needed.

-Geoff




From geoff at infradead.org  Mon Jan 26 13:48:48 2015
From: geoff at infradead.org (Geoff Levand)
Date: Mon, 26 Jan 2015 13:48:48 -0800
Subject: [PATCH 4/8] arm64: Add EL2 switch to soft_restart
In-Reply-To: <20150126190223.GO23313@leverpostej>
References: <cover.1421449714.git.geoff@infradead.org>
 <bb44d4eee4fab15278e4a8829a0981878bc9acd7.1421449714.git.geoff@infradead.org>
 <20150126190223.GO23313@leverpostej>
Message-ID: <1422308928.21823.16.camel@infradead.org>

Hi Mark,

On Mon, 2015-01-26 at 19:02 +0000, Mark Rutland wrote:
> On Sat, Jan 17, 2015 at 12:23:34AM +0000, Geoff Levand wrote:
> > When a CPU is reset it needs to be put into the exception level it had when it
> > entered the kernel.  Update cpu_reset() to accept an argument el2_switch which
> > signals cpu_reset() to enter the soft reset address at EL2.  If el2_switch is
> > not set the soft reset address will be entered at EL1.
> > 
> > Update cpu_soft_restart() and soft_restart() to pass the return of
> > is_hyp_mode_available() as the el2_switch value to cpu_reset().  Also update the
> > comments of cpu_reset(), cpu_soft_restart() and soft_restart() to reflect this
> > change.
> > 
> > Signed-off-by: Geoff Levand <geoff at infradead.org>
> > ---
> >  arch/arm64/include/asm/proc-fns.h |  4 ++--
> >  arch/arm64/kernel/process.c       | 10 ++++++++-
> >  arch/arm64/mm/proc.S              | 47 +++++++++++++++++++++++++++++----------
> >  3 files changed, 46 insertions(+), 15 deletions(-)
> > 
> > diff --git a/arch/arm64/include/asm/proc-fns.h b/arch/arm64/include/asm/proc-fns.h
> > index 9a8fd84..339394d 100644
> > --- a/arch/arm64/include/asm/proc-fns.h
> > +++ b/arch/arm64/include/asm/proc-fns.h
> > @@ -32,8 +32,8 @@ extern void cpu_cache_off(void);
> >  extern void cpu_do_idle(void);
> >  extern void cpu_do_switch_mm(unsigned long pgd_phys, struct mm_struct *mm);
> >  extern void cpu_reset(unsigned long addr) __attribute__((noreturn));
> > -void cpu_soft_restart(phys_addr_t cpu_reset,
> > -		unsigned long addr) __attribute__((noreturn));
> > +void cpu_soft_restart(phys_addr_t cpu_reset, unsigned long el2_switch,
> > +		      unsigned long addr) __attribute__((noreturn));
> >  extern void cpu_do_suspend(struct cpu_suspend_ctx *ptr);
> >  extern u64 cpu_do_resume(phys_addr_t ptr, u64 idmap_ttbr);
> >  
> > diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c
> > index fde9923..371bbf1 100644
> > --- a/arch/arm64/kernel/process.c
> > +++ b/arch/arm64/kernel/process.c
> > @@ -50,6 +50,7 @@
> >  #include <asm/mmu_context.h>
> >  #include <asm/processor.h>
> >  #include <asm/stacktrace.h>
> > +#include <asm/virt.h>
> >  
> >  #ifdef CONFIG_CC_STACKPROTECTOR
> >  #include <linux/stackprotector.h>
> > @@ -60,7 +61,14 @@ EXPORT_SYMBOL(__stack_chk_guard);
> >  void soft_restart(unsigned long addr)
> >  {
> >  	setup_mm_for_reboot();
> > -	cpu_soft_restart(virt_to_phys(cpu_reset), addr);
> > +
> > +	/* TODO: Remove this conditional when KVM can support CPU restart. */
> > +	if (IS_ENABLED(CONFIG_KVM))
> > +		cpu_soft_restart(virt_to_phys(cpu_reset), 0, addr);
> 
> If we haven't torn down KVM, doesn't that mean that KVM is active at EL2
> (with MMU and caches on) at this point?
> 
> If that's the case then we cannot possibly try to call kexec(), because
> we cannot touch the memory used by the page tables for those EL2
> mappings. Things will explode if we do.

This conditional is just if KVM, do things the old way (don't try to
switch exception levels).  It is to handle the system shutdown case.

Another patch in this series '[PATCH 7/8] arm64/kexec: Add checks for
KVM' assures kexec cannot happen when KVM is configured.

-Geoff





From horms at verge.net.au  Mon Jan 26 18:13:17 2015
From: horms at verge.net.au (Simon Horman)
Date: Tue, 27 Jan 2015 11:13:17 +0900
Subject: [ANNOUNCE] kexec-tools 2.0.9-rc1
Message-ID: <20150127021317.GA7909@verge.net.au>

Hi all,

I am happy to announce the release of kexec-tools 2.0.9-rc1.

This is a bug-fix pre-release.

So long as no serious problems arise I intend to release 2.0.9 around the
same time as the v3.19 kernel release, which I expect to occur in the next
week or so.  As such testing of 2.0.9-rc1 would be greatly appreciated.

I do not have any outstanding changes for 2.0.9 at this time.
And I would like to only accept bug fixes at this time. Taking
features patches once 2.0.9 has been released.

The pre-release can be downloaded from kernel.org:

	https://kernel.org/pub/linux/utils/kernel/kexec/kexec-tools-2.0.9-rc1.tar.xz
	https://kernel.org/pub/linux/utils/kernel/kexec/

I have also tagged it in git:

	https://git.kernel.org/pub/scm/utils/kernel/kexec/kexec-tools.git

Thanks to everyone who has contributed to kexec.


From dyoung at redhat.com  Mon Jan 26 22:21:41 2015
From: dyoung at redhat.com (Dave Young)
Date: Tue, 27 Jan 2015 14:21:41 +0800
Subject: [RFD] efi assisted kdump
In-Reply-To: <20150124170344.3b06a23e@hananiah.suse.cz>
References: <20150124132637.GB2139@darkstar.redhat.com>
 <20150124170344.3b06a23e@hananiah.suse.cz>
Message-ID: <20150127062141.GC2506@darkstar.redhat.com>

Hi, Petr

On 01/24/15 at 05:03pm, Petr Tesarik wrote:
> On Sat, 24 Jan 2015 21:26:37 +0800
> Dave Young <dyoung at redhat.com> wrote:
> 
> > Hi,
> > 
> > Kdump has several limitations currently such as kdump kernel reboot will bypass
> > device shutdown path so device drivers should reset during initialization.
> > 
> >[...]
> > 
> > ppc64 has a feature "firmware assisted kdump", see below documentation:
> > Documentation/powerpc/firmware-assisted-dump.txt
> 
> Hi Dave,
> 
> while I'm no expert on either UEFI or IBM POWER, I'd like to warn you
> that fadump (firmware-assisted dump) on PPC is not quite optimal in its
> current form. One of the things that have always irritated me are
> excessive RAM requirements.
> 
> The problem is that there is only one reboot in fadump - after saving
> the dump, the secondary kernel discards the saved area and continues
> booting as usual. However, many kernel structuers must be already
> allocated at that point, e.g. the memmap array(s), but they are sized
> by the total RAM, not the limited amount available to the 2nd kernel.

Thanks for the comment.

I'm not sure I understand the RAM requirements you mentioned. I think you
are worrying about freeing oldmem to be used by 2nd kernel.

But we do not need to do same as power we can just reboot another time
as long as vmcore is saved. The main advantage is the capture kernel can
boot with all devices being reset. 

> 
> This issue is not so bad on a typical IBM POWER system, because these
> machines tend to be partitioned into many small LPARs, where each has
> relatively little RAM. But on a machine with 6TB RAM, you'll have to
> reserve approx. 84G merely for the memmap array on the secondary
> kernel's boot, which looks just a bit wasteful...
> 
> Maybe the two approaches can be combined somehow, so that the
> kdump system uses kexec() to transition into the production system, but
> that only works if a normal kexec does not have any of the issues
> you're trying to address.
> 
> Just my two cents,
> Petr Tesarik
> 
> _______________________________________________
> kexec mailing list
> kexec at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/kexec


From dyoung at redhat.com  Mon Jan 26 22:24:34 2015
From: dyoung at redhat.com (Dave Young)
Date: Tue, 27 Jan 2015 14:24:34 +0800
Subject: [RFD] efi assisted kdump
In-Reply-To: <20150126140843.GA25559@redhat.com>
References: <20150124132637.GB2139@darkstar.redhat.com>
 <20150126140843.GA25559@redhat.com>
Message-ID: <20150127062433.GD2506@darkstar.redhat.com>

On 01/26/15 at 09:08am, Vivek Goyal wrote:
> On Sat, Jan 24, 2015 at 09:26:37PM +0800, Dave Young wrote:
> > Hi,
> > 
> > Kdump has several limitations currently such as kdump kernel reboot will bypass
> > device shutdown path so device drivers should reset during initialization.
> > 
> > * One of such problem we encounter now is the iommu
> > issue, 1st kernel's on fly DMA requests cause 2nd kernel hang. There's some effort
> > on this area, Zhenhua Li posted patches to resolve the intel iommu issue which is
> > under review. But that is just one case there's other possible problems in the future.
> > 
> > * There's no serial console on most machines in the market especially for desktop
> > machines and laptops. kms enabled kernel need drm layer driver for framebuffer
> > console, after kernel crashing we need a console to see the 2nd kernel output
> > ideally a serial console because we can not switch back to VGA mode.
> > 
> > ppc64 has a feature "firmware assisted kdump", see below documentation:
> > Documentation/powerpc/firmware-assisted-dump.txt
> > 
> > In case UEFI machines I wonder if we can do similar things, basic idea is doing
> > minimum thing based on original kdump process.
> > 
> > kernel reserve crashkernel memory during early boot
> >  -> user (kdump service) save necessary informations in some way so that second
> >     kernel boot can access ie. efi runtime variables or in reserved memory
> >     * crashkernel memory ranges infomation
> >     * collect infomations and save elf notes for 2nd kernel vmcore initialization
> 
> I think anything related to vmcore initilization can be in memory
> somewhere and should not be a problem.
> 
> But we do have to think about anything which is passed in bootparams or
> command line to second kenrnel with current mechanism, how will it be
> passed to second kernel.

Hi, Vivek,

Since we are rebooting via efi maybe we can just choose another boot entry
in grub.cfg with the proper cmdline?

Thanks
Dave 


From dyoung at redhat.com  Mon Jan 26 22:37:42 2015
From: dyoung at redhat.com (Dave Young)
Date: Tue, 27 Jan 2015 14:37:42 +0800
Subject: [RFD] efi assisted kdump
In-Reply-To: <20150126163941.GD3320@codeblueprint.co.uk>
References: <20150124132637.GB2139@darkstar.redhat.com>
 <20150126163941.GD3320@codeblueprint.co.uk>
Message-ID: <20150127063742.GE2506@darkstar.redhat.com>

Hi, Matt

Thanks for the feedback.

On 01/26/15 at 04:39pm, Matt Fleming wrote:
> On Sat, 24 Jan, at 09:26:37PM, Dave Young wrote:
> > 
> > There's several things I need to ask for help from EFI gurus to see if it is doable:
> > 
> > * Make sure in case EFI warm reboot the memory of previous kernel can be retained.
>  
> The only way to reserve memory across a reboot is with EFI capsules.

I know your capsules patchset, but I did not look into the details.

Do you means the old memory will be not reliable in case booting without capsules
even for warm reboot?

> 
> > * How to determine if kernel is boot with EFI warm reboot in stub
> 
> The presence of a capsule would indicate whether we were invoked from a
> crash handler.

Ok, another problem is for efi capsule handling, can we do it in efi stub, or I
in a efi application, which stage is the doable?

>  
> > * What is the right way to pass informations from 1st kernel to 2nd kernel.
> >   Is it ok for saving something with efi variables? 
> 
> EFI capsules would the be most natural mechanism, but you could
> conceivably do it with EFI variables (there's an upper limit on the size
> of variable data, though).

Will think about it after I checking the capsule machanisms details.

> 
> > * Other possible problems what I missed
> 
> Support for runtime EFI capsules was rather spotty last time I looked.
> Things may have moved on, but it's definitely something to watch out
> for.
> 
> There have been patches on linux-efi in recent months for making use of
> EFI capsules,
> 
>   https://lkml.kernel.org/r/1412692886-25306-1-git-send-email-matt at console-pimps.org
>   https://lkml.kernel.org/r/1414984030-13859-1-git-send-email-hock.leong.kweh at intel.com
> 
> I could take a look at resubmitting the first series above if that would
> be useful for doing kexec across a reset.

Thanks a lot. I will take a look at how capsules works firstly.
Dave


From mtk.manpages at gmail.com  Tue Jan 27 00:07:09 2015
From: mtk.manpages at gmail.com (Michael Kerrisk (man-pages))
Date: Tue, 27 Jan 2015 09:07:09 +0100
Subject: Edited kexec_load(2) [kexec_file_load()] man page for review
In-Reply-To: <54B91271.3000600@gmail.com>
References: <545FBDDD.9060801@gmail.com> <20141111213037.GA31445@redhat.com>
 <54ADA284.30502@gmail.com> <20150112221634.GD16162@redhat.com>
 <54B91271.3000600@gmail.com>
Message-ID: <CAKgNAkg14Aa=vDZSZQTu6gUhCUdTNkhNQ52fL3X1XbvnNJcx5g@mail.gmail.com>

Hello Vivek,

Ping!

Cheers,

Michael


On 16 January 2015 at 14:30, Michael Kerrisk (man-pages)
<mtk.manpages at gmail.com> wrote:
> Hello Vivek,
>
> Thanks for your comments! I've added some further text to
> the page based on those comments. See some follow-up
> questions below.
>
> On 01/12/2015 11:16 PM, Vivek Goyal wrote:
>> On Wed, Jan 07, 2015 at 10:17:56PM +0100, Michael Kerrisk (man-pages) wrote:
>>
>> [..]
>>>>> .BR KEXEC_ON_CRASH " (since Linux 2.6.13)"
>>>>> Execute the new kernel automatically on a system crash.
>>>>> .\" FIXME Explain in more detail how KEXEC_ON_CRASH is actually used
>>>
>>> I wasn't expecting that you would respond to the FIXMEs that were
>>> not labeled "kexec_file_load", but I was hoping you might ;-). Thanks!
>>> I have a few additional questions to your nice notes.
>>>
>>>> Upon boot first kernel reserves a chunk of contiguous memory (if
>>>> crashkernel=<> command line paramter is passed). This memory is
>>>> is used to load the crash kernel (Kernel which will be booted into
>>>> if first kernel crashes).
>>>
>>
>> Hi Michael,
>>
>>> Can I just confirm: is it in all cases only possible to use kexec_load()
>>> and kexec_file_load() if the kernel was booted with the 'crashkernel'
>>> parameter set?
>>
>> As of now, only kexec_load() and kexec_file_load() system calls can
>> make use of memory reserved by crashkernel=<> kernel parameter. And
>> this is used only if we are trying to load a crash kernel (KEXEC_ON_CRASH
>> flag specified).
>
> Okay.
>
>>>> Location of this reserved memory is exported to user space through
>>>> /proc/iomem file.
>>>
>>> Is that export via an entry labeled "Crash kernel" in the
>>> /proc/iomem file?
>>
>> Yes.
>
> Okay -- thanks.
>
>>>> User space can parse it and prepare list of segments
>>>> specifying this reserved memory as destination.
>>>
>>> I'm not quite clear on "specifying this reserved memory as destination".
>>> Is that done by specifying the address in the kexec_segment.mem fields?
>>
>> You are absolutely right. User space can specify in kexec_segment.mem
>> field the memory location where it expecting a particular segment to
>> be loaded by kernel.
>>
>>>
>>>> Once kernel sees the flag KEXEC_ON_CRASH, it makes sure that all the
>>>> segments are destined for reserved memory otherwise kernel load operation
>>>> fails.
>>>
>>> Could you point me to where this checking is done? Also, what is the
>>> error (errno) that occurs when the load operation fails? (I think the
>>> answers to these questions are "at the start of kimage_alloc_init()"
>>> and "EADDRNOTAVAIL", but I'd like to confirm.)
>>
>> This checking happens in sanity_check_segment_list() which is called
>> by kimage_alloc_init().
>>
>> And yes, error code returned is -EADDRNOTAVAIL.
>
> Thanks. I added EADDRNOTAVAIL to the ERRORS.
>
>>>> [..]
>>>>> struct kexec_segment {
>>>>>     void   *buf;        /* Buffer in user space */
>>>>>     size_t  bufsz;      /* Buffer length in user space */
>>>>>     void   *mem;        /* Physical address of kernel */
>>>>>     size_t  memsz;      /* Physical address length */
>>>>> };
>>>>> .fi
>>>>> .in
>>>>> .PP
>>>>> .\" FIXME Explain the details of how the kernel image defined by segments
>>>>> .\" is copied from the calling process into previously reserved memory.
>>>>
>>>> Kernel image defined by segments is copied into kernel either in regular
>>>> memory
>>>
>>> Could you clarify what you mean by "regular memory"?
>>
>> I meant memory which is not reserved memory.
>
> Okay.
>
>>>> or in reserved memory (if KEXEC_ON_CRASH is set). Kernel first
>>>> copies list of segments in kernel memory and then goes does various
>>>> sanity checks on the segments. If everything looks line, kernel copies
>>>> segment data to kernel memory.
>>>>
>>>> In case of normal kexec, segment data is loaded in any available memory
>>>> and segment data is moved to final destination at the kexec reboot time.
>>>
>>> By "moved to final destination", do you mean "moved from user space to the
>>> final kernel-space destination"?
>>
>> No. Segment data moves from user space to kernel space once kexec_load()
>> call finishes successfully. But when user does reboot (kexec -e), at that
>> time kernel moves that segment data to its final location. Kernel could
>> not place the segment at its final location during kexec_load() time as
>> that memory is already in use by running kernel. But once we are about
>> to reboot to new kernel, we can overwrite the old kernel's memory.
>
> Got it.
>
>>>> In case of kexec on panic (KEXEC_ON_CRASH flag set), segment data is
>>>> directly loaded to reserved memory and after crash kexec simply jumps
>>>
>>> By "directly", I assume you mean "at the time of the kexec_laod() call",
>>> right?
>>
>> Yes.
>
> Thanks.
>
> So, returning to the kexeec_segment structure:
>
>            struct kexec_segment {
>                void   *buf;        /* Buffer in user space */
>                size_t  bufsz;      /* Buffer length in user space */
>                void   *mem;        /* Physical address of kernel */
>                size_t  memsz;      /* Physical address length */
>            };
>
> Are the following statements correct:
> * buf + bufsz identify a memory region in the caller's virtual
>   address space that is the source of the copy
> * mem + memsz specify the target memory region of the copy
> * mem is  physical memory address, as seen from kernel space
> * the number of bytes copied from userspace is min(bufsz, memsz)
> * if bufsz > memsz, then excess bytes in the user-space buffer
>   are ignored.
> * if memsz > bufsz, then excess bytes in the target kernel buffer
>   are filled with zeros.
> ?
>
> Also, it seems to me that 'mem' need not be page aligned.
> Is that correct? Should the man page say something about that?
> (E.g., is it generally desirable that 'mem' should be page aligned?)
>
> Likewise, 'memsz' doesn't need to be a page multiple, IIUC.
> Should the man page say anything about this? For example, should
> it note that the initialized kernel segment will be of size:
>
>      (mem % PAGE_SIZE + memsz) rounded up to the next multiple of PAGE_SIZE
>
> And should it note that if 'mem' is not a multiple of the page size, then
> the initial bytes (mem % PAGE_SIZE)) in the first page of the kernel segment
> will be zeros?
>
> (Hopefully I have read kimage_load_normal_segment() correctly.)
>
> And one further question. Other than the fact that they are used with
> different system calls, what is the difference between KEXEC_ON_CRASH
> and KEXEC_FILE_ON_CRASH?
>
> Thanks,
>
> Michael
>
> --
> Michael Kerrisk
> Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
> Linux/UNIX System Programming Training: http://man7.org/training/



-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/


From ptesarik at suse.cz  Tue Jan 27 01:14:28 2015
From: ptesarik at suse.cz (Petr Tesarik)
Date: Tue, 27 Jan 2015 10:14:28 +0100
Subject: [RFD] efi assisted kdump
In-Reply-To: <20150127062141.GC2506@darkstar.redhat.com>
References: <20150124132637.GB2139@darkstar.redhat.com>
 <20150124170344.3b06a23e@hananiah.suse.cz>
 <20150127062141.GC2506@darkstar.redhat.com>
Message-ID: <20150127101428.75a736bb@hananiah.suse.cz>

On Tue, 27 Jan 2015 14:21:41 +0800
Dave Young <dyoung at redhat.com> wrote:

> Hi, Petr
> 
> On 01/24/15 at 05:03pm, Petr Tesarik wrote:
> > On Sat, 24 Jan 2015 21:26:37 +0800
> > Dave Young <dyoung at redhat.com> wrote:
> > 
> > > Hi,
> > > 
> > > Kdump has several limitations currently such as kdump kernel reboot will bypass
> > > device shutdown path so device drivers should reset during initialization.
> > > 
> > >[...]
> > > 
> > > ppc64 has a feature "firmware assisted kdump", see below documentation:
> > > Documentation/powerpc/firmware-assisted-dump.txt
> > 
> > Hi Dave,
> > 
> > while I'm no expert on either UEFI or IBM POWER, I'd like to warn you
> > that fadump (firmware-assisted dump) on PPC is not quite optimal in its
> > current form. One of the things that have always irritated me are
> > excessive RAM requirements.
> > 
> > The problem is that there is only one reboot in fadump - after saving
> > the dump, the secondary kernel discards the saved area and continues
> > booting as usual. However, many kernel structuers must be already
> > allocated at that point, e.g. the memmap array(s), but they are sized
> > by the total RAM, not the limited amount available to the 2nd kernel.
> 
> Thanks for the comment.
> 
> I'm not sure I understand the RAM requirements you mentioned. I think you
> are worrying about freeing oldmem to be used by 2nd kernel.
> 
> But we do not need to do same as power we can just reboot another time
> as long as vmcore is saved. The main advantage is the capture kernel can
> boot with all devices being reset. 

Yes, that's true. If you're fine with rebooting the machine twice (once
for the dump-taking and once for bringing up the normal system), then
there's no concern. The IBM folks apparently wanted to save some
downtime associated with going through the boot-up sequence. In fact,
you could kexec back into the normal system instead of rebooting, if
kexec can be used.

All I wanted to say is that fadump has its own issues, and we'd better
not repeat them with UEFI-based dumps. ;-)

Petr Tesarik


From vgoyal at redhat.com  Tue Jan 27 06:24:59 2015
From: vgoyal at redhat.com (Vivek Goyal)
Date: Tue, 27 Jan 2015 09:24:59 -0500
Subject: Edited kexec_load(2) [kexec_file_load()] man page for review
In-Reply-To: <54B91271.3000600@gmail.com>
References: <545FBDDD.9060801@gmail.com> <20141111213037.GA31445@redhat.com>
 <54ADA284.30502@gmail.com> <20150112221634.GD16162@redhat.com>
 <54B91271.3000600@gmail.com>
Message-ID: <20150127142459.GA12851@redhat.com>

On Fri, Jan 16, 2015 at 02:30:25PM +0100, Michael Kerrisk (man-pages) wrote:
[..]
> 

Hi Michael,

Please find my responses below. Sorry, I got stuck in other work and
forgot about this thread.

> So, returning to the kexeec_segment structure:
> 
>            struct kexec_segment {
>                void   *buf;        /* Buffer in user space */
>                size_t  bufsz;      /* Buffer length in user space */
>                void   *mem;        /* Physical address of kernel */
>                size_t  memsz;      /* Physical address length */
>            };
> 
> Are the following statements correct:
> * buf + bufsz identify a memory region in the caller's virtual 
>   address space that is the source of the copy

Yes.

> * mem + memsz specify the target memory region of the copy

Yes.

> * mem is  physical memory address, as seen from kernel space

Yes.

> * the number of bytes copied from userspace is min(bufsz, memsz)

Yes. bufsz can not be more than memsz. There is a check to validate
this in kernel.

	result = -EINVAL;
	for (i = 0; i < nr_segments; i++) {
		if (image->segment[i].bufsz > image->segment[i].memsz)
			return result;
	}

> * if bufsz > memsz, then excess bytes in the user-space buffer 
>   are ignored.

You will get -EINVAL.

> * if memsz > bufsz, then excess bytes in the target kernel buffer
>   are filled with zeros.

Yes.

> Also, it seems to me that 'mem' need not be page aligned.
> Is that correct? Should the man page say something about that?
> (E.g., is it generally desirable that 'mem' should be page aligned?)

mem and memsz need to be page aligned. There is a check for that too.

	mstart = image->segment[i].mem;
	mend   = mstart + image->segment[i].memsz;
	if ((mstart & ~PAGE_MASK) || (mend & ~PAGE_MASK))
		return result;

> 
> Likewise, 'memsz' doesn't need to beta page multiple, IIUC.
> Should the man page say anything about this? For example, should 
> it note that the initialized kernel segment will be of size:
> 
>      (mem % PAGE_SIZE + memsz) rounded up to the next multiple of PAGE_SIZE
> 
> And should it note that if 'mem' is not a multiple of the page size, then
> the initial bytes (mem % PAGE_SIZE)) in the first page of the kernel segment 
> will be zeros?
> 
> (Hopefully I have read kimage_load_normal_segment() correctly.)

Both mem and memsz need to be page aligned.

> 
> And one further question. Other than the fact that they are used with 
> different system calls, what is the difference between KEXEC_ON_CRASH 
> and KEXEC_FILE_ON_CRASH?

Right now I can't think of any other difference. They both tell respective
system call that this kernel needs to be loaded in reserved memory region
for crash kernel.

Thanks
Vivek


From mark.rutland at arm.com  Tue Jan 27 08:46:20 2015
From: mark.rutland at arm.com (Mark Rutland)
Date: Tue, 27 Jan 2015 16:46:20 +0000
Subject: [PATCH 4/8] arm64: Add EL2 switch to soft_restart
In-Reply-To: <1422308928.21823.16.camel@infradead.org>
References: <cover.1421449714.git.geoff@infradead.org>
 <bb44d4eee4fab15278e4a8829a0981878bc9acd7.1421449714.git.geoff@infradead.org>
 <20150126190223.GO23313@leverpostej>
 <1422308928.21823.16.camel@infradead.org>
Message-ID: <20150127164619.GL17721@leverpostej>

On Mon, Jan 26, 2015 at 09:48:48PM +0000, Geoff Levand wrote:
> Hi Mark,
> 
> On Mon, 2015-01-26 at 19:02 +0000, Mark Rutland wrote:
> > On Sat, Jan 17, 2015 at 12:23:34AM +0000, Geoff Levand wrote:
> > > When a CPU is reset it needs to be put into the exception level it had when it
> > > entered the kernel.  Update cpu_reset() to accept an argument el2_switch which
> > > signals cpu_reset() to enter the soft reset address at EL2.  If el2_switch is
> > > not set the soft reset address will be entered at EL1.
> > > 
> > > Update cpu_soft_restart() and soft_restart() to pass the return of
> > > is_hyp_mode_available() as the el2_switch value to cpu_reset().  Also update the
> > > comments of cpu_reset(), cpu_soft_restart() and soft_restart() to reflect this
> > > change.
> > > 
> > > Signed-off-by: Geoff Levand <geoff at infradead.org>
> > > ---
> > >  arch/arm64/include/asm/proc-fns.h |  4 ++--
> > >  arch/arm64/kernel/process.c       | 10 ++++++++-
> > >  arch/arm64/mm/proc.S              | 47 +++++++++++++++++++++++++++++----------
> > >  3 files changed, 46 insertions(+), 15 deletions(-)
> > > 
> > > diff --git a/arch/arm64/include/asm/proc-fns.h b/arch/arm64/include/asm/proc-fns.h
> > > index 9a8fd84..339394d 100644
> > > --- a/arch/arm64/include/asm/proc-fns.h
> > > +++ b/arch/arm64/include/asm/proc-fns.h
> > > @@ -32,8 +32,8 @@ extern void cpu_cache_off(void);
> > >  extern void cpu_do_idle(void);
> > >  extern void cpu_do_switch_mm(unsigned long pgd_phys, struct mm_struct *mm);
> > >  extern void cpu_reset(unsigned long addr) __attribute__((noreturn));
> > > -void cpu_soft_restart(phys_addr_t cpu_reset,
> > > -		unsigned long addr) __attribute__((noreturn));
> > > +void cpu_soft_restart(phys_addr_t cpu_reset, unsigned long el2_switch,
> > > +		      unsigned long addr) __attribute__((noreturn));
> > >  extern void cpu_do_suspend(struct cpu_suspend_ctx *ptr);
> > >  extern u64 cpu_do_resume(phys_addr_t ptr, u64 idmap_ttbr);
> > >  
> > > diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c
> > > index fde9923..371bbf1 100644
> > > --- a/arch/arm64/kernel/process.c
> > > +++ b/arch/arm64/kernel/process.c
> > > @@ -50,6 +50,7 @@
> > >  #include <asm/mmu_context.h>
> > >  #include <asm/processor.h>
> > >  #include <asm/stacktrace.h>
> > > +#include <asm/virt.h>
> > >  
> > >  #ifdef CONFIG_CC_STACKPROTECTOR
> > >  #include <linux/stackprotector.h>
> > > @@ -60,7 +61,14 @@ EXPORT_SYMBOL(__stack_chk_guard);
> > >  void soft_restart(unsigned long addr)
> > >  {
> > >  	setup_mm_for_reboot();
> > > -	cpu_soft_restart(virt_to_phys(cpu_reset), addr);
> > > +
> > > +	/* TODO: Remove this conditional when KVM can support CPU restart. */
> > > +	if (IS_ENABLED(CONFIG_KVM))
> > > +		cpu_soft_restart(virt_to_phys(cpu_reset), 0, addr);
> > 
> > If we haven't torn down KVM, doesn't that mean that KVM is active at EL2
> > (with MMU and caches on) at this point?
> > 
> > If that's the case then we cannot possibly try to call kexec(), because
> > we cannot touch the memory used by the page tables for those EL2
> > mappings. Things will explode if we do.
> 
> This conditional is just if KVM, do things the old way (don't try to
> switch exception levels).  It is to handle the system shutdown case.

Having grepped treewide for soft_restart, other than kexec there are no
users for arm64. So surely kexec is the only case to cater for at the
moment?

> Another patch in this series '[PATCH 7/8] arm64/kexec: Add checks for
> KVM' assures kexec cannot happen when KVM is configured.

It would be better to just move this earlier (or event better, implement
kvm teardown).

Mark.


From catalin.marinas at arm.com  Tue Jan 27 09:39:47 2015
From: catalin.marinas at arm.com (Catalin Marinas)
Date: Tue, 27 Jan 2015 17:39:47 +0000
Subject: [PATCH 3/8] arm64: Add new hcall HVC_CALL_FUNC
In-Reply-To: <3f1795bc1dad1d0510a9d24a168ecd5d83666507.1421449714.git.geoff@infradead.org>
References: <cover.1421449714.git.geoff@infradead.org>
 <3f1795bc1dad1d0510a9d24a168ecd5d83666507.1421449714.git.geoff@infradead.org>
Message-ID: <20150127173947.GJ3226@e104818-lin.cambridge.arm.com>

On Sat, Jan 17, 2015 at 12:23:34AM +0000, Geoff Levand wrote:
> diff --git a/arch/arm64/include/asm/virt.h b/arch/arm64/include/asm/virt.h
> index 99c319c..4f23a48 100644
> --- a/arch/arm64/include/asm/virt.h
> +++ b/arch/arm64/include/asm/virt.h
> @@ -41,6 +41,19 @@
>  
>  #define HVC_CALL_HYP 3
>  
> +/*
> + * HVC_CALL_FUNC - Execute a function at EL2.
> + *
> + * @x0: Physical address of the function to be executed.
> + * @x1: Passed as the first argument to the function.
> + * @x2: Passed as the second argument to the function.
> + * @x3: Passed as the third argument to the function.
> + *
> + * The called function must preserve the contents of register x18.

Can you pick a register that's normally callee saved?

> + */
> +
> +#define HVC_CALL_FUNC 4
> +
>  #ifndef __ASSEMBLY__
>  
>  /*
> diff --git a/arch/arm64/kernel/hyp-stub.S b/arch/arm64/kernel/hyp-stub.S
> index e3db3fd..b5d36e7 100644
> --- a/arch/arm64/kernel/hyp-stub.S
> +++ b/arch/arm64/kernel/hyp-stub.S
> @@ -66,9 +66,20 @@ el1_sync:
>  	mrs	x0, vbar_el2
>  	b	2f
>  
> -1:	cmp	x18, #HVC_SET_VECTORS
> -	b.ne	2f
> -	msr	vbar_el2, x0
> +1:	cmp     x18, #HVC_SET_VECTORS

This line doesn't seem to have any change, apart from some whitespace.
Or did you want to drop the label?

> +	b.ne    1f
> +	msr     vbar_el2, x0
> +	b       2f
> +
> +1:	cmp     x18, #HVC_CALL_FUNC
> +	b.ne    2f
> +	mov     x18, lr
> +	mov     lr, x0
> +	mov     x0, x1
> +	mov     x1, x2
> +	mov     x2, x3
> +	blr     lr
> +	mov     lr, x18
>  
>  2:	eret
>  ENDPROC(el1_sync)

What is the calling convention for this HVC? You mentioned x18 above but
what about other registers that the called function may corrupt (x18 is
a temporary register, so it's not expected to be callee saved).

-- 
Catalin


From catalin.marinas at arm.com  Tue Jan 27 09:57:40 2015
From: catalin.marinas at arm.com (Catalin Marinas)
Date: Tue, 27 Jan 2015 17:57:40 +0000
Subject: [PATCH 4/8] arm64: Add EL2 switch to soft_restart
In-Reply-To: <bb44d4eee4fab15278e4a8829a0981878bc9acd7.1421449714.git.geoff@infradead.org>
References: <cover.1421449714.git.geoff@infradead.org>
 <bb44d4eee4fab15278e4a8829a0981878bc9acd7.1421449714.git.geoff@infradead.org>
Message-ID: <20150127175740.GK3226@e104818-lin.cambridge.arm.com>

On Sat, Jan 17, 2015 at 12:23:34AM +0000, Geoff Levand wrote:
>  ENTRY(cpu_reset)
> -	mrs	x1, sctlr_el1
> -	bic	x1, x1, #1
> -	msr	sctlr_el1, x1			// disable the MMU
> +	mrs	x2, sctlr_el1
> +	bic	x2, x2, #1
> +	msr	sctlr_el1, x2			// disable the MMU
>  	isb
> -	ret	x0
> +
> +	cbz	x0, 1f				// el2_switch?
> +	mov	x0, x1
> +	mov	x1, xzr
> +	mov	x2, xzr
> +	mov	x3, xzr
> +	hvc	#HVC_CALL_FUNC			// no return

If that's the only user of HVC_CALL_FUNC, why do we bother with
arguments, calling convention?

-- 
Catalin


From mark.rutland at arm.com  Tue Jan 27 10:00:51 2015
From: mark.rutland at arm.com (Mark Rutland)
Date: Tue, 27 Jan 2015 18:00:51 +0000
Subject: [PATCH 3/8] arm64: Add new hcall HVC_CALL_FUNC
In-Reply-To: <20150127173947.GJ3226@e104818-lin.cambridge.arm.com>
References: <cover.1421449714.git.geoff@infradead.org>
 <3f1795bc1dad1d0510a9d24a168ecd5d83666507.1421449714.git.geoff@infradead.org>
 <20150127173947.GJ3226@e104818-lin.cambridge.arm.com>
Message-ID: <20150127180051.GN17721@leverpostej>

On Tue, Jan 27, 2015 at 05:39:47PM +0000, Catalin Marinas wrote:
> On Sat, Jan 17, 2015 at 12:23:34AM +0000, Geoff Levand wrote:
> > diff --git a/arch/arm64/include/asm/virt.h b/arch/arm64/include/asm/virt.h
> > index 99c319c..4f23a48 100644
> > --- a/arch/arm64/include/asm/virt.h
> > +++ b/arch/arm64/include/asm/virt.h
> > @@ -41,6 +41,19 @@
> >  
> >  #define HVC_CALL_HYP 3
> >  
> > +/*
> > + * HVC_CALL_FUNC - Execute a function at EL2.
> > + *
> > + * @x0: Physical address of the function to be executed.
> > + * @x1: Passed as the first argument to the function.
> > + * @x2: Passed as the second argument to the function.
> > + * @x3: Passed as the third argument to the function.
> > + *
> > + * The called function must preserve the contents of register x18.
> 
> Can you pick a register that's normally callee saved?

We're in the hyp-stub, so we don't have a stack in EL2. Therefore we
can't stack any of the existing callee-saved register values in order to
be able to use them.

One way to avoid that would be to have asm block which issues the HVC at
EL1 stack/unstack the LR around the HVC. Then we're free to corrupt the
LR at EL2 in order to call the provided function.

[...]

> > +1:	cmp     x18, #HVC_CALL_FUNC
> > +	b.ne    2f
> > +	mov     x18, lr
> > +	mov     lr, x0
> > +	mov     x0, x1
> > +	mov     x1, x2
> > +	mov     x2, x3
> > +	blr     lr
> > +	mov     lr, x18
> >  
> >  2:	eret
> >  ENDPROC(el1_sync)
> 
> What is the calling convention for this HVC? You mentioned x18 above but
> what about other registers that the called function may corrupt (x18 is
> a temporary register, so it's not expected to be callee saved).

Other than x18, the usual PCS rules apply here. We don't have a stack,
so the function we call can't make a nested call to anything else.

Mark.


From geoff at infradead.org  Tue Jan 27 10:34:57 2015
From: geoff at infradead.org (Geoff Levand)
Date: Tue, 27 Jan 2015 10:34:57 -0800
Subject: [PATCH 4/8] arm64: Add EL2 switch to soft_restart
In-Reply-To: <20150127164619.GL17721@leverpostej>
References: <cover.1421449714.git.geoff@infradead.org>
 <bb44d4eee4fab15278e4a8829a0981878bc9acd7.1421449714.git.geoff@infradead.org>
 <20150126190223.GO23313@leverpostej>
 <1422308928.21823.16.camel@infradead.org>
 <20150127164619.GL17721@leverpostej>
Message-ID: <1422383697.21823.19.camel@infradead.org>

Hi Mark,

On Tue, 2015-01-27 at 16:46 +0000, Mark Rutland wrote:
> On Mon, Jan 26, 2015 at 09:48:48PM +0000, Geoff Levand wrote:
> > This conditional is just if KVM, do things the old way (don't try to
> > switch exception levels).  It is to handle the system shutdown case.
> 
> Having grepped treewide for soft_restart, other than kexec there are no
> users for arm64. So surely kexec is the only case to cater for at the
> moment?

Yes, I think you're right, and so it seems we can drop this patch and
just have the 'Add checks for KVM' patch.

> > Another patch in this series '[PATCH 7/8] arm64/kexec: Add checks for
> > KVM' assures kexec cannot happen when KVM is configured.
> 
> It would be better to just move this earlier (or event better, implement
> kvm teardown).

Yes, I hope we don't really need to have any KVM work-arounds.

-Geoff





From geoff at infradead.org  Tue Jan 27 11:33:18 2015
From: geoff at infradead.org (Geoff Levand)
Date: Tue, 27 Jan 2015 11:33:18 -0800
Subject: [PATCH V2 1/8] arm64: Fold proc-macros.S into assembler.h
In-Reply-To: <e81815fb841db3ea40ce8e2fd18cf021ec9c6cf7.1421449714.git.geoff@infradead.org>
References: <cover.1421449714.git.geoff@infradead.org>
 <e81815fb841db3ea40ce8e2fd18cf021ec9c6cf7.1421449714.git.geoff@infradead.org>
Message-ID: <1422387198.21823.23.camel@infradead.org>

To allow the assembler macros defined in arch/arm64/mm/proc-macros.S to be used
outside the mm code move the contents of proc-macros.S to
arch/arm64/include/asm/assembler.h, delete proc-macros.S, and fix up all
references to proc-macros.S.

Signed-off-by: Geoff Levand <geoff at infradead.org>
---
 arch/arm64/include/asm/assembler.h | 37 +++++++++++++++++++++++++-
 arch/arm64/mm/cache.S              |  2 --
 arch/arm64/mm/proc-macros.S        | 54 --------------------------------------
 arch/arm64/mm/proc.S               |  2 --
 4 files changed, 36 insertions(+), 59 deletions(-)
 delete mode 100644 arch/arm64/mm/proc-macros.S

diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h
index 5901480..80436d3 100644
--- a/arch/arm64/include/asm/assembler.h
+++ b/arch/arm64/include/asm/assembler.h
@@ -1,5 +1,5 @@
 /*
- * Based on arch/arm/include/asm/assembler.h
+ * Based on arch/arm/include/asm/assembler.h, arch/arm/mm/proc-macros.S
  *
  * Copyright (C) 1996-2000 Russell King
  * Copyright (C) 2012 ARM Ltd.
@@ -20,6 +20,7 @@
 #error "Only include this from assembly code"
 #endif
 
+#include <asm/asm-offsets.h>
 #include <asm/ptrace.h>
 #include <asm/thread_info.h>
 
@@ -155,3 +156,37 @@ lr	.req	x30		// link register
 #endif
 	orr	\rd, \lbits, \hbits, lsl #32
 	.endm
+
+/*
+ * vma_vm_mm - get mm pointer from vma pointer (vma->vm_mm)
+ */
+	.macro	vma_vm_mm, rd, rn
+	ldr	\rd, [\rn, #VMA_VM_MM]
+	.endm
+
+/*
+ * mmid - get context id from mm pointer (mm->context.id)
+ */
+	.macro	mmid, rd, rn
+	ldr	\rd, [\rn, #MM_CONTEXT_ID]
+	.endm
+
+/*
+ * dcache_line_size - get the minimum D-cache line size from the CTR register.
+ */
+	.macro	dcache_line_size, reg, tmp
+	mrs	\tmp, ctr_el0			// read CTR
+	ubfm	\tmp, \tmp, #16, #19		// cache line size encoding
+	mov	\reg, #4			// bytes per word
+	lsl	\reg, \reg, \tmp		// actual cache line size
+	.endm
+
+/*
+ * icache_line_size - get the minimum I-cache line size from the CTR register.
+ */
+	.macro	icache_line_size, reg, tmp
+	mrs	\tmp, ctr_el0			// read CTR
+	and	\tmp, \tmp, #0xf		// cache line size encoding
+	mov	\reg, #4			// bytes per word
+	lsl	\reg, \reg, \tmp		// actual cache line size
+	.endm
diff --git a/arch/arm64/mm/cache.S b/arch/arm64/mm/cache.S
index 2560e1e..2d7a67c 100644
--- a/arch/arm64/mm/cache.S
+++ b/arch/arm64/mm/cache.S
@@ -24,8 +24,6 @@
 #include <asm/cpufeature.h>
 #include <asm/alternative-asm.h>
 
-#include "proc-macros.S"
-
 /*
  *	__flush_dcache_all()
  *
diff --git a/arch/arm64/mm/proc-macros.S b/arch/arm64/mm/proc-macros.S
deleted file mode 100644
index 005d29e..0000000
--- a/arch/arm64/mm/proc-macros.S
+++ /dev/null
@@ -1,54 +0,0 @@
-/*
- * Based on arch/arm/mm/proc-macros.S
- *
- * Copyright (C) 2012 ARM Ltd.
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program.  If not, see <http://www.gnu.org/licenses/>.
- */
-
-#include <asm/asm-offsets.h>
-#include <asm/thread_info.h>
-
-/*
- * vma_vm_mm - get mm pointer from vma pointer (vma->vm_mm)
- */
-	.macro	vma_vm_mm, rd, rn
-	ldr	\rd, [\rn, #VMA_VM_MM]
-	.endm
-
-/*
- * mmid - get context id from mm pointer (mm->context.id)
- */
-	.macro	mmid, rd, rn
-	ldr	\rd, [\rn, #MM_CONTEXT_ID]
-	.endm
-
-/*
- * dcache_line_size - get the minimum D-cache line size from the CTR register.
- */
-	.macro	dcache_line_size, reg, tmp
-	mrs	\tmp, ctr_el0			// read CTR
-	ubfm	\tmp, \tmp, #16, #19		// cache line size encoding
-	mov	\reg, #4			// bytes per word
-	lsl	\reg, \reg, \tmp		// actual cache line size
-	.endm
-
-/*
- * icache_line_size - get the minimum I-cache line size from the CTR register.
- */
-	.macro	icache_line_size, reg, tmp
-	mrs	\tmp, ctr_el0			// read CTR
-	and	\tmp, \tmp, #0xf		// cache line size encoding
-	mov	\reg, #4			// bytes per word
-	lsl	\reg, \reg, \tmp		// actual cache line size
-	.endm
diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S
index 28eebfb..fe69f6e 100644
--- a/arch/arm64/mm/proc.S
+++ b/arch/arm64/mm/proc.S
@@ -26,8 +26,6 @@
 #include <asm/pgtable-hwdef.h>
 #include <asm/pgtable.h>
 
-#include "proc-macros.S"
-
 #ifdef CONFIG_ARM64_64K_PAGES
 #define TCR_TG_FLAGS	TCR_TG0_64K | TCR_TG1_64K
 #else
-- 
2.1.0





From mtk.manpages at gmail.com  Wed Jan 28 00:04:38 2015
From: mtk.manpages at gmail.com (Michael Kerrisk (man-pages))
Date: Wed, 28 Jan 2015 09:04:38 +0100
Subject: Edited kexec_load(2) [kexec_file_load()] man page for review
In-Reply-To: <20150127142459.GA12851@redhat.com>
References: <545FBDDD.9060801@gmail.com> <20141111213037.GA31445@redhat.com>
 <54ADA284.30502@gmail.com> <20150112221634.GD16162@redhat.com>
 <54B91271.3000600@gmail.com> <20150127142459.GA12851@redhat.com>
Message-ID: <54C89816.8030709@gmail.com>

Hi Vivek,

On 01/27/2015 03:24 PM, Vivek Goyal wrote:
> On Fri, Jan 16, 2015 at 02:30:25PM +0100, Michael Kerrisk (man-pages) wrote:
> [..]
>>
> 
> Hi Michael,
> 
> Please find my responses below. Sorry, I got stuck in other work and
> forgot about this thread.
> 
>> So, returning to the kexeec_segment structure:
>>
>>            struct kexec_segment {
>>                void   *buf;        /* Buffer in user space */
>>                size_t  bufsz;      /* Buffer length in user space */
>>                void   *mem;        /* Physical address of kernel */
>>                size_t  memsz;      /* Physical address length */
>>            };
>>
>> Are the following statements correct:
>> * buf + bufsz identify a memory region in the caller's virtual 
>>   address space that is the source of the copy
> 
> Yes.

Okay.

>> * mem + memsz specify the target memory region of the copy
> 
> Yes.

Okay.

>> * mem is  physical memory address, as seen from kernel space
> 
> Yes.

Okay.

>> * the number of bytes copied from userspace is min(bufsz, memsz)
> 
> Yes. bufsz can not be more than memsz. There is a check to validate
> this in kernel.
> 
> 	result = -EINVAL;
> 	for (i = 0; i < nr_segments; i++) {
> 		if (image->segment[i].bufsz > image->segment[i].memsz)
> 			return result;
> 	}

Okay. So it's more precise to leave discussion of min(bufz, memsz) 
out of the man page just to say: bufsz bytes are transferred; 
if bufsz < memsz, then the excess bytes in the target region are 
filled with zeros. Right?

>> * if bufsz > memsz, then excess bytes in the user-space buffer 
>>   are ignored.
> 
> You will get -EINVAL.

Okay.

>> * if memsz > bufsz, then excess bytes in the target kernel buffer
>>   are filled with zeros.
> 
> Yes.

Okay.

>> Also, it seems to me that 'mem' need not be page aligned.
>> Is that correct? Should the man page say something about that?
>> (E.g., is it generally desirable that 'mem' should be page aligned?)
> 
> mem and memsz need to be page aligned. There is a check for that too.
> 
> 	mstart = image->segment[i].mem;
> 	mend   = mstart + image->segment[i].memsz;
> 	if ((mstart & ~PAGE_MASK) || (mend & ~PAGE_MASK))
> 		return result;
> 
>>
>> Likewise, 'memsz' doesn't need to beta page multiple, IIUC.
>> Should the man page say anything about this? For example, should 
>> it note that the initialized kernel segment will be of size:
>>
>>      (mem % PAGE_SIZE + memsz) rounded up to the next multiple of PAGE_SIZE
>>
>> And should it note that if 'mem' is not a multiple of the page size, then
>> the initial bytes (mem % PAGE_SIZE)) in the first page of the kernel segment 
>> will be zeros?
>>
>> (Hopefully I have read kimage_load_normal_segment() correctly.)
> 
> Both mem and memsz need to be page aligned.

And the error if not is EADDRNOTAVAIL, right?

>> And one further question. Other than the fact that they are used with 
>> different system calls, what is the difference between KEXEC_ON_CRASH 
>> and KEXEC_FILE_ON_CRASH?
> 
> Right now I can't think of any other difference. They both tell respective
> system call that this kernel needs to be loaded in reserved memory region
> for crash kernel.

Okay.

I've made various adjustments to the page in the light of your comments 
above. Thanks!

Cheers,

Michael



-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/


From vgoyal at redhat.com  Wed Jan 28 06:48:03 2015
From: vgoyal at redhat.com (Vivek Goyal)
Date: Wed, 28 Jan 2015 09:48:03 -0500
Subject: Edited kexec_load(2) [kexec_file_load()] man page for review
In-Reply-To: <54C89816.8030709@gmail.com>
References: <545FBDDD.9060801@gmail.com> <20141111213037.GA31445@redhat.com>
 <54ADA284.30502@gmail.com> <20150112221634.GD16162@redhat.com>
 <54B91271.3000600@gmail.com> <20150127142459.GA12851@redhat.com>
 <54C89816.8030709@gmail.com>
Message-ID: <20150128144803.GC15342@redhat.com>

On Wed, Jan 28, 2015 at 09:04:38AM +0100, Michael Kerrisk (man-pages) wrote:

Hi Michael,

[..]
> >> * the number of bytes copied from userspace is min(bufsz, memsz)
> > 
> > Yes. bufsz can not be more than memsz. There is a check to validate
> > this in kernel.
> > 
> > 	result = -EINVAL;
> > 	for (i = 0; i < nr_segments; i++) {
> > 		if (image->segment[i].bufsz > image->segment[i].memsz)
> > 			return result;
> > 	}
> 
> Okay. So it's more precise to leave discussion of min(bufz, memsz) 
> out of the man page just to say: bufsz bytes are transferred; 
> if bufsz < memsz, then the excess bytes in the target region are 
> filled with zeros. Right?

Sounds good.

[..]
> > Both mem and memsz need to be page aligned.
> 
> And the error if not is EADDRNOTAVAIL, right?

Yes.

> 
> >> And one further question. Other than the fact that they are used with 
> >> different system calls, what is the difference between KEXEC_ON_CRASH 
> >> and KEXEC_FILE_ON_CRASH?
> > 
> > Right now I can't think of any other difference. They both tell respective
> > system call that this kernel needs to be loaded in reserved memory region
> > for crash kernel.
> 
> Okay.
> 
> I've made various adjustments to the page in the light of your comments 
> above. Thanks!

Thank you for following it up and improving kexec man page.

Thanks
Vivek


From mtk.manpages at gmail.com  Wed Jan 28 07:49:34 2015
From: mtk.manpages at gmail.com (Michael Kerrisk (man-pages))
Date: Wed, 28 Jan 2015 16:49:34 +0100
Subject: Edited kexec_load(2) [kexec_file_load()] man page for review
In-Reply-To: <20150128144803.GC15342@redhat.com>
References: <545FBDDD.9060801@gmail.com> <20141111213037.GA31445@redhat.com>
 <54ADA284.30502@gmail.com> <20150112221634.GD16162@redhat.com>
 <54B91271.3000600@gmail.com> <20150127142459.GA12851@redhat.com>
 <54C89816.8030709@gmail.com> <20150128144803.GC15342@redhat.com>
Message-ID: <CAKgNAkhhVq5fuqwH0aPdgR1bKkAPPPuS84hfAFg3grGEJNxS4w@mail.gmail.com>

[Dropping Andi into CC, which I should have done to start with, since
he wrote the original page, and might also have some comments]

Hello Vivek,

>> I've made various adjustments to the page in the light of your comments
>> above. Thanks!
>
> Thank you for following it up and improving kexec man page.

You're welcome. So, by now, I've made quite a lot of changes
(including adding a number of cases under ERRORS). I think the revised
kexec_load/kexec_file_load page is pretty much ready to go, but would
you be willing to give the text below a check over first?

Thanks

Michael

====

.\" Copyright (C) 2010 Intel Corporation, Author: Andi Kleen
.\" and Copyright 2014, Vivek Goyal <vgoyal at redhat.com>
.\" and Copyright (c) 2015, Michael Kerrisk <mtk.manpages at gmail.com>
.\"
.\" %%%LICENSE_START(VERBATIM)
.\" Permission is granted to make and distribute verbatim copies of this
.\" manual provided the copyright notice and this permission notice are
.\" preserved on all copies.
.\"
.\" Permission is granted to copy and distribute modified versions of this
.\" manual under the conditions for verbatim copying, provided that the
.\" entire resulting derived work is distributed under the terms of a
.\" permission notice identical to this one.
.\"
.\" Since the Linux kernel and libraries are constantly changing, this
.\" manual page may be incorrect or out-of-date.  The author(s) assume no
.\" responsibility for errors or omissions, or for damages resulting from
.\" the use of the information contained herein.  The author(s) may not
.\" have taken the same level of care in the production of this manual,
.\" which is licensed free of charge, as they might when working
.\" professionally.
.\"
.\" Formatted or processed versions of this manual, if unaccompanied by
.\" the source, must acknowledge the copyright and authors of this work.
.\" %%%LICENSE_END
.\"
.TH KEXEC_LOAD 2 2014-08-19 "Linux" "Linux Programmer's Manual"
.SH NAME
kexec_load, kexec_file_load \- load a new kernel for later execution
.SH SYNOPSIS
.nf
.B #include <linux/kexec.h>

.BI "long kexec_load(unsigned long " entry ", unsigned long " nr_segments ","
.BI "                struct kexec_segment *" segments \
", unsigned long " flags ");"

.BI "long kexec_file_load(int " kernel_fd ", int " initrd_fd ","
.br
.BI "                    unsigned long " cmdline_len  \
", const char *" cmdline ","
.BI "                    unsigned long " flags ");"

.fi
.IR Note :
There are no glibc wrappers for these system calls; see NOTES.
.SH DESCRIPTION
The
.BR kexec_load ()
system call loads a new kernel that can be executed later by
.BR reboot (2).
.PP
The
.I flags
argument is a bit mask that controls the operation of the call.
The following values can be specified in
.IR flags :
.TP
.BR KEXEC_ON_CRASH " (since Linux 2.6.13)"
Execute the new kernel automatically on a system crash.
This "crash kernel" is loaded into an area of reserved memory that
is determined at boot time using the
.I craskkernel
kernel command-line parameter.
The location of this reserved memory is exported to user space via the
.I /proc/iomem
file, in an entry labeled "Crash kernel".
A user-space application can parse this file and prepare a list of
segments (see below) that specify this reserved memory as destination.
If this flag is specified, the kernel checks that the
target segments specified in
.I segments
fall within the reserved region.
.TP
.BR KEXEC_PRESERVE_CONTEXT " (since Linux 2.6.27)"
Preserve the system hardware and
software states before executing the new kernel.
This could be used for system suspend.
This flag is available only if the kernel was configured with
.BR CONFIG_KEXEC_JUMP ,
and is effective only if
.I nr_segments
is greater than 0.
.PP
The high-order bits (corresponding to the mask 0xffff0000) of
.I flags
contain the architecture of the to-be-executed kernel.
Specify (OR) the constant
.B KEXEC_ARCH_DEFAULT
to use the current architecture,
or one of the following architecture constants
.BR KEXEC_ARCH_386 ,
.BR KEXEC_ARCH_68K ,
.BR KEXEC_ARCH_X86_64 ,
.BR KEXEC_ARCH_PPC ,
.BR KEXEC_ARCH_PPC64 ,
.BR KEXEC_ARCH_IA_64 ,
.BR KEXEC_ARCH_ARM ,
.BR KEXEC_ARCH_S390 ,
.BR KEXEC_ARCH_SH ,
.BR KEXEC_ARCH_MIPS ,
and
.BR KEXEC_ARCH_MIPS_LE .
The architecture must be executable on the CPU of the system.

The
.I entry
argument is the physical entry address in the kernel image.
The
.I nr_segments
argument is the number of segments pointed to by the
.I segments
pointer;
the kernel imposes an (arbitrary) limit of 16 on the number of segments.
The
.I segments
argument is an array of
.I kexec_segment
structures which define the kernel layout:
.in +4n
.nf

struct kexec_segment {
    void   *buf;        /* Buffer in user space */
    size_t  bufsz;      /* Buffer length in user space */
    void   *mem;        /* Physical address of kernel */
    size_t  memsz;      /* Physical address length */
};
.fi
.in
.PP
The kernel image defined by
.I segments
is copied from the calling process into
the kernel either in regular
memory or in reserved memory (if
.BR KEXEC_ON_CRASH
is set).
The kernel first performs various sanity checks on the
information passed in
.IR segments .
If these checks pass, the kernel copies the segment data to kernel memory.
Each segment specified in
.I segments
is copied as follows:
.IP * 3
.I buf
and
.I bufsz
identify a memory region in the caller's virtual address space
that is the source of the copy.
The value in
.I bufsz
may not exceed the value in the
.I memsz
field.
.IP *
.I mem
and
.I memsz
specify a physical address range that is the target of the copy.
The values specified in both fields must be multiples of
the system page size.
.IP *
.I bufsz
bytes are copied from the source buffer to the target kernel buffer.
If
.I bufsz
is less than
.IR memsz ,
then the excess bytes in the kernel buffer are zeroed out.
.PP
In case of a normal kexec (i.e., the
.BR KEXEC_ON_CRASH
flag is not set), the segment data is loaded in any available memory
and is moved to the final destination at kexec reboot time (e.g., when the
.BR kexec (8)
command is executed with the
.I \-e
option).

In case of kexec on panic (i.e., the
.BR KEXEC_ON_CRASH
flag is set), the segment data is
loaded to reserved memory at the time of the call, and, after a crash,
the kexec mechanism simply passes control to that kernel.

The
.BR kexec_load ()
system call is available only if the kernel was configured with
.BR CONFIG_KEXEC .
.SS kexec_file_load()
The
.BR kexec_file_load ()
system call is similar to
.BR kexec_load (),
but it takes a different set of arguments.
It reads the kernel to be loaded from the file referred to by the descriptor
.IR kernel_fd ,
and the initrd (initial RAM disk)
to be loaded from file referred to by the descriptor
.IR initrd_fd .
The
.IR cmdline
argument is a pointer to a buffer containing the command line
for the new kernel.
The
.IR cmdline_len
argument specifies size of the buffer.
The last byte in the buffer must be a null byte (\(aq\\0\(aq).

The
.IR flags
argument is a bit mask which modifies the behavior of the call.
The following values can be specified in
.IR flags :
.TP
.BR KEXEC_FILE_UNLOAD
Unload the currently loaded kernel.
.TP
.BR KEXEC_FILE_ON_CRASH
Load the new kernel in the memory region reserved for the crash kernel
(as for
.BR KEXEC_ON_CRASH).
This kernel is booted if the currently running kernel crashes.
.TP
.BR KEXEC_FILE_NO_INITRAMFS
Loading initrd/initramfs is optional.
Specify this flag if no initramfs is being loaded.
If this flag is set, the value passed in
.IR initrd_fd
is ignored.
.PP
The
.BR kexec_file_load ()
.\" See also http://lwn.net/Articles/603116/
system call was added to provide support for systems
where "kexec" loading should be restricted to
only kernels that are signed.
This system call is available only if the kernel was configured with
.BR CONFIG_KEXEC_FILE .
.SH RETURN VALUE
On success, these system calls returns 0.
On error, \-1 is returned and
.I errno
is set to indicate the error.
.SH ERRORS
.TP
.B EADDRNOTAVAIL
.\" See kernel/kexec.::sanity_check_segment_list in the 3.19 kernel source
The
.B KEXEC_ON_CRASH
flags was specified, but the region specified by the
.I mem
and
.I memsz
fields of one of the
.I segments
entries lies outside the range of memory reserved for the crash kernel.
.TP
.B EADDRNOTAVAIL
The value in a
.I mem
or
.I memsz
field in one of the
.I segments
entries is not a multiple of the system page size.
.TP
.B EBADF
.I kernel_fd
or
.I initrd_fd
is not a valid file descriptor.
.TP
.B EBUSY
Another crash kernel is already being loaded
or a crash kernel is already in use.
.TP
.B EINVAL
.I flags
is invalid.
.TP
.B EINVAL
The value of a
.I bufsz
field in one of the
.I segments
entries exceeds the value in the corresponding
.I memsz
field.
.TP
.B EINVAL
.IR nr_segments
exceeds
.BR KEXEC_SEGMENT_MAX
(16).
.TP
.B EINVAL
Two or more of the kernel target buffers overlap.
.TP
.B EINVAL
The value in
.I cmdline[cmdline_len-1]
is not \(aq\\0\(aq.
.TP
.B EINVAL
The file referred to by
.I kernel_fd
or
.I initrd_fd
is empty (length zero).
.TP
.B ENOMEM
Could not allocate memory.
.TP
.B ENOEXEC
.I kernel_fd
does not refer to an open file, or the kernel can't load this file.
.TP
.B EPERM
The caller does not have the
.BR CAP_SYS_BOOT
capability.
.SH VERSIONS
The
.BR kexec_load ()
system call first appeared in Linux 2.6.13.
The
.BR kexec_file_load ()
system call first appeared in Linux 3.17.
.SH CONFORMING TO
These system calls are Linux-specific.
.SH NOTES
Currently, there is no glibc support for these system calls.
Call them using
.BR syscall (2).
.SH SEE ALSO
.BR reboot (2),
.BR syscall (2),
.BR kexec (8)

The kernel source files
.IR Documentation/kdump/kdump.txt
and
.IR Documentation/kernel-parameters.txt .

-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/


From vgoyal at redhat.com  Wed Jan 28 12:34:02 2015
From: vgoyal at redhat.com (Vivek Goyal)
Date: Wed, 28 Jan 2015 15:34:02 -0500
Subject: Edited kexec_load(2) [kexec_file_load()] man page for review
In-Reply-To: <CAKgNAkhhVq5fuqwH0aPdgR1bKkAPPPuS84hfAFg3grGEJNxS4w@mail.gmail.com>
References: <545FBDDD.9060801@gmail.com> <20141111213037.GA31445@redhat.com>
 <54ADA284.30502@gmail.com> <20150112221634.GD16162@redhat.com>
 <54B91271.3000600@gmail.com> <20150127142459.GA12851@redhat.com>
 <54C89816.8030709@gmail.com> <20150128144803.GC15342@redhat.com>
 <CAKgNAkhhVq5fuqwH0aPdgR1bKkAPPPuS84hfAFg3grGEJNxS4w@mail.gmail.com>
Message-ID: <20150128203402.GG15342@redhat.com>

On Wed, Jan 28, 2015 at 04:49:34PM +0100, Michael Kerrisk (man-pages) wrote:
> [Dropping Andi into CC, which I should have done to start with, since
> he wrote the original page, and might also have some comments]
> 
> Hello Vivek,
> 
> >> I've made various adjustments to the page in the light of your comments
> >> above. Thanks!
> >
> > Thank you for following it up and improving kexec man page.
> 
> You're welcome. So, by now, I've made quite a lot of changes
> (including adding a number of cases under ERRORS). I think the revised
> kexec_load/kexec_file_load page is pretty much ready to go, but would
> you be willing to give the text below a check over first?
> 

Hi Michael,

I had a quick look and it looks good to me.

Thanks
Vivek


From lkml14 at scotdoyle.com  Wed Jan 28 13:14:03 2015
From: lkml14 at scotdoyle.com (Scot Doyle)
Date: Wed, 28 Jan 2015 21:14:03 +0000 (UTC)
Subject: Edited kexec_load(2) [kexec_file_load()] man page for review
In-Reply-To: <20150128203402.GG15342@redhat.com>
References: <545FBDDD.9060801@gmail.com> <20141111213037.GA31445@redhat.com>
 <54ADA284.30502@gmail.com> <20150112221634.GD16162@redhat.com>
 <54B91271.3000600@gmail.com> <20150127142459.GA12851@redhat.com>
 <54C89816.8030709@gmail.com> <20150128144803.GC15342@redhat.com>
 <CAKgNAkhhVq5fuqwH0aPdgR1bKkAPPPuS84hfAFg3grGEJNxS4w@mail.gmail.com>
 <20150128203402.GG15342@redhat.com>
Message-ID: <alpine.DEB.2.11.1501282104490.9275@localhost.localdomain>

On Wed, 28 Jan 2015, Vivek Goyal wrote:
> On Wed, Jan 28, 2015 at 04:49:34PM +0100, Michael Kerrisk (man-pages) wrote:
> > Hello Vivek,
> > 
> > >> I've made various adjustments to the page in the light of your comments
> > >> above. Thanks!
> > >
> > > Thank you for following it up and improving kexec man page.
> > 
> > You're welcome. So, by now, I've made quite a lot of changes
> > (including adding a number of cases under ERRORS). I think the revised
> > kexec_load/kexec_file_load page is pretty much ready to go, but would
> > you be willing to give the text below a check over first?
> > 
> 
> Hi Michael,
> 
> I had a quick look and it looks good to me.
> 
> Thanks
> Vivek

When I tested, kexec_file_load required CONFIG_RELOCATABLE. Is the same 
true for kexec_load? Would it make sense to note this in the man pages 
along with the need for CONFIG_KEXEC_FILE, etc? Or as an error message?

Thanks,
Scot



From vgoyal at redhat.com  Wed Jan 28 13:31:25 2015
From: vgoyal at redhat.com (Vivek Goyal)
Date: Wed, 28 Jan 2015 16:31:25 -0500
Subject: Edited kexec_load(2) [kexec_file_load()] man page for review
In-Reply-To: <alpine.DEB.2.11.1501282104490.9275@localhost.localdomain>
References: <20141111213037.GA31445@redhat.com> <54ADA284.30502@gmail.com>
 <20150112221634.GD16162@redhat.com> <54B91271.3000600@gmail.com>
 <20150127142459.GA12851@redhat.com> <54C89816.8030709@gmail.com>
 <20150128144803.GC15342@redhat.com>
 <CAKgNAkhhVq5fuqwH0aPdgR1bKkAPPPuS84hfAFg3grGEJNxS4w@mail.gmail.com>
 <20150128203402.GG15342@redhat.com>
 <alpine.DEB.2.11.1501282104490.9275@localhost.localdomain>
Message-ID: <20150128213125.GH15342@redhat.com>

On Wed, Jan 28, 2015 at 09:14:03PM +0000, Scot Doyle wrote:
> On Wed, 28 Jan 2015, Vivek Goyal wrote:
> > On Wed, Jan 28, 2015 at 04:49:34PM +0100, Michael Kerrisk (man-pages) wrote:
> > > Hello Vivek,
> > > 
> > > >> I've made various adjustments to the page in the light of your comments
> > > >> above. Thanks!
> > > >
> > > > Thank you for following it up and improving kexec man page.
> > > 
> > > You're welcome. So, by now, I've made quite a lot of changes
> > > (including adding a number of cases under ERRORS). I think the revised
> > > kexec_load/kexec_file_load page is pretty much ready to go, but would
> > > you be willing to give the text below a check over first?
> > > 
> > 
> > Hi Michael,
> > 
> > I had a quick look and it looks good to me.
> > 
> > Thanks
> > Vivek
> 
> When I tested, kexec_file_load required CONFIG_RELOCATABLE. Is the same 
> true for kexec_load? Would it make sense to note this in the man pages 
> along with the need for CONFIG_KEXEC_FILE, etc? Or as an error message?

Hmm.., I can't see an explicity dependency between RELOCATABLE and
KEXEC. Both KEXEC and KEXEC_FILE should be able to load a kernel
even if it had RELOCATABLE=n.

Just that kernel will run from the address it has been built for.

Thanks
Vivek


From lkml14 at scotdoyle.com  Wed Jan 28 14:10:59 2015
From: lkml14 at scotdoyle.com (Scot Doyle)
Date: Wed, 28 Jan 2015 22:10:59 +0000 (UTC)
Subject: Edited kexec_load(2) [kexec_file_load()] man page for review
In-Reply-To: <20150128213125.GH15342@redhat.com>
References: <20141111213037.GA31445@redhat.com> <54ADA284.30502@gmail.com>
 <20150112221634.GD16162@redhat.com> <54B91271.3000600@gmail.com>
 <20150127142459.GA12851@redhat.com> <54C89816.8030709@gmail.com>
 <20150128144803.GC15342@redhat.com>
 <CAKgNAkhhVq5fuqwH0aPdgR1bKkAPPPuS84hfAFg3grGEJNxS4w@mail.gmail.com>
 <20150128203402.GG15342@redhat.com>
 <alpine.DEB.2.11.1501282104490.9275@localhost.localdomain>
 <20150128213125.GH15342@redhat.com>
Message-ID: <alpine.DEB.2.11.1501282207570.5656@localhost.localdomain>

On Wed, 28 Jan 2015, Vivek Goyal wrote:
> On Wed, Jan 28, 2015 at 09:14:03PM +0000, Scot Doyle wrote:
> > On Wed, 28 Jan 2015, Vivek Goyal wrote:
> > > On Wed, Jan 28, 2015 at 04:49:34PM +0100, Michael Kerrisk (man-pages) wrote:
> > > > Hello Vivek,
> > > > 
> > > > >> I've made various adjustments to the page in the light of your comments
> > > > >> above. Thanks!
> > > > >
> > > > > Thank you for following it up and improving kexec man page.
> > > > 
> > > > You're welcome. So, by now, I've made quite a lot of changes
> > > > (including adding a number of cases under ERRORS). I think the revised
> > > > kexec_load/kexec_file_load page is pretty much ready to go, but would
> > > > you be willing to give the text below a check over first?
> > > > 
> > > 
> > > Hi Michael,
> > > 
> > > I had a quick look and it looks good to me.
> > > 
> > > Thanks
> > > Vivek
> > 
> > When I tested, kexec_file_load required CONFIG_RELOCATABLE. Is the same 
> > true for kexec_load? Would it make sense to note this in the man pages 
> > along with the need for CONFIG_KEXEC_FILE, etc? Or as an error message?
> 
> Hmm.., I can't see an explicity dependency between RELOCATABLE and
> KEXEC. Both KEXEC and KEXEC_FILE should be able to load a kernel
> even if it had RELOCATABLE=n.
> 
> Just that kernel will run from the address it has been built for.
> 
> Thanks
> Vivek

Confusing, right? kexec_file_load returns -ENOEXEC and dmesg says 
"kexec-bzImage64: XLF_CAN_BE_LOADED_ABOVE_4G is not set." which leads to
arch/x86/boot/header.S line 396:

#if defined(CONFIG_RELOCATABLE) && defined(CONFIG_X86_64)  
   /* kernel/boot_param/ramdisk could be loaded above 4g */
# define XLF1 XLF_CAN_BE_LOADED_ABOVE_4G
#else
# define XLF1 0
#endif



From vgoyal at redhat.com  Wed Jan 28 14:25:26 2015
From: vgoyal at redhat.com (Vivek Goyal)
Date: Wed, 28 Jan 2015 17:25:26 -0500
Subject: Edited kexec_load(2) [kexec_file_load()] man page for review
In-Reply-To: <alpine.DEB.2.11.1501282207570.5656@localhost.localdomain>
References: <20150112221634.GD16162@redhat.com> <54B91271.3000600@gmail.com>
 <20150127142459.GA12851@redhat.com> <54C89816.8030709@gmail.com>
 <20150128144803.GC15342@redhat.com>
 <CAKgNAkhhVq5fuqwH0aPdgR1bKkAPPPuS84hfAFg3grGEJNxS4w@mail.gmail.com>
 <20150128203402.GG15342@redhat.com>
 <alpine.DEB.2.11.1501282104490.9275@localhost.localdomain>
 <20150128213125.GH15342@redhat.com>
 <alpine.DEB.2.11.1501282207570.5656@localhost.localdomain>
Message-ID: <20150128222526.GJ15342@redhat.com>

On Wed, Jan 28, 2015 at 10:10:59PM +0000, Scot Doyle wrote:
> On Wed, 28 Jan 2015, Vivek Goyal wrote:
> > On Wed, Jan 28, 2015 at 09:14:03PM +0000, Scot Doyle wrote:
> > > On Wed, 28 Jan 2015, Vivek Goyal wrote:
> > > > On Wed, Jan 28, 2015 at 04:49:34PM +0100, Michael Kerrisk (man-pages) wrote:
> > > > > Hello Vivek,
> > > > > 
> > > > > >> I've made various adjustments to the page in the light of your comments
> > > > > >> above. Thanks!
> > > > > >
> > > > > > Thank you for following it up and improving kexec man page.
> > > > > 
> > > > > You're welcome. So, by now, I've made quite a lot of changes
> > > > > (including adding a number of cases under ERRORS). I think the revised
> > > > > kexec_load/kexec_file_load page is pretty much ready to go, but would
> > > > > you be willing to give the text below a check over first?
> > > > > 
> > > > 
> > > > Hi Michael,
> > > > 
> > > > I had a quick look and it looks good to me.
> > > > 
> > > > Thanks
> > > > Vivek
> > > 
> > > When I tested, kexec_file_load required CONFIG_RELOCATABLE. Is the same 
> > > true for kexec_load? Would it make sense to note this in the man pages 
> > > along with the need for CONFIG_KEXEC_FILE, etc? Or as an error message?
> > 
> > Hmm.., I can't see an explicity dependency between RELOCATABLE and
> > KEXEC. Both KEXEC and KEXEC_FILE should be able to load a kernel
> > even if it had RELOCATABLE=n.
> > 
> > Just that kernel will run from the address it has been built for.
> > 
> > Thanks
> > Vivek
> 
> Confusing, right? kexec_file_load returns -ENOEXEC and dmesg says 
> "kexec-bzImage64: XLF_CAN_BE_LOADED_ABOVE_4G is not set." which leads to
> arch/x86/boot/header.S line 396:
> 
> #if defined(CONFIG_RELOCATABLE) && defined(CONFIG_X86_64)  
>    /* kernel/boot_param/ramdisk could be loaded above 4g */
> # define XLF1 XLF_CAN_BE_LOADED_ABOVE_4G
> #else
> # define XLF1 0
> #endif

Ah, this one. Actually generic kexec file loading implementation does not
impose this restriction. It is the image specific loader part which
decides what kind of bzImage it can load.

Current implementation (kexec-bzimage64.c), is only supporting loading
bzImages which are 64bit and can be loaded above 4G. This simplifies
the implementation of loader.

But there is nothing which prevents one from implementing other image
loaders.

So instead of saying that kexec_file_load() depends on CONFIG_RELOCATABLE,
it might be better to say in man page that currently this system call
supports only loading a bzImage which is 64bit and which can be loaded
above 4G too.

Thanks
Vivek


From lkml14 at scotdoyle.com  Wed Jan 28 17:27:20 2015
From: lkml14 at scotdoyle.com (Scot Doyle)
Date: Thu, 29 Jan 2015 01:27:20 +0000 (UTC)
Subject: Edited kexec_load(2) [kexec_file_load()] man page for review
In-Reply-To: <20150128222526.GJ15342@redhat.com>
References: <20150112221634.GD16162@redhat.com> <54B91271.3000600@gmail.com>
 <20150127142459.GA12851@redhat.com> <54C89816.8030709@gmail.com>
 <20150128144803.GC15342@redhat.com>
 <CAKgNAkhhVq5fuqwH0aPdgR1bKkAPPPuS84hfAFg3grGEJNxS4w@mail.gmail.com>
 <20150128203402.GG15342@redhat.com>
 <alpine.DEB.2.11.1501282104490.9275@localhost.localdomain>
 <20150128213125.GH15342@redhat.com>
 <alpine.DEB.2.11.1501282207570.5656@localhost.localdomain>
 <20150128222526.GJ15342@redhat.com>
Message-ID: <alpine.DEB.2.11.1501290010120.6022@localhost.localdomain>

On Wed, 28 Jan 2015, Vivek Goyal wrote:
> On Wed, Jan 28, 2015 at 10:10:59PM +0000, Scot Doyle wrote:
> > On Wed, 28 Jan 2015, Vivek Goyal wrote:
> > > On Wed, Jan 28, 2015 at 09:14:03PM +0000, Scot Doyle wrote:
> > > > When I tested, kexec_file_load required CONFIG_RELOCATABLE. Is the same 
> > > > true for kexec_load? Would it make sense to note this in the man pages 
> > > > along with the need for CONFIG_KEXEC_FILE, etc? Or as an error message?
> > > 
> > > Hmm.., I can't see an explicity dependency between RELOCATABLE and
> > > KEXEC. Both KEXEC and KEXEC_FILE should be able to load a kernel
> > > even if it had RELOCATABLE=n.
> > > 
> > > Just that kernel will run from the address it has been built for.
> > > 
> > > Thanks
> > > Vivek
> > 
> > Confusing, right? kexec_file_load returns -ENOEXEC and dmesg says 
> > "kexec-bzImage64: XLF_CAN_BE_LOADED_ABOVE_4G is not set." which leads to
> > arch/x86/boot/header.S line 396:
> > 
> > #if defined(CONFIG_RELOCATABLE) && defined(CONFIG_X86_64)  
> >    /* kernel/boot_param/ramdisk could be loaded above 4g */
> > # define XLF1 XLF_CAN_BE_LOADED_ABOVE_4G
> > #else
> > # define XLF1 0
> > #endif
> 
> Ah, this one. Actually generic kexec file loading implementation does not
> impose this restriction. It is the image specific loader part which
> decides what kind of bzImage it can load.
> 
> Current implementation (kexec-bzimage64.c), is only supporting loading
> bzImages which are 64bit and can be loaded above 4G. This simplifies
> the implementation of loader.
> 
> But there is nothing which prevents one from implementing other image
> loaders.
> 
> So instead of saying that kexec_file_load() depends on CONFIG_RELOCATABLE,
> it might be better to say in man page that currently this system call
> supports only loading a bzImage which is 64bit and which can be loaded
> above 4G too.
> 
> Thanks
> Vivek

Thanks, I agree, and think it would make sense to list them as part of the
page's ENOEXEC error.


From mtk.manpages at gmail.com  Wed Jan 28 21:39:56 2015
From: mtk.manpages at gmail.com (Michael Kerrisk (man-pages))
Date: Thu, 29 Jan 2015 06:39:56 +0100
Subject: Edited kexec_load(2) [kexec_file_load()] man page for review
In-Reply-To: <alpine.DEB.2.11.1501290010120.6022@localhost.localdomain>
References: <20150112221634.GD16162@redhat.com> <54B91271.3000600@gmail.com>
 <20150127142459.GA12851@redhat.com> <54C89816.8030709@gmail.com>
 <20150128144803.GC15342@redhat.com>
 <CAKgNAkhhVq5fuqwH0aPdgR1bKkAPPPuS84hfAFg3grGEJNxS4w@mail.gmail.com>
 <20150128203402.GG15342@redhat.com>
 <alpine.DEB.2.11.1501282104490.9275@localhost.localdomain>
 <20150128213125.GH15342@redhat.com>
 <alpine.DEB.2.11.1501282207570.5656@localhost.localdomain>
 <20150128222526.GJ15342@redhat.com>
 <alpine.DEB.2.11.1501290010120.6022@localhost.localdomain>
Message-ID: <CAKgNAkjBBRmcpu8DuaGCCRhvXP_AFotK5xrHx2HaR6g_8omULw@mail.gmail.com>

On 29 January 2015 at 02:27, Scot Doyle <lkml14 at scotdoyle.com> wrote:
> On Wed, 28 Jan 2015, Vivek Goyal wrote:
>> On Wed, Jan 28, 2015 at 10:10:59PM +0000, Scot Doyle wrote:
>> > On Wed, 28 Jan 2015, Vivek Goyal wrote:
>> > > On Wed, Jan 28, 2015 at 09:14:03PM +0000, Scot Doyle wrote:
>> > > > When I tested, kexec_file_load required CONFIG_RELOCATABLE. Is the same
>> > > > true for kexec_load? Would it make sense to note this in the man pages
>> > > > along with the need for CONFIG_KEXEC_FILE, etc? Or as an error message?
>> > >
>> > > Hmm.., I can't see an explicity dependency between RELOCATABLE and
>> > > KEXEC. Both KEXEC and KEXEC_FILE should be able to load a kernel
>> > > even if it had RELOCATABLE=n.
>> > >
>> > > Just that kernel will run from the address it has been built for.
>> > >
>> > > Thanks
>> > > Vivek
>> >
>> > Confusing, right? kexec_file_load returns -ENOEXEC and dmesg says
>> > "kexec-bzImage64: XLF_CAN_BE_LOADED_ABOVE_4G is not set." which leads to
>> > arch/x86/boot/header.S line 396:
>> >
>> > #if defined(CONFIG_RELOCATABLE) && defined(CONFIG_X86_64)
>> >    /* kernel/boot_param/ramdisk could be loaded above 4g */
>> > # define XLF1 XLF_CAN_BE_LOADED_ABOVE_4G
>> > #else
>> > # define XLF1 0
>> > #endif
>>
>> Ah, this one. Actually generic kexec file loading implementation does not
>> impose this restriction. It is the image specific loader part which
>> decides what kind of bzImage it can load.
>>
>> Current implementation (kexec-bzimage64.c), is only supporting loading
>> bzImages which are 64bit and can be loaded above 4G. This simplifies
>> the implementation of loader.
>>
>> But there is nothing which prevents one from implementing other image
>> loaders.
>>
>> So instead of saying that kexec_file_load() depends on CONFIG_RELOCATABLE,
>> it might be better to say in man page that currently this system call
>> supports only loading a bzImage which is 64bit and which can be loaded
>> above 4G too.
>>
>> Thanks
>> Vivek
>
> Thanks, I agree, and think it would make sense to list them as part of the
> page's ENOEXEC error.

Scott, could you then phras a couple of sentences that capture thge
details, so I can add it to the ENOEXEC error?

Thanks,

Michael


-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/


From takahiro.akashi at linaro.org  Thu Jan 29 01:36:15 2015
From: takahiro.akashi at linaro.org (AKASHI Takahiro)
Date: Thu, 29 Jan 2015 18:36:15 +0900
Subject: [PATCH 7/8] arm64/kexec: Add checks for KVM
In-Reply-To: <20150126191942.GQ23313@leverpostej>
References: <cover.1421449714.git.geoff@infradead.org>
 <a3353cf2ac47f49c20bbbb88405301cbaa359805.1421449714.git.geoff@infradead.org>
 <20150126191942.GQ23313@leverpostej>
Message-ID: <54C9FF0F.9050300@linaro.org>

Hello,

On 01/27/2015 04:19 AM, Mark Rutland wrote:
> On Sat, Jan 17, 2015 at 12:23:34AM +0000, Geoff Levand wrote:
>> Add runtime checks that fail the arm64 kexec syscall for situations that would
>> result in system instability do to problems in the KVM kernel support.
>> These checks should be removed when the KVM problems are resolved fixed.
>>
>> Signed-off-by: Geoff Levand <geoff at infradead.org>
>> ---
>>   arch/arm64/kernel/machine_kexec.c | 10 ++++++++++
>>   1 file changed, 10 insertions(+)
>>
>> diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c
>> index 3d84759..a36459d 100644
>> --- a/arch/arm64/kernel/machine_kexec.c
>> +++ b/arch/arm64/kernel/machine_kexec.c
>> @@ -16,6 +16,9 @@
>>   #include <asm/cacheflush.h>
>>   #include <asm/system_misc.h>
>>
>> +/* TODO: Remove this include when KVM can support a kexec reboot. */
>> +#include <asm/virt.h>
>> +
>>   /* Global variables for the relocate_kernel routine. */
>>   extern const unsigned char relocate_new_kernel[];
>>   extern const unsigned long relocate_new_kernel_size;
>> @@ -100,6 +103,13 @@ int machine_kexec_prepare(struct kimage *image)
>>
>>   	kexec_image_info(image);
>>
>> +	/* TODO: Remove this message when KVM can support a kexec reboot. */
>> +	if (IS_ENABLED(CONFIG_KVM) && is_hyp_mode_available()) {
>> +		pr_err("%s: Your kernel is configured with KVM support (CONFIG_KVM=y) which currently does not allow for kexec re-boot.\n",
>> +			__func__);
>> +		return -ENOSYS;
>> +	}
>
> If you really don't want to implement KVM teardown, surely this should
> be at the start of the series, so we don't have a point in the middle
> where things may explode in this case?

I'm going to fix this KVM issue (teardown) in cooperation with Geoff.

Looking into kvm init code, kvm_arch_init() in particular,
I guess that teardown function (kvm_arch_exit()) should do
   (reverting kvm_timer_hyp_init() per cpu)
- stop arch timer

   (reverting cpu_init_hyp_mode() per cpu)
- flush TLB
- jump into identical mapping (using boot_hyp_pgd?)
- disable MMU?
- restore vbar_el2 to __hyp_stub_vectors (or NULL?)

   (reverting kvm_mmu_init())
- Do we need to free page tables and a bounce(trampoline) page?

Is this good enough for safely shutting down kvm?
Do I miss anything essential, or can I skip anyhing?

I really appreciate your comments.
-Takahiro AKASHI

> Mark.
>
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
>


From takahiro.akashi at linaro.org  Thu Jan 29 01:57:48 2015
From: takahiro.akashi at linaro.org (AKASHI Takahiro)
Date: Thu, 29 Jan 2015 18:57:48 +0900
Subject: [PATCH 7/8] arm64/kexec: Add checks for KVM
In-Reply-To: <20150126191942.GQ23313@leverpostej>
References: <cover.1421449714.git.geoff@infradead.org>
 <a3353cf2ac47f49c20bbbb88405301cbaa359805.1421449714.git.geoff@infradead.org>
 <20150126191942.GQ23313@leverpostej>
Message-ID: <54CA041C.6020403@linaro.org>

Hello,

On 01/27/2015 04:19 AM, Mark Rutland wrote:
> On Sat, Jan 17, 2015 at 12:23:34AM +0000, Geoff Levand wrote:
>> Add runtime checks that fail the arm64 kexec syscall for situations that would
>> result in system instability do to problems in the KVM kernel support.
>> These checks should be removed when the KVM problems are resolved fixed.
>>
>> Signed-off-by: Geoff Levand <geoff at infradead.org>
>> ---
>>   arch/arm64/kernel/machine_kexec.c | 10 ++++++++++
>>   1 file changed, 10 insertions(+)
>>
>> diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c
>> index 3d84759..a36459d 100644
>> --- a/arch/arm64/kernel/machine_kexec.c
>> +++ b/arch/arm64/kernel/machine_kexec.c
>> @@ -16,6 +16,9 @@
>>   #include <asm/cacheflush.h>
>>   #include <asm/system_misc.h>
>>
>> +/* TODO: Remove this include when KVM can support a kexec reboot. */
>> +#include <asm/virt.h>
>> +
>>   /* Global variables for the relocate_kernel routine. */
>>   extern const unsigned char relocate_new_kernel[];
>>   extern const unsigned long relocate_new_kernel_size;
>> @@ -100,6 +103,13 @@ int machine_kexec_prepare(struct kimage *image)
>>
>>   	kexec_image_info(image);
>>
>> +	/* TODO: Remove this message when KVM can support a kexec reboot. */
>> +	if (IS_ENABLED(CONFIG_KVM) && is_hyp_mode_available()) {
>> +		pr_err("%s: Your kernel is configured with KVM support (CONFIG_KVM=y) which currently does not allow for kexec re-boot.\n",
>> +			__func__);
>> +		return -ENOSYS;
>> +	}
>
> If you really don't want to implement KVM teardown, surely this should
> be at the start of the series, so we don't have a point in the middle
> where things may explode in this case?

I'm going to fix this KVM issue (teardown) in cooperation with Geoff.

Looking into kvm init code, kvm_arch_init() in particular,
I guess that teardown function (kvm_arch_exit()) should do
   (reverting kvm_timer_hyp_init() per cpu)
- stop arch timer

   (reverting cpu_init_hyp_mode() per cpu)
- flush TLB
- jump into identical mapping (using boot_hyp_pgd?)
- disable MMU?
- restore vbar_el2 to __hyp_stub_vectors (or NULL?)

   (reverting kvm_mmu_init())
- Do we need to free page tables and a bounce(trampoline) page?

Is this good enough for safely shutting down kvm?
Do I miss anything essential, or can I skip anyhing?

I really appreciate your comments.
-Takahiro AKASHI

> Mark.
>
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
>


From marc.zyngier at arm.com  Thu Jan 29 02:59:45 2015
From: marc.zyngier at arm.com (Marc Zyngier)
Date: Thu, 29 Jan 2015 10:59:45 +0000
Subject: [PATCH 7/8] arm64/kexec: Add checks for KVM
In-Reply-To: <54CA041C.6020403@linaro.org>
References: <cover.1421449714.git.geoff@infradead.org>
 <a3353cf2ac47f49c20bbbb88405301cbaa359805.1421449714.git.geoff@infradead.org>
 <20150126191942.GQ23313@leverpostej> <54CA041C.6020403@linaro.org>
Message-ID: <54CA12A1.8090508@arm.com>


On 29/01/15 09:57, AKASHI Takahiro wrote:
> Hello,
> 
> On 01/27/2015 04:19 AM, Mark Rutland wrote:
>> On Sat, Jan 17, 2015 at 12:23:34AM +0000, Geoff Levand wrote:
>>> Add runtime checks that fail the arm64 kexec syscall for situations that would
>>> result in system instability do to problems in the KVM kernel support.
>>> These checks should be removed when the KVM problems are resolved fixed.
>>>
>>> Signed-off-by: Geoff Levand <geoff at infradead.org>
>>> ---
>>>   arch/arm64/kernel/machine_kexec.c | 10 ++++++++++
>>>   1 file changed, 10 insertions(+)
>>>
>>> diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c
>>> index 3d84759..a36459d 100644
>>> --- a/arch/arm64/kernel/machine_kexec.c
>>> +++ b/arch/arm64/kernel/machine_kexec.c
>>> @@ -16,6 +16,9 @@
>>>   #include <asm/cacheflush.h>
>>>   #include <asm/system_misc.h>
>>>
>>> +/* TODO: Remove this include when KVM can support a kexec reboot. */
>>> +#include <asm/virt.h>
>>> +
>>>   /* Global variables for the relocate_kernel routine. */
>>>   extern const unsigned char relocate_new_kernel[];
>>>   extern const unsigned long relocate_new_kernel_size;
>>> @@ -100,6 +103,13 @@ int machine_kexec_prepare(struct kimage *image)
>>>
>>>   	kexec_image_info(image);
>>>
>>> +	/* TODO: Remove this message when KVM can support a kexec reboot. */
>>> +	if (IS_ENABLED(CONFIG_KVM) && is_hyp_mode_available()) {
>>> +		pr_err("%s: Your kernel is configured with KVM support (CONFIG_KVM=y) which currently does not allow for kexec re-boot.\n",
>>> +			__func__);
>>> +		return -ENOSYS;
>>> +	}
>>
>> If you really don't want to implement KVM teardown, surely this should
>> be at the start of the series, so we don't have a point in the middle
>> where things may explode in this case?
> 
> I'm going to fix this KVM issue (teardown) in cooperation with Geoff.
> 
> Looking into kvm init code, kvm_arch_init() in particular,
> I guess that teardown function (kvm_arch_exit()) should do
>    (reverting kvm_timer_hyp_init() per cpu)
> - stop arch timer

No need, there shouldn't be any guest running at that point.

>    (reverting cpu_init_hyp_mode() per cpu)
> - flush TLB
> - jump into identical mapping (using boot_hyp_pgd?)
> - disable MMU?

Yes, and for that, you need to go back to an idmap

> - restore vbar_el2 to __hyp_stub_vectors (or NULL?)

It doesn't matter, you want to stay in HYP for the next kernel.

>    (reverting kvm_mmu_init())
> - Do we need to free page tables and a bounce(trampoline) page?

I don't think that's useful, you've killed the kernel already.

> Is this good enough for safely shutting down kvm?
> Do I miss anything essential, or can I skip anyhing?

I've outlined the steps a couple of days ago there:
http://www.spinics.net/lists/arm-kernel/msg395177.html

and the outline above should give you a nice list of things to look at.
All in all, it is pretty straightforward, provided that you're careful
enough about (commit 5a677ce044f has most of the information, just read
it backward... ;-).

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...


From lkml14 at scotdoyle.com  Thu Jan 29 08:06:39 2015
From: lkml14 at scotdoyle.com (Scot Doyle)
Date: Thu, 29 Jan 2015 16:06:39 +0000 (UTC)
Subject: Edited kexec_load(2) [kexec_file_load()] man page for review
In-Reply-To: <CAKgNAkjBBRmcpu8DuaGCCRhvXP_AFotK5xrHx2HaR6g_8omULw@mail.gmail.com>
References: <20150112221634.GD16162@redhat.com> <54B91271.3000600@gmail.com>
 <20150127142459.GA12851@redhat.com> <54C89816.8030709@gmail.com>
 <20150128144803.GC15342@redhat.com>
 <CAKgNAkhhVq5fuqwH0aPdgR1bKkAPPPuS84hfAFg3grGEJNxS4w@mail.gmail.com>
 <20150128203402.GG15342@redhat.com>
 <alpine.DEB.2.11.1501282104490.9275@localhost.localdomain>
 <20150128213125.GH15342@redhat.com>
 <alpine.DEB.2.11.1501282207570.5656@localhost.localdomain>
 <20150128222526.GJ15342@redhat.com>
 <alpine.DEB.2.11.1501290010120.6022@localhost.localdomain>
 <CAKgNAkjBBRmcpu8DuaGCCRhvXP_AFotK5xrHx2HaR6g_8omULw@mail.gmail.com>
Message-ID: <alpine.DEB.2.11.1501291429001.1597@localhost.localdomain>

On Thu, 29 Jan 2015, Michael Kerrisk (man-pages) wrote:
> On 29 January 2015 at 02:27, Scot Doyle <lkml14 at scotdoyle.com> wrote:
> > On Wed, 28 Jan 2015, Vivek Goyal wrote:
> >> On Wed, Jan 28, 2015 at 10:10:59PM +0000, Scot Doyle wrote:
> >> > On Wed, 28 Jan 2015, Vivek Goyal wrote:
> >> > > On Wed, Jan 28, 2015 at 09:14:03PM +0000, Scot Doyle wrote:
> >> > > > When I tested, kexec_file_load required CONFIG_RELOCATABLE. Is the same
> >> > > > true for kexec_load? Would it make sense to note this in the man pages
> >> > > > along with the need for CONFIG_KEXEC_FILE, etc? Or as an error message?
> >> > >
> >> > > Hmm.., I can't see an explicity dependency between RELOCATABLE and
> >> > > KEXEC. Both KEXEC and KEXEC_FILE should be able to load a kernel
> >> > > even if it had RELOCATABLE=n.
> >> > >
> >> > > Just that kernel will run from the address it has been built for.
> >> > >
> >> > > Thanks
> >> > > Vivek
> >> >
> >> > Confusing, right? kexec_file_load returns -ENOEXEC and dmesg says
> >> > "kexec-bzImage64: XLF_CAN_BE_LOADED_ABOVE_4G is not set." which leads to
> >> > arch/x86/boot/header.S line 396:
> >> >
> >> > #if defined(CONFIG_RELOCATABLE) && defined(CONFIG_X86_64)
> >> >    /* kernel/boot_param/ramdisk could be loaded above 4g */
> >> > # define XLF1 XLF_CAN_BE_LOADED_ABOVE_4G
> >> > #else
> >> > # define XLF1 0
> >> > #endif
> >>
> >> Ah, this one. Actually generic kexec file loading implementation does not
> >> impose this restriction. It is the image specific loader part which
> >> decides what kind of bzImage it can load.
> >>
> >> Current implementation (kexec-bzimage64.c), is only supporting loading
> >> bzImages which are 64bit and can be loaded above 4G. This simplifies
> >> the implementation of loader.
> >>
> >> But there is nothing which prevents one from implementing other image
> >> loaders.
> >>
> >> So instead of saying that kexec_file_load() depends on CONFIG_RELOCATABLE,
> >> it might be better to say in man page that currently this system call
> >> supports only loading a bzImage which is 64bit and which can be loaded
> >> above 4G too.
> >>
> >> Thanks
> >> Vivek
> >
> > Thanks, I agree, and think it would make sense to list them as part of the
> > page's ENOEXEC error.
> 
> Scott, could you then phras a couple of sentences that capture thge
> details, so I can add it to the ENOEXEC error?
> 
> Thanks,
> 
> Michael

Yes, maybe something like "kernel_fd does not refer to an open file, or 
the file type is not supported. Currently, the file must be a bzImage
and contain an x86 kernel loadable above 4G in memory (see 
Documentation/x86/boot.txt)."?

boot.txt explains that loading above 4G implies 64-bit and is specified 
via a bit in xloadflags added in Linux 3.8.


From mark.rutland at arm.com  Thu Jan 29 10:47:47 2015
From: mark.rutland at arm.com (Mark Rutland)
Date: Thu, 29 Jan 2015 18:47:47 +0000
Subject: [PATCH 7/8] arm64/kexec: Add checks for KVM
In-Reply-To: <54CA12A1.8090508@arm.com>
References: <cover.1421449714.git.geoff@infradead.org>
 <a3353cf2ac47f49c20bbbb88405301cbaa359805.1421449714.git.geoff@infradead.org>
 <20150126191942.GQ23313@leverpostej> <54CA041C.6020403@linaro.org>
 <54CA12A1.8090508@arm.com>
Message-ID: <20150129184747.GV17721@leverpostej>

On Thu, Jan 29, 2015 at 10:59:45AM +0000, Marc Zyngier wrote:
> 
> On 29/01/15 09:57, AKASHI Takahiro wrote:
> > Hello,
> > 
> > On 01/27/2015 04:19 AM, Mark Rutland wrote:
> >> On Sat, Jan 17, 2015 at 12:23:34AM +0000, Geoff Levand wrote:
> >>> Add runtime checks that fail the arm64 kexec syscall for situations that would
> >>> result in system instability do to problems in the KVM kernel support.
> >>> These checks should be removed when the KVM problems are resolved fixed.
> >>>
> >>> Signed-off-by: Geoff Levand <geoff at infradead.org>
> >>> ---
> >>>   arch/arm64/kernel/machine_kexec.c | 10 ++++++++++
> >>>   1 file changed, 10 insertions(+)
> >>>
> >>> diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c
> >>> index 3d84759..a36459d 100644
> >>> --- a/arch/arm64/kernel/machine_kexec.c
> >>> +++ b/arch/arm64/kernel/machine_kexec.c
> >>> @@ -16,6 +16,9 @@
> >>>   #include <asm/cacheflush.h>
> >>>   #include <asm/system_misc.h>
> >>>
> >>> +/* TODO: Remove this include when KVM can support a kexec reboot. */
> >>> +#include <asm/virt.h>
> >>> +
> >>>   /* Global variables for the relocate_kernel routine. */
> >>>   extern const unsigned char relocate_new_kernel[];
> >>>   extern const unsigned long relocate_new_kernel_size;
> >>> @@ -100,6 +103,13 @@ int machine_kexec_prepare(struct kimage *image)
> >>>
> >>>   	kexec_image_info(image);
> >>>
> >>> +	/* TODO: Remove this message when KVM can support a kexec reboot. */
> >>> +	if (IS_ENABLED(CONFIG_KVM) && is_hyp_mode_available()) {
> >>> +		pr_err("%s: Your kernel is configured with KVM support (CONFIG_KVM=y) which currently does not allow for kexec re-boot.\n",
> >>> +			__func__);
> >>> +		return -ENOSYS;
> >>> +	}
> >>
> >> If you really don't want to implement KVM teardown, surely this should
> >> be at the start of the series, so we don't have a point in the middle
> >> where things may explode in this case?
> > 
> > I'm going to fix this KVM issue (teardown) in cooperation with Geoff.
> > 
> > Looking into kvm init code, kvm_arch_init() in particular,
> > I guess that teardown function (kvm_arch_exit()) should do
> >    (reverting kvm_timer_hyp_init() per cpu)
> > - stop arch timer
> 
> No need, there shouldn't be any guest running at that point.
> 
> >    (reverting cpu_init_hyp_mode() per cpu)
> > - flush TLB
> > - jump into identical mapping (using boot_hyp_pgd?)
> > - disable MMU?
> 
> Yes, and for that, you need to go back to an idmap
> 
> > - restore vbar_el2 to __hyp_stub_vectors (or NULL?)
> 
> It doesn't matter, you want to stay in HYP for the next kernel.

Well, it depends on how the teardown is implemented. I'd imagined we'd
have KVM tear itself down (restoring the hyp stub) as part of shut down.
Later kexec/soft_restart would call the stub to get back to EL2.

That way kexec doesn't need to know anything about KVM, and it has one
path that works regardless of whether KVM is compiled into the kernel.

Mark.


From takahiro.akashi at linaro.org  Thu Jan 29 22:10:53 2015
From: takahiro.akashi at linaro.org (AKASHI Takahiro)
Date: Fri, 30 Jan 2015 15:10:53 +0900
Subject: [PATCH 7/8] arm64/kexec: Add checks for KVM
In-Reply-To: <20150129184747.GV17721@leverpostej>
References: <cover.1421449714.git.geoff@infradead.org>
 <a3353cf2ac47f49c20bbbb88405301cbaa359805.1421449714.git.geoff@infradead.org>
 <20150126191942.GQ23313@leverpostej> <54CA041C.6020403@linaro.org>
 <54CA12A1.8090508@arm.com> <20150129184747.GV17721@leverpostej>
Message-ID: <54CB206D.7040705@linaro.org>

Hello Marc, Mark

Thank you for your useful comments.
I need to look at Marc's commit(5a677ce044f) more carefully
about trampoline code.

On 01/30/2015 03:47 AM, Mark Rutland wrote:
> On Thu, Jan 29, 2015 at 10:59:45AM +0000, Marc Zyngier wrote:
>>
>> On 29/01/15 09:57, AKASHI Takahiro wrote:
>>> Hello,
>>>
>>> On 01/27/2015 04:19 AM, Mark Rutland wrote:
>>>> On Sat, Jan 17, 2015 at 12:23:34AM +0000, Geoff Levand wrote:
>>>>> Add runtime checks that fail the arm64 kexec syscall for situations that would
>>>>> result in system instability do to problems in the KVM kernel support.
>>>>> These checks should be removed when the KVM problems are resolved fixed.
>>>>>
>>>>> Signed-off-by: Geoff Levand <geoff at infradead.org>
>>>>> ---
>>>>>    arch/arm64/kernel/machine_kexec.c | 10 ++++++++++
>>>>>    1 file changed, 10 insertions(+)
>>>>>
>>>>> diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c
>>>>> index 3d84759..a36459d 100644
>>>>> --- a/arch/arm64/kernel/machine_kexec.c
>>>>> +++ b/arch/arm64/kernel/machine_kexec.c
>>>>> @@ -16,6 +16,9 @@
>>>>>    #include <asm/cacheflush.h>
>>>>>    #include <asm/system_misc.h>
>>>>>
>>>>> +/* TODO: Remove this include when KVM can support a kexec reboot. */
>>>>> +#include <asm/virt.h>
>>>>> +
>>>>>    /* Global variables for the relocate_kernel routine. */
>>>>>    extern const unsigned char relocate_new_kernel[];
>>>>>    extern const unsigned long relocate_new_kernel_size;
>>>>> @@ -100,6 +103,13 @@ int machine_kexec_prepare(struct kimage *image)
>>>>>
>>>>>    	kexec_image_info(image);
>>>>>
>>>>> +	/* TODO: Remove this message when KVM can support a kexec reboot. */
>>>>> +	if (IS_ENABLED(CONFIG_KVM) && is_hyp_mode_available()) {
>>>>> +		pr_err("%s: Your kernel is configured with KVM support (CONFIG_KVM=y) which currently does not allow for kexec re-boot.\n",
>>>>> +			__func__);
>>>>> +		return -ENOSYS;
>>>>> +	}
>>>>
>>>> If you really don't want to implement KVM teardown, surely this should
>>>> be at the start of the series, so we don't have a point in the middle
>>>> where things may explode in this case?
>>>
>>> I'm going to fix this KVM issue (teardown) in cooperation with Geoff.
>>>
>>> Looking into kvm init code, kvm_arch_init() in particular,
>>> I guess that teardown function (kvm_arch_exit()) should do
>>>     (reverting kvm_timer_hyp_init() per cpu)
>>> - stop arch timer
>>
>> No need, there shouldn't be any guest running at that point.
>>
>>>     (reverting cpu_init_hyp_mode() per cpu)
>>> - flush TLB
>>> - jump into identical mapping (using boot_hyp_pgd?)
>>> - disable MMU?
>>
>> Yes, and for that, you need to go back to an idmap
>>
>>> - restore vbar_el2 to __hyp_stub_vectors (or NULL?)
>>
>> It doesn't matter, you want to stay in HYP for the next kernel.
>
> Well, it depends on how the teardown is implemented. I'd imagined we'd
> have KVM tear itself down (restoring the hyp stub) as part of shut down.
> Later kexec/soft_restart would call the stub to get back to EL2.
>
> That way kexec doesn't need to know anything about KVM, and it has one
> path that works regardless of whether KVM is compiled into the kernel.

Initially, I thought that we would define kvm_arch_exit() and call it
somewhere in the middle of kexec path (no idea yet).
But Geoff suggested me to implement a new hvc call, HVC_CPU_SHUTDOWN(??),
and make it called via cpu_notifier(CPU_DYING_FROZEN) initiated by
machine_shutdown() from kernel_kexec().
(As you know, a hook is already there for cpu online in kvm/arm.c.)

Is this the right place to put teardown function?
(I'm not sure if this hook will be called also on a boot cpu.)

Thanks,
-Takahiro AKASHI



> Mark.
>


From mark.rutland at arm.com  Fri Jan 30 04:14:45 2015
From: mark.rutland at arm.com (Mark Rutland)
Date: Fri, 30 Jan 2015 12:14:45 +0000
Subject: [PATCH 7/8] arm64/kexec: Add checks for KVM
In-Reply-To: <54CB206D.7040705@linaro.org>
References: <cover.1421449714.git.geoff@infradead.org>
 <a3353cf2ac47f49c20bbbb88405301cbaa359805.1421449714.git.geoff@infradead.org>
 <20150126191942.GQ23313@leverpostej> <54CA041C.6020403@linaro.org>
 <54CA12A1.8090508@arm.com> <20150129184747.GV17721@leverpostej>
 <54CB206D.7040705@linaro.org>
Message-ID: <20150130121444.GA9730@leverpostej>

On Fri, Jan 30, 2015 at 06:10:53AM +0000, AKASHI Takahiro wrote:
> Hello Marc, Mark
> 
> Thank you for your useful comments.
> I need to look at Marc's commit(5a677ce044f) more carefully
> about trampoline code.
> 
> On 01/30/2015 03:47 AM, Mark Rutland wrote:
> > On Thu, Jan 29, 2015 at 10:59:45AM +0000, Marc Zyngier wrote:
> >>
> >> On 29/01/15 09:57, AKASHI Takahiro wrote:
> >>> Hello,
> >>>
> >>> On 01/27/2015 04:19 AM, Mark Rutland wrote:
> >>>> On Sat, Jan 17, 2015 at 12:23:34AM +0000, Geoff Levand wrote:
> >>>>> Add runtime checks that fail the arm64 kexec syscall for situations that would
> >>>>> result in system instability do to problems in the KVM kernel support.
> >>>>> These checks should be removed when the KVM problems are resolved fixed.
> >>>>>
> >>>>> Signed-off-by: Geoff Levand <geoff at infradead.org>
> >>>>> ---
> >>>>>    arch/arm64/kernel/machine_kexec.c | 10 ++++++++++
> >>>>>    1 file changed, 10 insertions(+)
> >>>>>
> >>>>> diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c
> >>>>> index 3d84759..a36459d 100644
> >>>>> --- a/arch/arm64/kernel/machine_kexec.c
> >>>>> +++ b/arch/arm64/kernel/machine_kexec.c
> >>>>> @@ -16,6 +16,9 @@
> >>>>>    #include <asm/cacheflush.h>
> >>>>>    #include <asm/system_misc.h>
> >>>>>
> >>>>> +/* TODO: Remove this include when KVM can support a kexec reboot. */
> >>>>> +#include <asm/virt.h>
> >>>>> +
> >>>>>    /* Global variables for the relocate_kernel routine. */
> >>>>>    extern const unsigned char relocate_new_kernel[];
> >>>>>    extern const unsigned long relocate_new_kernel_size;
> >>>>> @@ -100,6 +103,13 @@ int machine_kexec_prepare(struct kimage *image)
> >>>>>
> >>>>>    	kexec_image_info(image);
> >>>>>
> >>>>> +	/* TODO: Remove this message when KVM can support a kexec reboot. */
> >>>>> +	if (IS_ENABLED(CONFIG_KVM) && is_hyp_mode_available()) {
> >>>>> +		pr_err("%s: Your kernel is configured with KVM support (CONFIG_KVM=y) which currently does not allow for kexec re-boot.\n",
> >>>>> +			__func__);
> >>>>> +		return -ENOSYS;
> >>>>> +	}
> >>>>
> >>>> If you really don't want to implement KVM teardown, surely this should
> >>>> be at the start of the series, so we don't have a point in the middle
> >>>> where things may explode in this case?
> >>>
> >>> I'm going to fix this KVM issue (teardown) in cooperation with Geoff.
> >>>
> >>> Looking into kvm init code, kvm_arch_init() in particular,
> >>> I guess that teardown function (kvm_arch_exit()) should do
> >>>     (reverting kvm_timer_hyp_init() per cpu)
> >>> - stop arch timer
> >>
> >> No need, there shouldn't be any guest running at that point.
> >>
> >>>     (reverting cpu_init_hyp_mode() per cpu)
> >>> - flush TLB
> >>> - jump into identical mapping (using boot_hyp_pgd?)
> >>> - disable MMU?
> >>
> >> Yes, and for that, you need to go back to an idmap
> >>
> >>> - restore vbar_el2 to __hyp_stub_vectors (or NULL?)
> >>
> >> It doesn't matter, you want to stay in HYP for the next kernel.
> >
> > Well, it depends on how the teardown is implemented. I'd imagined we'd
> > have KVM tear itself down (restoring the hyp stub) as part of shut down.
> > Later kexec/soft_restart would call the stub to get back to EL2.
> >
> > That way kexec doesn't need to know anything about KVM, and it has one
> > path that works regardless of whether KVM is compiled into the kernel.
> 
> Initially, I thought that we would define kvm_arch_exit() and call it
> somewhere in the middle of kexec path (no idea yet).
> But Geoff suggested me to implement a new hvc call, HVC_CPU_SHUTDOWN(??),
> and make it called via cpu_notifier(CPU_DYING_FROZEN) initiated by
> machine_shutdown() from kernel_kexec().
> (As you know, a hook is already there for cpu online in kvm/arm.c.)

I think it would make far more sense to have the KVM teardown entirely
contained within KVM (e.g. in kvm_arch_exit). That way there's no need
for an interdependency between KVM and kexec, and we'd be a step closer
to KVM as a module.

I'm not keen on having KVM teardown in the middle of the kexec path.

Mark.


From linux at rasmusvillemoes.dk  Fri Jan 30 05:32:52 2015
From: linux at rasmusvillemoes.dk (Rasmus Villemoes)
Date: Fri, 30 Jan 2015 14:32:52 +0100
Subject: [PATCH] kernel/kexec.c: Use strchrnul
Message-ID: <1422624772-20285-1-git-send-email-linux@rasmusvillemoes.dk>

strchrnul exists precisely for this use case.

Signed-off-by: Rasmus Villemoes <linux at rasmusvillemoes.dk>
---
 kernel/kexec.c | 5 +----
 1 file changed, 1 insertion(+), 4 deletions(-)

diff --git a/kernel/kexec.c b/kernel/kexec.c
index 9a8a01abbaed..70e7050f7ab9 100644
--- a/kernel/kexec.c
+++ b/kernel/kexec.c
@@ -1792,12 +1792,9 @@ static __init char *get_last_crashkernel(char *cmdline,
 	/* find crashkernel and use the last one if there are more */
 	p = strstr(p, name);
 	while (p) {
-		char *end_p = strchr(p, ' ');
+		char *end_p = strchrnul(p, ' ');
 		char *q;
 
-		if (!end_p)
-			end_p = p + strlen(p);
-
 		if (!suffix) {
 			int i;
 
-- 
2.1.3



From mtk.manpages at gmail.com  Fri Jan 30 07:25:06 2015
From: mtk.manpages at gmail.com (Michael Kerrisk (man-pages))
Date: Fri, 30 Jan 2015 16:25:06 +0100
Subject: Edited kexec_load(2) [kexec_file_load()] man page for review
In-Reply-To: <alpine.DEB.2.11.1501291429001.1597@localhost.localdomain>
References: <20150112221634.GD16162@redhat.com> <54B91271.3000600@gmail.com>
 <20150127142459.GA12851@redhat.com> <54C89816.8030709@gmail.com>
 <20150128144803.GC15342@redhat.com>
 <CAKgNAkhhVq5fuqwH0aPdgR1bKkAPPPuS84hfAFg3grGEJNxS4w@mail.gmail.com>
 <20150128203402.GG15342@redhat.com>
 <alpine.DEB.2.11.1501282104490.9275@localhost.localdomain>
 <20150128213125.GH15342@redhat.com>
 <alpine.DEB.2.11.1501282207570.5656@localhost.localdomain>
 <20150128222526.GJ15342@redhat.com>
 <alpine.DEB.2.11.1501290010120.6022@localhost.localdomain>
 <CAKgNAkjBBRmcpu8DuaGCCRhvXP_AFotK5xrHx2HaR6g_8omULw@mail.gmail.com>
 <alpine.DEB.2.11.1501291429001.1597@localhost.localdomain>
Message-ID: <54CBA252.4050003@gmail.com>

On 01/29/2015 05:06 PM, Scot Doyle wrote:
> On Thu, 29 Jan 2015, Michael Kerrisk (man-pages) wrote:
>> On 29 January 2015 at 02:27, Scot Doyle <lkml14 at scotdoyle.com> wrote:
>>> On Wed, 28 Jan 2015, Vivek Goyal wrote:
>>>> On Wed, Jan 28, 2015 at 10:10:59PM +0000, Scot Doyle wrote:
>>>>> On Wed, 28 Jan 2015, Vivek Goyal wrote:
>>>>>> On Wed, Jan 28, 2015 at 09:14:03PM +0000, Scot Doyle wrote:
>>>>>>> When I tested, kexec_file_load required CONFIG_RELOCATABLE. Is the same
>>>>>>> true for kexec_load? Would it make sense to note this in the man pages
>>>>>>> along with the need for CONFIG_KEXEC_FILE, etc? Or as an error message?
>>>>>>
>>>>>> Hmm.., I can't see an explicity dependency between RELOCATABLE and
>>>>>> KEXEC. Both KEXEC and KEXEC_FILE should be able to load a kernel
>>>>>> even if it had RELOCATABLE=n.
>>>>>>
>>>>>> Just that kernel will run from the address it has been built for.
>>>>>>
>>>>>> Thanks
>>>>>> Vivek
>>>>>
>>>>> Confusing, right? kexec_file_load returns -ENOEXEC and dmesg says
>>>>> "kexec-bzImage64: XLF_CAN_BE_LOADED_ABOVE_4G is not set." which leads to
>>>>> arch/x86/boot/header.S line 396:
>>>>>
>>>>> #if defined(CONFIG_RELOCATABLE) && defined(CONFIG_X86_64)
>>>>>    /* kernel/boot_param/ramdisk could be loaded above 4g */
>>>>> # define XLF1 XLF_CAN_BE_LOADED_ABOVE_4G
>>>>> #else
>>>>> # define XLF1 0
>>>>> #endif
>>>>
>>>> Ah, this one. Actually generic kexec file loading implementation does not
>>>> impose this restriction. It is the image specific loader part which
>>>> decides what kind of bzImage it can load.
>>>>
>>>> Current implementation (kexec-bzimage64.c), is only supporting loading
>>>> bzImages which are 64bit and can be loaded above 4G. This simplifies
>>>> the implementation of loader.
>>>>
>>>> But there is nothing which prevents one from implementing other image
>>>> loaders.
>>>>
>>>> So instead of saying that kexec_file_load() depends on CONFIG_RELOCATABLE,
>>>> it might be better to say in man page that currently this system call
>>>> supports only loading a bzImage which is 64bit and which can be loaded
>>>> above 4G too.
>>>>
>>>> Thanks
>>>> Vivek
>>>
>>> Thanks, I agree, and think it would make sense to list them as part of the
>>> page's ENOEXEC error.
>>
>> Scott, could you then phras a couple of sentences that capture thge
>> details, so I can add it to the ENOEXEC error?
>>
>> Thanks,
>>
>> Michael
> 
> Yes, maybe something like "kernel_fd does not refer to an open file, or 
> the file type is not supported. Currently, the file must be a bzImage
> and contain an x86 kernel loadable above 4G in memory (see 
> Documentation/x86/boot.txt)."?
> 
> boot.txt explains that loading above 4G implies 64-bit and is specified 
> via a bit in xloadflags added in Linux 3.8.

Added and pushed. Thanks, Scott.

Cheers,

Michael


-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/


From geoff at infradead.org  Fri Jan 30 11:48:24 2015
From: geoff at infradead.org (Geoff Levand)
Date: Fri, 30 Jan 2015 11:48:24 -0800
Subject: [PATCH 7/8] arm64/kexec: Add checks for KVM
In-Reply-To: <54CB206D.7040705@linaro.org>
References: <cover.1421449714.git.geoff@infradead.org>
 <a3353cf2ac47f49c20bbbb88405301cbaa359805.1421449714.git.geoff@infradead.org>
 <20150126191942.GQ23313@leverpostej> <54CA041C.6020403@linaro.org>
 <54CA12A1.8090508@arm.com> <20150129184747.GV17721@leverpostej>
 <54CB206D.7040705@linaro.org>
Message-ID: <1422647304.21823.31.camel@infradead.org>

Hi Takahiro.

On Fri, 2015-01-30 at 15:10 +0900, AKASHI Takahiro wrote:
> Initially, I thought that we would define kvm_arch_exit() and call it
> somewhere in the middle of kexec path (no idea yet).
> But Geoff suggested me to implement a new hvc call, HVC_CPU_SHUTDOWN(??),
> and make it called via cpu_notifier(CPU_DYING_FROZEN) initiated by
> machine_shutdown() from kernel_kexec().

As an initial implementation we can hook into the CPU_DYING_FROZEN
notifier sent to hyp_init_cpu_notify().  The longer term solution
should use kvm_arch_hardware_enable() and kvm_arch_hardware_disable().

The calls to cpu_notifier(CPU_DYING_FROZEN) are part of cpu hot
plug, and independent of kexec.  If someone were to add spin-table
cpu un-plug, then it would be used for that also.  It seems we should
be able to test without kexec by using cpu hot plug.

To tear down KVM you need to get back to hyp mode, and hence
the need for HVC_CPU_SHUTDOWN.  The sequence I envisioned would
be like this:

cpu_notifier(CPU_DYING_FROZEN)
 -> kvm_cpu_shutdown() 
    prepare for hvc
    -> HVC_CPU_SHUTDOWN
       now in hyp mode, do KVM tear down, restore default exception vectors

Once the default exception vectors are restored soft_restart()
can then execute the cpu_reset routine in EL2.

Some notes are here for those with access:  https://cards.linaro.org/browse/KWG-611

-Geoff



From geoff at infradead.org  Fri Jan 30 13:47:11 2015
From: geoff at infradead.org (Geoff Levand)
Date: Fri, 30 Jan 2015 13:47:11 -0800
Subject: [PATCH 4/8] arm64: Add EL2 switch to soft_restart
In-Reply-To: <20150127175740.GK3226@e104818-lin.cambridge.arm.com>
References: <cover.1421449714.git.geoff@infradead.org>
 <bb44d4eee4fab15278e4a8829a0981878bc9acd7.1421449714.git.geoff@infradead.org>
 <20150127175740.GK3226@e104818-lin.cambridge.arm.com>
Message-ID: <1422654431.21823.33.camel@infradead.org>

Hi Catalin,

On Tue, 2015-01-27 at 17:57 +0000, Catalin Marinas wrote:
> On Sat, Jan 17, 2015 at 12:23:34AM +0000, Geoff Levand wrote:
> >  ENTRY(cpu_reset)
> > -	mrs	x1, sctlr_el1
> > -	bic	x1, x1, #1
> > -	msr	sctlr_el1, x1			// disable the MMU
> > +	mrs	x2, sctlr_el1
> > +	bic	x2, x2, #1
> > +	msr	sctlr_el1, x2			// disable the MMU
> >  	isb
> > -	ret	x0
> > +
> > +	cbz	x0, 1f				// el2_switch?
> > +	mov	x0, x1
> > +	mov	x1, xzr
> > +	mov	x2, xzr
> > +	mov	x3, xzr
> > +	hvc	#HVC_CALL_FUNC			// no return
> 
> If that's the only user of HVC_CALL_FUNC, why do we bother with
> arguments, calling convention?

It was intended that HVC_CALL_FUNC be a mechanism to call a generic
function in hyp mode.  cpu_reset() is the only user of it now.

As Mark explained in another post, the use of x18 by the hyp stub
is to avoid the complication of setting up a stack for EL2.  We
thought this solution was acceptable since there are relatively
few HVC_CALL_FUNC calls and we could assure they would do the
right thing.

-Geoff 






From geoff at infradead.org  Fri Jan 30 13:52:31 2015
From: geoff at infradead.org (Geoff Levand)
Date: Fri, 30 Jan 2015 13:52:31 -0800
Subject: [PATCH 3/8] arm64: Add new hcall HVC_CALL_FUNC
In-Reply-To: <20150127173947.GJ3226@e104818-lin.cambridge.arm.com>
References: <cover.1421449714.git.geoff@infradead.org>
 <3f1795bc1dad1d0510a9d24a168ecd5d83666507.1421449714.git.geoff@infradead.org>
 <20150127173947.GJ3226@e104818-lin.cambridge.arm.com>
Message-ID: <1422654751.21823.35.camel@infradead.org>

Hi Catalin,


On Tue, 2015-01-27 at 17:39 +0000, Catalin Marinas wrote:
> On Sat, Jan 17, 2015 at 12:23:34AM +0000, Geoff Levand wrote:
> > diff --git a/arch/arm64/include/asm/virt.h b/arch/arm64/include/asm/virt.h
> > index 99c319c..4f23a48 100644
> > --- a/arch/arm64/include/asm/virt.h
> > +++ b/arch/arm64/include/asm/virt.h
> > @@ -41,6 +41,19 @@
> >  
> >  #define HVC_CALL_HYP 3
> >  
> > +/*
> > + * HVC_CALL_FUNC - Execute a function at EL2.
> > + *
> > + * @x0: Physical address of the function to be executed.
> > + * @x1: Passed as the first argument to the function.
> > + * @x2: Passed as the second argument to the function.
> > + * @x3: Passed as the third argument to the function.
> > + *
> > + * The called function must preserve the contents of register x18.
> 
> Can you pick a register that's normally callee saved?

Mark covered this in his reply.

> > + */
> > +
> > +#define HVC_CALL_FUNC 4
> > +
> >  #ifndef __ASSEMBLY__
> >  
> >  /*
> > diff --git a/arch/arm64/kernel/hyp-stub.S b/arch/arm64/kernel/hyp-stub.S
> > index e3db3fd..b5d36e7 100644
> > --- a/arch/arm64/kernel/hyp-stub.S
> > +++ b/arch/arm64/kernel/hyp-stub.S
> > @@ -66,9 +66,20 @@ el1_sync:
> >  	mrs	x0, vbar_el2
> >  	b	2f
> >  
> > -1:	cmp	x18, #HVC_SET_VECTORS
> > -	b.ne	2f
> > -	msr	vbar_el2, x0
> > +1:	cmp     x18, #HVC_SET_VECTORS
> 
> This line doesn't seem to have any change, apart from some whitespace.
> Or did you want to drop the label?

Some whitespace problems got in there from so many rebases.  I went
through the series and cleaned them up.


> > +	b.ne    1f
> > +	msr     vbar_el2, x0
> > +	b       2f
> > +
> > +1:	cmp     x18, #HVC_CALL_FUNC
> > +	b.ne    2f
> > +	mov     x18, lr
> > +	mov     lr, x0
> > +	mov     x0, x1
> > +	mov     x1, x2
> > +	mov     x2, x3
> > +	blr     lr
> > +	mov     lr, x18
> >  
> >  2:	eret
> >  ENDPROC(el1_sync)
> 
> What is the calling convention for this HVC? You mentioned x18 above but
> what about other registers that the called function may corrupt (x18 is
> a temporary register, so it's not expected to be callee saved).

Again, Mark covered this in his reply.




From geoff at infradead.org  Fri Jan 30 15:31:21 2015
From: geoff at infradead.org (Geoff Levand)
Date: Fri, 30 Jan 2015 15:31:21 -0800
Subject: [PATCH 2/8] arm64: Convert hcalls to use ISS field
In-Reply-To: <20150126182621.GH14048@e104818-lin.cambridge.arm.com>
References: <cover.1421449714.git.geoff@infradead.org>
 <6719b2520095926ae0e8ac888663b87f2098f565.1421449714.git.geoff@infradead.org>
 <20150126182621.GH14048@e104818-lin.cambridge.arm.com>
Message-ID: <1422660681.21823.38.camel@infradead.org>

On Mon, 2015-01-26 at 18:26 +0000, Catalin Marinas wrote:
> On Sat, Jan 17, 2015 at 12:23:34AM +0000, Geoff Levand wrote:
> > To allow for additional hcalls to be defined and to make the arm64 hcall API
> > more consistent across exception vector routines, change the hcall implementations
> > to use the ISS field of the ESR_EL2 register to specify the hcall type.
> > 
> > The existing arm64 hcall implementations are limited in that they only allow
> > for two distinct hcalls; with the x0 register either zero, or not zero.  Also,
> > the API of the hyp-stub exception vector routines and the KVM exception vector
> > routines differ; hyp-stub uses a non-zero value in x0 to implement
> > __hyp_set_vectors, whereas KVM uses it to implement kvm_call_hyp.
> > 
> > Define three new preprocessor macros HVC_GET_VECTORS, HVC_SET_VECTORS and
> > HVC_CALL_HYP and to be used as hcall type specifiers and convert the
> > existing __hyp_get_vectors(), __hyp_set_vectors() and kvm_call_hyp() routines
> > to use these new macros when executing an HVC call.  Also change the
> > corresponding hyp-stub and KVM el1_sync exception vector routines to use these
> > new macros.
> > 
> > Signed-off-by: Geoff Levand <geoff at infradead.org>
> 
> Using the #imm value for HVC to separate what gets called looks fine to
> me. However, I'd like to see a review from Marc/Christoffer on this
> patch.

Marc, Christopher, comments please?

> Some comments below:
> 
> > diff --git a/arch/arm64/include/asm/virt.h b/arch/arm64/include/asm/virt.h
> > index 7a5df52..99c319c 100644
> > --- a/arch/arm64/include/asm/virt.h
> > +++ b/arch/arm64/include/asm/virt.h
> > @@ -21,6 +21,26 @@
> >  #define BOOT_CPU_MODE_EL1	(0xe11)
> >  #define BOOT_CPU_MODE_EL2	(0xe12)
> >  
> > +/*
> > + * HVC_GET_VECTORS - Return the value of the vbar_el2 register.
> > + */
> > +
> > +#define HVC_GET_VECTORS 1
> > +
> > +/*
> > + * HVC_SET_VECTORS - Set the value of the vbar_el2 register.
> > + *
> > + * @x0: Physical address of the new vector table.
> > + */
> > +
> > +#define HVC_SET_VECTORS 2
> > +
> > +/*
> > + * HVC_CALL_HYP - Execute a hyp routine.
> > + */
> > +
> > +#define HVC_CALL_HYP 3
> 
> I think you can ignore this case (make it the default), just define it
> as 0 as that's the normal use-case after initialisation and avoid
> checking it explicitly.

OK, I changed this so that HVC_CALL_HYP is the default at 0.

> >  /*
> > diff --git a/arch/arm64/kernel/hyp-stub.S b/arch/arm64/kernel/hyp-stub.S
> > index a272f33..e3db3fd 100644
> > --- a/arch/arm64/kernel/hyp-stub.S
> > +++ b/arch/arm64/kernel/hyp-stub.S
> > @@ -22,6 +22,7 @@
> >  #include <linux/irqchip/arm-gic-v3.h>
> >  
> >  #include <asm/assembler.h>
> > +#include <asm/kvm_arm.h>
> >  #include <asm/ptrace.h>
> >  #include <asm/virt.h>
> >  
> > @@ -53,14 +54,22 @@ ENDPROC(__hyp_stub_vectors)
> >  	.align 11
> >  
> >  el1_sync:
> > -	mrs	x1, esr_el2
> > -	lsr	x1, x1, #26
> > -	cmp	x1, #0x16
> > -	b.ne	2f				// Not an HVC trap
> > -	cbz	x0, 1f
> > -	msr	vbar_el2, x0			// Set vbar_el2
> > +	mrs	x18, esr_el2
> > +	lsr	x17, x18, #ESR_ELx_EC_SHIFT
> > +	and	x18, x18, #ESR_ELx_ISS_MASK
> > +
> > +	cmp     x17, #ESR_ELx_EC_HVC64
> > +	b.ne    2f				// Not an HVC trap
> > +
> > +	cmp	x18, #HVC_GET_VECTORS
> > +	b.ne	1f
> > +	mrs	x0, vbar_el2
> >  	b	2f
> > -1:	mrs	x0, vbar_el2			// Return vbar_el2
> > +
> > +1:	cmp	x18, #HVC_SET_VECTORS
> > +	b.ne	2f
> > +	msr	vbar_el2, x0
> > +
> >  2:	eret
> >  ENDPROC(el1_sync)
> 
> You seem to be using x17 and x18 here freely. Do you have any guarantees
> that the caller saved/restored those registers? I guess you assume they
> are temporary registers and the caller first branches to a function
> (like __kvm_hyp_call) and expects them to be corrupted. But I'm not sure
> that's always the case. Take for example the __invoke_psci_fn_hvc where
> the function is in C (we should change this for other reasons).

Yes, I assume the compiler will not expect them to be preserved.  I
missed __invoke_psci_fn_hvc.  Can we just add x17 and x18 to the
clobbered list?

        asm volatile(
                        __asmeq("%0", "x0")
                        __asmeq("%1", "x1")
                        __asmeq("%2", "x2")
                        __asmeq("%3", "x3")
                        "hvc    #0\n"
                : "+r" (function_id)
-               : "r" (arg0), "r" (arg1), "r" (arg2));
+               : "r" (arg0), "r" (arg1), "r" (arg2)
+               : "x17", "x18");


> > diff --git a/arch/arm64/kvm/hyp.S b/arch/arm64/kvm/hyp.S
> > index c0d8202..1916c89 100644
> > --- a/arch/arm64/kvm/hyp.S
> > +++ b/arch/arm64/kvm/hyp.S
> > @@ -27,6 +27,7 @@
> >  #include <asm/kvm_asm.h>
> >  #include <asm/kvm_mmu.h>
> >  #include <asm/memory.h>
> > +#include <asm/virt.h>
> >  
> >  #define CPU_GP_REG_OFFSET(x)	(CPU_GP_REGS + x)
> >  #define CPU_XREG_OFFSET(x)	CPU_GP_REG_OFFSET(CPU_USER_PT_REGS + 8*x)
> > @@ -1106,12 +1107,9 @@ __hyp_panic_str:
> >   * in Hyp mode (see init_hyp_mode in arch/arm/kvm/arm.c).  Return values are
> >   * passed in r0 and r1.
> >   *
> > - * A function pointer with a value of 0 has a special meaning, and is
> > - * used to implement __hyp_get_vectors in the same way as in
> > - * arch/arm64/kernel/hyp_stub.S.
> >   */
> >  ENTRY(kvm_call_hyp)
> > -	hvc	#0
> > +	hvc	#HVC_CALL_HYP
> >  	ret
> >  ENDPROC(kvm_call_hyp)
> >  
> > @@ -1142,6 +1140,7 @@ el1_sync:					// Guest trapped into EL2
> >  
> >  	mrs	x1, esr_el2
> >  	lsr	x2, x1, #ESR_ELx_EC_SHIFT
> > +	and	x0, x1, #ESR_ELx_ISS_MASK
> >  
> >  	cmp	x2, #ESR_ELx_EC_HVC64
> >  	b.ne	el1_trap
> > @@ -1150,15 +1149,19 @@ el1_sync:					// Guest trapped into EL2
> >  	cbnz	x3, el1_trap			// called HVC
> >  
> >  	/* Here, we're pretty sure the host called HVC. */
> > +	mov	x18, x0
> 
> Same comment here about corrupting x18. If it is safe, maybe add some
> comments in the calling place.

I added a comment regarding this to virt.h where the HVC_XXX macros
are defined.  I'll post that fixed up patch for review.

> 
> >  	pop	x2, x3
> >  	pop	x0, x1
> >  
> > -	/* Check for __hyp_get_vectors */
> > -	cbnz	x0, 1f
> > +	cmp	x18, #HVC_GET_VECTORS
> > +	b.ne	1f
> >  	mrs	x0, vbar_el2
> >  	b	2f
> >  
> > -1:	push	lr, xzr
> > +1:	cmp	x18, #HVC_CALL_HYP
> > +	b.ne	2f
> > +
> > +	push	lr, xzr
> 
> At this point, we expect either HVC_GET_VECTORS or HVC_CALL_HYP. I think
> you can simply assume HVC_CALL_HYP as default and ignore the additional
> cmp.

OK, did that.

-Geoff




From geoff at infradead.org  Fri Jan 30 15:33:48 2015
From: geoff at infradead.org (Geoff Levand)
Date: Fri, 30 Jan 2015 15:33:48 -0800
Subject: [PATCH v2 2/8] arm64: Convert hcalls to use ISS field
In-Reply-To: <6719b2520095926ae0e8ac888663b87f2098f565.1421449714.git.geoff@infradead.org>
References: <cover.1421449714.git.geoff@infradead.org>
 <6719b2520095926ae0e8ac888663b87f2098f565.1421449714.git.geoff@infradead.org>
Message-ID: <1422660828.21823.40.camel@infradead.org>

To allow for additional hcalls to be defined and to make the arm64 hcall API
more consistent across exception vector routines, change the hcall implementations
to use the ISS field of the ESR_EL2 register to specify the hcall type.

The existing arm64 hcall implementations are limited in that they only allow
for two distinct hcalls; with the x0 register either zero, or not zero.  Also,
the API of the hyp-stub exception vector routines and the KVM exception vector
routines differ; hyp-stub uses a non-zero value in x0 to implement
__hyp_set_vectors, whereas KVM uses it to implement kvm_call_hyp.

Define three new preprocessor macros HVC_CALL_HYP, HVC_GET_VECTORS, and
HVC_SET_VECTORS to be used as hcall type specifiers and convert the
existing __hyp_get_vectors(), __hyp_set_vectors() and kvm_call_hyp() routines
to use these new macros when executing an HVC call.  Also change the
corresponding hyp-stub and KVM el1_sync exception vector routines to use these
new macros.

Signed-off-by: Geoff Levand <geoff at infradead.org>
---
 arch/arm64/include/asm/virt.h | 27 +++++++++++++++++++++++++++
 arch/arm64/kernel/hyp-stub.S  | 32 +++++++++++++++++++++-----------
 arch/arm64/kernel/psci.c      |  3 ++-
 arch/arm64/kvm/hyp.S          | 16 +++++++++-------
 4 files changed, 59 insertions(+), 19 deletions(-)

diff --git a/arch/arm64/include/asm/virt.h b/arch/arm64/include/asm/virt.h
index 7a5df52..eb10368 100644
--- a/arch/arm64/include/asm/virt.h
+++ b/arch/arm64/include/asm/virt.h
@@ -18,6 +18,33 @@
 #ifndef __ASM__VIRT_H
 #define __ASM__VIRT_H
 
+/*
+ * The arm64 hcall implementation uses the ISS field of the ESR_EL2 register to
+ * specify the hcall type.  The exception handlers are allowed to use registers
+ * x17 and x18 in their implementation.  Any routine issuing an hcall must not
+ * expect these registers to be preserved.
+ */
+
+/*
+ * HVC_CALL_HYP - Execute a hyp routine.
+ */
+
+#define HVC_CALL_HYP 0
+
+/*
+ * HVC_GET_VECTORS - Return the value of the vbar_el2 register.
+ */
+
+#define HVC_GET_VECTORS 1
+
+/*
+ * HVC_SET_VECTORS - Set the value of the vbar_el2 register.
+ *
+ * @x0: Physical address of the new vector table.
+ */
+
+#define HVC_SET_VECTORS 2
+
 #define BOOT_CPU_MODE_EL1	(0xe11)
 #define BOOT_CPU_MODE_EL2	(0xe12)
 
diff --git a/arch/arm64/kernel/hyp-stub.S b/arch/arm64/kernel/hyp-stub.S
index a272f33..017ab519 100644
--- a/arch/arm64/kernel/hyp-stub.S
+++ b/arch/arm64/kernel/hyp-stub.S
@@ -22,6 +22,7 @@
 #include <linux/irqchip/arm-gic-v3.h>
 
 #include <asm/assembler.h>
+#include <asm/kvm_arm.h>
 #include <asm/ptrace.h>
 #include <asm/virt.h>
 
@@ -53,14 +54,22 @@ ENDPROC(__hyp_stub_vectors)
 	.align 11
 
 el1_sync:
-	mrs	x1, esr_el2
-	lsr	x1, x1, #26
-	cmp	x1, #0x16
+	mrs	x18, esr_el2
+	lsr	x17, x18, #ESR_ELx_EC_SHIFT
+	and	x18, x18, #ESR_ELx_ISS_MASK
+
+	cmp	x17, #ESR_ELx_EC_HVC64
 	b.ne	2f				// Not an HVC trap
-	cbz	x0, 1f
-	msr	vbar_el2, x0			// Set vbar_el2
+
+	cmp	x18, #HVC_GET_VECTORS
+	b.ne	1f
+	mrs	x0, vbar_el2
 	b	2f
-1:	mrs	x0, vbar_el2			// Return vbar_el2
+
+1:	cmp	x18, #HVC_SET_VECTORS
+	b.ne	2f
+	msr	vbar_el2, x0
+
 2:	eret
 ENDPROC(el1_sync)
 
@@ -100,11 +109,12 @@ ENDPROC(\label)
  * initialisation entry point.
  */
 
-ENTRY(__hyp_get_vectors)
-	mov	x0, xzr
-	// fall through
 ENTRY(__hyp_set_vectors)
-	hvc	#0
+	hvc	#HVC_SET_VECTORS
 	ret
-ENDPROC(__hyp_get_vectors)
 ENDPROC(__hyp_set_vectors)
+
+ENTRY(__hyp_get_vectors)
+	hvc	#HVC_GET_VECTORS
+	ret
+ENDPROC(__hyp_get_vectors)
diff --git a/arch/arm64/kernel/psci.c b/arch/arm64/kernel/psci.c
index 3425f31..7043fd7 100644
--- a/arch/arm64/kernel/psci.c
+++ b/arch/arm64/kernel/psci.c
@@ -123,7 +123,8 @@ static noinline int __invoke_psci_fn_hvc(u64 function_id, u64 arg0, u64 arg1,
 			__asmeq("%3", "x3")
 			"hvc	#0\n"
 		: "+r" (function_id)
-		: "r" (arg0), "r" (arg1), "r" (arg2));
+		: "r" (arg0), "r" (arg1), "r" (arg2)
+		: "x17", "x18");
 
 	return function_id;
 }
diff --git a/arch/arm64/kvm/hyp.S b/arch/arm64/kvm/hyp.S
index c0d8202..42c9851 100644
--- a/arch/arm64/kvm/hyp.S
+++ b/arch/arm64/kvm/hyp.S
@@ -27,6 +27,7 @@
 #include <asm/kvm_asm.h>
 #include <asm/kvm_mmu.h>
 #include <asm/memory.h>
+#include <asm/virt.h>
 
 #define CPU_GP_REG_OFFSET(x)	(CPU_GP_REGS + x)
 #define CPU_XREG_OFFSET(x)	CPU_GP_REG_OFFSET(CPU_USER_PT_REGS + 8*x)
@@ -1106,12 +1107,9 @@ __hyp_panic_str:
  * in Hyp mode (see init_hyp_mode in arch/arm/kvm/arm.c).  Return values are
  * passed in r0 and r1.
  *
- * A function pointer with a value of 0 has a special meaning, and is
- * used to implement __hyp_get_vectors in the same way as in
- * arch/arm64/kernel/hyp_stub.S.
  */
 ENTRY(kvm_call_hyp)
-	hvc	#0
+	hvc	#HVC_CALL_HYP
 	ret
 ENDPROC(kvm_call_hyp)
 
@@ -1142,6 +1140,7 @@ el1_sync:					// Guest trapped into EL2
 
 	mrs	x1, esr_el2
 	lsr	x2, x1, #ESR_ELx_EC_SHIFT
+	and	x0, x1, #ESR_ELx_ISS_MASK
 
 	cmp	x2, #ESR_ELx_EC_HVC64
 	b.ne	el1_trap
@@ -1150,15 +1149,18 @@ el1_sync:					// Guest trapped into EL2
 	cbnz	x3, el1_trap			// called HVC
 
 	/* Here, we're pretty sure the host called HVC. */
+	mov	x18, x0
 	pop	x2, x3
 	pop	x0, x1
 
-	/* Check for __hyp_get_vectors */
-	cbnz	x0, 1f
+	cmp	x18, #HVC_GET_VECTORS
+	b.ne	1f
 	mrs	x0, vbar_el2
 	b	2f
 
-1:	push	lr, xzr
+1:	/* Default to HVC_CALL_HYP. */
+
+	push	lr, xzr
 
 	/*
 	 * Compute the function address in EL2, and shuffle the parameters.
-- 
2.1.0





From yszhou4tech at gmail.com  Sat Jan 31 08:10:06 2015
From: yszhou4tech at gmail.com (Yousong Zhou)
Date: Sun,  1 Feb 2015 00:10:06 +0800
Subject: [PATCH 1/3] configure.ac: quote result of AC_CHECK_LIB()
Message-ID: <1422720608-28700-1-git-send-email-yszhou4tech@gmail.com>

Fix the following error when searching for lzma support.

	checking for lzma_code in -llzma... ./configure: line 4756: ac_fn_c_try_link: command not found

Signed-off-by: Yousong Zhou <yszhou4tech at gmail.com>
---
 configure.ac |   12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/configure.ac b/configure.ac
index 4e15cb0..212e838 100644
--- a/configure.ac
+++ b/configure.ac
@@ -152,22 +152,22 @@ AC_CHECK_PROG([DIRNAME],  dirname,  dirname,  "no", [$PATH])
 dnl See if I have a usable copy of zlib available
 if test "$with_zlib" = yes ; then
 	AC_CHECK_HEADER(zlib.h,
-		AC_CHECK_LIB(z, inflateInit_, ,
-		AC_MSG_NOTICE([zlib support disabled])))
+		[AC_CHECK_LIB(z, inflateInit_, ,
+		AC_MSG_NOTICE([zlib support disabled]))])
 fi
 
 dnl See if I have a usable copy of lzma available
 if test "$with_lzma" = yes ; then
 	AC_CHECK_HEADER(lzma.h,
-		AC_CHECK_LIB(lzma, lzma_code, ,
-		AC_MSG_NOTICE([lzma support disabled])))
+		[AC_CHECK_LIB(lzma, lzma_code, ,
+		AC_MSG_NOTICE([lzma support disabled]))])
 fi
 
 dnl find Xen control stack libraries
 if test "$with_xen" = yes ; then
 	AC_CHECK_HEADER(xenctrl.h,
-		AC_CHECK_LIB(xenctrl, xc_kexec_load, ,
-		AC_MSG_NOTICE([Xen support disabled])))
+		[AC_CHECK_LIB(xenctrl, xc_kexec_load, ,
+		AC_MSG_NOTICE([Xen support disabled]))])
 fi
 
 dnl ---Sanity checks
-- 
1.7.10.4



From yszhou4tech at gmail.com  Sat Jan 31 08:10:07 2015
From: yszhou4tech at gmail.com (Yousong Zhou)
Date: Sun,  1 Feb 2015 00:10:07 +0800
Subject: [PATCH 2/3] Fix zlib/lzma decompression.
In-Reply-To: <1422720608-28700-1-git-send-email-yszhou4tech@gmail.com>
References: <1422720608-28700-1-git-send-email-yszhou4tech@gmail.com>
Message-ID: <1422720608-28700-2-git-send-email-yszhou4tech@gmail.com>

Let {zlib,lzma}_decompress_file() return NULL if anything wrong happened
to allow the other method to have a chance to run.

Signed-off-by: Yousong Zhou <yszhou4tech at gmail.com>
---
 kexec/lzma.c |   33 ++++++++++++++++++++++-----------
 kexec/zlib.c |   57 +++++++++++++++++++++++++++++++++++----------------------
 2 files changed, 57 insertions(+), 33 deletions(-)

diff --git a/kexec/lzma.c b/kexec/lzma.c
index 939aeb3..5bfccb7 100644
--- a/kexec/lzma.c
+++ b/kexec/lzma.c
@@ -162,13 +162,16 @@ char *lzma_decompress_file(const char *filename, off_t *r_size)
 	off_t size, allocated;
 	ssize_t result;
 
-	if (!filename) {
-		*r_size = 0;
-		return 0;
-	}
+	dbgprintf("Try LZMA decompression.\n");
+
+	*r_size = 0;
+	if (!filename)
+		return NULL;
+
 	fp = lzopen(filename, "rb");
 	if (fp == 0) {
-		die("Cannot open `%s'\n", filename);
+		dbgprintf("Cannot open `%s'\n", filename);
+		return NULL;
 	}
 	size = 0;
 	allocated = 65536;
@@ -183,17 +186,25 @@ char *lzma_decompress_file(const char *filename, off_t *r_size)
 			if ((errno == EINTR) || (errno == EAGAIN))
 				continue;
 
-			die ("read on %s of %ld bytes failed\n",
-				filename, (allocated - size) + 0UL);
+			dbgprintf("%s: read on %s of %ld bytes failed\n",
+				__func__, filename, (allocated - size) + 0UL);
+			break;
 		}
 		size += result;
-	} while(result > 0);
-	result = lzclose(fp);
-	if (result != LZMA_OK) {
-		die ("Close of %s failed\n", filename);
+	} while (result > 0);
+
+	if (lzclose(fp) != LZMA_OK) {
+		dbgprintf("%s: Close of %s failed\n", __func__, filename);
+		goto fail;
 	}
+	if (result < 0)
+		goto fail;
+
 	*r_size =  size;
 	return buf;
+fail:
+	free(buf);
+	return NULL;
 }
 #else
 char *lzma_decompress_file(const char *UNUSED(filename), off_t *UNUSED(r_size))
diff --git a/kexec/zlib.c b/kexec/zlib.c
index d44df12..7170ac3 100644
--- a/kexec/zlib.c
+++ b/kexec/zlib.c
@@ -15,29 +15,39 @@
 #include <ctype.h>
 #include <zlib.h>
 
+static void _gzerror(gzFile fp, int *errnum, const char **errmsg)
+{
+	*errmsg = gzerror(fp, errnum);
+	if (*errnum == Z_ERRNO) {
+		*errmsg = strerror(*errnum);
+	}
+}
+
 char *zlib_decompress_file(const char *filename, off_t *r_size)
 {
 	gzFile fp;
 	int errnum;
 	const char *msg;
 	char *buf;
-	off_t size, allocated;
+	off_t size = 0, allocated;
 	ssize_t result;
 
+	dbgprintf("Try gzip decompression.\n");
+
+	*r_size = 0;
 	if (!filename) {
-		*r_size = 0;
-		return 0;
+		return NULL;
 	}
 	fp = gzopen(filename, "rb");
 	if (fp == 0) {
-		msg = gzerror(fp, &errnum);
-		if (errnum == Z_ERRNO) {
-			msg = strerror(errno);
-		}
-		fprintf(stderr, "Cannot open `%s': %s\n", filename, msg);
+		_gzerror(fp, &errnum, &msg);
+		dbgprintf("Cannot open `%s': %s\n", filename, msg);
+		return NULL;
+	}
+	if (gzdirect(fp)) {
+		/* It's not in gzip format */
 		return NULL;
 	}
-	size = 0;
 	allocated = 65536;
 	buf = xmalloc(allocated);
 	do {
@@ -49,25 +59,28 @@ char *zlib_decompress_file(const char *filename, off_t *r_size)
 		if (result < 0) {
 			if ((errno == EINTR) || (errno == EAGAIN))
 				continue;
-
-			msg = gzerror(fp, &errnum);
-			if (errnum == Z_ERRNO) {
-				msg = strerror(errno);
-			}
-			die ("read on %s of %ld bytes failed: %s\n",
-				filename, (allocated - size) + 0UL, msg);
+			_gzerror(fp, &errnum, &msg);
+			dbgprintf("Read on %s of %ld bytes failed: %s\n",
+					filename, (allocated - size) + 0UL, msg);
+			size = 0;
+			goto fail;
 		}
 		size += result;
 	} while(result > 0);
+
+fail:
 	result = gzclose(fp);
 	if (result != Z_OK) {
-		msg = gzerror(fp, &errnum);
-		if (errnum == Z_ERRNO) {
-			msg = strerror(errno);
-		}
-		die ("Close of %s failed: %s\n", filename, msg);
+		_gzerror(fp, &errnum, &msg);
+		dbgprintf(" Close of %s failed: %s\n", filename, msg);
+	}
+
+	if (size > 0) {
+		*r_size = size;
+	} else {
+		free(buf);
+		buf = NULL;
 	}
-	*r_size =  size;
 	return buf;
 }
 #else
-- 
1.7.10.4



From yszhou4tech at gmail.com  Sat Jan 31 08:10:08 2015
From: yszhou4tech at gmail.com (Yousong Zhou)
Date: Sun,  1 Feb 2015 00:10:08 +0800
Subject: [PATCH 3/3] Fix a few compilation warnings.
In-Reply-To: <1422720608-28700-1-git-send-email-yszhou4tech@gmail.com>
References: <1422720608-28700-1-git-send-email-yszhou4tech@gmail.com>
Message-ID: <1422720608-28700-3-git-send-email-yszhou4tech@gmail.com>


Signed-off-by: Yousong Zhou <yszhou4tech at gmail.com>
---
 kexec/arch/mips/crashdump-mips.c |    3 ++-
 kexec/arch/mips/kexec-elf-mips.c |    2 +-
 kexec/kexec-elf-rel.c            |    3 +++
 kexec/kexec.h                    |    2 +-
 4 files changed, 7 insertions(+), 3 deletions(-)

diff --git a/kexec/arch/mips/crashdump-mips.c b/kexec/arch/mips/crashdump-mips.c
index e7840e0..98c9f7c 100644
--- a/kexec/arch/mips/crashdump-mips.c
+++ b/kexec/arch/mips/crashdump-mips.c
@@ -22,6 +22,7 @@
 #include <stdlib.h>
 #include <errno.h>
 #include <limits.h>
+#include <inttypes.h>
 #include <elf.h>
 #include <sys/types.h>
 #include <sys/stat.h>
@@ -52,7 +53,7 @@ static int get_kernel_paddr(struct crash_elf_info *elf_info)
 
 	if (parse_iomem_single("Kernel code\n", &start, NULL) == 0) {
 		elf_info->kern_paddr_start = start;
-		dbgprintf("kernel load physical addr start = 0x%lx\n", start);
+		dbgprintf("kernel load physical addr start = 0x%" PRIu64 "\n", start);
 		return 0;
 	}
 
diff --git a/kexec/arch/mips/kexec-elf-mips.c b/kexec/arch/mips/kexec-elf-mips.c
index a27d986..8a6419a 100644
--- a/kexec/arch/mips/kexec-elf-mips.c
+++ b/kexec/arch/mips/kexec-elf-mips.c
@@ -158,7 +158,7 @@ int elf_mips_load(int argc, char **argv, const char *buf, off_t len,
 	if (info->kexec_flags & KEXEC_ON_CRASH)
 		/* In case of crashdump segment[0] is kernel.
 		 * Put cmdline just after it. */
-		cmdline_addr = info->segment[0].mem +
+		cmdline_addr = (unsigned long)info->segment[0].mem +
 				info->segment[0].memsz;
 	else
 		cmdline_addr = 0;
diff --git a/kexec/kexec-elf-rel.c b/kexec/kexec-elf-rel.c
index c625f30..5a5fcd1 100644
--- a/kexec/kexec-elf-rel.c
+++ b/kexec/kexec-elf-rel.c
@@ -347,6 +347,9 @@ int elf_rel_load(struct mem_ehdr *ehdr, struct kexec_info *info,
 			else if (shdr->sh_type == SHT_RELA) {
 				rel = elf_rela(ehdr, ptr);
 			}
+			else {
+				die("Unexpected sh_type: %d\n", shdr->sh_type);
+			}
 			/* the location to change */
 			location = section->sh_data + rel.r_offset;
 
diff --git a/kexec/kexec.h b/kexec/kexec.h
index 4be2b2f..2c85052 100644
--- a/kexec/kexec.h
+++ b/kexec/kexec.h
@@ -245,7 +245,7 @@ extern int file_types;
 
 extern void dbgprint_mem_range(const char *prefix, struct memory_range *mr, int nr_mr);
 extern void die(const char *fmt, ...)
-	__attribute__ ((format (printf, 1, 2)));
+	__attribute__ ((format (printf, 1, 2), noreturn));
 extern void *xmalloc(size_t size);
 extern void *xrealloc(void *ptr, size_t size);
 extern char *slurp_file(const char *filename, off_t *r_size);
-- 
1.7.10.4



