From eric.devolder at oracle.com  Thu Jun  1 07:18:52 2017
From: eric.devolder at oracle.com (Eric DeVolder)
Date: Thu, 1 Jun 2017 09:18:52 -0500
Subject: makedumpfile PATCH] Fix the use of Xen physical and machine
 addresses
In-Reply-To: <20170531090750.GA19333@olila.local.net-space.pl>
References: <20170523093452.GQ30364@olila.local.net-space.pl>
 <b1e80585-7a71-b1d1-7d09-f52da8a82b7e@oracle.com>
 <20170530232815.5233981e@hananiah.suse.cz>
 <20170531090750.GA19333@olila.local.net-space.pl>
Message-ID: <5f866d24-4aaa-ef80-4327-779780132b12@oracle.com>



On 05/31/2017 04:07 AM, Daniel Kiper wrote:
> On Tue, May 30, 2017 at 11:28:15PM +0200, Petr Tesarik wrote:
>> On Tue, 30 May 2017 14:30:43 -0500
>> Eric DeVolder <eric.devolder at oracle.com> wrote:
>>
>>> Hi,
>>> Testing is underway. Generally working but I do have a couple of fails
>>> I'm looking into:
>>>
>>>   > FAIL xen-4.1.2-rc3-pre_domU-pv_xl_linux-2.6.39-x86_64.tar.bz2
>>                              ^^^^^^^
>>
>> Is this a PV DomU xc_core style dump? If so, then it has never been
>> supported by makedumpfile, because the file format is very different,
>> with no program headers, but a special section called ".xen_pages". In
>> fact, the file format itself does not cover this type of dump.
> 
> Ahhh... IIRC, you are right. Thanks for pointing this out. Eric, please
> double check it and if PV __GUESTS__ are not supported by makedumpfile
> skip tests for them. Just test dom0 and Xen hypervisor stuff.

I concur, this dump format is not supported by makedumpfile.

I'm still working on validating 32-bit dump files.

eric



From eric.devolder at oracle.com  Thu Jun  1 09:00:41 2017
From: eric.devolder at oracle.com (Eric DeVolder)
Date: Thu, 1 Jun 2017 11:00:41 -0500
Subject: makedumpfile PATCH] Fix the use of Xen physical and machine
 addresses
In-Reply-To: <5f866d24-4aaa-ef80-4327-779780132b12@oracle.com>
References: <20170523093452.GQ30364@olila.local.net-space.pl>
 <b1e80585-7a71-b1d1-7d09-f52da8a82b7e@oracle.com>
 <20170530232815.5233981e@hananiah.suse.cz>
 <20170531090750.GA19333@olila.local.net-space.pl>
 <5f866d24-4aaa-ef80-4327-779780132b12@oracle.com>
Message-ID: <54051391-6363-ec1b-0cc9-3f156c32908c@oracle.com>



On 06/01/2017 09:18 AM, Eric DeVolder wrote:
> 
> 
> On 05/31/2017 04:07 AM, Daniel Kiper wrote:
>> On Tue, May 30, 2017 at 11:28:15PM +0200, Petr Tesarik wrote:
>>> On Tue, 30 May 2017 14:30:43 -0500
>>> Eric DeVolder <eric.devolder at oracle.com> wrote:
>>>
>>>> Hi,
>>>> Testing is underway. Generally working but I do have a couple of fails
>>>> I'm looking into:
>>>>
>>>>   > FAIL xen-4.1.2-rc3-pre_domU-pv_xl_linux-2.6.39-x86_64.tar.bz2
>>>                              ^^^^^^^
>>>
>>> Is this a PV DomU xc_core style dump? If so, then it has never been
>>> supported by makedumpfile, because the file format is very different,
>>> with no program headers, but a special section called ".xen_pages". In
>>> fact, the file format itself does not cover this type of dump.
>>
>> Ahhh... IIRC, you are right. Thanks for pointing this out. Eric, please
>> double check it and if PV __GUESTS__ are not supported by makedumpfile
>> skip tests for them. Just test dom0 and Xen hypervisor stuff.
> 
> I concur, this dump format is not supported by makedumpfile.
> 
> I'm still working on validating 32-bit dump files.

I have validated the 32-bit dump files with this patch. All is good.
eric



From ats-kumagai at wm.jp.nec.com  Thu Jun  1 20:04:30 2017
From: ats-kumagai at wm.jp.nec.com (Atsushi Kumagai)
Date: Fri, 2 Jun 2017 03:04:30 +0000
Subject: makedumpfile PATCH] Fix the use of Xen physical and machine
 addresses
In-Reply-To: <54051391-6363-ec1b-0cc9-3f156c32908c@oracle.com>
References: <20170523093452.GQ30364@olila.local.net-space.pl>
 <b1e80585-7a71-b1d1-7d09-f52da8a82b7e@oracle.com>
 <20170530232815.5233981e@hananiah.suse.cz>
 <20170531090750.GA19333@olila.local.net-space.pl>
 <5f866d24-4aaa-ef80-4327-779780132b12@oracle.com>
 <54051391-6363-ec1b-0cc9-3f156c32908c@oracle.com>
Message-ID: <0910DD04CBD6DE4193FCF86B9C00BE9701EDD48C@BPXM01GP.gisp.nec.co.jp>

Thanks everyone, I'll merge this patch into v1.6.2

Regards,
Atsushi Kumagai

>On 06/01/2017 09:18 AM, Eric DeVolder wrote:
>>
>>
>> On 05/31/2017 04:07 AM, Daniel Kiper wrote:
>>> On Tue, May 30, 2017 at 11:28:15PM +0200, Petr Tesarik wrote:
>>>> On Tue, 30 May 2017 14:30:43 -0500
>>>> Eric DeVolder <eric.devolder at oracle.com> wrote:
>>>>
>>>>> Hi,
>>>>> Testing is underway. Generally working but I do have a couple of fails
>>>>> I'm looking into:
>>>>>
>>>>>   > FAIL xen-4.1.2-rc3-pre_domU-pv_xl_linux-2.6.39-x86_64.tar.bz2
>>>>                              ^^^^^^^
>>>>
>>>> Is this a PV DomU xc_core style dump? If so, then it has never been
>>>> supported by makedumpfile, because the file format is very different,
>>>> with no program headers, but a special section called ".xen_pages". In
>>>> fact, the file format itself does not cover this type of dump.
>>>
>>> Ahhh... IIRC, you are right. Thanks for pointing this out. Eric, please
>>> double check it and if PV __GUESTS__ are not supported by makedumpfile
>>> skip tests for them. Just test dom0 and Xen hypervisor stuff.
>>
>> I concur, this dump format is not supported by makedumpfile.
>>
>> I'm still working on validating 32-bit dump files.
>
>I have validated the 32-bit dump files with this patch. All is good.
>eric
>
>
>_______________________________________________
>kexec mailing list
>kexec at lists.infradead.org
>http://lists.infradead.org/mailman/listinfo/kexec



From james.morse at arm.com  Fri Jun  2 01:23:28 2017
From: james.morse at arm.com (James Morse)
Date: Fri, 02 Jun 2017 09:23:28 +0100
Subject: [PATCH v3 0/2] kexec-tools: arm64: Enable D-cache in purgatory
In-Reply-To: <cover.1495514291.git.panand@redhat.com>
References: <cover.1495514291.git.panand@redhat.com>
Message-ID: <59312080.1090906@arm.com>

Hi Pratyush,

On 23/05/17 06:02, Pratyush Anand wrote:
> It takes more that 2 minutes to verify SHA in purgatory when vmlinuz image
> is around 13MB and initramfs is around 30MB. It takes more than 20 second
> even when we have -O2 optimization enabled. However, if dcache is enabled
> during purgatory execution then, it takes just a second in SHA
> verification.
> 
> Therefore, these patches adds support for dcache enabling facility during
> purgatory execution.

I'm still not convinced we need this. Moving the SHA verification to happen
before the dcache+mmu are disabled would also solve the delay problem, and we
can print an error message or fail the syscall.

For kexec we don't expect memory corruption, what are we testing for?
I can see the use for kdump, but the kdump-kernel is unmapped so the kernel
can't accidentally write over it.

(we discussed all this last time, but it fizzled-out. If you and the
 kexec-tools maintainer think its necessary, fine by me!)

I have some comments on making this code easier to maintain..


Thanks,

James


From james.morse at arm.com  Fri Jun  2 01:24:03 2017
From: james.morse at arm.com (James Morse)
Date: Fri, 02 Jun 2017 09:24:03 +0100
Subject: [PATCH v3 1/2] kexec: arm64: create identity page table to be
 used in purgatory
In-Reply-To: <3355da35facd39ed02e66c0d7d50f86692d565a1.1495514291.git.panand@redhat.com>
References: <cover.1495514291.git.panand@redhat.com>
 <3355da35facd39ed02e66c0d7d50f86692d565a1.1495514291.git.panand@redhat.com>
Message-ID: <593120A3.8040806@arm.com>

Hi Pratyush,

On 23/05/17 06:02, Pratyush Anand wrote:
> Purgatory sha verification is very slow when D-cache is not enabled there.
> We need to enable MMU as well to enable D-Cache.Therefore,we need to
> have an identity mapped page table in purgatory.
> 
> Since debugging is very difficult in purgatory therefore we prefer to do as
> much work as possible in kexec.
> 
> This patch prepares page table for purgatory in advance. We support only 4K
> page table,because it will be available on most of the platform. This page
> table is passed to the purgatory as a new segment.
> 
> VA bit is fixed as 48, page table level is 3 where 3rd level page table
> contains 2M block entries.

This had me confused: You actually have 4 levels of page tables, but the last
level is never used, because the third level always contains block entries.

If you want to use the kernel's macros for shifts and masks (which I think you
should), you must use the same terminology.

Because you wrote this:
> +#define PGTABLE_LEVELS		3

The calculated PGDIR_SHIFT is what the kernel would call PUD_SHIFT with 48bit VA.

> +#define ARM64_HW_PGTABLE_LEVEL_SHIFT(n) ((PAGE_SHIFT - 3) * (4 - (n)) + 3)
> +#define PGDIR_SHIFT		ARM64_HW_PGTABLE_LEVEL_SHIFT(4 - PGTABLE_LEVELS)

Which then explains why you have to invent a new name for PGD_SHIFT, and
calculate it manually:
> +#define EXTRA_SHIFT		(PGDIR_SHIFT + PAGE_SHIFT - 3)


kexec-tools is closely coupled to the kernel, can we try to make this as similar
as possible so that code (and reviewers!) can easily move between the two.

(some other comments below)

Thanks,


James

> diff --git a/kexec/arch/arm64/kexec-arm64.c b/kexec/arch/arm64/kexec-arm64.c
> index 62f37585b788..af5dab331e97 100644
> --- a/kexec/arch/arm64/kexec-arm64.c
> +++ b/kexec/arch/arm64/kexec-arm64.c
> @@ -33,6 +34,10 @@
>  #define PROP_ELFCOREHDR "linux,elfcorehdr"
>  #define PROP_USABLE_MEM_RANGE "linux,usable-memory-range"
>  
> +typedef unsigned long guest_addr_t;
> +typedef unsigned long host_addr_t;

What do guest and host mean here?
I assumed everything that happens in purgatory would have something like
phys_addr_t (for both physical and virtual addresses).


> +static int next_tbl_cnt = 1;
> +
>  /* Global varables the core kexec routines expect. */

Nit: variables


>  
>  unsigned char reuse_initrd;
> @@ -519,6 +524,118 @@ unsigned long arm64_locate_kernel_segment(struct kexec_info *info)
>  	return hole;
>  }
>  
> +static host_addr_t *create_table_entry(host_addr_t *pgtbl_buf,
> +		guest_addr_t pgtbl_mem, host_addr_t *tbl,
> +		guest_addr_t virt, int shift,
> +		unsigned long ptrs)
> +{
> +	unsigned long index, desc, offset;
> +
> +	index = (virt >> shift) & (ptrs - 1);
> +	/* check if we have allocated a table already for this index */
> +	if (tbl[index]) {
> +		/*
> +		 * table index will have entry as per purgatory (guest)page
> +		 * table memory. Find out corresponding buffer address of
> +		 * table (host).
> +		 */
> +		desc = tbl[index] & ~3UL;
> +		offset = desc - pgtbl_mem;
> +		return &pgtbl_buf[offset >> 3];
> +	}
> +
> +	if (next_tbl_cnt >= MAX_PGTBLE_SZ / PAGE_SIZE)
> +		die("%s:No more memory for page table\n", __func__);
> +	/*
> +	 * Always write page table content as per page table memory
> +	 * allocated for purgatory(guest) area, but return corresponding
> +	 * buffer area allocated in kexec (host).
> +	 */
> +
> +	tbl[index] = (pgtbl_mem + PAGE_SIZE * next_tbl_cnt) | PMD_TYPE_TABLE;
> +
> +	return &pgtbl_buf[(next_tbl_cnt++ * PAGE_SIZE) >> 3];
> +}
> +
> +static void create_block_entry(host_addr_t *tbl, unsigned long flags,
> +				guest_addr_t virt)
> +{
> +	unsigned long index;
> +	unsigned long desc;
> +
> +	index = pmd_index(virt);
> +	desc = virt & PMD_MASK;
> +	desc |= flags;
> +	tbl[index] = desc;
> +}
> +
> +static void create_identity_entry(host_addr_t *pgtbl_buf,
> +		guest_addr_t pgtbl_mem, guest_addr_t virt,
> +		unsigned long flags)
> +{
> +	host_addr_t *tbl = pgtbl_buf;
> +	tbl = create_table_entry(pgtbl_buf, pgtbl_mem, tbl, virt,
> +			EXTRA_SHIFT, EXTRA_PTRS);
> +	tbl = create_table_entry(pgtbl_buf, pgtbl_mem, tbl, virt,
> +			IDENTITY_SHIFT, PTRS_PER_PTE);

Are 'extra' and 'identity' names for levels in the page tables? If so can we use
the same names as the kernel.


> +	create_block_entry(tbl, flags, virt);
> +}
> +
> +/**
> + * arm64_create_pgtbl_segment - Create page table segments to be used by
> + * purgatory. Page table will have entries to access memory area of all
> + * those segments which becomes part of sha verification in purgatory.
> + * Additionally, we also create page table for purgatory segment as well.
> + */
> +
> +static int arm64_create_pgtbl_segment(struct kexec_info *info,
> +		unsigned long hole_min, unsigned long hole_max)
> +{
> +	host_addr_t *pgtbl_buf;
> +	guest_addr_t pgtbl_mem, mstart, mend;
> +	int i;
> +	unsigned long purgatory_base, purgatory_len;
> +
> +	pgtbl_buf = xmalloc(MAX_PGTBLE_SZ);
> +	memset(pgtbl_buf, 0, MAX_PGTBLE_SZ);
> +	pgtbl_mem = add_buffer_phys_virt(info, pgtbl_buf, MAX_PGTBLE_SZ,
> +			MAX_PGTBLE_SZ, PAGE_SIZE, hole_min, hole_max, 1, 0);
> +	for (i = 0; i < info->nr_segments; i++) {
> +		if (info->segment[i].mem == (void *)info->rhdr.rel_addr) {
> +			purgatory_base = (unsigned long)info->segment[i].mem;
> +			purgatory_len = info->segment[i].memsz;
> +		}
> +		mstart = (unsigned long)info->segment[i].mem;
> +		mend = mstart + info->segment[i].memsz;
> +		mstart &= ~(SECTION_SIZE - 1);
> +		while (mstart < mend) {
> +			create_identity_entry(pgtbl_buf, pgtbl_mem,
> +					mstart, MMU_FLAGS_NORMAL);
> +			mstart += SECTION_SIZE;
> +		}
> +	}
> +
> +	/* we will need pgtble_base in purgatory for enabling d-cache */
> +	elf_rel_set_symbol(&info->rhdr, "pgtble_base", &pgtbl_mem,
> +		sizeof(pgtbl_mem));
> +	/*
> +	 * We need to disable d-cache before we exit from purgatory.
> +	 * Since, only d-cache flush by VAs is recommended, therefore we
> +	 * will also need memory location of all those area which will be
> +	 * accessed in purgatory with enabled d-cache. sha256_regions
> +	 * already have start and length for all the segments except
> +	 * purgatory. Therefore, we will need to pass start and length of
> +	 * purgatory additionally.
> +	 */
> +	elf_rel_set_symbol(&info->rhdr, "purgatory_base", &purgatory_base,
> +		sizeof(purgatory_base));
> +	elf_rel_set_symbol(&info->rhdr, "purgatory_len", &purgatory_len,
> +		sizeof(purgatory_len));
> +
> +	return 0;
> +}
> +
>  /**
>   * arm64_load_other_segments - Prepare the dtb, initrd and purgatory segments.
>   */
> @@ -630,6 +747,8 @@ int arm64_load_other_segments(struct kexec_info *info,
>  	elf_rel_set_symbol(&info->rhdr, "arm64_dtb_addr", &dtb_base,
>  		sizeof(dtb_base));
>  
> +	arm64_create_pgtbl_segment(info, hole_min, hole_max);

hole_min->hole_max is the region from the end of the Kernel image, to the end of
usable memory. I can't see how this window is made smaller as the dtb, initramfs
then page tables are added to it... (kvmtool updates the equivalent values as it
adds things)

Does kexec-tools have some other accounting for what memory is in use? (in which
case why are these values needed?)


>  	return 0;
>  }
>  
> diff --git a/kexec/arch/arm64/kexec-mmu.h b/kexec/arch/arm64/kexec-mmu.h
> new file mode 100644
> index 000000000000..55354b5e3002
> --- /dev/null
> +++ b/kexec/arch/arm64/kexec-mmu.h
> @@ -0,0 +1,58 @@
> +#if !defined(KEXEC_MMU_H)
> +#define KEXEC_MMU_H

As lots of the symbols names and values below were copied from the kernel, we
should copy the copyright statement too. Something like:

/*
 * Based on linux v4.11's arch/arm64/include/asm/pgtable-hwdef.h
 * Copyright (C) 2012 ARM Ltd.
 */


> +/*
> + * kexec creates identity page table to be used in purgatory so that

> + * dcache verification becomes faster.

... so that SHA verification of the image can make use of the dcache.


> + *
> + * These are the definitions to be used by page table creation routine.
> + *
> + * Only 4K page table, 3 level, 2M block mapping, 48bit VA is supported
> + */
> +#define PAGE_SHIFT		12
> +#define PGTABLE_LEVELS		3
> +#define ARM64_HW_PGTABLE_LEVEL_SHIFT(n) ((PAGE_SHIFT - 3) * (4 - (n)) + 3)
> +#define PGDIR_SHIFT		ARM64_HW_PGTABLE_LEVEL_SHIFT(4 - PGTABLE_LEVELS)

> +#define EXTRA_SHIFT		(PGDIR_SHIFT + PAGE_SHIFT - 3)
> +#define EXTRA_PTRS		(1 << (48 - EXTRA_SHIFT))

This looks like the level above PGD? What do you need this for if you have only
3 levels of page tables?


> +#define PUD_SHIFT		PGDIR_SHIFT
> +#define PMD_SHIFT		ARM64_HW_PGTABLE_LEVEL_SHIFT(2)
> +#define PMD_SIZE		(1UL << PMD_SHIFT)
> +#define PMD_MASK		(~(PMD_SIZE-1))
> +#define IDENTITY_SHIFT		PUD_SHIFT
> +#define PTRS_PER_PTE		(1 << (PAGE_SHIFT - 3))
> +#define PTRS_PER_PMD		PTRS_PER_PTE
> +#define pmd_index(addr)		(((addr) >> PMD_SHIFT) & (PTRS_PER_PMD - 1))
> +#define PMD_TYPE_TABLE		(3UL << 0)
> +#define PMD_TYPE_SECT		(1UL << 0)
> +#define PMD_SECT_AF		(1UL << 10)
> +#define PMD_ATTRINDX(t)		((unsigned long)(t) << 2)
> +#define MT_NORMAL		0
> +#define PMD_FLAGS_NORMAL	(PMD_TYPE_SECT | PMD_SECT_AF)
> +#define MMU_FLAGS_NORMAL	(PMD_ATTRINDX(MT_NORMAL) | PMD_FLAGS_NORMAL)
> +#define SECTION_SHIFT		PMD_SHIFT
> +#define SECTION_SIZE		(1UL << SECTION_SHIFT)
> +#define PAGE_SIZE		(1 << PAGE_SHIFT)
> +/*
> + * Since we are using 3 level of page tables, therefore minimum number of
> + * table will be 3. Each entry in level 3 page table can map 2MB memory
> + * area. Thus a level 3 page table indexed by bit 29:21 can map a total of
> + * 1G memory area. Therefore, if any segment crosses 1G boundary, then we
> + * will need one more level 3 table. Similarly, level 2 page table indexed
> + * by bit 38:30 can map a total of 512G memory area. If segment addresses
> + * are more than 512G apart then we will need two more table for each such
> + * block.
> + * Lets consider 2G as the maximum size of crashkernel.

crashkernel+initramfs ?

A good assumption. Can we test this is true when the image is loaded?
(otherwise its going to be really fun to debug!)


> This 2G memory
> + * location might not be allocated at 1G aligned boundary, so in that case
> + * we need to have 5 table size resrved to map any location of the crash

(Nit: reserved)


> + * kernel region.
> + *
> + * If we will ever wish to support uart debugging in purgatory then that
> + * might cross the boundary and therefore additional 2 more table space. In
> + * that case, we will need a total of 7 table space.
> + *
> + * As of now keep it fixed at 5. Increase it if any platform either
> + * supports uart or more than 2G of crash kernel size.
> + */
> +#define MAX_PGTBLE_SZ	(5 * PAGE_SIZE)
> +
> +#endif
> 


From ard.biesheuvel at linaro.org  Fri Jun  2 02:55:07 2017
From: ard.biesheuvel at linaro.org (Ard Biesheuvel)
Date: Fri, 2 Jun 2017 09:55:07 +0000
Subject: [PATCH v3 0/2] kexec-tools: arm64: Enable D-cache in purgatory
In-Reply-To: <59312080.1090906@arm.com>
References: <cover.1495514291.git.panand@redhat.com> <59312080.1090906@arm.com>
Message-ID: <CAKv+Gu--N4Pef6z=G4ePTgF3CcKhmmSfrvJBY-5p0CLL_PZ=3Q@mail.gmail.com>

On 2 June 2017 at 08:23, James Morse <james.morse at arm.com> wrote:
> Hi Pratyush,
>
> On 23/05/17 06:02, Pratyush Anand wrote:
>> It takes more that 2 minutes to verify SHA in purgatory when vmlinuz image
>> is around 13MB and initramfs is around 30MB. It takes more than 20 second
>> even when we have -O2 optimization enabled. However, if dcache is enabled
>> during purgatory execution then, it takes just a second in SHA
>> verification.
>>
>> Therefore, these patches adds support for dcache enabling facility during
>> purgatory execution.
>
> I'm still not convinced we need this. Moving the SHA verification to happen
> before the dcache+mmu are disabled would also solve the delay problem, and we
> can print an error message or fail the syscall.
>
> For kexec we don't expect memory corruption, what are we testing for?

This is a very good question. SHA-256 is quite a heavy hammer if all
you need is CRC style error detection. Note that SHA-256 uses 256
bytes of round keys, which means that in the absence of a cache, each
64 byte chunk of data processed involves (re)reading 320 bytes from
DRAM. That also means you could write a SHA-256 implementation for
AArch64 that keeps the round keys in NEON registers instead, and it
would probably be a lot faster.

> I can see the use for kdump, but the kdump-kernel is unmapped so the kernel
> can't accidentally write over it.
>
> (we discussed all this last time, but it fizzled-out. If you and the
>  kexec-tools maintainer think its necessary, fine by me!)
>


From bhupesh.linux at gmail.com  Fri Jun  2 04:15:23 2017
From: bhupesh.linux at gmail.com (Bhupesh SHARMA)
Date: Fri, 2 Jun 2017 16:45:23 +0530
Subject: [PATCH v3 0/2] kexec-tools: arm64: Enable D-cache in purgatory
In-Reply-To: <CAKv+Gu--N4Pef6z=G4ePTgF3CcKhmmSfrvJBY-5p0CLL_PZ=3Q@mail.gmail.com>
References: <cover.1495514291.git.panand@redhat.com> <59312080.1090906@arm.com>
 <CAKv+Gu--N4Pef6z=G4ePTgF3CcKhmmSfrvJBY-5p0CLL_PZ=3Q@mail.gmail.com>
Message-ID: <CAFTCetS17w0-BJJzTfi8uNS67xos4MwJz3R=qdz2Oe7+fcXk3g@mail.gmail.com>

Hi Ard, James

On Fri, Jun 2, 2017 at 3:25 PM, Ard Biesheuvel
<ard.biesheuvel at linaro.org> wrote:
> On 2 June 2017 at 08:23, James Morse <james.morse at arm.com> wrote:
>> Hi Pratyush,
>>
>> On 23/05/17 06:02, Pratyush Anand wrote:
>>> It takes more that 2 minutes to verify SHA in purgatory when vmlinuz image
>>> is around 13MB and initramfs is around 30MB. It takes more than 20 second
>>> even when we have -O2 optimization enabled. However, if dcache is enabled
>>> during purgatory execution then, it takes just a second in SHA
>>> verification.
>>>
>>> Therefore, these patches adds support for dcache enabling facility during
>>> purgatory execution.
>>
>> I'm still not convinced we need this. Moving the SHA verification to happen
>> before the dcache+mmu are disabled would also solve the delay problem, and we
>> can print an error message or fail the syscall.
>>
>> For kexec we don't expect memory corruption, what are we testing for?
>
> This is a very good question. SHA-256 is quite a heavy hammer if all
> you need is CRC style error detection. Note that SHA-256 uses 256
> bytes of round keys, which means that in the absence of a cache, each
> 64 byte chunk of data processed involves (re)reading 320 bytes from
> DRAM. That also means you could write a SHA-256 implementation for
> AArch64 that keeps the round keys in NEON registers instead, and it
> would probably be a lot faster.

AFAICR the sha-256 implementation was proposed to boot a signed
kexec/kdump kernel to circumvent kexec from violating UEFI secure boot
restrictions (see [1]).

As Matthew Garret rightly noted (see[2]), secure Boot, if enabled, is
explicitly designed to stop you booting modified kernels unless you've
added your own keys.

But if you boot a signed Linux distribution with kexec enabled without
using the SHA like feature in the purgatory (like, say, Ubuntu) then
you're able to boot a modified Windows kernel that will still believe
it was booted securely.

So, CRC wouldn't possibly fulfil the functionality we are trying to
achieve with SHA-256 in the purgatory.

However, having seen the power of using the inbuilt CRC instructions
from the ARM64 ISA on a SoC which supports it, I can vouch that the
native ISA implementations are much faster than other approaches.

However, using the SHA-256 implementation (as you rightly noted) would
employ NEON registers and can be faster, however I remember some SoC
vendors disabling co-processor extensions in their ARM implementations
in the past, so I am not sure we can assume that NEON extensions would
be available in all ARMv8 implementations by default.

[1] https://lwn.net/Articles/603116/
[2] http://mjg59.dreamwidth.org/28746.html

Regards,
Bhupesh

>> I can see the use for kdump, but the kdump-kernel is unmapped so the kernel
>> can't accidentally write over it.
>>
>> (we discussed all this last time, but it fizzled-out. If you and the
>>  kexec-tools maintainer think its necessary, fine by me!)
>>
>
> _______________________________________________
> kexec mailing list
> kexec at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/kexec


From ard.biesheuvel at linaro.org  Fri Jun  2 04:36:02 2017
From: ard.biesheuvel at linaro.org (Ard Biesheuvel)
Date: Fri, 2 Jun 2017 11:36:02 +0000
Subject: [PATCH v3 0/2] kexec-tools: arm64: Enable D-cache in purgatory
In-Reply-To: <CAFTCetS17w0-BJJzTfi8uNS67xos4MwJz3R=qdz2Oe7+fcXk3g@mail.gmail.com>
References: <cover.1495514291.git.panand@redhat.com> <59312080.1090906@arm.com>
 <CAKv+Gu--N4Pef6z=G4ePTgF3CcKhmmSfrvJBY-5p0CLL_PZ=3Q@mail.gmail.com>
 <CAFTCetS17w0-BJJzTfi8uNS67xos4MwJz3R=qdz2Oe7+fcXk3g@mail.gmail.com>
Message-ID: <CAKv+Gu86EEFzVzG4h0f6sBk=ar4Ffv_KuNhyKNi_Zf7k3KvWcg@mail.gmail.com>

On 2 June 2017 at 11:15, Bhupesh SHARMA <bhupesh.linux at gmail.com> wrote:
> Hi Ard, James
>
> On Fri, Jun 2, 2017 at 3:25 PM, Ard Biesheuvel
> <ard.biesheuvel at linaro.org> wrote:
>> On 2 June 2017 at 08:23, James Morse <james.morse at arm.com> wrote:
>>> Hi Pratyush,
>>>
>>> On 23/05/17 06:02, Pratyush Anand wrote:
>>>> It takes more that 2 minutes to verify SHA in purgatory when vmlinuz image
>>>> is around 13MB and initramfs is around 30MB. It takes more than 20 second
>>>> even when we have -O2 optimization enabled. However, if dcache is enabled
>>>> during purgatory execution then, it takes just a second in SHA
>>>> verification.
>>>>
>>>> Therefore, these patches adds support for dcache enabling facility during
>>>> purgatory execution.
>>>
>>> I'm still not convinced we need this. Moving the SHA verification to happen
>>> before the dcache+mmu are disabled would also solve the delay problem, and we
>>> can print an error message or fail the syscall.
>>>
>>> For kexec we don't expect memory corruption, what are we testing for?
>>
>> This is a very good question. SHA-256 is quite a heavy hammer if all
>> you need is CRC style error detection. Note that SHA-256 uses 256
>> bytes of round keys, which means that in the absence of a cache, each
>> 64 byte chunk of data processed involves (re)reading 320 bytes from
>> DRAM. That also means you could write a SHA-256 implementation for
>> AArch64 that keeps the round keys in NEON registers instead, and it
>> would probably be a lot faster.
>
> AFAICR the sha-256 implementation was proposed to boot a signed
> kexec/kdump kernel to circumvent kexec from violating UEFI secure boot
> restrictions (see [1]).
>
> As Matthew Garret rightly noted (see[2]), secure Boot, if enabled, is
> explicitly designed to stop you booting modified kernels unless you've
> added your own keys.
>
> But if you boot a signed Linux distribution with kexec enabled without
> using the SHA like feature in the purgatory (like, say, Ubuntu) then
> you're able to boot a modified Windows kernel that will still believe
> it was booted securely.
>
> So, CRC wouldn't possibly fulfil the functionality we are trying to
> achieve with SHA-256 in the purgatory.
>

OK. But it appears that kexec_load_file() generates the hashes, and
the purgatory just double checks them? That means there is wiggle room
in terms of hash implementation, even though a non-cryptographic hash
may be out of the question.

> However, having seen the power of using the inbuilt CRC instructions
> from the ARM64 ISA on a SoC which supports it, I can vouch that the
> native ISA implementations are much faster than other approaches.
>
> However, using the SHA-256 implementation (as you rightly noted) would
> employ NEON registers and can be faster, however I remember some SoC
> vendors disabling co-processor extensions in their ARM implementations
> in the past, so I am not sure we can assume that NEON extensions would
> be available in all ARMv8 implementations by default.
>

Alternatively, a SHA-256 implementation that uses movz/movk sequences
instead of ldr instructions to load the round constants would already
be 5x faster, given that we don't need page tables to enable the
I-cache.


From ard.biesheuvel at linaro.org  Fri Jun  2 04:44:31 2017
From: ard.biesheuvel at linaro.org (Ard Biesheuvel)
Date: Fri, 2 Jun 2017 11:44:31 +0000
Subject: [PATCH v3 0/2] kexec-tools: arm64: Enable D-cache in purgatory
In-Reply-To: <CAKv+Gu86EEFzVzG4h0f6sBk=ar4Ffv_KuNhyKNi_Zf7k3KvWcg@mail.gmail.com>
References: <cover.1495514291.git.panand@redhat.com> <59312080.1090906@arm.com>
 <CAKv+Gu--N4Pef6z=G4ePTgF3CcKhmmSfrvJBY-5p0CLL_PZ=3Q@mail.gmail.com>
 <CAFTCetS17w0-BJJzTfi8uNS67xos4MwJz3R=qdz2Oe7+fcXk3g@mail.gmail.com>
 <CAKv+Gu86EEFzVzG4h0f6sBk=ar4Ffv_KuNhyKNi_Zf7k3KvWcg@mail.gmail.com>
Message-ID: <CAKv+Gu-0p6iWnY9fr67YrYUgiLwArwkNUKio-7XjT0uMafcQ-w@mail.gmail.com>

On 2 June 2017 at 11:36, Ard Biesheuvel <ard.biesheuvel at linaro.org> wrote:
> On 2 June 2017 at 11:15, Bhupesh SHARMA <bhupesh.linux at gmail.com> wrote:
>> Hi Ard, James
>>
>> On Fri, Jun 2, 2017 at 3:25 PM, Ard Biesheuvel
>> <ard.biesheuvel at linaro.org> wrote:
>>> On 2 June 2017 at 08:23, James Morse <james.morse at arm.com> wrote:
>>>> Hi Pratyush,
>>>>
>>>> On 23/05/17 06:02, Pratyush Anand wrote:
>>>>> It takes more that 2 minutes to verify SHA in purgatory when vmlinuz image
>>>>> is around 13MB and initramfs is around 30MB. It takes more than 20 second
>>>>> even when we have -O2 optimization enabled. However, if dcache is enabled
>>>>> during purgatory execution then, it takes just a second in SHA
>>>>> verification.
>>>>>
>>>>> Therefore, these patches adds support for dcache enabling facility during
>>>>> purgatory execution.
>>>>
>>>> I'm still not convinced we need this. Moving the SHA verification to happen
>>>> before the dcache+mmu are disabled would also solve the delay problem, and we
>>>> can print an error message or fail the syscall.
>>>>
>>>> For kexec we don't expect memory corruption, what are we testing for?
>>>
>>> This is a very good question. SHA-256 is quite a heavy hammer if all
>>> you need is CRC style error detection. Note that SHA-256 uses 256
>>> bytes of round keys, which means that in the absence of a cache, each
>>> 64 byte chunk of data processed involves (re)reading 320 bytes from
>>> DRAM. That also means you could write a SHA-256 implementation for
>>> AArch64 that keeps the round keys in NEON registers instead, and it
>>> would probably be a lot faster.
>>
>> AFAICR the sha-256 implementation was proposed to boot a signed
>> kexec/kdump kernel to circumvent kexec from violating UEFI secure boot
>> restrictions (see [1]).
>>
>> As Matthew Garret rightly noted (see[2]), secure Boot, if enabled, is
>> explicitly designed to stop you booting modified kernels unless you've
>> added your own keys.
>>
>> But if you boot a signed Linux distribution with kexec enabled without
>> using the SHA like feature in the purgatory (like, say, Ubuntu) then
>> you're able to boot a modified Windows kernel that will still believe
>> it was booted securely.
>>
>> So, CRC wouldn't possibly fulfil the functionality we are trying to
>> achieve with SHA-256 in the purgatory.
>>
>
> OK. But it appears that kexec_load_file() generates the hashes, and
> the purgatory just double checks them? That means there is wiggle room
> in terms of hash implementation, even though a non-cryptographic hash
> may be out of the question.
>
>> However, having seen the power of using the inbuilt CRC instructions
>> from the ARM64 ISA on a SoC which supports it, I can vouch that the
>> native ISA implementations are much faster than other approaches.
>>
>> However, using the SHA-256 implementation (as you rightly noted) would
>> employ NEON registers and can be faster, however I remember some SoC
>> vendors disabling co-processor extensions in their ARM implementations
>> in the past, so I am not sure we can assume that NEON extensions would
>> be available in all ARMv8 implementations by default.
>>
>
> Alternatively, a SHA-256 implementation that uses movz/movk sequences
> instead of ldr instructions to load the round constants would already
> be 5x faster, given that we don't need page tables to enable the
> I-cache.

Actually, looking at the C code and the objdump of the kernel's
sha256_generic driver, it is likely that it is already doing this, and
none of the points I made actually make a lot of sense ...

Pratyush: I assume you are already enabling the I-cache in the purgatory?


From panand at redhat.com  Fri Jun  2 07:32:25 2017
From: panand at redhat.com (Pratyush Anand)
Date: Fri, 2 Jun 2017 20:02:25 +0530
Subject: [PATCH v3 0/2] kexec-tools: arm64: Enable D-cache in purgatory
In-Reply-To: <CAKv+Gu-0p6iWnY9fr67YrYUgiLwArwkNUKio-7XjT0uMafcQ-w@mail.gmail.com>
References: <cover.1495514291.git.panand@redhat.com> <59312080.1090906@arm.com>
 <CAKv+Gu--N4Pef6z=G4ePTgF3CcKhmmSfrvJBY-5p0CLL_PZ=3Q@mail.gmail.com>
 <CAFTCetS17w0-BJJzTfi8uNS67xos4MwJz3R=qdz2Oe7+fcXk3g@mail.gmail.com>
 <CAKv+Gu86EEFzVzG4h0f6sBk=ar4Ffv_KuNhyKNi_Zf7k3KvWcg@mail.gmail.com>
 <CAKv+Gu-0p6iWnY9fr67YrYUgiLwArwkNUKio-7XjT0uMafcQ-w@mail.gmail.com>
Message-ID: <2b80fca3-b462-a1b5-36db-e8b8b8fdc126@redhat.com>

Hi Ard,

Thanks for all the inputs.

On Friday 02 June 2017 05:14 PM, Ard Biesheuvel wrote:
>> Alternatively, a SHA-256 implementation that uses movz/movk sequences
>> instead of ldr instructions to load the round constants would already
>> be 5x faster, given that we don't need page tables to enable the
>> I-cache.
> Actually, looking at the C code and the objdump of the kernel's
> sha256_generic driver, it is likely that it is already doing this, and
> none of the points I made actually make a lot of sense ...
>
> Pratyush: I assume you are already enabling the I-cache in the purgatory?

It's not enabled yet. But, I had tried first with enabling only I-cache, but 
that did not make much difference in execution time.

~Pratyush




From panand at redhat.com  Fri Jun  2 07:29:09 2017
From: panand at redhat.com (Pratyush Anand)
Date: Fri, 2 Jun 2017 19:59:09 +0530
Subject: [PATCH v3 1/2] kexec: arm64: create identity page table to be
 used in purgatory
In-Reply-To: <593120A3.8040806@arm.com>
References: <cover.1495514291.git.panand@redhat.com>
 <3355da35facd39ed02e66c0d7d50f86692d565a1.1495514291.git.panand@redhat.com>
 <593120A3.8040806@arm.com>
Message-ID: <ce0087e5-4a50-0150-3e17-96b6a25c51e4@redhat.com>

Hi James,

Thanks for taking out time to review the code.

On Friday 02 June 2017 01:54 PM, James Morse wrote:
> Hi Pratyush,
>
> On 23/05/17 06:02, Pratyush Anand wrote:
>> Purgatory sha verification is very slow when D-cache is not enabled there.
>> We need to enable MMU as well to enable D-Cache.Therefore,we need to
>> have an identity mapped page table in purgatory.
>>
>> Since debugging is very difficult in purgatory therefore we prefer to do as
>> much work as possible in kexec.
>>
>> This patch prepares page table for purgatory in advance. We support only 4K
>> page table,because it will be available on most of the platform. This page
>> table is passed to the purgatory as a new segment.
>>
>> VA bit is fixed as 48, page table level is 3 where 3rd level page table
>> contains 2M block entries.
>
> This had me confused: You actually have 4 levels of page tables, but the last
> level is never used, because the third level always contains block entries.
>
> If you want to use the kernel's macros for shifts and masks (which I think you
> should), you must use the same terminology.
>
> Because you wrote this:
>> +#define PGTABLE_LEVELS		3
>
> The calculated PGDIR_SHIFT is what the kernel would call PUD_SHIFT with 48bit VA.
>
>> +#define ARM64_HW_PGTABLE_LEVEL_SHIFT(n) ((PAGE_SHIFT - 3) * (4 - (n)) + 3)
>> +#define PGDIR_SHIFT		ARM64_HW_PGTABLE_LEVEL_SHIFT(4 - PGTABLE_LEVELS)
>
> Which then explains why you have to invent a new name for PGD_SHIFT, and
> calculate it manually:
>> +#define EXTRA_SHIFT		(PGDIR_SHIFT + PAGE_SHIFT - 3)
>
>
> kexec-tools is closely coupled to the kernel, can we try to make this as similar
> as possible so that code (and reviewers!) can easily move between the two.

OK..I will try to sort out the differences
...
>
> (some other comments below)
>
> Thanks,
>
>
> James
>
>> diff --git a/kexec/arch/arm64/kexec-arm64.c b/kexec/arch/arm64/kexec-arm64.c
>> index 62f37585b788..af5dab331e97 100644
>> --- a/kexec/arch/arm64/kexec-arm64.c
>> +++ b/kexec/arch/arm64/kexec-arm64.c
>> @@ -33,6 +34,10 @@
>>  #define PROP_ELFCOREHDR "linux,elfcorehdr"
>>  #define PROP_USABLE_MEM_RANGE "linux,usable-memory-range"
>>
>> +typedef unsigned long guest_addr_t;
>> +typedef unsigned long host_addr_t;
>
> What do guest and host mean here?

guest is something which will be in purgatory domain and host is something in 
kexec domain. I remember, you had suggested to differentiate these two address 
domain for better code readability.

> I assumed everything that happens in purgatory would have something like
> phys_addr_t (for both physical and virtual addresses).

Yes, so guest_addr_t is like phys_addr_t.  I can replace, if there are some 
better names to differentiate between these two address domain.

>
>
>> +static int next_tbl_cnt = 1;
>> +
>>  /* Global varables the core kexec routines expect. */
>
> Nit: variables

OK.

>
>
>>
>>  unsigned char reuse_initrd;
>> @@ -519,6 +524,118 @@ unsigned long arm64_locate_kernel_segment(struct kexec_info *info)
>>  	return hole;
>>  }
>>
>> +static host_addr_t *create_table_entry(host_addr_t *pgtbl_buf,
>> +		guest_addr_t pgtbl_mem, host_addr_t *tbl,
>> +		guest_addr_t virt, int shift,
>> +		unsigned long ptrs)
>> +{
>> +	unsigned long index, desc, offset;
>> +
>> +	index = (virt >> shift) & (ptrs - 1);
>> +	/* check if we have allocated a table already for this index */
>> +	if (tbl[index]) {
>> +		/*
>> +		 * table index will have entry as per purgatory (guest)page
>> +		 * table memory. Find out corresponding buffer address of
>> +		 * table (host).
>> +		 */
>> +		desc = tbl[index] & ~3UL;
>> +		offset = desc - pgtbl_mem;
>> +		return &pgtbl_buf[offset >> 3];
>> +	}
>> +
>> +	if (next_tbl_cnt >= MAX_PGTBLE_SZ / PAGE_SIZE)
>> +		die("%s:No more memory for page table\n", __func__);
>> +	/*
>> +	 * Always write page table content as per page table memory
>> +	 * allocated for purgatory(guest) area, but return corresponding
>> +	 * buffer area allocated in kexec (host).
>> +	 */
>> +
>> +	tbl[index] = (pgtbl_mem + PAGE_SIZE * next_tbl_cnt) | PMD_TYPE_TABLE;
>> +
>> +	return &pgtbl_buf[(next_tbl_cnt++ * PAGE_SIZE) >> 3];
>> +}
>> +
>> +static void create_block_entry(host_addr_t *tbl, unsigned long flags,
>> +				guest_addr_t virt)
>> +{
>> +	unsigned long index;
>> +	unsigned long desc;
>> +
>> +	index = pmd_index(virt);
>> +	desc = virt & PMD_MASK;
>> +	desc |= flags;
>> +	tbl[index] = desc;
>> +}
>> +
>> +static void create_identity_entry(host_addr_t *pgtbl_buf,
>> +		guest_addr_t pgtbl_mem, guest_addr_t virt,
>> +		unsigned long flags)
>> +{
>> +	host_addr_t *tbl = pgtbl_buf;
>> +	tbl = create_table_entry(pgtbl_buf, pgtbl_mem, tbl, virt,
>> +			EXTRA_SHIFT, EXTRA_PTRS);
>> +	tbl = create_table_entry(pgtbl_buf, pgtbl_mem, tbl, virt,
>> +			IDENTITY_SHIFT, PTRS_PER_PTE);
>
> Are 'extra' and 'identity' names for levels in the page tables? If so can we use
> the same names as the kernel.
>

OK,will make it similar as per 4 level page (so 3 level swapper page) in kernel.

>
>> +	create_block_entry(tbl, flags, virt);
>> +}
>> +
>> +/**
>> + * arm64_create_pgtbl_segment - Create page table segments to be used by
>> + * purgatory. Page table will have entries to access memory area of all
>> + * those segments which becomes part of sha verification in purgatory.
>> + * Additionally, we also create page table for purgatory segment as well.
>> + */
>> +
>> +static int arm64_create_pgtbl_segment(struct kexec_info *info,
>> +		unsigned long hole_min, unsigned long hole_max)
>> +{
>> +	host_addr_t *pgtbl_buf;
>> +	guest_addr_t pgtbl_mem, mstart, mend;
>> +	int i;
>> +	unsigned long purgatory_base, purgatory_len;
>> +
>> +	pgtbl_buf = xmalloc(MAX_PGTBLE_SZ);
>> +	memset(pgtbl_buf, 0, MAX_PGTBLE_SZ);
>> +	pgtbl_mem = add_buffer_phys_virt(info, pgtbl_buf, MAX_PGTBLE_SZ,
>> +			MAX_PGTBLE_SZ, PAGE_SIZE, hole_min, hole_max, 1, 0);
>> +	for (i = 0; i < info->nr_segments; i++) {
>> +		if (info->segment[i].mem == (void *)info->rhdr.rel_addr) {
>> +			purgatory_base = (unsigned long)info->segment[i].mem;
>> +			purgatory_len = info->segment[i].memsz;
>> +		}
>> +		mstart = (unsigned long)info->segment[i].mem;
>> +		mend = mstart + info->segment[i].memsz;
>> +		mstart &= ~(SECTION_SIZE - 1);
>> +		while (mstart < mend) {
>> +			create_identity_entry(pgtbl_buf, pgtbl_mem,
>> +					mstart, MMU_FLAGS_NORMAL);
>> +			mstart += SECTION_SIZE;
>> +		}
>> +	}
>> +
>> +	/* we will need pgtble_base in purgatory for enabling d-cache */
>> +	elf_rel_set_symbol(&info->rhdr, "pgtble_base", &pgtbl_mem,
>> +		sizeof(pgtbl_mem));
>> +	/*
>> +	 * We need to disable d-cache before we exit from purgatory.
>> +	 * Since, only d-cache flush by VAs is recommended, therefore we
>> +	 * will also need memory location of all those area which will be
>> +	 * accessed in purgatory with enabled d-cache. sha256_regions
>> +	 * already have start and length for all the segments except
>> +	 * purgatory. Therefore, we will need to pass start and length of
>> +	 * purgatory additionally.n
>> +	 */
>> +	elf_rel_set_symbol(&info->rhdr, "purgatory_base", &purgatory_base,
>> +		sizeof(purgatory_base));
>> +	elf_rel_set_symbol(&info->rhdr, "purgatory_len", &purgatory_len,
>> +		sizeof(purgatory_len));
>> +
>> +	return 0;
>> +}
>> +
>>  /**
>>   * arm64_load_other_segments - Prepare the dtb, initrd and purgatory segments.
>>   */
>> @@ -630,6 +747,8 @@ int arm64_load_other_segments(struct kexec_info *info,
>>  	elf_rel_set_symbol(&info->rhdr, "arm64_dtb_addr", &dtb_base,
>>  		sizeof(dtb_base));
>>
>> +	arm64_create_pgtbl_segment(info, hole_min, hole_max);
>
> hole_min->hole_max is the region from the end of the Kernel image, to the end of
> usable memory. I can't see how this window is made smaller as the dtb, initramfs
> then page tables are added to it... (kvmtool updates the equivalent values as it
> adds things)
>
> Does kexec-tools have some other accounting for what memory is in use? (in which
> case why are these values needed?)
>

Please see locate_hole() and add_segment_phys_virt().

locate_hole() finds a hole from info->memory_range lying between hole_min and 
hole_max. add_segment_phys_virt() assigned allocated hole to info->segment. 
locate_hole()  uses info->segment as well to know about consumed regions.

>
>>  	return 0;
>>  }
>>
>> diff --git a/kexec/arch/arm64/kexec-mmu.h b/kexec/arch/arm64/kexec-mmu.h
>> new file mode 100644
>> index 000000000000..55354b5e3002
>> --- /dev/null
>> +++ b/kexec/arch/arm64/kexec-mmu.h
>> @@ -0,0 +1,58 @@
>> +#if !defined(KEXEC_MMU_H)
>> +#define KEXEC_MMU_H
>
> As lots of the symbols names and values below were copied from the kernel, we
> should copy the copyright statement too. Something like:
>
> /*
>  * Based on linux v4.11's arch/arm64/include/asm/pgtable-hwdef.h
>  * Copyright (C) 2012 ARM Ltd.
>  */
>

OK

>
>> +/*
>> + * kexec creates identity page table to be used in purgatory so that
>
>> + * dcache verification becomes faster.
>
> ... so that SHA verification of the image can make use of the dcache.

OK.. :-)

>
>
>> + *
>> + * These are the definitions to be used by page table creation routine.
>> + *
>> + * Only 4K page table, 3 level, 2M block mapping, 48bit VA is supported
>> + */
>> +#define PAGE_SHIFT		12
>> +#define PGTABLE_LEVELS		3
>> +#define ARM64_HW_PGTABLE_LEVEL_SHIFT(n) ((PAGE_SHIFT - 3) * (4 - (n)) + 3)
>> +#define PGDIR_SHIFT		ARM64_HW_PGTABLE_LEVEL_SHIFT(4 - PGTABLE_LEVELS)
>
>> +#define EXTRA_SHIFT		(PGDIR_SHIFT + PAGE_SHIFT - 3)
>> +#define EXTRA_PTRS		(1 << (48 - EXTRA_SHIFT))
>
> This looks like the level above PGD? What do you need this for if you have only
> 3 levels of page tables?

As said above, will remove these names now.

>
>
>> +#define PUD_SHIFT		PGDIR_SHIFT
>> +#define PMD_SHIFT		ARM64_HW_PGTABLE_LEVEL_SHIFT(2)
>> +#define PMD_SIZE		(1UL << PMD_SHIFT)
>> +#define PMD_MASK		(~(PMD_SIZE-1))
>> +#define IDENTITY_SHIFT		PUD_SHIFT
>> +#define PTRS_PER_PTE		(1 << (PAGE_SHIFT - 3))
>> +#define PTRS_PER_PMD		PTRS_PER_PTE
>> +#define pmd_index(addr)		(((addr) >> PMD_SHIFT) & (PTRS_PER_PMD - 1))
>> +#define PMD_TYPE_TABLE		(3UL << 0)
>> +#define PMD_TYPE_SECT		(1UL << 0)
>> +#define PMD_SECT_AF		(1UL << 10)
>> +#define PMD_ATTRINDX(t)		((unsigned long)(t) << 2)
>> +#define MT_NORMAL		0
>> +#define PMD_FLAGS_NORMAL	(PMD_TYPE_SECT | PMD_SECT_AF)
>> +#define MMU_FLAGS_NORMAL	(PMD_ATTRINDX(MT_NORMAL) | PMD_FLAGS_NORMAL)
>> +#define SECTION_SHIFT		PMD_SHIFT
>> +#define SECTION_SIZE		(1UL << SECTION_SHIFT)
>> +#define PAGE_SIZE		(1 << PAGE_SHIFT)
>> +/*
>> + * Since we are using 3 level of page tables, therefore minimum number of
>> + * table will be 3. Each entry in level 3 page table can map 2MB memory
>> + * area. Thus a level 3 page table indexed by bit 29:21 can map a total of
>> + * 1G memory area. Therefore, if any segment crosses 1G boundary, then we
>> + * will need one more level 3 table. Similarly, level 2 page table indexed
>> + * by bit 38:30 can map a total of 512G memory area. If segment addresses
>> + * are more than 512G apart then we will need two more table for each such
>> + * block.
>> + * Lets consider 2G as the maximum size of crashkernel.
>
> crashkernel+initramfs ?

OK

>
> A good assumption. Can we test this is true when the image is loaded?
> (otherwise its going to be really fun to debug!)
>

That we already do. If we cross that boundary we will need additional space 
and it will die() during kexec load.
         if (next_tbl_cnt >= MAX_PGTBLE_SZ / PAGE_SIZE)
                 die("%s:No more memory for page table\n", __func__);
[It is difficult to implement, get going without MMU enabled in that case]

>
>> This 2G memory
>> + * location might not be allocated at 1G aligned boundary, so in that case
>> + * we need to have 5 table size resrved to map any location of the crash
>
> (Nit: reserved)

OK.

>
>
>> + * kernel region.
>> + *
>> + * If we will ever wish to support uart debugging in purgatory then that
>> + * might cross the boundary and therefore additional 2 more table space. In
>> + * that case, we will need a total of 7 table space.
>> + *
>> + * As of now keep it fixed at 5. Increase it if any platform either
>> + * supports uart or more than 2G of crash kernel size.
>> + */
>> +#define MAX_PGTBLE_SZ	(5 * PAGE_SIZE)
>> +
>> +#endif
>>


~Pratyush


From panand at redhat.com  Fri Jun  2 07:42:57 2017
From: panand at redhat.com (Pratyush Anand)
Date: Fri, 2 Jun 2017 20:12:57 +0530
Subject: [PATCH v3 0/2] kexec-tools: arm64: Enable D-cache in purgatory
In-Reply-To: <59312080.1090906@arm.com>
References: <cover.1495514291.git.panand@redhat.com> <59312080.1090906@arm.com>
Message-ID: <1963984d-e78e-a5b2-f1a2-fb0a789a505e@redhat.com>

Hi James,

On Friday 02 June 2017 01:53 PM, James Morse wrote:
> Hi Pratyush,
>
> On 23/05/17 06:02, Pratyush Anand wrote:
>> It takes more that 2 minutes to verify SHA in purgatory when vmlinuz image
>> is around 13MB and initramfs is around 30MB. It takes more than 20 second
>> even when we have -O2 optimization enabled. However, if dcache is enabled
>> during purgatory execution then, it takes just a second in SHA
>> verification.
>>
>> Therefore, these patches adds support for dcache enabling facility during
>> purgatory execution.
>
> I'm still not convinced we need this. Moving the SHA verification to happen
> before the dcache+mmu are disabled would also solve the delay problem,

Humm..I am not sure, if we can do that.

When we leave kernel (and enter into purgatory), icache+dcache+mmu are already 
disabled. I think, that would be possible when we will be in a position to use 
in-kernel purgatory.

> and we
> can print an error message or fail the syscall.
>
> For kexec we don't expect memory corruption, what are we testing for?
> I can see the use for kdump, but the kdump-kernel is unmapped so the kernel
> can't accidentally write over it.
>
> (we discussed all this last time, but it fizzled-out. If you and the
>  kexec-tools maintainer think its necessary, fine by me!)

Yes, there had already been discussion and MAINTAINERs have discouraged 
none-purgatory implementation.

>
> I have some comments on making this code easier to maintain..
>

Thanks.

I have implemented your review comments and have archived the code in

https://github.com/pratyushanand/kexec-tools.git : purgatory-enable-dcache

I will be posting the next version only when someone complains about ARM64 
kdump behavior that it is not as fast as x86.

Thanks for all your time on this series. That really helped me to understand 
the arm64 page table in a better way.

~Pratyush





From james.morse at arm.com  Fri Jun  2 09:36:08 2017
From: james.morse at arm.com (James Morse)
Date: Fri, 02 Jun 2017 17:36:08 +0100
Subject: [PATCH v3 0/2] kexec-tools: arm64: Enable D-cache in purgatory
In-Reply-To: <CAFTCetS17w0-BJJzTfi8uNS67xos4MwJz3R=qdz2Oe7+fcXk3g@mail.gmail.com>
References: <cover.1495514291.git.panand@redhat.com> <59312080.1090906@arm.com>
 <CAKv+Gu--N4Pef6z=G4ePTgF3CcKhmmSfrvJBY-5p0CLL_PZ=3Q@mail.gmail.com>
 <CAFTCetS17w0-BJJzTfi8uNS67xos4MwJz3R=qdz2Oe7+fcXk3g@mail.gmail.com>
Message-ID: <593193F8.1080509@arm.com>

Hi Bhupesh,

On 02/06/17 12:15, Bhupesh SHARMA wrote:
> On Fri, Jun 2, 2017 at 3:25 PM, Ard Biesheuvel
> <ard.biesheuvel at linaro.org> wrote:
>> On 2 June 2017 at 08:23, James Morse <james.morse at arm.com> wrote:
>>> On 23/05/17 06:02, Pratyush Anand wrote:
>>>> It takes more that 2 minutes to verify SHA in purgatory when vmlinuz image
>>>> is around 13MB and initramfs is around 30MB. It takes more than 20 second
>>>> even when we have -O2 optimization enabled. However, if dcache is enabled
>>>> during purgatory execution then, it takes just a second in SHA
>>>> verification.
>>>>
>>>> Therefore, these patches adds support for dcache enabling facility during
>>>> purgatory execution.
>>>
>>> I'm still not convinced we need this. Moving the SHA verification to happen
>>> before the dcache+mmu are disabled would also solve the delay problem, and we
>>> can print an error message or fail the syscall.
>>>
>>> For kexec we don't expect memory corruption, what are we testing for?
>>
>> This is a very good question. SHA-256 is quite a heavy hammer if all
>> you need is CRC style error detection.

Thanks for the history links.

We don't (yet) support KEXEC_FILE or KEXEC_VERIFY_SIG, and arm64 doesn't have an
in-kernel purgatory (which looks to be required for kexec_file under secure-boot).


> AFAICR the sha-256 implementation was proposed to boot a signed
> kexec/kdump kernel to circumvent kexec from violating UEFI secure boot
> restrictions (see [1]).

The beginning of the kexec-tools git history is 'kexec-tools-1.101' in 2006, it
had util_lib/sha256.c. It looks like SecureBoot arrived in 2011 with v2.3.1 of UEFI.
I can see how x86 picked up on this checksum for secure-boot as kexec-tools
already did this work, (some of the files under arch/x86/purgatory note their
kexec-tools origin), my question is why did it do it in the first place?
If the reason is accidental writes, we mitigate this on arm64 by unmapping the
kdump region instead of just marking it read-only.


> As Matthew Garret rightly noted (see[2]), secure Boot, if enabled, is
> explicitly designed to stop you booting modified kernels unless you've
> added your own keys.

> So, CRC wouldn't possibly fulfil the functionality we are trying to
> achieve with SHA-256 in the purgatory.

Is this still true for a purgatory provided by user-space?


Thanks,

James



From maheshvenkateshwaran at gmail.com  Sat Jun  3 06:41:31 2017
From: maheshvenkateshwaran at gmail.com (Mahesh V)
Date: Sat, 3 Jun 2017 19:11:31 +0530
Subject: crash kernel not booting
Message-ID: <CAEwzPJ0GZuzPXGLB2NcZt=JQq3+BQFeAeD-VxX280rY+wEQHbQ@mail.gmail.com>

Hi folks,

Have mostly exhausted all my options to make my ARM crash kernel boot :(.
Nothings worked so far. Any pointers as to how should I proceed further?

Following are my .config parameters that I set.
+CONFIG_KEXEC=y
+CONFIG_CRASH_DUMP=y
+CONFIG_DEBUG_LL=y
+CONFIG_DEBUG_LL_UART_NONE=y
+CONFIG_EARLY_PRINTK=y

defaults. If I set CONFIG_ARM_PATCH_PHYS_VIRT, I get a compile error
('__pv_phys_offset' undeclared here (not in a function))

CONFIG_PHYS_OFFSET=0x41500000
# CONFIG_ARM_PATCH_PHYS_VIRT is not set
# CONFIG_AUTO_ZRELADDR is not set


I am trying to load the uncompressed vmlinux (not uImage or zImage) to
0x44800000 location in my RAM (crash kernel location)

iomem has the following entry
44800000-5fbfffff : System RAM
  44800000-4ddfffff : Crash kernel


I used the following kexec command to load the kernel to that location
/aruba/bin/kexec -d -p /aruba/bin/vmlinux
--command-line="console=ttyHSL1,9600n8 rdinit=/sbin/init ubi.mtd=aos0
ubi.mtd=aos1 ubi.mtd=ubifs crashkernel=150m at 1096m earlyprintk noisy"
--atags --mem-min=0x44800000


The crash kernel vmlinux does load at that location. Verified using dd
command (44808000 == 1149272064)
/aruba/bin # dd if=/dev/mem skip=1149272064 bs=1 count=20  | od -x
20+0000000     457f    464c    0101    0001    0000    0000    0000    0000
0 records in
20+00000020     0002    0028
 records out
0000024


However when I object dump the vmlinux, it has addresses pointing to c0000000
Is this the reason why my crash kernel never boots (because the
locations in  vmlinux is c0000000 and not 0x44800000 which is my start
address)

these are the last few lines after crashing the main kernel

[ 7152.560043] Code: e3a02001 e5832000 f57ff04f e3a03000 (e5c32000)
[ 7152.634583] Loading crashdump kernel...
[ 7152.678819] image start ---- image->start 44808000
[ 7152.736082]  kexec start address 44808000,
[ 7152.786098]  reboot_code_buffer 0xc3300000,
[ 7152.837144]  reboot_code_buffer_phys 44800000,
[ 7152.891315]  kexec_mach_type 4704
[ 7152.931927] Bye!
[ 7152.953920] Inside __soft_restart !! c037fb68 4187fb68
[ 7153.016338] __soft_restart final address 44800000

code snippet from process.c in static void __soft_restart(void *addr) function
        phys_reset = (phys_reset_t)(unsigned long)virt_to_phys(cpu_reset);
        printk(KERN_EMERG "__soft_restart final address %p \n",
(unsigned char *)addr);

        phys_reset((unsigned long)addr);
        /* Should never get here. */
        BUG();


After cpu_reset I do not see any prints or kernel messages. How do I
debug further ?
Any pointers would help


objdump of my vmlinux

[mv_iap43_2 linux-3.4.0 (IAP-OCTOMORE)]$
../../../../aos-cmn/tools/armv7-openwrt-linux-gnu-uClibc-0.9.33.2/bin/arm-openwrt-linux-objdump
-d vmlinux | less

vmlinux:     file format elf32-littlearm


Disassembly of section .head.text:

c0008000 <stext>:
c0008000:       e321f0d3        msr     CPSR_c, #211    ; 0xd3
c0008004:       ee109f10        mrc     15, 0, r9, cr0, cr0, {0}
c0008008:       eb0daf8c        bl      c0373e40 <__lookup_processor_type>
c000800c:       e1b0a005        movs    sl, r5
c0008010:       e28f302c        add     r3, pc, #44     ; 0x2c
c0008014:       e8930110        ldm     r3, {r4, r8}
c0008018:       e0434004        sub     r4, r3, r4
c000801c:       e0888004        add     r8, r8, r4
c0008020:       eb000009        bl      c000804c <__vet_atags>
c0008024:       eb1fdff5        bl      c0800000 <__init_begin>
c0008028:       eb000159        bl      c0008594 <__create_page_tables>
c000802c:       e59fd00c        ldr     sp, [pc, #12]   ; c0008040 <stext+0x40>
c0008030:       e28fe004        add     lr, pc, #4
c0008034:       e1a08004        mov     r8, r4
c0008038:       e28af010        add     pc, sl, #16
c000803c:       ea0daf74        b       c0373e14 <__enable_mmu>
c0008040:       c080005c        .word   0xc080005c
c0008044:       c0008044        .word   0xc0008044
c0008048:       c0000000        .word   0xc0000000


 Thanks in advance
regards,
Mahesh


From piliu at redhat.com  Sun Jun  4 23:17:58 2017
From: piliu at redhat.com (Pingfan Liu)
Date: Mon,  5 Jun 2017 14:17:58 +0800
Subject: [makedumpfile PATCH 1/2] fold the calc of time delta into a func
Message-ID: <1496643479-28351-1-git-send-email-piliu@redhat.com>

This piece of code will be used by more than one function after
introducing the next patch

Signed-off-by: Pingfan Liu <piliu at redhat.com>
---
 print_info.c | 28 ++++++++++++++++------------
 1 file changed, 16 insertions(+), 12 deletions(-)

diff --git a/print_info.c b/print_info.c
index 832196d..79dfe5e 100644
--- a/print_info.c
+++ b/print_info.c
@@ -337,6 +337,19 @@ print_usage(void)
 	MSG("\n");
 }
 
+static void calc_delta(struct timeval *tv_start, struct timeval *delta)
+{
+	struct timeval tv_end;
+
+	gettimeofday(&tv_end, NULL);
+	delta->tv_sec = tv_end.tv_sec - tv_start->tv_sec;
+	delta->tv_usec = tv_end.tv_usec - tv_start->tv_usec;
+	if (delta->tv_usec < 0) {
+		delta->tv_sec--;
+		delta->tv_usec += 1000000;
+	}
+}
+
 void
 print_progress(const char *msg, unsigned long current, unsigned long end)
 {
@@ -369,19 +382,10 @@ print_progress(const char *msg, unsigned long current, unsigned long end)
 void
 print_execution_time(char *step_name, struct timeval *tv_start)
 {
-	struct timeval tv_end;
-	time_t diff_sec;
-	suseconds_t diff_usec;
+	struct timeval delta;
 
-	gettimeofday(&tv_end, NULL);
-
-	diff_sec  = tv_end.tv_sec - tv_start->tv_sec;
-	diff_usec = tv_end.tv_usec - tv_start->tv_usec;
-	if (diff_usec < 0) {
-		diff_sec--;
-		diff_usec += 1000000;
-	}
+	calc_delta(tv_start, &delta);
 	REPORT_MSG("STEP [%s] : %ld.%06ld seconds\n",
-		   step_name, diff_sec, diff_usec);
+		   step_name, delta.tv_sec, delta.tv_usec);
 }
 
-- 
2.7.4



From piliu at redhat.com  Sun Jun  4 23:17:59 2017
From: piliu at redhat.com (Pingfan Liu)
Date: Mon,  5 Jun 2017 14:17:59 +0800
Subject: [makedumpfile PATCH 2/2] print_info: show the remaining time of dump
 progress
In-Reply-To: <1496643479-28351-1-git-send-email-piliu@redhat.com>
References: <1496643479-28351-1-git-send-email-piliu@redhat.com>
Message-ID: <1496643479-28351-2-git-send-email-piliu@redhat.com>

If the dump progress is slow, it is better to show  an estimate of
remaining time of the progress to the user. The estimator simply uses
the average speed to work.

Signed-off-by: Pingfan Liu <piliu at redhat.com>
---
 makedumpfile.c | 46 +++++++++++++++++++++++-----------------------
 print_info.c   | 38 ++++++++++++++++++++++++++++++++------
 print_info.h   |  3 ++-
 3 files changed, 57 insertions(+), 30 deletions(-)

diff --git a/makedumpfile.c b/makedumpfile.c
index 301772a..35a36ce 100644
--- a/makedumpfile.c
+++ b/makedumpfile.c
@@ -5175,7 +5175,7 @@ _exclude_free_page(struct cycle *cycle)
 
 	for (num_nodes = 1; num_nodes <= vt.numnodes; num_nodes++) {
 
-		print_progress(PROGRESS_FREE_PAGES, num_nodes - 1, vt.numnodes);
+		print_progress(PROGRESS_FREE_PAGES, num_nodes - 1, vt.numnodes, NULL);
 
 		node_zones = pgdat + OFFSET(pglist_data.node_zones);
 
@@ -5188,7 +5188,7 @@ _exclude_free_page(struct cycle *cycle)
 		for (i = 0; i < nr_zones; i++) {
 
 			print_progress(PROGRESS_FREE_PAGES, i + nr_zones * (num_nodes - 1),
-					nr_zones * vt.numnodes);
+					nr_zones * vt.numnodes, NULL);
 
 			zone = node_zones + (i * SIZE(zone));
 			if (!readmem(VADDR, zone + OFFSET(zone.spanned_pages),
@@ -5216,7 +5216,7 @@ _exclude_free_page(struct cycle *cycle)
 	/*
 	 * print [100 %]
 	 */
-	print_progress(PROGRESS_FREE_PAGES, vt.numnodes, vt.numnodes);
+	print_progress(PROGRESS_FREE_PAGES, vt.numnodes, vt.numnodes, NULL);
 	print_execution_time(PROGRESS_FREE_PAGES, &tv_start);
 
 	return TRUE;
@@ -5403,7 +5403,7 @@ create_1st_bitmap_file(void)
 	for (i = 0; get_pt_load(i, &phys_start, &phys_end, NULL, NULL); i++) {
 
 		if (!info->flag_mem_usage)
-			print_progress(PROGRESS_HOLES, i, num_pt_loads);
+			print_progress(PROGRESS_HOLES, i, num_pt_loads, NULL);
 
 		pfn_start = paddr_to_pfn(phys_start);
 		pfn_end   = paddr_to_pfn(phys_end);
@@ -5422,7 +5422,7 @@ create_1st_bitmap_file(void)
 	 * print 100 %
 	 */
 	if (!info->flag_mem_usage) {
-		print_progress(PROGRESS_HOLES, info->max_mapnr, info->max_mapnr);
+		print_progress(PROGRESS_HOLES, info->max_mapnr, info->max_mapnr, NULL);
 		print_execution_time(PROGRESS_HOLES, &tv_start);
 	}
 
@@ -5548,7 +5548,7 @@ create_bitmap_from_memhole(struct cycle *cycle, struct dump_bitmap *bitmap, int
 		pfn_start = MAX(paddr_to_pfn(phys_start), cycle->start_pfn);
 		pfn_end = MIN(paddr_to_pfn(phys_end), cycle->end_pfn);
 
-		print_progress(PROGRESS_HOLES, i, num_pt_loads);
+		print_progress(PROGRESS_HOLES, i, num_pt_loads, NULL);
 
 		if (pfn_start >= pfn_end)
 			continue;
@@ -5587,7 +5587,7 @@ create_bitmap_from_memhole(struct cycle *cycle, struct dump_bitmap *bitmap, int
 	/*
 	 * print 100 %
 	 */
-	print_progress(PROGRESS_HOLES, info->max_mapnr, info->max_mapnr);
+	print_progress(PROGRESS_HOLES, info->max_mapnr, info->max_mapnr, NULL);
 	print_execution_time(PROGRESS_HOLES, &tv_start);
 
 	return TRUE;
@@ -5842,7 +5842,7 @@ exclude_unnecessary_pages(struct cycle *cycle)
 	for (mm = 0; mm < info->num_mem_map; mm++) {
 
 		if (!info->flag_mem_usage)
-			print_progress(PROGRESS_UNN_PAGES, mm, info->num_mem_map);
+			print_progress(PROGRESS_UNN_PAGES, mm, info->num_mem_map, NULL);
 
 		mmd = &info->mem_map_data[mm];
 
@@ -5860,7 +5860,7 @@ exclude_unnecessary_pages(struct cycle *cycle)
 	 * print [100 %]
 	 */
 	if (!info->flag_mem_usage) {
-		print_progress(PROGRESS_UNN_PAGES, info->num_mem_map, info->num_mem_map);
+		print_progress(PROGRESS_UNN_PAGES, info->num_mem_map, info->num_mem_map, NULL);
 		print_execution_time(PROGRESS_UNN_PAGES, &tv_start);
 	}
 
@@ -7334,7 +7334,7 @@ write_elf_pages_cyclic(struct cache_data *cd_header, struct cache_data *cd_page)
 				}
 
 				if ((num_dumped % per) == 0)
-					print_progress(PROGRESS_COPY, num_dumped, num_dumpable);
+					print_progress(PROGRESS_COPY, num_dumped, num_dumpable, &tv_start);
 
 				num_dumped++;
 
@@ -7453,7 +7453,7 @@ write_elf_pages_cyclic(struct cache_data *cd_header, struct cache_data *cd_page)
 	/*
 	 * print [100 %]
 	 */
-	print_progress(PROGRESS_COPY, num_dumpable, num_dumpable);
+	print_progress(PROGRESS_COPY, num_dumpable, num_dumpable, &tv_start);
 	print_execution_time(PROGRESS_COPY, &tv_start);
 	PROGRESS_MSG("\n");
 
@@ -7931,7 +7931,7 @@ write_kdump_pages_parallel_cyclic(struct cache_data *cd_header,
 		}
 
 		if ((num_dumped % per) == 0)
-			print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable);
+			print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable, &tv_start);
 
 		num_dumped++;
 
@@ -7967,7 +7967,7 @@ finish:
 	/*
 	 * print [100 %]
 	 */
-	print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable);
+	print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable, &tv_start);
 	print_execution_time(PROGRESS_COPY, &tv_start);
 	PROGRESS_MSG("\n");
 
@@ -8084,7 +8084,7 @@ write_kdump_pages_cyclic(struct cache_data *cd_header, struct cache_data *cd_pag
 	for (pfn = start_pfn; pfn < end_pfn; pfn++) {
 
 		if ((num_dumped % per) == 0)
-			print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable);
+			print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable, &tv_start);
 
 		/*
 		 * Check the excluded page.
@@ -8164,7 +8164,7 @@ out:
 		free(wrkmem);
 #endif
 
-	print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable);
+	print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable, &tv_start);
 	print_execution_time(PROGRESS_COPY, &tv_start);
 
 	return ret;
@@ -8622,7 +8622,7 @@ write_kdump_pages_and_bitmap_cyclic(struct cache_data *cd_header, struct cache_d
 	/*
 	 * print [100 %]
 	 */
-	print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable);
+	print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable, &tv_start);
 	print_execution_time(PROGRESS_COPY, &tv_start);
 	PROGRESS_MSG("\n");
 
@@ -9216,7 +9216,7 @@ exclude_xen3_user_domain(void)
 	 */
 	for (i = 0; get_pt_load(i, &phys_start, &phys_end, NULL, NULL); i++) {
 
-		print_progress(PROGRESS_XEN_DOMAIN, i, num_pt_loads);
+		print_progress(PROGRESS_XEN_DOMAIN, i, num_pt_loads, NULL);
 
 		pfn     = paddr_to_pfn(phys_start);
 		pfn_end = paddr_to_pfn(phys_end);
@@ -9224,7 +9224,7 @@ exclude_xen3_user_domain(void)
 
 		for (j = 0; pfn < pfn_end; pfn++, j++) {
 			print_progress(PROGRESS_XEN_DOMAIN, j + (size * i),
-					size * num_pt_loads);
+					size * num_pt_loads, NULL);
 
 			if (!allocated_in_map(pfn)) {
 				clear_bit_on_2nd_bitmap(pfn, NULL);
@@ -9280,7 +9280,7 @@ exclude_xen4_user_domain(void)
 	 */
 	for (i = 0; get_pt_load(i, &phys_start, &phys_end, NULL, NULL); i++) {
 
-		print_progress(PROGRESS_XEN_DOMAIN, i, num_pt_loads);
+		print_progress(PROGRESS_XEN_DOMAIN, i, num_pt_loads, NULL);
 
 		pfn     = paddr_to_pfn(phys_start);
 		pfn_end = paddr_to_pfn(phys_end);
@@ -9288,7 +9288,7 @@ exclude_xen4_user_domain(void)
 
 		for (j = 0; pfn < pfn_end; pfn++, j++) {
 			print_progress(PROGRESS_XEN_DOMAIN, j + (size * i),
-					size * num_pt_loads);
+					size * num_pt_loads, NULL);
 
 			page_info_addr = info->frame_table_vaddr + pfn * SIZE(page_info);
 			if (!readmem(VADDR_XEN,
@@ -9356,7 +9356,7 @@ exclude_xen_user_domain(void)
 	/*
 	 * print [100 %]
 	 */
-	print_progress(PROGRESS_XEN_DOMAIN, 1, 1);
+	print_progress(PROGRESS_XEN_DOMAIN, 1, 1, NULL);
 	print_execution_time(PROGRESS_XEN_DOMAIN, &tv_start);
 
 	return ret;
@@ -10342,7 +10342,7 @@ reassemble_kdump_pages(void)
 
 			num_dumped++;
 
-			print_progress(PROGRESS_COPY, num_dumped, num_dumpable);
+			print_progress(PROGRESS_COPY, num_dumped, num_dumpable, &tv_start);
 
 			if (lseek(fd, offset_ph_org, SEEK_SET) < 0) {
 				ERRMSG("Can't seek a file(%s). %s\n",
@@ -10439,7 +10439,7 @@ reassemble_kdump_pages(void)
 						    size_eraseinfo))
 			goto out;
 	}
-	print_progress(PROGRESS_COPY, num_dumpable, num_dumpable);
+	print_progress(PROGRESS_COPY, num_dumpable, num_dumpable, &tv_start);
 	print_execution_time(PROGRESS_COPY, &tv_start);
 
 	ret = TRUE;
diff --git a/print_info.c b/print_info.c
index 79dfe5e..10a3548 100644
--- a/print_info.c
+++ b/print_info.c
@@ -16,7 +16,7 @@
 #include "print_info.h"
 #include <time.h>
 
-#define PROGRESS_MAXLEN		"35"
+#define PROGRESS_MAXLEN		"55"
 
 int message_level;
 int flag_strerr_message;
@@ -350,14 +350,34 @@ static void calc_delta(struct timeval *tv_start, struct timeval *delta)
 	}
 }
 
+/* produce less than 11 bytes on msg */
+static int eta_to_human_short (int secs, char* msg)
+{
+	if (secs < 100)
+		sprintf(msg, "%ds", secs);
+	else if (secs < 100 * 60)
+		sprintf(msg, "%dm%ds", secs / 60, secs % 60);
+	else if (secs < 48 * 3600)
+		sprintf(msg, "%dh%dm", secs / 3600, (secs / 60) % 60);
+	else if (secs < 100 * 86400)
+		sprintf(msg, "%dd%dh", secs / 86400, (secs / 3600) % 24);
+	else
+		sprintf(msg, ">2day");
+	return 0;
+}
+
+
 void
-print_progress(const char *msg, unsigned long current, unsigned long end)
+print_progress(const char *msg, unsigned long current, unsigned long end, struct timeval *start)
 {
 	float progress;
 	time_t tm;
 	static time_t last_time = 0;
 	static unsigned int lapse = 0;
 	static const char *spinner = "/|\\-";
+	struct timeval delta;
+	double eta;
+	char eta_msg[32] = "eta: ";
 
 	if (current < end) {
 		tm = time(NULL);
@@ -368,13 +388,19 @@ print_progress(const char *msg, unsigned long current, unsigned long end)
 	} else
 		progress = 100;
 
+	if (start != NULL) {
+		calc_delta(start, &delta);
+		eta = delta.tv_sec + delta.tv_usec/1e6;
+		eta = (100 - progress) * eta/progress;
+		eta_to_human_short(eta, eta_msg+5);
+	}
 	if (flag_ignore_r_char) {
-		PROGRESS_MSG("%-" PROGRESS_MAXLEN "s: [%5.1f %%] %c\n",
-			     msg, progress, spinner[lapse % 4]);
+		PROGRESS_MSG("%-" PROGRESS_MAXLEN "s: [%5.1f %%] %c  %16s\n",
+			     msg, progress, spinner[lapse % 4], eta_msg);
 	} else {
 		PROGRESS_MSG("\r");
-		PROGRESS_MSG("%-" PROGRESS_MAXLEN "s: [%5.1f %%] %c",
-			     msg, progress, spinner[lapse % 4]);
+		PROGRESS_MSG("%-" PROGRESS_MAXLEN "s: [%5.1f %%] %c  %16s",
+			     msg, progress, spinner[lapse % 4], eta_msg);
 	}
 	lapse++;
 }
diff --git a/print_info.h b/print_info.h
index 01e3706..1ce3593 100644
--- a/print_info.h
+++ b/print_info.h
@@ -25,7 +25,8 @@ extern int flag_ignore_r_char;
 
 void show_version(void);
 void print_usage(void);
-void print_progress(const char *msg, unsigned long current, unsigned long end);
+void print_progress(const char *msg, unsigned long current, unsigned long end, struct timeval *start);
+
 void print_execution_time(char *step_name, struct timeval *tv_start);
 
 /*
-- 
2.7.4



From ats-kumagai at wm.jp.nec.com  Wed Jun  7 00:06:07 2017
From: ats-kumagai at wm.jp.nec.com (Atsushi Kumagai)
Date: Wed, 7 Jun 2017 07:06:07 +0000
Subject: [makedumpfile PATCH 2/2] print_info: show the remaining time of
 dump progress
In-Reply-To: <1496643479-28351-2-git-send-email-piliu@redhat.com>
References: <1496643479-28351-1-git-send-email-piliu@redhat.com>
 <1496643479-28351-2-git-send-email-piliu@redhat.com>
Message-ID: <0910DD04CBD6DE4193FCF86B9C00BE9701EDE6B5@BPXM01GP.gisp.nec.co.jp>

Hello Pingfan,

>If the dump progress is slow, it is better to show  an estimate of
>remaining time of the progress to the user. The estimator simply uses
>the average speed to work.

I like it, thanks for your work.
I have a minor comment.

[snip]
> void
>-print_progress(const char *msg, unsigned long current, unsigned long end)
>+print_progress(const char *msg, unsigned long current, unsigned long end, struct timeval *start)
> {
> 	float progress;
> 	time_t tm;
> 	static time_t last_time = 0;
> 	static unsigned int lapse = 0;
> 	static const char *spinner = "/|\\-";
>+	struct timeval delta;
>+	double eta;
>+	char eta_msg[32] = "eta: ";

I thinks this "eta: " should be set in eta_to_human_short(), because...

> 	if (current < end) {
> 		tm = time(NULL);
>@@ -368,13 +388,19 @@ print_progress(const char *msg, unsigned long current, unsigned long end)
> 	} else
> 		progress = 100;
>
>+	if (start != NULL) {
>+		calc_delta(start, &delta);
>+		eta = delta.tv_sec + delta.tv_usec/1e6;
>+		eta = (100 - progress) * eta/progress;
>+		eta_to_human_short(eta, eta_msg+5);

the offset for "eta: " is here as a magic number.
It has low maintainability since the number is independent of
the actual string length.


Thanks,
Atsushi Kumagai

>+	}
> 	if (flag_ignore_r_char) {
>-		PROGRESS_MSG("%-" PROGRESS_MAXLEN "s: [%5.1f %%] %c\n",
>-			     msg, progress, spinner[lapse % 4]);
>+		PROGRESS_MSG("%-" PROGRESS_MAXLEN "s: [%5.1f %%] %c  %16s\n",
>+			     msg, progress, spinner[lapse % 4], eta_msg);
> 	} else {
> 		PROGRESS_MSG("\r");
>-		PROGRESS_MSG("%-" PROGRESS_MAXLEN "s: [%5.1f %%] %c",
>-			     msg, progress, spinner[lapse % 4]);
>+		PROGRESS_MSG("%-" PROGRESS_MAXLEN "s: [%5.1f %%] %c  %16s",
>+			     msg, progress, spinner[lapse % 4], eta_msg);
> 	}
> 	lapse++;
> }
>diff --git a/print_info.h b/print_info.h
>index 01e3706..1ce3593 100644
>--- a/print_info.h
>+++ b/print_info.h
>@@ -25,7 +25,8 @@ extern int flag_ignore_r_char;
>
> void show_version(void);
> void print_usage(void);
>-void print_progress(const char *msg, unsigned long current, unsigned long end);
>+void print_progress(const char *msg, unsigned long current, unsigned long end, struct timeval *start);
>+
> void print_execution_time(char *step_name, struct timeval *tv_start);
>
> /*
>--
>2.7.4
>
>
>_______________________________________________
>kexec mailing list
>kexec at lists.infradead.org
>http://lists.infradead.org/mailman/listinfo/kexec



From piliu at redhat.com  Wed Jun  7 00:19:05 2017
From: piliu at redhat.com (Pingfan Liu)
Date: Wed, 7 Jun 2017 03:19:05 -0400 (EDT)
Subject: [makedumpfile PATCH 2/2] print_info: show the remaining time of
 dump progress
In-Reply-To: <0910DD04CBD6DE4193FCF86B9C00BE9701EDE6B5@BPXM01GP.gisp.nec.co.jp>
References: <1496643479-28351-1-git-send-email-piliu@redhat.com>
 <1496643479-28351-2-git-send-email-piliu@redhat.com>
 <0910DD04CBD6DE4193FCF86B9C00BE9701EDE6B5@BPXM01GP.gisp.nec.co.jp>
Message-ID: <1962253196.19841266.1496819945660.JavaMail.zimbra@redhat.com>





----- Original Message -----
> From: "Atsushi Kumagai" <ats-kumagai at wm.jp.nec.com>
> To: "Pingfan Liu" <piliu at redhat.com>
> Cc: kexec at lists.infradead.org
> Sent: Wednesday, June 7, 2017 3:06:07 PM
> Subject: RE: [makedumpfile PATCH 2/2] print_info: show the remaining time of dump progress
> 
> Hello Pingfan,
> 
> >If the dump progress is slow, it is better to show  an estimate of
> >remaining time of the progress to the user. The estimator simply uses
> >the average speed to work.
> 
> I like it, thanks for your work.
> I have a minor comment.
> 
> [snip]
> > void
> >-print_progress(const char *msg, unsigned long current, unsigned long end)
> >+print_progress(const char *msg, unsigned long current, unsigned long end,
> >struct timeval *start)
> > {
> > 	float progress;
> > 	time_t tm;
> > 	static time_t last_time = 0;
> > 	static unsigned int lapse = 0;
> > 	static const char *spinner = "/|\\-";
> >+	struct timeval delta;
> >+	double eta;
> >+	char eta_msg[32] = "eta: ";
> 
> I thinks this "eta: " should be set in eta_to_human_short(), because...
> 
> > 	if (current < end) {
> > 		tm = time(NULL);
> >@@ -368,13 +388,19 @@ print_progress(const char *msg, unsigned long current,
> >unsigned long end)
> > 	} else
> > 		progress = 100;
> >
> >+	if (start != NULL) {
> >+		calc_delta(start, &delta);
> >+		eta = delta.tv_sec + delta.tv_usec/1e6;
> >+		eta = (100 - progress) * eta/progress;
> >+		eta_to_human_short(eta, eta_msg+5);
> 
> the offset for "eta: " is here as a magic number.
> It has low maintainability since the number is independent of
> the actual string length.
> 
> 
Thanks for your review. I will modify according to your comment.


Regards,
Pingfan



From piliu at redhat.com  Wed Jun  7 01:09:49 2017
From: piliu at redhat.com (Pingfan Liu)
Date: Wed,  7 Jun 2017 16:09:49 +0800
Subject: [makedumpfile PATCHv2 1/2] fold the calc of time delta into a func
Message-ID: <1496822990-17848-1-git-send-email-piliu@redhat.com>

This piece of code will be used by more than one function after
introducing the next patch

Signed-off-by: Pingfan Liu <piliu at redhat.com>
---
No changes from v1

 print_info.c | 28 ++++++++++++++++------------
 1 file changed, 16 insertions(+), 12 deletions(-)

diff --git a/print_info.c b/print_info.c
index 832196d..79dfe5e 100644
--- a/print_info.c
+++ b/print_info.c
@@ -337,6 +337,19 @@ print_usage(void)
 	MSG("\n");
 }
 
+static void calc_delta(struct timeval *tv_start, struct timeval *delta)
+{
+	struct timeval tv_end;
+
+	gettimeofday(&tv_end, NULL);
+	delta->tv_sec = tv_end.tv_sec - tv_start->tv_sec;
+	delta->tv_usec = tv_end.tv_usec - tv_start->tv_usec;
+	if (delta->tv_usec < 0) {
+		delta->tv_sec--;
+		delta->tv_usec += 1000000;
+	}
+}
+
 void
 print_progress(const char *msg, unsigned long current, unsigned long end)
 {
@@ -369,19 +382,10 @@ print_progress(const char *msg, unsigned long current, unsigned long end)
 void
 print_execution_time(char *step_name, struct timeval *tv_start)
 {
-	struct timeval tv_end;
-	time_t diff_sec;
-	suseconds_t diff_usec;
+	struct timeval delta;
 
-	gettimeofday(&tv_end, NULL);
-
-	diff_sec  = tv_end.tv_sec - tv_start->tv_sec;
-	diff_usec = tv_end.tv_usec - tv_start->tv_usec;
-	if (diff_usec < 0) {
-		diff_sec--;
-		diff_usec += 1000000;
-	}
+	calc_delta(tv_start, &delta);
 	REPORT_MSG("STEP [%s] : %ld.%06ld seconds\n",
-		   step_name, diff_sec, diff_usec);
+		   step_name, delta.tv_sec, delta.tv_usec);
 }
 
-- 
2.7.4



From piliu at redhat.com  Wed Jun  7 01:09:50 2017
From: piliu at redhat.com (Pingfan Liu)
Date: Wed,  7 Jun 2017 16:09:50 +0800
Subject: [makedumpfile PATCHv2 2/2] print_info: show the remaining time of
 dump progress
In-Reply-To: <1496822990-17848-1-git-send-email-piliu@redhat.com>
References: <1496822990-17848-1-git-send-email-piliu@redhat.com>
Message-ID: <1496822990-17848-2-git-send-email-piliu@redhat.com>

If the dump progress is slow, it is better to show  an estimate of
remaining time of the progress to the user. The estimator simply uses
the average speed to work.

Signed-off-by: Pingfan Liu <piliu at redhat.com>
---
v1 -> v2: fold  eta string into eta_to_human_short()


 makedumpfile.c | 46 +++++++++++++++++++++++-----------------------
 print_info.c   | 41 +++++++++++++++++++++++++++++++++++------
 print_info.h   |  3 ++-
 3 files changed, 60 insertions(+), 30 deletions(-)

diff --git a/makedumpfile.c b/makedumpfile.c
index 301772a..35a36ce 100644
--- a/makedumpfile.c
+++ b/makedumpfile.c
@@ -5175,7 +5175,7 @@ _exclude_free_page(struct cycle *cycle)
 
 	for (num_nodes = 1; num_nodes <= vt.numnodes; num_nodes++) {
 
-		print_progress(PROGRESS_FREE_PAGES, num_nodes - 1, vt.numnodes);
+		print_progress(PROGRESS_FREE_PAGES, num_nodes - 1, vt.numnodes, NULL);
 
 		node_zones = pgdat + OFFSET(pglist_data.node_zones);
 
@@ -5188,7 +5188,7 @@ _exclude_free_page(struct cycle *cycle)
 		for (i = 0; i < nr_zones; i++) {
 
 			print_progress(PROGRESS_FREE_PAGES, i + nr_zones * (num_nodes - 1),
-					nr_zones * vt.numnodes);
+					nr_zones * vt.numnodes, NULL);
 
 			zone = node_zones + (i * SIZE(zone));
 			if (!readmem(VADDR, zone + OFFSET(zone.spanned_pages),
@@ -5216,7 +5216,7 @@ _exclude_free_page(struct cycle *cycle)
 	/*
 	 * print [100 %]
 	 */
-	print_progress(PROGRESS_FREE_PAGES, vt.numnodes, vt.numnodes);
+	print_progress(PROGRESS_FREE_PAGES, vt.numnodes, vt.numnodes, NULL);
 	print_execution_time(PROGRESS_FREE_PAGES, &tv_start);
 
 	return TRUE;
@@ -5403,7 +5403,7 @@ create_1st_bitmap_file(void)
 	for (i = 0; get_pt_load(i, &phys_start, &phys_end, NULL, NULL); i++) {
 
 		if (!info->flag_mem_usage)
-			print_progress(PROGRESS_HOLES, i, num_pt_loads);
+			print_progress(PROGRESS_HOLES, i, num_pt_loads, NULL);
 
 		pfn_start = paddr_to_pfn(phys_start);
 		pfn_end   = paddr_to_pfn(phys_end);
@@ -5422,7 +5422,7 @@ create_1st_bitmap_file(void)
 	 * print 100 %
 	 */
 	if (!info->flag_mem_usage) {
-		print_progress(PROGRESS_HOLES, info->max_mapnr, info->max_mapnr);
+		print_progress(PROGRESS_HOLES, info->max_mapnr, info->max_mapnr, NULL);
 		print_execution_time(PROGRESS_HOLES, &tv_start);
 	}
 
@@ -5548,7 +5548,7 @@ create_bitmap_from_memhole(struct cycle *cycle, struct dump_bitmap *bitmap, int
 		pfn_start = MAX(paddr_to_pfn(phys_start), cycle->start_pfn);
 		pfn_end = MIN(paddr_to_pfn(phys_end), cycle->end_pfn);
 
-		print_progress(PROGRESS_HOLES, i, num_pt_loads);
+		print_progress(PROGRESS_HOLES, i, num_pt_loads, NULL);
 
 		if (pfn_start >= pfn_end)
 			continue;
@@ -5587,7 +5587,7 @@ create_bitmap_from_memhole(struct cycle *cycle, struct dump_bitmap *bitmap, int
 	/*
 	 * print 100 %
 	 */
-	print_progress(PROGRESS_HOLES, info->max_mapnr, info->max_mapnr);
+	print_progress(PROGRESS_HOLES, info->max_mapnr, info->max_mapnr, NULL);
 	print_execution_time(PROGRESS_HOLES, &tv_start);
 
 	return TRUE;
@@ -5842,7 +5842,7 @@ exclude_unnecessary_pages(struct cycle *cycle)
 	for (mm = 0; mm < info->num_mem_map; mm++) {
 
 		if (!info->flag_mem_usage)
-			print_progress(PROGRESS_UNN_PAGES, mm, info->num_mem_map);
+			print_progress(PROGRESS_UNN_PAGES, mm, info->num_mem_map, NULL);
 
 		mmd = &info->mem_map_data[mm];
 
@@ -5860,7 +5860,7 @@ exclude_unnecessary_pages(struct cycle *cycle)
 	 * print [100 %]
 	 */
 	if (!info->flag_mem_usage) {
-		print_progress(PROGRESS_UNN_PAGES, info->num_mem_map, info->num_mem_map);
+		print_progress(PROGRESS_UNN_PAGES, info->num_mem_map, info->num_mem_map, NULL);
 		print_execution_time(PROGRESS_UNN_PAGES, &tv_start);
 	}
 
@@ -7334,7 +7334,7 @@ write_elf_pages_cyclic(struct cache_data *cd_header, struct cache_data *cd_page)
 				}
 
 				if ((num_dumped % per) == 0)
-					print_progress(PROGRESS_COPY, num_dumped, num_dumpable);
+					print_progress(PROGRESS_COPY, num_dumped, num_dumpable, &tv_start);
 
 				num_dumped++;
 
@@ -7453,7 +7453,7 @@ write_elf_pages_cyclic(struct cache_data *cd_header, struct cache_data *cd_page)
 	/*
 	 * print [100 %]
 	 */
-	print_progress(PROGRESS_COPY, num_dumpable, num_dumpable);
+	print_progress(PROGRESS_COPY, num_dumpable, num_dumpable, &tv_start);
 	print_execution_time(PROGRESS_COPY, &tv_start);
 	PROGRESS_MSG("\n");
 
@@ -7931,7 +7931,7 @@ write_kdump_pages_parallel_cyclic(struct cache_data *cd_header,
 		}
 
 		if ((num_dumped % per) == 0)
-			print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable);
+			print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable, &tv_start);
 
 		num_dumped++;
 
@@ -7967,7 +7967,7 @@ finish:
 	/*
 	 * print [100 %]
 	 */
-	print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable);
+	print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable, &tv_start);
 	print_execution_time(PROGRESS_COPY, &tv_start);
 	PROGRESS_MSG("\n");
 
@@ -8084,7 +8084,7 @@ write_kdump_pages_cyclic(struct cache_data *cd_header, struct cache_data *cd_pag
 	for (pfn = start_pfn; pfn < end_pfn; pfn++) {
 
 		if ((num_dumped % per) == 0)
-			print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable);
+			print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable, &tv_start);
 
 		/*
 		 * Check the excluded page.
@@ -8164,7 +8164,7 @@ out:
 		free(wrkmem);
 #endif
 
-	print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable);
+	print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable, &tv_start);
 	print_execution_time(PROGRESS_COPY, &tv_start);
 
 	return ret;
@@ -8622,7 +8622,7 @@ write_kdump_pages_and_bitmap_cyclic(struct cache_data *cd_header, struct cache_d
 	/*
 	 * print [100 %]
 	 */
-	print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable);
+	print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable, &tv_start);
 	print_execution_time(PROGRESS_COPY, &tv_start);
 	PROGRESS_MSG("\n");
 
@@ -9216,7 +9216,7 @@ exclude_xen3_user_domain(void)
 	 */
 	for (i = 0; get_pt_load(i, &phys_start, &phys_end, NULL, NULL); i++) {
 
-		print_progress(PROGRESS_XEN_DOMAIN, i, num_pt_loads);
+		print_progress(PROGRESS_XEN_DOMAIN, i, num_pt_loads, NULL);
 
 		pfn     = paddr_to_pfn(phys_start);
 		pfn_end = paddr_to_pfn(phys_end);
@@ -9224,7 +9224,7 @@ exclude_xen3_user_domain(void)
 
 		for (j = 0; pfn < pfn_end; pfn++, j++) {
 			print_progress(PROGRESS_XEN_DOMAIN, j + (size * i),
-					size * num_pt_loads);
+					size * num_pt_loads, NULL);
 
 			if (!allocated_in_map(pfn)) {
 				clear_bit_on_2nd_bitmap(pfn, NULL);
@@ -9280,7 +9280,7 @@ exclude_xen4_user_domain(void)
 	 */
 	for (i = 0; get_pt_load(i, &phys_start, &phys_end, NULL, NULL); i++) {
 
-		print_progress(PROGRESS_XEN_DOMAIN, i, num_pt_loads);
+		print_progress(PROGRESS_XEN_DOMAIN, i, num_pt_loads, NULL);
 
 		pfn     = paddr_to_pfn(phys_start);
 		pfn_end = paddr_to_pfn(phys_end);
@@ -9288,7 +9288,7 @@ exclude_xen4_user_domain(void)
 
 		for (j = 0; pfn < pfn_end; pfn++, j++) {
 			print_progress(PROGRESS_XEN_DOMAIN, j + (size * i),
-					size * num_pt_loads);
+					size * num_pt_loads, NULL);
 
 			page_info_addr = info->frame_table_vaddr + pfn * SIZE(page_info);
 			if (!readmem(VADDR_XEN,
@@ -9356,7 +9356,7 @@ exclude_xen_user_domain(void)
 	/*
 	 * print [100 %]
 	 */
-	print_progress(PROGRESS_XEN_DOMAIN, 1, 1);
+	print_progress(PROGRESS_XEN_DOMAIN, 1, 1, NULL);
 	print_execution_time(PROGRESS_XEN_DOMAIN, &tv_start);
 
 	return ret;
@@ -10342,7 +10342,7 @@ reassemble_kdump_pages(void)
 
 			num_dumped++;
 
-			print_progress(PROGRESS_COPY, num_dumped, num_dumpable);
+			print_progress(PROGRESS_COPY, num_dumped, num_dumpable, &tv_start);
 
 			if (lseek(fd, offset_ph_org, SEEK_SET) < 0) {
 				ERRMSG("Can't seek a file(%s). %s\n",
@@ -10439,7 +10439,7 @@ reassemble_kdump_pages(void)
 						    size_eraseinfo))
 			goto out;
 	}
-	print_progress(PROGRESS_COPY, num_dumpable, num_dumpable);
+	print_progress(PROGRESS_COPY, num_dumpable, num_dumpable, &tv_start);
 	print_execution_time(PROGRESS_COPY, &tv_start);
 
 	ret = TRUE;
diff --git a/print_info.c b/print_info.c
index 79dfe5e..4e0bb5c 100644
--- a/print_info.c
+++ b/print_info.c
@@ -15,8 +15,9 @@
  */
 #include "print_info.h"
 #include <time.h>
+#include <string.h>
 
-#define PROGRESS_MAXLEN		"35"
+#define PROGRESS_MAXLEN		"50"
 
 int message_level;
 int flag_strerr_message;
@@ -350,14 +351,36 @@ static void calc_delta(struct timeval *tv_start, struct timeval *delta)
 	}
 }
 
+/* produce less than 12 bytes on msg */
+static int eta_to_human_short (int secs, char* msg)
+{
+	strcpy(msg, "eta: ");
+	msg += strlen("eta: ");
+	if (secs < 100)
+		sprintf(msg, "%ds", secs);
+	else if (secs < 100 * 60)
+		sprintf(msg, "%dm%ds", secs / 60, secs % 60);
+	else if (secs < 48 * 3600)
+		sprintf(msg, "%dh%dm", secs / 3600, (secs / 60) % 60);
+	else if (secs < 100 * 86400)
+		sprintf(msg, "%dd%dh", secs / 86400, (secs / 3600) % 24);
+	else
+		sprintf(msg, ">2day");
+	return 0;
+}
+
+
 void
-print_progress(const char *msg, unsigned long current, unsigned long end)
+print_progress(const char *msg, unsigned long current, unsigned long end, struct timeval *start)
 {
 	float progress;
 	time_t tm;
 	static time_t last_time = 0;
 	static unsigned int lapse = 0;
 	static const char *spinner = "/|\\-";
+	struct timeval delta;
+	double eta;
+	char eta_msg[16] = " ";
 
 	if (current < end) {
 		tm = time(NULL);
@@ -368,13 +391,19 @@ print_progress(const char *msg, unsigned long current, unsigned long end)
 	} else
 		progress = 100;
 
+	if (start != NULL) {
+		calc_delta(start, &delta);
+		eta = delta.tv_sec + delta.tv_usec/1e6;
+		eta = (100 - progress) * eta/progress;
+		eta_to_human_short(eta, eta_msg);
+	}
 	if (flag_ignore_r_char) {
-		PROGRESS_MSG("%-" PROGRESS_MAXLEN "s: [%5.1f %%] %c\n",
-			     msg, progress, spinner[lapse % 4]);
+		PROGRESS_MSG("%-" PROGRESS_MAXLEN "s: [%5.1f %%] %c  %16s\n",
+			     msg, progress, spinner[lapse % 4], eta_msg);
 	} else {
 		PROGRESS_MSG("\r");
-		PROGRESS_MSG("%-" PROGRESS_MAXLEN "s: [%5.1f %%] %c",
-			     msg, progress, spinner[lapse % 4]);
+		PROGRESS_MSG("%-" PROGRESS_MAXLEN "s: [%5.1f %%] %c  %16s",
+			     msg, progress, spinner[lapse % 4], eta_msg);
 	}
 	lapse++;
 }
diff --git a/print_info.h b/print_info.h
index 01e3706..1ce3593 100644
--- a/print_info.h
+++ b/print_info.h
@@ -25,7 +25,8 @@ extern int flag_ignore_r_char;
 
 void show_version(void);
 void print_usage(void);
-void print_progress(const char *msg, unsigned long current, unsigned long end);
+void print_progress(const char *msg, unsigned long current, unsigned long end, struct timeval *start);
+
 void print_execution_time(char *step_name, struct timeval *tv_start);
 
 /*
-- 
2.7.4



From abhishek.shah at broadcom.com  Wed Jun  7 07:27:25 2017
From: abhishek.shah at broadcom.com (Abhishek Shah)
Date: Wed, 7 Jun 2017 19:57:25 +0530
Subject: makedumpfile: support for newer kernels [v4.9 onwards]
Message-ID: <CAKUFe6ajSLEH-RHf483oLRGF8Z1MXN0W-X8kDg7VHHjbus+fDg@mail.gmail.com>

Hi All,

I am using makedumpfile utility to compress the ramdump taken by kexec/kdump,
i.e. /proc/vmcore, on arm64 platform with kernel v4.12-rc3.

/proc/vmcore works fine with crash utility without passing it though
makedumpfile, but the file generated by makedumpfile is not getting
processed by crash utility; I am getting following error :

crash: seek error: kernel virtual address: ffff8008000c16a0  type:
"memory section"

I am using kernel v4.12-RC3; makedumpfile currently seems to support
kernel up to 4.8 as per documentation.
Is there plan to support newer kernel? can you provide some pointers
to modifications required for newer kernel support?


Regards,
Abhishek


From danielwa at cisco.com  Wed Jun  7 09:20:51 2017
From: danielwa at cisco.com (Daniel Walker)
Date: Wed, 7 Jun 2017 09:20:51 -0700
Subject: crash_kexec in oops_end() and panic()
Message-ID: <72f60925-c2a6-8811-a75a-5f443486e781@cisco.com>


Hi,

These two paths seem to be duplicating each other. We have an issue 
where we're using mtdoops to collect kernel logs on oops and panic, we 
also have a crash kernel (which also collects these logs). mtdoops saves 
logs differently for oops and panic, since oops isn't always fatal it 
schedules a write to the flash. Since panic() is always fatal is writes 
the logs immediately. In oops_end() the crash kernel runs immediately 
while still signaling an OOPS condition to mtdoops. Since mtdoops 
schedules a write to flash later, there is no later since the crash 
kernel runs immediately, we end up without getting the logs

I'm wondering what the significance is to have these two paths ? 
oops_end() could just call into panic() or a modified panic_with_regs() 
then we would collapse multiple paths. There is what I would call a hack 
in kexec_should_crash() which checks if there are 
crash_kexec_post_notifiers and it runs panic() if they exist. This 
wouldn't be needed if we always called panic() . I also wonder if there 
are other things in panic() which we should be running , but don't get 
run because of these two paths.


Any info appreciated.


Daniel



From ebiederm at xmission.com  Wed Jun  7 09:46:27 2017
From: ebiederm at xmission.com (Eric W. Biederman)
Date: Wed, 07 Jun 2017 11:46:27 -0500
Subject: crash_kexec in oops_end() and panic()
In-Reply-To: <72f60925-c2a6-8811-a75a-5f443486e781@cisco.com> (Daniel Walker's
 message of "Wed, 7 Jun 2017 09:20:51 -0700")
References: <72f60925-c2a6-8811-a75a-5f443486e781@cisco.com>
Message-ID: <87bmpzoie4.fsf@xmission.com>

Daniel Walker <danielwa at cisco.com> writes:

> Hi,
>
> These two paths seem to be duplicating each other. We have an issue
> where we're using mtdoops to collect kernel logs on oops and panic, we
> also have a crash kernel (which also collects these logs). mtdoops
> saves logs differently for oops and panic, since oops isn't always
> fatal it schedules a write to the flash. Since panic() is always fatal
> is writes the logs immediately. In oops_end() the crash kernel runs
> immediately while still signaling an OOPS condition to mtdoops. Since
> mtdoops schedules a write to flash later, there is no later since the
> crash kernel runs immediately, we end up without getting the logs
>
> I'm wondering what the significance is to have these two paths ?
> oops_end() could just call into panic() or a modified
> panic_with_regs() then we would collapse multiple paths. There is what
> I would call a hack in kexec_should_crash() which checks if there are
> crash_kexec_post_notifiers and it runs panic() if they exist. This
> wouldn't be needed if we always called panic() . I also wonder if
> there are other things in panic() which we should be running , but
> don't get run because of these two paths.

crash_kexec_post_notifiers is a horrible hack it is broken by design and
no one should use it.

Looking at the history and it still seems valid is the point of
kexec_should_crash is so that crash_kexec could be called with the
registers at the time of the crash.

The code that is run in kexec on panic path the less well it works.
This has been a known fact for years.

Please figure out how to depend on less code running in a broken
kernel.  Trying to figure out how to run more code is not the solution
to making the kernel reliable at the time of a crash.

Eric


From danielwa at cisco.com  Wed Jun  7 10:00:15 2017
From: danielwa at cisco.com (Daniel Walker)
Date: Wed, 7 Jun 2017 10:00:15 -0700
Subject: crash_kexec in oops_end() and panic()
In-Reply-To: <87bmpzoie4.fsf@xmission.com>
References: <72f60925-c2a6-8811-a75a-5f443486e781@cisco.com>
 <87bmpzoie4.fsf@xmission.com>
Message-ID: <f8c4c044-6b2d-d9ec-aff9-009e2a545440@cisco.com>

On 06/07/2017 09:46 AM, Eric W. Biederman wrote:
> Daniel Walker <danielwa at cisco.com> writes:
>
>> Hi,
>>
>> These two paths seem to be duplicating each other. We have an issue
>> where we're using mtdoops to collect kernel logs on oops and panic, we
>> also have a crash kernel (which also collects these logs). mtdoops
>> saves logs differently for oops and panic, since oops isn't always
>> fatal it schedules a write to the flash. Since panic() is always fatal
>> is writes the logs immediately. In oops_end() the crash kernel runs
>> immediately while still signaling an OOPS condition to mtdoops. Since
>> mtdoops schedules a write to flash later, there is no later since the
>> crash kernel runs immediately, we end up without getting the logs
>>
>> I'm wondering what the significance is to have these two paths ?
>> oops_end() could just call into panic() or a modified
>> panic_with_regs() then we would collapse multiple paths. There is what
>> I would call a hack in kexec_should_crash() which checks if there are
>> crash_kexec_post_notifiers and it runs panic() if they exist. This
>> wouldn't be needed if we always called panic() . I also wonder if
>> there are other things in panic() which we should be running , but
>> don't get run because of these two paths.
> crash_kexec_post_notifiers is a horrible hack it is broken by design and
> no one should use it.
>
> Looking at the history and it still seems valid is the point of
> kexec_should_crash is so that crash_kexec could be called with the
> registers at the time of the crash.

This is why I mention panic_with_regs() , it's not that hard to just 
send them into panic().

>
> The code that is run in kexec on panic path the less well it works.
> This has been a known fact for years.
>
> Please figure out how to depend on less code running in a broken
> kernel.  Trying to figure out how to run more code is not the solution
> to making the kernel reliable at the time of a crash.
>
> Eric


While this may be true, your cutting short an already existing panic() 
path which has been around and is well used. I would think if it's good 
enough for everyone else it should be good enough for the crash kernel path.

Daniel



From ebiederm at xmission.com  Wed Jun  7 10:11:08 2017
From: ebiederm at xmission.com (Eric W. Biederman)
Date: Wed, 07 Jun 2017 12:11:08 -0500
Subject: crash_kexec in oops_end() and panic()
In-Reply-To: <f8c4c044-6b2d-d9ec-aff9-009e2a545440@cisco.com> (Daniel Walker's
 message of "Wed, 7 Jun 2017 10:00:15 -0700")
References: <72f60925-c2a6-8811-a75a-5f443486e781@cisco.com>
 <87bmpzoie4.fsf@xmission.com>
 <f8c4c044-6b2d-d9ec-aff9-009e2a545440@cisco.com>
Message-ID: <87vao7lo43.fsf@xmission.com>

Daniel Walker <danielwa at cisco.com> writes:

> On 06/07/2017 09:46 AM, Eric W. Biederman wrote:
>> Daniel Walker <danielwa at cisco.com> writes:
>>
>>> Hi,
>>>
>>> These two paths seem to be duplicating each other. We have an issue
>>> where we're using mtdoops to collect kernel logs on oops and panic, we
>>> also have a crash kernel (which also collects these logs). mtdoops
>>> saves logs differently for oops and panic, since oops isn't always
>>> fatal it schedules a write to the flash. Since panic() is always fatal
>>> is writes the logs immediately. In oops_end() the crash kernel runs
>>> immediately while still signaling an OOPS condition to mtdoops. Since
>>> mtdoops schedules a write to flash later, there is no later since the
>>> crash kernel runs immediately, we end up without getting the logs
>>>
>>> I'm wondering what the significance is to have these two paths ?
>>> oops_end() could just call into panic() or a modified
>>> panic_with_regs() then we would collapse multiple paths. There is what
>>> I would call a hack in kexec_should_crash() which checks if there are
>>> crash_kexec_post_notifiers and it runs panic() if they exist. This
>>> wouldn't be needed if we always called panic() . I also wonder if
>>> there are other things in panic() which we should be running , but
>>> don't get run because of these two paths.
>> crash_kexec_post_notifiers is a horrible hack it is broken by design and
>> no one should use it.
>>
>> Looking at the history and it still seems valid is the point of
>> kexec_should_crash is so that crash_kexec could be called with the
>> registers at the time of the crash.
>
> This is why I mention panic_with_regs() , it's not that hard to just
> send them into panic().

Given your reasoning adding more useless code that is likely to
destabalize the code path I am not particularly interested.

>> The code that is run in kexec on panic path the less well it works.
>> This has been a known fact for years.
>>
>> Please figure out how to depend on less code running in a broken
>> kernel.  Trying to figure out how to run more code is not the solution
>> to making the kernel reliable at the time of a crash.
>>
>> Eric
>
>
> While this may be true, your cutting short an already existing panic()
> path which has been around and is well used. I would think if it's
> good enough for everyone else it should be good enough for the crash
> kernel path.

You are talking about decisions over a decade old at this point.  So
this isn't some new thing you are complaing about.  In fact mtdoops is
much newer.    So if it isn't working that is the code with the bug.

Furthermore we have lots of emperical evidence that doing lots of things
in the kernel at crash time doesn't work in practice.

Without kexec-on-panic there just isn't anything else the code can do,
so it makes sense to do a lot of flaky things.

I am not at all sympathetic with the notion of reorganizing the code so
that more code can run in the kexec on panic path and reduce it's
changes of working.  We have been there.  We have the scars.  That is
the reason why kexec is used at all.

There used to be a core dumper in the kernel it only worked in the lab
never in the field.

What I hear you asking for is to put a core dumper (or something as
liable to failure) back in the kernel without acknowledging any of the
reasons why that doesn't work in practice.  I am hearing you arguing for
making the code buggier.  I am hearing that and I am saying not interested.

Eric


From danielwa at cisco.com  Wed Jun  7 10:38:50 2017
From: danielwa at cisco.com (Daniel Walker)
Date: Wed, 7 Jun 2017 10:38:50 -0700
Subject: crash_kexec in oops_end() and panic()
In-Reply-To: <87vao7lo43.fsf@xmission.com>
References: <72f60925-c2a6-8811-a75a-5f443486e781@cisco.com>
 <87bmpzoie4.fsf@xmission.com>
 <f8c4c044-6b2d-d9ec-aff9-009e2a545440@cisco.com>
 <87vao7lo43.fsf@xmission.com>
Message-ID: <51265567-0f3b-4899-5976-8677736d36a8@cisco.com>

On 06/07/2017 10:11 AM, Eric W. Biederman wrote:
> Daniel Walker <danielwa at cisco.com> writes:
>
>> On 06/07/2017 09:46 AM, Eric W. Biederman wrote:
>>> Daniel Walker <danielwa at cisco.com> writes:
>>>
>>>> Hi,
>>>>
>>>> These two paths seem to be duplicating each other. We have an issue
>>>> where we're using mtdoops to collect kernel logs on oops and panic, we
>>>> also have a crash kernel (which also collects these logs). mtdoops
>>>> saves logs differently for oops and panic, since oops isn't always
>>>> fatal it schedules a write to the flash. Since panic() is always fatal
>>>> is writes the logs immediately. In oops_end() the crash kernel runs
>>>> immediately while still signaling an OOPS condition to mtdoops. Since
>>>> mtdoops schedules a write to flash later, there is no later since the
>>>> crash kernel runs immediately, we end up without getting the logs
>>>>
>>>> I'm wondering what the significance is to have these two paths ?
>>>> oops_end() could just call into panic() or a modified
>>>> panic_with_regs() then we would collapse multiple paths. There is what
>>>> I would call a hack in kexec_should_crash() which checks if there are
>>>> crash_kexec_post_notifiers and it runs panic() if they exist. This
>>>> wouldn't be needed if we always called panic() . I also wonder if
>>>> there are other things in panic() which we should be running , but
>>>> don't get run because of these two paths.
>>> crash_kexec_post_notifiers is a horrible hack it is broken by design and
>>> no one should use it.
>>>
>>> Looking at the history and it still seems valid is the point of
>>> kexec_should_crash is so that crash_kexec could be called with the
>>> registers at the time of the crash.
>> This is why I mention panic_with_regs() , it's not that hard to just
>> send them into panic().
> Given your reasoning adding more useless code that is likely to
> destabalize the code path I am not particularly interested.
>
>>> The code that is run in kexec on panic path the less well it works.
>>> This has been a known fact for years.
>>>
>>> Please figure out how to depend on less code running in a broken
>>> kernel.  Trying to figure out how to run more code is not the solution
>>> to making the kernel reliable at the time of a crash.
>>>
>>> Eric
>>
>> While this may be true, your cutting short an already existing panic()
>> path which has been around and is well used. I would think if it's
>> good enough for everyone else it should be good enough for the crash
>> kernel path.
> You are talking about decisions over a decade old at this point.  So
> this isn't some new thing you are complaing about.  In fact mtdoops is
> much newer.    So if it isn't working that is the code with the bug.
>
> Furthermore we have lots of emperical evidence that doing lots of things
> in the kernel at crash time doesn't work in practice.
>
> Without kexec-on-panic there just isn't anything else the code can do,
> so it makes sense to do a lot of flaky things.
>
> I am not at all sympathetic with the notion of reorganizing the code so
> that more code can run in the kexec on panic path and reduce it's
> changes of working.  We have been there.  We have the scars.  That is
> the reason why kexec is used at all.
>
> There used to be a core dumper in the kernel it only worked in the lab
> never in the field.
>
> What I hear you asking for is to put a core dumper (or something as
> liable to failure) back in the kernel without acknowledging any of the
> reasons why that doesn't work in practice.  I am hearing you arguing for
> making the code buggier.  I am hearing that and I am saying not interested.

panic() already runs crash_kexec(). I'm not familiar with this "dumper" 
, can you explain the connection? I'm just suggesting we combine two 
similar code paths. The panic() call does minimal additional code prior 
to calling crash_kexec , but I'm not sure it's significant enough to 
cause reliability to change. Is there something specific in panic() 
which you can point to which you know would cause a problem ?

It seems your suggesting crash_kexec() inside panic() never , or rarely, 
works in the field. I think it's worked fine a Cisco more than once.

Daniel


From thomas.lendacky at amd.com  Wed Jun  7 12:13:10 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Wed, 07 Jun 2017 14:13:10 -0500
Subject: [PATCH v6 00/34] x86: Secure Memory Encryption (AMD)
Message-ID: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>

This patch series provides support for AMD's new Secure Memory Encryption (SME)
feature.

SME can be used to mark individual pages of memory as encrypted through the
page tables. A page of memory that is marked encrypted will be automatically
decrypted when read from DRAM and will be automatically encrypted when
written to DRAM. Details on SME can found in the links below.

The SME feature is identified through a CPUID function and enabled through
the SYSCFG MSR. Once enabled, page table entries will determine how the
memory is accessed. If a page table entry has the memory encryption mask set,
then that memory will be accessed as encrypted memory. The memory encryption
mask (as well as other related information) is determined from settings
returned through the same CPUID function that identifies the presence of the
feature.

The approach that this patch series takes is to encrypt everything possible
starting early in the boot where the kernel is encrypted. Using the page
table macros the encryption mask can be incorporated into all page table
entries and page allocations. By updating the protection map, userspace
allocations are also marked encrypted. Certain data must be accounted for
as having been placed in memory before SME was enabled (EFI, initrd, etc.)
and accessed accordingly.

This patch series is a pre-cursor to another AMD processor feature called
Secure Encrypted Virtualization (SEV). The support for SEV will build upon
the SME support and will be submitted later. Details on SEV can be found
in the links below.

The following links provide additional detail:

AMD Memory Encryption whitepaper:
   http://amd-dev.wpengine.netdna-cdn.com/wordpress/media/2013/12/AMD_Memory_Encryption_Whitepaper_v7-Public.pdf

AMD64 Architecture Programmer's Manual:
   http://support.amd.com/TechDocs/24593.pdf
   SME is section 7.10
   SEV is section 15.34

---

This patch series is based off of the master branch of tip.
  Commit 53614fbd7961 ("Merge branch 'WIP.x86/fpu'")

Source code is also available at https://github.com/codomania/tip/tree/sme-v6


Still to do:
- Kdump support, including using memremap() instead of ioremap_cache()

Changes since v5:
- Added support for 5-level paging
- Added IOMMU support
- Created a generic asm/mem_encrypt.h in order to remove a bunch of
  #ifndef/#define entries
- Removed changes to the __va() macro and defined a function to return
  the true physical address in cr3
- Removed sysfs support as it was determined not to be needed
- General code cleanup based on feedback
- General cleanup of patch subjects and descriptions

Changes since v4:
- Re-worked mapping of setup data to not use a fixed list. Rather, check
  dynamically whether the requested early_memremap()/memremap() call
  needs to be mapped decrypted.
- Moved SME cpu feature into scattered features
- Moved some declarations into header files
- Cleared the encryption mask from the __PHYSICAL_MASK so that users
  of macros such as pmd_pfn_mask() don't have to worry/know about the
  encryption mask
- Updated some return types and values related to EFI and e820 functions
  so that an error could be returned
- During cpu shutdown, removed cache disabling and added a check for kexec
  in progress to use wbinvd followed immediately by halt in order to avoid
  any memory corruption
- Update how persistent memory is identified
- Added a function to find command line arguments and their values
- Added sysfs support
- General code cleanup based on feedback
- General cleanup of patch subjects and descriptions


Changes since v3:
- Broke out some of the patches into smaller individual patches
- Updated Documentation
- Added a message to indicate why the IOMMU was disabled
- Updated CPU feature support for SME by taking into account whether
  BIOS has enabled SME
- Eliminated redundant functions
- Added some warning messages for DMA usage of bounce buffers when SME
  is active
- Added support for persistent memory
- Added support to determine when setup data is being mapped and be sure
  to map it un-encrypted
- Added CONFIG support to set the default action of whether to activate
  SME if it is supported/enabled
- Added support for (re)booting with kexec

Changes since v2:
- Updated Documentation
- Make the encryption mask available outside of arch/x86 through a
  standard include file
- Conversion of assembler routines to C where possible (not everything
  could be converted, e.g. the routine that does the actual encryption
  needs to be copied into a safe location and it is difficult to
  determine the actual length of the function in order to copy it)
- Fix SME feature use of scattered CPUID feature
- Creation of SME specific functions for things like encrypting
  the setup data, ramdisk, etc.
- New take on early_memremap / memremap encryption support
- Additional support for accessing video buffers (fbdev/gpu) as
  un-encrypted
- Disable IOMMU for now - need to investigate further in relation to
  how it needs to be programmed relative to accessing physical memory

Changes since v1:
- Added Documentation.
- Removed AMD vendor check for setting the PAT write protect mode
- Updated naming of trampoline flag for SME as well as moving of the
  SME check to before paging is enabled.
- Change to early_memremap to identify the data being mapped as either
  boot data or kernel data.  The idea being that boot data will have
  been placed in memory as un-encrypted data and would need to be accessed
  as such.
- Updated debugfs support for the bootparams to access the data properly.
- Do not set the SYSCFG[MEME] bit, only check it.  The setting of the
  MemEncryptionModeEn bit results in a reduction of physical address size
  of the processor.  It is possible that BIOS could have configured resources
  resources into a range that will now not be addressable.  To prevent this,
  rely on BIOS to set the SYSCFG[MEME] bit and only then enable memory
  encryption support in the kernel.

Tom Lendacky (34):
      x86: Document AMD Secure Memory Encryption (SME)
      x86/mm/pat: Set write-protect cache mode for full PAT support
      x86, mpparse, x86/acpi, x86/PCI, x86/dmi, SFI: Use memremap for RAM mappings
      x86/CPU/AMD: Add the Secure Memory Encryption CPU feature
      x86/CPU/AMD: Handle SME reduction in physical address size
      x86/mm: Add Secure Memory Encryption (SME) support
      x86/mm: Don't use phys_to_virt in ioremap() if SME is active
      x86/mm: Add support to enable SME in early boot processing
      x86/mm: Simplify p[gum]d_page() macros
      x86, x86/mm, x86/xen, olpc: Use __va() against just the physical address in cr3
      x86/mm: Provide general kernel support for memory encryption
      x86/mm: Extend early_memremap() support with additional attrs
      x86/mm: Add support for early encrypt/decrypt of memory
      x86/mm: Insure that boot memory areas are mapped properly
      x86/boot/e820: Add support to determine the E820 type of an address
      efi: Add an EFI table address match function
      efi: Update efi_mem_type() to return an error rather than 0
      x86/efi: Update EFI pagetable creation to work with SME
      x86/mm: Add support to access boot related data in the clear
      x86, mpparse: Use memremap to map the mpf and mpc data
      x86/mm: Add support to access persistent memory in the clear
      x86/mm: Add support for changing the memory encryption attribute
      x86, realmode: Decrypt trampoline area if memory encryption is active
      x86, swiotlb: Add memory encryption support
      swiotlb: Add warnings for use of bounce buffers with SME
      iommu/amd: Allow the AMD IOMMU to work with memory encryption
      x86, realmode: Check for memory encryption on the APs
      x86, drm, fbdev: Do not specify encrypted memory for video mappings
      kvm: x86: svm: Support Secure Memory Encryption within KVM
      x86/mm, kexec: Allow kexec to be used with SME
      x86/mm: Use proper encryption attributes with /dev/mem
      x86/mm: Add support to encrypt the kernel in-place
      x86/boot: Add early cmdline parsing for options with arguments
      x86/mm: Add support to make use of Secure Memory Encryption


 Documentation/admin-guide/kernel-parameters.txt |   11 
 Documentation/x86/amd-memory-encryption.txt     |   68 ++
 arch/ia64/kernel/efi.c                          |    4 
 arch/x86/Kconfig                                |   26 +
 arch/x86/boot/compressed/pagetable.c            |    7 
 arch/x86/include/asm/cmdline.h                  |    2 
 arch/x86/include/asm/cpufeatures.h              |    1 
 arch/x86/include/asm/dma-mapping.h              |    5 
 arch/x86/include/asm/dmi.h                      |    8 
 arch/x86/include/asm/e820/api.h                 |    2 
 arch/x86/include/asm/fixmap.h                   |   20 +
 arch/x86/include/asm/init.h                     |    1 
 arch/x86/include/asm/io.h                       |    7 
 arch/x86/include/asm/kexec.h                    |    8 
 arch/x86/include/asm/kvm_host.h                 |    2 
 arch/x86/include/asm/mem_encrypt.h              |  112 ++++
 arch/x86/include/asm/msr-index.h                |    2 
 arch/x86/include/asm/page_types.h               |    2 
 arch/x86/include/asm/pgtable.h                  |   28 +
 arch/x86/include/asm/pgtable_types.h            |   54 +-
 arch/x86/include/asm/processor.h                |    3 
 arch/x86/include/asm/realmode.h                 |   12 
 arch/x86/include/asm/set_memory.h               |    3 
 arch/x86/include/asm/special_insns.h            |    9 
 arch/x86/include/asm/vga.h                      |   14 
 arch/x86/kernel/acpi/boot.c                     |    6 
 arch/x86/kernel/cpu/amd.c                       |   17 +
 arch/x86/kernel/cpu/scattered.c                 |    1 
 arch/x86/kernel/e820.c                          |   26 +
 arch/x86/kernel/espfix_64.c                     |    2 
 arch/x86/kernel/head64.c                        |   42 +
 arch/x86/kernel/head_64.S                       |   80 ++-
 arch/x86/kernel/kdebugfs.c                      |   34 -
 arch/x86/kernel/ksysfs.c                        |   28 -
 arch/x86/kernel/machine_kexec_64.c              |   35 +
 arch/x86/kernel/mpparse.c                       |  108 +++-
 arch/x86/kernel/pci-dma.c                       |   11 
 arch/x86/kernel/pci-nommu.c                     |    2 
 arch/x86/kernel/pci-swiotlb.c                   |   15 -
 arch/x86/kernel/process.c                       |   17 +
 arch/x86/kernel/setup.c                         |    9 
 arch/x86/kvm/mmu.c                              |   12 
 arch/x86/kvm/mmu.h                              |    2 
 arch/x86/kvm/svm.c                              |   35 +
 arch/x86/kvm/vmx.c                              |    3 
 arch/x86/kvm/x86.c                              |    3 
 arch/x86/lib/cmdline.c                          |  105 ++++
 arch/x86/mm/Makefile                            |    3 
 arch/x86/mm/fault.c                             |   10 
 arch/x86/mm/ident_map.c                         |   12 
 arch/x86/mm/ioremap.c                           |  277 +++++++++-
 arch/x86/mm/kasan_init_64.c                     |    4 
 arch/x86/mm/mem_encrypt.c                       |  667 +++++++++++++++++++++++
 arch/x86/mm/mem_encrypt_boot.S                  |  150 +++++
 arch/x86/mm/pageattr.c                          |   67 ++
 arch/x86/mm/pat.c                               |    9 
 arch/x86/pci/common.c                           |    4 
 arch/x86/platform/efi/efi.c                     |    6 
 arch/x86/platform/efi/efi_64.c                  |   15 -
 arch/x86/platform/olpc/olpc-xo1-pm.c            |    2 
 arch/x86/power/hibernate_64.c                   |    2 
 arch/x86/realmode/init.c                        |   15 +
 arch/x86/realmode/rm/trampoline_64.S            |   24 +
 arch/x86/xen/mmu_pv.c                           |    6 
 drivers/firmware/dmi-sysfs.c                    |    5 
 drivers/firmware/efi/efi.c                      |   33 +
 drivers/firmware/pcdp.c                         |    4 
 drivers/gpu/drm/drm_gem.c                       |    2 
 drivers/gpu/drm/drm_vm.c                        |    4 
 drivers/gpu/drm/ttm/ttm_bo_vm.c                 |    7 
 drivers/gpu/drm/udl/udl_fb.c                    |    4 
 drivers/iommu/amd_iommu.c                       |   36 +
 drivers/iommu/amd_iommu_init.c                  |   18 -
 drivers/iommu/amd_iommu_proto.h                 |   10 
 drivers/iommu/amd_iommu_types.h                 |    2 
 drivers/sfi/sfi_core.c                          |   22 -
 drivers/video/fbdev/core/fbmem.c                |   12 
 include/asm-generic/early_ioremap.h             |    2 
 include/asm-generic/mem_encrypt.h               |   45 ++
 include/asm-generic/pgtable.h                   |    8 
 include/linux/dma-mapping.h                     |    9 
 include/linux/efi.h                             |    9 
 include/linux/io.h                              |    2 
 include/linux/kexec.h                           |   14 
 include/linux/mem_encrypt.h                     |   18 +
 include/linux/swiotlb.h                         |    1 
 init/main.c                                     |   13 
 kernel/kexec_core.c                             |    6 
 kernel/memremap.c                               |   20 +
 lib/swiotlb.c                                   |   59 ++
 mm/early_ioremap.c                              |   30 +
 91 files changed, 2411 insertions(+), 261 deletions(-)
 create mode 100644 Documentation/x86/amd-memory-encryption.txt
 create mode 100644 arch/x86/include/asm/mem_encrypt.h
 create mode 100644 arch/x86/mm/mem_encrypt.c
 create mode 100644 arch/x86/mm/mem_encrypt_boot.S
 create mode 100644 include/asm-generic/mem_encrypt.h
 create mode 100644 include/linux/mem_encrypt.h

-- 
Tom Lendacky


From thomas.lendacky at amd.com  Wed Jun  7 12:13:21 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Wed, 07 Jun 2017 14:13:21 -0500
Subject: [PATCH v6 01/34] x86: Document AMD Secure Memory Encryption (SME)
In-Reply-To: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170607191320.28645.51540.stgit@tlendack-t1.amdoffice.net>

Create a Documentation entry to describe the AMD Secure Memory
Encryption (SME) feature and add documentation for the mem_encrypt=
kernel parameter.

Reviewed-by: Borislav Petkov <bp at suse.de>
Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 Documentation/admin-guide/kernel-parameters.txt |   11 ++++
 Documentation/x86/amd-memory-encryption.txt     |   68 +++++++++++++++++++++++
 2 files changed, 79 insertions(+)
 create mode 100644 Documentation/x86/amd-memory-encryption.txt

diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index 4e4c340..abb65da 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -2188,6 +2188,17 @@
 			memory contents and reserves bad memory
 			regions that are detected.
 
+	mem_encrypt=	[X86-64] AMD Secure Memory Encryption (SME) control
+			Valid arguments: on, off
+			Default (depends on kernel configuration option):
+			  on  (CONFIG_AMD_MEM_ENCRYPT_ACTIVE_BY_DEFAULT=y)
+			  off (CONFIG_AMD_MEM_ENCRYPT_ACTIVE_BY_DEFAULT=n)
+			mem_encrypt=on:		Activate SME
+			mem_encrypt=off:	Do not activate SME
+
+			Refer to Documentation/x86/amd-memory-encryption.txt
+			for details on when memory encryption can be activated.
+
 	mem_sleep_default=	[SUSPEND] Default system suspend mode:
 			s2idle  - Suspend-To-Idle
 			shallow - Power-On Suspend or equivalent (if supported)
diff --git a/Documentation/x86/amd-memory-encryption.txt b/Documentation/x86/amd-memory-encryption.txt
new file mode 100644
index 0000000..f512ab7
--- /dev/null
+++ b/Documentation/x86/amd-memory-encryption.txt
@@ -0,0 +1,68 @@
+Secure Memory Encryption (SME) is a feature found on AMD processors.
+
+SME provides the ability to mark individual pages of memory as encrypted using
+the standard x86 page tables.  A page that is marked encrypted will be
+automatically decrypted when read from DRAM and encrypted when written to
+DRAM.  SME can therefore be used to protect the contents of DRAM from physical
+attacks on the system.
+
+A page is encrypted when a page table entry has the encryption bit set (see
+below on how to determine its position).  The encryption bit can also be
+specified in the cr3 register, allowing the PGD table to be encrypted. Each
+successive level of page tables can also be encrypted by setting the encryption
+bit in the page table entry that points to the next table. This allows the full
+page table hierarchy to be encrypted. Note, this means that just because the
+encryption bit is set in cr3, doesn't imply the full hierarchy is encyrpted.
+Each page table entry in the hierarchy needs to have the encryption bit set to
+achieve that. So, theoretically, you could have the encryption bit set in cr3
+so that the PGD is encrypted, but not set the encryption bit in the PGD entry
+for a PUD which results in the PUD pointed to by that entry to not be
+encrypted.
+
+Support for SME can be determined through the CPUID instruction. The CPUID
+function 0x8000001f reports information related to SME:
+
+	0x8000001f[eax]:
+		Bit[0] indicates support for SME
+	0x8000001f[ebx]:
+		Bits[5:0]  pagetable bit number used to activate memory
+			   encryption
+		Bits[11:6] reduction in physical address space, in bits, when
+			   memory encryption is enabled (this only affects
+			   system physical addresses, not guest physical
+			   addresses)
+
+If support for SME is present, MSR 0xc00100010 (MSR_K8_SYSCFG) can be used to
+determine if SME is enabled and/or to enable memory encryption:
+
+	0xc0010010:
+		Bit[23]   0 = memory encryption features are disabled
+			  1 = memory encryption features are enabled
+
+Linux relies on BIOS to set this bit if BIOS has determined that the reduction
+in the physical address space as a result of enabling memory encryption (see
+CPUID information above) will not conflict with the address space resource
+requirements for the system.  If this bit is not set upon Linux startup then
+Linux itself will not set it and memory encryption will not be possible.
+
+The state of SME in the Linux kernel can be documented as follows:
+	- Supported:
+	  The CPU supports SME (determined through CPUID instruction).
+
+	- Enabled:
+	  Supported and bit 23 of MSR_K8_SYSCFG is set.
+
+	- Active:
+	  Supported, Enabled and the Linux kernel is actively applying
+	  the encryption bit to page table entries (the SME mask in the
+	  kernel is non-zero).
+
+SME can also be enabled and activated in the BIOS. If SME is enabled and
+activated in the BIOS, then all memory accesses will be encrypted and it will
+not be necessary to activate the Linux memory encryption support.  If the BIOS
+merely enables SME (sets bit 23 of the MSR_K8_SYSCFG), then Linux can activate
+memory encryption by default (CONFIG_AMD_MEM_ENCRYPT_ACTIVE_BY_DEFAULT=y) or
+by supplying mem_encrypt=on on the kernel command line.  However, if BIOS does
+not enable SME, then Linux will not be able to activate memory encryption, even
+if configured to do so by default or the mem_encrypt=on command line parameter
+is specified.



From thomas.lendacky at amd.com  Wed Jun  7 12:13:33 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Wed, 07 Jun 2017 14:13:33 -0500
Subject: [PATCH v6 02/34] x86/mm/pat: Set write-protect cache mode for full
 PAT support
In-Reply-To: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170607191333.28645.95909.stgit@tlendack-t1.amdoffice.net>

For processors that support PAT, set the write-protect cache mode
(_PAGE_CACHE_MODE_WP) entry to the actual write-protect value (x05).

Acked-by: Borislav Petkov <bp at suse.de>
Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/mm/pat.c |    6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/arch/x86/mm/pat.c b/arch/x86/mm/pat.c
index 9b78685..6753d9c 100644
--- a/arch/x86/mm/pat.c
+++ b/arch/x86/mm/pat.c
@@ -295,7 +295,7 @@ static void init_cache_modes(void)
  * pat_init - Initialize PAT MSR and PAT table
  *
  * This function initializes PAT MSR and PAT table with an OS-defined value
- * to enable additional cache attributes, WC and WT.
+ * to enable additional cache attributes, WC, WT and WP.
  *
  * This function must be called on all CPUs using the specific sequence of
  * operations defined in Intel SDM. mtrr_rendezvous_handler() provides this
@@ -356,7 +356,7 @@ void pat_init(void)
 		 *      010    2    UC-: _PAGE_CACHE_MODE_UC_MINUS
 		 *      011    3    UC : _PAGE_CACHE_MODE_UC
 		 *      100    4    WB : Reserved
-		 *      101    5    WC : Reserved
+		 *      101    5    WP : _PAGE_CACHE_MODE_WP
 		 *      110    6    UC-: Reserved
 		 *      111    7    WT : _PAGE_CACHE_MODE_WT
 		 *
@@ -364,7 +364,7 @@ void pat_init(void)
 		 * corresponding types in the presence of PAT errata.
 		 */
 		pat = PAT(0, WB) | PAT(1, WC) | PAT(2, UC_MINUS) | PAT(3, UC) |
-		      PAT(4, WB) | PAT(5, WC) | PAT(6, UC_MINUS) | PAT(7, WT);
+		      PAT(4, WB) | PAT(5, WP) | PAT(6, UC_MINUS) | PAT(7, WT);
 	}
 
 	if (!boot_cpu_done) {



From thomas.lendacky at amd.com  Wed Jun  7 12:13:41 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Wed, 07 Jun 2017 14:13:41 -0500
Subject: [PATCH v6 03/34] x86, mpparse, x86/acpi, x86/PCI, x86/dmi,
 SFI: Use memremap for RAM mappings
In-Reply-To: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170607191341.28645.70439.stgit@tlendack-t1.amdoffice.net>

The ioremap() function is intended for mapping MMIO. For RAM, the
memremap() function should be used. Convert calls from ioremap() to
memremap() when re-mapping RAM.

This will be used later by SME to control how the encryption mask is
applied to memory mappings, with certain memory locations being mapped
decrypted vs encrypted.

Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/include/asm/dmi.h   |    8 ++++----
 arch/x86/kernel/acpi/boot.c  |    6 +++---
 arch/x86/kernel/kdebugfs.c   |   34 +++++++++++-----------------------
 arch/x86/kernel/ksysfs.c     |   28 ++++++++++++++--------------
 arch/x86/kernel/mpparse.c    |   10 +++++-----
 arch/x86/pci/common.c        |    4 ++--
 drivers/firmware/dmi-sysfs.c |    5 +++--
 drivers/firmware/pcdp.c      |    4 ++--
 drivers/sfi/sfi_core.c       |   22 +++++++++++-----------
 9 files changed, 55 insertions(+), 66 deletions(-)

diff --git a/arch/x86/include/asm/dmi.h b/arch/x86/include/asm/dmi.h
index 3c69fed..a8e15b0 100644
--- a/arch/x86/include/asm/dmi.h
+++ b/arch/x86/include/asm/dmi.h
@@ -13,9 +13,9 @@ static __always_inline __init void *dmi_alloc(unsigned len)
 }
 
 /* Use early IO mappings for DMI because it's initialized early */
-#define dmi_early_remap		early_ioremap
-#define dmi_early_unmap		early_iounmap
-#define dmi_remap		ioremap_cache
-#define dmi_unmap		iounmap
+#define dmi_early_remap		early_memremap
+#define dmi_early_unmap		early_memunmap
+#define dmi_remap(_x, _l)	memremap(_x, _l, MEMREMAP_WB)
+#define dmi_unmap(_x)		memunmap(_x)
 
 #endif /* _ASM_X86_DMI_H */
diff --git a/arch/x86/kernel/acpi/boot.c b/arch/x86/kernel/acpi/boot.c
index 6bb6806..850160a 100644
--- a/arch/x86/kernel/acpi/boot.c
+++ b/arch/x86/kernel/acpi/boot.c
@@ -115,7 +115,7 @@
 #define	ACPI_INVALID_GSI		INT_MIN
 
 /*
- * This is just a simple wrapper around early_ioremap(),
+ * This is just a simple wrapper around early_memremap(),
  * with sanity checks for phys == 0 and size == 0.
  */
 char *__init __acpi_map_table(unsigned long phys, unsigned long size)
@@ -124,7 +124,7 @@ char *__init __acpi_map_table(unsigned long phys, unsigned long size)
 	if (!phys || !size)
 		return NULL;
 
-	return early_ioremap(phys, size);
+	return early_memremap(phys, size);
 }
 
 void __init __acpi_unmap_table(char *map, unsigned long size)
@@ -132,7 +132,7 @@ void __init __acpi_unmap_table(char *map, unsigned long size)
 	if (!map || !size)
 		return;
 
-	early_iounmap(map, size);
+	early_memunmap(map, size);
 }
 
 #ifdef CONFIG_X86_LOCAL_APIC
diff --git a/arch/x86/kernel/kdebugfs.c b/arch/x86/kernel/kdebugfs.c
index 38b6458..fd6f8fb 100644
--- a/arch/x86/kernel/kdebugfs.c
+++ b/arch/x86/kernel/kdebugfs.c
@@ -33,7 +33,6 @@ static ssize_t setup_data_read(struct file *file, char __user *user_buf,
 	struct setup_data_node *node = file->private_data;
 	unsigned long remain;
 	loff_t pos = *ppos;
-	struct page *pg;
 	void *p;
 	u64 pa;
 
@@ -47,18 +46,13 @@ static ssize_t setup_data_read(struct file *file, char __user *user_buf,
 		count = node->len - pos;
 
 	pa = node->paddr + sizeof(struct setup_data) + pos;
-	pg = pfn_to_page((pa + count - 1) >> PAGE_SHIFT);
-	if (PageHighMem(pg)) {
-		p = ioremap_cache(pa, count);
-		if (!p)
-			return -ENXIO;
-	} else
-		p = __va(pa);
+	p = memremap(pa, count, MEMREMAP_WB);
+	if (!p)
+		return -ENOMEM;
 
 	remain = copy_to_user(user_buf, p, count);
 
-	if (PageHighMem(pg))
-		iounmap(p);
+	memunmap(p);
 
 	if (remain)
 		return -EFAULT;
@@ -109,7 +103,6 @@ static int __init create_setup_data_nodes(struct dentry *parent)
 	struct setup_data *data;
 	int error;
 	struct dentry *d;
-	struct page *pg;
 	u64 pa_data;
 	int no = 0;
 
@@ -126,16 +119,12 @@ static int __init create_setup_data_nodes(struct dentry *parent)
 			goto err_dir;
 		}
 
-		pg = pfn_to_page((pa_data+sizeof(*data)-1) >> PAGE_SHIFT);
-		if (PageHighMem(pg)) {
-			data = ioremap_cache(pa_data, sizeof(*data));
-			if (!data) {
-				kfree(node);
-				error = -ENXIO;
-				goto err_dir;
-			}
-		} else
-			data = __va(pa_data);
+		data = memremap(pa_data, sizeof(*data), MEMREMAP_WB);
+		if (!data) {
+			kfree(node);
+			error = -ENOMEM;
+			goto err_dir;
+		}
 
 		node->paddr = pa_data;
 		node->type = data->type;
@@ -143,8 +132,7 @@ static int __init create_setup_data_nodes(struct dentry *parent)
 		error = create_setup_data_node(d, no, node);
 		pa_data = data->next;
 
-		if (PageHighMem(pg))
-			iounmap(data);
+		memunmap(data);
 		if (error)
 			goto err_dir;
 		no++;
diff --git a/arch/x86/kernel/ksysfs.c b/arch/x86/kernel/ksysfs.c
index 4afc67f..ee51db9 100644
--- a/arch/x86/kernel/ksysfs.c
+++ b/arch/x86/kernel/ksysfs.c
@@ -16,8 +16,8 @@
 #include <linux/stat.h>
 #include <linux/slab.h>
 #include <linux/mm.h>
+#include <linux/io.h>
 
-#include <asm/io.h>
 #include <asm/setup.h>
 
 static ssize_t version_show(struct kobject *kobj,
@@ -79,12 +79,12 @@ static int get_setup_data_paddr(int nr, u64 *paddr)
 			*paddr = pa_data;
 			return 0;
 		}
-		data = ioremap_cache(pa_data, sizeof(*data));
+		data = memremap(pa_data, sizeof(*data), MEMREMAP_WB);
 		if (!data)
 			return -ENOMEM;
 
 		pa_data = data->next;
-		iounmap(data);
+		memunmap(data);
 		i++;
 	}
 	return -EINVAL;
@@ -97,17 +97,17 @@ static int __init get_setup_data_size(int nr, size_t *size)
 	u64 pa_data = boot_params.hdr.setup_data;
 
 	while (pa_data) {
-		data = ioremap_cache(pa_data, sizeof(*data));
+		data = memremap(pa_data, sizeof(*data), MEMREMAP_WB);
 		if (!data)
 			return -ENOMEM;
 		if (nr == i) {
 			*size = data->len;
-			iounmap(data);
+			memunmap(data);
 			return 0;
 		}
 
 		pa_data = data->next;
-		iounmap(data);
+		memunmap(data);
 		i++;
 	}
 	return -EINVAL;
@@ -127,12 +127,12 @@ static ssize_t type_show(struct kobject *kobj,
 	ret = get_setup_data_paddr(nr, &paddr);
 	if (ret)
 		return ret;
-	data = ioremap_cache(paddr, sizeof(*data));
+	data = memremap(paddr, sizeof(*data), MEMREMAP_WB);
 	if (!data)
 		return -ENOMEM;
 
 	ret = sprintf(buf, "0x%x\n", data->type);
-	iounmap(data);
+	memunmap(data);
 	return ret;
 }
 
@@ -154,7 +154,7 @@ static ssize_t setup_data_data_read(struct file *fp,
 	ret = get_setup_data_paddr(nr, &paddr);
 	if (ret)
 		return ret;
-	data = ioremap_cache(paddr, sizeof(*data));
+	data = memremap(paddr, sizeof(*data), MEMREMAP_WB);
 	if (!data)
 		return -ENOMEM;
 
@@ -170,15 +170,15 @@ static ssize_t setup_data_data_read(struct file *fp,
 		goto out;
 
 	ret = count;
-	p = ioremap_cache(paddr + sizeof(*data), data->len);
+	p = memremap(paddr + sizeof(*data), data->len, MEMREMAP_WB);
 	if (!p) {
 		ret = -ENOMEM;
 		goto out;
 	}
 	memcpy(buf, p + off, count);
-	iounmap(p);
+	memunmap(p);
 out:
-	iounmap(data);
+	memunmap(data);
 	return ret;
 }
 
@@ -250,13 +250,13 @@ static int __init get_setup_data_total_num(u64 pa_data, int *nr)
 	*nr = 0;
 	while (pa_data) {
 		*nr += 1;
-		data = ioremap_cache(pa_data, sizeof(*data));
+		data = memremap(pa_data, sizeof(*data), MEMREMAP_WB);
 		if (!data) {
 			ret = -ENOMEM;
 			goto out;
 		}
 		pa_data = data->next;
-		iounmap(data);
+		memunmap(data);
 	}
 
 out:
diff --git a/arch/x86/kernel/mpparse.c b/arch/x86/kernel/mpparse.c
index 0d904d7..fd37f39 100644
--- a/arch/x86/kernel/mpparse.c
+++ b/arch/x86/kernel/mpparse.c
@@ -436,9 +436,9 @@ static unsigned long __init get_mpc_size(unsigned long physptr)
 	struct mpc_table *mpc;
 	unsigned long size;
 
-	mpc = early_ioremap(physptr, PAGE_SIZE);
+	mpc = early_memremap(physptr, PAGE_SIZE);
 	size = mpc->length;
-	early_iounmap(mpc, PAGE_SIZE);
+	early_memunmap(mpc, PAGE_SIZE);
 	apic_printk(APIC_VERBOSE, "  mpc: %lx-%lx\n", physptr, physptr + size);
 
 	return size;
@@ -450,7 +450,7 @@ static int __init check_physptr(struct mpf_intel *mpf, unsigned int early)
 	unsigned long size;
 
 	size = get_mpc_size(mpf->physptr);
-	mpc = early_ioremap(mpf->physptr, size);
+	mpc = early_memremap(mpf->physptr, size);
 	/*
 	 * Read the physical hardware table.  Anything here will
 	 * override the defaults.
@@ -461,10 +461,10 @@ static int __init check_physptr(struct mpf_intel *mpf, unsigned int early)
 #endif
 		pr_err("BIOS bug, MP table errors detected!...\n");
 		pr_cont("... disabling SMP support. (tell your hw vendor)\n");
-		early_iounmap(mpc, size);
+		early_memunmap(mpc, size);
 		return -1;
 	}
-	early_iounmap(mpc, size);
+	early_memunmap(mpc, size);
 
 	if (early)
 		return -1;
diff --git a/arch/x86/pci/common.c b/arch/x86/pci/common.c
index 190e718..08cf71c 100644
--- a/arch/x86/pci/common.c
+++ b/arch/x86/pci/common.c
@@ -691,7 +691,7 @@ int pcibios_add_device(struct pci_dev *dev)
 
 	pa_data = boot_params.hdr.setup_data;
 	while (pa_data) {
-		data = ioremap(pa_data, sizeof(*rom));
+		data = memremap(pa_data, sizeof(*rom), MEMREMAP_WB);
 		if (!data)
 			return -ENOMEM;
 
@@ -710,7 +710,7 @@ int pcibios_add_device(struct pci_dev *dev)
 			}
 		}
 		pa_data = data->next;
-		iounmap(data);
+		memunmap(data);
 	}
 	set_dma_domain_ops(dev);
 	set_dev_domain_options(dev);
diff --git a/drivers/firmware/dmi-sysfs.c b/drivers/firmware/dmi-sysfs.c
index ef76e5e..d5de6ee 100644
--- a/drivers/firmware/dmi-sysfs.c
+++ b/drivers/firmware/dmi-sysfs.c
@@ -25,6 +25,7 @@
 #include <linux/slab.h>
 #include <linux/list.h>
 #include <linux/io.h>
+#include <asm/dmi.h>
 
 #define MAX_ENTRY_TYPE 255 /* Most of these aren't used, but we consider
 			      the top entry type is only 8 bits */
@@ -380,7 +381,7 @@ static ssize_t dmi_sel_raw_read_phys32(struct dmi_sysfs_entry *entry,
 	u8 __iomem *mapped;
 	ssize_t wrote = 0;
 
-	mapped = ioremap(sel->access_method_address, sel->area_length);
+	mapped = dmi_remap(sel->access_method_address, sel->area_length);
 	if (!mapped)
 		return -EIO;
 
@@ -390,7 +391,7 @@ static ssize_t dmi_sel_raw_read_phys32(struct dmi_sysfs_entry *entry,
 		wrote++;
 	}
 
-	iounmap(mapped);
+	dmi_unmap(mapped);
 	return wrote;
 }
 
diff --git a/drivers/firmware/pcdp.c b/drivers/firmware/pcdp.c
index 75273a25..e83d6ae 100644
--- a/drivers/firmware/pcdp.c
+++ b/drivers/firmware/pcdp.c
@@ -95,7 +95,7 @@
 	if (efi.hcdp == EFI_INVALID_TABLE_ADDR)
 		return -ENODEV;
 
-	pcdp = early_ioremap(efi.hcdp, 4096);
+	pcdp = early_memremap(efi.hcdp, 4096);
 	printk(KERN_INFO "PCDP: v%d at 0x%lx\n", pcdp->rev, efi.hcdp);
 
 	if (strstr(cmdline, "console=hcdp")) {
@@ -131,6 +131,6 @@
 	}
 
 out:
-	early_iounmap(pcdp, 4096);
+	early_memunmap(pcdp, 4096);
 	return rc;
 }
diff --git a/drivers/sfi/sfi_core.c b/drivers/sfi/sfi_core.c
index 296db7a..d5ce534 100644
--- a/drivers/sfi/sfi_core.c
+++ b/drivers/sfi/sfi_core.c
@@ -86,13 +86,13 @@
 /*
  * FW creates and saves the SFI tables in memory. When these tables get
  * used, they may need to be mapped to virtual address space, and the mapping
- * can happen before or after the ioremap() is ready, so a flag is needed
+ * can happen before or after the memremap() is ready, so a flag is needed
  * to indicating this
  */
-static u32 sfi_use_ioremap __read_mostly;
+static u32 sfi_use_memremap __read_mostly;
 
 /*
- * sfi_un/map_memory calls early_ioremap/iounmap which is a __init function
+ * sfi_un/map_memory calls early_memremap/memunmap which is a __init function
  * and introduces section mismatch. So use __ref to make it calm.
  */
 static void __iomem * __ref sfi_map_memory(u64 phys, u32 size)
@@ -100,10 +100,10 @@ static void __iomem * __ref sfi_map_memory(u64 phys, u32 size)
 	if (!phys || !size)
 		return NULL;
 
-	if (sfi_use_ioremap)
-		return ioremap_cache(phys, size);
+	if (sfi_use_memremap)
+		return memremap(phys, size, MEMREMAP_WB);
 	else
-		return early_ioremap(phys, size);
+		return early_memremap(phys, size);
 }
 
 static void __ref sfi_unmap_memory(void __iomem *virt, u32 size)
@@ -111,10 +111,10 @@ static void __ref sfi_unmap_memory(void __iomem *virt, u32 size)
 	if (!virt || !size)
 		return;
 
-	if (sfi_use_ioremap)
-		iounmap(virt);
+	if (sfi_use_memremap)
+		memunmap(virt);
 	else
-		early_iounmap(virt, size);
+		early_memunmap(virt, size);
 }
 
 static void sfi_print_table_header(unsigned long long pa,
@@ -507,8 +507,8 @@ void __init sfi_init_late(void)
 	length = syst_va->header.len;
 	sfi_unmap_memory(syst_va, sizeof(struct sfi_table_simple));
 
-	/* Use ioremap now after it is ready */
-	sfi_use_ioremap = 1;
+	/* Use memremap now after it is ready */
+	sfi_use_memremap = 1;
 	syst_va = sfi_map_memory(syst_pa, length);
 
 	sfi_acpi_init();



From thomas.lendacky at amd.com  Wed Jun  7 12:14:04 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Wed, 07 Jun 2017 14:14:04 -0500
Subject: [PATCH v6 05/34] x86/CPU/AMD: Handle SME reduction in physical
 address size
In-Reply-To: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170607191404.28645.41148.stgit@tlendack-t1.amdoffice.net>

When System Memory Encryption (SME) is enabled, the physical address
space is reduced. Adjust the x86_phys_bits value to reflect this
reduction.

Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/kernel/cpu/amd.c |   10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/arch/x86/kernel/cpu/amd.c b/arch/x86/kernel/cpu/amd.c
index c47ceee..5bdcbd4 100644
--- a/arch/x86/kernel/cpu/amd.c
+++ b/arch/x86/kernel/cpu/amd.c
@@ -613,15 +613,19 @@ static void early_init_amd(struct cpuinfo_x86 *c)
 		set_cpu_bug(c, X86_BUG_AMD_E400);
 
 	/*
-	 * BIOS support is required for SME. If BIOS has not enabled SME
-	 * then don't advertise the feature (set in scattered.c)
+	 * BIOS support is required for SME. If BIOS has enabld SME then
+	 * adjust x86_phys_bits by the SME physical address space reduction
+	 * value. If BIOS has not enabled SME then don't advertise the
+	 * feature (set in scattered.c).
 	 */
 	if (cpu_has(c, X86_FEATURE_SME)) {
 		u64 msr;
 
 		/* Check if SME is enabled */
 		rdmsrl(MSR_K8_SYSCFG, msr);
-		if (!(msr & MSR_K8_SYSCFG_MEM_ENCRYPT))
+		if (msr & MSR_K8_SYSCFG_MEM_ENCRYPT)
+			c->x86_phys_bits -= (cpuid_ebx(0x8000001f) >> 6) & 0x3f;
+		else
 			clear_cpu_cap(c, X86_FEATURE_SME);
 	}
 }



From thomas.lendacky at amd.com  Wed Jun  7 12:13:53 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Wed, 07 Jun 2017 14:13:53 -0500
Subject: [PATCH v6 04/34] x86/CPU/AMD: Add the Secure Memory Encryption CPU
 feature
In-Reply-To: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170607191353.28645.85058.stgit@tlendack-t1.amdoffice.net>

Update the CPU features to include identifying and reporting on the
Secure Memory Encryption (SME) feature.  SME is identified by CPUID
0x8000001f, but requires BIOS support to enable it (set bit 23 of
MSR_K8_SYSCFG).  Only show the SME feature as available if reported by
CPUID and enabled by BIOS.

Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/include/asm/cpufeatures.h |    1 +
 arch/x86/include/asm/msr-index.h   |    2 ++
 arch/x86/kernel/cpu/amd.c          |   13 +++++++++++++
 arch/x86/kernel/cpu/scattered.c    |    1 +
 4 files changed, 17 insertions(+)

diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h
index 2701e5f..2b692df 100644
--- a/arch/x86/include/asm/cpufeatures.h
+++ b/arch/x86/include/asm/cpufeatures.h
@@ -196,6 +196,7 @@
 
 #define X86_FEATURE_HW_PSTATE	( 7*32+ 8) /* AMD HW-PState */
 #define X86_FEATURE_PROC_FEEDBACK ( 7*32+ 9) /* AMD ProcFeedbackInterface */
+#define X86_FEATURE_SME		( 7*32+10) /* AMD Secure Memory Encryption */
 
 #define X86_FEATURE_INTEL_PPIN	( 7*32+14) /* Intel Processor Inventory Number */
 #define X86_FEATURE_INTEL_PT	( 7*32+15) /* Intel Processor Trace */
diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h
index 18b1623..460ac01 100644
--- a/arch/x86/include/asm/msr-index.h
+++ b/arch/x86/include/asm/msr-index.h
@@ -352,6 +352,8 @@
 #define MSR_K8_TOP_MEM1			0xc001001a
 #define MSR_K8_TOP_MEM2			0xc001001d
 #define MSR_K8_SYSCFG			0xc0010010
+#define MSR_K8_SYSCFG_MEM_ENCRYPT_BIT	23
+#define MSR_K8_SYSCFG_MEM_ENCRYPT	BIT_ULL(MSR_K8_SYSCFG_MEM_ENCRYPT_BIT)
 #define MSR_K8_INT_PENDING_MSG		0xc0010055
 /* C1E active bits in int pending message */
 #define K8_INTP_C1E_ACTIVE_MASK		0x18000000
diff --git a/arch/x86/kernel/cpu/amd.c b/arch/x86/kernel/cpu/amd.c
index bb5abe8..c47ceee 100644
--- a/arch/x86/kernel/cpu/amd.c
+++ b/arch/x86/kernel/cpu/amd.c
@@ -611,6 +611,19 @@ static void early_init_amd(struct cpuinfo_x86 *c)
 	 */
 	if (cpu_has_amd_erratum(c, amd_erratum_400))
 		set_cpu_bug(c, X86_BUG_AMD_E400);
+
+	/*
+	 * BIOS support is required for SME. If BIOS has not enabled SME
+	 * then don't advertise the feature (set in scattered.c)
+	 */
+	if (cpu_has(c, X86_FEATURE_SME)) {
+		u64 msr;
+
+		/* Check if SME is enabled */
+		rdmsrl(MSR_K8_SYSCFG, msr);
+		if (!(msr & MSR_K8_SYSCFG_MEM_ENCRYPT))
+			clear_cpu_cap(c, X86_FEATURE_SME);
+	}
 }
 
 static void init_amd_k8(struct cpuinfo_x86 *c)
diff --git a/arch/x86/kernel/cpu/scattered.c b/arch/x86/kernel/cpu/scattered.c
index 23c2350..05459ad 100644
--- a/arch/x86/kernel/cpu/scattered.c
+++ b/arch/x86/kernel/cpu/scattered.c
@@ -31,6 +31,7 @@ struct cpuid_bit {
 	{ X86_FEATURE_HW_PSTATE,	CPUID_EDX,  7, 0x80000007, 0 },
 	{ X86_FEATURE_CPB,		CPUID_EDX,  9, 0x80000007, 0 },
 	{ X86_FEATURE_PROC_FEEDBACK,    CPUID_EDX, 11, 0x80000007, 0 },
+	{ X86_FEATURE_SME,		CPUID_EAX,  0, 0x8000001f, 0 },
 	{ 0, 0, 0, 0, 0 }
 };
 



From thomas.lendacky at amd.com  Wed Jun  7 12:14:16 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Wed, 07 Jun 2017 14:14:16 -0500
Subject: [PATCH v6 06/34] x86/mm: Add Secure Memory Encryption (SME) support
In-Reply-To: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170607191416.28645.58145.stgit@tlendack-t1.amdoffice.net>

Add support for Secure Memory Encryption (SME). This initial support
provides a Kconfig entry to build the SME support into the kernel and
defines the memory encryption mask that will be used in subsequent
patches to mark pages as encrypted.

Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/Kconfig                   |   22 ++++++++++++++++++++++
 arch/x86/include/asm/mem_encrypt.h |   35 +++++++++++++++++++++++++++++++++++
 arch/x86/mm/Makefile               |    1 +
 arch/x86/mm/mem_encrypt.c          |   21 +++++++++++++++++++++
 include/asm-generic/mem_encrypt.h  |   27 +++++++++++++++++++++++++++
 include/linux/mem_encrypt.h        |   18 ++++++++++++++++++
 6 files changed, 124 insertions(+)
 create mode 100644 arch/x86/include/asm/mem_encrypt.h
 create mode 100644 arch/x86/mm/mem_encrypt.c
 create mode 100644 include/asm-generic/mem_encrypt.h
 create mode 100644 include/linux/mem_encrypt.h

diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index 4ccfacc..11f2fdb 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -1407,6 +1407,28 @@ config X86_DIRECT_GBPAGES
 	  supports them), so don't confuse the user by printing
 	  that we have them enabled.
 
+config AMD_MEM_ENCRYPT
+	bool "AMD Secure Memory Encryption (SME) support"
+	depends on X86_64 && CPU_SUP_AMD
+	---help---
+	  Say yes to enable support for the encryption of system memory.
+	  This requires an AMD processor that supports Secure Memory
+	  Encryption (SME).
+
+config AMD_MEM_ENCRYPT_ACTIVE_BY_DEFAULT
+	bool "Activate AMD Secure Memory Encryption (SME) by default"
+	default y
+	depends on AMD_MEM_ENCRYPT
+	---help---
+	  Say yes to have system memory encrypted by default if running on
+	  an AMD processor that supports Secure Memory Encryption (SME).
+
+	  If set to Y, then the encryption of system memory can be
+	  deactivated with the mem_encrypt=off command line option.
+
+	  If set to N, then the encryption of system memory can be
+	  activated with the mem_encrypt=on command line option.
+
 # Common NUMA Features
 config NUMA
 	bool "Numa Memory Allocation and Scheduler Support"
diff --git a/arch/x86/include/asm/mem_encrypt.h b/arch/x86/include/asm/mem_encrypt.h
new file mode 100644
index 0000000..5008fd9
--- /dev/null
+++ b/arch/x86/include/asm/mem_encrypt.h
@@ -0,0 +1,35 @@
+/*
+ * AMD Memory Encryption Support
+ *
+ * Copyright (C) 2016 Advanced Micro Devices, Inc.
+ *
+ * Author: Tom Lendacky <thomas.lendacky at amd.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#ifndef __X86_MEM_ENCRYPT_H__
+#define __X86_MEM_ENCRYPT_H__
+
+#ifndef __ASSEMBLY__
+
+#ifdef CONFIG_AMD_MEM_ENCRYPT
+
+extern unsigned long sme_me_mask;
+
+#else	/* !CONFIG_AMD_MEM_ENCRYPT */
+
+#define sme_me_mask	0UL
+
+#endif	/* CONFIG_AMD_MEM_ENCRYPT */
+
+static inline bool sme_active(void)
+{
+	return !!sme_me_mask;
+}
+
+#endif	/* __ASSEMBLY__ */
+
+#endif	/* __X86_MEM_ENCRYPT_H__ */
diff --git a/arch/x86/mm/Makefile b/arch/x86/mm/Makefile
index 96d2b84..44d4d21 100644
--- a/arch/x86/mm/Makefile
+++ b/arch/x86/mm/Makefile
@@ -39,3 +39,4 @@ obj-$(CONFIG_X86_INTEL_MPX)	+= mpx.o
 obj-$(CONFIG_X86_INTEL_MEMORY_PROTECTION_KEYS) += pkeys.o
 obj-$(CONFIG_RANDOMIZE_MEMORY) += kaslr.o
 
+obj-$(CONFIG_AMD_MEM_ENCRYPT)	+= mem_encrypt.o
diff --git a/arch/x86/mm/mem_encrypt.c b/arch/x86/mm/mem_encrypt.c
new file mode 100644
index 0000000..b99d469
--- /dev/null
+++ b/arch/x86/mm/mem_encrypt.c
@@ -0,0 +1,21 @@
+/*
+ * AMD Memory Encryption Support
+ *
+ * Copyright (C) 2016 Advanced Micro Devices, Inc.
+ *
+ * Author: Tom Lendacky <thomas.lendacky at amd.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include <linux/linkage.h>
+
+/*
+ * Since SME related variables are set early in the boot process they must
+ * reside in the .data section so as not to be zeroed out when the .bss
+ * section is later cleared.
+ */
+unsigned long sme_me_mask __section(.data) = 0;
+EXPORT_SYMBOL_GPL(sme_me_mask);
diff --git a/include/asm-generic/mem_encrypt.h b/include/asm-generic/mem_encrypt.h
new file mode 100644
index 0000000..563c918
--- /dev/null
+++ b/include/asm-generic/mem_encrypt.h
@@ -0,0 +1,27 @@
+/*
+ * AMD Memory Encryption Support
+ *
+ * Copyright (C) 2017 Advanced Micro Devices, Inc.
+ *
+ * Author: Tom Lendacky <thomas.lendacky at amd.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#ifndef __ASM_GENERIC_MEM_ENCRYPT_H__
+#define __ASM_GENERIC_MEM_ENCRYPT_H__
+
+#ifndef __ASSEMBLY__
+
+#define sme_me_mask	0UL
+
+static inline bool sme_active(void)
+{
+	return false;
+}
+
+#endif	/* __ASSEMBLY__ */
+
+#endif	/* __MEM_ENCRYPT_H__ */
diff --git a/include/linux/mem_encrypt.h b/include/linux/mem_encrypt.h
new file mode 100644
index 0000000..1d8e063
--- /dev/null
+++ b/include/linux/mem_encrypt.h
@@ -0,0 +1,18 @@
+/*
+ * AMD Memory Encryption Support
+ *
+ * Copyright (C) 2016 Advanced Micro Devices, Inc.
+ *
+ * Author: Tom Lendacky <thomas.lendacky at amd.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#ifndef __MEM_ENCRYPT_H__
+#define __MEM_ENCRYPT_H__
+
+#include <asm/mem_encrypt.h>
+
+#endif	/* __MEM_ENCRYPT_H__ */



From thomas.lendacky at amd.com  Wed Jun  7 12:14:23 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Wed, 07 Jun 2017 14:14:23 -0500
Subject: [PATCH v6 07/34] x86/mm: Don't use phys_to_virt in ioremap() if SME
 is active
In-Reply-To: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170607191423.28645.14413.stgit@tlendack-t1.amdoffice.net>

Currently there is a check if the address being mapped is in the ISA
range (is_ISA_range()), and if it is then phys_to_virt() is used to
perform the mapping.  When SME is active, however, this will result
in the mapping having the encryption bit set when it is expected that
an ioremap() should not have the encryption bit set. So only use the
phys_to_virt() function if SME is not active

Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/mm/ioremap.c |    7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/arch/x86/mm/ioremap.c b/arch/x86/mm/ioremap.c
index bbc558b..2a0fa89 100644
--- a/arch/x86/mm/ioremap.c
+++ b/arch/x86/mm/ioremap.c
@@ -21,6 +21,7 @@
 #include <asm/tlbflush.h>
 #include <asm/pgalloc.h>
 #include <asm/pat.h>
+#include <asm/mem_encrypt.h>
 
 #include "physaddr.h"
 
@@ -106,9 +107,11 @@ static void __iomem *__ioremap_caller(resource_size_t phys_addr,
 	}
 
 	/*
-	 * Don't remap the low PCI/ISA area, it's always mapped..
+	 * Don't remap the low PCI/ISA area, it's always mapped.
+	 *   But if SME is active, skip this so that the encryption bit
+	 *   doesn't get set.
 	 */
-	if (is_ISA_range(phys_addr, last_addr))
+	if (is_ISA_range(phys_addr, last_addr) && !sme_active())
 		return (__force void __iomem *)phys_to_virt(phys_addr);
 
 	/*



From thomas.lendacky at amd.com  Wed Jun  7 12:14:36 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Wed, 07 Jun 2017 14:14:36 -0500
Subject: [PATCH v6 08/34] x86/mm: Add support to enable SME in early boot
 processing
In-Reply-To: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170607191436.28645.59706.stgit@tlendack-t1.amdoffice.net>

Add support to the early boot code to use Secure Memory Encryption (SME).
Since the kernel has been loaded into memory in a decrypted state, encrypt
the kernel in place and update the early pagetables with the memory
encryption mask so that new pagetable entries will use memory encryption.

The routines to set the encryption mask and perform the encryption are
stub routines for now with functionality to be added in a later patch.

Because of the need to have the routines available to head_64.S, the
mem_encrypt.c is always built and #ifdefs in mem_encrypt.c will provide
functionality or stub routines depending on CONFIG_AMD_MEM_ENCRYPT.

Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/kernel/head_64.S |   61 ++++++++++++++++++++++++++++++++++++++++++++-
 arch/x86/mm/Makefile      |    4 +--
 arch/x86/mm/mem_encrypt.c |   26 +++++++++++++++++++
 3 files changed, 86 insertions(+), 5 deletions(-)

diff --git a/arch/x86/kernel/head_64.S b/arch/x86/kernel/head_64.S
index ac9d327..222630c 100644
--- a/arch/x86/kernel/head_64.S
+++ b/arch/x86/kernel/head_64.S
@@ -91,6 +91,23 @@ startup_64:
 	jnz	bad_address
 
 	/*
+	 * Activate Secure Memory Encryption (SME), if supported and enabled.
+	 * The real_mode_data address is in %rsi and that register can be
+	 * clobbered by the called function so be sure to save it.
+	 * Save the returned mask in %r12 for later use.
+	 */
+	push	%rsi
+	call	sme_enable
+	pop	%rsi
+	movq	%rax, %r12
+
+	/*
+	 * Add the memory encryption mask to %rbp to include it in the page
+	 * table fixups.
+	 */
+	addq	%r12, %rbp
+
+	/*
 	 * Fixup the physical addresses in the page table
 	 */
 	addq	%rbp, early_level4_pgt + (L4_START_KERNEL*8)(%rip)
@@ -113,6 +130,7 @@ startup_64:
 	shrq	$PGDIR_SHIFT, %rax
 
 	leaq	(PAGE_SIZE + _KERNPG_TABLE)(%rbx), %rdx
+	addq	%r12, %rdx
 	movq	%rdx, 0(%rbx,%rax,8)
 	movq	%rdx, 8(%rbx,%rax,8)
 
@@ -129,6 +147,7 @@ startup_64:
 	movq	%rdi, %rax
 	shrq	$PMD_SHIFT, %rdi
 	addq	$(__PAGE_KERNEL_LARGE_EXEC & ~_PAGE_GLOBAL), %rax
+	addq	%r12, %rax
 	leaq	(_end - 1)(%rip), %rcx
 	shrq	$PMD_SHIFT, %rcx
 	subq	%rdi, %rcx
@@ -142,6 +161,12 @@ startup_64:
 	decl	%ecx
 	jnz	1b
 
+	/*
+	 * Determine if any fixups are required. This includes fixups
+	 * based on where the kernel was loaded and whether SME is
+	 * active. If %rbp is zero, then we can skip both the fixups
+	 * and the call to encrypt the kernel.
+	 */
 	test %rbp, %rbp
 	jz .Lskip_fixup
 
@@ -162,11 +187,30 @@ startup_64:
 	cmp	%r8, %rdi
 	jne	1b
 
-	/* Fixup phys_base */
+	/*
+	 * Fixup phys_base - remove the memory encryption mask from %rbp
+	 * to obtain the true physical address.
+	 */
+	subq	%r12, %rbp
 	addq	%rbp, phys_base(%rip)
 
+	/*
+	 * Encrypt the kernel if SME is active.
+	 * The real_mode_data address is in %rsi and that register can be
+	 * clobbered by the called function so be sure to save it.
+	 */
+	push	%rsi
+	call	sme_encrypt_kernel
+	pop	%rsi
+
 .Lskip_fixup:
+	/*
+	 * The encryption mask is in %r12. We ADD this to %rax to be sure
+	 * that the encryption mask is part of the value that will be
+	 * stored in %cr3.
+	 */
 	movq	$(early_level4_pgt - __START_KERNEL_map), %rax
+	addq	%r12, %rax
 	jmp 1f
 ENTRY(secondary_startup_64)
 	/*
@@ -186,7 +230,20 @@ ENTRY(secondary_startup_64)
 	/* Sanitize CPU configuration */
 	call verify_cpu
 
-	movq	$(init_level4_pgt - __START_KERNEL_map), %rax
+	/*
+	 * Get the SME encryption mask.
+	 *  The encryption mask will be returned in %rax so we do an ADD
+	 *  below to be sure that the encryption mask is part of the
+	 *  value that will stored in %cr3.
+	 *
+	 * The real_mode_data address is in %rsi and that register can be
+	 * clobbered by the called function so be sure to save it.
+	 */
+	push	%rsi
+	call	sme_get_me_mask
+	pop	%rsi
+
+	addq	$(init_level4_pgt - __START_KERNEL_map), %rax
 1:
 
 	/* Enable PAE mode and PGE */
diff --git a/arch/x86/mm/Makefile b/arch/x86/mm/Makefile
index 44d4d21..88ee454 100644
--- a/arch/x86/mm/Makefile
+++ b/arch/x86/mm/Makefile
@@ -2,7 +2,7 @@
 KCOV_INSTRUMENT_tlb.o	:= n
 
 obj-y	:=  init.o init_$(BITS).o fault.o ioremap.o extable.o pageattr.o mmap.o \
-	    pat.o pgtable.o physaddr.o gup.o setup_nx.o tlb.o
+	    pat.o pgtable.o physaddr.o gup.o setup_nx.o tlb.o mem_encrypt.o
 
 # Make sure __phys_addr has no stackprotector
 nostackp := $(call cc-option, -fno-stack-protector)
@@ -38,5 +38,3 @@ obj-$(CONFIG_NUMA_EMU)		+= numa_emulation.o
 obj-$(CONFIG_X86_INTEL_MPX)	+= mpx.o
 obj-$(CONFIG_X86_INTEL_MEMORY_PROTECTION_KEYS) += pkeys.o
 obj-$(CONFIG_RANDOMIZE_MEMORY) += kaslr.o
-
-obj-$(CONFIG_AMD_MEM_ENCRYPT)	+= mem_encrypt.o
diff --git a/arch/x86/mm/mem_encrypt.c b/arch/x86/mm/mem_encrypt.c
index b99d469..cc00d8b 100644
--- a/arch/x86/mm/mem_encrypt.c
+++ b/arch/x86/mm/mem_encrypt.c
@@ -11,6 +11,9 @@
  */
 
 #include <linux/linkage.h>
+#include <linux/init.h>
+
+#ifdef CONFIG_AMD_MEM_ENCRYPT
 
 /*
  * Since SME related variables are set early in the boot process they must
@@ -19,3 +22,26 @@
  */
 unsigned long sme_me_mask __section(.data) = 0;
 EXPORT_SYMBOL_GPL(sme_me_mask);
+
+void __init sme_encrypt_kernel(void)
+{
+}
+
+unsigned long __init sme_enable(void)
+{
+	return sme_me_mask;
+}
+
+unsigned long sme_get_me_mask(void)
+{
+	return sme_me_mask;
+}
+
+#else	/* !CONFIG_AMD_MEM_ENCRYPT */
+
+void __init sme_encrypt_kernel(void)	{ }
+unsigned long __init sme_enable(void)	{ return 0; }
+
+unsigned long sme_get_me_mask(void)	{ return 0; }
+
+#endif	/* CONFIG_AMD_MEM_ENCRYPT */



From thomas.lendacky at amd.com  Wed Jun  7 12:14:45 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Wed, 07 Jun 2017 14:14:45 -0500
Subject: [PATCH v6 09/34] x86/mm: Simplify p[gum]d_page() macros
In-Reply-To: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170607191445.28645.15664.stgit@tlendack-t1.amdoffice.net>

Create a pgd_pfn() macro similar to the p[um]d_pfn() macros and then
use the p[gum]d_pfn() macros in the p[gum]d_page() macros instead of
duplicating the code.

Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/include/asm/pgtable.h |   16 +++++++++-------
 1 file changed, 9 insertions(+), 7 deletions(-)

diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h
index f5af95a..96b6b83 100644
--- a/arch/x86/include/asm/pgtable.h
+++ b/arch/x86/include/asm/pgtable.h
@@ -195,6 +195,11 @@ static inline unsigned long p4d_pfn(p4d_t p4d)
 	return (p4d_val(p4d) & p4d_pfn_mask(p4d)) >> PAGE_SHIFT;
 }
 
+static inline unsigned long pgd_pfn(pgd_t pgd)
+{
+	return (pgd_val(pgd) & PTE_PFN_MASK) >> PAGE_SHIFT;
+}
+
 static inline int p4d_large(p4d_t p4d)
 {
 	/* No 512 GiB pages yet */
@@ -699,8 +704,7 @@ static inline unsigned long pmd_page_vaddr(pmd_t pmd)
  * Currently stuck as a macro due to indirect forward reference to
  * linux/mmzone.h's __section_mem_map_addr() definition:
  */
-#define pmd_page(pmd)		\
-	pfn_to_page((pmd_val(pmd) & pmd_pfn_mask(pmd)) >> PAGE_SHIFT)
+#define pmd_page(pmd)	pfn_to_page(pmd_pfn(pmd))
 
 /*
  * the pmd page can be thought of an array like this: pmd_t[PTRS_PER_PMD]
@@ -768,8 +772,7 @@ static inline unsigned long pud_page_vaddr(pud_t pud)
  * Currently stuck as a macro due to indirect forward reference to
  * linux/mmzone.h's __section_mem_map_addr() definition:
  */
-#define pud_page(pud)		\
-	pfn_to_page((pud_val(pud) & pud_pfn_mask(pud)) >> PAGE_SHIFT)
+#define pud_page(pud)	pfn_to_page(pud_pfn(pud))
 
 /* Find an entry in the second-level page table.. */
 static inline pmd_t *pmd_offset(pud_t *pud, unsigned long address)
@@ -819,8 +822,7 @@ static inline unsigned long p4d_page_vaddr(p4d_t p4d)
  * Currently stuck as a macro due to indirect forward reference to
  * linux/mmzone.h's __section_mem_map_addr() definition:
  */
-#define p4d_page(p4d)		\
-	pfn_to_page((p4d_val(p4d) & p4d_pfn_mask(p4d)) >> PAGE_SHIFT)
+#define p4d_page(p4d)	pfn_to_page(p4d_pfn(p4d))
 
 /* Find an entry in the third-level page table.. */
 static inline pud_t *pud_offset(p4d_t *p4d, unsigned long address)
@@ -854,7 +856,7 @@ static inline unsigned long pgd_page_vaddr(pgd_t pgd)
  * Currently stuck as a macro due to indirect forward reference to
  * linux/mmzone.h's __section_mem_map_addr() definition:
  */
-#define pgd_page(pgd)		pfn_to_page(pgd_val(pgd) >> PAGE_SHIFT)
+#define pgd_page(pgd)	pfn_to_page(pgd_pfn(pgd))
 
 /* to find an entry in a page-table-directory. */
 static inline p4d_t *p4d_offset(pgd_t *pgd, unsigned long address)



From thomas.lendacky at amd.com  Wed Jun  7 12:15:08 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Wed, 07 Jun 2017 14:15:08 -0500
Subject: [PATCH v6 11/34] x86/mm: Provide general kernel support for memory
 encryption
In-Reply-To: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170607191508.28645.85684.stgit@tlendack-t1.amdoffice.net>

Changes to the existing page table macros will allow the SME support to
be enabled in a simple fashion with minimal changes to files that use these
macros.  Since the memory encryption mask will now be part of the regular
pagetable macros, we introduce two new macros (_PAGE_TABLE_NOENC and
_KERNPG_TABLE_NOENC) to allow for early pagetable creation/initialization
without the encryption mask before SME becomes active.  Two new pgprot()
macros are defined to allow setting or clearing the page encryption mask.

The FIXMAP_PAGE_NOCACHE define is introduced for use with MMIO.  SME does
not support encryption for MMIO areas so this define removes the encryption
mask from the page attribute.

Two new macros are introduced (__sme_pa() / __sme_pa_nodebug()) to allow
creating a physical address with the encryption mask.  These are used when
working with the cr3 register so that the PGD can be encrypted. The current
__va() macro is updated so that the virtual address is generated based off
of the physical address without the encryption mask thus allowing the same
virtual address to be generated regardless of whether encryption is enabled
for that physical location or not.

Also, an early initialization function is added for SME.  If SME is active,
this function:
 - Updates the early_pmd_flags so that early page faults create mappings
   with the encryption mask.
 - Updates the __supported_pte_mask to include the encryption mask.
 - Updates the protection_map entries to include the encryption mask so
   that user-space allocations will automatically have the encryption mask
   applied.

Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/boot/compressed/pagetable.c |    7 +++++
 arch/x86/include/asm/fixmap.h        |    7 +++++
 arch/x86/include/asm/mem_encrypt.h   |   25 +++++++++++++++++++
 arch/x86/include/asm/page_types.h    |    2 +-
 arch/x86/include/asm/pgtable.h       |    9 +++++++
 arch/x86/include/asm/pgtable_types.h |   45 ++++++++++++++++++++++------------
 arch/x86/include/asm/processor.h     |    3 ++
 arch/x86/kernel/espfix_64.c          |    2 +-
 arch/x86/kernel/head64.c             |   10 +++++++-
 arch/x86/kernel/head_64.S            |   18 +++++++-------
 arch/x86/mm/kasan_init_64.c          |    4 ++-
 arch/x86/mm/mem_encrypt.c            |   18 ++++++++++++++
 arch/x86/mm/pageattr.c               |    3 ++
 include/asm-generic/mem_encrypt.h    |    8 ++++++
 include/asm-generic/pgtable.h        |    8 ++++++
 15 files changed, 138 insertions(+), 31 deletions(-)

diff --git a/arch/x86/boot/compressed/pagetable.c b/arch/x86/boot/compressed/pagetable.c
index 1d78f17..05455ff 100644
--- a/arch/x86/boot/compressed/pagetable.c
+++ b/arch/x86/boot/compressed/pagetable.c
@@ -15,6 +15,13 @@
 #define __pa(x)  ((unsigned long)(x))
 #define __va(x)  ((void *)((unsigned long)(x)))
 
+/*
+ * The pgtable.h and mm/ident_map.c includes make use of the SME related
+ * information which is not used in the compressed image support. Un-define
+ * the SME support to avoid any compile and link errors.
+ */
+#undef CONFIG_AMD_MEM_ENCRYPT
+
 #include "misc.h"
 
 /* These actually do the work of building the kernel identity maps. */
diff --git a/arch/x86/include/asm/fixmap.h b/arch/x86/include/asm/fixmap.h
index b65155c..d9ff226 100644
--- a/arch/x86/include/asm/fixmap.h
+++ b/arch/x86/include/asm/fixmap.h
@@ -157,6 +157,13 @@ static inline void __set_fixmap(enum fixed_addresses idx,
 }
 #endif
 
+/*
+ * FIXMAP_PAGE_NOCACHE is used for MMIO. Memory encryption is not
+ * supported for MMIO addresses, so make sure that the memory encryption
+ * mask is not part of the page attributes.
+ */
+#define FIXMAP_PAGE_NOCACHE PAGE_KERNEL_IO_NOCACHE
+
 #include <asm-generic/fixmap.h>
 
 #define __late_set_fixmap(idx, phys, flags) __set_fixmap(idx, phys, flags)
diff --git a/arch/x86/include/asm/mem_encrypt.h b/arch/x86/include/asm/mem_encrypt.h
index 5008fd9..f1c4c29 100644
--- a/arch/x86/include/asm/mem_encrypt.h
+++ b/arch/x86/include/asm/mem_encrypt.h
@@ -15,14 +15,22 @@
 
 #ifndef __ASSEMBLY__
 
+#include <linux/init.h>
+
 #ifdef CONFIG_AMD_MEM_ENCRYPT
 
 extern unsigned long sme_me_mask;
 
+void __init sme_early_init(void);
+
 #else	/* !CONFIG_AMD_MEM_ENCRYPT */
 
 #define sme_me_mask	0UL
 
+static inline void __init sme_early_init(void)
+{
+}
+
 #endif	/* CONFIG_AMD_MEM_ENCRYPT */
 
 static inline bool sme_active(void)
@@ -30,6 +38,23 @@ static inline bool sme_active(void)
 	return !!sme_me_mask;
 }
 
+/*
+ * The __sme_pa() and __sme_pa_nodebug() macros are meant for use when
+ * writing to or comparing values from the cr3 register.  Having the
+ * encryption mask set in cr3 enables the PGD entry to be encrypted and
+ * avoid special case handling of PGD allocations.
+ */
+#define __sme_pa(x)		(__pa(x) | sme_me_mask)
+#define __sme_pa_nodebug(x)	(__pa_nodebug(x) | sme_me_mask)
+
+/*
+ * The __sme_set() and __sme_clr() macros are useful for adding or removing
+ * the encryption mask from a value (e.g. when dealing with pagetable
+ * entries).
+ */
+#define __sme_set(x)		((unsigned long)(x) | sme_me_mask)
+#define __sme_clr(x)		((unsigned long)(x) & ~sme_me_mask)
+
 #endif	/* __ASSEMBLY__ */
 
 #endif	/* __X86_MEM_ENCRYPT_H__ */
diff --git a/arch/x86/include/asm/page_types.h b/arch/x86/include/asm/page_types.h
index 7bd0099..fead0a5 100644
--- a/arch/x86/include/asm/page_types.h
+++ b/arch/x86/include/asm/page_types.h
@@ -15,7 +15,7 @@
 #define PUD_PAGE_SIZE		(_AC(1, UL) << PUD_SHIFT)
 #define PUD_PAGE_MASK		(~(PUD_PAGE_SIZE-1))
 
-#define __PHYSICAL_MASK		((phys_addr_t)((1ULL << __PHYSICAL_MASK_SHIFT) - 1))
+#define __PHYSICAL_MASK		((phys_addr_t)(__sme_clr((1ULL << __PHYSICAL_MASK_SHIFT) - 1)))
 #define __VIRTUAL_MASK		((1UL << __VIRTUAL_MASK_SHIFT) - 1)
 
 /* Cast *PAGE_MASK to a signed type so that it is sign-extended if
diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h
index 96b6b83..3f789ec 100644
--- a/arch/x86/include/asm/pgtable.h
+++ b/arch/x86/include/asm/pgtable.h
@@ -3,6 +3,7 @@
 
 #include <asm/page.h>
 #include <asm/pgtable_types.h>
+#include <asm/mem_encrypt.h>
 
 /*
  * Macro to mark a page protection value as UC-
@@ -13,6 +14,12 @@
 		     cachemode2protval(_PAGE_CACHE_MODE_UC_MINUS)))	\
 	 : (prot))
 
+/*
+ * Macros to add or remove encryption attribute
+ */
+#define pgprot_encrypted(prot)	__pgprot(__sme_set(pgprot_val(prot)))
+#define pgprot_decrypted(prot)	__pgprot(__sme_clr(pgprot_val(prot)))
+
 #ifndef __ASSEMBLY__
 #include <asm/x86_init.h>
 
@@ -38,6 +45,8 @@
 
 extern struct mm_struct *pgd_page_get_mm(struct page *page);
 
+extern pmdval_t early_pmd_flags;
+
 #ifdef CONFIG_PARAVIRT
 #include <asm/paravirt.h>
 #else  /* !CONFIG_PARAVIRT */
diff --git a/arch/x86/include/asm/pgtable_types.h b/arch/x86/include/asm/pgtable_types.h
index bf9638e..d3ae99c 100644
--- a/arch/x86/include/asm/pgtable_types.h
+++ b/arch/x86/include/asm/pgtable_types.h
@@ -2,7 +2,9 @@
 #define _ASM_X86_PGTABLE_DEFS_H
 
 #include <linux/const.h>
+
 #include <asm/page_types.h>
+#include <asm/mem_encrypt.h>
 
 #define FIRST_USER_ADDRESS	0UL
 
@@ -121,10 +123,10 @@
 
 #define _PAGE_PROTNONE	(_AT(pteval_t, 1) << _PAGE_BIT_PROTNONE)
 
-#define _PAGE_TABLE	(_PAGE_PRESENT | _PAGE_RW | _PAGE_USER |	\
-			 _PAGE_ACCESSED | _PAGE_DIRTY)
-#define _KERNPG_TABLE	(_PAGE_PRESENT | _PAGE_RW | _PAGE_ACCESSED |	\
-			 _PAGE_DIRTY)
+#define _PAGE_TABLE_NOENC	(_PAGE_PRESENT | _PAGE_RW | _PAGE_USER |\
+				 _PAGE_ACCESSED | _PAGE_DIRTY)
+#define _KERNPG_TABLE_NOENC	(_PAGE_PRESENT | _PAGE_RW |		\
+				 _PAGE_ACCESSED | _PAGE_DIRTY)
 
 /*
  * Set of bits not changed in pte_modify.  The pte's
@@ -191,18 +193,29 @@ enum page_cache_mode {
 #define __PAGE_KERNEL_IO		(__PAGE_KERNEL)
 #define __PAGE_KERNEL_IO_NOCACHE	(__PAGE_KERNEL_NOCACHE)
 
-#define PAGE_KERNEL			__pgprot(__PAGE_KERNEL)
-#define PAGE_KERNEL_RO			__pgprot(__PAGE_KERNEL_RO)
-#define PAGE_KERNEL_EXEC		__pgprot(__PAGE_KERNEL_EXEC)
-#define PAGE_KERNEL_RX			__pgprot(__PAGE_KERNEL_RX)
-#define PAGE_KERNEL_NOCACHE		__pgprot(__PAGE_KERNEL_NOCACHE)
-#define PAGE_KERNEL_LARGE		__pgprot(__PAGE_KERNEL_LARGE)
-#define PAGE_KERNEL_LARGE_EXEC		__pgprot(__PAGE_KERNEL_LARGE_EXEC)
-#define PAGE_KERNEL_VSYSCALL		__pgprot(__PAGE_KERNEL_VSYSCALL)
-#define PAGE_KERNEL_VVAR		__pgprot(__PAGE_KERNEL_VVAR)
-
-#define PAGE_KERNEL_IO			__pgprot(__PAGE_KERNEL_IO)
-#define PAGE_KERNEL_IO_NOCACHE		__pgprot(__PAGE_KERNEL_IO_NOCACHE)
+#ifndef __ASSEMBLY__
+
+#define _PAGE_ENC	(_AT(pteval_t, sme_me_mask))
+
+#define _PAGE_TABLE	(_PAGE_PRESENT | _PAGE_RW | _PAGE_USER |	\
+			 _PAGE_ACCESSED | _PAGE_DIRTY | _PAGE_ENC)
+#define _KERNPG_TABLE	(_PAGE_PRESENT | _PAGE_RW | _PAGE_ACCESSED |	\
+			 _PAGE_DIRTY | _PAGE_ENC)
+
+#define PAGE_KERNEL		__pgprot(__PAGE_KERNEL | _PAGE_ENC)
+#define PAGE_KERNEL_RO		__pgprot(__PAGE_KERNEL_RO | _PAGE_ENC)
+#define PAGE_KERNEL_EXEC	__pgprot(__PAGE_KERNEL_EXEC | _PAGE_ENC)
+#define PAGE_KERNEL_RX		__pgprot(__PAGE_KERNEL_RX | _PAGE_ENC)
+#define PAGE_KERNEL_NOCACHE	__pgprot(__PAGE_KERNEL_NOCACHE | _PAGE_ENC)
+#define PAGE_KERNEL_LARGE	__pgprot(__PAGE_KERNEL_LARGE | _PAGE_ENC)
+#define PAGE_KERNEL_LARGE_EXEC	__pgprot(__PAGE_KERNEL_LARGE_EXEC | _PAGE_ENC)
+#define PAGE_KERNEL_VSYSCALL	__pgprot(__PAGE_KERNEL_VSYSCALL | _PAGE_ENC)
+#define PAGE_KERNEL_VVAR	__pgprot(__PAGE_KERNEL_VVAR | _PAGE_ENC)
+
+#define PAGE_KERNEL_IO		__pgprot(__PAGE_KERNEL_IO)
+#define PAGE_KERNEL_IO_NOCACHE	__pgprot(__PAGE_KERNEL_IO_NOCACHE)
+
+#endif	/* __ASSEMBLY__ */
 
 /*         xwr */
 #define __P000	PAGE_NONE
diff --git a/arch/x86/include/asm/processor.h b/arch/x86/include/asm/processor.h
index 3cada99..61e055d 100644
--- a/arch/x86/include/asm/processor.h
+++ b/arch/x86/include/asm/processor.h
@@ -22,6 +22,7 @@
 #include <asm/nops.h>
 #include <asm/special_insns.h>
 #include <asm/fpu/types.h>
+#include <asm/mem_encrypt.h>
 
 #include <linux/personality.h>
 #include <linux/cache.h>
@@ -233,7 +234,7 @@ static inline void native_cpuid(unsigned int *eax, unsigned int *ebx,
 
 static inline void load_cr3(pgd_t *pgdir)
 {
-	write_cr3(__pa(pgdir));
+	write_cr3(__sme_pa(pgdir));
 }
 
 #ifdef CONFIG_X86_32
diff --git a/arch/x86/kernel/espfix_64.c b/arch/x86/kernel/espfix_64.c
index 8e598a1..0955ec7 100644
--- a/arch/x86/kernel/espfix_64.c
+++ b/arch/x86/kernel/espfix_64.c
@@ -195,7 +195,7 @@ void init_espfix_ap(int cpu)
 
 	pte_p = pte_offset_kernel(&pmd, addr);
 	stack_page = page_address(alloc_pages_node(node, GFP_KERNEL, 0));
-	pte = __pte(__pa(stack_page) | (__PAGE_KERNEL_RO & ptemask));
+	pte = __pte(__pa(stack_page) | ((__PAGE_KERNEL_RO | _PAGE_ENC) & ptemask));
 	for (n = 0; n < ESPFIX_PTE_CLONES; n++)
 		set_pte(&pte_p[n*PTE_STRIDE], pte);
 
diff --git a/arch/x86/kernel/head64.c b/arch/x86/kernel/head64.c
index dc03624..00ae2c5 100644
--- a/arch/x86/kernel/head64.c
+++ b/arch/x86/kernel/head64.c
@@ -29,6 +29,7 @@
 #include <asm/bootparam_utils.h>
 #include <asm/microcode.h>
 #include <asm/kasan.h>
+#include <asm/mem_encrypt.h>
 
 /*
  * Manage page tables very early on.
@@ -43,7 +44,7 @@ static void __init reset_early_page_tables(void)
 {
 	memset(early_level4_pgt, 0, sizeof(pgd_t)*(PTRS_PER_PGD-1));
 	next_early_pgt = 0;
-	write_cr3(__pa_nodebug(early_level4_pgt));
+	write_cr3(__sme_pa_nodebug(early_level4_pgt));
 }
 
 /* Create a new PMD entry */
@@ -158,6 +159,13 @@ asmlinkage __visible void __init x86_64_start_kernel(char * real_mode_data)
 
 	clear_page(init_level4_pgt);
 
+	/*
+	 * SME support may update early_pmd_flags to include the memory
+	 * encryption mask, so it needs to be called before anything
+	 * that may generate a page fault.
+	 */
+	sme_early_init();
+
 	kasan_early_init();
 
 	for (i = 0; i < NUM_EXCEPTION_VECTORS; i++)
diff --git a/arch/x86/kernel/head_64.S b/arch/x86/kernel/head_64.S
index 222630c..1fe944b 100644
--- a/arch/x86/kernel/head_64.S
+++ b/arch/x86/kernel/head_64.S
@@ -129,7 +129,7 @@ startup_64:
 	movq	%rdi, %rax
 	shrq	$PGDIR_SHIFT, %rax
 
-	leaq	(PAGE_SIZE + _KERNPG_TABLE)(%rbx), %rdx
+	leaq	(PAGE_SIZE + _KERNPG_TABLE_NOENC)(%rbx), %rdx
 	addq	%r12, %rdx
 	movq	%rdx, 0(%rbx,%rax,8)
 	movq	%rdx, 8(%rbx,%rax,8)
@@ -476,7 +476,7 @@ GLOBAL(name)
 	__INITDATA
 NEXT_PAGE(early_level4_pgt)
 	.fill	511,8,0
-	.quad	level3_kernel_pgt - __START_KERNEL_map + _PAGE_TABLE
+	.quad	level3_kernel_pgt - __START_KERNEL_map + _PAGE_TABLE_NOENC
 
 NEXT_PAGE(early_dynamic_pgts)
 	.fill	512*EARLY_DYNAMIC_PAGE_TABLES,8,0
@@ -488,15 +488,15 @@ NEXT_PAGE(init_level4_pgt)
 	.fill	512,8,0
 #else
 NEXT_PAGE(init_level4_pgt)
-	.quad   level3_ident_pgt - __START_KERNEL_map + _KERNPG_TABLE
+	.quad   level3_ident_pgt - __START_KERNEL_map + _KERNPG_TABLE_NOENC
 	.org    init_level4_pgt + L4_PAGE_OFFSET*8, 0
-	.quad   level3_ident_pgt - __START_KERNEL_map + _KERNPG_TABLE
+	.quad   level3_ident_pgt - __START_KERNEL_map + _KERNPG_TABLE_NOENC
 	.org    init_level4_pgt + L4_START_KERNEL*8, 0
 	/* (2^48-(2*1024*1024*1024))/(2^39) = 511 */
-	.quad   level3_kernel_pgt - __START_KERNEL_map + _PAGE_TABLE
+	.quad   level3_kernel_pgt - __START_KERNEL_map + _PAGE_TABLE_NOENC
 
 NEXT_PAGE(level3_ident_pgt)
-	.quad	level2_ident_pgt - __START_KERNEL_map + _KERNPG_TABLE
+	.quad	level2_ident_pgt - __START_KERNEL_map + _KERNPG_TABLE_NOENC
 	.fill	511, 8, 0
 NEXT_PAGE(level2_ident_pgt)
 	/* Since I easily can, map the first 1G.
@@ -508,8 +508,8 @@ NEXT_PAGE(level2_ident_pgt)
 NEXT_PAGE(level3_kernel_pgt)
 	.fill	L3_START_KERNEL,8,0
 	/* (2^48-(2*1024*1024*1024)-((2^39)*511))/(2^30) = 510 */
-	.quad	level2_kernel_pgt - __START_KERNEL_map + _KERNPG_TABLE
-	.quad	level2_fixmap_pgt - __START_KERNEL_map + _PAGE_TABLE
+	.quad	level2_kernel_pgt - __START_KERNEL_map + _KERNPG_TABLE_NOENC
+	.quad	level2_fixmap_pgt - __START_KERNEL_map + _PAGE_TABLE_NOENC
 
 NEXT_PAGE(level2_kernel_pgt)
 	/*
@@ -527,7 +527,7 @@ NEXT_PAGE(level2_kernel_pgt)
 
 NEXT_PAGE(level2_fixmap_pgt)
 	.fill	506,8,0
-	.quad	level1_fixmap_pgt - __START_KERNEL_map + _PAGE_TABLE
+	.quad	level1_fixmap_pgt - __START_KERNEL_map + _PAGE_TABLE_NOENC
 	/* 8MB reserved for vsyscalls + a 2MB hole = 4 + 1 entries */
 	.fill	5,8,0
 
diff --git a/arch/x86/mm/kasan_init_64.c b/arch/x86/mm/kasan_init_64.c
index 0c7d812..6f1837a 100644
--- a/arch/x86/mm/kasan_init_64.c
+++ b/arch/x86/mm/kasan_init_64.c
@@ -92,7 +92,7 @@ static int kasan_die_handler(struct notifier_block *self,
 void __init kasan_early_init(void)
 {
 	int i;
-	pteval_t pte_val = __pa_nodebug(kasan_zero_page) | __PAGE_KERNEL;
+	pteval_t pte_val = __pa_nodebug(kasan_zero_page) | __PAGE_KERNEL | _PAGE_ENC;
 	pmdval_t pmd_val = __pa_nodebug(kasan_zero_pte) | _KERNPG_TABLE;
 	pudval_t pud_val = __pa_nodebug(kasan_zero_pmd) | _KERNPG_TABLE;
 	p4dval_t p4d_val = __pa_nodebug(kasan_zero_pud) | _KERNPG_TABLE;
@@ -158,7 +158,7 @@ void __init kasan_init(void)
 	 */
 	memset(kasan_zero_page, 0, PAGE_SIZE);
 	for (i = 0; i < PTRS_PER_PTE; i++) {
-		pte_t pte = __pte(__pa(kasan_zero_page) | __PAGE_KERNEL_RO);
+		pte_t pte = __pte(__pa(kasan_zero_page) | __PAGE_KERNEL_RO | _PAGE_ENC);
 		set_pte(&kasan_zero_pte[i], pte);
 	}
 	/* Flush TLBs again to be sure that write protection applied. */
diff --git a/arch/x86/mm/mem_encrypt.c b/arch/x86/mm/mem_encrypt.c
index cc00d8b..8ca93e5 100644
--- a/arch/x86/mm/mem_encrypt.c
+++ b/arch/x86/mm/mem_encrypt.c
@@ -15,6 +15,8 @@
 
 #ifdef CONFIG_AMD_MEM_ENCRYPT
 
+#include <linux/mm.h>
+
 /*
  * Since SME related variables are set early in the boot process they must
  * reside in the .data section so as not to be zeroed out when the .bss
@@ -23,6 +25,22 @@
 unsigned long sme_me_mask __section(.data) = 0;
 EXPORT_SYMBOL_GPL(sme_me_mask);
 
+void __init sme_early_init(void)
+{
+	unsigned int i;
+
+	if (!sme_me_mask)
+		return;
+
+	early_pmd_flags = __sme_set(early_pmd_flags);
+
+	__supported_pte_mask = __sme_set(__supported_pte_mask);
+
+	/* Update the protection map with memory encryption mask */
+	for (i = 0; i < ARRAY_SIZE(protection_map); i++)
+		protection_map[i] = pgprot_encrypted(protection_map[i]);
+}
+
 void __init sme_encrypt_kernel(void)
 {
 }
diff --git a/arch/x86/mm/pageattr.c b/arch/x86/mm/pageattr.c
index c8520b2..e7d3866 100644
--- a/arch/x86/mm/pageattr.c
+++ b/arch/x86/mm/pageattr.c
@@ -2014,6 +2014,9 @@ int kernel_map_pages_in_pgd(pgd_t *pgd, u64 pfn, unsigned long address,
 	if (!(page_flags & _PAGE_RW))
 		cpa.mask_clr = __pgprot(_PAGE_RW);
 
+	if (!(page_flags & _PAGE_ENC))
+		cpa.mask_clr = pgprot_encrypted(cpa.mask_clr);
+
 	cpa.mask_set = __pgprot(_PAGE_PRESENT | page_flags);
 
 	retval = __change_page_attr_set_clr(&cpa, 0);
diff --git a/include/asm-generic/mem_encrypt.h b/include/asm-generic/mem_encrypt.h
index 563c918..b55c3f9 100644
--- a/include/asm-generic/mem_encrypt.h
+++ b/include/asm-generic/mem_encrypt.h
@@ -22,6 +22,14 @@ static inline bool sme_active(void)
 	return false;
 }
 
+/*
+ * The __sme_set() and __sme_clr() macros are useful for adding or removing
+ * the encryption mask from a value (e.g. when dealing with pagetable
+ * entries).
+ */
+#define __sme_set(x)		(x)
+#define __sme_clr(x)		(x)
+
 #endif	/* __ASSEMBLY__ */
 
 #endif	/* __MEM_ENCRYPT_H__ */
diff --git a/include/asm-generic/pgtable.h b/include/asm-generic/pgtable.h
index 7dfa767..882cb5d 100644
--- a/include/asm-generic/pgtable.h
+++ b/include/asm-generic/pgtable.h
@@ -424,6 +424,14 @@ static inline int pud_same(pud_t pud_a, pud_t pud_b)
 #define pgprot_device pgprot_noncached
 #endif
 
+#ifndef pgprot_encrypted
+#define pgprot_encrypted(prot)	(prot)
+#endif
+
+#ifndef pgprot_decrypted
+#define pgprot_decrypted(prot)	(prot)
+#endif
+
 #ifndef pgprot_modify
 #define pgprot_modify pgprot_modify
 static inline pgprot_t pgprot_modify(pgprot_t oldprot, pgprot_t newprot)



From thomas.lendacky at amd.com  Wed Jun  7 12:14:53 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Wed, 07 Jun 2017 14:14:53 -0500
Subject: [PATCH v6 10/34] x86, x86/mm, x86/xen,
 olpc: Use __va() against just the physical address in cr3
In-Reply-To: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170607191453.28645.92256.stgit@tlendack-t1.amdoffice.net>

The cr3 register entry can contain the SME encryption bit that indicates
the PGD is encrypted.  The encryption bit should not be used when creating
a virtual address for the PGD table.

Create a new function, read_cr3_pa(), that will extract the physical
address from the cr3 register. This function is then used where a virtual
address of the PGD needs to be created/used from the cr3 register.

Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/include/asm/special_insns.h |    9 +++++++++
 arch/x86/kernel/head64.c             |    2 +-
 arch/x86/mm/fault.c                  |   10 +++++-----
 arch/x86/mm/ioremap.c                |    2 +-
 arch/x86/platform/olpc/olpc-xo1-pm.c |    2 +-
 arch/x86/power/hibernate_64.c        |    2 +-
 arch/x86/xen/mmu_pv.c                |    6 +++---
 7 files changed, 21 insertions(+), 12 deletions(-)

diff --git a/arch/x86/include/asm/special_insns.h b/arch/x86/include/asm/special_insns.h
index 12af3e3..d8e8ace 100644
--- a/arch/x86/include/asm/special_insns.h
+++ b/arch/x86/include/asm/special_insns.h
@@ -234,6 +234,15 @@ static inline void clwb(volatile void *__p)
 
 #define nop() asm volatile ("nop")
 
+static inline unsigned long native_read_cr3_pa(void)
+{
+	return (native_read_cr3() & PHYSICAL_PAGE_MASK);
+}
+
+static inline unsigned long read_cr3_pa(void)
+{
+	return (read_cr3() & PHYSICAL_PAGE_MASK);
+}
 
 #endif /* __KERNEL__ */
 
diff --git a/arch/x86/kernel/head64.c b/arch/x86/kernel/head64.c
index 43b7002..dc03624 100644
--- a/arch/x86/kernel/head64.c
+++ b/arch/x86/kernel/head64.c
@@ -55,7 +55,7 @@ int __init early_make_pgtable(unsigned long address)
 	pmdval_t pmd, *pmd_p;
 
 	/* Invalid address or early pgt is done ?  */
-	if (physaddr >= MAXMEM || read_cr3() != __pa_nodebug(early_level4_pgt))
+	if (physaddr >= MAXMEM || read_cr3_pa() != __pa_nodebug(early_level4_pgt))
 		return -1;
 
 again:
diff --git a/arch/x86/mm/fault.c b/arch/x86/mm/fault.c
index 8ad91a0..2a1fa10c 100644
--- a/arch/x86/mm/fault.c
+++ b/arch/x86/mm/fault.c
@@ -346,7 +346,7 @@ static noinline int vmalloc_fault(unsigned long address)
 	 * Do _not_ use "current" here. We might be inside
 	 * an interrupt in the middle of a task switch..
 	 */
-	pgd_paddr = read_cr3();
+	pgd_paddr = read_cr3_pa();
 	pmd_k = vmalloc_sync_one(__va(pgd_paddr), address);
 	if (!pmd_k)
 		return -1;
@@ -388,7 +388,7 @@ static bool low_pfn(unsigned long pfn)
 
 static void dump_pagetable(unsigned long address)
 {
-	pgd_t *base = __va(read_cr3());
+	pgd_t *base = __va(read_cr3_pa());
 	pgd_t *pgd = &base[pgd_index(address)];
 	p4d_t *p4d;
 	pud_t *pud;
@@ -451,7 +451,7 @@ static noinline int vmalloc_fault(unsigned long address)
 	 * happen within a race in page table update. In the later
 	 * case just flush:
 	 */
-	pgd = (pgd_t *)__va(read_cr3()) + pgd_index(address);
+	pgd = (pgd_t *)__va(read_cr3_pa()) + pgd_index(address);
 	pgd_ref = pgd_offset_k(address);
 	if (pgd_none(*pgd_ref))
 		return -1;
@@ -555,7 +555,7 @@ static int bad_address(void *p)
 
 static void dump_pagetable(unsigned long address)
 {
-	pgd_t *base = __va(read_cr3() & PHYSICAL_PAGE_MASK);
+	pgd_t *base = __va(read_cr3_pa());
 	pgd_t *pgd = base + pgd_index(address);
 	p4d_t *p4d;
 	pud_t *pud;
@@ -700,7 +700,7 @@ static int is_f00f_bug(struct pt_regs *regs, unsigned long address)
 		pgd_t *pgd;
 		pte_t *pte;
 
-		pgd = __va(read_cr3() & PHYSICAL_PAGE_MASK);
+		pgd = __va(read_cr3_pa());
 		pgd += pgd_index(address);
 
 		pte = lookup_address_in_pgd(pgd, address, &level);
diff --git a/arch/x86/mm/ioremap.c b/arch/x86/mm/ioremap.c
index 2a0fa89..e6305dd 100644
--- a/arch/x86/mm/ioremap.c
+++ b/arch/x86/mm/ioremap.c
@@ -427,7 +427,7 @@ void unxlate_dev_mem_ptr(phys_addr_t phys, void *addr)
 static inline pmd_t * __init early_ioremap_pmd(unsigned long addr)
 {
 	/* Don't assume we're using swapper_pg_dir at this point */
-	pgd_t *base = __va(read_cr3());
+	pgd_t *base = __va(read_cr3_pa());
 	pgd_t *pgd = &base[pgd_index(addr)];
 	p4d_t *p4d = p4d_offset(pgd, addr);
 	pud_t *pud = pud_offset(p4d, addr);
diff --git a/arch/x86/platform/olpc/olpc-xo1-pm.c b/arch/x86/platform/olpc/olpc-xo1-pm.c
index c5350fd..0668aaf 100644
--- a/arch/x86/platform/olpc/olpc-xo1-pm.c
+++ b/arch/x86/platform/olpc/olpc-xo1-pm.c
@@ -77,7 +77,7 @@ static int xo1_power_state_enter(suspend_state_t pm_state)
 
 asmlinkage __visible int xo1_do_sleep(u8 sleep_state)
 {
-	void *pgd_addr = __va(read_cr3());
+	void *pgd_addr = __va(read_cr3_pa());
 
 	/* Program wakeup mask (using dword access to CS5536_PM1_EN) */
 	outl(wakeup_mask << 16, acpi_base + CS5536_PM1_STS);
diff --git a/arch/x86/power/hibernate_64.c b/arch/x86/power/hibernate_64.c
index a6e21fe..0a7650d 100644
--- a/arch/x86/power/hibernate_64.c
+++ b/arch/x86/power/hibernate_64.c
@@ -150,7 +150,7 @@ static int relocate_restore_code(void)
 	memcpy((void *)relocated_restore_code, &core_restore_code, PAGE_SIZE);
 
 	/* Make the page containing the relocated code executable */
-	pgd = (pgd_t *)__va(read_cr3()) + pgd_index(relocated_restore_code);
+	pgd = (pgd_t *)__va(read_cr3_pa()) + pgd_index(relocated_restore_code);
 	p4d = p4d_offset(pgd, relocated_restore_code);
 	if (p4d_large(*p4d)) {
 		set_p4d(p4d, __p4d(p4d_val(*p4d) & ~_PAGE_NX));
diff --git a/arch/x86/xen/mmu_pv.c b/arch/x86/xen/mmu_pv.c
index 1f386d7..2dc5243 100644
--- a/arch/x86/xen/mmu_pv.c
+++ b/arch/x86/xen/mmu_pv.c
@@ -2022,7 +2022,7 @@ static phys_addr_t __init xen_early_virt_to_phys(unsigned long vaddr)
 	pmd_t pmd;
 	pte_t pte;
 
-	pa = read_cr3();
+	pa = read_cr3_pa();
 	pgd = native_make_pgd(xen_read_phys_ulong(pa + pgd_index(vaddr) *
 						       sizeof(pgd)));
 	if (!pgd_present(pgd))
@@ -2102,7 +2102,7 @@ void __init xen_relocate_p2m(void)
 	pt_phys = pmd_phys + PFN_PHYS(n_pmd);
 	p2m_pfn = PFN_DOWN(pt_phys) + n_pt;
 
-	pgd = __va(read_cr3());
+	pgd = __va(read_cr3_pa());
 	new_p2m = (unsigned long *)(2 * PGDIR_SIZE);
 	idx_p4d = 0;
 	save_pud = n_pud;
@@ -2209,7 +2209,7 @@ static void __init xen_write_cr3_init(unsigned long cr3)
 {
 	unsigned long pfn = PFN_DOWN(__pa(swapper_pg_dir));
 
-	BUG_ON(read_cr3() != __pa(initial_page_table));
+	BUG_ON(read_cr3_pa() != __pa(initial_page_table));
 	BUG_ON(cr3 != __pa(swapper_pg_dir));
 
 	/*



From thomas.lendacky at amd.com  Wed Jun  7 12:15:16 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Wed, 07 Jun 2017 14:15:16 -0500
Subject: [PATCH v6 12/34] x86/mm: Extend early_memremap() support with
 additional attrs
In-Reply-To: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170607191516.28645.12130.stgit@tlendack-t1.amdoffice.net>

Add early_memremap() support to be able to specify encrypted and
decrypted mappings with and without write-protection. The use of
write-protection is necessary when encrypting data "in place". The
write-protect attribute is considered cacheable for loads, but not
stores. This implies that the hardware will never give the core a
dirty line with this memtype.

Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/Kconfig                     |    4 +++
 arch/x86/include/asm/fixmap.h        |   13 ++++++++++
 arch/x86/include/asm/pgtable_types.h |    8 ++++++
 arch/x86/mm/ioremap.c                |   44 ++++++++++++++++++++++++++++++++++
 include/asm-generic/early_ioremap.h  |    2 ++
 mm/early_ioremap.c                   |   10 ++++++++
 6 files changed, 81 insertions(+)

diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index 11f2fdb..8002530 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -1429,6 +1429,10 @@ config AMD_MEM_ENCRYPT_ACTIVE_BY_DEFAULT
 	  If set to N, then the encryption of system memory can be
 	  activated with the mem_encrypt=on command line option.
 
+config ARCH_USE_MEMREMAP_PROT
+	def_bool y
+	depends on AMD_MEM_ENCRYPT
+
 # Common NUMA Features
 config NUMA
 	bool "Numa Memory Allocation and Scheduler Support"
diff --git a/arch/x86/include/asm/fixmap.h b/arch/x86/include/asm/fixmap.h
index d9ff226..dcd9fb5 100644
--- a/arch/x86/include/asm/fixmap.h
+++ b/arch/x86/include/asm/fixmap.h
@@ -164,6 +164,19 @@ static inline void __set_fixmap(enum fixed_addresses idx,
  */
 #define FIXMAP_PAGE_NOCACHE PAGE_KERNEL_IO_NOCACHE
 
+/*
+ * Early memremap routines used for in-place encryption. The mappings created
+ * by these routines are intended to be used as temporary mappings.
+ */
+void __init *early_memremap_encrypted(resource_size_t phys_addr,
+				      unsigned long size);
+void __init *early_memremap_encrypted_wp(resource_size_t phys_addr,
+					 unsigned long size);
+void __init *early_memremap_decrypted(resource_size_t phys_addr,
+				      unsigned long size);
+void __init *early_memremap_decrypted_wp(resource_size_t phys_addr,
+					 unsigned long size);
+
 #include <asm-generic/fixmap.h>
 
 #define __late_set_fixmap(idx, phys, flags) __set_fixmap(idx, phys, flags)
diff --git a/arch/x86/include/asm/pgtable_types.h b/arch/x86/include/asm/pgtable_types.h
index d3ae99c..ce8cb1c 100644
--- a/arch/x86/include/asm/pgtable_types.h
+++ b/arch/x86/include/asm/pgtable_types.h
@@ -161,6 +161,7 @@ enum page_cache_mode {
 
 #define _PAGE_CACHE_MASK	(_PAGE_PAT | _PAGE_PCD | _PAGE_PWT)
 #define _PAGE_NOCACHE		(cachemode2protval(_PAGE_CACHE_MODE_UC))
+#define _PAGE_CACHE_WP		(cachemode2protval(_PAGE_CACHE_MODE_WP))
 
 #define PAGE_NONE	__pgprot(_PAGE_PROTNONE | _PAGE_ACCESSED)
 #define PAGE_SHARED	__pgprot(_PAGE_PRESENT | _PAGE_RW | _PAGE_USER | \
@@ -189,6 +190,7 @@ enum page_cache_mode {
 #define __PAGE_KERNEL_VVAR		(__PAGE_KERNEL_RO | _PAGE_USER)
 #define __PAGE_KERNEL_LARGE		(__PAGE_KERNEL | _PAGE_PSE)
 #define __PAGE_KERNEL_LARGE_EXEC	(__PAGE_KERNEL_EXEC | _PAGE_PSE)
+#define __PAGE_KERNEL_WP		(__PAGE_KERNEL | _PAGE_CACHE_WP)
 
 #define __PAGE_KERNEL_IO		(__PAGE_KERNEL)
 #define __PAGE_KERNEL_IO_NOCACHE	(__PAGE_KERNEL_NOCACHE)
@@ -202,6 +204,12 @@ enum page_cache_mode {
 #define _KERNPG_TABLE	(_PAGE_PRESENT | _PAGE_RW | _PAGE_ACCESSED |	\
 			 _PAGE_DIRTY | _PAGE_ENC)
 
+#define __PAGE_KERNEL_ENC	(__PAGE_KERNEL | _PAGE_ENC)
+#define __PAGE_KERNEL_ENC_WP	(__PAGE_KERNEL_WP | _PAGE_ENC)
+
+#define __PAGE_KERNEL_NOENC	(__PAGE_KERNEL)
+#define __PAGE_KERNEL_NOENC_WP	(__PAGE_KERNEL_WP)
+
 #define PAGE_KERNEL		__pgprot(__PAGE_KERNEL | _PAGE_ENC)
 #define PAGE_KERNEL_RO		__pgprot(__PAGE_KERNEL_RO | _PAGE_ENC)
 #define PAGE_KERNEL_EXEC	__pgprot(__PAGE_KERNEL_EXEC | _PAGE_ENC)
diff --git a/arch/x86/mm/ioremap.c b/arch/x86/mm/ioremap.c
index e6305dd..792db75 100644
--- a/arch/x86/mm/ioremap.c
+++ b/arch/x86/mm/ioremap.c
@@ -422,6 +422,50 @@ void unxlate_dev_mem_ptr(phys_addr_t phys, void *addr)
 	iounmap((void __iomem *)((unsigned long)addr & PAGE_MASK));
 }
 
+#ifdef CONFIG_ARCH_USE_MEMREMAP_PROT
+/* Remap memory with encryption */
+void __init *early_memremap_encrypted(resource_size_t phys_addr,
+				      unsigned long size)
+{
+	return early_memremap_prot(phys_addr, size, __PAGE_KERNEL_ENC);
+}
+
+/*
+ * Remap memory with encryption and write-protected - cannot be called
+ * before pat_init() is called
+ */
+void __init *early_memremap_encrypted_wp(resource_size_t phys_addr,
+					 unsigned long size)
+{
+	/* Be sure the write-protect PAT entry is set for write-protect */
+	if (__pte2cachemode_tbl[_PAGE_CACHE_MODE_WP] != _PAGE_CACHE_MODE_WP)
+		return NULL;
+
+	return early_memremap_prot(phys_addr, size, __PAGE_KERNEL_ENC_WP);
+}
+
+/* Remap memory without encryption */
+void __init *early_memremap_decrypted(resource_size_t phys_addr,
+				      unsigned long size)
+{
+	return early_memremap_prot(phys_addr, size, __PAGE_KERNEL_NOENC);
+}
+
+/*
+ * Remap memory without encryption and write-protected - cannot be called
+ * before pat_init() is called
+ */
+void __init *early_memremap_decrypted_wp(resource_size_t phys_addr,
+					 unsigned long size)
+{
+	/* Be sure the write-protect PAT entry is set for write-protect */
+	if (__pte2cachemode_tbl[_PAGE_CACHE_MODE_WP] != _PAGE_CACHE_MODE_WP)
+		return NULL;
+
+	return early_memremap_prot(phys_addr, size, __PAGE_KERNEL_NOENC_WP);
+}
+#endif	/* CONFIG_ARCH_USE_MEMREMAP_PROT */
+
 static pte_t bm_pte[PAGE_SIZE/sizeof(pte_t)] __page_aligned_bss;
 
 static inline pmd_t * __init early_ioremap_pmd(unsigned long addr)
diff --git a/include/asm-generic/early_ioremap.h b/include/asm-generic/early_ioremap.h
index 734ad4d..2edef8d 100644
--- a/include/asm-generic/early_ioremap.h
+++ b/include/asm-generic/early_ioremap.h
@@ -13,6 +13,8 @@ extern void *early_memremap(resource_size_t phys_addr,
 			    unsigned long size);
 extern void *early_memremap_ro(resource_size_t phys_addr,
 			       unsigned long size);
+extern void *early_memremap_prot(resource_size_t phys_addr,
+				 unsigned long size, unsigned long prot_val);
 extern void early_iounmap(void __iomem *addr, unsigned long size);
 extern void early_memunmap(void *addr, unsigned long size);
 
diff --git a/mm/early_ioremap.c b/mm/early_ioremap.c
index 6d5717b..d7d30da 100644
--- a/mm/early_ioremap.c
+++ b/mm/early_ioremap.c
@@ -226,6 +226,16 @@ void __init early_iounmap(void __iomem *addr, unsigned long size)
 }
 #endif
 
+#ifdef CONFIG_ARCH_USE_MEMREMAP_PROT
+void __init *
+early_memremap_prot(resource_size_t phys_addr, unsigned long size,
+		    unsigned long prot_val)
+{
+	return (__force void *)__early_ioremap(phys_addr, size,
+					       __pgprot(prot_val));
+}
+#endif
+
 #define MAX_MAP_CHUNK	(NR_FIX_BTMAPS << PAGE_SHIFT)
 
 void __init copy_from_early_mem(void *dest, phys_addr_t src, unsigned long size)



From thomas.lendacky at amd.com  Wed Jun  7 12:15:27 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Wed, 07 Jun 2017 14:15:27 -0500
Subject: [PATCH v6 13/34] x86/mm: Add support for early encrypt/decrypt of
 memory
In-Reply-To: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170607191527.28645.84593.stgit@tlendack-t1.amdoffice.net>

Add support to be able to either encrypt or decrypt data in place during
the early stages of booting the kernel. This does not change the memory
encryption attribute - it is used for ensuring that data present in either
an encrypted or decrypted memory area is in the proper state (for example
the initrd will have been loaded by the boot loader and will not be
encrypted, but the memory that it resides in is marked as encrypted).

Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/include/asm/mem_encrypt.h |   15 +++++++
 arch/x86/mm/mem_encrypt.c          |   76 ++++++++++++++++++++++++++++++++++++
 2 files changed, 91 insertions(+)

diff --git a/arch/x86/include/asm/mem_encrypt.h b/arch/x86/include/asm/mem_encrypt.h
index f1c4c29..7c395cf 100644
--- a/arch/x86/include/asm/mem_encrypt.h
+++ b/arch/x86/include/asm/mem_encrypt.h
@@ -21,12 +21,27 @@
 
 extern unsigned long sme_me_mask;
 
+void __init sme_early_encrypt(resource_size_t paddr,
+			      unsigned long size);
+void __init sme_early_decrypt(resource_size_t paddr,
+			      unsigned long size);
+
 void __init sme_early_init(void);
 
 #else	/* !CONFIG_AMD_MEM_ENCRYPT */
 
 #define sme_me_mask	0UL
 
+static inline void __init sme_early_encrypt(resource_size_t paddr,
+					    unsigned long size)
+{
+}
+
+static inline void __init sme_early_decrypt(resource_size_t paddr,
+					    unsigned long size)
+{
+}
+
 static inline void __init sme_early_init(void)
 {
 }
diff --git a/arch/x86/mm/mem_encrypt.c b/arch/x86/mm/mem_encrypt.c
index 8ca93e5..18c0887 100644
--- a/arch/x86/mm/mem_encrypt.c
+++ b/arch/x86/mm/mem_encrypt.c
@@ -17,6 +17,9 @@
 
 #include <linux/mm.h>
 
+#include <asm/tlbflush.h>
+#include <asm/fixmap.h>
+
 /*
  * Since SME related variables are set early in the boot process they must
  * reside in the .data section so as not to be zeroed out when the .bss
@@ -25,6 +28,79 @@
 unsigned long sme_me_mask __section(.data) = 0;
 EXPORT_SYMBOL_GPL(sme_me_mask);
 
+/* Buffer used for early in-place encryption by BSP, no locking needed */
+static char sme_early_buffer[PAGE_SIZE] __aligned(PAGE_SIZE);
+
+/*
+ * This routine does not change the underlying encryption setting of the
+ * page(s) that map this memory. It assumes that eventually the memory is
+ * meant to be accessed as either encrypted or decrypted but the contents
+ * are currently not in the desired state.
+ *
+ * This routine follows the steps outlined in the AMD64 Architecture
+ * Programmer's Manual Volume 2, Section 7.10.8 Encrypt-in-Place.
+ */
+static void __init __sme_early_enc_dec(resource_size_t paddr,
+				       unsigned long size, bool enc)
+{
+	void *src, *dst;
+	size_t len;
+
+	if (!sme_me_mask)
+		return;
+
+	local_flush_tlb();
+	wbinvd();
+
+	/*
+	 * There are limited number of early mapping slots, so map (at most)
+	 * one page at time.
+	 */
+	while (size) {
+		len = min_t(size_t, sizeof(sme_early_buffer), size);
+
+		/*
+		 * Create mappings for the current and desired format of
+		 * the memory. Use a write-protected mapping for the source.
+		 */
+		src = enc ? early_memremap_decrypted_wp(paddr, len) :
+			    early_memremap_encrypted_wp(paddr, len);
+
+		dst = enc ? early_memremap_encrypted(paddr, len) :
+			    early_memremap_decrypted(paddr, len);
+
+		/*
+		 * If a mapping can't be obtained to perform the operation,
+		 * then eventual access of that area in the desired mode
+		 * will cause a crash.
+		 */
+		BUG_ON(!src || !dst);
+
+		/*
+		 * Use a temporary buffer, of cache-line multiple size, to
+		 * avoid data corruption as documented in the APM.
+		 */
+		memcpy(sme_early_buffer, src, len);
+		memcpy(dst, sme_early_buffer, len);
+
+		early_memunmap(dst, len);
+		early_memunmap(src, len);
+
+		paddr += len;
+		size -= len;
+	}
+}
+
+void __init sme_early_encrypt(resource_size_t paddr, unsigned long size)
+{
+	__sme_early_enc_dec(paddr, size, true);
+}
+
+void __init sme_early_decrypt(resource_size_t paddr, unsigned long size)
+{
+	__sme_early_enc_dec(paddr, size, false);
+}
+
 void __init sme_early_init(void)
 {
 	unsigned int i;



From thomas.lendacky at amd.com  Wed Jun  7 12:15:39 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Wed, 07 Jun 2017 14:15:39 -0500
Subject: [PATCH v6 14/34] x86/mm: Insure that boot memory areas are mapped
 properly
In-Reply-To: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170607191539.28645.70161.stgit@tlendack-t1.amdoffice.net>

The boot data and command line data are present in memory in a decrypted
state and are copied early in the boot process.  The early page fault
support will map these areas as encrypted, so before attempting to copy
them, add decrypted mappings so the data is accessed properly when copied.

For the initrd, encrypt this data in place. Since the future mapping of the
initrd area will be mapped as encrypted the data will be accessed properly.

Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/include/asm/mem_encrypt.h |   11 +++++
 arch/x86/include/asm/pgtable.h     |    3 +
 arch/x86/kernel/head64.c           |   30 ++++++++++++--
 arch/x86/kernel/setup.c            |    9 ++++
 arch/x86/mm/mem_encrypt.c          |   77 ++++++++++++++++++++++++++++++++++++
 5 files changed, 126 insertions(+), 4 deletions(-)

diff --git a/arch/x86/include/asm/mem_encrypt.h b/arch/x86/include/asm/mem_encrypt.h
index 7c395cf..61a7049 100644
--- a/arch/x86/include/asm/mem_encrypt.h
+++ b/arch/x86/include/asm/mem_encrypt.h
@@ -26,6 +26,9 @@ void __init sme_early_encrypt(resource_size_t paddr,
 void __init sme_early_decrypt(resource_size_t paddr,
 			      unsigned long size);
 
+void __init sme_map_bootdata(char *real_mode_data);
+void __init sme_unmap_bootdata(char *real_mode_data);
+
 void __init sme_early_init(void);
 
 #else	/* !CONFIG_AMD_MEM_ENCRYPT */
@@ -42,6 +45,14 @@ static inline void __init sme_early_decrypt(resource_size_t paddr,
 {
 }
 
+static inline void __init sme_map_bootdata(char *real_mode_data)
+{
+}
+
+static inline void __init sme_unmap_bootdata(char *real_mode_data)
+{
+}
+
 static inline void __init sme_early_init(void)
 {
 }
diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h
index 3f789ec..16657e7 100644
--- a/arch/x86/include/asm/pgtable.h
+++ b/arch/x86/include/asm/pgtable.h
@@ -23,6 +23,9 @@
 #ifndef __ASSEMBLY__
 #include <asm/x86_init.h>
 
+extern pgd_t early_level4_pgt[PTRS_PER_PGD];
+int __init __early_make_pgtable(unsigned long address, pmdval_t pmd);
+
 void ptdump_walk_pgd_level(struct seq_file *m, pgd_t *pgd);
 void ptdump_walk_pgd_level_checkwx(void);
 
diff --git a/arch/x86/kernel/head64.c b/arch/x86/kernel/head64.c
index 00ae2c5..f1fe5df 100644
--- a/arch/x86/kernel/head64.c
+++ b/arch/x86/kernel/head64.c
@@ -34,7 +34,6 @@
 /*
  * Manage page tables very early on.
  */
-extern pgd_t early_level4_pgt[PTRS_PER_PGD];
 extern pmd_t early_dynamic_pgts[EARLY_DYNAMIC_PAGE_TABLES][PTRS_PER_PMD];
 static unsigned int __initdata next_early_pgt = 2;
 pmdval_t early_pmd_flags = __PAGE_KERNEL_LARGE & ~(_PAGE_GLOBAL | _PAGE_NX);
@@ -48,12 +47,12 @@ static void __init reset_early_page_tables(void)
 }
 
 /* Create a new PMD entry */
-int __init early_make_pgtable(unsigned long address)
+int __init __early_make_pgtable(unsigned long address, pmdval_t pmd)
 {
 	unsigned long physaddr = address - __PAGE_OFFSET;
 	pgdval_t pgd, *pgd_p;
 	pudval_t pud, *pud_p;
-	pmdval_t pmd, *pmd_p;
+	pmdval_t *pmd_p;
 
 	/* Invalid address or early pgt is done ?  */
 	if (physaddr >= MAXMEM || read_cr3_pa() != __pa_nodebug(early_level4_pgt))
@@ -95,12 +94,21 @@ int __init early_make_pgtable(unsigned long address)
 		memset(pmd_p, 0, sizeof(*pmd_p) * PTRS_PER_PMD);
 		*pud_p = (pudval_t)pmd_p - __START_KERNEL_map + phys_base + _KERNPG_TABLE;
 	}
-	pmd = (physaddr & PMD_MASK) + early_pmd_flags;
 	pmd_p[pmd_index(address)] = pmd;
 
 	return 0;
 }
 
+int __init early_make_pgtable(unsigned long address)
+{
+	unsigned long physaddr = address - __PAGE_OFFSET;
+	pmdval_t pmd;
+
+	pmd = (physaddr & PMD_MASK) + early_pmd_flags;
+
+	return __early_make_pgtable(address, pmd);
+}
+
 /* Don't add a printk in there. printk relies on the PDA which is not initialized 
    yet. */
 static void __init clear_bss(void)
@@ -123,6 +131,12 @@ static void __init copy_bootdata(char *real_mode_data)
 	char * command_line;
 	unsigned long cmd_line_ptr;
 
+	/*
+	 * If SME is active, this will create decrypted mappings of the
+	 * boot data in advance of the copy operations.
+	 */
+	sme_map_bootdata(real_mode_data);
+
 	memcpy(&boot_params, real_mode_data, sizeof boot_params);
 	sanitize_boot_params(&boot_params);
 	cmd_line_ptr = get_cmd_line_ptr();
@@ -130,6 +144,14 @@ static void __init copy_bootdata(char *real_mode_data)
 		command_line = __va(cmd_line_ptr);
 		memcpy(boot_command_line, command_line, COMMAND_LINE_SIZE);
 	}
+
+	/*
+	 * The old boot data is no longer needed and won't be reserved,
+	 * freeing up that memory for use by the system. If SME is active,
+	 * we need to remove the mappings that were created so that the
+	 * memory doesn't remain mapped as decrypted.
+	 */
+	sme_unmap_bootdata(real_mode_data);
 }
 
 asmlinkage __visible void __init x86_64_start_kernel(char * real_mode_data)
diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
index f818236..d1414a1 100644
--- a/arch/x86/kernel/setup.c
+++ b/arch/x86/kernel/setup.c
@@ -115,6 +115,7 @@
 #include <asm/microcode.h>
 #include <asm/mmu_context.h>
 #include <asm/kaslr.h>
+#include <asm/mem_encrypt.h>
 
 /*
  * max_low_pfn_mapped: highest direct mapped pfn under 4GB
@@ -374,6 +375,14 @@ static void __init reserve_initrd(void)
 	    !ramdisk_image || !ramdisk_size)
 		return;		/* No initrd provided by bootloader */
 
+	/*
+	 * If SME is active, this memory will be marked encrypted by the
+	 * kernel when it is accessed (including relocation). However, the
+	 * ramdisk image was loaded decrypted by the bootloader, so make
+	 * sure that it is encrypted before accessing it.
+	 */
+	sme_early_encrypt(ramdisk_image, ramdisk_end - ramdisk_image);
+
 	initrd_start = 0;
 
 	mapped_size = memblock_mem_size(max_pfn_mapped);
diff --git a/arch/x86/mm/mem_encrypt.c b/arch/x86/mm/mem_encrypt.c
index 18c0887..2321f05 100644
--- a/arch/x86/mm/mem_encrypt.c
+++ b/arch/x86/mm/mem_encrypt.c
@@ -19,6 +19,8 @@
 
 #include <asm/tlbflush.h>
 #include <asm/fixmap.h>
+#include <asm/setup.h>
+#include <asm/bootparam.h>
 
 /*
  * Since SME related variables are set early in the boot process they must
@@ -101,6 +103,81 @@ void __init sme_early_decrypt(resource_size_t paddr, unsigned long size)
 	__sme_early_enc_dec(paddr, size, false);
 }
 
+static void __init sme_early_pgtable_flush(void)
+{
+	write_cr3(__sme_pa_nodebug(early_level4_pgt));
+}
+
+static void __init __sme_early_map_unmap_mem(void *vaddr, unsigned long size,
+					     bool map)
+{
+	unsigned long paddr = (unsigned long)vaddr - __PAGE_OFFSET;
+	pmdval_t pmd_flags, pmd;
+
+	/* Use early_pmd_flags but remove the encryption mask */
+	pmd_flags = __sme_clr(early_pmd_flags);
+
+	do {
+		pmd = map ? (paddr & PMD_MASK) + pmd_flags : 0;
+		__early_make_pgtable((unsigned long)vaddr, pmd);
+
+		vaddr += PMD_SIZE;
+		paddr += PMD_SIZE;
+		size = (size <= PMD_SIZE) ? 0 : size - PMD_SIZE;
+	} while (size);
+}
+
+static void __init __sme_map_unmap_bootdata(char *real_mode_data, bool map)
+{
+	struct boot_params *boot_data;
+	unsigned long cmdline_paddr;
+
+	__sme_early_map_unmap_mem(real_mode_data, sizeof(boot_params), map);
+	boot_data = (struct boot_params *)real_mode_data;
+
+	/*
+	 * Determine the command line address only after having established
+	 * the decrypted mapping.
+	 */
+	cmdline_paddr = boot_data->hdr.cmd_line_ptr |
+			((u64)boot_data->ext_cmd_line_ptr << 32);
+
+	if (cmdline_paddr)
+		__sme_early_map_unmap_mem(__va(cmdline_paddr),
+					  COMMAND_LINE_SIZE, map);
+}
+
+void __init sme_unmap_bootdata(char *real_mode_data)
+{
+	/* If SME is not active, the bootdata is in the correct state */
+	if (!sme_active())
+		return;
+
+	/*
+	 * The bootdata and command line aren't needed anymore so clear
+	 * any mapping of them.
+	 */
+	__sme_map_unmap_bootdata(real_mode_data, false);
+
+	sme_early_pgtable_flush();
+}
+
+void __init sme_map_bootdata(char *real_mode_data)
+{
+	/* If SME is not active, the bootdata is in the correct state */
+	if (!sme_active())
+		return;
+
+	/*
+	 * The bootdata and command line will not be encrypted, so they
+	 * need to be mapped as decrypted memory so they can be copied
+	 * properly.
+	 */
+	__sme_map_unmap_bootdata(real_mode_data, true);
+
+	sme_early_pgtable_flush();
+}
+
 void __init sme_early_init(void)
 {
 	unsigned int i;



From thomas.lendacky at amd.com  Wed Jun  7 12:15:49 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Wed, 07 Jun 2017 14:15:49 -0500
Subject: [PATCH v6 15/34] x86/boot/e820: Add support to determine the E820
 type of an address
In-Reply-To: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170607191549.28645.40861.stgit@tlendack-t1.amdoffice.net>

Add a function that will return the E820 type associated with an address
range.

Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/include/asm/e820/api.h |    2 ++
 arch/x86/kernel/e820.c          |   26 +++++++++++++++++++++++---
 2 files changed, 25 insertions(+), 3 deletions(-)

diff --git a/arch/x86/include/asm/e820/api.h b/arch/x86/include/asm/e820/api.h
index 8e0f8b8..3641f5f 100644
--- a/arch/x86/include/asm/e820/api.h
+++ b/arch/x86/include/asm/e820/api.h
@@ -38,6 +38,8 @@
 extern void e820__reallocate_tables(void);
 extern void e820__register_nosave_regions(unsigned long limit_pfn);
 
+extern int  e820__get_entry_type(u64 start, u64 end);
+
 /*
  * Returns true iff the specified range [start,end) is completely contained inside
  * the ISA region.
diff --git a/arch/x86/kernel/e820.c b/arch/x86/kernel/e820.c
index d78a586..46c9b65 100644
--- a/arch/x86/kernel/e820.c
+++ b/arch/x86/kernel/e820.c
@@ -84,7 +84,8 @@ bool e820__mapped_any(u64 start, u64 end, enum e820_type type)
  * Note: this function only works correctly once the E820 table is sorted and
  * not-overlapping (at least for the range specified), which is the case normally.
  */
-bool __init e820__mapped_all(u64 start, u64 end, enum e820_type type)
+static struct e820_entry *__e820__mapped_all(u64 start, u64 end,
+					     enum e820_type type)
 {
 	int i;
 
@@ -110,9 +111,28 @@ bool __init e820__mapped_all(u64 start, u64 end, enum e820_type type)
 		 * coverage of the desired range exists:
 		 */
 		if (start >= end)
-			return 1;
+			return entry;
 	}
-	return 0;
+
+	return NULL;
+}
+
+/*
+ * This function checks if the entire range <start,end> is mapped with type.
+ */
+bool __init e820__mapped_all(u64 start, u64 end, enum e820_type type)
+{
+	return __e820__mapped_all(start, end, type);
+}
+
+/*
+ * This function returns the type associated with the range <start,end>.
+ */
+int e820__get_entry_type(u64 start, u64 end)
+{
+	struct e820_entry *entry = __e820__mapped_all(start, end, 0);
+
+	return entry ? entry->type : -EINVAL;
 }
 
 /*



From thomas.lendacky at amd.com  Wed Jun  7 12:16:03 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Wed, 07 Jun 2017 14:16:03 -0500
Subject: [PATCH v6 16/34] efi: Add an EFI table address match function
In-Reply-To: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170607191603.28645.2332.stgit@tlendack-t1.amdoffice.net>

Add a function that will determine if a supplied physical address matches
the address of an EFI table.

Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 drivers/firmware/efi/efi.c |   33 +++++++++++++++++++++++++++++++++
 include/linux/efi.h        |    7 +++++++
 2 files changed, 40 insertions(+)

diff --git a/drivers/firmware/efi/efi.c b/drivers/firmware/efi/efi.c
index b372aad..983675d 100644
--- a/drivers/firmware/efi/efi.c
+++ b/drivers/firmware/efi/efi.c
@@ -55,6 +55,25 @@ struct efi __read_mostly efi = {
 };
 EXPORT_SYMBOL(efi);
 
+static unsigned long *efi_tables[] = {
+	&efi.mps,
+	&efi.acpi,
+	&efi.acpi20,
+	&efi.smbios,
+	&efi.smbios3,
+	&efi.sal_systab,
+	&efi.boot_info,
+	&efi.hcdp,
+	&efi.uga,
+	&efi.uv_systab,
+	&efi.fw_vendor,
+	&efi.runtime,
+	&efi.config_table,
+	&efi.esrt,
+	&efi.properties_table,
+	&efi.mem_attr_table,
+};
+
 static bool disable_runtime;
 static int __init setup_noefi(char *arg)
 {
@@ -854,6 +873,20 @@ int efi_status_to_err(efi_status_t status)
 	return err;
 }
 
+bool efi_is_table_address(unsigned long phys_addr)
+{
+	unsigned int i;
+
+	if (phys_addr == EFI_INVALID_TABLE_ADDR)
+		return false;
+
+	for (i = 0; i < ARRAY_SIZE(efi_tables); i++)
+		if (*(efi_tables[i]) == phys_addr)
+			return true;
+
+	return false;
+}
+
 #ifdef CONFIG_KEXEC
 static int update_efi_random_seed(struct notifier_block *nb,
 				  unsigned long code, void *unused)
diff --git a/include/linux/efi.h b/include/linux/efi.h
index ec36f42..504fa85 100644
--- a/include/linux/efi.h
+++ b/include/linux/efi.h
@@ -1079,6 +1079,8 @@ static inline bool efi_enabled(int feature)
 	return test_bit(feature, &efi.flags) != 0;
 }
 extern void efi_reboot(enum reboot_mode reboot_mode, const char *__unused);
+
+extern bool efi_is_table_address(unsigned long phys_addr);
 #else
 static inline bool efi_enabled(int feature)
 {
@@ -1092,6 +1094,11 @@ static inline bool efi_enabled(int feature)
 {
 	return false;
 }
+
+static inline bool efi_is_table_address(unsigned long phys_addr)
+{
+	return false;
+}
 #endif
 
 extern int efi_status_to_err(efi_status_t status);



From thomas.lendacky at amd.com  Wed Jun  7 12:16:17 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Wed, 07 Jun 2017 14:16:17 -0500
Subject: [PATCH v6 17/34] efi: Update efi_mem_type() to return an error
 rather than 0
In-Reply-To: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170607191617.28645.8923.stgit@tlendack-t1.amdoffice.net>

The efi_mem_type() function currently returns a 0, which maps to
EFI_RESERVED_TYPE, if the function is unable to find a memmap entry for
the supplied physical address. Returning EFI_RESERVED_TYPE implies that
a memmap entry exists, when it doesn't.  Instead of returning 0, change
the function to return a negative error value when no memmap entry is
found.

Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/ia64/kernel/efi.c      |    4 ++--
 arch/x86/platform/efi/efi.c |    6 +++---
 include/linux/efi.h         |    2 +-
 3 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/arch/ia64/kernel/efi.c b/arch/ia64/kernel/efi.c
index 1212956..8141600 100644
--- a/arch/ia64/kernel/efi.c
+++ b/arch/ia64/kernel/efi.c
@@ -757,14 +757,14 @@ static void __init handle_palo(unsigned long phys_addr)
 	return 0;
 }
 
-u32
+int
 efi_mem_type (unsigned long phys_addr)
 {
 	efi_memory_desc_t *md = efi_memory_descriptor(phys_addr);
 
 	if (md)
 		return md->type;
-	return 0;
+	return -EINVAL;
 }
 
 u64
diff --git a/arch/x86/platform/efi/efi.c b/arch/x86/platform/efi/efi.c
index 43b96f5..a6a26cc 100644
--- a/arch/x86/platform/efi/efi.c
+++ b/arch/x86/platform/efi/efi.c
@@ -1034,12 +1034,12 @@ void __init efi_enter_virtual_mode(void)
 /*
  * Convenience functions to obtain memory types and attributes
  */
-u32 efi_mem_type(unsigned long phys_addr)
+int efi_mem_type(unsigned long phys_addr)
 {
 	efi_memory_desc_t *md;
 
 	if (!efi_enabled(EFI_MEMMAP))
-		return 0;
+		return -ENOTSUPP;
 
 	for_each_efi_memory_desc(md) {
 		if ((md->phys_addr <= phys_addr) &&
@@ -1047,7 +1047,7 @@ u32 efi_mem_type(unsigned long phys_addr)
 				  (md->num_pages << EFI_PAGE_SHIFT))))
 			return md->type;
 	}
-	return 0;
+	return -EINVAL;
 }
 
 static int __init arch_parse_efi_cmdline(char *str)
diff --git a/include/linux/efi.h b/include/linux/efi.h
index 504fa85..8bcb271 100644
--- a/include/linux/efi.h
+++ b/include/linux/efi.h
@@ -973,7 +973,7 @@ static inline void efi_esrt_init(void) { }
 extern int efi_config_parse_tables(void *config_tables, int count, int sz,
 				   efi_config_table_type_t *arch_tables);
 extern u64 efi_get_iobase (void);
-extern u32 efi_mem_type (unsigned long phys_addr);
+extern int efi_mem_type(unsigned long phys_addr);
 extern u64 efi_mem_attributes (unsigned long phys_addr);
 extern u64 efi_mem_attribute (unsigned long phys_addr, unsigned long size);
 extern int __init efi_uart_console_only (void);



From thomas.lendacky at amd.com  Wed Jun  7 12:16:27 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Wed, 07 Jun 2017 14:16:27 -0500
Subject: [PATCH v6 18/34] x86/efi: Update EFI pagetable creation to work
 with SME
In-Reply-To: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170607191627.28645.4398.stgit@tlendack-t1.amdoffice.net>

When SME is active, pagetable entries created for EFI need to have the
encryption mask set as necessary.

When the new pagetable pages are allocated they are mapped encrypted. So,
update the efi_pgt value that will be used in cr3 to include the encryption
mask so that the PGD table can be read successfully. The pagetable mapping
as well as the kernel are also added to the pagetable mapping as encrypted.
All other EFI mappings are mapped decrypted (tables, etc.).

Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/platform/efi/efi_64.c |   15 +++++++++++----
 1 file changed, 11 insertions(+), 4 deletions(-)

diff --git a/arch/x86/platform/efi/efi_64.c b/arch/x86/platform/efi/efi_64.c
index eb8dff1..ed37fa3 100644
--- a/arch/x86/platform/efi/efi_64.c
+++ b/arch/x86/platform/efi/efi_64.c
@@ -327,7 +327,7 @@ void efi_sync_low_kernel_mappings(void)
 
 int __init efi_setup_page_tables(unsigned long pa_memmap, unsigned num_pages)
 {
-	unsigned long pfn, text;
+	unsigned long pfn, text, pf;
 	struct page *page;
 	unsigned npages;
 	pgd_t *pgd;
@@ -335,7 +335,12 @@ int __init efi_setup_page_tables(unsigned long pa_memmap, unsigned num_pages)
 	if (efi_enabled(EFI_OLD_MEMMAP))
 		return 0;
 
-	efi_scratch.efi_pgt = (pgd_t *)__pa(efi_pgd);
+	/*
+	 * Since the PGD is encrypted, set the encryption mask so that when
+	 * this value is loaded into cr3 the PGD will be decrypted during
+	 * the pagetable walk.
+	 */
+	efi_scratch.efi_pgt = (pgd_t *)__sme_pa(efi_pgd);
 	pgd = efi_pgd;
 
 	/*
@@ -345,7 +350,8 @@ int __init efi_setup_page_tables(unsigned long pa_memmap, unsigned num_pages)
 	 * phys_efi_set_virtual_address_map().
 	 */
 	pfn = pa_memmap >> PAGE_SHIFT;
-	if (kernel_map_pages_in_pgd(pgd, pfn, pa_memmap, num_pages, _PAGE_NX | _PAGE_RW)) {
+	pf = _PAGE_NX | _PAGE_RW | _PAGE_ENC;
+	if (kernel_map_pages_in_pgd(pgd, pfn, pa_memmap, num_pages, pf)) {
 		pr_err("Error ident-mapping new memmap (0x%lx)!\n", pa_memmap);
 		return 1;
 	}
@@ -388,7 +394,8 @@ int __init efi_setup_page_tables(unsigned long pa_memmap, unsigned num_pages)
 	text = __pa(_text);
 	pfn = text >> PAGE_SHIFT;
 
-	if (kernel_map_pages_in_pgd(pgd, pfn, text, npages, _PAGE_RW)) {
+	pf = _PAGE_RW | _PAGE_ENC;
+	if (kernel_map_pages_in_pgd(pgd, pfn, text, npages, pf)) {
 		pr_err("Failed to map kernel text 1:1\n");
 		return 1;
 	}



From thomas.lendacky at amd.com  Wed Jun  7 12:16:43 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Wed, 07 Jun 2017 14:16:43 -0500
Subject: [PATCH v6 20/34] x86,
 mpparse: Use memremap to map the mpf and mpc data
In-Reply-To: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170607191643.28645.91679.stgit@tlendack-t1.amdoffice.net>

The SMP MP-table is built by UEFI and placed in memory in a decrypted
state. These tables are accessed using a mix of early_memremap(),
early_memunmap(), phys_to_virt() and virt_to_phys(). Change all accesses
to use early_memremap()/early_memunmap(). This allows for proper setting
of the encryption mask so that the data can be successfully accessed when
SME is active.

Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/kernel/mpparse.c |   98 ++++++++++++++++++++++++++++++++-------------
 1 file changed, 70 insertions(+), 28 deletions(-)

diff --git a/arch/x86/kernel/mpparse.c b/arch/x86/kernel/mpparse.c
index fd37f39..44b5d582 100644
--- a/arch/x86/kernel/mpparse.c
+++ b/arch/x86/kernel/mpparse.c
@@ -429,7 +429,7 @@ static inline void __init construct_default_ISA_mptable(int mpc_default_type)
 	}
 }
 
-static struct mpf_intel *mpf_found;
+static unsigned long mpf_base;
 
 static unsigned long __init get_mpc_size(unsigned long physptr)
 {
@@ -451,6 +451,7 @@ static int __init check_physptr(struct mpf_intel *mpf, unsigned int early)
 
 	size = get_mpc_size(mpf->physptr);
 	mpc = early_memremap(mpf->physptr, size);
+
 	/*
 	 * Read the physical hardware table.  Anything here will
 	 * override the defaults.
@@ -497,12 +498,12 @@ static int __init check_physptr(struct mpf_intel *mpf, unsigned int early)
  */
 void __init default_get_smp_config(unsigned int early)
 {
-	struct mpf_intel *mpf = mpf_found;
+	struct mpf_intel *mpf;
 
 	if (!smp_found_config)
 		return;
 
-	if (!mpf)
+	if (!mpf_base)
 		return;
 
 	if (acpi_lapic && early)
@@ -515,6 +516,12 @@ void __init default_get_smp_config(unsigned int early)
 	if (acpi_lapic && acpi_ioapic)
 		return;
 
+	mpf = early_memremap(mpf_base, sizeof(*mpf));
+	if (!mpf) {
+		pr_err("MPTABLE: mpf early_memremap() failed\n");
+		return;
+	}
+
 	pr_info("Intel MultiProcessor Specification v1.%d\n",
 		mpf->specification);
 #if defined(CONFIG_X86_LOCAL_APIC) && defined(CONFIG_X86_32)
@@ -529,7 +536,7 @@ void __init default_get_smp_config(unsigned int early)
 	/*
 	 * Now see if we need to read further.
 	 */
-	if (mpf->feature1 != 0) {
+	if (mpf->feature1) {
 		if (early) {
 			/*
 			 * local APIC has default address
@@ -542,8 +549,10 @@ void __init default_get_smp_config(unsigned int early)
 		construct_default_ISA_mptable(mpf->feature1);
 
 	} else if (mpf->physptr) {
-		if (check_physptr(mpf, early))
+		if (check_physptr(mpf, early)) {
+			early_memunmap(mpf, sizeof(*mpf));
 			return;
+		}
 	} else
 		BUG();
 
@@ -552,6 +561,8 @@ void __init default_get_smp_config(unsigned int early)
 	/*
 	 * Only use the first configuration found.
 	 */
+
+	early_memunmap(mpf, sizeof(*mpf));
 }
 
 static void __init smp_reserve_memory(struct mpf_intel *mpf)
@@ -561,15 +572,16 @@ static void __init smp_reserve_memory(struct mpf_intel *mpf)
 
 static int __init smp_scan_config(unsigned long base, unsigned long length)
 {
-	unsigned int *bp = phys_to_virt(base);
+	unsigned int *bp;
 	struct mpf_intel *mpf;
-	unsigned long mem;
+	int ret = 0;
 
 	apic_printk(APIC_VERBOSE, "Scan for SMP in [mem %#010lx-%#010lx]\n",
 		    base, base + length - 1);
 	BUILD_BUG_ON(sizeof(*mpf) != 16);
 
 	while (length > 0) {
+		bp = early_memremap(base, length);
 		mpf = (struct mpf_intel *)bp;
 		if ((*bp == SMP_MAGIC_IDENT) &&
 		    (mpf->length == 1) &&
@@ -579,24 +591,26 @@ static int __init smp_scan_config(unsigned long base, unsigned long length)
 #ifdef CONFIG_X86_LOCAL_APIC
 			smp_found_config = 1;
 #endif
-			mpf_found = mpf;
+			mpf_base = base;
 
-			pr_info("found SMP MP-table at [mem %#010llx-%#010llx] mapped at [%p]\n",
-				(unsigned long long) virt_to_phys(mpf),
-				(unsigned long long) virt_to_phys(mpf) +
-				sizeof(*mpf) - 1, mpf);
+			pr_info("found SMP MP-table at [mem %#010lx-%#010lx] mapped at [%p]\n",
+				base, base + sizeof(*mpf) - 1, mpf);
 
-			mem = virt_to_phys(mpf);
-			memblock_reserve(mem, sizeof(*mpf));
+			memblock_reserve(base, sizeof(*mpf));
 			if (mpf->physptr)
 				smp_reserve_memory(mpf);
 
-			return 1;
+			ret = 1;
 		}
-		bp += 4;
+		early_memunmap(bp, length);
+
+		if (ret)
+			break;
+
+		base += 16;
 		length -= 16;
 	}
-	return 0;
+	return ret;
 }
 
 void __init default_find_smp_config(void)
@@ -838,29 +852,40 @@ static int __init update_mp_table(void)
 	char oem[10];
 	struct mpf_intel *mpf;
 	struct mpc_table *mpc, *mpc_new;
+	unsigned long size;
 
 	if (!enable_update_mptable)
 		return 0;
 
-	mpf = mpf_found;
-	if (!mpf)
+	if (!mpf_base)
+		return 0;
+
+	mpf = early_memremap(mpf_base, sizeof(*mpf));
+	if (!mpf) {
+		pr_err("MPTABLE: mpf early_memremap() failed\n");
 		return 0;
+	}
 
 	/*
 	 * Now see if we need to go further.
 	 */
-	if (mpf->feature1 != 0)
-		return 0;
+	if (mpf->feature1)
+		goto do_unmap_mpf;
 
 	if (!mpf->physptr)
-		return 0;
+		goto do_unmap_mpf;
 
-	mpc = phys_to_virt(mpf->physptr);
+	size = get_mpc_size(mpf->physptr);
+	mpc = early_memremap(mpf->physptr, size);
+	if (!mpc) {
+		pr_err("MPTABLE: mpc early_memremap() failed\n");
+		goto do_unmap_mpf;
+	}
 
 	if (!smp_check_mpc(mpc, oem, str))
-		return 0;
+		goto do_unmap_mpc;
 
-	pr_info("mpf: %llx\n", (u64)virt_to_phys(mpf));
+	pr_info("mpf: %llx\n", (u64)mpf_base);
 	pr_info("physptr: %x\n", mpf->physptr);
 
 	if (mpc_new_phys && mpc->length > mpc_new_length) {
@@ -878,21 +903,32 @@ static int __init update_mp_table(void)
 		new = mpf_checksum((unsigned char *)mpc, mpc->length);
 		if (old == new) {
 			pr_info("mpc is readonly, please try alloc_mptable instead\n");
-			return 0;
+			goto do_unmap_mpc;
 		}
 		pr_info("use in-position replacing\n");
 	} else {
+		mpc_new = early_memremap(mpc_new_phys, mpc_new_length);
+		if (!mpc_new) {
+			pr_err("MPTABLE: new mpc early_memremap() failed\n");
+			goto do_unmap_mpc;
+		}
 		mpf->physptr = mpc_new_phys;
-		mpc_new = phys_to_virt(mpc_new_phys);
 		memcpy(mpc_new, mpc, mpc->length);
+		early_memunmap(mpc, size);
 		mpc = mpc_new;
+		size = mpc_new_length;
 		/* check if we can modify that */
 		if (mpc_new_phys - mpf->physptr) {
 			struct mpf_intel *mpf_new;
 			/* steal 16 bytes from [0, 1k) */
+			mpf_new = early_memremap(0x400 - 16, sizeof(*mpf_new));
+			if (!mpf_new) {
+				pr_err("MPTABLE: new mpf early_memremap() failed\n");
+				goto do_unmap_mpc;
+			}
 			pr_info("mpf new: %x\n", 0x400 - 16);
-			mpf_new = phys_to_virt(0x400 - 16);
 			memcpy(mpf_new, mpf, 16);
+			early_memunmap(mpf, sizeof(*mpf));
 			mpf = mpf_new;
 			mpf->physptr = mpc_new_phys;
 		}
@@ -909,6 +945,12 @@ static int __init update_mp_table(void)
 	 */
 	replace_intsrc_all(mpc, mpc_new_phys, mpc_new_length);
 
+do_unmap_mpc:
+	early_memunmap(mpc, size);
+
+do_unmap_mpf:
+	early_memunmap(mpf, sizeof(*mpf));
+
 	return 0;
 }
 



From thomas.lendacky at amd.com  Wed Jun  7 12:16:36 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Wed, 07 Jun 2017 14:16:36 -0500
Subject: [PATCH v6 19/34] x86/mm: Add support to access boot related data in
 the clear
In-Reply-To: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170607191636.28645.98914.stgit@tlendack-t1.amdoffice.net>

Boot data (such as EFI related data) is not encrypted when the system is
booted because UEFI/BIOS does not run with SME active. In order to access
this data properly it needs to be mapped decrypted.

Update early_memremap() to provide an arch specific routine to modify the
pagetable protection attributes before they are applied to the new
mapping. This is used to remove the encryption mask for boot related data.

Update memremap() to provide an arch specific routine to determine if RAM
remapping is allowed.  RAM remapping will cause an encrypted mapping to be
generated. By preventing RAM remapping, ioremap_cache() will be used
instead, which will provide a decrypted mapping of the boot related data.

Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/include/asm/io.h |    4 +
 arch/x86/mm/ioremap.c     |  179 +++++++++++++++++++++++++++++++++++++++++++++
 include/linux/io.h        |    2 +
 kernel/memremap.c         |   20 ++++-
 mm/early_ioremap.c        |   18 ++++-
 5 files changed, 216 insertions(+), 7 deletions(-)

diff --git a/arch/x86/include/asm/io.h b/arch/x86/include/asm/io.h
index 7afb0e2..9eac5a5 100644
--- a/arch/x86/include/asm/io.h
+++ b/arch/x86/include/asm/io.h
@@ -381,4 +381,8 @@ extern int __must_check arch_phys_wc_add(unsigned long base,
 #define arch_io_reserve_memtype_wc arch_io_reserve_memtype_wc
 #endif
 
+extern bool arch_memremap_can_ram_remap(resource_size_t offset, size_t size,
+					unsigned long flags);
+#define arch_memremap_can_ram_remap arch_memremap_can_ram_remap
+
 #endif /* _ASM_X86_IO_H */
diff --git a/arch/x86/mm/ioremap.c b/arch/x86/mm/ioremap.c
index 792db75..34ed59d 100644
--- a/arch/x86/mm/ioremap.c
+++ b/arch/x86/mm/ioremap.c
@@ -13,6 +13,7 @@
 #include <linux/slab.h>
 #include <linux/vmalloc.h>
 #include <linux/mmiotrace.h>
+#include <linux/efi.h>
 
 #include <asm/set_memory.h>
 #include <asm/e820/api.h>
@@ -22,6 +23,7 @@
 #include <asm/pgalloc.h>
 #include <asm/pat.h>
 #include <asm/mem_encrypt.h>
+#include <asm/setup.h>
 
 #include "physaddr.h"
 
@@ -422,6 +424,183 @@ void unxlate_dev_mem_ptr(phys_addr_t phys, void *addr)
 	iounmap((void __iomem *)((unsigned long)addr & PAGE_MASK));
 }
 
+/*
+ * Examine the physical address to determine if it is an area of memory
+ * that should be mapped decrypted.  If the memory is not part of the
+ * kernel usable area it was accessed and created decrypted, so these
+ * areas should be mapped decrypted.
+ */
+static bool memremap_should_map_decrypted(resource_size_t phys_addr,
+					  unsigned long size)
+{
+	/* Check if the address is outside kernel usable area */
+	switch (e820__get_entry_type(phys_addr, phys_addr + size - 1)) {
+	case E820_TYPE_RESERVED:
+	case E820_TYPE_ACPI:
+	case E820_TYPE_NVS:
+	case E820_TYPE_UNUSABLE:
+		return true;
+	default:
+		break;
+	}
+
+	return false;
+}
+
+/*
+ * Examine the physical address to determine if it is EFI data. Check
+ * it against the boot params structure and EFI tables and memory types.
+ */
+static bool memremap_is_efi_data(resource_size_t phys_addr,
+				 unsigned long size)
+{
+	u64 paddr;
+
+	/* Check if the address is part of EFI boot/runtime data */
+	if (!efi_enabled(EFI_BOOT))
+		return false;
+
+	paddr = boot_params.efi_info.efi_memmap_hi;
+	paddr <<= 32;
+	paddr |= boot_params.efi_info.efi_memmap;
+	if (phys_addr == paddr)
+		return true;
+
+	paddr = boot_params.efi_info.efi_systab_hi;
+	paddr <<= 32;
+	paddr |= boot_params.efi_info.efi_systab;
+	if (phys_addr == paddr)
+		return true;
+
+	if (efi_is_table_address(phys_addr))
+		return true;
+
+	switch (efi_mem_type(phys_addr)) {
+	case EFI_BOOT_SERVICES_DATA:
+	case EFI_RUNTIME_SERVICES_DATA:
+		return true;
+	default:
+		break;
+	}
+
+	return false;
+}
+
+/*
+ * Examine the physical address to determine if it is boot data by checking
+ * it against the boot params setup_data chain.
+ */
+static bool memremap_is_setup_data(resource_size_t phys_addr,
+				   unsigned long size)
+{
+	struct setup_data *data;
+	u64 paddr, paddr_next;
+
+	paddr = boot_params.hdr.setup_data;
+	while (paddr) {
+		unsigned int len;
+
+		if (phys_addr == paddr)
+			return true;
+
+		data = memremap(paddr, sizeof(*data),
+				MEMREMAP_WB | MEMREMAP_DEC);
+
+		paddr_next = data->next;
+		len = data->len;
+
+		memunmap(data);
+
+		if ((phys_addr > paddr) && (phys_addr < (paddr + len)))
+			return true;
+
+		paddr = paddr_next;
+	}
+
+	return false;
+}
+
+/*
+ * Examine the physical address to determine if it is boot data by checking
+ * it against the boot params setup_data chain (early boot version).
+ */
+static bool __init early_memremap_is_setup_data(resource_size_t phys_addr,
+						unsigned long size)
+{
+	struct setup_data *data;
+	u64 paddr, paddr_next;
+
+	paddr = boot_params.hdr.setup_data;
+	while (paddr) {
+		unsigned int len;
+
+		if (phys_addr == paddr)
+			return true;
+
+		data = early_memremap_decrypted(paddr, sizeof(*data));
+
+		paddr_next = data->next;
+		len = data->len;
+
+		early_memunmap(data, sizeof(*data));
+
+		if ((phys_addr > paddr) && (phys_addr < (paddr + len)))
+			return true;
+
+		paddr = paddr_next;
+	}
+
+	return false;
+}
+
+/*
+ * Architecture function to determine if RAM remap is allowed. By default, a
+ * RAM remap will map the data as encrypted. Determine if a RAM remap should
+ * not be done so that the data will be mapped decrypted.
+ */
+bool arch_memremap_can_ram_remap(resource_size_t phys_addr, unsigned long size,
+				 unsigned long flags)
+{
+	if (!sme_active())
+		return true;
+
+	if (flags & MEMREMAP_ENC)
+		return true;
+
+	if (flags & MEMREMAP_DEC)
+		return false;
+
+	if (memremap_is_setup_data(phys_addr, size) ||
+	    memremap_is_efi_data(phys_addr, size) ||
+	    memremap_should_map_decrypted(phys_addr, size))
+		return false;
+
+	return true;
+}
+
+/*
+ * Architecture override of __weak function to adjust the protection attributes
+ * used when remapping memory. By default, early_memremap() will map the data
+ * as encrypted. Determine if an encrypted mapping should not be done and set
+ * the appropriate protection attributes.
+ */
+pgprot_t __init early_memremap_pgprot_adjust(resource_size_t phys_addr,
+					     unsigned long size,
+					     pgprot_t prot)
+{
+	if (!sme_active())
+		return prot;
+
+	if (early_memremap_is_setup_data(phys_addr, size) ||
+	    memremap_is_efi_data(phys_addr, size) ||
+	    memremap_should_map_decrypted(phys_addr, size))
+		prot = pgprot_decrypted(prot);
+	else
+		prot = pgprot_encrypted(prot);
+
+	return prot;
+}
+
 #ifdef CONFIG_ARCH_USE_MEMREMAP_PROT
 /* Remap memory with encryption */
 void __init *early_memremap_encrypted(resource_size_t phys_addr,
diff --git a/include/linux/io.h b/include/linux/io.h
index 2195d9e..32e30e8 100644
--- a/include/linux/io.h
+++ b/include/linux/io.h
@@ -157,6 +157,8 @@ enum {
 	MEMREMAP_WB = 1 << 0,
 	MEMREMAP_WT = 1 << 1,
 	MEMREMAP_WC = 1 << 2,
+	MEMREMAP_ENC = 1 << 3,
+	MEMREMAP_DEC = 1 << 4,
 };
 
 void *memremap(resource_size_t offset, size_t size, unsigned long flags);
diff --git a/kernel/memremap.c b/kernel/memremap.c
index 23a6483..9b4fbd5 100644
--- a/kernel/memremap.c
+++ b/kernel/memremap.c
@@ -34,13 +34,24 @@ static void *arch_memremap_wb(resource_size_t offset, unsigned long size)
 }
 #endif
 
-static void *try_ram_remap(resource_size_t offset, size_t size)
+#ifndef arch_memremap_can_ram_remap
+static bool arch_memremap_can_ram_remap(resource_size_t offset, size_t size,
+					unsigned long flags)
+{
+	return true;
+}
+#endif
+
+static void *try_ram_remap(resource_size_t offset, size_t size,
+			   unsigned long flags)
 {
 	unsigned long pfn = PHYS_PFN(offset);
 
 	/* In the simple case just return the existing linear address */
-	if (pfn_valid(pfn) && !PageHighMem(pfn_to_page(pfn)))
+	if (pfn_valid(pfn) && !PageHighMem(pfn_to_page(pfn)) &&
+	    arch_memremap_can_ram_remap(offset, size, flags))
 		return __va(offset);
+
 	return NULL; /* fallback to arch_memremap_wb */
 }
 
@@ -48,7 +59,8 @@ static void *try_ram_remap(resource_size_t offset, size_t size)
  * memremap() - remap an iomem_resource as cacheable memory
  * @offset: iomem resource start address
  * @size: size of remap
- * @flags: any of MEMREMAP_WB, MEMREMAP_WT and MEMREMAP_WC
+ * @flags: any of MEMREMAP_WB, MEMREMAP_WT, MEMREMAP_WC,
+ *		  MEMREMAP_ENC, MEMREMAP_DEC
  *
  * memremap() is "ioremap" for cases where it is known that the resource
  * being mapped does not have i/o side effects and the __iomem
@@ -95,7 +107,7 @@ void *memremap(resource_size_t offset, size_t size, unsigned long flags)
 		 * the requested range is potentially in System RAM.
 		 */
 		if (is_ram == REGION_INTERSECTS)
-			addr = try_ram_remap(offset, size);
+			addr = try_ram_remap(offset, size, flags);
 		if (!addr)
 			addr = arch_memremap_wb(offset, size);
 	}
diff --git a/mm/early_ioremap.c b/mm/early_ioremap.c
index d7d30da..b1dd4a9 100644
--- a/mm/early_ioremap.c
+++ b/mm/early_ioremap.c
@@ -30,6 +30,13 @@ static int __init early_ioremap_debug_setup(char *str)
 
 static int after_paging_init __initdata;
 
+pgprot_t __init __weak early_memremap_pgprot_adjust(resource_size_t phys_addr,
+						    unsigned long size,
+						    pgprot_t prot)
+{
+	return prot;
+}
+
 void __init __weak early_ioremap_shutdown(void)
 {
 }
@@ -215,14 +222,19 @@ void __init early_iounmap(void __iomem *addr, unsigned long size)
 void __init *
 early_memremap(resource_size_t phys_addr, unsigned long size)
 {
-	return (__force void *)__early_ioremap(phys_addr, size,
-					       FIXMAP_PAGE_NORMAL);
+	pgprot_t prot = early_memremap_pgprot_adjust(phys_addr, size,
+						     FIXMAP_PAGE_NORMAL);
+
+	return (__force void *)__early_ioremap(phys_addr, size, prot);
 }
 #ifdef FIXMAP_PAGE_RO
 void __init *
 early_memremap_ro(resource_size_t phys_addr, unsigned long size)
 {
-	return (__force void *)__early_ioremap(phys_addr, size, FIXMAP_PAGE_RO);
+	pgprot_t prot = early_memremap_pgprot_adjust(phys_addr, size,
+						     FIXMAP_PAGE_RO);
+
+	return (__force void *)__early_ioremap(phys_addr, size, prot);
 }
 #endif
 



From thomas.lendacky at amd.com  Wed Jun  7 12:17:21 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Wed, 07 Jun 2017 14:17:21 -0500
Subject: [PATCH v6 24/34] x86, swiotlb: Add memory encryption support
In-Reply-To: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170607191721.28645.96519.stgit@tlendack-t1.amdoffice.net>

Since DMA addresses will effectively look like 48-bit addresses when the
memory encryption mask is set, SWIOTLB is needed if the DMA mask of the
device performing the DMA does not support 48-bits. SWIOTLB will be
initialized to create decrypted bounce buffers for use by these devices.

Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/include/asm/dma-mapping.h |    5 ++-
 arch/x86/include/asm/mem_encrypt.h |    5 +++
 arch/x86/kernel/pci-dma.c          |   11 +++++--
 arch/x86/kernel/pci-nommu.c        |    2 +
 arch/x86/kernel/pci-swiotlb.c      |   15 ++++++++--
 arch/x86/mm/mem_encrypt.c          |   22 ++++++++++++++
 include/linux/swiotlb.h            |    1 +
 init/main.c                        |   13 ++++++++
 lib/swiotlb.c                      |   56 +++++++++++++++++++++++++++++++-----
 9 files changed, 113 insertions(+), 17 deletions(-)

diff --git a/arch/x86/include/asm/dma-mapping.h b/arch/x86/include/asm/dma-mapping.h
index 08a0838..d75430a 100644
--- a/arch/x86/include/asm/dma-mapping.h
+++ b/arch/x86/include/asm/dma-mapping.h
@@ -12,6 +12,7 @@
 #include <asm/io.h>
 #include <asm/swiotlb.h>
 #include <linux/dma-contiguous.h>
+#include <asm/mem_encrypt.h>
 
 #ifdef CONFIG_ISA
 # define ISA_DMA_BIT_MASK DMA_BIT_MASK(24)
@@ -62,12 +63,12 @@ static inline bool dma_capable(struct device *dev, dma_addr_t addr, size_t size)
 
 static inline dma_addr_t phys_to_dma(struct device *dev, phys_addr_t paddr)
 {
-	return paddr;
+	return __sme_set(paddr);
 }
 
 static inline phys_addr_t dma_to_phys(struct device *dev, dma_addr_t daddr)
 {
-	return daddr;
+	return __sme_clr(daddr);
 }
 #endif /* CONFIG_X86_DMA_REMAP */
 
diff --git a/arch/x86/include/asm/mem_encrypt.h b/arch/x86/include/asm/mem_encrypt.h
index 61a7049..f1215a4 100644
--- a/arch/x86/include/asm/mem_encrypt.h
+++ b/arch/x86/include/asm/mem_encrypt.h
@@ -31,6 +31,11 @@ void __init sme_early_decrypt(resource_size_t paddr,
 
 void __init sme_early_init(void);
 
+/* Architecture __weak replacement functions */
+void __init mem_encrypt_init(void);
+
+void swiotlb_set_mem_attributes(void *vaddr, unsigned long size);
+
 #else	/* !CONFIG_AMD_MEM_ENCRYPT */
 
 #define sme_me_mask	0UL
diff --git a/arch/x86/kernel/pci-dma.c b/arch/x86/kernel/pci-dma.c
index 3a216ec..72d96d4 100644
--- a/arch/x86/kernel/pci-dma.c
+++ b/arch/x86/kernel/pci-dma.c
@@ -93,9 +93,12 @@ void *dma_generic_alloc_coherent(struct device *dev, size_t size,
 	if (gfpflags_allow_blocking(flag)) {
 		page = dma_alloc_from_contiguous(dev, count, get_order(size),
 						 flag);
-		if (page && page_to_phys(page) + size > dma_mask) {
-			dma_release_from_contiguous(dev, page, count);
-			page = NULL;
+		if (page) {
+			addr = phys_to_dma(dev, page_to_phys(page));
+			if (addr + size > dma_mask) {
+				dma_release_from_contiguous(dev, page, count);
+				page = NULL;
+			}
 		}
 	}
 	/* fallback */
@@ -104,7 +107,7 @@ void *dma_generic_alloc_coherent(struct device *dev, size_t size,
 	if (!page)
 		return NULL;
 
-	addr = page_to_phys(page);
+	addr = phys_to_dma(dev, page_to_phys(page));
 	if (addr + size > dma_mask) {
 		__free_pages(page, get_order(size));
 
diff --git a/arch/x86/kernel/pci-nommu.c b/arch/x86/kernel/pci-nommu.c
index a88952e..98b576a 100644
--- a/arch/x86/kernel/pci-nommu.c
+++ b/arch/x86/kernel/pci-nommu.c
@@ -30,7 +30,7 @@ static dma_addr_t nommu_map_page(struct device *dev, struct page *page,
 				 enum dma_data_direction dir,
 				 unsigned long attrs)
 {
-	dma_addr_t bus = page_to_phys(page) + offset;
+	dma_addr_t bus = phys_to_dma(dev, page_to_phys(page)) + offset;
 	WARN_ON(size == 0);
 	if (!check_addr("map_single", dev, bus, size))
 		return DMA_ERROR_CODE;
diff --git a/arch/x86/kernel/pci-swiotlb.c b/arch/x86/kernel/pci-swiotlb.c
index 1e23577..cc1e106 100644
--- a/arch/x86/kernel/pci-swiotlb.c
+++ b/arch/x86/kernel/pci-swiotlb.c
@@ -12,6 +12,8 @@
 #include <asm/dma.h>
 #include <asm/xen/swiotlb-xen.h>
 #include <asm/iommu_table.h>
+#include <asm/mem_encrypt.h>
+
 int swiotlb __read_mostly;
 
 void *x86_swiotlb_alloc_coherent(struct device *hwdev, size_t size,
@@ -79,8 +81,8 @@ int __init pci_swiotlb_detect_override(void)
 		  pci_swiotlb_late_init);
 
 /*
- * if 4GB or more detected (and iommu=off not set) return 1
- * and set swiotlb to 1.
+ * If 4GB or more detected (and iommu=off not set) or if SME is active
+ * then set swiotlb to 1 and return 1.
  */
 int __init pci_swiotlb_detect_4gb(void)
 {
@@ -89,6 +91,15 @@ int __init pci_swiotlb_detect_4gb(void)
 	if (!no_iommu && max_possible_pfn > MAX_DMA32_PFN)
 		swiotlb = 1;
 #endif
+
+	/*
+	 * If SME is active then swiotlb will be set to 1 so that bounce
+	 * buffers are allocated and used for devices that do not support
+	 * the addressing range required for the encryption mask.
+	 */
+	if (sme_active())
+		swiotlb = 1;
+
 	return swiotlb;
 }
 IOMMU_INIT(pci_swiotlb_detect_4gb,
diff --git a/arch/x86/mm/mem_encrypt.c b/arch/x86/mm/mem_encrypt.c
index 2321f05..5d7c51d 100644
--- a/arch/x86/mm/mem_encrypt.c
+++ b/arch/x86/mm/mem_encrypt.c
@@ -16,11 +16,14 @@
 #ifdef CONFIG_AMD_MEM_ENCRYPT
 
 #include <linux/mm.h>
+#include <linux/dma-mapping.h>
+#include <linux/swiotlb.h>
 
 #include <asm/tlbflush.h>
 #include <asm/fixmap.h>
 #include <asm/setup.h>
 #include <asm/bootparam.h>
+#include <asm/set_memory.h>
 
 /*
  * Since SME related variables are set early in the boot process they must
@@ -194,6 +197,25 @@ void __init sme_early_init(void)
 		protection_map[i] = pgprot_encrypted(protection_map[i]);
 }
 
+/* Architecture __weak replacement functions */
+void __init mem_encrypt_init(void)
+{
+	if (!sme_me_mask)
+		return;
+
+	/* Call into SWIOTLB to update the SWIOTLB DMA buffers */
+	swiotlb_update_mem_attributes();
+}
+
+void swiotlb_set_mem_attributes(void *vaddr, unsigned long size)
+{
+	WARN(PAGE_ALIGN(size) != size,
+	     "size is not page-aligned (%#lx)\n", size);
+
+	/* Make the SWIOTLB buffer area decrypted */
+	set_memory_decrypted((unsigned long)vaddr, size >> PAGE_SHIFT);
+}
+
 void __init sme_encrypt_kernel(void)
 {
 }
diff --git a/include/linux/swiotlb.h b/include/linux/swiotlb.h
index 4ee479f..15e7160 100644
--- a/include/linux/swiotlb.h
+++ b/include/linux/swiotlb.h
@@ -35,6 +35,7 @@ enum swiotlb_force {
 extern unsigned long swiotlb_nr_tbl(void);
 unsigned long swiotlb_size_or_default(void);
 extern int swiotlb_late_init_with_tbl(char *tlb, unsigned long nslabs);
+extern void __init swiotlb_update_mem_attributes(void);
 
 /*
  * Enumeration for sync targets
diff --git a/init/main.c b/init/main.c
index df58a41..7125b5f 100644
--- a/init/main.c
+++ b/init/main.c
@@ -488,6 +488,10 @@ void __init __weak thread_stack_cache_init(void)
 }
 #endif
 
+void __init __weak mem_encrypt_init(void)
+{
+}
+
 /*
  * Set up kernel memory allocators
  */
@@ -640,6 +644,15 @@ asmlinkage __visible void __init start_kernel(void)
 	 */
 	locking_selftest();
 
+	/*
+	 * This needs to be called before any devices perform DMA
+	 * operations that might use the SWIOTLB bounce buffers.
+	 * This call will mark the bounce buffers as decrypted so
+	 * that their usage will not cause "plain-text" data to be
+	 * decrypted when accessed.
+	 */
+	mem_encrypt_init();
+
 #ifdef CONFIG_BLK_DEV_INITRD
 	if (initrd_start && !initrd_below_start_ok &&
 	    page_to_pfn(virt_to_page((void *)initrd_start)) < min_low_pfn) {
diff --git a/lib/swiotlb.c b/lib/swiotlb.c
index a8d74a7..74d6557 100644
--- a/lib/swiotlb.c
+++ b/lib/swiotlb.c
@@ -30,6 +30,7 @@
 #include <linux/highmem.h>
 #include <linux/gfp.h>
 #include <linux/scatterlist.h>
+#include <linux/mem_encrypt.h>
 
 #include <asm/io.h>
 #include <asm/dma.h>
@@ -155,6 +156,17 @@ unsigned long swiotlb_size_or_default(void)
 	return size ? size : (IO_TLB_DEFAULT_SIZE);
 }
 
+void __weak swiotlb_set_mem_attributes(void *vaddr, unsigned long size)
+{
+}
+
+/* For swiotlb, clear memory encryption mask from dma addresses */
+static dma_addr_t swiotlb_phys_to_dma(struct device *hwdev,
+				      phys_addr_t address)
+{
+	return __sme_clr(phys_to_dma(hwdev, address));
+}
+
 /* Note that this doesn't work with highmem page */
 static dma_addr_t swiotlb_virt_to_bus(struct device *hwdev,
 				      volatile void *address)
@@ -183,6 +195,31 @@ void swiotlb_print_info(void)
 	       bytes >> 20, vstart, vend - 1);
 }
 
+/*
+ * Early SWIOTLB allocation may be too early to allow an architecture to
+ * perform the desired operations.  This function allows the architecture to
+ * call SWIOTLB when the operations are possible.  It needs to be called
+ * before the SWIOTLB memory is used.
+ */
+void __init swiotlb_update_mem_attributes(void)
+{
+	void *vaddr;
+	unsigned long bytes;
+
+	if (no_iotlb_memory || late_alloc)
+		return;
+
+	vaddr = phys_to_virt(io_tlb_start);
+	bytes = PAGE_ALIGN(io_tlb_nslabs << IO_TLB_SHIFT);
+	swiotlb_set_mem_attributes(vaddr, bytes);
+	memset(vaddr, 0, bytes);
+
+	vaddr = phys_to_virt(io_tlb_overflow_buffer);
+	bytes = PAGE_ALIGN(io_tlb_overflow);
+	swiotlb_set_mem_attributes(vaddr, bytes);
+	memset(vaddr, 0, bytes);
+}
+
 int __init swiotlb_init_with_tbl(char *tlb, unsigned long nslabs, int verbose)
 {
 	void *v_overflow_buffer;
@@ -320,6 +357,7 @@ int __init swiotlb_init_with_tbl(char *tlb, unsigned long nslabs, int verbose)
 	io_tlb_start = virt_to_phys(tlb);
 	io_tlb_end = io_tlb_start + bytes;
 
+	swiotlb_set_mem_attributes(tlb, bytes);
 	memset(tlb, 0, bytes);
 
 	/*
@@ -330,6 +368,8 @@ int __init swiotlb_init_with_tbl(char *tlb, unsigned long nslabs, int verbose)
 	if (!v_overflow_buffer)
 		goto cleanup2;
 
+	swiotlb_set_mem_attributes(v_overflow_buffer, io_tlb_overflow);
+	memset(v_overflow_buffer, 0, io_tlb_overflow);
 	io_tlb_overflow_buffer = virt_to_phys(v_overflow_buffer);
 
 	/*
@@ -581,7 +621,7 @@ phys_addr_t swiotlb_tbl_map_single(struct device *hwdev,
 		return SWIOTLB_MAP_ERROR;
 	}
 
-	start_dma_addr = phys_to_dma(hwdev, io_tlb_start);
+	start_dma_addr = swiotlb_phys_to_dma(hwdev, io_tlb_start);
 	return swiotlb_tbl_map_single(hwdev, start_dma_addr, phys, size,
 				      dir, attrs);
 }
@@ -702,7 +742,7 @@ void swiotlb_tbl_sync_single(struct device *hwdev, phys_addr_t tlb_addr,
 			goto err_warn;
 
 		ret = phys_to_virt(paddr);
-		dev_addr = phys_to_dma(hwdev, paddr);
+		dev_addr = swiotlb_phys_to_dma(hwdev, paddr);
 
 		/* Confirm address can be DMA'd by device */
 		if (dev_addr + size - 1 > dma_mask) {
@@ -812,10 +852,10 @@ dma_addr_t swiotlb_map_page(struct device *dev, struct page *page,
 	map = map_single(dev, phys, size, dir, attrs);
 	if (map == SWIOTLB_MAP_ERROR) {
 		swiotlb_full(dev, size, dir, 1);
-		return phys_to_dma(dev, io_tlb_overflow_buffer);
+		return swiotlb_phys_to_dma(dev, io_tlb_overflow_buffer);
 	}
 
-	dev_addr = phys_to_dma(dev, map);
+	dev_addr = swiotlb_phys_to_dma(dev, map);
 
 	/* Ensure that the address returned is DMA'ble */
 	if (dma_capable(dev, dev_addr, size))
@@ -824,7 +864,7 @@ dma_addr_t swiotlb_map_page(struct device *dev, struct page *page,
 	attrs |= DMA_ATTR_SKIP_CPU_SYNC;
 	swiotlb_tbl_unmap_single(dev, map, size, dir, attrs);
 
-	return phys_to_dma(dev, io_tlb_overflow_buffer);
+	return swiotlb_phys_to_dma(dev, io_tlb_overflow_buffer);
 }
 EXPORT_SYMBOL_GPL(swiotlb_map_page);
 
@@ -958,7 +998,7 @@ void swiotlb_unmap_page(struct device *hwdev, dma_addr_t dev_addr,
 				sg_dma_len(sgl) = 0;
 				return 0;
 			}
-			sg->dma_address = phys_to_dma(hwdev, map);
+			sg->dma_address = swiotlb_phys_to_dma(hwdev, map);
 		} else
 			sg->dma_address = dev_addr;
 		sg_dma_len(sg) = sg->length;
@@ -1026,7 +1066,7 @@ void swiotlb_unmap_page(struct device *hwdev, dma_addr_t dev_addr,
 int
 swiotlb_dma_mapping_error(struct device *hwdev, dma_addr_t dma_addr)
 {
-	return (dma_addr == phys_to_dma(hwdev, io_tlb_overflow_buffer));
+	return (dma_addr == swiotlb_phys_to_dma(hwdev, io_tlb_overflow_buffer));
 }
 EXPORT_SYMBOL(swiotlb_dma_mapping_error);
 
@@ -1039,6 +1079,6 @@ void swiotlb_unmap_page(struct device *hwdev, dma_addr_t dev_addr,
 int
 swiotlb_dma_supported(struct device *hwdev, u64 mask)
 {
-	return phys_to_dma(hwdev, io_tlb_end - 1) <= mask;
+	return swiotlb_phys_to_dma(hwdev, io_tlb_end - 1) <= mask;
 }
 EXPORT_SYMBOL(swiotlb_dma_supported);



From thomas.lendacky at amd.com  Wed Jun  7 12:17:45 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Wed, 07 Jun 2017 14:17:45 -0500
Subject: [PATCH v6 26/34] iommu/amd: Allow the AMD IOMMU to work with memory
 encryption
In-Reply-To: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170607191745.28645.81756.stgit@tlendack-t1.amdoffice.net>

The IOMMU is programmed with physical addresses for the various tables
and buffers that are used to communicate between the device and the
driver. When the driver allocates this memory it is encrypted. In order
for the IOMMU to access the memory as encrypted the encryption mask needs
to be included in these physical addresses during configuration.

The PTE entries created by the IOMMU should also include the encryption
mask so that when the device behind the IOMMU performs a DMA, the DMA
will be performed to encrypted memory.

Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/include/asm/mem_encrypt.h |    7 +++++++
 arch/x86/mm/mem_encrypt.c          |   30 ++++++++++++++++++++++++++++++
 drivers/iommu/amd_iommu.c          |   36 +++++++++++++++++++-----------------
 drivers/iommu/amd_iommu_init.c     |   18 ++++++++++++------
 drivers/iommu/amd_iommu_proto.h    |   10 ++++++++++
 drivers/iommu/amd_iommu_types.h    |    2 +-
 include/asm-generic/mem_encrypt.h  |    5 +++++
 7 files changed, 84 insertions(+), 24 deletions(-)

diff --git a/arch/x86/include/asm/mem_encrypt.h b/arch/x86/include/asm/mem_encrypt.h
index c7a2525..d86e544 100644
--- a/arch/x86/include/asm/mem_encrypt.h
+++ b/arch/x86/include/asm/mem_encrypt.h
@@ -31,6 +31,8 @@ void __init sme_early_decrypt(resource_size_t paddr,
 
 void __init sme_early_init(void);
 
+bool sme_iommu_supported(void);
+
 /* Architecture __weak replacement functions */
 void __init mem_encrypt_init(void);
 
@@ -62,6 +64,11 @@ static inline void __init sme_early_init(void)
 {
 }
 
+static inline bool sme_iommu_supported(void)
+{
+	return true;
+}
+
 #endif	/* CONFIG_AMD_MEM_ENCRYPT */
 
 static inline bool sme_active(void)
diff --git a/arch/x86/mm/mem_encrypt.c b/arch/x86/mm/mem_encrypt.c
index 5d7c51d..018b58a 100644
--- a/arch/x86/mm/mem_encrypt.c
+++ b/arch/x86/mm/mem_encrypt.c
@@ -197,6 +197,36 @@ void __init sme_early_init(void)
 		protection_map[i] = pgprot_encrypted(protection_map[i]);
 }
 
+bool sme_iommu_supported(void)
+{
+	struct cpuinfo_x86 *c = &boot_cpu_data;
+
+	if (!sme_me_mask || (c->x86 != 0x17))
+		return true;
+
+	/* For Fam17h, a specific level of support is required */
+	switch (c->microcode & 0xf000) {
+	case 0x0000:
+		return false;
+	case 0x1000:
+		switch (c->microcode & 0x0f00) {
+		case 0x0000:
+			return false;
+		case 0x0100:
+			if ((c->microcode & 0xff) < 0x26)
+				return false;
+			break;
+		case 0x0200:
+			if ((c->microcode & 0xff) < 0x05)
+				return false;
+			break;
+		}
+		break;
+	}
+
+	return true;
+}
+
 /* Architecture __weak replacement functions */
 void __init mem_encrypt_init(void)
 {
diff --git a/drivers/iommu/amd_iommu.c b/drivers/iommu/amd_iommu.c
index 63cacf5..94eb130 100644
--- a/drivers/iommu/amd_iommu.c
+++ b/drivers/iommu/amd_iommu.c
@@ -544,7 +544,7 @@ static void dump_dte_entry(u16 devid)
 
 static void dump_command(unsigned long phys_addr)
 {
-	struct iommu_cmd *cmd = phys_to_virt(phys_addr);
+	struct iommu_cmd *cmd = iommu_phys_to_virt(phys_addr);
 	int i;
 
 	for (i = 0; i < 4; ++i)
@@ -863,13 +863,15 @@ static void copy_cmd_to_buffer(struct amd_iommu *iommu,
 	writel(tail, iommu->mmio_base + MMIO_CMD_TAIL_OFFSET);
 }
 
-static void build_completion_wait(struct iommu_cmd *cmd, u64 address)
+static void build_completion_wait(struct iommu_cmd *cmd, volatile u64 *sem)
 {
+	u64 address = iommu_virt_to_phys((void *)sem);
+
 	WARN_ON(address & 0x7ULL);
 
 	memset(cmd, 0, sizeof(*cmd));
-	cmd->data[0] = lower_32_bits(__pa(address)) | CMD_COMPL_WAIT_STORE_MASK;
-	cmd->data[1] = upper_32_bits(__pa(address));
+	cmd->data[0] = lower_32_bits(address) | CMD_COMPL_WAIT_STORE_MASK;
+	cmd->data[1] = upper_32_bits(address);
 	cmd->data[2] = 1;
 	CMD_SET_TYPE(cmd, CMD_COMPL_WAIT);
 }
@@ -1033,7 +1035,7 @@ static int __iommu_queue_command_sync(struct amd_iommu *iommu,
 
 		iommu->cmd_sem = 0;
 
-		build_completion_wait(&sync_cmd, (u64)&iommu->cmd_sem);
+		build_completion_wait(&sync_cmd, &iommu->cmd_sem);
 		copy_cmd_to_buffer(iommu, &sync_cmd, tail);
 
 		if ((ret = wait_on_sem(&iommu->cmd_sem)) != 0)
@@ -1083,7 +1085,7 @@ static int iommu_completion_wait(struct amd_iommu *iommu)
 		return 0;
 
 
-	build_completion_wait(&cmd, (u64)&iommu->cmd_sem);
+	build_completion_wait(&cmd, &iommu->cmd_sem);
 
 	spin_lock_irqsave(&iommu->lock, flags);
 
@@ -1328,7 +1330,7 @@ static bool increase_address_space(struct protection_domain *domain,
 		return false;
 
 	*pte             = PM_LEVEL_PDE(domain->mode,
-					virt_to_phys(domain->pt_root));
+					iommu_virt_to_phys(domain->pt_root));
 	domain->pt_root  = pte;
 	domain->mode    += 1;
 	domain->updated  = true;
@@ -1365,7 +1367,7 @@ static u64 *alloc_pte(struct protection_domain *domain,
 			if (!page)
 				return NULL;
 
-			__npte = PM_LEVEL_PDE(level, virt_to_phys(page));
+			__npte = PM_LEVEL_PDE(level, iommu_virt_to_phys(page));
 
 			/* pte could have been changed somewhere. */
 			if (cmpxchg64(pte, __pte, __npte) != __pte) {
@@ -1481,10 +1483,10 @@ static int iommu_map_page(struct protection_domain *dom,
 			return -EBUSY;
 
 	if (count > 1) {
-		__pte = PAGE_SIZE_PTE(phys_addr, page_size);
+		__pte = PAGE_SIZE_PTE(__sme_set(phys_addr), page_size);
 		__pte |= PM_LEVEL_ENC(7) | IOMMU_PTE_P | IOMMU_PTE_FC;
 	} else
-		__pte = phys_addr | IOMMU_PTE_P | IOMMU_PTE_FC;
+		__pte = __sme_set(phys_addr) | IOMMU_PTE_P | IOMMU_PTE_FC;
 
 	if (prot & IOMMU_PROT_IR)
 		__pte |= IOMMU_PTE_IR;
@@ -1700,7 +1702,7 @@ static void free_gcr3_tbl_level1(u64 *tbl)
 		if (!(tbl[i] & GCR3_VALID))
 			continue;
 
-		ptr = __va(tbl[i] & PAGE_MASK);
+		ptr = iommu_phys_to_virt(tbl[i] & PAGE_MASK);
 
 		free_page((unsigned long)ptr);
 	}
@@ -1715,7 +1717,7 @@ static void free_gcr3_tbl_level2(u64 *tbl)
 		if (!(tbl[i] & GCR3_VALID))
 			continue;
 
-		ptr = __va(tbl[i] & PAGE_MASK);
+		ptr = iommu_phys_to_virt(tbl[i] & PAGE_MASK);
 
 		free_gcr3_tbl_level1(ptr);
 	}
@@ -1807,7 +1809,7 @@ static void set_dte_entry(u16 devid, struct protection_domain *domain, bool ats)
 	u64 flags = 0;
 
 	if (domain->mode != PAGE_MODE_NONE)
-		pte_root = virt_to_phys(domain->pt_root);
+		pte_root = iommu_virt_to_phys(domain->pt_root);
 
 	pte_root |= (domain->mode & DEV_ENTRY_MODE_MASK)
 		    << DEV_ENTRY_MODE_SHIFT;
@@ -1819,7 +1821,7 @@ static void set_dte_entry(u16 devid, struct protection_domain *domain, bool ats)
 		flags |= DTE_FLAG_IOTLB;
 
 	if (domain->flags & PD_IOMMUV2_MASK) {
-		u64 gcr3 = __pa(domain->gcr3_tbl);
+		u64 gcr3 = iommu_virt_to_phys(domain->gcr3_tbl);
 		u64 glx  = domain->glx;
 		u64 tmp;
 
@@ -3470,10 +3472,10 @@ static u64 *__get_gcr3_pte(u64 *root, int level, int pasid, bool alloc)
 			if (root == NULL)
 				return NULL;
 
-			*pte = __pa(root) | GCR3_VALID;
+			*pte = iommu_virt_to_phys(root) | GCR3_VALID;
 		}
 
-		root = __va(*pte & PAGE_MASK);
+		root = iommu_phys_to_virt(*pte & PAGE_MASK);
 
 		level -= 1;
 	}
@@ -3652,7 +3654,7 @@ static void set_dte_irq_entry(u16 devid, struct irq_remap_table *table)
 
 	dte	= amd_iommu_dev_table[devid].data[2];
 	dte	&= ~DTE_IRQ_PHYS_ADDR_MASK;
-	dte	|= virt_to_phys(table->table);
+	dte	|= iommu_virt_to_phys(table->table);
 	dte	|= DTE_IRQ_REMAP_INTCTL;
 	dte	|= DTE_IRQ_TABLE_LEN;
 	dte	|= DTE_IRQ_REMAP_ENABLE;
diff --git a/drivers/iommu/amd_iommu_init.c b/drivers/iommu/amd_iommu_init.c
index 5a11328..2870a6b 100644
--- a/drivers/iommu/amd_iommu_init.c
+++ b/drivers/iommu/amd_iommu_init.c
@@ -29,6 +29,7 @@
 #include <linux/export.h>
 #include <linux/iommu.h>
 #include <linux/kmemleak.h>
+#include <linux/mem_encrypt.h>
 #include <asm/pci-direct.h>
 #include <asm/iommu.h>
 #include <asm/gart.h>
@@ -346,7 +347,7 @@ static void iommu_set_device_table(struct amd_iommu *iommu)
 
 	BUG_ON(iommu->mmio_base == NULL);
 
-	entry = virt_to_phys(amd_iommu_dev_table);
+	entry = iommu_virt_to_phys(amd_iommu_dev_table);
 	entry |= (dev_table_size >> 12) - 1;
 	memcpy_toio(iommu->mmio_base + MMIO_DEV_TABLE_OFFSET,
 			&entry, sizeof(entry));
@@ -602,7 +603,7 @@ static void iommu_enable_command_buffer(struct amd_iommu *iommu)
 
 	BUG_ON(iommu->cmd_buf == NULL);
 
-	entry = (u64)virt_to_phys(iommu->cmd_buf);
+	entry = iommu_virt_to_phys(iommu->cmd_buf);
 	entry |= MMIO_CMD_SIZE_512;
 
 	memcpy_toio(iommu->mmio_base + MMIO_CMD_BUF_OFFSET,
@@ -631,7 +632,7 @@ static void iommu_enable_event_buffer(struct amd_iommu *iommu)
 
 	BUG_ON(iommu->evt_buf == NULL);
 
-	entry = (u64)virt_to_phys(iommu->evt_buf) | EVT_LEN_MASK;
+	entry = iommu_virt_to_phys(iommu->evt_buf) | EVT_LEN_MASK;
 
 	memcpy_toio(iommu->mmio_base + MMIO_EVT_BUF_OFFSET,
 		    &entry, sizeof(entry));
@@ -664,7 +665,7 @@ static void iommu_enable_ppr_log(struct amd_iommu *iommu)
 	if (iommu->ppr_log == NULL)
 		return;
 
-	entry = (u64)virt_to_phys(iommu->ppr_log) | PPR_LOG_SIZE_512;
+	entry = iommu_virt_to_phys(iommu->ppr_log) | PPR_LOG_SIZE_512;
 
 	memcpy_toio(iommu->mmio_base + MMIO_PPR_LOG_OFFSET,
 		    &entry, sizeof(entry));
@@ -744,10 +745,10 @@ static int iommu_init_ga_log(struct amd_iommu *iommu)
 	if (!iommu->ga_log_tail)
 		goto err_out;
 
-	entry = (u64)virt_to_phys(iommu->ga_log) | GA_LOG_SIZE_512;
+	entry = iommu_virt_to_phys(iommu->ga_log) | GA_LOG_SIZE_512;
 	memcpy_toio(iommu->mmio_base + MMIO_GA_LOG_BASE_OFFSET,
 		    &entry, sizeof(entry));
-	entry = ((u64)virt_to_phys(iommu->ga_log) & 0xFFFFFFFFFFFFFULL) & ~7ULL;
+	entry = (iommu_virt_to_phys(iommu->ga_log) & 0xFFFFFFFFFFFFFULL) & ~7ULL;
 	memcpy_toio(iommu->mmio_base + MMIO_GA_LOG_TAIL_OFFSET,
 		    &entry, sizeof(entry));
 	writel(0x00, iommu->mmio_base + MMIO_GA_HEAD_OFFSET);
@@ -2552,6 +2553,11 @@ int __init amd_iommu_detect(void)
 	if (amd_iommu_disabled)
 		return -ENODEV;
 
+	if (!sme_iommu_supported()) {
+		pr_notice("AMD-Vi: IOMMU not supported when SME is active\n");
+		return -ENODEV;
+	}
+
 	ret = iommu_go_to_state(IOMMU_IVRS_DETECTED);
 	if (ret)
 		return ret;
diff --git a/drivers/iommu/amd_iommu_proto.h b/drivers/iommu/amd_iommu_proto.h
index 466260f..3f12fb2 100644
--- a/drivers/iommu/amd_iommu_proto.h
+++ b/drivers/iommu/amd_iommu_proto.h
@@ -87,4 +87,14 @@ static inline bool iommu_feature(struct amd_iommu *iommu, u64 f)
 	return !!(iommu->features & f);
 }
 
+static inline u64 iommu_virt_to_phys(void *vaddr)
+{
+	return (u64)__sme_set(virt_to_phys(vaddr));
+}
+
+static inline void *iommu_phys_to_virt(unsigned long paddr)
+{
+	return phys_to_virt(__sme_clr(paddr));
+}
+
 #endif /* _ASM_X86_AMD_IOMMU_PROTO_H  */
diff --git a/drivers/iommu/amd_iommu_types.h b/drivers/iommu/amd_iommu_types.h
index 4de8f41..3ce587d 100644
--- a/drivers/iommu/amd_iommu_types.h
+++ b/drivers/iommu/amd_iommu_types.h
@@ -343,7 +343,7 @@
 
 #define IOMMU_PAGE_MASK (((1ULL << 52) - 1) & ~0xfffULL)
 #define IOMMU_PTE_PRESENT(pte) ((pte) & IOMMU_PTE_P)
-#define IOMMU_PTE_PAGE(pte) (phys_to_virt((pte) & IOMMU_PAGE_MASK))
+#define IOMMU_PTE_PAGE(pte) (iommu_phys_to_virt((pte) & IOMMU_PAGE_MASK))
 #define IOMMU_PTE_MODE(pte) (((pte) >> 9) & 0x07)
 
 #define IOMMU_PROT_MASK 0x03
diff --git a/include/asm-generic/mem_encrypt.h b/include/asm-generic/mem_encrypt.h
index fb02ff0..bbc49e1 100644
--- a/include/asm-generic/mem_encrypt.h
+++ b/include/asm-generic/mem_encrypt.h
@@ -27,6 +27,11 @@ static inline u64 sme_dma_mask(void)
 	return 0ULL;
 }
 
+static inline bool sme_iommu_supported(void)
+{
+	return true;
+}
+
 /*
  * The __sme_set() and __sme_clr() macros are useful for adding or removing
  * the encryption mask from a value (e.g. when dealing with pagetable



From thomas.lendacky at amd.com  Wed Jun  7 12:17:32 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Wed, 07 Jun 2017 14:17:32 -0500
Subject: [PATCH v6 25/34] swiotlb: Add warnings for use of bounce buffers
 with SME
In-Reply-To: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170607191732.28645.42876.stgit@tlendack-t1.amdoffice.net>

Add warnings to let the user know when bounce buffers are being used for
DMA when SME is active.  Since the bounce buffers are not in encrypted
memory, these notifications are to allow the user to determine some
appropriate action - if necessary.

Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/include/asm/mem_encrypt.h |    8 ++++++++
 include/asm-generic/mem_encrypt.h  |    5 +++++
 include/linux/dma-mapping.h        |    9 +++++++++
 lib/swiotlb.c                      |    3 +++
 4 files changed, 25 insertions(+)

diff --git a/arch/x86/include/asm/mem_encrypt.h b/arch/x86/include/asm/mem_encrypt.h
index f1215a4..c7a2525 100644
--- a/arch/x86/include/asm/mem_encrypt.h
+++ b/arch/x86/include/asm/mem_encrypt.h
@@ -69,6 +69,14 @@ static inline bool sme_active(void)
 	return !!sme_me_mask;
 }
 
+static inline u64 sme_dma_mask(void)
+{
+	if (!sme_me_mask)
+		return 0ULL;
+
+	return ((u64)sme_me_mask << 1) - 1;
+}
+
 /*
  * The __sme_pa() and __sme_pa_nodebug() macros are meant for use when
  * writing to or comparing values from the cr3 register.  Having the
diff --git a/include/asm-generic/mem_encrypt.h b/include/asm-generic/mem_encrypt.h
index b55c3f9..fb02ff0 100644
--- a/include/asm-generic/mem_encrypt.h
+++ b/include/asm-generic/mem_encrypt.h
@@ -22,6 +22,11 @@ static inline bool sme_active(void)
 	return false;
 }
 
+static inline u64 sme_dma_mask(void)
+{
+	return 0ULL;
+}
+
 /*
  * The __sme_set() and __sme_clr() macros are useful for adding or removing
  * the encryption mask from a value (e.g. when dealing with pagetable
diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h
index 4f3eece..e2c5fda 100644
--- a/include/linux/dma-mapping.h
+++ b/include/linux/dma-mapping.h
@@ -10,6 +10,7 @@
 #include <linux/scatterlist.h>
 #include <linux/kmemcheck.h>
 #include <linux/bug.h>
+#include <linux/mem_encrypt.h>
 
 /**
  * List of possible attributes associated with a DMA mapping. The semantics
@@ -577,6 +578,10 @@ static inline int dma_set_mask(struct device *dev, u64 mask)
 
 	if (!dev->dma_mask || !dma_supported(dev, mask))
 		return -EIO;
+
+	if (sme_active() && (mask < sme_dma_mask()))
+		dev_warn(dev, "SME is active, device will require DMA bounce buffers\n");
+
 	*dev->dma_mask = mask;
 	return 0;
 }
@@ -596,6 +601,10 @@ static inline int dma_set_coherent_mask(struct device *dev, u64 mask)
 {
 	if (!dma_supported(dev, mask))
 		return -EIO;
+
+	if (sme_active() && (mask < sme_dma_mask()))
+		dev_warn(dev, "SME is active, device will require DMA bounce buffers\n");
+
 	dev->coherent_dma_mask = mask;
 	return 0;
 }
diff --git a/lib/swiotlb.c b/lib/swiotlb.c
index 74d6557..f78906a 100644
--- a/lib/swiotlb.c
+++ b/lib/swiotlb.c
@@ -509,6 +509,9 @@ phys_addr_t swiotlb_tbl_map_single(struct device *hwdev,
 	if (no_iotlb_memory)
 		panic("Can not allocate SWIOTLB buffer earlier and can't now provide you with the DMA bounce buffer");
 
+	if (sme_active())
+		pr_warn_once("SME is active and system is using DMA bounce buffers\n");
+
 	mask = dma_get_seg_boundary(hwdev);
 
 	tbl_dma_addr &= mask;



From thomas.lendacky at amd.com  Wed Jun  7 12:17:54 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Wed, 07 Jun 2017 14:17:54 -0500
Subject: [PATCH v6 27/34] x86, realmode: Check for memory encryption on the APs
In-Reply-To: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170607191754.28645.22939.stgit@tlendack-t1.amdoffice.net>

Add support to check if memory encryption is active in the kernel and that
it has been enabled on the AP. If memory encryption is active in the kernel
but has not been enabled on the AP, then set the memory encryption bit (bit
23) of MSR_K8_SYSCFG to enable memory encryption on that AP and allow the
AP to continue start up.

Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/include/asm/realmode.h      |   12 ++++++++++++
 arch/x86/realmode/init.c             |    4 ++++
 arch/x86/realmode/rm/trampoline_64.S |   24 ++++++++++++++++++++++++
 3 files changed, 40 insertions(+)

diff --git a/arch/x86/include/asm/realmode.h b/arch/x86/include/asm/realmode.h
index 230e190..90d9152 100644
--- a/arch/x86/include/asm/realmode.h
+++ b/arch/x86/include/asm/realmode.h
@@ -1,6 +1,15 @@
 #ifndef _ARCH_X86_REALMODE_H
 #define _ARCH_X86_REALMODE_H
 
+/*
+ * Flag bit definitions for use with the flags field of the trampoline header
+ * in the CONFIG_X86_64 variant.
+ */
+#define TH_FLAGS_SME_ACTIVE_BIT		0
+#define TH_FLAGS_SME_ACTIVE		BIT(TH_FLAGS_SME_ACTIVE_BIT)
+
+#ifndef __ASSEMBLY__
+
 #include <linux/types.h>
 #include <asm/io.h>
 
@@ -38,6 +47,7 @@ struct trampoline_header {
 	u64 start;
 	u64 efer;
 	u32 cr4;
+	u32 flags;
 #endif
 };
 
@@ -69,4 +79,6 @@ static inline size_t real_mode_size_needed(void)
 void set_real_mode_mem(phys_addr_t mem, size_t size);
 void reserve_real_mode(void);
 
+#endif /* __ASSEMBLY__ */
+
 #endif /* _ARCH_X86_REALMODE_H */
diff --git a/arch/x86/realmode/init.c b/arch/x86/realmode/init.c
index 195ba29..60373d0 100644
--- a/arch/x86/realmode/init.c
+++ b/arch/x86/realmode/init.c
@@ -101,6 +101,10 @@ static void __init setup_real_mode(void)
 	trampoline_cr4_features = &trampoline_header->cr4;
 	*trampoline_cr4_features = mmu_cr4_features;
 
+	trampoline_header->flags = 0;
+	if (sme_active())
+		trampoline_header->flags |= TH_FLAGS_SME_ACTIVE;
+
 	trampoline_pgd = (u64 *) __va(real_mode_header->trampoline_pgd);
 	trampoline_pgd[0] = trampoline_pgd_entry.pgd;
 	trampoline_pgd[511] = init_level4_pgt[511].pgd;
diff --git a/arch/x86/realmode/rm/trampoline_64.S b/arch/x86/realmode/rm/trampoline_64.S
index dac7b20..614fd70 100644
--- a/arch/x86/realmode/rm/trampoline_64.S
+++ b/arch/x86/realmode/rm/trampoline_64.S
@@ -30,6 +30,7 @@
 #include <asm/msr.h>
 #include <asm/segment.h>
 #include <asm/processor-flags.h>
+#include <asm/realmode.h>
 #include "realmode.h"
 
 	.text
@@ -92,6 +93,28 @@ ENTRY(startup_32)
 	movl	%edx, %fs
 	movl	%edx, %gs
 
+	/*
+	 * Check for memory encryption support. This is a safety net in
+	 * case BIOS hasn't done the necessary step of setting the bit in
+	 * the MSR for this AP. If SME is active and we've gotten this far
+	 * then it is safe for us to set the MSR bit and continue. If we
+	 * don't we'll eventually crash trying to execute encrypted
+	 * instructions.
+	 */
+	bt	$TH_FLAGS_SME_ACTIVE_BIT, pa_tr_flags
+	jnc	.Ldone
+	movl	$MSR_K8_SYSCFG, %ecx
+	rdmsr
+	bts	$MSR_K8_SYSCFG_MEM_ENCRYPT_BIT, %eax
+	jc	.Ldone
+
+	/*
+	 * Memory encryption is enabled but the SME enable bit for this
+	 * CPU has has not been set.  It is safe to set it, so do so.
+	 */
+	wrmsr
+.Ldone:
+
 	movl	pa_tr_cr4, %eax
 	movl	%eax, %cr4		# Enable PAE mode
 
@@ -147,6 +170,7 @@ GLOBAL(trampoline_header)
 	tr_start:		.space	8
 	GLOBAL(tr_efer)		.space	8
 	GLOBAL(tr_cr4)		.space	4
+	GLOBAL(tr_flags)	.space	4
 END(trampoline_header)
 
 #include "trampoline_common.S"



From thomas.lendacky at amd.com  Wed Jun  7 12:18:04 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Wed, 07 Jun 2017 14:18:04 -0500
Subject: [PATCH v6 28/34] x86, drm,
 fbdev: Do not specify encrypted memory for video mappings
In-Reply-To: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170607191804.28645.72185.stgit@tlendack-t1.amdoffice.net>

Since video memory needs to be accessed decrypted, be sure that the
memory encryption mask is not set for the video ranges.

Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/include/asm/vga.h       |   14 +++++++++++++-
 arch/x86/mm/pageattr.c           |    2 ++
 drivers/gpu/drm/drm_gem.c        |    2 ++
 drivers/gpu/drm/drm_vm.c         |    4 ++++
 drivers/gpu/drm/ttm/ttm_bo_vm.c  |    7 +++++--
 drivers/gpu/drm/udl/udl_fb.c     |    4 ++++
 drivers/video/fbdev/core/fbmem.c |   12 ++++++++++++
 7 files changed, 42 insertions(+), 3 deletions(-)

diff --git a/arch/x86/include/asm/vga.h b/arch/x86/include/asm/vga.h
index c4b9dc2..9f42bee 100644
--- a/arch/x86/include/asm/vga.h
+++ b/arch/x86/include/asm/vga.h
@@ -7,12 +7,24 @@
 #ifndef _ASM_X86_VGA_H
 #define _ASM_X86_VGA_H
 
+#include <asm/set_memory.h>
+
 /*
  *	On the PC, we can just recalculate addresses and then
  *	access the videoram directly without any black magic.
+ *	To support memory encryption however, we need to access
+ *	the videoram as decrypted memory.
  */
 
-#define VGA_MAP_MEM(x, s) (unsigned long)phys_to_virt(x)
+#define VGA_MAP_MEM(x, s)					\
+({								\
+	unsigned long start = (unsigned long)phys_to_virt(x);	\
+								\
+	if (IS_ENABLED(CONFIG_AMD_MEM_ENCRYPT))			\
+		set_memory_decrypted(start, (s) >> PAGE_SHIFT);	\
+								\
+	start;							\
+})
 
 #define vga_readb(x) (*(x))
 #define vga_writeb(x, y) (*(y) = (x))
diff --git a/arch/x86/mm/pageattr.c b/arch/x86/mm/pageattr.c
index d9e09fb..13fc5db 100644
--- a/arch/x86/mm/pageattr.c
+++ b/arch/x86/mm/pageattr.c
@@ -1825,11 +1825,13 @@ int set_memory_encrypted(unsigned long addr, int numpages)
 {
 	return __set_memory_enc_dec(addr, numpages, true);
 }
+EXPORT_SYMBOL_GPL(set_memory_encrypted);
 
 int set_memory_decrypted(unsigned long addr, int numpages)
 {
 	return __set_memory_enc_dec(addr, numpages, false);
 }
+EXPORT_SYMBOL_GPL(set_memory_decrypted);
 
 int set_pages_uc(struct page *page, int numpages)
 {
diff --git a/drivers/gpu/drm/drm_gem.c b/drivers/gpu/drm/drm_gem.c
index b1e28c9..019f48c 100644
--- a/drivers/gpu/drm/drm_gem.c
+++ b/drivers/gpu/drm/drm_gem.c
@@ -36,6 +36,7 @@
 #include <linux/pagemap.h>
 #include <linux/shmem_fs.h>
 #include <linux/dma-buf.h>
+#include <linux/mem_encrypt.h>
 #include <drm/drmP.h>
 #include <drm/drm_vma_manager.h>
 #include <drm/drm_gem.h>
@@ -928,6 +929,7 @@ int drm_gem_mmap_obj(struct drm_gem_object *obj, unsigned long obj_size,
 	vma->vm_ops = dev->driver->gem_vm_ops;
 	vma->vm_private_data = obj;
 	vma->vm_page_prot = pgprot_writecombine(vm_get_page_prot(vma->vm_flags));
+	vma->vm_page_prot = pgprot_decrypted(vma->vm_page_prot);
 
 	/* Take a ref for this mapping of the object, so that the fault
 	 * handler can dereference the mmap offset's pointer to the object.
diff --git a/drivers/gpu/drm/drm_vm.c b/drivers/gpu/drm/drm_vm.c
index 1170b32..ed4bcbf 100644
--- a/drivers/gpu/drm/drm_vm.c
+++ b/drivers/gpu/drm/drm_vm.c
@@ -40,6 +40,7 @@
 #include <linux/efi.h>
 #include <linux/slab.h>
 #endif
+#include <linux/mem_encrypt.h>
 #include <asm/pgtable.h>
 #include "drm_internal.h"
 #include "drm_legacy.h"
@@ -58,6 +59,9 @@ static pgprot_t drm_io_prot(struct drm_local_map *map,
 {
 	pgprot_t tmp = vm_get_page_prot(vma->vm_flags);
 
+	/* We don't want graphics memory to be mapped encrypted */
+	tmp = pgprot_decrypted(tmp);
+
 #if defined(__i386__) || defined(__x86_64__) || defined(__powerpc__)
 	if (map->type == _DRM_REGISTERS && !(map->flags & _DRM_WRITE_COMBINING))
 		tmp = pgprot_noncached(tmp);
diff --git a/drivers/gpu/drm/ttm/ttm_bo_vm.c b/drivers/gpu/drm/ttm/ttm_bo_vm.c
index 9f53df9..622dab6 100644
--- a/drivers/gpu/drm/ttm/ttm_bo_vm.c
+++ b/drivers/gpu/drm/ttm/ttm_bo_vm.c
@@ -39,6 +39,7 @@
 #include <linux/rbtree.h>
 #include <linux/module.h>
 #include <linux/uaccess.h>
+#include <linux/mem_encrypt.h>
 
 #define TTM_BO_VM_NUM_PREFAULT 16
 
@@ -230,9 +231,11 @@ static int ttm_bo_vm_fault(struct vm_fault *vmf)
 	 * first page.
 	 */
 	for (i = 0; i < TTM_BO_VM_NUM_PREFAULT; ++i) {
-		if (bo->mem.bus.is_iomem)
+		if (bo->mem.bus.is_iomem) {
+			/* Iomem should not be marked encrypted */
+			cvma.vm_page_prot = pgprot_decrypted(cvma.vm_page_prot);
 			pfn = bdev->driver->io_mem_pfn(bo, page_offset);
-		else {
+		} else {
 			page = ttm->pages[page_offset];
 			if (unlikely(!page && i == 0)) {
 				retval = VM_FAULT_OOM;
diff --git a/drivers/gpu/drm/udl/udl_fb.c b/drivers/gpu/drm/udl/udl_fb.c
index 4a65003..92e1690 100644
--- a/drivers/gpu/drm/udl/udl_fb.c
+++ b/drivers/gpu/drm/udl/udl_fb.c
@@ -14,6 +14,7 @@
 #include <linux/slab.h>
 #include <linux/fb.h>
 #include <linux/dma-buf.h>
+#include <linux/mem_encrypt.h>
 
 #include <drm/drmP.h>
 #include <drm/drm_crtc.h>
@@ -169,6 +170,9 @@ static int udl_fb_mmap(struct fb_info *info, struct vm_area_struct *vma)
 	pr_notice("mmap() framebuffer addr:%lu size:%lu\n",
 		  pos, size);
 
+	/* We don't want the framebuffer to be mapped encrypted */
+	vma->vm_page_prot = pgprot_decrypted(vma->vm_page_prot);
+
 	while (size > 0) {
 		page = vmalloc_to_pfn((void *)pos);
 		if (remap_pfn_range(vma, start, page, PAGE_SIZE, PAGE_SHARED))
diff --git a/drivers/video/fbdev/core/fbmem.c b/drivers/video/fbdev/core/fbmem.c
index 069fe79..b5e7c33 100644
--- a/drivers/video/fbdev/core/fbmem.c
+++ b/drivers/video/fbdev/core/fbmem.c
@@ -32,6 +32,7 @@
 #include <linux/device.h>
 #include <linux/efi.h>
 #include <linux/fb.h>
+#include <linux/mem_encrypt.h>
 
 #include <asm/fb.h>
 
@@ -1405,6 +1406,12 @@ static long fb_compat_ioctl(struct file *file, unsigned int cmd,
 	mutex_lock(&info->mm_lock);
 	if (fb->fb_mmap) {
 		int res;
+
+		/*
+		 * The framebuffer needs to be accessed decrypted, be sure
+		 * SME protection is removed ahead of the call
+		 */
+		vma->vm_page_prot = pgprot_decrypted(vma->vm_page_prot);
 		res = fb->fb_mmap(info, vma);
 		mutex_unlock(&info->mm_lock);
 		return res;
@@ -1430,6 +1437,11 @@ static long fb_compat_ioctl(struct file *file, unsigned int cmd,
 	mutex_unlock(&info->mm_lock);
 
 	vma->vm_page_prot = vm_get_page_prot(vma->vm_flags);
+	/*
+	 * The framebuffer needs to be accessed decrypted, be sure
+	 * SME protection is removed
+	 */
+	vma->vm_page_prot = pgprot_decrypted(vma->vm_page_prot);
 	fb_pgprotect(file, vma, start);
 
 	return vm_iomap_memory(vma, start, len);



From thomas.lendacky at amd.com  Wed Jun  7 12:18:15 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Wed, 07 Jun 2017 14:18:15 -0500
Subject: [PATCH v6 29/34] kvm: x86: svm: Support Secure Memory Encryption
 within KVM
In-Reply-To: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170607191815.28645.9054.stgit@tlendack-t1.amdoffice.net>

Update the KVM support to work with SME. The VMCB has a number of fields
where physical addresses are used and these addresses must contain the
memory encryption mask in order to properly access the encrypted memory.
Also, use the memory encryption mask when creating and using the nested
page tables.

Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/include/asm/kvm_host.h |    2 +-
 arch/x86/kvm/mmu.c              |   12 ++++++++----
 arch/x86/kvm/mmu.h              |    2 +-
 arch/x86/kvm/svm.c              |   35 ++++++++++++++++++-----------------
 arch/x86/kvm/vmx.c              |    3 ++-
 arch/x86/kvm/x86.c              |    3 ++-
 6 files changed, 32 insertions(+), 25 deletions(-)

diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index 695605e..6d1267f 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -1069,7 +1069,7 @@ struct kvm_arch_async_pf {
 void kvm_mmu_uninit_vm(struct kvm *kvm);
 void kvm_mmu_set_mask_ptes(u64 user_mask, u64 accessed_mask,
 		u64 dirty_mask, u64 nx_mask, u64 x_mask, u64 p_mask,
-		u64 acc_track_mask);
+		u64 acc_track_mask, u64 me_mask);
 
 void kvm_mmu_reset_context(struct kvm_vcpu *vcpu);
 void kvm_mmu_slot_remove_write_access(struct kvm *kvm,
diff --git a/arch/x86/kvm/mmu.c b/arch/x86/kvm/mmu.c
index 5d3376f..892b7bd 100644
--- a/arch/x86/kvm/mmu.c
+++ b/arch/x86/kvm/mmu.c
@@ -107,7 +107,7 @@ enum {
 	(((address) >> PT32_LEVEL_SHIFT(level)) & ((1 << PT32_LEVEL_BITS) - 1))
 
 
-#define PT64_BASE_ADDR_MASK (((1ULL << 52) - 1) & ~(u64)(PAGE_SIZE-1))
+#define PT64_BASE_ADDR_MASK __sme_clr((((1ULL << 52) - 1) & ~(u64)(PAGE_SIZE-1)))
 #define PT64_DIR_BASE_ADDR_MASK \
 	(PT64_BASE_ADDR_MASK & ~((1ULL << (PAGE_SHIFT + PT64_LEVEL_BITS)) - 1))
 #define PT64_LVL_ADDR_MASK(level) \
@@ -125,7 +125,7 @@ enum {
 					    * PT32_LEVEL_BITS))) - 1))
 
 #define PT64_PERM_MASK (PT_PRESENT_MASK | PT_WRITABLE_MASK | shadow_user_mask \
-			| shadow_x_mask | shadow_nx_mask)
+			| shadow_x_mask | shadow_nx_mask | shadow_me_mask)
 
 #define ACC_EXEC_MASK    1
 #define ACC_WRITE_MASK   PT_WRITABLE_MASK
@@ -184,6 +184,7 @@ struct kvm_shadow_walk_iterator {
 static u64 __read_mostly shadow_dirty_mask;
 static u64 __read_mostly shadow_mmio_mask;
 static u64 __read_mostly shadow_present_mask;
+static u64 __read_mostly shadow_me_mask;
 
 /*
  * The mask/value to distinguish a PTE that has been marked not-present for
@@ -317,7 +318,7 @@ static bool check_mmio_spte(struct kvm_vcpu *vcpu, u64 spte)
 
 void kvm_mmu_set_mask_ptes(u64 user_mask, u64 accessed_mask,
 		u64 dirty_mask, u64 nx_mask, u64 x_mask, u64 p_mask,
-		u64 acc_track_mask)
+		u64 acc_track_mask, u64 me_mask)
 {
 	if (acc_track_mask != 0)
 		acc_track_mask |= SPTE_SPECIAL_MASK;
@@ -330,6 +331,7 @@ void kvm_mmu_set_mask_ptes(u64 user_mask, u64 accessed_mask,
 	shadow_present_mask = p_mask;
 	shadow_acc_track_mask = acc_track_mask;
 	WARN_ON(shadow_accessed_mask != 0 && shadow_acc_track_mask != 0);
+	shadow_me_mask = me_mask;
 }
 EXPORT_SYMBOL_GPL(kvm_mmu_set_mask_ptes);
 
@@ -2398,7 +2400,8 @@ static void link_shadow_page(struct kvm_vcpu *vcpu, u64 *sptep,
 	BUILD_BUG_ON(VMX_EPT_WRITABLE_MASK != PT_WRITABLE_MASK);
 
 	spte = __pa(sp->spt) | shadow_present_mask | PT_WRITABLE_MASK |
-	       shadow_user_mask | shadow_x_mask | shadow_accessed_mask;
+	       shadow_user_mask | shadow_x_mask | shadow_accessed_mask |
+	       shadow_me_mask;
 
 	mmu_spte_set(sptep, spte);
 
@@ -2700,6 +2703,7 @@ static int set_spte(struct kvm_vcpu *vcpu, u64 *sptep,
 		pte_access &= ~ACC_WRITE_MASK;
 
 	spte |= (u64)pfn << PAGE_SHIFT;
+	spte |= shadow_me_mask;
 
 	if (pte_access & ACC_WRITE_MASK) {
 
diff --git a/arch/x86/kvm/mmu.h b/arch/x86/kvm/mmu.h
index 2797580..9694ff9 100644
--- a/arch/x86/kvm/mmu.h
+++ b/arch/x86/kvm/mmu.h
@@ -48,7 +48,7 @@
 
 static inline u64 rsvd_bits(int s, int e)
 {
-	return ((1ULL << (e - s + 1)) - 1) << s;
+	return __sme_clr(((1ULL << (e - s + 1)) - 1) << s);
 }
 
 void kvm_mmu_set_mmio_spte_mask(u64 mmio_mask);
diff --git a/arch/x86/kvm/svm.c b/arch/x86/kvm/svm.c
index ba9891a..d2e9fca 100644
--- a/arch/x86/kvm/svm.c
+++ b/arch/x86/kvm/svm.c
@@ -1138,9 +1138,9 @@ static void avic_init_vmcb(struct vcpu_svm *svm)
 {
 	struct vmcb *vmcb = svm->vmcb;
 	struct kvm_arch *vm_data = &svm->vcpu.kvm->arch;
-	phys_addr_t bpa = page_to_phys(svm->avic_backing_page);
-	phys_addr_t lpa = page_to_phys(vm_data->avic_logical_id_table_page);
-	phys_addr_t ppa = page_to_phys(vm_data->avic_physical_id_table_page);
+	phys_addr_t bpa = __sme_set(page_to_phys(svm->avic_backing_page));
+	phys_addr_t lpa = __sme_set(page_to_phys(vm_data->avic_logical_id_table_page));
+	phys_addr_t ppa = __sme_set(page_to_phys(vm_data->avic_physical_id_table_page));
 
 	vmcb->control.avic_backing_page = bpa & AVIC_HPA_MASK;
 	vmcb->control.avic_logical_id = lpa & AVIC_HPA_MASK;
@@ -1203,8 +1203,8 @@ static void init_vmcb(struct vcpu_svm *svm)
 		set_intercept(svm, INTERCEPT_MWAIT);
 	}
 
-	control->iopm_base_pa = iopm_base;
-	control->msrpm_base_pa = __pa(svm->msrpm);
+	control->iopm_base_pa = __sme_set(iopm_base);
+	control->msrpm_base_pa = __sme_set(__pa(svm->msrpm));
 	control->int_ctl = V_INTR_MASKING_MASK;
 
 	init_seg(&save->es);
@@ -1338,9 +1338,9 @@ static int avic_init_backing_page(struct kvm_vcpu *vcpu)
 		return -EINVAL;
 
 	new_entry = READ_ONCE(*entry);
-	new_entry = (page_to_phys(svm->avic_backing_page) &
-		     AVIC_PHYSICAL_ID_ENTRY_BACKING_PAGE_MASK) |
-		     AVIC_PHYSICAL_ID_ENTRY_VALID_MASK;
+	new_entry = __sme_set((page_to_phys(svm->avic_backing_page) &
+			      AVIC_PHYSICAL_ID_ENTRY_BACKING_PAGE_MASK) |
+			      AVIC_PHYSICAL_ID_ENTRY_VALID_MASK);
 	WRITE_ONCE(*entry, new_entry);
 
 	svm->avic_physical_id_cache = entry;
@@ -1608,7 +1608,7 @@ static struct kvm_vcpu *svm_create_vcpu(struct kvm *kvm, unsigned int id)
 
 	svm->vmcb = page_address(page);
 	clear_page(svm->vmcb);
-	svm->vmcb_pa = page_to_pfn(page) << PAGE_SHIFT;
+	svm->vmcb_pa = __sme_set(page_to_pfn(page) << PAGE_SHIFT);
 	svm->asid_generation = 0;
 	init_vmcb(svm);
 
@@ -1636,7 +1636,7 @@ static void svm_free_vcpu(struct kvm_vcpu *vcpu)
 {
 	struct vcpu_svm *svm = to_svm(vcpu);
 
-	__free_page(pfn_to_page(svm->vmcb_pa >> PAGE_SHIFT));
+	__free_page(pfn_to_page(__sme_clr(svm->vmcb_pa) >> PAGE_SHIFT));
 	__free_pages(virt_to_page(svm->msrpm), MSRPM_ALLOC_ORDER);
 	__free_page(virt_to_page(svm->nested.hsave));
 	__free_pages(virt_to_page(svm->nested.msrpm), MSRPM_ALLOC_ORDER);
@@ -2303,7 +2303,7 @@ static u64 nested_svm_get_tdp_pdptr(struct kvm_vcpu *vcpu, int index)
 	u64 pdpte;
 	int ret;
 
-	ret = kvm_vcpu_read_guest_page(vcpu, gpa_to_gfn(cr3), &pdpte,
+	ret = kvm_vcpu_read_guest_page(vcpu, gpa_to_gfn(__sme_clr(cr3)), &pdpte,
 				       offset_in_page(cr3) + index * 8, 8);
 	if (ret)
 		return 0;
@@ -2315,7 +2315,7 @@ static void nested_svm_set_tdp_cr3(struct kvm_vcpu *vcpu,
 {
 	struct vcpu_svm *svm = to_svm(vcpu);
 
-	svm->vmcb->control.nested_cr3 = root;
+	svm->vmcb->control.nested_cr3 = __sme_set(root);
 	mark_dirty(svm->vmcb, VMCB_NPT);
 	svm_flush_tlb(vcpu);
 }
@@ -2803,7 +2803,7 @@ static bool nested_svm_vmrun_msrpm(struct vcpu_svm *svm)
 		svm->nested.msrpm[p] = svm->msrpm[p] | value;
 	}
 
-	svm->vmcb->control.msrpm_base_pa = __pa(svm->nested.msrpm);
+	svm->vmcb->control.msrpm_base_pa = __sme_set(__pa(svm->nested.msrpm));
 
 	return true;
 }
@@ -4435,7 +4435,7 @@ static int svm_ir_list_add(struct vcpu_svm *svm, struct amd_iommu_pi_data *pi)
 	pr_debug("SVM: %s: use GA mode for irq %u\n", __func__,
 		 irq.vector);
 	*svm = to_svm(vcpu);
-	vcpu_info->pi_desc_addr = page_to_phys((*svm)->avic_backing_page);
+	vcpu_info->pi_desc_addr = __sme_set(page_to_phys((*svm)->avic_backing_page));
 	vcpu_info->vector = irq.vector;
 
 	return 0;
@@ -4486,7 +4486,8 @@ static int svm_update_pi_irte(struct kvm *kvm, unsigned int host_irq,
 			struct amd_iommu_pi_data pi;
 
 			/* Try to enable guest_mode in IRTE */
-			pi.base = page_to_phys(svm->avic_backing_page) & AVIC_HPA_MASK;
+			pi.base = __sme_set(page_to_phys(svm->avic_backing_page) &
+					    AVIC_HPA_MASK);
 			pi.ga_tag = AVIC_GATAG(kvm->arch.avic_vm_id,
 						     svm->vcpu.vcpu_id);
 			pi.is_guest_mode = true;
@@ -4911,7 +4912,7 @@ static void svm_set_cr3(struct kvm_vcpu *vcpu, unsigned long root)
 {
 	struct vcpu_svm *svm = to_svm(vcpu);
 
-	svm->vmcb->save.cr3 = root;
+	svm->vmcb->save.cr3 = __sme_set(root);
 	mark_dirty(svm->vmcb, VMCB_CR);
 	svm_flush_tlb(vcpu);
 }
@@ -4920,7 +4921,7 @@ static void set_tdp_cr3(struct kvm_vcpu *vcpu, unsigned long root)
 {
 	struct vcpu_svm *svm = to_svm(vcpu);
 
-	svm->vmcb->control.nested_cr3 = root;
+	svm->vmcb->control.nested_cr3 = __sme_set(root);
 	mark_dirty(svm->vmcb, VMCB_NPT);
 
 	/* Also sync guest cr3 here in case we live migrate */
diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c
index 9b4b5d6..dd3dd26 100644
--- a/arch/x86/kvm/vmx.c
+++ b/arch/x86/kvm/vmx.c
@@ -6443,7 +6443,8 @@ void vmx_enable_tdp(void)
 		enable_ept_ad_bits ? VMX_EPT_DIRTY_BIT : 0ull,
 		0ull, VMX_EPT_EXECUTABLE_MASK,
 		cpu_has_vmx_ept_execute_only() ? 0ull : VMX_EPT_READABLE_MASK,
-		enable_ept_ad_bits ? 0ull : VMX_EPT_RWX_MASK);
+		enable_ept_ad_bits ? 0ull : VMX_EPT_RWX_MASK,
+		0ull);
 
 	ept_set_mmio_spte_mask();
 	kvm_enable_tdp();
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index a2cd099..d232b98 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -66,6 +66,7 @@
 #include <asm/pvclock.h>
 #include <asm/div64.h>
 #include <asm/irq_remapping.h>
+#include <asm/mem_encrypt.h>
 
 #define CREATE_TRACE_POINTS
 #include "trace.h"
@@ -6095,7 +6096,7 @@ int kvm_arch_init(void *opaque)
 
 	kvm_mmu_set_mask_ptes(PT_USER_MASK, PT_ACCESSED_MASK,
 			PT_DIRTY_MASK, PT64_NX_MASK, 0,
-			PT_PRESENT_MASK, 0);
+			PT_PRESENT_MASK, 0, sme_me_mask);
 	kvm_timer_init();
 
 	perf_register_guest_info_callbacks(&kvm_guest_cbs);



From thomas.lendacky at amd.com  Wed Jun  7 12:18:27 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Wed, 07 Jun 2017 14:18:27 -0500
Subject: [PATCH v6 30/34] x86/mm, kexec: Allow kexec to be used with SME
In-Reply-To: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170607191827.28645.93989.stgit@tlendack-t1.amdoffice.net>

Provide support so that kexec can be used to boot a kernel when SME is
enabled.

Support is needed to allocate pages for kexec without encryption.  This
is needed in order to be able to reboot in the kernel in the same manner
as originally booted.

Additionally, when shutting down all of the CPUs we need to be sure to
flush the caches and then halt. This is needed when booting from a state
where SME was not active into a state where SME is active (or vice-versa).
Without these steps, it is possible for cache lines to exist for the same
physical location but tagged both with and without the encryption bit. This
can cause random memory corruption when caches are flushed depending on
which cacheline is written last.

Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/include/asm/init.h          |    1 +
 arch/x86/include/asm/kexec.h         |    8 ++++++++
 arch/x86/include/asm/pgtable_types.h |    1 +
 arch/x86/kernel/machine_kexec_64.c   |   35 +++++++++++++++++++++++++++++++++-
 arch/x86/kernel/process.c            |   17 +++++++++++++++--
 arch/x86/mm/ident_map.c              |   12 ++++++++----
 include/linux/kexec.h                |   14 ++++++++++++++
 kernel/kexec_core.c                  |    6 ++++++
 8 files changed, 87 insertions(+), 7 deletions(-)

diff --git a/arch/x86/include/asm/init.h b/arch/x86/include/asm/init.h
index 474eb8c..05c4aa0 100644
--- a/arch/x86/include/asm/init.h
+++ b/arch/x86/include/asm/init.h
@@ -7,6 +7,7 @@ struct x86_mapping_info {
 	unsigned long page_flag;	 /* page flag for PMD or PUD entry */
 	unsigned long offset;		 /* ident mapping offset */
 	bool direct_gbpages;		 /* PUD level 1GB page support */
+	unsigned long kernpg_flag;	 /* kernel pagetable flag override */
 };
 
 int kernel_ident_mapping_init(struct x86_mapping_info *info, pgd_t *pgd_page,
diff --git a/arch/x86/include/asm/kexec.h b/arch/x86/include/asm/kexec.h
index 70ef205..e8183ac 100644
--- a/arch/x86/include/asm/kexec.h
+++ b/arch/x86/include/asm/kexec.h
@@ -207,6 +207,14 @@ struct kexec_entry64_regs {
 	uint64_t r15;
 	uint64_t rip;
 };
+
+extern int arch_kexec_post_alloc_pages(void *vaddr, unsigned int pages,
+				       gfp_t gfp);
+#define arch_kexec_post_alloc_pages arch_kexec_post_alloc_pages
+
+extern void arch_kexec_pre_free_pages(void *vaddr, unsigned int pages);
+#define arch_kexec_pre_free_pages arch_kexec_pre_free_pages
+
 #endif
 
 typedef void crash_vmclear_fn(void);
diff --git a/arch/x86/include/asm/pgtable_types.h b/arch/x86/include/asm/pgtable_types.h
index ce8cb1c..0f326f4 100644
--- a/arch/x86/include/asm/pgtable_types.h
+++ b/arch/x86/include/asm/pgtable_types.h
@@ -213,6 +213,7 @@ enum page_cache_mode {
 #define PAGE_KERNEL		__pgprot(__PAGE_KERNEL | _PAGE_ENC)
 #define PAGE_KERNEL_RO		__pgprot(__PAGE_KERNEL_RO | _PAGE_ENC)
 #define PAGE_KERNEL_EXEC	__pgprot(__PAGE_KERNEL_EXEC | _PAGE_ENC)
+#define PAGE_KERNEL_EXEC_NOENC	__pgprot(__PAGE_KERNEL_EXEC)
 #define PAGE_KERNEL_RX		__pgprot(__PAGE_KERNEL_RX | _PAGE_ENC)
 #define PAGE_KERNEL_NOCACHE	__pgprot(__PAGE_KERNEL_NOCACHE | _PAGE_ENC)
 #define PAGE_KERNEL_LARGE	__pgprot(__PAGE_KERNEL_LARGE | _PAGE_ENC)
diff --git a/arch/x86/kernel/machine_kexec_64.c b/arch/x86/kernel/machine_kexec_64.c
index 6f5ca4e..35e069a 100644
--- a/arch/x86/kernel/machine_kexec_64.c
+++ b/arch/x86/kernel/machine_kexec_64.c
@@ -87,7 +87,7 @@ static int init_transition_pgtable(struct kimage *image, pgd_t *pgd)
 		set_pmd(pmd, __pmd(__pa(pte) | _KERNPG_TABLE));
 	}
 	pte = pte_offset_kernel(pmd, vaddr);
-	set_pte(pte, pfn_pte(paddr >> PAGE_SHIFT, PAGE_KERNEL_EXEC));
+	set_pte(pte, pfn_pte(paddr >> PAGE_SHIFT, PAGE_KERNEL_EXEC_NOENC));
 	return 0;
 err:
 	free_transition_pgtable(image);
@@ -115,6 +115,7 @@ static int init_pgtable(struct kimage *image, unsigned long start_pgtable)
 		.alloc_pgt_page	= alloc_pgt_page,
 		.context	= image,
 		.page_flag	= __PAGE_KERNEL_LARGE_EXEC,
+		.kernpg_flag	= _KERNPG_TABLE_NOENC,
 	};
 	unsigned long mstart, mend;
 	pgd_t *level4p;
@@ -602,3 +603,35 @@ void arch_kexec_unprotect_crashkres(void)
 {
 	kexec_mark_crashkres(false);
 }
+
+int arch_kexec_post_alloc_pages(void *vaddr, unsigned int pages, gfp_t gfp)
+{
+	int ret;
+
+	if (sme_active()) {
+		/*
+		 * If SME is active we need to be sure that kexec pages are
+		 * not encrypted because when we boot to the new kernel the
+		 * pages won't be accessed encrypted (initially).
+		 */
+		ret = set_memory_decrypted((unsigned long)vaddr, pages);
+		if (ret)
+			return ret;
+
+		if (gfp & __GFP_ZERO)
+			memset(vaddr, 0, pages * PAGE_SIZE);
+	}
+
+	return 0;
+}
+
+void arch_kexec_pre_free_pages(void *vaddr, unsigned int pages)
+{
+	if (sme_active()) {
+		/*
+		 * If SME is active we need to reset the pages back to being
+		 * an encrypted mapping before freeing them.
+		 */
+		set_memory_encrypted((unsigned long)vaddr, pages);
+	}
+}
diff --git a/arch/x86/kernel/process.c b/arch/x86/kernel/process.c
index 0bb8842..fdad0fb 100644
--- a/arch/x86/kernel/process.c
+++ b/arch/x86/kernel/process.c
@@ -355,6 +355,7 @@ bool xen_set_default_idle(void)
 	return ret;
 }
 #endif
+
 void stop_this_cpu(void *dummy)
 {
 	local_irq_disable();
@@ -365,8 +366,20 @@ void stop_this_cpu(void *dummy)
 	disable_local_APIC();
 	mcheck_cpu_clear(this_cpu_ptr(&cpu_info));
 
-	for (;;)
-		halt();
+	for (;;) {
+		/*
+		 * Use wbinvd followed by hlt to stop the processor. This
+		 * provides support for kexec on a processor that supports
+		 * SME. With kexec, going from SME inactive to SME active
+		 * requires clearing cache entries so that addresses without
+		 * the encryption bit set don't corrupt the same physical
+		 * address that has the encryption bit set when caches are
+		 * flushed. To achieve this a wbinvd is performed followed by
+		 * a hlt. Even if the processor is not in the kexec/SME
+		 * scenario this only adds a wbinvd to a halting processor.
+		 */
+		asm volatile("wbinvd; hlt" : : : "memory");
+	}
 }
 
 /*
diff --git a/arch/x86/mm/ident_map.c b/arch/x86/mm/ident_map.c
index adab159..31cea98 100644
--- a/arch/x86/mm/ident_map.c
+++ b/arch/x86/mm/ident_map.c
@@ -51,7 +51,7 @@ static int ident_pud_init(struct x86_mapping_info *info, pud_t *pud_page,
 		if (!pmd)
 			return -ENOMEM;
 		ident_pmd_init(info, pmd, addr, next);
-		set_pud(pud, __pud(__pa(pmd) | _KERNPG_TABLE));
+		set_pud(pud, __pud(__pa(pmd) | info->kernpg_flag));
 	}
 
 	return 0;
@@ -79,7 +79,7 @@ static int ident_p4d_init(struct x86_mapping_info *info, p4d_t *p4d_page,
 		if (!pud)
 			return -ENOMEM;
 		ident_pud_init(info, pud, addr, next);
-		set_p4d(p4d, __p4d(__pa(pud) | _KERNPG_TABLE));
+		set_p4d(p4d, __p4d(__pa(pud) | info->kernpg_flag));
 	}
 
 	return 0;
@@ -93,6 +93,10 @@ int kernel_ident_mapping_init(struct x86_mapping_info *info, pgd_t *pgd_page,
 	unsigned long next;
 	int result;
 
+	/* Set the default pagetable flags if not supplied */
+	if (!info->kernpg_flag)
+		info->kernpg_flag = _KERNPG_TABLE;
+
 	for (; addr < end; addr = next) {
 		pgd_t *pgd = pgd_page + pgd_index(addr);
 		p4d_t *p4d;
@@ -116,14 +120,14 @@ int kernel_ident_mapping_init(struct x86_mapping_info *info, pgd_t *pgd_page,
 		if (result)
 			return result;
 		if (IS_ENABLED(CONFIG_X86_5LEVEL)) {
-			set_pgd(pgd, __pgd(__pa(p4d) | _KERNPG_TABLE));
+			set_pgd(pgd, __pgd(__pa(p4d) | info->kernpg_flag));
 		} else {
 			/*
 			 * With p4d folded, pgd is equal to p4d.
 			 * The pgd entry has to point to the pud page table in this case.
 			 */
 			pud_t *pud = pud_offset(p4d, 0);
-			set_pgd(pgd, __pgd(__pa(pud) | _KERNPG_TABLE));
+			set_pgd(pgd, __pgd(__pa(pud) | info->kernpg_flag));
 		}
 	}
 
diff --git a/include/linux/kexec.h b/include/linux/kexec.h
index c9481eb..5d17fd6 100644
--- a/include/linux/kexec.h
+++ b/include/linux/kexec.h
@@ -334,6 +334,20 @@ static inline void *boot_phys_to_virt(unsigned long entry)
 	return phys_to_virt(boot_phys_to_phys(entry));
 }
 
+#ifndef arch_kexec_post_alloc_pages
+static inline int arch_kexec_post_alloc_pages(void *vaddr, unsigned int pages,
+					      gfp_t gfp)
+{
+	return 0;
+}
+#endif
+
+#ifndef arch_kexec_pre_free_pages
+static inline void arch_kexec_pre_free_pages(void *vaddr, unsigned int pages)
+{
+}
+#endif
+
 #else /* !CONFIG_KEXEC_CORE */
 struct pt_regs;
 struct task_struct;
diff --git a/kernel/kexec_core.c b/kernel/kexec_core.c
index ae1a3ba..ecab7b3 100644
--- a/kernel/kexec_core.c
+++ b/kernel/kexec_core.c
@@ -309,6 +309,9 @@ static struct page *kimage_alloc_pages(gfp_t gfp_mask, unsigned int order)
 		count = 1 << order;
 		for (i = 0; i < count; i++)
 			SetPageReserved(pages + i);
+
+		arch_kexec_post_alloc_pages(page_address(pages), count,
+					    gfp_mask);
 	}
 
 	return pages;
@@ -320,6 +323,9 @@ static void kimage_free_pages(struct page *page)
 
 	order = page_private(page);
 	count = 1 << order;
+
+	arch_kexec_pre_free_pages(page_address(page), count);
+
 	for (i = 0; i < count; i++)
 		ClearPageReserved(page + i);
 	__free_pages(page, order);



From thomas.lendacky at amd.com  Wed Jun  7 12:18:42 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Wed, 07 Jun 2017 14:18:42 -0500
Subject: [PATCH v6 31/34] x86/mm: Use proper encryption attributes with
 /dev/mem
In-Reply-To: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170607191842.28645.56563.stgit@tlendack-t1.amdoffice.net>

When accessing memory using /dev/mem (or /dev/kmem) use the proper
encryption attributes when mapping the memory.

To insure the proper attributes are applied when reading or writing
/dev/mem, update the xlate_dev_mem_ptr() function to use memremap()
which will essentially perform the same steps of applying __va for
RAM or using ioremap() for if not RAM.

To insure the proper attributes are applied when mmapping /dev/mem,
update the phys_mem_access_prot() to call phys_mem_access_encrypted(),
a new function which will check if the memory should be mapped encrypted
or not. If it is not to be mapped encrypted then the VMA protection
value is updated to remove the encryption bit.

Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/include/asm/io.h |    3 +++
 arch/x86/mm/ioremap.c     |   18 +++++++++---------
 arch/x86/mm/pat.c         |    3 +++
 3 files changed, 15 insertions(+), 9 deletions(-)

diff --git a/arch/x86/include/asm/io.h b/arch/x86/include/asm/io.h
index 9eac5a5..db163d7 100644
--- a/arch/x86/include/asm/io.h
+++ b/arch/x86/include/asm/io.h
@@ -385,4 +385,7 @@ extern bool arch_memremap_can_ram_remap(resource_size_t offset, size_t size,
 					unsigned long flags);
 #define arch_memremap_can_ram_remap arch_memremap_can_ram_remap
 
+extern bool phys_mem_access_encrypted(unsigned long phys_addr,
+				      unsigned long size);
+
 #endif /* _ASM_X86_IO_H */
diff --git a/arch/x86/mm/ioremap.c b/arch/x86/mm/ioremap.c
index 99cda55..56dd5b2 100644
--- a/arch/x86/mm/ioremap.c
+++ b/arch/x86/mm/ioremap.c
@@ -404,12 +404,10 @@ void *xlate_dev_mem_ptr(phys_addr_t phys)
 	unsigned long offset = phys & ~PAGE_MASK;
 	void *vaddr;
 
-	/* If page is RAM, we can use __va. Otherwise ioremap and unmap. */
-	if (page_is_ram(start >> PAGE_SHIFT))
-		return __va(phys);
+	/* memremap() maps if RAM, otherwise falls back to ioremap() */
+	vaddr = memremap(start, PAGE_SIZE, MEMREMAP_WB);
 
-	vaddr = ioremap_cache(start, PAGE_SIZE);
-	/* Only add the offset on success and return NULL if the ioremap() failed: */
+	/* Only add the offset on success and return NULL if memremap() failed */
 	if (vaddr)
 		vaddr += offset;
 
@@ -418,10 +416,7 @@ void *xlate_dev_mem_ptr(phys_addr_t phys)
 
 void unxlate_dev_mem_ptr(phys_addr_t phys, void *addr)
 {
-	if (page_is_ram(phys >> PAGE_SHIFT))
-		return;
-
-	iounmap((void __iomem *)((unsigned long)addr & PAGE_MASK));
+	memunmap((void *)((unsigned long)addr & PAGE_MASK));
 }
 
 /*
@@ -630,6 +625,11 @@ pgprot_t __init early_memremap_pgprot_adjust(resource_size_t phys_addr,
 	return prot;
 }
 
+bool phys_mem_access_encrypted(unsigned long phys_addr, unsigned long size)
+{
+	return arch_memremap_can_ram_remap(phys_addr, size, 0);
+}
+
 #ifdef CONFIG_ARCH_USE_MEMREMAP_PROT
 /* Remap memory with encryption */
 void __init *early_memremap_encrypted(resource_size_t phys_addr,
diff --git a/arch/x86/mm/pat.c b/arch/x86/mm/pat.c
index 6753d9c..b970c95 100644
--- a/arch/x86/mm/pat.c
+++ b/arch/x86/mm/pat.c
@@ -748,6 +748,9 @@ void arch_io_free_memtype_wc(resource_size_t start, resource_size_t size)
 pgprot_t phys_mem_access_prot(struct file *file, unsigned long pfn,
 				unsigned long size, pgprot_t vma_prot)
 {
+	if (!phys_mem_access_encrypted(pfn << PAGE_SHIFT, size))
+		vma_prot = pgprot_decrypted(vma_prot);
+
 	return vma_prot;
 }
 



From thomas.lendacky at amd.com  Wed Jun  7 12:18:54 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Wed, 07 Jun 2017 14:18:54 -0500
Subject: [PATCH v6 32/34] x86/mm: Add support to encrypt the kernel in-place
In-Reply-To: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170607191853.28645.85024.stgit@tlendack-t1.amdoffice.net>

Add the support to encrypt the kernel in-place. This is done by creating
new page mappings for the kernel - a decrypted write-protected mapping
and an encrypted mapping. The kernel is encrypted by copying it through
a temporary buffer.

Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/include/asm/mem_encrypt.h |    6 +
 arch/x86/mm/Makefile               |    2 
 arch/x86/mm/mem_encrypt.c          |  314 ++++++++++++++++++++++++++++++++++++
 arch/x86/mm/mem_encrypt_boot.S     |  150 +++++++++++++++++
 4 files changed, 472 insertions(+)
 create mode 100644 arch/x86/mm/mem_encrypt_boot.S

diff --git a/arch/x86/include/asm/mem_encrypt.h b/arch/x86/include/asm/mem_encrypt.h
index d86e544..e0a8edc 100644
--- a/arch/x86/include/asm/mem_encrypt.h
+++ b/arch/x86/include/asm/mem_encrypt.h
@@ -21,6 +21,12 @@
 
 extern unsigned long sme_me_mask;
 
+void sme_encrypt_execute(unsigned long encrypted_kernel_vaddr,
+			 unsigned long decrypted_kernel_vaddr,
+			 unsigned long kernel_len,
+			 unsigned long encryption_wa,
+			 unsigned long encryption_pgd);
+
 void __init sme_early_encrypt(resource_size_t paddr,
 			      unsigned long size);
 void __init sme_early_decrypt(resource_size_t paddr,
diff --git a/arch/x86/mm/Makefile b/arch/x86/mm/Makefile
index 88ee454..47b26ea 100644
--- a/arch/x86/mm/Makefile
+++ b/arch/x86/mm/Makefile
@@ -38,3 +38,5 @@ obj-$(CONFIG_NUMA_EMU)		+= numa_emulation.o
 obj-$(CONFIG_X86_INTEL_MPX)	+= mpx.o
 obj-$(CONFIG_X86_INTEL_MEMORY_PROTECTION_KEYS) += pkeys.o
 obj-$(CONFIG_RANDOMIZE_MEMORY) += kaslr.o
+
+obj-$(CONFIG_AMD_MEM_ENCRYPT)	+= mem_encrypt_boot.o
diff --git a/arch/x86/mm/mem_encrypt.c b/arch/x86/mm/mem_encrypt.c
index 018b58a..6129477 100644
--- a/arch/x86/mm/mem_encrypt.c
+++ b/arch/x86/mm/mem_encrypt.c
@@ -24,6 +24,8 @@
 #include <asm/setup.h>
 #include <asm/bootparam.h>
 #include <asm/set_memory.h>
+#include <asm/cacheflush.h>
+#include <asm/sections.h>
 
 /*
  * Since SME related variables are set early in the boot process they must
@@ -246,8 +248,320 @@ void swiotlb_set_mem_attributes(void *vaddr, unsigned long size)
 	set_memory_decrypted((unsigned long)vaddr, size >> PAGE_SHIFT);
 }
 
+static void __init sme_clear_pgd(pgd_t *pgd_base, unsigned long start,
+				 unsigned long end)
+{
+	unsigned long pgd_start, pgd_end, pgd_size;
+	pgd_t *pgd_p;
+
+	pgd_start = start & PGDIR_MASK;
+	pgd_end = end & PGDIR_MASK;
+
+	pgd_size = (((pgd_end - pgd_start) / PGDIR_SIZE) + 1);
+	pgd_size *= sizeof(pgd_t);
+
+	pgd_p = pgd_base + pgd_index(start);
+
+	memset(pgd_p, 0, pgd_size);
+}
+
+#ifndef CONFIG_X86_5LEVEL
+#define native_make_p4d(_x)	(p4d_t) { .pgd = native_make_pgd(_x) }
+#endif
+
+#define PGD_FLAGS	_KERNPG_TABLE_NOENC
+#define P4D_FLAGS	_KERNPG_TABLE_NOENC
+#define PUD_FLAGS	_KERNPG_TABLE_NOENC
+#define PMD_FLAGS	(__PAGE_KERNEL_LARGE_EXEC & ~_PAGE_GLOBAL)
+
+static void __init *sme_populate_pgd(pgd_t *pgd_base, void *pgtable_area,
+				     unsigned long vaddr, pmdval_t pmd_val)
+{
+	pgd_t *pgd_p;
+	p4d_t *p4d_p;
+	pud_t *pud_p;
+	pmd_t *pmd_p;
+
+	pgd_p = pgd_base + pgd_index(vaddr);
+	if (native_pgd_val(*pgd_p)) {
+		if (IS_ENABLED(CONFIG_X86_5LEVEL))
+			p4d_p = (p4d_t *)(native_pgd_val(*pgd_p) & ~PTE_FLAGS_MASK);
+		else
+			pud_p = (pud_t *)(native_pgd_val(*pgd_p) & ~PTE_FLAGS_MASK);
+	} else {
+		pgd_t pgd;
+
+		if (IS_ENABLED(CONFIG_X86_5LEVEL)) {
+			p4d_p = pgtable_area;
+			memset(p4d_p, 0, sizeof(*p4d_p) * PTRS_PER_P4D);
+			pgtable_area += sizeof(*p4d_p) * PTRS_PER_P4D;
+
+			pgd = native_make_pgd((pgdval_t)p4d_p + PGD_FLAGS);
+		} else {
+			pud_p = pgtable_area;
+			memset(pud_p, 0, sizeof(*pud_p) * PTRS_PER_PUD);
+			pgtable_area += sizeof(*pud_p) * PTRS_PER_PUD;
+
+			pgd = native_make_pgd((pgdval_t)pud_p + PGD_FLAGS);
+		}
+		native_set_pgd(pgd_p, pgd);
+	}
+
+	if (IS_ENABLED(CONFIG_X86_5LEVEL)) {
+		p4d_p += p4d_index(vaddr);
+		if (native_p4d_val(*p4d_p)) {
+			pud_p = (pud_t *)(native_p4d_val(*p4d_p) & ~PTE_FLAGS_MASK);
+		} else {
+			p4d_t p4d;
+
+			pud_p = pgtable_area;
+			memset(pud_p, 0, sizeof(*pud_p) * PTRS_PER_PUD);
+			pgtable_area += sizeof(*pud_p) * PTRS_PER_PUD;
+
+			p4d = native_make_p4d((p4dval_t)pud_p + P4D_FLAGS);
+			native_set_p4d(p4d_p, p4d);
+		}
+	}
+
+	pud_p += pud_index(vaddr);
+	if (native_pud_val(*pud_p)) {
+		if (native_pud_val(*pud_p) & _PAGE_PSE)
+			goto out;
+
+		pmd_p = (pmd_t *)(native_pud_val(*pud_p) & ~PTE_FLAGS_MASK);
+	} else {
+		pud_t pud;
+
+		pmd_p = pgtable_area;
+		memset(pmd_p, 0, sizeof(*pmd_p) * PTRS_PER_PMD);
+		pgtable_area += sizeof(*pmd_p) * PTRS_PER_PMD;
+
+		pud = native_make_pud((pudval_t)pmd_p + PUD_FLAGS);
+		native_set_pud(pud_p, pud);
+	}
+
+	pmd_p += pmd_index(vaddr);
+	if (!native_pmd_val(*pmd_p) || !(native_pmd_val(*pmd_p) & _PAGE_PSE))
+		native_set_pmd(pmd_p, native_make_pmd(pmd_val));
+
+out:
+	return pgtable_area;
+}
+
+static unsigned long __init sme_pgtable_calc(unsigned long len)
+{
+	unsigned long p4d_size, pud_size, pmd_size;
+	unsigned long total;
+
+	/*
+	 * Perform a relatively simplistic calculation of the pagetable
+	 * entries that are needed. That mappings will be covered by 2MB
+	 * PMD entries so we can conservatively calculate the required
+	 * number of P4D, PUD and PMD structures needed to perform the
+	 * mappings. Incrementing the count for each covers the case where
+	 * the addresses cross entries.
+	 */
+	if (IS_ENABLED(CONFIG_X86_5LEVEL)) {
+		p4d_size = (ALIGN(len, PGDIR_SIZE) / PGDIR_SIZE) + 1;
+		p4d_size *= sizeof(p4d_t) * PTRS_PER_P4D;
+		pud_size = (ALIGN(len, P4D_SIZE) / P4D_SIZE) + 1;
+		pud_size *= sizeof(pud_t) * PTRS_PER_PUD;
+	} else {
+		p4d_size = 0;
+		pud_size = (ALIGN(len, PGDIR_SIZE) / PGDIR_SIZE) + 1;
+		pud_size *= sizeof(pud_t) * PTRS_PER_PUD;
+	}
+	pmd_size = (ALIGN(len, PUD_SIZE) / PUD_SIZE) + 1;
+	pmd_size *= sizeof(pmd_t) * PTRS_PER_PMD;
+
+	total = p4d_size + pud_size + pmd_size;
+
+	/*
+	 * Now calculate the added pagetable structures needed to populate
+	 * the new pagetables.
+	 */
+	if (IS_ENABLED(CONFIG_X86_5LEVEL)) {
+		p4d_size = ALIGN(total, PGDIR_SIZE) / PGDIR_SIZE;
+		p4d_size *= sizeof(p4d_t) * PTRS_PER_P4D;
+		pud_size = ALIGN(total, P4D_SIZE) / P4D_SIZE;
+		pud_size *= sizeof(pud_t) * PTRS_PER_PUD;
+	} else {
+		p4d_size = 0;
+		pud_size = ALIGN(total, PGDIR_SIZE) / PGDIR_SIZE;
+		pud_size *= sizeof(pud_t) * PTRS_PER_PUD;
+	}
+	pmd_size = ALIGN(total, PUD_SIZE) / PUD_SIZE;
+	pmd_size *= sizeof(pmd_t) * PTRS_PER_PMD;
+
+	total += p4d_size + pud_size + pmd_size;
+
+	return total;
+}
+
 void __init sme_encrypt_kernel(void)
 {
+	unsigned long workarea_start, workarea_end, workarea_len;
+	unsigned long execute_start, execute_end, execute_len;
+	unsigned long kernel_start, kernel_end, kernel_len;
+	unsigned long pgtable_area_len;
+	unsigned long paddr, pmd_flags;
+	unsigned long decrypted_base;
+	void *pgtable_area;
+	pgd_t *pgd;
+
+	if (!sme_active())
+		return;
+
+	/*
+	 * Prepare for encrypting the kernel by building new pagetables with
+	 * the necessary attributes needed to encrypt the kernel in place.
+	 *
+	 *   One range of virtual addresses will map the memory occupied
+	 *   by the kernel as encrypted.
+	 *
+	 *   Another range of virtual addresses will map the memory occupied
+	 *   by the kernel as decrypted and write-protected.
+	 *
+	 *     The use of write-protect attribute will prevent any of the
+	 *     memory from being cached.
+	 */
+
+	/* Physical addresses gives us the identity mapped virtual addresses */
+	kernel_start = __pa_symbol(_text);
+	kernel_end = ALIGN(__pa_symbol(_end), PMD_PAGE_SIZE);
+	kernel_len = kernel_end - kernel_start;
+
+	/* Set the encryption workarea to be immediately after the kernel */
+	workarea_start = kernel_end;
+
+	/*
+	 * Calculate required number of workarea bytes needed:
+	 *   executable encryption area size:
+	 *     stack page (PAGE_SIZE)
+	 *     encryption routine page (PAGE_SIZE)
+	 *     intermediate copy buffer (PMD_PAGE_SIZE)
+	 *   pagetable structures for the encryption of the kernel
+	 *   pagetable structures for workarea (in case not currently mapped)
+	 */
+	execute_start = workarea_start;
+	execute_end = execute_start + (PAGE_SIZE * 2) + PMD_PAGE_SIZE;
+	execute_len = execute_end - execute_start;
+
+	/*
+	 * One PGD for both encrypted and decrypted mappings and a set of
+	 * PUDs and PMDs for each of the encrypted and decrypted mappings.
+	 */
+	pgtable_area_len = sizeof(pgd_t) * PTRS_PER_PGD;
+	pgtable_area_len += sme_pgtable_calc(execute_end - kernel_start) * 2;
+
+	/* PUDs and PMDs needed in the current pagetables for the workarea */
+	pgtable_area_len += sme_pgtable_calc(execute_len + pgtable_area_len);
+
+	/*
+	 * The total workarea includes the executable encryption area and
+	 * the pagetable area.
+	 */
+	workarea_len = execute_len + pgtable_area_len;
+	workarea_end = workarea_start + workarea_len;
+
+	/*
+	 * Set the address to the start of where newly created pagetable
+	 * structures (PGDs, PUDs and PMDs) will be allocated. New pagetable
+	 * structures are created when the workarea is added to the current
+	 * pagetables and when the new encrypted and decrypted kernel
+	 * mappings are populated.
+	 */
+	pgtable_area = (void *)execute_end;
+
+	/*
+	 * Make sure the current pagetable structure has entries for
+	 * addressing the workarea.
+	 */
+	pgd = (pgd_t *)native_read_cr3_pa();
+	paddr = workarea_start;
+	while (paddr < workarea_end) {
+		pgtable_area = sme_populate_pgd(pgd, pgtable_area,
+						paddr,
+						paddr + PMD_FLAGS);
+
+		paddr += PMD_PAGE_SIZE;
+	}
+
+	/* Flush the TLB - no globals so cr3 is enough */
+	native_write_cr3(native_read_cr3());
+
+	/*
+	 * A new pagetable structure is being built to allow for the kernel
+	 * to be encrypted. It starts with an empty PGD that will then be
+	 * populated with new PUDs and PMDs as the encrypted and decrypted
+	 * kernel mappings are created.
+	 */
+	pgd = pgtable_area;
+	memset(pgd, 0, sizeof(*pgd) * PTRS_PER_PGD);
+	pgtable_area += sizeof(*pgd) * PTRS_PER_PGD;
+
+	/* Add encrypted kernel (identity) mappings */
+	pmd_flags = PMD_FLAGS | _PAGE_ENC;
+	paddr = kernel_start;
+	while (paddr < kernel_end) {
+		pgtable_area = sme_populate_pgd(pgd, pgtable_area,
+						paddr,
+						paddr + pmd_flags);
+
+		paddr += PMD_PAGE_SIZE;
+	}
+
+	/*
+	 * A different PGD index/entry must be used to get different
+	 * pagetable entries for the decrypted mapping. Choose the next
+	 * PGD index and convert it to a virtual address to be used as
+	 * the base of the mapping.
+	 */
+	decrypted_base = (pgd_index(workarea_end) + 1) & (PTRS_PER_PGD - 1);
+	decrypted_base <<= PGDIR_SHIFT;
+
+	/* Add decrypted, write-protected kernel (non-identity) mappings */
+	pmd_flags = (PMD_FLAGS & ~_PAGE_CACHE_MASK) | (_PAGE_PAT | _PAGE_PWT);
+	paddr = kernel_start;
+	while (paddr < kernel_end) {
+		pgtable_area = sme_populate_pgd(pgd, pgtable_area,
+						paddr + decrypted_base,
+						paddr + pmd_flags);
+
+		paddr += PMD_PAGE_SIZE;
+	}
+
+	/* Add decrypted workarea mappings to both kernel mappings */
+	paddr = workarea_start;
+	while (paddr < workarea_end) {
+		pgtable_area = sme_populate_pgd(pgd, pgtable_area,
+						paddr,
+						paddr + PMD_FLAGS);
+
+		pgtable_area = sme_populate_pgd(pgd, pgtable_area,
+						paddr + decrypted_base,
+						paddr + PMD_FLAGS);
+
+		paddr += PMD_PAGE_SIZE;
+	}
+
+	/* Perform the encryption */
+	sme_encrypt_execute(kernel_start, kernel_start + decrypted_base,
+			    kernel_len, workarea_start, (unsigned long)pgd);
+
+	/*
+	 * At this point we are running encrypted.  Remove the mappings for
+	 * the decrypted areas - all that is needed for this is to remove
+	 * the PGD entry/entries.
+	 */
+	sme_clear_pgd(pgd, kernel_start + decrypted_base,
+		      kernel_end + decrypted_base);
+
+	sme_clear_pgd(pgd, workarea_start + decrypted_base,
+		      workarea_end + decrypted_base);
+
+	/* Flush the TLB - no globals so cr3 is enough */
+	native_write_cr3(native_read_cr3());
 }
 
 unsigned long __init sme_enable(void)
diff --git a/arch/x86/mm/mem_encrypt_boot.S b/arch/x86/mm/mem_encrypt_boot.S
new file mode 100644
index 0000000..7720b00
--- /dev/null
+++ b/arch/x86/mm/mem_encrypt_boot.S
@@ -0,0 +1,150 @@
+/*
+ * AMD Memory Encryption Support
+ *
+ * Copyright (C) 2016 Advanced Micro Devices, Inc.
+ *
+ * Author: Tom Lendacky <thomas.lendacky at amd.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include <linux/linkage.h>
+#include <asm/pgtable.h>
+#include <asm/page.h>
+#include <asm/processor-flags.h>
+#include <asm/msr-index.h>
+
+	.text
+	.code64
+ENTRY(sme_encrypt_execute)
+
+	/*
+	 * Entry parameters:
+	 *   RDI - virtual address for the encrypted kernel mapping
+	 *   RSI - virtual address for the decrypted kernel mapping
+	 *   RDX - length of kernel
+	 *   RCX - virtual address of the encryption workarea, including:
+	 *     - stack page (PAGE_SIZE)
+	 *     - encryption routine page (PAGE_SIZE)
+	 *     - intermediate copy buffer (PMD_PAGE_SIZE)
+	 *    R8 - physcial address of the pagetables to use for encryption
+	 */
+
+	push	%rbp
+	push	%r12
+
+	/* Set up a one page stack in the non-encrypted memory area */
+	movq	%rsp, %rbp		/* Save current stack pointer */
+	movq	%rcx, %rax		/* Workarea stack page */
+	movq	%rax, %rsp		/* Set new stack pointer */
+	addq	$PAGE_SIZE, %rsp	/* Stack grows from the bottom */
+	addq	$PAGE_SIZE, %rax	/* Workarea encryption routine */
+
+	movq	%rdi, %r10		/* Encrypted kernel */
+	movq	%rsi, %r11		/* Decrypted kernel */
+	movq	%rdx, %r12		/* Kernel length */
+
+	/* Copy encryption routine into the workarea */
+	movq	%rax, %rdi				/* Workarea encryption routine */
+	leaq	__enc_copy(%rip), %rsi			/* Encryption routine */
+	movq	$(.L__enc_copy_end - __enc_copy), %rcx	/* Encryption routine length */
+	rep	movsb
+
+	/* Setup registers for call */
+	movq	%r10, %rdi		/* Encrypted kernel */
+	movq	%r11, %rsi		/* Decrypted kernel */
+	movq	%r8, %rdx		/* Pagetables used for encryption */
+	movq	%r12, %rcx		/* Kernel length */
+	movq	%rax, %r8		/* Workarea encryption routine */
+	addq	$PAGE_SIZE, %r8		/* Workarea intermediate copy buffer */
+
+	call	*%rax			/* Call the encryption routine */
+
+	movq	%rbp, %rsp		/* Restore original stack pointer */
+
+	pop	%r12
+	pop	%rbp
+
+	ret
+ENDPROC(sme_encrypt_execute)
+
+ENTRY(__enc_copy)
+/*
+ * Routine used to encrypt kernel.
+ *   This routine must be run outside of the kernel proper since
+ *   the kernel will be encrypted during the process. So this
+ *   routine is defined here and then copied to an area outside
+ *   of the kernel where it will remain and run decrypted
+ *   during execution.
+ *
+ *   On entry the registers must be:
+ *     RDI - virtual address for the encrypted kernel mapping
+ *     RSI - virtual address for the decrypted kernel mapping
+ *     RDX - address of the pagetables to use for encryption
+ *     RCX - length of kernel
+ *      R8 - intermediate copy buffer
+ *
+ *     RAX - points to this routine
+ *
+ * The kernel will be encrypted by copying from the non-encrypted
+ * kernel space to an intermediate buffer and then copying from the
+ * intermediate buffer back to the encrypted kernel space. The physical
+ * addresses of the two kernel space mappings are the same which
+ * results in the kernel being encrypted "in place".
+ */
+	/* Enable the new page tables */
+	mov	%rdx, %cr3
+
+	/* Flush any global TLBs */
+	mov	%cr4, %rdx
+	andq	$~X86_CR4_PGE, %rdx
+	mov	%rdx, %cr4
+	orq	$X86_CR4_PGE, %rdx
+	mov	%rdx, %cr4
+
+	/* Set the PAT register PA5 entry to write-protect */
+	push	%rcx
+	movl	$MSR_IA32_CR_PAT, %ecx
+	rdmsr
+	push	%rdx			/* Save original PAT value */
+	andl	$0xffff00ff, %edx	/* Clear PA5 */
+	orl	$0x00000500, %edx	/* Set PA5 to WP */
+	wrmsr
+	pop	%rdx			/* RDX contains original PAT value */
+	pop	%rcx
+
+	movq	%rcx, %r9		/* Save kernel length */
+	movq	%rdi, %r10		/* Save encrypted kernel address */
+	movq	%rsi, %r11		/* Save decrypted kernel address */
+
+	wbinvd				/* Invalidate any cache entries */
+
+	/* Copy/encrypt 2MB at a time */
+1:
+	movq	%r11, %rsi		/* Source - decrypted kernel */
+	movq	%r8, %rdi		/* Dest   - intermediate copy buffer */
+	movq	$PMD_PAGE_SIZE, %rcx	/* 2MB length */
+	rep	movsb
+
+	movq	%r8, %rsi		/* Source - intermediate copy buffer */
+	movq	%r10, %rdi		/* Dest   - encrypted kernel */
+	movq	$PMD_PAGE_SIZE, %rcx	/* 2MB length */
+	rep	movsb
+
+	addq	$PMD_PAGE_SIZE, %r11
+	addq	$PMD_PAGE_SIZE, %r10
+	subq	$PMD_PAGE_SIZE, %r9	/* Kernel length decrement */
+	jnz	1b			/* Kernel length not zero? */
+
+	/* Restore PAT register */
+	push	%rdx			/* Save original PAT value */
+	movl	$MSR_IA32_CR_PAT, %ecx
+	rdmsr
+	pop	%rdx			/* Restore original PAT value */
+	wrmsr
+
+	ret
+.L__enc_copy_end:
+ENDPROC(__enc_copy)



From thomas.lendacky at amd.com  Wed Jun  7 12:19:05 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Wed, 07 Jun 2017 14:19:05 -0500
Subject: [PATCH v6 33/34] x86/boot: Add early cmdline parsing for options
 with arguments
In-Reply-To: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170607191905.28645.58429.stgit@tlendack-t1.amdoffice.net>

Add a cmdline_find_option() function to look for cmdline options that
take arguments. The argument is returned in a supplied buffer and the
argument length (regardless of whether it fits in the supplied buffer)
is returned, with -1 indicating not found.

Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/include/asm/cmdline.h |    2 +
 arch/x86/lib/cmdline.c         |  105 ++++++++++++++++++++++++++++++++++++++++
 2 files changed, 107 insertions(+)

diff --git a/arch/x86/include/asm/cmdline.h b/arch/x86/include/asm/cmdline.h
index e01f7f7..84ae170 100644
--- a/arch/x86/include/asm/cmdline.h
+++ b/arch/x86/include/asm/cmdline.h
@@ -2,5 +2,7 @@
 #define _ASM_X86_CMDLINE_H
 
 int cmdline_find_option_bool(const char *cmdline_ptr, const char *option);
+int cmdline_find_option(const char *cmdline_ptr, const char *option,
+			char *buffer, int bufsize);
 
 #endif /* _ASM_X86_CMDLINE_H */
diff --git a/arch/x86/lib/cmdline.c b/arch/x86/lib/cmdline.c
index 5cc78bf..3261abb 100644
--- a/arch/x86/lib/cmdline.c
+++ b/arch/x86/lib/cmdline.c
@@ -104,7 +104,112 @@ static inline int myisspace(u8 c)
 	return 0;	/* Buffer overrun */
 }
 
+/*
+ * Find a non-boolean option (i.e. option=argument). In accordance with
+ * standard Linux practice, if this option is repeated, this returns the
+ * last instance on the command line.
+ *
+ * @cmdline: the cmdline string
+ * @max_cmdline_size: the maximum size of cmdline
+ * @option: option string to look for
+ * @buffer: memory buffer to return the option argument
+ * @bufsize: size of the supplied memory buffer
+ *
+ * Returns the length of the argument (regardless of if it was
+ * truncated to fit in the buffer), or -1 on not found.
+ */
+static int
+__cmdline_find_option(const char *cmdline, int max_cmdline_size,
+		      const char *option, char *buffer, int bufsize)
+{
+	char c;
+	int pos = 0, len = -1;
+	const char *opptr = NULL;
+	char *bufptr = buffer;
+	enum {
+		st_wordstart = 0,	/* Start of word/after whitespace */
+		st_wordcmp,	/* Comparing this word */
+		st_wordskip,	/* Miscompare, skip */
+		st_bufcpy,	/* Copying this to buffer */
+	} state = st_wordstart;
+
+	if (!cmdline)
+		return -1;      /* No command line */
+
+	/*
+	 * This 'pos' check ensures we do not overrun
+	 * a non-NULL-terminated 'cmdline'
+	 */
+	while (pos++ < max_cmdline_size) {
+		c = *(char *)cmdline++;
+		if (!c)
+			break;
+
+		switch (state) {
+		case st_wordstart:
+			if (myisspace(c))
+				break;
+
+			state = st_wordcmp;
+			opptr = option;
+			/* fall through */
+
+		case st_wordcmp:
+			if ((c == '=') && !*opptr) {
+				/*
+				 * We matched all the way to the end of the
+				 * option we were looking for, prepare to
+				 * copy the argument.
+				 */
+				len = 0;
+				bufptr = buffer;
+				state = st_bufcpy;
+				break;
+			} else if (c == *opptr++) {
+				/*
+				 * We are currently matching, so continue
+				 * to the next character on the cmdline.
+				 */
+				break;
+			}
+			state = st_wordskip;
+			/* fall through */
+
+		case st_wordskip:
+			if (myisspace(c))
+				state = st_wordstart;
+			break;
+
+		case st_bufcpy:
+			if (myisspace(c)) {
+				state = st_wordstart;
+			} else {
+				/*
+				 * Increment len, but don't overrun the
+				 * supplied buffer and leave room for the
+				 * NULL terminator.
+				 */
+				if (++len < bufsize)
+					*bufptr++ = c;
+			}
+			break;
+		}
+	}
+
+	if (bufsize)
+		*bufptr = '\0';
+
+	return len;
+}
+
 int cmdline_find_option_bool(const char *cmdline, const char *option)
 {
 	return __cmdline_find_option_bool(cmdline, COMMAND_LINE_SIZE, option);
 }
+
+int cmdline_find_option(const char *cmdline, const char *option, char *buffer,
+			int bufsize)
+{
+	return __cmdline_find_option(cmdline, COMMAND_LINE_SIZE, option,
+				     buffer, bufsize);
+}



From thomas.lendacky at amd.com  Wed Jun  7 12:19:16 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Wed, 07 Jun 2017 14:19:16 -0500
Subject: [PATCH v6 34/34] x86/mm: Add support to make use of Secure Memory
 Encryption
In-Reply-To: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170607191916.28645.87015.stgit@tlendack-t1.amdoffice.net>

Add support to check if SME has been enabled and if memory encryption
should be activated (checking of command line option based on the
configuration of the default state).  If memory encryption is to be
activated, then the encryption mask is set and the kernel is encrypted
"in place."

Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/kernel/head_64.S |    1 
 arch/x86/mm/mem_encrypt.c |   93 +++++++++++++++++++++++++++++++++++++++++++--
 2 files changed, 89 insertions(+), 5 deletions(-)

diff --git a/arch/x86/kernel/head_64.S b/arch/x86/kernel/head_64.S
index 1fe944b..660bf8e 100644
--- a/arch/x86/kernel/head_64.S
+++ b/arch/x86/kernel/head_64.S
@@ -97,6 +97,7 @@ startup_64:
 	 * Save the returned mask in %r12 for later use.
 	 */
 	push	%rsi
+	movq	%rsi, %rdi
 	call	sme_enable
 	pop	%rsi
 	movq	%rax, %r12
diff --git a/arch/x86/mm/mem_encrypt.c b/arch/x86/mm/mem_encrypt.c
index 6129477..d624058 100644
--- a/arch/x86/mm/mem_encrypt.c
+++ b/arch/x86/mm/mem_encrypt.c
@@ -12,6 +12,7 @@
 
 #include <linux/linkage.h>
 #include <linux/init.h>
+#include <asm/bootparam.h>
 
 #ifdef CONFIG_AMD_MEM_ENCRYPT
 
@@ -22,10 +23,23 @@
 #include <asm/tlbflush.h>
 #include <asm/fixmap.h>
 #include <asm/setup.h>
-#include <asm/bootparam.h>
 #include <asm/set_memory.h>
 #include <asm/cacheflush.h>
 #include <asm/sections.h>
+#include <asm/mem_encrypt.h>
+#include <asm/processor-flags.h>
+#include <asm/msr.h>
+#include <asm/cmdline.h>
+
+static char sme_cmdline_arg[] __initdata = "mem_encrypt";
+static char sme_cmdline_on[]  __initdata = "on";
+static char sme_cmdline_off[] __initdata = "off";
+
+/*
+ * Some SME functions run very early causing issues with the stack-protector
+ * support. Provide a way to turn off this support on a per-function basis.
+ */
+#define SME_NOSTACKP __attribute__((__optimize__("no-stack-protector")))
 
 /*
  * Since SME related variables are set early in the boot process they must
@@ -237,6 +251,8 @@ void __init mem_encrypt_init(void)
 
 	/* Call into SWIOTLB to update the SWIOTLB DMA buffers */
 	swiotlb_update_mem_attributes();
+
+	pr_info("AMD Secure Memory Encryption (SME) active\n");
 }
 
 void swiotlb_set_mem_attributes(void *vaddr, unsigned long size)
@@ -564,8 +580,75 @@ void __init sme_encrypt_kernel(void)
 	native_write_cr3(native_read_cr3());
 }
 
-unsigned long __init sme_enable(void)
+unsigned long __init SME_NOSTACKP sme_enable(struct boot_params *bp)
 {
+	const char *cmdline_ptr, *cmdline_arg, *cmdline_on, *cmdline_off;
+	unsigned int eax, ebx, ecx, edx;
+	bool active_by_default;
+	unsigned long me_mask;
+	char buffer[16];
+	u64 msr;
+
+	/* Check for the SME support leaf */
+	eax = 0x80000000;
+	ecx = 0;
+	native_cpuid(&eax, &ebx, &ecx, &edx);
+	if (eax < 0x8000001f)
+		goto out;
+
+	/*
+	 * Check for the SME feature:
+	 *   CPUID Fn8000_001F[EAX] - Bit 0
+	 *     Secure Memory Encryption support
+	 *   CPUID Fn8000_001F[EBX] - Bits 5:0
+	 *     Pagetable bit position used to indicate encryption
+	 */
+	eax = 0x8000001f;
+	ecx = 0;
+	native_cpuid(&eax, &ebx, &ecx, &edx);
+	if (!(eax & 1))
+		goto out;
+
+	me_mask = 1UL << (ebx & 0x3f);
+
+	/* Check if SME is enabled */
+	msr = __rdmsr(MSR_K8_SYSCFG);
+	if (!(msr & MSR_K8_SYSCFG_MEM_ENCRYPT))
+		goto out;
+
+	/*
+	 * Fixups have not been applied to phys_base yet and we're running
+	 * identity mapped, so we must obtain the address to the SME command
+	 * line argument data using rip-relative addressing.
+	 */
+	asm ("lea sme_cmdline_arg(%%rip), %0"
+	     : "=r" (cmdline_arg)
+	     : "p" (sme_cmdline_arg));
+	asm ("lea sme_cmdline_on(%%rip), %0"
+	     : "=r" (cmdline_on)
+	     : "p" (sme_cmdline_on));
+	asm ("lea sme_cmdline_off(%%rip), %0"
+	     : "=r" (cmdline_off)
+	     : "p" (sme_cmdline_off));
+
+	if (IS_ENABLED(CONFIG_AMD_MEM_ENCRYPT_ACTIVE_BY_DEFAULT))
+		active_by_default = true;
+	else
+		active_by_default = false;
+
+	cmdline_ptr = (const char *)((u64)bp->hdr.cmd_line_ptr |
+				     ((u64)bp->ext_cmd_line_ptr << 32));
+
+	cmdline_find_option(cmdline_ptr, cmdline_arg, buffer, sizeof(buffer));
+
+	if (strncmp(buffer, cmdline_on, sizeof(buffer)) == 0)
+		sme_me_mask = me_mask;
+	else if (strncmp(buffer, cmdline_off, sizeof(buffer)) == 0)
+		sme_me_mask = 0;
+	else
+		sme_me_mask = active_by_default ? me_mask : 0;
+
+out:
 	return sme_me_mask;
 }
 
@@ -576,9 +659,9 @@ unsigned long sme_get_me_mask(void)
 
 #else	/* !CONFIG_AMD_MEM_ENCRYPT */
 
-void __init sme_encrypt_kernel(void)	{ }
-unsigned long __init sme_enable(void)	{ return 0; }
+void __init sme_encrypt_kernel(void)			{ }
+unsigned long __init sme_enable(struct boot_params *bp)	{ return 0; }
 
-unsigned long sme_get_me_mask(void)	{ return 0; }
+unsigned long sme_get_me_mask(void)			{ return 0; }
 
 #endif	/* CONFIG_AMD_MEM_ENCRYPT */



From thomas.lendacky at amd.com  Wed Jun  7 12:17:09 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Wed, 07 Jun 2017 14:17:09 -0500
Subject: [PATCH v6 23/34] x86,
 realmode: Decrypt trampoline area if memory encryption is active
In-Reply-To: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170607191709.28645.69034.stgit@tlendack-t1.amdoffice.net>

When Secure Memory Encryption is enabled, the trampoline area must not
be encrypted. A CPU running in real mode will not be able to decrypt
memory that has been encrypted because it will not be able to use addresses
with the memory encryption mask.

A recent change that added a new system_state value exposed a warning
issued by early_ioreamp() when the system_state was not SYSTEM_BOOTING.
At the stage where the trampoline area is decrypted, the system_state is
now SYSTEM_SCHEDULING. The check was changed to issue a warning if the
system_state is greater than or equal to SYSTEM_RUNNING.

Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/realmode/init.c |   11 +++++++++++
 mm/early_ioremap.c       |    2 +-
 2 files changed, 12 insertions(+), 1 deletion(-)

diff --git a/arch/x86/realmode/init.c b/arch/x86/realmode/init.c
index a163a90..195ba29 100644
--- a/arch/x86/realmode/init.c
+++ b/arch/x86/realmode/init.c
@@ -6,6 +6,7 @@
 #include <asm/pgtable.h>
 #include <asm/realmode.h>
 #include <asm/tlbflush.h>
+#include <asm/mem_encrypt.h>
 
 struct real_mode_header *real_mode_header;
 u32 *trampoline_cr4_features;
@@ -130,6 +131,16 @@ static void __init set_real_mode_permissions(void)
 	unsigned long text_start =
 		(unsigned long) __va(real_mode_header->text_start);
 
+	/*
+	 * If SME is active, the trampoline area will need to be in
+	 * decrypted memory in order to bring up other processors
+	 * successfully.
+	 */
+	if (sme_active()) {
+		sme_early_decrypt(__pa(base), size);
+		set_memory_decrypted((unsigned long)base, size >> PAGE_SHIFT);
+	}
+
 	set_memory_nx((unsigned long) base, size >> PAGE_SHIFT);
 	set_memory_ro((unsigned long) base, ro_size >> PAGE_SHIFT);
 	set_memory_x((unsigned long) text_start, text_size >> PAGE_SHIFT);
diff --git a/mm/early_ioremap.c b/mm/early_ioremap.c
index b1dd4a9..01d13ae 100644
--- a/mm/early_ioremap.c
+++ b/mm/early_ioremap.c
@@ -110,7 +110,7 @@ static int __init check_early_ioremap_leak(void)
 	enum fixed_addresses idx;
 	int i, slot;
 
-	WARN_ON(system_state != SYSTEM_BOOTING);
+	WARN_ON(system_state >= SYSTEM_RUNNING);
 
 	slot = -1;
 	for (i = 0; i < FIX_BTMAPS_SLOTS; i++) {



From thomas.lendacky at amd.com  Wed Jun  7 12:17:00 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Wed, 07 Jun 2017 14:17:00 -0500
Subject: [PATCH v6 22/34] x86/mm: Add support for changing the memory
 encryption attribute
In-Reply-To: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170607191659.28645.1669.stgit@tlendack-t1.amdoffice.net>

Add support for changing the memory encryption attribute for one or more
memory pages. This will be useful when we have to change the AP trampoline
area to not be encrypted. Or when we need to change the SWIOTLB area to
not be encrypted in support of devices that can't support the encryption
mask range.

Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/include/asm/set_memory.h |    3 ++
 arch/x86/mm/pageattr.c            |   62 +++++++++++++++++++++++++++++++++++++
 2 files changed, 65 insertions(+)

diff --git a/arch/x86/include/asm/set_memory.h b/arch/x86/include/asm/set_memory.h
index eaec6c3..cd71273 100644
--- a/arch/x86/include/asm/set_memory.h
+++ b/arch/x86/include/asm/set_memory.h
@@ -11,6 +11,7 @@
  * Executability : eXeutable, NoteXecutable
  * Read/Write    : ReadOnly, ReadWrite
  * Presence      : NotPresent
+ * Encryption    : Encrypted, Decrypted
  *
  * Within a category, the attributes are mutually exclusive.
  *
@@ -42,6 +43,8 @@
 int set_memory_wb(unsigned long addr, int numpages);
 int set_memory_np(unsigned long addr, int numpages);
 int set_memory_4k(unsigned long addr, int numpages);
+int set_memory_encrypted(unsigned long addr, int numpages);
+int set_memory_decrypted(unsigned long addr, int numpages);
 
 int set_memory_array_uc(unsigned long *addr, int addrinarray);
 int set_memory_array_wc(unsigned long *addr, int addrinarray);
diff --git a/arch/x86/mm/pageattr.c b/arch/x86/mm/pageattr.c
index e7d3866..d9e09fb 100644
--- a/arch/x86/mm/pageattr.c
+++ b/arch/x86/mm/pageattr.c
@@ -1769,6 +1769,68 @@ int set_memory_4k(unsigned long addr, int numpages)
 					__pgprot(0), 1, 0, NULL);
 }
 
+static int __set_memory_enc_dec(unsigned long addr, int numpages, bool enc)
+{
+	struct cpa_data cpa;
+	unsigned long start;
+	int ret;
+
+	/* Nothing to do if the SME is not active */
+	if (!sme_active())
+		return 0;
+
+	/* Should not be working on unaligned addresses */
+	if (WARN_ONCE(addr & ~PAGE_MASK, "misaligned address: %#lx\n", addr))
+		addr &= PAGE_MASK;
+
+	start = addr;
+
+	memset(&cpa, 0, sizeof(cpa));
+	cpa.vaddr = &addr;
+	cpa.numpages = numpages;
+	cpa.mask_set = enc ? __pgprot(_PAGE_ENC) : __pgprot(0);
+	cpa.mask_clr = enc ? __pgprot(0) : __pgprot(_PAGE_ENC);
+	cpa.pgd = init_mm.pgd;
+
+	/* Must avoid aliasing mappings in the highmem code */
+	kmap_flush_unused();
+	vm_unmap_aliases();
+
+	/*
+	 * Before changing the encryption attribute, we need to flush caches.
+	 */
+	if (static_cpu_has(X86_FEATURE_CLFLUSH))
+		cpa_flush_range(start, numpages, 1);
+	else
+		cpa_flush_all(1);
+
+	ret = __change_page_attr_set_clr(&cpa, 1);
+
+	/*
+	 * After changing the encryption attribute, we need to flush TLBs
+	 * again in case any speculative TLB caching occurred (but no need
+	 * to flush caches again).  We could just use cpa_flush_all(), but
+	 * in case TLB flushing gets optimized in the cpa_flush_range()
+	 * path use the same logic as above.
+	 */
+	if (static_cpu_has(X86_FEATURE_CLFLUSH))
+		cpa_flush_range(start, numpages, 0);
+	else
+		cpa_flush_all(0);
+
+	return ret;
+}
+
+int set_memory_encrypted(unsigned long addr, int numpages)
+{
+	return __set_memory_enc_dec(addr, numpages, true);
+}
+
+int set_memory_decrypted(unsigned long addr, int numpages)
+{
+	return __set_memory_enc_dec(addr, numpages, false);
+}
+
 int set_pages_uc(struct page *page, int numpages)
 {
 	unsigned long addr = (unsigned long)page_address(page);



From thomas.lendacky at amd.com  Wed Jun  7 12:16:51 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Wed, 07 Jun 2017 14:16:51 -0500
Subject: [PATCH v6 21/34] x86/mm: Add support to access persistent memory in
 the clear
In-Reply-To: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170607191651.28645.70995.stgit@tlendack-t1.amdoffice.net>

Persistent memory is expected to persist across reboots. The encryption
key used by SME will change across reboots which will result in corrupted
persistent memory.  Persistent memory is handed out by block devices
through memory remapping functions, so be sure not to map this memory as
encrypted.

Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/mm/ioremap.c |   31 ++++++++++++++++++++++++++++++-
 1 file changed, 30 insertions(+), 1 deletion(-)

diff --git a/arch/x86/mm/ioremap.c b/arch/x86/mm/ioremap.c
index 34ed59d..99cda55 100644
--- a/arch/x86/mm/ioremap.c
+++ b/arch/x86/mm/ioremap.c
@@ -428,17 +428,46 @@ void unxlate_dev_mem_ptr(phys_addr_t phys, void *addr)
  * Examine the physical address to determine if it is an area of memory
  * that should be mapped decrypted.  If the memory is not part of the
  * kernel usable area it was accessed and created decrypted, so these
- * areas should be mapped decrypted.
+ * areas should be mapped decrypted. And since the encryption key can
+ * change across reboots, persistent memory should also be mapped
+ * decrypted.
  */
 static bool memremap_should_map_decrypted(resource_size_t phys_addr,
 					  unsigned long size)
 {
+	int is_pmem;
+
+	/*
+	 * Check if the address is part of a persistent memory region.
+	 * This check covers areas added by E820, EFI and ACPI.
+	 */
+	is_pmem = region_intersects(phys_addr, size, IORESOURCE_MEM,
+				    IORES_DESC_PERSISTENT_MEMORY);
+	if (is_pmem != REGION_DISJOINT)
+		return true;
+
+	/*
+	 * Check if the non-volatile attribute is set for an EFI
+	 * reserved area.
+	 */
+	if (efi_enabled(EFI_BOOT)) {
+		switch (efi_mem_type(phys_addr)) {
+		case EFI_RESERVED_TYPE:
+			if (efi_mem_attributes(phys_addr) & EFI_MEMORY_NV)
+				return true;
+			break;
+		default:
+			break;
+		}
+	}
+
 	/* Check if the address is outside kernel usable area */
 	switch (e820__get_entry_type(phys_addr, phys_addr + size - 1)) {
 	case E820_TYPE_RESERVED:
 	case E820_TYPE_ACPI:
 	case E820_TYPE_NVS:
 	case E820_TYPE_UNUSABLE:
+	case E820_TYPE_PRAM:
 		return true;
 	default:
 		break;



From boris.ostrovsky at oracle.com  Wed Jun  7 15:06:50 2017
From: boris.ostrovsky at oracle.com (Boris Ostrovsky)
Date: Wed, 7 Jun 2017 18:06:50 -0400
Subject: [PATCH v6 10/34] x86, x86/mm, x86/xen, olpc: Use __va() against
 just the physical address in cr3
In-Reply-To: <20170607191453.28645.92256.stgit@tlendack-t1.amdoffice.net>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
 <20170607191453.28645.92256.stgit@tlendack-t1.amdoffice.net>
Message-ID: <b15e8924-4069-b5fa-adb2-86c164b1dd36@oracle.com>

On 06/07/2017 03:14 PM, Tom Lendacky wrote:
> The cr3 register entry can contain the SME encryption bit that indicates
> the PGD is encrypted.  The encryption bit should not be used when creating
> a virtual address for the PGD table.
>
> Create a new function, read_cr3_pa(), that will extract the physical
> address from the cr3 register. This function is then used where a virtual
> address of the PGD needs to be created/used from the cr3 register.
>
> Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
> ---
>  arch/x86/include/asm/special_insns.h |    9 +++++++++
>  arch/x86/kernel/head64.c             |    2 +-
>  arch/x86/mm/fault.c                  |   10 +++++-----
>  arch/x86/mm/ioremap.c                |    2 +-
>  arch/x86/platform/olpc/olpc-xo1-pm.c |    2 +-
>  arch/x86/power/hibernate_64.c        |    2 +-
>  arch/x86/xen/mmu_pv.c                |    6 +++---
>  7 files changed, 21 insertions(+), 12 deletions(-)
>
> diff --git a/arch/x86/include/asm/special_insns.h b/arch/x86/include/asm/special_insns.h
> index 12af3e3..d8e8ace 100644
> --- a/arch/x86/include/asm/special_insns.h
> +++ b/arch/x86/include/asm/special_insns.h
> @@ -234,6 +234,15 @@ static inline void clwb(volatile void *__p)
>  
>  #define nop() asm volatile ("nop")
>  
> +static inline unsigned long native_read_cr3_pa(void)
> +{
> +	return (native_read_cr3() & PHYSICAL_PAGE_MASK);
> +}
> +
> +static inline unsigned long read_cr3_pa(void)
> +{
> +	return (read_cr3() & PHYSICAL_PAGE_MASK);
> +}
>  
>  #endif /* __KERNEL__ */
>  
> diff --git a/arch/x86/kernel/head64.c b/arch/x86/kernel/head64.c
> index 43b7002..dc03624 100644
> --- a/arch/x86/kernel/head64.c
> +++ b/arch/x86/kernel/head64.c
> @@ -55,7 +55,7 @@ int __init early_make_pgtable(unsigned long address)
>  	pmdval_t pmd, *pmd_p;
>  
>  	/* Invalid address or early pgt is done ?  */
> -	if (physaddr >= MAXMEM || read_cr3() != __pa_nodebug(early_level4_pgt))
> +	if (physaddr >= MAXMEM || read_cr3_pa() != __pa_nodebug(early_level4_pgt))
>  		return -1;
>  
>  again:
> diff --git a/arch/x86/mm/fault.c b/arch/x86/mm/fault.c
> index 8ad91a0..2a1fa10c 100644
> --- a/arch/x86/mm/fault.c
> +++ b/arch/x86/mm/fault.c
> @@ -346,7 +346,7 @@ static noinline int vmalloc_fault(unsigned long address)
>  	 * Do _not_ use "current" here. We might be inside
>  	 * an interrupt in the middle of a task switch..
>  	 */
> -	pgd_paddr = read_cr3();
> +	pgd_paddr = read_cr3_pa();
>  	pmd_k = vmalloc_sync_one(__va(pgd_paddr), address);
>  	if (!pmd_k)
>  		return -1;
> @@ -388,7 +388,7 @@ static bool low_pfn(unsigned long pfn)
>  
>  static void dump_pagetable(unsigned long address)
>  {
> -	pgd_t *base = __va(read_cr3());
> +	pgd_t *base = __va(read_cr3_pa());
>  	pgd_t *pgd = &base[pgd_index(address)];
>  	p4d_t *p4d;
>  	pud_t *pud;
> @@ -451,7 +451,7 @@ static noinline int vmalloc_fault(unsigned long address)
>  	 * happen within a race in page table update. In the later
>  	 * case just flush:
>  	 */
> -	pgd = (pgd_t *)__va(read_cr3()) + pgd_index(address);
> +	pgd = (pgd_t *)__va(read_cr3_pa()) + pgd_index(address);
>  	pgd_ref = pgd_offset_k(address);
>  	if (pgd_none(*pgd_ref))
>  		return -1;
> @@ -555,7 +555,7 @@ static int bad_address(void *p)
>  
>  static void dump_pagetable(unsigned long address)
>  {
> -	pgd_t *base = __va(read_cr3() & PHYSICAL_PAGE_MASK);
> +	pgd_t *base = __va(read_cr3_pa());
>  	pgd_t *pgd = base + pgd_index(address);
>  	p4d_t *p4d;
>  	pud_t *pud;
> @@ -700,7 +700,7 @@ static int is_f00f_bug(struct pt_regs *regs, unsigned long address)
>  		pgd_t *pgd;
>  		pte_t *pte;
>  
> -		pgd = __va(read_cr3() & PHYSICAL_PAGE_MASK);
> +		pgd = __va(read_cr3_pa());
>  		pgd += pgd_index(address);
>  
>  		pte = lookup_address_in_pgd(pgd, address, &level);
> diff --git a/arch/x86/mm/ioremap.c b/arch/x86/mm/ioremap.c
> index 2a0fa89..e6305dd 100644
> --- a/arch/x86/mm/ioremap.c
> +++ b/arch/x86/mm/ioremap.c
> @@ -427,7 +427,7 @@ void unxlate_dev_mem_ptr(phys_addr_t phys, void *addr)
>  static inline pmd_t * __init early_ioremap_pmd(unsigned long addr)
>  {
>  	/* Don't assume we're using swapper_pg_dir at this point */
> -	pgd_t *base = __va(read_cr3());
> +	pgd_t *base = __va(read_cr3_pa());
>  	pgd_t *pgd = &base[pgd_index(addr)];
>  	p4d_t *p4d = p4d_offset(pgd, addr);
>  	pud_t *pud = pud_offset(p4d, addr);
> diff --git a/arch/x86/platform/olpc/olpc-xo1-pm.c b/arch/x86/platform/olpc/olpc-xo1-pm.c
> index c5350fd..0668aaf 100644
> --- a/arch/x86/platform/olpc/olpc-xo1-pm.c
> +++ b/arch/x86/platform/olpc/olpc-xo1-pm.c
> @@ -77,7 +77,7 @@ static int xo1_power_state_enter(suspend_state_t pm_state)
>  
>  asmlinkage __visible int xo1_do_sleep(u8 sleep_state)
>  {
> -	void *pgd_addr = __va(read_cr3());
> +	void *pgd_addr = __va(read_cr3_pa());
>  
>  	/* Program wakeup mask (using dword access to CS5536_PM1_EN) */
>  	outl(wakeup_mask << 16, acpi_base + CS5536_PM1_STS);
> diff --git a/arch/x86/power/hibernate_64.c b/arch/x86/power/hibernate_64.c
> index a6e21fe..0a7650d 100644
> --- a/arch/x86/power/hibernate_64.c
> +++ b/arch/x86/power/hibernate_64.c
> @@ -150,7 +150,7 @@ static int relocate_restore_code(void)
>  	memcpy((void *)relocated_restore_code, &core_restore_code, PAGE_SIZE);
>  
>  	/* Make the page containing the relocated code executable */
> -	pgd = (pgd_t *)__va(read_cr3()) + pgd_index(relocated_restore_code);
> +	pgd = (pgd_t *)__va(read_cr3_pa()) + pgd_index(relocated_restore_code);
>  	p4d = p4d_offset(pgd, relocated_restore_code);
>  	if (p4d_large(*p4d)) {
>  		set_p4d(p4d, __p4d(p4d_val(*p4d) & ~_PAGE_NX));
> diff --git a/arch/x86/xen/mmu_pv.c b/arch/x86/xen/mmu_pv.c
> index 1f386d7..2dc5243 100644
> --- a/arch/x86/xen/mmu_pv.c
> +++ b/arch/x86/xen/mmu_pv.c
> @@ -2022,7 +2022,7 @@ static phys_addr_t __init xen_early_virt_to_phys(unsigned long vaddr)
>  	pmd_t pmd;
>  	pte_t pte;
>  
> -	pa = read_cr3();
> +	pa = read_cr3_pa();
>  	pgd = native_make_pgd(xen_read_phys_ulong(pa + pgd_index(vaddr) *
>  						       sizeof(pgd)));
>  	if (!pgd_present(pgd))
> @@ -2102,7 +2102,7 @@ void __init xen_relocate_p2m(void)
>  	pt_phys = pmd_phys + PFN_PHYS(n_pmd);
>  	p2m_pfn = PFN_DOWN(pt_phys) + n_pt;
>  
> -	pgd = __va(read_cr3());
> +	pgd = __va(read_cr3_pa());
>  	new_p2m = (unsigned long *)(2 * PGDIR_SIZE);
>  	idx_p4d = 0;
>  	save_pud = n_pud;
> @@ -2209,7 +2209,7 @@ static void __init xen_write_cr3_init(unsigned long cr3)
>  {
>  	unsigned long pfn = PFN_DOWN(__pa(swapper_pg_dir));
>  
> -	BUG_ON(read_cr3() != __pa(initial_page_table));
> +	BUG_ON(read_cr3_pa() != __pa(initial_page_table));
>  	BUG_ON(cr3 != __pa(swapper_pg_dir));
>  
>  	/*


(Please copy Xen maintainers when modifying xen-related files.)

Given that page tables for Xen PV guests are controlled by the
hypervisor I don't think this change (although harmless) is necessary.
What may be needed is making sure X86_FEATURE_SME is not set for PV guests.

-boris



From commendsarnex at gmail.com  Wed Jun  7 19:38:14 2017
From: commendsarnex at gmail.com (Nick Sarnie)
Date: Wed, 7 Jun 2017 22:38:14 -0400
Subject: [PATCH v6 26/34] iommu/amd: Allow the AMD IOMMU to work with
 memory encryption
In-Reply-To: <20170607191745.28645.81756.stgit@tlendack-t1.amdoffice.net>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
 <20170607191745.28645.81756.stgit@tlendack-t1.amdoffice.net>
Message-ID: <CAOcCaLZ5QNx+CdnLn4eHtYOJOezJAsmq2whf8mggOdeMQDWOhw@mail.gmail.com>

On Wed, Jun 7, 2017 at 3:17 PM, Tom Lendacky <thomas.lendacky at amd.com> wrote:
> The IOMMU is programmed with physical addresses for the various tables
> and buffers that are used to communicate between the device and the
> driver. When the driver allocates this memory it is encrypted. In order
> for the IOMMU to access the memory as encrypted the encryption mask needs
> to be included in these physical addresses during configuration.
>
> The PTE entries created by the IOMMU should also include the encryption
> mask so that when the device behind the IOMMU performs a DMA, the DMA
> will be performed to encrypted memory.
>
> Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
> ---
>  arch/x86/include/asm/mem_encrypt.h |    7 +++++++
>  arch/x86/mm/mem_encrypt.c          |   30 ++++++++++++++++++++++++++++++
>  drivers/iommu/amd_iommu.c          |   36 +++++++++++++++++++-----------------
>  drivers/iommu/amd_iommu_init.c     |   18 ++++++++++++------
>  drivers/iommu/amd_iommu_proto.h    |   10 ++++++++++
>  drivers/iommu/amd_iommu_types.h    |    2 +-
>  include/asm-generic/mem_encrypt.h  |    5 +++++
>  7 files changed, 84 insertions(+), 24 deletions(-)
>
> diff --git a/arch/x86/include/asm/mem_encrypt.h b/arch/x86/include/asm/mem_encrypt.h
> index c7a2525..d86e544 100644
> --- a/arch/x86/include/asm/mem_encrypt.h
> +++ b/arch/x86/include/asm/mem_encrypt.h
> @@ -31,6 +31,8 @@ void __init sme_early_decrypt(resource_size_t paddr,
>
>  void __init sme_early_init(void);
>
> +bool sme_iommu_supported(void);
> +
>  /* Architecture __weak replacement functions */
>  void __init mem_encrypt_init(void);
>
> @@ -62,6 +64,11 @@ static inline void __init sme_early_init(void)
>  {
>  }
>
> +static inline bool sme_iommu_supported(void)
> +{
> +       return true;
> +}
> +
>  #endif /* CONFIG_AMD_MEM_ENCRYPT */
>
>  static inline bool sme_active(void)
> diff --git a/arch/x86/mm/mem_encrypt.c b/arch/x86/mm/mem_encrypt.c
> index 5d7c51d..018b58a 100644
> --- a/arch/x86/mm/mem_encrypt.c
> +++ b/arch/x86/mm/mem_encrypt.c
> @@ -197,6 +197,36 @@ void __init sme_early_init(void)
>                 protection_map[i] = pgprot_encrypted(protection_map[i]);
>  }
>
> +bool sme_iommu_supported(void)
> +{
> +       struct cpuinfo_x86 *c = &boot_cpu_data;
> +
> +       if (!sme_me_mask || (c->x86 != 0x17))
> +               return true;
> +
> +       /* For Fam17h, a specific level of support is required */
> +       switch (c->microcode & 0xf000) {
> +       case 0x0000:
> +               return false;
> +       case 0x1000:
> +               switch (c->microcode & 0x0f00) {
> +               case 0x0000:
> +                       return false;
> +               case 0x0100:
> +                       if ((c->microcode & 0xff) < 0x26)
> +                               return false;
> +                       break;
> +               case 0x0200:
> +                       if ((c->microcode & 0xff) < 0x05)
> +                               return false;
> +                       break;
> +               }
> +               break;
> +       }
> +
> +       return true;
> +}
> +
>  /* Architecture __weak replacement functions */
>  void __init mem_encrypt_init(void)
>  {
> diff --git a/drivers/iommu/amd_iommu.c b/drivers/iommu/amd_iommu.c
> index 63cacf5..94eb130 100644
> --- a/drivers/iommu/amd_iommu.c
> +++ b/drivers/iommu/amd_iommu.c
> @@ -544,7 +544,7 @@ static void dump_dte_entry(u16 devid)
>
>  static void dump_command(unsigned long phys_addr)
>  {
> -       struct iommu_cmd *cmd = phys_to_virt(phys_addr);
> +       struct iommu_cmd *cmd = iommu_phys_to_virt(phys_addr);
>         int i;
>
>         for (i = 0; i < 4; ++i)
> @@ -863,13 +863,15 @@ static void copy_cmd_to_buffer(struct amd_iommu *iommu,
>         writel(tail, iommu->mmio_base + MMIO_CMD_TAIL_OFFSET);
>  }
>
> -static void build_completion_wait(struct iommu_cmd *cmd, u64 address)
> +static void build_completion_wait(struct iommu_cmd *cmd, volatile u64 *sem)
>  {
> +       u64 address = iommu_virt_to_phys((void *)sem);
> +
>         WARN_ON(address & 0x7ULL);
>
>         memset(cmd, 0, sizeof(*cmd));
> -       cmd->data[0] = lower_32_bits(__pa(address)) | CMD_COMPL_WAIT_STORE_MASK;
> -       cmd->data[1] = upper_32_bits(__pa(address));
> +       cmd->data[0] = lower_32_bits(address) | CMD_COMPL_WAIT_STORE_MASK;
> +       cmd->data[1] = upper_32_bits(address);
>         cmd->data[2] = 1;
>         CMD_SET_TYPE(cmd, CMD_COMPL_WAIT);
>  }
> @@ -1033,7 +1035,7 @@ static int __iommu_queue_command_sync(struct amd_iommu *iommu,
>
>                 iommu->cmd_sem = 0;
>
> -               build_completion_wait(&sync_cmd, (u64)&iommu->cmd_sem);
> +               build_completion_wait(&sync_cmd, &iommu->cmd_sem);
>                 copy_cmd_to_buffer(iommu, &sync_cmd, tail);
>
>                 if ((ret = wait_on_sem(&iommu->cmd_sem)) != 0)
> @@ -1083,7 +1085,7 @@ static int iommu_completion_wait(struct amd_iommu *iommu)
>                 return 0;
>
>
> -       build_completion_wait(&cmd, (u64)&iommu->cmd_sem);
> +       build_completion_wait(&cmd, &iommu->cmd_sem);
>
>         spin_lock_irqsave(&iommu->lock, flags);
>
> @@ -1328,7 +1330,7 @@ static bool increase_address_space(struct protection_domain *domain,
>                 return false;
>
>         *pte             = PM_LEVEL_PDE(domain->mode,
> -                                       virt_to_phys(domain->pt_root));
> +                                       iommu_virt_to_phys(domain->pt_root));
>         domain->pt_root  = pte;
>         domain->mode    += 1;
>         domain->updated  = true;
> @@ -1365,7 +1367,7 @@ static u64 *alloc_pte(struct protection_domain *domain,
>                         if (!page)
>                                 return NULL;
>
> -                       __npte = PM_LEVEL_PDE(level, virt_to_phys(page));
> +                       __npte = PM_LEVEL_PDE(level, iommu_virt_to_phys(page));
>
>                         /* pte could have been changed somewhere. */
>                         if (cmpxchg64(pte, __pte, __npte) != __pte) {
> @@ -1481,10 +1483,10 @@ static int iommu_map_page(struct protection_domain *dom,
>                         return -EBUSY;
>
>         if (count > 1) {
> -               __pte = PAGE_SIZE_PTE(phys_addr, page_size);
> +               __pte = PAGE_SIZE_PTE(__sme_set(phys_addr), page_size);
>                 __pte |= PM_LEVEL_ENC(7) | IOMMU_PTE_P | IOMMU_PTE_FC;
>         } else
> -               __pte = phys_addr | IOMMU_PTE_P | IOMMU_PTE_FC;
> +               __pte = __sme_set(phys_addr) | IOMMU_PTE_P | IOMMU_PTE_FC;
>
>         if (prot & IOMMU_PROT_IR)
>                 __pte |= IOMMU_PTE_IR;
> @@ -1700,7 +1702,7 @@ static void free_gcr3_tbl_level1(u64 *tbl)
>                 if (!(tbl[i] & GCR3_VALID))
>                         continue;
>
> -               ptr = __va(tbl[i] & PAGE_MASK);
> +               ptr = iommu_phys_to_virt(tbl[i] & PAGE_MASK);
>
>                 free_page((unsigned long)ptr);
>         }
> @@ -1715,7 +1717,7 @@ static void free_gcr3_tbl_level2(u64 *tbl)
>                 if (!(tbl[i] & GCR3_VALID))
>                         continue;
>
> -               ptr = __va(tbl[i] & PAGE_MASK);
> +               ptr = iommu_phys_to_virt(tbl[i] & PAGE_MASK);
>
>                 free_gcr3_tbl_level1(ptr);
>         }
> @@ -1807,7 +1809,7 @@ static void set_dte_entry(u16 devid, struct protection_domain *domain, bool ats)
>         u64 flags = 0;
>
>         if (domain->mode != PAGE_MODE_NONE)
> -               pte_root = virt_to_phys(domain->pt_root);
> +               pte_root = iommu_virt_to_phys(domain->pt_root);
>
>         pte_root |= (domain->mode & DEV_ENTRY_MODE_MASK)
>                     << DEV_ENTRY_MODE_SHIFT;
> @@ -1819,7 +1821,7 @@ static void set_dte_entry(u16 devid, struct protection_domain *domain, bool ats)
>                 flags |= DTE_FLAG_IOTLB;
>
>         if (domain->flags & PD_IOMMUV2_MASK) {
> -               u64 gcr3 = __pa(domain->gcr3_tbl);
> +               u64 gcr3 = iommu_virt_to_phys(domain->gcr3_tbl);
>                 u64 glx  = domain->glx;
>                 u64 tmp;
>
> @@ -3470,10 +3472,10 @@ static u64 *__get_gcr3_pte(u64 *root, int level, int pasid, bool alloc)
>                         if (root == NULL)
>                                 return NULL;
>
> -                       *pte = __pa(root) | GCR3_VALID;
> +                       *pte = iommu_virt_to_phys(root) | GCR3_VALID;
>                 }
>
> -               root = __va(*pte & PAGE_MASK);
> +               root = iommu_phys_to_virt(*pte & PAGE_MASK);
>
>                 level -= 1;
>         }
> @@ -3652,7 +3654,7 @@ static void set_dte_irq_entry(u16 devid, struct irq_remap_table *table)
>
>         dte     = amd_iommu_dev_table[devid].data[2];
>         dte     &= ~DTE_IRQ_PHYS_ADDR_MASK;
> -       dte     |= virt_to_phys(table->table);
> +       dte     |= iommu_virt_to_phys(table->table);
>         dte     |= DTE_IRQ_REMAP_INTCTL;
>         dte     |= DTE_IRQ_TABLE_LEN;
>         dte     |= DTE_IRQ_REMAP_ENABLE;
> diff --git a/drivers/iommu/amd_iommu_init.c b/drivers/iommu/amd_iommu_init.c
> index 5a11328..2870a6b 100644
> --- a/drivers/iommu/amd_iommu_init.c
> +++ b/drivers/iommu/amd_iommu_init.c
> @@ -29,6 +29,7 @@
>  #include <linux/export.h>
>  #include <linux/iommu.h>
>  #include <linux/kmemleak.h>
> +#include <linux/mem_encrypt.h>
>  #include <asm/pci-direct.h>
>  #include <asm/iommu.h>
>  #include <asm/gart.h>
> @@ -346,7 +347,7 @@ static void iommu_set_device_table(struct amd_iommu *iommu)
>
>         BUG_ON(iommu->mmio_base == NULL);
>
> -       entry = virt_to_phys(amd_iommu_dev_table);
> +       entry = iommu_virt_to_phys(amd_iommu_dev_table);
>         entry |= (dev_table_size >> 12) - 1;
>         memcpy_toio(iommu->mmio_base + MMIO_DEV_TABLE_OFFSET,
>                         &entry, sizeof(entry));
> @@ -602,7 +603,7 @@ static void iommu_enable_command_buffer(struct amd_iommu *iommu)
>
>         BUG_ON(iommu->cmd_buf == NULL);
>
> -       entry = (u64)virt_to_phys(iommu->cmd_buf);
> +       entry = iommu_virt_to_phys(iommu->cmd_buf);
>         entry |= MMIO_CMD_SIZE_512;
>
>         memcpy_toio(iommu->mmio_base + MMIO_CMD_BUF_OFFSET,
> @@ -631,7 +632,7 @@ static void iommu_enable_event_buffer(struct amd_iommu *iommu)
>
>         BUG_ON(iommu->evt_buf == NULL);
>
> -       entry = (u64)virt_to_phys(iommu->evt_buf) | EVT_LEN_MASK;
> +       entry = iommu_virt_to_phys(iommu->evt_buf) | EVT_LEN_MASK;
>
>         memcpy_toio(iommu->mmio_base + MMIO_EVT_BUF_OFFSET,
>                     &entry, sizeof(entry));
> @@ -664,7 +665,7 @@ static void iommu_enable_ppr_log(struct amd_iommu *iommu)
>         if (iommu->ppr_log == NULL)
>                 return;
>
> -       entry = (u64)virt_to_phys(iommu->ppr_log) | PPR_LOG_SIZE_512;
> +       entry = iommu_virt_to_phys(iommu->ppr_log) | PPR_LOG_SIZE_512;
>
>         memcpy_toio(iommu->mmio_base + MMIO_PPR_LOG_OFFSET,
>                     &entry, sizeof(entry));
> @@ -744,10 +745,10 @@ static int iommu_init_ga_log(struct amd_iommu *iommu)
>         if (!iommu->ga_log_tail)
>                 goto err_out;
>
> -       entry = (u64)virt_to_phys(iommu->ga_log) | GA_LOG_SIZE_512;
> +       entry = iommu_virt_to_phys(iommu->ga_log) | GA_LOG_SIZE_512;
>         memcpy_toio(iommu->mmio_base + MMIO_GA_LOG_BASE_OFFSET,
>                     &entry, sizeof(entry));
> -       entry = ((u64)virt_to_phys(iommu->ga_log) & 0xFFFFFFFFFFFFFULL) & ~7ULL;
> +       entry = (iommu_virt_to_phys(iommu->ga_log) & 0xFFFFFFFFFFFFFULL) & ~7ULL;
>         memcpy_toio(iommu->mmio_base + MMIO_GA_LOG_TAIL_OFFSET,
>                     &entry, sizeof(entry));
>         writel(0x00, iommu->mmio_base + MMIO_GA_HEAD_OFFSET);
> @@ -2552,6 +2553,11 @@ int __init amd_iommu_detect(void)
>         if (amd_iommu_disabled)
>                 return -ENODEV;
>
> +       if (!sme_iommu_supported()) {
> +               pr_notice("AMD-Vi: IOMMU not supported when SME is active\n");
> +               return -ENODEV;
> +       }
> +
>         ret = iommu_go_to_state(IOMMU_IVRS_DETECTED);
>         if (ret)
>                 return ret;
> diff --git a/drivers/iommu/amd_iommu_proto.h b/drivers/iommu/amd_iommu_proto.h
> index 466260f..3f12fb2 100644
> --- a/drivers/iommu/amd_iommu_proto.h
> +++ b/drivers/iommu/amd_iommu_proto.h
> @@ -87,4 +87,14 @@ static inline bool iommu_feature(struct amd_iommu *iommu, u64 f)
>         return !!(iommu->features & f);
>  }
>
> +static inline u64 iommu_virt_to_phys(void *vaddr)
> +{
> +       return (u64)__sme_set(virt_to_phys(vaddr));
> +}
> +
> +static inline void *iommu_phys_to_virt(unsigned long paddr)
> +{
> +       return phys_to_virt(__sme_clr(paddr));
> +}
> +
>  #endif /* _ASM_X86_AMD_IOMMU_PROTO_H  */
> diff --git a/drivers/iommu/amd_iommu_types.h b/drivers/iommu/amd_iommu_types.h
> index 4de8f41..3ce587d 100644
> --- a/drivers/iommu/amd_iommu_types.h
> +++ b/drivers/iommu/amd_iommu_types.h
> @@ -343,7 +343,7 @@
>
>  #define IOMMU_PAGE_MASK (((1ULL << 52) - 1) & ~0xfffULL)
>  #define IOMMU_PTE_PRESENT(pte) ((pte) & IOMMU_PTE_P)
> -#define IOMMU_PTE_PAGE(pte) (phys_to_virt((pte) & IOMMU_PAGE_MASK))
> +#define IOMMU_PTE_PAGE(pte) (iommu_phys_to_virt((pte) & IOMMU_PAGE_MASK))
>  #define IOMMU_PTE_MODE(pte) (((pte) >> 9) & 0x07)
>
>  #define IOMMU_PROT_MASK 0x03
> diff --git a/include/asm-generic/mem_encrypt.h b/include/asm-generic/mem_encrypt.h
> index fb02ff0..bbc49e1 100644
> --- a/include/asm-generic/mem_encrypt.h
> +++ b/include/asm-generic/mem_encrypt.h
> @@ -27,6 +27,11 @@ static inline u64 sme_dma_mask(void)
>         return 0ULL;
>  }
>
> +static inline bool sme_iommu_supported(void)
> +{
> +       return true;
> +}
> +
>  /*
>   * The __sme_set() and __sme_clr() macros are useful for adding or removing
>   * the encryption mask from a value (e.g. when dealing with pagetable
>
> _______________________________________________
> iommu mailing list
> iommu at lists.linux-foundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/iommu


Hi Tom,

This sounds like a cool feature. I'm trying to test it on my Ryzen
system, but c->microcode & 0xf000 is evaluating as 0, so IOMMU is not
being enabled on my system. I'm using the latest microcode for AGESA
1.0.0.6, 0x08001126. Is this work reliant on a future microcode
update, or is there some other issue?

Thanks,
Sarnex


From commendsarnex at gmail.com  Wed Jun  7 19:40:03 2017
From: commendsarnex at gmail.com (Nick Sarnie)
Date: Wed, 7 Jun 2017 22:40:03 -0400
Subject: [PATCH v6 00/34] x86: Secure Memory Encryption (AMD)
In-Reply-To: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
Message-ID: <CAOcCaLYWoOu0c-Fkee-=wegNqkzUp9pLFLmaFrXuhiXRnUZ3Xw@mail.gmail.com>

On Wed, Jun 7, 2017 at 3:13 PM, Tom Lendacky <thomas.lendacky at amd.com> wrote:
> This patch series provides support for AMD's new Secure Memory Encryption (SME)
> feature.
>
> SME can be used to mark individual pages of memory as encrypted through the
> page tables. A page of memory that is marked encrypted will be automatically
> decrypted when read from DRAM and will be automatically encrypted when
> written to DRAM. Details on SME can found in the links below.
>
> The SME feature is identified through a CPUID function and enabled through
> the SYSCFG MSR. Once enabled, page table entries will determine how the
> memory is accessed. If a page table entry has the memory encryption mask set,
> then that memory will be accessed as encrypted memory. The memory encryption
> mask (as well as other related information) is determined from settings
> returned through the same CPUID function that identifies the presence of the
> feature.
>
> The approach that this patch series takes is to encrypt everything possible
> starting early in the boot where the kernel is encrypted. Using the page
> table macros the encryption mask can be incorporated into all page table
> entries and page allocations. By updating the protection map, userspace
> allocations are also marked encrypted. Certain data must be accounted for
> as having been placed in memory before SME was enabled (EFI, initrd, etc.)
> and accessed accordingly.
>
> This patch series is a pre-cursor to another AMD processor feature called
> Secure Encrypted Virtualization (SEV). The support for SEV will build upon
> the SME support and will be submitted later. Details on SEV can be found
> in the links below.
>
> The following links provide additional detail:
>
> AMD Memory Encryption whitepaper:
>    http://amd-dev.wpengine.netdna-cdn.com/wordpress/media/2013/12/AMD_Memory_Encryption_Whitepaper_v7-Public.pdf
>
> AMD64 Architecture Programmer's Manual:
>    http://support.amd.com/TechDocs/24593.pdf
>    SME is section 7.10
>    SEV is section 15.34
>
> ---
>
> This patch series is based off of the master branch of tip.
>   Commit 53614fbd7961 ("Merge branch 'WIP.x86/fpu'")
>
> Source code is also available at https://github.com/codomania/tip/tree/sme-v6
>
>
> Still to do:
> - Kdump support, including using memremap() instead of ioremap_cache()
>
> Changes since v5:
> - Added support for 5-level paging
> - Added IOMMU support
> - Created a generic asm/mem_encrypt.h in order to remove a bunch of
>   #ifndef/#define entries
> - Removed changes to the __va() macro and defined a function to return
>   the true physical address in cr3
> - Removed sysfs support as it was determined not to be needed
> - General code cleanup based on feedback
> - General cleanup of patch subjects and descriptions
>
> Changes since v4:
> - Re-worked mapping of setup data to not use a fixed list. Rather, check
>   dynamically whether the requested early_memremap()/memremap() call
>   needs to be mapped decrypted.
> - Moved SME cpu feature into scattered features
> - Moved some declarations into header files
> - Cleared the encryption mask from the __PHYSICAL_MASK so that users
>   of macros such as pmd_pfn_mask() don't have to worry/know about the
>   encryption mask
> - Updated some return types and values related to EFI and e820 functions
>   so that an error could be returned
> - During cpu shutdown, removed cache disabling and added a check for kexec
>   in progress to use wbinvd followed immediately by halt in order to avoid
>   any memory corruption
> - Update how persistent memory is identified
> - Added a function to find command line arguments and their values
> - Added sysfs support
> - General code cleanup based on feedback
> - General cleanup of patch subjects and descriptions
>
>
> Changes since v3:
> - Broke out some of the patches into smaller individual patches
> - Updated Documentation
> - Added a message to indicate why the IOMMU was disabled
> - Updated CPU feature support for SME by taking into account whether
>   BIOS has enabled SME
> - Eliminated redundant functions
> - Added some warning messages for DMA usage of bounce buffers when SME
>   is active
> - Added support for persistent memory
> - Added support to determine when setup data is being mapped and be sure
>   to map it un-encrypted
> - Added CONFIG support to set the default action of whether to activate
>   SME if it is supported/enabled
> - Added support for (re)booting with kexec
>
> Changes since v2:
> - Updated Documentation
> - Make the encryption mask available outside of arch/x86 through a
>   standard include file
> - Conversion of assembler routines to C where possible (not everything
>   could be converted, e.g. the routine that does the actual encryption
>   needs to be copied into a safe location and it is difficult to
>   determine the actual length of the function in order to copy it)
> - Fix SME feature use of scattered CPUID feature
> - Creation of SME specific functions for things like encrypting
>   the setup data, ramdisk, etc.
> - New take on early_memremap / memremap encryption support
> - Additional support for accessing video buffers (fbdev/gpu) as
>   un-encrypted
> - Disable IOMMU for now - need to investigate further in relation to
>   how it needs to be programmed relative to accessing physical memory
>
> Changes since v1:
> - Added Documentation.
> - Removed AMD vendor check for setting the PAT write protect mode
> - Updated naming of trampoline flag for SME as well as moving of the
>   SME check to before paging is enabled.
> - Change to early_memremap to identify the data being mapped as either
>   boot data or kernel data.  The idea being that boot data will have
>   been placed in memory as un-encrypted data and would need to be accessed
>   as such.
> - Updated debugfs support for the bootparams to access the data properly.
> - Do not set the SYSCFG[MEME] bit, only check it.  The setting of the
>   MemEncryptionModeEn bit results in a reduction of physical address size
>   of the processor.  It is possible that BIOS could have configured resources
>   resources into a range that will now not be addressable.  To prevent this,
>   rely on BIOS to set the SYSCFG[MEME] bit and only then enable memory
>   encryption support in the kernel.
>
> Tom Lendacky (34):
>       x86: Document AMD Secure Memory Encryption (SME)
>       x86/mm/pat: Set write-protect cache mode for full PAT support
>       x86, mpparse, x86/acpi, x86/PCI, x86/dmi, SFI: Use memremap for RAM mappings
>       x86/CPU/AMD: Add the Secure Memory Encryption CPU feature
>       x86/CPU/AMD: Handle SME reduction in physical address size
>       x86/mm: Add Secure Memory Encryption (SME) support
>       x86/mm: Don't use phys_to_virt in ioremap() if SME is active
>       x86/mm: Add support to enable SME in early boot processing
>       x86/mm: Simplify p[gum]d_page() macros
>       x86, x86/mm, x86/xen, olpc: Use __va() against just the physical address in cr3
>       x86/mm: Provide general kernel support for memory encryption
>       x86/mm: Extend early_memremap() support with additional attrs
>       x86/mm: Add support for early encrypt/decrypt of memory
>       x86/mm: Insure that boot memory areas are mapped properly
>       x86/boot/e820: Add support to determine the E820 type of an address
>       efi: Add an EFI table address match function
>       efi: Update efi_mem_type() to return an error rather than 0
>       x86/efi: Update EFI pagetable creation to work with SME
>       x86/mm: Add support to access boot related data in the clear
>       x86, mpparse: Use memremap to map the mpf and mpc data
>       x86/mm: Add support to access persistent memory in the clear
>       x86/mm: Add support for changing the memory encryption attribute
>       x86, realmode: Decrypt trampoline area if memory encryption is active
>       x86, swiotlb: Add memory encryption support
>       swiotlb: Add warnings for use of bounce buffers with SME
>       iommu/amd: Allow the AMD IOMMU to work with memory encryption
>       x86, realmode: Check for memory encryption on the APs
>       x86, drm, fbdev: Do not specify encrypted memory for video mappings
>       kvm: x86: svm: Support Secure Memory Encryption within KVM
>       x86/mm, kexec: Allow kexec to be used with SME
>       x86/mm: Use proper encryption attributes with /dev/mem
>       x86/mm: Add support to encrypt the kernel in-place
>       x86/boot: Add early cmdline parsing for options with arguments
>       x86/mm: Add support to make use of Secure Memory Encryption
>
>
>  Documentation/admin-guide/kernel-parameters.txt |   11
>  Documentation/x86/amd-memory-encryption.txt     |   68 ++
>  arch/ia64/kernel/efi.c                          |    4
>  arch/x86/Kconfig                                |   26 +
>  arch/x86/boot/compressed/pagetable.c            |    7
>  arch/x86/include/asm/cmdline.h                  |    2
>  arch/x86/include/asm/cpufeatures.h              |    1
>  arch/x86/include/asm/dma-mapping.h              |    5
>  arch/x86/include/asm/dmi.h                      |    8
>  arch/x86/include/asm/e820/api.h                 |    2
>  arch/x86/include/asm/fixmap.h                   |   20 +
>  arch/x86/include/asm/init.h                     |    1
>  arch/x86/include/asm/io.h                       |    7
>  arch/x86/include/asm/kexec.h                    |    8
>  arch/x86/include/asm/kvm_host.h                 |    2
>  arch/x86/include/asm/mem_encrypt.h              |  112 ++++
>  arch/x86/include/asm/msr-index.h                |    2
>  arch/x86/include/asm/page_types.h               |    2
>  arch/x86/include/asm/pgtable.h                  |   28 +
>  arch/x86/include/asm/pgtable_types.h            |   54 +-
>  arch/x86/include/asm/processor.h                |    3
>  arch/x86/include/asm/realmode.h                 |   12
>  arch/x86/include/asm/set_memory.h               |    3
>  arch/x86/include/asm/special_insns.h            |    9
>  arch/x86/include/asm/vga.h                      |   14
>  arch/x86/kernel/acpi/boot.c                     |    6
>  arch/x86/kernel/cpu/amd.c                       |   17 +
>  arch/x86/kernel/cpu/scattered.c                 |    1
>  arch/x86/kernel/e820.c                          |   26 +
>  arch/x86/kernel/espfix_64.c                     |    2
>  arch/x86/kernel/head64.c                        |   42 +
>  arch/x86/kernel/head_64.S                       |   80 ++-
>  arch/x86/kernel/kdebugfs.c                      |   34 -
>  arch/x86/kernel/ksysfs.c                        |   28 -
>  arch/x86/kernel/machine_kexec_64.c              |   35 +
>  arch/x86/kernel/mpparse.c                       |  108 +++-
>  arch/x86/kernel/pci-dma.c                       |   11
>  arch/x86/kernel/pci-nommu.c                     |    2
>  arch/x86/kernel/pci-swiotlb.c                   |   15 -
>  arch/x86/kernel/process.c                       |   17 +
>  arch/x86/kernel/setup.c                         |    9
>  arch/x86/kvm/mmu.c                              |   12
>  arch/x86/kvm/mmu.h                              |    2
>  arch/x86/kvm/svm.c                              |   35 +
>  arch/x86/kvm/vmx.c                              |    3
>  arch/x86/kvm/x86.c                              |    3
>  arch/x86/lib/cmdline.c                          |  105 ++++
>  arch/x86/mm/Makefile                            |    3
>  arch/x86/mm/fault.c                             |   10
>  arch/x86/mm/ident_map.c                         |   12
>  arch/x86/mm/ioremap.c                           |  277 +++++++++-
>  arch/x86/mm/kasan_init_64.c                     |    4
>  arch/x86/mm/mem_encrypt.c                       |  667 +++++++++++++++++++++++
>  arch/x86/mm/mem_encrypt_boot.S                  |  150 +++++
>  arch/x86/mm/pageattr.c                          |   67 ++
>  arch/x86/mm/pat.c                               |    9
>  arch/x86/pci/common.c                           |    4
>  arch/x86/platform/efi/efi.c                     |    6
>  arch/x86/platform/efi/efi_64.c                  |   15 -
>  arch/x86/platform/olpc/olpc-xo1-pm.c            |    2
>  arch/x86/power/hibernate_64.c                   |    2
>  arch/x86/realmode/init.c                        |   15 +
>  arch/x86/realmode/rm/trampoline_64.S            |   24 +
>  arch/x86/xen/mmu_pv.c                           |    6
>  drivers/firmware/dmi-sysfs.c                    |    5
>  drivers/firmware/efi/efi.c                      |   33 +
>  drivers/firmware/pcdp.c                         |    4
>  drivers/gpu/drm/drm_gem.c                       |    2
>  drivers/gpu/drm/drm_vm.c                        |    4
>  drivers/gpu/drm/ttm/ttm_bo_vm.c                 |    7
>  drivers/gpu/drm/udl/udl_fb.c                    |    4
>  drivers/iommu/amd_iommu.c                       |   36 +
>  drivers/iommu/amd_iommu_init.c                  |   18 -
>  drivers/iommu/amd_iommu_proto.h                 |   10
>  drivers/iommu/amd_iommu_types.h                 |    2
>  drivers/sfi/sfi_core.c                          |   22 -
>  drivers/video/fbdev/core/fbmem.c                |   12
>  include/asm-generic/early_ioremap.h             |    2
>  include/asm-generic/mem_encrypt.h               |   45 ++
>  include/asm-generic/pgtable.h                   |    8
>  include/linux/dma-mapping.h                     |    9
>  include/linux/efi.h                             |    9
>  include/linux/io.h                              |    2
>  include/linux/kexec.h                           |   14
>  include/linux/mem_encrypt.h                     |   18 +
>  include/linux/swiotlb.h                         |    1
>  init/main.c                                     |   13
>  kernel/kexec_core.c                             |    6
>  kernel/memremap.c                               |   20 +
>  lib/swiotlb.c                                   |   59 ++
>  mm/early_ioremap.c                              |   30 +
>  91 files changed, 2411 insertions(+), 261 deletions(-)
>  create mode 100644 Documentation/x86/amd-memory-encryption.txt
>  create mode 100644 arch/x86/include/asm/mem_encrypt.h
>  create mode 100644 arch/x86/mm/mem_encrypt.c
>  create mode 100644 arch/x86/mm/mem_encrypt_boot.S
>  create mode 100644 include/asm-generic/mem_encrypt.h
>  create mode 100644 include/linux/mem_encrypt.h
>
> --
> Tom Lendacky
> _______________________________________________
> iommu mailing list
> iommu at lists.linux-foundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/iommu


Hi Tom,

Thanks for your work on this. This may be a stupid question, but is
using bounce buffers for the GPU(s) expected to reduce performance in
any/a noticeable way? I'm hitting another issue which I've already
sent mail about so I can't test it for myself at the moment,

Thanks,
Sarnex


From panand at redhat.com  Wed Jun  7 22:13:03 2017
From: panand at redhat.com (Pratyush Anand)
Date: Thu, 8 Jun 2017 10:43:03 +0530
Subject: makedumpfile: support for newer kernels [v4.9 onwards]
In-Reply-To: <0ae1e066-821e-3b11-ee44-b615245cb1ea@redhat.com>
References: <CAEzMDiD-w91Q+Fjy_KgXDcwo9YGasa9ggqW2dEhF4ffV7RstqQ@mail.gmail.com>
 <0bd0a35a-88cd-3f68-846f-2a2ad1e7afcb@redhat.com>
 <CAEzMDiBKdA15Gn2kgs=BgzCM=bzaBXCQP_tM_Vmfc4uxGL0SRg@mail.gmail.com>
 <0ae1e066-821e-3b11-ee44-b615245cb1ea@redhat.com>
Message-ID: <3b72a75c-012e-2779-74a4-d36ca1ef9e56@redhat.com>



On Thursday 08 June 2017 10:31 AM, Pratyush Anand wrote:
>
>
> On Thursday 08 June 2017 10:23 AM, Abhishek Shah wrote:
>> Hi Pratyush,
>> I have posted the same question on the list as well....
>> Maybe I will copy your response there and continue discussion...
>
> Ah..I was on PTO, and just chasing with volume of mails.
>
> I see your mail at list :-)
>
>> I am using yocto(2.3 version) to build makedumpfile, which fetches v1.6.1
>> makedumpfile and also applys some patches including arm64 specific patches.

What are these ARM64 specific patches? I believe, you do not need any topup in 
makedumpfile for ARM64 if you use v1.6.1.

>> We have VA_BITS set as 48 bits... I will get other information and post on the
>> list.

OK.

>>
>> Regards,
>> Abhishek
>>
>> On Jun 8, 2017 10:05 AM, "Pratyush Anand" <panand at redhat.com
>> <mailto:panand at redhat.com>> wrote:
>>
>>     Hi Abhishek,
>>
>>
>>     On Monday 05 June 2017 04:51 PM, Abhishek Shah wrote:
>>
>>         Hi Atsushi,
>>
>>         I am using makedumpfile utility to compress the ramdump taken by
>>         kexec/kdump,
>>         i.e. /proc/vmcore, on arm64 platform.
>>
>>         /proc/vmcore works fine with crash utility without passing it though
>>         makedumpfile, but the file generated by makedumpfile is not getting
>>         processed
>>         by crash utility; I am getting following error:
>>         crash: seek error: kernel virtual address: ffff8008000c16a0  type:
>> "memory
>>         section"
>>
>>         Makedumpfile currently seems to support kernel up to 4.8 as per
>>         documentation.
>>         Is there plan to support newer kernel? can you provide some pointers to
>>         modifications required for newer kernel support, if I can help?
>>
>>
>>     Are you using makedumpfile-1.6.1? I expect that it should work with latest
>>     ARM64 kernel as well. If it does not work, can you please share your
>>     kernel configuration specially about,
>>
>>     VA_BITS, PAGE_SHIFT, RANDOMIZE_BASE etc.
>>
>>     ..and please keep mailing list in CC so that you can get answer from more
>>     people. Also, other can get benefit from this conversation.
>>
>>     ~Pratyush
>>
>>

~Pratyush


From panand at redhat.com  Wed Jun  7 22:08:51 2017
From: panand at redhat.com (Pratyush Anand)
Date: Thu, 8 Jun 2017 10:38:51 +0530
Subject: crash kernel not booting
In-Reply-To: <CAEwzPJ0GZuzPXGLB2NcZt=JQq3+BQFeAeD-VxX280rY+wEQHbQ@mail.gmail.com>
References: <CAEwzPJ0GZuzPXGLB2NcZt=JQq3+BQFeAeD-VxX280rY+wEQHbQ@mail.gmail.com>
Message-ID: <ff3df802-eb41-d3ff-c850-772a63c3c167@redhat.com>



On Saturday 03 June 2017 07:11 PM, Mahesh V wrote:
> Hi folks,
>
> Have mostly exhausted all my options to make my ARM crash kernel boot :(.
> Nothings worked so far. Any pointers as to how should I proceed further?
>
> Following are my .config parameters that I set.
> +CONFIG_KEXEC=y
> +CONFIG_CRASH_DUMP=y
> +CONFIG_DEBUG_LL=y
> +CONFIG_DEBUG_LL_UART_NONE=y
> +CONFIG_EARLY_PRINTK=y
>
> defaults. If I set CONFIG_ARM_PATCH_PHYS_VIRT, I get a compile error
> ('__pv_phys_offset' undeclared here (not in a function))
>
> CONFIG_PHYS_OFFSET=0x41500000
> # CONFIG_ARM_PATCH_PHYS_VIRT is not set
> # CONFIG_AUTO_ZRELADDR is not set
>
>
> I am trying to load the uncompressed vmlinux (not uImage or zImage) to
> 0x44800000 location in my RAM (crash kernel location)
>
> iomem has the following entry
> 44800000-5fbfffff : System RAM
>   44800000-4ddfffff : Crash kernel
>
>
> I used the following kexec command to load the kernel to that location
> /aruba/bin/kexec -d -p /aruba/bin/vmlinux
> --command-line="console=ttyHSL1,9600n8 rdinit=/sbin/init ubi.mtd=aos0
> ubi.mtd=aos1 ubi.mtd=ubifs crashkernel=150m at 1096m earlyprintk noisy"
> --atags --mem-min=0x44800000
>

I see mail on the list now.

As I had suggested,
- No need to use crashkernel=150m at 1096m in the command line of second kernel
- It seems that your earlyprintk is still not working. So, it would be good to 
fix that first, so that you can know if your crash kernel itself crashed very 
early.

>
> The crash kernel vmlinux does load at that location. Verified using dd
> command (44808000 == 1149272064)
> /aruba/bin # dd if=/dev/mem skip=1149272064 bs=1 count=20  | od -x
> 20+0000000     457f    464c    0101    0001    0000    0000    0000    0000
> 0 records in
> 20+00000020     0002    0028
>  records out
> 0000024
>
>
> However when I object dump the vmlinux, it has addresses pointing to c0000000
> Is this the reason why my crash kernel never boots (because the
> locations in  vmlinux is c0000000 and not 0x44800000 which is my start
> address)
>
> these are the last few lines after crashing the main kernel
>
> [ 7152.560043] Code: e3a02001 e5832000 f57ff04f e3a03000 (e5c32000)
> [ 7152.634583] Loading crashdump kernel...
> [ 7152.678819] image start ---- image->start 44808000
> [ 7152.736082]  kexec start address 44808000,
> [ 7152.786098]  reboot_code_buffer 0xc3300000,
> [ 7152.837144]  reboot_code_buffer_phys 44800000,
> [ 7152.891315]  kexec_mach_type 4704
> [ 7152.931927] Bye!
> [ 7152.953920] Inside __soft_restart !! c037fb68 4187fb68
> [ 7153.016338] __soft_restart final address 44800000
>
> code snippet from process.c in static void __soft_restart(void *addr) function
>         phys_reset = (phys_reset_t)(unsigned long)virt_to_phys(cpu_reset);
>         printk(KERN_EMERG "__soft_restart final address %p \n",
> (unsigned char *)addr);
>
>         phys_reset((unsigned long)addr);
>         /* Should never get here. */
>         BUG();
>
>
> After cpu_reset I do not see any prints or kernel messages. How do I
> debug further ?
> Any pointers would help
>
>
> objdump of my vmlinux
>
> [mv_iap43_2 linux-3.4.0 (IAP-OCTOMORE)]$
> ../../../../aos-cmn/tools/armv7-openwrt-linux-gnu-uClibc-0.9.33.2/bin/arm-openwrt-linux-objdump
> -d vmlinux | less
>
> vmlinux:     file format elf32-littlearm
>
>
> Disassembly of section .head.text:
>
> c0008000 <stext>:
> c0008000:       e321f0d3        msr     CPSR_c, #211    ; 0xd3
> c0008004:       ee109f10        mrc     15, 0, r9, cr0, cr0, {0}
> c0008008:       eb0daf8c        bl      c0373e40 <__lookup_processor_type>
> c000800c:       e1b0a005        movs    sl, r5
> c0008010:       e28f302c        add     r3, pc, #44     ; 0x2c
> c0008014:       e8930110        ldm     r3, {r4, r8}
> c0008018:       e0434004        sub     r4, r3, r4
> c000801c:       e0888004        add     r8, r8, r4
> c0008020:       eb000009        bl      c000804c <__vet_atags>
> c0008024:       eb1fdff5        bl      c0800000 <__init_begin>
> c0008028:       eb000159        bl      c0008594 <__create_page_tables>
> c000802c:       e59fd00c        ldr     sp, [pc, #12]   ; c0008040 <stext+0x40>
> c0008030:       e28fe004        add     lr, pc, #4
> c0008034:       e1a08004        mov     r8, r4
> c0008038:       e28af010        add     pc, sl, #16
> c000803c:       ea0daf74        b       c0373e14 <__enable_mmu>
> c0008040:       c080005c        .word   0xc080005c
> c0008044:       c0008044        .word   0xc0008044
> c0008048:       c0000000        .word   0xc0000000
>
>
>  Thanks in advance
> regards,
> Mahesh
>
> _______________________________________________
> kexec mailing list
> kexec at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/kexec
>


~Pratyush


From abrs612 at gmail.com  Wed Jun  7 23:00:31 2017
From: abrs612 at gmail.com (Abhishek Shah)
Date: Thu, 8 Jun 2017 11:30:31 +0530
Subject: makedumpfile: support for newer kernels [v4.9 onwards]
In-Reply-To: <3b72a75c-012e-2779-74a4-d36ca1ef9e56@redhat.com>
References: <CAEzMDiD-w91Q+Fjy_KgXDcwo9YGasa9ggqW2dEhF4ffV7RstqQ@mail.gmail.com>
 <0bd0a35a-88cd-3f68-846f-2a2ad1e7afcb@redhat.com>
 <CAEzMDiBKdA15Gn2kgs=BgzCM=bzaBXCQP_tM_Vmfc4uxGL0SRg@mail.gmail.com>
 <0ae1e066-821e-3b11-ee44-b615245cb1ea@redhat.com>
 <3b72a75c-012e-2779-74a4-d36ca1ef9e56@redhat.com>
Message-ID: <CAEzMDiARnp-FMqUcoVdCBSSApaRbenvrk7cawgyo8kJJvy0jXg@mail.gmail.com>

Hi Pratyush,

>>> I am using yocto(2.3 version) to build makedumpfile, which fetches v1.6.1
>>> makedumpfile and also applys some patches including arm64 specific
>>> patches.
>
>
> What are these ARM64 specific patches? I believe, you do not need any topup
> in makedumpfile for ARM64 if you use v1.6.1.

Sorry, I got confused with yocto kexec recipe.
There are NO arm64 specific patches in yocto for makedumpfile.

>>> We have VA_BITS set as 48 bits... I will get other information and post
>>> on the
>>> list.
>
>
> OK.
>
Kernel config:
ARM64_VA_BITS_48 [=y]
ARM64_PAGE_SHIFT [=12]
RANDOMIZE_BASE [=n]
ARM64_4K_PAGES [=y]

Regards,
Abhishek


From luto at kernel.org  Wed Jun  7 23:05:42 2017
From: luto at kernel.org (Andy Lutomirski)
Date: Wed, 7 Jun 2017 23:05:42 -0700
Subject: [PATCH v6 10/34] x86, x86/mm, x86/xen, olpc: Use __va() against
 just the physical address in cr3
In-Reply-To: <20170607191453.28645.92256.stgit@tlendack-t1.amdoffice.net>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
 <20170607191453.28645.92256.stgit@tlendack-t1.amdoffice.net>
Message-ID: <CALCETrVVhMf=zkiDNn_-hKDZLGXKFiwxuWkPmD5RJgHa5VUMiQ@mail.gmail.com>

On Wed, Jun 7, 2017 at 12:14 PM, Tom Lendacky <thomas.lendacky at amd.com> wrote:
> The cr3 register entry can contain the SME encryption bit that indicates
> the PGD is encrypted.  The encryption bit should not be used when creating
> a virtual address for the PGD table.
>
> Create a new function, read_cr3_pa(), that will extract the physical
> address from the cr3 register. This function is then used where a virtual
> address of the PGD needs to be created/used from the cr3 register.

This is going to conflict with:

https://git.kernel.org/pub/scm/linux/kernel/git/luto/linux.git/commit/?h=x86/pcid&id=555c81e5d01a62b629ec426a2f50d27e2127c1df

We're both encountering the fact that CR3 munges the page table PA
with some other stuff, and some readers want to see the actual CR3
value and other readers just want the PA.  The thing I prefer about my
patch is that I get rid of read_cr3() entirely, forcing the patch to
update every single reader, making review and conflict resolution much
safer.

I'd be willing to send a patch tomorrow that just does the split into
__read_cr3() and read_cr3_pa() (I like your name better) and then we
can both base on top of it.  Would that make sense?

Also:

> +static inline unsigned long read_cr3_pa(void)
> +{
> +       return (read_cr3() & PHYSICAL_PAGE_MASK);
> +}

Is there any guarantee that the magic encryption bit is masked out in
PHYSICAL_PAGE_MASK?  The docs make it sound like it could be any bit.
(But if it's one of the low 12 bits, that would be quite confusing.)

--Andy


From panand at redhat.com  Wed Jun  7 23:36:39 2017
From: panand at redhat.com (Pratyush Anand)
Date: Thu, 8 Jun 2017 12:06:39 +0530
Subject: [Makedumpfile PATCH v4 1/2] makedumpfile: add runtime kaslr
 offset if it exists
In-Reply-To: <302c16ede3723264ca0abb5eef9c9f553ffcfbd7.1495767738.git.panand@redhat.com>
References: <cover.1495767738.git.panand@redhat.com>
 <302c16ede3723264ca0abb5eef9c9f553ffcfbd7.1495767738.git.panand@redhat.com>
Message-ID: <f603a814-d112-fa67-ac89-0b9173b4ed7c@redhat.com>

Hi Atsushi,

On Friday 26 May 2017 08:36 AM, Pratyush Anand wrote:
> +#define get_kaslr_offset(X)	FALSE
>  #define get_xen_basic_info_arch(X) get_xen_basic_info_arm64(X)
>  #define get_xen_info_arch(X) get_xen_info_arm64(X)
>  #define is_phys_addr(X)		stub_true_ul(X)
> @@ -851,6 +857,7 @@ unsigned long long vaddr_to_paddr_arm(unsigned long vaddr);
>  #define get_phys_base()		get_phys_base_arm()
>  #define get_machdep_info()	get_machdep_info_arm()
>  #define get_versiondep_info()	stub_true()
> +#define get_kaslr_offset(X)	FALSE


It seems that using stub_false() would be better than FALSE.

I see, its not yet in upstream/devel branch.
So, Either you can edit them before applying or may be I can send v5.

Please let me know.

~Pratyush


From panand at redhat.com  Thu Jun  8 00:09:46 2017
From: panand at redhat.com (Pratyush Anand)
Date: Thu, 8 Jun 2017 12:39:46 +0530
Subject: [makedumpfile PATCHv2 2/2] print_info: show the remaining time of
 dump progress
In-Reply-To: <1496822990-17848-2-git-send-email-piliu@redhat.com>
References: <1496822990-17848-1-git-send-email-piliu@redhat.com>
 <1496822990-17848-2-git-send-email-piliu@redhat.com>
Message-ID: <817b653f-9191-70c2-21b9-6fba9a196399@redhat.com>



On Wednesday 07 June 2017 01:39 PM, Pingfan Liu wrote:
> If the dump progress is slow, it is better to show  an estimate of
> remaining time of the progress to the user. The estimator simply uses
> the average speed to work.

Nice improvement.

It could have been better to write about how does log looks before and after 
this patch.

>
> Signed-off-by: Pingfan Liu <piliu at redhat.com>
> ---
> v1 -> v2: fold  eta string into eta_to_human_short()
>
>
>  makedumpfile.c | 46 +++++++++++++++++++++++-----------------------
>  print_info.c   | 41 +++++++++++++++++++++++++++++++++++------
>  print_info.h   |  3 ++-
>  3 files changed, 60 insertions(+), 30 deletions(-)
>
> diff --git a/makedumpfile.c b/makedumpfile.c
> index 301772a..35a36ce 100644
> --- a/makedumpfile.c
> +++ b/makedumpfile.c
> @@ -5175,7 +5175,7 @@ _exclude_free_page(struct cycle *cycle)
>
>  	for (num_nodes = 1; num_nodes <= vt.numnodes; num_nodes++) {
>
> -		print_progress(PROGRESS_FREE_PAGES, num_nodes - 1, vt.numnodes);
> +		print_progress(PROGRESS_FREE_PAGES, num_nodes - 1, vt.numnodes, NULL);
>
>  		node_zones = pgdat + OFFSET(pglist_data.node_zones);
>
> @@ -5188,7 +5188,7 @@ _exclude_free_page(struct cycle *cycle)
>  		for (i = 0; i < nr_zones; i++) {
>
>  			print_progress(PROGRESS_FREE_PAGES, i + nr_zones * (num_nodes - 1),
> -					nr_zones * vt.numnodes);
> +					nr_zones * vt.numnodes, NULL);
>
>  			zone = node_zones + (i * SIZE(zone));
>  			if (!readmem(VADDR, zone + OFFSET(zone.spanned_pages),
> @@ -5216,7 +5216,7 @@ _exclude_free_page(struct cycle *cycle)
>  	/*
>  	 * print [100 %]
>  	 */
> -	print_progress(PROGRESS_FREE_PAGES, vt.numnodes, vt.numnodes);
> +	print_progress(PROGRESS_FREE_PAGES, vt.numnodes, vt.numnodes, NULL);
>  	print_execution_time(PROGRESS_FREE_PAGES, &tv_start);
>
>  	return TRUE;
> @@ -5403,7 +5403,7 @@ create_1st_bitmap_file(void)
>  	for (i = 0; get_pt_load(i, &phys_start, &phys_end, NULL, NULL); i++) {
>
>  		if (!info->flag_mem_usage)
> -			print_progress(PROGRESS_HOLES, i, num_pt_loads);
> +			print_progress(PROGRESS_HOLES, i, num_pt_loads, NULL);
>
>  		pfn_start = paddr_to_pfn(phys_start);
>  		pfn_end   = paddr_to_pfn(phys_end);
> @@ -5422,7 +5422,7 @@ create_1st_bitmap_file(void)
>  	 * print 100 %
>  	 */
>  	if (!info->flag_mem_usage) {
> -		print_progress(PROGRESS_HOLES, info->max_mapnr, info->max_mapnr);
> +		print_progress(PROGRESS_HOLES, info->max_mapnr, info->max_mapnr, NULL);
>  		print_execution_time(PROGRESS_HOLES, &tv_start);
>  	}
>
> @@ -5548,7 +5548,7 @@ create_bitmap_from_memhole(struct cycle *cycle, struct dump_bitmap *bitmap, int
>  		pfn_start = MAX(paddr_to_pfn(phys_start), cycle->start_pfn);
>  		pfn_end = MIN(paddr_to_pfn(phys_end), cycle->end_pfn);
>
> -		print_progress(PROGRESS_HOLES, i, num_pt_loads);
> +		print_progress(PROGRESS_HOLES, i, num_pt_loads, NULL);
>
>  		if (pfn_start >= pfn_end)
>  			continue;
> @@ -5587,7 +5587,7 @@ create_bitmap_from_memhole(struct cycle *cycle, struct dump_bitmap *bitmap, int
>  	/*
>  	 * print 100 %
>  	 */
> -	print_progress(PROGRESS_HOLES, info->max_mapnr, info->max_mapnr);
> +	print_progress(PROGRESS_HOLES, info->max_mapnr, info->max_mapnr, NULL);
>  	print_execution_time(PROGRESS_HOLES, &tv_start);
>
>  	return TRUE;
> @@ -5842,7 +5842,7 @@ exclude_unnecessary_pages(struct cycle *cycle)
>  	for (mm = 0; mm < info->num_mem_map; mm++) {
>
>  		if (!info->flag_mem_usage)
> -			print_progress(PROGRESS_UNN_PAGES, mm, info->num_mem_map);
> +			print_progress(PROGRESS_UNN_PAGES, mm, info->num_mem_map, NULL);
>
>  		mmd = &info->mem_map_data[mm];
>
> @@ -5860,7 +5860,7 @@ exclude_unnecessary_pages(struct cycle *cycle)
>  	 * print [100 %]
>  	 */
>  	if (!info->flag_mem_usage) {
> -		print_progress(PROGRESS_UNN_PAGES, info->num_mem_map, info->num_mem_map);
> +		print_progress(PROGRESS_UNN_PAGES, info->num_mem_map, info->num_mem_map, NULL);
>  		print_execution_time(PROGRESS_UNN_PAGES, &tv_start);
>  	}
>
> @@ -7334,7 +7334,7 @@ write_elf_pages_cyclic(struct cache_data *cd_header, struct cache_data *cd_page)
>  				}
>
>  				if ((num_dumped % per) == 0)
> -					print_progress(PROGRESS_COPY, num_dumped, num_dumpable);
> +					print_progress(PROGRESS_COPY, num_dumped, num_dumpable, &tv_start);
>
>  				num_dumped++;
>
> @@ -7453,7 +7453,7 @@ write_elf_pages_cyclic(struct cache_data *cd_header, struct cache_data *cd_page)
>  	/*
>  	 * print [100 %]
>  	 */
> -	print_progress(PROGRESS_COPY, num_dumpable, num_dumpable);
> +	print_progress(PROGRESS_COPY, num_dumpable, num_dumpable, &tv_start);
>  	print_execution_time(PROGRESS_COPY, &tv_start);
>  	PROGRESS_MSG("\n");
>
> @@ -7931,7 +7931,7 @@ write_kdump_pages_parallel_cyclic(struct cache_data *cd_header,
>  		}
>
>  		if ((num_dumped % per) == 0)
> -			print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable);
> +			print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable, &tv_start);
>
>  		num_dumped++;
>
> @@ -7967,7 +7967,7 @@ finish:
>  	/*
>  	 * print [100 %]
>  	 */
> -	print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable);
> +	print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable, &tv_start);
>  	print_execution_time(PROGRESS_COPY, &tv_start);
>  	PROGRESS_MSG("\n");
>
> @@ -8084,7 +8084,7 @@ write_kdump_pages_cyclic(struct cache_data *cd_header, struct cache_data *cd_pag
>  	for (pfn = start_pfn; pfn < end_pfn; pfn++) {
>
>  		if ((num_dumped % per) == 0)
> -			print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable);
> +			print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable, &tv_start);
>
>  		/*
>  		 * Check the excluded page.
> @@ -8164,7 +8164,7 @@ out:
>  		free(wrkmem);
>  #endif
>
> -	print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable);
> +	print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable, &tv_start);
>  	print_execution_time(PROGRESS_COPY, &tv_start);
>
>  	return ret;
> @@ -8622,7 +8622,7 @@ write_kdump_pages_and_bitmap_cyclic(struct cache_data *cd_header, struct cache_d
>  	/*
>  	 * print [100 %]
>  	 */
> -	print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable);
> +	print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable, &tv_start);
>  	print_execution_time(PROGRESS_COPY, &tv_start);
>  	PROGRESS_MSG("\n");
>
> @@ -9216,7 +9216,7 @@ exclude_xen3_user_domain(void)
>  	 */
>  	for (i = 0; get_pt_load(i, &phys_start, &phys_end, NULL, NULL); i++) {
>
> -		print_progress(PROGRESS_XEN_DOMAIN, i, num_pt_loads);
> +		print_progress(PROGRESS_XEN_DOMAIN, i, num_pt_loads, NULL);
>
>  		pfn     = paddr_to_pfn(phys_start);
>  		pfn_end = paddr_to_pfn(phys_end);
> @@ -9224,7 +9224,7 @@ exclude_xen3_user_domain(void)
>
>  		for (j = 0; pfn < pfn_end; pfn++, j++) {
>  			print_progress(PROGRESS_XEN_DOMAIN, j + (size * i),
> -					size * num_pt_loads);
> +					size * num_pt_loads, NULL);
>
>  			if (!allocated_in_map(pfn)) {
>  				clear_bit_on_2nd_bitmap(pfn, NULL);
> @@ -9280,7 +9280,7 @@ exclude_xen4_user_domain(void)
>  	 */
>  	for (i = 0; get_pt_load(i, &phys_start, &phys_end, NULL, NULL); i++) {
>
> -		print_progress(PROGRESS_XEN_DOMAIN, i, num_pt_loads);
> +		print_progress(PROGRESS_XEN_DOMAIN, i, num_pt_loads, NULL);
>
>  		pfn     = paddr_to_pfn(phys_start);
>  		pfn_end = paddr_to_pfn(phys_end);
> @@ -9288,7 +9288,7 @@ exclude_xen4_user_domain(void)
>
>  		for (j = 0; pfn < pfn_end; pfn++, j++) {
>  			print_progress(PROGRESS_XEN_DOMAIN, j + (size * i),
> -					size * num_pt_loads);
> +					size * num_pt_loads, NULL);
>
>  			page_info_addr = info->frame_table_vaddr + pfn * SIZE(page_info);
>  			if (!readmem(VADDR_XEN,
> @@ -9356,7 +9356,7 @@ exclude_xen_user_domain(void)
>  	/*
>  	 * print [100 %]
>  	 */
> -	print_progress(PROGRESS_XEN_DOMAIN, 1, 1);
> +	print_progress(PROGRESS_XEN_DOMAIN, 1, 1, NULL);
>  	print_execution_time(PROGRESS_XEN_DOMAIN, &tv_start);
>
>  	return ret;
> @@ -10342,7 +10342,7 @@ reassemble_kdump_pages(void)
>
>  			num_dumped++;
>
> -			print_progress(PROGRESS_COPY, num_dumped, num_dumpable);
> +			print_progress(PROGRESS_COPY, num_dumped, num_dumpable, &tv_start);
>
>  			if (lseek(fd, offset_ph_org, SEEK_SET) < 0) {
>  				ERRMSG("Can't seek a file(%s). %s\n",
> @@ -10439,7 +10439,7 @@ reassemble_kdump_pages(void)
>  						    size_eraseinfo))
>  			goto out;
>  	}
> -	print_progress(PROGRESS_COPY, num_dumpable, num_dumpable);
> +	print_progress(PROGRESS_COPY, num_dumpable, num_dumpable, &tv_start);
>  	print_execution_time(PROGRESS_COPY, &tv_start);
>
>  	ret = TRUE;
> diff --git a/print_info.c b/print_info.c
> index 79dfe5e..4e0bb5c 100644
> --- a/print_info.c
> +++ b/print_info.c
> @@ -15,8 +15,9 @@
>   */
>  #include "print_info.h"
>  #include <time.h>
> +#include <string.h>
>
> -#define PROGRESS_MAXLEN		"35"
> +#define PROGRESS_MAXLEN		"50"
>
>  int message_level;
>  int flag_strerr_message;
> @@ -350,14 +351,36 @@ static void calc_delta(struct timeval *tv_start, struct timeval *delta)
>  	}
>  }
>
> +/* produce less than 12 bytes on msg */
> +static int eta_to_human_short (int secs, char* msg)
> +{
> +	strcpy(msg, "eta: ");
> +	msg += strlen("eta: ");
> +	if (secs < 100)
> +		sprintf(msg, "%ds", secs);
> +	else if (secs < 100 * 60)
> +		sprintf(msg, "%dm%ds", secs / 60, secs % 60);
> +	else if (secs < 48 * 3600)
> +		sprintf(msg, "%dh%dm", secs / 3600, (secs / 60) % 60);
> +	else if (secs < 100 * 86400)
> +		sprintf(msg, "%dd%dh", secs / 86400, (secs / 3600) % 24);
> +	else
> +		sprintf(msg, ">2day");
> +	return 0;
> +}
> +
> +
>  void
> -print_progress(const char *msg, unsigned long current, unsigned long end)
> +print_progress(const char *msg, unsigned long current, unsigned long end, struct timeval *start)
>  {
>  	float progress;
>  	time_t tm;
>  	static time_t last_time = 0;
>  	static unsigned int lapse = 0;
>  	static const char *spinner = "/|\\-";
> +	struct timeval delta;
> +	double eta;
> +	char eta_msg[16] = " ";
>
>  	if (current < end) {
>  		tm = time(NULL);
> @@ -368,13 +391,19 @@ print_progress(const char *msg, unsigned long current, unsigned long end)
>  	} else
>  		progress = 100;
>
> +	if (start != NULL) {
> +		calc_delta(start, &delta);
> +		eta = delta.tv_sec + delta.tv_usec/1e6;

space around /
and better to write hex value as 0x1e6

Sorry, I could not understand what is this magic number 1e6?


> +		eta = (100 - progress) * eta/progress;

space around /

> +		eta_to_human_short(eta, eta_msg);
> +	}
>  	if (flag_ignore_r_char) {
> -		PROGRESS_MSG("%-" PROGRESS_MAXLEN "s: [%5.1f %%] %c\n",
> -			     msg, progress, spinner[lapse % 4]);
> +		PROGRESS_MSG("%-" PROGRESS_MAXLEN "s: [%5.1f %%] %c  %16s\n",
> +			     msg, progress, spinner[lapse % 4], eta_msg);
>  	} else {
>  		PROGRESS_MSG("\r");
> -		PROGRESS_MSG("%-" PROGRESS_MAXLEN "s: [%5.1f %%] %c",
> -			     msg, progress, spinner[lapse % 4]);
> +		PROGRESS_MSG("%-" PROGRESS_MAXLEN "s: [%5.1f %%] %c  %16s",
> +			     msg, progress, spinner[lapse % 4], eta_msg);
>  	}
>  	lapse++;
>  }
> diff --git a/print_info.h b/print_info.h
> index 01e3706..1ce3593 100644
> --- a/print_info.h
> +++ b/print_info.h
> @@ -25,7 +25,8 @@ extern int flag_ignore_r_char;
>
>  void show_version(void);
>  void print_usage(void);
> -void print_progress(const char *msg, unsigned long current, unsigned long end);
> +void print_progress(const char *msg, unsigned long current, unsigned long end, struct timeval *start);
> +
>  void print_execution_time(char *step_name, struct timeval *tv_start);
>
>  /*
>

~Pratyush


From hch at infradead.org  Thu Jun  8 00:58:29 2017
From: hch at infradead.org (Christoph Hellwig)
Date: Thu, 8 Jun 2017 00:58:29 -0700
Subject: [PATCH v6 25/34] swiotlb: Add warnings for use of bounce buffers
 with SME
In-Reply-To: <20170607191732.28645.42876.stgit@tlendack-t1.amdoffice.net>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
 <20170607191732.28645.42876.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170608075829.GA2446@infradead.org>

On Wed, Jun 07, 2017 at 02:17:32PM -0500, Tom Lendacky wrote:
> Add warnings to let the user know when bounce buffers are being used for
> DMA when SME is active.  Since the bounce buffers are not in encrypted
> memory, these notifications are to allow the user to determine some
> appropriate action - if necessary.

And what would the action be?  Do we need a boot or other option to
disallow this fallback for people who care deeply?


From ats-kumagai at wm.jp.nec.com  Thu Jun  8 00:59:21 2017
From: ats-kumagai at wm.jp.nec.com (Atsushi Kumagai)
Date: Thu, 8 Jun 2017 07:59:21 +0000
Subject: [Makedumpfile PATCH v4 1/2] makedumpfile: add runtime kaslr
 offset if it exists
In-Reply-To: <f603a814-d112-fa67-ac89-0b9173b4ed7c@redhat.com>
References: <cover.1495767738.git.panand@redhat.com>
 <302c16ede3723264ca0abb5eef9c9f553ffcfbd7.1495767738.git.panand@redhat.com>
 <f603a814-d112-fa67-ac89-0b9173b4ed7c@redhat.com>
Message-ID: <0910DD04CBD6DE4193FCF86B9C00BE9701EDF05E@BPXM01GP.gisp.nec.co.jp>

>Hi Atsushi,
>
>On Friday 26 May 2017 08:36 AM, Pratyush Anand wrote:
>> +#define get_kaslr_offset(X)	FALSE
>>  #define get_xen_basic_info_arch(X) get_xen_basic_info_arm64(X)
>>  #define get_xen_info_arch(X) get_xen_info_arm64(X)
>>  #define is_phys_addr(X)		stub_true_ul(X)
>> @@ -851,6 +857,7 @@ unsigned long long vaddr_to_paddr_arm(unsigned long vaddr);
>>  #define get_phys_base()		get_phys_base_arm()
>>  #define get_machdep_info()	get_machdep_info_arm()
>>  #define get_versiondep_info()	stub_true()
>> +#define get_kaslr_offset(X)	FALSE
>
>
>It seems that using stub_false() would be better than FALSE.
>
>I see, its not yet in upstream/devel branch.
>So, Either you can edit them before applying or may be I can send v5.
>
>Please let me know.

I'll fix it and push it to the devel branch.
Thanks for your pointing out.

Regards,
Atsushi Kumagai



From panand at redhat.com  Thu Jun  8 01:10:36 2017
From: panand at redhat.com (Pratyush Anand)
Date: Thu, 8 Jun 2017 13:40:36 +0530
Subject: [Makedumpfile PATCH v4 1/2] makedumpfile: add runtime kaslr
 offset if it exists
In-Reply-To: <0910DD04CBD6DE4193FCF86B9C00BE9701EDF05E@BPXM01GP.gisp.nec.co.jp>
References: <cover.1495767738.git.panand@redhat.com>
 <302c16ede3723264ca0abb5eef9c9f553ffcfbd7.1495767738.git.panand@redhat.com>
 <f603a814-d112-fa67-ac89-0b9173b4ed7c@redhat.com>
 <0910DD04CBD6DE4193FCF86B9C00BE9701EDF05E@BPXM01GP.gisp.nec.co.jp>
Message-ID: <ea6e024e-c5ce-739a-7b4b-aa2e0af7fbc2@redhat.com>



On Thursday 08 June 2017 01:29 PM, Atsushi Kumagai wrote:
>> Hi Atsushi,
>>
>> On Friday 26 May 2017 08:36 AM, Pratyush Anand wrote:
>>> +#define get_kaslr_offset(X)	FALSE
>>>  #define get_xen_basic_info_arch(X) get_xen_basic_info_arm64(X)
>>>  #define get_xen_info_arch(X) get_xen_info_arm64(X)
>>>  #define is_phys_addr(X)		stub_true_ul(X)
>>> @@ -851,6 +857,7 @@ unsigned long long vaddr_to_paddr_arm(unsigned long vaddr);
>>>  #define get_phys_base()		get_phys_base_arm()
>>>  #define get_machdep_info()	get_machdep_info_arm()
>>>  #define get_versiondep_info()	stub_true()
>>> +#define get_kaslr_offset(X)	FALSE
>>
>>
>> It seems that using stub_false() would be better than FALSE.
>>
>> I see, its not yet in upstream/devel branch.
>> So, Either you can edit them before applying or may be I can send v5.
>>
>> Please let me know.
>
> I'll fix it and push it to the devel branch.

Thanks :-)

> Thanks for your pointing out.


~Pratyush


From lkp at intel.com  Wed Jun  7 21:24:39 2017
From: lkp at intel.com (kbuild test robot)
Date: Thu, 8 Jun 2017 12:24:39 +0800
Subject: [PATCH v6 19/34] x86/mm: Add support to access boot related data
 in the clear
In-Reply-To: <20170607191636.28645.98914.stgit@tlendack-t1.amdoffice.net>
Message-ID: <201706081213.vVLC8kRm%fengguang.wu@intel.com>

Hi Tom,

[auto build test ERROR on linus/master]
[also build test ERROR on v4.12-rc4 next-20170607]
[cannot apply to tip/x86/core]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]

url:    https://github.com/0day-ci/linux/commits/Tom-Lendacky/x86-Secure-Memory-Encryption-AMD/20170608-104147
config: i386-randconfig-x077-06040719 (attached as .config)
compiler: gcc-6 (Debian 6.2.0-3) 6.2.0 20160901
reproduce:
        # save the attached .config to linux build tree
        make ARCH=i386 

All error/warnings (new ones prefixed by >>):

   In file included from arch/x86/include/asm/dma.h:12:0,
                    from include/linux/bootmem.h:9,
                    from arch/x86/mm/ioremap.c:9:
>> arch/x86/include/asm/io.h:386:37: error: conflicting types for 'arch_memremap_can_ram_remap'
    #define arch_memremap_can_ram_remap arch_memremap_can_ram_remap
                                        ^
>> arch/x86/mm/ioremap.c:561:6: note: in expansion of macro 'arch_memremap_can_ram_remap'
    bool arch_memremap_can_ram_remap(resource_size_t phys_addr, unsigned long size,
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~
   arch/x86/include/asm/io.h:384:13: note: previous declaration of 'arch_memremap_can_ram_remap' was here
    extern bool arch_memremap_can_ram_remap(resource_size_t offset, size_t size,
                ^~~~~~~~~~~~~~~~~~~~~~~~~~~
--
   In file included from arch/x86/include/asm/dma.h:12:0,
                    from include/linux/bootmem.h:9,
                    from arch/x86//mm/ioremap.c:9:
>> arch/x86/include/asm/io.h:386:37: error: conflicting types for 'arch_memremap_can_ram_remap'
    #define arch_memremap_can_ram_remap arch_memremap_can_ram_remap
                                        ^
   arch/x86//mm/ioremap.c:561:6: note: in expansion of macro 'arch_memremap_can_ram_remap'
    bool arch_memremap_can_ram_remap(resource_size_t phys_addr, unsigned long size,
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~
   arch/x86/include/asm/io.h:384:13: note: previous declaration of 'arch_memremap_can_ram_remap' was here
    extern bool arch_memremap_can_ram_remap(resource_size_t offset, size_t size,
                ^~~~~~~~~~~~~~~~~~~~~~~~~~~

vim +/arch_memremap_can_ram_remap +386 arch/x86/include/asm/io.h

   380	extern void arch_io_free_memtype_wc(resource_size_t start, resource_size_t size);
   381	#define arch_io_reserve_memtype_wc arch_io_reserve_memtype_wc
   382	#endif
   383	
   384	extern bool arch_memremap_can_ram_remap(resource_size_t offset, size_t size,
   385						unsigned long flags);
 > 386	#define arch_memremap_can_ram_remap arch_memremap_can_ram_remap
   387	
   388	#endif /* _ASM_X86_IO_H */

---
0-DAY kernel test infrastructure                Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all                   Intel Corporation
-------------- next part --------------
A non-text attachment was scrubbed...
Name: .config.gz
Type: application/gzip
Size: 23705 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/kexec/attachments/20170608/9dcb4c88/attachment-0001.gz>

From lkp at intel.com  Wed Jun  7 22:53:24 2017
From: lkp at intel.com (kbuild test robot)
Date: Thu, 8 Jun 2017 13:53:24 +0800
Subject: [PATCH v6 25/34] swiotlb: Add warnings for use of bounce buffers
 with SME
In-Reply-To: <20170607191732.28645.42876.stgit@tlendack-t1.amdoffice.net>
Message-ID: <201706081348.u0hG73ce%fengguang.wu@intel.com>

Hi Tom,

[auto build test ERROR on linus/master]
[also build test ERROR on v4.12-rc4 next-20170607]
[cannot apply to tip/x86/core]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]

url:    https://github.com/0day-ci/linux/commits/Tom-Lendacky/x86-Secure-Memory-Encryption-AMD/20170608-104147
config: sparc-defconfig (attached as .config)
compiler: sparc-linux-gcc (GCC) 6.2.0
reproduce:
        wget https://raw.githubusercontent.com/01org/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
        chmod +x ~/bin/make.cross
        # save the attached .config to linux build tree
        make.cross ARCH=sparc 

All errors (new ones prefixed by >>):

   In file included from include/linux/dma-mapping.h:13:0,
                    from include/linux/skbuff.h:34,
                    from include/linux/filter.h:12,
                    from kernel//bpf/core.c:24:
>> include/linux/mem_encrypt.h:16:29: fatal error: asm/mem_encrypt.h: No such file or directory
    #include <asm/mem_encrypt.h>
                                ^
   compilation terminated.

vim +16 include/linux/mem_encrypt.h

2d7c2ec4 Tom Lendacky 2017-06-07  10   * published by the Free Software Foundation.
2d7c2ec4 Tom Lendacky 2017-06-07  11   */
2d7c2ec4 Tom Lendacky 2017-06-07  12  
2d7c2ec4 Tom Lendacky 2017-06-07  13  #ifndef __MEM_ENCRYPT_H__
2d7c2ec4 Tom Lendacky 2017-06-07  14  #define __MEM_ENCRYPT_H__
2d7c2ec4 Tom Lendacky 2017-06-07  15  
2d7c2ec4 Tom Lendacky 2017-06-07 @16  #include <asm/mem_encrypt.h>
2d7c2ec4 Tom Lendacky 2017-06-07  17  
2d7c2ec4 Tom Lendacky 2017-06-07  18  #endif	/* __MEM_ENCRYPT_H__ */

:::::: The code at line 16 was first introduced by commit
:::::: 2d7c2ec4c60e83432b27bfb32042706f404d4158 x86/mm: Add Secure Memory Encryption (SME) support

:::::: TO: Tom Lendacky <thomas.lendacky at amd.com>
:::::: CC: 0day robot <fengguang.wu at intel.com>

---
0-DAY kernel test infrastructure                Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all                   Intel Corporation
-------------- next part --------------
A non-text attachment was scrubbed...
Name: .config.gz
Type: application/gzip
Size: 12016 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/kexec/attachments/20170608/4de593a1/attachment.gz>

From lkp at intel.com  Thu Jun  8 00:39:03 2017
From: lkp at intel.com (kbuild test robot)
Date: Thu, 8 Jun 2017 15:39:03 +0800
Subject: [PATCH v6 10/34] x86, x86/mm, x86/xen, olpc: Use __va() against
 just the physical address in cr3
In-Reply-To: <20170607191453.28645.92256.stgit@tlendack-t1.amdoffice.net>
Message-ID: <201706081509.rlvRTFuv%fengguang.wu@intel.com>

Hi Tom,

[auto build test ERROR on linus/master]
[also build test ERROR on v4.12-rc4 next-20170607]
[cannot apply to tip/x86/core]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]

url:    https://github.com/0day-ci/linux/commits/Tom-Lendacky/x86-Secure-Memory-Encryption-AMD/20170608-104147
config: um-x86_64_defconfig (attached as .config)
compiler: gcc-6 (Debian 6.2.0-3) 6.2.0 20160901
reproduce:
        # save the attached .config to linux build tree
        make ARCH=um SUBARCH=x86_64

All errors (new ones prefixed by >>):

   In file included from arch/x86/include/asm/cacheflush.h:6:0,
                    from include/linux/highmem.h:11,
                    from net/core/sock.c:116:
   arch/x86/include/asm/special_insns.h: In function 'native_read_cr3_pa':
>> arch/x86/include/asm/special_insns.h:239:30: error: 'PHYSICAL_PAGE_MASK' undeclared (first use in this function)
     return (native_read_cr3() & PHYSICAL_PAGE_MASK);
                                 ^~~~~~~~~~~~~~~~~~
   arch/x86/include/asm/special_insns.h:239:30: note: each undeclared identifier is reported only once for each function it appears in
   arch/x86/include/asm/special_insns.h: In function 'read_cr3_pa':
   arch/x86/include/asm/special_insns.h:244:23: error: 'PHYSICAL_PAGE_MASK' undeclared (first use in this function)
     return (read_cr3() & PHYSICAL_PAGE_MASK);
                          ^~~~~~~~~~~~~~~~~~

vim +/PHYSICAL_PAGE_MASK +239 arch/x86/include/asm/special_insns.h

   233	}
   234	
   235	#define nop() asm volatile ("nop")
   236	
   237	static inline unsigned long native_read_cr3_pa(void)
   238	{
 > 239		return (native_read_cr3() & PHYSICAL_PAGE_MASK);
   240	}
   241	
   242	static inline unsigned long read_cr3_pa(void)

---
0-DAY kernel test infrastructure                Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all                   Intel Corporation
-------------- next part --------------
A non-text attachment was scrubbed...
Name: .config.gz
Type: application/gzip
Size: 7762 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/kexec/attachments/20170608/91c01990/attachment-0001.gz>

From thomas.lendacky at amd.com  Thu Jun  8 06:42:51 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Thu, 8 Jun 2017 08:42:51 -0500
Subject: [PATCH v6 10/34] x86, x86/mm, x86/xen, olpc: Use __va() against
 just the physical address in cr3
In-Reply-To: <b15e8924-4069-b5fa-adb2-86c164b1dd36@oracle.com>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
 <20170607191453.28645.92256.stgit@tlendack-t1.amdoffice.net>
 <b15e8924-4069-b5fa-adb2-86c164b1dd36@oracle.com>
Message-ID: <4a7376fb-abfc-8edd-42b7-38de461ac65e@amd.com>

On 6/7/2017 5:06 PM, Boris Ostrovsky wrote:
> On 06/07/2017 03:14 PM, Tom Lendacky wrote:
>> The cr3 register entry can contain the SME encryption bit that indicates
>> the PGD is encrypted.  The encryption bit should not be used when creating
>> a virtual address for the PGD table.
>>
>> Create a new function, read_cr3_pa(), that will extract the physical
>> address from the cr3 register. This function is then used where a virtual
>> address of the PGD needs to be created/used from the cr3 register.
>>
>> Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
>> ---
>>   arch/x86/include/asm/special_insns.h |    9 +++++++++
>>   arch/x86/kernel/head64.c             |    2 +-
>>   arch/x86/mm/fault.c                  |   10 +++++-----
>>   arch/x86/mm/ioremap.c                |    2 +-
>>   arch/x86/platform/olpc/olpc-xo1-pm.c |    2 +-
>>   arch/x86/power/hibernate_64.c        |    2 +-
>>   arch/x86/xen/mmu_pv.c                |    6 +++---
>>   7 files changed, 21 insertions(+), 12 deletions(-)
>>

...

>> diff --git a/arch/x86/xen/mmu_pv.c b/arch/x86/xen/mmu_pv.c
>> index 1f386d7..2dc5243 100644
>> --- a/arch/x86/xen/mmu_pv.c
>> +++ b/arch/x86/xen/mmu_pv.c
>> @@ -2022,7 +2022,7 @@ static phys_addr_t __init xen_early_virt_to_phys(unsigned long vaddr)
>>   	pmd_t pmd;
>>   	pte_t pte;
>>   
>> -	pa = read_cr3();
>> +	pa = read_cr3_pa();
>>   	pgd = native_make_pgd(xen_read_phys_ulong(pa + pgd_index(vaddr) *
>>   						       sizeof(pgd)));
>>   	if (!pgd_present(pgd))
>> @@ -2102,7 +2102,7 @@ void __init xen_relocate_p2m(void)
>>   	pt_phys = pmd_phys + PFN_PHYS(n_pmd);
>>   	p2m_pfn = PFN_DOWN(pt_phys) + n_pt;
>>   
>> -	pgd = __va(read_cr3());
>> +	pgd = __va(read_cr3_pa());
>>   	new_p2m = (unsigned long *)(2 * PGDIR_SIZE);
>>   	idx_p4d = 0;
>>   	save_pud = n_pud;
>> @@ -2209,7 +2209,7 @@ static void __init xen_write_cr3_init(unsigned long cr3)
>>   {
>>   	unsigned long pfn = PFN_DOWN(__pa(swapper_pg_dir));
>>   
>> -	BUG_ON(read_cr3() != __pa(initial_page_table));
>> +	BUG_ON(read_cr3_pa() != __pa(initial_page_table));
>>   	BUG_ON(cr3 != __pa(swapper_pg_dir));
>>   
>>   	/*
> 
> 
> (Please copy Xen maintainers when modifying xen-related files.)

Sorry about that, missed adding the Xen maintainers when I added this
change.

> 
> Given that page tables for Xen PV guests are controlled by the
> hypervisor I don't think this change (although harmless) is necessary.

I can back this change out if the Xen maintainers think that's best.

> What may be needed is making sure X86_FEATURE_SME is not set for PV guests.

And that may be something that Xen will need to control through either
CPUID or MSR support for the PV guests.

Thanks,
Tom

> 
> -boris
> 


From thomas.lendacky at amd.com  Thu Jun  8 07:26:38 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Thu, 8 Jun 2017 09:26:38 -0500
Subject: [PATCH v6 26/34] iommu/amd: Allow the AMD IOMMU to work with
 memory encryption
In-Reply-To: <CAOcCaLZ5QNx+CdnLn4eHtYOJOezJAsmq2whf8mggOdeMQDWOhw@mail.gmail.com>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
 <20170607191745.28645.81756.stgit@tlendack-t1.amdoffice.net>
 <CAOcCaLZ5QNx+CdnLn4eHtYOJOezJAsmq2whf8mggOdeMQDWOhw@mail.gmail.com>
Message-ID: <9ca73103-f202-b5b0-b449-79a4b8a8fd72@amd.com>

On 6/7/2017 9:38 PM, Nick Sarnie wrote:
> On Wed, Jun 7, 2017 at 3:17 PM, Tom Lendacky <thomas.lendacky at amd.com> wrote:
>> The IOMMU is programmed with physical addresses for the various tables
>> and buffers that are used to communicate between the device and the
>> driver. When the driver allocates this memory it is encrypted. In order
>> for the IOMMU to access the memory as encrypted the encryption mask needs
>> to be included in these physical addresses during configuration.
>>
>> The PTE entries created by the IOMMU should also include the encryption
>> mask so that when the device behind the IOMMU performs a DMA, the DMA
>> will be performed to encrypted memory.
>>
>> Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
>> ---
>>   arch/x86/include/asm/mem_encrypt.h |    7 +++++++
>>   arch/x86/mm/mem_encrypt.c          |   30 ++++++++++++++++++++++++++++++
>>   drivers/iommu/amd_iommu.c          |   36 +++++++++++++++++++-----------------
>>   drivers/iommu/amd_iommu_init.c     |   18 ++++++++++++------
>>   drivers/iommu/amd_iommu_proto.h    |   10 ++++++++++
>>   drivers/iommu/amd_iommu_types.h    |    2 +-
>>   include/asm-generic/mem_encrypt.h  |    5 +++++
>>   7 files changed, 84 insertions(+), 24 deletions(-)
>>
>> diff --git a/arch/x86/include/asm/mem_encrypt.h b/arch/x86/include/asm/mem_encrypt.h
>> index c7a2525..d86e544 100644
>> --- a/arch/x86/include/asm/mem_encrypt.h
>> +++ b/arch/x86/include/asm/mem_encrypt.h
>> @@ -31,6 +31,8 @@ void __init sme_early_decrypt(resource_size_t paddr,
>>
>>   void __init sme_early_init(void);
>>
>> +bool sme_iommu_supported(void);
>> +
>>   /* Architecture __weak replacement functions */
>>   void __init mem_encrypt_init(void);
>>
>> @@ -62,6 +64,11 @@ static inline void __init sme_early_init(void)
>>   {
>>   }
>>
>> +static inline bool sme_iommu_supported(void)
>> +{
>> +       return true;
>> +}
>> +
>>   #endif /* CONFIG_AMD_MEM_ENCRYPT */
>>
>>   static inline bool sme_active(void)
>> diff --git a/arch/x86/mm/mem_encrypt.c b/arch/x86/mm/mem_encrypt.c
>> index 5d7c51d..018b58a 100644
>> --- a/arch/x86/mm/mem_encrypt.c
>> +++ b/arch/x86/mm/mem_encrypt.c
>> @@ -197,6 +197,36 @@ void __init sme_early_init(void)
>>                  protection_map[i] = pgprot_encrypted(protection_map[i]);
>>   }
>>
>> +bool sme_iommu_supported(void)
>> +{
>> +       struct cpuinfo_x86 *c = &boot_cpu_data;
>> +
>> +       if (!sme_me_mask || (c->x86 != 0x17))
>> +               return true;
>> +
>> +       /* For Fam17h, a specific level of support is required */
>> +       switch (c->microcode & 0xf000) {
>> +       case 0x0000:
>> +               return false;
>> +       case 0x1000:
>> +               switch (c->microcode & 0x0f00) {
>> +               case 0x0000:
>> +                       return false;
>> +               case 0x0100:
>> +                       if ((c->microcode & 0xff) < 0x26)
>> +                               return false;
>> +                       break;
>> +               case 0x0200:
>> +                       if ((c->microcode & 0xff) < 0x05)
>> +                               return false;
>> +                       break;
>> +               }
>> +               break;
>> +       }
>> +
>> +       return true;
>> +}
>> +
>>   /* Architecture __weak replacement functions */
>>   void __init mem_encrypt_init(void)
>>   {
> 

...

> 
> Hi Tom,
> 
> This sounds like a cool feature. I'm trying to test it on my Ryzen
> system, but c->microcode & 0xf000 is evaluating as 0, so IOMMU is not
> being enabled on my system. I'm using the latest microcode for AGESA
> 1.0.0.6, 0x08001126. Is this work reliant on a future microcode
> update, or is there some other issue?

This is my mistake. I moved the check and didn't re-test. At this point
the c->microcode field hasn't been filled in so I'll need to read
MSR_AMD64_PATCH_LEVEL directly in the sme_iommu_supported() function.

Thanks,
Tom

> 
> Thanks,
> Sarnex
> 


From thomas.lendacky at amd.com  Thu Jun  8 09:14:38 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Thu, 8 Jun 2017 11:14:38 -0500
Subject: [PATCH v6 00/34] x86: Secure Memory Encryption (AMD)
In-Reply-To: <CAOcCaLYWoOu0c-Fkee-=wegNqkzUp9pLFLmaFrXuhiXRnUZ3Xw@mail.gmail.com>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
 <CAOcCaLYWoOu0c-Fkee-=wegNqkzUp9pLFLmaFrXuhiXRnUZ3Xw@mail.gmail.com>
Message-ID: <2dd424fe-0af1-d82f-b608-271ea5e1f62b@amd.com>

On 6/7/2017 9:40 PM, Nick Sarnie wrote:
> On Wed, Jun 7, 2017 at 3:13 PM, Tom Lendacky <thomas.lendacky at amd.com> wrote:
>> This patch series provides support for AMD's new Secure Memory Encryption (SME)
>> feature.
>>
>> SME can be used to mark individual pages of memory as encrypted through the
>> page tables. A page of memory that is marked encrypted will be automatically
>> decrypted when read from DRAM and will be automatically encrypted when
>> written to DRAM. Details on SME can found in the links below.
>>
>> The SME feature is identified through a CPUID function and enabled through
>> the SYSCFG MSR. Once enabled, page table entries will determine how the
>> memory is accessed. If a page table entry has the memory encryption mask set,
>> then that memory will be accessed as encrypted memory. The memory encryption
>> mask (as well as other related information) is determined from settings
>> returned through the same CPUID function that identifies the presence of the
>> feature.
>>
>> The approach that this patch series takes is to encrypt everything possible
>> starting early in the boot where the kernel is encrypted. Using the page
>> table macros the encryption mask can be incorporated into all page table
>> entries and page allocations. By updating the protection map, userspace
>> allocations are also marked encrypted. Certain data must be accounted for
>> as having been placed in memory before SME was enabled (EFI, initrd, etc.)
>> and accessed accordingly.
>>
>> This patch series is a pre-cursor to another AMD processor feature called
>> Secure Encrypted Virtualization (SEV). The support for SEV will build upon
>> the SME support and will be submitted later. Details on SEV can be found
>> in the links below.
>>
>> The following links provide additional detail:
>>
>> AMD Memory Encryption whitepaper:
>>     http://amd-dev.wpengine.netdna-cdn.com/wordpress/media/2013/12/AMD_Memory_Encryption_Whitepaper_v7-Public.pdf
>>
>> AMD64 Architecture Programmer's Manual:
>>     http://support.amd.com/TechDocs/24593.pdf
>>     SME is section 7.10
>>     SEV is section 15.34
>>
>> ---
>>

...

> 
> 
> Hi Tom,
> 
> Thanks for your work on this. This may be a stupid question, but is
> using bounce buffers for the GPU(s) expected to reduce performance in
> any/a noticeable way? I'm hitting another issue which I've already
> sent mail about so I can't test it for myself at the moment,

That all depends on the workload, how much DMA is being performed, etc.
But it is extra overhead to use bounce buffers.

Thanks,
Tom

> 
> Thanks,
> Sarnex
> 


From boris.ostrovsky at oracle.com  Thu Jun  8 13:51:04 2017
From: boris.ostrovsky at oracle.com (Boris Ostrovsky)
Date: Thu, 8 Jun 2017 16:51:04 -0400
Subject: [PATCH v6 10/34] x86, x86/mm, x86/xen, olpc: Use __va() against
 just the physical address in cr3
In-Reply-To: <4a7376fb-abfc-8edd-42b7-38de461ac65e@amd.com>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
 <20170607191453.28645.92256.stgit@tlendack-t1.amdoffice.net>
 <b15e8924-4069-b5fa-adb2-86c164b1dd36@oracle.com>
 <4a7376fb-abfc-8edd-42b7-38de461ac65e@amd.com>
Message-ID: <67fe69ac-a213-8de3-db28-0e54bba95127@oracle.com>


>
>> What may be needed is making sure X86_FEATURE_SME is not set for PV
>> guests.
>
> And that may be something that Xen will need to control through either
> CPUID or MSR support for the PV guests.


Only on newer versions of Xen. On earlier versions (2-3 years old) leaf
0x80000007 is passed to the guest unchanged. And so is MSR_K8_SYSCFG.

-boris


From thomas.lendacky at amd.com  Thu Jun  8 14:02:37 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Thu, 8 Jun 2017 16:02:37 -0500
Subject: [PATCH v6 10/34] x86, x86/mm, x86/xen, olpc: Use __va() against
 just the physical address in cr3
In-Reply-To: <67fe69ac-a213-8de3-db28-0e54bba95127@oracle.com>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
 <20170607191453.28645.92256.stgit@tlendack-t1.amdoffice.net>
 <b15e8924-4069-b5fa-adb2-86c164b1dd36@oracle.com>
 <4a7376fb-abfc-8edd-42b7-38de461ac65e@amd.com>
 <67fe69ac-a213-8de3-db28-0e54bba95127@oracle.com>
Message-ID: <fcb196c8-f1eb-a38c-336c-7bd3929b029e@amd.com>

On 6/8/2017 3:51 PM, Boris Ostrovsky wrote:
> 
>>
>>> What may be needed is making sure X86_FEATURE_SME is not set for PV
>>> guests.
>>
>> And that may be something that Xen will need to control through either
>> CPUID or MSR support for the PV guests.
> 
> 
> Only on newer versions of Xen. On earlier versions (2-3 years old) leaf
> 0x80000007 is passed to the guest unchanged. And so is MSR_K8_SYSCFG.

The SME feature is in leaf 0x8000001f, is that leaf passed to the guest
unchanged?

Thanks,
Tom

> 
> -boris
> 


From thomas.lendacky at amd.com  Thu Jun  8 14:09:36 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Thu, 8 Jun 2017 16:09:36 -0500
Subject: [PATCH v6 25/34] swiotlb: Add warnings for use of bounce buffers
 with SME
In-Reply-To: <201706081348.u0hG73ce%fengguang.wu@intel.com>
References: <201706081348.u0hG73ce%fengguang.wu@intel.com>
Message-ID: <56d5d7bf-45f9-6d66-d71a-166ecb60296c@amd.com>

On 6/8/2017 12:53 AM, kbuild test robot wrote:
> Hi Tom,
> 
> [auto build test ERROR on linus/master]
> [also build test ERROR on v4.12-rc4 next-20170607]
> [cannot apply to tip/x86/core]
> [if your patch is applied to the wrong git tree, please drop us a note to help improve the system]
> 
> url:    https://github.com/0day-ci/linux/commits/Tom-Lendacky/x86-Secure-Memory-Encryption-AMD/20170608-104147
> config: sparc-defconfig (attached as .config)
> compiler: sparc-linux-gcc (GCC) 6.2.0
> reproduce:
>          wget https://raw.githubusercontent.com/01org/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
>          chmod +x ~/bin/make.cross
>          # save the attached .config to linux build tree
>          make.cross ARCH=sparc
> 
> All errors (new ones prefixed by >>):
> 
>     In file included from include/linux/dma-mapping.h:13:0,
>                      from include/linux/skbuff.h:34,
>                      from include/linux/filter.h:12,
>                      from kernel//bpf/core.c:24:
>>> include/linux/mem_encrypt.h:16:29: fatal error: asm/mem_encrypt.h: No such file or directory
>      #include <asm/mem_encrypt.h>
>                                  ^
>     compilation terminated.

Okay, I had the wrong understanding of the asm-generic directory. The
next series will fix this so it is not an issue for other arches.

Thanks,
Tom

> 
> vim +16 include/linux/mem_encrypt.h
> 
> 2d7c2ec4 Tom Lendacky 2017-06-07  10   * published by the Free Software Foundation.
> 2d7c2ec4 Tom Lendacky 2017-06-07  11   */
> 2d7c2ec4 Tom Lendacky 2017-06-07  12
> 2d7c2ec4 Tom Lendacky 2017-06-07  13  #ifndef __MEM_ENCRYPT_H__
> 2d7c2ec4 Tom Lendacky 2017-06-07  14  #define __MEM_ENCRYPT_H__
> 2d7c2ec4 Tom Lendacky 2017-06-07  15
> 2d7c2ec4 Tom Lendacky 2017-06-07 @16  #include <asm/mem_encrypt.h>
> 2d7c2ec4 Tom Lendacky 2017-06-07  17
> 2d7c2ec4 Tom Lendacky 2017-06-07  18  #endif	/* __MEM_ENCRYPT_H__ */
> 
> :::::: The code at line 16 was first introduced by commit
> :::::: 2d7c2ec4c60e83432b27bfb32042706f404d4158 x86/mm: Add Secure Memory Encryption (SME) support
> 
> :::::: TO: Tom Lendacky <thomas.lendacky at amd.com>
> :::::: CC: 0day robot <fengguang.wu at intel.com>
> 
> ---
> 0-DAY kernel test infrastructure                Open Source Technology Center
> https://lists.01.org/pipermail/kbuild-all                   Intel Corporation
> 


From boris.ostrovsky at oracle.com  Thu Jun  8 14:17:29 2017
From: boris.ostrovsky at oracle.com (Boris Ostrovsky)
Date: Thu, 8 Jun 2017 17:17:29 -0400
Subject: [PATCH v6 10/34] x86, x86/mm, x86/xen, olpc: Use __va() against
 just the physical address in cr3
In-Reply-To: <fcb196c8-f1eb-a38c-336c-7bd3929b029e@amd.com>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
 <20170607191453.28645.92256.stgit@tlendack-t1.amdoffice.net>
 <b15e8924-4069-b5fa-adb2-86c164b1dd36@oracle.com>
 <4a7376fb-abfc-8edd-42b7-38de461ac65e@amd.com>
 <67fe69ac-a213-8de3-db28-0e54bba95127@oracle.com>
 <fcb196c8-f1eb-a38c-336c-7bd3929b029e@amd.com>
Message-ID: <12c7e511-996d-cf60-3a3b-0be7b41bd85b@oracle.com>

On 06/08/2017 05:02 PM, Tom Lendacky wrote:
> On 6/8/2017 3:51 PM, Boris Ostrovsky wrote:
>>
>>>
>>>> What may be needed is making sure X86_FEATURE_SME is not set for PV
>>>> guests.
>>>
>>> And that may be something that Xen will need to control through either
>>> CPUID or MSR support for the PV guests.
>>
>>
>> Only on newer versions of Xen. On earlier versions (2-3 years old) leaf
>> 0x80000007 is passed to the guest unchanged. And so is MSR_K8_SYSCFG.
>
> The SME feature is in leaf 0x8000001f, is that leaf passed to the guest
> unchanged?

Oh, I misread the patch where X86_FEATURE_SME is defined. Then all
versions, including the current one, pass it unchanged.

All that's needed is setup_clear_cpu_cap(X86_FEATURE_SME) in
xen_init_capabilities().


-boris


From andrew.cooper3 at citrix.com  Thu Jun  8 15:01:37 2017
From: andrew.cooper3 at citrix.com (Andrew Cooper)
Date: Thu, 8 Jun 2017 23:01:37 +0100
Subject: [Xen-devel] [PATCH v6 10/34] x86, x86/mm, x86/xen, olpc: Use
 __va() against just the physical address in cr3
In-Reply-To: <12c7e511-996d-cf60-3a3b-0be7b41bd85b@oracle.com>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
 <20170607191453.28645.92256.stgit@tlendack-t1.amdoffice.net>
 <b15e8924-4069-b5fa-adb2-86c164b1dd36@oracle.com>
 <4a7376fb-abfc-8edd-42b7-38de461ac65e@amd.com>
 <67fe69ac-a213-8de3-db28-0e54bba95127@oracle.com>
 <fcb196c8-f1eb-a38c-336c-7bd3929b029e@amd.com>
 <12c7e511-996d-cf60-3a3b-0be7b41bd85b@oracle.com>
Message-ID: <d37917b1-8e49-e8a8-b9ac-59491331640f@citrix.com>

On 08/06/2017 22:17, Boris Ostrovsky wrote:
> On 06/08/2017 05:02 PM, Tom Lendacky wrote:
>> On 6/8/2017 3:51 PM, Boris Ostrovsky wrote:
>>>>> What may be needed is making sure X86_FEATURE_SME is not set for PV
>>>>> guests.
>>>> And that may be something that Xen will need to control through either
>>>> CPUID or MSR support for the PV guests.
>>>
>>> Only on newer versions of Xen. On earlier versions (2-3 years old) leaf
>>> 0x80000007 is passed to the guest unchanged. And so is MSR_K8_SYSCFG.
>> The SME feature is in leaf 0x8000001f, is that leaf passed to the guest
>> unchanged?
> Oh, I misread the patch where X86_FEATURE_SME is defined. Then all
> versions, including the current one, pass it unchanged.
>
> All that's needed is setup_clear_cpu_cap(X86_FEATURE_SME) in
> xen_init_capabilities().

AMD processors still don't support CPUID Faulting (or at least, I
couldn't find any reference to it in the latest docs), so we cannot
actually hide SME from a guest which goes looking at native CPUID. 
Furthermore, I'm not aware of any CPUID masking support covering that leaf.

However, if Linux is using the paravirtual cpuid hook, things are
slightly better.

On Xen 4.9 and later, no guests will see the feature.  On earlier
versions of Xen (before I fixed the logic), plain domUs will not see the
feature, while dom0 will.

For safely, I'd recommend unilaterally clobbering the feature as Boris
suggested.  There is no way SME will be supportable on a per-PV guest
basis, although (as far as I am aware) Xen as a whole would be able to
encompass itself and all of its PV guests inside one single SME instance.

~Andrew


From thomas.lendacky at amd.com  Thu Jun  8 15:38:18 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Thu, 8 Jun 2017 17:38:18 -0500
Subject: [PATCH v6 10/34] x86, x86/mm, x86/xen, olpc: Use __va() against
 just the physical address in cr3
In-Reply-To: <CALCETrVVhMf=zkiDNn_-hKDZLGXKFiwxuWkPmD5RJgHa5VUMiQ@mail.gmail.com>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
 <20170607191453.28645.92256.stgit@tlendack-t1.amdoffice.net>
 <CALCETrVVhMf=zkiDNn_-hKDZLGXKFiwxuWkPmD5RJgHa5VUMiQ@mail.gmail.com>
Message-ID: <85355c4c-2cc1-daac-d8fe-ac6965b34606@amd.com>

On 6/8/2017 1:05 AM, Andy Lutomirski wrote:
> On Wed, Jun 7, 2017 at 12:14 PM, Tom Lendacky <thomas.lendacky at amd.com> wrote:
>> The cr3 register entry can contain the SME encryption bit that indicates
>> the PGD is encrypted.  The encryption bit should not be used when creating
>> a virtual address for the PGD table.
>>
>> Create a new function, read_cr3_pa(), that will extract the physical
>> address from the cr3 register. This function is then used where a virtual
>> address of the PGD needs to be created/used from the cr3 register.
> 
> This is going to conflict with:
> 
> https://git.kernel.org/pub/scm/linux/kernel/git/luto/linux.git/commit/?h=x86/pcid&id=555c81e5d01a62b629ec426a2f50d27e2127c1df
> 
> We're both encountering the fact that CR3 munges the page table PA
> with some other stuff, and some readers want to see the actual CR3
> value and other readers just want the PA.  The thing I prefer about my
> patch is that I get rid of read_cr3() entirely, forcing the patch to
> update every single reader, making review and conflict resolution much
> safer.
> 
> I'd be willing to send a patch tomorrow that just does the split into
> __read_cr3() and read_cr3_pa() (I like your name better) and then we
> can both base on top of it.  Would that make sense?

That makes sense to me.

> 
> Also:
> 
>> +static inline unsigned long read_cr3_pa(void)
>> +{
>> +       return (read_cr3() & PHYSICAL_PAGE_MASK);
>> +}
> 
> Is there any guarantee that the magic encryption bit is masked out in
> PHYSICAL_PAGE_MASK?  The docs make it sound like it could be any bit.
> (But if it's one of the low 12 bits, that would be quite confusing.)

Right now it's bit 47 and we're steering away from any of the currently
reserved bits so we should be safe.

Thanks,
Tom

> 
> --Andy
> 


From thomas.lendacky at amd.com  Thu Jun  8 16:04:03 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Thu, 8 Jun 2017 18:04:03 -0500
Subject: [PATCH v6 25/34] swiotlb: Add warnings for use of bounce buffers
 with SME
In-Reply-To: <20170608075829.GA2446@infradead.org>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
 <20170607191732.28645.42876.stgit@tlendack-t1.amdoffice.net>
 <20170608075829.GA2446@infradead.org>
Message-ID: <8d78db0c-7c3e-9304-7d04-31095246ec83@amd.com>

On 6/8/2017 2:58 AM, Christoph Hellwig wrote:
> On Wed, Jun 07, 2017 at 02:17:32PM -0500, Tom Lendacky wrote:
>> Add warnings to let the user know when bounce buffers are being used for
>> DMA when SME is active.  Since the bounce buffers are not in encrypted
>> memory, these notifications are to allow the user to determine some
>> appropriate action - if necessary.
> 
> And what would the action be?  Do we need a boot or other option to
> disallow this fallback for people who care deeply?

The action could be to enable the IOMMU so that lower DMA addresses are
used, or to replace the device with one that supports 64-bit DMA or, if
the device is not used much, you could just ignore it.

I'm not sure we need an option to disallow the fallback.  Are you
thinking along the lines of disabling the device?

Thanks,
Tom

> 


From xlpang at redhat.com  Thu Jun  8 19:17:05 2017
From: xlpang at redhat.com (Xunlei Pang)
Date: Fri,  9 Jun 2017 10:17:05 +0800
Subject: [PATCH] s390/crash: Fix KEXEC_NOTE_BYTES definition
Message-ID: <1496974625-10891-1-git-send-email-xlpang@redhat.com>

S390 KEXEC_NOTE_BYTES is not used by note_buf_t as before, which
is now defined as follows:
    typedef u32 note_buf_t[CRASH_CORE_NOTE_BYTES/4];
It was changed by the CONFIG_CRASH_CORE feature.

This patch gets rid of all the old KEXEC_NOTE_BYTES stuff, and
renames KEXEC_NOTE_BYTES to CRASH_CORE_NOTE_BYTES for S390.

Fixes: 692f66f26a4c ("crash: move crashkernel parsing and vmcore related code under CONFIG_CRASH_CORE")
Cc: Dave Young <dyoung at redhat.com>
Cc: Dave Anderson <anderson at redhat.com>
Cc: Hari Bathini <hbathini at linux.vnet.ibm.com>
Cc: Gustavo Luiz Duarte <gustavold at linux.vnet.ibm.com>
Signed-off-by: Xunlei Pang <xlpang at redhat.com>
---
 arch/s390/include/asm/kexec.h |  2 +-
 include/linux/crash_core.h    |  7 +++++++
 include/linux/kexec.h         | 11 +----------
 3 files changed, 9 insertions(+), 11 deletions(-)

diff --git a/arch/s390/include/asm/kexec.h b/arch/s390/include/asm/kexec.h
index 2f924bc..352deb8 100644
--- a/arch/s390/include/asm/kexec.h
+++ b/arch/s390/include/asm/kexec.h
@@ -47,7 +47,7 @@
  * Seven notes plus zero note at the end: prstatus, fpregset, timer,
  * tod_cmp, tod_reg, control regs, and prefix
  */
-#define KEXEC_NOTE_BYTES \
+#define CRASH_CORE_NOTE_BYTES \
 	(ALIGN(sizeof(struct elf_note), 4) * 8 + \
 	 ALIGN(sizeof("CORE"), 4) * 7 + \
 	 ALIGN(sizeof(struct elf_prstatus), 4) + \
diff --git a/include/linux/crash_core.h b/include/linux/crash_core.h
index e9de6b4..dbc6e5c 100644
--- a/include/linux/crash_core.h
+++ b/include/linux/crash_core.h
@@ -10,9 +10,16 @@
 #define CRASH_CORE_NOTE_NAME_BYTES ALIGN(sizeof(CRASH_CORE_NOTE_NAME), 4)
 #define CRASH_CORE_NOTE_DESC_BYTES ALIGN(sizeof(struct elf_prstatus), 4)
 
+/*
+ * The per-cpu notes area is a list of notes terminated by a "NULL"
+ * note header.  For kdump, the code in vmcore.c runs in the context
+ * of the second kernel to combine them into one note.
+ */
+#ifndef CRASH_CORE_NOTE_BYTES
 #define CRASH_CORE_NOTE_BYTES	   ((CRASH_CORE_NOTE_HEAD_BYTES * 2) +	\
 				     CRASH_CORE_NOTE_NAME_BYTES +	\
 				     CRASH_CORE_NOTE_DESC_BYTES)
+#endif
 
 #define VMCOREINFO_BYTES	   PAGE_SIZE
 #define VMCOREINFO_NOTE_NAME	   "VMCOREINFO"
diff --git a/include/linux/kexec.h b/include/linux/kexec.h
index 3ea8275..133df03 100644
--- a/include/linux/kexec.h
+++ b/include/linux/kexec.h
@@ -14,7 +14,6 @@
 
 #if !defined(__ASSEMBLY__)
 
-#include <linux/crash_core.h>
 #include <asm/io.h>
 
 #include <uapi/linux/kexec.h>
@@ -25,6 +24,7 @@
 #include <linux/ioport.h>
 #include <linux/module.h>
 #include <asm/kexec.h>
+#include <linux/crash_core.h>
 
 /* Verify architecture specific macros are defined */
 
@@ -63,15 +63,6 @@
 #define KEXEC_CORE_NOTE_NAME	CRASH_CORE_NOTE_NAME
 
 /*
- * The per-cpu notes area is a list of notes terminated by a "NULL"
- * note header.  For kdump, the code in vmcore.c runs in the context
- * of the second kernel to combine them into one note.
- */
-#ifndef KEXEC_NOTE_BYTES
-#define KEXEC_NOTE_BYTES	CRASH_CORE_NOTE_BYTES
-#endif
-
-/*
  * This structure is used to hold the arguments that are used when loading
  * kernel binaries.
  */
-- 
1.8.3.1



From dyoung at redhat.com  Thu Jun  8 19:29:04 2017
From: dyoung at redhat.com (Dave Young)
Date: Fri, 9 Jun 2017 10:29:04 +0800
Subject: [PATCH] s390/crash: Fix KEXEC_NOTE_BYTES definition
In-Reply-To: <1496974625-10891-1-git-send-email-xlpang@redhat.com>
References: <1496974625-10891-1-git-send-email-xlpang@redhat.com>
Message-ID: <20170609022904.GB3688@dhcp-128-65.nay.redhat.com>

On 06/09/17 at 10:17am, Xunlei Pang wrote:
> S390 KEXEC_NOTE_BYTES is not used by note_buf_t as before, which
> is now defined as follows:
>     typedef u32 note_buf_t[CRASH_CORE_NOTE_BYTES/4];
> It was changed by the CONFIG_CRASH_CORE feature.
> 
> This patch gets rid of all the old KEXEC_NOTE_BYTES stuff, and
> renames KEXEC_NOTE_BYTES to CRASH_CORE_NOTE_BYTES for S390.
> 
> Fixes: 692f66f26a4c ("crash: move crashkernel parsing and vmcore related code under CONFIG_CRASH_CORE")
> Cc: Dave Young <dyoung at redhat.com>
> Cc: Dave Anderson <anderson at redhat.com>
> Cc: Hari Bathini <hbathini at linux.vnet.ibm.com>
> Cc: Gustavo Luiz Duarte <gustavold at linux.vnet.ibm.com>
> Signed-off-by: Xunlei Pang <xlpang at redhat.com>
> ---
>  arch/s390/include/asm/kexec.h |  2 +-
>  include/linux/crash_core.h    |  7 +++++++
>  include/linux/kexec.h         | 11 +----------
>  3 files changed, 9 insertions(+), 11 deletions(-)
> 
> diff --git a/arch/s390/include/asm/kexec.h b/arch/s390/include/asm/kexec.h
> index 2f924bc..352deb8 100644
> --- a/arch/s390/include/asm/kexec.h
> +++ b/arch/s390/include/asm/kexec.h
> @@ -47,7 +47,7 @@
>   * Seven notes plus zero note at the end: prstatus, fpregset, timer,
>   * tod_cmp, tod_reg, control regs, and prefix
>   */
> -#define KEXEC_NOTE_BYTES \
> +#define CRASH_CORE_NOTE_BYTES \
>  	(ALIGN(sizeof(struct elf_note), 4) * 8 + \
>  	 ALIGN(sizeof("CORE"), 4) * 7 + \
>  	 ALIGN(sizeof(struct elf_prstatus), 4) + \
> diff --git a/include/linux/crash_core.h b/include/linux/crash_core.h
> index e9de6b4..dbc6e5c 100644
> --- a/include/linux/crash_core.h
> +++ b/include/linux/crash_core.h
> @@ -10,9 +10,16 @@
>  #define CRASH_CORE_NOTE_NAME_BYTES ALIGN(sizeof(CRASH_CORE_NOTE_NAME), 4)
>  #define CRASH_CORE_NOTE_DESC_BYTES ALIGN(sizeof(struct elf_prstatus), 4)
>  
> +/*
> + * The per-cpu notes area is a list of notes terminated by a "NULL"
> + * note header.  For kdump, the code in vmcore.c runs in the context
> + * of the second kernel to combine them into one note.
> + */
> +#ifndef CRASH_CORE_NOTE_BYTES
>  #define CRASH_CORE_NOTE_BYTES	   ((CRASH_CORE_NOTE_HEAD_BYTES * 2) +	\
>  				     CRASH_CORE_NOTE_NAME_BYTES +	\
>  				     CRASH_CORE_NOTE_DESC_BYTES)
> +#endif
>  
>  #define VMCOREINFO_BYTES	   PAGE_SIZE
>  #define VMCOREINFO_NOTE_NAME	   "VMCOREINFO"
> diff --git a/include/linux/kexec.h b/include/linux/kexec.h
> index 3ea8275..133df03 100644
> --- a/include/linux/kexec.h
> +++ b/include/linux/kexec.h
> @@ -14,7 +14,6 @@
>  
>  #if !defined(__ASSEMBLY__)
>  
> -#include <linux/crash_core.h>
>  #include <asm/io.h>
>  
>  #include <uapi/linux/kexec.h>
> @@ -25,6 +24,7 @@
>  #include <linux/ioport.h>
>  #include <linux/module.h>
>  #include <asm/kexec.h>
> +#include <linux/crash_core.h>
>  
>  /* Verify architecture specific macros are defined */
>  
> @@ -63,15 +63,6 @@
>  #define KEXEC_CORE_NOTE_NAME	CRASH_CORE_NOTE_NAME
>  
>  /*
> - * The per-cpu notes area is a list of notes terminated by a "NULL"
> - * note header.  For kdump, the code in vmcore.c runs in the context
> - * of the second kernel to combine them into one note.
> - */
> -#ifndef KEXEC_NOTE_BYTES
> -#define KEXEC_NOTE_BYTES	CRASH_CORE_NOTE_BYTES
> -#endif

It is still not clear how does s390 use the crash_notes except this macro.
But from code point of view we do need to update this as well after the
crash_core splitting.

Acked-by: Dave Young <dyoung at redhat.com>

Thanks
Dave


From hbathini at linux.vnet.ibm.com  Thu Jun  8 21:12:10 2017
From: hbathini at linux.vnet.ibm.com (Hari Bathini)
Date: Fri, 9 Jun 2017 09:42:10 +0530
Subject: [PATCH] s390/crash: Fix KEXEC_NOTE_BYTES definition
In-Reply-To: <1496974625-10891-1-git-send-email-xlpang@redhat.com>
References: <1496974625-10891-1-git-send-email-xlpang@redhat.com>
Message-ID: <945499f7-3639-d803-adfe-8aa473695488@linux.vnet.ibm.com>



On Friday 09 June 2017 07:47 AM, Xunlei Pang wrote:
> S390 KEXEC_NOTE_BYTES is not used by note_buf_t as before, which
> is now defined as follows:
>      typedef u32 note_buf_t[CRASH_CORE_NOTE_BYTES/4];
> It was changed by the CONFIG_CRASH_CORE feature.
>
> This patch gets rid of all the old KEXEC_NOTE_BYTES stuff, and
> renames KEXEC_NOTE_BYTES to CRASH_CORE_NOTE_BYTES for S390.
>
> Fixes: 692f66f26a4c ("crash: move crashkernel parsing and vmcore related code under CONFIG_CRASH_CORE")
> Cc: Dave Young <dyoung at redhat.com>
> Cc: Dave Anderson <anderson at redhat.com>
> Cc: Hari Bathini <hbathini at linux.vnet.ibm.com>
> Cc: Gustavo Luiz Duarte <gustavold at linux.vnet.ibm.com>
> Signed-off-by: Xunlei Pang <xlpang at redhat.com>
> ---
>   arch/s390/include/asm/kexec.h |  2 +-
>   include/linux/crash_core.h    |  7 +++++++
>   include/linux/kexec.h         | 11 +----------
>   3 files changed, 9 insertions(+), 11 deletions(-)
>
> diff --git a/arch/s390/include/asm/kexec.h b/arch/s390/include/asm/kexec.h
> index 2f924bc..352deb8 100644
> --- a/arch/s390/include/asm/kexec.h
> +++ b/arch/s390/include/asm/kexec.h
> @@ -47,7 +47,7 @@
>    * Seven notes plus zero note at the end: prstatus, fpregset, timer,
>    * tod_cmp, tod_reg, control regs, and prefix
>    */
> -#define KEXEC_NOTE_BYTES \
> +#define CRASH_CORE_NOTE_BYTES \
>   	(ALIGN(sizeof(struct elf_note), 4) * 8 + \
>   	 ALIGN(sizeof("CORE"), 4) * 7 + \
>   	 ALIGN(sizeof(struct elf_prstatus), 4) + \
> diff --git a/include/linux/crash_core.h b/include/linux/crash_core.h
> index e9de6b4..dbc6e5c 100644
> --- a/include/linux/crash_core.h
> +++ b/include/linux/crash_core.h
> @@ -10,9 +10,16 @@
>   #define CRASH_CORE_NOTE_NAME_BYTES ALIGN(sizeof(CRASH_CORE_NOTE_NAME), 4)
>   #define CRASH_CORE_NOTE_DESC_BYTES ALIGN(sizeof(struct elf_prstatus), 4)
>
> +/*
> + * The per-cpu notes area is a list of notes terminated by a "NULL"
> + * note header.  For kdump, the code in vmcore.c runs in the context
> + * of the second kernel to combine them into one note.
> + */
> +#ifndef CRASH_CORE_NOTE_BYTES
>   #define CRASH_CORE_NOTE_BYTES	   ((CRASH_CORE_NOTE_HEAD_BYTES * 2) +	\
>   				     CRASH_CORE_NOTE_NAME_BYTES +	\
>   				     CRASH_CORE_NOTE_DESC_BYTES)
> +#endif
>
>   #define VMCOREINFO_BYTES	   PAGE_SIZE
>   #define VMCOREINFO_NOTE_NAME	   "VMCOREINFO"
> diff --git a/include/linux/kexec.h b/include/linux/kexec.h
> index 3ea8275..133df03 100644
> --- a/include/linux/kexec.h
> +++ b/include/linux/kexec.h
> @@ -14,7 +14,6 @@
>
>   #if !defined(__ASSEMBLY__)
>
> -#include <linux/crash_core.h>
>   #include <asm/io.h>
>
>   #include <uapi/linux/kexec.h>
> @@ -25,6 +24,7 @@
>   #include <linux/ioport.h>
>   #include <linux/module.h>
>   #include <asm/kexec.h>
> +#include <linux/crash_core.h>
>
>   /* Verify architecture specific macros are defined */
>
> @@ -63,15 +63,6 @@
>   #define KEXEC_CORE_NOTE_NAME	CRASH_CORE_NOTE_NAME
>
>   /*
> - * The per-cpu notes area is a list of notes terminated by a "NULL"
> - * note header.  For kdump, the code in vmcore.c runs in the context
> - * of the second kernel to combine them into one note.
> - */
> -#ifndef KEXEC_NOTE_BYTES
> -#define KEXEC_NOTE_BYTES	CRASH_CORE_NOTE_BYTES
> -#endif
> -
> -/*
>    * This structure is used to hold the arguments that are used when loading
>    * kernel binaries.
>    */

Looks good to me. Thanks for the fix..

- Hari



From dyoung at redhat.com  Fri Jun  9 00:45:29 2017
From: dyoung at redhat.com (Dave Young)
Date: Fri, 9 Jun 2017 15:45:29 +0800
Subject: [PATCH] s390/crash: Fix KEXEC_NOTE_BYTES definition
In-Reply-To: <20170609022904.GB3688@dhcp-128-65.nay.redhat.com>
References: <1496974625-10891-1-git-send-email-xlpang@redhat.com>
 <20170609022904.GB3688@dhcp-128-65.nay.redhat.com>
Message-ID: <20170609074529.GA9485@dhcp-128-65.nay.redhat.com>

On 06/09/17 at 10:29am, Dave Young wrote:
> On 06/09/17 at 10:17am, Xunlei Pang wrote:
> > S390 KEXEC_NOTE_BYTES is not used by note_buf_t as before, which
> > is now defined as follows:
> >     typedef u32 note_buf_t[CRASH_CORE_NOTE_BYTES/4];
> > It was changed by the CONFIG_CRASH_CORE feature.
> > 
> > This patch gets rid of all the old KEXEC_NOTE_BYTES stuff, and
> > renames KEXEC_NOTE_BYTES to CRASH_CORE_NOTE_BYTES for S390.
> > 
> > Fixes: 692f66f26a4c ("crash: move crashkernel parsing and vmcore related code under CONFIG_CRASH_CORE")
> > Cc: Dave Young <dyoung at redhat.com>
> > Cc: Dave Anderson <anderson at redhat.com>
> > Cc: Hari Bathini <hbathini at linux.vnet.ibm.com>
> > Cc: Gustavo Luiz Duarte <gustavold at linux.vnet.ibm.com>
> > Signed-off-by: Xunlei Pang <xlpang at redhat.com>
> > ---
> >  arch/s390/include/asm/kexec.h |  2 +-
> >  include/linux/crash_core.h    |  7 +++++++
> >  include/linux/kexec.h         | 11 +----------
> >  3 files changed, 9 insertions(+), 11 deletions(-)
> > 
> > diff --git a/arch/s390/include/asm/kexec.h b/arch/s390/include/asm/kexec.h
> > index 2f924bc..352deb8 100644
> > --- a/arch/s390/include/asm/kexec.h
> > +++ b/arch/s390/include/asm/kexec.h
> > @@ -47,7 +47,7 @@
> >   * Seven notes plus zero note at the end: prstatus, fpregset, timer,
> >   * tod_cmp, tod_reg, control regs, and prefix
> >   */
> > -#define KEXEC_NOTE_BYTES \
> > +#define CRASH_CORE_NOTE_BYTES \
> >  	(ALIGN(sizeof(struct elf_note), 4) * 8 + \
> >  	 ALIGN(sizeof("CORE"), 4) * 7 + \
> >  	 ALIGN(sizeof(struct elf_prstatus), 4) + \

I found that in mainline since below commit, above define should be
useless, but if distribution with older kernel does need your fix, so in
mainline the right fix should be dropping the s390 part about these
macros usage.

Anyway this need a comment from Michael.

commit 8a07dd02d7615d91d65d6235f7232e3f9b5d347f
Author: Martin Schwidefsky <schwidefsky at de.ibm.com>
Date:   Wed Oct 14 15:53:06 2015 +0200

    s390/kdump: remove code to create ELF notes in the crashed system
    
    The s390 architecture can store the CPU registers of the crashed
system
    after the kdump kernel has been started and this is the preferred
way.
    Remove the remaining code fragments that deal with storing CPU
registers
    while the crashed system is still active.
    
    Acked-by: Michael Holzheu <holzheu at linux.vnet.ibm.com>
    Signed-off-by: Martin Schwidefsky <schwidefsky at de.ibm.com>


> > diff --git a/include/linux/crash_core.h b/include/linux/crash_core.h
> > index e9de6b4..dbc6e5c 100644
> > --- a/include/linux/crash_core.h
> > +++ b/include/linux/crash_core.h
> > @@ -10,9 +10,16 @@
> >  #define CRASH_CORE_NOTE_NAME_BYTES ALIGN(sizeof(CRASH_CORE_NOTE_NAME), 4)
> >  #define CRASH_CORE_NOTE_DESC_BYTES ALIGN(sizeof(struct elf_prstatus), 4)
> >  
> > +/*
> > + * The per-cpu notes area is a list of notes terminated by a "NULL"
> > + * note header.  For kdump, the code in vmcore.c runs in the context
> > + * of the second kernel to combine them into one note.
> > + */
> > +#ifndef CRASH_CORE_NOTE_BYTES
> >  #define CRASH_CORE_NOTE_BYTES	   ((CRASH_CORE_NOTE_HEAD_BYTES * 2) +	\
> >  				     CRASH_CORE_NOTE_NAME_BYTES +	\
> >  				     CRASH_CORE_NOTE_DESC_BYTES)
> > +#endif
> >  
> >  #define VMCOREINFO_BYTES	   PAGE_SIZE
> >  #define VMCOREINFO_NOTE_NAME	   "VMCOREINFO"
> > diff --git a/include/linux/kexec.h b/include/linux/kexec.h
> > index 3ea8275..133df03 100644
> > --- a/include/linux/kexec.h
> > +++ b/include/linux/kexec.h
> > @@ -14,7 +14,6 @@
> >  
> >  #if !defined(__ASSEMBLY__)
> >  
> > -#include <linux/crash_core.h>
> >  #include <asm/io.h>
> >  
> >  #include <uapi/linux/kexec.h>
> > @@ -25,6 +24,7 @@
> >  #include <linux/ioport.h>
> >  #include <linux/module.h>
> >  #include <asm/kexec.h>
> > +#include <linux/crash_core.h>
> >  
> >  /* Verify architecture specific macros are defined */
> >  
> > @@ -63,15 +63,6 @@
> >  #define KEXEC_CORE_NOTE_NAME	CRASH_CORE_NOTE_NAME
> >  
> >  /*
> > - * The per-cpu notes area is a list of notes terminated by a "NULL"
> > - * note header.  For kdump, the code in vmcore.c runs in the context
> > - * of the second kernel to combine them into one note.
> > - */
> > -#ifndef KEXEC_NOTE_BYTES
> > -#define KEXEC_NOTE_BYTES	CRASH_CORE_NOTE_BYTES
> > -#endif
> 
> It is still not clear how does s390 use the crash_notes except this macro.
> But from code point of view we do need to update this as well after the
> crash_core splitting.
> 
> Acked-by: Dave Young <dyoung at redhat.com>

Hold on the ack because of the new findings, wait for Michael's
feedback.

Thanks
Dave


From bp at alien8.de  Fri Jun  9 03:55:40 2017
From: bp at alien8.de (Borislav Petkov)
Date: Fri, 9 Jun 2017 12:55:40 +0200
Subject: [PATCH v6 04/34] x86/CPU/AMD: Add the Secure Memory Encryption
 CPU feature
In-Reply-To: <20170607191353.28645.85058.stgit@tlendack-t1.amdoffice.net>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
 <20170607191353.28645.85058.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170609105540.jpkle2qja4kyin6i@pd.tnic>

On Wed, Jun 07, 2017 at 02:13:53PM -0500, Tom Lendacky wrote:
> Update the CPU features to include identifying and reporting on the
> Secure Memory Encryption (SME) feature.  SME is identified by CPUID
> 0x8000001f, but requires BIOS support to enable it (set bit 23 of
> MSR_K8_SYSCFG).  Only show the SME feature as available if reported by
> CPUID and enabled by BIOS.
> 
> Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
> ---
>  arch/x86/include/asm/cpufeatures.h |    1 +
>  arch/x86/include/asm/msr-index.h   |    2 ++
>  arch/x86/kernel/cpu/amd.c          |   13 +++++++++++++
>  arch/x86/kernel/cpu/scattered.c    |    1 +
>  4 files changed, 17 insertions(+)

Reviewed-by: Borislav Petkov <bp at suse.de>

-- 
Regards/Gruss,
    Boris.

Good mailing practices for 400: avoid top-posting and trim the reply.


From horms at verge.net.au  Fri Jun  9 09:28:15 2017
From: horms at verge.net.au (Simon Horman)
Date: Fri, 9 Jun 2017 18:28:15 +0200
Subject: [ANNOUNCE] kexec-tools 2.0.15-rc1
Message-ID: <20170609162815.GA21274@verge.net.au>

Hi all,

I am happy to announce the release of kexec-tools 2.0.15-rc1.

This is an incremental feature pre-release.

So long as no serious problems arise I intend to release kexec-tools 2.0.15
in a weeks time. Testing of 2.0.15-rc1 would be greatly appreciated.

I do not have any outstanding changes for 2.0.15 at this time.
And I would like to only accept bug fixes at this time and take take
features patches once 2.0.15 has been released.

The pre-release can be downloaded from kernel.org:

	https://kernel.org/pub/linux/utils/kernel/kexec/kexec-tools-2.0.15-rc1.tar.xz
	https://kernel.org/pub/linux/utils/kernel/kexec/

I have also tagged it in git:

	https://git.kernel.org/pub/scm/utils/kernel/kexec/kexec-tools.git

Thanks to everyone who has contributed to kexec.


For reference the changes since v2.0.14 are:

302e1e362753 kexec-tools 2.0.15-rc1
fab91841c717 Handle additional e820 memmap type strings
c504ff5d85aa arm64: kdump: Add support for binary image files
5f955585c7c9 arm64: kdump: add DT properties to crash dump kernel's dtb
defad947feff arm64: kdump: set up other segments
1591926df5a6 arm64: kdump: set up kernel image segment
0bd5219da953 arm64: kdump: add elf core header segment
c0672c93edcb arm64: kdump: identify memory regions
a17234fe94bc arm64: change return values on error to negative
ef26cc33b8d6 arm64: identify PHYS_OFFSET correctly
325804055e99 kexec: generalize and rename get_kernel_stext_sym()
c95df0e099b1 kexec: extend the semantics of kexec_iomem_for_each_line
59d3e5b5ad6f Fix broken Xen support in configure.ac
4a6d67d9e938 x86: Support large number of memory ranges
0516f46adbf3 crashdump: Remove stray get_crashkernel_region() declaration
82a49747e5ad ppc: Fix format warning with die()
1550f81bf188 x86/x86_64: Fix format warning with die()
47cc70157c66 Don't use %L width specifier with integer values
2f6f6d6fef78 vmcore-dmesg: Define _GNU_SOURCE
896fb2aa30c6 arm64: add uImage support
a0c575793b86 uImage: use 'char *' instead of 'unsigned char *' for uImage_probe()
f25146afc5a9 uImage: use 'char *' instead of 'unsigned char *' for uImage_load()
b3d533c1f499 uImage: Add new IH_ARCH_xxx definitions
67234243bb91 uImage: Fix uImage_load() for little-endian machines
0cc1891c4dc8 uImage: fix realloc() pointer confusion
ed15ba1b9977 build_mem_phdrs(): check if p_paddr is invalid
263e45ccf27b Only print debug message when failed to serach for kernel symbol from /proc/kallsyms
7dac152d5b47 gitignore: add two generated files in purgatory
05ae4fb2e354 crashdump/sh: Add get_crash_kernel_load_range() function
796f0ffa134d crashdump/s390: Add get_crash_kernel_load_range() function
e4280e22c8c4 crashdump/ppc64: Add get_crash_kernel_load_range() function
7fc80cfcd913 crashdump/ppc: Add get_crash_kernel_load_range() function
d2caf4c4c43b crashdump/mips: Add get_crash_kernel_load_range() function
14d71e51e5c9 crashdump/m68k: Add get_crash_kernel_load_range() function
5c80bd9be295 crashdump/ia64: Add get_crash_kernel_load_range() function
b6af22826f60 crashdump/cris: Add get_crash_kernel_load_range() function
d43610084164 crashdump/arm64: Add get_crash_kernel_load_range() function
cfcf60c38182 crashdump/arm: Add get_crash_kernel_load_range() function
76b31203222a kexec: Add option to get crash kernel region size
e49623b0787d purgatory: Add purgatory.map and purgatory.ro.sym to clean recipe
ceedb33e6cd3 kexec: Remove redundant space from help message
2cf7cb9a6080 kexec: implemented XEN KEXEC STATUS to determine if an image is loaded
4eaa36cd01a9 alpha: add missing __NR_kexec_load definition
24aa2d93cac3 kexec: Increase the upper limit for RAM segments
f63d8530b9b6 ppc64: Reduce number of ELF LOAD segments
9da19c0a6f49 kexec-tools 2.0.14.git


From bp at alien8.de  Fri Jun  9 09:30:03 2017
From: bp at alien8.de (Borislav Petkov)
Date: Fri, 9 Jun 2017 18:30:03 +0200
Subject: [PATCH v6 05/34] x86/CPU/AMD: Handle SME reduction in physical
 address size
In-Reply-To: <20170607191404.28645.41148.stgit@tlendack-t1.amdoffice.net>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
 <20170607191404.28645.41148.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170609163003.g2s6aqzkikt6nt4a@pd.tnic>

On Wed, Jun 07, 2017 at 02:14:04PM -0500, Tom Lendacky wrote:
> When System Memory Encryption (SME) is enabled, the physical address
> space is reduced. Adjust the x86_phys_bits value to reflect this
> reduction.
> 
> Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
> ---
>  arch/x86/kernel/cpu/amd.c |   10 +++++++---
>  1 file changed, 7 insertions(+), 3 deletions(-)

Reviewed-by: Borislav Petkov <bp at suse.de>

-- 
Regards/Gruss,
    Boris.

Good mailing practices for 400: avoid top-posting and trim the reply.


From bp at alien8.de  Fri Jun  9 09:43:09 2017
From: bp at alien8.de (Borislav Petkov)
Date: Fri, 9 Jun 2017 18:43:09 +0200
Subject: [PATCH v6 06/34] x86/mm: Add Secure Memory Encryption (SME)
 support
In-Reply-To: <20170607191416.28645.58145.stgit@tlendack-t1.amdoffice.net>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
 <20170607191416.28645.58145.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170609164309.45xgyzibdkzmdbde@pd.tnic>

On Wed, Jun 07, 2017 at 02:14:16PM -0500, Tom Lendacky wrote:
> Add support for Secure Memory Encryption (SME). This initial support
> provides a Kconfig entry to build the SME support into the kernel and
> defines the memory encryption mask that will be used in subsequent
> patches to mark pages as encrypted.
> 
> Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
> ---
>  arch/x86/Kconfig                   |   22 ++++++++++++++++++++++
>  arch/x86/include/asm/mem_encrypt.h |   35 +++++++++++++++++++++++++++++++++++
>  arch/x86/mm/Makefile               |    1 +
>  arch/x86/mm/mem_encrypt.c          |   21 +++++++++++++++++++++
>  include/asm-generic/mem_encrypt.h  |   27 +++++++++++++++++++++++++++
>  include/linux/mem_encrypt.h        |   18 ++++++++++++++++++
>  6 files changed, 124 insertions(+)
>  create mode 100644 arch/x86/include/asm/mem_encrypt.h
>  create mode 100644 arch/x86/mm/mem_encrypt.c
>  create mode 100644 include/asm-generic/mem_encrypt.h
>  create mode 100644 include/linux/mem_encrypt.h

Reviewed-by: Borislav Petkov <bp at suse.de>

-- 
Regards/Gruss,
    Boris.

Good mailing practices for 400: avoid top-posting and trim the reply.


From thomas.lendacky at amd.com  Fri Jun  9 11:36:35 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Fri, 9 Jun 2017 13:36:35 -0500
Subject: [Xen-devel] [PATCH v6 10/34] x86, x86/mm, x86/xen, olpc: Use
 __va() against just the physical address in cr3
In-Reply-To: <d37917b1-8e49-e8a8-b9ac-59491331640f@citrix.com>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
 <20170607191453.28645.92256.stgit@tlendack-t1.amdoffice.net>
 <b15e8924-4069-b5fa-adb2-86c164b1dd36@oracle.com>
 <4a7376fb-abfc-8edd-42b7-38de461ac65e@amd.com>
 <67fe69ac-a213-8de3-db28-0e54bba95127@oracle.com>
 <fcb196c8-f1eb-a38c-336c-7bd3929b029e@amd.com>
 <12c7e511-996d-cf60-3a3b-0be7b41bd85b@oracle.com>
 <d37917b1-8e49-e8a8-b9ac-59491331640f@citrix.com>
Message-ID: <9725c503-2e33-2365-87f5-f017e1cbe9b6@amd.com>

On 6/8/2017 5:01 PM, Andrew Cooper wrote:
> On 08/06/2017 22:17, Boris Ostrovsky wrote:
>> On 06/08/2017 05:02 PM, Tom Lendacky wrote:
>>> On 6/8/2017 3:51 PM, Boris Ostrovsky wrote:
>>>>>> What may be needed is making sure X86_FEATURE_SME is not set for PV
>>>>>> guests.
>>>>> And that may be something that Xen will need to control through either
>>>>> CPUID or MSR support for the PV guests.
>>>>
>>>> Only on newer versions of Xen. On earlier versions (2-3 years old) leaf
>>>> 0x80000007 is passed to the guest unchanged. And so is MSR_K8_SYSCFG.
>>> The SME feature is in leaf 0x8000001f, is that leaf passed to the guest
>>> unchanged?
>> Oh, I misread the patch where X86_FEATURE_SME is defined. Then all
>> versions, including the current one, pass it unchanged.
>>
>> All that's needed is setup_clear_cpu_cap(X86_FEATURE_SME) in
>> xen_init_capabilities().
> 
> AMD processors still don't support CPUID Faulting (or at least, I
> couldn't find any reference to it in the latest docs), so we cannot
> actually hide SME from a guest which goes looking at native CPUID.
> Furthermore, I'm not aware of any CPUID masking support covering that leaf.
> 
> However, if Linux is using the paravirtual cpuid hook, things are
> slightly better.
> 
> On Xen 4.9 and later, no guests will see the feature.  On earlier
> versions of Xen (before I fixed the logic), plain domUs will not see the
> feature, while dom0 will.
> 
> For safely, I'd recommend unilaterally clobbering the feature as Boris
> suggested.  There is no way SME will be supportable on a per-PV guest

That may be too late. Early boot support in head_64.S will make calls to
check for the feature (through CPUID and MSR), set the sme_me_mask and
encrypt the kernel in place. Is there another way to approach this?

> basis, although (as far as I am aware) Xen as a whole would be able to
> encompass itself and all of its PV guests inside one single SME instance.

Yes, that is correct.

Thanks,
Tom

> 
> ~Andrew
> 


From boris.ostrovsky at oracle.com  Fri Jun  9 11:43:33 2017
From: boris.ostrovsky at oracle.com (Boris Ostrovsky)
Date: Fri, 9 Jun 2017 14:43:33 -0400
Subject: [Xen-devel] [PATCH v6 10/34] x86, x86/mm, x86/xen, olpc: Use
 __va() against just the physical address in cr3
In-Reply-To: <9725c503-2e33-2365-87f5-f017e1cbe9b6@amd.com>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
 <20170607191453.28645.92256.stgit@tlendack-t1.amdoffice.net>
 <b15e8924-4069-b5fa-adb2-86c164b1dd36@oracle.com>
 <4a7376fb-abfc-8edd-42b7-38de461ac65e@amd.com>
 <67fe69ac-a213-8de3-db28-0e54bba95127@oracle.com>
 <fcb196c8-f1eb-a38c-336c-7bd3929b029e@amd.com>
 <12c7e511-996d-cf60-3a3b-0be7b41bd85b@oracle.com>
 <d37917b1-8e49-e8a8-b9ac-59491331640f@citrix.com>
 <9725c503-2e33-2365-87f5-f017e1cbe9b6@amd.com>
Message-ID: <8e8eac45-95be-f1b5-6f44-f131d275f7bc@oracle.com>

On 06/09/2017 02:36 PM, Tom Lendacky wrote:
> On 6/8/2017 5:01 PM, Andrew Cooper wrote:
>> On 08/06/2017 22:17, Boris Ostrovsky wrote:
>>> On 06/08/2017 05:02 PM, Tom Lendacky wrote:
>>>> On 6/8/2017 3:51 PM, Boris Ostrovsky wrote:
>>>>>>> What may be needed is making sure X86_FEATURE_SME is not set for PV
>>>>>>> guests.
>>>>>> And that may be something that Xen will need to control through
>>>>>> either
>>>>>> CPUID or MSR support for the PV guests.
>>>>>
>>>>> Only on newer versions of Xen. On earlier versions (2-3 years old)
>>>>> leaf
>>>>> 0x80000007 is passed to the guest unchanged. And so is MSR_K8_SYSCFG.
>>>> The SME feature is in leaf 0x8000001f, is that leaf passed to the
>>>> guest
>>>> unchanged?
>>> Oh, I misread the patch where X86_FEATURE_SME is defined. Then all
>>> versions, including the current one, pass it unchanged.
>>>
>>> All that's needed is setup_clear_cpu_cap(X86_FEATURE_SME) in
>>> xen_init_capabilities().
>>
>> AMD processors still don't support CPUID Faulting (or at least, I
>> couldn't find any reference to it in the latest docs), so we cannot
>> actually hide SME from a guest which goes looking at native CPUID.
>> Furthermore, I'm not aware of any CPUID masking support covering that
>> leaf.
>>
>> However, if Linux is using the paravirtual cpuid hook, things are
>> slightly better.
>>
>> On Xen 4.9 and later, no guests will see the feature.  On earlier
>> versions of Xen (before I fixed the logic), plain domUs will not see the
>> feature, while dom0 will.
>>
>> For safely, I'd recommend unilaterally clobbering the feature as Boris
>> suggested.  There is no way SME will be supportable on a per-PV guest
>
> That may be too late. Early boot support in head_64.S will make calls to
> check for the feature (through CPUID and MSR), set the sme_me_mask and
> encrypt the kernel in place. Is there another way to approach this?


PV guests don't go through Linux x86 early boot code. They start at
xen_start_kernel() (well, xen-head.S:startup_xen(), really) and  merge
with baremetal path at x86_64_start_reservations() (for 64-bit).


-boris

>
>> basis, although (as far as I am aware) Xen as a whole would be able to
>> encompass itself and all of its PV guests inside one single SME
>> instance.
>
> Yes, that is correct.
>
> Thanks,
> Tom
>
>>
>> ~Andrew
>>



From luto at kernel.org  Fri Jun  9 11:46:32 2017
From: luto at kernel.org (Andy Lutomirski)
Date: Fri, 9 Jun 2017 11:46:32 -0700
Subject: [PATCH v6 10/34] x86, x86/mm, x86/xen, olpc: Use __va() against
 just the physical address in cr3
In-Reply-To: <85355c4c-2cc1-daac-d8fe-ac6965b34606@amd.com>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
 <20170607191453.28645.92256.stgit@tlendack-t1.amdoffice.net>
 <CALCETrVVhMf=zkiDNn_-hKDZLGXKFiwxuWkPmD5RJgHa5VUMiQ@mail.gmail.com>
 <85355c4c-2cc1-daac-d8fe-ac6965b34606@amd.com>
Message-ID: <CALCETrVyHbgJxk6hTRuM0CCpWTvFSX0MxuUQjKOf=7hYCt-yEg@mail.gmail.com>

On Thu, Jun 8, 2017 at 3:38 PM, Tom Lendacky <thomas.lendacky at amd.com> wrote:
> On 6/8/2017 1:05 AM, Andy Lutomirski wrote:
>>
>> On Wed, Jun 7, 2017 at 12:14 PM, Tom Lendacky <thomas.lendacky at amd.com>
>> wrote:
>>>
>>> The cr3 register entry can contain the SME encryption bit that indicates
>>> the PGD is encrypted.  The encryption bit should not be used when
>>> creating
>>> a virtual address for the PGD table.
>>>
>>> Create a new function, read_cr3_pa(), that will extract the physical
>>> address from the cr3 register. This function is then used where a virtual
>>> address of the PGD needs to be created/used from the cr3 register.
>>
>>
>> This is going to conflict with:
>>
>>
>> https://git.kernel.org/pub/scm/linux/kernel/git/luto/linux.git/commit/?h=x86/pcid&id=555c81e5d01a62b629ec426a2f50d27e2127c1df
>>
>> We're both encountering the fact that CR3 munges the page table PA
>> with some other stuff, and some readers want to see the actual CR3
>> value and other readers just want the PA.  The thing I prefer about my
>> patch is that I get rid of read_cr3() entirely, forcing the patch to
>> update every single reader, making review and conflict resolution much
>> safer.
>>
>> I'd be willing to send a patch tomorrow that just does the split into
>> __read_cr3() and read_cr3_pa() (I like your name better) and then we
>> can both base on top of it.  Would that make sense?
>
>
> That makes sense to me.

Draft patch:

https://git.kernel.org/pub/scm/linux/kernel/git/luto/linux.git/commit/?h=x86/read_cr3&id=9adebbc1071f066421a27b4f6e040190f1049624

>
>>
>> Also:
>>
>>> +static inline unsigned long read_cr3_pa(void)
>>> +{
>>> +       return (read_cr3() & PHYSICAL_PAGE_MASK);
>>> +}
>>
>>
>> Is there any guarantee that the magic encryption bit is masked out in
>> PHYSICAL_PAGE_MASK?  The docs make it sound like it could be any bit.
>> (But if it's one of the low 12 bits, that would be quite confusing.)
>
>
> Right now it's bit 47 and we're steering away from any of the currently
> reserved bits so we should be safe.

Should the SME init code check that it's a usable bit (i.e. outside
our physical address mask and not one of the bottom twelve bits)?  If
some future CPU daftly picks, say, bit 12, we'll regret it if we
enable SME.

--Andy


From andrew.cooper3 at citrix.com  Fri Jun  9 11:54:04 2017
From: andrew.cooper3 at citrix.com (Andrew Cooper)
Date: Fri, 9 Jun 2017 19:54:04 +0100
Subject: [Xen-devel] [PATCH v6 10/34] x86, x86/mm, x86/xen, olpc: Use
 __va() against just the physical address in cr3
In-Reply-To: <8e8eac45-95be-f1b5-6f44-f131d275f7bc@oracle.com>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
 <20170607191453.28645.92256.stgit@tlendack-t1.amdoffice.net>
 <b15e8924-4069-b5fa-adb2-86c164b1dd36@oracle.com>
 <4a7376fb-abfc-8edd-42b7-38de461ac65e@amd.com>
 <67fe69ac-a213-8de3-db28-0e54bba95127@oracle.com>
 <fcb196c8-f1eb-a38c-336c-7bd3929b029e@amd.com>
 <12c7e511-996d-cf60-3a3b-0be7b41bd85b@oracle.com>
 <d37917b1-8e49-e8a8-b9ac-59491331640f@citrix.com>
 <9725c503-2e33-2365-87f5-f017e1cbe9b6@amd.com>
 <8e8eac45-95be-f1b5-6f44-f131d275f7bc@oracle.com>
Message-ID: <2d507507-8736-89ff-0579-c2eee4b3ac34@citrix.com>

On 09/06/17 19:43, Boris Ostrovsky wrote:
> On 06/09/2017 02:36 PM, Tom Lendacky wrote:
>>> basis, although (as far as I am aware) Xen as a whole would be able to
>>> encompass itself and all of its PV guests inside one single SME
>>> instance.
>> Yes, that is correct.

Thinking more about this, it would only be possible if all the PV guests
were SME-aware and understood not to choke when it finds a frame with a
high address bit set.

I expect the only viable way to implement this (should we wish) is to
have PV guests explicitly signal support (probably via an ELF note),
after which it needs to know about the existence of SME, the meaning of
the encrypted bit in PTEs, and to defer all configuration responsibility
to Xen.

~Andrew


From thomas.lendacky at amd.com  Fri Jun  9 11:59:21 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Fri, 9 Jun 2017 13:59:21 -0500
Subject: [Xen-devel] [PATCH v6 10/34] x86, x86/mm, x86/xen, olpc: Use
 __va() against just the physical address in cr3
In-Reply-To: <8e8eac45-95be-f1b5-6f44-f131d275f7bc@oracle.com>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
 <20170607191453.28645.92256.stgit@tlendack-t1.amdoffice.net>
 <b15e8924-4069-b5fa-adb2-86c164b1dd36@oracle.com>
 <4a7376fb-abfc-8edd-42b7-38de461ac65e@amd.com>
 <67fe69ac-a213-8de3-db28-0e54bba95127@oracle.com>
 <fcb196c8-f1eb-a38c-336c-7bd3929b029e@amd.com>
 <12c7e511-996d-cf60-3a3b-0be7b41bd85b@oracle.com>
 <d37917b1-8e49-e8a8-b9ac-59491331640f@citrix.com>
 <9725c503-2e33-2365-87f5-f017e1cbe9b6@amd.com>
 <8e8eac45-95be-f1b5-6f44-f131d275f7bc@oracle.com>
Message-ID: <33f20df0-bf71-bd9d-7a7e-4fb5e8793400@amd.com>

On 6/9/2017 1:43 PM, Boris Ostrovsky wrote:
> On 06/09/2017 02:36 PM, Tom Lendacky wrote:
>> On 6/8/2017 5:01 PM, Andrew Cooper wrote:
>>> On 08/06/2017 22:17, Boris Ostrovsky wrote:
>>>> On 06/08/2017 05:02 PM, Tom Lendacky wrote:
>>>>> On 6/8/2017 3:51 PM, Boris Ostrovsky wrote:
>>>>>>>> What may be needed is making sure X86_FEATURE_SME is not set for PV
>>>>>>>> guests.
>>>>>>> And that may be something that Xen will need to control through
>>>>>>> either
>>>>>>> CPUID or MSR support for the PV guests.
>>>>>>
>>>>>> Only on newer versions of Xen. On earlier versions (2-3 years old)
>>>>>> leaf
>>>>>> 0x80000007 is passed to the guest unchanged. And so is MSR_K8_SYSCFG.
>>>>> The SME feature is in leaf 0x8000001f, is that leaf passed to the
>>>>> guest
>>>>> unchanged?
>>>> Oh, I misread the patch where X86_FEATURE_SME is defined. Then all
>>>> versions, including the current one, pass it unchanged.
>>>>
>>>> All that's needed is setup_clear_cpu_cap(X86_FEATURE_SME) in
>>>> xen_init_capabilities().
>>>
>>> AMD processors still don't support CPUID Faulting (or at least, I
>>> couldn't find any reference to it in the latest docs), so we cannot
>>> actually hide SME from a guest which goes looking at native CPUID.
>>> Furthermore, I'm not aware of any CPUID masking support covering that
>>> leaf.
>>>
>>> However, if Linux is using the paravirtual cpuid hook, things are
>>> slightly better.
>>>
>>> On Xen 4.9 and later, no guests will see the feature.  On earlier
>>> versions of Xen (before I fixed the logic), plain domUs will not see the
>>> feature, while dom0 will.
>>>
>>> For safely, I'd recommend unilaterally clobbering the feature as Boris
>>> suggested.  There is no way SME will be supportable on a per-PV guest
>>
>> That may be too late. Early boot support in head_64.S will make calls to
>> check for the feature (through CPUID and MSR), set the sme_me_mask and
>> encrypt the kernel in place. Is there another way to approach this?
> 
> 
> PV guests don't go through Linux x86 early boot code. They start at
> xen_start_kernel() (well, xen-head.S:startup_xen(), really) and  merge
> with baremetal path at x86_64_start_reservations() (for 64-bit).
> 

Ok, I don't think anything needs to be done then. The sme_me_mask is set
in sme_enable() which is only called from head_64.S. If the sme_me_mask
isn't set then SME won't be active. The feature will just report the
capability of the processor, but that doesn't mean it is active. If you
still want the feature to be clobbered we can do that, though.

Thanks,
Tom

> 
> -boris
> 
>>
>>> basis, although (as far as I am aware) Xen as a whole would be able to
>>> encompass itself and all of its PV guests inside one single SME
>>> instance.
>>
>> Yes, that is correct.
>>
>> Thanks,
>> Tom
>>
>>>
>>> ~Andrew
>>>
> 


From boris.ostrovsky at oracle.com  Fri Jun  9 12:42:24 2017
From: boris.ostrovsky at oracle.com (Boris Ostrovsky)
Date: Fri, 9 Jun 2017 15:42:24 -0400
Subject: [Xen-devel] [PATCH v6 10/34] x86, x86/mm, x86/xen, olpc: Use
 __va() against just the physical address in cr3
In-Reply-To: <33f20df0-bf71-bd9d-7a7e-4fb5e8793400@amd.com>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
 <20170607191453.28645.92256.stgit@tlendack-t1.amdoffice.net>
 <b15e8924-4069-b5fa-adb2-86c164b1dd36@oracle.com>
 <4a7376fb-abfc-8edd-42b7-38de461ac65e@amd.com>
 <67fe69ac-a213-8de3-db28-0e54bba95127@oracle.com>
 <fcb196c8-f1eb-a38c-336c-7bd3929b029e@amd.com>
 <12c7e511-996d-cf60-3a3b-0be7b41bd85b@oracle.com>
 <d37917b1-8e49-e8a8-b9ac-59491331640f@citrix.com>
 <9725c503-2e33-2365-87f5-f017e1cbe9b6@amd.com>
 <8e8eac45-95be-f1b5-6f44-f131d275f7bc@oracle.com>
 <33f20df0-bf71-bd9d-7a7e-4fb5e8793400@amd.com>
Message-ID: <d1931930-78d6-ebb1-755c-80bc88a397ed@oracle.com>


>>
>> PV guests don't go through Linux x86 early boot code. They start at
>> xen_start_kernel() (well, xen-head.S:startup_xen(), really) and  merge
>> with baremetal path at x86_64_start_reservations() (for 64-bit).
>>
>
> Ok, I don't think anything needs to be done then. The sme_me_mask is set
> in sme_enable() which is only called from head_64.S. If the sme_me_mask
> isn't set then SME won't be active. The feature will just report the
> capability of the processor, but that doesn't mean it is active. If you
> still want the feature to be clobbered we can do that, though.

I'd prefer to explicitly clear to avoid any ambiguity.

-boris


From thomas.lendacky at amd.com  Fri Jun  9 14:20:35 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Fri, 9 Jun 2017 16:20:35 -0500
Subject: [PATCH v6 10/34] x86, x86/mm, x86/xen, olpc: Use __va() against
 just the physical address in cr3
In-Reply-To: <CALCETrVyHbgJxk6hTRuM0CCpWTvFSX0MxuUQjKOf=7hYCt-yEg@mail.gmail.com>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
 <20170607191453.28645.92256.stgit@tlendack-t1.amdoffice.net>
 <CALCETrVVhMf=zkiDNn_-hKDZLGXKFiwxuWkPmD5RJgHa5VUMiQ@mail.gmail.com>
 <85355c4c-2cc1-daac-d8fe-ac6965b34606@amd.com>
 <CALCETrVyHbgJxk6hTRuM0CCpWTvFSX0MxuUQjKOf=7hYCt-yEg@mail.gmail.com>
Message-ID: <601d5972-72b0-6556-0054-7275cb763816@amd.com>

On 6/9/2017 1:46 PM, Andy Lutomirski wrote:
> On Thu, Jun 8, 2017 at 3:38 PM, Tom Lendacky <thomas.lendacky at amd.com> wrote:
>> On 6/8/2017 1:05 AM, Andy Lutomirski wrote:
>>>
>>> On Wed, Jun 7, 2017 at 12:14 PM, Tom Lendacky <thomas.lendacky at amd.com>
>>> wrote:
>>>>
>>>> The cr3 register entry can contain the SME encryption bit that indicates
>>>> the PGD is encrypted.  The encryption bit should not be used when
>>>> creating
>>>> a virtual address for the PGD table.
>>>>
>>>> Create a new function, read_cr3_pa(), that will extract the physical
>>>> address from the cr3 register. This function is then used where a virtual
>>>> address of the PGD needs to be created/used from the cr3 register.
>>>
>>>
>>> This is going to conflict with:
>>>
>>>
>>> https://git.kernel.org/pub/scm/linux/kernel/git/luto/linux.git/commit/?h=x86/pcid&id=555c81e5d01a62b629ec426a2f50d27e2127c1df
>>>
>>> We're both encountering the fact that CR3 munges the page table PA
>>> with some other stuff, and some readers want to see the actual CR3
>>> value and other readers just want the PA.  The thing I prefer about my
>>> patch is that I get rid of read_cr3() entirely, forcing the patch to
>>> update every single reader, making review and conflict resolution much
>>> safer.
>>>
>>> I'd be willing to send a patch tomorrow that just does the split into
>>> __read_cr3() and read_cr3_pa() (I like your name better) and then we
>>> can both base on top of it.  Would that make sense?
>>
>>
>> That makes sense to me.
> 
> Draft patch:
> 
> https://git.kernel.org/pub/scm/linux/kernel/git/luto/linux.git/commit/?h=x86/read_cr3&id=9adebbc1071f066421a27b4f6e040190f1049624

Looks good to me. I'll look at how to best mask off the encryption bit
in CR3_ADDR_MASK for SME support.  I should be able to just do an
__sme_clr() against it.

> 
>>
>>>
>>> Also:
>>>
>>>> +static inline unsigned long read_cr3_pa(void)
>>>> +{
>>>> +       return (read_cr3() & PHYSICAL_PAGE_MASK);
>>>> +}
>>>
>>>
>>> Is there any guarantee that the magic encryption bit is masked out in
>>> PHYSICAL_PAGE_MASK?  The docs make it sound like it could be any bit.
>>> (But if it's one of the low 12 bits, that would be quite confusing.)
>>
>>
>> Right now it's bit 47 and we're steering away from any of the currently
>> reserved bits so we should be safe.
> 
> Should the SME init code check that it's a usable bit (i.e. outside
> our physical address mask and not one of the bottom twelve bits)?  If
> some future CPU daftly picks, say, bit 12, we'll regret it if we
> enable SME.

I think I can safely say that it will never be any of the lower 12 bits,
but let me talk to some of the hardware folks and see about the other
end of the range.

Thanks,
Tom

> 
> --Andy
> 


From bp at alien8.de  Sat Jun 10 03:44:08 2017
From: bp at alien8.de (Borislav Petkov)
Date: Sat, 10 Jun 2017 12:44:08 +0200
Subject: [PATCH v6 09/34] x86/mm: Simplify p[gum]d_page() macros
In-Reply-To: <20170607191445.28645.15664.stgit@tlendack-t1.amdoffice.net>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
 <20170607191445.28645.15664.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170610104408.xmmlwtafyhbczxyi@pd.tnic>

On Wed, Jun 07, 2017 at 02:14:45PM -0500, Tom Lendacky wrote:
> Create a pgd_pfn() macro similar to the p[um]d_pfn() macros and then
> use the p[gum]d_pfn() macros in the p[gum]d_page() macros instead of
> duplicating the code.
> 
> Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
> ---
>  arch/x86/include/asm/pgtable.h |   16 +++++++++-------
>  1 file changed, 9 insertions(+), 7 deletions(-)

For patches 7-9:

Reviewed-by: Borislav Petkov <bp at suse.de>

-- 
Regards/Gruss,
    Boris.

Good mailing practices for 400: avoid top-posting and trim the reply.


From bp at alien8.de  Sat Jun 10 08:56:25 2017
From: bp at alien8.de (Borislav Petkov)
Date: Sat, 10 Jun 2017 17:56:25 +0200
Subject: [PATCH v6 13/34] x86/mm: Add support for early encrypt/decrypt
 of memory
In-Reply-To: <20170607191527.28645.84593.stgit@tlendack-t1.amdoffice.net>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
 <20170607191527.28645.84593.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170610155624.q3jh6q2wpnepxpsw@pd.tnic>

On Wed, Jun 07, 2017 at 02:15:27PM -0500, Tom Lendacky wrote:
> Add support to be able to either encrypt or decrypt data in place during
> the early stages of booting the kernel. This does not change the memory
> encryption attribute - it is used for ensuring that data present in either
> an encrypted or decrypted memory area is in the proper state (for example
> the initrd will have been loaded by the boot loader and will not be
> encrypted, but the memory that it resides in is marked as encrypted).
> 
> Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
> ---
>  arch/x86/include/asm/mem_encrypt.h |   15 +++++++
>  arch/x86/mm/mem_encrypt.c          |   76 ++++++++++++++++++++++++++++++++++++
>  2 files changed, 91 insertions(+)

Patches 11-13:

Reviewed-by: Borislav Petkov <bp at suse.de>

-- 
Regards/Gruss,
    Boris.

Good mailing practices for 400: avoid top-posting and trim the reply.


From bp at alien8.de  Sat Jun 10 09:01:19 2017
From: bp at alien8.de (Borislav Petkov)
Date: Sat, 10 Jun 2017 18:01:19 +0200
Subject: [PATCH v6 14/34] x86/mm: Insure that boot memory areas are
 mapped properly
In-Reply-To: <20170607191539.28645.70161.stgit@tlendack-t1.amdoffice.net>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
 <20170607191539.28645.70161.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170610160119.bnx5ir5dj3i27igx@pd.tnic>

On Wed, Jun 07, 2017 at 02:15:39PM -0500, Tom Lendacky wrote:
> The boot data and command line data are present in memory in a decrypted
> state and are copied early in the boot process.  The early page fault
> support will map these areas as encrypted, so before attempting to copy
> them, add decrypted mappings so the data is accessed properly when copied.
> 
> For the initrd, encrypt this data in place. Since the future mapping of the
> initrd area will be mapped as encrypted the data will be accessed properly.
> 
> Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
> ---
>  arch/x86/include/asm/mem_encrypt.h |   11 +++++
>  arch/x86/include/asm/pgtable.h     |    3 +
>  arch/x86/kernel/head64.c           |   30 ++++++++++++--
>  arch/x86/kernel/setup.c            |    9 ++++
>  arch/x86/mm/mem_encrypt.c          |   77 ++++++++++++++++++++++++++++++++++++
>  5 files changed, 126 insertions(+), 4 deletions(-)

Some cleanups ontop in case you get to send v7:

diff --git a/arch/x86/include/asm/mem_encrypt.h b/arch/x86/include/asm/mem_encrypt.h
index 61a704945294..5959a42dd4d5 100644
--- a/arch/x86/include/asm/mem_encrypt.h
+++ b/arch/x86/include/asm/mem_encrypt.h
@@ -45,13 +45,8 @@ static inline void __init sme_early_decrypt(resource_size_t paddr,
 {
 }
 
-static inline void __init sme_map_bootdata(char *real_mode_data)
-{
-}
-
-static inline void __init sme_unmap_bootdata(char *real_mode_data)
-{
-}
+static inline void __init sme_map_bootdata(char *real_mode_data)	{ }
+static inline void __init sme_unmap_bootdata(char *real_mode_data)	{ }
 
 static inline void __init sme_early_init(void)
 {
diff --git a/arch/x86/mm/mem_encrypt.c b/arch/x86/mm/mem_encrypt.c
index 2321f05045e5..32ebbe0ab04d 100644
--- a/arch/x86/mm/mem_encrypt.c
+++ b/arch/x86/mm/mem_encrypt.c
@@ -132,6 +132,10 @@ static void __init __sme_map_unmap_bootdata(char *real_mode_data, bool map)
 	struct boot_params *boot_data;
 	unsigned long cmdline_paddr;
 
+	/* If SME is not active, the bootdata is in the correct state */
+	if (!sme_active())
+		return;
+
 	__sme_early_map_unmap_mem(real_mode_data, sizeof(boot_params), map);
 	boot_data = (struct boot_params *)real_mode_data;
 
@@ -142,40 +146,22 @@ static void __init __sme_map_unmap_bootdata(char *real_mode_data, bool map)
 	cmdline_paddr = boot_data->hdr.cmd_line_ptr |
 			((u64)boot_data->ext_cmd_line_ptr << 32);
 
-	if (cmdline_paddr)
-		__sme_early_map_unmap_mem(__va(cmdline_paddr),
-					  COMMAND_LINE_SIZE, map);
+	if (!cmdline_paddr)
+		return;
+
+	__sme_early_map_unmap_mem(__va(cmdline_paddr), COMMAND_LINE_SIZE, map);
+
+	sme_early_pgtable_flush();
 }
 
 void __init sme_unmap_bootdata(char *real_mode_data)
 {
-	/* If SME is not active, the bootdata is in the correct state */
-	if (!sme_active())
-		return;
-
-	/*
-	 * The bootdata and command line aren't needed anymore so clear
-	 * any mapping of them.
-	 */
 	__sme_map_unmap_bootdata(real_mode_data, false);
-
-	sme_early_pgtable_flush();
 }
 
 void __init sme_map_bootdata(char *real_mode_data)
 {
-	/* If SME is not active, the bootdata is in the correct state */
-	if (!sme_active())
-		return;
-
-	/*
-	 * The bootdata and command line will not be encrypted, so they
-	 * need to be mapped as decrypted memory so they can be copied
-	 * properly.
-	 */
 	__sme_map_unmap_bootdata(real_mode_data, true);
-
-	sme_early_pgtable_flush();
 }
 
 void __init sme_early_init(void)

-- 
Regards/Gruss,
    Boris.

Good mailing practices for 400: avoid top-posting and trim the reply.


From xpang at redhat.com  Sun Jun 11 01:43:23 2017
From: xpang at redhat.com (Xunlei Pang)
Date: Sun, 11 Jun 2017 16:43:23 +0800
Subject: [PATCH] s390/crash: Fix KEXEC_NOTE_BYTES definition
In-Reply-To: <20170609074529.GA9485@dhcp-128-65.nay.redhat.com>
References: <1496974625-10891-1-git-send-email-xlpang@redhat.com>
 <20170609022904.GB3688@dhcp-128-65.nay.redhat.com>
 <20170609074529.GA9485@dhcp-128-65.nay.redhat.com>
Message-ID: <593D02AB.1030706@redhat.com>

On 06/09/2017 at 03:45 PM, Dave Young wrote:
> On 06/09/17 at 10:29am, Dave Young wrote:
>> On 06/09/17 at 10:17am, Xunlei Pang wrote:
>>> S390 KEXEC_NOTE_BYTES is not used by note_buf_t as before, which
>>> is now defined as follows:
>>>     typedef u32 note_buf_t[CRASH_CORE_NOTE_BYTES/4];
>>> It was changed by the CONFIG_CRASH_CORE feature.
>>>
>>> This patch gets rid of all the old KEXEC_NOTE_BYTES stuff, and
>>> renames KEXEC_NOTE_BYTES to CRASH_CORE_NOTE_BYTES for S390.
>>>
>>> Fixes: 692f66f26a4c ("crash: move crashkernel parsing and vmcore related code under CONFIG_CRASH_CORE")
>>> Cc: Dave Young <dyoung at redhat.com>
>>> Cc: Dave Anderson <anderson at redhat.com>
>>> Cc: Hari Bathini <hbathini at linux.vnet.ibm.com>
>>> Cc: Gustavo Luiz Duarte <gustavold at linux.vnet.ibm.com>
>>> Signed-off-by: Xunlei Pang <xlpang at redhat.com>
>>> ---
>>>  arch/s390/include/asm/kexec.h |  2 +-
>>>  include/linux/crash_core.h    |  7 +++++++
>>>  include/linux/kexec.h         | 11 +----------
>>>  3 files changed, 9 insertions(+), 11 deletions(-)
>>>
>>> diff --git a/arch/s390/include/asm/kexec.h b/arch/s390/include/asm/kexec.h
>>> index 2f924bc..352deb8 100644
>>> --- a/arch/s390/include/asm/kexec.h
>>> +++ b/arch/s390/include/asm/kexec.h
>>> @@ -47,7 +47,7 @@
>>>   * Seven notes plus zero note at the end: prstatus, fpregset, timer,
>>>   * tod_cmp, tod_reg, control regs, and prefix
>>>   */
>>> -#define KEXEC_NOTE_BYTES \
>>> +#define CRASH_CORE_NOTE_BYTES \
>>>  	(ALIGN(sizeof(struct elf_note), 4) * 8 + \
>>>  	 ALIGN(sizeof("CORE"), 4) * 7 + \
>>>  	 ALIGN(sizeof(struct elf_prstatus), 4) + \
> I found that in mainline since below commit, above define should be
> useless, but if distribution with older kernel does need your fix, so in
> mainline the right fix should be dropping the s390 part about these
> macros usage.

Indeed, then I think we can remove this special definition of S390 to avoid confusion.

Regards,
Xunlei

>
> Anyway this need a comment from Michael.
>
> commit 8a07dd02d7615d91d65d6235f7232e3f9b5d347f
> Author: Martin Schwidefsky <schwidefsky at de.ibm.com>
> Date:   Wed Oct 14 15:53:06 2015 +0200
>
>     s390/kdump: remove code to create ELF notes in the crashed system
>     
>     The s390 architecture can store the CPU registers of the crashed
> system
>     after the kdump kernel has been started and this is the preferred
> way.
>     Remove the remaining code fragments that deal with storing CPU
> registers
>     while the crashed system is still active.
>     
>     Acked-by: Michael Holzheu <holzheu at linux.vnet.ibm.com>
>     Signed-off-by: Martin Schwidefsky <schwidefsky at de.ibm.com>
>
>
>>> diff --git a/include/linux/crash_core.h b/include/linux/crash_core.h
>>> index e9de6b4..dbc6e5c 100644
>>> --- a/include/linux/crash_core.h
>>> +++ b/include/linux/crash_core.h
>>> @@ -10,9 +10,16 @@
>>>  #define CRASH_CORE_NOTE_NAME_BYTES ALIGN(sizeof(CRASH_CORE_NOTE_NAME), 4)
>>>  #define CRASH_CORE_NOTE_DESC_BYTES ALIGN(sizeof(struct elf_prstatus), 4)
>>>  
>>> +/*
>>> + * The per-cpu notes area is a list of notes terminated by a "NULL"
>>> + * note header.  For kdump, the code in vmcore.c runs in the context
>>> + * of the second kernel to combine them into one note.
>>> + */
>>> +#ifndef CRASH_CORE_NOTE_BYTES
>>>  #define CRASH_CORE_NOTE_BYTES	   ((CRASH_CORE_NOTE_HEAD_BYTES * 2) +	\
>>>  				     CRASH_CORE_NOTE_NAME_BYTES +	\
>>>  				     CRASH_CORE_NOTE_DESC_BYTES)
>>> +#endif
>>>  
>>>  #define VMCOREINFO_BYTES	   PAGE_SIZE
>>>  #define VMCOREINFO_NOTE_NAME	   "VMCOREINFO"
>>> diff --git a/include/linux/kexec.h b/include/linux/kexec.h
>>> index 3ea8275..133df03 100644
>>> --- a/include/linux/kexec.h
>>> +++ b/include/linux/kexec.h
>>> @@ -14,7 +14,6 @@
>>>  
>>>  #if !defined(__ASSEMBLY__)
>>>  
>>> -#include <linux/crash_core.h>
>>>  #include <asm/io.h>
>>>  
>>>  #include <uapi/linux/kexec.h>
>>> @@ -25,6 +24,7 @@
>>>  #include <linux/ioport.h>
>>>  #include <linux/module.h>
>>>  #include <asm/kexec.h>
>>> +#include <linux/crash_core.h>
>>>  
>>>  /* Verify architecture specific macros are defined */
>>>  
>>> @@ -63,15 +63,6 @@
>>>  #define KEXEC_CORE_NOTE_NAME	CRASH_CORE_NOTE_NAME
>>>  
>>>  /*
>>> - * The per-cpu notes area is a list of notes terminated by a "NULL"
>>> - * note header.  For kdump, the code in vmcore.c runs in the context
>>> - * of the second kernel to combine them into one note.
>>> - */
>>> -#ifndef KEXEC_NOTE_BYTES
>>> -#define KEXEC_NOTE_BYTES	CRASH_CORE_NOTE_BYTES
>>> -#endif
>> It is still not clear how does s390 use the crash_notes except this macro.
>> But from code point of view we do need to update this as well after the
>> crash_core splitting.
>>
>> Acked-by: Dave Young <dyoung at redhat.com>
> Hold on the ack because of the new findings, wait for Michael's
> feedback.
>
> Thanks
> Dave



From bp at alien8.de  Sun Jun 11 12:44:11 2017
From: bp at alien8.de (Borislav Petkov)
Date: Sun, 11 Jun 2017 21:44:11 +0200
Subject: [PATCH v6 18/34] x86/efi: Update EFI pagetable creation to work
 with SME
In-Reply-To: <20170607191627.28645.4398.stgit@tlendack-t1.amdoffice.net>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
 <20170607191627.28645.4398.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170611194411.shbwwf34ftumipab@pd.tnic>

On Wed, Jun 07, 2017 at 02:16:27PM -0500, Tom Lendacky wrote:
> When SME is active, pagetable entries created for EFI need to have the
> encryption mask set as necessary.
> 
> When the new pagetable pages are allocated they are mapped encrypted. So,
> update the efi_pgt value that will be used in cr3 to include the encryption
> mask so that the PGD table can be read successfully. The pagetable mapping
> as well as the kernel are also added to the pagetable mapping as encrypted.
> All other EFI mappings are mapped decrypted (tables, etc.).
> 
> Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
> ---
>  arch/x86/platform/efi/efi_64.c |   15 +++++++++++----
>  1 file changed, 11 insertions(+), 4 deletions(-)

patches 15-18:

Reviewed-by: Borislav Petkov <bp at suse.de>

-- 
Regards/Gruss,
    Boris.

Good mailing practices for 400: avoid top-posting and trim the reply.


From thomas.lendacky at amd.com  Mon Jun 12 06:31:58 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Mon, 12 Jun 2017 08:31:58 -0500
Subject: [PATCH v6 14/34] x86/mm: Insure that boot memory areas are mapped
 properly
In-Reply-To: <20170610160119.bnx5ir5dj3i27igx@pd.tnic>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
 <20170607191539.28645.70161.stgit@tlendack-t1.amdoffice.net>
 <20170610160119.bnx5ir5dj3i27igx@pd.tnic>
Message-ID: <64d7cb4e-64fe-0882-ad17-fc3918c3a09a@amd.com>

On 6/10/2017 11:01 AM, Borislav Petkov wrote:
> On Wed, Jun 07, 2017 at 02:15:39PM -0500, Tom Lendacky wrote:
>> The boot data and command line data are present in memory in a decrypted
>> state and are copied early in the boot process.  The early page fault
>> support will map these areas as encrypted, so before attempting to copy
>> them, add decrypted mappings so the data is accessed properly when copied.
>>
>> For the initrd, encrypt this data in place. Since the future mapping of the
>> initrd area will be mapped as encrypted the data will be accessed properly.
>>
>> Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
>> ---
>>   arch/x86/include/asm/mem_encrypt.h |   11 +++++
>>   arch/x86/include/asm/pgtable.h     |    3 +
>>   arch/x86/kernel/head64.c           |   30 ++++++++++++--
>>   arch/x86/kernel/setup.c            |    9 ++++
>>   arch/x86/mm/mem_encrypt.c          |   77 ++++++++++++++++++++++++++++++++++++
>>   5 files changed, 126 insertions(+), 4 deletions(-)
> 
> Some cleanups ontop in case you get to send v7:

There will be a v7.

> 
> diff --git a/arch/x86/include/asm/mem_encrypt.h b/arch/x86/include/asm/mem_encrypt.h
> index 61a704945294..5959a42dd4d5 100644
> --- a/arch/x86/include/asm/mem_encrypt.h
> +++ b/arch/x86/include/asm/mem_encrypt.h
> @@ -45,13 +45,8 @@ static inline void __init sme_early_decrypt(resource_size_t paddr,
>   {
>   }
>   
> -static inline void __init sme_map_bootdata(char *real_mode_data)
> -{
> -}
> -
> -static inline void __init sme_unmap_bootdata(char *real_mode_data)
> -{
> -}
> +static inline void __init sme_map_bootdata(char *real_mode_data)	{ }
> +static inline void __init sme_unmap_bootdata(char *real_mode_data)	{ }
>   
>   static inline void __init sme_early_init(void)
>   {
> diff --git a/arch/x86/mm/mem_encrypt.c b/arch/x86/mm/mem_encrypt.c
> index 2321f05045e5..32ebbe0ab04d 100644
> --- a/arch/x86/mm/mem_encrypt.c
> +++ b/arch/x86/mm/mem_encrypt.c
> @@ -132,6 +132,10 @@ static void __init __sme_map_unmap_bootdata(char *real_mode_data, bool map)
>   	struct boot_params *boot_data;
>   	unsigned long cmdline_paddr;
>   
> +	/* If SME is not active, the bootdata is in the correct state */
> +	if (!sme_active())
> +		return;
> +
>   	__sme_early_map_unmap_mem(real_mode_data, sizeof(boot_params), map);
>   	boot_data = (struct boot_params *)real_mode_data;
>   
> @@ -142,40 +146,22 @@ static void __init __sme_map_unmap_bootdata(char *real_mode_data, bool map)
>   	cmdline_paddr = boot_data->hdr.cmd_line_ptr |
>   			((u64)boot_data->ext_cmd_line_ptr << 32);
>   
> -	if (cmdline_paddr)
> -		__sme_early_map_unmap_mem(__va(cmdline_paddr),
> -					  COMMAND_LINE_SIZE, map);
> +	if (!cmdline_paddr)
> +		return;
> +
> +	__sme_early_map_unmap_mem(__va(cmdline_paddr), COMMAND_LINE_SIZE, map);
> +
> +	sme_early_pgtable_flush();

Yup, overall it definitely simplifies things.

I have to call sme_early_pgtable_flush() even if cmdline_paddr is NULL,
so I'll either keep the if and have one flush at the end or I can move
the flush into __sme_early_map_unmap_mem(). I'm leaning towards the
latter.

Thanks,
Tom

>   }
>   
>   void __init sme_unmap_bootdata(char *real_mode_data)
>   {
> -	/* If SME is not active, the bootdata is in the correct state */
> -	if (!sme_active())
> -		return;
> -
> -	/*
> -	 * The bootdata and command line aren't needed anymore so clear
> -	 * any mapping of them.
> -	 */
>   	__sme_map_unmap_bootdata(real_mode_data, false);
> -
> -	sme_early_pgtable_flush();
>   }
>   
>   void __init sme_map_bootdata(char *real_mode_data)
>   {
> -	/* If SME is not active, the bootdata is in the correct state */
> -	if (!sme_active())
> -		return;
> -
> -	/*
> -	 * The bootdata and command line will not be encrypted, so they
> -	 * need to be mapped as decrypted memory so they can be copied
> -	 * properly.
> -	 */
>   	__sme_map_unmap_bootdata(real_mode_data, true);
> -
> -	sme_early_pgtable_flush();
>   }
>   
>   void __init sme_early_init(void)
> 


From colona at arista.com  Mon Jun 12 18:00:46 2017
From: colona at arista.com (Ivan Delalande)
Date: Mon, 12 Jun 2017 18:00:46 -0700
Subject: [PATCH] Add the --partial-dmesg option to dump only non-cleared dmesg
Message-ID: <20170613010046.GA25025@visor>

This option will make --dump-dmesg save only the part of the dmesg
buffer since it was last cleared on the crashed kernel (with
dmesg --clear and such) instead of the whole buffer since boot or
wrap-around. It works on kernels with commit f468908bb55a ("printk:
add clear_idx symbol to vmcoreinfo") merged in v4.6 and otherwise
will default to the regular behavior.

Signed-off-by: Ivan Delalande <colona at arista.com>
---
 makedumpfile.8 |  8 +++++++-
 makedumpfile.c | 21 +++++++++++++++++++--
 makedumpfile.h |  3 +++
 3 files changed, 29 insertions(+), 3 deletions(-)

diff --git a/makedumpfile.8 b/makedumpfile.8
index 9932364..76a8d6e 100644
--- a/makedumpfile.8
+++ b/makedumpfile.8
@@ -20,7 +20,7 @@ makedumpfile \- make a small dumpfile of kdump
 .br
 \fBmakedumpfile\fR    [\fIOPTION\fR] [\-\-xen-syms \fIXEN-SYMS\fR|\-\-xen-vmcoreinfo \fIVMCOREINFO\fR] \fIVMCORE\fR \fIDUMPFILE\fR
 .br
-\fBmakedumpfile\fR \-\-dump-dmesg [\-x \fIVMLINUX\fR|\-i \fIVMCOREINFO\fR] \fIVMCORE\fR \fILOGFILE\fR
+\fBmakedumpfile\fR \-\-dump-dmesg [\-\-partial-dmesg] [\-x \fIVMLINUX\fR|\-i \fIVMCOREINFO\fR] \fIVMCORE\fR \fILOGFILE\fR
 .br
 \fBmakedumpfile\fR    [\fIOPTION\fR] \-x \fIVMLINUX\fR \-\-diskset=\fIVMCORE1\fR \-\-diskset=\fIVMCORE2\fR [\-\-diskset=\fIVMCORE3\fR ..] \fIDUMPFILE\fR
 .br
@@ -586,6 +586,12 @@ it is necessary to specfiy [\-x \fIVMLINUX\fR] or [\-i \fIVMCOREINFO\fR].
 
 
 .TP
+\fB\-\-partial-dmesg\fR
+This option will make --dump-dmesg extract only dmesg logs since that buffer was
+last cleared on the crashed kernel, through "dmesg --clear" for example.
+
+
+.TP
 \fB\-\-mem-usage\fR
 This option is only for x86_64.
 This option is used to show the page numbers of current system in different
diff --git a/makedumpfile.c b/makedumpfile.c
index 95cc5ef..7c88203 100644
--- a/makedumpfile.c
+++ b/makedumpfile.c
@@ -1498,6 +1498,7 @@ get_symbol_info(void)
 	SYMBOL_INIT(log_buf_len, "log_buf_len");
 	SYMBOL_INIT(log_end, "log_end");
 	SYMBOL_INIT(log_first_idx, "log_first_idx");
+	SYMBOL_INIT(clear_idx, "clear_idx");
 	SYMBOL_INIT(log_next_idx, "log_next_idx");
 	SYMBOL_INIT(max_pfn, "max_pfn");
 	SYMBOL_INIT(modules, "modules");
@@ -2115,6 +2116,7 @@ write_vmcoreinfo_data(void)
 	WRITE_SYMBOL("log_buf_len", log_buf_len);
 	WRITE_SYMBOL("log_end", log_end);
 	WRITE_SYMBOL("log_first_idx", log_first_idx);
+	WRITE_SYMBOL("clear_idx", clear_idx);
 	WRITE_SYMBOL("log_next_idx", log_next_idx);
 	WRITE_SYMBOL("max_pfn", max_pfn);
 	WRITE_SYMBOL("high_memory", high_memory);
@@ -2509,6 +2511,7 @@ read_vmcoreinfo(void)
 	READ_SYMBOL("log_buf_len", log_buf_len);
 	READ_SYMBOL("log_end", log_end);
 	READ_SYMBOL("log_first_idx", log_first_idx);
+	READ_SYMBOL("clear_idx", clear_idx);
 	READ_SYMBOL("log_next_idx", log_next_idx);
 	READ_SYMBOL("max_pfn", max_pfn);
 	READ_SYMBOL("high_memory", high_memory);
@@ -5025,6 +5028,7 @@ dump_dmesg()
 	int log_buf_len, length_log, length_oldlog, ret = FALSE;
 	unsigned long index, log_buf, log_end;
 	unsigned int idx, log_first_idx, log_next_idx;
+	unsigned long long first_idx_sym;
 	unsigned long log_end_2_6_24;
 	unsigned      log_end_2_6_25;
 	char *log_buffer = NULL, *log_ptr = NULL;
@@ -5058,7 +5062,13 @@ dump_dmesg()
 			ERRMSG("Can't find variable-length record symbols");
 			return FALSE;
 		} else {
-			if (!readmem(VADDR, SYMBOL(log_first_idx), &log_first_idx,
+			if (info->flag_partial_dmesg
+			    && SYMBOL(clear_idx) != NOT_FOUND_SYMBOL)
+				first_idx_sym = SYMBOL(clear_idx);
+			else
+				first_idx_sym = SYMBOL(log_first_idx);
+
+			if (!readmem(VADDR, first_idx_sym, &log_first_idx,
 			    sizeof(log_first_idx))) {
 				ERRMSG("Can't get log_first_idx.\n");
 				return FALSE;
@@ -5102,7 +5112,10 @@ dump_dmesg()
 	DEBUG_MSG("log_buf       : %lx\n", log_buf);
 	DEBUG_MSG("log_end       : %lx\n", log_end);
 	DEBUG_MSG("log_buf_len   : %d\n", log_buf_len);
-	DEBUG_MSG("log_first_idx : %u\n", log_first_idx);
+	if (info->flag_partial_dmesg)
+		DEBUG_MSG("clear_idx : %u\n", log_first_idx);
+	else
+		DEBUG_MSG("log_first_idx : %u\n", log_first_idx);
 	DEBUG_MSG("log_next_idx  : %u\n", log_next_idx);
 
 	if ((log_buffer = malloc(log_buf_len)) == NULL) {
@@ -11022,6 +11035,7 @@ static struct option longopts[] = {
 	{"message-level", required_argument, NULL, OPT_MESSAGE_LEVEL},
 	{"vtop", required_argument, NULL, OPT_VTOP},
 	{"dump-dmesg", no_argument, NULL, OPT_DUMP_DMESG},
+	{"partial-dmesg", no_argument, NULL, OPT_PARTIAL_DMESG},
 	{"config", required_argument, NULL, OPT_CONFIG},
 	{"help", no_argument, NULL, OPT_HELP},
 	{"diskset", required_argument, NULL, OPT_DISKSET},
@@ -11135,6 +11149,9 @@ main(int argc, char *argv[])
 		case OPT_DUMP_DMESG:
 			info->flag_dmesg = 1;
 			break;
+		case OPT_PARTIAL_DMESG:
+			info->flag_partial_dmesg = 1;
+			break;
 		case OPT_MEM_USAGE:
 		       info->flag_mem_usage = 1;
 		       break;
diff --git a/makedumpfile.h b/makedumpfile.h
index 6ea8cc4..d3b7532 100644
--- a/makedumpfile.h
+++ b/makedumpfile.h
@@ -1241,6 +1241,7 @@ struct DumpInfo {
 	int		flag_force;	     /* overwrite existing stuff */
 	int		flag_exclude_xen_dom;/* exclude Domain-U from xen-kdump */
 	int             flag_dmesg;          /* dump the dmesg log out of the vmcore file */
+	int             flag_partial_dmesg;  /* dmesg dump only from the last cleared index*/
 	int             flag_mem_usage;  /*show the page number of memory in different use*/
 	int		flag_use_printk_log; /* did we read printk_log symbol name? */
 	int		flag_nospace;	     /* the flag of "No space on device" error */
@@ -1529,6 +1530,7 @@ struct symbol_table {
 	unsigned long long	log_buf_len;
 	unsigned long long	log_end;
 	unsigned long long	log_first_idx;
+	unsigned long long	clear_idx;
 	unsigned long long	log_next_idx;
 	unsigned long long	max_pfn;
 	unsigned long long	node_remap_start_vaddr;
@@ -2268,6 +2270,7 @@ struct elf_prstatus {
 #define OPT_SPLITBLOCK_SIZE	OPT_START+14
 #define OPT_WORKING_DIR         OPT_START+15
 #define OPT_NUM_THREADS	OPT_START+16
+#define OPT_PARTIAL_DMESG	OPT_START+17
 
 /*
  * Function Prototype.
-- 
2.13.1


-- 
Ivan Delalande
Kernel Team - Vancouver - #? A3Y20
Phone: 7342 | Mobile: 604 600 5313


From panand at redhat.com  Mon Jun 12 21:49:53 2017
From: panand at redhat.com (Pratyush Anand)
Date: Tue, 13 Jun 2017 10:19:53 +0530
Subject: [ANNOUNCE] kexec-tools 2.0.15-rc1
In-Reply-To: <20170609162815.GA21274@verge.net.au>
References: <20170609162815.GA21274@verge.net.au>
Message-ID: <6b2a836d-1dfa-d631-a597-430eb16ce9ef@redhat.com>



On Friday 09 June 2017 09:58 PM, Simon Horman wrote:
> Hi all,
>
> I am happy to announce the release of kexec-tools 2.0.15-rc1.
>
> This is an incremental feature pre-release.
>
> So long as no serious problems arise I intend to release kexec-tools 2.0.15
> in a weeks time. Testing of 2.0.15-rc1 would be greatly appreciated.

I tested it with one of the ARM64 platform and it worked fine.


>
> I do not have any outstanding changes for 2.0.15 at this time.
> And I would like to only accept bug fixes at this time and take take
> features patches once 2.0.15 has been released.
>
> The pre-release can be downloaded from kernel.org:
>
> 	https://kernel.org/pub/linux/utils/kernel/kexec/kexec-tools-2.0.15-rc1.tar.xz
> 	https://kernel.org/pub/linux/utils/kernel/kexec/
>
> I have also tagged it in git:
>
> 	https://git.kernel.org/pub/scm/utils/kernel/kexec/kexec-tools.git
>
> Thanks to everyone who has contributed to kexec.
>
>
> For reference the changes since v2.0.14 are:
>
> 302e1e362753 kexec-tools 2.0.15-rc1
> fab91841c717 Handle additional e820 memmap type strings
> c504ff5d85aa arm64: kdump: Add support for binary image files
> 5f955585c7c9 arm64: kdump: add DT properties to crash dump kernel's dtb
> defad947feff arm64: kdump: set up other segments
> 1591926df5a6 arm64: kdump: set up kernel image segment
> 0bd5219da953 arm64: kdump: add elf core header segment
> c0672c93edcb arm64: kdump: identify memory regions
> a17234fe94bc arm64: change return values on error to negative
> ef26cc33b8d6 arm64: identify PHYS_OFFSET correctly
> 325804055e99 kexec: generalize and rename get_kernel_stext_sym()
> c95df0e099b1 kexec: extend the semantics of kexec_iomem_for_each_line
> 59d3e5b5ad6f Fix broken Xen support in configure.ac
> 4a6d67d9e938 x86: Support large number of memory ranges
> 0516f46adbf3 crashdump: Remove stray get_crashkernel_region() declaration
> 82a49747e5ad ppc: Fix format warning with die()
> 1550f81bf188 x86/x86_64: Fix format warning with die()
> 47cc70157c66 Don't use %L width specifier with integer values
> 2f6f6d6fef78 vmcore-dmesg: Define _GNU_SOURCE
> 896fb2aa30c6 arm64: add uImage support
> a0c575793b86 uImage: use 'char *' instead of 'unsigned char *' for uImage_probe()
> f25146afc5a9 uImage: use 'char *' instead of 'unsigned char *' for uImage_load()
> b3d533c1f499 uImage: Add new IH_ARCH_xxx definitions
> 67234243bb91 uImage: Fix uImage_load() for little-endian machines
> 0cc1891c4dc8 uImage: fix realloc() pointer confusion
> ed15ba1b9977 build_mem_phdrs(): check if p_paddr is invalid
> 263e45ccf27b Only print debug message when failed to serach for kernel symbol from /proc/kallsyms
> 7dac152d5b47 gitignore: add two generated files in purgatory
> 05ae4fb2e354 crashdump/sh: Add get_crash_kernel_load_range() function
> 796f0ffa134d crashdump/s390: Add get_crash_kernel_load_range() function
> e4280e22c8c4 crashdump/ppc64: Add get_crash_kernel_load_range() function
> 7fc80cfcd913 crashdump/ppc: Add get_crash_kernel_load_range() function
> d2caf4c4c43b crashdump/mips: Add get_crash_kernel_load_range() function
> 14d71e51e5c9 crashdump/m68k: Add get_crash_kernel_load_range() function
> 5c80bd9be295 crashdump/ia64: Add get_crash_kernel_load_range() function
> b6af22826f60 crashdump/cris: Add get_crash_kernel_load_range() function
> d43610084164 crashdump/arm64: Add get_crash_kernel_load_range() function
> cfcf60c38182 crashdump/arm: Add get_crash_kernel_load_range() function
> 76b31203222a kexec: Add option to get crash kernel region size
> e49623b0787d purgatory: Add purgatory.map and purgatory.ro.sym to clean recipe
> ceedb33e6cd3 kexec: Remove redundant space from help message
> 2cf7cb9a6080 kexec: implemented XEN KEXEC STATUS to determine if an image is loaded
> 4eaa36cd01a9 alpha: add missing __NR_kexec_load definition
> 24aa2d93cac3 kexec: Increase the upper limit for RAM segments
> f63d8530b9b6 ppc64: Reduce number of ELF LOAD segments
> 9da19c0a6f49 kexec-tools 2.0.14.git
>
> _______________________________________________
> kexec mailing list
> kexec at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/kexec
>

~Pratyush


From ats-kumagai at wm.jp.nec.com  Mon Jun 12 23:21:19 2017
From: ats-kumagai at wm.jp.nec.com (Atsushi Kumagai)
Date: Tue, 13 Jun 2017 06:21:19 +0000
Subject: [makedumpfile PATCHv2 2/2] print_info: show the remaining time
 of dump progress
In-Reply-To: <817b653f-9191-70c2-21b9-6fba9a196399@redhat.com>
References: <1496822990-17848-1-git-send-email-piliu@redhat.com>
 <1496822990-17848-2-git-send-email-piliu@redhat.com>
 <817b653f-9191-70c2-21b9-6fba9a196399@redhat.com>
Message-ID: <0910DD04CBD6DE4193FCF86B9C00BE9701EDFCE4@BPXM01GP.gisp.nec.co.jp>

Hello Pingfan,

>On Wednesday 07 June 2017 01:39 PM, Pingfan Liu wrote:
>> If the dump progress is slow, it is better to show  an estimate of
>> remaining time of the progress to the user. The estimator simply uses
>> the average speed to work.
>
>Nice improvement.
>
>It could have been better to write about how does log looks before and after
>this patch.

I have no additional comments on v2 patch, so I'll merge a next version
immediately if you update the description.

Thanks,
Atsushi Kumagai

>>
>> Signed-off-by: Pingfan Liu <piliu at redhat.com>
>> ---
>> v1 -> v2: fold  eta string into eta_to_human_short()
>>
>>
>>  makedumpfile.c | 46 +++++++++++++++++++++++-----------------------
>>  print_info.c   | 41 +++++++++++++++++++++++++++++++++++------
>>  print_info.h   |  3 ++-
>>  3 files changed, 60 insertions(+), 30 deletions(-)
>>
>> diff --git a/makedumpfile.c b/makedumpfile.c
>> index 301772a..35a36ce 100644
>> --- a/makedumpfile.c
>> +++ b/makedumpfile.c
>> @@ -5175,7 +5175,7 @@ _exclude_free_page(struct cycle *cycle)
>>
>>  	for (num_nodes = 1; num_nodes <= vt.numnodes; num_nodes++) {
>>
>> -		print_progress(PROGRESS_FREE_PAGES, num_nodes - 1, vt.numnodes);
>> +		print_progress(PROGRESS_FREE_PAGES, num_nodes - 1, vt.numnodes, NULL);
>>
>>  		node_zones = pgdat + OFFSET(pglist_data.node_zones);
>>
>> @@ -5188,7 +5188,7 @@ _exclude_free_page(struct cycle *cycle)
>>  		for (i = 0; i < nr_zones; i++) {
>>
>>  			print_progress(PROGRESS_FREE_PAGES, i + nr_zones * (num_nodes - 1),
>> -					nr_zones * vt.numnodes);
>> +					nr_zones * vt.numnodes, NULL);
>>
>>  			zone = node_zones + (i * SIZE(zone));
>>  			if (!readmem(VADDR, zone + OFFSET(zone.spanned_pages),
>> @@ -5216,7 +5216,7 @@ _exclude_free_page(struct cycle *cycle)
>>  	/*
>>  	 * print [100 %]
>>  	 */
>> -	print_progress(PROGRESS_FREE_PAGES, vt.numnodes, vt.numnodes);
>> +	print_progress(PROGRESS_FREE_PAGES, vt.numnodes, vt.numnodes, NULL);
>>  	print_execution_time(PROGRESS_FREE_PAGES, &tv_start);
>>
>>  	return TRUE;
>> @@ -5403,7 +5403,7 @@ create_1st_bitmap_file(void)
>>  	for (i = 0; get_pt_load(i, &phys_start, &phys_end, NULL, NULL); i++) {
>>
>>  		if (!info->flag_mem_usage)
>> -			print_progress(PROGRESS_HOLES, i, num_pt_loads);
>> +			print_progress(PROGRESS_HOLES, i, num_pt_loads, NULL);
>>
>>  		pfn_start = paddr_to_pfn(phys_start);
>>  		pfn_end   = paddr_to_pfn(phys_end);
>> @@ -5422,7 +5422,7 @@ create_1st_bitmap_file(void)
>>  	 * print 100 %
>>  	 */
>>  	if (!info->flag_mem_usage) {
>> -		print_progress(PROGRESS_HOLES, info->max_mapnr, info->max_mapnr);
>> +		print_progress(PROGRESS_HOLES, info->max_mapnr, info->max_mapnr, NULL);
>>  		print_execution_time(PROGRESS_HOLES, &tv_start);
>>  	}
>>
>> @@ -5548,7 +5548,7 @@ create_bitmap_from_memhole(struct cycle *cycle, struct dump_bitmap *bitmap, int
>>  		pfn_start = MAX(paddr_to_pfn(phys_start), cycle->start_pfn);
>>  		pfn_end = MIN(paddr_to_pfn(phys_end), cycle->end_pfn);
>>
>> -		print_progress(PROGRESS_HOLES, i, num_pt_loads);
>> +		print_progress(PROGRESS_HOLES, i, num_pt_loads, NULL);
>>
>>  		if (pfn_start >= pfn_end)
>>  			continue;
>> @@ -5587,7 +5587,7 @@ create_bitmap_from_memhole(struct cycle *cycle, struct dump_bitmap *bitmap, int
>>  	/*
>>  	 * print 100 %
>>  	 */
>> -	print_progress(PROGRESS_HOLES, info->max_mapnr, info->max_mapnr);
>> +	print_progress(PROGRESS_HOLES, info->max_mapnr, info->max_mapnr, NULL);
>>  	print_execution_time(PROGRESS_HOLES, &tv_start);
>>
>>  	return TRUE;
>> @@ -5842,7 +5842,7 @@ exclude_unnecessary_pages(struct cycle *cycle)
>>  	for (mm = 0; mm < info->num_mem_map; mm++) {
>>
>>  		if (!info->flag_mem_usage)
>> -			print_progress(PROGRESS_UNN_PAGES, mm, info->num_mem_map);
>> +			print_progress(PROGRESS_UNN_PAGES, mm, info->num_mem_map, NULL);
>>
>>  		mmd = &info->mem_map_data[mm];
>>
>> @@ -5860,7 +5860,7 @@ exclude_unnecessary_pages(struct cycle *cycle)
>>  	 * print [100 %]
>>  	 */
>>  	if (!info->flag_mem_usage) {
>> -		print_progress(PROGRESS_UNN_PAGES, info->num_mem_map, info->num_mem_map);
>> +		print_progress(PROGRESS_UNN_PAGES, info->num_mem_map, info->num_mem_map, NULL);
>>  		print_execution_time(PROGRESS_UNN_PAGES, &tv_start);
>>  	}
>>
>> @@ -7334,7 +7334,7 @@ write_elf_pages_cyclic(struct cache_data *cd_header, struct cache_data *cd_page)
>>  				}
>>
>>  				if ((num_dumped % per) == 0)
>> -					print_progress(PROGRESS_COPY, num_dumped, num_dumpable);
>> +					print_progress(PROGRESS_COPY, num_dumped, num_dumpable, &tv_start);
>>
>>  				num_dumped++;
>>
>> @@ -7453,7 +7453,7 @@ write_elf_pages_cyclic(struct cache_data *cd_header, struct cache_data *cd_page)
>>  	/*
>>  	 * print [100 %]
>>  	 */
>> -	print_progress(PROGRESS_COPY, num_dumpable, num_dumpable);
>> +	print_progress(PROGRESS_COPY, num_dumpable, num_dumpable, &tv_start);
>>  	print_execution_time(PROGRESS_COPY, &tv_start);
>>  	PROGRESS_MSG("\n");
>>
>> @@ -7931,7 +7931,7 @@ write_kdump_pages_parallel_cyclic(struct cache_data *cd_header,
>>  		}
>>
>>  		if ((num_dumped % per) == 0)
>> -			print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable);
>> +			print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable, &tv_start);
>>
>>  		num_dumped++;
>>
>> @@ -7967,7 +7967,7 @@ finish:
>>  	/*
>>  	 * print [100 %]
>>  	 */
>> -	print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable);
>> +	print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable, &tv_start);
>>  	print_execution_time(PROGRESS_COPY, &tv_start);
>>  	PROGRESS_MSG("\n");
>>
>> @@ -8084,7 +8084,7 @@ write_kdump_pages_cyclic(struct cache_data *cd_header, struct cache_data *cd_pag
>>  	for (pfn = start_pfn; pfn < end_pfn; pfn++) {
>>
>>  		if ((num_dumped % per) == 0)
>> -			print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable);
>> +			print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable, &tv_start);
>>
>>  		/*
>>  		 * Check the excluded page.
>> @@ -8164,7 +8164,7 @@ out:
>>  		free(wrkmem);
>>  #endif
>>
>> -	print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable);
>> +	print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable, &tv_start);
>>  	print_execution_time(PROGRESS_COPY, &tv_start);
>>
>>  	return ret;
>> @@ -8622,7 +8622,7 @@ write_kdump_pages_and_bitmap_cyclic(struct cache_data *cd_header, struct cache_d
>>  	/*
>>  	 * print [100 %]
>>  	 */
>> -	print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable);
>> +	print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable, &tv_start);
>>  	print_execution_time(PROGRESS_COPY, &tv_start);
>>  	PROGRESS_MSG("\n");
>>
>> @@ -9216,7 +9216,7 @@ exclude_xen3_user_domain(void)
>>  	 */
>>  	for (i = 0; get_pt_load(i, &phys_start, &phys_end, NULL, NULL); i++) {
>>
>> -		print_progress(PROGRESS_XEN_DOMAIN, i, num_pt_loads);
>> +		print_progress(PROGRESS_XEN_DOMAIN, i, num_pt_loads, NULL);
>>
>>  		pfn     = paddr_to_pfn(phys_start);
>>  		pfn_end = paddr_to_pfn(phys_end);
>> @@ -9224,7 +9224,7 @@ exclude_xen3_user_domain(void)
>>
>>  		for (j = 0; pfn < pfn_end; pfn++, j++) {
>>  			print_progress(PROGRESS_XEN_DOMAIN, j + (size * i),
>> -					size * num_pt_loads);
>> +					size * num_pt_loads, NULL);
>>
>>  			if (!allocated_in_map(pfn)) {
>>  				clear_bit_on_2nd_bitmap(pfn, NULL);
>> @@ -9280,7 +9280,7 @@ exclude_xen4_user_domain(void)
>>  	 */
>>  	for (i = 0; get_pt_load(i, &phys_start, &phys_end, NULL, NULL); i++) {
>>
>> -		print_progress(PROGRESS_XEN_DOMAIN, i, num_pt_loads);
>> +		print_progress(PROGRESS_XEN_DOMAIN, i, num_pt_loads, NULL);
>>
>>  		pfn     = paddr_to_pfn(phys_start);
>>  		pfn_end = paddr_to_pfn(phys_end);
>> @@ -9288,7 +9288,7 @@ exclude_xen4_user_domain(void)
>>
>>  		for (j = 0; pfn < pfn_end; pfn++, j++) {
>>  			print_progress(PROGRESS_XEN_DOMAIN, j + (size * i),
>> -					size * num_pt_loads);
>> +					size * num_pt_loads, NULL);
>>
>>  			page_info_addr = info->frame_table_vaddr + pfn * SIZE(page_info);
>>  			if (!readmem(VADDR_XEN,
>> @@ -9356,7 +9356,7 @@ exclude_xen_user_domain(void)
>>  	/*
>>  	 * print [100 %]
>>  	 */
>> -	print_progress(PROGRESS_XEN_DOMAIN, 1, 1);
>> +	print_progress(PROGRESS_XEN_DOMAIN, 1, 1, NULL);
>>  	print_execution_time(PROGRESS_XEN_DOMAIN, &tv_start);
>>
>>  	return ret;
>> @@ -10342,7 +10342,7 @@ reassemble_kdump_pages(void)
>>
>>  			num_dumped++;
>>
>> -			print_progress(PROGRESS_COPY, num_dumped, num_dumpable);
>> +			print_progress(PROGRESS_COPY, num_dumped, num_dumpable, &tv_start);
>>
>>  			if (lseek(fd, offset_ph_org, SEEK_SET) < 0) {
>>  				ERRMSG("Can't seek a file(%s). %s\n",
>> @@ -10439,7 +10439,7 @@ reassemble_kdump_pages(void)
>>  						    size_eraseinfo))
>>  			goto out;
>>  	}
>> -	print_progress(PROGRESS_COPY, num_dumpable, num_dumpable);
>> +	print_progress(PROGRESS_COPY, num_dumpable, num_dumpable, &tv_start);
>>  	print_execution_time(PROGRESS_COPY, &tv_start);
>>
>>  	ret = TRUE;
>> diff --git a/print_info.c b/print_info.c
>> index 79dfe5e..4e0bb5c 100644
>> --- a/print_info.c
>> +++ b/print_info.c
>> @@ -15,8 +15,9 @@
>>   */
>>  #include "print_info.h"
>>  #include <time.h>
>> +#include <string.h>
>>
>> -#define PROGRESS_MAXLEN		"35"
>> +#define PROGRESS_MAXLEN		"50"
>>
>>  int message_level;
>>  int flag_strerr_message;
>> @@ -350,14 +351,36 @@ static void calc_delta(struct timeval *tv_start, struct timeval *delta)
>>  	}
>>  }
>>
>> +/* produce less than 12 bytes on msg */
>> +static int eta_to_human_short (int secs, char* msg)
>> +{
>> +	strcpy(msg, "eta: ");
>> +	msg += strlen("eta: ");
>> +	if (secs < 100)
>> +		sprintf(msg, "%ds", secs);
>> +	else if (secs < 100 * 60)
>> +		sprintf(msg, "%dm%ds", secs / 60, secs % 60);
>> +	else if (secs < 48 * 3600)
>> +		sprintf(msg, "%dh%dm", secs / 3600, (secs / 60) % 60);
>> +	else if (secs < 100 * 86400)
>> +		sprintf(msg, "%dd%dh", secs / 86400, (secs / 3600) % 24);
>> +	else
>> +		sprintf(msg, ">2day");
>> +	return 0;
>> +}
>> +
>> +
>>  void
>> -print_progress(const char *msg, unsigned long current, unsigned long end)
>> +print_progress(const char *msg, unsigned long current, unsigned long end, struct timeval *start)
>>  {
>>  	float progress;
>>  	time_t tm;
>>  	static time_t last_time = 0;
>>  	static unsigned int lapse = 0;
>>  	static const char *spinner = "/|\\-";
>> +	struct timeval delta;
>> +	double eta;
>> +	char eta_msg[16] = " ";
>>
>>  	if (current < end) {
>>  		tm = time(NULL);
>> @@ -368,13 +391,19 @@ print_progress(const char *msg, unsigned long current, unsigned long end)
>>  	} else
>>  		progress = 100;
>>
>> +	if (start != NULL) {
>> +		calc_delta(start, &delta);
>> +		eta = delta.tv_sec + delta.tv_usec/1e6;
>
>space around /
>and better to write hex value as 0x1e6
>
>Sorry, I could not understand what is this magic number 1e6?
>
>
>> +		eta = (100 - progress) * eta/progress;
>
>space around /
>
>> +		eta_to_human_short(eta, eta_msg);
>> +	}
>>  	if (flag_ignore_r_char) {
>> -		PROGRESS_MSG("%-" PROGRESS_MAXLEN "s: [%5.1f %%] %c\n",
>> -			     msg, progress, spinner[lapse % 4]);
>> +		PROGRESS_MSG("%-" PROGRESS_MAXLEN "s: [%5.1f %%] %c  %16s\n",
>> +			     msg, progress, spinner[lapse % 4], eta_msg);
>>  	} else {
>>  		PROGRESS_MSG("\r");
>> -		PROGRESS_MSG("%-" PROGRESS_MAXLEN "s: [%5.1f %%] %c",
>> -			     msg, progress, spinner[lapse % 4]);
>> +		PROGRESS_MSG("%-" PROGRESS_MAXLEN "s: [%5.1f %%] %c  %16s",
>> +			     msg, progress, spinner[lapse % 4], eta_msg);
>>  	}
>>  	lapse++;
>>  }
>> diff --git a/print_info.h b/print_info.h
>> index 01e3706..1ce3593 100644
>> --- a/print_info.h
>> +++ b/print_info.h
>> @@ -25,7 +25,8 @@ extern int flag_ignore_r_char;
>>
>>  void show_version(void);
>>  void print_usage(void);
>> -void print_progress(const char *msg, unsigned long current, unsigned long end);
>> +void print_progress(const char *msg, unsigned long current, unsigned long end, struct timeval *start);
>> +
>>  void print_execution_time(char *step_name, struct timeval *tv_start);
>>
>>  /*
>>
>
>~Pratyush
>
>_______________________________________________
>kexec mailing list
>kexec at lists.infradead.org
>http://lists.infradead.org/mailman/listinfo/kexec




From ats-kumagai at wm.jp.nec.com  Mon Jun 12 23:20:57 2017
From: ats-kumagai at wm.jp.nec.com (Atsushi Kumagai)
Date: Tue, 13 Jun 2017 06:20:57 +0000
Subject: [PATCH] Add the --partial-dmesg option to dump only non-cleared
 dmesg
In-Reply-To: <20170613010046.GA25025@visor>
References: <20170613010046.GA25025@visor>
Message-ID: <0910DD04CBD6DE4193FCF86B9C00BE9701EDFCCA@BPXM01GP.gisp.nec.co.jp>

Hello Ivan,

>This option will make --dump-dmesg save only the part of the dmesg
>buffer since it was last cleared on the crashed kernel (with
>dmesg --clear and such) instead of the whole buffer since boot or
>wrap-around. It works on kernels with commit f468908bb55a ("printk:
>add clear_idx symbol to vmcoreinfo") merged in v4.6 and otherwise
>will default to the regular behavior.

It works fine, thanks. 
I think check_param_for_creating_dumpfile() should be modified
to reject a single --partial-dmesg, it should be always specified
with --dump-dmesg.

Thanks,
Atsushi Kumagai

>Signed-off-by: Ivan Delalande <colona at arista.com>
>---
> makedumpfile.8 |  8 +++++++-
> makedumpfile.c | 21 +++++++++++++++++++--
> makedumpfile.h |  3 +++
> 3 files changed, 29 insertions(+), 3 deletions(-)
>
>diff --git a/makedumpfile.8 b/makedumpfile.8
>index 9932364..76a8d6e 100644
>--- a/makedumpfile.8
>+++ b/makedumpfile.8
>@@ -20,7 +20,7 @@ makedumpfile \- make a small dumpfile of kdump
> .br
> \fBmakedumpfile\fR    [\fIOPTION\fR] [\-\-xen-syms \fIXEN-SYMS\fR|\-\-xen-vmcoreinfo \fIVMCOREINFO\fR] \fIVMCORE\fR
>\fIDUMPFILE\fR
> .br
>-\fBmakedumpfile\fR \-\-dump-dmesg [\-x \fIVMLINUX\fR|\-i \fIVMCOREINFO\fR] \fIVMCORE\fR \fILOGFILE\fR
>+\fBmakedumpfile\fR \-\-dump-dmesg [\-\-partial-dmesg] [\-x \fIVMLINUX\fR|\-i \fIVMCOREINFO\fR] \fIVMCORE\fR
>\fILOGFILE\fR
> .br
> \fBmakedumpfile\fR    [\fIOPTION\fR] \-x \fIVMLINUX\fR \-\-diskset=\fIVMCORE1\fR \-\-diskset=\fIVMCORE2\fR
>[\-\-diskset=\fIVMCORE3\fR ..] \fIDUMPFILE\fR
> .br
>@@ -586,6 +586,12 @@ it is necessary to specfiy [\-x \fIVMLINUX\fR] or [\-i \fIVMCOREINFO\fR].
>
>
> .TP
>+\fB\-\-partial-dmesg\fR
>+This option will make --dump-dmesg extract only dmesg logs since that buffer was
>+last cleared on the crashed kernel, through "dmesg --clear" for example.
>+
>+
>+.TP
> \fB\-\-mem-usage\fR
> This option is only for x86_64.
> This option is used to show the page numbers of current system in different
>diff --git a/makedumpfile.c b/makedumpfile.c
>index 95cc5ef..7c88203 100644
>--- a/makedumpfile.c
>+++ b/makedumpfile.c
>@@ -1498,6 +1498,7 @@ get_symbol_info(void)
> 	SYMBOL_INIT(log_buf_len, "log_buf_len");
> 	SYMBOL_INIT(log_end, "log_end");
> 	SYMBOL_INIT(log_first_idx, "log_first_idx");
>+	SYMBOL_INIT(clear_idx, "clear_idx");
> 	SYMBOL_INIT(log_next_idx, "log_next_idx");
> 	SYMBOL_INIT(max_pfn, "max_pfn");
> 	SYMBOL_INIT(modules, "modules");
>@@ -2115,6 +2116,7 @@ write_vmcoreinfo_data(void)
> 	WRITE_SYMBOL("log_buf_len", log_buf_len);
> 	WRITE_SYMBOL("log_end", log_end);
> 	WRITE_SYMBOL("log_first_idx", log_first_idx);
>+	WRITE_SYMBOL("clear_idx", clear_idx);
> 	WRITE_SYMBOL("log_next_idx", log_next_idx);
> 	WRITE_SYMBOL("max_pfn", max_pfn);
> 	WRITE_SYMBOL("high_memory", high_memory);
>@@ -2509,6 +2511,7 @@ read_vmcoreinfo(void)
> 	READ_SYMBOL("log_buf_len", log_buf_len);
> 	READ_SYMBOL("log_end", log_end);
> 	READ_SYMBOL("log_first_idx", log_first_idx);
>+	READ_SYMBOL("clear_idx", clear_idx);
> 	READ_SYMBOL("log_next_idx", log_next_idx);
> 	READ_SYMBOL("max_pfn", max_pfn);
> 	READ_SYMBOL("high_memory", high_memory);
>@@ -5025,6 +5028,7 @@ dump_dmesg()
> 	int log_buf_len, length_log, length_oldlog, ret = FALSE;
> 	unsigned long index, log_buf, log_end;
> 	unsigned int idx, log_first_idx, log_next_idx;
>+	unsigned long long first_idx_sym;
> 	unsigned long log_end_2_6_24;
> 	unsigned      log_end_2_6_25;
> 	char *log_buffer = NULL, *log_ptr = NULL;
>@@ -5058,7 +5062,13 @@ dump_dmesg()
> 			ERRMSG("Can't find variable-length record symbols");
> 			return FALSE;
> 		} else {
>-			if (!readmem(VADDR, SYMBOL(log_first_idx), &log_first_idx,
>+			if (info->flag_partial_dmesg
>+			    && SYMBOL(clear_idx) != NOT_FOUND_SYMBOL)
>+				first_idx_sym = SYMBOL(clear_idx);
>+			else
>+				first_idx_sym = SYMBOL(log_first_idx);
>+
>+			if (!readmem(VADDR, first_idx_sym, &log_first_idx,
> 			    sizeof(log_first_idx))) {
> 				ERRMSG("Can't get log_first_idx.\n");
> 				return FALSE;
>@@ -5102,7 +5112,10 @@ dump_dmesg()
> 	DEBUG_MSG("log_buf       : %lx\n", log_buf);
> 	DEBUG_MSG("log_end       : %lx\n", log_end);
> 	DEBUG_MSG("log_buf_len   : %d\n", log_buf_len);
>-	DEBUG_MSG("log_first_idx : %u\n", log_first_idx);
>+	if (info->flag_partial_dmesg)
>+		DEBUG_MSG("clear_idx : %u\n", log_first_idx);
>+	else
>+		DEBUG_MSG("log_first_idx : %u\n", log_first_idx);
> 	DEBUG_MSG("log_next_idx  : %u\n", log_next_idx);
>
> 	if ((log_buffer = malloc(log_buf_len)) == NULL) {
>@@ -11022,6 +11035,7 @@ static struct option longopts[] = {
> 	{"message-level", required_argument, NULL, OPT_MESSAGE_LEVEL},
> 	{"vtop", required_argument, NULL, OPT_VTOP},
> 	{"dump-dmesg", no_argument, NULL, OPT_DUMP_DMESG},
>+	{"partial-dmesg", no_argument, NULL, OPT_PARTIAL_DMESG},
> 	{"config", required_argument, NULL, OPT_CONFIG},
> 	{"help", no_argument, NULL, OPT_HELP},
> 	{"diskset", required_argument, NULL, OPT_DISKSET},
>@@ -11135,6 +11149,9 @@ main(int argc, char *argv[])
> 		case OPT_DUMP_DMESG:
> 			info->flag_dmesg = 1;
> 			break;
>+		case OPT_PARTIAL_DMESG:
>+			info->flag_partial_dmesg = 1;
>+			break;
> 		case OPT_MEM_USAGE:
> 		       info->flag_mem_usage = 1;
> 		       break;
>diff --git a/makedumpfile.h b/makedumpfile.h
>index 6ea8cc4..d3b7532 100644
>--- a/makedumpfile.h
>+++ b/makedumpfile.h
>@@ -1241,6 +1241,7 @@ struct DumpInfo {
> 	int		flag_force;	     /* overwrite existing stuff */
> 	int		flag_exclude_xen_dom;/* exclude Domain-U from xen-kdump */
> 	int             flag_dmesg;          /* dump the dmesg log out of the vmcore file */
>+	int             flag_partial_dmesg;  /* dmesg dump only from the last cleared index*/
> 	int             flag_mem_usage;  /*show the page number of memory in different use*/
> 	int		flag_use_printk_log; /* did we read printk_log symbol name? */
> 	int		flag_nospace;	     /* the flag of "No space on device" error */
>@@ -1529,6 +1530,7 @@ struct symbol_table {
> 	unsigned long long	log_buf_len;
> 	unsigned long long	log_end;
> 	unsigned long long	log_first_idx;
>+	unsigned long long	clear_idx;
> 	unsigned long long	log_next_idx;
> 	unsigned long long	max_pfn;
> 	unsigned long long	node_remap_start_vaddr;
>@@ -2268,6 +2270,7 @@ struct elf_prstatus {
> #define OPT_SPLITBLOCK_SIZE	OPT_START+14
> #define OPT_WORKING_DIR         OPT_START+15
> #define OPT_NUM_THREADS	OPT_START+16
>+#define OPT_PARTIAL_DMESG	OPT_START+17
>
> /*
>  * Function Prototype.
>--
>2.13.1
>
>
>--
>Ivan Delalande
>Kernel Team - Vancouver - #? A3Y20
>Phone: 7342 | Mobile: 604 600 5313


From piliu at redhat.com  Tue Jun 13 00:36:15 2017
From: piliu at redhat.com (Pingfan Liu)
Date: Tue, 13 Jun 2017 03:36:15 -0400 (EDT)
Subject: [makedumpfile PATCHv2 2/2] print_info: show the remaining time
 of dump progress
In-Reply-To: <817b653f-9191-70c2-21b9-6fba9a196399@redhat.com>
References: <1496822990-17848-1-git-send-email-piliu@redhat.com>
 <1496822990-17848-2-git-send-email-piliu@redhat.com>
 <817b653f-9191-70c2-21b9-6fba9a196399@redhat.com>
Message-ID: <102818823.21629841.1497339375650.JavaMail.zimbra@redhat.com>





----- Original Message -----
> From: "Pratyush Anand" <panand at redhat.com>
> To: kexec at lists.infradead.org
> Sent: Thursday, June 8, 2017 3:09:46 PM
> Subject: Re: [makedumpfile PATCHv2 2/2] print_info: show the remaining time of dump progress
> 
> 
> 
> On Wednesday 07 June 2017 01:39 PM, Pingfan Liu wrote:
> > If the dump progress is slow, it is better to show  an estimate of
> > remaining time of the progress to the user. The estimator simply uses
> > the average speed to work.
> 
> Nice improvement.
> 
> It could have been better to write about how does log looks before and after
> this patch.
>
The progress bar looks like: Copying data                                      : [  1.8 %] /          eta: 39s
I will update the commit log.
> >
> > Signed-off-by: Pingfan Liu <piliu at redhat.com>
> > ---
> > v1 -> v2: fold  eta string into eta_to_human_short()
> >
> >
> >  makedumpfile.c | 46 +++++++++++++++++++++++-----------------------
> >  print_info.c   | 41 +++++++++++++++++++++++++++++++++++------
> >  print_info.h   |  3 ++-
> >  3 files changed, 60 insertions(+), 30 deletions(-)
> >
> > diff --git a/makedumpfile.c b/makedumpfile.c
> > index 301772a..35a36ce 100644
> > --- a/makedumpfile.c
> > +++ b/makedumpfile.c
> > @@ -5175,7 +5175,7 @@ _exclude_free_page(struct cycle *cycle)
> >
> >  	for (num_nodes = 1; num_nodes <= vt.numnodes; num_nodes++) {
> >
> > -		print_progress(PROGRESS_FREE_PAGES, num_nodes - 1, vt.numnodes);
> > +		print_progress(PROGRESS_FREE_PAGES, num_nodes - 1, vt.numnodes, NULL);
> >
> >  		node_zones = pgdat + OFFSET(pglist_data.node_zones);
> >
> > @@ -5188,7 +5188,7 @@ _exclude_free_page(struct cycle *cycle)
> >  		for (i = 0; i < nr_zones; i++) {
> >
> >  			print_progress(PROGRESS_FREE_PAGES, i + nr_zones * (num_nodes - 1),
> > -					nr_zones * vt.numnodes);
> > +					nr_zones * vt.numnodes, NULL);
> >
> >  			zone = node_zones + (i * SIZE(zone));
> >  			if (!readmem(VADDR, zone + OFFSET(zone.spanned_pages),
> > @@ -5216,7 +5216,7 @@ _exclude_free_page(struct cycle *cycle)
> >  	/*
> >  	 * print [100 %]
> >  	 */
> > -	print_progress(PROGRESS_FREE_PAGES, vt.numnodes, vt.numnodes);
> > +	print_progress(PROGRESS_FREE_PAGES, vt.numnodes, vt.numnodes, NULL);
> >  	print_execution_time(PROGRESS_FREE_PAGES, &tv_start);
> >
> >  	return TRUE;
> > @@ -5403,7 +5403,7 @@ create_1st_bitmap_file(void)
> >  	for (i = 0; get_pt_load(i, &phys_start, &phys_end, NULL, NULL); i++) {
> >
> >  		if (!info->flag_mem_usage)
> > -			print_progress(PROGRESS_HOLES, i, num_pt_loads);
> > +			print_progress(PROGRESS_HOLES, i, num_pt_loads, NULL);
> >
> >  		pfn_start = paddr_to_pfn(phys_start);
> >  		pfn_end   = paddr_to_pfn(phys_end);
> > @@ -5422,7 +5422,7 @@ create_1st_bitmap_file(void)
> >  	 * print 100 %
> >  	 */
> >  	if (!info->flag_mem_usage) {
> > -		print_progress(PROGRESS_HOLES, info->max_mapnr, info->max_mapnr);
> > +		print_progress(PROGRESS_HOLES, info->max_mapnr, info->max_mapnr, NULL);
> >  		print_execution_time(PROGRESS_HOLES, &tv_start);
> >  	}
> >
> > @@ -5548,7 +5548,7 @@ create_bitmap_from_memhole(struct cycle *cycle,
> > struct dump_bitmap *bitmap, int
> >  		pfn_start = MAX(paddr_to_pfn(phys_start), cycle->start_pfn);
> >  		pfn_end = MIN(paddr_to_pfn(phys_end), cycle->end_pfn);
> >
> > -		print_progress(PROGRESS_HOLES, i, num_pt_loads);
> > +		print_progress(PROGRESS_HOLES, i, num_pt_loads, NULL);
> >
> >  		if (pfn_start >= pfn_end)
> >  			continue;
> > @@ -5587,7 +5587,7 @@ create_bitmap_from_memhole(struct cycle *cycle,
> > struct dump_bitmap *bitmap, int
> >  	/*
> >  	 * print 100 %
> >  	 */
> > -	print_progress(PROGRESS_HOLES, info->max_mapnr, info->max_mapnr);
> > +	print_progress(PROGRESS_HOLES, info->max_mapnr, info->max_mapnr, NULL);
> >  	print_execution_time(PROGRESS_HOLES, &tv_start);
> >
> >  	return TRUE;
> > @@ -5842,7 +5842,7 @@ exclude_unnecessary_pages(struct cycle *cycle)
> >  	for (mm = 0; mm < info->num_mem_map; mm++) {
> >
> >  		if (!info->flag_mem_usage)
> > -			print_progress(PROGRESS_UNN_PAGES, mm, info->num_mem_map);
> > +			print_progress(PROGRESS_UNN_PAGES, mm, info->num_mem_map, NULL);
> >
> >  		mmd = &info->mem_map_data[mm];
> >
> > @@ -5860,7 +5860,7 @@ exclude_unnecessary_pages(struct cycle *cycle)
> >  	 * print [100 %]
> >  	 */
> >  	if (!info->flag_mem_usage) {
> > -		print_progress(PROGRESS_UNN_PAGES, info->num_mem_map,
> > info->num_mem_map);
> > +		print_progress(PROGRESS_UNN_PAGES, info->num_mem_map, info->num_mem_map,
> > NULL);
> >  		print_execution_time(PROGRESS_UNN_PAGES, &tv_start);
> >  	}
> >
> > @@ -7334,7 +7334,7 @@ write_elf_pages_cyclic(struct cache_data *cd_header,
> > struct cache_data *cd_page)
> >  				}
> >
> >  				if ((num_dumped % per) == 0)
> > -					print_progress(PROGRESS_COPY, num_dumped, num_dumpable);
> > +					print_progress(PROGRESS_COPY, num_dumped, num_dumpable, &tv_start);
> >
> >  				num_dumped++;
> >
> > @@ -7453,7 +7453,7 @@ write_elf_pages_cyclic(struct cache_data *cd_header,
> > struct cache_data *cd_page)
> >  	/*
> >  	 * print [100 %]
> >  	 */
> > -	print_progress(PROGRESS_COPY, num_dumpable, num_dumpable);
> > +	print_progress(PROGRESS_COPY, num_dumpable, num_dumpable, &tv_start);
> >  	print_execution_time(PROGRESS_COPY, &tv_start);
> >  	PROGRESS_MSG("\n");
> >
> > @@ -7931,7 +7931,7 @@ write_kdump_pages_parallel_cyclic(struct cache_data
> > *cd_header,
> >  		}
> >
> >  		if ((num_dumped % per) == 0)
> > -			print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable);
> > +			print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable,
> > &tv_start);
> >
> >  		num_dumped++;
> >
> > @@ -7967,7 +7967,7 @@ finish:
> >  	/*
> >  	 * print [100 %]
> >  	 */
> > -	print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable);
> > +	print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable, &tv_start);
> >  	print_execution_time(PROGRESS_COPY, &tv_start);
> >  	PROGRESS_MSG("\n");
> >
> > @@ -8084,7 +8084,7 @@ write_kdump_pages_cyclic(struct cache_data
> > *cd_header, struct cache_data *cd_pag
> >  	for (pfn = start_pfn; pfn < end_pfn; pfn++) {
> >
> >  		if ((num_dumped % per) == 0)
> > -			print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable);
> > +			print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable,
> > &tv_start);
> >
> >  		/*
> >  		 * Check the excluded page.
> > @@ -8164,7 +8164,7 @@ out:
> >  		free(wrkmem);
> >  #endif
> >
> > -	print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable);
> > +	print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable, &tv_start);
> >  	print_execution_time(PROGRESS_COPY, &tv_start);
> >
> >  	return ret;
> > @@ -8622,7 +8622,7 @@ write_kdump_pages_and_bitmap_cyclic(struct cache_data
> > *cd_header, struct cache_d
> >  	/*
> >  	 * print [100 %]
> >  	 */
> > -	print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable);
> > +	print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable, &tv_start);
> >  	print_execution_time(PROGRESS_COPY, &tv_start);
> >  	PROGRESS_MSG("\n");
> >
> > @@ -9216,7 +9216,7 @@ exclude_xen3_user_domain(void)
> >  	 */
> >  	for (i = 0; get_pt_load(i, &phys_start, &phys_end, NULL, NULL); i++) {
> >
> > -		print_progress(PROGRESS_XEN_DOMAIN, i, num_pt_loads);
> > +		print_progress(PROGRESS_XEN_DOMAIN, i, num_pt_loads, NULL);
> >
> >  		pfn     = paddr_to_pfn(phys_start);
> >  		pfn_end = paddr_to_pfn(phys_end);
> > @@ -9224,7 +9224,7 @@ exclude_xen3_user_domain(void)
> >
> >  		for (j = 0; pfn < pfn_end; pfn++, j++) {
> >  			print_progress(PROGRESS_XEN_DOMAIN, j + (size * i),
> > -					size * num_pt_loads);
> > +					size * num_pt_loads, NULL);
> >
> >  			if (!allocated_in_map(pfn)) {
> >  				clear_bit_on_2nd_bitmap(pfn, NULL);
> > @@ -9280,7 +9280,7 @@ exclude_xen4_user_domain(void)
> >  	 */
> >  	for (i = 0; get_pt_load(i, &phys_start, &phys_end, NULL, NULL); i++) {
> >
> > -		print_progress(PROGRESS_XEN_DOMAIN, i, num_pt_loads);
> > +		print_progress(PROGRESS_XEN_DOMAIN, i, num_pt_loads, NULL);
> >
> >  		pfn     = paddr_to_pfn(phys_start);
> >  		pfn_end = paddr_to_pfn(phys_end);
> > @@ -9288,7 +9288,7 @@ exclude_xen4_user_domain(void)
> >
> >  		for (j = 0; pfn < pfn_end; pfn++, j++) {
> >  			print_progress(PROGRESS_XEN_DOMAIN, j + (size * i),
> > -					size * num_pt_loads);
> > +					size * num_pt_loads, NULL);
> >
> >  			page_info_addr = info->frame_table_vaddr + pfn * SIZE(page_info);
> >  			if (!readmem(VADDR_XEN,
> > @@ -9356,7 +9356,7 @@ exclude_xen_user_domain(void)
> >  	/*
> >  	 * print [100 %]
> >  	 */
> > -	print_progress(PROGRESS_XEN_DOMAIN, 1, 1);
> > +	print_progress(PROGRESS_XEN_DOMAIN, 1, 1, NULL);
> >  	print_execution_time(PROGRESS_XEN_DOMAIN, &tv_start);
> >
> >  	return ret;
> > @@ -10342,7 +10342,7 @@ reassemble_kdump_pages(void)
> >
> >  			num_dumped++;
> >
> > -			print_progress(PROGRESS_COPY, num_dumped, num_dumpable);
> > +			print_progress(PROGRESS_COPY, num_dumped, num_dumpable, &tv_start);
> >
> >  			if (lseek(fd, offset_ph_org, SEEK_SET) < 0) {
> >  				ERRMSG("Can't seek a file(%s). %s\n",
> > @@ -10439,7 +10439,7 @@ reassemble_kdump_pages(void)
> >  						    size_eraseinfo))
> >  			goto out;
> >  	}
> > -	print_progress(PROGRESS_COPY, num_dumpable, num_dumpable);
> > +	print_progress(PROGRESS_COPY, num_dumpable, num_dumpable, &tv_start);
> >  	print_execution_time(PROGRESS_COPY, &tv_start);
> >
> >  	ret = TRUE;
> > diff --git a/print_info.c b/print_info.c
> > index 79dfe5e..4e0bb5c 100644
> > --- a/print_info.c
> > +++ b/print_info.c
> > @@ -15,8 +15,9 @@
> >   */
> >  #include "print_info.h"
> >  #include <time.h>
> > +#include <string.h>
> >
> > -#define PROGRESS_MAXLEN		"35"
> > +#define PROGRESS_MAXLEN		"50"
> >
> >  int message_level;
> >  int flag_strerr_message;
> > @@ -350,14 +351,36 @@ static void calc_delta(struct timeval *tv_start,
> > struct timeval *delta)
> >  	}
> >  }
> >
> > +/* produce less than 12 bytes on msg */
> > +static int eta_to_human_short (int secs, char* msg)
> > +{
> > +	strcpy(msg, "eta: ");
> > +	msg += strlen("eta: ");
> > +	if (secs < 100)
> > +		sprintf(msg, "%ds", secs);
> > +	else if (secs < 100 * 60)
> > +		sprintf(msg, "%dm%ds", secs / 60, secs % 60);
> > +	else if (secs < 48 * 3600)
> > +		sprintf(msg, "%dh%dm", secs / 3600, (secs / 60) % 60);
> > +	else if (secs < 100 * 86400)
> > +		sprintf(msg, "%dd%dh", secs / 86400, (secs / 3600) % 24);
> > +	else
> > +		sprintf(msg, ">2day");
> > +	return 0;
> > +}
> > +
> > +
> >  void
> > -print_progress(const char *msg, unsigned long current, unsigned long end)
> > +print_progress(const char *msg, unsigned long current, unsigned long end,
> > struct timeval *start)
> >  {
> >  	float progress;
> >  	time_t tm;
> >  	static time_t last_time = 0;
> >  	static unsigned int lapse = 0;
> >  	static const char *spinner = "/|\\-";
> > +	struct timeval delta;
> > +	double eta;
> > +	char eta_msg[16] = " ";
> >
> >  	if (current < end) {
> >  		tm = time(NULL);
> > @@ -368,13 +391,19 @@ print_progress(const char *msg, unsigned long
> > current, unsigned long end)
> >  	} else
> >  		progress = 100;
> >
> > +	if (start != NULL) {
> > +		calc_delta(start, &delta);
> > +		eta = delta.tv_sec + delta.tv_usec/1e6;
> 
> space around /
> and better to write hex value as 0x1e6
> 
> Sorry, I could not understand what is this magic number 1e6?
> 
Ah, it is 10^6.

> 
> > +		eta = (100 - progress) * eta/progress;
> 
> space around /
> 
Adopted.

Thank you for review.

Regards,
Pingfan

> > +		eta_to_human_short(eta, eta_msg);
> > +	}
> >  	if (flag_ignore_r_char) {
> > -		PROGRESS_MSG("%-" PROGRESS_MAXLEN "s: [%5.1f %%] %c\n",
> > -			     msg, progress, spinner[lapse % 4]);
> > +		PROGRESS_MSG("%-" PROGRESS_MAXLEN "s: [%5.1f %%] %c  %16s\n",
> > +			     msg, progress, spinner[lapse % 4], eta_msg);
> >  	} else {
> >  		PROGRESS_MSG("\r");
> > -		PROGRESS_MSG("%-" PROGRESS_MAXLEN "s: [%5.1f %%] %c",
> > -			     msg, progress, spinner[lapse % 4]);
> > +		PROGRESS_MSG("%-" PROGRESS_MAXLEN "s: [%5.1f %%] %c  %16s",
> > +			     msg, progress, spinner[lapse % 4], eta_msg);
> >  	}
> >  	lapse++;
> >  }
> > diff --git a/print_info.h b/print_info.h
> > index 01e3706..1ce3593 100644
> > --- a/print_info.h
> > +++ b/print_info.h
> > @@ -25,7 +25,8 @@ extern int flag_ignore_r_char;
> >
> >  void show_version(void);
> >  void print_usage(void);
> > -void print_progress(const char *msg, unsigned long current, unsigned long
> > end);
> > +void print_progress(const char *msg, unsigned long current, unsigned long
> > end, struct timeval *start);
> > +
> >  void print_execution_time(char *step_name, struct timeval *tv_start);
> >
> >  /*
> >
> 
> ~Pratyush
> 
> _______________________________________________
> kexec mailing list
> kexec at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/kexec
> 


From panand at redhat.com  Tue Jun 13 01:06:40 2017
From: panand at redhat.com (Pratyush Anand)
Date: Tue, 13 Jun 2017 13:36:40 +0530
Subject: makedumpfile: support for newer kernels [v4.9 onwards]
In-Reply-To: <CAEzMDiARnp-FMqUcoVdCBSSApaRbenvrk7cawgyo8kJJvy0jXg@mail.gmail.com>
References: <CAEzMDiD-w91Q+Fjy_KgXDcwo9YGasa9ggqW2dEhF4ffV7RstqQ@mail.gmail.com>
 <0bd0a35a-88cd-3f68-846f-2a2ad1e7afcb@redhat.com>
 <CAEzMDiBKdA15Gn2kgs=BgzCM=bzaBXCQP_tM_Vmfc4uxGL0SRg@mail.gmail.com>
 <0ae1e066-821e-3b11-ee44-b615245cb1ea@redhat.com>
 <3b72a75c-012e-2779-74a4-d36ca1ef9e56@redhat.com>
 <CAEzMDiARnp-FMqUcoVdCBSSApaRbenvrk7cawgyo8kJJvy0jXg@mail.gmail.com>
Message-ID: <364d6133-158b-0f28-0278-dc0da33b2916@redhat.com>

Hi Abhishek,

On Thursday 08 June 2017 11:30 AM, Abhishek Shah wrote:
> Hi Pratyush,
>
>>>> I am using yocto(2.3 version) to build makedumpfile, which fetches v1.6.1
>>>> makedumpfile and also applys some patches including arm64 specific
>>>> patches.
>>
>>
>> What are these ARM64 specific patches? I believe, you do not need any topup
>> in makedumpfile for ARM64 if you use v1.6.1.
>
> Sorry, I got confused with yocto kexec recipe.
> There are NO arm64 specific patches in yocto for makedumpfile.
>
>>>> We have VA_BITS set as 48 bits... I will get other information and post
>>>> on the
>>>> list.
>>
>>
>> OK.
>>
> Kernel config:
> ARM64_VA_BITS_48 [=y]
> ARM64_PAGE_SHIFT [=12]
> RANDOMIZE_BASE [=n]
> ARM64_4K_PAGES [=y]

These are the only variable configs on which arm64 makedumpfile depends. I 
tested with above configuration and things were working fine.

I used 4.11 based kernel and  crash-7.1.9.

BTW, which version of crash utility do you use. If you are not at latest can 
you please try with latest.

~Pratyush

CCed Dave if he has some input.
@Dave, problem reported by Abhishek is:

 > /proc/vmcore works fine with crash utility without passing it though
 > makedumpfile, but the file generated by makedumpfile is not getting
 > processed by crash utility; I am getting following error:
 > crash: seek error: kernel virtual address: ffff8008000c16a0  type: "memory
 > section"


From anderson at redhat.com  Tue Jun 13 06:39:02 2017
From: anderson at redhat.com (Dave Anderson)
Date: Tue, 13 Jun 2017 09:39:02 -0400 (EDT)
Subject: makedumpfile: support for newer kernels [v4.9 onwards]
In-Reply-To: <mailman.34714.1497341231.26984.kexec@lists.infradead.org>
References: <mailman.34714.1497341231.26984.kexec@lists.infradead.org>
Message-ID: <958641941.25783513.1497361142979.JavaMail.zimbra@redhat.com>


----- Original Message -----

> Hi Abhishek,
> 
> On Thursday 08 June 2017 11:30 AM, Abhishek Shah wrote:
> > Hi Pratyush,
> >
> >>>> I am using yocto(2.3 version) to build makedumpfile, which fetches v1.6.1
> >>>> makedumpfile and also applys some patches including arm64 specific patches.
> >>
> >>
> >> What are these ARM64 specific patches? I believe, you do not need any topup
> >> in makedumpfile for ARM64 if you use v1.6.1.
> >
> > Sorry, I got confused with yocto kexec recipe.
> > There are NO arm64 specific patches in yocto for makedumpfile.
> >
> >>>> We have VA_BITS set as 48 bits... I will get other information and post
> >>>> on the list.
> >>
> >>
> >> OK.
> >>
> > Kernel config:
> > ARM64_VA_BITS_48 [=y]
> > ARM64_PAGE_SHIFT [=12]
> > RANDOMIZE_BASE [=n]
> > ARM64_4K_PAGES [=y]
> 
> These are the only variable configs on which arm64 makedumpfile depends. I
> tested with above configuration and things were working fine.
> 
> I used 4.11 based kernel and  crash-7.1.9.
> 
> BTW, which version of crash utility do you use. If you are not at latest can
> you please try with latest.
> 
> ~Pratyush
> 
> CCed Dave if he has some input.
> @Dave, problem reported by Abhishek is:
> 
>  > /proc/vmcore works fine with crash utility without passing it though
>  > makedumpfile, but the file generated by makedumpfile is not getting
>  > processed by crash utility; I am getting following error:
>  > crash: seek error: kernel virtual address: ffff8008000c16a0  type: "memory
>  > section"


A "seek error" means that the page for the physical address that was translated
from ffff8008000c16a0 could not be found in the dumpfile.  The output from 
"crash --d ..." might yield some additional clues. 

However, given that crash apparently works with the 4.12 /proc/vmcore that the
dumpfile was generated from, it seemingly points to makedumpfile.  That's always 
the first thing I ask when seeing this kind of bug, i.e., does crash work with the
same kernel on a live system, or even better, with the initial ELF vmcore file
from which it was generated.  If it does, then it's usually a makedumpfile issue,
because the virtual address gets translated to a physical address by the upper-level
crash code, and then passed down into whatever the underlying memory source is.  
If it could be read from /proc/vmcore, then it should accessible from makedumpfile's
compressed kdump.

That being said, I haven't tested ARM64 on any 4.12-rc kernels.

Dave


From abrs612 at gmail.com  Tue Jun 13 08:17:08 2017
From: abrs612 at gmail.com (Abhishek Shah)
Date: Tue, 13 Jun 2017 20:47:08 +0530
Subject: makedumpfile: support for newer kernels [v4.9 onwards]
In-Reply-To: <364d6133-158b-0f28-0278-dc0da33b2916@redhat.com>
References: <CAEzMDiD-w91Q+Fjy_KgXDcwo9YGasa9ggqW2dEhF4ffV7RstqQ@mail.gmail.com>
 <0bd0a35a-88cd-3f68-846f-2a2ad1e7afcb@redhat.com>
 <CAEzMDiBKdA15Gn2kgs=BgzCM=bzaBXCQP_tM_Vmfc4uxGL0SRg@mail.gmail.com>
 <0ae1e066-821e-3b11-ee44-b615245cb1ea@redhat.com>
 <3b72a75c-012e-2779-74a4-d36ca1ef9e56@redhat.com>
 <CAEzMDiARnp-FMqUcoVdCBSSApaRbenvrk7cawgyo8kJJvy0jXg@mail.gmail.com>
 <364d6133-158b-0f28-0278-dc0da33b2916@redhat.com>
Message-ID: <CAEzMDiAq1U3SRkvwh5nqETDBoZs8DojhmMTDSG7B-Uzbh1a3Nw@mail.gmail.com>

Hi Pratyush/Dave,

On Tue, Jun 13, 2017 at 1:36 PM, Pratyush Anand <panand at redhat.com> wrote:
> Hi Abhishek,
>
> On Thursday 08 June 2017 11:30 AM, Abhishek Shah wrote:
>>
>> Hi Pratyush,
>>
>> Kernel config:
>> ARM64_VA_BITS_48 [=y]
>> ARM64_PAGE_SHIFT [=12]
>> RANDOMIZE_BASE [=n]
>> ARM64_4K_PAGES [=y]
>
> These are the only variable configs on which arm64 makedumpfile depends. I
> tested with above configuration and things were working fine.
>
> I used 4.11 based kernel and  crash-7.1.9.
>
> BTW, which version of crash utility do you use. If you are not at latest can
> you please try with latest.
>
I was using crash 7.1.9++ with head commit
5c52842a58a2602dba81de71831af98b2b53c6e0;
I have upgraded my kernel and built the crash utility again; and
following is my setup details now:

kernel: 4.12- rc5
yocto based rootfs: Pyro[2.3] which has kexec 2.0.14 and makedumpfile 1.6.1.
crash: 7.1.9++ [Head: 4d517ad28acd845fe6e91360e645cf0446a4757b]

The output generated by makedumpfile gets processed by crash utility
if I use following command:
makedumpfile -E /proc/vmcore vmcore

But, the output generated by following command (which I used while
reporting the issue) gives the "seek error" while being processed by
crash.
makedumpfile /proc/vmcore vmcore

(As a result?), output from following command gives "seek error" as
well while being processed by crash:
makedumpfile -c /proc/vmcore vmcore

makedumpfile Manual suggests:
"-E
Create DUMPFILE in the ELF format.
This option cannot be specified with -c option, because the ELF format
does not support compressed data."
So only elf formatted output from makedumpfile gets processed by crash.

@Pratyush, Thanks for testing and verifying the kernel configuration
for arm64 makedumpfile.
When you say things were working fine for you,  what exact
makedumpfile command did you use??

Regards,
Abhishek


From colona at arista.com  Tue Jun 13 17:58:54 2017
From: colona at arista.com (Ivan Delalande)
Date: Tue, 13 Jun 2017 17:58:54 -0700
Subject: [makedumpfile PATCH v2] Add the --partial-dmesg option to dump only
 non-cleared dmesg
In-Reply-To: <20170613010046.GA25025@visor>
Message-ID: <20170614005854.GA32055@visor>

This option will make --dump-dmesg save only the part of the dmesg
buffer since it was last cleared on the crashed kernel (with
dmesg --clear and such) instead of the whole buffer since boot or
wrap-around. It works on kernels with commit f468908bb55a ("printk:
add clear_idx symbol to vmcoreinfo") merged in v4.6 and otherwise
will default to the regular behavior.

Signed-off-by: Ivan Delalande <colona at arista.com>
---
 makedumpfile.8 |  8 +++++++-
 makedumpfile.c | 24 ++++++++++++++++++++++--
 makedumpfile.h |  3 +++
 3 files changed, 32 insertions(+), 3 deletions(-)

diff --git a/makedumpfile.8 b/makedumpfile.8
index 9932364..76a8d6e 100644
--- a/makedumpfile.8
+++ b/makedumpfile.8
@@ -20,7 +20,7 @@ makedumpfile \- make a small dumpfile of kdump
 .br
 \fBmakedumpfile\fR    [\fIOPTION\fR] [\-\-xen-syms \fIXEN-SYMS\fR|\-\-xen-vmcoreinfo \fIVMCOREINFO\fR] \fIVMCORE\fR \fIDUMPFILE\fR
 .br
-\fBmakedumpfile\fR \-\-dump-dmesg [\-x \fIVMLINUX\fR|\-i \fIVMCOREINFO\fR] \fIVMCORE\fR \fILOGFILE\fR
+\fBmakedumpfile\fR \-\-dump-dmesg [\-\-partial-dmesg] [\-x \fIVMLINUX\fR|\-i \fIVMCOREINFO\fR] \fIVMCORE\fR \fILOGFILE\fR
 .br
 \fBmakedumpfile\fR    [\fIOPTION\fR] \-x \fIVMLINUX\fR \-\-diskset=\fIVMCORE1\fR \-\-diskset=\fIVMCORE2\fR [\-\-diskset=\fIVMCORE3\fR ..] \fIDUMPFILE\fR
 .br
@@ -586,6 +586,12 @@ it is necessary to specfiy [\-x \fIVMLINUX\fR] or [\-i \fIVMCOREINFO\fR].
 
 
 .TP
+\fB\-\-partial-dmesg\fR
+This option will make --dump-dmesg extract only dmesg logs since that buffer was
+last cleared on the crashed kernel, through "dmesg --clear" for example.
+
+
+.TP
 \fB\-\-mem-usage\fR
 This option is only for x86_64.
 This option is used to show the page numbers of current system in different
diff --git a/makedumpfile.c b/makedumpfile.c
index 95cc5ef..6c5fa4c 100644
--- a/makedumpfile.c
+++ b/makedumpfile.c
@@ -1498,6 +1498,7 @@ get_symbol_info(void)
 	SYMBOL_INIT(log_buf_len, "log_buf_len");
 	SYMBOL_INIT(log_end, "log_end");
 	SYMBOL_INIT(log_first_idx, "log_first_idx");
+	SYMBOL_INIT(clear_idx, "clear_idx");
 	SYMBOL_INIT(log_next_idx, "log_next_idx");
 	SYMBOL_INIT(max_pfn, "max_pfn");
 	SYMBOL_INIT(modules, "modules");
@@ -2115,6 +2116,7 @@ write_vmcoreinfo_data(void)
 	WRITE_SYMBOL("log_buf_len", log_buf_len);
 	WRITE_SYMBOL("log_end", log_end);
 	WRITE_SYMBOL("log_first_idx", log_first_idx);
+	WRITE_SYMBOL("clear_idx", clear_idx);
 	WRITE_SYMBOL("log_next_idx", log_next_idx);
 	WRITE_SYMBOL("max_pfn", max_pfn);
 	WRITE_SYMBOL("high_memory", high_memory);
@@ -2509,6 +2511,7 @@ read_vmcoreinfo(void)
 	READ_SYMBOL("log_buf_len", log_buf_len);
 	READ_SYMBOL("log_end", log_end);
 	READ_SYMBOL("log_first_idx", log_first_idx);
+	READ_SYMBOL("clear_idx", clear_idx);
 	READ_SYMBOL("log_next_idx", log_next_idx);
 	READ_SYMBOL("max_pfn", max_pfn);
 	READ_SYMBOL("high_memory", high_memory);
@@ -5025,6 +5028,7 @@ dump_dmesg()
 	int log_buf_len, length_log, length_oldlog, ret = FALSE;
 	unsigned long index, log_buf, log_end;
 	unsigned int idx, log_first_idx, log_next_idx;
+	unsigned long long first_idx_sym;
 	unsigned long log_end_2_6_24;
 	unsigned      log_end_2_6_25;
 	char *log_buffer = NULL, *log_ptr = NULL;
@@ -5058,7 +5062,13 @@ dump_dmesg()
 			ERRMSG("Can't find variable-length record symbols");
 			return FALSE;
 		} else {
-			if (!readmem(VADDR, SYMBOL(log_first_idx), &log_first_idx,
+			if (info->flag_partial_dmesg
+			    && SYMBOL(clear_idx) != NOT_FOUND_SYMBOL)
+				first_idx_sym = SYMBOL(clear_idx);
+			else
+				first_idx_sym = SYMBOL(log_first_idx);
+
+			if (!readmem(VADDR, first_idx_sym, &log_first_idx,
 			    sizeof(log_first_idx))) {
 				ERRMSG("Can't get log_first_idx.\n");
 				return FALSE;
@@ -5102,7 +5112,10 @@ dump_dmesg()
 	DEBUG_MSG("log_buf       : %lx\n", log_buf);
 	DEBUG_MSG("log_end       : %lx\n", log_end);
 	DEBUG_MSG("log_buf_len   : %d\n", log_buf_len);
-	DEBUG_MSG("log_first_idx : %u\n", log_first_idx);
+	if (info->flag_partial_dmesg)
+		DEBUG_MSG("clear_idx : %u\n", log_first_idx);
+	else
+		DEBUG_MSG("log_first_idx : %u\n", log_first_idx);
 	DEBUG_MSG("log_next_idx  : %u\n", log_next_idx);
 
 	if ((log_buffer = malloc(log_buf_len)) == NULL) {
@@ -10618,6 +10631,9 @@ check_param_for_creating_dumpfile(int argc, char *argv[])
 		}
 	}
 
+	if (info->flag_partial_dmesg && !info->flag_dmesg)
+		return FALSE;
+
 	if ((argc == optind + 2) && !info->flag_flatten
 				 && !info->flag_split
 				 && !info->flag_sadump_diskset) {
@@ -11022,6 +11038,7 @@ static struct option longopts[] = {
 	{"message-level", required_argument, NULL, OPT_MESSAGE_LEVEL},
 	{"vtop", required_argument, NULL, OPT_VTOP},
 	{"dump-dmesg", no_argument, NULL, OPT_DUMP_DMESG},
+	{"partial-dmesg", no_argument, NULL, OPT_PARTIAL_DMESG},
 	{"config", required_argument, NULL, OPT_CONFIG},
 	{"help", no_argument, NULL, OPT_HELP},
 	{"diskset", required_argument, NULL, OPT_DISKSET},
@@ -11135,6 +11152,9 @@ main(int argc, char *argv[])
 		case OPT_DUMP_DMESG:
 			info->flag_dmesg = 1;
 			break;
+		case OPT_PARTIAL_DMESG:
+			info->flag_partial_dmesg = 1;
+			break;
 		case OPT_MEM_USAGE:
 		       info->flag_mem_usage = 1;
 		       break;
diff --git a/makedumpfile.h b/makedumpfile.h
index 6ea8cc4..d3b7532 100644
--- a/makedumpfile.h
+++ b/makedumpfile.h
@@ -1241,6 +1241,7 @@ struct DumpInfo {
 	int		flag_force;	     /* overwrite existing stuff */
 	int		flag_exclude_xen_dom;/* exclude Domain-U from xen-kdump */
 	int             flag_dmesg;          /* dump the dmesg log out of the vmcore file */
+	int             flag_partial_dmesg;  /* dmesg dump only from the last cleared index*/
 	int             flag_mem_usage;  /*show the page number of memory in different use*/
 	int		flag_use_printk_log; /* did we read printk_log symbol name? */
 	int		flag_nospace;	     /* the flag of "No space on device" error */
@@ -1529,6 +1530,7 @@ struct symbol_table {
 	unsigned long long	log_buf_len;
 	unsigned long long	log_end;
 	unsigned long long	log_first_idx;
+	unsigned long long	clear_idx;
 	unsigned long long	log_next_idx;
 	unsigned long long	max_pfn;
 	unsigned long long	node_remap_start_vaddr;
@@ -2268,6 +2270,7 @@ struct elf_prstatus {
 #define OPT_SPLITBLOCK_SIZE	OPT_START+14
 #define OPT_WORKING_DIR         OPT_START+15
 #define OPT_NUM_THREADS	OPT_START+16
+#define OPT_PARTIAL_DMESG	OPT_START+17
 
 /*
  * Function Prototype.
-- 
2.13.1


From ats-kumagai at wm.jp.nec.com  Wed Jun 14 00:52:02 2017
From: ats-kumagai at wm.jp.nec.com (Atsushi Kumagai)
Date: Wed, 14 Jun 2017 07:52:02 +0000
Subject: [makedumpfile PATCH v2] Add the --partial-dmesg option to dump
 only non-cleared dmesg
In-Reply-To: <20170614005854.GA32055@visor>
References: <20170613010046.GA25025@visor> <20170614005854.GA32055@visor>
Message-ID: <0910DD04CBD6DE4193FCF86B9C00BE9701EE06D5@BPXM01GP.gisp.nec.co.jp>

Hello Ivan,

>This option will make --dump-dmesg save only the part of the dmesg
>buffer since it was last cleared on the crashed kernel (with
>dmesg --clear and such) instead of the whole buffer since boot or
>wrap-around. It works on kernels with commit f468908bb55a ("printk:
>add clear_idx symbol to vmcoreinfo") merged in v4.6 and otherwise
>will default to the regular behavior.
>
>Signed-off-by: Ivan Delalande <colona at arista.com>

Thanks, I'll merge this patch into v1.6.2.

Regards,
Atsushi Kumagai

>---
> makedumpfile.8 |  8 +++++++-
> makedumpfile.c | 24 ++++++++++++++++++++++--
> makedumpfile.h |  3 +++
> 3 files changed, 32 insertions(+), 3 deletions(-)
>
>diff --git a/makedumpfile.8 b/makedumpfile.8
>index 9932364..76a8d6e 100644
>--- a/makedumpfile.8
>+++ b/makedumpfile.8
>@@ -20,7 +20,7 @@ makedumpfile \- make a small dumpfile of kdump
> .br
> \fBmakedumpfile\fR    [\fIOPTION\fR] [\-\-xen-syms \fIXEN-SYMS\fR|\-\-xen-vmcoreinfo \fIVMCOREINFO\fR] \fIVMCORE\fR
>\fIDUMPFILE\fR
> .br
>-\fBmakedumpfile\fR \-\-dump-dmesg [\-x \fIVMLINUX\fR|\-i \fIVMCOREINFO\fR] \fIVMCORE\fR \fILOGFILE\fR
>+\fBmakedumpfile\fR \-\-dump-dmesg [\-\-partial-dmesg] [\-x \fIVMLINUX\fR|\-i \fIVMCOREINFO\fR] \fIVMCORE\fR
>\fILOGFILE\fR
> .br
> \fBmakedumpfile\fR    [\fIOPTION\fR] \-x \fIVMLINUX\fR \-\-diskset=\fIVMCORE1\fR \-\-diskset=\fIVMCORE2\fR
>[\-\-diskset=\fIVMCORE3\fR ..] \fIDUMPFILE\fR
> .br
>@@ -586,6 +586,12 @@ it is necessary to specfiy [\-x \fIVMLINUX\fR] or [\-i \fIVMCOREINFO\fR].
>
>
> .TP
>+\fB\-\-partial-dmesg\fR
>+This option will make --dump-dmesg extract only dmesg logs since that buffer was
>+last cleared on the crashed kernel, through "dmesg --clear" for example.
>+
>+
>+.TP
> \fB\-\-mem-usage\fR
> This option is only for x86_64.
> This option is used to show the page numbers of current system in different
>diff --git a/makedumpfile.c b/makedumpfile.c
>index 95cc5ef..6c5fa4c 100644
>--- a/makedumpfile.c
>+++ b/makedumpfile.c
>@@ -1498,6 +1498,7 @@ get_symbol_info(void)
> 	SYMBOL_INIT(log_buf_len, "log_buf_len");
> 	SYMBOL_INIT(log_end, "log_end");
> 	SYMBOL_INIT(log_first_idx, "log_first_idx");
>+	SYMBOL_INIT(clear_idx, "clear_idx");
> 	SYMBOL_INIT(log_next_idx, "log_next_idx");
> 	SYMBOL_INIT(max_pfn, "max_pfn");
> 	SYMBOL_INIT(modules, "modules");
>@@ -2115,6 +2116,7 @@ write_vmcoreinfo_data(void)
> 	WRITE_SYMBOL("log_buf_len", log_buf_len);
> 	WRITE_SYMBOL("log_end", log_end);
> 	WRITE_SYMBOL("log_first_idx", log_first_idx);
>+	WRITE_SYMBOL("clear_idx", clear_idx);
> 	WRITE_SYMBOL("log_next_idx", log_next_idx);
> 	WRITE_SYMBOL("max_pfn", max_pfn);
> 	WRITE_SYMBOL("high_memory", high_memory);
>@@ -2509,6 +2511,7 @@ read_vmcoreinfo(void)
> 	READ_SYMBOL("log_buf_len", log_buf_len);
> 	READ_SYMBOL("log_end", log_end);
> 	READ_SYMBOL("log_first_idx", log_first_idx);
>+	READ_SYMBOL("clear_idx", clear_idx);
> 	READ_SYMBOL("log_next_idx", log_next_idx);
> 	READ_SYMBOL("max_pfn", max_pfn);
> 	READ_SYMBOL("high_memory", high_memory);
>@@ -5025,6 +5028,7 @@ dump_dmesg()
> 	int log_buf_len, length_log, length_oldlog, ret = FALSE;
> 	unsigned long index, log_buf, log_end;
> 	unsigned int idx, log_first_idx, log_next_idx;
>+	unsigned long long first_idx_sym;
> 	unsigned long log_end_2_6_24;
> 	unsigned      log_end_2_6_25;
> 	char *log_buffer = NULL, *log_ptr = NULL;
>@@ -5058,7 +5062,13 @@ dump_dmesg()
> 			ERRMSG("Can't find variable-length record symbols");
> 			return FALSE;
> 		} else {
>-			if (!readmem(VADDR, SYMBOL(log_first_idx), &log_first_idx,
>+			if (info->flag_partial_dmesg
>+			    && SYMBOL(clear_idx) != NOT_FOUND_SYMBOL)
>+				first_idx_sym = SYMBOL(clear_idx);
>+			else
>+				first_idx_sym = SYMBOL(log_first_idx);
>+
>+			if (!readmem(VADDR, first_idx_sym, &log_first_idx,
> 			    sizeof(log_first_idx))) {
> 				ERRMSG("Can't get log_first_idx.\n");
> 				return FALSE;
>@@ -5102,7 +5112,10 @@ dump_dmesg()
> 	DEBUG_MSG("log_buf       : %lx\n", log_buf);
> 	DEBUG_MSG("log_end       : %lx\n", log_end);
> 	DEBUG_MSG("log_buf_len   : %d\n", log_buf_len);
>-	DEBUG_MSG("log_first_idx : %u\n", log_first_idx);
>+	if (info->flag_partial_dmesg)
>+		DEBUG_MSG("clear_idx : %u\n", log_first_idx);
>+	else
>+		DEBUG_MSG("log_first_idx : %u\n", log_first_idx);
> 	DEBUG_MSG("log_next_idx  : %u\n", log_next_idx);
>
> 	if ((log_buffer = malloc(log_buf_len)) == NULL) {
>@@ -10618,6 +10631,9 @@ check_param_for_creating_dumpfile(int argc, char *argv[])
> 		}
> 	}
>
>+	if (info->flag_partial_dmesg && !info->flag_dmesg)
>+		return FALSE;
>+
> 	if ((argc == optind + 2) && !info->flag_flatten
> 				 && !info->flag_split
> 				 && !info->flag_sadump_diskset) {
>@@ -11022,6 +11038,7 @@ static struct option longopts[] = {
> 	{"message-level", required_argument, NULL, OPT_MESSAGE_LEVEL},
> 	{"vtop", required_argument, NULL, OPT_VTOP},
> 	{"dump-dmesg", no_argument, NULL, OPT_DUMP_DMESG},
>+	{"partial-dmesg", no_argument, NULL, OPT_PARTIAL_DMESG},
> 	{"config", required_argument, NULL, OPT_CONFIG},
> 	{"help", no_argument, NULL, OPT_HELP},
> 	{"diskset", required_argument, NULL, OPT_DISKSET},
>@@ -11135,6 +11152,9 @@ main(int argc, char *argv[])
> 		case OPT_DUMP_DMESG:
> 			info->flag_dmesg = 1;
> 			break;
>+		case OPT_PARTIAL_DMESG:
>+			info->flag_partial_dmesg = 1;
>+			break;
> 		case OPT_MEM_USAGE:
> 		       info->flag_mem_usage = 1;
> 		       break;
>diff --git a/makedumpfile.h b/makedumpfile.h
>index 6ea8cc4..d3b7532 100644
>--- a/makedumpfile.h
>+++ b/makedumpfile.h
>@@ -1241,6 +1241,7 @@ struct DumpInfo {
> 	int		flag_force;	     /* overwrite existing stuff */
> 	int		flag_exclude_xen_dom;/* exclude Domain-U from xen-kdump */
> 	int             flag_dmesg;          /* dump the dmesg log out of the vmcore file */
>+	int             flag_partial_dmesg;  /* dmesg dump only from the last cleared index*/
> 	int             flag_mem_usage;  /*show the page number of memory in different use*/
> 	int		flag_use_printk_log; /* did we read printk_log symbol name? */
> 	int		flag_nospace;	     /* the flag of "No space on device" error */
>@@ -1529,6 +1530,7 @@ struct symbol_table {
> 	unsigned long long	log_buf_len;
> 	unsigned long long	log_end;
> 	unsigned long long	log_first_idx;
>+	unsigned long long	clear_idx;
> 	unsigned long long	log_next_idx;
> 	unsigned long long	max_pfn;
> 	unsigned long long	node_remap_start_vaddr;
>@@ -2268,6 +2270,7 @@ struct elf_prstatus {
> #define OPT_SPLITBLOCK_SIZE	OPT_START+14
> #define OPT_WORKING_DIR         OPT_START+15
> #define OPT_NUM_THREADS	OPT_START+16
>+#define OPT_PARTIAL_DMESG	OPT_START+17
>
> /*
>  * Function Prototype.
>--
>2.13.1



From piliu at redhat.com  Wed Jun 14 02:41:01 2017
From: piliu at redhat.com (Pingfan Liu)
Date: Wed, 14 Jun 2017 17:41:01 +0800
Subject: [makedumpfile PATCHv3 1/2] fold the calc of time delta into a func
Message-ID: <1497433262-5687-1-git-send-email-piliu@redhat.com>

This piece of code will be used by more than one function after
introducing the next patch

Signed-off-by: Pingfan Liu <piliu at redhat.com>
---
 print_info.c | 28 ++++++++++++++++------------
 1 file changed, 16 insertions(+), 12 deletions(-)

diff --git a/print_info.c b/print_info.c
index 832196d..79dfe5e 100644
--- a/print_info.c
+++ b/print_info.c
@@ -337,6 +337,19 @@ print_usage(void)
 	MSG("\n");
 }
 
+static void calc_delta(struct timeval *tv_start, struct timeval *delta)
+{
+	struct timeval tv_end;
+
+	gettimeofday(&tv_end, NULL);
+	delta->tv_sec = tv_end.tv_sec - tv_start->tv_sec;
+	delta->tv_usec = tv_end.tv_usec - tv_start->tv_usec;
+	if (delta->tv_usec < 0) {
+		delta->tv_sec--;
+		delta->tv_usec += 1000000;
+	}
+}
+
 void
 print_progress(const char *msg, unsigned long current, unsigned long end)
 {
@@ -369,19 +382,10 @@ print_progress(const char *msg, unsigned long current, unsigned long end)
 void
 print_execution_time(char *step_name, struct timeval *tv_start)
 {
-	struct timeval tv_end;
-	time_t diff_sec;
-	suseconds_t diff_usec;
+	struct timeval delta;
 
-	gettimeofday(&tv_end, NULL);
-
-	diff_sec  = tv_end.tv_sec - tv_start->tv_sec;
-	diff_usec = tv_end.tv_usec - tv_start->tv_usec;
-	if (diff_usec < 0) {
-		diff_sec--;
-		diff_usec += 1000000;
-	}
+	calc_delta(tv_start, &delta);
 	REPORT_MSG("STEP [%s] : %ld.%06ld seconds\n",
-		   step_name, diff_sec, diff_usec);
+		   step_name, delta.tv_sec, delta.tv_usec);
 }
 
-- 
2.7.4



From piliu at redhat.com  Wed Jun 14 02:41:31 2017
From: piliu at redhat.com (Pingfan Liu)
Date: Wed, 14 Jun 2017 17:41:31 +0800
Subject: [makedumpfile PATCHv3 1/2] fold the calc of time delta into a func
Message-ID: <1497433292-6015-1-git-send-email-piliu@redhat.com>

This piece of code will be used by more than one function after
introducing the next patch

Signed-off-by: Pingfan Liu <piliu at redhat.com>
---
 print_info.c | 28 ++++++++++++++++------------
 1 file changed, 16 insertions(+), 12 deletions(-)

diff --git a/print_info.c b/print_info.c
index 832196d..79dfe5e 100644
--- a/print_info.c
+++ b/print_info.c
@@ -337,6 +337,19 @@ print_usage(void)
 	MSG("\n");
 }
 
+static void calc_delta(struct timeval *tv_start, struct timeval *delta)
+{
+	struct timeval tv_end;
+
+	gettimeofday(&tv_end, NULL);
+	delta->tv_sec = tv_end.tv_sec - tv_start->tv_sec;
+	delta->tv_usec = tv_end.tv_usec - tv_start->tv_usec;
+	if (delta->tv_usec < 0) {
+		delta->tv_sec--;
+		delta->tv_usec += 1000000;
+	}
+}
+
 void
 print_progress(const char *msg, unsigned long current, unsigned long end)
 {
@@ -369,19 +382,10 @@ print_progress(const char *msg, unsigned long current, unsigned long end)
 void
 print_execution_time(char *step_name, struct timeval *tv_start)
 {
-	struct timeval tv_end;
-	time_t diff_sec;
-	suseconds_t diff_usec;
+	struct timeval delta;
 
-	gettimeofday(&tv_end, NULL);
-
-	diff_sec  = tv_end.tv_sec - tv_start->tv_sec;
-	diff_usec = tv_end.tv_usec - tv_start->tv_usec;
-	if (diff_usec < 0) {
-		diff_sec--;
-		diff_usec += 1000000;
-	}
+	calc_delta(tv_start, &delta);
 	REPORT_MSG("STEP [%s] : %ld.%06ld seconds\n",
-		   step_name, diff_sec, diff_usec);
+		   step_name, delta.tv_sec, delta.tv_usec);
 }
 
-- 
2.7.4



From piliu at redhat.com  Wed Jun 14 02:41:32 2017
From: piliu at redhat.com (Pingfan Liu)
Date: Wed, 14 Jun 2017 17:41:32 +0800
Subject: [makedumpfile PATCH v3 2/2] print_info: show the remaining time of
 dump progress
In-Reply-To: <1497433292-6015-1-git-send-email-piliu@redhat.com>
References: <1497433292-6015-1-git-send-email-piliu@redhat.com>
Message-ID: <1497433292-6015-2-git-send-email-piliu@redhat.com>

If the dump progress is slow, it is better to show  an estimate of
remaining time of the progress to the user. The estimator simply uses
the average speed to work.
Before this patch, the bar looks like:
  Copying data                                      : [  1.8 %] /
After this patch:
  Copying data                                      : [  1.8 %] /          eta: 39s

Signed-off-by: Pingfan Liu <piliu at redhat.com>
---
v2 -> v3: improve the commit log

 makedumpfile.c | 46 +++++++++++++++++++++++-----------------------
 print_info.c   | 41 +++++++++++++++++++++++++++++++++++------
 print_info.h   |  3 ++-
 3 files changed, 60 insertions(+), 30 deletions(-)

diff --git a/makedumpfile.c b/makedumpfile.c
index 301772a..35a36ce 100644
--- a/makedumpfile.c
+++ b/makedumpfile.c
@@ -5175,7 +5175,7 @@ _exclude_free_page(struct cycle *cycle)
 
 	for (num_nodes = 1; num_nodes <= vt.numnodes; num_nodes++) {
 
-		print_progress(PROGRESS_FREE_PAGES, num_nodes - 1, vt.numnodes);
+		print_progress(PROGRESS_FREE_PAGES, num_nodes - 1, vt.numnodes, NULL);
 
 		node_zones = pgdat + OFFSET(pglist_data.node_zones);
 
@@ -5188,7 +5188,7 @@ _exclude_free_page(struct cycle *cycle)
 		for (i = 0; i < nr_zones; i++) {
 
 			print_progress(PROGRESS_FREE_PAGES, i + nr_zones * (num_nodes - 1),
-					nr_zones * vt.numnodes);
+					nr_zones * vt.numnodes, NULL);
 
 			zone = node_zones + (i * SIZE(zone));
 			if (!readmem(VADDR, zone + OFFSET(zone.spanned_pages),
@@ -5216,7 +5216,7 @@ _exclude_free_page(struct cycle *cycle)
 	/*
 	 * print [100 %]
 	 */
-	print_progress(PROGRESS_FREE_PAGES, vt.numnodes, vt.numnodes);
+	print_progress(PROGRESS_FREE_PAGES, vt.numnodes, vt.numnodes, NULL);
 	print_execution_time(PROGRESS_FREE_PAGES, &tv_start);
 
 	return TRUE;
@@ -5403,7 +5403,7 @@ create_1st_bitmap_file(void)
 	for (i = 0; get_pt_load(i, &phys_start, &phys_end, NULL, NULL); i++) {
 
 		if (!info->flag_mem_usage)
-			print_progress(PROGRESS_HOLES, i, num_pt_loads);
+			print_progress(PROGRESS_HOLES, i, num_pt_loads, NULL);
 
 		pfn_start = paddr_to_pfn(phys_start);
 		pfn_end   = paddr_to_pfn(phys_end);
@@ -5422,7 +5422,7 @@ create_1st_bitmap_file(void)
 	 * print 100 %
 	 */
 	if (!info->flag_mem_usage) {
-		print_progress(PROGRESS_HOLES, info->max_mapnr, info->max_mapnr);
+		print_progress(PROGRESS_HOLES, info->max_mapnr, info->max_mapnr, NULL);
 		print_execution_time(PROGRESS_HOLES, &tv_start);
 	}
 
@@ -5548,7 +5548,7 @@ create_bitmap_from_memhole(struct cycle *cycle, struct dump_bitmap *bitmap, int
 		pfn_start = MAX(paddr_to_pfn(phys_start), cycle->start_pfn);
 		pfn_end = MIN(paddr_to_pfn(phys_end), cycle->end_pfn);
 
-		print_progress(PROGRESS_HOLES, i, num_pt_loads);
+		print_progress(PROGRESS_HOLES, i, num_pt_loads, NULL);
 
 		if (pfn_start >= pfn_end)
 			continue;
@@ -5587,7 +5587,7 @@ create_bitmap_from_memhole(struct cycle *cycle, struct dump_bitmap *bitmap, int
 	/*
 	 * print 100 %
 	 */
-	print_progress(PROGRESS_HOLES, info->max_mapnr, info->max_mapnr);
+	print_progress(PROGRESS_HOLES, info->max_mapnr, info->max_mapnr, NULL);
 	print_execution_time(PROGRESS_HOLES, &tv_start);
 
 	return TRUE;
@@ -5842,7 +5842,7 @@ exclude_unnecessary_pages(struct cycle *cycle)
 	for (mm = 0; mm < info->num_mem_map; mm++) {
 
 		if (!info->flag_mem_usage)
-			print_progress(PROGRESS_UNN_PAGES, mm, info->num_mem_map);
+			print_progress(PROGRESS_UNN_PAGES, mm, info->num_mem_map, NULL);
 
 		mmd = &info->mem_map_data[mm];
 
@@ -5860,7 +5860,7 @@ exclude_unnecessary_pages(struct cycle *cycle)
 	 * print [100 %]
 	 */
 	if (!info->flag_mem_usage) {
-		print_progress(PROGRESS_UNN_PAGES, info->num_mem_map, info->num_mem_map);
+		print_progress(PROGRESS_UNN_PAGES, info->num_mem_map, info->num_mem_map, NULL);
 		print_execution_time(PROGRESS_UNN_PAGES, &tv_start);
 	}
 
@@ -7334,7 +7334,7 @@ write_elf_pages_cyclic(struct cache_data *cd_header, struct cache_data *cd_page)
 				}
 
 				if ((num_dumped % per) == 0)
-					print_progress(PROGRESS_COPY, num_dumped, num_dumpable);
+					print_progress(PROGRESS_COPY, num_dumped, num_dumpable, &tv_start);
 
 				num_dumped++;
 
@@ -7453,7 +7453,7 @@ write_elf_pages_cyclic(struct cache_data *cd_header, struct cache_data *cd_page)
 	/*
 	 * print [100 %]
 	 */
-	print_progress(PROGRESS_COPY, num_dumpable, num_dumpable);
+	print_progress(PROGRESS_COPY, num_dumpable, num_dumpable, &tv_start);
 	print_execution_time(PROGRESS_COPY, &tv_start);
 	PROGRESS_MSG("\n");
 
@@ -7931,7 +7931,7 @@ write_kdump_pages_parallel_cyclic(struct cache_data *cd_header,
 		}
 
 		if ((num_dumped % per) == 0)
-			print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable);
+			print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable, &tv_start);
 
 		num_dumped++;
 
@@ -7967,7 +7967,7 @@ finish:
 	/*
 	 * print [100 %]
 	 */
-	print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable);
+	print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable, &tv_start);
 	print_execution_time(PROGRESS_COPY, &tv_start);
 	PROGRESS_MSG("\n");
 
@@ -8084,7 +8084,7 @@ write_kdump_pages_cyclic(struct cache_data *cd_header, struct cache_data *cd_pag
 	for (pfn = start_pfn; pfn < end_pfn; pfn++) {
 
 		if ((num_dumped % per) == 0)
-			print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable);
+			print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable, &tv_start);
 
 		/*
 		 * Check the excluded page.
@@ -8164,7 +8164,7 @@ out:
 		free(wrkmem);
 #endif
 
-	print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable);
+	print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable, &tv_start);
 	print_execution_time(PROGRESS_COPY, &tv_start);
 
 	return ret;
@@ -8622,7 +8622,7 @@ write_kdump_pages_and_bitmap_cyclic(struct cache_data *cd_header, struct cache_d
 	/*
 	 * print [100 %]
 	 */
-	print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable);
+	print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable, &tv_start);
 	print_execution_time(PROGRESS_COPY, &tv_start);
 	PROGRESS_MSG("\n");
 
@@ -9216,7 +9216,7 @@ exclude_xen3_user_domain(void)
 	 */
 	for (i = 0; get_pt_load(i, &phys_start, &phys_end, NULL, NULL); i++) {
 
-		print_progress(PROGRESS_XEN_DOMAIN, i, num_pt_loads);
+		print_progress(PROGRESS_XEN_DOMAIN, i, num_pt_loads, NULL);
 
 		pfn     = paddr_to_pfn(phys_start);
 		pfn_end = paddr_to_pfn(phys_end);
@@ -9224,7 +9224,7 @@ exclude_xen3_user_domain(void)
 
 		for (j = 0; pfn < pfn_end; pfn++, j++) {
 			print_progress(PROGRESS_XEN_DOMAIN, j + (size * i),
-					size * num_pt_loads);
+					size * num_pt_loads, NULL);
 
 			if (!allocated_in_map(pfn)) {
 				clear_bit_on_2nd_bitmap(pfn, NULL);
@@ -9280,7 +9280,7 @@ exclude_xen4_user_domain(void)
 	 */
 	for (i = 0; get_pt_load(i, &phys_start, &phys_end, NULL, NULL); i++) {
 
-		print_progress(PROGRESS_XEN_DOMAIN, i, num_pt_loads);
+		print_progress(PROGRESS_XEN_DOMAIN, i, num_pt_loads, NULL);
 
 		pfn     = paddr_to_pfn(phys_start);
 		pfn_end = paddr_to_pfn(phys_end);
@@ -9288,7 +9288,7 @@ exclude_xen4_user_domain(void)
 
 		for (j = 0; pfn < pfn_end; pfn++, j++) {
 			print_progress(PROGRESS_XEN_DOMAIN, j + (size * i),
-					size * num_pt_loads);
+					size * num_pt_loads, NULL);
 
 			page_info_addr = info->frame_table_vaddr + pfn * SIZE(page_info);
 			if (!readmem(VADDR_XEN,
@@ -9356,7 +9356,7 @@ exclude_xen_user_domain(void)
 	/*
 	 * print [100 %]
 	 */
-	print_progress(PROGRESS_XEN_DOMAIN, 1, 1);
+	print_progress(PROGRESS_XEN_DOMAIN, 1, 1, NULL);
 	print_execution_time(PROGRESS_XEN_DOMAIN, &tv_start);
 
 	return ret;
@@ -10342,7 +10342,7 @@ reassemble_kdump_pages(void)
 
 			num_dumped++;
 
-			print_progress(PROGRESS_COPY, num_dumped, num_dumpable);
+			print_progress(PROGRESS_COPY, num_dumped, num_dumpable, &tv_start);
 
 			if (lseek(fd, offset_ph_org, SEEK_SET) < 0) {
 				ERRMSG("Can't seek a file(%s). %s\n",
@@ -10439,7 +10439,7 @@ reassemble_kdump_pages(void)
 						    size_eraseinfo))
 			goto out;
 	}
-	print_progress(PROGRESS_COPY, num_dumpable, num_dumpable);
+	print_progress(PROGRESS_COPY, num_dumpable, num_dumpable, &tv_start);
 	print_execution_time(PROGRESS_COPY, &tv_start);
 
 	ret = TRUE;
diff --git a/print_info.c b/print_info.c
index 79dfe5e..e0e6a27 100644
--- a/print_info.c
+++ b/print_info.c
@@ -15,8 +15,9 @@
  */
 #include "print_info.h"
 #include <time.h>
+#include <string.h>
 
-#define PROGRESS_MAXLEN		"35"
+#define PROGRESS_MAXLEN		"50"
 
 int message_level;
 int flag_strerr_message;
@@ -350,14 +351,36 @@ static void calc_delta(struct timeval *tv_start, struct timeval *delta)
 	}
 }
 
+/* produce less than 12 bytes on msg */
+static int eta_to_human_short (int secs, char* msg)
+{
+	strcpy(msg, "eta: ");
+	msg += strlen("eta: ");
+	if (secs < 100)
+		sprintf(msg, "%ds", secs);
+	else if (secs < 100 * 60)
+		sprintf(msg, "%dm%ds", secs / 60, secs % 60);
+	else if (secs < 48 * 3600)
+		sprintf(msg, "%dh%dm", secs / 3600, (secs / 60) % 60);
+	else if (secs < 100 * 86400)
+		sprintf(msg, "%dd%dh", secs / 86400, (secs / 3600) % 24);
+	else
+		sprintf(msg, ">2day");
+	return 0;
+}
+
+
 void
-print_progress(const char *msg, unsigned long current, unsigned long end)
+print_progress(const char *msg, unsigned long current, unsigned long end, struct timeval *start)
 {
 	float progress;
 	time_t tm;
 	static time_t last_time = 0;
 	static unsigned int lapse = 0;
 	static const char *spinner = "/|\\-";
+	struct timeval delta;
+	double eta;
+	char eta_msg[16] = " ";
 
 	if (current < end) {
 		tm = time(NULL);
@@ -368,13 +391,19 @@ print_progress(const char *msg, unsigned long current, unsigned long end)
 	} else
 		progress = 100;
 
+	if (start != NULL) {
+		calc_delta(start, &delta);
+		eta = delta.tv_sec + delta.tv_usec / 1e6;
+		eta = (100 - progress) * eta / progress;
+		eta_to_human_short(eta, eta_msg);
+	}
 	if (flag_ignore_r_char) {
-		PROGRESS_MSG("%-" PROGRESS_MAXLEN "s: [%5.1f %%] %c\n",
-			     msg, progress, spinner[lapse % 4]);
+		PROGRESS_MSG("%-" PROGRESS_MAXLEN "s: [%5.1f %%] %c  %16s\n",
+			     msg, progress, spinner[lapse % 4], eta_msg);
 	} else {
 		PROGRESS_MSG("\r");
-		PROGRESS_MSG("%-" PROGRESS_MAXLEN "s: [%5.1f %%] %c",
-			     msg, progress, spinner[lapse % 4]);
+		PROGRESS_MSG("%-" PROGRESS_MAXLEN "s: [%5.1f %%] %c  %16s",
+			     msg, progress, spinner[lapse % 4], eta_msg);
 	}
 	lapse++;
 }
diff --git a/print_info.h b/print_info.h
index 01e3706..1ce3593 100644
--- a/print_info.h
+++ b/print_info.h
@@ -25,7 +25,8 @@ extern int flag_ignore_r_char;
 
 void show_version(void);
 void print_usage(void);
-void print_progress(const char *msg, unsigned long current, unsigned long end);
+void print_progress(const char *msg, unsigned long current, unsigned long end, struct timeval *start);
+
 void print_execution_time(char *step_name, struct timeval *tv_start);
 
 /*
-- 
2.7.4



From bp at alien8.de  Wed Jun 14 09:07:54 2017
From: bp at alien8.de (Borislav Petkov)
Date: Wed, 14 Jun 2017 18:07:54 +0200
Subject: [PATCH v6 20/34] x86, mpparse: Use memremap to map the mpf and
 mpc data
In-Reply-To: <20170607191643.28645.91679.stgit@tlendack-t1.amdoffice.net>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
 <20170607191643.28645.91679.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170614160754.c4ywbf5ktqwgc4ij@pd.tnic>

On Wed, Jun 07, 2017 at 02:16:43PM -0500, Tom Lendacky wrote:
> The SMP MP-table is built by UEFI and placed in memory in a decrypted
> state. These tables are accessed using a mix of early_memremap(),
> early_memunmap(), phys_to_virt() and virt_to_phys(). Change all accesses
> to use early_memremap()/early_memunmap(). This allows for proper setting
> of the encryption mask so that the data can be successfully accessed when
> SME is active.
> 
> Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
> ---
>  arch/x86/kernel/mpparse.c |   98 ++++++++++++++++++++++++++++++++-------------
>  1 file changed, 70 insertions(+), 28 deletions(-)

...

> @@ -515,6 +516,12 @@ void __init default_get_smp_config(unsigned int early)
>  	if (acpi_lapic && acpi_ioapic)
>  		return;
>  
> +	mpf = early_memremap(mpf_base, sizeof(*mpf));
> +	if (!mpf) {
> +		pr_err("MPTABLE: mpf early_memremap() failed\n");

If you're going to introduce new prefixes then add:

#undef pr_fmt
#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt

at the beginning of the file so that they all say "mpparse:" instead.

And pls make that message more user-friendly: "Error mapping MP table"
or so.

-- 
Regards/Gruss,
    Boris.

Good mailing practices for 400: avoid top-posting and trim the reply.


From bp at alien8.de  Wed Jun 14 09:24:16 2017
From: bp at alien8.de (Borislav Petkov)
Date: Wed, 14 Jun 2017 18:24:16 +0200
Subject: [PATCH v6 23/34] x86, realmode: Decrypt trampoline area if
 memory encryption is active
In-Reply-To: <20170607191709.28645.69034.stgit@tlendack-t1.amdoffice.net>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
 <20170607191709.28645.69034.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170614162416.ksa54esy5ql7sjgz@pd.tnic>

On Wed, Jun 07, 2017 at 02:17:09PM -0500, Tom Lendacky wrote:
> When Secure Memory Encryption is enabled, the trampoline area must not
> be encrypted. A CPU running in real mode will not be able to decrypt
> memory that has been encrypted because it will not be able to use addresses
> with the memory encryption mask.
> 
> A recent change that added a new system_state value exposed a warning
> issued by early_ioreamp() when the system_state was not SYSTEM_BOOTING.
> At the stage where the trampoline area is decrypted, the system_state is
> now SYSTEM_SCHEDULING. The check was changed to issue a warning if the
> system_state is greater than or equal to SYSTEM_RUNNING.

This piece along with the hunk touching system_state absolutely needs to
be a separate patch as it is unrelated.

-- 
Regards/Gruss,
    Boris.

Good mailing practices for 400: avoid top-posting and trim the reply.


From bp at alien8.de  Wed Jun 14 09:25:48 2017
From: bp at alien8.de (Borislav Petkov)
Date: Wed, 14 Jun 2017 18:25:48 +0200
Subject: [PATCH v6 22/34] x86/mm: Add support for changing the memory
 encryption attribute
In-Reply-To: <20170607191659.28645.1669.stgit@tlendack-t1.amdoffice.net>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
 <20170607191659.28645.1669.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170614162548.crwm44epddyrtgm6@pd.tnic>

On Wed, Jun 07, 2017 at 02:17:00PM -0500, Tom Lendacky wrote:
> Add support for changing the memory encryption attribute for one or more
> memory pages. This will be useful when we have to change the AP trampoline
> area to not be encrypted. Or when we need to change the SWIOTLB area to
> not be encrypted in support of devices that can't support the encryption
> mask range.
> 
> Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
> ---
>  arch/x86/include/asm/set_memory.h |    3 ++
>  arch/x86/mm/pageattr.c            |   62 +++++++++++++++++++++++++++++++++++++
>  2 files changed, 65 insertions(+)

Patches 21-22:

Reviewed-by: Borislav Petkov <bp at suse.de>

-- 
Regards/Gruss,
    Boris.

Good mailing practices for 400: avoid top-posting and trim the reply.


From thomas.lendacky at amd.com  Wed Jun 14 09:38:57 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Wed, 14 Jun 2017 11:38:57 -0500
Subject: [PATCH v6 23/34] x86, realmode: Decrypt trampoline area if memory
 encryption is active
In-Reply-To: <20170614162416.ksa54esy5ql7sjgz@pd.tnic>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
 <20170607191709.28645.69034.stgit@tlendack-t1.amdoffice.net>
 <20170614162416.ksa54esy5ql7sjgz@pd.tnic>
Message-ID: <dffd4702-9598-40ef-c9ad-ef970df13e32@amd.com>

On 6/14/2017 11:24 AM, Borislav Petkov wrote:
> On Wed, Jun 07, 2017 at 02:17:09PM -0500, Tom Lendacky wrote:
>> When Secure Memory Encryption is enabled, the trampoline area must not
>> be encrypted. A CPU running in real mode will not be able to decrypt
>> memory that has been encrypted because it will not be able to use addresses
>> with the memory encryption mask.
>>
>> A recent change that added a new system_state value exposed a warning
>> issued by early_ioreamp() when the system_state was not SYSTEM_BOOTING.
>> At the stage where the trampoline area is decrypted, the system_state is
>> now SYSTEM_SCHEDULING. The check was changed to issue a warning if the
>> system_state is greater than or equal to SYSTEM_RUNNING.
> 
> This piece along with the hunk touching system_state absolutely needs to
> be a separate patch as it is unrelated.

Yup, will do.

Thanks,
Tom

> 


From bp at alien8.de  Wed Jun 14 09:39:04 2017
From: bp at alien8.de (Borislav Petkov)
Date: Wed, 14 Jun 2017 18:39:04 +0200
Subject: [PATCH v6 23/34] x86, realmode: Decrypt trampoline area if
 memory encryption is active
Message-ID: <20170614163903.fvlvscewnuk2u75x@pd.tnic>

On Wed, Jun 14, 2017 at 06:24:16PM +0200, Borislav Petkov wrote:
> On Wed, Jun 07, 2017 at 02:17:09PM -0500, Tom Lendacky wrote:
> > When Secure Memory Encryption is enabled, the trampoline area must not
> > be encrypted. A CPU running in real mode will not be able to decrypt
> > memory that has been encrypted because it will not be able to use addresses
> > with the memory encryption mask.
> > 
> > A recent change that added a new system_state value exposed a warning
> > issued by early_ioreamp() when the system_state was not SYSTEM_BOOTING.
> > At the stage where the trampoline area is decrypted, the system_state is
> > now SYSTEM_SCHEDULING. The check was changed to issue a warning if the
> > system_state is greater than or equal to SYSTEM_RUNNING.
> 
> This piece along with the hunk touching system_state absolutely needs to
> be a separate patch as it is unrelated.

Btw, pls send this now and separate from the patchset as it is a bugfix
that should go into sched/core.

Thanks.

-- 
Regards/Gruss,
    Boris.

Good mailing practices for 400: avoid top-posting and trim the reply.


From bp at alien8.de  Wed Jun 14 09:45:53 2017
From: bp at alien8.de (Borislav Petkov)
Date: Wed, 14 Jun 2017 18:45:53 +0200
Subject: [PATCH v6 24/34] x86, swiotlb: Add memory encryption support
In-Reply-To: <20170607191721.28645.96519.stgit@tlendack-t1.amdoffice.net>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
 <20170607191721.28645.96519.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170614164553.jwcfgugpizz5pc2e@pd.tnic>

On Wed, Jun 07, 2017 at 02:17:21PM -0500, Tom Lendacky wrote:
> Since DMA addresses will effectively look like 48-bit addresses when the
> memory encryption mask is set, SWIOTLB is needed if the DMA mask of the
> device performing the DMA does not support 48-bits. SWIOTLB will be
> initialized to create decrypted bounce buffers for use by these devices.
> 
> Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
> ---

...


> diff --git a/init/main.c b/init/main.c
> index df58a41..7125b5f 100644
> --- a/init/main.c
> +++ b/init/main.c
> @@ -488,6 +488,10 @@ void __init __weak thread_stack_cache_init(void)
>  }
>  #endif
>  
> +void __init __weak mem_encrypt_init(void)
> +{
> +}

void __init __weak mem_encrypt_init(void) { }

saves some real estate. Please do that for the rest of the stubs you're
adding, for the next version.

> +
>  /*
>   * Set up kernel memory allocators
>   */
> @@ -640,6 +644,15 @@ asmlinkage __visible void __init start_kernel(void)
>  	 */
>  	locking_selftest();
>  
> +	/*
> +	 * This needs to be called before any devices perform DMA
> +	 * operations that might use the SWIOTLB bounce buffers.
> +	 * This call will mark the bounce buffers as decrypted so
> +	 * that their usage will not cause "plain-text" data to be
> +	 * decrypted when accessed.

s/This call/It/

> +	 */
> +	mem_encrypt_init();
> +
>  #ifdef CONFIG_BLK_DEV_INITRD
>  	if (initrd_start && !initrd_below_start_ok &&
>  	    page_to_pfn(virt_to_page((void *)initrd_start)) < min_low_pfn) {
> diff --git a/lib/swiotlb.c b/lib/swiotlb.c
> index a8d74a7..74d6557 100644
> --- a/lib/swiotlb.c
> +++ b/lib/swiotlb.c
> @@ -30,6 +30,7 @@
>  #include <linux/highmem.h>
>  #include <linux/gfp.h>
>  #include <linux/scatterlist.h>
> +#include <linux/mem_encrypt.h>
>  
>  #include <asm/io.h>
>  #include <asm/dma.h>
> @@ -155,6 +156,17 @@ unsigned long swiotlb_size_or_default(void)
>  	return size ? size : (IO_TLB_DEFAULT_SIZE);
>  }
>  
> +void __weak swiotlb_set_mem_attributes(void *vaddr, unsigned long size)
> +{
> +}

As above.

-- 
Regards/Gruss,
    Boris.

Good mailing practices for 400: avoid top-posting and trim the reply.


From bp at alien8.de  Wed Jun 14 09:50:52 2017
From: bp at alien8.de (Borislav Petkov)
Date: Wed, 14 Jun 2017 18:50:52 +0200
Subject: [PATCH v6 25/34] swiotlb: Add warnings for use of bounce buffers
 with SME
In-Reply-To: <20170607191732.28645.42876.stgit@tlendack-t1.amdoffice.net>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
 <20170607191732.28645.42876.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170614165052.fyn5t4gkq5leczcc@pd.tnic>

On Wed, Jun 07, 2017 at 02:17:32PM -0500, Tom Lendacky wrote:
> Add warnings to let the user know when bounce buffers are being used for
> DMA when SME is active.  Since the bounce buffers are not in encrypted
> memory, these notifications are to allow the user to determine some
> appropriate action - if necessary.
> 
> Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
> ---
>  arch/x86/include/asm/mem_encrypt.h |    8 ++++++++
>  include/asm-generic/mem_encrypt.h  |    5 +++++
>  include/linux/dma-mapping.h        |    9 +++++++++
>  lib/swiotlb.c                      |    3 +++
>  4 files changed, 25 insertions(+)
> 
> diff --git a/arch/x86/include/asm/mem_encrypt.h b/arch/x86/include/asm/mem_encrypt.h
> index f1215a4..c7a2525 100644
> --- a/arch/x86/include/asm/mem_encrypt.h
> +++ b/arch/x86/include/asm/mem_encrypt.h
> @@ -69,6 +69,14 @@ static inline bool sme_active(void)
>  	return !!sme_me_mask;
>  }
>  
> +static inline u64 sme_dma_mask(void)
> +{
> +	if (!sme_me_mask)
> +		return 0ULL;
> +
> +	return ((u64)sme_me_mask << 1) - 1;
> +}
> +
>  /*
>   * The __sme_pa() and __sme_pa_nodebug() macros are meant for use when
>   * writing to or comparing values from the cr3 register.  Having the
> diff --git a/include/asm-generic/mem_encrypt.h b/include/asm-generic/mem_encrypt.h
> index b55c3f9..fb02ff0 100644
> --- a/include/asm-generic/mem_encrypt.h
> +++ b/include/asm-generic/mem_encrypt.h
> @@ -22,6 +22,11 @@ static inline bool sme_active(void)
>  	return false;
>  }
>  
> +static inline u64 sme_dma_mask(void)
> +{
> +	return 0ULL;
> +}
> +
>  /*
>   * The __sme_set() and __sme_clr() macros are useful for adding or removing
>   * the encryption mask from a value (e.g. when dealing with pagetable
> diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h
> index 4f3eece..e2c5fda 100644
> --- a/include/linux/dma-mapping.h
> +++ b/include/linux/dma-mapping.h
> @@ -10,6 +10,7 @@
>  #include <linux/scatterlist.h>
>  #include <linux/kmemcheck.h>
>  #include <linux/bug.h>
> +#include <linux/mem_encrypt.h>
>  
>  /**
>   * List of possible attributes associated with a DMA mapping. The semantics
> @@ -577,6 +578,10 @@ static inline int dma_set_mask(struct device *dev, u64 mask)
>  
>  	if (!dev->dma_mask || !dma_supported(dev, mask))
>  		return -EIO;
> +
> +	if (sme_active() && (mask < sme_dma_mask()))
> +		dev_warn(dev, "SME is active, device will require DMA bounce buffers\n");

Something looks strange here:

you're checking sme_active() before calling sme_dma_mask() and yet in
it, you're checking !sme_me_mask again. What gives?

Why not move the sme_active() check into sme_dma_mask() and thus
simplify callers?

-- 
Regards/Gruss,
    Boris.

Good mailing practices for 400: avoid top-posting and trim the reply.


From thomas.lendacky at amd.com  Wed Jun 14 10:06:54 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Wed, 14 Jun 2017 12:06:54 -0500
Subject: [PATCH v6 20/34] x86, mpparse: Use memremap to map the mpf and
 mpc data
In-Reply-To: <20170614160754.c4ywbf5ktqwgc4ij@pd.tnic>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
 <20170607191643.28645.91679.stgit@tlendack-t1.amdoffice.net>
 <20170614160754.c4ywbf5ktqwgc4ij@pd.tnic>
Message-ID: <86f31710-76d0-5fee-f4a7-8cdb4b9b9a8e@amd.com>

On 6/14/2017 11:07 AM, Borislav Petkov wrote:
> On Wed, Jun 07, 2017 at 02:16:43PM -0500, Tom Lendacky wrote:
>> The SMP MP-table is built by UEFI and placed in memory in a decrypted
>> state. These tables are accessed using a mix of early_memremap(),
>> early_memunmap(), phys_to_virt() and virt_to_phys(). Change all accesses
>> to use early_memremap()/early_memunmap(). This allows for proper setting
>> of the encryption mask so that the data can be successfully accessed when
>> SME is active.
>>
>> Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
>> ---
>>   arch/x86/kernel/mpparse.c |   98 ++++++++++++++++++++++++++++++++-------------
>>   1 file changed, 70 insertions(+), 28 deletions(-)
> 
> ...
> 
>> @@ -515,6 +516,12 @@ void __init default_get_smp_config(unsigned int early)
>>   	if (acpi_lapic && acpi_ioapic)
>>   		return;
>>   
>> +	mpf = early_memremap(mpf_base, sizeof(*mpf));
>> +	if (!mpf) {
>> +		pr_err("MPTABLE: mpf early_memremap() failed\n");
> 
> If you're going to introduce new prefixes then add:

This isn't new...  there are a number of messages issued in this file
with that prefix, so I was just following convention.  Changing the
prefix could be a follow-on patch.

> 
> #undef pr_fmt
> #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
> 
> at the beginning of the file so that they all say "mpparse:" instead.
> 
> And pls make that message more user-friendly: "Error mapping MP table"
> or so.

Can do.

Thanks,
Tom

> 


From bp at alien8.de  Wed Jun 14 10:27:18 2017
From: bp at alien8.de (Borislav Petkov)
Date: Wed, 14 Jun 2017 19:27:18 +0200
Subject: [PATCH v6 20/34] x86, mpparse: Use memremap to map the mpf and
 mpc data
In-Reply-To: <86f31710-76d0-5fee-f4a7-8cdb4b9b9a8e@amd.com>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
 <20170607191643.28645.91679.stgit@tlendack-t1.amdoffice.net>
 <20170614160754.c4ywbf5ktqwgc4ij@pd.tnic>
 <86f31710-76d0-5fee-f4a7-8cdb4b9b9a8e@amd.com>
Message-ID: <20170614172718.3opzawxsgobiy2li@pd.tnic>

On Wed, Jun 14, 2017 at 12:06:54PM -0500, Tom Lendacky wrote:
> This isn't new...  there are a number of messages issued in this file
> with that prefix, so I was just following convention.

The "convention" that some of the messages are prefixed and some aren't?

:-)

> Changing the prefix could be a follow-on patch.

Ok. As some of those print statements have prefixes and some don't,
let's unify them.

Thanks.

-- 
Regards/Gruss,
    Boris.

Good mailing practices for 400: avoid top-posting and trim the reply.


From bp at alien8.de  Wed Jun 14 10:42:08 2017
From: bp at alien8.de (Borislav Petkov)
Date: Wed, 14 Jun 2017 19:42:08 +0200
Subject: [PATCH v6 26/34] iommu/amd: Allow the AMD IOMMU to work with
 memory encryption
In-Reply-To: <20170607191745.28645.81756.stgit@tlendack-t1.amdoffice.net>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
 <20170607191745.28645.81756.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170614174208.p2yr5exs4b6pjxhf@pd.tnic>

On Wed, Jun 07, 2017 at 02:17:45PM -0500, Tom Lendacky wrote:
> The IOMMU is programmed with physical addresses for the various tables
> and buffers that are used to communicate between the device and the
> driver. When the driver allocates this memory it is encrypted. In order
> for the IOMMU to access the memory as encrypted the encryption mask needs
> to be included in these physical addresses during configuration.
> 
> The PTE entries created by the IOMMU should also include the encryption
> mask so that when the device behind the IOMMU performs a DMA, the DMA
> will be performed to encrypted memory.
> 
> Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
> ---
>  arch/x86/include/asm/mem_encrypt.h |    7 +++++++
>  arch/x86/mm/mem_encrypt.c          |   30 ++++++++++++++++++++++++++++++
>  drivers/iommu/amd_iommu.c          |   36 +++++++++++++++++++-----------------
>  drivers/iommu/amd_iommu_init.c     |   18 ++++++++++++------
>  drivers/iommu/amd_iommu_proto.h    |   10 ++++++++++
>  drivers/iommu/amd_iommu_types.h    |    2 +-
>  include/asm-generic/mem_encrypt.h  |    5 +++++
>  7 files changed, 84 insertions(+), 24 deletions(-)
> 
> diff --git a/arch/x86/include/asm/mem_encrypt.h b/arch/x86/include/asm/mem_encrypt.h
> index c7a2525..d86e544 100644
> --- a/arch/x86/include/asm/mem_encrypt.h
> +++ b/arch/x86/include/asm/mem_encrypt.h
> @@ -31,6 +31,8 @@ void __init sme_early_decrypt(resource_size_t paddr,
>  
>  void __init sme_early_init(void);
>  
> +bool sme_iommu_supported(void);
> +
>  /* Architecture __weak replacement functions */
>  void __init mem_encrypt_init(void);
>  
> @@ -62,6 +64,11 @@ static inline void __init sme_early_init(void)
>  {
>  }
>  
> +static inline bool sme_iommu_supported(void)
> +{
> +	return true;
> +}

Some more file real-estate saving:

static inline bool sme_iommu_supported(void) 	{ return true; }

> +
>  #endif	/* CONFIG_AMD_MEM_ENCRYPT */
>  
>  static inline bool sme_active(void)
> diff --git a/arch/x86/mm/mem_encrypt.c b/arch/x86/mm/mem_encrypt.c
> index 5d7c51d..018b58a 100644
> --- a/arch/x86/mm/mem_encrypt.c
> +++ b/arch/x86/mm/mem_encrypt.c
> @@ -197,6 +197,36 @@ void __init sme_early_init(void)
>  		protection_map[i] = pgprot_encrypted(protection_map[i]);
>  }
>  
> +bool sme_iommu_supported(void)

Why is this one exported with all the header file declarations if it is
going to be used in the iommu code only? IOW, you can make it a static
function there and save yourself all the exporting.

> +{
> +	struct cpuinfo_x86 *c = &boot_cpu_data;
> +
> +	if (!sme_me_mask || (c->x86 != 0x17))

me_mask or sme_active()?

Or is the IOMMU "disabled" in a way the moment the BIOS decides that SME
can be enabled?

Also, family checks are always a bad idea for enablement. Why do we need
the family check? Because future families will work with the IOMMU? :-)

> +		return true;
> +
> +	/* For Fam17h, a specific level of support is required */
> +	switch (c->microcode & 0xf000) {

Also, you said in another mail on this subthread that c->microcode
is not yet set. Are you saying, that the iommu init gunk runs before
init_amd(), where we do set c->microcode?

If so, we can move the setting to early_init_amd() or so.

> +	case 0x0000:
> +		return false;
> +	case 0x1000:
> +		switch (c->microcode & 0x0f00) {
> +		case 0x0000:
> +			return false;
> +		case 0x0100:
> +			if ((c->microcode & 0xff) < 0x26)
> +				return false;
> +			break;
> +		case 0x0200:
> +			if ((c->microcode & 0xff) < 0x05)
> +				return false;
> +			break;
> +		}

So this is the microcode revision, why those complex compares? Can't you
simply check a range of values?

> +		break;
> +	}
> +
> +	return true;
> +}
> +
>  /* Architecture __weak replacement functions */
>  void __init mem_encrypt_init(void)
>  {
> diff --git a/drivers/iommu/amd_iommu.c b/drivers/iommu/amd_iommu.c
> index 63cacf5..94eb130 100644
> --- a/drivers/iommu/amd_iommu.c
> +++ b/drivers/iommu/amd_iommu.c
> @@ -544,7 +544,7 @@ static void dump_dte_entry(u16 devid)
>  
>  static void dump_command(unsigned long phys_addr)
>  {
> -	struct iommu_cmd *cmd = phys_to_virt(phys_addr);
> +	struct iommu_cmd *cmd = iommu_phys_to_virt(phys_addr);
>  	int i;
>  
>  	for (i = 0; i < 4; ++i)
> @@ -863,13 +863,15 @@ static void copy_cmd_to_buffer(struct amd_iommu *iommu,
>  	writel(tail, iommu->mmio_base + MMIO_CMD_TAIL_OFFSET);
>  }
>  
> -static void build_completion_wait(struct iommu_cmd *cmd, u64 address)
> +static void build_completion_wait(struct iommu_cmd *cmd, volatile u64 *sem)

WARNING: Use of volatile is usually wrong: see Documentation/process/volatile-considered-harmful.rst
#134: FILE: drivers/iommu/amd_iommu.c:866:
+static void build_completion_wait(struct iommu_cmd *cmd, volatile u64 *sem)

>  {
> +	u64 address = iommu_virt_to_phys((void *)sem);
> +
>  	WARN_ON(address & 0x7ULL);
>  
>  	memset(cmd, 0, sizeof(*cmd));
> -	cmd->data[0] = lower_32_bits(__pa(address)) | CMD_COMPL_WAIT_STORE_MASK;
> -	cmd->data[1] = upper_32_bits(__pa(address));
> +	cmd->data[0] = lower_32_bits(address) | CMD_COMPL_WAIT_STORE_MASK;
> +	cmd->data[1] = upper_32_bits(address);
>  	cmd->data[2] = 1;
>  	CMD_SET_TYPE(cmd, CMD_COMPL_WAIT);

<... snip stuff which Joerg needs to review... >

> diff --git a/include/asm-generic/mem_encrypt.h b/include/asm-generic/mem_encrypt.h
> index fb02ff0..bbc49e1 100644
> --- a/include/asm-generic/mem_encrypt.h
> +++ b/include/asm-generic/mem_encrypt.h
> @@ -27,6 +27,11 @@ static inline u64 sme_dma_mask(void)
>  	return 0ULL;
>  }
>  
> +static inline bool sme_iommu_supported(void)
> +{
> +	return true;
> +}

Save some more file real-estate... you get the idea by now, I'm sure.

:-)

-- 
Regards/Gruss,
    Boris.

Good mailing practices for 400: avoid top-posting and trim the reply.


From thomas.lendacky at amd.com  Wed Jun 14 12:38:48 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Wed, 14 Jun 2017 14:38:48 -0500
Subject: [PATCH v6 24/34] x86, swiotlb: Add memory encryption support
In-Reply-To: <20170614164553.jwcfgugpizz5pc2e@pd.tnic>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
 <20170607191721.28645.96519.stgit@tlendack-t1.amdoffice.net>
 <20170614164553.jwcfgugpizz5pc2e@pd.tnic>
Message-ID: <c1a59a06-2850-b215-5c08-27adff6fecde@amd.com>

On 6/14/2017 11:45 AM, Borislav Petkov wrote:
> On Wed, Jun 07, 2017 at 02:17:21PM -0500, Tom Lendacky wrote:
>> Since DMA addresses will effectively look like 48-bit addresses when the
>> memory encryption mask is set, SWIOTLB is needed if the DMA mask of the
>> device performing the DMA does not support 48-bits. SWIOTLB will be
>> initialized to create decrypted bounce buffers for use by these devices.
>>
>> Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
>> ---
> 
> ...
> 
> 
>> diff --git a/init/main.c b/init/main.c
>> index df58a41..7125b5f 100644
>> --- a/init/main.c
>> +++ b/init/main.c
>> @@ -488,6 +488,10 @@ void __init __weak thread_stack_cache_init(void)
>>   }
>>   #endif
>>   
>> +void __init __weak mem_encrypt_init(void)
>> +{
>> +}
> 
> void __init __weak mem_encrypt_init(void) { }
> 
> saves some real estate. Please do that for the rest of the stubs you're
> adding, for the next version.

Ok, will do.

Thanks,
Tom

> 
>> +
>>   /*
>>    * Set up kernel memory allocators
>>    */
>> @@ -640,6 +644,15 @@ asmlinkage __visible void __init start_kernel(void)
>>   	 */
>>   	locking_selftest();
>>   
>> +	/*
>> +	 * This needs to be called before any devices perform DMA
>> +	 * operations that might use the SWIOTLB bounce buffers.
>> +	 * This call will mark the bounce buffers as decrypted so
>> +	 * that their usage will not cause "plain-text" data to be
>> +	 * decrypted when accessed.
> 
> s/This call/It/
> 
>> +	 */
>> +	mem_encrypt_init();
>> +
>>   #ifdef CONFIG_BLK_DEV_INITRD
>>   	if (initrd_start && !initrd_below_start_ok &&
>>   	    page_to_pfn(virt_to_page((void *)initrd_start)) < min_low_pfn) {
>> diff --git a/lib/swiotlb.c b/lib/swiotlb.c
>> index a8d74a7..74d6557 100644
>> --- a/lib/swiotlb.c
>> +++ b/lib/swiotlb.c
>> @@ -30,6 +30,7 @@
>>   #include <linux/highmem.h>
>>   #include <linux/gfp.h>
>>   #include <linux/scatterlist.h>
>> +#include <linux/mem_encrypt.h>
>>   
>>   #include <asm/io.h>
>>   #include <asm/dma.h>
>> @@ -155,6 +156,17 @@ unsigned long swiotlb_size_or_default(void)
>>   	return size ? size : (IO_TLB_DEFAULT_SIZE);
>>   }
>>   
>> +void __weak swiotlb_set_mem_attributes(void *vaddr, unsigned long size)
>> +{
>> +}
> 
> As above.
> 


From thomas.lendacky at amd.com  Wed Jun 14 12:49:02 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Wed, 14 Jun 2017 14:49:02 -0500
Subject: [PATCH v6 25/34] swiotlb: Add warnings for use of bounce buffers
 with SME
In-Reply-To: <20170614165052.fyn5t4gkq5leczcc@pd.tnic>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
 <20170607191732.28645.42876.stgit@tlendack-t1.amdoffice.net>
 <20170614165052.fyn5t4gkq5leczcc@pd.tnic>
Message-ID: <33d1debc-c684-cba1-7d95-493678f086d0@amd.com>

On 6/14/2017 11:50 AM, Borislav Petkov wrote:
> On Wed, Jun 07, 2017 at 02:17:32PM -0500, Tom Lendacky wrote:
>> Add warnings to let the user know when bounce buffers are being used for
>> DMA when SME is active.  Since the bounce buffers are not in encrypted
>> memory, these notifications are to allow the user to determine some
>> appropriate action - if necessary.
>>
>> Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
>> ---
>>   arch/x86/include/asm/mem_encrypt.h |    8 ++++++++
>>   include/asm-generic/mem_encrypt.h  |    5 +++++
>>   include/linux/dma-mapping.h        |    9 +++++++++
>>   lib/swiotlb.c                      |    3 +++
>>   4 files changed, 25 insertions(+)
>>
>> diff --git a/arch/x86/include/asm/mem_encrypt.h b/arch/x86/include/asm/mem_encrypt.h
>> index f1215a4..c7a2525 100644
>> --- a/arch/x86/include/asm/mem_encrypt.h
>> +++ b/arch/x86/include/asm/mem_encrypt.h
>> @@ -69,6 +69,14 @@ static inline bool sme_active(void)
>>   	return !!sme_me_mask;
>>   }
>>   
>> +static inline u64 sme_dma_mask(void)
>> +{
>> +	if (!sme_me_mask)
>> +		return 0ULL;
>> +
>> +	return ((u64)sme_me_mask << 1) - 1;
>> +}
>> +
>>   /*
>>    * The __sme_pa() and __sme_pa_nodebug() macros are meant for use when
>>    * writing to or comparing values from the cr3 register.  Having the
>> diff --git a/include/asm-generic/mem_encrypt.h b/include/asm-generic/mem_encrypt.h
>> index b55c3f9..fb02ff0 100644
>> --- a/include/asm-generic/mem_encrypt.h
>> +++ b/include/asm-generic/mem_encrypt.h
>> @@ -22,6 +22,11 @@ static inline bool sme_active(void)
>>   	return false;
>>   }
>>   
>> +static inline u64 sme_dma_mask(void)
>> +{
>> +	return 0ULL;
>> +}
>> +
>>   /*
>>    * The __sme_set() and __sme_clr() macros are useful for adding or removing
>>    * the encryption mask from a value (e.g. when dealing with pagetable
>> diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h
>> index 4f3eece..e2c5fda 100644
>> --- a/include/linux/dma-mapping.h
>> +++ b/include/linux/dma-mapping.h
>> @@ -10,6 +10,7 @@
>>   #include <linux/scatterlist.h>
>>   #include <linux/kmemcheck.h>
>>   #include <linux/bug.h>
>> +#include <linux/mem_encrypt.h>
>>   
>>   /**
>>    * List of possible attributes associated with a DMA mapping. The semantics
>> @@ -577,6 +578,10 @@ static inline int dma_set_mask(struct device *dev, u64 mask)
>>   
>>   	if (!dev->dma_mask || !dma_supported(dev, mask))
>>   		return -EIO;
>> +
>> +	if (sme_active() && (mask < sme_dma_mask()))
>> +		dev_warn(dev, "SME is active, device will require DMA bounce buffers\n");
> 
> Something looks strange here:
> 
> you're checking sme_active() before calling sme_dma_mask() and yet in
> it, you're checking !sme_me_mask again. What gives?
> 

I guess I don't need the sme_active() check since the second part of the
if statement can only ever be true if SME is active (since mask is
unsigned).

Thanks,
Tom

> Why not move the sme_active() check into sme_dma_mask() and thus
> simplify callers?
> 


From thomas.lendacky at amd.com  Wed Jun 14 13:40:28 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Wed, 14 Jun 2017 15:40:28 -0500
Subject: [PATCH v6 26/34] iommu/amd: Allow the AMD IOMMU to work with
 memory encryption
In-Reply-To: <20170614174208.p2yr5exs4b6pjxhf@pd.tnic>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
 <20170607191745.28645.81756.stgit@tlendack-t1.amdoffice.net>
 <20170614174208.p2yr5exs4b6pjxhf@pd.tnic>
Message-ID: <0611d01a-19f8-d6ae-2682-932789855518@amd.com>

On 6/14/2017 12:42 PM, Borislav Petkov wrote:
> On Wed, Jun 07, 2017 at 02:17:45PM -0500, Tom Lendacky wrote:
>> The IOMMU is programmed with physical addresses for the various tables
>> and buffers that are used to communicate between the device and the
>> driver. When the driver allocates this memory it is encrypted. In order
>> for the IOMMU to access the memory as encrypted the encryption mask needs
>> to be included in these physical addresses during configuration.
>>
>> The PTE entries created by the IOMMU should also include the encryption
>> mask so that when the device behind the IOMMU performs a DMA, the DMA
>> will be performed to encrypted memory.
>>
>> Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
>> ---
>>   arch/x86/include/asm/mem_encrypt.h |    7 +++++++
>>   arch/x86/mm/mem_encrypt.c          |   30 ++++++++++++++++++++++++++++++
>>   drivers/iommu/amd_iommu.c          |   36 +++++++++++++++++++-----------------
>>   drivers/iommu/amd_iommu_init.c     |   18 ++++++++++++------
>>   drivers/iommu/amd_iommu_proto.h    |   10 ++++++++++
>>   drivers/iommu/amd_iommu_types.h    |    2 +-
>>   include/asm-generic/mem_encrypt.h  |    5 +++++
>>   7 files changed, 84 insertions(+), 24 deletions(-)
>>
>> diff --git a/arch/x86/include/asm/mem_encrypt.h b/arch/x86/include/asm/mem_encrypt.h
>> index c7a2525..d86e544 100644
>> --- a/arch/x86/include/asm/mem_encrypt.h
>> +++ b/arch/x86/include/asm/mem_encrypt.h
>> @@ -31,6 +31,8 @@ void __init sme_early_decrypt(resource_size_t paddr,
>>   
>>   void __init sme_early_init(void);
>>   
>> +bool sme_iommu_supported(void);
>> +
>>   /* Architecture __weak replacement functions */
>>   void __init mem_encrypt_init(void);
>>   
>> @@ -62,6 +64,11 @@ static inline void __init sme_early_init(void)
>>   {
>>   }
>>   
>> +static inline bool sme_iommu_supported(void)
>> +{
>> +	return true;
>> +}
> 
> Some more file real-estate saving:
> 
> static inline bool sme_iommu_supported(void) 	{ return true; }
> 
>> +
>>   #endif	/* CONFIG_AMD_MEM_ENCRYPT */
>>   
>>   static inline bool sme_active(void)
>> diff --git a/arch/x86/mm/mem_encrypt.c b/arch/x86/mm/mem_encrypt.c
>> index 5d7c51d..018b58a 100644
>> --- a/arch/x86/mm/mem_encrypt.c
>> +++ b/arch/x86/mm/mem_encrypt.c
>> @@ -197,6 +197,36 @@ void __init sme_early_init(void)
>>   		protection_map[i] = pgprot_encrypted(protection_map[i]);
>>   }
>>   
>> +bool sme_iommu_supported(void)
> 
> Why is this one exported with all the header file declarations if it is
> going to be used in the iommu code only? IOW, you can make it a static
> function there and save yourself all the exporting.

I was trying to keep all the logic for it here in the SME related files
rather than put it in the iommu code itself. But it is easy enough to
move if you think it's worth it.

> 
>> +{
>> +	struct cpuinfo_x86 *c = &boot_cpu_data;
>> +
>> +	if (!sme_me_mask || (c->x86 != 0x17))
> 
> me_mask or sme_active()?

I like using sme_active() outside of the SME-specific files and using
sme_me_mask in the SME-specific files to save any changes that will have
to be made once SEV comes around.

> 
> Or is the IOMMU "disabled" in a way the moment the BIOS decides that SME
> can be enabled?

There's a fix in the AGESA layer of the BIOS that permits the IOMMU to
function properly when SME is enabled.  Unfortunately, the only easy way
to determine if that fix is available is through the patch level check.

> 
> Also, family checks are always a bad idea for enablement. Why do we need
> the family check? Because future families will work with the IOMMU? :-)

Yes, any future family that supports SME will (should) work with the
IOMMU without having to check patch levels.

> 
>> +		return true;
>> +
>> +	/* For Fam17h, a specific level of support is required */
>> +	switch (c->microcode & 0xf000) {
> 
> Also, you said in another mail on this subthread that c->microcode
> is not yet set. Are you saying, that the iommu init gunk runs before
> init_amd(), where we do set c->microcode?
> 
> If so, we can move the setting to early_init_amd() or so.

I'll look into that.

> 
>> +	case 0x0000:
>> +		return false;
>> +	case 0x1000:
>> +		switch (c->microcode & 0x0f00) {
>> +		case 0x0000:
>> +			return false;
>> +		case 0x0100:
>> +			if ((c->microcode & 0xff) < 0x26)
>> +				return false;
>> +			break;
>> +		case 0x0200:
>> +			if ((c->microcode & 0xff) < 0x05)
>> +				return false;
>> +			break;
>> +		}
> 
> So this is the microcode revision, why those complex compares? Can't you
> simply check a range of values?

I'll look into simplifying the checks.

> 
>> +		break;
>> +	}
>> +
>> +	return true;
>> +}
>> +
>>   /* Architecture __weak replacement functions */
>>   void __init mem_encrypt_init(void)
>>   {
>> diff --git a/drivers/iommu/amd_iommu.c b/drivers/iommu/amd_iommu.c
>> index 63cacf5..94eb130 100644
>> --- a/drivers/iommu/amd_iommu.c
>> +++ b/drivers/iommu/amd_iommu.c
>> @@ -544,7 +544,7 @@ static void dump_dte_entry(u16 devid)
>>   
>>   static void dump_command(unsigned long phys_addr)
>>   {
>> -	struct iommu_cmd *cmd = phys_to_virt(phys_addr);
>> +	struct iommu_cmd *cmd = iommu_phys_to_virt(phys_addr);
>>   	int i;
>>   
>>   	for (i = 0; i < 4; ++i)
>> @@ -863,13 +863,15 @@ static void copy_cmd_to_buffer(struct amd_iommu *iommu,
>>   	writel(tail, iommu->mmio_base + MMIO_CMD_TAIL_OFFSET);
>>   }
>>   
>> -static void build_completion_wait(struct iommu_cmd *cmd, u64 address)
>> +static void build_completion_wait(struct iommu_cmd *cmd, volatile u64 *sem)
> 
> WARNING: Use of volatile is usually wrong: see Documentation/process/volatile-considered-harmful.rst
> #134: FILE: drivers/iommu/amd_iommu.c:866:
> +static void build_completion_wait(struct iommu_cmd *cmd, volatile u64 *sem)
> 

The semaphore area is written to by the device so the use of volatile is
appropriate in this case.

Thanks,
Tom

>>   {
>> +	u64 address = iommu_virt_to_phys((void *)sem);
>> +
>>   	WARN_ON(address & 0x7ULL);
>>   
>>   	memset(cmd, 0, sizeof(*cmd));
>> -	cmd->data[0] = lower_32_bits(__pa(address)) | CMD_COMPL_WAIT_STORE_MASK;
>> -	cmd->data[1] = upper_32_bits(__pa(address));
>> +	cmd->data[0] = lower_32_bits(address) | CMD_COMPL_WAIT_STORE_MASK;
>> +	cmd->data[1] = upper_32_bits(address);
>>   	cmd->data[2] = 1;
>>   	CMD_SET_TYPE(cmd, CMD_COMPL_WAIT);
> 
> <... snip stuff which Joerg needs to review... >
> 
>> diff --git a/include/asm-generic/mem_encrypt.h b/include/asm-generic/mem_encrypt.h
>> index fb02ff0..bbc49e1 100644
>> --- a/include/asm-generic/mem_encrypt.h
>> +++ b/include/asm-generic/mem_encrypt.h
>> @@ -27,6 +27,11 @@ static inline u64 sme_dma_mask(void)
>>   	return 0ULL;
>>   }
>>   
>> +static inline bool sme_iommu_supported(void)
>> +{
>> +	return true;
>> +}
> 
> Save some more file real-estate... you get the idea by now, I'm sure.
> 
> :-)
> 


From panand at redhat.com  Wed Jun 14 19:37:12 2017
From: panand at redhat.com (Pratyush Anand)
Date: Thu, 15 Jun 2017 08:07:12 +0530
Subject: [makedumpfile PATCH v3 2/2] print_info: show the remaining time
 of dump progress
In-Reply-To: <1497433292-6015-2-git-send-email-piliu@redhat.com>
References: <1497433292-6015-1-git-send-email-piliu@redhat.com>
 <1497433292-6015-2-git-send-email-piliu@redhat.com>
Message-ID: <7cb91f63-828d-9e98-c846-19c9215c9bed@redhat.com>



On Wednesday 14 June 2017 03:11 PM, Pingfan Liu wrote:
> If the dump progress is slow, it is better to show  an estimate of
> remaining time of the progress to the user. The estimator simply uses
> the average speed to work.
> Before this patch, the bar looks like:
>   Copying data                                      : [  1.8 %] /
> After this patch:
>   Copying data                                      : [  1.8 %] /          eta: 39s
>
> Signed-off-by: Pingfan Liu <piliu at redhat.com>
> ---
> v2 -> v3: improve the commit log

for the series

Reviewed-by: Pratyush Anand <panand at redhat.com>

>
>  makedumpfile.c | 46 +++++++++++++++++++++++-----------------------
>  print_info.c   | 41 +++++++++++++++++++++++++++++++++++------
>  print_info.h   |  3 ++-
>  3 files changed, 60 insertions(+), 30 deletions(-)
>
> diff --git a/makedumpfile.c b/makedumpfile.c
> index 301772a..35a36ce 100644
> --- a/makedumpfile.c
> +++ b/makedumpfile.c
> @@ -5175,7 +5175,7 @@ _exclude_free_page(struct cycle *cycle)
>
>  	for (num_nodes = 1; num_nodes <= vt.numnodes; num_nodes++) {
>
> -		print_progress(PROGRESS_FREE_PAGES, num_nodes - 1, vt.numnodes);
> +		print_progress(PROGRESS_FREE_PAGES, num_nodes - 1, vt.numnodes, NULL);
>
>  		node_zones = pgdat + OFFSET(pglist_data.node_zones);
>
> @@ -5188,7 +5188,7 @@ _exclude_free_page(struct cycle *cycle)
>  		for (i = 0; i < nr_zones; i++) {
>
>  			print_progress(PROGRESS_FREE_PAGES, i + nr_zones * (num_nodes - 1),
> -					nr_zones * vt.numnodes);
> +					nr_zones * vt.numnodes, NULL);
>
>  			zone = node_zones + (i * SIZE(zone));
>  			if (!readmem(VADDR, zone + OFFSET(zone.spanned_pages),
> @@ -5216,7 +5216,7 @@ _exclude_free_page(struct cycle *cycle)
>  	/*
>  	 * print [100 %]
>  	 */
> -	print_progress(PROGRESS_FREE_PAGES, vt.numnodes, vt.numnodes);
> +	print_progress(PROGRESS_FREE_PAGES, vt.numnodes, vt.numnodes, NULL);
>  	print_execution_time(PROGRESS_FREE_PAGES, &tv_start);
>
>  	return TRUE;
> @@ -5403,7 +5403,7 @@ create_1st_bitmap_file(void)
>  	for (i = 0; get_pt_load(i, &phys_start, &phys_end, NULL, NULL); i++) {
>
>  		if (!info->flag_mem_usage)
> -			print_progress(PROGRESS_HOLES, i, num_pt_loads);
> +			print_progress(PROGRESS_HOLES, i, num_pt_loads, NULL);
>
>  		pfn_start = paddr_to_pfn(phys_start);
>  		pfn_end   = paddr_to_pfn(phys_end);
> @@ -5422,7 +5422,7 @@ create_1st_bitmap_file(void)
>  	 * print 100 %
>  	 */
>  	if (!info->flag_mem_usage) {
> -		print_progress(PROGRESS_HOLES, info->max_mapnr, info->max_mapnr);
> +		print_progress(PROGRESS_HOLES, info->max_mapnr, info->max_mapnr, NULL);
>  		print_execution_time(PROGRESS_HOLES, &tv_start);
>  	}
>
> @@ -5548,7 +5548,7 @@ create_bitmap_from_memhole(struct cycle *cycle, struct dump_bitmap *bitmap, int
>  		pfn_start = MAX(paddr_to_pfn(phys_start), cycle->start_pfn);
>  		pfn_end = MIN(paddr_to_pfn(phys_end), cycle->end_pfn);
>
> -		print_progress(PROGRESS_HOLES, i, num_pt_loads);
> +		print_progress(PROGRESS_HOLES, i, num_pt_loads, NULL);
>
>  		if (pfn_start >= pfn_end)
>  			continue;
> @@ -5587,7 +5587,7 @@ create_bitmap_from_memhole(struct cycle *cycle, struct dump_bitmap *bitmap, int
>  	/*
>  	 * print 100 %
>  	 */
> -	print_progress(PROGRESS_HOLES, info->max_mapnr, info->max_mapnr);
> +	print_progress(PROGRESS_HOLES, info->max_mapnr, info->max_mapnr, NULL);
>  	print_execution_time(PROGRESS_HOLES, &tv_start);
>
>  	return TRUE;
> @@ -5842,7 +5842,7 @@ exclude_unnecessary_pages(struct cycle *cycle)
>  	for (mm = 0; mm < info->num_mem_map; mm++) {
>
>  		if (!info->flag_mem_usage)
> -			print_progress(PROGRESS_UNN_PAGES, mm, info->num_mem_map);
> +			print_progress(PROGRESS_UNN_PAGES, mm, info->num_mem_map, NULL);
>
>  		mmd = &info->mem_map_data[mm];
>
> @@ -5860,7 +5860,7 @@ exclude_unnecessary_pages(struct cycle *cycle)
>  	 * print [100 %]
>  	 */
>  	if (!info->flag_mem_usage) {
> -		print_progress(PROGRESS_UNN_PAGES, info->num_mem_map, info->num_mem_map);
> +		print_progress(PROGRESS_UNN_PAGES, info->num_mem_map, info->num_mem_map, NULL);
>  		print_execution_time(PROGRESS_UNN_PAGES, &tv_start);
>  	}
>
> @@ -7334,7 +7334,7 @@ write_elf_pages_cyclic(struct cache_data *cd_header, struct cache_data *cd_page)
>  				}
>
>  				if ((num_dumped % per) == 0)
> -					print_progress(PROGRESS_COPY, num_dumped, num_dumpable);
> +					print_progress(PROGRESS_COPY, num_dumped, num_dumpable, &tv_start);
>
>  				num_dumped++;
>
> @@ -7453,7 +7453,7 @@ write_elf_pages_cyclic(struct cache_data *cd_header, struct cache_data *cd_page)
>  	/*
>  	 * print [100 %]
>  	 */
> -	print_progress(PROGRESS_COPY, num_dumpable, num_dumpable);
> +	print_progress(PROGRESS_COPY, num_dumpable, num_dumpable, &tv_start);
>  	print_execution_time(PROGRESS_COPY, &tv_start);
>  	PROGRESS_MSG("\n");
>
> @@ -7931,7 +7931,7 @@ write_kdump_pages_parallel_cyclic(struct cache_data *cd_header,
>  		}
>
>  		if ((num_dumped % per) == 0)
> -			print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable);
> +			print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable, &tv_start);
>
>  		num_dumped++;
>
> @@ -7967,7 +7967,7 @@ finish:
>  	/*
>  	 * print [100 %]
>  	 */
> -	print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable);
> +	print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable, &tv_start);
>  	print_execution_time(PROGRESS_COPY, &tv_start);
>  	PROGRESS_MSG("\n");
>
> @@ -8084,7 +8084,7 @@ write_kdump_pages_cyclic(struct cache_data *cd_header, struct cache_data *cd_pag
>  	for (pfn = start_pfn; pfn < end_pfn; pfn++) {
>
>  		if ((num_dumped % per) == 0)
> -			print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable);
> +			print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable, &tv_start);
>
>  		/*
>  		 * Check the excluded page.
> @@ -8164,7 +8164,7 @@ out:
>  		free(wrkmem);
>  #endif
>
> -	print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable);
> +	print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable, &tv_start);
>  	print_execution_time(PROGRESS_COPY, &tv_start);
>
>  	return ret;
> @@ -8622,7 +8622,7 @@ write_kdump_pages_and_bitmap_cyclic(struct cache_data *cd_header, struct cache_d
>  	/*
>  	 * print [100 %]
>  	 */
> -	print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable);
> +	print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable, &tv_start);
>  	print_execution_time(PROGRESS_COPY, &tv_start);
>  	PROGRESS_MSG("\n");
>
> @@ -9216,7 +9216,7 @@ exclude_xen3_user_domain(void)
>  	 */
>  	for (i = 0; get_pt_load(i, &phys_start, &phys_end, NULL, NULL); i++) {
>
> -		print_progress(PROGRESS_XEN_DOMAIN, i, num_pt_loads);
> +		print_progress(PROGRESS_XEN_DOMAIN, i, num_pt_loads, NULL);
>
>  		pfn     = paddr_to_pfn(phys_start);
>  		pfn_end = paddr_to_pfn(phys_end);
> @@ -9224,7 +9224,7 @@ exclude_xen3_user_domain(void)
>
>  		for (j = 0; pfn < pfn_end; pfn++, j++) {
>  			print_progress(PROGRESS_XEN_DOMAIN, j + (size * i),
> -					size * num_pt_loads);
> +					size * num_pt_loads, NULL);
>
>  			if (!allocated_in_map(pfn)) {
>  				clear_bit_on_2nd_bitmap(pfn, NULL);
> @@ -9280,7 +9280,7 @@ exclude_xen4_user_domain(void)
>  	 */
>  	for (i = 0; get_pt_load(i, &phys_start, &phys_end, NULL, NULL); i++) {
>
> -		print_progress(PROGRESS_XEN_DOMAIN, i, num_pt_loads);
> +		print_progress(PROGRESS_XEN_DOMAIN, i, num_pt_loads, NULL);
>
>  		pfn     = paddr_to_pfn(phys_start);
>  		pfn_end = paddr_to_pfn(phys_end);
> @@ -9288,7 +9288,7 @@ exclude_xen4_user_domain(void)
>
>  		for (j = 0; pfn < pfn_end; pfn++, j++) {
>  			print_progress(PROGRESS_XEN_DOMAIN, j + (size * i),
> -					size * num_pt_loads);
> +					size * num_pt_loads, NULL);
>
>  			page_info_addr = info->frame_table_vaddr + pfn * SIZE(page_info);
>  			if (!readmem(VADDR_XEN,
> @@ -9356,7 +9356,7 @@ exclude_xen_user_domain(void)
>  	/*
>  	 * print [100 %]
>  	 */
> -	print_progress(PROGRESS_XEN_DOMAIN, 1, 1);
> +	print_progress(PROGRESS_XEN_DOMAIN, 1, 1, NULL);
>  	print_execution_time(PROGRESS_XEN_DOMAIN, &tv_start);
>
>  	return ret;
> @@ -10342,7 +10342,7 @@ reassemble_kdump_pages(void)
>
>  			num_dumped++;
>
> -			print_progress(PROGRESS_COPY, num_dumped, num_dumpable);
> +			print_progress(PROGRESS_COPY, num_dumped, num_dumpable, &tv_start);
>
>  			if (lseek(fd, offset_ph_org, SEEK_SET) < 0) {
>  				ERRMSG("Can't seek a file(%s). %s\n",
> @@ -10439,7 +10439,7 @@ reassemble_kdump_pages(void)
>  						    size_eraseinfo))
>  			goto out;
>  	}
> -	print_progress(PROGRESS_COPY, num_dumpable, num_dumpable);
> +	print_progress(PROGRESS_COPY, num_dumpable, num_dumpable, &tv_start);
>  	print_execution_time(PROGRESS_COPY, &tv_start);
>
>  	ret = TRUE;
> diff --git a/print_info.c b/print_info.c
> index 79dfe5e..e0e6a27 100644
> --- a/print_info.c
> +++ b/print_info.c
> @@ -15,8 +15,9 @@
>   */
>  #include "print_info.h"
>  #include <time.h>
> +#include <string.h>
>
> -#define PROGRESS_MAXLEN		"35"
> +#define PROGRESS_MAXLEN		"50"
>
>  int message_level;
>  int flag_strerr_message;
> @@ -350,14 +351,36 @@ static void calc_delta(struct timeval *tv_start, struct timeval *delta)
>  	}
>  }
>
> +/* produce less than 12 bytes on msg */
> +static int eta_to_human_short (int secs, char* msg)
> +{
> +	strcpy(msg, "eta: ");
> +	msg += strlen("eta: ");
> +	if (secs < 100)
> +		sprintf(msg, "%ds", secs);
> +	else if (secs < 100 * 60)
> +		sprintf(msg, "%dm%ds", secs / 60, secs % 60);
> +	else if (secs < 48 * 3600)
> +		sprintf(msg, "%dh%dm", secs / 3600, (secs / 60) % 60);
> +	else if (secs < 100 * 86400)
> +		sprintf(msg, "%dd%dh", secs / 86400, (secs / 3600) % 24);
> +	else
> +		sprintf(msg, ">2day");
> +	return 0;
> +}
> +
> +
>  void
> -print_progress(const char *msg, unsigned long current, unsigned long end)
> +print_progress(const char *msg, unsigned long current, unsigned long end, struct timeval *start)
>  {
>  	float progress;
>  	time_t tm;
>  	static time_t last_time = 0;
>  	static unsigned int lapse = 0;
>  	static const char *spinner = "/|\\-";
> +	struct timeval delta;
> +	double eta;
> +	char eta_msg[16] = " ";
>
>  	if (current < end) {
>  		tm = time(NULL);
> @@ -368,13 +391,19 @@ print_progress(const char *msg, unsigned long current, unsigned long end)
>  	} else
>  		progress = 100;
>
> +	if (start != NULL) {
> +		calc_delta(start, &delta);
> +		eta = delta.tv_sec + delta.tv_usec / 1e6;
> +		eta = (100 - progress) * eta / progress;
> +		eta_to_human_short(eta, eta_msg);
> +	}
>  	if (flag_ignore_r_char) {
> -		PROGRESS_MSG("%-" PROGRESS_MAXLEN "s: [%5.1f %%] %c\n",
> -			     msg, progress, spinner[lapse % 4]);
> +		PROGRESS_MSG("%-" PROGRESS_MAXLEN "s: [%5.1f %%] %c  %16s\n",
> +			     msg, progress, spinner[lapse % 4], eta_msg);
>  	} else {
>  		PROGRESS_MSG("\r");
> -		PROGRESS_MSG("%-" PROGRESS_MAXLEN "s: [%5.1f %%] %c",
> -			     msg, progress, spinner[lapse % 4]);
> +		PROGRESS_MSG("%-" PROGRESS_MAXLEN "s: [%5.1f %%] %c  %16s",
> +			     msg, progress, spinner[lapse % 4], eta_msg);
>  	}
>  	lapse++;
>  }
> diff --git a/print_info.h b/print_info.h
> index 01e3706..1ce3593 100644
> --- a/print_info.h
> +++ b/print_info.h
> @@ -25,7 +25,8 @@ extern int flag_ignore_r_char;
>
>  void show_version(void);
>  void print_usage(void);
> -void print_progress(const char *msg, unsigned long current, unsigned long end);
> +void print_progress(const char *msg, unsigned long current, unsigned long end, struct timeval *start);
> +
>  void print_execution_time(char *step_name, struct timeval *tv_start);
>
>  /*
>


From ats-kumagai at wm.jp.nec.com  Wed Jun 14 19:58:52 2017
From: ats-kumagai at wm.jp.nec.com (Atsushi Kumagai)
Date: Thu, 15 Jun 2017 02:58:52 +0000
Subject: [makedumpfile PATCH v3 2/2] print_info: show the remaining time
 of dump progress
In-Reply-To: <7cb91f63-828d-9e98-c846-19c9215c9bed@redhat.com>
References: <1497433292-6015-1-git-send-email-piliu@redhat.com>
 <1497433292-6015-2-git-send-email-piliu@redhat.com>
 <7cb91f63-828d-9e98-c846-19c9215c9bed@redhat.com>
Message-ID: <0910DD04CBD6DE4193FCF86B9C00BE9701EE17F1@BPXM01GP.gisp.nec.co.jp>

Hello Pingfan,

>On Wednesday 14 June 2017 03:11 PM, Pingfan Liu wrote:
>> If the dump progress is slow, it is better to show  an estimate of
>> remaining time of the progress to the user. The estimator simply uses
>> the average speed to work.
>> Before this patch, the bar looks like:
>>   Copying data                                      : [  1.8 %] /
>> After this patch:
>>   Copying data                                      : [  1.8 %] /          eta: 39s
>>
>> Signed-off-by: Pingfan Liu <piliu at redhat.com>
>> ---
>> v2 -> v3: improve the commit log
>
>for the series
>
>Reviewed-by: Pratyush Anand <panand at redhat.com>

Thanks, I'll merge v3 patches into v1.6.2.

Regards,
Atsushi Kumagai

>>
>>  makedumpfile.c | 46 +++++++++++++++++++++++-----------------------
>>  print_info.c   | 41 +++++++++++++++++++++++++++++++++++------
>>  print_info.h   |  3 ++-
>>  3 files changed, 60 insertions(+), 30 deletions(-)
>>
>> diff --git a/makedumpfile.c b/makedumpfile.c
>> index 301772a..35a36ce 100644
>> --- a/makedumpfile.c
>> +++ b/makedumpfile.c
>> @@ -5175,7 +5175,7 @@ _exclude_free_page(struct cycle *cycle)
>>
>>  	for (num_nodes = 1; num_nodes <= vt.numnodes; num_nodes++) {
>>
>> -		print_progress(PROGRESS_FREE_PAGES, num_nodes - 1, vt.numnodes);
>> +		print_progress(PROGRESS_FREE_PAGES, num_nodes - 1, vt.numnodes, NULL);
>>
>>  		node_zones = pgdat + OFFSET(pglist_data.node_zones);
>>
>> @@ -5188,7 +5188,7 @@ _exclude_free_page(struct cycle *cycle)
>>  		for (i = 0; i < nr_zones; i++) {
>>
>>  			print_progress(PROGRESS_FREE_PAGES, i + nr_zones * (num_nodes - 1),
>> -					nr_zones * vt.numnodes);
>> +					nr_zones * vt.numnodes, NULL);
>>
>>  			zone = node_zones + (i * SIZE(zone));
>>  			if (!readmem(VADDR, zone + OFFSET(zone.spanned_pages),
>> @@ -5216,7 +5216,7 @@ _exclude_free_page(struct cycle *cycle)
>>  	/*
>>  	 * print [100 %]
>>  	 */
>> -	print_progress(PROGRESS_FREE_PAGES, vt.numnodes, vt.numnodes);
>> +	print_progress(PROGRESS_FREE_PAGES, vt.numnodes, vt.numnodes, NULL);
>>  	print_execution_time(PROGRESS_FREE_PAGES, &tv_start);
>>
>>  	return TRUE;
>> @@ -5403,7 +5403,7 @@ create_1st_bitmap_file(void)
>>  	for (i = 0; get_pt_load(i, &phys_start, &phys_end, NULL, NULL); i++) {
>>
>>  		if (!info->flag_mem_usage)
>> -			print_progress(PROGRESS_HOLES, i, num_pt_loads);
>> +			print_progress(PROGRESS_HOLES, i, num_pt_loads, NULL);
>>
>>  		pfn_start = paddr_to_pfn(phys_start);
>>  		pfn_end   = paddr_to_pfn(phys_end);
>> @@ -5422,7 +5422,7 @@ create_1st_bitmap_file(void)
>>  	 * print 100 %
>>  	 */
>>  	if (!info->flag_mem_usage) {
>> -		print_progress(PROGRESS_HOLES, info->max_mapnr, info->max_mapnr);
>> +		print_progress(PROGRESS_HOLES, info->max_mapnr, info->max_mapnr, NULL);
>>  		print_execution_time(PROGRESS_HOLES, &tv_start);
>>  	}
>>
>> @@ -5548,7 +5548,7 @@ create_bitmap_from_memhole(struct cycle *cycle, struct dump_bitmap *bitmap, int
>>  		pfn_start = MAX(paddr_to_pfn(phys_start), cycle->start_pfn);
>>  		pfn_end = MIN(paddr_to_pfn(phys_end), cycle->end_pfn);
>>
>> -		print_progress(PROGRESS_HOLES, i, num_pt_loads);
>> +		print_progress(PROGRESS_HOLES, i, num_pt_loads, NULL);
>>
>>  		if (pfn_start >= pfn_end)
>>  			continue;
>> @@ -5587,7 +5587,7 @@ create_bitmap_from_memhole(struct cycle *cycle, struct dump_bitmap *bitmap, int
>>  	/*
>>  	 * print 100 %
>>  	 */
>> -	print_progress(PROGRESS_HOLES, info->max_mapnr, info->max_mapnr);
>> +	print_progress(PROGRESS_HOLES, info->max_mapnr, info->max_mapnr, NULL);
>>  	print_execution_time(PROGRESS_HOLES, &tv_start);
>>
>>  	return TRUE;
>> @@ -5842,7 +5842,7 @@ exclude_unnecessary_pages(struct cycle *cycle)
>>  	for (mm = 0; mm < info->num_mem_map; mm++) {
>>
>>  		if (!info->flag_mem_usage)
>> -			print_progress(PROGRESS_UNN_PAGES, mm, info->num_mem_map);
>> +			print_progress(PROGRESS_UNN_PAGES, mm, info->num_mem_map, NULL);
>>
>>  		mmd = &info->mem_map_data[mm];
>>
>> @@ -5860,7 +5860,7 @@ exclude_unnecessary_pages(struct cycle *cycle)
>>  	 * print [100 %]
>>  	 */
>>  	if (!info->flag_mem_usage) {
>> -		print_progress(PROGRESS_UNN_PAGES, info->num_mem_map, info->num_mem_map);
>> +		print_progress(PROGRESS_UNN_PAGES, info->num_mem_map, info->num_mem_map, NULL);
>>  		print_execution_time(PROGRESS_UNN_PAGES, &tv_start);
>>  	}
>>
>> @@ -7334,7 +7334,7 @@ write_elf_pages_cyclic(struct cache_data *cd_header, struct cache_data *cd_page)
>>  				}
>>
>>  				if ((num_dumped % per) == 0)
>> -					print_progress(PROGRESS_COPY, num_dumped, num_dumpable);
>> +					print_progress(PROGRESS_COPY, num_dumped, num_dumpable, &tv_start);
>>
>>  				num_dumped++;
>>
>> @@ -7453,7 +7453,7 @@ write_elf_pages_cyclic(struct cache_data *cd_header, struct cache_data *cd_page)
>>  	/*
>>  	 * print [100 %]
>>  	 */
>> -	print_progress(PROGRESS_COPY, num_dumpable, num_dumpable);
>> +	print_progress(PROGRESS_COPY, num_dumpable, num_dumpable, &tv_start);
>>  	print_execution_time(PROGRESS_COPY, &tv_start);
>>  	PROGRESS_MSG("\n");
>>
>> @@ -7931,7 +7931,7 @@ write_kdump_pages_parallel_cyclic(struct cache_data *cd_header,
>>  		}
>>
>>  		if ((num_dumped % per) == 0)
>> -			print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable);
>> +			print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable, &tv_start);
>>
>>  		num_dumped++;
>>
>> @@ -7967,7 +7967,7 @@ finish:
>>  	/*
>>  	 * print [100 %]
>>  	 */
>> -	print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable);
>> +	print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable, &tv_start);
>>  	print_execution_time(PROGRESS_COPY, &tv_start);
>>  	PROGRESS_MSG("\n");
>>
>> @@ -8084,7 +8084,7 @@ write_kdump_pages_cyclic(struct cache_data *cd_header, struct cache_data *cd_pag
>>  	for (pfn = start_pfn; pfn < end_pfn; pfn++) {
>>
>>  		if ((num_dumped % per) == 0)
>> -			print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable);
>> +			print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable, &tv_start);
>>
>>  		/*
>>  		 * Check the excluded page.
>> @@ -8164,7 +8164,7 @@ out:
>>  		free(wrkmem);
>>  #endif
>>
>> -	print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable);
>> +	print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable, &tv_start);
>>  	print_execution_time(PROGRESS_COPY, &tv_start);
>>
>>  	return ret;
>> @@ -8622,7 +8622,7 @@ write_kdump_pages_and_bitmap_cyclic(struct cache_data *cd_header, struct cache_d
>>  	/*
>>  	 * print [100 %]
>>  	 */
>> -	print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable);
>> +	print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable, &tv_start);
>>  	print_execution_time(PROGRESS_COPY, &tv_start);
>>  	PROGRESS_MSG("\n");
>>
>> @@ -9216,7 +9216,7 @@ exclude_xen3_user_domain(void)
>>  	 */
>>  	for (i = 0; get_pt_load(i, &phys_start, &phys_end, NULL, NULL); i++) {
>>
>> -		print_progress(PROGRESS_XEN_DOMAIN, i, num_pt_loads);
>> +		print_progress(PROGRESS_XEN_DOMAIN, i, num_pt_loads, NULL);
>>
>>  		pfn     = paddr_to_pfn(phys_start);
>>  		pfn_end = paddr_to_pfn(phys_end);
>> @@ -9224,7 +9224,7 @@ exclude_xen3_user_domain(void)
>>
>>  		for (j = 0; pfn < pfn_end; pfn++, j++) {
>>  			print_progress(PROGRESS_XEN_DOMAIN, j + (size * i),
>> -					size * num_pt_loads);
>> +					size * num_pt_loads, NULL);
>>
>>  			if (!allocated_in_map(pfn)) {
>>  				clear_bit_on_2nd_bitmap(pfn, NULL);
>> @@ -9280,7 +9280,7 @@ exclude_xen4_user_domain(void)
>>  	 */
>>  	for (i = 0; get_pt_load(i, &phys_start, &phys_end, NULL, NULL); i++) {
>>
>> -		print_progress(PROGRESS_XEN_DOMAIN, i, num_pt_loads);
>> +		print_progress(PROGRESS_XEN_DOMAIN, i, num_pt_loads, NULL);
>>
>>  		pfn     = paddr_to_pfn(phys_start);
>>  		pfn_end = paddr_to_pfn(phys_end);
>> @@ -9288,7 +9288,7 @@ exclude_xen4_user_domain(void)
>>
>>  		for (j = 0; pfn < pfn_end; pfn++, j++) {
>>  			print_progress(PROGRESS_XEN_DOMAIN, j + (size * i),
>> -					size * num_pt_loads);
>> +					size * num_pt_loads, NULL);
>>
>>  			page_info_addr = info->frame_table_vaddr + pfn * SIZE(page_info);
>>  			if (!readmem(VADDR_XEN,
>> @@ -9356,7 +9356,7 @@ exclude_xen_user_domain(void)
>>  	/*
>>  	 * print [100 %]
>>  	 */
>> -	print_progress(PROGRESS_XEN_DOMAIN, 1, 1);
>> +	print_progress(PROGRESS_XEN_DOMAIN, 1, 1, NULL);
>>  	print_execution_time(PROGRESS_XEN_DOMAIN, &tv_start);
>>
>>  	return ret;
>> @@ -10342,7 +10342,7 @@ reassemble_kdump_pages(void)
>>
>>  			num_dumped++;
>>
>> -			print_progress(PROGRESS_COPY, num_dumped, num_dumpable);
>> +			print_progress(PROGRESS_COPY, num_dumped, num_dumpable, &tv_start);
>>
>>  			if (lseek(fd, offset_ph_org, SEEK_SET) < 0) {
>>  				ERRMSG("Can't seek a file(%s). %s\n",
>> @@ -10439,7 +10439,7 @@ reassemble_kdump_pages(void)
>>  						    size_eraseinfo))
>>  			goto out;
>>  	}
>> -	print_progress(PROGRESS_COPY, num_dumpable, num_dumpable);
>> +	print_progress(PROGRESS_COPY, num_dumpable, num_dumpable, &tv_start);
>>  	print_execution_time(PROGRESS_COPY, &tv_start);
>>
>>  	ret = TRUE;
>> diff --git a/print_info.c b/print_info.c
>> index 79dfe5e..e0e6a27 100644
>> --- a/print_info.c
>> +++ b/print_info.c
>> @@ -15,8 +15,9 @@
>>   */
>>  #include "print_info.h"
>>  #include <time.h>
>> +#include <string.h>
>>
>> -#define PROGRESS_MAXLEN		"35"
>> +#define PROGRESS_MAXLEN		"50"
>>
>>  int message_level;
>>  int flag_strerr_message;
>> @@ -350,14 +351,36 @@ static void calc_delta(struct timeval *tv_start, struct timeval *delta)
>>  	}
>>  }
>>
>> +/* produce less than 12 bytes on msg */
>> +static int eta_to_human_short (int secs, char* msg)
>> +{
>> +	strcpy(msg, "eta: ");
>> +	msg += strlen("eta: ");
>> +	if (secs < 100)
>> +		sprintf(msg, "%ds", secs);
>> +	else if (secs < 100 * 60)
>> +		sprintf(msg, "%dm%ds", secs / 60, secs % 60);
>> +	else if (secs < 48 * 3600)
>> +		sprintf(msg, "%dh%dm", secs / 3600, (secs / 60) % 60);
>> +	else if (secs < 100 * 86400)
>> +		sprintf(msg, "%dd%dh", secs / 86400, (secs / 3600) % 24);
>> +	else
>> +		sprintf(msg, ">2day");
>> +	return 0;
>> +}
>> +
>> +
>>  void
>> -print_progress(const char *msg, unsigned long current, unsigned long end)
>> +print_progress(const char *msg, unsigned long current, unsigned long end, struct timeval *start)
>>  {
>>  	float progress;
>>  	time_t tm;
>>  	static time_t last_time = 0;
>>  	static unsigned int lapse = 0;
>>  	static const char *spinner = "/|\\-";
>> +	struct timeval delta;
>> +	double eta;
>> +	char eta_msg[16] = " ";
>>
>>  	if (current < end) {
>>  		tm = time(NULL);
>> @@ -368,13 +391,19 @@ print_progress(const char *msg, unsigned long current, unsigned long end)
>>  	} else
>>  		progress = 100;
>>
>> +	if (start != NULL) {
>> +		calc_delta(start, &delta);
>> +		eta = delta.tv_sec + delta.tv_usec / 1e6;
>> +		eta = (100 - progress) * eta / progress;
>> +		eta_to_human_short(eta, eta_msg);
>> +	}
>>  	if (flag_ignore_r_char) {
>> -		PROGRESS_MSG("%-" PROGRESS_MAXLEN "s: [%5.1f %%] %c\n",
>> -			     msg, progress, spinner[lapse % 4]);
>> +		PROGRESS_MSG("%-" PROGRESS_MAXLEN "s: [%5.1f %%] %c  %16s\n",
>> +			     msg, progress, spinner[lapse % 4], eta_msg);
>>  	} else {
>>  		PROGRESS_MSG("\r");
>> -		PROGRESS_MSG("%-" PROGRESS_MAXLEN "s: [%5.1f %%] %c",
>> -			     msg, progress, spinner[lapse % 4]);
>> +		PROGRESS_MSG("%-" PROGRESS_MAXLEN "s: [%5.1f %%] %c  %16s",
>> +			     msg, progress, spinner[lapse % 4], eta_msg);
>>  	}
>>  	lapse++;
>>  }
>> diff --git a/print_info.h b/print_info.h
>> index 01e3706..1ce3593 100644
>> --- a/print_info.h
>> +++ b/print_info.h
>> @@ -25,7 +25,8 @@ extern int flag_ignore_r_char;
>>
>>  void show_version(void);
>>  void print_usage(void);
>> -void print_progress(const char *msg, unsigned long current, unsigned long end);
>> +void print_progress(const char *msg, unsigned long current, unsigned long end, struct timeval *start);
>> +
>>  void print_execution_time(char *step_name, struct timeval *tv_start);
>>
>>  /*
>>




From bp at alien8.de  Thu Jun 15 02:08:32 2017
From: bp at alien8.de (Borislav Petkov)
Date: Thu, 15 Jun 2017 11:08:32 +0200
Subject: [PATCH v6 25/34] swiotlb: Add warnings for use of bounce buffers
 with SME
In-Reply-To: <33d1debc-c684-cba1-7d95-493678f086d0@amd.com>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
 <20170607191732.28645.42876.stgit@tlendack-t1.amdoffice.net>
 <20170614165052.fyn5t4gkq5leczcc@pd.tnic>
 <33d1debc-c684-cba1-7d95-493678f086d0@amd.com>
Message-ID: <20170615090832.ncmq2kgom32cchhw@pd.tnic>

On Wed, Jun 14, 2017 at 02:49:02PM -0500, Tom Lendacky wrote:
> I guess I don't need the sme_active() check since the second part of the
> if statement can only ever be true if SME is active (since mask is
> unsigned).

... and you can define sme_me_mask as an u64 directly (it is that already,
practically, as we don't do SME on 32-bit) and then get rid of the cast.

-- 
Regards/Gruss,
    Boris.

Good mailing practices for 400: avoid top-posting and trim the reply.


From bp at alien8.de  Thu Jun 15 02:41:12 2017
From: bp at alien8.de (Borislav Petkov)
Date: Thu, 15 Jun 2017 11:41:12 +0200
Subject: [PATCH v6 26/34] iommu/amd: Allow the AMD IOMMU to work with
 memory encryption
In-Reply-To: <0611d01a-19f8-d6ae-2682-932789855518@amd.com>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
 <20170607191745.28645.81756.stgit@tlendack-t1.amdoffice.net>
 <20170614174208.p2yr5exs4b6pjxhf@pd.tnic>
 <0611d01a-19f8-d6ae-2682-932789855518@amd.com>
Message-ID: <20170615094111.wga334kg2bhxqib3@pd.tnic>

On Wed, Jun 14, 2017 at 03:40:28PM -0500, Tom Lendacky wrote:
> I was trying to keep all the logic for it here in the SME related files
> rather than put it in the iommu code itself. But it is easy enough to
> move if you think it's worth it.

Yes please - the less needlessly global symbols, the better.

> > Also, you said in another mail on this subthread that c->microcode
> > is not yet set. Are you saying, that the iommu init gunk runs before
> > init_amd(), where we do set c->microcode?
> > 
> > If so, we can move the setting to early_init_amd() or so.
> 
> I'll look into that.

And I don't think c->microcode is not set by the time we init the iommu
because, AFAICT, we do the latter in pci_iommu_init() and that's a
rootfs_initcall() which happens later then the CPU init stuff.

> I'll look into simplifying the checks.

Something like this maybe?

	if (rev >= 0x1205)
		return true;

	if (rev <= 0x11ff && rev >= 0x1126)
		return true;

	return false;

> > WARNING: Use of volatile is usually wrong: see Documentation/process/volatile-considered-harmful.rst
> > #134: FILE: drivers/iommu/amd_iommu.c:866:
> > +static void build_completion_wait(struct iommu_cmd *cmd, volatile u64 *sem)
> > 
> 
> The semaphore area is written to by the device so the use of volatile is
> appropriate in this case.

Do you mean this is like the last exception case in that document above:

"
  - Pointers to data structures in coherent memory which might be modified
    by I/O devices can, sometimes, legitimately be volatile.  A ring buffer
    used by a network adapter, where that adapter changes pointers to
    indicate which descriptors have been processed, is an example of this
    type of situation."

?

If so, it did work fine until now, without the volatile. Why is it
needed now, all of a sudden?

-- 
Regards/Gruss,
    Boris.

Good mailing practices for 400: avoid top-posting and trim the reply.


From bp at alien8.de  Thu Jun 15 02:55:59 2017
From: bp at alien8.de (Borislav Petkov)
Date: Thu, 15 Jun 2017 11:55:59 +0200
Subject: [PATCH v6 29/34] kvm: x86: svm: Support Secure Memory Encryption
 within KVM
In-Reply-To: <20170607191815.28645.9054.stgit@tlendack-t1.amdoffice.net>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
 <20170607191815.28645.9054.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170615095559.mhf7amisrbzujvul@pd.tnic>

On Wed, Jun 07, 2017 at 02:18:15PM -0500, Tom Lendacky wrote:
> Update the KVM support to work with SME. The VMCB has a number of fields
> where physical addresses are used and these addresses must contain the
> memory encryption mask in order to properly access the encrypted memory.
> Also, use the memory encryption mask when creating and using the nested
> page tables.
> 
> Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
> ---
>  arch/x86/include/asm/kvm_host.h |    2 +-
>  arch/x86/kvm/mmu.c              |   12 ++++++++----
>  arch/x86/kvm/mmu.h              |    2 +-
>  arch/x86/kvm/svm.c              |   35 ++++++++++++++++++-----------------
>  arch/x86/kvm/vmx.c              |    3 ++-
>  arch/x86/kvm/x86.c              |    3 ++-
>  6 files changed, 32 insertions(+), 25 deletions(-)

Patches 27-29:

Reviewed-by: Borislav Petkov <bp at suse.de>

-- 
Regards/Gruss,
    Boris.

Good mailing practices for 400: avoid top-posting and trim the reply.


From bp at alien8.de  Thu Jun 15 03:03:45 2017
From: bp at alien8.de (Borislav Petkov)
Date: Thu, 15 Jun 2017 12:03:45 +0200
Subject: [PATCH v6 30/34] x86/mm, kexec: Allow kexec to be used with SME
In-Reply-To: <20170607191827.28645.93989.stgit@tlendack-t1.amdoffice.net>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
 <20170607191827.28645.93989.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170615100345.76pn5ruf6cm3ktpe@pd.tnic>

On Wed, Jun 07, 2017 at 02:18:27PM -0500, Tom Lendacky wrote:
> Provide support so that kexec can be used to boot a kernel when SME is
> enabled.
> 
> Support is needed to allocate pages for kexec without encryption.  This
> is needed in order to be able to reboot in the kernel in the same manner
> as originally booted.
> 
> Additionally, when shutting down all of the CPUs we need to be sure to
> flush the caches and then halt. This is needed when booting from a state
> where SME was not active into a state where SME is active (or vice-versa).
> Without these steps, it is possible for cache lines to exist for the same
> physical location but tagged both with and without the encryption bit. This
> can cause random memory corruption when caches are flushed depending on
> which cacheline is written last.
> 
> Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
> ---
>  arch/x86/include/asm/init.h          |    1 +
>  arch/x86/include/asm/kexec.h         |    8 ++++++++
>  arch/x86/include/asm/pgtable_types.h |    1 +
>  arch/x86/kernel/machine_kexec_64.c   |   35 +++++++++++++++++++++++++++++++++-
>  arch/x86/kernel/process.c            |   17 +++++++++++++++--
>  arch/x86/mm/ident_map.c              |   12 ++++++++----
>  include/linux/kexec.h                |   14 ++++++++++++++
>  kernel/kexec_core.c                  |    6 ++++++
>  8 files changed, 87 insertions(+), 7 deletions(-)
> 
> diff --git a/arch/x86/include/asm/init.h b/arch/x86/include/asm/init.h
> index 474eb8c..05c4aa0 100644
> --- a/arch/x86/include/asm/init.h
> +++ b/arch/x86/include/asm/init.h
> @@ -7,6 +7,7 @@ struct x86_mapping_info {
>  	unsigned long page_flag;	 /* page flag for PMD or PUD entry */
>  	unsigned long offset;		 /* ident mapping offset */
>  	bool direct_gbpages;		 /* PUD level 1GB page support */
> +	unsigned long kernpg_flag;	 /* kernel pagetable flag override */
>  };
>  
>  int kernel_ident_mapping_init(struct x86_mapping_info *info, pgd_t *pgd_page,
> diff --git a/arch/x86/include/asm/kexec.h b/arch/x86/include/asm/kexec.h
> index 70ef205..e8183ac 100644
> --- a/arch/x86/include/asm/kexec.h
> +++ b/arch/x86/include/asm/kexec.h
> @@ -207,6 +207,14 @@ struct kexec_entry64_regs {
>  	uint64_t r15;
>  	uint64_t rip;
>  };
> +
> +extern int arch_kexec_post_alloc_pages(void *vaddr, unsigned int pages,
> +				       gfp_t gfp);
> +#define arch_kexec_post_alloc_pages arch_kexec_post_alloc_pages
> +
> +extern void arch_kexec_pre_free_pages(void *vaddr, unsigned int pages);
> +#define arch_kexec_pre_free_pages arch_kexec_pre_free_pages
> +
>  #endif
>  
>  typedef void crash_vmclear_fn(void);
> diff --git a/arch/x86/include/asm/pgtable_types.h b/arch/x86/include/asm/pgtable_types.h
> index ce8cb1c..0f326f4 100644
> --- a/arch/x86/include/asm/pgtable_types.h
> +++ b/arch/x86/include/asm/pgtable_types.h
> @@ -213,6 +213,7 @@ enum page_cache_mode {
>  #define PAGE_KERNEL		__pgprot(__PAGE_KERNEL | _PAGE_ENC)
>  #define PAGE_KERNEL_RO		__pgprot(__PAGE_KERNEL_RO | _PAGE_ENC)
>  #define PAGE_KERNEL_EXEC	__pgprot(__PAGE_KERNEL_EXEC | _PAGE_ENC)
> +#define PAGE_KERNEL_EXEC_NOENC	__pgprot(__PAGE_KERNEL_EXEC)
>  #define PAGE_KERNEL_RX		__pgprot(__PAGE_KERNEL_RX | _PAGE_ENC)
>  #define PAGE_KERNEL_NOCACHE	__pgprot(__PAGE_KERNEL_NOCACHE | _PAGE_ENC)
>  #define PAGE_KERNEL_LARGE	__pgprot(__PAGE_KERNEL_LARGE | _PAGE_ENC)
> diff --git a/arch/x86/kernel/machine_kexec_64.c b/arch/x86/kernel/machine_kexec_64.c
> index 6f5ca4e..35e069a 100644
> --- a/arch/x86/kernel/machine_kexec_64.c
> +++ b/arch/x86/kernel/machine_kexec_64.c
> @@ -87,7 +87,7 @@ static int init_transition_pgtable(struct kimage *image, pgd_t *pgd)
>  		set_pmd(pmd, __pmd(__pa(pte) | _KERNPG_TABLE));
>  	}
>  	pte = pte_offset_kernel(pmd, vaddr);
> -	set_pte(pte, pfn_pte(paddr >> PAGE_SHIFT, PAGE_KERNEL_EXEC));
> +	set_pte(pte, pfn_pte(paddr >> PAGE_SHIFT, PAGE_KERNEL_EXEC_NOENC));
>  	return 0;
>  err:
>  	free_transition_pgtable(image);
> @@ -115,6 +115,7 @@ static int init_pgtable(struct kimage *image, unsigned long start_pgtable)
>  		.alloc_pgt_page	= alloc_pgt_page,
>  		.context	= image,
>  		.page_flag	= __PAGE_KERNEL_LARGE_EXEC,
> +		.kernpg_flag	= _KERNPG_TABLE_NOENC,
>  	};
>  	unsigned long mstart, mend;
>  	pgd_t *level4p;
> @@ -602,3 +603,35 @@ void arch_kexec_unprotect_crashkres(void)
>  {
>  	kexec_mark_crashkres(false);
>  }
> +
> +int arch_kexec_post_alloc_pages(void *vaddr, unsigned int pages, gfp_t gfp)
> +{
> +	int ret;
> +
> +	if (sme_active()) {

What happened to flipping the logic and saving an indentation level here?

> +		/*
> +		 * If SME is active we need to be sure that kexec pages are
> +		 * not encrypted because when we boot to the new kernel the
> +		 * pages won't be accessed encrypted (initially).
> +		 */
> +		ret = set_memory_decrypted((unsigned long)vaddr, pages);
> +		if (ret)
> +			return ret;
> +
> +		if (gfp & __GFP_ZERO)
> +			memset(vaddr, 0, pages * PAGE_SIZE);
> +	}

This is still zeroing the memory a second time. That function has missed
all my comments from last time.

-- 
Regards/Gruss,
    Boris.

Good mailing practices for 400: avoid top-posting and trim the reply.


From thomas.lendacky at amd.com  Thu Jun 15 06:23:17 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Thu, 15 Jun 2017 08:23:17 -0500
Subject: [PATCH v6 25/34] swiotlb: Add warnings for use of bounce buffers
 with SME
In-Reply-To: <20170615090832.ncmq2kgom32cchhw@pd.tnic>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
 <20170607191732.28645.42876.stgit@tlendack-t1.amdoffice.net>
 <20170614165052.fyn5t4gkq5leczcc@pd.tnic>
 <33d1debc-c684-cba1-7d95-493678f086d0@amd.com>
 <20170615090832.ncmq2kgom32cchhw@pd.tnic>
Message-ID: <a6fb0334-42e4-a08f-ac95-f33c48b0711f@amd.com>

On 6/15/2017 4:08 AM, Borislav Petkov wrote:
> On Wed, Jun 14, 2017 at 02:49:02PM -0500, Tom Lendacky wrote:
>> I guess I don't need the sme_active() check since the second part of the
>> if statement can only ever be true if SME is active (since mask is
>> unsigned).
> 
> ... and you can define sme_me_mask as an u64 directly (it is that already,
> practically, as we don't do SME on 32-bit) and then get rid of the cast.

Let me look into that. There are so many places that are expecting an
unsigned long I'll have to see how that all works out from a build
perspective.

Thanks,
Tom

> 


From thomas.lendacky at amd.com  Thu Jun 15 07:59:45 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Thu, 15 Jun 2017 09:59:45 -0500
Subject: [PATCH v6 26/34] iommu/amd: Allow the AMD IOMMU to work with
 memory encryption
In-Reply-To: <20170615094111.wga334kg2bhxqib3@pd.tnic>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
 <20170607191745.28645.81756.stgit@tlendack-t1.amdoffice.net>
 <20170614174208.p2yr5exs4b6pjxhf@pd.tnic>
 <0611d01a-19f8-d6ae-2682-932789855518@amd.com>
 <20170615094111.wga334kg2bhxqib3@pd.tnic>
Message-ID: <921153f5-1528-31d8-b815-f0419e819aeb@amd.com>

On 6/15/2017 4:41 AM, Borislav Petkov wrote:
> On Wed, Jun 14, 2017 at 03:40:28PM -0500, Tom Lendacky wrote:
>> I was trying to keep all the logic for it here in the SME related files
>> rather than put it in the iommu code itself. But it is easy enough to
>> move if you think it's worth it.
> 
> Yes please - the less needlessly global symbols, the better.

Ok.

> 
>>> Also, you said in another mail on this subthread that c->microcode
>>> is not yet set. Are you saying, that the iommu init gunk runs before
>>> init_amd(), where we do set c->microcode?
>>>
>>> If so, we can move the setting to early_init_amd() or so.
>>
>> I'll look into that.
> 
> And I don't think c->microcode is not set by the time we init the iommu
> because, AFAICT, we do the latter in pci_iommu_init() and that's a
> rootfs_initcall() which happens later then the CPU init stuff.

Actually the detection routine, amd_iommu_detect(), is part of the
IOMMU_INIT_FINISH macro support which is called early through mm_init()
from start_kernel() and that routine is called before init_amd().

> 
>> I'll look into simplifying the checks.
> 
> Something like this maybe?
> 
> 	if (rev >= 0x1205)
> 		return true;
> 
> 	if (rev <= 0x11ff && rev >= 0x1126)
> 		return true;
> 
> 	return false;

Yup, something like that.

> 
>>> WARNING: Use of volatile is usually wrong: see Documentation/process/volatile-considered-harmful.rst
>>> #134: FILE: drivers/iommu/amd_iommu.c:866:
>>> +static void build_completion_wait(struct iommu_cmd *cmd, volatile u64 *sem)
>>>
>>
>> The semaphore area is written to by the device so the use of volatile is
>> appropriate in this case.
> 
> Do you mean this is like the last exception case in that document above:
> 
> "
>    - Pointers to data structures in coherent memory which might be modified
>      by I/O devices can, sometimes, legitimately be volatile.  A ring buffer
>      used by a network adapter, where that adapter changes pointers to
>      indicate which descriptors have been processed, is an example of this
>      type of situation."
> 
> ?
> 
> If so, it did work fine until now, without the volatile. Why is it
> needed now, all of a sudden?

If you run checkpatch against the whole amd_iommu.c file you'll see that
same warning for the wait_on_sem() function.  The checkpatch warning
shows up now because I modified the build_completion_wait() function as
part of the support to use iommu_virt_to_phys().

Since I'm casting the arg to iommu_virt_to_phys() no matter what I can
avoid the signature change to the build_completion_wait() function and
avoid this confusion in the future.

Thanks,
Tom

> 


From bp at alien8.de  Thu Jun 15 08:33:22 2017
From: bp at alien8.de (Borislav Petkov)
Date: Thu, 15 Jun 2017 17:33:22 +0200
Subject: [PATCH v6 26/34] iommu/amd: Allow the AMD IOMMU to work with
 memory encryption
In-Reply-To: <921153f5-1528-31d8-b815-f0419e819aeb@amd.com>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
 <20170607191745.28645.81756.stgit@tlendack-t1.amdoffice.net>
 <20170614174208.p2yr5exs4b6pjxhf@pd.tnic>
 <0611d01a-19f8-d6ae-2682-932789855518@amd.com>
 <20170615094111.wga334kg2bhxqib3@pd.tnic>
 <921153f5-1528-31d8-b815-f0419e819aeb@amd.com>
Message-ID: <20170615153322.nwylo3dzn4fdx6n6@pd.tnic>

On Thu, Jun 15, 2017 at 09:59:45AM -0500, Tom Lendacky wrote:
> Actually the detection routine, amd_iommu_detect(), is part of the
> IOMMU_INIT_FINISH macro support which is called early through mm_init()
> from start_kernel() and that routine is called before init_amd().

Ah, we do that there too:

	for (p = __iommu_table; p < __iommu_table_end; p++) {

Can't say that that code with the special section and whatnot is
obvious. :-\

Oh, well, early_init_amd() then. That is called in
start_kernel->setup_arch->early_cpu_init and thus before mm_init().

> > If so, it did work fine until now, without the volatile. Why is it
> > needed now, all of a sudden?
> 
> If you run checkpatch against the whole amd_iommu.c file you'll see that

I'm, of course, not talking about the signature change: I'm *actually*
questioning the need to make this argument volatile, all of a sudden.

If there's a need, please explain why. It worked fine until now. If it
didn't, we would've seen it.

If it is a bug, then it needs a proper explanation, a *separate* patch
and so on. But not like now, a drive-by change in an IOMMU enablement
patch.

If it is wrong, then wait_on_sem() needs to be fixed too. AFAICT,
wait_on_sem() gets called in both cases with interrupts disabled, while
holding a lock so I'd like to pls know why, even in that case, does this
variable need to be volatile.

-- 
Regards/Gruss,
    Boris.

Good mailing practices for 400: avoid top-posting and trim the reply.


From thomas.lendacky at amd.com  Thu Jun 15 09:33:41 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Thu, 15 Jun 2017 11:33:41 -0500
Subject: [PATCH v6 26/34] iommu/amd: Allow the AMD IOMMU to work with
 memory encryption
In-Reply-To: <20170615153322.nwylo3dzn4fdx6n6@pd.tnic>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
 <20170607191745.28645.81756.stgit@tlendack-t1.amdoffice.net>
 <20170614174208.p2yr5exs4b6pjxhf@pd.tnic>
 <0611d01a-19f8-d6ae-2682-932789855518@amd.com>
 <20170615094111.wga334kg2bhxqib3@pd.tnic>
 <921153f5-1528-31d8-b815-f0419e819aeb@amd.com>
 <20170615153322.nwylo3dzn4fdx6n6@pd.tnic>
Message-ID: <3db2c52d-5e63-a1df-edd4-975bce7f29c2@amd.com>

On 6/15/2017 10:33 AM, Borislav Petkov wrote:
> On Thu, Jun 15, 2017 at 09:59:45AM -0500, Tom Lendacky wrote:
>> Actually the detection routine, amd_iommu_detect(), is part of the
>> IOMMU_INIT_FINISH macro support which is called early through mm_init()
>> from start_kernel() and that routine is called before init_amd().
> 
> Ah, we do that there too:
> 
> 	for (p = __iommu_table; p < __iommu_table_end; p++) {
> 
> Can't say that that code with the special section and whatnot is
> obvious. :-\
> 
> Oh, well, early_init_amd() then. That is called in
> start_kernel->setup_arch->early_cpu_init and thus before mm_init().
> 
>>> If so, it did work fine until now, without the volatile. Why is it
>>> needed now, all of a sudden?
>>
>> If you run checkpatch against the whole amd_iommu.c file you'll see that
> 
> I'm, of course, not talking about the signature change: I'm *actually*
> questioning the need to make this argument volatile, all of a sudden.

Understood.

> 
> If there's a need, please explain why. It worked fine until now. If it
> didn't, we would've seen it.

The original reason for the change was to try and make the use of
iommu_virt_to_phys() straight forward.  Removing the cast and changing
build_completion_wait() to take a u64 * (without volatile) resulted in a
warning because cmd_sem is defined in the amd_iommu struct as volatile,
which required a cast on the call to iommu_virt_to_phys() anyway. Since
it worked fine previously and the whole volatile thing is beyond the
scope of this patchset, I'll change back to the original method of how
the function was called.

> 
> If it is a bug, then it needs a proper explanation, a *separate* patch
> and so on. But not like now, a drive-by change in an IOMMU enablement
> patch.
> 
> If it is wrong, then wait_on_sem() needs to be fixed too. AFAICT,
> wait_on_sem() gets called in both cases with interrupts disabled, while
> holding a lock so I'd like to pls know why, even in that case, does this
> variable need to be volatile

Changing the signature back reverts to the original way, so this can be
looked at separate from this patchset then.

Thanks,
Tom

> 


From thomas.lendacky at amd.com  Thu Jun 15 10:43:13 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Thu, 15 Jun 2017 12:43:13 -0500
Subject: [PATCH v6 30/34] x86/mm, kexec: Allow kexec to be used with SME
In-Reply-To: <20170615100345.76pn5ruf6cm3ktpe@pd.tnic>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
 <20170607191827.28645.93989.stgit@tlendack-t1.amdoffice.net>
 <20170615100345.76pn5ruf6cm3ktpe@pd.tnic>
Message-ID: <8973bfc5-9d18-f98e-4c38-6452f5d81ebe@amd.com>

On 6/15/2017 5:03 AM, Borislav Petkov wrote:
> On Wed, Jun 07, 2017 at 02:18:27PM -0500, Tom Lendacky wrote:
>> Provide support so that kexec can be used to boot a kernel when SME is
>> enabled.
>>
>> Support is needed to allocate pages for kexec without encryption.  This
>> is needed in order to be able to reboot in the kernel in the same manner
>> as originally booted.
>>
>> Additionally, when shutting down all of the CPUs we need to be sure to
>> flush the caches and then halt. This is needed when booting from a state
>> where SME was not active into a state where SME is active (or vice-versa).
>> Without these steps, it is possible for cache lines to exist for the same
>> physical location but tagged both with and without the encryption bit. This
>> can cause random memory corruption when caches are flushed depending on
>> which cacheline is written last.
>>
>> Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
>> ---
>>   arch/x86/include/asm/init.h          |    1 +
>>   arch/x86/include/asm/kexec.h         |    8 ++++++++
>>   arch/x86/include/asm/pgtable_types.h |    1 +
>>   arch/x86/kernel/machine_kexec_64.c   |   35 +++++++++++++++++++++++++++++++++-
>>   arch/x86/kernel/process.c            |   17 +++++++++++++++--
>>   arch/x86/mm/ident_map.c              |   12 ++++++++----
>>   include/linux/kexec.h                |   14 ++++++++++++++
>>   kernel/kexec_core.c                  |    6 ++++++
>>   8 files changed, 87 insertions(+), 7 deletions(-)
>>
>> diff --git a/arch/x86/include/asm/init.h b/arch/x86/include/asm/init.h
>> index 474eb8c..05c4aa0 100644
>> --- a/arch/x86/include/asm/init.h
>> +++ b/arch/x86/include/asm/init.h
>> @@ -7,6 +7,7 @@ struct x86_mapping_info {
>>   	unsigned long page_flag;	 /* page flag for PMD or PUD entry */
>>   	unsigned long offset;		 /* ident mapping offset */
>>   	bool direct_gbpages;		 /* PUD level 1GB page support */
>> +	unsigned long kernpg_flag;	 /* kernel pagetable flag override */
>>   };
>>   
>>   int kernel_ident_mapping_init(struct x86_mapping_info *info, pgd_t *pgd_page,
>> diff --git a/arch/x86/include/asm/kexec.h b/arch/x86/include/asm/kexec.h
>> index 70ef205..e8183ac 100644
>> --- a/arch/x86/include/asm/kexec.h
>> +++ b/arch/x86/include/asm/kexec.h
>> @@ -207,6 +207,14 @@ struct kexec_entry64_regs {
>>   	uint64_t r15;
>>   	uint64_t rip;
>>   };
>> +
>> +extern int arch_kexec_post_alloc_pages(void *vaddr, unsigned int pages,
>> +				       gfp_t gfp);
>> +#define arch_kexec_post_alloc_pages arch_kexec_post_alloc_pages
>> +
>> +extern void arch_kexec_pre_free_pages(void *vaddr, unsigned int pages);
>> +#define arch_kexec_pre_free_pages arch_kexec_pre_free_pages
>> +
>>   #endif
>>   
>>   typedef void crash_vmclear_fn(void);
>> diff --git a/arch/x86/include/asm/pgtable_types.h b/arch/x86/include/asm/pgtable_types.h
>> index ce8cb1c..0f326f4 100644
>> --- a/arch/x86/include/asm/pgtable_types.h
>> +++ b/arch/x86/include/asm/pgtable_types.h
>> @@ -213,6 +213,7 @@ enum page_cache_mode {
>>   #define PAGE_KERNEL		__pgprot(__PAGE_KERNEL | _PAGE_ENC)
>>   #define PAGE_KERNEL_RO		__pgprot(__PAGE_KERNEL_RO | _PAGE_ENC)
>>   #define PAGE_KERNEL_EXEC	__pgprot(__PAGE_KERNEL_EXEC | _PAGE_ENC)
>> +#define PAGE_KERNEL_EXEC_NOENC	__pgprot(__PAGE_KERNEL_EXEC)
>>   #define PAGE_KERNEL_RX		__pgprot(__PAGE_KERNEL_RX | _PAGE_ENC)
>>   #define PAGE_KERNEL_NOCACHE	__pgprot(__PAGE_KERNEL_NOCACHE | _PAGE_ENC)
>>   #define PAGE_KERNEL_LARGE	__pgprot(__PAGE_KERNEL_LARGE | _PAGE_ENC)
>> diff --git a/arch/x86/kernel/machine_kexec_64.c b/arch/x86/kernel/machine_kexec_64.c
>> index 6f5ca4e..35e069a 100644
>> --- a/arch/x86/kernel/machine_kexec_64.c
>> +++ b/arch/x86/kernel/machine_kexec_64.c
>> @@ -87,7 +87,7 @@ static int init_transition_pgtable(struct kimage *image, pgd_t *pgd)
>>   		set_pmd(pmd, __pmd(__pa(pte) | _KERNPG_TABLE));
>>   	}
>>   	pte = pte_offset_kernel(pmd, vaddr);
>> -	set_pte(pte, pfn_pte(paddr >> PAGE_SHIFT, PAGE_KERNEL_EXEC));
>> +	set_pte(pte, pfn_pte(paddr >> PAGE_SHIFT, PAGE_KERNEL_EXEC_NOENC));
>>   	return 0;
>>   err:
>>   	free_transition_pgtable(image);
>> @@ -115,6 +115,7 @@ static int init_pgtable(struct kimage *image, unsigned long start_pgtable)
>>   		.alloc_pgt_page	= alloc_pgt_page,
>>   		.context	= image,
>>   		.page_flag	= __PAGE_KERNEL_LARGE_EXEC,
>> +		.kernpg_flag	= _KERNPG_TABLE_NOENC,
>>   	};
>>   	unsigned long mstart, mend;
>>   	pgd_t *level4p;
>> @@ -602,3 +603,35 @@ void arch_kexec_unprotect_crashkres(void)
>>   {
>>   	kexec_mark_crashkres(false);
>>   }
>> +
>> +int arch_kexec_post_alloc_pages(void *vaddr, unsigned int pages, gfp_t gfp)
>> +{
>> +	int ret;
>> +
>> +	if (sme_active()) {
> 
> What happened to flipping the logic and saving an indentation level here?
> 
>> +		/*
>> +		 * If SME is active we need to be sure that kexec pages are
>> +		 * not encrypted because when we boot to the new kernel the
>> +		 * pages won't be accessed encrypted (initially).
>> +		 */
>> +		ret = set_memory_decrypted((unsigned long)vaddr, pages);
>> +		if (ret)
>> +			return ret;
>> +
>> +		if (gfp & __GFP_ZERO)
>> +			memset(vaddr, 0, pages * PAGE_SIZE);
>> +	}
> 
> This is still zeroing the memory a second time. That function has missed
> all my comments from last time.

Hmmm...  not sure what happened, I thought I made changes here.  I'll
take care of it.

Thanks,
Tom

> 


From konrad.wilk at oracle.com  Thu Jun 15 13:13:29 2017
From: konrad.wilk at oracle.com (Konrad Rzeszutek Wilk)
Date: Thu, 15 Jun 2017 16:13:29 -0400
Subject: [PATCH v6 26/34] iommu/amd: Allow the AMD IOMMU to work with
 memory encryption
In-Reply-To: <20170615153322.nwylo3dzn4fdx6n6@pd.tnic>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
 <20170607191745.28645.81756.stgit@tlendack-t1.amdoffice.net>
 <20170614174208.p2yr5exs4b6pjxhf@pd.tnic>
 <0611d01a-19f8-d6ae-2682-932789855518@amd.com>
 <20170615094111.wga334kg2bhxqib3@pd.tnic>
 <921153f5-1528-31d8-b815-f0419e819aeb@amd.com>
 <20170615153322.nwylo3dzn4fdx6n6@pd.tnic>
Message-ID: <C1A52990-84AA-4258-B864-0121F5F5C4B5@oracle.com>

On June 15, 2017 11:33:22 AM EDT, Borislav Petkov <bp at alien8.de> wrote:
>On Thu, Jun 15, 2017 at 09:59:45AM -0500, Tom Lendacky wrote:
>> Actually the detection routine, amd_iommu_detect(), is part of the
>> IOMMU_INIT_FINISH macro support which is called early through
>mm_init()
>> from start_kernel() and that routine is called before init_amd().
>
>Ah, we do that there too:
>
>	for (p = __iommu_table; p < __iommu_table_end; p++) {
>
>Can't say that that code with the special section and whatnot is
>obvious. :-\
>

Patches to make it more obvious would be always welcome!



Thanks!


From ats-kumagai at wm.jp.nec.com  Fri Jun 16 01:18:15 2017
From: ats-kumagai at wm.jp.nec.com (Atsushi Kumagai)
Date: Fri, 16 Jun 2017 08:18:15 +0000
Subject: makedumpfile: support for newer kernels [v4.9 onwards]
In-Reply-To: <CAEzMDiAq1U3SRkvwh5nqETDBoZs8DojhmMTDSG7B-Uzbh1a3Nw@mail.gmail.com>
References: <CAEzMDiD-w91Q+Fjy_KgXDcwo9YGasa9ggqW2dEhF4ffV7RstqQ@mail.gmail.com>
 <0bd0a35a-88cd-3f68-846f-2a2ad1e7afcb@redhat.com>
 <CAEzMDiBKdA15Gn2kgs=BgzCM=bzaBXCQP_tM_Vmfc4uxGL0SRg@mail.gmail.com>
 <0ae1e066-821e-3b11-ee44-b615245cb1ea@redhat.com>
 <3b72a75c-012e-2779-74a4-d36ca1ef9e56@redhat.com>
 <CAEzMDiARnp-FMqUcoVdCBSSApaRbenvrk7cawgyo8kJJvy0jXg@mail.gmail.com>
 <364d6133-158b-0f28-0278-dc0da33b2916@redhat.com>
 <CAEzMDiAq1U3SRkvwh5nqETDBoZs8DojhmMTDSG7B-Uzbh1a3Nw@mail.gmail.com>
Message-ID: <0910DD04CBD6DE4193FCF86B9C00BE9701EE2111@BPXM01GP.gisp.nec.co.jp>

Hello,

I would like to inform you that I can't reply on this thread next week
because of vacation, Thanks for your cooperation.

BTW, I answer the first question:

>Is there plan to support newer kernel?

I'm going to release the next version in July, it will support kernel 4.11.

Thanks,
Atsushi Kumagai

>
>Hi Pratyush/Dave,
>
>On Tue, Jun 13, 2017 at 1:36 PM, Pratyush Anand <panand at redhat.com> wrote:
>> Hi Abhishek,
>>
>> On Thursday 08 June 2017 11:30 AM, Abhishek Shah wrote:
>>>
>>> Hi Pratyush,
>>>
>>> Kernel config:
>>> ARM64_VA_BITS_48 [=y]
>>> ARM64_PAGE_SHIFT [=12]
>>> RANDOMIZE_BASE [=n]
>>> ARM64_4K_PAGES [=y]
>>
>> These are the only variable configs on which arm64 makedumpfile depends. I
>> tested with above configuration and things were working fine.
>>
>> I used 4.11 based kernel and  crash-7.1.9.
>>
>> BTW, which version of crash utility do you use. If you are not at latest can
>> you please try with latest.
>>
>I was using crash 7.1.9++ with head commit
>5c52842a58a2602dba81de71831af98b2b53c6e0;
>I have upgraded my kernel and built the crash utility again; and
>following is my setup details now:
>
>kernel: 4.12- rc5
>yocto based rootfs: Pyro[2.3] which has kexec 2.0.14 and makedumpfile 1.6.1.
>crash: 7.1.9++ [Head: 4d517ad28acd845fe6e91360e645cf0446a4757b]
>
>The output generated by makedumpfile gets processed by crash utility
>if I use following command:
>makedumpfile -E /proc/vmcore vmcore
>
>But, the output generated by following command (which I used while
>reporting the issue) gives the "seek error" while being processed by
>crash.
>makedumpfile /proc/vmcore vmcore
>
>(As a result?), output from following command gives "seek error" as
>well while being processed by crash:
>makedumpfile -c /proc/vmcore vmcore
>
>makedumpfile Manual suggests:
>"-E
>Create DUMPFILE in the ELF format.
>This option cannot be specified with -c option, because the ELF format
>does not support compressed data."
>So only elf formatted output from makedumpfile gets processed by crash.
>
>@Pratyush, Thanks for testing and verifying the kernel configuration
>for arm64 makedumpfile.
>When you say things were working fine for you,  what exact
>makedumpfile command did you use??
>
>Regards,
>Abhishek
>
>_______________________________________________
>kexec mailing list
>kexec at lists.infradead.org
>http://lists.infradead.org/mailman/listinfo/kexec




From panand at redhat.com  Fri Jun 16 01:34:49 2017
From: panand at redhat.com (Pratyush Anand)
Date: Fri, 16 Jun 2017 14:04:49 +0530
Subject: makedumpfile: support for newer kernels [v4.9 onwards]
In-Reply-To: <CAEzMDiAq1U3SRkvwh5nqETDBoZs8DojhmMTDSG7B-Uzbh1a3Nw@mail.gmail.com>
References: <CAEzMDiD-w91Q+Fjy_KgXDcwo9YGasa9ggqW2dEhF4ffV7RstqQ@mail.gmail.com>
 <0bd0a35a-88cd-3f68-846f-2a2ad1e7afcb@redhat.com>
 <CAEzMDiBKdA15Gn2kgs=BgzCM=bzaBXCQP_tM_Vmfc4uxGL0SRg@mail.gmail.com>
 <0ae1e066-821e-3b11-ee44-b615245cb1ea@redhat.com>
 <3b72a75c-012e-2779-74a4-d36ca1ef9e56@redhat.com>
 <CAEzMDiARnp-FMqUcoVdCBSSApaRbenvrk7cawgyo8kJJvy0jXg@mail.gmail.com>
 <364d6133-158b-0f28-0278-dc0da33b2916@redhat.com>
 <CAEzMDiAq1U3SRkvwh5nqETDBoZs8DojhmMTDSG7B-Uzbh1a3Nw@mail.gmail.com>
Message-ID: <148b498a-cd13-bcb2-de04-d74b1c2a0f65@redhat.com>



On Tuesday 13 June 2017 08:47 PM, Abhishek Shah wrote:
> @Pratyush, Thanks for testing and verifying the kernel configuration
> for arm64 makedumpfile.
> When you say things were working fine for you,  what exact
> makedumpfile command did you use??

makedumpfile -l --message-level 1 -d 31 /proc/vmcore /var/crash/vmcore

~Pratyush


From horms at verge.net.au  Fri Jun 16 07:12:55 2017
From: horms at verge.net.au (Simon Horman)
Date: Fri, 16 Jun 2017 16:12:55 +0200
Subject: [ANNOUNCE] kexec-tools 2.0.15
Message-ID: <20170616141255.GA28512@verge.net.au>

Hi all,

I am happy to announce the release of kexec-tools 2.0.15.

This is a feature release. It would have coincided with the release of the
v4.11 Linux Kernel, however, due to an oversight on my part is is rather late.

This release can be downloaded from kernel.org:

	http://kernel.org/pub/linux/utils/kernel/kexec/kexec-tools-2.0.15.tar.xz
	http://kernel.org/pub/linux/utils/kernel/kexec/

It is also tagged it in git:

	git://git.kernel.org/pub/scm/utils/kernel/kexec/kexec-tools.git

Thanks to everyone who has contributed to kexec.


From thomas.lendacky at amd.com  Fri Jun 16 11:49:48 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Fri, 16 Jun 2017 13:49:48 -0500
Subject: [PATCH v7 00/36] x86: Secure Memory Encryption (AMD)
Message-ID: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>

This patch series provides support for AMD's new Secure Memory Encryption (SME)
feature.

SME can be used to mark individual pages of memory as encrypted through the
page tables. A page of memory that is marked encrypted will be automatically
decrypted when read from DRAM and will be automatically encrypted when
written to DRAM. Details on SME can found in the links below.

The SME feature is identified through a CPUID function and enabled through
the SYSCFG MSR. Once enabled, page table entries will determine how the
memory is accessed. If a page table entry has the memory encryption mask set,
then that memory will be accessed as encrypted memory. The memory encryption
mask (as well as other related information) is determined from settings
returned through the same CPUID function that identifies the presence of the
feature.

The approach that this patch series takes is to encrypt everything possible
starting early in the boot where the kernel is encrypted. Using the page
table macros the encryption mask can be incorporated into all page table
entries and page allocations. By updating the protection map, userspace
allocations are also marked encrypted. Certain data must be accounted for
as having been placed in memory before SME was enabled (EFI, initrd, etc.)
and accessed accordingly.

This patch series is a pre-cursor to another AMD processor feature called
Secure Encrypted Virtualization (SEV). The support for SEV will build upon
the SME support and will be submitted later. Details on SEV can be found
in the links below.

The following links provide additional detail:

AMD Memory Encryption whitepaper:
   http://amd-dev.wpengine.netdna-cdn.com/wordpress/media/2013/12/AMD_Memory_Encryption_Whitepaper_v7-Public.pdf

AMD64 Architecture Programmer's Manual:
   http://support.amd.com/TechDocs/24593.pdf
   SME is section 7.10
   SEV is section 15.34

---

This patch series is based off of the master branch of tip:
  https://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git master

  Commit 3d55328fd1f8 ("Merge branch 'WIP.x86/fpu'")

Source code is also available at https://github.com/codomania/tip/tree/sme-v7


Still to do:
- Kdump support, including using memremap() instead of ioremap_cache()

Changes since v6:
- Fixed the asm include file issue that caused build errors on other archs
- Rebased the CR3 register changes on top of Andy Lutomirski's patch
- Added a patch to clear the SME cpu feature if running as a PV guest under
  Xen
- Added a patch to obtain the AMD microcode level earlier in the boot
  instead of directly reading the MSR
- Refactor patch #8 ("x86/mm: Add support to enable SME in early boot
  processing") because the 5-level paging support moved the code into the
  new C-function __startup_64()
- Removed need to decrypt trampoline area in-place (set memory attributes
  before copying the trampoline code)
- General code cleanup based on feedback

Changes since v5:
- Added support for 5-level paging
- Added IOMMU support
- Created a generic asm/mem_encrypt.h in order to remove a bunch of
  #ifndef/#define entries
- Removed changes to the __va() macro and defined a function to return
  the true physical address in cr3
- Removed sysfs support as it was determined not to be needed
- General code cleanup based on feedback
- General cleanup of patch subjects and descriptions

Changes since v4:
- Re-worked mapping of setup data to not use a fixed list. Rather, check
  dynamically whether the requested early_memremap()/memremap() call
  needs to be mapped decrypted.
- Moved SME cpu feature into scattered features
- Moved some declarations into header files
- Cleared the encryption mask from the __PHYSICAL_MASK so that users
  of macros such as pmd_pfn_mask() don't have to worry/know about the
  encryption mask
- Updated some return types and values related to EFI and e820 functions
  so that an error could be returned
- During cpu shutdown, removed cache disabling and added a check for kexec
  in progress to use wbinvd followed immediately by halt in order to avoid
  any memory corruption
- Update how persistent memory is identified
- Added a function to find command line arguments and their values
- Added sysfs support
- General code cleanup based on feedback
- General cleanup of patch subjects and descriptions


Changes since v3:
- Broke out some of the patches into smaller individual patches
- Updated Documentation
- Added a message to indicate why the IOMMU was disabled
- Updated CPU feature support for SME by taking into account whether
  BIOS has enabled SME
- Eliminated redundant functions
- Added some warning messages for DMA usage of bounce buffers when SME
  is active
- Added support for persistent memory
- Added support to determine when setup data is being mapped and be sure
  to map it un-encrypted
- Added CONFIG support to set the default action of whether to activate
  SME if it is supported/enabled
- Added support for (re)booting with kexec

Changes since v2:
- Updated Documentation
- Make the encryption mask available outside of arch/x86 through a
  standard include file
- Conversion of assembler routines to C where possible (not everything
  could be converted, e.g. the routine that does the actual encryption
  needs to be copied into a safe location and it is difficult to
  determine the actual length of the function in order to copy it)
- Fix SME feature use of scattered CPUID feature
- Creation of SME specific functions for things like encrypting
  the setup data, ramdisk, etc.
- New take on early_memremap / memremap encryption support
- Additional support for accessing video buffers (fbdev/gpu) as
  un-encrypted
- Disable IOMMU for now - need to investigate further in relation to
  how it needs to be programmed relative to accessing physical memory

Changes since v1:
- Added Documentation.
- Removed AMD vendor check for setting the PAT write protect mode
- Updated naming of trampoline flag for SME as well as moving of the
  SME check to before paging is enabled.
- Change to early_memremap to identify the data being mapped as either
  boot data or kernel data.  The idea being that boot data will have
  been placed in memory as un-encrypted data and would need to be accessed
  as such.
- Updated debugfs support for the bootparams to access the data properly.
- Do not set the SYSCFG[MEME] bit, only check it.  The setting of the
  MemEncryptionModeEn bit results in a reduction of physical address size
  of the processor.  It is possible that BIOS could have configured resources
  resources into a range that will now not be addressable.  To prevent this,
  rely on BIOS to set the SYSCFG[MEME] bit and only then enable memory
  encryption support in the kernel.

Tom Lendacky (36):
      x86: Document AMD Secure Memory Encryption (SME)
      x86/mm/pat: Set write-protect cache mode for full PAT support
      x86, mpparse, x86/acpi, x86/PCI, x86/dmi, SFI: Use memremap for RAM mappings
      x86/CPU/AMD: Add the Secure Memory Encryption CPU feature
      x86/CPU/AMD: Handle SME reduction in physical address size
      x86/mm: Add Secure Memory Encryption (SME) support
      x86/mm: Don't use phys_to_virt in ioremap() if SME is active
      x86/mm: Add support to enable SME in early boot processing
      x86/mm: Simplify p[gum]d_page() macros
      x86/mm: Provide general kernel support for memory encryption
      x86/mm: Add SME support for read_cr3_pa()
      x86/mm: Extend early_memremap() support with additional attrs
      x86/mm: Add support for early encrypt/decrypt of memory
      x86/mm: Insure that boot memory areas are mapped properly
      x86/boot/e820: Add support to determine the E820 type of an address
      efi: Add an EFI table address match function
      efi: Update efi_mem_type() to return an error rather than 0
      x86/efi: Update EFI pagetable creation to work with SME
      x86/mm: Add support to access boot related data in the clear
      x86, mpparse: Use memremap to map the mpf and mpc data
      x86/mm: Add support to access persistent memory in the clear
      x86/mm: Add support for changing the memory encryption attribute
      x86, realmode: Decrypt trampoline area if memory encryption is active
      x86, swiotlb: Add memory encryption support
      swiotlb: Add warnings for use of bounce buffers with SME
      x86/CPU/AMD: Make the microcode level available earlier in the boot
      iommu/amd: Allow the AMD IOMMU to work with memory encryption
      x86, realmode: Check for memory encryption on the APs
      x86, drm, fbdev: Do not specify encrypted memory for video mappings
      kvm: x86: svm: Support Secure Memory Encryption within KVM
      x86/mm, kexec: Allow kexec to be used with SME
      xen/x86: Remove SME feature in PV guests
      x86/mm: Use proper encryption attributes with /dev/mem
      x86/mm: Add support to encrypt the kernel in-place
      x86/boot: Add early cmdline parsing for options with arguments
      x86/mm: Add support to make use of Secure Memory Encryption


 Documentation/admin-guide/kernel-parameters.txt |   11 
 Documentation/x86/amd-memory-encryption.txt     |   68 ++
 arch/ia64/kernel/efi.c                          |    4 
 arch/x86/Kconfig                                |   30 +
 arch/x86/boot/compressed/pagetable.c            |    7 
 arch/x86/include/asm/cmdline.h                  |    2 
 arch/x86/include/asm/cpufeatures.h              |    1 
 arch/x86/include/asm/dma-mapping.h              |    5 
 arch/x86/include/asm/dmi.h                      |    8 
 arch/x86/include/asm/e820/api.h                 |    2 
 arch/x86/include/asm/fixmap.h                   |   20 +
 arch/x86/include/asm/init.h                     |    1 
 arch/x86/include/asm/io.h                       |    8 
 arch/x86/include/asm/kexec.h                    |    8 
 arch/x86/include/asm/kvm_host.h                 |    2 
 arch/x86/include/asm/mem_encrypt.h              |   80 +++
 arch/x86/include/asm/msr-index.h                |    2 
 arch/x86/include/asm/page_types.h               |    3 
 arch/x86/include/asm/pgtable.h                  |   28 +
 arch/x86/include/asm/pgtable_types.h            |   54 +-
 arch/x86/include/asm/processor-flags.h          |    3 
 arch/x86/include/asm/processor.h                |    8 
 arch/x86/include/asm/realmode.h                 |   12 
 arch/x86/include/asm/set_memory.h               |    3 
 arch/x86/include/asm/vga.h                      |   14 -
 arch/x86/kernel/acpi/boot.c                     |    6 
 arch/x86/kernel/cpu/amd.c                       |   25 +
 arch/x86/kernel/cpu/scattered.c                 |    1 
 arch/x86/kernel/e820.c                          |   26 +
 arch/x86/kernel/espfix_64.c                     |    2 
 arch/x86/kernel/head64.c                        |   74 ++-
 arch/x86/kernel/head_64.S                       |   59 ++
 arch/x86/kernel/kdebugfs.c                      |   34 -
 arch/x86/kernel/ksysfs.c                        |   28 +
 arch/x86/kernel/machine_kexec_64.c              |   22 +
 arch/x86/kernel/mpparse.c                       |  108 +++-
 arch/x86/kernel/pci-dma.c                       |   11 
 arch/x86/kernel/pci-nommu.c                     |    2 
 arch/x86/kernel/pci-swiotlb.c                   |   15 -
 arch/x86/kernel/process.c                       |   17 +
 arch/x86/kernel/setup.c                         |    9 
 arch/x86/kvm/mmu.c                              |   12 
 arch/x86/kvm/mmu.h                              |    2 
 arch/x86/kvm/svm.c                              |   35 +
 arch/x86/kvm/vmx.c                              |    3 
 arch/x86/kvm/x86.c                              |    3 
 arch/x86/lib/cmdline.c                          |  105 ++++
 arch/x86/mm/Makefile                            |    3 
 arch/x86/mm/ident_map.c                         |   12 
 arch/x86/mm/ioremap.c                           |  275 ++++++++++
 arch/x86/mm/kasan_init_64.c                     |    6 
 arch/x86/mm/mem_encrypt.c                       |  627 +++++++++++++++++++++++
 arch/x86/mm/mem_encrypt_boot.S                  |  150 ++++++
 arch/x86/mm/pageattr.c                          |   67 ++
 arch/x86/mm/pat.c                               |    9 
 arch/x86/pci/common.c                           |    4 
 arch/x86/platform/efi/efi.c                     |    6 
 arch/x86/platform/efi/efi_64.c                  |   15 -
 arch/x86/realmode/init.c                        |   12 
 arch/x86/realmode/rm/trampoline_64.S            |   24 +
 arch/x86/xen/enlighten_pv.c                     |    1 
 drivers/firmware/dmi-sysfs.c                    |    5 
 drivers/firmware/efi/efi.c                      |   33 +
 drivers/firmware/pcdp.c                         |    4 
 drivers/gpu/drm/drm_gem.c                       |    2 
 drivers/gpu/drm/drm_vm.c                        |    4 
 drivers/gpu/drm/ttm/ttm_bo_vm.c                 |    7 
 drivers/gpu/drm/udl/udl_fb.c                    |    4 
 drivers/iommu/amd_iommu.c                       |   30 +
 drivers/iommu/amd_iommu_init.c                  |   34 +
 drivers/iommu/amd_iommu_proto.h                 |   10 
 drivers/iommu/amd_iommu_types.h                 |    2 
 drivers/sfi/sfi_core.c                          |   22 -
 drivers/video/fbdev/core/fbmem.c                |   12 
 include/asm-generic/early_ioremap.h             |    2 
 include/asm-generic/pgtable.h                   |    8 
 include/linux/dma-mapping.h                     |   11 
 include/linux/efi.h                             |    9 
 include/linux/io.h                              |    2 
 include/linux/kexec.h                           |   14 +
 include/linux/mem_encrypt.h                     |   51 ++
 include/linux/swiotlb.h                         |    1 
 init/main.c                                     |   10 
 kernel/kexec_core.c                             |   12 
 kernel/memremap.c                               |   20 +
 lib/swiotlb.c                                   |   57 ++
 mm/early_ioremap.c                              |   28 +
 87 files changed, 2324 insertions(+), 264 deletions(-)
 create mode 100644 Documentation/x86/amd-memory-encryption.txt
 create mode 100644 arch/x86/include/asm/mem_encrypt.h
 create mode 100644 arch/x86/mm/mem_encrypt.c
 create mode 100644 arch/x86/mm/mem_encrypt_boot.S
 create mode 100644 include/linux/mem_encrypt.h

-- 
Tom Lendacky


From thomas.lendacky at amd.com  Fri Jun 16 11:49:58 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Fri, 16 Jun 2017 13:49:58 -0500
Subject: [PATCH v7 01/36] x86: Document AMD Secure Memory Encryption (SME)
In-Reply-To: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
References: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170616184958.18967.93012.stgit@tlendack-t1.amdoffice.net>

Create a Documentation entry to describe the AMD Secure Memory
Encryption (SME) feature and add documentation for the mem_encrypt=
kernel parameter.

Reviewed-by: Borislav Petkov <bp at suse.de>
Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 Documentation/admin-guide/kernel-parameters.txt |   11 ++++
 Documentation/x86/amd-memory-encryption.txt     |   68 +++++++++++++++++++++++
 2 files changed, 79 insertions(+)
 create mode 100644 Documentation/x86/amd-memory-encryption.txt

diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index ee5c65a..9edc0b7 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -2197,6 +2197,17 @@
 			memory contents and reserves bad memory
 			regions that are detected.
 
+	mem_encrypt=	[X86-64] AMD Secure Memory Encryption (SME) control
+			Valid arguments: on, off
+			Default (depends on kernel configuration option):
+			  on  (CONFIG_AMD_MEM_ENCRYPT_ACTIVE_BY_DEFAULT=y)
+			  off (CONFIG_AMD_MEM_ENCRYPT_ACTIVE_BY_DEFAULT=n)
+			mem_encrypt=on:		Activate SME
+			mem_encrypt=off:	Do not activate SME
+
+			Refer to Documentation/x86/amd-memory-encryption.txt
+			for details on when memory encryption can be activated.
+
 	mem_sleep_default=	[SUSPEND] Default system suspend mode:
 			s2idle  - Suspend-To-Idle
 			shallow - Power-On Suspend or equivalent (if supported)
diff --git a/Documentation/x86/amd-memory-encryption.txt b/Documentation/x86/amd-memory-encryption.txt
new file mode 100644
index 0000000..f512ab7
--- /dev/null
+++ b/Documentation/x86/amd-memory-encryption.txt
@@ -0,0 +1,68 @@
+Secure Memory Encryption (SME) is a feature found on AMD processors.
+
+SME provides the ability to mark individual pages of memory as encrypted using
+the standard x86 page tables.  A page that is marked encrypted will be
+automatically decrypted when read from DRAM and encrypted when written to
+DRAM.  SME can therefore be used to protect the contents of DRAM from physical
+attacks on the system.
+
+A page is encrypted when a page table entry has the encryption bit set (see
+below on how to determine its position).  The encryption bit can also be
+specified in the cr3 register, allowing the PGD table to be encrypted. Each
+successive level of page tables can also be encrypted by setting the encryption
+bit in the page table entry that points to the next table. This allows the full
+page table hierarchy to be encrypted. Note, this means that just because the
+encryption bit is set in cr3, doesn't imply the full hierarchy is encyrpted.
+Each page table entry in the hierarchy needs to have the encryption bit set to
+achieve that. So, theoretically, you could have the encryption bit set in cr3
+so that the PGD is encrypted, but not set the encryption bit in the PGD entry
+for a PUD which results in the PUD pointed to by that entry to not be
+encrypted.
+
+Support for SME can be determined through the CPUID instruction. The CPUID
+function 0x8000001f reports information related to SME:
+
+	0x8000001f[eax]:
+		Bit[0] indicates support for SME
+	0x8000001f[ebx]:
+		Bits[5:0]  pagetable bit number used to activate memory
+			   encryption
+		Bits[11:6] reduction in physical address space, in bits, when
+			   memory encryption is enabled (this only affects
+			   system physical addresses, not guest physical
+			   addresses)
+
+If support for SME is present, MSR 0xc00100010 (MSR_K8_SYSCFG) can be used to
+determine if SME is enabled and/or to enable memory encryption:
+
+	0xc0010010:
+		Bit[23]   0 = memory encryption features are disabled
+			  1 = memory encryption features are enabled
+
+Linux relies on BIOS to set this bit if BIOS has determined that the reduction
+in the physical address space as a result of enabling memory encryption (see
+CPUID information above) will not conflict with the address space resource
+requirements for the system.  If this bit is not set upon Linux startup then
+Linux itself will not set it and memory encryption will not be possible.
+
+The state of SME in the Linux kernel can be documented as follows:
+	- Supported:
+	  The CPU supports SME (determined through CPUID instruction).
+
+	- Enabled:
+	  Supported and bit 23 of MSR_K8_SYSCFG is set.
+
+	- Active:
+	  Supported, Enabled and the Linux kernel is actively applying
+	  the encryption bit to page table entries (the SME mask in the
+	  kernel is non-zero).
+
+SME can also be enabled and activated in the BIOS. If SME is enabled and
+activated in the BIOS, then all memory accesses will be encrypted and it will
+not be necessary to activate the Linux memory encryption support.  If the BIOS
+merely enables SME (sets bit 23 of the MSR_K8_SYSCFG), then Linux can activate
+memory encryption by default (CONFIG_AMD_MEM_ENCRYPT_ACTIVE_BY_DEFAULT=y) or
+by supplying mem_encrypt=on on the kernel command line.  However, if BIOS does
+not enable SME, then Linux will not be able to activate memory encryption, even
+if configured to do so by default or the mem_encrypt=on command line parameter
+is specified.



From thomas.lendacky at amd.com  Fri Jun 16 11:50:09 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Fri, 16 Jun 2017 13:50:09 -0500
Subject: [PATCH v7 02/36] x86/mm/pat: Set write-protect cache mode for full
 PAT support
In-Reply-To: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
References: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170616185008.18967.91275.stgit@tlendack-t1.amdoffice.net>

For processors that support PAT, set the write-protect cache mode
(_PAGE_CACHE_MODE_WP) entry to the actual write-protect value (x05).

Acked-by: Borislav Petkov <bp at suse.de>
Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/mm/pat.c |    6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/arch/x86/mm/pat.c b/arch/x86/mm/pat.c
index 9b78685..6753d9c 100644
--- a/arch/x86/mm/pat.c
+++ b/arch/x86/mm/pat.c
@@ -295,7 +295,7 @@ static void init_cache_modes(void)
  * pat_init - Initialize PAT MSR and PAT table
  *
  * This function initializes PAT MSR and PAT table with an OS-defined value
- * to enable additional cache attributes, WC and WT.
+ * to enable additional cache attributes, WC, WT and WP.
  *
  * This function must be called on all CPUs using the specific sequence of
  * operations defined in Intel SDM. mtrr_rendezvous_handler() provides this
@@ -356,7 +356,7 @@ void pat_init(void)
 		 *      010    2    UC-: _PAGE_CACHE_MODE_UC_MINUS
 		 *      011    3    UC : _PAGE_CACHE_MODE_UC
 		 *      100    4    WB : Reserved
-		 *      101    5    WC : Reserved
+		 *      101    5    WP : _PAGE_CACHE_MODE_WP
 		 *      110    6    UC-: Reserved
 		 *      111    7    WT : _PAGE_CACHE_MODE_WT
 		 *
@@ -364,7 +364,7 @@ void pat_init(void)
 		 * corresponding types in the presence of PAT errata.
 		 */
 		pat = PAT(0, WB) | PAT(1, WC) | PAT(2, UC_MINUS) | PAT(3, UC) |
-		      PAT(4, WB) | PAT(5, WC) | PAT(6, UC_MINUS) | PAT(7, WT);
+		      PAT(4, WB) | PAT(5, WP) | PAT(6, UC_MINUS) | PAT(7, WT);
 	}
 
 	if (!boot_cpu_done) {



From thomas.lendacky at amd.com  Fri Jun 16 11:50:23 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Fri, 16 Jun 2017 13:50:23 -0500
Subject: [PATCH v7 03/36] x86, mpparse, x86/acpi, x86/PCI, x86/dmi,
 SFI: Use memremap for RAM mappings
In-Reply-To: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
References: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170616185023.18967.72831.stgit@tlendack-t1.amdoffice.net>

The ioremap() function is intended for mapping MMIO. For RAM, the
memremap() function should be used. Convert calls from ioremap() to
memremap() when re-mapping RAM.

This will be used later by SME to control how the encryption mask is
applied to memory mappings, with certain memory locations being mapped
decrypted vs encrypted.

Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/include/asm/dmi.h   |    8 ++++----
 arch/x86/kernel/acpi/boot.c  |    6 +++---
 arch/x86/kernel/kdebugfs.c   |   34 +++++++++++-----------------------
 arch/x86/kernel/ksysfs.c     |   28 ++++++++++++++--------------
 arch/x86/kernel/mpparse.c    |   10 +++++-----
 arch/x86/pci/common.c        |    4 ++--
 drivers/firmware/dmi-sysfs.c |    5 +++--
 drivers/firmware/pcdp.c      |    4 ++--
 drivers/sfi/sfi_core.c       |   22 +++++++++++-----------
 9 files changed, 55 insertions(+), 66 deletions(-)

diff --git a/arch/x86/include/asm/dmi.h b/arch/x86/include/asm/dmi.h
index 3c69fed..a8e15b0 100644
--- a/arch/x86/include/asm/dmi.h
+++ b/arch/x86/include/asm/dmi.h
@@ -13,9 +13,9 @@ static __always_inline __init void *dmi_alloc(unsigned len)
 }
 
 /* Use early IO mappings for DMI because it's initialized early */
-#define dmi_early_remap		early_ioremap
-#define dmi_early_unmap		early_iounmap
-#define dmi_remap		ioremap_cache
-#define dmi_unmap		iounmap
+#define dmi_early_remap		early_memremap
+#define dmi_early_unmap		early_memunmap
+#define dmi_remap(_x, _l)	memremap(_x, _l, MEMREMAP_WB)
+#define dmi_unmap(_x)		memunmap(_x)
 
 #endif /* _ASM_X86_DMI_H */
diff --git a/arch/x86/kernel/acpi/boot.c b/arch/x86/kernel/acpi/boot.c
index 6bb6806..850160a 100644
--- a/arch/x86/kernel/acpi/boot.c
+++ b/arch/x86/kernel/acpi/boot.c
@@ -115,7 +115,7 @@
 #define	ACPI_INVALID_GSI		INT_MIN
 
 /*
- * This is just a simple wrapper around early_ioremap(),
+ * This is just a simple wrapper around early_memremap(),
  * with sanity checks for phys == 0 and size == 0.
  */
 char *__init __acpi_map_table(unsigned long phys, unsigned long size)
@@ -124,7 +124,7 @@ char *__init __acpi_map_table(unsigned long phys, unsigned long size)
 	if (!phys || !size)
 		return NULL;
 
-	return early_ioremap(phys, size);
+	return early_memremap(phys, size);
 }
 
 void __init __acpi_unmap_table(char *map, unsigned long size)
@@ -132,7 +132,7 @@ void __init __acpi_unmap_table(char *map, unsigned long size)
 	if (!map || !size)
 		return;
 
-	early_iounmap(map, size);
+	early_memunmap(map, size);
 }
 
 #ifdef CONFIG_X86_LOCAL_APIC
diff --git a/arch/x86/kernel/kdebugfs.c b/arch/x86/kernel/kdebugfs.c
index 38b6458..fd6f8fb 100644
--- a/arch/x86/kernel/kdebugfs.c
+++ b/arch/x86/kernel/kdebugfs.c
@@ -33,7 +33,6 @@ static ssize_t setup_data_read(struct file *file, char __user *user_buf,
 	struct setup_data_node *node = file->private_data;
 	unsigned long remain;
 	loff_t pos = *ppos;
-	struct page *pg;
 	void *p;
 	u64 pa;
 
@@ -47,18 +46,13 @@ static ssize_t setup_data_read(struct file *file, char __user *user_buf,
 		count = node->len - pos;
 
 	pa = node->paddr + sizeof(struct setup_data) + pos;
-	pg = pfn_to_page((pa + count - 1) >> PAGE_SHIFT);
-	if (PageHighMem(pg)) {
-		p = ioremap_cache(pa, count);
-		if (!p)
-			return -ENXIO;
-	} else
-		p = __va(pa);
+	p = memremap(pa, count, MEMREMAP_WB);
+	if (!p)
+		return -ENOMEM;
 
 	remain = copy_to_user(user_buf, p, count);
 
-	if (PageHighMem(pg))
-		iounmap(p);
+	memunmap(p);
 
 	if (remain)
 		return -EFAULT;
@@ -109,7 +103,6 @@ static int __init create_setup_data_nodes(struct dentry *parent)
 	struct setup_data *data;
 	int error;
 	struct dentry *d;
-	struct page *pg;
 	u64 pa_data;
 	int no = 0;
 
@@ -126,16 +119,12 @@ static int __init create_setup_data_nodes(struct dentry *parent)
 			goto err_dir;
 		}
 
-		pg = pfn_to_page((pa_data+sizeof(*data)-1) >> PAGE_SHIFT);
-		if (PageHighMem(pg)) {
-			data = ioremap_cache(pa_data, sizeof(*data));
-			if (!data) {
-				kfree(node);
-				error = -ENXIO;
-				goto err_dir;
-			}
-		} else
-			data = __va(pa_data);
+		data = memremap(pa_data, sizeof(*data), MEMREMAP_WB);
+		if (!data) {
+			kfree(node);
+			error = -ENOMEM;
+			goto err_dir;
+		}
 
 		node->paddr = pa_data;
 		node->type = data->type;
@@ -143,8 +132,7 @@ static int __init create_setup_data_nodes(struct dentry *parent)
 		error = create_setup_data_node(d, no, node);
 		pa_data = data->next;
 
-		if (PageHighMem(pg))
-			iounmap(data);
+		memunmap(data);
 		if (error)
 			goto err_dir;
 		no++;
diff --git a/arch/x86/kernel/ksysfs.c b/arch/x86/kernel/ksysfs.c
index 4afc67f..ee51db9 100644
--- a/arch/x86/kernel/ksysfs.c
+++ b/arch/x86/kernel/ksysfs.c
@@ -16,8 +16,8 @@
 #include <linux/stat.h>
 #include <linux/slab.h>
 #include <linux/mm.h>
+#include <linux/io.h>
 
-#include <asm/io.h>
 #include <asm/setup.h>
 
 static ssize_t version_show(struct kobject *kobj,
@@ -79,12 +79,12 @@ static int get_setup_data_paddr(int nr, u64 *paddr)
 			*paddr = pa_data;
 			return 0;
 		}
-		data = ioremap_cache(pa_data, sizeof(*data));
+		data = memremap(pa_data, sizeof(*data), MEMREMAP_WB);
 		if (!data)
 			return -ENOMEM;
 
 		pa_data = data->next;
-		iounmap(data);
+		memunmap(data);
 		i++;
 	}
 	return -EINVAL;
@@ -97,17 +97,17 @@ static int __init get_setup_data_size(int nr, size_t *size)
 	u64 pa_data = boot_params.hdr.setup_data;
 
 	while (pa_data) {
-		data = ioremap_cache(pa_data, sizeof(*data));
+		data = memremap(pa_data, sizeof(*data), MEMREMAP_WB);
 		if (!data)
 			return -ENOMEM;
 		if (nr == i) {
 			*size = data->len;
-			iounmap(data);
+			memunmap(data);
 			return 0;
 		}
 
 		pa_data = data->next;
-		iounmap(data);
+		memunmap(data);
 		i++;
 	}
 	return -EINVAL;
@@ -127,12 +127,12 @@ static ssize_t type_show(struct kobject *kobj,
 	ret = get_setup_data_paddr(nr, &paddr);
 	if (ret)
 		return ret;
-	data = ioremap_cache(paddr, sizeof(*data));
+	data = memremap(paddr, sizeof(*data), MEMREMAP_WB);
 	if (!data)
 		return -ENOMEM;
 
 	ret = sprintf(buf, "0x%x\n", data->type);
-	iounmap(data);
+	memunmap(data);
 	return ret;
 }
 
@@ -154,7 +154,7 @@ static ssize_t setup_data_data_read(struct file *fp,
 	ret = get_setup_data_paddr(nr, &paddr);
 	if (ret)
 		return ret;
-	data = ioremap_cache(paddr, sizeof(*data));
+	data = memremap(paddr, sizeof(*data), MEMREMAP_WB);
 	if (!data)
 		return -ENOMEM;
 
@@ -170,15 +170,15 @@ static ssize_t setup_data_data_read(struct file *fp,
 		goto out;
 
 	ret = count;
-	p = ioremap_cache(paddr + sizeof(*data), data->len);
+	p = memremap(paddr + sizeof(*data), data->len, MEMREMAP_WB);
 	if (!p) {
 		ret = -ENOMEM;
 		goto out;
 	}
 	memcpy(buf, p + off, count);
-	iounmap(p);
+	memunmap(p);
 out:
-	iounmap(data);
+	memunmap(data);
 	return ret;
 }
 
@@ -250,13 +250,13 @@ static int __init get_setup_data_total_num(u64 pa_data, int *nr)
 	*nr = 0;
 	while (pa_data) {
 		*nr += 1;
-		data = ioremap_cache(pa_data, sizeof(*data));
+		data = memremap(pa_data, sizeof(*data), MEMREMAP_WB);
 		if (!data) {
 			ret = -ENOMEM;
 			goto out;
 		}
 		pa_data = data->next;
-		iounmap(data);
+		memunmap(data);
 	}
 
 out:
diff --git a/arch/x86/kernel/mpparse.c b/arch/x86/kernel/mpparse.c
index 0d904d7..fd37f39 100644
--- a/arch/x86/kernel/mpparse.c
+++ b/arch/x86/kernel/mpparse.c
@@ -436,9 +436,9 @@ static unsigned long __init get_mpc_size(unsigned long physptr)
 	struct mpc_table *mpc;
 	unsigned long size;
 
-	mpc = early_ioremap(physptr, PAGE_SIZE);
+	mpc = early_memremap(physptr, PAGE_SIZE);
 	size = mpc->length;
-	early_iounmap(mpc, PAGE_SIZE);
+	early_memunmap(mpc, PAGE_SIZE);
 	apic_printk(APIC_VERBOSE, "  mpc: %lx-%lx\n", physptr, physptr + size);
 
 	return size;
@@ -450,7 +450,7 @@ static int __init check_physptr(struct mpf_intel *mpf, unsigned int early)
 	unsigned long size;
 
 	size = get_mpc_size(mpf->physptr);
-	mpc = early_ioremap(mpf->physptr, size);
+	mpc = early_memremap(mpf->physptr, size);
 	/*
 	 * Read the physical hardware table.  Anything here will
 	 * override the defaults.
@@ -461,10 +461,10 @@ static int __init check_physptr(struct mpf_intel *mpf, unsigned int early)
 #endif
 		pr_err("BIOS bug, MP table errors detected!...\n");
 		pr_cont("... disabling SMP support. (tell your hw vendor)\n");
-		early_iounmap(mpc, size);
+		early_memunmap(mpc, size);
 		return -1;
 	}
-	early_iounmap(mpc, size);
+	early_memunmap(mpc, size);
 
 	if (early)
 		return -1;
diff --git a/arch/x86/pci/common.c b/arch/x86/pci/common.c
index 190e718..08cf71c 100644
--- a/arch/x86/pci/common.c
+++ b/arch/x86/pci/common.c
@@ -691,7 +691,7 @@ int pcibios_add_device(struct pci_dev *dev)
 
 	pa_data = boot_params.hdr.setup_data;
 	while (pa_data) {
-		data = ioremap(pa_data, sizeof(*rom));
+		data = memremap(pa_data, sizeof(*rom), MEMREMAP_WB);
 		if (!data)
 			return -ENOMEM;
 
@@ -710,7 +710,7 @@ int pcibios_add_device(struct pci_dev *dev)
 			}
 		}
 		pa_data = data->next;
-		iounmap(data);
+		memunmap(data);
 	}
 	set_dma_domain_ops(dev);
 	set_dev_domain_options(dev);
diff --git a/drivers/firmware/dmi-sysfs.c b/drivers/firmware/dmi-sysfs.c
index ef76e5e..d5de6ee 100644
--- a/drivers/firmware/dmi-sysfs.c
+++ b/drivers/firmware/dmi-sysfs.c
@@ -25,6 +25,7 @@
 #include <linux/slab.h>
 #include <linux/list.h>
 #include <linux/io.h>
+#include <asm/dmi.h>
 
 #define MAX_ENTRY_TYPE 255 /* Most of these aren't used, but we consider
 			      the top entry type is only 8 bits */
@@ -380,7 +381,7 @@ static ssize_t dmi_sel_raw_read_phys32(struct dmi_sysfs_entry *entry,
 	u8 __iomem *mapped;
 	ssize_t wrote = 0;
 
-	mapped = ioremap(sel->access_method_address, sel->area_length);
+	mapped = dmi_remap(sel->access_method_address, sel->area_length);
 	if (!mapped)
 		return -EIO;
 
@@ -390,7 +391,7 @@ static ssize_t dmi_sel_raw_read_phys32(struct dmi_sysfs_entry *entry,
 		wrote++;
 	}
 
-	iounmap(mapped);
+	dmi_unmap(mapped);
 	return wrote;
 }
 
diff --git a/drivers/firmware/pcdp.c b/drivers/firmware/pcdp.c
index 75273a25..e83d6ae 100644
--- a/drivers/firmware/pcdp.c
+++ b/drivers/firmware/pcdp.c
@@ -95,7 +95,7 @@
 	if (efi.hcdp == EFI_INVALID_TABLE_ADDR)
 		return -ENODEV;
 
-	pcdp = early_ioremap(efi.hcdp, 4096);
+	pcdp = early_memremap(efi.hcdp, 4096);
 	printk(KERN_INFO "PCDP: v%d at 0x%lx\n", pcdp->rev, efi.hcdp);
 
 	if (strstr(cmdline, "console=hcdp")) {
@@ -131,6 +131,6 @@
 	}
 
 out:
-	early_iounmap(pcdp, 4096);
+	early_memunmap(pcdp, 4096);
 	return rc;
 }
diff --git a/drivers/sfi/sfi_core.c b/drivers/sfi/sfi_core.c
index 296db7a..d5ce534 100644
--- a/drivers/sfi/sfi_core.c
+++ b/drivers/sfi/sfi_core.c
@@ -86,13 +86,13 @@
 /*
  * FW creates and saves the SFI tables in memory. When these tables get
  * used, they may need to be mapped to virtual address space, and the mapping
- * can happen before or after the ioremap() is ready, so a flag is needed
+ * can happen before or after the memremap() is ready, so a flag is needed
  * to indicating this
  */
-static u32 sfi_use_ioremap __read_mostly;
+static u32 sfi_use_memremap __read_mostly;
 
 /*
- * sfi_un/map_memory calls early_ioremap/iounmap which is a __init function
+ * sfi_un/map_memory calls early_memremap/memunmap which is a __init function
  * and introduces section mismatch. So use __ref to make it calm.
  */
 static void __iomem * __ref sfi_map_memory(u64 phys, u32 size)
@@ -100,10 +100,10 @@ static void __iomem * __ref sfi_map_memory(u64 phys, u32 size)
 	if (!phys || !size)
 		return NULL;
 
-	if (sfi_use_ioremap)
-		return ioremap_cache(phys, size);
+	if (sfi_use_memremap)
+		return memremap(phys, size, MEMREMAP_WB);
 	else
-		return early_ioremap(phys, size);
+		return early_memremap(phys, size);
 }
 
 static void __ref sfi_unmap_memory(void __iomem *virt, u32 size)
@@ -111,10 +111,10 @@ static void __ref sfi_unmap_memory(void __iomem *virt, u32 size)
 	if (!virt || !size)
 		return;
 
-	if (sfi_use_ioremap)
-		iounmap(virt);
+	if (sfi_use_memremap)
+		memunmap(virt);
 	else
-		early_iounmap(virt, size);
+		early_memunmap(virt, size);
 }
 
 static void sfi_print_table_header(unsigned long long pa,
@@ -507,8 +507,8 @@ void __init sfi_init_late(void)
 	length = syst_va->header.len;
 	sfi_unmap_memory(syst_va, sizeof(struct sfi_table_simple));
 
-	/* Use ioremap now after it is ready */
-	sfi_use_ioremap = 1;
+	/* Use memremap now after it is ready */
+	sfi_use_memremap = 1;
 	syst_va = sfi_map_memory(syst_pa, length);
 
 	sfi_acpi_init();



From thomas.lendacky at amd.com  Fri Jun 16 11:50:34 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Fri, 16 Jun 2017 13:50:34 -0500
Subject: [PATCH v7 04/36] x86/CPU/AMD: Add the Secure Memory Encryption CPU
 feature
In-Reply-To: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
References: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170616185033.18967.46040.stgit@tlendack-t1.amdoffice.net>

Update the CPU features to include identifying and reporting on the
Secure Memory Encryption (SME) feature.  SME is identified by CPUID
0x8000001f, but requires BIOS support to enable it (set bit 23 of
MSR_K8_SYSCFG).  Only show the SME feature as available if reported by
CPUID and enabled by BIOS.

Reviewed-by: Borislav Petkov <bp at suse.de>
Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/include/asm/cpufeatures.h |    1 +
 arch/x86/include/asm/msr-index.h   |    2 ++
 arch/x86/kernel/cpu/amd.c          |   13 +++++++++++++
 arch/x86/kernel/cpu/scattered.c    |    1 +
 4 files changed, 17 insertions(+)

diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h
index 2701e5f..2b692df 100644
--- a/arch/x86/include/asm/cpufeatures.h
+++ b/arch/x86/include/asm/cpufeatures.h
@@ -196,6 +196,7 @@
 
 #define X86_FEATURE_HW_PSTATE	( 7*32+ 8) /* AMD HW-PState */
 #define X86_FEATURE_PROC_FEEDBACK ( 7*32+ 9) /* AMD ProcFeedbackInterface */
+#define X86_FEATURE_SME		( 7*32+10) /* AMD Secure Memory Encryption */
 
 #define X86_FEATURE_INTEL_PPIN	( 7*32+14) /* Intel Processor Inventory Number */
 #define X86_FEATURE_INTEL_PT	( 7*32+15) /* Intel Processor Trace */
diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h
index 18b1623..460ac01 100644
--- a/arch/x86/include/asm/msr-index.h
+++ b/arch/x86/include/asm/msr-index.h
@@ -352,6 +352,8 @@
 #define MSR_K8_TOP_MEM1			0xc001001a
 #define MSR_K8_TOP_MEM2			0xc001001d
 #define MSR_K8_SYSCFG			0xc0010010
+#define MSR_K8_SYSCFG_MEM_ENCRYPT_BIT	23
+#define MSR_K8_SYSCFG_MEM_ENCRYPT	BIT_ULL(MSR_K8_SYSCFG_MEM_ENCRYPT_BIT)
 #define MSR_K8_INT_PENDING_MSG		0xc0010055
 /* C1E active bits in int pending message */
 #define K8_INTP_C1E_ACTIVE_MASK		0x18000000
diff --git a/arch/x86/kernel/cpu/amd.c b/arch/x86/kernel/cpu/amd.c
index bb5abe8..c47ceee 100644
--- a/arch/x86/kernel/cpu/amd.c
+++ b/arch/x86/kernel/cpu/amd.c
@@ -611,6 +611,19 @@ static void early_init_amd(struct cpuinfo_x86 *c)
 	 */
 	if (cpu_has_amd_erratum(c, amd_erratum_400))
 		set_cpu_bug(c, X86_BUG_AMD_E400);
+
+	/*
+	 * BIOS support is required for SME. If BIOS has not enabled SME
+	 * then don't advertise the feature (set in scattered.c)
+	 */
+	if (cpu_has(c, X86_FEATURE_SME)) {
+		u64 msr;
+
+		/* Check if SME is enabled */
+		rdmsrl(MSR_K8_SYSCFG, msr);
+		if (!(msr & MSR_K8_SYSCFG_MEM_ENCRYPT))
+			clear_cpu_cap(c, X86_FEATURE_SME);
+	}
 }
 
 static void init_amd_k8(struct cpuinfo_x86 *c)
diff --git a/arch/x86/kernel/cpu/scattered.c b/arch/x86/kernel/cpu/scattered.c
index 23c2350..05459ad 100644
--- a/arch/x86/kernel/cpu/scattered.c
+++ b/arch/x86/kernel/cpu/scattered.c
@@ -31,6 +31,7 @@ struct cpuid_bit {
 	{ X86_FEATURE_HW_PSTATE,	CPUID_EDX,  7, 0x80000007, 0 },
 	{ X86_FEATURE_CPB,		CPUID_EDX,  9, 0x80000007, 0 },
 	{ X86_FEATURE_PROC_FEEDBACK,    CPUID_EDX, 11, 0x80000007, 0 },
+	{ X86_FEATURE_SME,		CPUID_EAX,  0, 0x8000001f, 0 },
 	{ 0, 0, 0, 0, 0 }
 };
 



From thomas.lendacky at amd.com  Fri Jun 16 11:50:45 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Fri, 16 Jun 2017 13:50:45 -0500
Subject: [PATCH v7 05/36] x86/CPU/AMD: Handle SME reduction in physical
 address size
In-Reply-To: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
References: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170616185045.18967.79495.stgit@tlendack-t1.amdoffice.net>

When System Memory Encryption (SME) is enabled, the physical address
space is reduced. Adjust the x86_phys_bits value to reflect this
reduction.

Reviewed-by: Borislav Petkov <bp at suse.de>
Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/kernel/cpu/amd.c |   10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/arch/x86/kernel/cpu/amd.c b/arch/x86/kernel/cpu/amd.c
index c47ceee..5bdcbd4 100644
--- a/arch/x86/kernel/cpu/amd.c
+++ b/arch/x86/kernel/cpu/amd.c
@@ -613,15 +613,19 @@ static void early_init_amd(struct cpuinfo_x86 *c)
 		set_cpu_bug(c, X86_BUG_AMD_E400);
 
 	/*
-	 * BIOS support is required for SME. If BIOS has not enabled SME
-	 * then don't advertise the feature (set in scattered.c)
+	 * BIOS support is required for SME. If BIOS has enabld SME then
+	 * adjust x86_phys_bits by the SME physical address space reduction
+	 * value. If BIOS has not enabled SME then don't advertise the
+	 * feature (set in scattered.c).
 	 */
 	if (cpu_has(c, X86_FEATURE_SME)) {
 		u64 msr;
 
 		/* Check if SME is enabled */
 		rdmsrl(MSR_K8_SYSCFG, msr);
-		if (!(msr & MSR_K8_SYSCFG_MEM_ENCRYPT))
+		if (msr & MSR_K8_SYSCFG_MEM_ENCRYPT)
+			c->x86_phys_bits -= (cpuid_ebx(0x8000001f) >> 6) & 0x3f;
+		else
 			clear_cpu_cap(c, X86_FEATURE_SME);
 	}
 }



From thomas.lendacky at amd.com  Fri Jun 16 11:50:54 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Fri, 16 Jun 2017 13:50:54 -0500
Subject: [PATCH v7 06/36] x86/mm: Add Secure Memory Encryption (SME) support
In-Reply-To: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
References: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170616185054.18967.52228.stgit@tlendack-t1.amdoffice.net>

Add support for Secure Memory Encryption (SME). This initial support
provides a Kconfig entry to build the SME support into the kernel and
defines the memory encryption mask that will be used in subsequent
patches to mark pages as encrypted.

Reviewed-by: Borislav Petkov <bp at suse.de>
Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/Kconfig                   |   26 ++++++++++++++++++++++++++
 arch/x86/include/asm/mem_encrypt.h |   30 ++++++++++++++++++++++++++++++
 arch/x86/mm/Makefile               |    1 +
 arch/x86/mm/mem_encrypt.c          |   21 +++++++++++++++++++++
 include/linux/mem_encrypt.h        |   35 +++++++++++++++++++++++++++++++++++
 5 files changed, 113 insertions(+)
 create mode 100644 arch/x86/include/asm/mem_encrypt.h
 create mode 100644 arch/x86/mm/mem_encrypt.c
 create mode 100644 include/linux/mem_encrypt.h

diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index 7c991d0..cf74791 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -1407,6 +1407,32 @@ config X86_DIRECT_GBPAGES
 	  supports them), so don't confuse the user by printing
 	  that we have them enabled.
 
+config ARCH_HAS_MEM_ENCRYPT
+	def_bool y
+	depends on X86
+
+config AMD_MEM_ENCRYPT
+	bool "AMD Secure Memory Encryption (SME) support"
+	depends on X86_64 && CPU_SUP_AMD
+	---help---
+	  Say yes to enable support for the encryption of system memory.
+	  This requires an AMD processor that supports Secure Memory
+	  Encryption (SME).
+
+config AMD_MEM_ENCRYPT_ACTIVE_BY_DEFAULT
+	bool "Activate AMD Secure Memory Encryption (SME) by default"
+	default y
+	depends on AMD_MEM_ENCRYPT
+	---help---
+	  Say yes to have system memory encrypted by default if running on
+	  an AMD processor that supports Secure Memory Encryption (SME).
+
+	  If set to Y, then the encryption of system memory can be
+	  deactivated with the mem_encrypt=off command line option.
+
+	  If set to N, then the encryption of system memory can be
+	  activated with the mem_encrypt=on command line option.
+
 # Common NUMA Features
 config NUMA
 	bool "Numa Memory Allocation and Scheduler Support"
diff --git a/arch/x86/include/asm/mem_encrypt.h b/arch/x86/include/asm/mem_encrypt.h
new file mode 100644
index 0000000..a105796
--- /dev/null
+++ b/arch/x86/include/asm/mem_encrypt.h
@@ -0,0 +1,30 @@
+/*
+ * AMD Memory Encryption Support
+ *
+ * Copyright (C) 2016 Advanced Micro Devices, Inc.
+ *
+ * Author: Tom Lendacky <thomas.lendacky at amd.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#ifndef __X86_MEM_ENCRYPT_H__
+#define __X86_MEM_ENCRYPT_H__
+
+#ifndef __ASSEMBLY__
+
+#ifdef CONFIG_AMD_MEM_ENCRYPT
+
+extern unsigned long sme_me_mask;
+
+#else	/* !CONFIG_AMD_MEM_ENCRYPT */
+
+#define sme_me_mask	0UL
+
+#endif	/* CONFIG_AMD_MEM_ENCRYPT */
+
+#endif	/* __ASSEMBLY__ */
+
+#endif	/* __X86_MEM_ENCRYPT_H__ */
diff --git a/arch/x86/mm/Makefile b/arch/x86/mm/Makefile
index 0fbdcb6..a94a7b6 100644
--- a/arch/x86/mm/Makefile
+++ b/arch/x86/mm/Makefile
@@ -39,3 +39,4 @@ obj-$(CONFIG_X86_INTEL_MPX)	+= mpx.o
 obj-$(CONFIG_X86_INTEL_MEMORY_PROTECTION_KEYS) += pkeys.o
 obj-$(CONFIG_RANDOMIZE_MEMORY) += kaslr.o
 
+obj-$(CONFIG_AMD_MEM_ENCRYPT)	+= mem_encrypt.o
diff --git a/arch/x86/mm/mem_encrypt.c b/arch/x86/mm/mem_encrypt.c
new file mode 100644
index 0000000..b99d469
--- /dev/null
+++ b/arch/x86/mm/mem_encrypt.c
@@ -0,0 +1,21 @@
+/*
+ * AMD Memory Encryption Support
+ *
+ * Copyright (C) 2016 Advanced Micro Devices, Inc.
+ *
+ * Author: Tom Lendacky <thomas.lendacky at amd.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include <linux/linkage.h>
+
+/*
+ * Since SME related variables are set early in the boot process they must
+ * reside in the .data section so as not to be zeroed out when the .bss
+ * section is later cleared.
+ */
+unsigned long sme_me_mask __section(.data) = 0;
+EXPORT_SYMBOL_GPL(sme_me_mask);
diff --git a/include/linux/mem_encrypt.h b/include/linux/mem_encrypt.h
new file mode 100644
index 0000000..59769f7
--- /dev/null
+++ b/include/linux/mem_encrypt.h
@@ -0,0 +1,35 @@
+/*
+ * AMD Memory Encryption Support
+ *
+ * Copyright (C) 2016 Advanced Micro Devices, Inc.
+ *
+ * Author: Tom Lendacky <thomas.lendacky at amd.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#ifndef __MEM_ENCRYPT_H__
+#define __MEM_ENCRYPT_H__
+
+#ifndef __ASSEMBLY__
+
+#ifdef CONFIG_ARCH_HAS_MEM_ENCRYPT
+
+#include <asm/mem_encrypt.h>
+
+#else	/* !CONFIG_ARCH_HAS_MEM_ENCRYPT */
+
+#define sme_me_mask	0UL
+
+#endif	/* CONFIG_ARCH_HAS_MEM_ENCRYPT */
+
+static inline bool sme_active(void)
+{
+	return !!sme_me_mask;
+}
+
+#endif	/* __ASSEMBLY__ */
+
+#endif	/* __MEM_ENCRYPT_H__ */



From thomas.lendacky at amd.com  Fri Jun 16 11:51:04 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Fri, 16 Jun 2017 13:51:04 -0500
Subject: [PATCH v7 07/36] x86/mm: Don't use phys_to_virt in ioremap() if SME
 is active
In-Reply-To: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
References: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170616185104.18967.7867.stgit@tlendack-t1.amdoffice.net>

Currently there is a check if the address being mapped is in the ISA
range (is_ISA_range()), and if it is then phys_to_virt() is used to
perform the mapping.  When SME is active, however, this will result
in the mapping having the encryption bit set when it is expected that
an ioremap() should not have the encryption bit set. So only use the
phys_to_virt() function if SME is not active

Reviewed-by: Borislav Petkov <bp at suse.de>
Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/mm/ioremap.c |    7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/arch/x86/mm/ioremap.c b/arch/x86/mm/ioremap.c
index 4c1b5fd..a382ba9 100644
--- a/arch/x86/mm/ioremap.c
+++ b/arch/x86/mm/ioremap.c
@@ -13,6 +13,7 @@
 #include <linux/slab.h>
 #include <linux/vmalloc.h>
 #include <linux/mmiotrace.h>
+#include <linux/mem_encrypt.h>
 
 #include <asm/set_memory.h>
 #include <asm/e820/api.h>
@@ -106,9 +107,11 @@ static void __iomem *__ioremap_caller(resource_size_t phys_addr,
 	}
 
 	/*
-	 * Don't remap the low PCI/ISA area, it's always mapped..
+	 * Don't remap the low PCI/ISA area, it's always mapped.
+	 *   But if SME is active, skip this so that the encryption bit
+	 *   doesn't get set.
 	 */
-	if (is_ISA_range(phys_addr, last_addr))
+	if (is_ISA_range(phys_addr, last_addr) && !sme_active())
 		return (__force void __iomem *)phys_to_virt(phys_addr);
 
 	/*



From thomas.lendacky at amd.com  Fri Jun 16 11:51:15 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Fri, 16 Jun 2017 13:51:15 -0500
Subject: [PATCH v7 08/36] x86/mm: Add support to enable SME in early boot
 processing
In-Reply-To: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
References: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170616185115.18967.79622.stgit@tlendack-t1.amdoffice.net>

Add support to the early boot code to use Secure Memory Encryption (SME).
Since the kernel has been loaded into memory in a decrypted state, encrypt
the kernel in place and update the early pagetables with the memory
encryption mask so that new pagetable entries will use memory encryption.

The routines to set the encryption mask and perform the encryption are
stub routines for now with functionality to be added in a later patch.

Because of the need to have the routines available to head_64.S, the
mem_encrypt.c is always built and #ifdefs in mem_encrypt.c will provide
functionality or stub routines depending on CONFIG_AMD_MEM_ENCRYPT.

Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/include/asm/mem_encrypt.h |    8 +++++++
 arch/x86/kernel/head64.c           |   33 +++++++++++++++++++++---------
 arch/x86/kernel/head_64.S          |   39 ++++++++++++++++++++++++++++++++++--
 arch/x86/mm/Makefile               |    4 +---
 arch/x86/mm/mem_encrypt.c          |   24 ++++++++++++++++++++++
 5 files changed, 93 insertions(+), 15 deletions(-)

diff --git a/arch/x86/include/asm/mem_encrypt.h b/arch/x86/include/asm/mem_encrypt.h
index a105796..988b336 100644
--- a/arch/x86/include/asm/mem_encrypt.h
+++ b/arch/x86/include/asm/mem_encrypt.h
@@ -15,16 +15,24 @@
 
 #ifndef __ASSEMBLY__
 
+#include <linux/init.h>
+
 #ifdef CONFIG_AMD_MEM_ENCRYPT
 
 extern unsigned long sme_me_mask;
 
+void __init sme_enable(void);
+
 #else	/* !CONFIG_AMD_MEM_ENCRYPT */
 
 #define sme_me_mask	0UL
 
+static inline void __init sme_enable(void) { }
+
 #endif	/* CONFIG_AMD_MEM_ENCRYPT */
 
+unsigned long sme_get_me_mask(void);
+
 #endif	/* __ASSEMBLY__ */
 
 #endif	/* __X86_MEM_ENCRYPT_H__ */
diff --git a/arch/x86/kernel/head64.c b/arch/x86/kernel/head64.c
index 2b2ac38..95979c3 100644
--- a/arch/x86/kernel/head64.c
+++ b/arch/x86/kernel/head64.c
@@ -14,6 +14,7 @@
 #include <linux/start_kernel.h>
 #include <linux/io.h>
 #include <linux/memblock.h>
+#include <linux/mem_encrypt.h>
 
 #include <asm/processor.h>
 #include <asm/proto.h>
@@ -46,6 +47,7 @@ static void __init *fixup_pointer(void *ptr, unsigned long physaddr)
 void __init __startup_64(unsigned long physaddr)
 {
 	unsigned long load_delta, *p;
+	unsigned long pgtable_flags;
 	pgdval_t *pgd;
 	p4dval_t *p4d;
 	pudval_t *pud;
@@ -66,6 +68,12 @@ void __init __startup_64(unsigned long physaddr)
 	if (load_delta & ~PMD_PAGE_MASK)
 		for (;;);
 
+	/* Activate Secure Memory Encryption (SME) if supported and enabled */
+	sme_enable();
+
+	/* Include the SME encryption mask in the fixup value */
+	load_delta += sme_get_me_mask();
+
 	/* Fixup the physical addresses in the page table */
 
 	pgd = fixup_pointer(&early_top_pgt, physaddr);
@@ -92,28 +100,30 @@ void __init __startup_64(unsigned long physaddr)
 
 	pud = fixup_pointer(early_dynamic_pgts[next_early_pgt++], physaddr);
 	pmd = fixup_pointer(early_dynamic_pgts[next_early_pgt++], physaddr);
+	pgtable_flags = _KERNPG_TABLE + sme_get_me_mask();
 
 	if (IS_ENABLED(CONFIG_X86_5LEVEL)) {
 		p4d = fixup_pointer(early_dynamic_pgts[next_early_pgt++], physaddr);
 
 		i = (physaddr >> PGDIR_SHIFT) % PTRS_PER_PGD;
-		pgd[i + 0] = (pgdval_t)p4d + _KERNPG_TABLE;
-		pgd[i + 1] = (pgdval_t)p4d + _KERNPG_TABLE;
+		pgd[i + 0] = (pgdval_t)p4d + pgtable_flags;
+		pgd[i + 1] = (pgdval_t)p4d + pgtable_flags;
 
 		i = (physaddr >> P4D_SHIFT) % PTRS_PER_P4D;
-		p4d[i + 0] = (pgdval_t)pud + _KERNPG_TABLE;
-		p4d[i + 1] = (pgdval_t)pud + _KERNPG_TABLE;
+		p4d[i + 0] = (pgdval_t)pud + pgtable_flags;
+		p4d[i + 1] = (pgdval_t)pud + pgtable_flags;
 	} else {
 		i = (physaddr >> PGDIR_SHIFT) % PTRS_PER_PGD;
-		pgd[i + 0] = (pgdval_t)pud + _KERNPG_TABLE;
-		pgd[i + 1] = (pgdval_t)pud + _KERNPG_TABLE;
+		pgd[i + 0] = (pgdval_t)pud + pgtable_flags;
+		pgd[i + 1] = (pgdval_t)pud + pgtable_flags;
 	}
 
 	i = (physaddr >> PUD_SHIFT) % PTRS_PER_PUD;
-	pud[i + 0] = (pudval_t)pmd + _KERNPG_TABLE;
-	pud[i + 1] = (pudval_t)pmd + _KERNPG_TABLE;
+	pud[i + 0] = (pudval_t)pmd + pgtable_flags;
+	pud[i + 1] = (pudval_t)pmd + pgtable_flags;
 
 	pmd_entry = __PAGE_KERNEL_LARGE_EXEC & ~_PAGE_GLOBAL;
+	pmd_entry += sme_get_me_mask();
 	pmd_entry +=  physaddr;
 
 	for (i = 0; i < DIV_ROUND_UP(_end - _text, PMD_SIZE); i++) {
@@ -134,9 +144,12 @@ void __init __startup_64(unsigned long physaddr)
 			pmd[i] += load_delta;
 	}
 
-	/* Fixup phys_base */
+	/*
+	 * Fixup phys_base - remove the memory encryption mask to obtain
+	 * the true physical address.
+	 */
 	p = fixup_pointer(&phys_base, physaddr);
-	*p += load_delta;
+	*p += load_delta - sme_get_me_mask();
 }
 
 /* Wipe all early page tables except for the kernel symbol map */
diff --git a/arch/x86/kernel/head_64.S b/arch/x86/kernel/head_64.S
index 6225550..ef12729 100644
--- a/arch/x86/kernel/head_64.S
+++ b/arch/x86/kernel/head_64.S
@@ -78,7 +78,29 @@ startup_64:
 	call	__startup_64
 	popq	%rsi
 
-	movq	$(early_top_pgt - __START_KERNEL_map), %rax
+	/*
+	 * Encrypt the kernel if SME is active.
+	 * The real_mode_data address is in %rsi and that register can be
+	 * clobbered by the called function so be sure to save it.
+	 */
+	push	%rsi
+	call	sme_encrypt_kernel
+	pop	%rsi
+
+	/*
+	 * Get the SME encryption mask.
+	 *  The encryption mask will be returned in %rax so we do an ADD
+	 *  below to be sure that the encryption mask is part of the
+	 *  value that will stored in %cr3.
+	 *
+	 * The real_mode_data address is in %rsi and that register can be
+	 * clobbered by the called function so be sure to save it.
+	 */
+	push	%rsi
+	call	sme_get_me_mask
+	pop	%rsi
+
+	addq	$(early_top_pgt - __START_KERNEL_map), %rax
 	jmp 1f
 ENTRY(secondary_startup_64)
 	/*
@@ -98,7 +120,20 @@ ENTRY(secondary_startup_64)
 	/* Sanitize CPU configuration */
 	call verify_cpu
 
-	movq	$(init_top_pgt - __START_KERNEL_map), %rax
+	/*
+	 * Get the SME encryption mask.
+	 *  The encryption mask will be returned in %rax so we do an ADD
+	 *  below to be sure that the encryption mask is part of the
+	 *  value that will stored in %cr3.
+	 *
+	 * The real_mode_data address is in %rsi and that register can be
+	 * clobbered by the called function so be sure to save it.
+	 */
+	push	%rsi
+	call	sme_get_me_mask
+	pop	%rsi
+
+	addq	$(init_top_pgt - __START_KERNEL_map), %rax
 1:
 
 	/* Enable PAE mode, PGE and LA57 */
diff --git a/arch/x86/mm/Makefile b/arch/x86/mm/Makefile
index a94a7b6..9e13841 100644
--- a/arch/x86/mm/Makefile
+++ b/arch/x86/mm/Makefile
@@ -2,7 +2,7 @@
 KCOV_INSTRUMENT_tlb.o	:= n
 
 obj-y	:=  init.o init_$(BITS).o fault.o ioremap.o extable.o pageattr.o mmap.o \
-	    pat.o pgtable.o physaddr.o setup_nx.o tlb.o
+	    pat.o pgtable.o physaddr.o setup_nx.o tlb.o mem_encrypt.o
 
 # Make sure __phys_addr has no stackprotector
 nostackp := $(call cc-option, -fno-stack-protector)
@@ -38,5 +38,3 @@ obj-$(CONFIG_NUMA_EMU)		+= numa_emulation.o
 obj-$(CONFIG_X86_INTEL_MPX)	+= mpx.o
 obj-$(CONFIG_X86_INTEL_MEMORY_PROTECTION_KEYS) += pkeys.o
 obj-$(CONFIG_RANDOMIZE_MEMORY) += kaslr.o
-
-obj-$(CONFIG_AMD_MEM_ENCRYPT)	+= mem_encrypt.o
diff --git a/arch/x86/mm/mem_encrypt.c b/arch/x86/mm/mem_encrypt.c
index b99d469..9a78277 100644
--- a/arch/x86/mm/mem_encrypt.c
+++ b/arch/x86/mm/mem_encrypt.c
@@ -11,6 +11,9 @@
  */
 
 #include <linux/linkage.h>
+#include <linux/init.h>
+
+#ifdef CONFIG_AMD_MEM_ENCRYPT
 
 /*
  * Since SME related variables are set early in the boot process they must
@@ -19,3 +22,24 @@
  */
 unsigned long sme_me_mask __section(.data) = 0;
 EXPORT_SYMBOL_GPL(sme_me_mask);
+
+void __init sme_encrypt_kernel(void)
+{
+}
+
+void __init sme_enable(void)
+{
+}
+
+unsigned long sme_get_me_mask(void)
+{
+	return sme_me_mask;
+}
+
+#else	/* !CONFIG_AMD_MEM_ENCRYPT */
+
+void __init sme_encrypt_kernel(void)	{ }
+
+unsigned long sme_get_me_mask(void)	{ return 0; }
+
+#endif	/* CONFIG_AMD_MEM_ENCRYPT */



From thomas.lendacky at amd.com  Fri Jun 16 11:51:25 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Fri, 16 Jun 2017 13:51:25 -0500
Subject: [PATCH v7 09/36] x86/mm: Simplify p[gum]d_page() macros
In-Reply-To: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
References: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170616185125.18967.33160.stgit@tlendack-t1.amdoffice.net>

Create a pgd_pfn() macro similar to the p[um]d_pfn() macros and then
use the p[gum]d_pfn() macros in the p[gum]d_page() macros instead of
duplicating the code.

Reviewed-by: Borislav Petkov <bp at suse.de>
Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/include/asm/pgtable.h |   16 +++++++++-------
 1 file changed, 9 insertions(+), 7 deletions(-)

diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h
index 77037b6..b64ea52 100644
--- a/arch/x86/include/asm/pgtable.h
+++ b/arch/x86/include/asm/pgtable.h
@@ -195,6 +195,11 @@ static inline unsigned long p4d_pfn(p4d_t p4d)
 	return (p4d_val(p4d) & p4d_pfn_mask(p4d)) >> PAGE_SHIFT;
 }
 
+static inline unsigned long pgd_pfn(pgd_t pgd)
+{
+	return (pgd_val(pgd) & PTE_PFN_MASK) >> PAGE_SHIFT;
+}
+
 static inline int p4d_large(p4d_t p4d)
 {
 	/* No 512 GiB pages yet */
@@ -704,8 +709,7 @@ static inline unsigned long pmd_page_vaddr(pmd_t pmd)
  * Currently stuck as a macro due to indirect forward reference to
  * linux/mmzone.h's __section_mem_map_addr() definition:
  */
-#define pmd_page(pmd)		\
-	pfn_to_page((pmd_val(pmd) & pmd_pfn_mask(pmd)) >> PAGE_SHIFT)
+#define pmd_page(pmd)	pfn_to_page(pmd_pfn(pmd))
 
 /*
  * the pmd page can be thought of an array like this: pmd_t[PTRS_PER_PMD]
@@ -773,8 +777,7 @@ static inline unsigned long pud_page_vaddr(pud_t pud)
  * Currently stuck as a macro due to indirect forward reference to
  * linux/mmzone.h's __section_mem_map_addr() definition:
  */
-#define pud_page(pud)		\
-	pfn_to_page((pud_val(pud) & pud_pfn_mask(pud)) >> PAGE_SHIFT)
+#define pud_page(pud)	pfn_to_page(pud_pfn(pud))
 
 /* Find an entry in the second-level page table.. */
 static inline pmd_t *pmd_offset(pud_t *pud, unsigned long address)
@@ -824,8 +827,7 @@ static inline unsigned long p4d_page_vaddr(p4d_t p4d)
  * Currently stuck as a macro due to indirect forward reference to
  * linux/mmzone.h's __section_mem_map_addr() definition:
  */
-#define p4d_page(p4d)		\
-	pfn_to_page((p4d_val(p4d) & p4d_pfn_mask(p4d)) >> PAGE_SHIFT)
+#define p4d_page(p4d)	pfn_to_page(p4d_pfn(p4d))
 
 /* Find an entry in the third-level page table.. */
 static inline pud_t *pud_offset(p4d_t *p4d, unsigned long address)
@@ -859,7 +861,7 @@ static inline unsigned long pgd_page_vaddr(pgd_t pgd)
  * Currently stuck as a macro due to indirect forward reference to
  * linux/mmzone.h's __section_mem_map_addr() definition:
  */
-#define pgd_page(pgd)		pfn_to_page(pgd_val(pgd) >> PAGE_SHIFT)
+#define pgd_page(pgd)	pfn_to_page(pgd_pfn(pgd))
 
 /* to find an entry in a page-table-directory. */
 static inline p4d_t *p4d_offset(pgd_t *pgd, unsigned long address)



From thomas.lendacky at amd.com  Fri Jun 16 11:51:40 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Fri, 16 Jun 2017 13:51:40 -0500
Subject: [PATCH v7 10/36] x86/mm: Provide general kernel support for memory
 encryption
In-Reply-To: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
References: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170616185140.18967.61905.stgit@tlendack-t1.amdoffice.net>

Changes to the existing page table macros will allow the SME support to
be enabled in a simple fashion with minimal changes to files that use these
macros.  Since the memory encryption mask will now be part of the regular
pagetable macros, we introduce two new macros (_PAGE_TABLE_NOENC and
_KERNPG_TABLE_NOENC) to allow for early pagetable creation/initialization
without the encryption mask before SME becomes active.  Two new pgprot()
macros are defined to allow setting or clearing the page encryption mask.

The FIXMAP_PAGE_NOCACHE define is introduced for use with MMIO.  SME does
not support encryption for MMIO areas so this define removes the encryption
mask from the page attribute.

Two new macros are introduced (__sme_pa() / __sme_pa_nodebug()) to allow
creating a physical address with the encryption mask.  These are used when
working with the cr3 register so that the PGD can be encrypted. The current
__va() macro is updated so that the virtual address is generated based off
of the physical address without the encryption mask thus allowing the same
virtual address to be generated regardless of whether encryption is enabled
for that physical location or not.

Also, an early initialization function is added for SME.  If SME is active,
this function:
 - Updates the early_pmd_flags so that early page faults create mappings
   with the encryption mask.
 - Updates the __supported_pte_mask to include the encryption mask.
 - Updates the protection_map entries to include the encryption mask so
   that user-space allocations will automatically have the encryption mask
   applied.

Reviewed-by: Borislav Petkov <bp at suse.de>
Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/boot/compressed/pagetable.c |    7 +++++
 arch/x86/include/asm/fixmap.h        |    7 +++++
 arch/x86/include/asm/mem_encrypt.h   |   13 ++++++++++
 arch/x86/include/asm/page_types.h    |    3 ++
 arch/x86/include/asm/pgtable.h       |    9 +++++++
 arch/x86/include/asm/pgtable_types.h |   45 ++++++++++++++++++++++------------
 arch/x86/include/asm/processor.h     |    3 ++
 arch/x86/kernel/espfix_64.c          |    2 +-
 arch/x86/kernel/head64.c             |   11 +++++++-
 arch/x86/kernel/head_64.S            |   20 ++++++++-------
 arch/x86/mm/kasan_init_64.c          |    4 ++-
 arch/x86/mm/mem_encrypt.c            |   18 ++++++++++++++
 arch/x86/mm/pageattr.c               |    3 ++
 include/asm-generic/pgtable.h        |    8 ++++++
 include/linux/mem_encrypt.h          |    8 ++++++
 15 files changed, 128 insertions(+), 33 deletions(-)

diff --git a/arch/x86/boot/compressed/pagetable.c b/arch/x86/boot/compressed/pagetable.c
index 8e69df9..246bf29 100644
--- a/arch/x86/boot/compressed/pagetable.c
+++ b/arch/x86/boot/compressed/pagetable.c
@@ -15,6 +15,13 @@
 #define __pa(x)  ((unsigned long)(x))
 #define __va(x)  ((void *)((unsigned long)(x)))
 
+/*
+ * The pgtable.h and mm/ident_map.c includes make use of the SME related
+ * information which is not used in the compressed image support. Un-define
+ * the SME support to avoid any compile and link errors.
+ */
+#undef CONFIG_AMD_MEM_ENCRYPT
+
 #include "misc.h"
 
 /* These actually do the work of building the kernel identity maps. */
diff --git a/arch/x86/include/asm/fixmap.h b/arch/x86/include/asm/fixmap.h
index b65155c..d9ff226 100644
--- a/arch/x86/include/asm/fixmap.h
+++ b/arch/x86/include/asm/fixmap.h
@@ -157,6 +157,13 @@ static inline void __set_fixmap(enum fixed_addresses idx,
 }
 #endif
 
+/*
+ * FIXMAP_PAGE_NOCACHE is used for MMIO. Memory encryption is not
+ * supported for MMIO addresses, so make sure that the memory encryption
+ * mask is not part of the page attributes.
+ */
+#define FIXMAP_PAGE_NOCACHE PAGE_KERNEL_IO_NOCACHE
+
 #include <asm-generic/fixmap.h>
 
 #define __late_set_fixmap(idx, phys, flags) __set_fixmap(idx, phys, flags)
diff --git a/arch/x86/include/asm/mem_encrypt.h b/arch/x86/include/asm/mem_encrypt.h
index 988b336..faae4e1 100644
--- a/arch/x86/include/asm/mem_encrypt.h
+++ b/arch/x86/include/asm/mem_encrypt.h
@@ -21,18 +21,31 @@
 
 extern unsigned long sme_me_mask;
 
+void __init sme_early_init(void);
+
 void __init sme_enable(void);
 
 #else	/* !CONFIG_AMD_MEM_ENCRYPT */
 
 #define sme_me_mask	0UL
 
+static inline void __init sme_early_init(void) { }
+
 static inline void __init sme_enable(void) { }
 
 #endif	/* CONFIG_AMD_MEM_ENCRYPT */
 
 unsigned long sme_get_me_mask(void);
 
+/*
+ * The __sme_pa() and __sme_pa_nodebug() macros are meant for use when
+ * writing to or comparing values from the cr3 register.  Having the
+ * encryption mask set in cr3 enables the PGD entry to be encrypted and
+ * avoid special case handling of PGD allocations.
+ */
+#define __sme_pa(x)		(__pa(x) | sme_me_mask)
+#define __sme_pa_nodebug(x)	(__pa_nodebug(x) | sme_me_mask)
+
 #endif	/* __ASSEMBLY__ */
 
 #endif	/* __X86_MEM_ENCRYPT_H__ */
diff --git a/arch/x86/include/asm/page_types.h b/arch/x86/include/asm/page_types.h
index 7bd0099..b98ed9d 100644
--- a/arch/x86/include/asm/page_types.h
+++ b/arch/x86/include/asm/page_types.h
@@ -3,6 +3,7 @@
 
 #include <linux/const.h>
 #include <linux/types.h>
+#include <linux/mem_encrypt.h>
 
 /* PAGE_SHIFT determines the page size */
 #define PAGE_SHIFT		12
@@ -15,7 +16,7 @@
 #define PUD_PAGE_SIZE		(_AC(1, UL) << PUD_SHIFT)
 #define PUD_PAGE_MASK		(~(PUD_PAGE_SIZE-1))
 
-#define __PHYSICAL_MASK		((phys_addr_t)((1ULL << __PHYSICAL_MASK_SHIFT) - 1))
+#define __PHYSICAL_MASK		((phys_addr_t)(__sme_clr((1ULL << __PHYSICAL_MASK_SHIFT) - 1)))
 #define __VIRTUAL_MASK		((1UL << __VIRTUAL_MASK_SHIFT) - 1)
 
 /* Cast *PAGE_MASK to a signed type so that it is sign-extended if
diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h
index b64ea52..c6452cb 100644
--- a/arch/x86/include/asm/pgtable.h
+++ b/arch/x86/include/asm/pgtable.h
@@ -1,6 +1,7 @@
 #ifndef _ASM_X86_PGTABLE_H
 #define _ASM_X86_PGTABLE_H
 
+#include <linux/mem_encrypt.h>
 #include <asm/page.h>
 #include <asm/pgtable_types.h>
 
@@ -13,6 +14,12 @@
 		     cachemode2protval(_PAGE_CACHE_MODE_UC_MINUS)))	\
 	 : (prot))
 
+/*
+ * Macros to add or remove encryption attribute
+ */
+#define pgprot_encrypted(prot)	__pgprot(__sme_set(pgprot_val(prot)))
+#define pgprot_decrypted(prot)	__pgprot(__sme_clr(pgprot_val(prot)))
+
 #ifndef __ASSEMBLY__
 #include <asm/x86_init.h>
 
@@ -38,6 +45,8 @@
 
 extern struct mm_struct *pgd_page_get_mm(struct page *page);
 
+extern pmdval_t early_pmd_flags;
+
 #ifdef CONFIG_PARAVIRT
 #include <asm/paravirt.h>
 #else  /* !CONFIG_PARAVIRT */
diff --git a/arch/x86/include/asm/pgtable_types.h b/arch/x86/include/asm/pgtable_types.h
index bf9638e..de32ca3 100644
--- a/arch/x86/include/asm/pgtable_types.h
+++ b/arch/x86/include/asm/pgtable_types.h
@@ -2,6 +2,8 @@
 #define _ASM_X86_PGTABLE_DEFS_H
 
 #include <linux/const.h>
+#include <linux/mem_encrypt.h>
+
 #include <asm/page_types.h>
 
 #define FIRST_USER_ADDRESS	0UL
@@ -121,10 +123,10 @@
 
 #define _PAGE_PROTNONE	(_AT(pteval_t, 1) << _PAGE_BIT_PROTNONE)
 
-#define _PAGE_TABLE	(_PAGE_PRESENT | _PAGE_RW | _PAGE_USER |	\
-			 _PAGE_ACCESSED | _PAGE_DIRTY)
-#define _KERNPG_TABLE	(_PAGE_PRESENT | _PAGE_RW | _PAGE_ACCESSED |	\
-			 _PAGE_DIRTY)
+#define _PAGE_TABLE_NOENC	(_PAGE_PRESENT | _PAGE_RW | _PAGE_USER |\
+				 _PAGE_ACCESSED | _PAGE_DIRTY)
+#define _KERNPG_TABLE_NOENC	(_PAGE_PRESENT | _PAGE_RW |		\
+				 _PAGE_ACCESSED | _PAGE_DIRTY)
 
 /*
  * Set of bits not changed in pte_modify.  The pte's
@@ -191,18 +193,29 @@ enum page_cache_mode {
 #define __PAGE_KERNEL_IO		(__PAGE_KERNEL)
 #define __PAGE_KERNEL_IO_NOCACHE	(__PAGE_KERNEL_NOCACHE)
 
-#define PAGE_KERNEL			__pgprot(__PAGE_KERNEL)
-#define PAGE_KERNEL_RO			__pgprot(__PAGE_KERNEL_RO)
-#define PAGE_KERNEL_EXEC		__pgprot(__PAGE_KERNEL_EXEC)
-#define PAGE_KERNEL_RX			__pgprot(__PAGE_KERNEL_RX)
-#define PAGE_KERNEL_NOCACHE		__pgprot(__PAGE_KERNEL_NOCACHE)
-#define PAGE_KERNEL_LARGE		__pgprot(__PAGE_KERNEL_LARGE)
-#define PAGE_KERNEL_LARGE_EXEC		__pgprot(__PAGE_KERNEL_LARGE_EXEC)
-#define PAGE_KERNEL_VSYSCALL		__pgprot(__PAGE_KERNEL_VSYSCALL)
-#define PAGE_KERNEL_VVAR		__pgprot(__PAGE_KERNEL_VVAR)
-
-#define PAGE_KERNEL_IO			__pgprot(__PAGE_KERNEL_IO)
-#define PAGE_KERNEL_IO_NOCACHE		__pgprot(__PAGE_KERNEL_IO_NOCACHE)
+#ifndef __ASSEMBLY__
+
+#define _PAGE_ENC	(_AT(pteval_t, sme_me_mask))
+
+#define _PAGE_TABLE	(_PAGE_PRESENT | _PAGE_RW | _PAGE_USER |	\
+			 _PAGE_ACCESSED | _PAGE_DIRTY | _PAGE_ENC)
+#define _KERNPG_TABLE	(_PAGE_PRESENT | _PAGE_RW | _PAGE_ACCESSED |	\
+			 _PAGE_DIRTY | _PAGE_ENC)
+
+#define PAGE_KERNEL		__pgprot(__PAGE_KERNEL | _PAGE_ENC)
+#define PAGE_KERNEL_RO		__pgprot(__PAGE_KERNEL_RO | _PAGE_ENC)
+#define PAGE_KERNEL_EXEC	__pgprot(__PAGE_KERNEL_EXEC | _PAGE_ENC)
+#define PAGE_KERNEL_RX		__pgprot(__PAGE_KERNEL_RX | _PAGE_ENC)
+#define PAGE_KERNEL_NOCACHE	__pgprot(__PAGE_KERNEL_NOCACHE | _PAGE_ENC)
+#define PAGE_KERNEL_LARGE	__pgprot(__PAGE_KERNEL_LARGE | _PAGE_ENC)
+#define PAGE_KERNEL_LARGE_EXEC	__pgprot(__PAGE_KERNEL_LARGE_EXEC | _PAGE_ENC)
+#define PAGE_KERNEL_VSYSCALL	__pgprot(__PAGE_KERNEL_VSYSCALL | _PAGE_ENC)
+#define PAGE_KERNEL_VVAR	__pgprot(__PAGE_KERNEL_VVAR | _PAGE_ENC)
+
+#define PAGE_KERNEL_IO		__pgprot(__PAGE_KERNEL_IO)
+#define PAGE_KERNEL_IO_NOCACHE	__pgprot(__PAGE_KERNEL_IO_NOCACHE)
+
+#endif	/* __ASSEMBLY__ */
 
 /*         xwr */
 #define __P000	PAGE_NONE
diff --git a/arch/x86/include/asm/processor.h b/arch/x86/include/asm/processor.h
index f3b1b27..8010c97 100644
--- a/arch/x86/include/asm/processor.h
+++ b/arch/x86/include/asm/processor.h
@@ -29,6 +29,7 @@
 #include <linux/math64.h>
 #include <linux/err.h>
 #include <linux/irqflags.h>
+#include <linux/mem_encrypt.h>
 
 /*
  * We handle most unaligned accesses in hardware.  On the other hand
@@ -241,7 +242,7 @@ static inline unsigned long read_cr3_pa(void)
 
 static inline void load_cr3(pgd_t *pgdir)
 {
-	write_cr3(__pa(pgdir));
+	write_cr3(__sme_pa(pgdir));
 }
 
 #ifdef CONFIG_X86_32
diff --git a/arch/x86/kernel/espfix_64.c b/arch/x86/kernel/espfix_64.c
index 6b91e2e..9c4e7ba 100644
--- a/arch/x86/kernel/espfix_64.c
+++ b/arch/x86/kernel/espfix_64.c
@@ -195,7 +195,7 @@ void init_espfix_ap(int cpu)
 
 	pte_p = pte_offset_kernel(&pmd, addr);
 	stack_page = page_address(alloc_pages_node(node, GFP_KERNEL, 0));
-	pte = __pte(__pa(stack_page) | (__PAGE_KERNEL_RO & ptemask));
+	pte = __pte(__pa(stack_page) | ((__PAGE_KERNEL_RO | _PAGE_ENC) & ptemask));
 	for (n = 0; n < ESPFIX_PTE_CLONES; n++)
 		set_pte(&pte_p[n*PTE_STRIDE], pte);
 
diff --git a/arch/x86/kernel/head64.c b/arch/x86/kernel/head64.c
index 95979c3..5d7363a 100644
--- a/arch/x86/kernel/head64.c
+++ b/arch/x86/kernel/head64.c
@@ -100,7 +100,7 @@ void __init __startup_64(unsigned long physaddr)
 
 	pud = fixup_pointer(early_dynamic_pgts[next_early_pgt++], physaddr);
 	pmd = fixup_pointer(early_dynamic_pgts[next_early_pgt++], physaddr);
-	pgtable_flags = _KERNPG_TABLE + sme_get_me_mask();
+	pgtable_flags = _KERNPG_TABLE_NOENC + sme_get_me_mask();
 
 	if (IS_ENABLED(CONFIG_X86_5LEVEL)) {
 		p4d = fixup_pointer(early_dynamic_pgts[next_early_pgt++], physaddr);
@@ -157,7 +157,7 @@ static void __init reset_early_page_tables(void)
 {
 	memset(early_top_pgt, 0, sizeof(pgd_t)*(PTRS_PER_PGD-1));
 	next_early_pgt = 0;
-	write_cr3(__pa_nodebug(early_top_pgt));
+	write_cr3(__sme_pa_nodebug(early_top_pgt));
 }
 
 /* Create a new PMD entry */
@@ -290,6 +290,13 @@ asmlinkage __visible void __init x86_64_start_kernel(char * real_mode_data)
 
 	clear_page(init_top_pgt);
 
+	/*
+	 * SME support may update early_pmd_flags to include the memory
+	 * encryption mask, so it needs to be called before anything
+	 * that may generate a page fault.
+	 */
+	sme_early_init();
+
 	kasan_early_init();
 
 	for (i = 0; i < NUM_EXCEPTION_VECTORS; i++)
diff --git a/arch/x86/kernel/head_64.S b/arch/x86/kernel/head_64.S
index ef12729..d56ccb71 100644
--- a/arch/x86/kernel/head_64.S
+++ b/arch/x86/kernel/head_64.S
@@ -370,9 +370,9 @@ GLOBAL(name)
 NEXT_PAGE(early_top_pgt)
 	.fill	511,8,0
 #ifdef CONFIG_X86_5LEVEL
-	.quad	level4_kernel_pgt - __START_KERNEL_map + _PAGE_TABLE
+	.quad	level4_kernel_pgt - __START_KERNEL_map + _PAGE_TABLE_NOENC
 #else
-	.quad	level3_kernel_pgt - __START_KERNEL_map + _PAGE_TABLE
+	.quad	level3_kernel_pgt - __START_KERNEL_map + _PAGE_TABLE_NOENC
 #endif
 
 NEXT_PAGE(early_dynamic_pgts)
@@ -385,15 +385,15 @@ NEXT_PAGE(init_top_pgt)
 	.fill	512,8,0
 #else
 NEXT_PAGE(init_top_pgt)
-	.quad   level3_ident_pgt - __START_KERNEL_map + _KERNPG_TABLE
+	.quad   level3_ident_pgt - __START_KERNEL_map + _KERNPG_TABLE_NOENC
 	.org    init_top_pgt + PGD_PAGE_OFFSET*8, 0
-	.quad   level3_ident_pgt - __START_KERNEL_map + _KERNPG_TABLE
+	.quad   level3_ident_pgt - __START_KERNEL_map + _KERNPG_TABLE_NOENC
 	.org    init_top_pgt + PGD_START_KERNEL*8, 0
 	/* (2^48-(2*1024*1024*1024))/(2^39) = 511 */
-	.quad   level3_kernel_pgt - __START_KERNEL_map + _PAGE_TABLE
+	.quad   level3_kernel_pgt - __START_KERNEL_map + _PAGE_TABLE_NOENC
 
 NEXT_PAGE(level3_ident_pgt)
-	.quad	level2_ident_pgt - __START_KERNEL_map + _KERNPG_TABLE
+	.quad	level2_ident_pgt - __START_KERNEL_map + _KERNPG_TABLE_NOENC
 	.fill	511, 8, 0
 NEXT_PAGE(level2_ident_pgt)
 	/* Since I easily can, map the first 1G.
@@ -405,14 +405,14 @@ NEXT_PAGE(level2_ident_pgt)
 #ifdef CONFIG_X86_5LEVEL
 NEXT_PAGE(level4_kernel_pgt)
 	.fill	511,8,0
-	.quad	level3_kernel_pgt - __START_KERNEL_map + _PAGE_TABLE
+	.quad	level3_kernel_pgt - __START_KERNEL_map + _PAGE_TABLE_NOENC
 #endif
 
 NEXT_PAGE(level3_kernel_pgt)
 	.fill	L3_START_KERNEL,8,0
 	/* (2^48-(2*1024*1024*1024)-((2^39)*511))/(2^30) = 510 */
-	.quad	level2_kernel_pgt - __START_KERNEL_map + _KERNPG_TABLE
-	.quad	level2_fixmap_pgt - __START_KERNEL_map + _PAGE_TABLE
+	.quad	level2_kernel_pgt - __START_KERNEL_map + _KERNPG_TABLE_NOENC
+	.quad	level2_fixmap_pgt - __START_KERNEL_map + _PAGE_TABLE_NOENC
 
 NEXT_PAGE(level2_kernel_pgt)
 	/*
@@ -430,7 +430,7 @@ NEXT_PAGE(level2_kernel_pgt)
 
 NEXT_PAGE(level2_fixmap_pgt)
 	.fill	506,8,0
-	.quad	level1_fixmap_pgt - __START_KERNEL_map + _PAGE_TABLE
+	.quad	level1_fixmap_pgt - __START_KERNEL_map + _PAGE_TABLE_NOENC
 	/* 8MB reserved for vsyscalls + a 2MB hole = 4 + 1 entries */
 	.fill	5,8,0
 
diff --git a/arch/x86/mm/kasan_init_64.c b/arch/x86/mm/kasan_init_64.c
index 88215ac..d7cc830 100644
--- a/arch/x86/mm/kasan_init_64.c
+++ b/arch/x86/mm/kasan_init_64.c
@@ -92,7 +92,7 @@ static int kasan_die_handler(struct notifier_block *self,
 void __init kasan_early_init(void)
 {
 	int i;
-	pteval_t pte_val = __pa_nodebug(kasan_zero_page) | __PAGE_KERNEL;
+	pteval_t pte_val = __pa_nodebug(kasan_zero_page) | __PAGE_KERNEL | _PAGE_ENC;
 	pmdval_t pmd_val = __pa_nodebug(kasan_zero_pte) | _KERNPG_TABLE;
 	pudval_t pud_val = __pa_nodebug(kasan_zero_pmd) | _KERNPG_TABLE;
 	p4dval_t p4d_val = __pa_nodebug(kasan_zero_pud) | _KERNPG_TABLE;
@@ -158,7 +158,7 @@ void __init kasan_init(void)
 	 */
 	memset(kasan_zero_page, 0, PAGE_SIZE);
 	for (i = 0; i < PTRS_PER_PTE; i++) {
-		pte_t pte = __pte(__pa(kasan_zero_page) | __PAGE_KERNEL_RO);
+		pte_t pte = __pte(__pa(kasan_zero_page) | __PAGE_KERNEL_RO | _PAGE_ENC);
 		set_pte(&kasan_zero_pte[i], pte);
 	}
 	/* Flush TLBs again to be sure that write protection applied. */
diff --git a/arch/x86/mm/mem_encrypt.c b/arch/x86/mm/mem_encrypt.c
index 9a78277..b2d1cdf 100644
--- a/arch/x86/mm/mem_encrypt.c
+++ b/arch/x86/mm/mem_encrypt.c
@@ -15,6 +15,8 @@
 
 #ifdef CONFIG_AMD_MEM_ENCRYPT
 
+#include <linux/mm.h>
+
 /*
  * Since SME related variables are set early in the boot process they must
  * reside in the .data section so as not to be zeroed out when the .bss
@@ -23,6 +25,22 @@
 unsigned long sme_me_mask __section(.data) = 0;
 EXPORT_SYMBOL_GPL(sme_me_mask);
 
+void __init sme_early_init(void)
+{
+	unsigned int i;
+
+	if (!sme_me_mask)
+		return;
+
+	early_pmd_flags = __sme_set(early_pmd_flags);
+
+	__supported_pte_mask = __sme_set(__supported_pte_mask);
+
+	/* Update the protection map with memory encryption mask */
+	for (i = 0; i < ARRAY_SIZE(protection_map); i++)
+		protection_map[i] = pgprot_encrypted(protection_map[i]);
+}
+
 void __init sme_encrypt_kernel(void)
 {
 }
diff --git a/arch/x86/mm/pageattr.c b/arch/x86/mm/pageattr.c
index c8520b2..e7d3866 100644
--- a/arch/x86/mm/pageattr.c
+++ b/arch/x86/mm/pageattr.c
@@ -2014,6 +2014,9 @@ int kernel_map_pages_in_pgd(pgd_t *pgd, u64 pfn, unsigned long address,
 	if (!(page_flags & _PAGE_RW))
 		cpa.mask_clr = __pgprot(_PAGE_RW);
 
+	if (!(page_flags & _PAGE_ENC))
+		cpa.mask_clr = pgprot_encrypted(cpa.mask_clr);
+
 	cpa.mask_set = __pgprot(_PAGE_PRESENT | page_flags);
 
 	retval = __change_page_attr_set_clr(&cpa, 0);
diff --git a/include/asm-generic/pgtable.h b/include/asm-generic/pgtable.h
index 7dfa767..882cb5d 100644
--- a/include/asm-generic/pgtable.h
+++ b/include/asm-generic/pgtable.h
@@ -424,6 +424,14 @@ static inline int pud_same(pud_t pud_a, pud_t pud_b)
 #define pgprot_device pgprot_noncached
 #endif
 
+#ifndef pgprot_encrypted
+#define pgprot_encrypted(prot)	(prot)
+#endif
+
+#ifndef pgprot_decrypted
+#define pgprot_decrypted(prot)	(prot)
+#endif
+
 #ifndef pgprot_modify
 #define pgprot_modify pgprot_modify
 static inline pgprot_t pgprot_modify(pgprot_t oldprot, pgprot_t newprot)
diff --git a/include/linux/mem_encrypt.h b/include/linux/mem_encrypt.h
index 59769f7..837c66b 100644
--- a/include/linux/mem_encrypt.h
+++ b/include/linux/mem_encrypt.h
@@ -30,6 +30,14 @@ static inline bool sme_active(void)
 	return !!sme_me_mask;
 }
 
+/*
+ * The __sme_set() and __sme_clr() macros are useful for adding or removing
+ * the encryption mask from a value (e.g. when dealing with pagetable
+ * entries).
+ */
+#define __sme_set(x)		((unsigned long)(x) | sme_me_mask)
+#define __sme_clr(x)		((unsigned long)(x) & ~sme_me_mask)
+
 #endif	/* __ASSEMBLY__ */
 
 #endif	/* __MEM_ENCRYPT_H__ */



From thomas.lendacky at amd.com  Fri Jun 16 11:51:55 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Fri, 16 Jun 2017 13:51:55 -0500
Subject: [PATCH v7 11/36] x86/mm: Add SME support for read_cr3_pa()
In-Reply-To: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
References: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170616185154.18967.71073.stgit@tlendack-t1.amdoffice.net>

The cr3 register entry can contain the SME encryption mask that indicates
the PGD is encrypted.  The encryption mask should not be used when
creating a virtual address from the cr3 register, so remove the SME
encryption mask in the read_cr3_pa() function.

During early boot SME will need to use a native version of read_cr3_pa(),
so create native_read_cr3_pa().

Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/include/asm/processor-flags.h |    3 ++-
 arch/x86/include/asm/processor.h       |    5 +++++
 2 files changed, 7 insertions(+), 1 deletion(-)

diff --git a/arch/x86/include/asm/processor-flags.h b/arch/x86/include/asm/processor-flags.h
index 79aa2f9..cb6999c 100644
--- a/arch/x86/include/asm/processor-flags.h
+++ b/arch/x86/include/asm/processor-flags.h
@@ -2,6 +2,7 @@
 #define _ASM_X86_PROCESSOR_FLAGS_H
 
 #include <uapi/asm/processor-flags.h>
+#include <linux/mem_encrypt.h>
 
 #ifdef CONFIG_VM86
 #define X86_VM_MASK	X86_EFLAGS_VM
@@ -33,7 +34,7 @@
  */
 #ifdef CONFIG_X86_64
 /* Mask off the address space ID bits. */
-#define CR3_ADDR_MASK 0x7FFFFFFFFFFFF000ull
+#define CR3_ADDR_MASK __sme_clr(0x7FFFFFFFFFFFF000ull)
 #define CR3_PCID_MASK 0xFFFull
 #else
 /*
diff --git a/arch/x86/include/asm/processor.h b/arch/x86/include/asm/processor.h
index 8010c97..ab878bd 100644
--- a/arch/x86/include/asm/processor.h
+++ b/arch/x86/include/asm/processor.h
@@ -240,6 +240,11 @@ static inline unsigned long read_cr3_pa(void)
 	return __read_cr3() & CR3_ADDR_MASK;
 }
 
+static inline unsigned long native_read_cr3_pa(void)
+{
+	return __native_read_cr3() & CR3_ADDR_MASK;
+}
+
 static inline void load_cr3(pgd_t *pgdir)
 {
 	write_cr3(__sme_pa(pgdir));



From thomas.lendacky at amd.com  Fri Jun 16 11:52:08 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Fri, 16 Jun 2017 13:52:08 -0500
Subject: [PATCH v7 12/36] x86/mm: Extend early_memremap() support with
 additional attrs
In-Reply-To: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
References: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170616185208.18967.72141.stgit@tlendack-t1.amdoffice.net>

Add early_memremap() support to be able to specify encrypted and
decrypted mappings with and without write-protection. The use of
write-protection is necessary when encrypting data "in place". The
write-protect attribute is considered cacheable for loads, but not
stores. This implies that the hardware will never give the core a
dirty line with this memtype.

Reviewed-by: Borislav Petkov <bp at suse.de>
Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/Kconfig                     |    4 +++
 arch/x86/include/asm/fixmap.h        |   13 ++++++++++
 arch/x86/include/asm/pgtable_types.h |    8 ++++++
 arch/x86/mm/ioremap.c                |   44 ++++++++++++++++++++++++++++++++++
 include/asm-generic/early_ioremap.h  |    2 ++
 mm/early_ioremap.c                   |   10 ++++++++
 6 files changed, 81 insertions(+)

diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index cf74791..0b09b88 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -1433,6 +1433,10 @@ config AMD_MEM_ENCRYPT_ACTIVE_BY_DEFAULT
 	  If set to N, then the encryption of system memory can be
 	  activated with the mem_encrypt=on command line option.
 
+config ARCH_USE_MEMREMAP_PROT
+	def_bool y
+	depends on AMD_MEM_ENCRYPT
+
 # Common NUMA Features
 config NUMA
 	bool "Numa Memory Allocation and Scheduler Support"
diff --git a/arch/x86/include/asm/fixmap.h b/arch/x86/include/asm/fixmap.h
index d9ff226..dcd9fb5 100644
--- a/arch/x86/include/asm/fixmap.h
+++ b/arch/x86/include/asm/fixmap.h
@@ -164,6 +164,19 @@ static inline void __set_fixmap(enum fixed_addresses idx,
  */
 #define FIXMAP_PAGE_NOCACHE PAGE_KERNEL_IO_NOCACHE
 
+/*
+ * Early memremap routines used for in-place encryption. The mappings created
+ * by these routines are intended to be used as temporary mappings.
+ */
+void __init *early_memremap_encrypted(resource_size_t phys_addr,
+				      unsigned long size);
+void __init *early_memremap_encrypted_wp(resource_size_t phys_addr,
+					 unsigned long size);
+void __init *early_memremap_decrypted(resource_size_t phys_addr,
+				      unsigned long size);
+void __init *early_memremap_decrypted_wp(resource_size_t phys_addr,
+					 unsigned long size);
+
 #include <asm-generic/fixmap.h>
 
 #define __late_set_fixmap(idx, phys, flags) __set_fixmap(idx, phys, flags)
diff --git a/arch/x86/include/asm/pgtable_types.h b/arch/x86/include/asm/pgtable_types.h
index de32ca3..32095af 100644
--- a/arch/x86/include/asm/pgtable_types.h
+++ b/arch/x86/include/asm/pgtable_types.h
@@ -161,6 +161,7 @@ enum page_cache_mode {
 
 #define _PAGE_CACHE_MASK	(_PAGE_PAT | _PAGE_PCD | _PAGE_PWT)
 #define _PAGE_NOCACHE		(cachemode2protval(_PAGE_CACHE_MODE_UC))
+#define _PAGE_CACHE_WP		(cachemode2protval(_PAGE_CACHE_MODE_WP))
 
 #define PAGE_NONE	__pgprot(_PAGE_PROTNONE | _PAGE_ACCESSED)
 #define PAGE_SHARED	__pgprot(_PAGE_PRESENT | _PAGE_RW | _PAGE_USER | \
@@ -189,6 +190,7 @@ enum page_cache_mode {
 #define __PAGE_KERNEL_VVAR		(__PAGE_KERNEL_RO | _PAGE_USER)
 #define __PAGE_KERNEL_LARGE		(__PAGE_KERNEL | _PAGE_PSE)
 #define __PAGE_KERNEL_LARGE_EXEC	(__PAGE_KERNEL_EXEC | _PAGE_PSE)
+#define __PAGE_KERNEL_WP		(__PAGE_KERNEL | _PAGE_CACHE_WP)
 
 #define __PAGE_KERNEL_IO		(__PAGE_KERNEL)
 #define __PAGE_KERNEL_IO_NOCACHE	(__PAGE_KERNEL_NOCACHE)
@@ -202,6 +204,12 @@ enum page_cache_mode {
 #define _KERNPG_TABLE	(_PAGE_PRESENT | _PAGE_RW | _PAGE_ACCESSED |	\
 			 _PAGE_DIRTY | _PAGE_ENC)
 
+#define __PAGE_KERNEL_ENC	(__PAGE_KERNEL | _PAGE_ENC)
+#define __PAGE_KERNEL_ENC_WP	(__PAGE_KERNEL_WP | _PAGE_ENC)
+
+#define __PAGE_KERNEL_NOENC	(__PAGE_KERNEL)
+#define __PAGE_KERNEL_NOENC_WP	(__PAGE_KERNEL_WP)
+
 #define PAGE_KERNEL		__pgprot(__PAGE_KERNEL | _PAGE_ENC)
 #define PAGE_KERNEL_RO		__pgprot(__PAGE_KERNEL_RO | _PAGE_ENC)
 #define PAGE_KERNEL_EXEC	__pgprot(__PAGE_KERNEL_EXEC | _PAGE_ENC)
diff --git a/arch/x86/mm/ioremap.c b/arch/x86/mm/ioremap.c
index a382ba9..4feda83 100644
--- a/arch/x86/mm/ioremap.c
+++ b/arch/x86/mm/ioremap.c
@@ -422,6 +422,50 @@ void unxlate_dev_mem_ptr(phys_addr_t phys, void *addr)
 	iounmap((void __iomem *)((unsigned long)addr & PAGE_MASK));
 }
 
+#ifdef CONFIG_ARCH_USE_MEMREMAP_PROT
+/* Remap memory with encryption */
+void __init *early_memremap_encrypted(resource_size_t phys_addr,
+				      unsigned long size)
+{
+	return early_memremap_prot(phys_addr, size, __PAGE_KERNEL_ENC);
+}
+
+/*
+ * Remap memory with encryption and write-protected - cannot be called
+ * before pat_init() is called
+ */
+void __init *early_memremap_encrypted_wp(resource_size_t phys_addr,
+					 unsigned long size)
+{
+	/* Be sure the write-protect PAT entry is set for write-protect */
+	if (__pte2cachemode_tbl[_PAGE_CACHE_MODE_WP] != _PAGE_CACHE_MODE_WP)
+		return NULL;
+
+	return early_memremap_prot(phys_addr, size, __PAGE_KERNEL_ENC_WP);
+}
+
+/* Remap memory without encryption */
+void __init *early_memremap_decrypted(resource_size_t phys_addr,
+				      unsigned long size)
+{
+	return early_memremap_prot(phys_addr, size, __PAGE_KERNEL_NOENC);
+}
+
+/*
+ * Remap memory without encryption and write-protected - cannot be called
+ * before pat_init() is called
+ */
+void __init *early_memremap_decrypted_wp(resource_size_t phys_addr,
+					 unsigned long size)
+{
+	/* Be sure the write-protect PAT entry is set for write-protect */
+	if (__pte2cachemode_tbl[_PAGE_CACHE_MODE_WP] != _PAGE_CACHE_MODE_WP)
+		return NULL;
+
+	return early_memremap_prot(phys_addr, size, __PAGE_KERNEL_NOENC_WP);
+}
+#endif	/* CONFIG_ARCH_USE_MEMREMAP_PROT */
+
 static pte_t bm_pte[PAGE_SIZE/sizeof(pte_t)] __page_aligned_bss;
 
 static inline pmd_t * __init early_ioremap_pmd(unsigned long addr)
diff --git a/include/asm-generic/early_ioremap.h b/include/asm-generic/early_ioremap.h
index 734ad4d..2edef8d 100644
--- a/include/asm-generic/early_ioremap.h
+++ b/include/asm-generic/early_ioremap.h
@@ -13,6 +13,8 @@ extern void *early_memremap(resource_size_t phys_addr,
 			    unsigned long size);
 extern void *early_memremap_ro(resource_size_t phys_addr,
 			       unsigned long size);
+extern void *early_memremap_prot(resource_size_t phys_addr,
+				 unsigned long size, unsigned long prot_val);
 extern void early_iounmap(void __iomem *addr, unsigned long size);
 extern void early_memunmap(void *addr, unsigned long size);
 
diff --git a/mm/early_ioremap.c b/mm/early_ioremap.c
index 6d5717b..d7d30da 100644
--- a/mm/early_ioremap.c
+++ b/mm/early_ioremap.c
@@ -226,6 +226,16 @@ void __init early_iounmap(void __iomem *addr, unsigned long size)
 }
 #endif
 
+#ifdef CONFIG_ARCH_USE_MEMREMAP_PROT
+void __init *
+early_memremap_prot(resource_size_t phys_addr, unsigned long size,
+		    unsigned long prot_val)
+{
+	return (__force void *)__early_ioremap(phys_addr, size,
+					       __pgprot(prot_val));
+}
+#endif
+
 #define MAX_MAP_CHUNK	(NR_FIX_BTMAPS << PAGE_SHIFT)
 
 void __init copy_from_early_mem(void *dest, phys_addr_t src, unsigned long size)



From thomas.lendacky at amd.com  Fri Jun 16 11:52:20 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Fri, 16 Jun 2017 13:52:20 -0500
Subject: [PATCH v7 13/36] x86/mm: Add support for early encrypt/decrypt of
 memory
In-Reply-To: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
References: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170616185220.18967.78204.stgit@tlendack-t1.amdoffice.net>

Add support to be able to either encrypt or decrypt data in place during
the early stages of booting the kernel. This does not change the memory
encryption attribute - it is used for ensuring that data present in either
an encrypted or decrypted memory area is in the proper state (for example
the initrd will have been loaded by the boot loader and will not be
encrypted, but the memory that it resides in is marked as encrypted).

Reviewed-by: Borislav Petkov <bp at suse.de>
Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/include/asm/mem_encrypt.h |   10 +++++
 arch/x86/mm/mem_encrypt.c          |   76 ++++++++++++++++++++++++++++++++++++
 2 files changed, 86 insertions(+)

diff --git a/arch/x86/include/asm/mem_encrypt.h b/arch/x86/include/asm/mem_encrypt.h
index faae4e1..6508ec9 100644
--- a/arch/x86/include/asm/mem_encrypt.h
+++ b/arch/x86/include/asm/mem_encrypt.h
@@ -21,6 +21,11 @@
 
 extern unsigned long sme_me_mask;
 
+void __init sme_early_encrypt(resource_size_t paddr,
+			      unsigned long size);
+void __init sme_early_decrypt(resource_size_t paddr,
+			      unsigned long size);
+
 void __init sme_early_init(void);
 
 void __init sme_enable(void);
@@ -29,6 +34,11 @@
 
 #define sme_me_mask	0UL
 
+static inline void __init sme_early_encrypt(resource_size_t paddr,
+					    unsigned long size) { }
+static inline void __init sme_early_decrypt(resource_size_t paddr,
+					    unsigned long size) { }
+
 static inline void __init sme_early_init(void) { }
 
 static inline void __init sme_enable(void) { }
diff --git a/arch/x86/mm/mem_encrypt.c b/arch/x86/mm/mem_encrypt.c
index b2d1cdf..b7671b9 100644
--- a/arch/x86/mm/mem_encrypt.c
+++ b/arch/x86/mm/mem_encrypt.c
@@ -17,6 +17,9 @@
 
 #include <linux/mm.h>
 
+#include <asm/tlbflush.h>
+#include <asm/fixmap.h>
+
 /*
  * Since SME related variables are set early in the boot process they must
  * reside in the .data section so as not to be zeroed out when the .bss
@@ -25,6 +28,79 @@
 unsigned long sme_me_mask __section(.data) = 0;
 EXPORT_SYMBOL_GPL(sme_me_mask);
 
+/* Buffer used for early in-place encryption by BSP, no locking needed */
+static char sme_early_buffer[PAGE_SIZE] __aligned(PAGE_SIZE);
+
+/*
+ * This routine does not change the underlying encryption setting of the
+ * page(s) that map this memory. It assumes that eventually the memory is
+ * meant to be accessed as either encrypted or decrypted but the contents
+ * are currently not in the desired state.
+ *
+ * This routine follows the steps outlined in the AMD64 Architecture
+ * Programmer's Manual Volume 2, Section 7.10.8 Encrypt-in-Place.
+ */
+static void __init __sme_early_enc_dec(resource_size_t paddr,
+				       unsigned long size, bool enc)
+{
+	void *src, *dst;
+	size_t len;
+
+	if (!sme_me_mask)
+		return;
+
+	local_flush_tlb();
+	wbinvd();
+
+	/*
+	 * There are limited number of early mapping slots, so map (at most)
+	 * one page at time.
+	 */
+	while (size) {
+		len = min_t(size_t, sizeof(sme_early_buffer), size);
+
+		/*
+		 * Create mappings for the current and desired format of
+		 * the memory. Use a write-protected mapping for the source.
+		 */
+		src = enc ? early_memremap_decrypted_wp(paddr, len) :
+			    early_memremap_encrypted_wp(paddr, len);
+
+		dst = enc ? early_memremap_encrypted(paddr, len) :
+			    early_memremap_decrypted(paddr, len);
+
+		/*
+		 * If a mapping can't be obtained to perform the operation,
+		 * then eventual access of that area in the desired mode
+		 * will cause a crash.
+		 */
+		BUG_ON(!src || !dst);
+
+		/*
+		 * Use a temporary buffer, of cache-line multiple size, to
+		 * avoid data corruption as documented in the APM.
+		 */
+		memcpy(sme_early_buffer, src, len);
+		memcpy(dst, sme_early_buffer, len);
+
+		early_memunmap(dst, len);
+		early_memunmap(src, len);
+
+		paddr += len;
+		size -= len;
+	}
+}
+
+void __init sme_early_encrypt(resource_size_t paddr, unsigned long size)
+{
+	__sme_early_enc_dec(paddr, size, true);
+}
+
+void __init sme_early_decrypt(resource_size_t paddr, unsigned long size)
+{
+	__sme_early_enc_dec(paddr, size, false);
+}
+
 void __init sme_early_init(void)
 {
 	unsigned int i;



From thomas.lendacky at amd.com  Fri Jun 16 11:52:32 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Fri, 16 Jun 2017 13:52:32 -0500
Subject: [PATCH v7 14/36] x86/mm: Insure that boot memory areas are mapped
 properly
In-Reply-To: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
References: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170616185232.18967.61753.stgit@tlendack-t1.amdoffice.net>

The boot data and command line data are present in memory in a decrypted
state and are copied early in the boot process.  The early page fault
support will map these areas as encrypted, so before attempting to copy
them, add decrypted mappings so the data is accessed properly when copied.

For the initrd, encrypt this data in place. Since the future mapping of
the initrd area will be mapped as encrypted the data will be accessed
properly.

Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/include/asm/mem_encrypt.h |    6 +++
 arch/x86/include/asm/pgtable.h     |    3 ++
 arch/x86/kernel/head64.c           |   30 +++++++++++++--
 arch/x86/kernel/setup.c            |    9 +++++
 arch/x86/mm/kasan_init_64.c        |    2 +
 arch/x86/mm/mem_encrypt.c          |   70 ++++++++++++++++++++++++++++++++++++
 6 files changed, 115 insertions(+), 5 deletions(-)

diff --git a/arch/x86/include/asm/mem_encrypt.h b/arch/x86/include/asm/mem_encrypt.h
index 6508ec9..3ffa5fa 100644
--- a/arch/x86/include/asm/mem_encrypt.h
+++ b/arch/x86/include/asm/mem_encrypt.h
@@ -26,6 +26,9 @@ void __init sme_early_encrypt(resource_size_t paddr,
 void __init sme_early_decrypt(resource_size_t paddr,
 			      unsigned long size);
 
+void __init sme_map_bootdata(char *real_mode_data);
+void __init sme_unmap_bootdata(char *real_mode_data);
+
 void __init sme_early_init(void);
 
 void __init sme_enable(void);
@@ -39,6 +42,9 @@ static inline void __init sme_early_encrypt(resource_size_t paddr,
 static inline void __init sme_early_decrypt(resource_size_t paddr,
 					    unsigned long size) { }
 
+static inline void __init sme_map_bootdata(char *real_mode_data) { }
+static inline void __init sme_unmap_bootdata(char *real_mode_data) { }
+
 static inline void __init sme_early_init(void) { }
 
 static inline void __init sme_enable(void) { }
diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h
index c6452cb..bbeae4a 100644
--- a/arch/x86/include/asm/pgtable.h
+++ b/arch/x86/include/asm/pgtable.h
@@ -23,6 +23,9 @@
 #ifndef __ASSEMBLY__
 #include <asm/x86_init.h>
 
+extern pgd_t early_top_pgt[PTRS_PER_PGD];
+int __init __early_make_pgtable(unsigned long address, pmdval_t pmd);
+
 void ptdump_walk_pgd_level(struct seq_file *m, pgd_t *pgd);
 void ptdump_walk_pgd_level_checkwx(void);
 
diff --git a/arch/x86/kernel/head64.c b/arch/x86/kernel/head64.c
index 5d7363a..9e94ed2 100644
--- a/arch/x86/kernel/head64.c
+++ b/arch/x86/kernel/head64.c
@@ -34,7 +34,6 @@
 /*
  * Manage page tables very early on.
  */
-extern pgd_t early_top_pgt[PTRS_PER_PGD];
 extern pmd_t early_dynamic_pgts[EARLY_DYNAMIC_PAGE_TABLES][PTRS_PER_PMD];
 static unsigned int __initdata next_early_pgt;
 pmdval_t early_pmd_flags = __PAGE_KERNEL_LARGE & ~(_PAGE_GLOBAL | _PAGE_NX);
@@ -161,13 +160,13 @@ static void __init reset_early_page_tables(void)
 }
 
 /* Create a new PMD entry */
-int __init early_make_pgtable(unsigned long address)
+int __init __early_make_pgtable(unsigned long address, pmdval_t pmd)
 {
 	unsigned long physaddr = address - __PAGE_OFFSET;
 	pgdval_t pgd, *pgd_p;
 	p4dval_t p4d, *p4d_p;
 	pudval_t pud, *pud_p;
-	pmdval_t pmd, *pmd_p;
+	pmdval_t *pmd_p;
 
 	/* Invalid address or early pgt is done ?  */
 	if (physaddr >= MAXMEM || read_cr3_pa() != __pa_nodebug(early_top_pgt))
@@ -226,12 +225,21 @@ int __init early_make_pgtable(unsigned long address)
 		memset(pmd_p, 0, sizeof(*pmd_p) * PTRS_PER_PMD);
 		*pud_p = (pudval_t)pmd_p - __START_KERNEL_map + phys_base + _KERNPG_TABLE;
 	}
-	pmd = (physaddr & PMD_MASK) + early_pmd_flags;
 	pmd_p[pmd_index(address)] = pmd;
 
 	return 0;
 }
 
+int __init early_make_pgtable(unsigned long address)
+{
+	unsigned long physaddr = address - __PAGE_OFFSET;
+	pmdval_t pmd;
+
+	pmd = (physaddr & PMD_MASK) + early_pmd_flags;
+
+	return __early_make_pgtable(address, pmd);
+}
+
 /* Don't add a printk in there. printk relies on the PDA which is not initialized 
    yet. */
 static void __init clear_bss(void)
@@ -254,6 +262,12 @@ static void __init copy_bootdata(char *real_mode_data)
 	char * command_line;
 	unsigned long cmd_line_ptr;
 
+	/*
+	 * If SME is active, this will create decrypted mappings of the
+	 * boot data in advance of the copy operations.
+	 */
+	sme_map_bootdata(real_mode_data);
+
 	memcpy(&boot_params, real_mode_data, sizeof boot_params);
 	sanitize_boot_params(&boot_params);
 	cmd_line_ptr = get_cmd_line_ptr();
@@ -261,6 +275,14 @@ static void __init copy_bootdata(char *real_mode_data)
 		command_line = __va(cmd_line_ptr);
 		memcpy(boot_command_line, command_line, COMMAND_LINE_SIZE);
 	}
+
+	/*
+	 * The old boot data is no longer needed and won't be reserved,
+	 * freeing up that memory for use by the system. If SME is active,
+	 * we need to remove the mappings that were created so that the
+	 * memory doesn't remain mapped as decrypted.
+	 */
+	sme_unmap_bootdata(real_mode_data);
 }
 
 asmlinkage __visible void __init x86_64_start_kernel(char * real_mode_data)
diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
index f818236..6a2f0a8 100644
--- a/arch/x86/kernel/setup.c
+++ b/arch/x86/kernel/setup.c
@@ -69,6 +69,7 @@
 #include <linux/crash_dump.h>
 #include <linux/tboot.h>
 #include <linux/jiffies.h>
+#include <linux/mem_encrypt.h>
 
 #include <linux/usb/xhci-dbgp.h>
 #include <video/edid.h>
@@ -374,6 +375,14 @@ static void __init reserve_initrd(void)
 	    !ramdisk_image || !ramdisk_size)
 		return;		/* No initrd provided by bootloader */
 
+	/*
+	 * If SME is active, this memory will be marked encrypted by the
+	 * kernel when it is accessed (including relocation). However, the
+	 * ramdisk image was loaded decrypted by the bootloader, so make
+	 * sure that it is encrypted before accessing it.
+	 */
+	sme_early_encrypt(ramdisk_image, ramdisk_end - ramdisk_image);
+
 	initrd_start = 0;
 
 	mapped_size = memblock_mem_size(max_pfn_mapped);
diff --git a/arch/x86/mm/kasan_init_64.c b/arch/x86/mm/kasan_init_64.c
index d7cc830..1b8791f 100644
--- a/arch/x86/mm/kasan_init_64.c
+++ b/arch/x86/mm/kasan_init_64.c
@@ -11,8 +11,8 @@
 #include <asm/e820/types.h>
 #include <asm/tlbflush.h>
 #include <asm/sections.h>
+#include <asm/pgtable.h>
 
-extern pgd_t early_top_pgt[PTRS_PER_PGD];
 extern struct range pfn_mapped[E820_MAX_ENTRIES];
 
 static int __init map_range(struct range *range)
diff --git a/arch/x86/mm/mem_encrypt.c b/arch/x86/mm/mem_encrypt.c
index b7671b9..ea5e3a6 100644
--- a/arch/x86/mm/mem_encrypt.c
+++ b/arch/x86/mm/mem_encrypt.c
@@ -19,6 +19,8 @@
 
 #include <asm/tlbflush.h>
 #include <asm/fixmap.h>
+#include <asm/setup.h>
+#include <asm/bootparam.h>
 
 /*
  * Since SME related variables are set early in the boot process they must
@@ -101,6 +103,74 @@ void __init sme_early_decrypt(resource_size_t paddr, unsigned long size)
 	__sme_early_enc_dec(paddr, size, false);
 }
 
+static void __init __sme_early_map_unmap_mem(void *vaddr, unsigned long size,
+					     bool map)
+{
+	unsigned long paddr = (unsigned long)vaddr - __PAGE_OFFSET;
+	pmdval_t pmd_flags, pmd;
+
+	/* Use early_pmd_flags but remove the encryption mask */
+	pmd_flags = __sme_clr(early_pmd_flags);
+
+	do {
+		pmd = map ? (paddr & PMD_MASK) + pmd_flags : 0;
+		__early_make_pgtable((unsigned long)vaddr, pmd);
+
+		vaddr += PMD_SIZE;
+		paddr += PMD_SIZE;
+		size = (size <= PMD_SIZE) ? 0 : size - PMD_SIZE;
+	} while (size);
+
+	write_cr3(__read_cr3());
+}
+
+static void __init __sme_map_unmap_bootdata(char *real_mode_data, bool map)
+{
+	struct boot_params *boot_data;
+	unsigned long cmdline_paddr;
+
+	/* If SME is not active, the bootdata is in the correct state */
+	if (!sme_active())
+		return;
+
+	if (!map) {
+		/*
+		 * If unmapping, get the command line address before
+		 * unmapping the real_mode_data.
+		 */
+		boot_data = (struct boot_params *)real_mode_data;
+		cmdline_paddr = boot_data->hdr.cmd_line_ptr |
+				((u64)boot_data->ext_cmd_line_ptr << 32);
+	}
+
+	__sme_early_map_unmap_mem(real_mode_data, sizeof(boot_params), map);
+
+	if (map) {
+		/*
+		 * If mapping, get the command line address after mapping
+		 * the real_mode_data.
+		 */
+		boot_data = (struct boot_params *)real_mode_data;
+		cmdline_paddr = boot_data->hdr.cmd_line_ptr |
+				((u64)boot_data->ext_cmd_line_ptr << 32);
+	}
+
+	if (!cmdline_paddr)
+		return;
+
+	__sme_early_map_unmap_mem(__va(cmdline_paddr), COMMAND_LINE_SIZE, map);
+}
+
+void __init sme_unmap_bootdata(char *real_mode_data)
+{
+	__sme_map_unmap_bootdata(real_mode_data, false);
+}
+
+void __init sme_map_bootdata(char *real_mode_data)
+{
+	__sme_map_unmap_bootdata(real_mode_data, true);
+}
+
 void __init sme_early_init(void)
 {
 	unsigned int i;



From thomas.lendacky at amd.com  Fri Jun 16 11:52:43 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Fri, 16 Jun 2017 13:52:43 -0500
Subject: [PATCH v7 15/36] x86/boot/e820: Add support to determine the E820
 type of an address
In-Reply-To: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
References: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170616185243.18967.44475.stgit@tlendack-t1.amdoffice.net>

Add a function that will return the E820 type associated with an address
range.

Reviewed-by: Borislav Petkov <bp at suse.de>
Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/include/asm/e820/api.h |    2 ++
 arch/x86/kernel/e820.c          |   26 +++++++++++++++++++++++---
 2 files changed, 25 insertions(+), 3 deletions(-)

diff --git a/arch/x86/include/asm/e820/api.h b/arch/x86/include/asm/e820/api.h
index 8e0f8b8..3641f5f 100644
--- a/arch/x86/include/asm/e820/api.h
+++ b/arch/x86/include/asm/e820/api.h
@@ -38,6 +38,8 @@
 extern void e820__reallocate_tables(void);
 extern void e820__register_nosave_regions(unsigned long limit_pfn);
 
+extern int  e820__get_entry_type(u64 start, u64 end);
+
 /*
  * Returns true iff the specified range [start,end) is completely contained inside
  * the ISA region.
diff --git a/arch/x86/kernel/e820.c b/arch/x86/kernel/e820.c
index d78a586..46c9b65 100644
--- a/arch/x86/kernel/e820.c
+++ b/arch/x86/kernel/e820.c
@@ -84,7 +84,8 @@ bool e820__mapped_any(u64 start, u64 end, enum e820_type type)
  * Note: this function only works correctly once the E820 table is sorted and
  * not-overlapping (at least for the range specified), which is the case normally.
  */
-bool __init e820__mapped_all(u64 start, u64 end, enum e820_type type)
+static struct e820_entry *__e820__mapped_all(u64 start, u64 end,
+					     enum e820_type type)
 {
 	int i;
 
@@ -110,9 +111,28 @@ bool __init e820__mapped_all(u64 start, u64 end, enum e820_type type)
 		 * coverage of the desired range exists:
 		 */
 		if (start >= end)
-			return 1;
+			return entry;
 	}
-	return 0;
+
+	return NULL;
+}
+
+/*
+ * This function checks if the entire range <start,end> is mapped with type.
+ */
+bool __init e820__mapped_all(u64 start, u64 end, enum e820_type type)
+{
+	return __e820__mapped_all(start, end, type);
+}
+
+/*
+ * This function returns the type associated with the range <start,end>.
+ */
+int e820__get_entry_type(u64 start, u64 end)
+{
+	struct e820_entry *entry = __e820__mapped_all(start, end, 0);
+
+	return entry ? entry->type : -EINVAL;
 }
 
 /*



From thomas.lendacky at amd.com  Fri Jun 16 11:53:06 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Fri, 16 Jun 2017 13:53:06 -0500
Subject: [PATCH v7 17/36] efi: Update efi_mem_type() to return an error
 rather than 0
In-Reply-To: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
References: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170616185306.18967.8964.stgit@tlendack-t1.amdoffice.net>

The efi_mem_type() function currently returns a 0, which maps to
EFI_RESERVED_TYPE, if the function is unable to find a memmap entry for
the supplied physical address. Returning EFI_RESERVED_TYPE implies that
a memmap entry exists, when it doesn't.  Instead of returning 0, change
the function to return a negative error value when no memmap entry is
found.

Reviewed-by: Borislav Petkov <bp at suse.de>
Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/ia64/kernel/efi.c      |    4 ++--
 arch/x86/platform/efi/efi.c |    6 +++---
 include/linux/efi.h         |    2 +-
 3 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/arch/ia64/kernel/efi.c b/arch/ia64/kernel/efi.c
index 1212956..8141600 100644
--- a/arch/ia64/kernel/efi.c
+++ b/arch/ia64/kernel/efi.c
@@ -757,14 +757,14 @@ static void __init handle_palo(unsigned long phys_addr)
 	return 0;
 }
 
-u32
+int
 efi_mem_type (unsigned long phys_addr)
 {
 	efi_memory_desc_t *md = efi_memory_descriptor(phys_addr);
 
 	if (md)
 		return md->type;
-	return 0;
+	return -EINVAL;
 }
 
 u64
diff --git a/arch/x86/platform/efi/efi.c b/arch/x86/platform/efi/efi.c
index f084d87..6217b23 100644
--- a/arch/x86/platform/efi/efi.c
+++ b/arch/x86/platform/efi/efi.c
@@ -1035,12 +1035,12 @@ void __init efi_enter_virtual_mode(void)
 /*
  * Convenience functions to obtain memory types and attributes
  */
-u32 efi_mem_type(unsigned long phys_addr)
+int efi_mem_type(unsigned long phys_addr)
 {
 	efi_memory_desc_t *md;
 
 	if (!efi_enabled(EFI_MEMMAP))
-		return 0;
+		return -ENOTSUPP;
 
 	for_each_efi_memory_desc(md) {
 		if ((md->phys_addr <= phys_addr) &&
@@ -1048,7 +1048,7 @@ u32 efi_mem_type(unsigned long phys_addr)
 				  (md->num_pages << EFI_PAGE_SHIFT))))
 			return md->type;
 	}
-	return 0;
+	return -EINVAL;
 }
 
 static int __init arch_parse_efi_cmdline(char *str)
diff --git a/include/linux/efi.h b/include/linux/efi.h
index 8e24f09..4e47f78 100644
--- a/include/linux/efi.h
+++ b/include/linux/efi.h
@@ -985,7 +985,7 @@ static inline void efi_esrt_init(void) { }
 extern int efi_config_parse_tables(void *config_tables, int count, int sz,
 				   efi_config_table_type_t *arch_tables);
 extern u64 efi_get_iobase (void);
-extern u32 efi_mem_type (unsigned long phys_addr);
+extern int efi_mem_type(unsigned long phys_addr);
 extern u64 efi_mem_attributes (unsigned long phys_addr);
 extern u64 efi_mem_attribute (unsigned long phys_addr, unsigned long size);
 extern int __init efi_uart_console_only (void);



From thomas.lendacky at amd.com  Fri Jun 16 11:52:53 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Fri, 16 Jun 2017 13:52:53 -0500
Subject: [PATCH v7 16/36] efi: Add an EFI table address match function
In-Reply-To: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
References: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170616185253.18967.55724.stgit@tlendack-t1.amdoffice.net>

Add a function that will determine if a supplied physical address matches
the address of an EFI table.

Reviewed-by: Borislav Petkov <bp at suse.de>
Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 drivers/firmware/efi/efi.c |   33 +++++++++++++++++++++++++++++++++
 include/linux/efi.h        |    7 +++++++
 2 files changed, 40 insertions(+)

diff --git a/drivers/firmware/efi/efi.c b/drivers/firmware/efi/efi.c
index b372aad..983675d 100644
--- a/drivers/firmware/efi/efi.c
+++ b/drivers/firmware/efi/efi.c
@@ -55,6 +55,25 @@ struct efi __read_mostly efi = {
 };
 EXPORT_SYMBOL(efi);
 
+static unsigned long *efi_tables[] = {
+	&efi.mps,
+	&efi.acpi,
+	&efi.acpi20,
+	&efi.smbios,
+	&efi.smbios3,
+	&efi.sal_systab,
+	&efi.boot_info,
+	&efi.hcdp,
+	&efi.uga,
+	&efi.uv_systab,
+	&efi.fw_vendor,
+	&efi.runtime,
+	&efi.config_table,
+	&efi.esrt,
+	&efi.properties_table,
+	&efi.mem_attr_table,
+};
+
 static bool disable_runtime;
 static int __init setup_noefi(char *arg)
 {
@@ -854,6 +873,20 @@ int efi_status_to_err(efi_status_t status)
 	return err;
 }
 
+bool efi_is_table_address(unsigned long phys_addr)
+{
+	unsigned int i;
+
+	if (phys_addr == EFI_INVALID_TABLE_ADDR)
+		return false;
+
+	for (i = 0; i < ARRAY_SIZE(efi_tables); i++)
+		if (*(efi_tables[i]) == phys_addr)
+			return true;
+
+	return false;
+}
+
 #ifdef CONFIG_KEXEC
 static int update_efi_random_seed(struct notifier_block *nb,
 				  unsigned long code, void *unused)
diff --git a/include/linux/efi.h b/include/linux/efi.h
index 8269bcb..8e24f09 100644
--- a/include/linux/efi.h
+++ b/include/linux/efi.h
@@ -1091,6 +1091,8 @@ static inline bool efi_enabled(int feature)
 	return test_bit(feature, &efi.flags) != 0;
 }
 extern void efi_reboot(enum reboot_mode reboot_mode, const char *__unused);
+
+extern bool efi_is_table_address(unsigned long phys_addr);
 #else
 static inline bool efi_enabled(int feature)
 {
@@ -1104,6 +1106,11 @@ static inline bool efi_enabled(int feature)
 {
 	return false;
 }
+
+static inline bool efi_is_table_address(unsigned long phys_addr)
+{
+	return false;
+}
 #endif
 
 extern int efi_status_to_err(efi_status_t status);



From thomas.lendacky at amd.com  Fri Jun 16 11:53:17 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Fri, 16 Jun 2017 13:53:17 -0500
Subject: [PATCH v7 18/36] x86/efi: Update EFI pagetable creation to work
 with SME
In-Reply-To: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
References: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170616185317.18967.99499.stgit@tlendack-t1.amdoffice.net>

When SME is active, pagetable entries created for EFI need to have the
encryption mask set as necessary.

When the new pagetable pages are allocated they are mapped encrypted. So,
update the efi_pgt value that will be used in cr3 to include the encryption
mask so that the PGD table can be read successfully. The pagetable mapping
as well as the kernel are also added to the pagetable mapping as encrypted.
All other EFI mappings are mapped decrypted (tables, etc.).

Reviewed-by: Borislav Petkov <bp at suse.de>
Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/platform/efi/efi_64.c |   15 +++++++++++----
 1 file changed, 11 insertions(+), 4 deletions(-)

diff --git a/arch/x86/platform/efi/efi_64.c b/arch/x86/platform/efi/efi_64.c
index 9bf72f5..12e8388 100644
--- a/arch/x86/platform/efi/efi_64.c
+++ b/arch/x86/platform/efi/efi_64.c
@@ -327,7 +327,7 @@ void efi_sync_low_kernel_mappings(void)
 
 int __init efi_setup_page_tables(unsigned long pa_memmap, unsigned num_pages)
 {
-	unsigned long pfn, text;
+	unsigned long pfn, text, pf;
 	struct page *page;
 	unsigned npages;
 	pgd_t *pgd;
@@ -335,7 +335,12 @@ int __init efi_setup_page_tables(unsigned long pa_memmap, unsigned num_pages)
 	if (efi_enabled(EFI_OLD_MEMMAP))
 		return 0;
 
-	efi_scratch.efi_pgt = (pgd_t *)__pa(efi_pgd);
+	/*
+	 * Since the PGD is encrypted, set the encryption mask so that when
+	 * this value is loaded into cr3 the PGD will be decrypted during
+	 * the pagetable walk.
+	 */
+	efi_scratch.efi_pgt = (pgd_t *)__sme_pa(efi_pgd);
 	pgd = efi_pgd;
 
 	/*
@@ -345,7 +350,8 @@ int __init efi_setup_page_tables(unsigned long pa_memmap, unsigned num_pages)
 	 * phys_efi_set_virtual_address_map().
 	 */
 	pfn = pa_memmap >> PAGE_SHIFT;
-	if (kernel_map_pages_in_pgd(pgd, pfn, pa_memmap, num_pages, _PAGE_NX | _PAGE_RW)) {
+	pf = _PAGE_NX | _PAGE_RW | _PAGE_ENC;
+	if (kernel_map_pages_in_pgd(pgd, pfn, pa_memmap, num_pages, pf)) {
 		pr_err("Error ident-mapping new memmap (0x%lx)!\n", pa_memmap);
 		return 1;
 	}
@@ -388,7 +394,8 @@ int __init efi_setup_page_tables(unsigned long pa_memmap, unsigned num_pages)
 	text = __pa(_text);
 	pfn = text >> PAGE_SHIFT;
 
-	if (kernel_map_pages_in_pgd(pgd, pfn, text, npages, _PAGE_RW)) {
+	pf = _PAGE_RW | _PAGE_ENC;
+	if (kernel_map_pages_in_pgd(pgd, pfn, text, npages, pf)) {
 		pr_err("Failed to map kernel text 1:1\n");
 		return 1;
 	}



From thomas.lendacky at amd.com  Fri Jun 16 11:53:26 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Fri, 16 Jun 2017 13:53:26 -0500
Subject: [PATCH v7 19/36] x86/mm: Add support to access boot related data in
 the clear
In-Reply-To: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
References: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170616185326.18967.43278.stgit@tlendack-t1.amdoffice.net>

Boot data (such as EFI related data) is not encrypted when the system is
booted because UEFI/BIOS does not run with SME active. In order to access
this data properly it needs to be mapped decrypted.

Update early_memremap() to provide an arch specific routine to modify the
pagetable protection attributes before they are applied to the new
mapping. This is used to remove the encryption mask for boot related data.

Update memremap() to provide an arch specific routine to determine if RAM
remapping is allowed.  RAM remapping will cause an encrypted mapping to be
generated. By preventing RAM remapping, ioremap_cache() will be used
instead, which will provide a decrypted mapping of the boot related data.

Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/include/asm/io.h |    5 +
 arch/x86/mm/ioremap.c     |  179 +++++++++++++++++++++++++++++++++++++++++++++
 include/linux/io.h        |    2 +
 kernel/memremap.c         |   20 ++++-
 mm/early_ioremap.c        |   18 ++++-
 5 files changed, 217 insertions(+), 7 deletions(-)

diff --git a/arch/x86/include/asm/io.h b/arch/x86/include/asm/io.h
index 7afb0e2..09c5557 100644
--- a/arch/x86/include/asm/io.h
+++ b/arch/x86/include/asm/io.h
@@ -381,4 +381,9 @@ extern int __must_check arch_phys_wc_add(unsigned long base,
 #define arch_io_reserve_memtype_wc arch_io_reserve_memtype_wc
 #endif
 
+extern bool arch_memremap_can_ram_remap(resource_size_t offset,
+					unsigned long size,
+					unsigned long flags);
+#define arch_memremap_can_ram_remap arch_memremap_can_ram_remap
+
 #endif /* _ASM_X86_IO_H */
diff --git a/arch/x86/mm/ioremap.c b/arch/x86/mm/ioremap.c
index 4feda83..f3fa007 100644
--- a/arch/x86/mm/ioremap.c
+++ b/arch/x86/mm/ioremap.c
@@ -14,6 +14,7 @@
 #include <linux/vmalloc.h>
 #include <linux/mmiotrace.h>
 #include <linux/mem_encrypt.h>
+#include <linux/efi.h>
 
 #include <asm/set_memory.h>
 #include <asm/e820/api.h>
@@ -22,6 +23,7 @@
 #include <asm/tlbflush.h>
 #include <asm/pgalloc.h>
 #include <asm/pat.h>
+#include <asm/setup.h>
 
 #include "physaddr.h"
 
@@ -422,6 +424,183 @@ void unxlate_dev_mem_ptr(phys_addr_t phys, void *addr)
 	iounmap((void __iomem *)((unsigned long)addr & PAGE_MASK));
 }
 
+/*
+ * Examine the physical address to determine if it is an area of memory
+ * that should be mapped decrypted.  If the memory is not part of the
+ * kernel usable area it was accessed and created decrypted, so these
+ * areas should be mapped decrypted.
+ */
+static bool memremap_should_map_decrypted(resource_size_t phys_addr,
+					  unsigned long size)
+{
+	/* Check if the address is outside kernel usable area */
+	switch (e820__get_entry_type(phys_addr, phys_addr + size - 1)) {
+	case E820_TYPE_RESERVED:
+	case E820_TYPE_ACPI:
+	case E820_TYPE_NVS:
+	case E820_TYPE_UNUSABLE:
+		return true;
+	default:
+		break;
+	}
+
+	return false;
+}
+
+/*
+ * Examine the physical address to determine if it is EFI data. Check
+ * it against the boot params structure and EFI tables and memory types.
+ */
+static bool memremap_is_efi_data(resource_size_t phys_addr,
+				 unsigned long size)
+{
+	u64 paddr;
+
+	/* Check if the address is part of EFI boot/runtime data */
+	if (!efi_enabled(EFI_BOOT))
+		return false;
+
+	paddr = boot_params.efi_info.efi_memmap_hi;
+	paddr <<= 32;
+	paddr |= boot_params.efi_info.efi_memmap;
+	if (phys_addr == paddr)
+		return true;
+
+	paddr = boot_params.efi_info.efi_systab_hi;
+	paddr <<= 32;
+	paddr |= boot_params.efi_info.efi_systab;
+	if (phys_addr == paddr)
+		return true;
+
+	if (efi_is_table_address(phys_addr))
+		return true;
+
+	switch (efi_mem_type(phys_addr)) {
+	case EFI_BOOT_SERVICES_DATA:
+	case EFI_RUNTIME_SERVICES_DATA:
+		return true;
+	default:
+		break;
+	}
+
+	return false;
+}
+
+/*
+ * Examine the physical address to determine if it is boot data by checking
+ * it against the boot params setup_data chain.
+ */
+static bool memremap_is_setup_data(resource_size_t phys_addr,
+				   unsigned long size)
+{
+	struct setup_data *data;
+	u64 paddr, paddr_next;
+
+	paddr = boot_params.hdr.setup_data;
+	while (paddr) {
+		unsigned int len;
+
+		if (phys_addr == paddr)
+			return true;
+
+		data = memremap(paddr, sizeof(*data),
+				MEMREMAP_WB | MEMREMAP_DEC);
+
+		paddr_next = data->next;
+		len = data->len;
+
+		memunmap(data);
+
+		if ((phys_addr > paddr) && (phys_addr < (paddr + len)))
+			return true;
+
+		paddr = paddr_next;
+	}
+
+	return false;
+}
+
+/*
+ * Examine the physical address to determine if it is boot data by checking
+ * it against the boot params setup_data chain (early boot version).
+ */
+static bool __init early_memremap_is_setup_data(resource_size_t phys_addr,
+						unsigned long size)
+{
+	struct setup_data *data;
+	u64 paddr, paddr_next;
+
+	paddr = boot_params.hdr.setup_data;
+	while (paddr) {
+		unsigned int len;
+
+		if (phys_addr == paddr)
+			return true;
+
+		data = early_memremap_decrypted(paddr, sizeof(*data));
+
+		paddr_next = data->next;
+		len = data->len;
+
+		early_memunmap(data, sizeof(*data));
+
+		if ((phys_addr > paddr) && (phys_addr < (paddr + len)))
+			return true;
+
+		paddr = paddr_next;
+	}
+
+	return false;
+}
+
+/*
+ * Architecture function to determine if RAM remap is allowed. By default, a
+ * RAM remap will map the data as encrypted. Determine if a RAM remap should
+ * not be done so that the data will be mapped decrypted.
+ */
+bool arch_memremap_can_ram_remap(resource_size_t phys_addr, unsigned long size,
+				 unsigned long flags)
+{
+	if (!sme_active())
+		return true;
+
+	if (flags & MEMREMAP_ENC)
+		return true;
+
+	if (flags & MEMREMAP_DEC)
+		return false;
+
+	if (memremap_is_setup_data(phys_addr, size) ||
+	    memremap_is_efi_data(phys_addr, size) ||
+	    memremap_should_map_decrypted(phys_addr, size))
+		return false;
+
+	return true;
+}
+
+/*
+ * Architecture override of __weak function to adjust the protection attributes
+ * used when remapping memory. By default, early_memremap() will map the data
+ * as encrypted. Determine if an encrypted mapping should not be done and set
+ * the appropriate protection attributes.
+ */
+pgprot_t __init early_memremap_pgprot_adjust(resource_size_t phys_addr,
+					     unsigned long size,
+					     pgprot_t prot)
+{
+	if (!sme_active())
+		return prot;
+
+	if (early_memremap_is_setup_data(phys_addr, size) ||
+	    memremap_is_efi_data(phys_addr, size) ||
+	    memremap_should_map_decrypted(phys_addr, size))
+		prot = pgprot_decrypted(prot);
+	else
+		prot = pgprot_encrypted(prot);
+
+	return prot;
+}
+
 #ifdef CONFIG_ARCH_USE_MEMREMAP_PROT
 /* Remap memory with encryption */
 void __init *early_memremap_encrypted(resource_size_t phys_addr,
diff --git a/include/linux/io.h b/include/linux/io.h
index 2195d9e..32e30e8 100644
--- a/include/linux/io.h
+++ b/include/linux/io.h
@@ -157,6 +157,8 @@ enum {
 	MEMREMAP_WB = 1 << 0,
 	MEMREMAP_WT = 1 << 1,
 	MEMREMAP_WC = 1 << 2,
+	MEMREMAP_ENC = 1 << 3,
+	MEMREMAP_DEC = 1 << 4,
 };
 
 void *memremap(resource_size_t offset, size_t size, unsigned long flags);
diff --git a/kernel/memremap.c b/kernel/memremap.c
index 23a6483..9b4fbd5 100644
--- a/kernel/memremap.c
+++ b/kernel/memremap.c
@@ -34,13 +34,24 @@ static void *arch_memremap_wb(resource_size_t offset, unsigned long size)
 }
 #endif
 
-static void *try_ram_remap(resource_size_t offset, size_t size)
+#ifndef arch_memremap_can_ram_remap
+static bool arch_memremap_can_ram_remap(resource_size_t offset, size_t size,
+					unsigned long flags)
+{
+	return true;
+}
+#endif
+
+static void *try_ram_remap(resource_size_t offset, size_t size,
+			   unsigned long flags)
 {
 	unsigned long pfn = PHYS_PFN(offset);
 
 	/* In the simple case just return the existing linear address */
-	if (pfn_valid(pfn) && !PageHighMem(pfn_to_page(pfn)))
+	if (pfn_valid(pfn) && !PageHighMem(pfn_to_page(pfn)) &&
+	    arch_memremap_can_ram_remap(offset, size, flags))
 		return __va(offset);
+
 	return NULL; /* fallback to arch_memremap_wb */
 }
 
@@ -48,7 +59,8 @@ static void *try_ram_remap(resource_size_t offset, size_t size)
  * memremap() - remap an iomem_resource as cacheable memory
  * @offset: iomem resource start address
  * @size: size of remap
- * @flags: any of MEMREMAP_WB, MEMREMAP_WT and MEMREMAP_WC
+ * @flags: any of MEMREMAP_WB, MEMREMAP_WT, MEMREMAP_WC,
+ *		  MEMREMAP_ENC, MEMREMAP_DEC
  *
  * memremap() is "ioremap" for cases where it is known that the resource
  * being mapped does not have i/o side effects and the __iomem
@@ -95,7 +107,7 @@ void *memremap(resource_size_t offset, size_t size, unsigned long flags)
 		 * the requested range is potentially in System RAM.
 		 */
 		if (is_ram == REGION_INTERSECTS)
-			addr = try_ram_remap(offset, size);
+			addr = try_ram_remap(offset, size, flags);
 		if (!addr)
 			addr = arch_memremap_wb(offset, size);
 	}
diff --git a/mm/early_ioremap.c b/mm/early_ioremap.c
index d7d30da..b1dd4a9 100644
--- a/mm/early_ioremap.c
+++ b/mm/early_ioremap.c
@@ -30,6 +30,13 @@ static int __init early_ioremap_debug_setup(char *str)
 
 static int after_paging_init __initdata;
 
+pgprot_t __init __weak early_memremap_pgprot_adjust(resource_size_t phys_addr,
+						    unsigned long size,
+						    pgprot_t prot)
+{
+	return prot;
+}
+
 void __init __weak early_ioremap_shutdown(void)
 {
 }
@@ -215,14 +222,19 @@ void __init early_iounmap(void __iomem *addr, unsigned long size)
 void __init *
 early_memremap(resource_size_t phys_addr, unsigned long size)
 {
-	return (__force void *)__early_ioremap(phys_addr, size,
-					       FIXMAP_PAGE_NORMAL);
+	pgprot_t prot = early_memremap_pgprot_adjust(phys_addr, size,
+						     FIXMAP_PAGE_NORMAL);
+
+	return (__force void *)__early_ioremap(phys_addr, size, prot);
 }
 #ifdef FIXMAP_PAGE_RO
 void __init *
 early_memremap_ro(resource_size_t phys_addr, unsigned long size)
 {
-	return (__force void *)__early_ioremap(phys_addr, size, FIXMAP_PAGE_RO);
+	pgprot_t prot = early_memremap_pgprot_adjust(phys_addr, size,
+						     FIXMAP_PAGE_RO);
+
+	return (__force void *)__early_ioremap(phys_addr, size, prot);
 }
 #endif
 



From thomas.lendacky at amd.com  Fri Jun 16 11:53:38 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Fri, 16 Jun 2017 13:53:38 -0500
Subject: [PATCH v7 20/36] x86,
 mpparse: Use memremap to map the mpf and mpc data
In-Reply-To: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
References: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170616185338.18967.80659.stgit@tlendack-t1.amdoffice.net>

The SMP MP-table is built by UEFI and placed in memory in a decrypted
state. These tables are accessed using a mix of early_memremap(),
early_memunmap(), phys_to_virt() and virt_to_phys(). Change all accesses
to use early_memremap()/early_memunmap(). This allows for proper setting
of the encryption mask so that the data can be successfully accessed when
SME is active.

Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/kernel/mpparse.c |   98 ++++++++++++++++++++++++++++++++-------------
 1 file changed, 70 insertions(+), 28 deletions(-)

diff --git a/arch/x86/kernel/mpparse.c b/arch/x86/kernel/mpparse.c
index fd37f39..5cbb317 100644
--- a/arch/x86/kernel/mpparse.c
+++ b/arch/x86/kernel/mpparse.c
@@ -429,7 +429,7 @@ static inline void __init construct_default_ISA_mptable(int mpc_default_type)
 	}
 }
 
-static struct mpf_intel *mpf_found;
+static unsigned long mpf_base;
 
 static unsigned long __init get_mpc_size(unsigned long physptr)
 {
@@ -451,6 +451,7 @@ static int __init check_physptr(struct mpf_intel *mpf, unsigned int early)
 
 	size = get_mpc_size(mpf->physptr);
 	mpc = early_memremap(mpf->physptr, size);
+
 	/*
 	 * Read the physical hardware table.  Anything here will
 	 * override the defaults.
@@ -497,12 +498,12 @@ static int __init check_physptr(struct mpf_intel *mpf, unsigned int early)
  */
 void __init default_get_smp_config(unsigned int early)
 {
-	struct mpf_intel *mpf = mpf_found;
+	struct mpf_intel *mpf;
 
 	if (!smp_found_config)
 		return;
 
-	if (!mpf)
+	if (!mpf_base)
 		return;
 
 	if (acpi_lapic && early)
@@ -515,6 +516,12 @@ void __init default_get_smp_config(unsigned int early)
 	if (acpi_lapic && acpi_ioapic)
 		return;
 
+	mpf = early_memremap(mpf_base, sizeof(*mpf));
+	if (!mpf) {
+		pr_err("MPTABLE: error mapping MP table\n");
+		return;
+	}
+
 	pr_info("Intel MultiProcessor Specification v1.%d\n",
 		mpf->specification);
 #if defined(CONFIG_X86_LOCAL_APIC) && defined(CONFIG_X86_32)
@@ -529,7 +536,7 @@ void __init default_get_smp_config(unsigned int early)
 	/*
 	 * Now see if we need to read further.
 	 */
-	if (mpf->feature1 != 0) {
+	if (mpf->feature1) {
 		if (early) {
 			/*
 			 * local APIC has default address
@@ -542,8 +549,10 @@ void __init default_get_smp_config(unsigned int early)
 		construct_default_ISA_mptable(mpf->feature1);
 
 	} else if (mpf->physptr) {
-		if (check_physptr(mpf, early))
+		if (check_physptr(mpf, early)) {
+			early_memunmap(mpf, sizeof(*mpf));
 			return;
+		}
 	} else
 		BUG();
 
@@ -552,6 +561,8 @@ void __init default_get_smp_config(unsigned int early)
 	/*
 	 * Only use the first configuration found.
 	 */
+
+	early_memunmap(mpf, sizeof(*mpf));
 }
 
 static void __init smp_reserve_memory(struct mpf_intel *mpf)
@@ -561,15 +572,16 @@ static void __init smp_reserve_memory(struct mpf_intel *mpf)
 
 static int __init smp_scan_config(unsigned long base, unsigned long length)
 {
-	unsigned int *bp = phys_to_virt(base);
+	unsigned int *bp;
 	struct mpf_intel *mpf;
-	unsigned long mem;
+	int ret = 0;
 
 	apic_printk(APIC_VERBOSE, "Scan for SMP in [mem %#010lx-%#010lx]\n",
 		    base, base + length - 1);
 	BUILD_BUG_ON(sizeof(*mpf) != 16);
 
 	while (length > 0) {
+		bp = early_memremap(base, length);
 		mpf = (struct mpf_intel *)bp;
 		if ((*bp == SMP_MAGIC_IDENT) &&
 		    (mpf->length == 1) &&
@@ -579,24 +591,26 @@ static int __init smp_scan_config(unsigned long base, unsigned long length)
 #ifdef CONFIG_X86_LOCAL_APIC
 			smp_found_config = 1;
 #endif
-			mpf_found = mpf;
+			mpf_base = base;
 
-			pr_info("found SMP MP-table at [mem %#010llx-%#010llx] mapped at [%p]\n",
-				(unsigned long long) virt_to_phys(mpf),
-				(unsigned long long) virt_to_phys(mpf) +
-				sizeof(*mpf) - 1, mpf);
+			pr_info("found SMP MP-table at [mem %#010lx-%#010lx] mapped at [%p]\n",
+				base, base + sizeof(*mpf) - 1, mpf);
 
-			mem = virt_to_phys(mpf);
-			memblock_reserve(mem, sizeof(*mpf));
+			memblock_reserve(base, sizeof(*mpf));
 			if (mpf->physptr)
 				smp_reserve_memory(mpf);
 
-			return 1;
+			ret = 1;
 		}
-		bp += 4;
+		early_memunmap(bp, length);
+
+		if (ret)
+			break;
+
+		base += 16;
 		length -= 16;
 	}
-	return 0;
+	return ret;
 }
 
 void __init default_find_smp_config(void)
@@ -838,29 +852,40 @@ static int __init update_mp_table(void)
 	char oem[10];
 	struct mpf_intel *mpf;
 	struct mpc_table *mpc, *mpc_new;
+	unsigned long size;
 
 	if (!enable_update_mptable)
 		return 0;
 
-	mpf = mpf_found;
-	if (!mpf)
+	if (!mpf_base)
+		return 0;
+
+	mpf = early_memremap(mpf_base, sizeof(*mpf));
+	if (!mpf) {
+		pr_err("MPTABLE: mpf early_memremap() failed\n");
 		return 0;
+	}
 
 	/*
 	 * Now see if we need to go further.
 	 */
-	if (mpf->feature1 != 0)
-		return 0;
+	if (mpf->feature1)
+		goto do_unmap_mpf;
 
 	if (!mpf->physptr)
-		return 0;
+		goto do_unmap_mpf;
 
-	mpc = phys_to_virt(mpf->physptr);
+	size = get_mpc_size(mpf->physptr);
+	mpc = early_memremap(mpf->physptr, size);
+	if (!mpc) {
+		pr_err("MPTABLE: mpc early_memremap() failed\n");
+		goto do_unmap_mpf;
+	}
 
 	if (!smp_check_mpc(mpc, oem, str))
-		return 0;
+		goto do_unmap_mpc;
 
-	pr_info("mpf: %llx\n", (u64)virt_to_phys(mpf));
+	pr_info("mpf: %llx\n", (u64)mpf_base);
 	pr_info("physptr: %x\n", mpf->physptr);
 
 	if (mpc_new_phys && mpc->length > mpc_new_length) {
@@ -878,21 +903,32 @@ static int __init update_mp_table(void)
 		new = mpf_checksum((unsigned char *)mpc, mpc->length);
 		if (old == new) {
 			pr_info("mpc is readonly, please try alloc_mptable instead\n");
-			return 0;
+			goto do_unmap_mpc;
 		}
 		pr_info("use in-position replacing\n");
 	} else {
+		mpc_new = early_memremap(mpc_new_phys, mpc_new_length);
+		if (!mpc_new) {
+			pr_err("MPTABLE: new mpc early_memremap() failed\n");
+			goto do_unmap_mpc;
+		}
 		mpf->physptr = mpc_new_phys;
-		mpc_new = phys_to_virt(mpc_new_phys);
 		memcpy(mpc_new, mpc, mpc->length);
+		early_memunmap(mpc, size);
 		mpc = mpc_new;
+		size = mpc_new_length;
 		/* check if we can modify that */
 		if (mpc_new_phys - mpf->physptr) {
 			struct mpf_intel *mpf_new;
 			/* steal 16 bytes from [0, 1k) */
+			mpf_new = early_memremap(0x400 - 16, sizeof(*mpf_new));
+			if (!mpf_new) {
+				pr_err("MPTABLE: new mpf early_memremap() failed\n");
+				goto do_unmap_mpc;
+			}
 			pr_info("mpf new: %x\n", 0x400 - 16);
-			mpf_new = phys_to_virt(0x400 - 16);
 			memcpy(mpf_new, mpf, 16);
+			early_memunmap(mpf, sizeof(*mpf));
 			mpf = mpf_new;
 			mpf->physptr = mpc_new_phys;
 		}
@@ -909,6 +945,12 @@ static int __init update_mp_table(void)
 	 */
 	replace_intsrc_all(mpc, mpc_new_phys, mpc_new_length);
 
+do_unmap_mpc:
+	early_memunmap(mpc, size);
+
+do_unmap_mpf:
+	early_memunmap(mpf, sizeof(*mpf));
+
 	return 0;
 }
 



From thomas.lendacky at amd.com  Fri Jun 16 11:53:49 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Fri, 16 Jun 2017 13:53:49 -0500
Subject: [PATCH v7 21/36] x86/mm: Add support to access persistent memory in
 the clear
In-Reply-To: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
References: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170616185349.18967.38054.stgit@tlendack-t1.amdoffice.net>

Persistent memory is expected to persist across reboots. The encryption
key used by SME will change across reboots which will result in corrupted
persistent memory.  Persistent memory is handed out by block devices
through memory remapping functions, so be sure not to map this memory as
encrypted.

Reviewed-by: Borislav Petkov <bp at suse.de>
Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/mm/ioremap.c |   31 ++++++++++++++++++++++++++++++-
 1 file changed, 30 insertions(+), 1 deletion(-)

diff --git a/arch/x86/mm/ioremap.c b/arch/x86/mm/ioremap.c
index f3fa007..0254b78 100644
--- a/arch/x86/mm/ioremap.c
+++ b/arch/x86/mm/ioremap.c
@@ -428,17 +428,46 @@ void unxlate_dev_mem_ptr(phys_addr_t phys, void *addr)
  * Examine the physical address to determine if it is an area of memory
  * that should be mapped decrypted.  If the memory is not part of the
  * kernel usable area it was accessed and created decrypted, so these
- * areas should be mapped decrypted.
+ * areas should be mapped decrypted. And since the encryption key can
+ * change across reboots, persistent memory should also be mapped
+ * decrypted.
  */
 static bool memremap_should_map_decrypted(resource_size_t phys_addr,
 					  unsigned long size)
 {
+	int is_pmem;
+
+	/*
+	 * Check if the address is part of a persistent memory region.
+	 * This check covers areas added by E820, EFI and ACPI.
+	 */
+	is_pmem = region_intersects(phys_addr, size, IORESOURCE_MEM,
+				    IORES_DESC_PERSISTENT_MEMORY);
+	if (is_pmem != REGION_DISJOINT)
+		return true;
+
+	/*
+	 * Check if the non-volatile attribute is set for an EFI
+	 * reserved area.
+	 */
+	if (efi_enabled(EFI_BOOT)) {
+		switch (efi_mem_type(phys_addr)) {
+		case EFI_RESERVED_TYPE:
+			if (efi_mem_attributes(phys_addr) & EFI_MEMORY_NV)
+				return true;
+			break;
+		default:
+			break;
+		}
+	}
+
 	/* Check if the address is outside kernel usable area */
 	switch (e820__get_entry_type(phys_addr, phys_addr + size - 1)) {
 	case E820_TYPE_RESERVED:
 	case E820_TYPE_ACPI:
 	case E820_TYPE_NVS:
 	case E820_TYPE_UNUSABLE:
+	case E820_TYPE_PRAM:
 		return true;
 	default:
 		break;



From thomas.lendacky at amd.com  Fri Jun 16 11:54:00 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Fri, 16 Jun 2017 13:54:00 -0500
Subject: [PATCH v7 22/36] x86/mm: Add support for changing the memory
 encryption attribute
In-Reply-To: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
References: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170616185400.18967.86653.stgit@tlendack-t1.amdoffice.net>

Add support for changing the memory encryption attribute for one or more
memory pages. This will be useful when we have to change the AP trampoline
area to not be encrypted. Or when we need to change the SWIOTLB area to
not be encrypted in support of devices that can't support the encryption
mask range.

Reviewed-by: Borislav Petkov <bp at suse.de>
Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/include/asm/set_memory.h |    3 ++
 arch/x86/mm/pageattr.c            |   62 +++++++++++++++++++++++++++++++++++++
 2 files changed, 65 insertions(+)

diff --git a/arch/x86/include/asm/set_memory.h b/arch/x86/include/asm/set_memory.h
index eaec6c3..cd71273 100644
--- a/arch/x86/include/asm/set_memory.h
+++ b/arch/x86/include/asm/set_memory.h
@@ -11,6 +11,7 @@
  * Executability : eXeutable, NoteXecutable
  * Read/Write    : ReadOnly, ReadWrite
  * Presence      : NotPresent
+ * Encryption    : Encrypted, Decrypted
  *
  * Within a category, the attributes are mutually exclusive.
  *
@@ -42,6 +43,8 @@
 int set_memory_wb(unsigned long addr, int numpages);
 int set_memory_np(unsigned long addr, int numpages);
 int set_memory_4k(unsigned long addr, int numpages);
+int set_memory_encrypted(unsigned long addr, int numpages);
+int set_memory_decrypted(unsigned long addr, int numpages);
 
 int set_memory_array_uc(unsigned long *addr, int addrinarray);
 int set_memory_array_wc(unsigned long *addr, int addrinarray);
diff --git a/arch/x86/mm/pageattr.c b/arch/x86/mm/pageattr.c
index e7d3866..d9e09fb 100644
--- a/arch/x86/mm/pageattr.c
+++ b/arch/x86/mm/pageattr.c
@@ -1769,6 +1769,68 @@ int set_memory_4k(unsigned long addr, int numpages)
 					__pgprot(0), 1, 0, NULL);
 }
 
+static int __set_memory_enc_dec(unsigned long addr, int numpages, bool enc)
+{
+	struct cpa_data cpa;
+	unsigned long start;
+	int ret;
+
+	/* Nothing to do if the SME is not active */
+	if (!sme_active())
+		return 0;
+
+	/* Should not be working on unaligned addresses */
+	if (WARN_ONCE(addr & ~PAGE_MASK, "misaligned address: %#lx\n", addr))
+		addr &= PAGE_MASK;
+
+	start = addr;
+
+	memset(&cpa, 0, sizeof(cpa));
+	cpa.vaddr = &addr;
+	cpa.numpages = numpages;
+	cpa.mask_set = enc ? __pgprot(_PAGE_ENC) : __pgprot(0);
+	cpa.mask_clr = enc ? __pgprot(0) : __pgprot(_PAGE_ENC);
+	cpa.pgd = init_mm.pgd;
+
+	/* Must avoid aliasing mappings in the highmem code */
+	kmap_flush_unused();
+	vm_unmap_aliases();
+
+	/*
+	 * Before changing the encryption attribute, we need to flush caches.
+	 */
+	if (static_cpu_has(X86_FEATURE_CLFLUSH))
+		cpa_flush_range(start, numpages, 1);
+	else
+		cpa_flush_all(1);
+
+	ret = __change_page_attr_set_clr(&cpa, 1);
+
+	/*
+	 * After changing the encryption attribute, we need to flush TLBs
+	 * again in case any speculative TLB caching occurred (but no need
+	 * to flush caches again).  We could just use cpa_flush_all(), but
+	 * in case TLB flushing gets optimized in the cpa_flush_range()
+	 * path use the same logic as above.
+	 */
+	if (static_cpu_has(X86_FEATURE_CLFLUSH))
+		cpa_flush_range(start, numpages, 0);
+	else
+		cpa_flush_all(0);
+
+	return ret;
+}
+
+int set_memory_encrypted(unsigned long addr, int numpages)
+{
+	return __set_memory_enc_dec(addr, numpages, true);
+}
+
+int set_memory_decrypted(unsigned long addr, int numpages)
+{
+	return __set_memory_enc_dec(addr, numpages, false);
+}
+
 int set_pages_uc(struct page *page, int numpages)
 {
 	unsigned long addr = (unsigned long)page_address(page);



From thomas.lendacky at amd.com  Fri Jun 16 11:54:12 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Fri, 16 Jun 2017 13:54:12 -0500
Subject: [PATCH v7 23/36] x86,
 realmode: Decrypt trampoline area if memory encryption is active
In-Reply-To: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
References: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170616185412.18967.91559.stgit@tlendack-t1.amdoffice.net>

When Secure Memory Encryption is enabled, the trampoline area must not
be encrypted. A CPU running in real mode will not be able to decrypt
memory that has been encrypted because it will not be able to use addresses
with the memory encryption mask.

Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/realmode/init.c |    8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/arch/x86/realmode/init.c b/arch/x86/realmode/init.c
index cd4be19..d6ddc7e 100644
--- a/arch/x86/realmode/init.c
+++ b/arch/x86/realmode/init.c
@@ -1,6 +1,7 @@
 #include <linux/io.h>
 #include <linux/slab.h>
 #include <linux/memblock.h>
+#include <linux/mem_encrypt.h>
 
 #include <asm/set_memory.h>
 #include <asm/pgtable.h>
@@ -59,6 +60,13 @@ static void __init setup_real_mode(void)
 
 	base = (unsigned char *)real_mode_header;
 
+	/*
+	 * If SME is active, the trampoline area will need to be in
+	 * decrypted memory in order to bring up other processors
+	 * successfully.
+	 */
+	set_memory_decrypted((unsigned long)base, size >> PAGE_SHIFT);
+
 	memcpy(base, real_mode_blob, size);
 
 	phys_base = __pa(base);



From thomas.lendacky at amd.com  Fri Jun 16 11:54:24 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Fri, 16 Jun 2017 13:54:24 -0500
Subject: [PATCH v7 24/36] x86, swiotlb: Add memory encryption support
In-Reply-To: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
References: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170616185423.18967.19605.stgit@tlendack-t1.amdoffice.net>

Since DMA addresses will effectively look like 48-bit addresses when the
memory encryption mask is set, SWIOTLB is needed if the DMA mask of the
device performing the DMA does not support 48-bits. SWIOTLB will be
initialized to create decrypted bounce buffers for use by these devices.

Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/include/asm/dma-mapping.h |    5 ++-
 arch/x86/include/asm/mem_encrypt.h |    5 +++
 arch/x86/kernel/pci-dma.c          |   11 +++++--
 arch/x86/kernel/pci-nommu.c        |    2 +
 arch/x86/kernel/pci-swiotlb.c      |   15 +++++++++-
 arch/x86/mm/mem_encrypt.c          |   22 +++++++++++++++
 include/linux/swiotlb.h            |    1 +
 init/main.c                        |   10 +++++++
 lib/swiotlb.c                      |   54 +++++++++++++++++++++++++++++++-----
 9 files changed, 108 insertions(+), 17 deletions(-)

diff --git a/arch/x86/include/asm/dma-mapping.h b/arch/x86/include/asm/dma-mapping.h
index 08a0838..191f9a5 100644
--- a/arch/x86/include/asm/dma-mapping.h
+++ b/arch/x86/include/asm/dma-mapping.h
@@ -12,6 +12,7 @@
 #include <asm/io.h>
 #include <asm/swiotlb.h>
 #include <linux/dma-contiguous.h>
+#include <linux/mem_encrypt.h>
 
 #ifdef CONFIG_ISA
 # define ISA_DMA_BIT_MASK DMA_BIT_MASK(24)
@@ -62,12 +63,12 @@ static inline bool dma_capable(struct device *dev, dma_addr_t addr, size_t size)
 
 static inline dma_addr_t phys_to_dma(struct device *dev, phys_addr_t paddr)
 {
-	return paddr;
+	return __sme_set(paddr);
 }
 
 static inline phys_addr_t dma_to_phys(struct device *dev, dma_addr_t daddr)
 {
-	return daddr;
+	return __sme_clr(daddr);
 }
 #endif /* CONFIG_X86_DMA_REMAP */
 
diff --git a/arch/x86/include/asm/mem_encrypt.h b/arch/x86/include/asm/mem_encrypt.h
index 3ffa5fa..af835cf 100644
--- a/arch/x86/include/asm/mem_encrypt.h
+++ b/arch/x86/include/asm/mem_encrypt.h
@@ -33,6 +33,11 @@ void __init sme_early_decrypt(resource_size_t paddr,
 
 void __init sme_enable(void);
 
+/* Architecture __weak replacement functions */
+void __init mem_encrypt_init(void);
+
+void swiotlb_set_mem_attributes(void *vaddr, unsigned long size);
+
 #else	/* !CONFIG_AMD_MEM_ENCRYPT */
 
 #define sme_me_mask	0UL
diff --git a/arch/x86/kernel/pci-dma.c b/arch/x86/kernel/pci-dma.c
index 3a216ec..72d96d4 100644
--- a/arch/x86/kernel/pci-dma.c
+++ b/arch/x86/kernel/pci-dma.c
@@ -93,9 +93,12 @@ void *dma_generic_alloc_coherent(struct device *dev, size_t size,
 	if (gfpflags_allow_blocking(flag)) {
 		page = dma_alloc_from_contiguous(dev, count, get_order(size),
 						 flag);
-		if (page && page_to_phys(page) + size > dma_mask) {
-			dma_release_from_contiguous(dev, page, count);
-			page = NULL;
+		if (page) {
+			addr = phys_to_dma(dev, page_to_phys(page));
+			if (addr + size > dma_mask) {
+				dma_release_from_contiguous(dev, page, count);
+				page = NULL;
+			}
 		}
 	}
 	/* fallback */
@@ -104,7 +107,7 @@ void *dma_generic_alloc_coherent(struct device *dev, size_t size,
 	if (!page)
 		return NULL;
 
-	addr = page_to_phys(page);
+	addr = phys_to_dma(dev, page_to_phys(page));
 	if (addr + size > dma_mask) {
 		__free_pages(page, get_order(size));
 
diff --git a/arch/x86/kernel/pci-nommu.c b/arch/x86/kernel/pci-nommu.c
index a88952e..98b576a 100644
--- a/arch/x86/kernel/pci-nommu.c
+++ b/arch/x86/kernel/pci-nommu.c
@@ -30,7 +30,7 @@ static dma_addr_t nommu_map_page(struct device *dev, struct page *page,
 				 enum dma_data_direction dir,
 				 unsigned long attrs)
 {
-	dma_addr_t bus = page_to_phys(page) + offset;
+	dma_addr_t bus = phys_to_dma(dev, page_to_phys(page)) + offset;
 	WARN_ON(size == 0);
 	if (!check_addr("map_single", dev, bus, size))
 		return DMA_ERROR_CODE;
diff --git a/arch/x86/kernel/pci-swiotlb.c b/arch/x86/kernel/pci-swiotlb.c
index 1e23577..6770775 100644
--- a/arch/x86/kernel/pci-swiotlb.c
+++ b/arch/x86/kernel/pci-swiotlb.c
@@ -6,12 +6,14 @@
 #include <linux/swiotlb.h>
 #include <linux/bootmem.h>
 #include <linux/dma-mapping.h>
+#include <linux/mem_encrypt.h>
 
 #include <asm/iommu.h>
 #include <asm/swiotlb.h>
 #include <asm/dma.h>
 #include <asm/xen/swiotlb-xen.h>
 #include <asm/iommu_table.h>
+
 int swiotlb __read_mostly;
 
 void *x86_swiotlb_alloc_coherent(struct device *hwdev, size_t size,
@@ -79,8 +81,8 @@ int __init pci_swiotlb_detect_override(void)
 		  pci_swiotlb_late_init);
 
 /*
- * if 4GB or more detected (and iommu=off not set) return 1
- * and set swiotlb to 1.
+ * If 4GB or more detected (and iommu=off not set) or if SME is active
+ * then set swiotlb to 1 and return 1.
  */
 int __init pci_swiotlb_detect_4gb(void)
 {
@@ -89,6 +91,15 @@ int __init pci_swiotlb_detect_4gb(void)
 	if (!no_iommu && max_possible_pfn > MAX_DMA32_PFN)
 		swiotlb = 1;
 #endif
+
+	/*
+	 * If SME is active then swiotlb will be set to 1 so that bounce
+	 * buffers are allocated and used for devices that do not support
+	 * the addressing range required for the encryption mask.
+	 */
+	if (sme_active())
+		swiotlb = 1;
+
 	return swiotlb;
 }
 IOMMU_INIT(pci_swiotlb_detect_4gb,
diff --git a/arch/x86/mm/mem_encrypt.c b/arch/x86/mm/mem_encrypt.c
index ea5e3a6..842c8a6 100644
--- a/arch/x86/mm/mem_encrypt.c
+++ b/arch/x86/mm/mem_encrypt.c
@@ -16,11 +16,14 @@
 #ifdef CONFIG_AMD_MEM_ENCRYPT
 
 #include <linux/mm.h>
+#include <linux/dma-mapping.h>
+#include <linux/swiotlb.h>
 
 #include <asm/tlbflush.h>
 #include <asm/fixmap.h>
 #include <asm/setup.h>
 #include <asm/bootparam.h>
+#include <asm/set_memory.h>
 
 /*
  * Since SME related variables are set early in the boot process they must
@@ -187,6 +190,25 @@ void __init sme_early_init(void)
 		protection_map[i] = pgprot_encrypted(protection_map[i]);
 }
 
+/* Architecture __weak replacement functions */
+void __init mem_encrypt_init(void)
+{
+	if (!sme_me_mask)
+		return;
+
+	/* Call into SWIOTLB to update the SWIOTLB DMA buffers */
+	swiotlb_update_mem_attributes();
+}
+
+void swiotlb_set_mem_attributes(void *vaddr, unsigned long size)
+{
+	WARN(PAGE_ALIGN(size) != size,
+	     "size is not page-aligned (%#lx)\n", size);
+
+	/* Make the SWIOTLB buffer area decrypted */
+	set_memory_decrypted((unsigned long)vaddr, size >> PAGE_SHIFT);
+}
+
 void __init sme_encrypt_kernel(void)
 {
 }
diff --git a/include/linux/swiotlb.h b/include/linux/swiotlb.h
index 4ee479f..15e7160 100644
--- a/include/linux/swiotlb.h
+++ b/include/linux/swiotlb.h
@@ -35,6 +35,7 @@ enum swiotlb_force {
 extern unsigned long swiotlb_nr_tbl(void);
 unsigned long swiotlb_size_or_default(void);
 extern int swiotlb_late_init_with_tbl(char *tlb, unsigned long nslabs);
+extern void __init swiotlb_update_mem_attributes(void);
 
 /*
  * Enumeration for sync targets
diff --git a/init/main.c b/init/main.c
index df58a41..5013cbc 100644
--- a/init/main.c
+++ b/init/main.c
@@ -488,6 +488,8 @@ void __init __weak thread_stack_cache_init(void)
 }
 #endif
 
+void __init __weak mem_encrypt_init(void) { }
+
 /*
  * Set up kernel memory allocators
  */
@@ -640,6 +642,14 @@ asmlinkage __visible void __init start_kernel(void)
 	 */
 	locking_selftest();
 
+	/*
+	 * This needs to be called before any devices perform DMA
+	 * operations that might use the SWIOTLB bounce buffers. It will
+	 * mark the bounce buffers as decrypted so that their usage will
+	 * not cause "plain-text" data to be decrypted when accessed.
+	 */
+	mem_encrypt_init();
+
 #ifdef CONFIG_BLK_DEV_INITRD
 	if (initrd_start && !initrd_below_start_ok &&
 	    page_to_pfn(virt_to_page((void *)initrd_start)) < min_low_pfn) {
diff --git a/lib/swiotlb.c b/lib/swiotlb.c
index a8d74a7..04ac91a 100644
--- a/lib/swiotlb.c
+++ b/lib/swiotlb.c
@@ -30,6 +30,7 @@
 #include <linux/highmem.h>
 #include <linux/gfp.h>
 #include <linux/scatterlist.h>
+#include <linux/mem_encrypt.h>
 
 #include <asm/io.h>
 #include <asm/dma.h>
@@ -155,6 +156,15 @@ unsigned long swiotlb_size_or_default(void)
 	return size ? size : (IO_TLB_DEFAULT_SIZE);
 }
 
+void __weak swiotlb_set_mem_attributes(void *vaddr, unsigned long size) { }
+
+/* For swiotlb, clear memory encryption mask from dma addresses */
+static dma_addr_t swiotlb_phys_to_dma(struct device *hwdev,
+				      phys_addr_t address)
+{
+	return __sme_clr(phys_to_dma(hwdev, address));
+}
+
 /* Note that this doesn't work with highmem page */
 static dma_addr_t swiotlb_virt_to_bus(struct device *hwdev,
 				      volatile void *address)
@@ -183,6 +193,31 @@ void swiotlb_print_info(void)
 	       bytes >> 20, vstart, vend - 1);
 }
 
+/*
+ * Early SWIOTLB allocation may be too early to allow an architecture to
+ * perform the desired operations.  This function allows the architecture to
+ * call SWIOTLB when the operations are possible.  It needs to be called
+ * before the SWIOTLB memory is used.
+ */
+void __init swiotlb_update_mem_attributes(void)
+{
+	void *vaddr;
+	unsigned long bytes;
+
+	if (no_iotlb_memory || late_alloc)
+		return;
+
+	vaddr = phys_to_virt(io_tlb_start);
+	bytes = PAGE_ALIGN(io_tlb_nslabs << IO_TLB_SHIFT);
+	swiotlb_set_mem_attributes(vaddr, bytes);
+	memset(vaddr, 0, bytes);
+
+	vaddr = phys_to_virt(io_tlb_overflow_buffer);
+	bytes = PAGE_ALIGN(io_tlb_overflow);
+	swiotlb_set_mem_attributes(vaddr, bytes);
+	memset(vaddr, 0, bytes);
+}
+
 int __init swiotlb_init_with_tbl(char *tlb, unsigned long nslabs, int verbose)
 {
 	void *v_overflow_buffer;
@@ -320,6 +355,7 @@ int __init swiotlb_init_with_tbl(char *tlb, unsigned long nslabs, int verbose)
 	io_tlb_start = virt_to_phys(tlb);
 	io_tlb_end = io_tlb_start + bytes;
 
+	swiotlb_set_mem_attributes(tlb, bytes);
 	memset(tlb, 0, bytes);
 
 	/*
@@ -330,6 +366,8 @@ int __init swiotlb_init_with_tbl(char *tlb, unsigned long nslabs, int verbose)
 	if (!v_overflow_buffer)
 		goto cleanup2;
 
+	swiotlb_set_mem_attributes(v_overflow_buffer, io_tlb_overflow);
+	memset(v_overflow_buffer, 0, io_tlb_overflow);
 	io_tlb_overflow_buffer = virt_to_phys(v_overflow_buffer);
 
 	/*
@@ -581,7 +619,7 @@ phys_addr_t swiotlb_tbl_map_single(struct device *hwdev,
 		return SWIOTLB_MAP_ERROR;
 	}
 
-	start_dma_addr = phys_to_dma(hwdev, io_tlb_start);
+	start_dma_addr = swiotlb_phys_to_dma(hwdev, io_tlb_start);
 	return swiotlb_tbl_map_single(hwdev, start_dma_addr, phys, size,
 				      dir, attrs);
 }
@@ -702,7 +740,7 @@ void swiotlb_tbl_sync_single(struct device *hwdev, phys_addr_t tlb_addr,
 			goto err_warn;
 
 		ret = phys_to_virt(paddr);
-		dev_addr = phys_to_dma(hwdev, paddr);
+		dev_addr = swiotlb_phys_to_dma(hwdev, paddr);
 
 		/* Confirm address can be DMA'd by device */
 		if (dev_addr + size - 1 > dma_mask) {
@@ -812,10 +850,10 @@ dma_addr_t swiotlb_map_page(struct device *dev, struct page *page,
 	map = map_single(dev, phys, size, dir, attrs);
 	if (map == SWIOTLB_MAP_ERROR) {
 		swiotlb_full(dev, size, dir, 1);
-		return phys_to_dma(dev, io_tlb_overflow_buffer);
+		return swiotlb_phys_to_dma(dev, io_tlb_overflow_buffer);
 	}
 
-	dev_addr = phys_to_dma(dev, map);
+	dev_addr = swiotlb_phys_to_dma(dev, map);
 
 	/* Ensure that the address returned is DMA'ble */
 	if (dma_capable(dev, dev_addr, size))
@@ -824,7 +862,7 @@ dma_addr_t swiotlb_map_page(struct device *dev, struct page *page,
 	attrs |= DMA_ATTR_SKIP_CPU_SYNC;
 	swiotlb_tbl_unmap_single(dev, map, size, dir, attrs);
 
-	return phys_to_dma(dev, io_tlb_overflow_buffer);
+	return swiotlb_phys_to_dma(dev, io_tlb_overflow_buffer);
 }
 EXPORT_SYMBOL_GPL(swiotlb_map_page);
 
@@ -958,7 +996,7 @@ void swiotlb_unmap_page(struct device *hwdev, dma_addr_t dev_addr,
 				sg_dma_len(sgl) = 0;
 				return 0;
 			}
-			sg->dma_address = phys_to_dma(hwdev, map);
+			sg->dma_address = swiotlb_phys_to_dma(hwdev, map);
 		} else
 			sg->dma_address = dev_addr;
 		sg_dma_len(sg) = sg->length;
@@ -1026,7 +1064,7 @@ void swiotlb_unmap_page(struct device *hwdev, dma_addr_t dev_addr,
 int
 swiotlb_dma_mapping_error(struct device *hwdev, dma_addr_t dma_addr)
 {
-	return (dma_addr == phys_to_dma(hwdev, io_tlb_overflow_buffer));
+	return (dma_addr == swiotlb_phys_to_dma(hwdev, io_tlb_overflow_buffer));
 }
 EXPORT_SYMBOL(swiotlb_dma_mapping_error);
 
@@ -1039,6 +1077,6 @@ void swiotlb_unmap_page(struct device *hwdev, dma_addr_t dev_addr,
 int
 swiotlb_dma_supported(struct device *hwdev, u64 mask)
 {
-	return phys_to_dma(hwdev, io_tlb_end - 1) <= mask;
+	return swiotlb_phys_to_dma(hwdev, io_tlb_end - 1) <= mask;
 }
 EXPORT_SYMBOL(swiotlb_dma_supported);



From thomas.lendacky at amd.com  Fri Jun 16 11:54:36 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Fri, 16 Jun 2017 13:54:36 -0500
Subject: [PATCH v7 25/36] swiotlb: Add warnings for use of bounce buffers
 with SME
In-Reply-To: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
References: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170616185435.18967.26665.stgit@tlendack-t1.amdoffice.net>

Add warnings to let the user know when bounce buffers are being used for
DMA when SME is active.  Since the bounce buffers are not in encrypted
memory, these notifications are to allow the user to determine some
appropriate action - if necessary.  Actions can range from utilizing an
IOMMU, replacing the device with another device that can support 64-bit
DMA, ignoring the message if the device isn't used much, etc.

Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 include/linux/dma-mapping.h |   11 +++++++++++
 include/linux/mem_encrypt.h |    8 ++++++++
 lib/swiotlb.c               |    3 +++
 3 files changed, 22 insertions(+)

diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h
index 4f3eece..ee2307e 100644
--- a/include/linux/dma-mapping.h
+++ b/include/linux/dma-mapping.h
@@ -10,6 +10,7 @@
 #include <linux/scatterlist.h>
 #include <linux/kmemcheck.h>
 #include <linux/bug.h>
+#include <linux/mem_encrypt.h>
 
 /**
  * List of possible attributes associated with a DMA mapping. The semantics
@@ -577,6 +578,11 @@ static inline int dma_set_mask(struct device *dev, u64 mask)
 
 	if (!dev->dma_mask || !dma_supported(dev, mask))
 		return -EIO;
+
+	/* Since mask is unsigned, this can only be true if SME is active */
+	if (mask < sme_dma_mask())
+		dev_warn(dev, "SME is active, device will require DMA bounce buffers\n");
+
 	*dev->dma_mask = mask;
 	return 0;
 }
@@ -596,6 +602,11 @@ static inline int dma_set_coherent_mask(struct device *dev, u64 mask)
 {
 	if (!dma_supported(dev, mask))
 		return -EIO;
+
+	/* Since mask is unsigned, this can only be true if SME is active */
+	if (mask < sme_dma_mask())
+		dev_warn(dev, "SME is active, device will require DMA bounce buffers\n");
+
 	dev->coherent_dma_mask = mask;
 	return 0;
 }
diff --git a/include/linux/mem_encrypt.h b/include/linux/mem_encrypt.h
index 837c66b..2168002 100644
--- a/include/linux/mem_encrypt.h
+++ b/include/linux/mem_encrypt.h
@@ -30,6 +30,14 @@ static inline bool sme_active(void)
 	return !!sme_me_mask;
 }
 
+static inline u64 sme_dma_mask(void)
+{
+	if (!sme_me_mask)
+		return 0ULL;
+
+	return ((u64)sme_me_mask << 1) - 1;
+}
+
 /*
  * The __sme_set() and __sme_clr() macros are useful for adding or removing
  * the encryption mask from a value (e.g. when dealing with pagetable
diff --git a/lib/swiotlb.c b/lib/swiotlb.c
index 04ac91a..8c6c83e 100644
--- a/lib/swiotlb.c
+++ b/lib/swiotlb.c
@@ -507,6 +507,9 @@ phys_addr_t swiotlb_tbl_map_single(struct device *hwdev,
 	if (no_iotlb_memory)
 		panic("Can not allocate SWIOTLB buffer earlier and can't now provide you with the DMA bounce buffer");
 
+	if (sme_active())
+		pr_warn_once("SME is active and system is using DMA bounce buffers\n");
+
 	mask = dma_get_seg_boundary(hwdev);
 
 	tbl_dma_addr &= mask;



From thomas.lendacky at amd.com  Fri Jun 16 11:54:47 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Fri, 16 Jun 2017 13:54:47 -0500
Subject: [PATCH v7 26/36] x86/CPU/AMD: Make the microcode level available
 earlier in the boot
In-Reply-To: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
References: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170616185447.18967.8792.stgit@tlendack-t1.amdoffice.net>

Move the setting of the cpuinfo_x86.microcode field from amd_init() to
early_amd_init() so that it is available earlier in the boot process. This
avoids having to read MSR_AMD64_PATCH_LEVEL directly during early boot.

Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/kernel/cpu/amd.c |    8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/arch/x86/kernel/cpu/amd.c b/arch/x86/kernel/cpu/amd.c
index 5bdcbd4..fdcf305 100644
--- a/arch/x86/kernel/cpu/amd.c
+++ b/arch/x86/kernel/cpu/amd.c
@@ -547,8 +547,12 @@ static void bsp_init_amd(struct cpuinfo_x86 *c)
 
 static void early_init_amd(struct cpuinfo_x86 *c)
 {
+	u32 dummy;
+
 	early_init_amd_mc(c);
 
+	rdmsr_safe(MSR_AMD64_PATCH_LEVEL, &c->microcode, &dummy);
+
 	/*
 	 * c->x86_power is 8000_0007 edx. Bit 8 is TSC runs at constant rate
 	 * with P/T states and does not stop in deep C-states
@@ -746,8 +750,6 @@ static void init_amd_bd(struct cpuinfo_x86 *c)
 
 static void init_amd(struct cpuinfo_x86 *c)
 {
-	u32 dummy;
-
 	early_init_amd(c);
 
 	/*
@@ -809,8 +811,6 @@ static void init_amd(struct cpuinfo_x86 *c)
 	if (c->x86 > 0x11)
 		set_cpu_cap(c, X86_FEATURE_ARAT);
 
-	rdmsr_safe(MSR_AMD64_PATCH_LEVEL, &c->microcode, &dummy);
-
 	/* 3DNow or LM implies PREFETCHW */
 	if (!cpu_has(c, X86_FEATURE_3DNOWPREFETCH))
 		if (cpu_has(c, X86_FEATURE_3DNOW) || cpu_has(c, X86_FEATURE_LM))



From thomas.lendacky at amd.com  Fri Jun 16 11:54:59 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Fri, 16 Jun 2017 13:54:59 -0500
Subject: [PATCH v7 27/36] iommu/amd: Allow the AMD IOMMU to work with memory
 encryption
In-Reply-To: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
References: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170616185459.18967.72790.stgit@tlendack-t1.amdoffice.net>

The IOMMU is programmed with physical addresses for the various tables
and buffers that are used to communicate between the device and the
driver. When the driver allocates this memory it is encrypted. In order
for the IOMMU to access the memory as encrypted the encryption mask needs
to be included in these physical addresses during configuration.

The PTE entries created by the IOMMU should also include the encryption
mask so that when the device behind the IOMMU performs a DMA, the DMA
will be performed to encrypted memory.

Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 drivers/iommu/amd_iommu.c       |   30 ++++++++++++++++--------------
 drivers/iommu/amd_iommu_init.c  |   34 ++++++++++++++++++++++++++++------
 drivers/iommu/amd_iommu_proto.h |   10 ++++++++++
 drivers/iommu/amd_iommu_types.h |    2 +-
 4 files changed, 55 insertions(+), 21 deletions(-)

diff --git a/drivers/iommu/amd_iommu.c b/drivers/iommu/amd_iommu.c
index 63cacf5..912008c 100644
--- a/drivers/iommu/amd_iommu.c
+++ b/drivers/iommu/amd_iommu.c
@@ -544,7 +544,7 @@ static void dump_dte_entry(u16 devid)
 
 static void dump_command(unsigned long phys_addr)
 {
-	struct iommu_cmd *cmd = phys_to_virt(phys_addr);
+	struct iommu_cmd *cmd = iommu_phys_to_virt(phys_addr);
 	int i;
 
 	for (i = 0; i < 4; ++i)
@@ -865,11 +865,13 @@ static void copy_cmd_to_buffer(struct amd_iommu *iommu,
 
 static void build_completion_wait(struct iommu_cmd *cmd, u64 address)
 {
+	u64 paddr = iommu_virt_to_phys((void *)address);
+
 	WARN_ON(address & 0x7ULL);
 
 	memset(cmd, 0, sizeof(*cmd));
-	cmd->data[0] = lower_32_bits(__pa(address)) | CMD_COMPL_WAIT_STORE_MASK;
-	cmd->data[1] = upper_32_bits(__pa(address));
+	cmd->data[0] = lower_32_bits(paddr) | CMD_COMPL_WAIT_STORE_MASK;
+	cmd->data[1] = upper_32_bits(paddr);
 	cmd->data[2] = 1;
 	CMD_SET_TYPE(cmd, CMD_COMPL_WAIT);
 }
@@ -1328,7 +1330,7 @@ static bool increase_address_space(struct protection_domain *domain,
 		return false;
 
 	*pte             = PM_LEVEL_PDE(domain->mode,
-					virt_to_phys(domain->pt_root));
+					iommu_virt_to_phys(domain->pt_root));
 	domain->pt_root  = pte;
 	domain->mode    += 1;
 	domain->updated  = true;
@@ -1365,7 +1367,7 @@ static u64 *alloc_pte(struct protection_domain *domain,
 			if (!page)
 				return NULL;
 
-			__npte = PM_LEVEL_PDE(level, virt_to_phys(page));
+			__npte = PM_LEVEL_PDE(level, iommu_virt_to_phys(page));
 
 			/* pte could have been changed somewhere. */
 			if (cmpxchg64(pte, __pte, __npte) != __pte) {
@@ -1481,10 +1483,10 @@ static int iommu_map_page(struct protection_domain *dom,
 			return -EBUSY;
 
 	if (count > 1) {
-		__pte = PAGE_SIZE_PTE(phys_addr, page_size);
+		__pte = PAGE_SIZE_PTE(__sme_set(phys_addr), page_size);
 		__pte |= PM_LEVEL_ENC(7) | IOMMU_PTE_P | IOMMU_PTE_FC;
 	} else
-		__pte = phys_addr | IOMMU_PTE_P | IOMMU_PTE_FC;
+		__pte = __sme_set(phys_addr) | IOMMU_PTE_P | IOMMU_PTE_FC;
 
 	if (prot & IOMMU_PROT_IR)
 		__pte |= IOMMU_PTE_IR;
@@ -1700,7 +1702,7 @@ static void free_gcr3_tbl_level1(u64 *tbl)
 		if (!(tbl[i] & GCR3_VALID))
 			continue;
 
-		ptr = __va(tbl[i] & PAGE_MASK);
+		ptr = iommu_phys_to_virt(tbl[i] & PAGE_MASK);
 
 		free_page((unsigned long)ptr);
 	}
@@ -1715,7 +1717,7 @@ static void free_gcr3_tbl_level2(u64 *tbl)
 		if (!(tbl[i] & GCR3_VALID))
 			continue;
 
-		ptr = __va(tbl[i] & PAGE_MASK);
+		ptr = iommu_phys_to_virt(tbl[i] & PAGE_MASK);
 
 		free_gcr3_tbl_level1(ptr);
 	}
@@ -1807,7 +1809,7 @@ static void set_dte_entry(u16 devid, struct protection_domain *domain, bool ats)
 	u64 flags = 0;
 
 	if (domain->mode != PAGE_MODE_NONE)
-		pte_root = virt_to_phys(domain->pt_root);
+		pte_root = iommu_virt_to_phys(domain->pt_root);
 
 	pte_root |= (domain->mode & DEV_ENTRY_MODE_MASK)
 		    << DEV_ENTRY_MODE_SHIFT;
@@ -1819,7 +1821,7 @@ static void set_dte_entry(u16 devid, struct protection_domain *domain, bool ats)
 		flags |= DTE_FLAG_IOTLB;
 
 	if (domain->flags & PD_IOMMUV2_MASK) {
-		u64 gcr3 = __pa(domain->gcr3_tbl);
+		u64 gcr3 = iommu_virt_to_phys(domain->gcr3_tbl);
 		u64 glx  = domain->glx;
 		u64 tmp;
 
@@ -3470,10 +3472,10 @@ static u64 *__get_gcr3_pte(u64 *root, int level, int pasid, bool alloc)
 			if (root == NULL)
 				return NULL;
 
-			*pte = __pa(root) | GCR3_VALID;
+			*pte = iommu_virt_to_phys(root) | GCR3_VALID;
 		}
 
-		root = __va(*pte & PAGE_MASK);
+		root = iommu_phys_to_virt(*pte & PAGE_MASK);
 
 		level -= 1;
 	}
@@ -3652,7 +3654,7 @@ static void set_dte_irq_entry(u16 devid, struct irq_remap_table *table)
 
 	dte	= amd_iommu_dev_table[devid].data[2];
 	dte	&= ~DTE_IRQ_PHYS_ADDR_MASK;
-	dte	|= virt_to_phys(table->table);
+	dte	|= iommu_virt_to_phys(table->table);
 	dte	|= DTE_IRQ_REMAP_INTCTL;
 	dte	|= DTE_IRQ_TABLE_LEN;
 	dte	|= DTE_IRQ_REMAP_ENABLE;
diff --git a/drivers/iommu/amd_iommu_init.c b/drivers/iommu/amd_iommu_init.c
index 5a11328..26862f1 100644
--- a/drivers/iommu/amd_iommu_init.c
+++ b/drivers/iommu/amd_iommu_init.c
@@ -29,6 +29,7 @@
 #include <linux/export.h>
 #include <linux/iommu.h>
 #include <linux/kmemleak.h>
+#include <linux/mem_encrypt.h>
 #include <asm/pci-direct.h>
 #include <asm/iommu.h>
 #include <asm/gart.h>
@@ -346,7 +347,7 @@ static void iommu_set_device_table(struct amd_iommu *iommu)
 
 	BUG_ON(iommu->mmio_base == NULL);
 
-	entry = virt_to_phys(amd_iommu_dev_table);
+	entry = iommu_virt_to_phys(amd_iommu_dev_table);
 	entry |= (dev_table_size >> 12) - 1;
 	memcpy_toio(iommu->mmio_base + MMIO_DEV_TABLE_OFFSET,
 			&entry, sizeof(entry));
@@ -602,7 +603,7 @@ static void iommu_enable_command_buffer(struct amd_iommu *iommu)
 
 	BUG_ON(iommu->cmd_buf == NULL);
 
-	entry = (u64)virt_to_phys(iommu->cmd_buf);
+	entry = iommu_virt_to_phys(iommu->cmd_buf);
 	entry |= MMIO_CMD_SIZE_512;
 
 	memcpy_toio(iommu->mmio_base + MMIO_CMD_BUF_OFFSET,
@@ -631,7 +632,7 @@ static void iommu_enable_event_buffer(struct amd_iommu *iommu)
 
 	BUG_ON(iommu->evt_buf == NULL);
 
-	entry = (u64)virt_to_phys(iommu->evt_buf) | EVT_LEN_MASK;
+	entry = iommu_virt_to_phys(iommu->evt_buf) | EVT_LEN_MASK;
 
 	memcpy_toio(iommu->mmio_base + MMIO_EVT_BUF_OFFSET,
 		    &entry, sizeof(entry));
@@ -664,7 +665,7 @@ static void iommu_enable_ppr_log(struct amd_iommu *iommu)
 	if (iommu->ppr_log == NULL)
 		return;
 
-	entry = (u64)virt_to_phys(iommu->ppr_log) | PPR_LOG_SIZE_512;
+	entry = iommu_virt_to_phys(iommu->ppr_log) | PPR_LOG_SIZE_512;
 
 	memcpy_toio(iommu->mmio_base + MMIO_PPR_LOG_OFFSET,
 		    &entry, sizeof(entry));
@@ -744,10 +745,10 @@ static int iommu_init_ga_log(struct amd_iommu *iommu)
 	if (!iommu->ga_log_tail)
 		goto err_out;
 
-	entry = (u64)virt_to_phys(iommu->ga_log) | GA_LOG_SIZE_512;
+	entry = iommu_virt_to_phys(iommu->ga_log) | GA_LOG_SIZE_512;
 	memcpy_toio(iommu->mmio_base + MMIO_GA_LOG_BASE_OFFSET,
 		    &entry, sizeof(entry));
-	entry = ((u64)virt_to_phys(iommu->ga_log) & 0xFFFFFFFFFFFFFULL) & ~7ULL;
+	entry = (iommu_virt_to_phys(iommu->ga_log) & 0xFFFFFFFFFFFFFULL) & ~7ULL;
 	memcpy_toio(iommu->mmio_base + MMIO_GA_LOG_TAIL_OFFSET,
 		    &entry, sizeof(entry));
 	writel(0x00, iommu->mmio_base + MMIO_GA_HEAD_OFFSET);
@@ -2535,6 +2536,24 @@ static int __init amd_iommu_init(void)
 	return ret;
 }
 
+static bool amd_iommu_supports_sme(void)
+{
+	if (!sme_active() || (boot_cpu_data.x86 != 0x17))
+		return true;
+
+	/* For Fam17h, a specific level of support is required */
+	if (boot_cpu_data.microcode >= 0x08001205)
+		return true;
+
+	if ((boot_cpu_data.microcode >= 0x08001126) &&
+	    (boot_cpu_data.microcode <= 0x080011ff))
+		return true;
+
+	pr_notice("AMD-Vi: IOMMU not currently supported when SME is active\n");
+
+	return false;
+}
+
 /****************************************************************************
  *
  * Early detect code. This code runs at IOMMU detection time in the DMA
@@ -2552,6 +2571,9 @@ int __init amd_iommu_detect(void)
 	if (amd_iommu_disabled)
 		return -ENODEV;
 
+	if (!amd_iommu_supports_sme())
+		return -ENODEV;
+
 	ret = iommu_go_to_state(IOMMU_IVRS_DETECTED);
 	if (ret)
 		return ret;
diff --git a/drivers/iommu/amd_iommu_proto.h b/drivers/iommu/amd_iommu_proto.h
index 466260f..3f12fb2 100644
--- a/drivers/iommu/amd_iommu_proto.h
+++ b/drivers/iommu/amd_iommu_proto.h
@@ -87,4 +87,14 @@ static inline bool iommu_feature(struct amd_iommu *iommu, u64 f)
 	return !!(iommu->features & f);
 }
 
+static inline u64 iommu_virt_to_phys(void *vaddr)
+{
+	return (u64)__sme_set(virt_to_phys(vaddr));
+}
+
+static inline void *iommu_phys_to_virt(unsigned long paddr)
+{
+	return phys_to_virt(__sme_clr(paddr));
+}
+
 #endif /* _ASM_X86_AMD_IOMMU_PROTO_H  */
diff --git a/drivers/iommu/amd_iommu_types.h b/drivers/iommu/amd_iommu_types.h
index 4de8f41..3ce587d 100644
--- a/drivers/iommu/amd_iommu_types.h
+++ b/drivers/iommu/amd_iommu_types.h
@@ -343,7 +343,7 @@
 
 #define IOMMU_PAGE_MASK (((1ULL << 52) - 1) & ~0xfffULL)
 #define IOMMU_PTE_PRESENT(pte) ((pte) & IOMMU_PTE_P)
-#define IOMMU_PTE_PAGE(pte) (phys_to_virt((pte) & IOMMU_PAGE_MASK))
+#define IOMMU_PTE_PAGE(pte) (iommu_phys_to_virt((pte) & IOMMU_PAGE_MASK))
 #define IOMMU_PTE_MODE(pte) (((pte) >> 9) & 0x07)
 
 #define IOMMU_PROT_MASK 0x03



From thomas.lendacky at amd.com  Fri Jun 16 11:55:10 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Fri, 16 Jun 2017 13:55:10 -0500
Subject: [PATCH v7 28/36] x86, realmode: Check for memory encryption on the APs
In-Reply-To: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
References: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170616185510.18967.23512.stgit@tlendack-t1.amdoffice.net>

Add support to check if memory encryption is active in the kernel and that
it has been enabled on the AP. If memory encryption is active in the kernel
but has not been enabled on the AP, then set the memory encryption bit (bit
23) of MSR_K8_SYSCFG to enable memory encryption on that AP and allow the
AP to continue start up.

Reviewed-by: Borislav Petkov <bp at suse.de>
Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/include/asm/realmode.h      |   12 ++++++++++++
 arch/x86/realmode/init.c             |    4 ++++
 arch/x86/realmode/rm/trampoline_64.S |   24 ++++++++++++++++++++++++
 3 files changed, 40 insertions(+)

diff --git a/arch/x86/include/asm/realmode.h b/arch/x86/include/asm/realmode.h
index 230e190..90d9152 100644
--- a/arch/x86/include/asm/realmode.h
+++ b/arch/x86/include/asm/realmode.h
@@ -1,6 +1,15 @@
 #ifndef _ARCH_X86_REALMODE_H
 #define _ARCH_X86_REALMODE_H
 
+/*
+ * Flag bit definitions for use with the flags field of the trampoline header
+ * in the CONFIG_X86_64 variant.
+ */
+#define TH_FLAGS_SME_ACTIVE_BIT		0
+#define TH_FLAGS_SME_ACTIVE		BIT(TH_FLAGS_SME_ACTIVE_BIT)
+
+#ifndef __ASSEMBLY__
+
 #include <linux/types.h>
 #include <asm/io.h>
 
@@ -38,6 +47,7 @@ struct trampoline_header {
 	u64 start;
 	u64 efer;
 	u32 cr4;
+	u32 flags;
 #endif
 };
 
@@ -69,4 +79,6 @@ static inline size_t real_mode_size_needed(void)
 void set_real_mode_mem(phys_addr_t mem, size_t size);
 void reserve_real_mode(void);
 
+#endif /* __ASSEMBLY__ */
+
 #endif /* _ARCH_X86_REALMODE_H */
diff --git a/arch/x86/realmode/init.c b/arch/x86/realmode/init.c
index d6ddc7e..1f71980 100644
--- a/arch/x86/realmode/init.c
+++ b/arch/x86/realmode/init.c
@@ -108,6 +108,10 @@ static void __init setup_real_mode(void)
 	trampoline_cr4_features = &trampoline_header->cr4;
 	*trampoline_cr4_features = mmu_cr4_features;
 
+	trampoline_header->flags = 0;
+	if (sme_active())
+		trampoline_header->flags |= TH_FLAGS_SME_ACTIVE;
+
 	trampoline_pgd = (u64 *) __va(real_mode_header->trampoline_pgd);
 	trampoline_pgd[0] = trampoline_pgd_entry.pgd;
 	trampoline_pgd[511] = init_top_pgt[511].pgd;
diff --git a/arch/x86/realmode/rm/trampoline_64.S b/arch/x86/realmode/rm/trampoline_64.S
index dac7b20..614fd70 100644
--- a/arch/x86/realmode/rm/trampoline_64.S
+++ b/arch/x86/realmode/rm/trampoline_64.S
@@ -30,6 +30,7 @@
 #include <asm/msr.h>
 #include <asm/segment.h>
 #include <asm/processor-flags.h>
+#include <asm/realmode.h>
 #include "realmode.h"
 
 	.text
@@ -92,6 +93,28 @@ ENTRY(startup_32)
 	movl	%edx, %fs
 	movl	%edx, %gs
 
+	/*
+	 * Check for memory encryption support. This is a safety net in
+	 * case BIOS hasn't done the necessary step of setting the bit in
+	 * the MSR for this AP. If SME is active and we've gotten this far
+	 * then it is safe for us to set the MSR bit and continue. If we
+	 * don't we'll eventually crash trying to execute encrypted
+	 * instructions.
+	 */
+	bt	$TH_FLAGS_SME_ACTIVE_BIT, pa_tr_flags
+	jnc	.Ldone
+	movl	$MSR_K8_SYSCFG, %ecx
+	rdmsr
+	bts	$MSR_K8_SYSCFG_MEM_ENCRYPT_BIT, %eax
+	jc	.Ldone
+
+	/*
+	 * Memory encryption is enabled but the SME enable bit for this
+	 * CPU has has not been set.  It is safe to set it, so do so.
+	 */
+	wrmsr
+.Ldone:
+
 	movl	pa_tr_cr4, %eax
 	movl	%eax, %cr4		# Enable PAE mode
 
@@ -147,6 +170,7 @@ GLOBAL(trampoline_header)
 	tr_start:		.space	8
 	GLOBAL(tr_efer)		.space	8
 	GLOBAL(tr_cr4)		.space	4
+	GLOBAL(tr_flags)	.space	4
 END(trampoline_header)
 
 #include "trampoline_common.S"



From thomas.lendacky at amd.com  Fri Jun 16 11:55:33 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Fri, 16 Jun 2017 13:55:33 -0500
Subject: [PATCH v7 30/36] kvm: x86: svm: Support Secure Memory Encryption
 within KVM
In-Reply-To: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
References: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170616185532.18967.63164.stgit@tlendack-t1.amdoffice.net>

Update the KVM support to work with SME. The VMCB has a number of fields
where physical addresses are used and these addresses must contain the
memory encryption mask in order to properly access the encrypted memory.
Also, use the memory encryption mask when creating and using the nested
page tables.

Reviewed-by: Borislav Petkov <bp at suse.de>
Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/include/asm/kvm_host.h |    2 +-
 arch/x86/kvm/mmu.c              |   12 ++++++++----
 arch/x86/kvm/mmu.h              |    2 +-
 arch/x86/kvm/svm.c              |   35 ++++++++++++++++++-----------------
 arch/x86/kvm/vmx.c              |    3 ++-
 arch/x86/kvm/x86.c              |    3 ++-
 6 files changed, 32 insertions(+), 25 deletions(-)

diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index 695605e..6d1267f 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -1069,7 +1069,7 @@ struct kvm_arch_async_pf {
 void kvm_mmu_uninit_vm(struct kvm *kvm);
 void kvm_mmu_set_mask_ptes(u64 user_mask, u64 accessed_mask,
 		u64 dirty_mask, u64 nx_mask, u64 x_mask, u64 p_mask,
-		u64 acc_track_mask);
+		u64 acc_track_mask, u64 me_mask);
 
 void kvm_mmu_reset_context(struct kvm_vcpu *vcpu);
 void kvm_mmu_slot_remove_write_access(struct kvm *kvm,
diff --git a/arch/x86/kvm/mmu.c b/arch/x86/kvm/mmu.c
index cb82259..e85888c 100644
--- a/arch/x86/kvm/mmu.c
+++ b/arch/x86/kvm/mmu.c
@@ -107,7 +107,7 @@ enum {
 	(((address) >> PT32_LEVEL_SHIFT(level)) & ((1 << PT32_LEVEL_BITS) - 1))
 
 
-#define PT64_BASE_ADDR_MASK (((1ULL << 52) - 1) & ~(u64)(PAGE_SIZE-1))
+#define PT64_BASE_ADDR_MASK __sme_clr((((1ULL << 52) - 1) & ~(u64)(PAGE_SIZE-1)))
 #define PT64_DIR_BASE_ADDR_MASK \
 	(PT64_BASE_ADDR_MASK & ~((1ULL << (PAGE_SHIFT + PT64_LEVEL_BITS)) - 1))
 #define PT64_LVL_ADDR_MASK(level) \
@@ -125,7 +125,7 @@ enum {
 					    * PT32_LEVEL_BITS))) - 1))
 
 #define PT64_PERM_MASK (PT_PRESENT_MASK | PT_WRITABLE_MASK | shadow_user_mask \
-			| shadow_x_mask | shadow_nx_mask)
+			| shadow_x_mask | shadow_nx_mask | shadow_me_mask)
 
 #define ACC_EXEC_MASK    1
 #define ACC_WRITE_MASK   PT_WRITABLE_MASK
@@ -184,6 +184,7 @@ struct kvm_shadow_walk_iterator {
 static u64 __read_mostly shadow_dirty_mask;
 static u64 __read_mostly shadow_mmio_mask;
 static u64 __read_mostly shadow_present_mask;
+static u64 __read_mostly shadow_me_mask;
 
 /*
  * The mask/value to distinguish a PTE that has been marked not-present for
@@ -317,7 +318,7 @@ static bool check_mmio_spte(struct kvm_vcpu *vcpu, u64 spte)
 
 void kvm_mmu_set_mask_ptes(u64 user_mask, u64 accessed_mask,
 		u64 dirty_mask, u64 nx_mask, u64 x_mask, u64 p_mask,
-		u64 acc_track_mask)
+		u64 acc_track_mask, u64 me_mask)
 {
 	if (acc_track_mask != 0)
 		acc_track_mask |= SPTE_SPECIAL_MASK;
@@ -330,6 +331,7 @@ void kvm_mmu_set_mask_ptes(u64 user_mask, u64 accessed_mask,
 	shadow_present_mask = p_mask;
 	shadow_acc_track_mask = acc_track_mask;
 	WARN_ON(shadow_accessed_mask != 0 && shadow_acc_track_mask != 0);
+	shadow_me_mask = me_mask;
 }
 EXPORT_SYMBOL_GPL(kvm_mmu_set_mask_ptes);
 
@@ -2398,7 +2400,8 @@ static void link_shadow_page(struct kvm_vcpu *vcpu, u64 *sptep,
 	BUILD_BUG_ON(VMX_EPT_WRITABLE_MASK != PT_WRITABLE_MASK);
 
 	spte = __pa(sp->spt) | shadow_present_mask | PT_WRITABLE_MASK |
-	       shadow_user_mask | shadow_x_mask | shadow_accessed_mask;
+	       shadow_user_mask | shadow_x_mask | shadow_accessed_mask |
+	       shadow_me_mask;
 
 	mmu_spte_set(sptep, spte);
 
@@ -2700,6 +2703,7 @@ static int set_spte(struct kvm_vcpu *vcpu, u64 *sptep,
 		pte_access &= ~ACC_WRITE_MASK;
 
 	spte |= (u64)pfn << PAGE_SHIFT;
+	spte |= shadow_me_mask;
 
 	if (pte_access & ACC_WRITE_MASK) {
 
diff --git a/arch/x86/kvm/mmu.h b/arch/x86/kvm/mmu.h
index 330bf3a..08b779d 100644
--- a/arch/x86/kvm/mmu.h
+++ b/arch/x86/kvm/mmu.h
@@ -48,7 +48,7 @@
 
 static inline u64 rsvd_bits(int s, int e)
 {
-	return ((1ULL << (e - s + 1)) - 1) << s;
+	return __sme_clr(((1ULL << (e - s + 1)) - 1) << s);
 }
 
 void kvm_mmu_set_mmio_spte_mask(u64 mmio_mask);
diff --git a/arch/x86/kvm/svm.c b/arch/x86/kvm/svm.c
index ba9891a..d2e9fca 100644
--- a/arch/x86/kvm/svm.c
+++ b/arch/x86/kvm/svm.c
@@ -1138,9 +1138,9 @@ static void avic_init_vmcb(struct vcpu_svm *svm)
 {
 	struct vmcb *vmcb = svm->vmcb;
 	struct kvm_arch *vm_data = &svm->vcpu.kvm->arch;
-	phys_addr_t bpa = page_to_phys(svm->avic_backing_page);
-	phys_addr_t lpa = page_to_phys(vm_data->avic_logical_id_table_page);
-	phys_addr_t ppa = page_to_phys(vm_data->avic_physical_id_table_page);
+	phys_addr_t bpa = __sme_set(page_to_phys(svm->avic_backing_page));
+	phys_addr_t lpa = __sme_set(page_to_phys(vm_data->avic_logical_id_table_page));
+	phys_addr_t ppa = __sme_set(page_to_phys(vm_data->avic_physical_id_table_page));
 
 	vmcb->control.avic_backing_page = bpa & AVIC_HPA_MASK;
 	vmcb->control.avic_logical_id = lpa & AVIC_HPA_MASK;
@@ -1203,8 +1203,8 @@ static void init_vmcb(struct vcpu_svm *svm)
 		set_intercept(svm, INTERCEPT_MWAIT);
 	}
 
-	control->iopm_base_pa = iopm_base;
-	control->msrpm_base_pa = __pa(svm->msrpm);
+	control->iopm_base_pa = __sme_set(iopm_base);
+	control->msrpm_base_pa = __sme_set(__pa(svm->msrpm));
 	control->int_ctl = V_INTR_MASKING_MASK;
 
 	init_seg(&save->es);
@@ -1338,9 +1338,9 @@ static int avic_init_backing_page(struct kvm_vcpu *vcpu)
 		return -EINVAL;
 
 	new_entry = READ_ONCE(*entry);
-	new_entry = (page_to_phys(svm->avic_backing_page) &
-		     AVIC_PHYSICAL_ID_ENTRY_BACKING_PAGE_MASK) |
-		     AVIC_PHYSICAL_ID_ENTRY_VALID_MASK;
+	new_entry = __sme_set((page_to_phys(svm->avic_backing_page) &
+			      AVIC_PHYSICAL_ID_ENTRY_BACKING_PAGE_MASK) |
+			      AVIC_PHYSICAL_ID_ENTRY_VALID_MASK);
 	WRITE_ONCE(*entry, new_entry);
 
 	svm->avic_physical_id_cache = entry;
@@ -1608,7 +1608,7 @@ static struct kvm_vcpu *svm_create_vcpu(struct kvm *kvm, unsigned int id)
 
 	svm->vmcb = page_address(page);
 	clear_page(svm->vmcb);
-	svm->vmcb_pa = page_to_pfn(page) << PAGE_SHIFT;
+	svm->vmcb_pa = __sme_set(page_to_pfn(page) << PAGE_SHIFT);
 	svm->asid_generation = 0;
 	init_vmcb(svm);
 
@@ -1636,7 +1636,7 @@ static void svm_free_vcpu(struct kvm_vcpu *vcpu)
 {
 	struct vcpu_svm *svm = to_svm(vcpu);
 
-	__free_page(pfn_to_page(svm->vmcb_pa >> PAGE_SHIFT));
+	__free_page(pfn_to_page(__sme_clr(svm->vmcb_pa) >> PAGE_SHIFT));
 	__free_pages(virt_to_page(svm->msrpm), MSRPM_ALLOC_ORDER);
 	__free_page(virt_to_page(svm->nested.hsave));
 	__free_pages(virt_to_page(svm->nested.msrpm), MSRPM_ALLOC_ORDER);
@@ -2303,7 +2303,7 @@ static u64 nested_svm_get_tdp_pdptr(struct kvm_vcpu *vcpu, int index)
 	u64 pdpte;
 	int ret;
 
-	ret = kvm_vcpu_read_guest_page(vcpu, gpa_to_gfn(cr3), &pdpte,
+	ret = kvm_vcpu_read_guest_page(vcpu, gpa_to_gfn(__sme_clr(cr3)), &pdpte,
 				       offset_in_page(cr3) + index * 8, 8);
 	if (ret)
 		return 0;
@@ -2315,7 +2315,7 @@ static void nested_svm_set_tdp_cr3(struct kvm_vcpu *vcpu,
 {
 	struct vcpu_svm *svm = to_svm(vcpu);
 
-	svm->vmcb->control.nested_cr3 = root;
+	svm->vmcb->control.nested_cr3 = __sme_set(root);
 	mark_dirty(svm->vmcb, VMCB_NPT);
 	svm_flush_tlb(vcpu);
 }
@@ -2803,7 +2803,7 @@ static bool nested_svm_vmrun_msrpm(struct vcpu_svm *svm)
 		svm->nested.msrpm[p] = svm->msrpm[p] | value;
 	}
 
-	svm->vmcb->control.msrpm_base_pa = __pa(svm->nested.msrpm);
+	svm->vmcb->control.msrpm_base_pa = __sme_set(__pa(svm->nested.msrpm));
 
 	return true;
 }
@@ -4435,7 +4435,7 @@ static int svm_ir_list_add(struct vcpu_svm *svm, struct amd_iommu_pi_data *pi)
 	pr_debug("SVM: %s: use GA mode for irq %u\n", __func__,
 		 irq.vector);
 	*svm = to_svm(vcpu);
-	vcpu_info->pi_desc_addr = page_to_phys((*svm)->avic_backing_page);
+	vcpu_info->pi_desc_addr = __sme_set(page_to_phys((*svm)->avic_backing_page));
 	vcpu_info->vector = irq.vector;
 
 	return 0;
@@ -4486,7 +4486,8 @@ static int svm_update_pi_irte(struct kvm *kvm, unsigned int host_irq,
 			struct amd_iommu_pi_data pi;
 
 			/* Try to enable guest_mode in IRTE */
-			pi.base = page_to_phys(svm->avic_backing_page) & AVIC_HPA_MASK;
+			pi.base = __sme_set(page_to_phys(svm->avic_backing_page) &
+					    AVIC_HPA_MASK);
 			pi.ga_tag = AVIC_GATAG(kvm->arch.avic_vm_id,
 						     svm->vcpu.vcpu_id);
 			pi.is_guest_mode = true;
@@ -4911,7 +4912,7 @@ static void svm_set_cr3(struct kvm_vcpu *vcpu, unsigned long root)
 {
 	struct vcpu_svm *svm = to_svm(vcpu);
 
-	svm->vmcb->save.cr3 = root;
+	svm->vmcb->save.cr3 = __sme_set(root);
 	mark_dirty(svm->vmcb, VMCB_CR);
 	svm_flush_tlb(vcpu);
 }
@@ -4920,7 +4921,7 @@ static void set_tdp_cr3(struct kvm_vcpu *vcpu, unsigned long root)
 {
 	struct vcpu_svm *svm = to_svm(vcpu);
 
-	svm->vmcb->control.nested_cr3 = root;
+	svm->vmcb->control.nested_cr3 = __sme_set(root);
 	mark_dirty(svm->vmcb, VMCB_NPT);
 
 	/* Also sync guest cr3 here in case we live migrate */
diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c
index 7dd53fb..53098cd 100644
--- a/arch/x86/kvm/vmx.c
+++ b/arch/x86/kvm/vmx.c
@@ -6452,7 +6452,8 @@ void vmx_enable_tdp(void)
 		enable_ept_ad_bits ? VMX_EPT_DIRTY_BIT : 0ull,
 		0ull, VMX_EPT_EXECUTABLE_MASK,
 		cpu_has_vmx_ept_execute_only() ? 0ull : VMX_EPT_READABLE_MASK,
-		enable_ept_ad_bits ? 0ull : VMX_EPT_RWX_MASK);
+		enable_ept_ad_bits ? 0ull : VMX_EPT_RWX_MASK,
+		0ull);
 
 	ept_set_mmio_spte_mask();
 	kvm_enable_tdp();
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 87d3cb9..559f710 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -54,6 +54,7 @@
 #include <linux/kvm_irqfd.h>
 #include <linux/irqbypass.h>
 #include <linux/sched/stat.h>
+#include <linux/mem_encrypt.h>
 
 #include <trace/events/kvm.h>
 
@@ -6095,7 +6096,7 @@ int kvm_arch_init(void *opaque)
 
 	kvm_mmu_set_mask_ptes(PT_USER_MASK, PT_ACCESSED_MASK,
 			PT_DIRTY_MASK, PT64_NX_MASK, 0,
-			PT_PRESENT_MASK, 0);
+			PT_PRESENT_MASK, 0, sme_me_mask);
 	kvm_timer_init();
 
 	perf_register_guest_info_callbacks(&kvm_guest_cbs);



From thomas.lendacky at amd.com  Fri Jun 16 11:55:23 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Fri, 16 Jun 2017 13:55:23 -0500
Subject: [PATCH v7 29/36] x86, drm,
 fbdev: Do not specify encrypted memory for video mappings
In-Reply-To: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
References: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170616185523.18967.34256.stgit@tlendack-t1.amdoffice.net>

Since video memory needs to be accessed decrypted, be sure that the
memory encryption mask is not set for the video ranges.

Reviewed-by: Borislav Petkov <bp at suse.de>
Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/include/asm/vga.h       |   14 +++++++++++++-
 arch/x86/mm/pageattr.c           |    2 ++
 drivers/gpu/drm/drm_gem.c        |    2 ++
 drivers/gpu/drm/drm_vm.c         |    4 ++++
 drivers/gpu/drm/ttm/ttm_bo_vm.c  |    7 +++++--
 drivers/gpu/drm/udl/udl_fb.c     |    4 ++++
 drivers/video/fbdev/core/fbmem.c |   12 ++++++++++++
 7 files changed, 42 insertions(+), 3 deletions(-)

diff --git a/arch/x86/include/asm/vga.h b/arch/x86/include/asm/vga.h
index c4b9dc2..9f42bee 100644
--- a/arch/x86/include/asm/vga.h
+++ b/arch/x86/include/asm/vga.h
@@ -7,12 +7,24 @@
 #ifndef _ASM_X86_VGA_H
 #define _ASM_X86_VGA_H
 
+#include <asm/set_memory.h>
+
 /*
  *	On the PC, we can just recalculate addresses and then
  *	access the videoram directly without any black magic.
+ *	To support memory encryption however, we need to access
+ *	the videoram as decrypted memory.
  */
 
-#define VGA_MAP_MEM(x, s) (unsigned long)phys_to_virt(x)
+#define VGA_MAP_MEM(x, s)					\
+({								\
+	unsigned long start = (unsigned long)phys_to_virt(x);	\
+								\
+	if (IS_ENABLED(CONFIG_AMD_MEM_ENCRYPT))			\
+		set_memory_decrypted(start, (s) >> PAGE_SHIFT);	\
+								\
+	start;							\
+})
 
 #define vga_readb(x) (*(x))
 #define vga_writeb(x, y) (*(y) = (x))
diff --git a/arch/x86/mm/pageattr.c b/arch/x86/mm/pageattr.c
index d9e09fb..13fc5db 100644
--- a/arch/x86/mm/pageattr.c
+++ b/arch/x86/mm/pageattr.c
@@ -1825,11 +1825,13 @@ int set_memory_encrypted(unsigned long addr, int numpages)
 {
 	return __set_memory_enc_dec(addr, numpages, true);
 }
+EXPORT_SYMBOL_GPL(set_memory_encrypted);
 
 int set_memory_decrypted(unsigned long addr, int numpages)
 {
 	return __set_memory_enc_dec(addr, numpages, false);
 }
+EXPORT_SYMBOL_GPL(set_memory_decrypted);
 
 int set_pages_uc(struct page *page, int numpages)
 {
diff --git a/drivers/gpu/drm/drm_gem.c b/drivers/gpu/drm/drm_gem.c
index b1e28c9..019f48c 100644
--- a/drivers/gpu/drm/drm_gem.c
+++ b/drivers/gpu/drm/drm_gem.c
@@ -36,6 +36,7 @@
 #include <linux/pagemap.h>
 #include <linux/shmem_fs.h>
 #include <linux/dma-buf.h>
+#include <linux/mem_encrypt.h>
 #include <drm/drmP.h>
 #include <drm/drm_vma_manager.h>
 #include <drm/drm_gem.h>
@@ -928,6 +929,7 @@ int drm_gem_mmap_obj(struct drm_gem_object *obj, unsigned long obj_size,
 	vma->vm_ops = dev->driver->gem_vm_ops;
 	vma->vm_private_data = obj;
 	vma->vm_page_prot = pgprot_writecombine(vm_get_page_prot(vma->vm_flags));
+	vma->vm_page_prot = pgprot_decrypted(vma->vm_page_prot);
 
 	/* Take a ref for this mapping of the object, so that the fault
 	 * handler can dereference the mmap offset's pointer to the object.
diff --git a/drivers/gpu/drm/drm_vm.c b/drivers/gpu/drm/drm_vm.c
index 1170b32..ed4bcbf 100644
--- a/drivers/gpu/drm/drm_vm.c
+++ b/drivers/gpu/drm/drm_vm.c
@@ -40,6 +40,7 @@
 #include <linux/efi.h>
 #include <linux/slab.h>
 #endif
+#include <linux/mem_encrypt.h>
 #include <asm/pgtable.h>
 #include "drm_internal.h"
 #include "drm_legacy.h"
@@ -58,6 +59,9 @@ static pgprot_t drm_io_prot(struct drm_local_map *map,
 {
 	pgprot_t tmp = vm_get_page_prot(vma->vm_flags);
 
+	/* We don't want graphics memory to be mapped encrypted */
+	tmp = pgprot_decrypted(tmp);
+
 #if defined(__i386__) || defined(__x86_64__) || defined(__powerpc__)
 	if (map->type == _DRM_REGISTERS && !(map->flags & _DRM_WRITE_COMBINING))
 		tmp = pgprot_noncached(tmp);
diff --git a/drivers/gpu/drm/ttm/ttm_bo_vm.c b/drivers/gpu/drm/ttm/ttm_bo_vm.c
index 9f53df9..622dab6 100644
--- a/drivers/gpu/drm/ttm/ttm_bo_vm.c
+++ b/drivers/gpu/drm/ttm/ttm_bo_vm.c
@@ -39,6 +39,7 @@
 #include <linux/rbtree.h>
 #include <linux/module.h>
 #include <linux/uaccess.h>
+#include <linux/mem_encrypt.h>
 
 #define TTM_BO_VM_NUM_PREFAULT 16
 
@@ -230,9 +231,11 @@ static int ttm_bo_vm_fault(struct vm_fault *vmf)
 	 * first page.
 	 */
 	for (i = 0; i < TTM_BO_VM_NUM_PREFAULT; ++i) {
-		if (bo->mem.bus.is_iomem)
+		if (bo->mem.bus.is_iomem) {
+			/* Iomem should not be marked encrypted */
+			cvma.vm_page_prot = pgprot_decrypted(cvma.vm_page_prot);
 			pfn = bdev->driver->io_mem_pfn(bo, page_offset);
-		else {
+		} else {
 			page = ttm->pages[page_offset];
 			if (unlikely(!page && i == 0)) {
 				retval = VM_FAULT_OOM;
diff --git a/drivers/gpu/drm/udl/udl_fb.c b/drivers/gpu/drm/udl/udl_fb.c
index 4a65003..92e1690 100644
--- a/drivers/gpu/drm/udl/udl_fb.c
+++ b/drivers/gpu/drm/udl/udl_fb.c
@@ -14,6 +14,7 @@
 #include <linux/slab.h>
 #include <linux/fb.h>
 #include <linux/dma-buf.h>
+#include <linux/mem_encrypt.h>
 
 #include <drm/drmP.h>
 #include <drm/drm_crtc.h>
@@ -169,6 +170,9 @@ static int udl_fb_mmap(struct fb_info *info, struct vm_area_struct *vma)
 	pr_notice("mmap() framebuffer addr:%lu size:%lu\n",
 		  pos, size);
 
+	/* We don't want the framebuffer to be mapped encrypted */
+	vma->vm_page_prot = pgprot_decrypted(vma->vm_page_prot);
+
 	while (size > 0) {
 		page = vmalloc_to_pfn((void *)pos);
 		if (remap_pfn_range(vma, start, page, PAGE_SIZE, PAGE_SHARED))
diff --git a/drivers/video/fbdev/core/fbmem.c b/drivers/video/fbdev/core/fbmem.c
index 069fe79..b5e7c33 100644
--- a/drivers/video/fbdev/core/fbmem.c
+++ b/drivers/video/fbdev/core/fbmem.c
@@ -32,6 +32,7 @@
 #include <linux/device.h>
 #include <linux/efi.h>
 #include <linux/fb.h>
+#include <linux/mem_encrypt.h>
 
 #include <asm/fb.h>
 
@@ -1405,6 +1406,12 @@ static long fb_compat_ioctl(struct file *file, unsigned int cmd,
 	mutex_lock(&info->mm_lock);
 	if (fb->fb_mmap) {
 		int res;
+
+		/*
+		 * The framebuffer needs to be accessed decrypted, be sure
+		 * SME protection is removed ahead of the call
+		 */
+		vma->vm_page_prot = pgprot_decrypted(vma->vm_page_prot);
 		res = fb->fb_mmap(info, vma);
 		mutex_unlock(&info->mm_lock);
 		return res;
@@ -1430,6 +1437,11 @@ static long fb_compat_ioctl(struct file *file, unsigned int cmd,
 	mutex_unlock(&info->mm_lock);
 
 	vma->vm_page_prot = vm_get_page_prot(vma->vm_flags);
+	/*
+	 * The framebuffer needs to be accessed decrypted, be sure
+	 * SME protection is removed
+	 */
+	vma->vm_page_prot = pgprot_decrypted(vma->vm_page_prot);
 	fb_pgprotect(file, vma, start);
 
 	return vm_iomap_memory(vma, start, len);



From thomas.lendacky at amd.com  Fri Jun 16 11:55:45 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Fri, 16 Jun 2017 13:55:45 -0500
Subject: [PATCH v7 31/36] x86/mm, kexec: Allow kexec to be used with SME
In-Reply-To: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
References: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170616185545.18967.90815.stgit@tlendack-t1.amdoffice.net>

Provide support so that kexec can be used to boot a kernel when SME is
enabled.

Support is needed to allocate pages for kexec without encryption.  This
is needed in order to be able to reboot in the kernel in the same manner
as originally booted.

Additionally, when shutting down all of the CPUs we need to be sure to
flush the caches and then halt. This is needed when booting from a state
where SME was not active into a state where SME is active (or vice-versa).
Without these steps, it is possible for cache lines to exist for the same
physical location but tagged both with and without the encryption bit. This
can cause random memory corruption when caches are flushed depending on
which cacheline is written last.

Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/include/asm/init.h          |    1 +
 arch/x86/include/asm/kexec.h         |    8 ++++++++
 arch/x86/include/asm/pgtable_types.h |    1 +
 arch/x86/kernel/machine_kexec_64.c   |   22 +++++++++++++++++++++-
 arch/x86/kernel/process.c            |   17 +++++++++++++++--
 arch/x86/mm/ident_map.c              |   12 ++++++++----
 include/linux/kexec.h                |   14 ++++++++++++++
 kernel/kexec_core.c                  |   12 +++++++++++-
 8 files changed, 79 insertions(+), 8 deletions(-)

diff --git a/arch/x86/include/asm/init.h b/arch/x86/include/asm/init.h
index 474eb8c..05c4aa0 100644
--- a/arch/x86/include/asm/init.h
+++ b/arch/x86/include/asm/init.h
@@ -7,6 +7,7 @@ struct x86_mapping_info {
 	unsigned long page_flag;	 /* page flag for PMD or PUD entry */
 	unsigned long offset;		 /* ident mapping offset */
 	bool direct_gbpages;		 /* PUD level 1GB page support */
+	unsigned long kernpg_flag;	 /* kernel pagetable flag override */
 };
 
 int kernel_ident_mapping_init(struct x86_mapping_info *info, pgd_t *pgd_page,
diff --git a/arch/x86/include/asm/kexec.h b/arch/x86/include/asm/kexec.h
index 70ef205..e8183ac 100644
--- a/arch/x86/include/asm/kexec.h
+++ b/arch/x86/include/asm/kexec.h
@@ -207,6 +207,14 @@ struct kexec_entry64_regs {
 	uint64_t r15;
 	uint64_t rip;
 };
+
+extern int arch_kexec_post_alloc_pages(void *vaddr, unsigned int pages,
+				       gfp_t gfp);
+#define arch_kexec_post_alloc_pages arch_kexec_post_alloc_pages
+
+extern void arch_kexec_pre_free_pages(void *vaddr, unsigned int pages);
+#define arch_kexec_pre_free_pages arch_kexec_pre_free_pages
+
 #endif
 
 typedef void crash_vmclear_fn(void);
diff --git a/arch/x86/include/asm/pgtable_types.h b/arch/x86/include/asm/pgtable_types.h
index 32095af..830992f 100644
--- a/arch/x86/include/asm/pgtable_types.h
+++ b/arch/x86/include/asm/pgtable_types.h
@@ -213,6 +213,7 @@ enum page_cache_mode {
 #define PAGE_KERNEL		__pgprot(__PAGE_KERNEL | _PAGE_ENC)
 #define PAGE_KERNEL_RO		__pgprot(__PAGE_KERNEL_RO | _PAGE_ENC)
 #define PAGE_KERNEL_EXEC	__pgprot(__PAGE_KERNEL_EXEC | _PAGE_ENC)
+#define PAGE_KERNEL_EXEC_NOENC	__pgprot(__PAGE_KERNEL_EXEC)
 #define PAGE_KERNEL_RX		__pgprot(__PAGE_KERNEL_RX | _PAGE_ENC)
 #define PAGE_KERNEL_NOCACHE	__pgprot(__PAGE_KERNEL_NOCACHE | _PAGE_ENC)
 #define PAGE_KERNEL_LARGE	__pgprot(__PAGE_KERNEL_LARGE | _PAGE_ENC)
diff --git a/arch/x86/kernel/machine_kexec_64.c b/arch/x86/kernel/machine_kexec_64.c
index cb0a304..9cf8daa 100644
--- a/arch/x86/kernel/machine_kexec_64.c
+++ b/arch/x86/kernel/machine_kexec_64.c
@@ -87,7 +87,7 @@ static int init_transition_pgtable(struct kimage *image, pgd_t *pgd)
 		set_pmd(pmd, __pmd(__pa(pte) | _KERNPG_TABLE));
 	}
 	pte = pte_offset_kernel(pmd, vaddr);
-	set_pte(pte, pfn_pte(paddr >> PAGE_SHIFT, PAGE_KERNEL_EXEC));
+	set_pte(pte, pfn_pte(paddr >> PAGE_SHIFT, PAGE_KERNEL_EXEC_NOENC));
 	return 0;
 err:
 	free_transition_pgtable(image);
@@ -115,6 +115,7 @@ static int init_pgtable(struct kimage *image, unsigned long start_pgtable)
 		.alloc_pgt_page	= alloc_pgt_page,
 		.context	= image,
 		.page_flag	= __PAGE_KERNEL_LARGE_EXEC,
+		.kernpg_flag	= _KERNPG_TABLE_NOENC,
 	};
 	unsigned long mstart, mend;
 	pgd_t *level4p;
@@ -602,3 +603,22 @@ void arch_kexec_unprotect_crashkres(void)
 {
 	kexec_mark_crashkres(false);
 }
+
+int arch_kexec_post_alloc_pages(void *vaddr, unsigned int pages, gfp_t gfp)
+{
+	/*
+	 * If SME is active we need to be sure that kexec pages are
+	 * not encrypted because when we boot to the new kernel the
+	 * pages won't be accessed encrypted (initially).
+	 */
+	return set_memory_decrypted((unsigned long)vaddr, pages);
+}
+
+void arch_kexec_pre_free_pages(void *vaddr, unsigned int pages)
+{
+	/*
+	 * If SME is active we need to reset the pages back to being
+	 * an encrypted mapping before freeing them.
+	 */
+	set_memory_encrypted((unsigned long)vaddr, pages);
+}
diff --git a/arch/x86/kernel/process.c b/arch/x86/kernel/process.c
index 0bb8842..fdad0fb 100644
--- a/arch/x86/kernel/process.c
+++ b/arch/x86/kernel/process.c
@@ -355,6 +355,7 @@ bool xen_set_default_idle(void)
 	return ret;
 }
 #endif
+
 void stop_this_cpu(void *dummy)
 {
 	local_irq_disable();
@@ -365,8 +366,20 @@ void stop_this_cpu(void *dummy)
 	disable_local_APIC();
 	mcheck_cpu_clear(this_cpu_ptr(&cpu_info));
 
-	for (;;)
-		halt();
+	for (;;) {
+		/*
+		 * Use wbinvd followed by hlt to stop the processor. This
+		 * provides support for kexec on a processor that supports
+		 * SME. With kexec, going from SME inactive to SME active
+		 * requires clearing cache entries so that addresses without
+		 * the encryption bit set don't corrupt the same physical
+		 * address that has the encryption bit set when caches are
+		 * flushed. To achieve this a wbinvd is performed followed by
+		 * a hlt. Even if the processor is not in the kexec/SME
+		 * scenario this only adds a wbinvd to a halting processor.
+		 */
+		asm volatile("wbinvd; hlt" : : : "memory");
+	}
 }
 
 /*
diff --git a/arch/x86/mm/ident_map.c b/arch/x86/mm/ident_map.c
index adab159..31cea98 100644
--- a/arch/x86/mm/ident_map.c
+++ b/arch/x86/mm/ident_map.c
@@ -51,7 +51,7 @@ static int ident_pud_init(struct x86_mapping_info *info, pud_t *pud_page,
 		if (!pmd)
 			return -ENOMEM;
 		ident_pmd_init(info, pmd, addr, next);
-		set_pud(pud, __pud(__pa(pmd) | _KERNPG_TABLE));
+		set_pud(pud, __pud(__pa(pmd) | info->kernpg_flag));
 	}
 
 	return 0;
@@ -79,7 +79,7 @@ static int ident_p4d_init(struct x86_mapping_info *info, p4d_t *p4d_page,
 		if (!pud)
 			return -ENOMEM;
 		ident_pud_init(info, pud, addr, next);
-		set_p4d(p4d, __p4d(__pa(pud) | _KERNPG_TABLE));
+		set_p4d(p4d, __p4d(__pa(pud) | info->kernpg_flag));
 	}
 
 	return 0;
@@ -93,6 +93,10 @@ int kernel_ident_mapping_init(struct x86_mapping_info *info, pgd_t *pgd_page,
 	unsigned long next;
 	int result;
 
+	/* Set the default pagetable flags if not supplied */
+	if (!info->kernpg_flag)
+		info->kernpg_flag = _KERNPG_TABLE;
+
 	for (; addr < end; addr = next) {
 		pgd_t *pgd = pgd_page + pgd_index(addr);
 		p4d_t *p4d;
@@ -116,14 +120,14 @@ int kernel_ident_mapping_init(struct x86_mapping_info *info, pgd_t *pgd_page,
 		if (result)
 			return result;
 		if (IS_ENABLED(CONFIG_X86_5LEVEL)) {
-			set_pgd(pgd, __pgd(__pa(p4d) | _KERNPG_TABLE));
+			set_pgd(pgd, __pgd(__pa(p4d) | info->kernpg_flag));
 		} else {
 			/*
 			 * With p4d folded, pgd is equal to p4d.
 			 * The pgd entry has to point to the pud page table in this case.
 			 */
 			pud_t *pud = pud_offset(p4d, 0);
-			set_pgd(pgd, __pgd(__pa(pud) | _KERNPG_TABLE));
+			set_pgd(pgd, __pgd(__pa(pud) | info->kernpg_flag));
 		}
 	}
 
diff --git a/include/linux/kexec.h b/include/linux/kexec.h
index c9481eb..5d17fd6 100644
--- a/include/linux/kexec.h
+++ b/include/linux/kexec.h
@@ -334,6 +334,20 @@ static inline void *boot_phys_to_virt(unsigned long entry)
 	return phys_to_virt(boot_phys_to_phys(entry));
 }
 
+#ifndef arch_kexec_post_alloc_pages
+static inline int arch_kexec_post_alloc_pages(void *vaddr, unsigned int pages,
+					      gfp_t gfp)
+{
+	return 0;
+}
+#endif
+
+#ifndef arch_kexec_pre_free_pages
+static inline void arch_kexec_pre_free_pages(void *vaddr, unsigned int pages)
+{
+}
+#endif
+
 #else /* !CONFIG_KEXEC_CORE */
 struct pt_regs;
 struct task_struct;
diff --git a/kernel/kexec_core.c b/kernel/kexec_core.c
index ae1a3ba..2df1c39 100644
--- a/kernel/kexec_core.c
+++ b/kernel/kexec_core.c
@@ -300,7 +300,7 @@ static struct page *kimage_alloc_pages(gfp_t gfp_mask, unsigned int order)
 {
 	struct page *pages;
 
-	pages = alloc_pages(gfp_mask, order);
+	pages = alloc_pages(gfp_mask & ~__GFP_ZERO, order);
 	if (pages) {
 		unsigned int count, i;
 
@@ -309,6 +309,13 @@ static struct page *kimage_alloc_pages(gfp_t gfp_mask, unsigned int order)
 		count = 1 << order;
 		for (i = 0; i < count; i++)
 			SetPageReserved(pages + i);
+
+		arch_kexec_post_alloc_pages(page_address(pages), count,
+					    gfp_mask);
+
+		if (gfp_mask & __GFP_ZERO)
+			for (i = 0; i < count; i++)
+				clear_highpage(pages + i);
 	}
 
 	return pages;
@@ -320,6 +327,9 @@ static void kimage_free_pages(struct page *page)
 
 	order = page_private(page);
 	count = 1 << order;
+
+	arch_kexec_pre_free_pages(page_address(page), count);
+
 	for (i = 0; i < count; i++)
 		ClearPageReserved(page + i);
 	__free_pages(page, order);



From thomas.lendacky at amd.com  Fri Jun 16 11:55:54 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Fri, 16 Jun 2017 13:55:54 -0500
Subject: [PATCH v7 32/36] xen/x86: Remove SME feature in PV guests
In-Reply-To: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
References: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170616185554.18967.82909.stgit@tlendack-t1.amdoffice.net>

Xen does not currently support SME for PV guests. Clear the SME cpu
capability in order to avoid any ambiguity.

Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/xen/enlighten_pv.c |    1 +
 1 file changed, 1 insertion(+)

diff --git a/arch/x86/xen/enlighten_pv.c b/arch/x86/xen/enlighten_pv.c
index f33eef4..e6ecf42 100644
--- a/arch/x86/xen/enlighten_pv.c
+++ b/arch/x86/xen/enlighten_pv.c
@@ -294,6 +294,7 @@ static void __init xen_init_capabilities(void)
 	setup_clear_cpu_cap(X86_FEATURE_MTRR);
 	setup_clear_cpu_cap(X86_FEATURE_ACC);
 	setup_clear_cpu_cap(X86_FEATURE_X2APIC);
+	setup_clear_cpu_cap(X86_FEATURE_SME);
 
 	if (!xen_initial_domain())
 		setup_clear_cpu_cap(X86_FEATURE_ACPI);



From thomas.lendacky at amd.com  Fri Jun 16 11:56:07 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Fri, 16 Jun 2017 13:56:07 -0500
Subject: [PATCH v7 33/36] x86/mm: Use proper encryption attributes with
 /dev/mem
In-Reply-To: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
References: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170616185607.18967.51412.stgit@tlendack-t1.amdoffice.net>

When accessing memory using /dev/mem (or /dev/kmem) use the proper
encryption attributes when mapping the memory.

To insure the proper attributes are applied when reading or writing
/dev/mem, update the xlate_dev_mem_ptr() function to use memremap()
which will essentially perform the same steps of applying __va for
RAM or using ioremap() for if not RAM.

To insure the proper attributes are applied when mmapping /dev/mem,
update the phys_mem_access_prot() to call phys_mem_access_encrypted(),
a new function which will check if the memory should be mapped encrypted
or not. If it is not to be mapped encrypted then the VMA protection
value is updated to remove the encryption bit.

Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/include/asm/io.h |    3 +++
 arch/x86/mm/ioremap.c     |   18 +++++++++---------
 arch/x86/mm/pat.c         |    3 +++
 3 files changed, 15 insertions(+), 9 deletions(-)

diff --git a/arch/x86/include/asm/io.h b/arch/x86/include/asm/io.h
index 09c5557..e080a39 100644
--- a/arch/x86/include/asm/io.h
+++ b/arch/x86/include/asm/io.h
@@ -386,4 +386,7 @@ extern bool arch_memremap_can_ram_remap(resource_size_t offset,
 					unsigned long flags);
 #define arch_memremap_can_ram_remap arch_memremap_can_ram_remap
 
+extern bool phys_mem_access_encrypted(unsigned long phys_addr,
+				      unsigned long size);
+
 #endif /* _ASM_X86_IO_H */
diff --git a/arch/x86/mm/ioremap.c b/arch/x86/mm/ioremap.c
index 0254b78..5d8b3cf 100644
--- a/arch/x86/mm/ioremap.c
+++ b/arch/x86/mm/ioremap.c
@@ -404,12 +404,10 @@ void *xlate_dev_mem_ptr(phys_addr_t phys)
 	unsigned long offset = phys & ~PAGE_MASK;
 	void *vaddr;
 
-	/* If page is RAM, we can use __va. Otherwise ioremap and unmap. */
-	if (page_is_ram(start >> PAGE_SHIFT))
-		return __va(phys);
+	/* memremap() maps if RAM, otherwise falls back to ioremap() */
+	vaddr = memremap(start, PAGE_SIZE, MEMREMAP_WB);
 
-	vaddr = ioremap_cache(start, PAGE_SIZE);
-	/* Only add the offset on success and return NULL if the ioremap() failed: */
+	/* Only add the offset on success and return NULL if memremap() failed */
 	if (vaddr)
 		vaddr += offset;
 
@@ -418,10 +416,7 @@ void *xlate_dev_mem_ptr(phys_addr_t phys)
 
 void unxlate_dev_mem_ptr(phys_addr_t phys, void *addr)
 {
-	if (page_is_ram(phys >> PAGE_SHIFT))
-		return;
-
-	iounmap((void __iomem *)((unsigned long)addr & PAGE_MASK));
+	memunmap((void *)((unsigned long)addr & PAGE_MASK));
 }
 
 /*
@@ -630,6 +625,11 @@ pgprot_t __init early_memremap_pgprot_adjust(resource_size_t phys_addr,
 	return prot;
 }
 
+bool phys_mem_access_encrypted(unsigned long phys_addr, unsigned long size)
+{
+	return arch_memremap_can_ram_remap(phys_addr, size, 0);
+}
+
 #ifdef CONFIG_ARCH_USE_MEMREMAP_PROT
 /* Remap memory with encryption */
 void __init *early_memremap_encrypted(resource_size_t phys_addr,
diff --git a/arch/x86/mm/pat.c b/arch/x86/mm/pat.c
index 6753d9c..b970c95 100644
--- a/arch/x86/mm/pat.c
+++ b/arch/x86/mm/pat.c
@@ -748,6 +748,9 @@ void arch_io_free_memtype_wc(resource_size_t start, resource_size_t size)
 pgprot_t phys_mem_access_prot(struct file *file, unsigned long pfn,
 				unsigned long size, pgprot_t vma_prot)
 {
+	if (!phys_mem_access_encrypted(pfn << PAGE_SHIFT, size))
+		vma_prot = pgprot_decrypted(vma_prot);
+
 	return vma_prot;
 }
 



From thomas.lendacky at amd.com  Fri Jun 16 11:56:30 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Fri, 16 Jun 2017 13:56:30 -0500
Subject: [PATCH v7 35/36] x86/boot: Add early cmdline parsing for options
 with arguments
In-Reply-To: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
References: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170616185630.18967.80046.stgit@tlendack-t1.amdoffice.net>

Add a cmdline_find_option() function to look for cmdline options that
take arguments. The argument is returned in a supplied buffer and the
argument length (regardless of whether it fits in the supplied buffer)
is returned, with -1 indicating not found.

Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/include/asm/cmdline.h |    2 +
 arch/x86/lib/cmdline.c         |  105 ++++++++++++++++++++++++++++++++++++++++
 2 files changed, 107 insertions(+)

diff --git a/arch/x86/include/asm/cmdline.h b/arch/x86/include/asm/cmdline.h
index e01f7f7..84ae170 100644
--- a/arch/x86/include/asm/cmdline.h
+++ b/arch/x86/include/asm/cmdline.h
@@ -2,5 +2,7 @@
 #define _ASM_X86_CMDLINE_H
 
 int cmdline_find_option_bool(const char *cmdline_ptr, const char *option);
+int cmdline_find_option(const char *cmdline_ptr, const char *option,
+			char *buffer, int bufsize);
 
 #endif /* _ASM_X86_CMDLINE_H */
diff --git a/arch/x86/lib/cmdline.c b/arch/x86/lib/cmdline.c
index 5cc78bf..3261abb 100644
--- a/arch/x86/lib/cmdline.c
+++ b/arch/x86/lib/cmdline.c
@@ -104,7 +104,112 @@ static inline int myisspace(u8 c)
 	return 0;	/* Buffer overrun */
 }
 
+/*
+ * Find a non-boolean option (i.e. option=argument). In accordance with
+ * standard Linux practice, if this option is repeated, this returns the
+ * last instance on the command line.
+ *
+ * @cmdline: the cmdline string
+ * @max_cmdline_size: the maximum size of cmdline
+ * @option: option string to look for
+ * @buffer: memory buffer to return the option argument
+ * @bufsize: size of the supplied memory buffer
+ *
+ * Returns the length of the argument (regardless of if it was
+ * truncated to fit in the buffer), or -1 on not found.
+ */
+static int
+__cmdline_find_option(const char *cmdline, int max_cmdline_size,
+		      const char *option, char *buffer, int bufsize)
+{
+	char c;
+	int pos = 0, len = -1;
+	const char *opptr = NULL;
+	char *bufptr = buffer;
+	enum {
+		st_wordstart = 0,	/* Start of word/after whitespace */
+		st_wordcmp,	/* Comparing this word */
+		st_wordskip,	/* Miscompare, skip */
+		st_bufcpy,	/* Copying this to buffer */
+	} state = st_wordstart;
+
+	if (!cmdline)
+		return -1;      /* No command line */
+
+	/*
+	 * This 'pos' check ensures we do not overrun
+	 * a non-NULL-terminated 'cmdline'
+	 */
+	while (pos++ < max_cmdline_size) {
+		c = *(char *)cmdline++;
+		if (!c)
+			break;
+
+		switch (state) {
+		case st_wordstart:
+			if (myisspace(c))
+				break;
+
+			state = st_wordcmp;
+			opptr = option;
+			/* fall through */
+
+		case st_wordcmp:
+			if ((c == '=') && !*opptr) {
+				/*
+				 * We matched all the way to the end of the
+				 * option we were looking for, prepare to
+				 * copy the argument.
+				 */
+				len = 0;
+				bufptr = buffer;
+				state = st_bufcpy;
+				break;
+			} else if (c == *opptr++) {
+				/*
+				 * We are currently matching, so continue
+				 * to the next character on the cmdline.
+				 */
+				break;
+			}
+			state = st_wordskip;
+			/* fall through */
+
+		case st_wordskip:
+			if (myisspace(c))
+				state = st_wordstart;
+			break;
+
+		case st_bufcpy:
+			if (myisspace(c)) {
+				state = st_wordstart;
+			} else {
+				/*
+				 * Increment len, but don't overrun the
+				 * supplied buffer and leave room for the
+				 * NULL terminator.
+				 */
+				if (++len < bufsize)
+					*bufptr++ = c;
+			}
+			break;
+		}
+	}
+
+	if (bufsize)
+		*bufptr = '\0';
+
+	return len;
+}
+
 int cmdline_find_option_bool(const char *cmdline, const char *option)
 {
 	return __cmdline_find_option_bool(cmdline, COMMAND_LINE_SIZE, option);
 }
+
+int cmdline_find_option(const char *cmdline, const char *option, char *buffer,
+			int bufsize)
+{
+	return __cmdline_find_option(cmdline, COMMAND_LINE_SIZE, option,
+				     buffer, bufsize);
+}



From thomas.lendacky at amd.com  Fri Jun 16 11:56:19 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Fri, 16 Jun 2017 13:56:19 -0500
Subject: [PATCH v7 34/36] x86/mm: Add support to encrypt the kernel in-place
In-Reply-To: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
References: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170616185619.18967.38945.stgit@tlendack-t1.amdoffice.net>

Add the support to encrypt the kernel in-place. This is done by creating
new page mappings for the kernel - a decrypted write-protected mapping
and an encrypted mapping. The kernel is encrypted by copying it through
a temporary buffer.

Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/include/asm/mem_encrypt.h |    6 +
 arch/x86/mm/Makefile               |    2 
 arch/x86/mm/mem_encrypt.c          |  314 ++++++++++++++++++++++++++++++++++++
 arch/x86/mm/mem_encrypt_boot.S     |  150 +++++++++++++++++
 4 files changed, 472 insertions(+)
 create mode 100644 arch/x86/mm/mem_encrypt_boot.S

diff --git a/arch/x86/include/asm/mem_encrypt.h b/arch/x86/include/asm/mem_encrypt.h
index af835cf..7da6de3 100644
--- a/arch/x86/include/asm/mem_encrypt.h
+++ b/arch/x86/include/asm/mem_encrypt.h
@@ -21,6 +21,12 @@
 
 extern unsigned long sme_me_mask;
 
+void sme_encrypt_execute(unsigned long encrypted_kernel_vaddr,
+			 unsigned long decrypted_kernel_vaddr,
+			 unsigned long kernel_len,
+			 unsigned long encryption_wa,
+			 unsigned long encryption_pgd);
+
 void __init sme_early_encrypt(resource_size_t paddr,
 			      unsigned long size);
 void __init sme_early_decrypt(resource_size_t paddr,
diff --git a/arch/x86/mm/Makefile b/arch/x86/mm/Makefile
index 9e13841..0633142 100644
--- a/arch/x86/mm/Makefile
+++ b/arch/x86/mm/Makefile
@@ -38,3 +38,5 @@ obj-$(CONFIG_NUMA_EMU)		+= numa_emulation.o
 obj-$(CONFIG_X86_INTEL_MPX)	+= mpx.o
 obj-$(CONFIG_X86_INTEL_MEMORY_PROTECTION_KEYS) += pkeys.o
 obj-$(CONFIG_RANDOMIZE_MEMORY) += kaslr.o
+
+obj-$(CONFIG_AMD_MEM_ENCRYPT)	+= mem_encrypt_boot.o
diff --git a/arch/x86/mm/mem_encrypt.c b/arch/x86/mm/mem_encrypt.c
index 842c8a6..6e87662 100644
--- a/arch/x86/mm/mem_encrypt.c
+++ b/arch/x86/mm/mem_encrypt.c
@@ -24,6 +24,8 @@
 #include <asm/setup.h>
 #include <asm/bootparam.h>
 #include <asm/set_memory.h>
+#include <asm/cacheflush.h>
+#include <asm/sections.h>
 
 /*
  * Since SME related variables are set early in the boot process they must
@@ -209,8 +211,320 @@ void swiotlb_set_mem_attributes(void *vaddr, unsigned long size)
 	set_memory_decrypted((unsigned long)vaddr, size >> PAGE_SHIFT);
 }
 
+static void __init sme_clear_pgd(pgd_t *pgd_base, unsigned long start,
+				 unsigned long end)
+{
+	unsigned long pgd_start, pgd_end, pgd_size;
+	pgd_t *pgd_p;
+
+	pgd_start = start & PGDIR_MASK;
+	pgd_end = end & PGDIR_MASK;
+
+	pgd_size = (((pgd_end - pgd_start) / PGDIR_SIZE) + 1);
+	pgd_size *= sizeof(pgd_t);
+
+	pgd_p = pgd_base + pgd_index(start);
+
+	memset(pgd_p, 0, pgd_size);
+}
+
+#ifndef CONFIG_X86_5LEVEL
+#define native_make_p4d(_x)	(p4d_t) { .pgd = native_make_pgd(_x) }
+#endif
+
+#define PGD_FLAGS	_KERNPG_TABLE_NOENC
+#define P4D_FLAGS	_KERNPG_TABLE_NOENC
+#define PUD_FLAGS	_KERNPG_TABLE_NOENC
+#define PMD_FLAGS	(__PAGE_KERNEL_LARGE_EXEC & ~_PAGE_GLOBAL)
+
+static void __init *sme_populate_pgd(pgd_t *pgd_base, void *pgtable_area,
+				     unsigned long vaddr, pmdval_t pmd_val)
+{
+	pgd_t *pgd_p;
+	p4d_t *p4d_p;
+	pud_t *pud_p;
+	pmd_t *pmd_p;
+
+	pgd_p = pgd_base + pgd_index(vaddr);
+	if (native_pgd_val(*pgd_p)) {
+		if (IS_ENABLED(CONFIG_X86_5LEVEL))
+			p4d_p = (p4d_t *)(native_pgd_val(*pgd_p) & ~PTE_FLAGS_MASK);
+		else
+			pud_p = (pud_t *)(native_pgd_val(*pgd_p) & ~PTE_FLAGS_MASK);
+	} else {
+		pgd_t pgd;
+
+		if (IS_ENABLED(CONFIG_X86_5LEVEL)) {
+			p4d_p = pgtable_area;
+			memset(p4d_p, 0, sizeof(*p4d_p) * PTRS_PER_P4D);
+			pgtable_area += sizeof(*p4d_p) * PTRS_PER_P4D;
+
+			pgd = native_make_pgd((pgdval_t)p4d_p + PGD_FLAGS);
+		} else {
+			pud_p = pgtable_area;
+			memset(pud_p, 0, sizeof(*pud_p) * PTRS_PER_PUD);
+			pgtable_area += sizeof(*pud_p) * PTRS_PER_PUD;
+
+			pgd = native_make_pgd((pgdval_t)pud_p + PGD_FLAGS);
+		}
+		native_set_pgd(pgd_p, pgd);
+	}
+
+	if (IS_ENABLED(CONFIG_X86_5LEVEL)) {
+		p4d_p += p4d_index(vaddr);
+		if (native_p4d_val(*p4d_p)) {
+			pud_p = (pud_t *)(native_p4d_val(*p4d_p) & ~PTE_FLAGS_MASK);
+		} else {
+			p4d_t p4d;
+
+			pud_p = pgtable_area;
+			memset(pud_p, 0, sizeof(*pud_p) * PTRS_PER_PUD);
+			pgtable_area += sizeof(*pud_p) * PTRS_PER_PUD;
+
+			p4d = native_make_p4d((p4dval_t)pud_p + P4D_FLAGS);
+			native_set_p4d(p4d_p, p4d);
+		}
+	}
+
+	pud_p += pud_index(vaddr);
+	if (native_pud_val(*pud_p)) {
+		if (native_pud_val(*pud_p) & _PAGE_PSE)
+			goto out;
+
+		pmd_p = (pmd_t *)(native_pud_val(*pud_p) & ~PTE_FLAGS_MASK);
+	} else {
+		pud_t pud;
+
+		pmd_p = pgtable_area;
+		memset(pmd_p, 0, sizeof(*pmd_p) * PTRS_PER_PMD);
+		pgtable_area += sizeof(*pmd_p) * PTRS_PER_PMD;
+
+		pud = native_make_pud((pudval_t)pmd_p + PUD_FLAGS);
+		native_set_pud(pud_p, pud);
+	}
+
+	pmd_p += pmd_index(vaddr);
+	if (!native_pmd_val(*pmd_p) || !(native_pmd_val(*pmd_p) & _PAGE_PSE))
+		native_set_pmd(pmd_p, native_make_pmd(pmd_val));
+
+out:
+	return pgtable_area;
+}
+
+static unsigned long __init sme_pgtable_calc(unsigned long len)
+{
+	unsigned long p4d_size, pud_size, pmd_size;
+	unsigned long total;
+
+	/*
+	 * Perform a relatively simplistic calculation of the pagetable
+	 * entries that are needed. That mappings will be covered by 2MB
+	 * PMD entries so we can conservatively calculate the required
+	 * number of P4D, PUD and PMD structures needed to perform the
+	 * mappings. Incrementing the count for each covers the case where
+	 * the addresses cross entries.
+	 */
+	if (IS_ENABLED(CONFIG_X86_5LEVEL)) {
+		p4d_size = (ALIGN(len, PGDIR_SIZE) / PGDIR_SIZE) + 1;
+		p4d_size *= sizeof(p4d_t) * PTRS_PER_P4D;
+		pud_size = (ALIGN(len, P4D_SIZE) / P4D_SIZE) + 1;
+		pud_size *= sizeof(pud_t) * PTRS_PER_PUD;
+	} else {
+		p4d_size = 0;
+		pud_size = (ALIGN(len, PGDIR_SIZE) / PGDIR_SIZE) + 1;
+		pud_size *= sizeof(pud_t) * PTRS_PER_PUD;
+	}
+	pmd_size = (ALIGN(len, PUD_SIZE) / PUD_SIZE) + 1;
+	pmd_size *= sizeof(pmd_t) * PTRS_PER_PMD;
+
+	total = p4d_size + pud_size + pmd_size;
+
+	/*
+	 * Now calculate the added pagetable structures needed to populate
+	 * the new pagetables.
+	 */
+	if (IS_ENABLED(CONFIG_X86_5LEVEL)) {
+		p4d_size = ALIGN(total, PGDIR_SIZE) / PGDIR_SIZE;
+		p4d_size *= sizeof(p4d_t) * PTRS_PER_P4D;
+		pud_size = ALIGN(total, P4D_SIZE) / P4D_SIZE;
+		pud_size *= sizeof(pud_t) * PTRS_PER_PUD;
+	} else {
+		p4d_size = 0;
+		pud_size = ALIGN(total, PGDIR_SIZE) / PGDIR_SIZE;
+		pud_size *= sizeof(pud_t) * PTRS_PER_PUD;
+	}
+	pmd_size = ALIGN(total, PUD_SIZE) / PUD_SIZE;
+	pmd_size *= sizeof(pmd_t) * PTRS_PER_PMD;
+
+	total += p4d_size + pud_size + pmd_size;
+
+	return total;
+}
+
 void __init sme_encrypt_kernel(void)
 {
+	unsigned long workarea_start, workarea_end, workarea_len;
+	unsigned long execute_start, execute_end, execute_len;
+	unsigned long kernel_start, kernel_end, kernel_len;
+	unsigned long pgtable_area_len;
+	unsigned long paddr, pmd_flags;
+	unsigned long decrypted_base;
+	void *pgtable_area;
+	pgd_t *pgd;
+
+	if (!sme_active())
+		return;
+
+	/*
+	 * Prepare for encrypting the kernel by building new pagetables with
+	 * the necessary attributes needed to encrypt the kernel in place.
+	 *
+	 *   One range of virtual addresses will map the memory occupied
+	 *   by the kernel as encrypted.
+	 *
+	 *   Another range of virtual addresses will map the memory occupied
+	 *   by the kernel as decrypted and write-protected.
+	 *
+	 *     The use of write-protect attribute will prevent any of the
+	 *     memory from being cached.
+	 */
+
+	/* Physical addresses gives us the identity mapped virtual addresses */
+	kernel_start = __pa_symbol(_text);
+	kernel_end = ALIGN(__pa_symbol(_end), PMD_PAGE_SIZE);
+	kernel_len = kernel_end - kernel_start;
+
+	/* Set the encryption workarea to be immediately after the kernel */
+	workarea_start = kernel_end;
+
+	/*
+	 * Calculate required number of workarea bytes needed:
+	 *   executable encryption area size:
+	 *     stack page (PAGE_SIZE)
+	 *     encryption routine page (PAGE_SIZE)
+	 *     intermediate copy buffer (PMD_PAGE_SIZE)
+	 *   pagetable structures for the encryption of the kernel
+	 *   pagetable structures for workarea (in case not currently mapped)
+	 */
+	execute_start = workarea_start;
+	execute_end = execute_start + (PAGE_SIZE * 2) + PMD_PAGE_SIZE;
+	execute_len = execute_end - execute_start;
+
+	/*
+	 * One PGD for both encrypted and decrypted mappings and a set of
+	 * PUDs and PMDs for each of the encrypted and decrypted mappings.
+	 */
+	pgtable_area_len = sizeof(pgd_t) * PTRS_PER_PGD;
+	pgtable_area_len += sme_pgtable_calc(execute_end - kernel_start) * 2;
+
+	/* PUDs and PMDs needed in the current pagetables for the workarea */
+	pgtable_area_len += sme_pgtable_calc(execute_len + pgtable_area_len);
+
+	/*
+	 * The total workarea includes the executable encryption area and
+	 * the pagetable area.
+	 */
+	workarea_len = execute_len + pgtable_area_len;
+	workarea_end = workarea_start + workarea_len;
+
+	/*
+	 * Set the address to the start of where newly created pagetable
+	 * structures (PGDs, PUDs and PMDs) will be allocated. New pagetable
+	 * structures are created when the workarea is added to the current
+	 * pagetables and when the new encrypted and decrypted kernel
+	 * mappings are populated.
+	 */
+	pgtable_area = (void *)execute_end;
+
+	/*
+	 * Make sure the current pagetable structure has entries for
+	 * addressing the workarea.
+	 */
+	pgd = (pgd_t *)native_read_cr3_pa();
+	paddr = workarea_start;
+	while (paddr < workarea_end) {
+		pgtable_area = sme_populate_pgd(pgd, pgtable_area,
+						paddr,
+						paddr + PMD_FLAGS);
+
+		paddr += PMD_PAGE_SIZE;
+	}
+
+	/* Flush the TLB - no globals so cr3 is enough */
+	native_write_cr3(__native_read_cr3());
+
+	/*
+	 * A new pagetable structure is being built to allow for the kernel
+	 * to be encrypted. It starts with an empty PGD that will then be
+	 * populated with new PUDs and PMDs as the encrypted and decrypted
+	 * kernel mappings are created.
+	 */
+	pgd = pgtable_area;
+	memset(pgd, 0, sizeof(*pgd) * PTRS_PER_PGD);
+	pgtable_area += sizeof(*pgd) * PTRS_PER_PGD;
+
+	/* Add encrypted kernel (identity) mappings */
+	pmd_flags = PMD_FLAGS | _PAGE_ENC;
+	paddr = kernel_start;
+	while (paddr < kernel_end) {
+		pgtable_area = sme_populate_pgd(pgd, pgtable_area,
+						paddr,
+						paddr + pmd_flags);
+
+		paddr += PMD_PAGE_SIZE;
+	}
+
+	/*
+	 * A different PGD index/entry must be used to get different
+	 * pagetable entries for the decrypted mapping. Choose the next
+	 * PGD index and convert it to a virtual address to be used as
+	 * the base of the mapping.
+	 */
+	decrypted_base = (pgd_index(workarea_end) + 1) & (PTRS_PER_PGD - 1);
+	decrypted_base <<= PGDIR_SHIFT;
+
+	/* Add decrypted, write-protected kernel (non-identity) mappings */
+	pmd_flags = (PMD_FLAGS & ~_PAGE_CACHE_MASK) | (_PAGE_PAT | _PAGE_PWT);
+	paddr = kernel_start;
+	while (paddr < kernel_end) {
+		pgtable_area = sme_populate_pgd(pgd, pgtable_area,
+						paddr + decrypted_base,
+						paddr + pmd_flags);
+
+		paddr += PMD_PAGE_SIZE;
+	}
+
+	/* Add decrypted workarea mappings to both kernel mappings */
+	paddr = workarea_start;
+	while (paddr < workarea_end) {
+		pgtable_area = sme_populate_pgd(pgd, pgtable_area,
+						paddr,
+						paddr + PMD_FLAGS);
+
+		pgtable_area = sme_populate_pgd(pgd, pgtable_area,
+						paddr + decrypted_base,
+						paddr + PMD_FLAGS);
+
+		paddr += PMD_PAGE_SIZE;
+	}
+
+	/* Perform the encryption */
+	sme_encrypt_execute(kernel_start, kernel_start + decrypted_base,
+			    kernel_len, workarea_start, (unsigned long)pgd);
+
+	/*
+	 * At this point we are running encrypted.  Remove the mappings for
+	 * the decrypted areas - all that is needed for this is to remove
+	 * the PGD entry/entries.
+	 */
+	sme_clear_pgd(pgd, kernel_start + decrypted_base,
+		      kernel_end + decrypted_base);
+
+	sme_clear_pgd(pgd, workarea_start + decrypted_base,
+		      workarea_end + decrypted_base);
+
+	/* Flush the TLB - no globals so cr3 is enough */
+	native_write_cr3(__native_read_cr3());
 }
 
 void __init sme_enable(void)
diff --git a/arch/x86/mm/mem_encrypt_boot.S b/arch/x86/mm/mem_encrypt_boot.S
new file mode 100644
index 0000000..7720b00
--- /dev/null
+++ b/arch/x86/mm/mem_encrypt_boot.S
@@ -0,0 +1,150 @@
+/*
+ * AMD Memory Encryption Support
+ *
+ * Copyright (C) 2016 Advanced Micro Devices, Inc.
+ *
+ * Author: Tom Lendacky <thomas.lendacky at amd.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include <linux/linkage.h>
+#include <asm/pgtable.h>
+#include <asm/page.h>
+#include <asm/processor-flags.h>
+#include <asm/msr-index.h>
+
+	.text
+	.code64
+ENTRY(sme_encrypt_execute)
+
+	/*
+	 * Entry parameters:
+	 *   RDI - virtual address for the encrypted kernel mapping
+	 *   RSI - virtual address for the decrypted kernel mapping
+	 *   RDX - length of kernel
+	 *   RCX - virtual address of the encryption workarea, including:
+	 *     - stack page (PAGE_SIZE)
+	 *     - encryption routine page (PAGE_SIZE)
+	 *     - intermediate copy buffer (PMD_PAGE_SIZE)
+	 *    R8 - physcial address of the pagetables to use for encryption
+	 */
+
+	push	%rbp
+	push	%r12
+
+	/* Set up a one page stack in the non-encrypted memory area */
+	movq	%rsp, %rbp		/* Save current stack pointer */
+	movq	%rcx, %rax		/* Workarea stack page */
+	movq	%rax, %rsp		/* Set new stack pointer */
+	addq	$PAGE_SIZE, %rsp	/* Stack grows from the bottom */
+	addq	$PAGE_SIZE, %rax	/* Workarea encryption routine */
+
+	movq	%rdi, %r10		/* Encrypted kernel */
+	movq	%rsi, %r11		/* Decrypted kernel */
+	movq	%rdx, %r12		/* Kernel length */
+
+	/* Copy encryption routine into the workarea */
+	movq	%rax, %rdi				/* Workarea encryption routine */
+	leaq	__enc_copy(%rip), %rsi			/* Encryption routine */
+	movq	$(.L__enc_copy_end - __enc_copy), %rcx	/* Encryption routine length */
+	rep	movsb
+
+	/* Setup registers for call */
+	movq	%r10, %rdi		/* Encrypted kernel */
+	movq	%r11, %rsi		/* Decrypted kernel */
+	movq	%r8, %rdx		/* Pagetables used for encryption */
+	movq	%r12, %rcx		/* Kernel length */
+	movq	%rax, %r8		/* Workarea encryption routine */
+	addq	$PAGE_SIZE, %r8		/* Workarea intermediate copy buffer */
+
+	call	*%rax			/* Call the encryption routine */
+
+	movq	%rbp, %rsp		/* Restore original stack pointer */
+
+	pop	%r12
+	pop	%rbp
+
+	ret
+ENDPROC(sme_encrypt_execute)
+
+ENTRY(__enc_copy)
+/*
+ * Routine used to encrypt kernel.
+ *   This routine must be run outside of the kernel proper since
+ *   the kernel will be encrypted during the process. So this
+ *   routine is defined here and then copied to an area outside
+ *   of the kernel where it will remain and run decrypted
+ *   during execution.
+ *
+ *   On entry the registers must be:
+ *     RDI - virtual address for the encrypted kernel mapping
+ *     RSI - virtual address for the decrypted kernel mapping
+ *     RDX - address of the pagetables to use for encryption
+ *     RCX - length of kernel
+ *      R8 - intermediate copy buffer
+ *
+ *     RAX - points to this routine
+ *
+ * The kernel will be encrypted by copying from the non-encrypted
+ * kernel space to an intermediate buffer and then copying from the
+ * intermediate buffer back to the encrypted kernel space. The physical
+ * addresses of the two kernel space mappings are the same which
+ * results in the kernel being encrypted "in place".
+ */
+	/* Enable the new page tables */
+	mov	%rdx, %cr3
+
+	/* Flush any global TLBs */
+	mov	%cr4, %rdx
+	andq	$~X86_CR4_PGE, %rdx
+	mov	%rdx, %cr4
+	orq	$X86_CR4_PGE, %rdx
+	mov	%rdx, %cr4
+
+	/* Set the PAT register PA5 entry to write-protect */
+	push	%rcx
+	movl	$MSR_IA32_CR_PAT, %ecx
+	rdmsr
+	push	%rdx			/* Save original PAT value */
+	andl	$0xffff00ff, %edx	/* Clear PA5 */
+	orl	$0x00000500, %edx	/* Set PA5 to WP */
+	wrmsr
+	pop	%rdx			/* RDX contains original PAT value */
+	pop	%rcx
+
+	movq	%rcx, %r9		/* Save kernel length */
+	movq	%rdi, %r10		/* Save encrypted kernel address */
+	movq	%rsi, %r11		/* Save decrypted kernel address */
+
+	wbinvd				/* Invalidate any cache entries */
+
+	/* Copy/encrypt 2MB at a time */
+1:
+	movq	%r11, %rsi		/* Source - decrypted kernel */
+	movq	%r8, %rdi		/* Dest   - intermediate copy buffer */
+	movq	$PMD_PAGE_SIZE, %rcx	/* 2MB length */
+	rep	movsb
+
+	movq	%r8, %rsi		/* Source - intermediate copy buffer */
+	movq	%r10, %rdi		/* Dest   - encrypted kernel */
+	movq	$PMD_PAGE_SIZE, %rcx	/* 2MB length */
+	rep	movsb
+
+	addq	$PMD_PAGE_SIZE, %r11
+	addq	$PMD_PAGE_SIZE, %r10
+	subq	$PMD_PAGE_SIZE, %r9	/* Kernel length decrement */
+	jnz	1b			/* Kernel length not zero? */
+
+	/* Restore PAT register */
+	push	%rdx			/* Save original PAT value */
+	movl	$MSR_IA32_CR_PAT, %ecx
+	rdmsr
+	pop	%rdx			/* Restore original PAT value */
+	wrmsr
+
+	ret
+.L__enc_copy_end:
+ENDPROC(__enc_copy)



From thomas.lendacky at amd.com  Fri Jun 16 11:56:39 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Fri, 16 Jun 2017 13:56:39 -0500
Subject: [PATCH v7 36/36] x86/mm: Add support to make use of Secure Memory
 Encryption
In-Reply-To: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
References: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170616185639.18967.41488.stgit@tlendack-t1.amdoffice.net>

Add support to check if SME has been enabled and if memory encryption
should be activated (checking of command line option based on the
configuration of the default state).  If memory encryption is to be
activated, then the encryption mask is set and the kernel is encrypted
"in place."

Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/include/asm/mem_encrypt.h |    6 ++-
 arch/x86/kernel/head64.c           |    4 +-
 arch/x86/mm/mem_encrypt.c          |   86 +++++++++++++++++++++++++++++++++++-
 3 files changed, 90 insertions(+), 6 deletions(-)

diff --git a/arch/x86/include/asm/mem_encrypt.h b/arch/x86/include/asm/mem_encrypt.h
index 7da6de3..aac9ed9 100644
--- a/arch/x86/include/asm/mem_encrypt.h
+++ b/arch/x86/include/asm/mem_encrypt.h
@@ -17,6 +17,8 @@
 
 #include <linux/init.h>
 
+#include <asm/bootparam.h>
+
 #ifdef CONFIG_AMD_MEM_ENCRYPT
 
 extern unsigned long sme_me_mask;
@@ -37,7 +39,7 @@ void __init sme_early_decrypt(resource_size_t paddr,
 
 void __init sme_early_init(void);
 
-void __init sme_enable(void);
+void __init sme_enable(struct boot_params *bp);
 
 /* Architecture __weak replacement functions */
 void __init mem_encrypt_init(void);
@@ -58,7 +60,7 @@ static inline void __init sme_unmap_bootdata(char *real_mode_data) { }
 
 static inline void __init sme_early_init(void) { }
 
-static inline void __init sme_enable(void) { }
+static inline void __init sme_enable(struct boot_params *bp) { }
 
 #endif	/* CONFIG_AMD_MEM_ENCRYPT */
 
diff --git a/arch/x86/kernel/head64.c b/arch/x86/kernel/head64.c
index 9e94ed2..1ff2e98 100644
--- a/arch/x86/kernel/head64.c
+++ b/arch/x86/kernel/head64.c
@@ -43,7 +43,7 @@ static void __init *fixup_pointer(void *ptr, unsigned long physaddr)
 	return ptr - (void *)_text + (void *)physaddr;
 }
 
-void __init __startup_64(unsigned long physaddr)
+void __init __startup_64(unsigned long physaddr, struct boot_params *bp)
 {
 	unsigned long load_delta, *p;
 	unsigned long pgtable_flags;
@@ -68,7 +68,7 @@ void __init __startup_64(unsigned long physaddr)
 		for (;;);
 
 	/* Activate Secure Memory Encryption (SME) if supported and enabled */
-	sme_enable();
+	sme_enable(bp);
 
 	/* Include the SME encryption mask in the fixup value */
 	load_delta += sme_get_me_mask();
diff --git a/arch/x86/mm/mem_encrypt.c b/arch/x86/mm/mem_encrypt.c
index 6e87662..13f780e 100644
--- a/arch/x86/mm/mem_encrypt.c
+++ b/arch/x86/mm/mem_encrypt.c
@@ -13,19 +13,34 @@
 #include <linux/linkage.h>
 #include <linux/init.h>
 
+#include <asm/bootparam.h>
+
 #ifdef CONFIG_AMD_MEM_ENCRYPT
 
 #include <linux/mm.h>
 #include <linux/dma-mapping.h>
 #include <linux/swiotlb.h>
+#include <linux/mem_encrypt.h>
 
 #include <asm/tlbflush.h>
 #include <asm/fixmap.h>
 #include <asm/setup.h>
-#include <asm/bootparam.h>
 #include <asm/set_memory.h>
 #include <asm/cacheflush.h>
 #include <asm/sections.h>
+#include <asm/processor-flags.h>
+#include <asm/msr.h>
+#include <asm/cmdline.h>
+
+/*
+ * Some SME functions run very early causing issues with the stack-protector
+ * support. Provide a way to turn off this support on a per-function basis.
+ */
+#define SME_NOSTACKP __attribute__((__optimize__("no-stack-protector")))
+
+static char sme_cmdline_arg[] __initdata = "mem_encrypt";
+static char sme_cmdline_on[]  __initdata = "on";
+static char sme_cmdline_off[] __initdata = "off";
 
 /*
  * Since SME related variables are set early in the boot process they must
@@ -200,6 +215,8 @@ void __init mem_encrypt_init(void)
 
 	/* Call into SWIOTLB to update the SWIOTLB DMA buffers */
 	swiotlb_update_mem_attributes();
+
+	pr_info("AMD Secure Memory Encryption (SME) active\n");
 }
 
 void swiotlb_set_mem_attributes(void *vaddr, unsigned long size)
@@ -527,8 +544,73 @@ void __init sme_encrypt_kernel(void)
 	native_write_cr3(__native_read_cr3());
 }
 
-void __init sme_enable(void)
+void __init SME_NOSTACKP sme_enable(struct boot_params *bp)
 {
+	const char *cmdline_ptr, *cmdline_arg, *cmdline_on, *cmdline_off;
+	unsigned int eax, ebx, ecx, edx;
+	bool active_by_default;
+	unsigned long me_mask;
+	char buffer[16];
+	u64 msr;
+
+	/* Check for the SME support leaf */
+	eax = 0x80000000;
+	ecx = 0;
+	native_cpuid(&eax, &ebx, &ecx, &edx);
+	if (eax < 0x8000001f)
+		return;
+
+	/*
+	 * Check for the SME feature:
+	 *   CPUID Fn8000_001F[EAX] - Bit 0
+	 *     Secure Memory Encryption support
+	 *   CPUID Fn8000_001F[EBX] - Bits 5:0
+	 *     Pagetable bit position used to indicate encryption
+	 */
+	eax = 0x8000001f;
+	ecx = 0;
+	native_cpuid(&eax, &ebx, &ecx, &edx);
+	if (!(eax & 1))
+		return;
+
+	me_mask = 1UL << (ebx & 0x3f);
+
+	/* Check if SME is enabled */
+	msr = __rdmsr(MSR_K8_SYSCFG);
+	if (!(msr & MSR_K8_SYSCFG_MEM_ENCRYPT))
+		return;
+
+	/*
+	 * Fixups have not been applied to phys_base yet and we're running
+	 * identity mapped, so we must obtain the address to the SME command
+	 * line argument data using rip-relative addressing.
+	 */
+	asm ("lea sme_cmdline_arg(%%rip), %0"
+	     : "=r" (cmdline_arg)
+	     : "p" (sme_cmdline_arg));
+	asm ("lea sme_cmdline_on(%%rip), %0"
+	     : "=r" (cmdline_on)
+	     : "p" (sme_cmdline_on));
+	asm ("lea sme_cmdline_off(%%rip), %0"
+	     : "=r" (cmdline_off)
+	     : "p" (sme_cmdline_off));
+
+	if (IS_ENABLED(CONFIG_AMD_MEM_ENCRYPT_ACTIVE_BY_DEFAULT))
+		active_by_default = true;
+	else
+		active_by_default = false;
+
+	cmdline_ptr = (const char *)((u64)bp->hdr.cmd_line_ptr |
+				     ((u64)bp->ext_cmd_line_ptr << 32));
+
+	cmdline_find_option(cmdline_ptr, cmdline_arg, buffer, sizeof(buffer));
+
+	if (strncmp(buffer, cmdline_on, sizeof(buffer)) == 0)
+		sme_me_mask = me_mask;
+	else if (strncmp(buffer, cmdline_off, sizeof(buffer)) == 0)
+		sme_me_mask = 0;
+	else
+		sme_me_mask = active_by_default ? me_mask : 0;
 }
 
 unsigned long sme_get_me_mask(void)



From jgross at suse.com  Sat Jun 17 03:40:17 2017
From: jgross at suse.com (Juergen Gross)
Date: Sat, 17 Jun 2017 12:40:17 +0200
Subject: [PATCH v7 32/36] xen/x86: Remove SME feature in PV guests
In-Reply-To: <20170616185554.18967.82909.stgit@tlendack-t1.amdoffice.net>
References: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
 <20170616185554.18967.82909.stgit@tlendack-t1.amdoffice.net>
Message-ID: <8dac5656-3d54-8b14-46ca-50072b507be3@suse.com>

On 16/06/17 20:55, Tom Lendacky wrote:
> Xen does not currently support SME for PV guests. Clear the SME cpu
> capability in order to avoid any ambiguity.
> 
> Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>

Reviewed-by: Juergen Gross <jgross at suse.com>


Juergen

> ---
>  arch/x86/xen/enlighten_pv.c |    1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/arch/x86/xen/enlighten_pv.c b/arch/x86/xen/enlighten_pv.c
> index f33eef4..e6ecf42 100644
> --- a/arch/x86/xen/enlighten_pv.c
> +++ b/arch/x86/xen/enlighten_pv.c
> @@ -294,6 +294,7 @@ static void __init xen_init_capabilities(void)
>  	setup_clear_cpu_cap(X86_FEATURE_MTRR);
>  	setup_clear_cpu_cap(X86_FEATURE_ACC);
>  	setup_clear_cpu_cap(X86_FEATURE_X2APIC);
> +	setup_clear_cpu_cap(X86_FEATURE_SME);
>  
>  	if (!xen_initial_domain())
>  		setup_clear_cpu_cap(X86_FEATURE_ACPI);
> 
> 



From lkp at intel.com  Sat Jun 17 07:08:20 2017
From: lkp at intel.com (kbuild test robot)
Date: Sat, 17 Jun 2017 22:08:20 +0800
Subject: [PATCH v7 29/36] x86, drm, fbdev: Do not specify encrypted
 memory for video mappings
In-Reply-To: <20170616185523.18967.34256.stgit@tlendack-t1.amdoffice.net>
Message-ID: <201706172210.QSL4W3Q1%fengguang.wu@intel.com>

Hi Tom,

[auto build test ERROR on next-20170616]
[cannot apply to tip/x86/core linus/master linux/master v4.9-rc8 v4.9-rc7 v4.9-rc6 v4.12-rc5]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]

url:    https://github.com/0day-ci/linux/commits/Tom-Lendacky/x86-Secure-Memory-Encryption-AMD/20170617-210824
config: sh-allmodconfig (attached as .config)
compiler: sh4-linux-gnu-gcc (Debian 6.1.1-9) 6.1.1 20160705
reproduce:
        wget https://raw.githubusercontent.com/01org/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
        chmod +x ~/bin/make.cross
        # save the attached .config to linux build tree
        make.cross ARCH=sh 

All errors (new ones prefixed by >>):

   drivers/gpu/drm/drm_gem.c: In function 'drm_gem_mmap_obj':
>> drivers/gpu/drm/drm_gem.c:932:22: error: implicit declaration of function 'pgprot_decrypted' [-Werror=implicit-function-declaration]
     vma->vm_page_prot = pgprot_decrypted(vma->vm_page_prot);
                         ^~~~~~~~~~~~~~~~
>> drivers/gpu/drm/drm_gem.c:932:20: error: incompatible types when assigning to type 'pgprot_t {aka struct <anonymous>}' from type 'int'
     vma->vm_page_prot = pgprot_decrypted(vma->vm_page_prot);
                       ^
   cc1: some warnings being treated as errors
--
   drivers/video/fbdev/core/fbmem.c: In function 'fb_mmap':
>> drivers/video/fbdev/core/fbmem.c:1414:23: error: implicit declaration of function 'pgprot_decrypted' [-Werror=implicit-function-declaration]
      vma->vm_page_prot = pgprot_decrypted(vma->vm_page_prot);
                          ^~~~~~~~~~~~~~~~
>> drivers/video/fbdev/core/fbmem.c:1414:21: error: incompatible types when assigning to type 'pgprot_t {aka struct <anonymous>}' from type 'int'
      vma->vm_page_prot = pgprot_decrypted(vma->vm_page_prot);
                        ^
   drivers/video/fbdev/core/fbmem.c:1444:20: error: incompatible types when assigning to type 'pgprot_t {aka struct <anonymous>}' from type 'int'
     vma->vm_page_prot = pgprot_decrypted(vma->vm_page_prot);
                       ^
   cc1: some warnings being treated as errors
--
   drivers/gpu/drm/udl/udl_fb.c: In function 'udl_fb_mmap':
>> drivers/gpu/drm/udl/udl_fb.c:174:22: error: implicit declaration of function 'pgprot_decrypted' [-Werror=implicit-function-declaration]
     vma->vm_page_prot = pgprot_decrypted(vma->vm_page_prot);
                         ^~~~~~~~~~~~~~~~
>> drivers/gpu/drm/udl/udl_fb.c:174:20: error: incompatible types when assigning to type 'pgprot_t {aka struct <anonymous>}' from type 'int'
     vma->vm_page_prot = pgprot_decrypted(vma->vm_page_prot);
                       ^
   cc1: some warnings being treated as errors

vim +/pgprot_decrypted +932 drivers/gpu/drm/drm_gem.c

   926			return -EINVAL;
   927	
   928		vma->vm_flags |= VM_IO | VM_PFNMAP | VM_DONTEXPAND | VM_DONTDUMP;
   929		vma->vm_ops = dev->driver->gem_vm_ops;
   930		vma->vm_private_data = obj;
   931		vma->vm_page_prot = pgprot_writecombine(vm_get_page_prot(vma->vm_flags));
 > 932		vma->vm_page_prot = pgprot_decrypted(vma->vm_page_prot);
   933	
   934		/* Take a ref for this mapping of the object, so that the fault
   935		 * handler can dereference the mmap offset's pointer to the object.

---
0-DAY kernel test infrastructure                Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all                   Intel Corporation
-------------- next part --------------
A non-text attachment was scrubbed...
Name: .config.gz
Type: application/gzip
Size: 45695 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/kexec/attachments/20170617/1c46b65d/attachment-0001.gz>

From abrs612 at gmail.com  Sun Jun 18 00:41:08 2017
From: abrs612 at gmail.com (Abhishek Shah)
Date: Sun, 18 Jun 2017 13:11:08 +0530
Subject: makedumpfile: support for newer kernels [v4.9 onwards]
In-Reply-To: <148b498a-cd13-bcb2-de04-d74b1c2a0f65@redhat.com>
References: <CAEzMDiD-w91Q+Fjy_KgXDcwo9YGasa9ggqW2dEhF4ffV7RstqQ@mail.gmail.com>
 <0bd0a35a-88cd-3f68-846f-2a2ad1e7afcb@redhat.com>
 <CAEzMDiBKdA15Gn2kgs=BgzCM=bzaBXCQP_tM_Vmfc4uxGL0SRg@mail.gmail.com>
 <0ae1e066-821e-3b11-ee44-b615245cb1ea@redhat.com>
 <3b72a75c-012e-2779-74a4-d36ca1ef9e56@redhat.com>
 <CAEzMDiARnp-FMqUcoVdCBSSApaRbenvrk7cawgyo8kJJvy0jXg@mail.gmail.com>
 <364d6133-158b-0f28-0278-dc0da33b2916@redhat.com>
 <CAEzMDiAq1U3SRkvwh5nqETDBoZs8DojhmMTDSG7B-Uzbh1a3Nw@mail.gmail.com>
 <148b498a-cd13-bcb2-de04-d74b1c2a0f65@redhat.com>
Message-ID: <CAEzMDiBU98VzS9Yye6UrCVM5-f9F64ZUqCOJPpnd+5ufdY5OsA@mail.gmail.com>

Hi Pratyush,

On Fri, Jun 16, 2017 at 2:04 PM, Pratyush Anand <panand at redhat.com> wrote:
>
>
> On Tuesday 13 June 2017 08:47 PM, Abhishek Shah wrote:
>>
>> @Pratyush, Thanks for testing and verifying the kernel configuration
>> for arm64 makedumpfile.
>> When you say things were working fine for you,  what exact
>> makedumpfile command did you use??
>
>
> makedumpfile -l --message-level 1 -d 31 /proc/vmcore /var/crash/vmcore
>
I am getting "seek error" as reported earlier if I use above command
to generate dump in my setup:
crash: seek error: kernel virtual address: ffff8008000c2020  type:
"memory section"


Regards,
Abhishek


From bp at alien8.de  Mon Jun 19 10:18:20 2017
From: bp at alien8.de (Borislav Petkov)
Date: Mon, 19 Jun 2017 19:18:20 +0200
Subject: [PATCH v6 26/34] iommu/amd: Allow the AMD IOMMU to work with
 memory encryption
In-Reply-To: <3db2c52d-5e63-a1df-edd4-975bce7f29c2@amd.com>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
 <20170607191745.28645.81756.stgit@tlendack-t1.amdoffice.net>
 <20170614174208.p2yr5exs4b6pjxhf@pd.tnic>
 <0611d01a-19f8-d6ae-2682-932789855518@amd.com>
 <20170615094111.wga334kg2bhxqib3@pd.tnic>
 <921153f5-1528-31d8-b815-f0419e819aeb@amd.com>
 <20170615153322.nwylo3dzn4fdx6n6@pd.tnic>
 <3db2c52d-5e63-a1df-edd4-975bce7f29c2@amd.com>
Message-ID: <20170619171820.tq4htttamb52pyx5@pd.tnic>

On Thu, Jun 15, 2017 at 11:33:41AM -0500, Tom Lendacky wrote:
> Changing the signature back reverts to the original way, so this can be
> looked at separate from this patchset then.

Right, the patch which added the volatile thing was this one:

  4bf5beef578e ("iommu/amd: Don't put completion-wait semaphore on stack")

and the commit message doesn't say why the thing needs to be volatile at
all.

Joerg?

-- 
Regards/Gruss,
    Boris.

Good mailing practices for 400: avoid top-posting and trim the reply.


From xiyou.wangcong at gmail.com  Mon Jun 19 15:23:14 2017
From: xiyou.wangcong at gmail.com (Cong Wang)
Date: Mon, 19 Jun 2017 15:23:14 -0700
Subject: [PATCH] vmcore-dmesg: avoid allocating large memory chunk for log buf
Message-ID: <1497910994-15129-1-git-send-email-xiyou.wangcong@gmail.com>

In kdump kernel, we got:

    Saving vmcore-dmesg.txt
    Failed to malloc 67108864 bytes for the logbuf: Cannot allocate memory

Apparently 64M is too luxury for a kdump kernel which only has 128M
in total.

We can avoid allocating such a large memory chunk, instead just allocate
a smaller chunk inside the loop each time.

Verify the result by comparing the vmcore-dmesg output with and without
this patch.

Cc: Simon Horman <horms at verge.net.au>
Signed-off-by: Cong Wang <xiyou.wangcong at gmail.com>
---
 vmcore-dmesg/vmcore-dmesg.c | 101 ++++++++++++++++++--------------------------
 1 file changed, 41 insertions(+), 60 deletions(-)

diff --git a/vmcore-dmesg/vmcore-dmesg.c b/vmcore-dmesg/vmcore-dmesg.c
index a8f56df..593557c 100644
--- a/vmcore-dmesg/vmcore-dmesg.c
+++ b/vmcore-dmesg/vmcore-dmesg.c
@@ -565,53 +565,12 @@ static inline uint64_t struct_val_u64(char *ptr, unsigned int offset)
 	return(file64_to_cpu(*(uint64_t *)(ptr + offset)));
 }
 
-/* human readable text of the record */
-static char *log_text(char *msg)
-{
-	return msg + log_sz;
-}
-
-/* get record by index; idx must point to valid msg */
-static char *log_from_idx(char *log_buf, uint32_t idx)
-{
-	char *msg = log_buf + idx;
-
-	/*
-	 * A length == 0 record is the end of buffer marker. Wrap around and
-	 * read the message at the start of the buffer.
-	 */
-	if (!struct_val_u16(msg, log_offset_len))
-		return log_buf;
-	return msg;
-}
-
-/* get next record; idx must point to valid msg */
-static uint32_t log_next(char *log_buf, uint32_t idx)
-{
-	char *msg = log_buf + idx;
-	uint16_t len;
-
-	/* length == 0 indicates the end of the buffer; wrap */
-	/*
-	 * A length == 0 record is the end of buffer marker. Wrap around and
-	 * read the message at the start of the buffer as *this* one, and
-	 * return the one after that.
-	 */
-	len = struct_val_u16(msg, log_offset_len);
-	if (!len) {
-		msg = log_buf;
-		return struct_val_u16(msg, log_offset_len);
-	}
-	return idx + len;
-}
-
 /* Read headers of log records and dump accordingly */
 static void dump_dmesg_structured(int fd)
 {
 #define OUT_BUF_SIZE	4096
 	uint64_t log_buf, log_buf_offset, ts_nsec;
 	uint32_t log_first_idx, log_next_idx, current_idx, len = 0, i;
-	int log_buf_len;
 	char *buf, out_buf[OUT_BUF_SIZE];
 	ssize_t ret;
 	char *msg;
@@ -659,34 +618,33 @@ static void dump_dmesg_structured(int fd)
 	}
 
 	log_buf = read_file_pointer(fd, vaddr_to_offset(log_buf_vaddr));
-	log_buf_len = read_file_s32(fd, vaddr_to_offset(log_buf_len_vaddr));
 
 	log_first_idx = read_file_u32(fd, vaddr_to_offset(log_first_idx_vaddr));
 	log_next_idx = read_file_u32(fd, vaddr_to_offset(log_next_idx_vaddr));
 
 	log_buf_offset = vaddr_to_offset(log_buf);
 
-	buf = calloc(1, log_buf_len);
+	buf = calloc(1, log_sz);
 	if (!buf) {
-		fprintf(stderr, "Failed to malloc %d bytes for the logbuf:"
-				" %s\n", log_buf_len, strerror(errno));
+		fprintf(stderr, "Failed to malloc %" PRId64 " bytes for the log:"
+				" %s\n", log_sz, strerror(errno));
 		exit(64);
 	}
 
-	ret = pread(fd, buf, log_buf_len, log_buf_offset);
-	if (ret != log_buf_len) {
-		fprintf(stderr, "Failed to read log buffer of size %d bytes:"
-			" %s\n", log_buf_len, strerror(errno));
-		exit(65);
-	}
-
 	/* Parse records and write out data at standard output */
 
 	current_idx = log_first_idx;
 	len = 0;
 	while (current_idx != log_next_idx) {
-		msg = log_from_idx(buf, current_idx);
-		ts_nsec = struct_val_u64(msg, log_offset_ts_nsec);
+		uint16_t loglen;
+
+		ret = pread(fd, buf, log_sz, log_buf_offset + current_idx);
+		if (ret != log_sz) {
+			fprintf(stderr, "Failed to read log of size %" PRId64 " bytes:"
+				" %s\n", log_sz, strerror(errno));
+			exit(65);
+		}
+		ts_nsec = struct_val_u64(buf, log_offset_ts_nsec);
 		imaxdiv_sec = imaxdiv(ts_nsec, 1000000000);
 		imaxdiv_usec = imaxdiv(imaxdiv_sec.rem, 1000);
 
@@ -695,9 +653,22 @@ static void dump_dmesg_structured(int fd)
 			(long long unsigned int)imaxdiv_usec.quot);
 
 		/* escape non-printable characters */
-		text_len = struct_val_u16(msg, log_offset_text_len);
+		text_len = struct_val_u16(buf, log_offset_text_len);
+		msg = calloc(1, text_len);
+		if (!msg) {
+			fprintf(stderr, "Failed to malloc %u bytes for log text:"
+				" %s\n", text_len, strerror(errno));
+			exit(64);
+		}
+
+		ret = pread(fd, msg, text_len, log_buf_offset + current_idx + log_sz);
+		if (ret != text_len) {
+			fprintf(stderr, "Failed to read log text of size %u bytes:"
+				" %s\n", text_len, strerror(errno));
+			exit(65);
+		}
 		for (i = 0; i < text_len; i++) {
-			unsigned char c = log_text(msg)[i];
+			unsigned char c = msg[i];
 
 			if (!isprint(c) && !isspace(c))
 				len += sprintf(out_buf + len, "\\x%02x", c);
@@ -711,11 +682,21 @@ static void dump_dmesg_structured(int fd)
 		}
 
 		out_buf[len++] = '\n';
-
-		/* Move to next record */
-		current_idx = log_next(buf, current_idx);
+		free(msg);
+		/* length == 0 indicates the end of the buffer; wrap */
+		/*
+		 * A length == 0 record is the end of buffer marker. Wrap around and
+		 * read the message at the start of the buffer as *this* one, and
+		 * return the one after that.
+		 */
+		loglen = struct_val_u16(buf, log_offset_len);
+		if (!loglen)
+			current_idx = log_first_idx;
+		else
+			/* Move to next record */
+			current_idx += loglen;
 	}
-
+	free(buf);
 	if (len)
 		write_to_stdout(out_buf, len);
 }
-- 
2.5.5



From bp at alien8.de  Tue Jun 20 00:21:25 2017
From: bp at alien8.de (Borislav Petkov)
Date: Tue, 20 Jun 2017 09:21:25 +0200
Subject: [PATCH v7 03/36] x86, mpparse, x86/acpi, x86/PCI, x86/dmi, SFI:
 Use memremap for RAM mappings
In-Reply-To: <20170616185023.18967.72831.stgit@tlendack-t1.amdoffice.net>
References: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
 <20170616185023.18967.72831.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170620072124.p6wztvxw5fj25a6m@pd.tnic>

On Fri, Jun 16, 2017 at 01:50:23PM -0500, Tom Lendacky wrote:
> The ioremap() function is intended for mapping MMIO. For RAM, the
> memremap() function should be used. Convert calls from ioremap() to
> memremap() when re-mapping RAM.
> 
> This will be used later by SME to control how the encryption mask is
> applied to memory mappings, with certain memory locations being mapped
> decrypted vs encrypted.
> 
> Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
> ---
>  arch/x86/include/asm/dmi.h   |    8 ++++----
>  arch/x86/kernel/acpi/boot.c  |    6 +++---
>  arch/x86/kernel/kdebugfs.c   |   34 +++++++++++-----------------------
>  arch/x86/kernel/ksysfs.c     |   28 ++++++++++++++--------------
>  arch/x86/kernel/mpparse.c    |   10 +++++-----
>  arch/x86/pci/common.c        |    4 ++--
>  drivers/firmware/dmi-sysfs.c |    5 +++--
>  drivers/firmware/pcdp.c      |    4 ++--
>  drivers/sfi/sfi_core.c       |   22 +++++++++++-----------
>  9 files changed, 55 insertions(+), 66 deletions(-)

Reviewed-by: Borislav Petkov <bp at suse.de>

-- 
Regards/Gruss,
    Boris.

Good mailing practices for 400: avoid top-posting and trim the reply.


From bp at alien8.de  Tue Jun 20 00:38:45 2017
From: bp at alien8.de (Borislav Petkov)
Date: Tue, 20 Jun 2017 09:38:45 +0200
Subject: [PATCH v7 08/36] x86/mm: Add support to enable SME in early boot
 processing
In-Reply-To: <20170616185115.18967.79622.stgit@tlendack-t1.amdoffice.net>
References: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
 <20170616185115.18967.79622.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170620073845.nteivabsgcdy7gv4@pd.tnic>

On Fri, Jun 16, 2017 at 01:51:15PM -0500, Tom Lendacky wrote:
> Add support to the early boot code to use Secure Memory Encryption (SME).
> Since the kernel has been loaded into memory in a decrypted state, encrypt
> the kernel in place and update the early pagetables with the memory
> encryption mask so that new pagetable entries will use memory encryption.
> 
> The routines to set the encryption mask and perform the encryption are
> stub routines for now with functionality to be added in a later patch.
> 
> Because of the need to have the routines available to head_64.S, the
> mem_encrypt.c is always built and #ifdefs in mem_encrypt.c will provide
> functionality or stub routines depending on CONFIG_AMD_MEM_ENCRYPT.
> 
> Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
> ---
>  arch/x86/include/asm/mem_encrypt.h |    8 +++++++
>  arch/x86/kernel/head64.c           |   33 +++++++++++++++++++++---------
>  arch/x86/kernel/head_64.S          |   39 ++++++++++++++++++++++++++++++++++--
>  arch/x86/mm/Makefile               |    4 +---
>  arch/x86/mm/mem_encrypt.c          |   24 ++++++++++++++++++++++
>  5 files changed, 93 insertions(+), 15 deletions(-)

...

> diff --git a/arch/x86/mm/mem_encrypt.c b/arch/x86/mm/mem_encrypt.c
> index b99d469..9a78277 100644
> --- a/arch/x86/mm/mem_encrypt.c
> +++ b/arch/x86/mm/mem_encrypt.c
> @@ -11,6 +11,9 @@
>   */
>  
>  #include <linux/linkage.h>
> +#include <linux/init.h>
> +
> +#ifdef CONFIG_AMD_MEM_ENCRYPT
>  
>  /*
>   * Since SME related variables are set early in the boot process they must
> @@ -19,3 +22,24 @@
>   */
>  unsigned long sme_me_mask __section(.data) = 0;
>  EXPORT_SYMBOL_GPL(sme_me_mask);
> +
> +void __init sme_encrypt_kernel(void)
> +{
> +}

Just the minor:

void __init sme_encrypt_kernel(void) { }

in case you have to respin.

Reviewed-by: Borislav Petkov <bp at suse.de>

-- 
Regards/Gruss,
    Boris.

Good mailing practices for 400: avoid top-posting and trim the reply.


From bp at alien8.de  Tue Jun 20 01:25:44 2017
From: bp at alien8.de (Borislav Petkov)
Date: Tue, 20 Jun 2017 10:25:44 +0200
Subject: [PATCH v7 11/36] x86/mm: Add SME support for read_cr3_pa()
In-Reply-To: <20170616185154.18967.71073.stgit@tlendack-t1.amdoffice.net>
References: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
 <20170616185154.18967.71073.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170620082544.uci6unwh4xum5k7n@pd.tnic>

On Fri, Jun 16, 2017 at 01:51:55PM -0500, Tom Lendacky wrote:
> The cr3 register entry can contain the SME encryption mask that indicates
> the PGD is encrypted.  The encryption mask should not be used when
> creating a virtual address from the cr3 register, so remove the SME
> encryption mask in the read_cr3_pa() function.
> 
> During early boot SME will need to use a native version of read_cr3_pa(),
> so create native_read_cr3_pa().
> 
> Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
> ---
>  arch/x86/include/asm/processor-flags.h |    3 ++-
>  arch/x86/include/asm/processor.h       |    5 +++++
>  2 files changed, 7 insertions(+), 1 deletion(-)

Reviewed-by: Borislav Petkov <bp at suse.de>

-- 
Regards/Gruss,
    Boris.

Good mailing practices for 400: avoid top-posting and trim the reply.


From lkp at intel.com  Sun Jun 11 12:46:54 2017
From: lkp at intel.com (kbuild test robot)
Date: Mon, 12 Jun 2017 03:46:54 +0800
Subject: [PATCH] s390/crash: Fix KEXEC_NOTE_BYTES definition
In-Reply-To: <1496974625-10891-1-git-send-email-xlpang@redhat.com>
Message-ID: <201706120335.KCTusZph%fengguang.wu@intel.com>

Hi Xunlei,

[auto build test ERROR on next-20170605]
[also build test ERROR on v4.12-rc4]
[cannot apply to linus/master linux/master s390/features v4.9-rc8 v4.9-rc7 v4.9-rc6]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]

url:    https://github.com/0day-ci/linux/commits/Xunlei-Pang/s390-crash-Fix-KEXEC_NOTE_BYTES-definition/20170610-173047
config: arm-sa1100 (attached as .config)
compiler: arm-linux-gnueabi-gcc (Debian 6.1.1-9) 6.1.1 20160705
reproduce:
        wget https://raw.githubusercontent.com/01org/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
        chmod +x ~/bin/make.cross
        # save the attached .config to linux build tree
        make.cross ARCH=arm 

All error/warnings (new ones prefixed by >>):

   In file included from arch/arm/include/asm/memory.h:378:0,
                    from arch/arm/include/asm/page.h:165,
                    from arch/arm/include/asm/thread_info.h:17,
                    from include/linux/thread_info.h:37,
                    from include/asm-generic/preempt.h:4,
                    from ./arch/arm/include/generated/asm/preempt.h:1,
                    from include/linux/preempt.h:80,
                    from include/linux/spinlock.h:50,
                    from include/linux/seqlock.h:35,
                    from include/linux/time.h:5,
                    from include/linux/efi.h:16,
                    from arch/arm/kernel/setup.c:10:
   arch/arm/include/asm/kexec.h: In function 'page_to_boot_pfn':
>> include/asm-generic/memory_model.h:63:14: error: implicit declaration of function 'page_to_section' [-Werror=implicit-function-declaration]
     int __sec = page_to_section(__pg);   \
                 ^
   include/asm-generic/memory_model.h:80:21: note: in expansion of macro '__page_to_pfn'
    #define page_to_pfn __page_to_pfn
                        ^~~~~~~~~~~~~
   arch/arm/include/asm/kexec.h:70:9: note: in expansion of macro 'page_to_pfn'
     return page_to_pfn(page) + (arch_phys_to_idmap_offset >> PAGE_SHIFT);
            ^~~~~~~~~~~
   In file included from include/linux/pid_namespace.h:6:0,
                    from include/linux/ptrace.h:9,
                    from include/uapi/linux/elfcore.h:7,
                    from include/linux/elfcore.h:9,
                    from include/linux/crash_core.h:5,
                    from include/linux/kexec.h:27,
                    from arch/arm/kernel/setup.c:24:
   include/linux/mm.h: At top level:
>> include/linux/mm.h:979:29: error: conflicting types for 'page_to_section'
    static inline unsigned long page_to_section(const struct page *page)
                                ^~~~~~~~~~~~~~~
   In file included from arch/arm/include/asm/memory.h:378:0,
                    from arch/arm/include/asm/page.h:165,
                    from arch/arm/include/asm/thread_info.h:17,
                    from include/linux/thread_info.h:37,
                    from include/asm-generic/preempt.h:4,
                    from ./arch/arm/include/generated/asm/preempt.h:1,
                    from include/linux/preempt.h:80,
                    from include/linux/spinlock.h:50,
                    from include/linux/seqlock.h:35,
                    from include/linux/time.h:5,
                    from include/linux/efi.h:16,
                    from arch/arm/kernel/setup.c:10:
   include/asm-generic/memory_model.h:63:14: note: previous implicit declaration of 'page_to_section' was here
     int __sec = page_to_section(__pg);   \
                 ^
   include/asm-generic/memory_model.h:80:21: note: in expansion of macro '__page_to_pfn'
    #define page_to_pfn __page_to_pfn
                        ^~~~~~~~~~~~~
   arch/arm/include/asm/kexec.h:70:9: note: in expansion of macro 'page_to_pfn'
     return page_to_pfn(page) + (arch_phys_to_idmap_offset >> PAGE_SHIFT);
            ^~~~~~~~~~~
   cc1: some warnings being treated as errors
--
   In file included from arch/arm/include/asm/memory.h:378:0,
                    from arch/arm/include/asm/page.h:165,
                    from arch/arm/include/asm/thread_info.h:17,
                    from include/linux/thread_info.h:37,
                    from include/asm-generic/preempt.h:4,
                    from ./arch/arm/include/generated/asm/preempt.h:1,
                    from include/linux/preempt.h:80,
                    from include/linux/spinlock.h:50,
                    from arch/arm/kernel/traps.c:18:
   arch/arm/include/asm/kexec.h: In function 'page_to_boot_pfn':
>> include/asm-generic/memory_model.h:63:14: error: implicit declaration of function 'page_to_section' [-Werror=implicit-function-declaration]
     int __sec = page_to_section(__pg);   \
                 ^
   include/asm-generic/memory_model.h:80:21: note: in expansion of macro '__page_to_pfn'
    #define page_to_pfn __page_to_pfn
                        ^~~~~~~~~~~~~
   arch/arm/include/asm/kexec.h:70:9: note: in expansion of macro 'page_to_pfn'
     return page_to_pfn(page) + (arch_phys_to_idmap_offset >> PAGE_SHIFT);
            ^~~~~~~~~~~
   include/asm-generic/memory_model.h:64:23: error: arithmetic on pointer to an incomplete type
     (unsigned long)(__pg - __section_mem_map_addr(__nr_to_section(__sec))); \
                          ^
   include/asm-generic/memory_model.h:80:21: note: in expansion of macro '__page_to_pfn'
    #define page_to_pfn __page_to_pfn
                        ^~~~~~~~~~~~~
   arch/arm/include/asm/kexec.h:70:9: note: in expansion of macro 'page_to_pfn'
     return page_to_pfn(page) + (arch_phys_to_idmap_offset >> PAGE_SHIFT);
            ^~~~~~~~~~~
   arch/arm/include/asm/kexec.h: In function 'boot_pfn_to_page':
>> include/asm-generic/memory_model.h:69:9: error: invalid use of undefined type 'struct page'
     struct mem_section *__sec = __pfn_to_section(__pfn); \
            ^
   include/asm-generic/memory_model.h:81:21: note: in expansion of macro '__pfn_to_page'
    #define pfn_to_page __pfn_to_page
                        ^~~~~~~~~~~~~
>> arch/arm/include/asm/kexec.h:76:9: note: in expansion of macro 'pfn_to_page'
     return pfn_to_page(boot_pfn - (arch_phys_to_idmap_offset >> PAGE_SHIFT));
            ^~~~~~~~~~~
   In file included from include/linux/pid_namespace.h:6:0,
                    from include/linux/ptrace.h:9,
                    from include/uapi/linux/elfcore.h:7,
                    from include/linux/elfcore.h:9,
                    from include/linux/crash_core.h:5,
                    from include/linux/kexec.h:27,
                    from arch/arm/kernel/traps.c:23:
   include/linux/mm.h: At top level:
>> include/linux/mm.h:979:29: error: conflicting types for 'page_to_section'
    static inline unsigned long page_to_section(const struct page *page)
                                ^~~~~~~~~~~~~~~
   In file included from arch/arm/include/asm/memory.h:378:0,
                    from arch/arm/include/asm/page.h:165,
                    from arch/arm/include/asm/thread_info.h:17,
                    from include/linux/thread_info.h:37,
                    from include/asm-generic/preempt.h:4,
                    from ./arch/arm/include/generated/asm/preempt.h:1,
                    from include/linux/preempt.h:80,
                    from include/linux/spinlock.h:50,
                    from arch/arm/kernel/traps.c:18:
   include/asm-generic/memory_model.h:63:14: note: previous implicit declaration of 'page_to_section' was here
     int __sec = page_to_section(__pg);   \
                 ^
   include/asm-generic/memory_model.h:80:21: note: in expansion of macro '__page_to_pfn'
    #define page_to_pfn __page_to_pfn
                        ^~~~~~~~~~~~~
   arch/arm/include/asm/kexec.h:70:9: note: in expansion of macro 'page_to_pfn'
     return page_to_pfn(page) + (arch_phys_to_idmap_offset >> PAGE_SHIFT);
            ^~~~~~~~~~~
   cc1: some warnings being treated as errors

vim +/pfn_to_page +76 arch/arm/include/asm/kexec.h

0719392a arch/arm/include/asm/kexec.h Russell King   2016-08-02  60  #define phys_to_boot_phys phys_to_boot_phys
0719392a arch/arm/include/asm/kexec.h Russell King   2016-08-02  61  
0719392a arch/arm/include/asm/kexec.h Russell King   2016-08-02  62  static inline phys_addr_t boot_phys_to_phys(unsigned long entry)
0719392a arch/arm/include/asm/kexec.h Russell King   2016-08-02  63  {
0719392a arch/arm/include/asm/kexec.h Russell King   2016-08-02  64  	return idmap_to_phys(entry);
0719392a arch/arm/include/asm/kexec.h Russell King   2016-08-02  65  }
0719392a arch/arm/include/asm/kexec.h Russell King   2016-08-02  66  #define boot_phys_to_phys boot_phys_to_phys
0719392a arch/arm/include/asm/kexec.h Russell King   2016-08-02  67  
0719392a arch/arm/include/asm/kexec.h Russell King   2016-08-02  68  static inline unsigned long page_to_boot_pfn(struct page *page)
0719392a arch/arm/include/asm/kexec.h Russell King   2016-08-02  69  {
0719392a arch/arm/include/asm/kexec.h Russell King   2016-08-02  70  	return page_to_pfn(page) + (arch_phys_to_idmap_offset >> PAGE_SHIFT);
0719392a arch/arm/include/asm/kexec.h Russell King   2016-08-02  71  }
0719392a arch/arm/include/asm/kexec.h Russell King   2016-08-02  72  #define page_to_boot_pfn page_to_boot_pfn
0719392a arch/arm/include/asm/kexec.h Russell King   2016-08-02  73  
0719392a arch/arm/include/asm/kexec.h Russell King   2016-08-02  74  static inline struct page *boot_pfn_to_page(unsigned long boot_pfn)
0719392a arch/arm/include/asm/kexec.h Russell King   2016-08-02  75  {
0719392a arch/arm/include/asm/kexec.h Russell King   2016-08-02 @76  	return pfn_to_page(boot_pfn - (arch_phys_to_idmap_offset >> PAGE_SHIFT));
0719392a arch/arm/include/asm/kexec.h Russell King   2016-08-02  77  }
0719392a arch/arm/include/asm/kexec.h Russell King   2016-08-02  78  #define boot_pfn_to_page boot_pfn_to_page
0719392a arch/arm/include/asm/kexec.h Russell King   2016-08-02  79  
c587e4a6 include/asm-arm/kexec.h      Richard Purdie 2007-02-06  80  #endif /* __ASSEMBLY__ */
c587e4a6 include/asm-arm/kexec.h      Richard Purdie 2007-02-06  81  
c587e4a6 include/asm-arm/kexec.h      Richard Purdie 2007-02-06  82  #endif /* CONFIG_KEXEC */
c587e4a6 include/asm-arm/kexec.h      Richard Purdie 2007-02-06  83  
c587e4a6 include/asm-arm/kexec.h      Richard Purdie 2007-02-06  84  #endif /* _ARM_KEXEC_H */

:::::: The code at line 76 was first introduced by commit
:::::: 0719392a61a9dbc2c850bc7bd1a17efba953fcf5 ARM: kexec: fix kexec for Keystone 2

:::::: TO: Russell King <rmk+kernel at arm.linux.org.uk>
:::::: CC: Linus Torvalds <torvalds at linux-foundation.org>

---
0-DAY kernel test infrastructure                Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all                   Intel Corporation
-------------- next part --------------
A non-text attachment was scrubbed...
Name: .config.gz
Type: application/gzip
Size: 23734 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/kexec/attachments/20170612/53a07126/attachment-0001.gz>

From bp at alien8.de  Tue Jun 20 08:30:56 2017
From: bp at alien8.de (Borislav Petkov)
Date: Tue, 20 Jun 2017 17:30:56 +0200
Subject: [PATCH v7 14/36] x86/mm: Insure that boot memory areas are
 mapped properly
In-Reply-To: <20170616185232.18967.61753.stgit@tlendack-t1.amdoffice.net>
References: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
 <20170616185232.18967.61753.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170620153056.bz2kvgvshnat6345@pd.tnic>

On Fri, Jun 16, 2017 at 01:52:32PM -0500, Tom Lendacky wrote:
> The boot data and command line data are present in memory in a decrypted
> state and are copied early in the boot process.  The early page fault
> support will map these areas as encrypted, so before attempting to copy
> them, add decrypted mappings so the data is accessed properly when copied.
> 
> For the initrd, encrypt this data in place. Since the future mapping of
> the initrd area will be mapped as encrypted the data will be accessed
> properly.
> 
> Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
> ---
>  arch/x86/include/asm/mem_encrypt.h |    6 +++
>  arch/x86/include/asm/pgtable.h     |    3 ++
>  arch/x86/kernel/head64.c           |   30 +++++++++++++--
>  arch/x86/kernel/setup.c            |    9 +++++
>  arch/x86/mm/kasan_init_64.c        |    2 +
>  arch/x86/mm/mem_encrypt.c          |   70 ++++++++++++++++++++++++++++++++++++
>  6 files changed, 115 insertions(+), 5 deletions(-)

...

> diff --git a/arch/x86/mm/mem_encrypt.c b/arch/x86/mm/mem_encrypt.c
> index b7671b9..ea5e3a6 100644
> --- a/arch/x86/mm/mem_encrypt.c
> +++ b/arch/x86/mm/mem_encrypt.c
> @@ -19,6 +19,8 @@
>  
>  #include <asm/tlbflush.h>
>  #include <asm/fixmap.h>
> +#include <asm/setup.h>
> +#include <asm/bootparam.h>
>  
>  /*
>   * Since SME related variables are set early in the boot process they must
> @@ -101,6 +103,74 @@ void __init sme_early_decrypt(resource_size_t paddr, unsigned long size)
>  	__sme_early_enc_dec(paddr, size, false);
>  }
>  
> +static void __init __sme_early_map_unmap_mem(void *vaddr, unsigned long size,
> +					     bool map)
> +{
> +	unsigned long paddr = (unsigned long)vaddr - __PAGE_OFFSET;
> +	pmdval_t pmd_flags, pmd;
> +
> +	/* Use early_pmd_flags but remove the encryption mask */
> +	pmd_flags = __sme_clr(early_pmd_flags);
> +
> +	do {
> +		pmd = map ? (paddr & PMD_MASK) + pmd_flags : 0;
> +		__early_make_pgtable((unsigned long)vaddr, pmd);
> +
> +		vaddr += PMD_SIZE;
> +		paddr += PMD_SIZE;
> +		size = (size <= PMD_SIZE) ? 0 : size - PMD_SIZE;
> +	} while (size);
> +
> +	write_cr3(__read_cr3());

local_flush_tlb() or __native_flush_tlb(). Probably the native variant
since this is early fun.

> +}
> +
> +static void __init __sme_map_unmap_bootdata(char *real_mode_data, bool map)
> +{
> +	struct boot_params *boot_data;
> +	unsigned long cmdline_paddr;
> +
> +	/* If SME is not active, the bootdata is in the correct state */
> +	if (!sme_active())
> +		return;
> +
> +	if (!map) {
> +		/*
> +		 * If unmapping, get the command line address before
> +		 * unmapping the real_mode_data.
> +		 */
> +		boot_data = (struct boot_params *)real_mode_data;
> +		cmdline_paddr = boot_data->hdr.cmd_line_ptr |
> +				((u64)boot_data->ext_cmd_line_ptr << 32);

Let it stick out:

	cmdline_paddr = bd->hdr.cmd_line_ptr | ((u64)bd->ext_cmd_line_ptr << 32);

> +	}
> +
> +	__sme_early_map_unmap_mem(real_mode_data, sizeof(boot_params), map);
> +
> +	if (map) {
> +		/*
> +		 * If mapping, get the command line address after mapping
> +		 * the real_mode_data.
> +		 */
> +		boot_data = (struct boot_params *)real_mode_data;
> +		cmdline_paddr = boot_data->hdr.cmd_line_ptr |
> +				((u64)boot_data->ext_cmd_line_ptr << 32);
> +	}
> +
> +	if (!cmdline_paddr)
> +		return;
> +
> +	__sme_early_map_unmap_mem(__va(cmdline_paddr), COMMAND_LINE_SIZE, map);

Ok, so from looking at this function now - it does different things
depending on whether we map or not. So it doesn't look like a worker
function anymore and you can move the stuff back to the original callers
below. Should make the whole flow more readable.

> +}
> +
> +void __init sme_unmap_bootdata(char *real_mode_data)
> +{
> +	__sme_map_unmap_bootdata(real_mode_data, false);
> +}
> +
> +void __init sme_map_bootdata(char *real_mode_data)
> +{
> +	__sme_map_unmap_bootdata(real_mode_data, true);
> +}
> +
>  void __init sme_early_init(void)
>  {
>  	unsigned int i;
> 

-- 
Regards/Gruss,
    Boris.

Good mailing practices for 400: avoid top-posting and trim the reply.


From thomas.lendacky at amd.com  Tue Jun 20 08:52:48 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Tue, 20 Jun 2017 10:52:48 -0500
Subject: [PATCH v7 08/36] x86/mm: Add support to enable SME in early boot
 processing
In-Reply-To: <20170620073845.nteivabsgcdy7gv4@pd.tnic>
References: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
 <20170616185115.18967.79622.stgit@tlendack-t1.amdoffice.net>
 <20170620073845.nteivabsgcdy7gv4@pd.tnic>
Message-ID: <49c62e8c-c4ae-6d05-e2a4-aa1fc6e2d717@amd.com>

On 6/20/2017 2:38 AM, Borislav Petkov wrote:
> On Fri, Jun 16, 2017 at 01:51:15PM -0500, Tom Lendacky wrote:
>> Add support to the early boot code to use Secure Memory Encryption (SME).
>> Since the kernel has been loaded into memory in a decrypted state, encrypt
>> the kernel in place and update the early pagetables with the memory
>> encryption mask so that new pagetable entries will use memory encryption.
>>
>> The routines to set the encryption mask and perform the encryption are
>> stub routines for now with functionality to be added in a later patch.
>>
>> Because of the need to have the routines available to head_64.S, the
>> mem_encrypt.c is always built and #ifdefs in mem_encrypt.c will provide
>> functionality or stub routines depending on CONFIG_AMD_MEM_ENCRYPT.
>>
>> Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
>> ---
>>   arch/x86/include/asm/mem_encrypt.h |    8 +++++++
>>   arch/x86/kernel/head64.c           |   33 +++++++++++++++++++++---------
>>   arch/x86/kernel/head_64.S          |   39 ++++++++++++++++++++++++++++++++++--
>>   arch/x86/mm/Makefile               |    4 +---
>>   arch/x86/mm/mem_encrypt.c          |   24 ++++++++++++++++++++++
>>   5 files changed, 93 insertions(+), 15 deletions(-)
> 
> ...
> 
>> diff --git a/arch/x86/mm/mem_encrypt.c b/arch/x86/mm/mem_encrypt.c
>> index b99d469..9a78277 100644
>> --- a/arch/x86/mm/mem_encrypt.c
>> +++ b/arch/x86/mm/mem_encrypt.c
>> @@ -11,6 +11,9 @@
>>    */
>>   
>>   #include <linux/linkage.h>
>> +#include <linux/init.h>
>> +
>> +#ifdef CONFIG_AMD_MEM_ENCRYPT
>>   
>>   /*
>>    * Since SME related variables are set early in the boot process they must
>> @@ -19,3 +22,24 @@
>>    */
>>   unsigned long sme_me_mask __section(.data) = 0;
>>   EXPORT_SYMBOL_GPL(sme_me_mask);
>> +
>> +void __init sme_encrypt_kernel(void)
>> +{
>> +}
> 
> Just the minor:
> 
> void __init sme_encrypt_kernel(void) { }
> 
> in case you have to respin.

I have to re-spin for the kbuild test error.  But given that this
function will be filled in later it's probably not worth doing the
space savings here.

Thanks,
Tom

> 
> Reviewed-by: Borislav Petkov <bp at suse.de>
> 


From luto at kernel.org  Tue Jun 20 09:17:26 2017
From: luto at kernel.org (Andy Lutomirski)
Date: Tue, 20 Jun 2017 09:17:26 -0700
Subject: [PATCH v7 11/36] x86/mm: Add SME support for read_cr3_pa()
In-Reply-To: <20170616185154.18967.71073.stgit@tlendack-t1.amdoffice.net>
References: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
 <20170616185154.18967.71073.stgit@tlendack-t1.amdoffice.net>
Message-ID: <CALCETrVkyj=wfcgNMVG_BU+xGb3yBNhxrDdSTxJLx7UYraVcUA@mail.gmail.com>

On Fri, Jun 16, 2017 at 11:51 AM, Tom Lendacky <thomas.lendacky at amd.com> wrote:
> The cr3 register entry can contain the SME encryption mask that indicates
> the PGD is encrypted.  The encryption mask should not be used when
> creating a virtual address from the cr3 register, so remove the SME
> encryption mask in the read_cr3_pa() function.
>
> During early boot SME will need to use a native version of read_cr3_pa(),
> so create native_read_cr3_pa().
>
> Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
> ---
>  arch/x86/include/asm/processor-flags.h |    3 ++-
>  arch/x86/include/asm/processor.h       |    5 +++++
>  2 files changed, 7 insertions(+), 1 deletion(-)
>
> diff --git a/arch/x86/include/asm/processor-flags.h b/arch/x86/include/asm/processor-flags.h
> index 79aa2f9..cb6999c 100644
> --- a/arch/x86/include/asm/processor-flags.h
> +++ b/arch/x86/include/asm/processor-flags.h
> @@ -2,6 +2,7 @@
>  #define _ASM_X86_PROCESSOR_FLAGS_H
>
>  #include <uapi/asm/processor-flags.h>
> +#include <linux/mem_encrypt.h>
>
>  #ifdef CONFIG_VM86
>  #define X86_VM_MASK    X86_EFLAGS_VM
> @@ -33,7 +34,7 @@
>   */
>  #ifdef CONFIG_X86_64
>  /* Mask off the address space ID bits. */
> -#define CR3_ADDR_MASK 0x7FFFFFFFFFFFF000ull
> +#define CR3_ADDR_MASK __sme_clr(0x7FFFFFFFFFFFF000ull)

Can you update the comment one line above, too?


From thomas.lendacky at amd.com  Tue Jun 20 09:23:03 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Tue, 20 Jun 2017 11:23:03 -0500
Subject: [PATCH v7 11/36] x86/mm: Add SME support for read_cr3_pa()
In-Reply-To: <CALCETrVkyj=wfcgNMVG_BU+xGb3yBNhxrDdSTxJLx7UYraVcUA@mail.gmail.com>
References: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
 <20170616185154.18967.71073.stgit@tlendack-t1.amdoffice.net>
 <CALCETrVkyj=wfcgNMVG_BU+xGb3yBNhxrDdSTxJLx7UYraVcUA@mail.gmail.com>
Message-ID: <9a1c0df8-ca12-eebc-5565-ced847989169@amd.com>

On 6/20/2017 11:17 AM, Andy Lutomirski wrote:
> On Fri, Jun 16, 2017 at 11:51 AM, Tom Lendacky <thomas.lendacky at amd.com> wrote:
>> The cr3 register entry can contain the SME encryption mask that indicates
>> the PGD is encrypted.  The encryption mask should not be used when
>> creating a virtual address from the cr3 register, so remove the SME
>> encryption mask in the read_cr3_pa() function.
>>
>> During early boot SME will need to use a native version of read_cr3_pa(),
>> so create native_read_cr3_pa().
>>
>> Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
>> ---
>>   arch/x86/include/asm/processor-flags.h |    3 ++-
>>   arch/x86/include/asm/processor.h       |    5 +++++
>>   2 files changed, 7 insertions(+), 1 deletion(-)
>>
>> diff --git a/arch/x86/include/asm/processor-flags.h b/arch/x86/include/asm/processor-flags.h
>> index 79aa2f9..cb6999c 100644
>> --- a/arch/x86/include/asm/processor-flags.h
>> +++ b/arch/x86/include/asm/processor-flags.h
>> @@ -2,6 +2,7 @@
>>   #define _ASM_X86_PROCESSOR_FLAGS_H
>>
>>   #include <uapi/asm/processor-flags.h>
>> +#include <linux/mem_encrypt.h>
>>
>>   #ifdef CONFIG_VM86
>>   #define X86_VM_MASK    X86_EFLAGS_VM
>> @@ -33,7 +34,7 @@
>>    */
>>   #ifdef CONFIG_X86_64
>>   /* Mask off the address space ID bits. */
>> -#define CR3_ADDR_MASK 0x7FFFFFFFFFFFF000ull
>> +#define CR3_ADDR_MASK __sme_clr(0x7FFFFFFFFFFFF000ull)
> 
> Can you update the comment one line above, too?

Yup, will do.

Thanks,
Tom

> 


From bp at alien8.de  Tue Jun 20 13:39:06 2017
From: bp at alien8.de (Borislav Petkov)
Date: Tue, 20 Jun 2017 22:39:06 +0200
Subject: [PATCH v7 19/36] x86/mm: Add support to access boot related data
 in the clear
In-Reply-To: <20170616185326.18967.43278.stgit@tlendack-t1.amdoffice.net>
References: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
 <20170616185326.18967.43278.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170620203906.fzkez2f7es6ow4gr@pd.tnic>

On Fri, Jun 16, 2017 at 01:53:26PM -0500, Tom Lendacky wrote:
> Boot data (such as EFI related data) is not encrypted when the system is
> booted because UEFI/BIOS does not run with SME active. In order to access
> this data properly it needs to be mapped decrypted.
> 
> Update early_memremap() to provide an arch specific routine to modify the
> pagetable protection attributes before they are applied to the new
> mapping. This is used to remove the encryption mask for boot related data.
> 
> Update memremap() to provide an arch specific routine to determine if RAM
> remapping is allowed.  RAM remapping will cause an encrypted mapping to be
> generated. By preventing RAM remapping, ioremap_cache() will be used
> instead, which will provide a decrypted mapping of the boot related data.
> 
> Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
> ---
>  arch/x86/include/asm/io.h |    5 +
>  arch/x86/mm/ioremap.c     |  179 +++++++++++++++++++++++++++++++++++++++++++++
>  include/linux/io.h        |    2 +
>  kernel/memremap.c         |   20 ++++-
>  mm/early_ioremap.c        |   18 ++++-
>  5 files changed, 217 insertions(+), 7 deletions(-)

Reviewed-by: Borislav Petkov <bp at suse.de>

-- 
Regards/Gruss,
    Boris.

Good mailing practices for 400: avoid top-posting and trim the reply.


From tglx at linutronix.de  Tue Jun 20 13:49:31 2017
From: tglx at linutronix.de (Thomas Gleixner)
Date: Tue, 20 Jun 2017 22:49:31 +0200 (CEST)
Subject: [PATCH v7 06/36] x86/mm: Add Secure Memory Encryption (SME)
 support
In-Reply-To: <20170616185054.18967.52228.stgit@tlendack-t1.amdoffice.net>
References: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
 <20170616185054.18967.52228.stgit@tlendack-t1.amdoffice.net>
Message-ID: <alpine.DEB.2.20.1706202244480.2157@nanos>

On Fri, 16 Jun 2017, Tom Lendacky wrote:
>  
> +config ARCH_HAS_MEM_ENCRYPT
> +	def_bool y
> +	depends on X86

That one is silly. The config switch is in the x86 KConfig file, so X86 is
on. If you intended to move this to some generic place outside of
x86/Kconfig then this should be

config ARCH_HAS_MEM_ENCRYPT
	bool

and x86/Kconfig should have

    	select ARCH_HAS_MEM_ENCRYPT

and that should be selected by AMD_MEM_ENCRYPT

> +config AMD_MEM_ENCRYPT
> +	bool "AMD Secure Memory Encryption (SME) support"
> +	depends on X86_64 && CPU_SUP_AMD
> +	---help---
> +	  Say yes to enable support for the encryption of system memory.
> +	  This requires an AMD processor that supports Secure Memory
> +	  Encryption (SME).

Thanks,

	tglx


From tglx at linutronix.de  Tue Jun 20 13:55:35 2017
From: tglx at linutronix.de (Thomas Gleixner)
Date: Tue, 20 Jun 2017 22:55:35 +0200 (CEST)
Subject: [PATCH v7 07/36] x86/mm: Don't use phys_to_virt in ioremap() if
 SME is active
In-Reply-To: <20170616185104.18967.7867.stgit@tlendack-t1.amdoffice.net>
References: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
 <20170616185104.18967.7867.stgit@tlendack-t1.amdoffice.net>
Message-ID: <alpine.DEB.2.20.1706202251110.2157@nanos>

On Fri, 16 Jun 2017, Tom Lendacky wrote:

> Currently there is a check if the address being mapped is in the ISA
> range (is_ISA_range()), and if it is then phys_to_virt() is used to
> perform the mapping.  When SME is active, however, this will result
> in the mapping having the encryption bit set when it is expected that
> an ioremap() should not have the encryption bit set. So only use the
> phys_to_virt() function if SME is not active

This does not make sense to me. What the heck has phys_to_virt() to do with
the encryption bit. Especially why would the encryption bit be set on that
mapping in the first place?

I'm probably missing something, but this want's some coherent explanation
understandable by mere mortals both in the changelog and the code comment.

Thanks,

	tglx


From tglx at linutronix.de  Wed Jun 21 00:16:05 2017
From: tglx at linutronix.de (Thomas Gleixner)
Date: Wed, 21 Jun 2017 09:16:05 +0200 (CEST)
Subject: [PATCH v7 08/36] x86/mm: Add support to enable SME in early boot
 processing
In-Reply-To: <20170616185115.18967.79622.stgit@tlendack-t1.amdoffice.net>
References: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
 <20170616185115.18967.79622.stgit@tlendack-t1.amdoffice.net>
Message-ID: <alpine.DEB.2.20.1706202259290.2157@nanos>

On Fri, 16 Jun 2017, Tom Lendacky wrote:
> diff --git a/arch/x86/include/asm/mem_encrypt.h b/arch/x86/include/asm/mem_encrypt.h
> index a105796..988b336 100644
> --- a/arch/x86/include/asm/mem_encrypt.h
> +++ b/arch/x86/include/asm/mem_encrypt.h
> @@ -15,16 +15,24 @@
>  
>  #ifndef __ASSEMBLY__
>  
> +#include <linux/init.h>
> +
>  #ifdef CONFIG_AMD_MEM_ENCRYPT
>  
>  extern unsigned long sme_me_mask;
>  
> +void __init sme_enable(void);
> +
>  #else	/* !CONFIG_AMD_MEM_ENCRYPT */
>  
>  #define sme_me_mask	0UL
>  
> +static inline void __init sme_enable(void) { }
> +
>  #endif	/* CONFIG_AMD_MEM_ENCRYPT */
>  
> +unsigned long sme_get_me_mask(void);

Why is this an unconditional function? Isn't the mask simply 0 when the MEM
ENCRYPT support is disabled?

> diff --git a/arch/x86/kernel/head_64.S b/arch/x86/kernel/head_64.S
> index 6225550..ef12729 100644
> --- a/arch/x86/kernel/head_64.S
> +++ b/arch/x86/kernel/head_64.S
> @@ -78,7 +78,29 @@ startup_64:
>  	call	__startup_64
>  	popq	%rsi
>  
> -	movq	$(early_top_pgt - __START_KERNEL_map), %rax
> +	/*
> +	 * Encrypt the kernel if SME is active.
> +	 * The real_mode_data address is in %rsi and that register can be
> +	 * clobbered by the called function so be sure to save it.
> +	 */
> +	push	%rsi
> +	call	sme_encrypt_kernel
> +	pop	%rsi

That does not make any sense. Neither the call to sme_encrypt_kernel() nor
the following call to sme_get_me_mask().

__startup_64() is already C code, so why can't you simply call that from
__startup_64() in C and return the mask from there?

> @@ -98,7 +120,20 @@ ENTRY(secondary_startup_64)
>  	/* Sanitize CPU configuration */
>  	call verify_cpu
>  
> -	movq	$(init_top_pgt - __START_KERNEL_map), %rax
> +	/*
> +	 * Get the SME encryption mask.
> +	 *  The encryption mask will be returned in %rax so we do an ADD
> +	 *  below to be sure that the encryption mask is part of the
> +	 *  value that will stored in %cr3.
> +	 *
> +	 * The real_mode_data address is in %rsi and that register can be
> +	 * clobbered by the called function so be sure to save it.
> +	 */
> +	push	%rsi
> +	call	sme_get_me_mask
> +	pop	%rsi

Do we really need a call here? The mask is established at this point, so
it's either 0 when the encryption stuff is not compiled in or it can be
retrieved from a variable which is accessible at this point.

> +
> +	addq	$(init_top_pgt - __START_KERNEL_map), %rax
>  1:
>  
>  	/* Enable PAE mode, PGE and LA57 */

Thanks,

	tglx


From tglx at linutronix.de  Wed Jun 21 00:18:59 2017
From: tglx at linutronix.de (Thomas Gleixner)
Date: Wed, 21 Jun 2017 09:18:59 +0200 (CEST)
Subject: [PATCH v7 10/36] x86/mm: Provide general kernel support for
 memory encryption
In-Reply-To: <20170616185140.18967.61905.stgit@tlendack-t1.amdoffice.net>
References: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
 <20170616185140.18967.61905.stgit@tlendack-t1.amdoffice.net>
Message-ID: <alpine.DEB.2.20.1706210916590.2328@nanos>

On Fri, 16 Jun 2017, Tom Lendacky wrote:
>  
> +#ifndef pgprot_encrypted
> +#define pgprot_encrypted(prot)	(prot)
> +#endif
> +
> +#ifndef pgprot_decrypted

That looks wrong. It's not decrypted it's rather unencrypted, right?

Thanks,

	tglx


From tglx at linutronix.de  Wed Jun 21 00:37:39 2017
From: tglx at linutronix.de (Thomas Gleixner)
Date: Wed, 21 Jun 2017 09:37:39 +0200 (CEST)
Subject: [PATCH v7 07/36] x86/mm: Don't use phys_to_virt in ioremap() if
 SME is active
In-Reply-To: <20170616185104.18967.7867.stgit@tlendack-t1.amdoffice.net>
References: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
 <20170616185104.18967.7867.stgit@tlendack-t1.amdoffice.net>
Message-ID: <alpine.DEB.2.20.1706210934540.2328@nanos>

On Fri, 16 Jun 2017, Tom Lendacky wrote:
> Currently there is a check if the address being mapped is in the ISA
> range (is_ISA_range()), and if it is then phys_to_virt() is used to
> perform the mapping.  When SME is active, however, this will result
> in the mapping having the encryption bit set when it is expected that
> an ioremap() should not have the encryption bit set. So only use the
> phys_to_virt() function if SME is not active
> 
> Reviewed-by: Borislav Petkov <bp at suse.de>
> Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
> ---
>  arch/x86/mm/ioremap.c |    7 +++++--
>  1 file changed, 5 insertions(+), 2 deletions(-)
> 
> diff --git a/arch/x86/mm/ioremap.c b/arch/x86/mm/ioremap.c
> index 4c1b5fd..a382ba9 100644
> --- a/arch/x86/mm/ioremap.c
> +++ b/arch/x86/mm/ioremap.c
> @@ -13,6 +13,7 @@
>  #include <linux/slab.h>
>  #include <linux/vmalloc.h>
>  #include <linux/mmiotrace.h>
> +#include <linux/mem_encrypt.h>
>  
>  #include <asm/set_memory.h>
>  #include <asm/e820/api.h>
> @@ -106,9 +107,11 @@ static void __iomem *__ioremap_caller(resource_size_t phys_addr,
>  	}
>  
>  	/*
> -	 * Don't remap the low PCI/ISA area, it's always mapped..
> +	 * Don't remap the low PCI/ISA area, it's always mapped.
> +	 *   But if SME is active, skip this so that the encryption bit
> +	 *   doesn't get set.
>  	 */
> -	if (is_ISA_range(phys_addr, last_addr))
> +	if (is_ISA_range(phys_addr, last_addr) && !sme_active())
>  		return (__force void __iomem *)phys_to_virt(phys_addr);

More thoughts about that.

Making this conditional on !sme_active() is not the best idea. I'd rather
remove that whole thing and make it unconditional so the code pathes get
always exercised and any subtle wreckage is detected on a broader base and
not only on that hard to access and debug SME capable machine owned by Joe
User.

Thanks,

	tglx


From bp at alien8.de  Wed Jun 21 01:23:01 2017
From: bp at alien8.de (Borislav Petkov)
Date: Wed, 21 Jun 2017 10:23:01 +0200
Subject: [PATCH v7 10/36] x86/mm: Provide general kernel support for
 memory encryption
In-Reply-To: <alpine.DEB.2.20.1706210916590.2328@nanos>
References: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
 <20170616185140.18967.61905.stgit@tlendack-t1.amdoffice.net>
 <alpine.DEB.2.20.1706210916590.2328@nanos>
Message-ID: <20170621082301.47lafavh6cstbwfl@pd.tnic>

On Wed, Jun 21, 2017 at 09:18:59AM +0200, Thomas Gleixner wrote:
> That looks wrong. It's not decrypted it's rather unencrypted, right?

Yeah, it previous versions of the patchset, "decrypted" and
"unencrypted" were both present so we settled on "decrypted" for the
nomenclature.

-- 
Regards/Gruss,
    Boris.

Good mailing practices for 400: avoid top-posting and trim the reply.


From bp at alien8.de  Wed Jun 21 01:39:18 2017
From: bp at alien8.de (Borislav Petkov)
Date: Wed, 21 Jun 2017 10:39:18 +0200
Subject: [PATCH v7 20/36] x86, mpparse: Use memremap to map the mpf and
 mpc data
In-Reply-To: <20170616185338.18967.80659.stgit@tlendack-t1.amdoffice.net>
References: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
 <20170616185338.18967.80659.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170621083918.oa2w5jsxs3si4oln@pd.tnic>

On Fri, Jun 16, 2017 at 01:53:38PM -0500, Tom Lendacky wrote:
> The SMP MP-table is built by UEFI and placed in memory in a decrypted
> state. These tables are accessed using a mix of early_memremap(),
> early_memunmap(), phys_to_virt() and virt_to_phys(). Change all accesses
> to use early_memremap()/early_memunmap(). This allows for proper setting
> of the encryption mask so that the data can be successfully accessed when
> SME is active.
> 
> Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
> ---
>  arch/x86/kernel/mpparse.c |   98 ++++++++++++++++++++++++++++++++-------------
>  1 file changed, 70 insertions(+), 28 deletions(-)

Reviewed-by: Borislav Petkov <bp at suse.de>

Please put the conversion to pr_fmt() on the TODO list for later.

Thanks.

-- 
Regards/Gruss,
    Boris.

Good mailing practices for 400: avoid top-posting and trim the reply.


From holzheu at linux.vnet.ibm.com  Wed Jun 21 02:00:31 2017
From: holzheu at linux.vnet.ibm.com (Michael Holzheu)
Date: Wed, 21 Jun 2017 11:00:31 +0200
Subject: [PATCH] s390/crash: Fix KEXEC_NOTE_BYTES definition
In-Reply-To: <1496974625-10891-1-git-send-email-xlpang@redhat.com>
References: <1496974625-10891-1-git-send-email-xlpang@redhat.com>
Message-ID: <20170621110031.76c15516@TP-holzheu>

Hi Xunlei,

Sorry for the late reply - I was on vacation up to now.
Give us some time to look into this issue.

Michael

Am Fri,  9 Jun 2017 10:17:05 +0800
schrieb Xunlei Pang <xlpang at redhat.com>:

> S390 KEXEC_NOTE_BYTES is not used by note_buf_t as before, which
> is now defined as follows:
>     typedef u32 note_buf_t[CRASH_CORE_NOTE_BYTES/4];
> It was changed by the CONFIG_CRASH_CORE feature.
> 
> This patch gets rid of all the old KEXEC_NOTE_BYTES stuff, and
> renames KEXEC_NOTE_BYTES to CRASH_CORE_NOTE_BYTES for S390.
> 
> Fixes: 692f66f26a4c ("crash: move crashkernel parsing and vmcore related code under CONFIG_CRASH_CORE")
> Cc: Dave Young <dyoung at redhat.com>
> Cc: Dave Anderson <anderson at redhat.com>
> Cc: Hari Bathini <hbathini at linux.vnet.ibm.com>
> Cc: Gustavo Luiz Duarte <gustavold at linux.vnet.ibm.com>
> Signed-off-by: Xunlei Pang <xlpang at redhat.com>
> ---
>  arch/s390/include/asm/kexec.h |  2 +-
>  include/linux/crash_core.h    |  7 +++++++
>  include/linux/kexec.h         | 11 +----------
>  3 files changed, 9 insertions(+), 11 deletions(-)
> 
> diff --git a/arch/s390/include/asm/kexec.h b/arch/s390/include/asm/kexec.h
> index 2f924bc..352deb8 100644
> --- a/arch/s390/include/asm/kexec.h
> +++ b/arch/s390/include/asm/kexec.h
> @@ -47,7 +47,7 @@
>   * Seven notes plus zero note at the end: prstatus, fpregset, timer,
>   * tod_cmp, tod_reg, control regs, and prefix
>   */
> -#define KEXEC_NOTE_BYTES \
> +#define CRASH_CORE_NOTE_BYTES \
>  	(ALIGN(sizeof(struct elf_note), 4) * 8 + \
>  	 ALIGN(sizeof("CORE"), 4) * 7 + \
>  	 ALIGN(sizeof(struct elf_prstatus), 4) + \
> diff --git a/include/linux/crash_core.h b/include/linux/crash_core.h
> index e9de6b4..dbc6e5c 100644
> --- a/include/linux/crash_core.h
> +++ b/include/linux/crash_core.h
> @@ -10,9 +10,16 @@
>  #define CRASH_CORE_NOTE_NAME_BYTES ALIGN(sizeof(CRASH_CORE_NOTE_NAME), 4)
>  #define CRASH_CORE_NOTE_DESC_BYTES ALIGN(sizeof(struct elf_prstatus), 4)
> 
> +/*
> + * The per-cpu notes area is a list of notes terminated by a "NULL"
> + * note header.  For kdump, the code in vmcore.c runs in the context
> + * of the second kernel to combine them into one note.
> + */
> +#ifndef CRASH_CORE_NOTE_BYTES
>  #define CRASH_CORE_NOTE_BYTES	   ((CRASH_CORE_NOTE_HEAD_BYTES * 2) +	\
>  				     CRASH_CORE_NOTE_NAME_BYTES +	\
>  				     CRASH_CORE_NOTE_DESC_BYTES)
> +#endif
> 
>  #define VMCOREINFO_BYTES	   PAGE_SIZE
>  #define VMCOREINFO_NOTE_NAME	   "VMCOREINFO"
> diff --git a/include/linux/kexec.h b/include/linux/kexec.h
> index 3ea8275..133df03 100644
> --- a/include/linux/kexec.h
> +++ b/include/linux/kexec.h
> @@ -14,7 +14,6 @@
> 
>  #if !defined(__ASSEMBLY__)
> 
> -#include <linux/crash_core.h>
>  #include <asm/io.h>
> 
>  #include <uapi/linux/kexec.h>
> @@ -25,6 +24,7 @@
>  #include <linux/ioport.h>
>  #include <linux/module.h>
>  #include <asm/kexec.h>
> +#include <linux/crash_core.h>
> 
>  /* Verify architecture specific macros are defined */
> 
> @@ -63,15 +63,6 @@
>  #define KEXEC_CORE_NOTE_NAME	CRASH_CORE_NOTE_NAME
> 
>  /*
> - * The per-cpu notes area is a list of notes terminated by a "NULL"
> - * note header.  For kdump, the code in vmcore.c runs in the context
> - * of the second kernel to combine them into one note.
> - */
> -#ifndef KEXEC_NOTE_BYTES
> -#define KEXEC_NOTE_BYTES	CRASH_CORE_NOTE_BYTES
> -#endif
> -
> -/*
>   * This structure is used to hold the arguments that are used when loading
>   * kernel binaries.
>   */



From bp at alien8.de  Wed Jun 21 02:54:48 2017
From: bp at alien8.de (Borislav Petkov)
Date: Wed, 21 Jun 2017 11:54:48 +0200
Subject: [PATCH v7 24/36] x86, swiotlb: Add memory encryption support
In-Reply-To: <20170616185423.18967.19605.stgit@tlendack-t1.amdoffice.net>
References: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
 <20170616185423.18967.19605.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170621095448.6c52kp2eves5uyzy@pd.tnic>

On Fri, Jun 16, 2017 at 01:54:24PM -0500, Tom Lendacky wrote:
> Since DMA addresses will effectively look like 48-bit addresses when the
> memory encryption mask is set, SWIOTLB is needed if the DMA mask of the
> device performing the DMA does not support 48-bits. SWIOTLB will be
> initialized to create decrypted bounce buffers for use by these devices.
> 
> Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
> ---
>  arch/x86/include/asm/dma-mapping.h |    5 ++-
>  arch/x86/include/asm/mem_encrypt.h |    5 +++
>  arch/x86/kernel/pci-dma.c          |   11 +++++--
>  arch/x86/kernel/pci-nommu.c        |    2 +
>  arch/x86/kernel/pci-swiotlb.c      |   15 +++++++++-
>  arch/x86/mm/mem_encrypt.c          |   22 +++++++++++++++
>  include/linux/swiotlb.h            |    1 +
>  init/main.c                        |   10 +++++++
>  lib/swiotlb.c                      |   54 +++++++++++++++++++++++++++++++-----
>  9 files changed, 108 insertions(+), 17 deletions(-)

Reviewed-by: Borislav Petkov <bp at suse.de>

-- 
Regards/Gruss,
    Boris.

Good mailing practices for 400: avoid top-posting and trim the reply.


From bp at alien8.de  Wed Jun 21 02:43:24 2017
From: bp at alien8.de (Borislav Petkov)
Date: Wed, 21 Jun 2017 11:43:24 +0200
Subject: [PATCH v7 23/36] x86, realmode: Decrypt trampoline area if
 memory encryption is active
In-Reply-To: <20170616185412.18967.91559.stgit@tlendack-t1.amdoffice.net>
References: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
 <20170616185412.18967.91559.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170621094324.n3dukli4gq6g5ftn@pd.tnic>

On Fri, Jun 16, 2017 at 01:54:12PM -0500, Tom Lendacky wrote:
> When Secure Memory Encryption is enabled, the trampoline area must not
> be encrypted. A CPU running in real mode will not be able to decrypt
> memory that has been encrypted because it will not be able to use addresses
> with the memory encryption mask.
> 
> Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
> ---
>  arch/x86/realmode/init.c |    8 ++++++++
>  1 file changed, 8 insertions(+)

Subject: x86/realmode: ...

other than that:

Reviewed-by: Borislav Petkov <bp at suse.de>

-- 
Regards/Gruss,
    Boris.

Good mailing practices for 400: avoid top-posting and trim the reply.


From bp at alien8.de  Wed Jun 21 03:50:26 2017
From: bp at alien8.de (Borislav Petkov)
Date: Wed, 21 Jun 2017 12:50:26 +0200
Subject: [PATCH v7 25/36] swiotlb: Add warnings for use of bounce buffers
 with SME
In-Reply-To: <20170616185435.18967.26665.stgit@tlendack-t1.amdoffice.net>
References: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
 <20170616185435.18967.26665.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170621105026.lcbtkklaenyi2wqe@pd.tnic>

On Fri, Jun 16, 2017 at 01:54:36PM -0500, Tom Lendacky wrote:
> Add warnings to let the user know when bounce buffers are being used for
> DMA when SME is active.  Since the bounce buffers are not in encrypted
> memory, these notifications are to allow the user to determine some
> appropriate action - if necessary.  Actions can range from utilizing an
> IOMMU, replacing the device with another device that can support 64-bit
> DMA, ignoring the message if the device isn't used much, etc.
> 
> Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
> ---
>  include/linux/dma-mapping.h |   11 +++++++++++
>  include/linux/mem_encrypt.h |    8 ++++++++
>  lib/swiotlb.c               |    3 +++
>  3 files changed, 22 insertions(+)
> 
> diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h
> index 4f3eece..ee2307e 100644
> --- a/include/linux/dma-mapping.h
> +++ b/include/linux/dma-mapping.h
> @@ -10,6 +10,7 @@
>  #include <linux/scatterlist.h>
>  #include <linux/kmemcheck.h>
>  #include <linux/bug.h>
> +#include <linux/mem_encrypt.h>
>  
>  /**
>   * List of possible attributes associated with a DMA mapping. The semantics
> @@ -577,6 +578,11 @@ static inline int dma_set_mask(struct device *dev, u64 mask)
>  
>  	if (!dev->dma_mask || !dma_supported(dev, mask))
>  		return -EIO;
> +
> +	/* Since mask is unsigned, this can only be true if SME is active */
> +	if (mask < sme_dma_mask())
> +		dev_warn(dev, "SME is active, device will require DMA bounce buffers\n");
> +
>  	*dev->dma_mask = mask;
>  	return 0;
>  }
> @@ -596,6 +602,11 @@ static inline int dma_set_coherent_mask(struct device *dev, u64 mask)
>  {
>  	if (!dma_supported(dev, mask))
>  		return -EIO;
> +
> +	/* Since mask is unsigned, this can only be true if SME is active */
> +	if (mask < sme_dma_mask())
> +		dev_warn(dev, "SME is active, device will require DMA bounce buffers\n");

Looks to me like those two checks above need to be a:

void sme_check_mask(struct device *dev, u64 mask)
{
        if (!sme_me_mask)
                return;

        /* Since mask is unsigned, this can only be true if SME is active */
        if (mask < (((u64)sme_me_mask << 1) - 1))
                dev_warn(dev, "SME is active, device will require DMA bounce buffers\n");
}

which gets called and sme_dma_mask() is not really needed.

-- 
Regards/Gruss,
    Boris.

Good mailing practices for 400: avoid top-posting and trim the reply.


From thomas.lendacky at amd.com  Wed Jun 21 06:26:44 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Wed, 21 Jun 2017 08:26:44 -0500
Subject: [PATCH v7 06/36] x86/mm: Add Secure Memory Encryption (SME)
 support
In-Reply-To: <alpine.DEB.2.20.1706202244480.2157@nanos>
References: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
 <20170616185054.18967.52228.stgit@tlendack-t1.amdoffice.net>
 <alpine.DEB.2.20.1706202244480.2157@nanos>
Message-ID: <efa51470-5d8f-6883-b9da-f2face12ea22@amd.com>

On 6/20/2017 3:49 PM, Thomas Gleixner wrote:
> On Fri, 16 Jun 2017, Tom Lendacky wrote:
>>   
>> +config ARCH_HAS_MEM_ENCRYPT
>> +	def_bool y
>> +	depends on X86
> 
> That one is silly. The config switch is in the x86 KConfig file, so X86 is
> on. If you intended to move this to some generic place outside of
> x86/Kconfig then this should be
> 
> config ARCH_HAS_MEM_ENCRYPT
> 	bool
> 
> and x86/Kconfig should have
> 
>      	select ARCH_HAS_MEM_ENCRYPT
> 
> and that should be selected by AMD_MEM_ENCRYPT

This is used for deciding whether to include the asm/mem_encrypt.h file
so it needs to be on whether AMD_MEM_ENCRYPT is configured or not. I'll
leave it in the x86/Kconfig file and remove the depends on line.

Thanks,
Tom

> 
>> +config AMD_MEM_ENCRYPT
>> +	bool "AMD Secure Memory Encryption (SME) support"
>> +	depends on X86_64 && CPU_SUP_AMD
>> +	---help---
>> +	  Say yes to enable support for the encryption of system memory.
>> +	  This requires an AMD processor that supports Secure Memory
>> +	  Encryption (SME).
> 
> Thanks,
> 
> 	tglx
> 


From thomas.lendacky at amd.com  Wed Jun 21 06:52:53 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Wed, 21 Jun 2017 08:52:53 -0500
Subject: [PATCH v7 07/36] x86/mm: Don't use phys_to_virt in ioremap() if
 SME is active
In-Reply-To: <alpine.DEB.2.20.1706202251110.2157@nanos>
References: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
 <20170616185104.18967.7867.stgit@tlendack-t1.amdoffice.net>
 <alpine.DEB.2.20.1706202251110.2157@nanos>
Message-ID: <c1072435-e3ad-21ea-b36f-505ff80ed599@amd.com>

On 6/20/2017 3:55 PM, Thomas Gleixner wrote:
> On Fri, 16 Jun 2017, Tom Lendacky wrote:
> 
>> Currently there is a check if the address being mapped is in the ISA
>> range (is_ISA_range()), and if it is then phys_to_virt() is used to
>> perform the mapping.  When SME is active, however, this will result
>> in the mapping having the encryption bit set when it is expected that
>> an ioremap() should not have the encryption bit set. So only use the
>> phys_to_virt() function if SME is not active
> 
> This does not make sense to me. What the heck has phys_to_virt() to do with
> the encryption bit. Especially why would the encryption bit be set on that
> mapping in the first place?

The default is that all entries that get added to the pagetables have
the encryption bit set unless specifically overridden.  Any __va() or
phys_to_virt() calls will result in a pagetable mapping that has the
encryption bit set.  For ioremap, the PAGE_KERNEL_IO protection is used
which will not/does not have the encryption bit set.

> 
> I'm probably missing something, but this want's some coherent explanation
> understandable by mere mortals both in the changelog and the code comment.

I'll add some additional info to the changelog and code.

Thanks,
Tom

> 
> Thanks,
> 
> 	tglx
> 


From thomas.lendacky at amd.com  Wed Jun 21 06:54:57 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Wed, 21 Jun 2017 08:54:57 -0500
Subject: [PATCH v7 07/36] x86/mm: Don't use phys_to_virt in ioremap() if
 SME is active
In-Reply-To: <alpine.DEB.2.20.1706210934540.2328@nanos>
References: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
 <20170616185104.18967.7867.stgit@tlendack-t1.amdoffice.net>
 <alpine.DEB.2.20.1706210934540.2328@nanos>
Message-ID: <859437c0-d190-240e-6aa5-5edb0b63aa9e@amd.com>

On 6/21/2017 2:37 AM, Thomas Gleixner wrote:
> On Fri, 16 Jun 2017, Tom Lendacky wrote:
>> Currently there is a check if the address being mapped is in the ISA
>> range (is_ISA_range()), and if it is then phys_to_virt() is used to
>> perform the mapping.  When SME is active, however, this will result
>> in the mapping having the encryption bit set when it is expected that
>> an ioremap() should not have the encryption bit set. So only use the
>> phys_to_virt() function if SME is not active
>>
>> Reviewed-by: Borislav Petkov <bp at suse.de>
>> Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
>> ---
>>   arch/x86/mm/ioremap.c |    7 +++++--
>>   1 file changed, 5 insertions(+), 2 deletions(-)
>>
>> diff --git a/arch/x86/mm/ioremap.c b/arch/x86/mm/ioremap.c
>> index 4c1b5fd..a382ba9 100644
>> --- a/arch/x86/mm/ioremap.c
>> +++ b/arch/x86/mm/ioremap.c
>> @@ -13,6 +13,7 @@
>>   #include <linux/slab.h>
>>   #include <linux/vmalloc.h>
>>   #include <linux/mmiotrace.h>
>> +#include <linux/mem_encrypt.h>
>>   
>>   #include <asm/set_memory.h>
>>   #include <asm/e820/api.h>
>> @@ -106,9 +107,11 @@ static void __iomem *__ioremap_caller(resource_size_t phys_addr,
>>   	}
>>   
>>   	/*
>> -	 * Don't remap the low PCI/ISA area, it's always mapped..
>> +	 * Don't remap the low PCI/ISA area, it's always mapped.
>> +	 *   But if SME is active, skip this so that the encryption bit
>> +	 *   doesn't get set.
>>   	 */
>> -	if (is_ISA_range(phys_addr, last_addr))
>> +	if (is_ISA_range(phys_addr, last_addr) && !sme_active())
>>   		return (__force void __iomem *)phys_to_virt(phys_addr);
> 
> More thoughts about that.
> 
> Making this conditional on !sme_active() is not the best idea. I'd rather
> remove that whole thing and make it unconditional so the code pathes get
> always exercised and any subtle wreckage is detected on a broader base and
> not only on that hard to access and debug SME capable machine owned by Joe
> User.

Ok, that sounds good.  I'll remove the check and usage of phys_to_virt()
and update the changelog with additional detail about that.

Thanks,
Tom

> 
> Thanks,
> 
> 	tglx
> 


From thomas.lendacky at amd.com  Wed Jun 21 08:14:35 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Wed, 21 Jun 2017 10:14:35 -0500
Subject: [PATCH v7 08/36] x86/mm: Add support to enable SME in early boot
 processing
In-Reply-To: <alpine.DEB.2.20.1706202259290.2157@nanos>
References: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
 <20170616185115.18967.79622.stgit@tlendack-t1.amdoffice.net>
 <alpine.DEB.2.20.1706202259290.2157@nanos>
Message-ID: <8d3c215f-cdad-5554-6e9c-5598e1081850@amd.com>

On 6/21/2017 2:16 AM, Thomas Gleixner wrote:
> On Fri, 16 Jun 2017, Tom Lendacky wrote:
>> diff --git a/arch/x86/include/asm/mem_encrypt.h b/arch/x86/include/asm/mem_encrypt.h
>> index a105796..988b336 100644
>> --- a/arch/x86/include/asm/mem_encrypt.h
>> +++ b/arch/x86/include/asm/mem_encrypt.h
>> @@ -15,16 +15,24 @@
>>   
>>   #ifndef __ASSEMBLY__
>>   
>> +#include <linux/init.h>
>> +
>>   #ifdef CONFIG_AMD_MEM_ENCRYPT
>>   
>>   extern unsigned long sme_me_mask;
>>   
>> +void __init sme_enable(void);
>> +
>>   #else	/* !CONFIG_AMD_MEM_ENCRYPT */
>>   
>>   #define sme_me_mask	0UL
>>   
>> +static inline void __init sme_enable(void) { }
>> +
>>   #endif	/* CONFIG_AMD_MEM_ENCRYPT */
>>   
>> +unsigned long sme_get_me_mask(void);
> 
> Why is this an unconditional function? Isn't the mask simply 0 when the MEM
> ENCRYPT support is disabled?

I made it unconditional because of the call from head_64.S. I can't make
use of the C level static inline function and since the mask is not a
variable if CONFIG_AMD_MEM_ENCRYPT is not configured (#defined to 0) I
can't reference the variable directly.

I could create a #define in head_64.S that changes this to load rax with
the variable if CONFIG_AMD_MEM_ENCRYPT is configured or a zero if it's
not or add a #ifdef at that point in the code directly. Thoughts on
that?

> 
>> diff --git a/arch/x86/kernel/head_64.S b/arch/x86/kernel/head_64.S
>> index 6225550..ef12729 100644
>> --- a/arch/x86/kernel/head_64.S
>> +++ b/arch/x86/kernel/head_64.S
>> @@ -78,7 +78,29 @@ startup_64:
>>   	call	__startup_64
>>   	popq	%rsi
>>   
>> -	movq	$(early_top_pgt - __START_KERNEL_map), %rax
>> +	/*
>> +	 * Encrypt the kernel if SME is active.
>> +	 * The real_mode_data address is in %rsi and that register can be
>> +	 * clobbered by the called function so be sure to save it.
>> +	 */
>> +	push	%rsi
>> +	call	sme_encrypt_kernel
>> +	pop	%rsi
> 
> That does not make any sense. Neither the call to sme_encrypt_kernel() nor
> the following call to sme_get_me_mask().
> 
> __startup_64() is already C code, so why can't you simply call that from
> __startup_64() in C and return the mask from there?

I was trying to keep it explicit as to what was happening, but I can
move those calls into __startup_64(). I'll still need the call to
sme_get_me_mask() in the secondary_startup_64 path, though (depending on
your thoughts to the above response).

> 
>> @@ -98,7 +120,20 @@ ENTRY(secondary_startup_64)
>>   	/* Sanitize CPU configuration */
>>   	call verify_cpu
>>   
>> -	movq	$(init_top_pgt - __START_KERNEL_map), %rax
>> +	/*
>> +	 * Get the SME encryption mask.
>> +	 *  The encryption mask will be returned in %rax so we do an ADD
>> +	 *  below to be sure that the encryption mask is part of the
>> +	 *  value that will stored in %cr3.
>> +	 *
>> +	 * The real_mode_data address is in %rsi and that register can be
>> +	 * clobbered by the called function so be sure to save it.
>> +	 */
>> +	push	%rsi
>> +	call	sme_get_me_mask
>> +	pop	%rsi
> 
> Do we really need a call here? The mask is established at this point, so
> it's either 0 when the encryption stuff is not compiled in or it can be
> retrieved from a variable which is accessible at this point.
> 

Same as above, this can be updated based on the decided approach.

Thanks,
Tom

>> +
>> +	addq	$(init_top_pgt - __START_KERNEL_map), %rax
>>   1:
>>   
>>   	/* Enable PAE mode, PGE and LA57 */
> 
> Thanks,
> 
> 	tglx
> 


From thomas.lendacky at amd.com  Wed Jun 21 08:37:41 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Wed, 21 Jun 2017 10:37:41 -0500
Subject: [PATCH v7 25/36] swiotlb: Add warnings for use of bounce buffers
 with SME
In-Reply-To: <20170621105026.lcbtkklaenyi2wqe@pd.tnic>
References: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
 <20170616185435.18967.26665.stgit@tlendack-t1.amdoffice.net>
 <20170621105026.lcbtkklaenyi2wqe@pd.tnic>
Message-ID: <7b9f8b6d-60f2-0a78-b582-0821401d17eb@amd.com>

On 6/21/2017 5:50 AM, Borislav Petkov wrote:
> On Fri, Jun 16, 2017 at 01:54:36PM -0500, Tom Lendacky wrote:
>> Add warnings to let the user know when bounce buffers are being used for
>> DMA when SME is active.  Since the bounce buffers are not in encrypted
>> memory, these notifications are to allow the user to determine some
>> appropriate action - if necessary.  Actions can range from utilizing an
>> IOMMU, replacing the device with another device that can support 64-bit
>> DMA, ignoring the message if the device isn't used much, etc.
>>
>> Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
>> ---
>>   include/linux/dma-mapping.h |   11 +++++++++++
>>   include/linux/mem_encrypt.h |    8 ++++++++
>>   lib/swiotlb.c               |    3 +++
>>   3 files changed, 22 insertions(+)
>>
>> diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h
>> index 4f3eece..ee2307e 100644
>> --- a/include/linux/dma-mapping.h
>> +++ b/include/linux/dma-mapping.h
>> @@ -10,6 +10,7 @@
>>   #include <linux/scatterlist.h>
>>   #include <linux/kmemcheck.h>
>>   #include <linux/bug.h>
>> +#include <linux/mem_encrypt.h>
>>   
>>   /**
>>    * List of possible attributes associated with a DMA mapping. The semantics
>> @@ -577,6 +578,11 @@ static inline int dma_set_mask(struct device *dev, u64 mask)
>>   
>>   	if (!dev->dma_mask || !dma_supported(dev, mask))
>>   		return -EIO;
>> +
>> +	/* Since mask is unsigned, this can only be true if SME is active */
>> +	if (mask < sme_dma_mask())
>> +		dev_warn(dev, "SME is active, device will require DMA bounce buffers\n");
>> +
>>   	*dev->dma_mask = mask;
>>   	return 0;
>>   }
>> @@ -596,6 +602,11 @@ static inline int dma_set_coherent_mask(struct device *dev, u64 mask)
>>   {
>>   	if (!dma_supported(dev, mask))
>>   		return -EIO;
>> +
>> +	/* Since mask is unsigned, this can only be true if SME is active */
>> +	if (mask < sme_dma_mask())
>> +		dev_warn(dev, "SME is active, device will require DMA bounce buffers\n");
> 
> Looks to me like those two checks above need to be a:
> 
> void sme_check_mask(struct device *dev, u64 mask)
> {
>          if (!sme_me_mask)
>                  return;
> 
>          /* Since mask is unsigned, this can only be true if SME is active */
>          if (mask < (((u64)sme_me_mask << 1) - 1))
>                  dev_warn(dev, "SME is active, device will require DMA bounce buffers\n");
> }
> 
> which gets called and sme_dma_mask() is not really needed.

Makes a lot of sense, I'll update the patch.

Thanks,
Tom

> 


From tglx at linutronix.de  Wed Jun 21 08:38:04 2017
From: tglx at linutronix.de (Thomas Gleixner)
Date: Wed, 21 Jun 2017 17:38:04 +0200 (CEST)
Subject: [PATCH v7 08/36] x86/mm: Add support to enable SME in early boot
 processing
In-Reply-To: <8d3c215f-cdad-5554-6e9c-5598e1081850@amd.com>
References: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
 <20170616185115.18967.79622.stgit@tlendack-t1.amdoffice.net>
 <alpine.DEB.2.20.1706202259290.2157@nanos>
 <8d3c215f-cdad-5554-6e9c-5598e1081850@amd.com>
Message-ID: <alpine.DEB.2.20.1706211720060.2328@nanos>

On Wed, 21 Jun 2017, Tom Lendacky wrote:
> On 6/21/2017 2:16 AM, Thomas Gleixner wrote:
> > Why is this an unconditional function? Isn't the mask simply 0 when the MEM
> > ENCRYPT support is disabled?
> 
> I made it unconditional because of the call from head_64.S. I can't make
> use of the C level static inline function and since the mask is not a
> variable if CONFIG_AMD_MEM_ENCRYPT is not configured (#defined to 0) I
> can't reference the variable directly.
> 
> I could create a #define in head_64.S that changes this to load rax with
> the variable if CONFIG_AMD_MEM_ENCRYPT is configured or a zero if it's
> not or add a #ifdef at that point in the code directly. Thoughts on
> that?

See below.

> > That does not make any sense. Neither the call to sme_encrypt_kernel() nor
> > the following call to sme_get_me_mask().
> > 
> > __startup_64() is already C code, so why can't you simply call that from
> > __startup_64() in C and return the mask from there?
> 
> I was trying to keep it explicit as to what was happening, but I can
> move those calls into __startup_64().

That's much preferred. And the return value wants to be documented in both
C and ASM code.

> I'll still need the call to sme_get_me_mask() in the secondary_startup_64
> path, though (depending on your thoughts to the above response).

        call verify_cpu

        movq    $(init_top_pgt - __START_KERNEL_map), %rax

So if you make that:

	/*
	 * Sanitize CPU configuration and retrieve the modifier
	 * for the initial pgdir entry which will be programmed
	 * into CR3. Depends on enabled SME encryption, normally 0.
	 */
	call __startup_secondary_64

        addq    $(init_top_pgt - __START_KERNEL_map), %rax

You can hide that stuff in C-code nicely without adding any cruft to the
ASM code.

Thanks,

	tglx


From bp at alien8.de  Wed Jun 21 09:59:22 2017
From: bp at alien8.de (Borislav Petkov)
Date: Wed, 21 Jun 2017 18:59:22 +0200
Subject: [PATCH v6 26/34] iommu/amd: Allow the AMD IOMMU to work with
 memory encryption
In-Reply-To: <20170621153721.GP30388@8bytes.org>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
 <20170607191745.28645.81756.stgit@tlendack-t1.amdoffice.net>
 <20170614174208.p2yr5exs4b6pjxhf@pd.tnic>
 <0611d01a-19f8-d6ae-2682-932789855518@amd.com>
 <20170615094111.wga334kg2bhxqib3@pd.tnic>
 <20170621153721.GP30388@8bytes.org>
Message-ID: <20170621165921.tv2jfhf5dz7hsjsy@pd.tnic>

On Wed, Jun 21, 2017 at 05:37:22PM +0200, Joerg Roedel wrote:
> > Do you mean this is like the last exception case in that document above:
> > 
> > "
> >   - Pointers to data structures in coherent memory which might be modified
> >     by I/O devices can, sometimes, legitimately be volatile.  A ring buffer
> >     used by a network adapter, where that adapter changes pointers to
> >     indicate which descriptors have been processed, is an example of this
> >     type of situation."
> > 
> > ?
> 
> So currently (without this patch) the build_completion_wait function
> does not take a volatile parameter, only wait_on_sem() does.
> 
> Wait_on_sem() needs it because its purpose is to poll a memory location
> which is changed by the iommu-hardware when its done with command
> processing.

Right, the reason above - memory modifiable by an IO device. You could
add a comment there explaining the need for the volatile.

> But the 'volatile' in build_completion_wait() looks unnecessary, because
> the function does not poll the memory location. It only uses the
> pointer, converts it to a physical address and writes it to the command
> to be queued.

Ok.

Thanks.

-- 
Regards/Gruss,
    Boris.

Good mailing practices for 400: avoid top-posting and trim the reply.


From holzheu at linux.vnet.ibm.com  Wed Jun 21 10:44:23 2017
From: holzheu at linux.vnet.ibm.com (Michael Holzheu)
Date: Wed, 21 Jun 2017 19:44:23 +0200
Subject: [PATCH] s390/crash: Fix KEXEC_NOTE_BYTES definition
In-Reply-To: <1496974625-10891-1-git-send-email-xlpang@redhat.com>
References: <1496974625-10891-1-git-send-email-xlpang@redhat.com>
Message-ID: <20170621194423.2d0318d7@TP-holzheu>

Am Fri,  9 Jun 2017 10:17:05 +0800
schrieb Xunlei Pang <xlpang at redhat.com>:

> S390 KEXEC_NOTE_BYTES is not used by note_buf_t as before, which
> is now defined as follows:
>     typedef u32 note_buf_t[CRASH_CORE_NOTE_BYTES/4];
> It was changed by the CONFIG_CRASH_CORE feature.
> 
> This patch gets rid of all the old KEXEC_NOTE_BYTES stuff, and
> renames KEXEC_NOTE_BYTES to CRASH_CORE_NOTE_BYTES for S390.
> 
> Fixes: 692f66f26a4c ("crash: move crashkernel parsing and vmcore related code under CONFIG_CRASH_CORE")
> Cc: Dave Young <dyoung at redhat.com>
> Cc: Dave Anderson <anderson at redhat.com>
> Cc: Hari Bathini <hbathini at linux.vnet.ibm.com>
> Cc: Gustavo Luiz Duarte <gustavold at linux.vnet.ibm.com>
> Signed-off-by: Xunlei Pang <xlpang at redhat.com>

Hello Xunlei,

As you already know on s390 we create the ELF header in the new kernel.
Therefore we don't use the per-cpu buffers for ELF notes to store
the register state.

For RHEL7 we still store the registers in machine_kexec.c:add_elf_notes().
Though we also use the ELF header from new kernel ...

We assume your original problem with the "kmem -s" failure
was caused by the memory overwrite due to the invalid size of the
"crash_notes" per-cpu buffers.

Therefore your patch looks good for RHEL7 but for upstream we propose the
patch below.
---
[PATCH] s390/crash: Remove unused KEXEC_NOTE_BYTES

After commmit 692f66f26a4c19 ("crash: move crashkernel parsing and vmcore
related code under CONFIG_CRASH_CORE") the KEXEC_NOTE_BYTES macro is not
used anymore and for s390 we create the ELF header in the new kernel
anyway. Therefore remove the macro.

Reported-by: Xunlei Pang <xpang at redhat.com>
Reviewed-by: Mikhail Zaslonko <zaslonko at linux.vnet.ibm.com>
Signed-off-by: Michael Holzheu <holzheu at linux.vnet.ibm.com>
---
 arch/s390/include/asm/kexec.h | 18 ------------------
 include/linux/crash_core.h    |  5 +++++
 include/linux/kexec.h         |  9 ---------
 3 files changed, 5 insertions(+), 27 deletions(-)

diff --git a/arch/s390/include/asm/kexec.h b/arch/s390/include/asm/kexec.h
index 2f924bc30e35..dccf24ee26d3 100644
--- a/arch/s390/include/asm/kexec.h
+++ b/arch/s390/include/asm/kexec.h
@@ -41,24 +41,6 @@
 /* The native architecture */
 #define KEXEC_ARCH KEXEC_ARCH_S390
 
-/*
- * Size for s390x ELF notes per CPU
- *
- * Seven notes plus zero note at the end: prstatus, fpregset, timer,
- * tod_cmp, tod_reg, control regs, and prefix
- */
-#define KEXEC_NOTE_BYTES \
-	(ALIGN(sizeof(struct elf_note), 4) * 8 + \
-	 ALIGN(sizeof("CORE"), 4) * 7 + \
-	 ALIGN(sizeof(struct elf_prstatus), 4) + \
-	 ALIGN(sizeof(elf_fpregset_t), 4) + \
-	 ALIGN(sizeof(u64), 4) + \
-	 ALIGN(sizeof(u64), 4) + \
-	 ALIGN(sizeof(u32), 4) + \
-	 ALIGN(sizeof(u64) * 16, 4) + \
-	 ALIGN(sizeof(u32), 4) \
-	)
-
 /* Provide a dummy definition to avoid build failures. */
 static inline void crash_setup_regs(struct pt_regs *newregs,
 					struct pt_regs *oldregs) { }
diff --git a/include/linux/crash_core.h b/include/linux/crash_core.h
index 541a197ba4a2..4090a42578a8 100644
--- a/include/linux/crash_core.h
+++ b/include/linux/crash_core.h
@@ -10,6 +10,11 @@
 #define CRASH_CORE_NOTE_NAME_BYTES ALIGN(sizeof(CRASH_CORE_NOTE_NAME), 4)
 #define CRASH_CORE_NOTE_DESC_BYTES ALIGN(sizeof(struct elf_prstatus), 4)
 
+/*
+ * The per-cpu notes area is a list of notes terminated by a "NULL"
+ * note header.  For kdump, the code in vmcore.c runs in the context
+ * of the second kernel to combine them into one note.
+ */
 #define CRASH_CORE_NOTE_BYTES	   ((CRASH_CORE_NOTE_HEAD_BYTES * 2) +	\
 				     CRASH_CORE_NOTE_NAME_BYTES +	\
 				     CRASH_CORE_NOTE_DESC_BYTES)
diff --git a/include/linux/kexec.h b/include/linux/kexec.h
index c9481ebcbc0c..65888418fb69 100644
--- a/include/linux/kexec.h
+++ b/include/linux/kexec.h
@@ -63,15 +63,6 @@
 #define KEXEC_CORE_NOTE_NAME	CRASH_CORE_NOTE_NAME
 
 /*
- * The per-cpu notes area is a list of notes terminated by a "NULL"
- * note header.  For kdump, the code in vmcore.c runs in the context
- * of the second kernel to combine them into one note.
- */
-#ifndef KEXEC_NOTE_BYTES
-#define KEXEC_NOTE_BYTES	CRASH_CORE_NOTE_BYTES
-#endif
-
-/*
  * This structure is used to hold the arguments that are used when loading
  * kernel binaries.
  */
-- 
2.11.2



From thomas.lendacky at amd.com  Wed Jun 21 11:30:19 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Wed, 21 Jun 2017 13:30:19 -0500
Subject: [PATCH v7 08/36] x86/mm: Add support to enable SME in early boot
 processing
In-Reply-To: <alpine.DEB.2.20.1706211720060.2328@nanos>
References: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
 <20170616185115.18967.79622.stgit@tlendack-t1.amdoffice.net>
 <alpine.DEB.2.20.1706202259290.2157@nanos>
 <8d3c215f-cdad-5554-6e9c-5598e1081850@amd.com>
 <alpine.DEB.2.20.1706211720060.2328@nanos>
Message-ID: <fc697503-ec54-f481-36b3-3d5bf63aaaee@amd.com>

On 6/21/2017 10:38 AM, Thomas Gleixner wrote:
> On Wed, 21 Jun 2017, Tom Lendacky wrote:
>> On 6/21/2017 2:16 AM, Thomas Gleixner wrote:
>>> Why is this an unconditional function? Isn't the mask simply 0 when the MEM
>>> ENCRYPT support is disabled?
>>
>> I made it unconditional because of the call from head_64.S. I can't make
>> use of the C level static inline function and since the mask is not a
>> variable if CONFIG_AMD_MEM_ENCRYPT is not configured (#defined to 0) I
>> can't reference the variable directly.
>>
>> I could create a #define in head_64.S that changes this to load rax with
>> the variable if CONFIG_AMD_MEM_ENCRYPT is configured or a zero if it's
>> not or add a #ifdef at that point in the code directly. Thoughts on
>> that?
> 
> See below.
> 
>>> That does not make any sense. Neither the call to sme_encrypt_kernel() nor
>>> the following call to sme_get_me_mask().
>>>
>>> __startup_64() is already C code, so why can't you simply call that from
>>> __startup_64() in C and return the mask from there?
>>
>> I was trying to keep it explicit as to what was happening, but I can
>> move those calls into __startup_64().
> 
> That's much preferred. And the return value wants to be documented in both
> C and ASM code.

Will do.

> 
>> I'll still need the call to sme_get_me_mask() in the secondary_startup_64
>> path, though (depending on your thoughts to the above response).
> 
>          call verify_cpu
> 
>          movq    $(init_top_pgt - __START_KERNEL_map), %rax
> 
> So if you make that:
> 
> 	/*
> 	 * Sanitize CPU configuration and retrieve the modifier
> 	 * for the initial pgdir entry which will be programmed
> 	 * into CR3. Depends on enabled SME encryption, normally 0.
> 	 */
> 	call __startup_secondary_64
> 
>          addq    $(init_top_pgt - __START_KERNEL_map), %rax
> 
> You can hide that stuff in C-code nicely without adding any cruft to the
> ASM code.
> 

Moving the call to verify_cpu into the C-code might be quite a bit of
change.  Currently, the verify_cpu code is included code and not a
global function.  I can still do the __startup_secondary_64() function
and then look to incorporate verify_cpu into both __startup_64() and
__startup_secondary_64() as a post-patch to this series. At least the
secondary path will have a base C routine to which modifications can
be made in the future if needed.  How does that sound?

Thanks,
Tom

> Thanks,
> 
> 	tglx
> 


From thomas.lendacky at amd.com  Wed Jun 21 11:40:14 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Wed, 21 Jun 2017 13:40:14 -0500
Subject: [PATCH v6 26/34] iommu/amd: Allow the AMD IOMMU to work with
 memory encryption
In-Reply-To: <20170621165921.tv2jfhf5dz7hsjsy@pd.tnic>
References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net>
 <20170607191745.28645.81756.stgit@tlendack-t1.amdoffice.net>
 <20170614174208.p2yr5exs4b6pjxhf@pd.tnic>
 <0611d01a-19f8-d6ae-2682-932789855518@amd.com>
 <20170615094111.wga334kg2bhxqib3@pd.tnic> <20170621153721.GP30388@8bytes.org>
 <20170621165921.tv2jfhf5dz7hsjsy@pd.tnic>
Message-ID: <26d48eb1-4d33-4abf-f169-3ce86aef22fe@amd.com>

On 6/21/2017 11:59 AM, Borislav Petkov wrote:
> On Wed, Jun 21, 2017 at 05:37:22PM +0200, Joerg Roedel wrote:
>>> Do you mean this is like the last exception case in that document above:
>>>
>>> "
>>>    - Pointers to data structures in coherent memory which might be modified
>>>      by I/O devices can, sometimes, legitimately be volatile.  A ring buffer
>>>      used by a network adapter, where that adapter changes pointers to
>>>      indicate which descriptors have been processed, is an example of this
>>>      type of situation."
>>>
>>> ?
>>
>> So currently (without this patch) the build_completion_wait function
>> does not take a volatile parameter, only wait_on_sem() does.
>>
>> Wait_on_sem() needs it because its purpose is to poll a memory location
>> which is changed by the iommu-hardware when its done with command
>> processing.
> 
> Right, the reason above - memory modifiable by an IO device. You could
> add a comment there explaining the need for the volatile.
> 
>> But the 'volatile' in build_completion_wait() looks unnecessary, because
>> the function does not poll the memory location. It only uses the
>> pointer, converts it to a physical address and writes it to the command
>> to be queued.
> 
> Ok.

Ok, so the (now) current version of the patch that doesn't change the
function signature is the right way to go.

Thanks,
Tom

> 
> Thanks.
> 


From tglx at linutronix.de  Wed Jun 21 11:52:47 2017
From: tglx at linutronix.de (Thomas Gleixner)
Date: Wed, 21 Jun 2017 20:52:47 +0200 (CEST)
Subject: [PATCH v7 08/36] x86/mm: Add support to enable SME in early boot
 processing
In-Reply-To: <fc697503-ec54-f481-36b3-3d5bf63aaaee@amd.com>
References: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
 <20170616185115.18967.79622.stgit@tlendack-t1.amdoffice.net>
 <alpine.DEB.2.20.1706202259290.2157@nanos>
 <8d3c215f-cdad-5554-6e9c-5598e1081850@amd.com>
 <alpine.DEB.2.20.1706211720060.2328@nanos>
 <fc697503-ec54-f481-36b3-3d5bf63aaaee@amd.com>
Message-ID: <alpine.DEB.2.20.1706212051120.2152@nanos>

On Wed, 21 Jun 2017, Tom Lendacky wrote:
> On 6/21/2017 10:38 AM, Thomas Gleixner wrote:
> > 	/*
> > 	 * Sanitize CPU configuration and retrieve the modifier
> > 	 * for the initial pgdir entry which will be programmed
> > 	 * into CR3. Depends on enabled SME encryption, normally 0.
> > 	 */
> > 	call __startup_secondary_64
> > 
> >          addq    $(init_top_pgt - __START_KERNEL_map), %rax
> > 
> > You can hide that stuff in C-code nicely without adding any cruft to the
> > ASM code.
> > 
> 
> Moving the call to verify_cpu into the C-code might be quite a bit of
> change.  Currently, the verify_cpu code is included code and not a
> global function.

Ah. Ok. I missed that.

> I can still do the __startup_secondary_64() function and then look to
> incorporate verify_cpu into both __startup_64() and
> __startup_secondary_64() as a post-patch to this series.

Yes, just having __startup_secondary_64() for now and there the extra bits
for that encryption stuff is fine.

> At least the secondary path will have a base C routine to which
> modifications can be made in the future if needed.  How does that sound?

Sounds like a plan.


From xpang at redhat.com  Thu Jun 22 02:12:07 2017
From: xpang at redhat.com (Xunlei Pang)
Date: Thu, 22 Jun 2017 17:12:07 +0800
Subject: [PATCH] s390/crash: Fix KEXEC_NOTE_BYTES definition
In-Reply-To: <20170621194423.2d0318d7@TP-holzheu>
References: <1496974625-10891-1-git-send-email-xlpang@redhat.com>
 <20170621194423.2d0318d7@TP-holzheu>
Message-ID: <594B89E7.9020501@redhat.com>

On 06/22/2017 at 01:44 AM, Michael Holzheu wrote:
> Am Fri,  9 Jun 2017 10:17:05 +0800
> schrieb Xunlei Pang <xlpang at redhat.com>:
>
>> S390 KEXEC_NOTE_BYTES is not used by note_buf_t as before, which
>> is now defined as follows:
>>     typedef u32 note_buf_t[CRASH_CORE_NOTE_BYTES/4];
>> It was changed by the CONFIG_CRASH_CORE feature.
>>
>> This patch gets rid of all the old KEXEC_NOTE_BYTES stuff, and
>> renames KEXEC_NOTE_BYTES to CRASH_CORE_NOTE_BYTES for S390.
>>
>> Fixes: 692f66f26a4c ("crash: move crashkernel parsing and vmcore related code under CONFIG_CRASH_CORE")
>> Cc: Dave Young <dyoung at redhat.com>
>> Cc: Dave Anderson <anderson at redhat.com>
>> Cc: Hari Bathini <hbathini at linux.vnet.ibm.com>
>> Cc: Gustavo Luiz Duarte <gustavold at linux.vnet.ibm.com>
>> Signed-off-by: Xunlei Pang <xlpang at redhat.com>
> Hello Xunlei,
>
> As you already know on s390 we create the ELF header in the new kernel.
> Therefore we don't use the per-cpu buffers for ELF notes to store
> the register state.
>
> For RHEL7 we still store the registers in machine_kexec.c:add_elf_notes().
> Though we also use the ELF header from new kernel ...
>
> We assume your original problem with the "kmem -s" failure
> was caused by the memory overwrite due to the invalid size of the
> "crash_notes" per-cpu buffers.
>
> Therefore your patch looks good for RHEL7 but for upstream we propose the
> patch below.

Hi Michael,

Yes, we already did this way.
Thanks for the confirmation, the patch below looks good to me.

Regards,
Xunlei

> ---
> [PATCH] s390/crash: Remove unused KEXEC_NOTE_BYTES
>
> After commmit 692f66f26a4c19 ("crash: move crashkernel parsing and vmcore
> related code under CONFIG_CRASH_CORE") the KEXEC_NOTE_BYTES macro is not
> used anymore and for s390 we create the ELF header in the new kernel
> anyway. Therefore remove the macro.
>
> Reported-by: Xunlei Pang <xpang at redhat.com>
> Reviewed-by: Mikhail Zaslonko <zaslonko at linux.vnet.ibm.com>
> Signed-off-by: Michael Holzheu <holzheu at linux.vnet.ibm.com>
> ---
>  arch/s390/include/asm/kexec.h | 18 ------------------
>  include/linux/crash_core.h    |  5 +++++
>  include/linux/kexec.h         |  9 ---------
>  3 files changed, 5 insertions(+), 27 deletions(-)
>
> diff --git a/arch/s390/include/asm/kexec.h b/arch/s390/include/asm/kexec.h
> index 2f924bc30e35..dccf24ee26d3 100644
> --- a/arch/s390/include/asm/kexec.h
> +++ b/arch/s390/include/asm/kexec.h
> @@ -41,24 +41,6 @@
>  /* The native architecture */
>  #define KEXEC_ARCH KEXEC_ARCH_S390
>  
> -/*
> - * Size for s390x ELF notes per CPU
> - *
> - * Seven notes plus zero note at the end: prstatus, fpregset, timer,
> - * tod_cmp, tod_reg, control regs, and prefix
> - */
> -#define KEXEC_NOTE_BYTES \
> -	(ALIGN(sizeof(struct elf_note), 4) * 8 + \
> -	 ALIGN(sizeof("CORE"), 4) * 7 + \
> -	 ALIGN(sizeof(struct elf_prstatus), 4) + \
> -	 ALIGN(sizeof(elf_fpregset_t), 4) + \
> -	 ALIGN(sizeof(u64), 4) + \
> -	 ALIGN(sizeof(u64), 4) + \
> -	 ALIGN(sizeof(u32), 4) + \
> -	 ALIGN(sizeof(u64) * 16, 4) + \
> -	 ALIGN(sizeof(u32), 4) \
> -	)
> -
>  /* Provide a dummy definition to avoid build failures. */
>  static inline void crash_setup_regs(struct pt_regs *newregs,
>  					struct pt_regs *oldregs) { }
> diff --git a/include/linux/crash_core.h b/include/linux/crash_core.h
> index 541a197ba4a2..4090a42578a8 100644
> --- a/include/linux/crash_core.h
> +++ b/include/linux/crash_core.h
> @@ -10,6 +10,11 @@
>  #define CRASH_CORE_NOTE_NAME_BYTES ALIGN(sizeof(CRASH_CORE_NOTE_NAME), 4)
>  #define CRASH_CORE_NOTE_DESC_BYTES ALIGN(sizeof(struct elf_prstatus), 4)
>  
> +/*
> + * The per-cpu notes area is a list of notes terminated by a "NULL"
> + * note header.  For kdump, the code in vmcore.c runs in the context
> + * of the second kernel to combine them into one note.
> + */
>  #define CRASH_CORE_NOTE_BYTES	   ((CRASH_CORE_NOTE_HEAD_BYTES * 2) +	\
>  				     CRASH_CORE_NOTE_NAME_BYTES +	\
>  				     CRASH_CORE_NOTE_DESC_BYTES)
> diff --git a/include/linux/kexec.h b/include/linux/kexec.h
> index c9481ebcbc0c..65888418fb69 100644
> --- a/include/linux/kexec.h
> +++ b/include/linux/kexec.h
> @@ -63,15 +63,6 @@
>  #define KEXEC_CORE_NOTE_NAME	CRASH_CORE_NOTE_NAME
>  
>  /*
> - * The per-cpu notes area is a list of notes terminated by a "NULL"
> - * note header.  For kdump, the code in vmcore.c runs in the context
> - * of the second kernel to combine them into one note.
> - */
> -#ifndef KEXEC_NOTE_BYTES
> -#define KEXEC_NOTE_BYTES	CRASH_CORE_NOTE_BYTES
> -#endif
> -
> -/*
>   * This structure is used to hold the arguments that are used when loading
>   * kernel binaries.
>   */



From bp at alien8.de  Thu Jun 22 02:40:33 2017
From: bp at alien8.de (Borislav Petkov)
Date: Thu, 22 Jun 2017 11:40:33 +0200
Subject: [PATCH v7 26/36] x86/CPU/AMD: Make the microcode level available
 earlier in the boot
In-Reply-To: <20170616185447.18967.8792.stgit@tlendack-t1.amdoffice.net>
References: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
 <20170616185447.18967.8792.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170622094033.gaqsujzi7ocpemko@pd.tnic>

On Fri, Jun 16, 2017 at 01:54:47PM -0500, Tom Lendacky wrote:
> Move the setting of the cpuinfo_x86.microcode field from amd_init() to
> early_amd_init() so that it is available earlier in the boot process. This
> avoids having to read MSR_AMD64_PATCH_LEVEL directly during early boot.
> 
> Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
> ---
>  arch/x86/kernel/cpu/amd.c |    8 ++++----
>  1 file changed, 4 insertions(+), 4 deletions(-)

Reviewed-by: Borislav Petkov <bp at suse.de>

-- 
Regards/Gruss,
    Boris.

Good mailing practices for 400: avoid top-posting and trim the reply.


From matt at codeblueprint.co.uk  Thu Jun 22 03:03:13 2017
From: matt at codeblueprint.co.uk (Matt Fleming)
Date: Thu, 22 Jun 2017 11:03:13 +0100
Subject: [PATCH v7 17/36] efi: Update efi_mem_type() to return an error
 rather than 0
In-Reply-To: <20170616185306.18967.8964.stgit@tlendack-t1.amdoffice.net>
References: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
 <20170616185306.18967.8964.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170622100313.GB3238@codeblueprint.co.uk>

On Fri, 16 Jun, at 01:53:06PM, Tom Lendacky wrote:
> The efi_mem_type() function currently returns a 0, which maps to
> EFI_RESERVED_TYPE, if the function is unable to find a memmap entry for
> the supplied physical address. Returning EFI_RESERVED_TYPE implies that
> a memmap entry exists, when it doesn't.  Instead of returning 0, change
> the function to return a negative error value when no memmap entry is
> found.
> 
> Reviewed-by: Borislav Petkov <bp at suse.de>
> Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
> ---
>  arch/ia64/kernel/efi.c      |    4 ++--
>  arch/x86/platform/efi/efi.c |    6 +++---
>  include/linux/efi.h         |    2 +-
>  3 files changed, 6 insertions(+), 6 deletions(-)

Reviewed-by: Matt Fleming <matt at codeblueprint.co.uk>


From bp at alien8.de  Thu Jun 22 03:56:37 2017
From: bp at alien8.de (Borislav Petkov)
Date: Thu, 22 Jun 2017 12:56:37 +0200
Subject: [PATCH v7 27/36] iommu/amd: Allow the AMD IOMMU to work with
 memory encryption
In-Reply-To: <20170616185459.18967.72790.stgit@tlendack-t1.amdoffice.net>
References: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
 <20170616185459.18967.72790.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170622105637.g7twdaae2v5eaown@pd.tnic>

On Fri, Jun 16, 2017 at 01:54:59PM -0500, Tom Lendacky wrote:
> The IOMMU is programmed with physical addresses for the various tables
> and buffers that are used to communicate between the device and the
> driver. When the driver allocates this memory it is encrypted. In order
> for the IOMMU to access the memory as encrypted the encryption mask needs
> to be included in these physical addresses during configuration.
> 
> The PTE entries created by the IOMMU should also include the encryption
> mask so that when the device behind the IOMMU performs a DMA, the DMA
> will be performed to encrypted memory.
> 
> Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
> ---
>  drivers/iommu/amd_iommu.c       |   30 ++++++++++++++++--------------
>  drivers/iommu/amd_iommu_init.c  |   34 ++++++++++++++++++++++++++++------
>  drivers/iommu/amd_iommu_proto.h |   10 ++++++++++
>  drivers/iommu/amd_iommu_types.h |    2 +-
>  4 files changed, 55 insertions(+), 21 deletions(-)

Reviewed-by: Borislav Petkov <bp at suse.de>

Btw, I'm assuming the virt_to_phys() difference on SME systems is only
needed in a handful of places. Otherwise, I'd suggest changing the
virt_to_phys() function/macro directly. But I guess most of the places
need the real physical address without the enc bit.

-- 
Regards/Gruss,
    Boris.

Good mailing practices for 400: avoid top-posting and trim the reply.


From matt at codeblueprint.co.uk  Thu Jun 22 03:57:20 2017
From: matt at codeblueprint.co.uk (Matt Fleming)
Date: Thu, 22 Jun 2017 11:57:20 +0100
Subject: [PATCH v7 16/36] efi: Add an EFI table address match function
In-Reply-To: <20170616185253.18967.55724.stgit@tlendack-t1.amdoffice.net>
References: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
 <20170616185253.18967.55724.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170622105720.GA3129@codeblueprint.co.uk>

On Fri, 16 Jun, at 01:52:53PM, Tom Lendacky wrote:
> Add a function that will determine if a supplied physical address matches
> the address of an EFI table.
> 
> Reviewed-by: Borislav Petkov <bp at suse.de>
> Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
> ---
>  drivers/firmware/efi/efi.c |   33 +++++++++++++++++++++++++++++++++
>  include/linux/efi.h        |    7 +++++++
>  2 files changed, 40 insertions(+)

Reviewed-by: Matt Fleming <matt at codeblueprint.co.uk>


From matt at codeblueprint.co.uk  Thu Jun 22 03:58:30 2017
From: matt at codeblueprint.co.uk (Matt Fleming)
Date: Thu, 22 Jun 2017 11:58:30 +0100
Subject: [PATCH v7 18/36] x86/efi: Update EFI pagetable creation to work
 with SME
In-Reply-To: <20170616185317.18967.99499.stgit@tlendack-t1.amdoffice.net>
References: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
 <20170616185317.18967.99499.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170622105830.GB3129@codeblueprint.co.uk>

On Fri, 16 Jun, at 01:53:17PM, Tom Lendacky wrote:
> When SME is active, pagetable entries created for EFI need to have the
> encryption mask set as necessary.
> 
> When the new pagetable pages are allocated they are mapped encrypted. So,
> update the efi_pgt value that will be used in cr3 to include the encryption
> mask so that the PGD table can be read successfully. The pagetable mapping
> as well as the kernel are also added to the pagetable mapping as encrypted.
> All other EFI mappings are mapped decrypted (tables, etc.).
> 
> Reviewed-by: Borislav Petkov <bp at suse.de>
> Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
> ---
>  arch/x86/platform/efi/efi_64.c |   15 +++++++++++----
>  1 file changed, 11 insertions(+), 4 deletions(-)
 
Reviewed-by: Matt Fleming <matt at codeblueprint.co.uk>


From matt at codeblueprint.co.uk  Thu Jun 22 04:04:05 2017
From: matt at codeblueprint.co.uk (Matt Fleming)
Date: Thu, 22 Jun 2017 12:04:05 +0100
Subject: [PATCH v7 19/36] x86/mm: Add support to access boot related data
 in the clear
In-Reply-To: <20170616185326.18967.43278.stgit@tlendack-t1.amdoffice.net>
References: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
 <20170616185326.18967.43278.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170622110405.GC3129@codeblueprint.co.uk>

On Fri, 16 Jun, at 01:53:26PM, Tom Lendacky wrote:
> Boot data (such as EFI related data) is not encrypted when the system is
> booted because UEFI/BIOS does not run with SME active. In order to access
> this data properly it needs to be mapped decrypted.
> 
> Update early_memremap() to provide an arch specific routine to modify the
> pagetable protection attributes before they are applied to the new
> mapping. This is used to remove the encryption mask for boot related data.
> 
> Update memremap() to provide an arch specific routine to determine if RAM
> remapping is allowed.  RAM remapping will cause an encrypted mapping to be
> generated. By preventing RAM remapping, ioremap_cache() will be used
> instead, which will provide a decrypted mapping of the boot related data.
> 
> Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
> ---
>  arch/x86/include/asm/io.h |    5 +
>  arch/x86/mm/ioremap.c     |  179 +++++++++++++++++++++++++++++++++++++++++++++
>  include/linux/io.h        |    2 +
>  kernel/memremap.c         |   20 ++++-
>  mm/early_ioremap.c        |   18 ++++-
>  5 files changed, 217 insertions(+), 7 deletions(-)

Reviewed-by: Matt Fleming <matt at codeblueprint.co.uk>


From thomas.lendacky at amd.com  Thu Jun 22 09:48:46 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Thu, 22 Jun 2017 11:48:46 -0500
Subject: [PATCH v7 27/36] iommu/amd: Allow the AMD IOMMU to work with
 memory encryption
In-Reply-To: <20170622105637.g7twdaae2v5eaown@pd.tnic>
References: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
 <20170616185459.18967.72790.stgit@tlendack-t1.amdoffice.net>
 <20170622105637.g7twdaae2v5eaown@pd.tnic>
Message-ID: <379702e6-31f1-2df1-8889-3498241aea49@amd.com>

On 6/22/2017 5:56 AM, Borislav Petkov wrote:
> On Fri, Jun 16, 2017 at 01:54:59PM -0500, Tom Lendacky wrote:
>> The IOMMU is programmed with physical addresses for the various tables
>> and buffers that are used to communicate between the device and the
>> driver. When the driver allocates this memory it is encrypted. In order
>> for the IOMMU to access the memory as encrypted the encryption mask needs
>> to be included in these physical addresses during configuration.
>>
>> The PTE entries created by the IOMMU should also include the encryption
>> mask so that when the device behind the IOMMU performs a DMA, the DMA
>> will be performed to encrypted memory.
>>
>> Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
>> ---
>>   drivers/iommu/amd_iommu.c       |   30 ++++++++++++++++--------------
>>   drivers/iommu/amd_iommu_init.c  |   34 ++++++++++++++++++++++++++++------
>>   drivers/iommu/amd_iommu_proto.h |   10 ++++++++++
>>   drivers/iommu/amd_iommu_types.h |    2 +-
>>   4 files changed, 55 insertions(+), 21 deletions(-)
> 
> Reviewed-by: Borislav Petkov <bp at suse.de>
> 
> Btw, I'm assuming the virt_to_phys() difference on SME systems is only
> needed in a handful of places. Otherwise, I'd suggest changing the
> virt_to_phys() function/macro directly. But I guess most of the places
> need the real physical address without the enc bit.

Correct.

Thanks,
Tom

> 


From bp at alien8.de  Fri Jun 23 01:56:07 2017
From: bp at alien8.de (Borislav Petkov)
Date: Fri, 23 Jun 2017 10:56:07 +0200
Subject: [PATCH v7 31/36] x86/mm, kexec: Allow kexec to be used with SME
In-Reply-To: <20170616185545.18967.90815.stgit@tlendack-t1.amdoffice.net>
References: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
 <20170616185545.18967.90815.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170623085606.ett5nuiow2ye7p3a@pd.tnic>

On Fri, Jun 16, 2017 at 01:55:45PM -0500, Tom Lendacky wrote:
> Provide support so that kexec can be used to boot a kernel when SME is
> enabled.
> 
> Support is needed to allocate pages for kexec without encryption.  This
> is needed in order to be able to reboot in the kernel in the same manner
> as originally booted.
> 
> Additionally, when shutting down all of the CPUs we need to be sure to
> flush the caches and then halt. This is needed when booting from a state
> where SME was not active into a state where SME is active (or vice-versa).
> Without these steps, it is possible for cache lines to exist for the same
> physical location but tagged both with and without the encryption bit. This
> can cause random memory corruption when caches are flushed depending on
> which cacheline is written last.
> 
> Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
> ---
>  arch/x86/include/asm/init.h          |    1 +
>  arch/x86/include/asm/kexec.h         |    8 ++++++++
>  arch/x86/include/asm/pgtable_types.h |    1 +
>  arch/x86/kernel/machine_kexec_64.c   |   22 +++++++++++++++++++++-
>  arch/x86/kernel/process.c            |   17 +++++++++++++++--
>  arch/x86/mm/ident_map.c              |   12 ++++++++----
>  include/linux/kexec.h                |   14 ++++++++++++++
>  kernel/kexec_core.c                  |   12 +++++++++++-
>  8 files changed, 79 insertions(+), 8 deletions(-)

...

> diff --git a/include/linux/kexec.h b/include/linux/kexec.h
> index c9481eb..5d17fd6 100644
> --- a/include/linux/kexec.h
> +++ b/include/linux/kexec.h
> @@ -334,6 +334,20 @@ static inline void *boot_phys_to_virt(unsigned long entry)
>  	return phys_to_virt(boot_phys_to_phys(entry));
>  }
>  
> +#ifndef arch_kexec_post_alloc_pages
> +static inline int arch_kexec_post_alloc_pages(void *vaddr, unsigned int pages,
> +					      gfp_t gfp)
> +{
> +	return 0;
> +}
> +#endif

Just a nitpick:

static inline int arch_kexec_post_alloc_pages(void *vaddr, unsigned int pages, gfp_t gfp) { return 0; }
static inline void arch_kexec_pre_free_pages(void *vaddr, unsigned int pages) { }

Other than that:

Reviewed-by: Borislav Petkov <bp at suse.de>

-- 
Regards/Gruss,
    Boris.

Good mailing practices for 400: avoid top-posting and trim the reply.


From hoeun.ryu at gmail.com  Fri Jun 23 01:55:38 2017
From: hoeun.ryu at gmail.com (Hoeun Ryu)
Date: Fri, 23 Jun 2017 17:55:38 +0900
Subject: [PATCH] kexec:arm: support zImage with appended device tree
Message-ID: <1498208138-19283-1-git-send-email-hoeun.ryu@gmail.com>

Arm linux supports zImage with appended dtb (CONFIG_ARM_APPENDED_DTB) and
the concatenated image is generated like `cat zImage dtb > zImage_w_dtb`.

This patch is to support the concatednated zImage. This changes the
priority of source of dtb file.

 1. --dtb dtb_file
 2. zImage_w_dtb	<= newly added
 3. /sys/firmware/fdt
 4. /proc/device-tree

Users don't need to specify dtb file in the command line and don't have to
have /sys/firmware/fdt or /proc/device-tree if dtb is available in zImage.

Signed-off-by: Hoeun Ryu <hoeun.ryu at gmail.com>
---
 kexec/arch/arm/kexec-arm.h        |  1 +
 kexec/arch/arm/kexec-zImage-arm.c | 47 ++++++++++++++++++++++++++++++---------
 2 files changed, 37 insertions(+), 11 deletions(-)

diff --git a/kexec/arch/arm/kexec-arm.h b/kexec/arch/arm/kexec-arm.h
index a74cce2..94695b7 100644
--- a/kexec/arch/arm/kexec-arm.h
+++ b/kexec/arch/arm/kexec-arm.h
@@ -4,6 +4,7 @@
 #include <sys/types.h>
 
 #define SYSFS_FDT "/sys/firmware/fdt"
+#define APPENDED_FDT ((char *)-1) /* it doesn't mean to be opened. */
 #define BOOT_BLOCK_VERSION 17
 #define BOOT_BLOCK_LAST_COMP_VERSION 16
 
diff --git a/kexec/arch/arm/kexec-zImage-arm.c b/kexec/arch/arm/kexec-zImage-arm.c
index 7f02b93..59f0003 100644
--- a/kexec/arch/arm/kexec-zImage-arm.c
+++ b/kexec/arch/arm/kexec-zImage-arm.c
@@ -390,6 +390,7 @@ int zImage_arm_load(int argc, char **argv, const char *buf, off_t len,
 	initrd_size = 0;
 	use_atags = 0;
 	dtb_file = NULL;
+	dtb_buf = NULL;
 	while((opt = getopt_long(argc, argv, short_options, options, 0)) != -1) {
 		switch(opt) {
 		default:
@@ -424,14 +425,6 @@ int zImage_arm_load(int argc, char **argv, const char *buf, off_t len,
 		return -1;
 	}
 
-	if (!use_atags && !dtb_file) {
-		int f;
-
-		f = have_sysfs_fdt();
-		if (f)
-			dtb_file = SYSFS_FDT;
-	}
-
 	if (command_line) {
 		command_line_len = strlen(command_line) + 1;
 		if (command_line_len > COMMAND_LINE_SIZE)
@@ -440,9 +433,6 @@ int zImage_arm_load(int argc, char **argv, const char *buf, off_t len,
 	if (ramdisk)
 		ramdisk_buf = slurp_file(ramdisk, &initrd_size);
 
-	if (dtb_file)
-		dtb_buf = slurp_file(dtb_file, &dtb_length);
-
 	if (len > 0x34) {
 		const struct zimage_header *hdr;
 		off_t size;
@@ -459,6 +449,25 @@ int zImage_arm_load(int argc, char **argv, const char *buf, off_t len,
 				  (unsigned long long)size,
 				  (unsigned long long)len);
 
+			/* check if zImage has an appended dtb */
+			if (!dtb_file) {
+				if (fdt_check_header(buf + size) == 0) {
+					/*
+					 * dtb_file won't be opened. It's set
+					 * just to pass NULL checking.
+					 */
+					dtb_file = APPENDED_FDT;
+					dtb_length = fdt_totalsize(buf + size);
+					dtb_buf = xmalloc(dtb_length);
+					memcpy(dtb_buf, buf+size, dtb_length);
+
+					dbgprintf("found appended dtb, size 0x%llx\n",
+						  (unsigned long long)dtb_length);
+
+
+				}
+			}
+
 			if (size > len) {
 				fprintf(stderr,
 					"zImage is truncated - file 0x%llx vs header 0x%llx\n",
@@ -575,6 +584,22 @@ int zImage_arm_load(int argc, char **argv, const char *buf, off_t len,
 		initrd_base = kernel_base + _ALIGN(len * 5, getpagesize());
 	}
 
+	if (!use_atags && !dtb_file) {
+		int f;
+
+		f = have_sysfs_fdt();
+		if (f)
+			dtb_file = SYSFS_FDT;
+	}
+
+	/*
+	 * dtb_buf can be allocated when checking zImage_with_dtb.
+	 * dtb_buf won't be allocated in the logic if dtb_file is handed over
+	 * in argv[].
+	 */
+	if (dtb_file && !dtb_buf)
+		dtb_buf = slurp_file(dtb_file, &dtb_length);
+
 	if (use_atags) {
 		/*
 		 * use ATAGs from /proc/atags
-- 
2.7.4



From bp at alien8.de  Fri Jun 23 02:09:57 2017
From: bp at alien8.de (Borislav Petkov)
Date: Fri, 23 Jun 2017 11:09:57 +0200
Subject: [PATCH v7 32/36] xen/x86: Remove SME feature in PV guests
In-Reply-To: <20170616185554.18967.82909.stgit@tlendack-t1.amdoffice.net>
References: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
 <20170616185554.18967.82909.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170623090957.yb2x225f4ok4w5qu@pd.tnic>

On Fri, Jun 16, 2017 at 01:55:54PM -0500, Tom Lendacky wrote:
> Xen does not currently support SME for PV guests. Clear the SME cpu

nitpick: s/cpu/CPU/

> capability in order to avoid any ambiguity.
> 
> Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
> ---
>  arch/x86/xen/enlighten_pv.c |    1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/arch/x86/xen/enlighten_pv.c b/arch/x86/xen/enlighten_pv.c
> index f33eef4..e6ecf42 100644
> --- a/arch/x86/xen/enlighten_pv.c
> +++ b/arch/x86/xen/enlighten_pv.c
> @@ -294,6 +294,7 @@ static void __init xen_init_capabilities(void)
>  	setup_clear_cpu_cap(X86_FEATURE_MTRR);
>  	setup_clear_cpu_cap(X86_FEATURE_ACC);
>  	setup_clear_cpu_cap(X86_FEATURE_X2APIC);
> +	setup_clear_cpu_cap(X86_FEATURE_SME);
>  
>  	if (!xen_initial_domain())
>  		setup_clear_cpu_cap(X86_FEATURE_ACPI);

Reviewed-by: Borislav Petkov <bp at suse.de>

-- 
Regards/Gruss,
    Boris.

Good mailing practices for 400: avoid top-posting and trim the reply.


From bp at alien8.de  Fri Jun 23 02:32:36 2017
From: bp at alien8.de (Borislav Petkov)
Date: Fri, 23 Jun 2017 11:32:36 +0200
Subject: [PATCH v7 33/36] x86/mm: Use proper encryption attributes with
 /dev/mem
In-Reply-To: <20170616185607.18967.51412.stgit@tlendack-t1.amdoffice.net>
References: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
 <20170616185607.18967.51412.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170623093236.ei7z2zmc6vjq5lod@pd.tnic>

On Fri, Jun 16, 2017 at 01:56:07PM -0500, Tom Lendacky wrote:
> When accessing memory using /dev/mem (or /dev/kmem) use the proper
> encryption attributes when mapping the memory.
> 
> To insure the proper attributes are applied when reading or writing
> /dev/mem, update the xlate_dev_mem_ptr() function to use memremap()
> which will essentially perform the same steps of applying __va for
> RAM or using ioremap() for if not RAM.
> 
> To insure the proper attributes are applied when mmapping /dev/mem,
> update the phys_mem_access_prot() to call phys_mem_access_encrypted(),
> a new function which will check if the memory should be mapped encrypted
> or not. If it is not to be mapped encrypted then the VMA protection
> value is updated to remove the encryption bit.
> 
> Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
> ---
>  arch/x86/include/asm/io.h |    3 +++
>  arch/x86/mm/ioremap.c     |   18 +++++++++---------
>  arch/x86/mm/pat.c         |    3 +++
>  3 files changed, 15 insertions(+), 9 deletions(-)

Reviewed-by: Borislav Petkov <bp at suse.de>

-- 
Regards/Gruss,
    Boris.

Good mailing practices for 400: avoid top-posting and trim the reply.


From bp at alien8.de  Fri Jun 23 03:00:13 2017
From: bp at alien8.de (Borislav Petkov)
Date: Fri, 23 Jun 2017 12:00:13 +0200
Subject: [PATCH v7 34/36] x86/mm: Add support to encrypt the kernel
 in-place
In-Reply-To: <20170616185619.18967.38945.stgit@tlendack-t1.amdoffice.net>
References: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
 <20170616185619.18967.38945.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170623100013.upd4or6esjvulmvg@pd.tnic>

On Fri, Jun 16, 2017 at 01:56:19PM -0500, Tom Lendacky wrote:
> Add the support to encrypt the kernel in-place. This is done by creating
> new page mappings for the kernel - a decrypted write-protected mapping
> and an encrypted mapping. The kernel is encrypted by copying it through
> a temporary buffer.
> 
> Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
> ---
>  arch/x86/include/asm/mem_encrypt.h |    6 +
>  arch/x86/mm/Makefile               |    2 
>  arch/x86/mm/mem_encrypt.c          |  314 ++++++++++++++++++++++++++++++++++++
>  arch/x86/mm/mem_encrypt_boot.S     |  150 +++++++++++++++++
>  4 files changed, 472 insertions(+)
>  create mode 100644 arch/x86/mm/mem_encrypt_boot.S
> 
> diff --git a/arch/x86/include/asm/mem_encrypt.h b/arch/x86/include/asm/mem_encrypt.h
> index af835cf..7da6de3 100644
> --- a/arch/x86/include/asm/mem_encrypt.h
> +++ b/arch/x86/include/asm/mem_encrypt.h
> @@ -21,6 +21,12 @@
>  
>  extern unsigned long sme_me_mask;
>  
> +void sme_encrypt_execute(unsigned long encrypted_kernel_vaddr,
> +			 unsigned long decrypted_kernel_vaddr,
> +			 unsigned long kernel_len,
> +			 unsigned long encryption_wa,
> +			 unsigned long encryption_pgd);
> +
>  void __init sme_early_encrypt(resource_size_t paddr,
>  			      unsigned long size);
>  void __init sme_early_decrypt(resource_size_t paddr,
> diff --git a/arch/x86/mm/Makefile b/arch/x86/mm/Makefile
> index 9e13841..0633142 100644
> --- a/arch/x86/mm/Makefile
> +++ b/arch/x86/mm/Makefile
> @@ -38,3 +38,5 @@ obj-$(CONFIG_NUMA_EMU)		+= numa_emulation.o
>  obj-$(CONFIG_X86_INTEL_MPX)	+= mpx.o
>  obj-$(CONFIG_X86_INTEL_MEMORY_PROTECTION_KEYS) += pkeys.o
>  obj-$(CONFIG_RANDOMIZE_MEMORY) += kaslr.o
> +
> +obj-$(CONFIG_AMD_MEM_ENCRYPT)	+= mem_encrypt_boot.o
> diff --git a/arch/x86/mm/mem_encrypt.c b/arch/x86/mm/mem_encrypt.c
> index 842c8a6..6e87662 100644
> --- a/arch/x86/mm/mem_encrypt.c
> +++ b/arch/x86/mm/mem_encrypt.c
> @@ -24,6 +24,8 @@
>  #include <asm/setup.h>
>  #include <asm/bootparam.h>
>  #include <asm/set_memory.h>
> +#include <asm/cacheflush.h>
> +#include <asm/sections.h>
>  
>  /*
>   * Since SME related variables are set early in the boot process they must
> @@ -209,8 +211,320 @@ void swiotlb_set_mem_attributes(void *vaddr, unsigned long size)
>  	set_memory_decrypted((unsigned long)vaddr, size >> PAGE_SHIFT);
>  }
>  
> +static void __init sme_clear_pgd(pgd_t *pgd_base, unsigned long start,
> +				 unsigned long end)
> +{
> +	unsigned long pgd_start, pgd_end, pgd_size;
> +	pgd_t *pgd_p;
> +
> +	pgd_start = start & PGDIR_MASK;
> +	pgd_end = end & PGDIR_MASK;
> +
> +	pgd_size = (((pgd_end - pgd_start) / PGDIR_SIZE) + 1);
> +	pgd_size *= sizeof(pgd_t);
> +
> +	pgd_p = pgd_base + pgd_index(start);
> +
> +	memset(pgd_p, 0, pgd_size);
> +}
> +
> +#ifndef CONFIG_X86_5LEVEL
> +#define native_make_p4d(_x)	(p4d_t) { .pgd = native_make_pgd(_x) }
> +#endif

Huh, why isn't this in arch/x86/include/asm/pgtable_types.h in the #else
branch of #if CONFIG_PGTABLE_LEVELS > 4 ?

Also

ERROR: Macros with complex values should be enclosed in parentheses
#105: FILE: arch/x86/mm/mem_encrypt.c:232:
+#define native_make_p4d(_x)    (p4d_t) { .pgd = native_make_pgd(_x) }

so why isn't it a function?

> +
> +#define PGD_FLAGS	_KERNPG_TABLE_NOENC
> +#define P4D_FLAGS	_KERNPG_TABLE_NOENC
> +#define PUD_FLAGS	_KERNPG_TABLE_NOENC
> +#define PMD_FLAGS	(__PAGE_KERNEL_LARGE_EXEC & ~_PAGE_GLOBAL)
> +
> +static void __init *sme_populate_pgd(pgd_t *pgd_base, void *pgtable_area,
> +				     unsigned long vaddr, pmdval_t pmd_val)
> +{
> +	pgd_t *pgd_p;
> +	p4d_t *p4d_p;
> +	pud_t *pud_p;
> +	pmd_t *pmd_p;
> +
> +	pgd_p = pgd_base + pgd_index(vaddr);
> +	if (native_pgd_val(*pgd_p)) {
> +		if (IS_ENABLED(CONFIG_X86_5LEVEL))

Err, I don't understand: so this is a Kconfig symbol and when it is
enabled at build time, you do a 5level pagetable.

But you can't stick a 5level pagetable to a hardware which doesn't know
about it.

Or do you mean that p4d layer folding at runtime to happen? (I admit, I
haven't looked at that in detail.) But then I'd hope that the generic
macros/functions would give you the ability to not care whether we have
a p4d or not and not add a whole bunch of ifdeffery to this code.

Hmmm.

-- 
Regards/Gruss,
    Boris.

Good mailing practices for 400: avoid top-posting and trim the reply.


From bp at alien8.de  Fri Jun 23 04:57:41 2017
From: bp at alien8.de (Borislav Petkov)
Date: Fri, 23 Jun 2017 13:57:41 +0200
Subject: [PATCH v7 35/36] x86/boot: Add early cmdline parsing for options
 with arguments
In-Reply-To: <20170616185630.18967.80046.stgit@tlendack-t1.amdoffice.net>
References: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
 <20170616185630.18967.80046.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170623115741.o4ejclshoy6yjogl@pd.tnic>

On Fri, Jun 16, 2017 at 01:56:30PM -0500, Tom Lendacky wrote:
> Add a cmdline_find_option() function to look for cmdline options that
> take arguments. The argument is returned in a supplied buffer and the
> argument length (regardless of whether it fits in the supplied buffer)
> is returned, with -1 indicating not found.
> 
> Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
> ---
>  arch/x86/include/asm/cmdline.h |    2 +
>  arch/x86/lib/cmdline.c         |  105 ++++++++++++++++++++++++++++++++++++++++
>  2 files changed, 107 insertions(+)

Reviewed-by: Borislav Petkov <bp at suse.de>

-- 
Regards/Gruss,
    Boris.

Good mailing practices for 400: avoid top-posting and trim the reply.


From bp at alien8.de  Fri Jun 23 10:39:37 2017
From: bp at alien8.de (Borislav Petkov)
Date: Fri, 23 Jun 2017 19:39:37 +0200
Subject: [PATCH v7 36/36] x86/mm: Add support to make use of Secure
 Memory Encryption
In-Reply-To: <20170616185639.18967.41488.stgit@tlendack-t1.amdoffice.net>
References: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
 <20170616185639.18967.41488.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170623173937.erotfiemyidyvarn@pd.tnic>

On Fri, Jun 16, 2017 at 01:56:39PM -0500, Tom Lendacky wrote:
> Add support to check if SME has been enabled and if memory encryption
> should be activated (checking of command line option based on the
> configuration of the default state).  If memory encryption is to be
> activated, then the encryption mask is set and the kernel is encrypted
> "in place."
> 
> Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
> ---
>  arch/x86/include/asm/mem_encrypt.h |    6 ++-
>  arch/x86/kernel/head64.c           |    4 +-
>  arch/x86/mm/mem_encrypt.c          |   86 +++++++++++++++++++++++++++++++++++-
>  3 files changed, 90 insertions(+), 6 deletions(-)

...

> +/*
> + * Some SME functions run very early causing issues with the stack-protector
> + * support. Provide a way to turn off this support on a per-function basis.
> + */
> +#define SME_NOSTACKP __attribute__((__optimize__("no-stack-protector")))

__nostackp

just like the others in include/linux/compiler-gcc.h.

> +
> +static char sme_cmdline_arg[] __initdata = "mem_encrypt";
> +static char sme_cmdline_on[]  __initdata = "on";
> +static char sme_cmdline_off[] __initdata = "off";
>  
>  /*
>   * Since SME related variables are set early in the boot process they must
> @@ -200,6 +215,8 @@ void __init mem_encrypt_init(void)
>  
>  	/* Call into SWIOTLB to update the SWIOTLB DMA buffers */
>  	swiotlb_update_mem_attributes();
> +
> +	pr_info("AMD Secure Memory Encryption (SME) active\n");
>  }
>  
>  void swiotlb_set_mem_attributes(void *vaddr, unsigned long size)
> @@ -527,8 +544,73 @@ void __init sme_encrypt_kernel(void)
>  	native_write_cr3(__native_read_cr3());
>  }
>  
> -void __init sme_enable(void)
> +void __init SME_NOSTACKP sme_enable(struct boot_params *bp)
>  {
> +	const char *cmdline_ptr, *cmdline_arg, *cmdline_on, *cmdline_off;
> +	unsigned int eax, ebx, ecx, edx;
> +	bool active_by_default;
> +	unsigned long me_mask;
> +	char buffer[16];
> +	u64 msr;
> +
> +	/* Check for the SME support leaf */
> +	eax = 0x80000000;
> +	ecx = 0;
> +	native_cpuid(&eax, &ebx, &ecx, &edx);
> +	if (eax < 0x8000001f)
> +		return;
> +
> +	/*
> +	 * Check for the SME feature:
> +	 *   CPUID Fn8000_001F[EAX] - Bit 0
> +	 *     Secure Memory Encryption support
> +	 *   CPUID Fn8000_001F[EBX] - Bits 5:0
> +	 *     Pagetable bit position used to indicate encryption
> +	 */
> +	eax = 0x8000001f;
> +	ecx = 0;
> +	native_cpuid(&eax, &ebx, &ecx, &edx);
> +	if (!(eax & 1))
> +		return;
> +
> +	me_mask = 1UL << (ebx & 0x3f);
> +
> +	/* Check if SME is enabled */
> +	msr = __rdmsr(MSR_K8_SYSCFG);
> +	if (!(msr & MSR_K8_SYSCFG_MEM_ENCRYPT))
> +		return;
> +
> +	/*
> +	 * Fixups have not been applied to phys_base yet and we're running
> +	 * identity mapped, so we must obtain the address to the SME command
> +	 * line argument data using rip-relative addressing.
> +	 */
> +	asm ("lea sme_cmdline_arg(%%rip), %0"
> +	     : "=r" (cmdline_arg)
> +	     : "p" (sme_cmdline_arg));
> +	asm ("lea sme_cmdline_on(%%rip), %0"
> +	     : "=r" (cmdline_on)
> +	     : "p" (sme_cmdline_on));
> +	asm ("lea sme_cmdline_off(%%rip), %0"
> +	     : "=r" (cmdline_off)
> +	     : "p" (sme_cmdline_off));
> +
> +	if (IS_ENABLED(CONFIG_AMD_MEM_ENCRYPT_ACTIVE_BY_DEFAULT))
> +		active_by_default = true;
> +	else
> +		active_by_default = false;
> +
> +	cmdline_ptr = (const char *)((u64)bp->hdr.cmd_line_ptr |
> +				     ((u64)bp->ext_cmd_line_ptr << 32));
> +
> +	cmdline_find_option(cmdline_ptr, cmdline_arg, buffer, sizeof(buffer));
> +
> +	if (strncmp(buffer, cmdline_on, sizeof(buffer)) == 0)

	if (!strncmp(...

> +		sme_me_mask = me_mask;
> +	else if (strncmp(buffer, cmdline_off, sizeof(buffer)) == 0)

	else if (!strncmp(...

-- 
Regards/Gruss,
    Boris.

Good mailing practices for 400: avoid top-posting and trim the reply.


From thomas.lendacky at amd.com  Fri Jun 23 10:44:46 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Fri, 23 Jun 2017 12:44:46 -0500
Subject: [PATCH v7 34/36] x86/mm: Add support to encrypt the kernel
 in-place
In-Reply-To: <20170623100013.upd4or6esjvulmvg@pd.tnic>
References: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
 <20170616185619.18967.38945.stgit@tlendack-t1.amdoffice.net>
 <20170623100013.upd4or6esjvulmvg@pd.tnic>
Message-ID: <af9a50f7-17ea-a840-6456-b6479e5d7e82@amd.com>

On 6/23/2017 5:00 AM, Borislav Petkov wrote:
> On Fri, Jun 16, 2017 at 01:56:19PM -0500, Tom Lendacky wrote:
>> Add the support to encrypt the kernel in-place. This is done by creating
>> new page mappings for the kernel - a decrypted write-protected mapping
>> and an encrypted mapping. The kernel is encrypted by copying it through
>> a temporary buffer.
>>
>> Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
>> ---
>>   arch/x86/include/asm/mem_encrypt.h |    6 +
>>   arch/x86/mm/Makefile               |    2
>>   arch/x86/mm/mem_encrypt.c          |  314 ++++++++++++++++++++++++++++++++++++
>>   arch/x86/mm/mem_encrypt_boot.S     |  150 +++++++++++++++++
>>   4 files changed, 472 insertions(+)
>>   create mode 100644 arch/x86/mm/mem_encrypt_boot.S
>>
>> diff --git a/arch/x86/include/asm/mem_encrypt.h b/arch/x86/include/asm/mem_encrypt.h
>> index af835cf..7da6de3 100644
>> --- a/arch/x86/include/asm/mem_encrypt.h
>> +++ b/arch/x86/include/asm/mem_encrypt.h
>> @@ -21,6 +21,12 @@
>>   
>>   extern unsigned long sme_me_mask;
>>   
>> +void sme_encrypt_execute(unsigned long encrypted_kernel_vaddr,
>> +			 unsigned long decrypted_kernel_vaddr,
>> +			 unsigned long kernel_len,
>> +			 unsigned long encryption_wa,
>> +			 unsigned long encryption_pgd);
>> +
>>   void __init sme_early_encrypt(resource_size_t paddr,
>>   			      unsigned long size);
>>   void __init sme_early_decrypt(resource_size_t paddr,
>> diff --git a/arch/x86/mm/Makefile b/arch/x86/mm/Makefile
>> index 9e13841..0633142 100644
>> --- a/arch/x86/mm/Makefile
>> +++ b/arch/x86/mm/Makefile
>> @@ -38,3 +38,5 @@ obj-$(CONFIG_NUMA_EMU)		+= numa_emulation.o
>>   obj-$(CONFIG_X86_INTEL_MPX)	+= mpx.o
>>   obj-$(CONFIG_X86_INTEL_MEMORY_PROTECTION_KEYS) += pkeys.o
>>   obj-$(CONFIG_RANDOMIZE_MEMORY) += kaslr.o
>> +
>> +obj-$(CONFIG_AMD_MEM_ENCRYPT)	+= mem_encrypt_boot.o
>> diff --git a/arch/x86/mm/mem_encrypt.c b/arch/x86/mm/mem_encrypt.c
>> index 842c8a6..6e87662 100644
>> --- a/arch/x86/mm/mem_encrypt.c
>> +++ b/arch/x86/mm/mem_encrypt.c
>> @@ -24,6 +24,8 @@
>>   #include <asm/setup.h>
>>   #include <asm/bootparam.h>
>>   #include <asm/set_memory.h>
>> +#include <asm/cacheflush.h>
>> +#include <asm/sections.h>
>>   
>>   /*
>>    * Since SME related variables are set early in the boot process they must
>> @@ -209,8 +211,320 @@ void swiotlb_set_mem_attributes(void *vaddr, unsigned long size)
>>   	set_memory_decrypted((unsigned long)vaddr, size >> PAGE_SHIFT);
>>   }
>>   
>> +static void __init sme_clear_pgd(pgd_t *pgd_base, unsigned long start,
>> +				 unsigned long end)
>> +{
>> +	unsigned long pgd_start, pgd_end, pgd_size;
>> +	pgd_t *pgd_p;
>> +
>> +	pgd_start = start & PGDIR_MASK;
>> +	pgd_end = end & PGDIR_MASK;
>> +
>> +	pgd_size = (((pgd_end - pgd_start) / PGDIR_SIZE) + 1);
>> +	pgd_size *= sizeof(pgd_t);
>> +
>> +	pgd_p = pgd_base + pgd_index(start);
>> +
>> +	memset(pgd_p, 0, pgd_size);
>> +}
>> +
>> +#ifndef CONFIG_X86_5LEVEL
>> +#define native_make_p4d(_x)	(p4d_t) { .pgd = native_make_pgd(_x) }
>> +#endif
> 
> Huh, why isn't this in arch/x86/include/asm/pgtable_types.h in the #else
> branch of #if CONFIG_PGTABLE_LEVELS > 4 ?

Normally the __p4d() macro would be used and that would be ok whether
CONFIG_X86_5LEVEL is defined or not. But since __p4d() is part of the
paravirt ops path I have to use native_make_p4d(). I'd be the only user
of the function and thought it would be best to localize it this way.

> 
> Also
> 
> ERROR: Macros with complex values should be enclosed in parentheses
> #105: FILE: arch/x86/mm/mem_encrypt.c:232:
> +#define native_make_p4d(_x)    (p4d_t) { .pgd = native_make_pgd(_x) }
> 
> so why isn't it a function?

I can define it as an inline function.

> 
>> +
>> +#define PGD_FLAGS	_KERNPG_TABLE_NOENC
>> +#define P4D_FLAGS	_KERNPG_TABLE_NOENC
>> +#define PUD_FLAGS	_KERNPG_TABLE_NOENC
>> +#define PMD_FLAGS	(__PAGE_KERNEL_LARGE_EXEC & ~_PAGE_GLOBAL)
>> +
>> +static void __init *sme_populate_pgd(pgd_t *pgd_base, void *pgtable_area,
>> +				     unsigned long vaddr, pmdval_t pmd_val)
>> +{
>> +	pgd_t *pgd_p;
>> +	p4d_t *p4d_p;
>> +	pud_t *pud_p;
>> +	pmd_t *pmd_p;
>> +
>> +	pgd_p = pgd_base + pgd_index(vaddr);
>> +	if (native_pgd_val(*pgd_p)) {
>> +		if (IS_ENABLED(CONFIG_X86_5LEVEL))
> 
> Err, I don't understand: so this is a Kconfig symbol and when it is
> enabled at build time, you do a 5level pagetable.
> 
> But you can't stick a 5level pagetable to a hardware which doesn't know
> about it.

True, 5-level will only be turned on for specific hardware which is why
I originally had this as only 4-level pagetables. But in a comment from
you back on the v5 version you said it needed to support 5-level. I
guess we should have discussed this more, but I also thought that should
our hardware ever support 5-level paging in the future then this would
be good to go.

> 
> Or do you mean that p4d layer folding at runtime to happen? (I admit, I
> haven't looked at that in detail.) But then I'd hope that the generic
> macros/functions would give you the ability to not care whether we have
> a p4d or not and not add a whole bunch of ifdeffery to this code.

The macros work great if you are not running identity mapped. You could
use p*d_offset() to move easily through the tables, but those functions
use __va() to generate table virtual addresses. I've seen where
boot/compressed/pagetable.c #defines __va() to work with identity mapped
pages but that would only work if I create a separate file just for this
function.

Given when this occurs it's very similar to what __startup_64() does in
regards to the IS_ENABLED(CONFIG_X86_5LEVEL) checks.

Thanks,
Tom

> 
> Hmmm.
> 


From dyoung at redhat.com  Sun Jun 25 19:52:56 2017
From: dyoung at redhat.com (Dave Young)
Date: Mon, 26 Jun 2017 10:52:56 +0800
Subject: [PATCH] kexec:arm: support zImage with appended device tree
In-Reply-To: <1498208138-19283-1-git-send-email-hoeun.ryu@gmail.com>
References: <1498208138-19283-1-git-send-email-hoeun.ryu@gmail.com>
Message-ID: <20170626025256.GA3378@dhcp-128-65.nay.redhat.com>

On 06/23/17 at 05:55pm, Hoeun Ryu wrote:
> Arm linux supports zImage with appended dtb (CONFIG_ARM_APPENDED_DTB) and
> the concatenated image is generated like `cat zImage dtb > zImage_w_dtb`.
> 
> This patch is to support the concatednated zImage. This changes the
> priority of source of dtb file.
> 
>  1. --dtb dtb_file
>  2. zImage_w_dtb	<= newly added
>  3. /sys/firmware/fdt
>  4. /proc/device-tree

The original default fdt is /sys/firmware/fdt, after your changes it
becomes zImage attached dtb, presumably one knows his zImage appended a
dtb it is fine, but if one does not notice it, it changes the behavior
he expects.

How about add a new option for zImage_dtb, like --zImage-dtb

> 
> Users don't need to specify dtb file in the command line and don't have to
> have /sys/firmware/fdt or /proc/device-tree if dtb is available in zImage.
> 
> Signed-off-by: Hoeun Ryu <hoeun.ryu at gmail.com>
> ---
>  kexec/arch/arm/kexec-arm.h        |  1 +
>  kexec/arch/arm/kexec-zImage-arm.c | 47 ++++++++++++++++++++++++++++++---------
>  2 files changed, 37 insertions(+), 11 deletions(-)
> 
> diff --git a/kexec/arch/arm/kexec-arm.h b/kexec/arch/arm/kexec-arm.h
> index a74cce2..94695b7 100644
> --- a/kexec/arch/arm/kexec-arm.h
> +++ b/kexec/arch/arm/kexec-arm.h
> @@ -4,6 +4,7 @@
>  #include <sys/types.h>
>  
>  #define SYSFS_FDT "/sys/firmware/fdt"
> +#define APPENDED_FDT ((char *)-1) /* it doesn't mean to be opened. */
>  #define BOOT_BLOCK_VERSION 17
>  #define BOOT_BLOCK_LAST_COMP_VERSION 16
>  
> diff --git a/kexec/arch/arm/kexec-zImage-arm.c b/kexec/arch/arm/kexec-zImage-arm.c
> index 7f02b93..59f0003 100644
> --- a/kexec/arch/arm/kexec-zImage-arm.c
> +++ b/kexec/arch/arm/kexec-zImage-arm.c
> @@ -390,6 +390,7 @@ int zImage_arm_load(int argc, char **argv, const char *buf, off_t len,
>  	initrd_size = 0;
>  	use_atags = 0;
>  	dtb_file = NULL;
> +	dtb_buf = NULL;
>  	while((opt = getopt_long(argc, argv, short_options, options, 0)) != -1) {
>  		switch(opt) {
>  		default:
> @@ -424,14 +425,6 @@ int zImage_arm_load(int argc, char **argv, const char *buf, off_t len,
>  		return -1;
>  	}
>  
> -	if (!use_atags && !dtb_file) {
> -		int f;
> -
> -		f = have_sysfs_fdt();
> -		if (f)
> -			dtb_file = SYSFS_FDT;
> -	}
> -
>  	if (command_line) {
>  		command_line_len = strlen(command_line) + 1;
>  		if (command_line_len > COMMAND_LINE_SIZE)
> @@ -440,9 +433,6 @@ int zImage_arm_load(int argc, char **argv, const char *buf, off_t len,
>  	if (ramdisk)
>  		ramdisk_buf = slurp_file(ramdisk, &initrd_size);
>  
> -	if (dtb_file)
> -		dtb_buf = slurp_file(dtb_file, &dtb_length);
> -
>  	if (len > 0x34) {
>  		const struct zimage_header *hdr;
>  		off_t size;
> @@ -459,6 +449,25 @@ int zImage_arm_load(int argc, char **argv, const char *buf, off_t len,
>  				  (unsigned long long)size,
>  				  (unsigned long long)len);
>  
> +			/* check if zImage has an appended dtb */
> +			if (!dtb_file) {
> +				if (fdt_check_header(buf + size) == 0) {
> +					/*
> +					 * dtb_file won't be opened. It's set
> +					 * just to pass NULL checking.
> +					 */
> +					dtb_file = APPENDED_FDT;
> +					dtb_length = fdt_totalsize(buf + size);
> +					dtb_buf = xmalloc(dtb_length);
> +					memcpy(dtb_buf, buf+size, dtb_length);
> +
> +					dbgprintf("found appended dtb, size 0x%llx\n",
> +						  (unsigned long long)dtb_length);
> +
> +
> +				}
> +			}
> +
>  			if (size > len) {
>  				fprintf(stderr,
>  					"zImage is truncated - file 0x%llx vs header 0x%llx\n",
> @@ -575,6 +584,22 @@ int zImage_arm_load(int argc, char **argv, const char *buf, off_t len,
>  		initrd_base = kernel_base + _ALIGN(len * 5, getpagesize());
>  	}
>  
> +	if (!use_atags && !dtb_file) {
> +		int f;
> +
> +		f = have_sysfs_fdt();
> +		if (f)
> +			dtb_file = SYSFS_FDT;
> +	}
> +
> +	/*
> +	 * dtb_buf can be allocated when checking zImage_with_dtb.
> +	 * dtb_buf won't be allocated in the logic if dtb_file is handed over
> +	 * in argv[].
> +	 */
> +	if (dtb_file && !dtb_buf)
> +		dtb_buf = slurp_file(dtb_file, &dtb_length);
> +
>  	if (use_atags) {
>  		/*
>  		 * use ATAGs from /proc/atags
> -- 
> 2.7.4
> 
> 
> _______________________________________________
> kexec mailing list
> kexec at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/kexec


From rmk at armlinux.org.uk  Mon Jun 26 02:14:12 2017
From: rmk at armlinux.org.uk (Russell King)
Date: Mon, 26 Jun 2017 10:14:12 +0100
Subject: [PATCH] kexec:arm: support zImage with appended device tree
In-Reply-To: <1498208138-19283-1-git-send-email-hoeun.ryu@gmail.com>
References: <1498208138-19283-1-git-send-email-hoeun.ryu@gmail.com>
Message-ID: <20170626091412.GE837@flint.armlinux.org.uk>

On Fri, Jun 23, 2017 at 05:55:38PM +0900, Hoeun Ryu wrote:
> Arm linux supports zImage with appended dtb (CONFIG_ARM_APPENDED_DTB) and
> the concatenated image is generated like `cat zImage dtb > zImage_w_dtb`.

We support that only for the purpose of allowing old boot loaders that
are not DT aware to load kernels that require DT.  If it weren't for
that, we wouldn't have it.

I don't see why we should propagate this hack to other systems such as
kexec, especially when they have native DT support.

-- 
Russell King


From panand at redhat.com  Mon Jun 26 02:15:08 2017
From: panand at redhat.com (Pratyush Anand)
Date: Mon, 26 Jun 2017 14:45:08 +0530
Subject: [PATCH] kexec:arm: support zImage with appended device tree
In-Reply-To: <1498208138-19283-1-git-send-email-hoeun.ryu@gmail.com>
References: <1498208138-19283-1-git-send-email-hoeun.ryu@gmail.com>
Message-ID: <cdce2336-228a-a794-cdde-8c869d7a8cf2@redhat.com>



On Friday 23 June 2017 02:25 PM, Hoeun Ryu wrote:
> Arm linux supports zImage with appended dtb (CONFIG_ARM_APPENDED_DTB) and
> the concatenated image is generated like `cat zImage dtb > zImage_w_dtb`.
>
> This patch is to support the concatednated zImage. This changes the
> priority of source of dtb file.
>
>  1. --dtb dtb_file
>  2. zImage_w_dtb	<= newly added
>  3. /sys/firmware/fdt
>  4. /proc/device-tree
>
> Users don't need to specify dtb file in the command line and don't have to
> have /sys/firmware/fdt or /proc/device-tree if dtb is available in zImage.

any specific reason to not to add the new method as last preferred option? 
Does it not work if there exist a /sys/firmware/fdt in case of a kernel booted 
with "dtb appended zImage"?

>
> Signed-off-by: Hoeun Ryu <hoeun.ryu at gmail.com>
> ---
>  kexec/arch/arm/kexec-arm.h        |  1 +
>  kexec/arch/arm/kexec-zImage-arm.c | 47 ++++++++++++++++++++++++++++++---------
>  2 files changed, 37 insertions(+), 11 deletions(-)
>
> diff --git a/kexec/arch/arm/kexec-arm.h b/kexec/arch/arm/kexec-arm.h
> index a74cce2..94695b7 100644
> --- a/kexec/arch/arm/kexec-arm.h
> +++ b/kexec/arch/arm/kexec-arm.h
> @@ -4,6 +4,7 @@
>  #include <sys/types.h>
>
>  #define SYSFS_FDT "/sys/firmware/fdt"
> +#define APPENDED_FDT ((char *)-1) /* it doesn't mean to be opened. */
>  #define BOOT_BLOCK_VERSION 17
>  #define BOOT_BLOCK_LAST_COMP_VERSION 16
>
> diff --git a/kexec/arch/arm/kexec-zImage-arm.c b/kexec/arch/arm/kexec-zImage-arm.c
> index 7f02b93..59f0003 100644
> --- a/kexec/arch/arm/kexec-zImage-arm.c
> +++ b/kexec/arch/arm/kexec-zImage-arm.c
> @@ -390,6 +390,7 @@ int zImage_arm_load(int argc, char **argv, const char *buf, off_t len,
>  	initrd_size = 0;
>  	use_atags = 0;
>  	dtb_file = NULL;
> +	dtb_buf = NULL;
>  	while((opt = getopt_long(argc, argv, short_options, options, 0)) != -1) {
>  		switch(opt) {
>  		default:
> @@ -424,14 +425,6 @@ int zImage_arm_load(int argc, char **argv, const char *buf, off_t len,
>  		return -1;
>  	}
>
> -	if (!use_atags && !dtb_file) {
> -		int f;
> -
> -		f = have_sysfs_fdt();
> -		if (f)
> -			dtb_file = SYSFS_FDT;
> -	}
> -
>  	if (command_line) {
>  		command_line_len = strlen(command_line) + 1;
>  		if (command_line_len > COMMAND_LINE_SIZE)
> @@ -440,9 +433,6 @@ int zImage_arm_load(int argc, char **argv, const char *buf, off_t len,
>  	if (ramdisk)
>  		ramdisk_buf = slurp_file(ramdisk, &initrd_size);
>
> -	if (dtb_file)
> -		dtb_buf = slurp_file(dtb_file, &dtb_length);
> -
>  	if (len > 0x34) {
>  		const struct zimage_header *hdr;
>  		off_t size;
> @@ -459,6 +449,25 @@ int zImage_arm_load(int argc, char **argv, const char *buf, off_t len,
>  				  (unsigned long long)size,
>  				  (unsigned long long)len);
>
> +			/* check if zImage has an appended dtb */
> +			if (!dtb_file) {
> +				if (fdt_check_header(buf + size) == 0) {

What if size >= len? I think, in that case there might not be a valid address 
at buf+size and reading those area would lead in segmentation fault.

> +					/*
> +					 * dtb_file won't be opened. It's set
> +					 * just to pass NULL checking.
> +					 */
> +					dtb_file = APPENDED_FDT;
> +					dtb_length = fdt_totalsize(buf + size);
> +					dtb_buf = xmalloc(dtb_length);
> +					memcpy(dtb_buf, buf+size, dtb_length);
> +
> +					dbgprintf("found appended dtb, size 0x%llx\n",
> +						  (unsigned long long)dtb_length);
> +
> +
> +				}
> +			}
> +
>  			if (size > len) {
>  				fprintf(stderr,
>  					"zImage is truncated - file 0x%llx vs header 0x%llx\n",
> @@ -575,6 +584,22 @@ int zImage_arm_load(int argc, char **argv, const char *buf, off_t len,
>  		initrd_base = kernel_base + _ALIGN(len * 5, getpagesize());
>  	}
>
> +	if (!use_atags && !dtb_file) {
> +		int f;
> +
> +		f = have_sysfs_fdt();
> +		if (f)
> +			dtb_file = SYSFS_FDT;
> +	}
> +
> +	/*
> +	 * dtb_buf can be allocated when checking zImage_with_dtb.
> +	 * dtb_buf won't be allocated in the logic if dtb_file is handed over
> +	 * in argv[].
> +	 */
> +	if (dtb_file && !dtb_buf)
> +		dtb_buf = slurp_file(dtb_file, &dtb_length);
> +
>  	if (use_atags) {
>  		/*
>  		 * use ATAGs from /proc/atags
>

--
Pratyush


From bp at alien8.de  Mon Jun 26 08:45:43 2017
From: bp at alien8.de (Borislav Petkov)
Date: Mon, 26 Jun 2017 17:45:43 +0200
Subject: [PATCH v7 34/36] x86/mm: Add support to encrypt the kernel
 in-place
In-Reply-To: <af9a50f7-17ea-a840-6456-b6479e5d7e82@amd.com>
References: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
 <20170616185619.18967.38945.stgit@tlendack-t1.amdoffice.net>
 <20170623100013.upd4or6esjvulmvg@pd.tnic>
 <af9a50f7-17ea-a840-6456-b6479e5d7e82@amd.com>
Message-ID: <20170626154543.fsuxfhxidytgo2ia@pd.tnic>

On Fri, Jun 23, 2017 at 12:44:46PM -0500, Tom Lendacky wrote:
> Normally the __p4d() macro would be used and that would be ok whether
> CONFIG_X86_5LEVEL is defined or not. But since __p4d() is part of the
> paravirt ops path I have to use native_make_p4d().

So __p4d is in !CONFIG_PARAVIRT path.

Regardless, we use the native_* variants in generic code to mean, not
paravirt. Just define it in a separate patch like the rest of the p4*
machinery and use it in your code. Sooner or later someone else will
need it.

> True, 5-level will only be turned on for specific hardware which is why
> I originally had this as only 4-level pagetables. But in a comment from
> you back on the v5 version you said it needed to support 5-level. I
> guess we should have discussed this more,

AFAIR, I said something along the lines of "what about 5-level page
tables?" and whether we care.

> but I also thought that should our hardware ever support 5-level
> paging in the future then this would be good to go.

There it is :-)

> The macros work great if you are not running identity mapped. You could
> use p*d_offset() to move easily through the tables, but those functions
> use __va() to generate table virtual addresses. I've seen where
> boot/compressed/pagetable.c #defines __va() to work with identity mapped
> pages but that would only work if I create a separate file just for this
> function.
> 
> Given when this occurs it's very similar to what __startup_64() does in
> regards to the IS_ENABLED(CONFIG_X86_5LEVEL) checks.

Ok.

-- 
Regards/Gruss,
    Boris.

Good mailing practices for 400: avoid top-posting and trim the reply.


From thomas.lendacky at amd.com  Mon Jun 26 09:34:49 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Mon, 26 Jun 2017 11:34:49 -0500
Subject: [PATCH v7 34/36] x86/mm: Add support to encrypt the kernel
 in-place
In-Reply-To: <20170626154543.fsuxfhxidytgo2ia@pd.tnic>
References: <20170616184947.18967.84890.stgit@tlendack-t1.amdoffice.net>
 <20170616185619.18967.38945.stgit@tlendack-t1.amdoffice.net>
 <20170623100013.upd4or6esjvulmvg@pd.tnic>
 <af9a50f7-17ea-a840-6456-b6479e5d7e82@amd.com>
 <20170626154543.fsuxfhxidytgo2ia@pd.tnic>
Message-ID: <aed44ee3-dd6c-e6e1-4de5-5629bf61e688@amd.com>

On 6/26/2017 10:45 AM, Borislav Petkov wrote:
> On Fri, Jun 23, 2017 at 12:44:46PM -0500, Tom Lendacky wrote:
>> Normally the __p4d() macro would be used and that would be ok whether
>> CONFIG_X86_5LEVEL is defined or not. But since __p4d() is part of the
>> paravirt ops path I have to use native_make_p4d().
> 
> So __p4d is in !CONFIG_PARAVIRT path.
> 
> Regardless, we use the native_* variants in generic code to mean, not
> paravirt. Just define it in a separate patch like the rest of the p4*
> machinery and use it in your code. Sooner or later someone else will
> need it.

Ok, will do.

> 
>> True, 5-level will only be turned on for specific hardware which is why
>> I originally had this as only 4-level pagetables. But in a comment from
>> you back on the v5 version you said it needed to support 5-level. I
>> guess we should have discussed this more,
> 
> AFAIR, I said something along the lines of "what about 5-level page
> tables?" and whether we care.

My bad, I took the meaning of that question the wrong way then.

Thanks,
Tom

> 
>> but I also thought that should our hardware ever support 5-level
>> paging in the future then this would be good to go.
> 
> There it is :-)
> 
>> The macros work great if you are not running identity mapped. You could
>> use p*d_offset() to move easily through the tables, but those functions
>> use __va() to generate table virtual addresses. I've seen where
>> boot/compressed/pagetable.c #defines __va() to work with identity mapped
>> pages but that would only work if I create a separate file just for this
>> function.
>>
>> Given when this occurs it's very similar to what __startup_64() does in
>> regards to the IS_ENABLED(CONFIG_X86_5LEVEL) checks.
> 
> Ok.
> 


From hoeun.ryu at gmail.com  Mon Jun 26 19:13:17 2017
From: hoeun.ryu at gmail.com (Hoeun Ryu)
Date: Tue, 27 Jun 2017 11:13:17 +0900
Subject: [PATCH] kexec:arm: support zImage with appended device tree
In-Reply-To: <20170626025256.GA3378@dhcp-128-65.nay.redhat.com>
References: <1498208138-19283-1-git-send-email-hoeun.ryu@gmail.com>
 <20170626025256.GA3378@dhcp-128-65.nay.redhat.com>
Message-ID: <1498529597.1734.2.camel@gmail.com>

On Mon, 2017-06-26 at 10:52 +0800, Dave Young wrote:
> On 06/23/17 at 05:55pm, Hoeun Ryu wrote:
> > 
> > Arm linux supports zImage with appended dtb
> > (CONFIG_ARM_APPENDED_DTB) and
> > the concatenated image is generated like `cat zImage dtb >
> > zImage_w_dtb`.
> > 
> > This patch is to support the concatednated zImage. This changes the
> > priority of source of dtb file.
> > 
> > ?1. --dtb dtb_file
> > ?2. zImage_w_dtb	<= newly added
> > ?3. /sys/firmware/fdt
> > ?4. /proc/device-tree
> The original default fdt is /sys/firmware/fdt, after your changes it
> becomes zImage attached dtb, presumably one knows his zImage appended
> a
> dtb it is fine, but if one does not notice it, it changes the
> behavior
> he expects.
> 
> How about add a new option for zImage_dtb, like --zImage-dtb
> 

It sounds good to me too.
I think I can accept your suggestion in the next version if the others
agree.

Thank you for the review.

> > 
> > 
> > Users don't need to specify dtb file in the command line and don't
> > have to
> > have /sys/firmware/fdt or /proc/device-tree if dtb is available in
> > zImage.
> > 
> > Signed-off-by: Hoeun Ryu <hoeun.ryu at gmail.com>
> > ---
> > ?kexec/arch/arm/kexec-arm.h????????|??1 +
> > ?kexec/arch/arm/kexec-zImage-arm.c | 47
> > ++++++++++++++++++++++++++++++---------
> > ?2 files changed, 37 insertions(+), 11 deletions(-)
> > 
> > diff --git a/kexec/arch/arm/kexec-arm.h b/kexec/arch/arm/kexec-
> > arm.h
> > index a74cce2..94695b7 100644
> > --- a/kexec/arch/arm/kexec-arm.h
> > +++ b/kexec/arch/arm/kexec-arm.h
> > @@ -4,6 +4,7 @@
> > ?#include <sys/types.h>
> > ?
> > ?#define SYSFS_FDT "/sys/firmware/fdt"
> > +#define APPENDED_FDT ((char *)-1) /* it doesn't mean to be opened.
> > */
> > ?#define BOOT_BLOCK_VERSION 17
> > ?#define BOOT_BLOCK_LAST_COMP_VERSION 16
> > ?
> > diff --git a/kexec/arch/arm/kexec-zImage-arm.c
> > b/kexec/arch/arm/kexec-zImage-arm.c
> > index 7f02b93..59f0003 100644
> > --- a/kexec/arch/arm/kexec-zImage-arm.c
> > +++ b/kexec/arch/arm/kexec-zImage-arm.c
> > @@ -390,6 +390,7 @@ int zImage_arm_load(int argc, char **argv,
> > const char *buf, off_t len,
> > ?	initrd_size = 0;
> > ?	use_atags = 0;
> > ?	dtb_file = NULL;
> > +	dtb_buf = NULL;
> > ?	while((opt = getopt_long(argc, argv, short_options,
> > options, 0)) != -1) {
> > ?		switch(opt) {
> > ?		default:
> > @@ -424,14 +425,6 @@ int zImage_arm_load(int argc, char **argv,
> > const char *buf, off_t len,
> > ?		return -1;
> > ?	}
> > ?
> > -	if (!use_atags && !dtb_file) {
> > -		int f;
> > -
> > -		f = have_sysfs_fdt();
> > -		if (f)
> > -			dtb_file = SYSFS_FDT;
> > -	}
> > -
> > ?	if (command_line) {
> > ?		command_line_len = strlen(command_line) + 1;
> > ?		if (command_line_len > COMMAND_LINE_SIZE)
> > @@ -440,9 +433,6 @@ int zImage_arm_load(int argc, char **argv,
> > const char *buf, off_t len,
> > ?	if (ramdisk)
> > ?		ramdisk_buf = slurp_file(ramdisk, &initrd_size);
> > ?
> > -	if (dtb_file)
> > -		dtb_buf = slurp_file(dtb_file, &dtb_length);
> > -
> > ?	if (len > 0x34) {
> > ?		const struct zimage_header *hdr;
> > ?		off_t size;
> > @@ -459,6 +449,25 @@ int zImage_arm_load(int argc, char **argv,
> > const char *buf, off_t len,
> > ?				??(unsigned long long)size,
> > ?				??(unsigned long long)len);
> > ?
> > +			/* check if zImage has an appended dtb */
> > +			if (!dtb_file) {
> > +				if (fdt_check_header(buf + size)
> > == 0) {
> > +					/*
> > +					?* dtb_file won't be
> > opened. It's set
> > +					?* just to pass NULL
> > checking.
> > +					?*/
> > +					dtb_file = APPENDED_FDT;
> > +					dtb_length =
> > fdt_totalsize(buf + size);
> > +					dtb_buf =
> > xmalloc(dtb_length);
> > +					memcpy(dtb_buf, buf+size,
> > dtb_length);
> > +
> > +					dbgprintf("found appended
> > dtb, size 0x%llx\n",
> > +						??(unsigned long
> > long)dtb_length);
> > +
> > +
> > +				}
> > +			}
> > +
> > ?			if (size > len) {
> > ?				fprintf(stderr,
> > ?					"zImage is truncated -
> > file 0x%llx vs header 0x%llx\n",
> > @@ -575,6 +584,22 @@ int zImage_arm_load(int argc, char **argv,
> > const char *buf, off_t len,
> > ?		initrd_base = kernel_base + _ALIGN(len * 5,
> > getpagesize());
> > ?	}
> > ?
> > +	if (!use_atags && !dtb_file) {
> > +		int f;
> > +
> > +		f = have_sysfs_fdt();
> > +		if (f)
> > +			dtb_file = SYSFS_FDT;
> > +	}
> > +
> > +	/*
> > +	?* dtb_buf can be allocated when checking zImage_with_dtb.
> > +	?* dtb_buf won't be allocated in the logic if dtb_file is
> > handed over
> > +	?* in argv[].
> > +	?*/
> > +	if (dtb_file && !dtb_buf)
> > +		dtb_buf = slurp_file(dtb_file, &dtb_length);
> > +
> > ?	if (use_atags) {
> > ?		/*
> > ?		?* use ATAGs from /proc/atags


From hoeun.ryu at gmail.com  Mon Jun 26 19:36:43 2017
From: hoeun.ryu at gmail.com (Hoeun Ryu)
Date: Tue, 27 Jun 2017 11:36:43 +0900
Subject: [PATCH] kexec:arm: support zImage with appended device tree
In-Reply-To: <20170626091412.GE837@flint.armlinux.org.uk>
References: <1498208138-19283-1-git-send-email-hoeun.ryu@gmail.com>
 <20170626091412.GE837@flint.armlinux.org.uk>
Message-ID: <1498531003.1734.13.camel@gmail.com>

On Mon, 2017-06-26 at 10:14 +0100, Russell King wrote:
> On Fri, Jun 23, 2017 at 05:55:38PM +0900, Hoeun Ryu wrote:
> > 
> > Arm linux supports zImage with appended dtb
> > (CONFIG_ARM_APPENDED_DTB) and
> > the concatenated image is generated like `cat zImage dtb >
> > zImage_w_dtb`.
> We support that only for the purpose of allowing old boot loaders
> that
> are not DT aware to load kernels that require DT.??If it weren't for
> that, we wouldn't have it.
> 
> I don't see why we should propagate this hack to other systems such
> as
> kexec, especially when they have native DT support.
> 

We have some cases when we would like to use different dtb from the
running system when using kexec and I think that's why kexec-tools
supports --dtb command line option.

For example, I have the second kernel for the crash dump with different
kernel configuration and the different dtb from the running system. I'd
like to exclude some nodes/properties from dtb like memory reservations
or unnecessary devices to keep the second kernel/dtb minimal.

The concatenated zImage for arm has a benefit (whether it's intended or
not) to make it possible for users to merge multiple boot images into a
simple single file.
What I'd like to do is just to support the concatenated zImage so that
users can use --load(-panic) zImage_with_dtb_from_the_running_system
instead of --load(-panic) zImage --dtb
different_dtb_from_the_running_system.

Thank you for the review.



From hoeun.ryu at gmail.com  Mon Jun 26 19:52:17 2017
From: hoeun.ryu at gmail.com (Hoeun Ryu)
Date: Tue, 27 Jun 2017 11:52:17 +0900
Subject: [PATCH] kexec:arm: support zImage with appended device tree
In-Reply-To: <cdce2336-228a-a794-cdde-8c869d7a8cf2@redhat.com>
References: <1498208138-19283-1-git-send-email-hoeun.ryu@gmail.com>
 <cdce2336-228a-a794-cdde-8c869d7a8cf2@redhat.com>
Message-ID: <1498531937.1734.24.camel@gmail.com>

On Mon, 2017-06-26 at 14:45 +0530, Pratyush Anand wrote:
> 
> On Friday 23 June 2017 02:25 PM, Hoeun Ryu wrote:
> > 
> > Arm linux supports zImage with appended dtb
> > (CONFIG_ARM_APPENDED_DTB) and
> > the concatenated image is generated like `cat zImage dtb >
> > zImage_w_dtb`.
> > 
> > This patch is to support the concatednated zImage. This changes the
> > priority of source of dtb file.
> > 
> > ?1. --dtb dtb_file
> > ?2. zImage_w_dtb	<= newly added
> > ?3. /sys/firmware/fdt
> > ?4. /proc/device-tree
> > 
> > Users don't need to specify dtb file in the command line and don't
> > have to
> > have /sys/firmware/fdt or /proc/device-tree if dtb is available in
> > zImage.
> any specific reason to not to add the new method as last preferred
> option??
> Does it not work if there exist a /sys/firmware/fdt in case of a
> kernel booted?
> with "dtb appended zImage"?
> 

I thought that DTBs from command line should be priotized over /sys or
/proc.
What I intended is to use command line `--load(-panic) zImage_with_dtb`
instead of `--load(-panic) zImage --dtb dtb_file`.
So I thought they should have the identical priority in both cases.

> > 
> > Signed-off-by: Hoeun Ryu <hoeun.ryu at gmail.com>
> > ---
> > ?kexec/arch/arm/kexec-arm.h????????|??1 +
> > ?kexec/arch/arm/kexec-zImage-arm.c | 47
> > ++++++++++++++++++++++++++++++---------
> > ?2 files changed, 37 insertions(+), 11 deletions(-)
> > 
> > diff --git a/kexec/arch/arm/kexec-arm.h b/kexec/arch/arm/kexec-
> > arm.h
> > index a74cce2..94695b7 100644
> > --- a/kexec/arch/arm/kexec-arm.h
> > +++ b/kexec/arch/arm/kexec-arm.h
> > @@ -4,6 +4,7 @@
> > ?#include <sys/types.h>
> > 
> > ?#define SYSFS_FDT "/sys/firmware/fdt"
> > +#define APPENDED_FDT ((char *)-1) /* it doesn't mean to be opened.
> > */
> > ?#define BOOT_BLOCK_VERSION 17
> > ?#define BOOT_BLOCK_LAST_COMP_VERSION 16
> > 
> > diff --git a/kexec/arch/arm/kexec-zImage-arm.c
> > b/kexec/arch/arm/kexec-zImage-arm.c
> > index 7f02b93..59f0003 100644
> > --- a/kexec/arch/arm/kexec-zImage-arm.c
> > +++ b/kexec/arch/arm/kexec-zImage-arm.c
> > @@ -390,6 +390,7 @@ int zImage_arm_load(int argc, char **argv,
> > const char *buf, off_t len,
> > ?	initrd_size = 0;
> > ?	use_atags = 0;
> > ?	dtb_file = NULL;
> > +	dtb_buf = NULL;
> > ?	while((opt = getopt_long(argc, argv, short_options,
> > options, 0)) != -1) {
> > ?		switch(opt) {
> > ?		default:
> > @@ -424,14 +425,6 @@ int zImage_arm_load(int argc, char **argv,
> > const char *buf, off_t len,
> > ?		return -1;
> > ?	}
> > 
> > -	if (!use_atags && !dtb_file) {
> > -		int f;
> > -
> > -		f = have_sysfs_fdt();
> > -		if (f)
> > -			dtb_file = SYSFS_FDT;
> > -	}
> > -
> > ?	if (command_line) {
> > ?		command_line_len = strlen(command_line) + 1;
> > ?		if (command_line_len > COMMAND_LINE_SIZE)
> > @@ -440,9 +433,6 @@ int zImage_arm_load(int argc, char **argv,
> > const char *buf, off_t len,
> > ?	if (ramdisk)
> > ?		ramdisk_buf = slurp_file(ramdisk, &initrd_size);
> > 
> > -	if (dtb_file)
> > -		dtb_buf = slurp_file(dtb_file, &dtb_length);
> > -
> > ?	if (len > 0x34) {
> > ?		const struct zimage_header *hdr;
> > ?		off_t size;
> > @@ -459,6 +449,25 @@ int zImage_arm_load(int argc, char **argv,
> > const char *buf, off_t len,
> > ?				??(unsigned long long)size,
> > ?				??(unsigned long long)len);
> > 
> > +			/* check if zImage has an appended dtb */
> > +			if (!dtb_file) {
> > +				if (fdt_check_header(buf + size)
> > == 0) {
> What if size >= len? I think, in that case there might not be a valid
> address?
> at buf+size and reading those area would lead in segmentation fault.
> 

Oh. I'll fix the bug in the next version.

Thank you for the review.

> > 
> > +					/*
> > +					?* dtb_file won't be
> > opened. It's set
> > +					?* just to pass NULL
> > checking.
> > +					?*/
> > +					dtb_file = APPENDED_FDT;
> > +					dtb_length =
> > fdt_totalsize(buf + size);
> > +					dtb_buf =
> > xmalloc(dtb_length);
> > +					memcpy(dtb_buf, buf+size,
> > dtb_length);
> > +
> > +					dbgprintf("found appended
> > dtb, size 0x%llx\n",
> > +						??(unsigned long
> > long)dtb_length);
> > +
> > +
> > +				}
> > +			}
> > +
> > ?			if (size > len) {
> > ?				fprintf(stderr,
> > ?					"zImage is truncated -
> > file 0x%llx vs header 0x%llx\n",
> > @@ -575,6 +584,22 @@ int zImage_arm_load(int argc, char **argv,
> > const char *buf, off_t len,
> > ?		initrd_base = kernel_base + _ALIGN(len * 5,
> > getpagesize());
> > ?	}
> > 
> > +	if (!use_atags && !dtb_file) {
> > +		int f;
> > +
> > +		f = have_sysfs_fdt();
> > +		if (f)
> > +			dtb_file = SYSFS_FDT;
> > +	}
> > +
> > +	/*
> > +	?* dtb_buf can be allocated when checking zImage_with_dtb.
> > +	?* dtb_buf won't be allocated in the logic if dtb_file is
> > handed over
> > +	?* in argv[].
> > +	?*/
> > +	if (dtb_file && !dtb_buf)
> > +		dtb_buf = slurp_file(dtb_file, &dtb_length);
> > +
> > ?	if (use_atags) {
> > ?		/*
> > ?		?* use ATAGs from /proc/atags
> > 
> --
> Pratyush


From rmk at armlinux.org.uk  Tue Jun 27 01:53:59 2017
From: rmk at armlinux.org.uk (Russell King)
Date: Tue, 27 Jun 2017 09:53:59 +0100
Subject: [PATCH] kexec:arm: support zImage with appended device tree
In-Reply-To: <1498531003.1734.13.camel@gmail.com>
References: <1498208138-19283-1-git-send-email-hoeun.ryu@gmail.com>
 <20170626091412.GE837@flint.armlinux.org.uk>
 <1498531003.1734.13.camel@gmail.com>
Message-ID: <20170627085359.GH837@flint.armlinux.org.uk>

On Tue, Jun 27, 2017 at 11:36:43AM +0900, Hoeun Ryu wrote:
> We have some cases when we would like to use different dtb from the
> running system when using kexec and I think that's why kexec-tools
> supports --dtb command line option.
> 
> For example, I have the second kernel for the crash dump with different
> kernel configuration and the different dtb from the running system. I'd
> like to exclude some nodes/properties from dtb like memory reservations
> or unnecessary devices to keep the second kernel/dtb minimal.
> 
> The concatenated zImage for arm has a benefit (whether it's intended or
> not) to make it possible for users to merge multiple boot images into a
> simple single file.
> What I'd like to do is just to support the concatenated zImage so that
> users can use --load(-panic) zImage_with_dtb_from_the_running_system
> instead of --load(-panic) zImage --dtb
> different_dtb_from_the_running_system.

Don't you mean --load(-panic) zImage_with_different_dtb_from_the_running_system
?

So, what you seem to be saying is that you think it's easier for the user
to do this:

# cat zImage some-other-dtb > zImage.dtb
# kexec --load-panic zImage.dtb

rather than:

# kexec --load-panic zImage --dtb some-other-dtb

?

What about the initrd?  Do you want to append that as well?

-- 
Russell King


From hoeun.ryu at gmail.com  Tue Jun 27 02:51:50 2017
From: hoeun.ryu at gmail.com (Hoeun Ryu)
Date: Tue, 27 Jun 2017 18:51:50 +0900
Subject: [PATCH] kexec:arm: support zImage with appended device tree
In-Reply-To: <20170627085359.GH837@flint.armlinux.org.uk>
References: <1498208138-19283-1-git-send-email-hoeun.ryu@gmail.com>
 <20170626091412.GE837@flint.armlinux.org.uk>
 <1498531003.1734.13.camel@gmail.com>
 <20170627085359.GH837@flint.armlinux.org.uk>
Message-ID: <BA3D72AC-402C-4148-B49A-560720267961@gmail.com>


> On Jun 27, 2017, at 5:53 PM, Russell King <rmk at armlinux.org.uk> wrote:
> 
>> On Tue, Jun 27, 2017 at 11:36:43AM +0900, Hoeun Ryu wrote:
>> We have some cases when we would like to use different dtb from the
>> running system when using kexec and I think that's why kexec-tools
>> supports --dtb command line option.
>> 
>> For example, I have the second kernel for the crash dump with different
>> kernel configuration and the different dtb from the running system. I'd
>> like to exclude some nodes/properties from dtb like memory reservations
>> or unnecessary devices to keep the second kernel/dtb minimal.
>> 
>> The concatenated zImage for arm has a benefit (whether it's intended or
>> not) to make it possible for users to merge multiple boot images into a
>> simple single file.
>> What I'd like to do is just to support the concatenated zImage so that
>> users can use --load(-panic) zImage_with_dtb_from_the_running_system
>> instead of --load(-panic) zImage --dtb
>> different_dtb_from_the_running_system.
> 
> Don't you mean --load(-panic) zImage_with_different_dtb_from_the_running_system
> ?
> 

I'm sorry. This is what I mean.

> So, what you seem to be saying is that you think it's easier for the user
> to do this:
> 
> # cat zImage some-other-dtb > zImage.dtb
> # kexec --load-panic zImage.dtb
> 
> rather than:
> 
> # kexec --load-panic zImage --dtb some-other-dtb
> 
> ?

Right.

> What about the initrd?  Do you want to append that as well?

I have thought of it.
I think It would be better to have it.
But I think this is the first step to do so.

> 
> -- 
> Russell King


From thomas.lendacky at amd.com  Tue Jun 27 07:56:08 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Tue, 27 Jun 2017 09:56:08 -0500
Subject: [PATCH v8 00/38] x86: Secure Memory Encryption (AMD)
Message-ID: <20170627145607.15908.26571.stgit@tlendack-t1.amdoffice.net>

This patch series provides support for AMD's new Secure Memory Encryption (SME)
feature.

SME can be used to mark individual pages of memory as encrypted through the
page tables. A page of memory that is marked encrypted will be automatically
decrypted when read from DRAM and will be automatically encrypted when
written to DRAM. Details on SME can found in the links below.

The SME feature is identified through a CPUID function and enabled through
the SYSCFG MSR. Once enabled, page table entries will determine how the
memory is accessed. If a page table entry has the memory encryption mask set,
then that memory will be accessed as encrypted memory. The memory encryption
mask (as well as other related information) is determined from settings
returned through the same CPUID function that identifies the presence of the
feature.

The approach that this patch series takes is to encrypt everything possible
starting early in the boot where the kernel is encrypted. Using the page
table macros the encryption mask can be incorporated into all page table
entries and page allocations. By updating the protection map, userspace
allocations are also marked encrypted. Certain data must be accounted for
as having been placed in memory before SME was enabled (EFI, initrd, etc.)
and accessed accordingly.

This patch series is a pre-cursor to another AMD processor feature called
Secure Encrypted Virtualization (SEV). The support for SEV will build upon
the SME support and will be submitted later. Details on SEV can be found
in the links below.

The following links provide additional detail:

AMD Memory Encryption whitepaper:
   http://amd-dev.wpengine.netdna-cdn.com/wordpress/media/2013/12/AMD_Memory_Encryption_Whitepaper_v7-Public.pdf

AMD64 Architecture Programmer's Manual:
   http://support.amd.com/TechDocs/24593.pdf
   SME is section 7.10
   SEV is section 15.34

---

This patch series is based off of the master branch of tip:
  https://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git master

  Commit 6ab5af989579 ("Merge branch 'irq/core'")

Source code is also available at https://github.com/codomania/tip/tree/sme-v8


Still to do:
- Kdump support, including using memremap() instead of ioremap_cache()

Changes since v7:
- Fixed kbuild test robot failure related to pgprot_decrypted() macro
  usage for some non-x86 archs
- Moved calls to encrypt the kernel and retrieve the encryption mask
  from assembler (head_64.S) into C (head64.c)
- Removed use of phys_to_virt() in __ioremap_caller() when address is in
  the ISA range. Now regular ioremap() processing occurs.
- Two new, small patches:
  - Introduced a native_make_p4d() for use when CONFIG_PGTABLE_LEVELS is
    not greater than 4
  - Introduced __nostackp GCC option to turn off stack protection on a
    per function basis
- General code cleanup based on feedback

Changes since v6:
- Fixed the asm include file issue that caused build errors on other archs
- Rebased the CR3 register changes on top of Andy Lutomirski's patch
- Added a patch to clear the SME cpu feature if running as a PV guest under
  Xen
- Added a patch to obtain the AMD microcode level earlier in the boot
  instead of directly reading the MSR
- Refactor patch #8 ("x86/mm: Add support to enable SME in early boot
  processing") because the 5-level paging support moved the code into the
  new C-function __startup_64()
- Removed need to decrypt trampoline area in-place (set memory attributes
  before copying the trampoline code)
- General code cleanup based on feedback

Changes since v5:
- Added support for 5-level paging
- Added IOMMU support
- Created a generic asm/mem_encrypt.h in order to remove a bunch of
  #ifndef/#define entries
- Removed changes to the __va() macro and defined a function to return
  the true physical address in cr3
- Removed sysfs support as it was determined not to be needed
- General code cleanup based on feedback
- General cleanup of patch subjects and descriptions

Changes since v4:
- Re-worked mapping of setup data to not use a fixed list. Rather, check
  dynamically whether the requested early_memremap()/memremap() call
  needs to be mapped decrypted.
- Moved SME cpu feature into scattered features
- Moved some declarations into header files
- Cleared the encryption mask from the __PHYSICAL_MASK so that users
  of macros such as pmd_pfn_mask() don't have to worry/know about the
  encryption mask
- Updated some return types and values related to EFI and e820 functions
  so that an error could be returned
- During cpu shutdown, removed cache disabling and added a check for kexec
  in progress to use wbinvd followed immediately by halt in order to avoid
  any memory corruption
- Update how persistent memory is identified
- Added a function to find command line arguments and their values
- Added sysfs support
- General code cleanup based on feedback
- General cleanup of patch subjects and descriptions


Changes since v3:
- Broke out some of the patches into smaller individual patches
- Updated Documentation
- Added a message to indicate why the IOMMU was disabled
- Updated CPU feature support for SME by taking into account whether
  BIOS has enabled SME
- Eliminated redundant functions
- Added some warning messages for DMA usage of bounce buffers when SME
  is active
- Added support for persistent memory
- Added support to determine when setup data is being mapped and be sure
  to map it un-encrypted
- Added CONFIG support to set the default action of whether to activate
  SME if it is supported/enabled
- Added support for (re)booting with kexec

Changes since v2:
- Updated Documentation
- Make the encryption mask available outside of arch/x86 through a
  standard include file
- Conversion of assembler routines to C where possible (not everything
  could be converted, e.g. the routine that does the actual encryption
  needs to be copied into a safe location and it is difficult to
  determine the actual length of the function in order to copy it)
- Fix SME feature use of scattered CPUID feature
- Creation of SME specific functions for things like encrypting
  the setup data, ramdisk, etc.
- New take on early_memremap / memremap encryption support
- Additional support for accessing video buffers (fbdev/gpu) as
  un-encrypted
- Disable IOMMU for now - need to investigate further in relation to
  how it needs to be programmed relative to accessing physical memory

Changes since v1:
- Added Documentation.
- Removed AMD vendor check for setting the PAT write protect mode
- Updated naming of trampoline flag for SME as well as moving of the
  SME check to before paging is enabled.
- Change to early_memremap to identify the data being mapped as either
  boot data or kernel data.  The idea being that boot data will have
  been placed in memory as un-encrypted data and would need to be accessed
  as such.
- Updated debugfs support for the bootparams to access the data properly.
- Do not set the SYSCFG[MEME] bit, only check it.  The setting of the
  MemEncryptionModeEn bit results in a reduction of physical address size
  of the processor.  It is possible that BIOS could have configured resources
  resources into a range that will now not be addressable.  To prevent this,
  rely on BIOS to set the SYSCFG[MEME] bit and only then enable memory
  encryption support in the kernel.

Tom Lendacky (38):
      x86: Document AMD Secure Memory Encryption (SME)
      x86/mm/pat: Set write-protect cache mode for full PAT support
      x86, mpparse, x86/acpi, x86/PCI, x86/dmi, SFI: Use memremap for RAM mappings
      x86/CPU/AMD: Add the Secure Memory Encryption CPU feature
      x86/CPU/AMD: Handle SME reduction in physical address size
      x86/mm: Add Secure Memory Encryption (SME) support
      x86/mm: Remove phys_to_virt() usage in ioremap()
      x86/mm: Add support to enable SME in early boot processing
      x86/mm: Simplify p[g4um]d_page() macros
      x86/mm: Provide general kernel support for memory encryption
      x86/mm: Add SME support for read_cr3_pa()
      x86/mm: Extend early_memremap() support with additional attrs
      x86/mm: Add support for early encrypt/decrypt of memory
      x86/mm: Insure that boot memory areas are mapped properly
      x86/boot/e820: Add support to determine the E820 type of an address
      efi: Add an EFI table address match function
      efi: Update efi_mem_type() to return an error rather than 0
      x86/efi: Update EFI pagetable creation to work with SME
      x86/mm: Add support to access boot related data in the clear
      x86, mpparse: Use memremap to map the mpf and mpc data
      x86/mm: Add support to access persistent memory in the clear
      x86/mm: Add support for changing the memory encryption attribute
      x86/realmode: Decrypt trampoline area if memory encryption is active
      x86, swiotlb: Add memory encryption support
      swiotlb: Add warnings for use of bounce buffers with SME
      x86/CPU/AMD: Make the microcode level available earlier in the boot
      iommu/amd: Allow the AMD IOMMU to work with memory encryption
      x86, realmode: Check for memory encryption on the APs
      x86, drm, fbdev: Do not specify encrypted memory for video mappings
      kvm: x86: svm: Support Secure Memory Encryption within KVM
      x86/mm, kexec: Allow kexec to be used with SME
      xen/x86: Remove SME feature in PV guests
      x86/mm: Use proper encryption attributes with /dev/mem
      x86/mm: Create native_make_p4d() for PGTABLE_LEVELS <= 4
      x86/mm: Add support to encrypt the kernel in-place
      x86/boot: Add early cmdline parsing for options with arguments
      compiler-gcc.h: Introduce __nostackp function attribute
      x86/mm: Add support to make use of Secure Memory Encryption


 Documentation/admin-guide/kernel-parameters.txt |   11 
 Documentation/x86/amd-memory-encryption.txt     |   68 +++
 arch/ia64/kernel/efi.c                          |    4 
 arch/x86/Kconfig                                |   29 +
 arch/x86/boot/compressed/pagetable.c            |    7 
 arch/x86/include/asm/cmdline.h                  |    2 
 arch/x86/include/asm/cpufeatures.h              |    1 
 arch/x86/include/asm/dma-mapping.h              |    5 
 arch/x86/include/asm/dmi.h                      |    8 
 arch/x86/include/asm/e820/api.h                 |    2 
 arch/x86/include/asm/fixmap.h                   |   20 +
 arch/x86/include/asm/init.h                     |    1 
 arch/x86/include/asm/io.h                       |    8 
 arch/x86/include/asm/kexec.h                    |    8 
 arch/x86/include/asm/kvm_host.h                 |    2 
 arch/x86/include/asm/mem_encrypt.h              |   80 +++
 arch/x86/include/asm/msr-index.h                |    2 
 arch/x86/include/asm/page_types.h               |    3 
 arch/x86/include/asm/pgtable.h                  |   28 +
 arch/x86/include/asm/pgtable_types.h            |   57 ++
 arch/x86/include/asm/processor-flags.h          |    5 
 arch/x86/include/asm/processor.h                |    8 
 arch/x86/include/asm/realmode.h                 |   12 
 arch/x86/include/asm/set_memory.h               |    3 
 arch/x86/include/asm/vga.h                      |   14 +
 arch/x86/kernel/acpi/boot.c                     |    6 
 arch/x86/kernel/cpu/amd.c                       |   25 +
 arch/x86/kernel/cpu/scattered.c                 |    1 
 arch/x86/kernel/e820.c                          |   26 +
 arch/x86/kernel/espfix_64.c                     |    2 
 arch/x86/kernel/head64.c                        |   93 +++-
 arch/x86/kernel/head_64.S                       |   40 +-
 arch/x86/kernel/kdebugfs.c                      |   34 -
 arch/x86/kernel/ksysfs.c                        |   28 +
 arch/x86/kernel/machine_kexec_64.c              |   22 +
 arch/x86/kernel/mpparse.c                       |  108 +++-
 arch/x86/kernel/pci-dma.c                       |   11 
 arch/x86/kernel/pci-nommu.c                     |    2 
 arch/x86/kernel/pci-swiotlb.c                   |   15 +
 arch/x86/kernel/process.c                       |   17 +
 arch/x86/kernel/setup.c                         |    9 
 arch/x86/kvm/mmu.c                              |   12 
 arch/x86/kvm/mmu.h                              |    2 
 arch/x86/kvm/svm.c                              |   35 +
 arch/x86/kvm/vmx.c                              |    3 
 arch/x86/kvm/x86.c                              |    3 
 arch/x86/lib/cmdline.c                          |  105 ++++
 arch/x86/mm/Makefile                            |    2 
 arch/x86/mm/ident_map.c                         |   12 
 arch/x86/mm/ioremap.c                           |  275 ++++++++++-
 arch/x86/mm/kasan_init_64.c                     |    6 
 arch/x86/mm/mem_encrypt.c                       |  593 +++++++++++++++++++++++
 arch/x86/mm/mem_encrypt_boot.S                  |  150 ++++++
 arch/x86/mm/pageattr.c                          |   67 +++
 arch/x86/mm/pat.c                               |    9 
 arch/x86/pci/common.c                           |    4 
 arch/x86/platform/efi/efi.c                     |    6 
 arch/x86/platform/efi/efi_64.c                  |   15 -
 arch/x86/realmode/init.c                        |   12 
 arch/x86/realmode/rm/trampoline_64.S            |   24 +
 arch/x86/xen/enlighten_pv.c                     |    1 
 drivers/firmware/dmi-sysfs.c                    |    5 
 drivers/firmware/efi/efi.c                      |   33 +
 drivers/firmware/pcdp.c                         |    4 
 drivers/gpu/drm/drm_gem.c                       |    2 
 drivers/gpu/drm/drm_vm.c                        |    4 
 drivers/gpu/drm/ttm/ttm_bo_vm.c                 |    7 
 drivers/gpu/drm/udl/udl_fb.c                    |    4 
 drivers/iommu/amd_iommu.c                       |   30 +
 drivers/iommu/amd_iommu_init.c                  |   34 +
 drivers/iommu/amd_iommu_proto.h                 |   10 
 drivers/iommu/amd_iommu_types.h                 |    2 
 drivers/sfi/sfi_core.c                          |   22 -
 drivers/video/fbdev/core/fbmem.c                |   12 
 include/asm-generic/early_ioremap.h             |    2 
 include/asm-generic/pgtable.h                   |   12 
 include/linux/compiler-gcc.h                    |    2 
 include/linux/compiler.h                        |    4 
 include/linux/dma-mapping.h                     |   13 +
 include/linux/efi.h                             |    9 
 include/linux/io.h                              |    2 
 include/linux/kexec.h                           |    8 
 include/linux/mem_encrypt.h                     |   48 ++
 include/linux/swiotlb.h                         |    1 
 init/main.c                                     |   10 
 kernel/kexec_core.c                             |   12 
 kernel/memremap.c                               |   20 +
 lib/swiotlb.c                                   |   57 ++
 mm/early_ioremap.c                              |   28 +
 89 files changed, 2293 insertions(+), 267 deletions(-)
 create mode 100644 Documentation/x86/amd-memory-encryption.txt
 create mode 100644 arch/x86/include/asm/mem_encrypt.h
 create mode 100644 arch/x86/mm/mem_encrypt.c
 create mode 100644 arch/x86/mm/mem_encrypt_boot.S
 create mode 100644 include/linux/mem_encrypt.h

-- 
Tom Lendacky


From thomas.lendacky at amd.com  Tue Jun 27 07:56:20 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Tue, 27 Jun 2017 09:56:20 -0500
Subject: [PATCH v8 01/38] x86: Document AMD Secure Memory Encryption (SME)
In-Reply-To: <20170627145607.15908.26571.stgit@tlendack-t1.amdoffice.net>
References: <20170627145607.15908.26571.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170627145620.15908.94963.stgit@tlendack-t1.amdoffice.net>

Create a Documentation entry to describe the AMD Secure Memory
Encryption (SME) feature and add documentation for the mem_encrypt=
kernel parameter.

Reviewed-by: Borislav Petkov <bp at suse.de>
Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 Documentation/admin-guide/kernel-parameters.txt |   11 ++++
 Documentation/x86/amd-memory-encryption.txt     |   68 +++++++++++++++++++++++
 2 files changed, 79 insertions(+)
 create mode 100644 Documentation/x86/amd-memory-encryption.txt

diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index 9b0b3de..51e03ee 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -2197,6 +2197,17 @@
 			memory contents and reserves bad memory
 			regions that are detected.
 
+	mem_encrypt=	[X86-64] AMD Secure Memory Encryption (SME) control
+			Valid arguments: on, off
+			Default (depends on kernel configuration option):
+			  on  (CONFIG_AMD_MEM_ENCRYPT_ACTIVE_BY_DEFAULT=y)
+			  off (CONFIG_AMD_MEM_ENCRYPT_ACTIVE_BY_DEFAULT=n)
+			mem_encrypt=on:		Activate SME
+			mem_encrypt=off:	Do not activate SME
+
+			Refer to Documentation/x86/amd-memory-encryption.txt
+			for details on when memory encryption can be activated.
+
 	mem_sleep_default=	[SUSPEND] Default system suspend mode:
 			s2idle  - Suspend-To-Idle
 			shallow - Power-On Suspend or equivalent (if supported)
diff --git a/Documentation/x86/amd-memory-encryption.txt b/Documentation/x86/amd-memory-encryption.txt
new file mode 100644
index 0000000..f512ab7
--- /dev/null
+++ b/Documentation/x86/amd-memory-encryption.txt
@@ -0,0 +1,68 @@
+Secure Memory Encryption (SME) is a feature found on AMD processors.
+
+SME provides the ability to mark individual pages of memory as encrypted using
+the standard x86 page tables.  A page that is marked encrypted will be
+automatically decrypted when read from DRAM and encrypted when written to
+DRAM.  SME can therefore be used to protect the contents of DRAM from physical
+attacks on the system.
+
+A page is encrypted when a page table entry has the encryption bit set (see
+below on how to determine its position).  The encryption bit can also be
+specified in the cr3 register, allowing the PGD table to be encrypted. Each
+successive level of page tables can also be encrypted by setting the encryption
+bit in the page table entry that points to the next table. This allows the full
+page table hierarchy to be encrypted. Note, this means that just because the
+encryption bit is set in cr3, doesn't imply the full hierarchy is encyrpted.
+Each page table entry in the hierarchy needs to have the encryption bit set to
+achieve that. So, theoretically, you could have the encryption bit set in cr3
+so that the PGD is encrypted, but not set the encryption bit in the PGD entry
+for a PUD which results in the PUD pointed to by that entry to not be
+encrypted.
+
+Support for SME can be determined through the CPUID instruction. The CPUID
+function 0x8000001f reports information related to SME:
+
+	0x8000001f[eax]:
+		Bit[0] indicates support for SME
+	0x8000001f[ebx]:
+		Bits[5:0]  pagetable bit number used to activate memory
+			   encryption
+		Bits[11:6] reduction in physical address space, in bits, when
+			   memory encryption is enabled (this only affects
+			   system physical addresses, not guest physical
+			   addresses)
+
+If support for SME is present, MSR 0xc00100010 (MSR_K8_SYSCFG) can be used to
+determine if SME is enabled and/or to enable memory encryption:
+
+	0xc0010010:
+		Bit[23]   0 = memory encryption features are disabled
+			  1 = memory encryption features are enabled
+
+Linux relies on BIOS to set this bit if BIOS has determined that the reduction
+in the physical address space as a result of enabling memory encryption (see
+CPUID information above) will not conflict with the address space resource
+requirements for the system.  If this bit is not set upon Linux startup then
+Linux itself will not set it and memory encryption will not be possible.
+
+The state of SME in the Linux kernel can be documented as follows:
+	- Supported:
+	  The CPU supports SME (determined through CPUID instruction).
+
+	- Enabled:
+	  Supported and bit 23 of MSR_K8_SYSCFG is set.
+
+	- Active:
+	  Supported, Enabled and the Linux kernel is actively applying
+	  the encryption bit to page table entries (the SME mask in the
+	  kernel is non-zero).
+
+SME can also be enabled and activated in the BIOS. If SME is enabled and
+activated in the BIOS, then all memory accesses will be encrypted and it will
+not be necessary to activate the Linux memory encryption support.  If the BIOS
+merely enables SME (sets bit 23 of the MSR_K8_SYSCFG), then Linux can activate
+memory encryption by default (CONFIG_AMD_MEM_ENCRYPT_ACTIVE_BY_DEFAULT=y) or
+by supplying mem_encrypt=on on the kernel command line.  However, if BIOS does
+not enable SME, then Linux will not be able to activate memory encryption, even
+if configured to do so by default or the mem_encrypt=on command line parameter
+is specified.



From thomas.lendacky at amd.com  Tue Jun 27 07:56:31 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Tue, 27 Jun 2017 09:56:31 -0500
Subject: [PATCH v8 02/38] x86/mm/pat: Set write-protect cache mode for full
 PAT support
In-Reply-To: <20170627145607.15908.26571.stgit@tlendack-t1.amdoffice.net>
References: <20170627145607.15908.26571.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170627145631.15908.28761.stgit@tlendack-t1.amdoffice.net>

For processors that support PAT, set the write-protect cache mode
(_PAGE_CACHE_MODE_WP) entry to the actual write-protect value (x05).

Acked-by: Borislav Petkov <bp at suse.de>
Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/mm/pat.c |    6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/arch/x86/mm/pat.c b/arch/x86/mm/pat.c
index 9b78685..6753d9c 100644
--- a/arch/x86/mm/pat.c
+++ b/arch/x86/mm/pat.c
@@ -295,7 +295,7 @@ static void init_cache_modes(void)
  * pat_init - Initialize PAT MSR and PAT table
  *
  * This function initializes PAT MSR and PAT table with an OS-defined value
- * to enable additional cache attributes, WC and WT.
+ * to enable additional cache attributes, WC, WT and WP.
  *
  * This function must be called on all CPUs using the specific sequence of
  * operations defined in Intel SDM. mtrr_rendezvous_handler() provides this
@@ -356,7 +356,7 @@ void pat_init(void)
 		 *      010    2    UC-: _PAGE_CACHE_MODE_UC_MINUS
 		 *      011    3    UC : _PAGE_CACHE_MODE_UC
 		 *      100    4    WB : Reserved
-		 *      101    5    WC : Reserved
+		 *      101    5    WP : _PAGE_CACHE_MODE_WP
 		 *      110    6    UC-: Reserved
 		 *      111    7    WT : _PAGE_CACHE_MODE_WT
 		 *
@@ -364,7 +364,7 @@ void pat_init(void)
 		 * corresponding types in the presence of PAT errata.
 		 */
 		pat = PAT(0, WB) | PAT(1, WC) | PAT(2, UC_MINUS) | PAT(3, UC) |
-		      PAT(4, WB) | PAT(5, WC) | PAT(6, UC_MINUS) | PAT(7, WT);
+		      PAT(4, WB) | PAT(5, WP) | PAT(6, UC_MINUS) | PAT(7, WT);
 	}
 
 	if (!boot_cpu_done) {



From thomas.lendacky at amd.com  Tue Jun 27 07:56:41 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Tue, 27 Jun 2017 09:56:41 -0500
Subject: [PATCH v8 03/38] x86, mpparse, x86/acpi, x86/PCI, x86/dmi,
 SFI: Use memremap for RAM mappings
In-Reply-To: <20170627145607.15908.26571.stgit@tlendack-t1.amdoffice.net>
References: <20170627145607.15908.26571.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170627145641.15908.7401.stgit@tlendack-t1.amdoffice.net>

The ioremap() function is intended for mapping MMIO. For RAM, the
memremap() function should be used. Convert calls from ioremap() to
memremap() when re-mapping RAM.

This will be used later by SME to control how the encryption mask is
applied to memory mappings, with certain memory locations being mapped
decrypted vs encrypted.

Reviewed-by: Borislav Petkov <bp at suse.de>
Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/include/asm/dmi.h   |    8 ++++----
 arch/x86/kernel/acpi/boot.c  |    6 +++---
 arch/x86/kernel/kdebugfs.c   |   34 +++++++++++-----------------------
 arch/x86/kernel/ksysfs.c     |   28 ++++++++++++++--------------
 arch/x86/kernel/mpparse.c    |   10 +++++-----
 arch/x86/pci/common.c        |    4 ++--
 drivers/firmware/dmi-sysfs.c |    5 +++--
 drivers/firmware/pcdp.c      |    4 ++--
 drivers/sfi/sfi_core.c       |   22 +++++++++++-----------
 9 files changed, 55 insertions(+), 66 deletions(-)

diff --git a/arch/x86/include/asm/dmi.h b/arch/x86/include/asm/dmi.h
index 3c69fed..a8e15b0 100644
--- a/arch/x86/include/asm/dmi.h
+++ b/arch/x86/include/asm/dmi.h
@@ -13,9 +13,9 @@ static __always_inline __init void *dmi_alloc(unsigned len)
 }
 
 /* Use early IO mappings for DMI because it's initialized early */
-#define dmi_early_remap		early_ioremap
-#define dmi_early_unmap		early_iounmap
-#define dmi_remap		ioremap_cache
-#define dmi_unmap		iounmap
+#define dmi_early_remap		early_memremap
+#define dmi_early_unmap		early_memunmap
+#define dmi_remap(_x, _l)	memremap(_x, _l, MEMREMAP_WB)
+#define dmi_unmap(_x)		memunmap(_x)
 
 #endif /* _ASM_X86_DMI_H */
diff --git a/arch/x86/kernel/acpi/boot.c b/arch/x86/kernel/acpi/boot.c
index 6bb6806..850160a 100644
--- a/arch/x86/kernel/acpi/boot.c
+++ b/arch/x86/kernel/acpi/boot.c
@@ -115,7 +115,7 @@
 #define	ACPI_INVALID_GSI		INT_MIN
 
 /*
- * This is just a simple wrapper around early_ioremap(),
+ * This is just a simple wrapper around early_memremap(),
  * with sanity checks for phys == 0 and size == 0.
  */
 char *__init __acpi_map_table(unsigned long phys, unsigned long size)
@@ -124,7 +124,7 @@ char *__init __acpi_map_table(unsigned long phys, unsigned long size)
 	if (!phys || !size)
 		return NULL;
 
-	return early_ioremap(phys, size);
+	return early_memremap(phys, size);
 }
 
 void __init __acpi_unmap_table(char *map, unsigned long size)
@@ -132,7 +132,7 @@ void __init __acpi_unmap_table(char *map, unsigned long size)
 	if (!map || !size)
 		return;
 
-	early_iounmap(map, size);
+	early_memunmap(map, size);
 }
 
 #ifdef CONFIG_X86_LOCAL_APIC
diff --git a/arch/x86/kernel/kdebugfs.c b/arch/x86/kernel/kdebugfs.c
index 38b6458..fd6f8fb 100644
--- a/arch/x86/kernel/kdebugfs.c
+++ b/arch/x86/kernel/kdebugfs.c
@@ -33,7 +33,6 @@ static ssize_t setup_data_read(struct file *file, char __user *user_buf,
 	struct setup_data_node *node = file->private_data;
 	unsigned long remain;
 	loff_t pos = *ppos;
-	struct page *pg;
 	void *p;
 	u64 pa;
 
@@ -47,18 +46,13 @@ static ssize_t setup_data_read(struct file *file, char __user *user_buf,
 		count = node->len - pos;
 
 	pa = node->paddr + sizeof(struct setup_data) + pos;
-	pg = pfn_to_page((pa + count - 1) >> PAGE_SHIFT);
-	if (PageHighMem(pg)) {
-		p = ioremap_cache(pa, count);
-		if (!p)
-			return -ENXIO;
-	} else
-		p = __va(pa);
+	p = memremap(pa, count, MEMREMAP_WB);
+	if (!p)
+		return -ENOMEM;
 
 	remain = copy_to_user(user_buf, p, count);
 
-	if (PageHighMem(pg))
-		iounmap(p);
+	memunmap(p);
 
 	if (remain)
 		return -EFAULT;
@@ -109,7 +103,6 @@ static int __init create_setup_data_nodes(struct dentry *parent)
 	struct setup_data *data;
 	int error;
 	struct dentry *d;
-	struct page *pg;
 	u64 pa_data;
 	int no = 0;
 
@@ -126,16 +119,12 @@ static int __init create_setup_data_nodes(struct dentry *parent)
 			goto err_dir;
 		}
 
-		pg = pfn_to_page((pa_data+sizeof(*data)-1) >> PAGE_SHIFT);
-		if (PageHighMem(pg)) {
-			data = ioremap_cache(pa_data, sizeof(*data));
-			if (!data) {
-				kfree(node);
-				error = -ENXIO;
-				goto err_dir;
-			}
-		} else
-			data = __va(pa_data);
+		data = memremap(pa_data, sizeof(*data), MEMREMAP_WB);
+		if (!data) {
+			kfree(node);
+			error = -ENOMEM;
+			goto err_dir;
+		}
 
 		node->paddr = pa_data;
 		node->type = data->type;
@@ -143,8 +132,7 @@ static int __init create_setup_data_nodes(struct dentry *parent)
 		error = create_setup_data_node(d, no, node);
 		pa_data = data->next;
 
-		if (PageHighMem(pg))
-			iounmap(data);
+		memunmap(data);
 		if (error)
 			goto err_dir;
 		no++;
diff --git a/arch/x86/kernel/ksysfs.c b/arch/x86/kernel/ksysfs.c
index 4afc67f..ee51db9 100644
--- a/arch/x86/kernel/ksysfs.c
+++ b/arch/x86/kernel/ksysfs.c
@@ -16,8 +16,8 @@
 #include <linux/stat.h>
 #include <linux/slab.h>
 #include <linux/mm.h>
+#include <linux/io.h>
 
-#include <asm/io.h>
 #include <asm/setup.h>
 
 static ssize_t version_show(struct kobject *kobj,
@@ -79,12 +79,12 @@ static int get_setup_data_paddr(int nr, u64 *paddr)
 			*paddr = pa_data;
 			return 0;
 		}
-		data = ioremap_cache(pa_data, sizeof(*data));
+		data = memremap(pa_data, sizeof(*data), MEMREMAP_WB);
 		if (!data)
 			return -ENOMEM;
 
 		pa_data = data->next;
-		iounmap(data);
+		memunmap(data);
 		i++;
 	}
 	return -EINVAL;
@@ -97,17 +97,17 @@ static int __init get_setup_data_size(int nr, size_t *size)
 	u64 pa_data = boot_params.hdr.setup_data;
 
 	while (pa_data) {
-		data = ioremap_cache(pa_data, sizeof(*data));
+		data = memremap(pa_data, sizeof(*data), MEMREMAP_WB);
 		if (!data)
 			return -ENOMEM;
 		if (nr == i) {
 			*size = data->len;
-			iounmap(data);
+			memunmap(data);
 			return 0;
 		}
 
 		pa_data = data->next;
-		iounmap(data);
+		memunmap(data);
 		i++;
 	}
 	return -EINVAL;
@@ -127,12 +127,12 @@ static ssize_t type_show(struct kobject *kobj,
 	ret = get_setup_data_paddr(nr, &paddr);
 	if (ret)
 		return ret;
-	data = ioremap_cache(paddr, sizeof(*data));
+	data = memremap(paddr, sizeof(*data), MEMREMAP_WB);
 	if (!data)
 		return -ENOMEM;
 
 	ret = sprintf(buf, "0x%x\n", data->type);
-	iounmap(data);
+	memunmap(data);
 	return ret;
 }
 
@@ -154,7 +154,7 @@ static ssize_t setup_data_data_read(struct file *fp,
 	ret = get_setup_data_paddr(nr, &paddr);
 	if (ret)
 		return ret;
-	data = ioremap_cache(paddr, sizeof(*data));
+	data = memremap(paddr, sizeof(*data), MEMREMAP_WB);
 	if (!data)
 		return -ENOMEM;
 
@@ -170,15 +170,15 @@ static ssize_t setup_data_data_read(struct file *fp,
 		goto out;
 
 	ret = count;
-	p = ioremap_cache(paddr + sizeof(*data), data->len);
+	p = memremap(paddr + sizeof(*data), data->len, MEMREMAP_WB);
 	if (!p) {
 		ret = -ENOMEM;
 		goto out;
 	}
 	memcpy(buf, p + off, count);
-	iounmap(p);
+	memunmap(p);
 out:
-	iounmap(data);
+	memunmap(data);
 	return ret;
 }
 
@@ -250,13 +250,13 @@ static int __init get_setup_data_total_num(u64 pa_data, int *nr)
 	*nr = 0;
 	while (pa_data) {
 		*nr += 1;
-		data = ioremap_cache(pa_data, sizeof(*data));
+		data = memremap(pa_data, sizeof(*data), MEMREMAP_WB);
 		if (!data) {
 			ret = -ENOMEM;
 			goto out;
 		}
 		pa_data = data->next;
-		iounmap(data);
+		memunmap(data);
 	}
 
 out:
diff --git a/arch/x86/kernel/mpparse.c b/arch/x86/kernel/mpparse.c
index 0d904d7..fd37f39 100644
--- a/arch/x86/kernel/mpparse.c
+++ b/arch/x86/kernel/mpparse.c
@@ -436,9 +436,9 @@ static unsigned long __init get_mpc_size(unsigned long physptr)
 	struct mpc_table *mpc;
 	unsigned long size;
 
-	mpc = early_ioremap(physptr, PAGE_SIZE);
+	mpc = early_memremap(physptr, PAGE_SIZE);
 	size = mpc->length;
-	early_iounmap(mpc, PAGE_SIZE);
+	early_memunmap(mpc, PAGE_SIZE);
 	apic_printk(APIC_VERBOSE, "  mpc: %lx-%lx\n", physptr, physptr + size);
 
 	return size;
@@ -450,7 +450,7 @@ static int __init check_physptr(struct mpf_intel *mpf, unsigned int early)
 	unsigned long size;
 
 	size = get_mpc_size(mpf->physptr);
-	mpc = early_ioremap(mpf->physptr, size);
+	mpc = early_memremap(mpf->physptr, size);
 	/*
 	 * Read the physical hardware table.  Anything here will
 	 * override the defaults.
@@ -461,10 +461,10 @@ static int __init check_physptr(struct mpf_intel *mpf, unsigned int early)
 #endif
 		pr_err("BIOS bug, MP table errors detected!...\n");
 		pr_cont("... disabling SMP support. (tell your hw vendor)\n");
-		early_iounmap(mpc, size);
+		early_memunmap(mpc, size);
 		return -1;
 	}
-	early_iounmap(mpc, size);
+	early_memunmap(mpc, size);
 
 	if (early)
 		return -1;
diff --git a/arch/x86/pci/common.c b/arch/x86/pci/common.c
index 190e718..08cf71c 100644
--- a/arch/x86/pci/common.c
+++ b/arch/x86/pci/common.c
@@ -691,7 +691,7 @@ int pcibios_add_device(struct pci_dev *dev)
 
 	pa_data = boot_params.hdr.setup_data;
 	while (pa_data) {
-		data = ioremap(pa_data, sizeof(*rom));
+		data = memremap(pa_data, sizeof(*rom), MEMREMAP_WB);
 		if (!data)
 			return -ENOMEM;
 
@@ -710,7 +710,7 @@ int pcibios_add_device(struct pci_dev *dev)
 			}
 		}
 		pa_data = data->next;
-		iounmap(data);
+		memunmap(data);
 	}
 	set_dma_domain_ops(dev);
 	set_dev_domain_options(dev);
diff --git a/drivers/firmware/dmi-sysfs.c b/drivers/firmware/dmi-sysfs.c
index ef76e5e..d5de6ee 100644
--- a/drivers/firmware/dmi-sysfs.c
+++ b/drivers/firmware/dmi-sysfs.c
@@ -25,6 +25,7 @@
 #include <linux/slab.h>
 #include <linux/list.h>
 #include <linux/io.h>
+#include <asm/dmi.h>
 
 #define MAX_ENTRY_TYPE 255 /* Most of these aren't used, but we consider
 			      the top entry type is only 8 bits */
@@ -380,7 +381,7 @@ static ssize_t dmi_sel_raw_read_phys32(struct dmi_sysfs_entry *entry,
 	u8 __iomem *mapped;
 	ssize_t wrote = 0;
 
-	mapped = ioremap(sel->access_method_address, sel->area_length);
+	mapped = dmi_remap(sel->access_method_address, sel->area_length);
 	if (!mapped)
 		return -EIO;
 
@@ -390,7 +391,7 @@ static ssize_t dmi_sel_raw_read_phys32(struct dmi_sysfs_entry *entry,
 		wrote++;
 	}
 
-	iounmap(mapped);
+	dmi_unmap(mapped);
 	return wrote;
 }
 
diff --git a/drivers/firmware/pcdp.c b/drivers/firmware/pcdp.c
index 75273a25..e83d6ae 100644
--- a/drivers/firmware/pcdp.c
+++ b/drivers/firmware/pcdp.c
@@ -95,7 +95,7 @@
 	if (efi.hcdp == EFI_INVALID_TABLE_ADDR)
 		return -ENODEV;
 
-	pcdp = early_ioremap(efi.hcdp, 4096);
+	pcdp = early_memremap(efi.hcdp, 4096);
 	printk(KERN_INFO "PCDP: v%d at 0x%lx\n", pcdp->rev, efi.hcdp);
 
 	if (strstr(cmdline, "console=hcdp")) {
@@ -131,6 +131,6 @@
 	}
 
 out:
-	early_iounmap(pcdp, 4096);
+	early_memunmap(pcdp, 4096);
 	return rc;
 }
diff --git a/drivers/sfi/sfi_core.c b/drivers/sfi/sfi_core.c
index 296db7a..d5ce534 100644
--- a/drivers/sfi/sfi_core.c
+++ b/drivers/sfi/sfi_core.c
@@ -86,13 +86,13 @@
 /*
  * FW creates and saves the SFI tables in memory. When these tables get
  * used, they may need to be mapped to virtual address space, and the mapping
- * can happen before or after the ioremap() is ready, so a flag is needed
+ * can happen before or after the memremap() is ready, so a flag is needed
  * to indicating this
  */
-static u32 sfi_use_ioremap __read_mostly;
+static u32 sfi_use_memremap __read_mostly;
 
 /*
- * sfi_un/map_memory calls early_ioremap/iounmap which is a __init function
+ * sfi_un/map_memory calls early_memremap/memunmap which is a __init function
  * and introduces section mismatch. So use __ref to make it calm.
  */
 static void __iomem * __ref sfi_map_memory(u64 phys, u32 size)
@@ -100,10 +100,10 @@ static void __iomem * __ref sfi_map_memory(u64 phys, u32 size)
 	if (!phys || !size)
 		return NULL;
 
-	if (sfi_use_ioremap)
-		return ioremap_cache(phys, size);
+	if (sfi_use_memremap)
+		return memremap(phys, size, MEMREMAP_WB);
 	else
-		return early_ioremap(phys, size);
+		return early_memremap(phys, size);
 }
 
 static void __ref sfi_unmap_memory(void __iomem *virt, u32 size)
@@ -111,10 +111,10 @@ static void __ref sfi_unmap_memory(void __iomem *virt, u32 size)
 	if (!virt || !size)
 		return;
 
-	if (sfi_use_ioremap)
-		iounmap(virt);
+	if (sfi_use_memremap)
+		memunmap(virt);
 	else
-		early_iounmap(virt, size);
+		early_memunmap(virt, size);
 }
 
 static void sfi_print_table_header(unsigned long long pa,
@@ -507,8 +507,8 @@ void __init sfi_init_late(void)
 	length = syst_va->header.len;
 	sfi_unmap_memory(syst_va, sizeof(struct sfi_table_simple));
 
-	/* Use ioremap now after it is ready */
-	sfi_use_ioremap = 1;
+	/* Use memremap now after it is ready */
+	sfi_use_memremap = 1;
 	syst_va = sfi_map_memory(syst_pa, length);
 
 	sfi_acpi_init();



From thomas.lendacky at amd.com  Tue Jun 27 07:56:49 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Tue, 27 Jun 2017 09:56:49 -0500
Subject: [PATCH v8 04/38] x86/CPU/AMD: Add the Secure Memory Encryption CPU
 feature
In-Reply-To: <20170627145607.15908.26571.stgit@tlendack-t1.amdoffice.net>
References: <20170627145607.15908.26571.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170627145649.15908.62309.stgit@tlendack-t1.amdoffice.net>

Update the CPU features to include identifying and reporting on the
Secure Memory Encryption (SME) feature.  SME is identified by CPUID
0x8000001f, but requires BIOS support to enable it (set bit 23 of
MSR_K8_SYSCFG).  Only show the SME feature as available if reported by
CPUID and enabled by BIOS.

Reviewed-by: Borislav Petkov <bp at suse.de>
Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/include/asm/cpufeatures.h |    1 +
 arch/x86/include/asm/msr-index.h   |    2 ++
 arch/x86/kernel/cpu/amd.c          |   13 +++++++++++++
 arch/x86/kernel/cpu/scattered.c    |    1 +
 4 files changed, 17 insertions(+)

diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h
index 2701e5f..2b692df 100644
--- a/arch/x86/include/asm/cpufeatures.h
+++ b/arch/x86/include/asm/cpufeatures.h
@@ -196,6 +196,7 @@
 
 #define X86_FEATURE_HW_PSTATE	( 7*32+ 8) /* AMD HW-PState */
 #define X86_FEATURE_PROC_FEEDBACK ( 7*32+ 9) /* AMD ProcFeedbackInterface */
+#define X86_FEATURE_SME		( 7*32+10) /* AMD Secure Memory Encryption */
 
 #define X86_FEATURE_INTEL_PPIN	( 7*32+14) /* Intel Processor Inventory Number */
 #define X86_FEATURE_INTEL_PT	( 7*32+15) /* Intel Processor Trace */
diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h
index 18b1623..460ac01 100644
--- a/arch/x86/include/asm/msr-index.h
+++ b/arch/x86/include/asm/msr-index.h
@@ -352,6 +352,8 @@
 #define MSR_K8_TOP_MEM1			0xc001001a
 #define MSR_K8_TOP_MEM2			0xc001001d
 #define MSR_K8_SYSCFG			0xc0010010
+#define MSR_K8_SYSCFG_MEM_ENCRYPT_BIT	23
+#define MSR_K8_SYSCFG_MEM_ENCRYPT	BIT_ULL(MSR_K8_SYSCFG_MEM_ENCRYPT_BIT)
 #define MSR_K8_INT_PENDING_MSG		0xc0010055
 /* C1E active bits in int pending message */
 #define K8_INTP_C1E_ACTIVE_MASK		0x18000000
diff --git a/arch/x86/kernel/cpu/amd.c b/arch/x86/kernel/cpu/amd.c
index bb5abe8..c47ceee 100644
--- a/arch/x86/kernel/cpu/amd.c
+++ b/arch/x86/kernel/cpu/amd.c
@@ -611,6 +611,19 @@ static void early_init_amd(struct cpuinfo_x86 *c)
 	 */
 	if (cpu_has_amd_erratum(c, amd_erratum_400))
 		set_cpu_bug(c, X86_BUG_AMD_E400);
+
+	/*
+	 * BIOS support is required for SME. If BIOS has not enabled SME
+	 * then don't advertise the feature (set in scattered.c)
+	 */
+	if (cpu_has(c, X86_FEATURE_SME)) {
+		u64 msr;
+
+		/* Check if SME is enabled */
+		rdmsrl(MSR_K8_SYSCFG, msr);
+		if (!(msr & MSR_K8_SYSCFG_MEM_ENCRYPT))
+			clear_cpu_cap(c, X86_FEATURE_SME);
+	}
 }
 
 static void init_amd_k8(struct cpuinfo_x86 *c)
diff --git a/arch/x86/kernel/cpu/scattered.c b/arch/x86/kernel/cpu/scattered.c
index 23c2350..05459ad 100644
--- a/arch/x86/kernel/cpu/scattered.c
+++ b/arch/x86/kernel/cpu/scattered.c
@@ -31,6 +31,7 @@ struct cpuid_bit {
 	{ X86_FEATURE_HW_PSTATE,	CPUID_EDX,  7, 0x80000007, 0 },
 	{ X86_FEATURE_CPB,		CPUID_EDX,  9, 0x80000007, 0 },
 	{ X86_FEATURE_PROC_FEEDBACK,    CPUID_EDX, 11, 0x80000007, 0 },
+	{ X86_FEATURE_SME,		CPUID_EAX,  0, 0x8000001f, 0 },
 	{ 0, 0, 0, 0, 0 }
 };
 



From thomas.lendacky at amd.com  Tue Jun 27 07:57:00 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Tue, 27 Jun 2017 09:57:00 -0500
Subject: [PATCH v8 05/38] x86/CPU/AMD: Handle SME reduction in physical
 address size
In-Reply-To: <20170627145607.15908.26571.stgit@tlendack-t1.amdoffice.net>
References: <20170627145607.15908.26571.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170627145700.15908.24923.stgit@tlendack-t1.amdoffice.net>

When System Memory Encryption (SME) is enabled, the physical address
space is reduced. Adjust the x86_phys_bits value to reflect this
reduction.

Reviewed-by: Borislav Petkov <bp at suse.de>
Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/kernel/cpu/amd.c |   10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/arch/x86/kernel/cpu/amd.c b/arch/x86/kernel/cpu/amd.c
index c47ceee..5bdcbd4 100644
--- a/arch/x86/kernel/cpu/amd.c
+++ b/arch/x86/kernel/cpu/amd.c
@@ -613,15 +613,19 @@ static void early_init_amd(struct cpuinfo_x86 *c)
 		set_cpu_bug(c, X86_BUG_AMD_E400);
 
 	/*
-	 * BIOS support is required for SME. If BIOS has not enabled SME
-	 * then don't advertise the feature (set in scattered.c)
+	 * BIOS support is required for SME. If BIOS has enabld SME then
+	 * adjust x86_phys_bits by the SME physical address space reduction
+	 * value. If BIOS has not enabled SME then don't advertise the
+	 * feature (set in scattered.c).
 	 */
 	if (cpu_has(c, X86_FEATURE_SME)) {
 		u64 msr;
 
 		/* Check if SME is enabled */
 		rdmsrl(MSR_K8_SYSCFG, msr);
-		if (!(msr & MSR_K8_SYSCFG_MEM_ENCRYPT))
+		if (msr & MSR_K8_SYSCFG_MEM_ENCRYPT)
+			c->x86_phys_bits -= (cpuid_ebx(0x8000001f) >> 6) & 0x3f;
+		else
 			clear_cpu_cap(c, X86_FEATURE_SME);
 	}
 }



From thomas.lendacky at amd.com  Tue Jun 27 07:57:13 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Tue, 27 Jun 2017 09:57:13 -0500
Subject: [PATCH v8 06/38] x86/mm: Add Secure Memory Encryption (SME) support
In-Reply-To: <20170627145607.15908.26571.stgit@tlendack-t1.amdoffice.net>
References: <20170627145607.15908.26571.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170627145713.15908.42762.stgit@tlendack-t1.amdoffice.net>

Add support for Secure Memory Encryption (SME). This initial support
provides a Kconfig entry to build the SME support into the kernel and
defines the memory encryption mask that will be used in subsequent
patches to mark pages as encrypted.

Reviewed-by: Borislav Petkov <bp at suse.de>
Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/Kconfig                   |   25 +++++++++++++++++++++++++
 arch/x86/include/asm/mem_encrypt.h |   30 ++++++++++++++++++++++++++++++
 arch/x86/mm/Makefile               |    1 +
 arch/x86/mm/mem_encrypt.c          |   21 +++++++++++++++++++++
 include/linux/mem_encrypt.h        |   35 +++++++++++++++++++++++++++++++++++
 5 files changed, 112 insertions(+)
 create mode 100644 arch/x86/include/asm/mem_encrypt.h
 create mode 100644 arch/x86/mm/mem_encrypt.c
 create mode 100644 include/linux/mem_encrypt.h

diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index 72028a1..3a59e9c 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -1409,6 +1409,31 @@ config X86_DIRECT_GBPAGES
 	  supports them), so don't confuse the user by printing
 	  that we have them enabled.
 
+config ARCH_HAS_MEM_ENCRYPT
+	def_bool y
+
+config AMD_MEM_ENCRYPT
+	bool "AMD Secure Memory Encryption (SME) support"
+	depends on X86_64 && CPU_SUP_AMD
+	---help---
+	  Say yes to enable support for the encryption of system memory.
+	  This requires an AMD processor that supports Secure Memory
+	  Encryption (SME).
+
+config AMD_MEM_ENCRYPT_ACTIVE_BY_DEFAULT
+	bool "Activate AMD Secure Memory Encryption (SME) by default"
+	default y
+	depends on AMD_MEM_ENCRYPT
+	---help---
+	  Say yes to have system memory encrypted by default if running on
+	  an AMD processor that supports Secure Memory Encryption (SME).
+
+	  If set to Y, then the encryption of system memory can be
+	  deactivated with the mem_encrypt=off command line option.
+
+	  If set to N, then the encryption of system memory can be
+	  activated with the mem_encrypt=on command line option.
+
 # Common NUMA Features
 config NUMA
 	bool "Numa Memory Allocation and Scheduler Support"
diff --git a/arch/x86/include/asm/mem_encrypt.h b/arch/x86/include/asm/mem_encrypt.h
new file mode 100644
index 0000000..a105796
--- /dev/null
+++ b/arch/x86/include/asm/mem_encrypt.h
@@ -0,0 +1,30 @@
+/*
+ * AMD Memory Encryption Support
+ *
+ * Copyright (C) 2016 Advanced Micro Devices, Inc.
+ *
+ * Author: Tom Lendacky <thomas.lendacky at amd.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#ifndef __X86_MEM_ENCRYPT_H__
+#define __X86_MEM_ENCRYPT_H__
+
+#ifndef __ASSEMBLY__
+
+#ifdef CONFIG_AMD_MEM_ENCRYPT
+
+extern unsigned long sme_me_mask;
+
+#else	/* !CONFIG_AMD_MEM_ENCRYPT */
+
+#define sme_me_mask	0UL
+
+#endif	/* CONFIG_AMD_MEM_ENCRYPT */
+
+#endif	/* __ASSEMBLY__ */
+
+#endif	/* __X86_MEM_ENCRYPT_H__ */
diff --git a/arch/x86/mm/Makefile b/arch/x86/mm/Makefile
index 0fbdcb6..a94a7b6 100644
--- a/arch/x86/mm/Makefile
+++ b/arch/x86/mm/Makefile
@@ -39,3 +39,4 @@ obj-$(CONFIG_X86_INTEL_MPX)	+= mpx.o
 obj-$(CONFIG_X86_INTEL_MEMORY_PROTECTION_KEYS) += pkeys.o
 obj-$(CONFIG_RANDOMIZE_MEMORY) += kaslr.o
 
+obj-$(CONFIG_AMD_MEM_ENCRYPT)	+= mem_encrypt.o
diff --git a/arch/x86/mm/mem_encrypt.c b/arch/x86/mm/mem_encrypt.c
new file mode 100644
index 0000000..b99d469
--- /dev/null
+++ b/arch/x86/mm/mem_encrypt.c
@@ -0,0 +1,21 @@
+/*
+ * AMD Memory Encryption Support
+ *
+ * Copyright (C) 2016 Advanced Micro Devices, Inc.
+ *
+ * Author: Tom Lendacky <thomas.lendacky at amd.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include <linux/linkage.h>
+
+/*
+ * Since SME related variables are set early in the boot process they must
+ * reside in the .data section so as not to be zeroed out when the .bss
+ * section is later cleared.
+ */
+unsigned long sme_me_mask __section(.data) = 0;
+EXPORT_SYMBOL_GPL(sme_me_mask);
diff --git a/include/linux/mem_encrypt.h b/include/linux/mem_encrypt.h
new file mode 100644
index 0000000..59769f7
--- /dev/null
+++ b/include/linux/mem_encrypt.h
@@ -0,0 +1,35 @@
+/*
+ * AMD Memory Encryption Support
+ *
+ * Copyright (C) 2016 Advanced Micro Devices, Inc.
+ *
+ * Author: Tom Lendacky <thomas.lendacky at amd.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#ifndef __MEM_ENCRYPT_H__
+#define __MEM_ENCRYPT_H__
+
+#ifndef __ASSEMBLY__
+
+#ifdef CONFIG_ARCH_HAS_MEM_ENCRYPT
+
+#include <asm/mem_encrypt.h>
+
+#else	/* !CONFIG_ARCH_HAS_MEM_ENCRYPT */
+
+#define sme_me_mask	0UL
+
+#endif	/* CONFIG_ARCH_HAS_MEM_ENCRYPT */
+
+static inline bool sme_active(void)
+{
+	return !!sme_me_mask;
+}
+
+#endif	/* __ASSEMBLY__ */
+
+#endif	/* __MEM_ENCRYPT_H__ */



From thomas.lendacky at amd.com  Tue Jun 27 07:57:25 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Tue, 27 Jun 2017 09:57:25 -0500
Subject: [PATCH v8 07/38] x86/mm: Remove phys_to_virt() usage in ioremap()
In-Reply-To: <20170627145607.15908.26571.stgit@tlendack-t1.amdoffice.net>
References: <20170627145607.15908.26571.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170627145725.15908.24612.stgit@tlendack-t1.amdoffice.net>

Currently there is a check if the address being mapped is in the ISA
range (is_ISA_range()), and if it is, then phys_to_virt() is used to
perform the mapping. When SME is active, the default is to add pagetable
mappings with the encryption bit set unless specifically overridden. The
resulting pagetable mapping from phys_to_virt() will result in a mapping
that has the encryption bit set. With SME, the use of ioremap() is
intended to generate pagetable mappings that do not have the encryption
bit set through the use of the PAGE_KERNEL_IO protection value.

Rather than special case the SME scenario, remove the ISA range check and
usage of phys_to_virt() and have ISA range mappings continue through the
remaining ioremap() path.

Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/mm/ioremap.c |    7 +------
 1 file changed, 1 insertion(+), 6 deletions(-)

diff --git a/arch/x86/mm/ioremap.c b/arch/x86/mm/ioremap.c
index 4c1b5fd..bfc3e2d 100644
--- a/arch/x86/mm/ioremap.c
+++ b/arch/x86/mm/ioremap.c
@@ -13,6 +13,7 @@
 #include <linux/slab.h>
 #include <linux/vmalloc.h>
 #include <linux/mmiotrace.h>
+#include <linux/mem_encrypt.h>
 
 #include <asm/set_memory.h>
 #include <asm/e820/api.h>
@@ -106,12 +107,6 @@ static void __iomem *__ioremap_caller(resource_size_t phys_addr,
 	}
 
 	/*
-	 * Don't remap the low PCI/ISA area, it's always mapped..
-	 */
-	if (is_ISA_range(phys_addr, last_addr))
-		return (__force void __iomem *)phys_to_virt(phys_addr);
-
-	/*
 	 * Don't allow anybody to remap normal RAM that we're using..
 	 */
 	pfn      = phys_addr >> PAGE_SHIFT;



From thomas.lendacky at amd.com  Tue Jun 27 07:57:39 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Tue, 27 Jun 2017 09:57:39 -0500
Subject: [PATCH v8 08/38] x86/mm: Add support to enable SME in early boot
 processing
In-Reply-To: <20170627145607.15908.26571.stgit@tlendack-t1.amdoffice.net>
References: <20170627145607.15908.26571.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170627145738.15908.16582.stgit@tlendack-t1.amdoffice.net>

Add support to the early boot code to use Secure Memory Encryption (SME).
Since the kernel has been loaded into memory in a decrypted state, encrypt
the kernel in place and update the early pagetables with the memory
encryption mask so that new pagetable entries will use memory encryption.

The routines to set the encryption mask and perform the encryption are
stub routines for now with functionality to be added in a later patch.

Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/include/asm/mem_encrypt.h |    8 +++++
 arch/x86/kernel/head64.c           |   53 +++++++++++++++++++++++++++++-------
 arch/x86/kernel/head_64.S          |   20 ++++++++++++--
 arch/x86/mm/mem_encrypt.c          |    9 ++++++
 include/linux/mem_encrypt.h        |    5 +++
 5 files changed, 82 insertions(+), 13 deletions(-)

diff --git a/arch/x86/include/asm/mem_encrypt.h b/arch/x86/include/asm/mem_encrypt.h
index a105796..475e34f 100644
--- a/arch/x86/include/asm/mem_encrypt.h
+++ b/arch/x86/include/asm/mem_encrypt.h
@@ -15,14 +15,22 @@
 
 #ifndef __ASSEMBLY__
 
+#include <linux/init.h>
+
 #ifdef CONFIG_AMD_MEM_ENCRYPT
 
 extern unsigned long sme_me_mask;
 
+void __init sme_encrypt_kernel(void);
+void __init sme_enable(void);
+
 #else	/* !CONFIG_AMD_MEM_ENCRYPT */
 
 #define sme_me_mask	0UL
 
+static inline void __init sme_encrypt_kernel(void) { }
+static inline void __init sme_enable(void) { }
+
 #endif	/* CONFIG_AMD_MEM_ENCRYPT */
 
 #endif	/* __ASSEMBLY__ */
diff --git a/arch/x86/kernel/head64.c b/arch/x86/kernel/head64.c
index 46c3c73..1f0ddcc 100644
--- a/arch/x86/kernel/head64.c
+++ b/arch/x86/kernel/head64.c
@@ -14,6 +14,7 @@
 #include <linux/start_kernel.h>
 #include <linux/io.h>
 #include <linux/memblock.h>
+#include <linux/mem_encrypt.h>
 
 #include <asm/processor.h>
 #include <asm/proto.h>
@@ -45,9 +46,10 @@ static void __head *fixup_pointer(void *ptr, unsigned long physaddr)
 	return ptr - (void *)_text + (void *)physaddr;
 }
 
-void __head __startup_64(unsigned long physaddr)
+unsigned long __head __startup_64(unsigned long physaddr)
 {
 	unsigned long load_delta, *p;
+	unsigned long pgtable_flags;
 	pgdval_t *pgd;
 	p4dval_t *p4d;
 	pudval_t *pud;
@@ -68,6 +70,12 @@ void __head __startup_64(unsigned long physaddr)
 	if (load_delta & ~PMD_PAGE_MASK)
 		for (;;);
 
+	/* Activate Secure Memory Encryption (SME) if supported and enabled */
+	sme_enable();
+
+	/* Include the SME encryption mask in the fixup value */
+	load_delta += sme_get_me_mask();
+
 	/* Fixup the physical addresses in the page table */
 
 	pgd = fixup_pointer(&early_top_pgt, physaddr);
@@ -94,28 +102,30 @@ void __head __startup_64(unsigned long physaddr)
 
 	pud = fixup_pointer(early_dynamic_pgts[next_early_pgt++], physaddr);
 	pmd = fixup_pointer(early_dynamic_pgts[next_early_pgt++], physaddr);
+	pgtable_flags = _KERNPG_TABLE + sme_get_me_mask();
 
 	if (IS_ENABLED(CONFIG_X86_5LEVEL)) {
 		p4d = fixup_pointer(early_dynamic_pgts[next_early_pgt++], physaddr);
 
 		i = (physaddr >> PGDIR_SHIFT) % PTRS_PER_PGD;
-		pgd[i + 0] = (pgdval_t)p4d + _KERNPG_TABLE;
-		pgd[i + 1] = (pgdval_t)p4d + _KERNPG_TABLE;
+		pgd[i + 0] = (pgdval_t)p4d + pgtable_flags;
+		pgd[i + 1] = (pgdval_t)p4d + pgtable_flags;
 
 		i = (physaddr >> P4D_SHIFT) % PTRS_PER_P4D;
-		p4d[i + 0] = (pgdval_t)pud + _KERNPG_TABLE;
-		p4d[i + 1] = (pgdval_t)pud + _KERNPG_TABLE;
+		p4d[i + 0] = (pgdval_t)pud + pgtable_flags;
+		p4d[i + 1] = (pgdval_t)pud + pgtable_flags;
 	} else {
 		i = (physaddr >> PGDIR_SHIFT) % PTRS_PER_PGD;
-		pgd[i + 0] = (pgdval_t)pud + _KERNPG_TABLE;
-		pgd[i + 1] = (pgdval_t)pud + _KERNPG_TABLE;
+		pgd[i + 0] = (pgdval_t)pud + pgtable_flags;
+		pgd[i + 1] = (pgdval_t)pud + pgtable_flags;
 	}
 
 	i = (physaddr >> PUD_SHIFT) % PTRS_PER_PUD;
-	pud[i + 0] = (pudval_t)pmd + _KERNPG_TABLE;
-	pud[i + 1] = (pudval_t)pmd + _KERNPG_TABLE;
+	pud[i + 0] = (pudval_t)pmd + pgtable_flags;
+	pud[i + 1] = (pudval_t)pmd + pgtable_flags;
 
 	pmd_entry = __PAGE_KERNEL_LARGE_EXEC & ~_PAGE_GLOBAL;
+	pmd_entry += sme_get_me_mask();
 	pmd_entry +=  physaddr;
 
 	for (i = 0; i < DIV_ROUND_UP(_end - _text, PMD_SIZE); i++) {
@@ -136,9 +146,30 @@ void __head __startup_64(unsigned long physaddr)
 			pmd[i] += load_delta;
 	}
 
-	/* Fixup phys_base */
+	/*
+	 * Fixup phys_base - remove the memory encryption mask to obtain
+	 * the true physical address.
+	 */
 	p = fixup_pointer(&phys_base, physaddr);
-	*p += load_delta;
+	*p += load_delta - sme_get_me_mask();
+
+	/* Encrypt the kernel (if SME is active) */
+	sme_encrypt_kernel();
+
+	/*
+	 * Return the SME encryption mask (if SME is active) to be used as a
+	 * modifier for the initial pgdir entry programmed into CR3.
+	 */
+	return sme_get_me_mask();
+}
+
+unsigned long __startup_secondary_64(void)
+{
+	/*
+	 * Return the SME encryption mask (if SME is active) to be used as a
+	 * modifier for the initial pgdir entry programmed into CR3.
+	 */
+	return sme_get_me_mask();
 }
 
 /* Wipe all early page tables except for the kernel symbol map */
diff --git a/arch/x86/kernel/head_64.S b/arch/x86/kernel/head_64.S
index 6225550..ec5d5e9 100644
--- a/arch/x86/kernel/head_64.S
+++ b/arch/x86/kernel/head_64.S
@@ -73,12 +73,19 @@ startup_64:
 	/* Sanitize CPU configuration */
 	call verify_cpu
 
+	/*
+	 * Perform pagetable fixups. Additionally, if SME is active, encrypt
+	 * the kernel and retrieve the modifier (SME encryption mask if SME
+	 * is active) to be added to the initial pgdir entry that will be
+	 * programmed into CR3.
+	 */
 	leaq	_text(%rip), %rdi
 	pushq	%rsi
 	call	__startup_64
 	popq	%rsi
 
-	movq	$(early_top_pgt - __START_KERNEL_map), %rax
+	/* Form the CR3 value being sure to include the CR3 modifier */
+	addq	$(early_top_pgt - __START_KERNEL_map), %rax
 	jmp 1f
 ENTRY(secondary_startup_64)
 	/*
@@ -98,7 +105,16 @@ ENTRY(secondary_startup_64)
 	/* Sanitize CPU configuration */
 	call verify_cpu
 
-	movq	$(init_top_pgt - __START_KERNEL_map), %rax
+	/*
+	 * Retrieve the modifier (SME encryption mask if SME is active) to be
+	 * added to the initial pgdir entry that will be programmed into CR3.
+	 */
+	pushq	%rsi
+	call	__startup_secondary_64
+	popq	%rsi
+
+	/* Form the CR3 value being sure to include the CR3 modifier */
+	addq	$(init_top_pgt - __START_KERNEL_map), %rax
 1:
 
 	/* Enable PAE mode, PGE and LA57 */
diff --git a/arch/x86/mm/mem_encrypt.c b/arch/x86/mm/mem_encrypt.c
index b99d469..3ac6f99 100644
--- a/arch/x86/mm/mem_encrypt.c
+++ b/arch/x86/mm/mem_encrypt.c
@@ -11,6 +11,7 @@
  */
 
 #include <linux/linkage.h>
+#include <linux/init.h>
 
 /*
  * Since SME related variables are set early in the boot process they must
@@ -19,3 +20,11 @@
  */
 unsigned long sme_me_mask __section(.data) = 0;
 EXPORT_SYMBOL_GPL(sme_me_mask);
+
+void __init sme_encrypt_kernel(void)
+{
+}
+
+void __init sme_enable(void)
+{
+}
diff --git a/include/linux/mem_encrypt.h b/include/linux/mem_encrypt.h
index 59769f7..570f4fc 100644
--- a/include/linux/mem_encrypt.h
+++ b/include/linux/mem_encrypt.h
@@ -30,6 +30,11 @@ static inline bool sme_active(void)
 	return !!sme_me_mask;
 }
 
+static inline unsigned long sme_get_me_mask(void)
+{
+	return sme_me_mask;
+}
+
 #endif	/* __ASSEMBLY__ */
 
 #endif	/* __MEM_ENCRYPT_H__ */



From thomas.lendacky at amd.com  Tue Jun 27 07:57:51 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Tue, 27 Jun 2017 09:57:51 -0500
Subject: [PATCH v8 09/38] x86/mm: Simplify p[g4um]d_page() macros
In-Reply-To: <20170627145607.15908.26571.stgit@tlendack-t1.amdoffice.net>
References: <20170627145607.15908.26571.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170627145751.15908.55617.stgit@tlendack-t1.amdoffice.net>

Create a pgd_pfn() macro similar to the p[4um]d_pfn() macros and then
use the p[g4um]d_pfn() macros in the p[g4um]d_page() macros instead of
duplicating the code.

Reviewed-by: Borislav Petkov <bp at suse.de>
Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/include/asm/pgtable.h |   16 +++++++++-------
 1 file changed, 9 insertions(+), 7 deletions(-)

diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h
index 77037b6..b64ea52 100644
--- a/arch/x86/include/asm/pgtable.h
+++ b/arch/x86/include/asm/pgtable.h
@@ -195,6 +195,11 @@ static inline unsigned long p4d_pfn(p4d_t p4d)
 	return (p4d_val(p4d) & p4d_pfn_mask(p4d)) >> PAGE_SHIFT;
 }
 
+static inline unsigned long pgd_pfn(pgd_t pgd)
+{
+	return (pgd_val(pgd) & PTE_PFN_MASK) >> PAGE_SHIFT;
+}
+
 static inline int p4d_large(p4d_t p4d)
 {
 	/* No 512 GiB pages yet */
@@ -704,8 +709,7 @@ static inline unsigned long pmd_page_vaddr(pmd_t pmd)
  * Currently stuck as a macro due to indirect forward reference to
  * linux/mmzone.h's __section_mem_map_addr() definition:
  */
-#define pmd_page(pmd)		\
-	pfn_to_page((pmd_val(pmd) & pmd_pfn_mask(pmd)) >> PAGE_SHIFT)
+#define pmd_page(pmd)	pfn_to_page(pmd_pfn(pmd))
 
 /*
  * the pmd page can be thought of an array like this: pmd_t[PTRS_PER_PMD]
@@ -773,8 +777,7 @@ static inline unsigned long pud_page_vaddr(pud_t pud)
  * Currently stuck as a macro due to indirect forward reference to
  * linux/mmzone.h's __section_mem_map_addr() definition:
  */
-#define pud_page(pud)		\
-	pfn_to_page((pud_val(pud) & pud_pfn_mask(pud)) >> PAGE_SHIFT)
+#define pud_page(pud)	pfn_to_page(pud_pfn(pud))
 
 /* Find an entry in the second-level page table.. */
 static inline pmd_t *pmd_offset(pud_t *pud, unsigned long address)
@@ -824,8 +827,7 @@ static inline unsigned long p4d_page_vaddr(p4d_t p4d)
  * Currently stuck as a macro due to indirect forward reference to
  * linux/mmzone.h's __section_mem_map_addr() definition:
  */
-#define p4d_page(p4d)		\
-	pfn_to_page((p4d_val(p4d) & p4d_pfn_mask(p4d)) >> PAGE_SHIFT)
+#define p4d_page(p4d)	pfn_to_page(p4d_pfn(p4d))
 
 /* Find an entry in the third-level page table.. */
 static inline pud_t *pud_offset(p4d_t *p4d, unsigned long address)
@@ -859,7 +861,7 @@ static inline unsigned long pgd_page_vaddr(pgd_t pgd)
  * Currently stuck as a macro due to indirect forward reference to
  * linux/mmzone.h's __section_mem_map_addr() definition:
  */
-#define pgd_page(pgd)		pfn_to_page(pgd_val(pgd) >> PAGE_SHIFT)
+#define pgd_page(pgd)	pfn_to_page(pgd_pfn(pgd))
 
 /* to find an entry in a page-table-directory. */
 static inline p4d_t *p4d_offset(pgd_t *pgd, unsigned long address)



From thomas.lendacky at amd.com  Tue Jun 27 07:58:01 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Tue, 27 Jun 2017 09:58:01 -0500
Subject: [PATCH v8 10/38] x86/mm: Provide general kernel support for memory
 encryption
In-Reply-To: <20170627145607.15908.26571.stgit@tlendack-t1.amdoffice.net>
References: <20170627145607.15908.26571.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170627145801.15908.37399.stgit@tlendack-t1.amdoffice.net>

Changes to the existing page table macros will allow the SME support to
be enabled in a simple fashion with minimal changes to files that use these
macros.  Since the memory encryption mask will now be part of the regular
pagetable macros, we introduce two new macros (_PAGE_TABLE_NOENC and
_KERNPG_TABLE_NOENC) to allow for early pagetable creation/initialization
without the encryption mask before SME becomes active.  Two new pgprot()
macros are defined to allow setting or clearing the page encryption mask.

The FIXMAP_PAGE_NOCACHE define is introduced for use with MMIO.  SME does
not support encryption for MMIO areas so this define removes the encryption
mask from the page attribute.

Two new macros are introduced (__sme_pa() / __sme_pa_nodebug()) to allow
creating a physical address with the encryption mask.  These are used when
working with the cr3 register so that the PGD can be encrypted. The current
__va() macro is updated so that the virtual address is generated based off
of the physical address without the encryption mask thus allowing the same
virtual address to be generated regardless of whether encryption is enabled
for that physical location or not.

Also, an early initialization function is added for SME.  If SME is active,
this function:
 - Updates the early_pmd_flags so that early page faults create mappings
   with the encryption mask.
 - Updates the __supported_pte_mask to include the encryption mask.
 - Updates the protection_map entries to include the encryption mask so
   that user-space allocations will automatically have the encryption mask
   applied.

Reviewed-by: Borislav Petkov <bp at suse.de>
Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/boot/compressed/pagetable.c |    7 +++++
 arch/x86/include/asm/fixmap.h        |    7 +++++
 arch/x86/include/asm/mem_encrypt.h   |   13 ++++++++++
 arch/x86/include/asm/page_types.h    |    3 ++
 arch/x86/include/asm/pgtable.h       |    9 +++++++
 arch/x86/include/asm/pgtable_types.h |   45 ++++++++++++++++++++++------------
 arch/x86/include/asm/processor.h     |    3 ++
 arch/x86/kernel/espfix_64.c          |    2 +-
 arch/x86/kernel/head64.c             |   11 +++++++-
 arch/x86/kernel/head_64.S            |   20 ++++++++-------
 arch/x86/mm/kasan_init_64.c          |    4 ++-
 arch/x86/mm/mem_encrypt.c            |   17 +++++++++++++
 arch/x86/mm/pageattr.c               |    3 ++
 include/asm-generic/pgtable.h        |   12 +++++++++
 include/linux/mem_encrypt.h          |    8 ++++++
 15 files changed, 131 insertions(+), 33 deletions(-)

diff --git a/arch/x86/boot/compressed/pagetable.c b/arch/x86/boot/compressed/pagetable.c
index 8e69df9..246bf29 100644
--- a/arch/x86/boot/compressed/pagetable.c
+++ b/arch/x86/boot/compressed/pagetable.c
@@ -15,6 +15,13 @@
 #define __pa(x)  ((unsigned long)(x))
 #define __va(x)  ((void *)((unsigned long)(x)))
 
+/*
+ * The pgtable.h and mm/ident_map.c includes make use of the SME related
+ * information which is not used in the compressed image support. Un-define
+ * the SME support to avoid any compile and link errors.
+ */
+#undef CONFIG_AMD_MEM_ENCRYPT
+
 #include "misc.h"
 
 /* These actually do the work of building the kernel identity maps. */
diff --git a/arch/x86/include/asm/fixmap.h b/arch/x86/include/asm/fixmap.h
index b65155c..d9ff226 100644
--- a/arch/x86/include/asm/fixmap.h
+++ b/arch/x86/include/asm/fixmap.h
@@ -157,6 +157,13 @@ static inline void __set_fixmap(enum fixed_addresses idx,
 }
 #endif
 
+/*
+ * FIXMAP_PAGE_NOCACHE is used for MMIO. Memory encryption is not
+ * supported for MMIO addresses, so make sure that the memory encryption
+ * mask is not part of the page attributes.
+ */
+#define FIXMAP_PAGE_NOCACHE PAGE_KERNEL_IO_NOCACHE
+
 #include <asm-generic/fixmap.h>
 
 #define __late_set_fixmap(idx, phys, flags) __set_fixmap(idx, phys, flags)
diff --git a/arch/x86/include/asm/mem_encrypt.h b/arch/x86/include/asm/mem_encrypt.h
index 475e34f..dbae7a5 100644
--- a/arch/x86/include/asm/mem_encrypt.h
+++ b/arch/x86/include/asm/mem_encrypt.h
@@ -21,6 +21,8 @@
 
 extern unsigned long sme_me_mask;
 
+void __init sme_early_init(void);
+
 void __init sme_encrypt_kernel(void);
 void __init sme_enable(void);
 
@@ -28,11 +30,22 @@
 
 #define sme_me_mask	0UL
 
+static inline void __init sme_early_init(void) { }
+
 static inline void __init sme_encrypt_kernel(void) { }
 static inline void __init sme_enable(void) { }
 
 #endif	/* CONFIG_AMD_MEM_ENCRYPT */
 
+/*
+ * The __sme_pa() and __sme_pa_nodebug() macros are meant for use when
+ * writing to or comparing values from the cr3 register.  Having the
+ * encryption mask set in cr3 enables the PGD entry to be encrypted and
+ * avoid special case handling of PGD allocations.
+ */
+#define __sme_pa(x)		(__pa(x) | sme_me_mask)
+#define __sme_pa_nodebug(x)	(__pa_nodebug(x) | sme_me_mask)
+
 #endif	/* __ASSEMBLY__ */
 
 #endif	/* __X86_MEM_ENCRYPT_H__ */
diff --git a/arch/x86/include/asm/page_types.h b/arch/x86/include/asm/page_types.h
index 7bd0099..b98ed9d 100644
--- a/arch/x86/include/asm/page_types.h
+++ b/arch/x86/include/asm/page_types.h
@@ -3,6 +3,7 @@
 
 #include <linux/const.h>
 #include <linux/types.h>
+#include <linux/mem_encrypt.h>
 
 /* PAGE_SHIFT determines the page size */
 #define PAGE_SHIFT		12
@@ -15,7 +16,7 @@
 #define PUD_PAGE_SIZE		(_AC(1, UL) << PUD_SHIFT)
 #define PUD_PAGE_MASK		(~(PUD_PAGE_SIZE-1))
 
-#define __PHYSICAL_MASK		((phys_addr_t)((1ULL << __PHYSICAL_MASK_SHIFT) - 1))
+#define __PHYSICAL_MASK		((phys_addr_t)(__sme_clr((1ULL << __PHYSICAL_MASK_SHIFT) - 1)))
 #define __VIRTUAL_MASK		((1UL << __VIRTUAL_MASK_SHIFT) - 1)
 
 /* Cast *PAGE_MASK to a signed type so that it is sign-extended if
diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h
index b64ea52..c6452cb 100644
--- a/arch/x86/include/asm/pgtable.h
+++ b/arch/x86/include/asm/pgtable.h
@@ -1,6 +1,7 @@
 #ifndef _ASM_X86_PGTABLE_H
 #define _ASM_X86_PGTABLE_H
 
+#include <linux/mem_encrypt.h>
 #include <asm/page.h>
 #include <asm/pgtable_types.h>
 
@@ -13,6 +14,12 @@
 		     cachemode2protval(_PAGE_CACHE_MODE_UC_MINUS)))	\
 	 : (prot))
 
+/*
+ * Macros to add or remove encryption attribute
+ */
+#define pgprot_encrypted(prot)	__pgprot(__sme_set(pgprot_val(prot)))
+#define pgprot_decrypted(prot)	__pgprot(__sme_clr(pgprot_val(prot)))
+
 #ifndef __ASSEMBLY__
 #include <asm/x86_init.h>
 
@@ -38,6 +45,8 @@
 
 extern struct mm_struct *pgd_page_get_mm(struct page *page);
 
+extern pmdval_t early_pmd_flags;
+
 #ifdef CONFIG_PARAVIRT
 #include <asm/paravirt.h>
 #else  /* !CONFIG_PARAVIRT */
diff --git a/arch/x86/include/asm/pgtable_types.h b/arch/x86/include/asm/pgtable_types.h
index bf9638e..de32ca3 100644
--- a/arch/x86/include/asm/pgtable_types.h
+++ b/arch/x86/include/asm/pgtable_types.h
@@ -2,6 +2,8 @@
 #define _ASM_X86_PGTABLE_DEFS_H
 
 #include <linux/const.h>
+#include <linux/mem_encrypt.h>
+
 #include <asm/page_types.h>
 
 #define FIRST_USER_ADDRESS	0UL
@@ -121,10 +123,10 @@
 
 #define _PAGE_PROTNONE	(_AT(pteval_t, 1) << _PAGE_BIT_PROTNONE)
 
-#define _PAGE_TABLE	(_PAGE_PRESENT | _PAGE_RW | _PAGE_USER |	\
-			 _PAGE_ACCESSED | _PAGE_DIRTY)
-#define _KERNPG_TABLE	(_PAGE_PRESENT | _PAGE_RW | _PAGE_ACCESSED |	\
-			 _PAGE_DIRTY)
+#define _PAGE_TABLE_NOENC	(_PAGE_PRESENT | _PAGE_RW | _PAGE_USER |\
+				 _PAGE_ACCESSED | _PAGE_DIRTY)
+#define _KERNPG_TABLE_NOENC	(_PAGE_PRESENT | _PAGE_RW |		\
+				 _PAGE_ACCESSED | _PAGE_DIRTY)
 
 /*
  * Set of bits not changed in pte_modify.  The pte's
@@ -191,18 +193,29 @@ enum page_cache_mode {
 #define __PAGE_KERNEL_IO		(__PAGE_KERNEL)
 #define __PAGE_KERNEL_IO_NOCACHE	(__PAGE_KERNEL_NOCACHE)
 
-#define PAGE_KERNEL			__pgprot(__PAGE_KERNEL)
-#define PAGE_KERNEL_RO			__pgprot(__PAGE_KERNEL_RO)
-#define PAGE_KERNEL_EXEC		__pgprot(__PAGE_KERNEL_EXEC)
-#define PAGE_KERNEL_RX			__pgprot(__PAGE_KERNEL_RX)
-#define PAGE_KERNEL_NOCACHE		__pgprot(__PAGE_KERNEL_NOCACHE)
-#define PAGE_KERNEL_LARGE		__pgprot(__PAGE_KERNEL_LARGE)
-#define PAGE_KERNEL_LARGE_EXEC		__pgprot(__PAGE_KERNEL_LARGE_EXEC)
-#define PAGE_KERNEL_VSYSCALL		__pgprot(__PAGE_KERNEL_VSYSCALL)
-#define PAGE_KERNEL_VVAR		__pgprot(__PAGE_KERNEL_VVAR)
-
-#define PAGE_KERNEL_IO			__pgprot(__PAGE_KERNEL_IO)
-#define PAGE_KERNEL_IO_NOCACHE		__pgprot(__PAGE_KERNEL_IO_NOCACHE)
+#ifndef __ASSEMBLY__
+
+#define _PAGE_ENC	(_AT(pteval_t, sme_me_mask))
+
+#define _PAGE_TABLE	(_PAGE_PRESENT | _PAGE_RW | _PAGE_USER |	\
+			 _PAGE_ACCESSED | _PAGE_DIRTY | _PAGE_ENC)
+#define _KERNPG_TABLE	(_PAGE_PRESENT | _PAGE_RW | _PAGE_ACCESSED |	\
+			 _PAGE_DIRTY | _PAGE_ENC)
+
+#define PAGE_KERNEL		__pgprot(__PAGE_KERNEL | _PAGE_ENC)
+#define PAGE_KERNEL_RO		__pgprot(__PAGE_KERNEL_RO | _PAGE_ENC)
+#define PAGE_KERNEL_EXEC	__pgprot(__PAGE_KERNEL_EXEC | _PAGE_ENC)
+#define PAGE_KERNEL_RX		__pgprot(__PAGE_KERNEL_RX | _PAGE_ENC)
+#define PAGE_KERNEL_NOCACHE	__pgprot(__PAGE_KERNEL_NOCACHE | _PAGE_ENC)
+#define PAGE_KERNEL_LARGE	__pgprot(__PAGE_KERNEL_LARGE | _PAGE_ENC)
+#define PAGE_KERNEL_LARGE_EXEC	__pgprot(__PAGE_KERNEL_LARGE_EXEC | _PAGE_ENC)
+#define PAGE_KERNEL_VSYSCALL	__pgprot(__PAGE_KERNEL_VSYSCALL | _PAGE_ENC)
+#define PAGE_KERNEL_VVAR	__pgprot(__PAGE_KERNEL_VVAR | _PAGE_ENC)
+
+#define PAGE_KERNEL_IO		__pgprot(__PAGE_KERNEL_IO)
+#define PAGE_KERNEL_IO_NOCACHE	__pgprot(__PAGE_KERNEL_IO_NOCACHE)
+
+#endif	/* __ASSEMBLY__ */
 
 /*         xwr */
 #define __P000	PAGE_NONE
diff --git a/arch/x86/include/asm/processor.h b/arch/x86/include/asm/processor.h
index f3b1b27..8010c97 100644
--- a/arch/x86/include/asm/processor.h
+++ b/arch/x86/include/asm/processor.h
@@ -29,6 +29,7 @@
 #include <linux/math64.h>
 #include <linux/err.h>
 #include <linux/irqflags.h>
+#include <linux/mem_encrypt.h>
 
 /*
  * We handle most unaligned accesses in hardware.  On the other hand
@@ -241,7 +242,7 @@ static inline unsigned long read_cr3_pa(void)
 
 static inline void load_cr3(pgd_t *pgdir)
 {
-	write_cr3(__pa(pgdir));
+	write_cr3(__sme_pa(pgdir));
 }
 
 #ifdef CONFIG_X86_32
diff --git a/arch/x86/kernel/espfix_64.c b/arch/x86/kernel/espfix_64.c
index 6b91e2e..9c4e7ba 100644
--- a/arch/x86/kernel/espfix_64.c
+++ b/arch/x86/kernel/espfix_64.c
@@ -195,7 +195,7 @@ void init_espfix_ap(int cpu)
 
 	pte_p = pte_offset_kernel(&pmd, addr);
 	stack_page = page_address(alloc_pages_node(node, GFP_KERNEL, 0));
-	pte = __pte(__pa(stack_page) | (__PAGE_KERNEL_RO & ptemask));
+	pte = __pte(__pa(stack_page) | ((__PAGE_KERNEL_RO | _PAGE_ENC) & ptemask));
 	for (n = 0; n < ESPFIX_PTE_CLONES; n++)
 		set_pte(&pte_p[n*PTE_STRIDE], pte);
 
diff --git a/arch/x86/kernel/head64.c b/arch/x86/kernel/head64.c
index 1f0ddcc..5cd0b72 100644
--- a/arch/x86/kernel/head64.c
+++ b/arch/x86/kernel/head64.c
@@ -102,7 +102,7 @@ unsigned long __head __startup_64(unsigned long physaddr)
 
 	pud = fixup_pointer(early_dynamic_pgts[next_early_pgt++], physaddr);
 	pmd = fixup_pointer(early_dynamic_pgts[next_early_pgt++], physaddr);
-	pgtable_flags = _KERNPG_TABLE + sme_get_me_mask();
+	pgtable_flags = _KERNPG_TABLE_NOENC + sme_get_me_mask();
 
 	if (IS_ENABLED(CONFIG_X86_5LEVEL)) {
 		p4d = fixup_pointer(early_dynamic_pgts[next_early_pgt++], physaddr);
@@ -177,7 +177,7 @@ static void __init reset_early_page_tables(void)
 {
 	memset(early_top_pgt, 0, sizeof(pgd_t)*(PTRS_PER_PGD-1));
 	next_early_pgt = 0;
-	write_cr3(__pa_nodebug(early_top_pgt));
+	write_cr3(__sme_pa_nodebug(early_top_pgt));
 }
 
 /* Create a new PMD entry */
@@ -310,6 +310,13 @@ asmlinkage __visible void __init x86_64_start_kernel(char * real_mode_data)
 
 	clear_page(init_top_pgt);
 
+	/*
+	 * SME support may update early_pmd_flags to include the memory
+	 * encryption mask, so it needs to be called before anything
+	 * that may generate a page fault.
+	 */
+	sme_early_init();
+
 	kasan_early_init();
 
 	for (i = 0; i < NUM_EXCEPTION_VECTORS; i++)
diff --git a/arch/x86/kernel/head_64.S b/arch/x86/kernel/head_64.S
index ec5d5e9..513cbb0 100644
--- a/arch/x86/kernel/head_64.S
+++ b/arch/x86/kernel/head_64.S
@@ -351,9 +351,9 @@ GLOBAL(name)
 NEXT_PAGE(early_top_pgt)
 	.fill	511,8,0
 #ifdef CONFIG_X86_5LEVEL
-	.quad	level4_kernel_pgt - __START_KERNEL_map + _PAGE_TABLE
+	.quad	level4_kernel_pgt - __START_KERNEL_map + _PAGE_TABLE_NOENC
 #else
-	.quad	level3_kernel_pgt - __START_KERNEL_map + _PAGE_TABLE
+	.quad	level3_kernel_pgt - __START_KERNEL_map + _PAGE_TABLE_NOENC
 #endif
 
 NEXT_PAGE(early_dynamic_pgts)
@@ -366,15 +366,15 @@ NEXT_PAGE(init_top_pgt)
 	.fill	512,8,0
 #else
 NEXT_PAGE(init_top_pgt)
-	.quad   level3_ident_pgt - __START_KERNEL_map + _KERNPG_TABLE
+	.quad   level3_ident_pgt - __START_KERNEL_map + _KERNPG_TABLE_NOENC
 	.org    init_top_pgt + PGD_PAGE_OFFSET*8, 0
-	.quad   level3_ident_pgt - __START_KERNEL_map + _KERNPG_TABLE
+	.quad   level3_ident_pgt - __START_KERNEL_map + _KERNPG_TABLE_NOENC
 	.org    init_top_pgt + PGD_START_KERNEL*8, 0
 	/* (2^48-(2*1024*1024*1024))/(2^39) = 511 */
-	.quad   level3_kernel_pgt - __START_KERNEL_map + _PAGE_TABLE
+	.quad   level3_kernel_pgt - __START_KERNEL_map + _PAGE_TABLE_NOENC
 
 NEXT_PAGE(level3_ident_pgt)
-	.quad	level2_ident_pgt - __START_KERNEL_map + _KERNPG_TABLE
+	.quad	level2_ident_pgt - __START_KERNEL_map + _KERNPG_TABLE_NOENC
 	.fill	511, 8, 0
 NEXT_PAGE(level2_ident_pgt)
 	/* Since I easily can, map the first 1G.
@@ -386,14 +386,14 @@ NEXT_PAGE(level2_ident_pgt)
 #ifdef CONFIG_X86_5LEVEL
 NEXT_PAGE(level4_kernel_pgt)
 	.fill	511,8,0
-	.quad	level3_kernel_pgt - __START_KERNEL_map + _PAGE_TABLE
+	.quad	level3_kernel_pgt - __START_KERNEL_map + _PAGE_TABLE_NOENC
 #endif
 
 NEXT_PAGE(level3_kernel_pgt)
 	.fill	L3_START_KERNEL,8,0
 	/* (2^48-(2*1024*1024*1024)-((2^39)*511))/(2^30) = 510 */
-	.quad	level2_kernel_pgt - __START_KERNEL_map + _KERNPG_TABLE
-	.quad	level2_fixmap_pgt - __START_KERNEL_map + _PAGE_TABLE
+	.quad	level2_kernel_pgt - __START_KERNEL_map + _KERNPG_TABLE_NOENC
+	.quad	level2_fixmap_pgt - __START_KERNEL_map + _PAGE_TABLE_NOENC
 
 NEXT_PAGE(level2_kernel_pgt)
 	/*
@@ -411,7 +411,7 @@ NEXT_PAGE(level2_kernel_pgt)
 
 NEXT_PAGE(level2_fixmap_pgt)
 	.fill	506,8,0
-	.quad	level1_fixmap_pgt - __START_KERNEL_map + _PAGE_TABLE
+	.quad	level1_fixmap_pgt - __START_KERNEL_map + _PAGE_TABLE_NOENC
 	/* 8MB reserved for vsyscalls + a 2MB hole = 4 + 1 entries */
 	.fill	5,8,0
 
diff --git a/arch/x86/mm/kasan_init_64.c b/arch/x86/mm/kasan_init_64.c
index 88215ac..d7cc830 100644
--- a/arch/x86/mm/kasan_init_64.c
+++ b/arch/x86/mm/kasan_init_64.c
@@ -92,7 +92,7 @@ static int kasan_die_handler(struct notifier_block *self,
 void __init kasan_early_init(void)
 {
 	int i;
-	pteval_t pte_val = __pa_nodebug(kasan_zero_page) | __PAGE_KERNEL;
+	pteval_t pte_val = __pa_nodebug(kasan_zero_page) | __PAGE_KERNEL | _PAGE_ENC;
 	pmdval_t pmd_val = __pa_nodebug(kasan_zero_pte) | _KERNPG_TABLE;
 	pudval_t pud_val = __pa_nodebug(kasan_zero_pmd) | _KERNPG_TABLE;
 	p4dval_t p4d_val = __pa_nodebug(kasan_zero_pud) | _KERNPG_TABLE;
@@ -158,7 +158,7 @@ void __init kasan_init(void)
 	 */
 	memset(kasan_zero_page, 0, PAGE_SIZE);
 	for (i = 0; i < PTRS_PER_PTE; i++) {
-		pte_t pte = __pte(__pa(kasan_zero_page) | __PAGE_KERNEL_RO);
+		pte_t pte = __pte(__pa(kasan_zero_page) | __PAGE_KERNEL_RO | _PAGE_ENC);
 		set_pte(&kasan_zero_pte[i], pte);
 	}
 	/* Flush TLBs again to be sure that write protection applied. */
diff --git a/arch/x86/mm/mem_encrypt.c b/arch/x86/mm/mem_encrypt.c
index 3ac6f99..f973d3d 100644
--- a/arch/x86/mm/mem_encrypt.c
+++ b/arch/x86/mm/mem_encrypt.c
@@ -12,6 +12,7 @@
 
 #include <linux/linkage.h>
 #include <linux/init.h>
+#include <linux/mm.h>
 
 /*
  * Since SME related variables are set early in the boot process they must
@@ -21,6 +22,22 @@
 unsigned long sme_me_mask __section(.data) = 0;
 EXPORT_SYMBOL_GPL(sme_me_mask);
 
+void __init sme_early_init(void)
+{
+	unsigned int i;
+
+	if (!sme_me_mask)
+		return;
+
+	early_pmd_flags = __sme_set(early_pmd_flags);
+
+	__supported_pte_mask = __sme_set(__supported_pte_mask);
+
+	/* Update the protection map with memory encryption mask */
+	for (i = 0; i < ARRAY_SIZE(protection_map); i++)
+		protection_map[i] = pgprot_encrypted(protection_map[i]);
+}
+
 void __init sme_encrypt_kernel(void)
 {
 }
diff --git a/arch/x86/mm/pageattr.c b/arch/x86/mm/pageattr.c
index c8520b2..e7d3866 100644
--- a/arch/x86/mm/pageattr.c
+++ b/arch/x86/mm/pageattr.c
@@ -2014,6 +2014,9 @@ int kernel_map_pages_in_pgd(pgd_t *pgd, u64 pfn, unsigned long address,
 	if (!(page_flags & _PAGE_RW))
 		cpa.mask_clr = __pgprot(_PAGE_RW);
 
+	if (!(page_flags & _PAGE_ENC))
+		cpa.mask_clr = pgprot_encrypted(cpa.mask_clr);
+
 	cpa.mask_set = __pgprot(_PAGE_PRESENT | page_flags);
 
 	retval = __change_page_attr_set_clr(&cpa, 0);
diff --git a/include/asm-generic/pgtable.h b/include/asm-generic/pgtable.h
index 7dfa767..4d7bb98 100644
--- a/include/asm-generic/pgtable.h
+++ b/include/asm-generic/pgtable.h
@@ -583,6 +583,18 @@ static inline void ptep_modify_prot_commit(struct mm_struct *mm,
 #endif /* CONFIG_MMU */
 
 /*
+ * No-op macros that just return the current protection value. Defined here
+ * because these macros can be used used even if CONFIG_MMU is not defined.
+ */
+#ifndef pgprot_encrypted
+#define pgprot_encrypted(prot)	(prot)
+#endif
+
+#ifndef pgprot_decrypted
+#define pgprot_decrypted(prot)	(prot)
+#endif
+
+/*
  * A facility to provide lazy MMU batching.  This allows PTE updates and
  * page invalidations to be delayed until a call to leave lazy MMU mode
  * is issued.  Some architectures may benefit from doing this, and it is
diff --git a/include/linux/mem_encrypt.h b/include/linux/mem_encrypt.h
index 570f4fc..1255f09 100644
--- a/include/linux/mem_encrypt.h
+++ b/include/linux/mem_encrypt.h
@@ -35,6 +35,14 @@ static inline unsigned long sme_get_me_mask(void)
 	return sme_me_mask;
 }
 
+/*
+ * The __sme_set() and __sme_clr() macros are useful for adding or removing
+ * the encryption mask from a value (e.g. when dealing with pagetable
+ * entries).
+ */
+#define __sme_set(x)		((unsigned long)(x) | sme_me_mask)
+#define __sme_clr(x)		((unsigned long)(x) & ~sme_me_mask)
+
 #endif	/* __ASSEMBLY__ */
 
 #endif	/* __MEM_ENCRYPT_H__ */



From thomas.lendacky at amd.com  Tue Jun 27 07:58:12 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Tue, 27 Jun 2017 09:58:12 -0500
Subject: [PATCH v8 11/38] x86/mm: Add SME support for read_cr3_pa()
In-Reply-To: <20170627145607.15908.26571.stgit@tlendack-t1.amdoffice.net>
References: <20170627145607.15908.26571.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170627145812.15908.39913.stgit@tlendack-t1.amdoffice.net>

The cr3 register entry can contain the SME encryption mask that indicates
the PGD is encrypted.  The encryption mask should not be used when
creating a virtual address from the cr3 register, so remove the SME
encryption mask in the read_cr3_pa() function.

During early boot SME will need to use a native version of read_cr3_pa(),
so create native_read_cr3_pa().

Reviewed-by: Borislav Petkov <bp at suse.de>
Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/include/asm/processor-flags.h |    5 +++--
 arch/x86/include/asm/processor.h       |    5 +++++
 2 files changed, 8 insertions(+), 2 deletions(-)

diff --git a/arch/x86/include/asm/processor-flags.h b/arch/x86/include/asm/processor-flags.h
index 79aa2f9..f5d3e50 100644
--- a/arch/x86/include/asm/processor-flags.h
+++ b/arch/x86/include/asm/processor-flags.h
@@ -2,6 +2,7 @@
 #define _ASM_X86_PROCESSOR_FLAGS_H
 
 #include <uapi/asm/processor-flags.h>
+#include <linux/mem_encrypt.h>
 
 #ifdef CONFIG_VM86
 #define X86_VM_MASK	X86_EFLAGS_VM
@@ -32,8 +33,8 @@
  * CR3_ADDR_MASK is the mask used by read_cr3_pa().
  */
 #ifdef CONFIG_X86_64
-/* Mask off the address space ID bits. */
-#define CR3_ADDR_MASK 0x7FFFFFFFFFFFF000ull
+/* Mask off the address space ID and SME encryption bits. */
+#define CR3_ADDR_MASK __sme_clr(0x7FFFFFFFFFFFF000ull)
 #define CR3_PCID_MASK 0xFFFull
 #else
 /*
diff --git a/arch/x86/include/asm/processor.h b/arch/x86/include/asm/processor.h
index 8010c97..ab878bd 100644
--- a/arch/x86/include/asm/processor.h
+++ b/arch/x86/include/asm/processor.h
@@ -240,6 +240,11 @@ static inline unsigned long read_cr3_pa(void)
 	return __read_cr3() & CR3_ADDR_MASK;
 }
 
+static inline unsigned long native_read_cr3_pa(void)
+{
+	return __native_read_cr3() & CR3_ADDR_MASK;
+}
+
 static inline void load_cr3(pgd_t *pgdir)
 {
 	write_cr3(__sme_pa(pgdir));



From thomas.lendacky at amd.com  Tue Jun 27 07:58:23 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Tue, 27 Jun 2017 09:58:23 -0500
Subject: [PATCH v8 12/38] x86/mm: Extend early_memremap() support with
 additional attrs
In-Reply-To: <20170627145607.15908.26571.stgit@tlendack-t1.amdoffice.net>
References: <20170627145607.15908.26571.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170627145823.15908.76242.stgit@tlendack-t1.amdoffice.net>

Add early_memremap() support to be able to specify encrypted and
decrypted mappings with and without write-protection. The use of
write-protection is necessary when encrypting data "in place". The
write-protect attribute is considered cacheable for loads, but not
stores. This implies that the hardware will never give the core a
dirty line with this memtype.

Reviewed-by: Borislav Petkov <bp at suse.de>
Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/Kconfig                     |    4 +++
 arch/x86/include/asm/fixmap.h        |   13 ++++++++++
 arch/x86/include/asm/pgtable_types.h |    8 ++++++
 arch/x86/mm/ioremap.c                |   44 ++++++++++++++++++++++++++++++++++
 include/asm-generic/early_ioremap.h  |    2 ++
 mm/early_ioremap.c                   |   10 ++++++++
 6 files changed, 81 insertions(+)

diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index 3a59e9c..a04081ce 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -1434,6 +1434,10 @@ config AMD_MEM_ENCRYPT_ACTIVE_BY_DEFAULT
 	  If set to N, then the encryption of system memory can be
 	  activated with the mem_encrypt=on command line option.
 
+config ARCH_USE_MEMREMAP_PROT
+	def_bool y
+	depends on AMD_MEM_ENCRYPT
+
 # Common NUMA Features
 config NUMA
 	bool "Numa Memory Allocation and Scheduler Support"
diff --git a/arch/x86/include/asm/fixmap.h b/arch/x86/include/asm/fixmap.h
index d9ff226..dcd9fb5 100644
--- a/arch/x86/include/asm/fixmap.h
+++ b/arch/x86/include/asm/fixmap.h
@@ -164,6 +164,19 @@ static inline void __set_fixmap(enum fixed_addresses idx,
  */
 #define FIXMAP_PAGE_NOCACHE PAGE_KERNEL_IO_NOCACHE
 
+/*
+ * Early memremap routines used for in-place encryption. The mappings created
+ * by these routines are intended to be used as temporary mappings.
+ */
+void __init *early_memremap_encrypted(resource_size_t phys_addr,
+				      unsigned long size);
+void __init *early_memremap_encrypted_wp(resource_size_t phys_addr,
+					 unsigned long size);
+void __init *early_memremap_decrypted(resource_size_t phys_addr,
+				      unsigned long size);
+void __init *early_memremap_decrypted_wp(resource_size_t phys_addr,
+					 unsigned long size);
+
 #include <asm-generic/fixmap.h>
 
 #define __late_set_fixmap(idx, phys, flags) __set_fixmap(idx, phys, flags)
diff --git a/arch/x86/include/asm/pgtable_types.h b/arch/x86/include/asm/pgtable_types.h
index de32ca3..32095af 100644
--- a/arch/x86/include/asm/pgtable_types.h
+++ b/arch/x86/include/asm/pgtable_types.h
@@ -161,6 +161,7 @@ enum page_cache_mode {
 
 #define _PAGE_CACHE_MASK	(_PAGE_PAT | _PAGE_PCD | _PAGE_PWT)
 #define _PAGE_NOCACHE		(cachemode2protval(_PAGE_CACHE_MODE_UC))
+#define _PAGE_CACHE_WP		(cachemode2protval(_PAGE_CACHE_MODE_WP))
 
 #define PAGE_NONE	__pgprot(_PAGE_PROTNONE | _PAGE_ACCESSED)
 #define PAGE_SHARED	__pgprot(_PAGE_PRESENT | _PAGE_RW | _PAGE_USER | \
@@ -189,6 +190,7 @@ enum page_cache_mode {
 #define __PAGE_KERNEL_VVAR		(__PAGE_KERNEL_RO | _PAGE_USER)
 #define __PAGE_KERNEL_LARGE		(__PAGE_KERNEL | _PAGE_PSE)
 #define __PAGE_KERNEL_LARGE_EXEC	(__PAGE_KERNEL_EXEC | _PAGE_PSE)
+#define __PAGE_KERNEL_WP		(__PAGE_KERNEL | _PAGE_CACHE_WP)
 
 #define __PAGE_KERNEL_IO		(__PAGE_KERNEL)
 #define __PAGE_KERNEL_IO_NOCACHE	(__PAGE_KERNEL_NOCACHE)
@@ -202,6 +204,12 @@ enum page_cache_mode {
 #define _KERNPG_TABLE	(_PAGE_PRESENT | _PAGE_RW | _PAGE_ACCESSED |	\
 			 _PAGE_DIRTY | _PAGE_ENC)
 
+#define __PAGE_KERNEL_ENC	(__PAGE_KERNEL | _PAGE_ENC)
+#define __PAGE_KERNEL_ENC_WP	(__PAGE_KERNEL_WP | _PAGE_ENC)
+
+#define __PAGE_KERNEL_NOENC	(__PAGE_KERNEL)
+#define __PAGE_KERNEL_NOENC_WP	(__PAGE_KERNEL_WP)
+
 #define PAGE_KERNEL		__pgprot(__PAGE_KERNEL | _PAGE_ENC)
 #define PAGE_KERNEL_RO		__pgprot(__PAGE_KERNEL_RO | _PAGE_ENC)
 #define PAGE_KERNEL_EXEC	__pgprot(__PAGE_KERNEL_EXEC | _PAGE_ENC)
diff --git a/arch/x86/mm/ioremap.c b/arch/x86/mm/ioremap.c
index bfc3e2d..26db273 100644
--- a/arch/x86/mm/ioremap.c
+++ b/arch/x86/mm/ioremap.c
@@ -414,6 +414,50 @@ void unxlate_dev_mem_ptr(phys_addr_t phys, void *addr)
 	iounmap((void __iomem *)((unsigned long)addr & PAGE_MASK));
 }
 
+#ifdef CONFIG_ARCH_USE_MEMREMAP_PROT
+/* Remap memory with encryption */
+void __init *early_memremap_encrypted(resource_size_t phys_addr,
+				      unsigned long size)
+{
+	return early_memremap_prot(phys_addr, size, __PAGE_KERNEL_ENC);
+}
+
+/*
+ * Remap memory with encryption and write-protected - cannot be called
+ * before pat_init() is called
+ */
+void __init *early_memremap_encrypted_wp(resource_size_t phys_addr,
+					 unsigned long size)
+{
+	/* Be sure the write-protect PAT entry is set for write-protect */
+	if (__pte2cachemode_tbl[_PAGE_CACHE_MODE_WP] != _PAGE_CACHE_MODE_WP)
+		return NULL;
+
+	return early_memremap_prot(phys_addr, size, __PAGE_KERNEL_ENC_WP);
+}
+
+/* Remap memory without encryption */
+void __init *early_memremap_decrypted(resource_size_t phys_addr,
+				      unsigned long size)
+{
+	return early_memremap_prot(phys_addr, size, __PAGE_KERNEL_NOENC);
+}
+
+/*
+ * Remap memory without encryption and write-protected - cannot be called
+ * before pat_init() is called
+ */
+void __init *early_memremap_decrypted_wp(resource_size_t phys_addr,
+					 unsigned long size)
+{
+	/* Be sure the write-protect PAT entry is set for write-protect */
+	if (__pte2cachemode_tbl[_PAGE_CACHE_MODE_WP] != _PAGE_CACHE_MODE_WP)
+		return NULL;
+
+	return early_memremap_prot(phys_addr, size, __PAGE_KERNEL_NOENC_WP);
+}
+#endif	/* CONFIG_ARCH_USE_MEMREMAP_PROT */
+
 static pte_t bm_pte[PAGE_SIZE/sizeof(pte_t)] __page_aligned_bss;
 
 static inline pmd_t * __init early_ioremap_pmd(unsigned long addr)
diff --git a/include/asm-generic/early_ioremap.h b/include/asm-generic/early_ioremap.h
index 734ad4d..2edef8d 100644
--- a/include/asm-generic/early_ioremap.h
+++ b/include/asm-generic/early_ioremap.h
@@ -13,6 +13,8 @@ extern void *early_memremap(resource_size_t phys_addr,
 			    unsigned long size);
 extern void *early_memremap_ro(resource_size_t phys_addr,
 			       unsigned long size);
+extern void *early_memremap_prot(resource_size_t phys_addr,
+				 unsigned long size, unsigned long prot_val);
 extern void early_iounmap(void __iomem *addr, unsigned long size);
 extern void early_memunmap(void *addr, unsigned long size);
 
diff --git a/mm/early_ioremap.c b/mm/early_ioremap.c
index 6d5717b..d7d30da 100644
--- a/mm/early_ioremap.c
+++ b/mm/early_ioremap.c
@@ -226,6 +226,16 @@ void __init early_iounmap(void __iomem *addr, unsigned long size)
 }
 #endif
 
+#ifdef CONFIG_ARCH_USE_MEMREMAP_PROT
+void __init *
+early_memremap_prot(resource_size_t phys_addr, unsigned long size,
+		    unsigned long prot_val)
+{
+	return (__force void *)__early_ioremap(phys_addr, size,
+					       __pgprot(prot_val));
+}
+#endif
+
 #define MAX_MAP_CHUNK	(NR_FIX_BTMAPS << PAGE_SHIFT)
 
 void __init copy_from_early_mem(void *dest, phys_addr_t src, unsigned long size)



From thomas.lendacky at amd.com  Tue Jun 27 07:58:44 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Tue, 27 Jun 2017 09:58:44 -0500
Subject: [PATCH v8 14/38] x86/mm: Insure that boot memory areas are mapped
 properly
In-Reply-To: <20170627145607.15908.26571.stgit@tlendack-t1.amdoffice.net>
References: <20170627145607.15908.26571.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170627145844.15908.49837.stgit@tlendack-t1.amdoffice.net>

The boot data and command line data are present in memory in a decrypted
state and are copied early in the boot process.  The early page fault
support will map these areas as encrypted, so before attempting to copy
them, add decrypted mappings so the data is accessed properly when copied.

For the initrd, encrypt this data in place. Since the future mapping of
the initrd area will be mapped as encrypted the data will be accessed
properly.

Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/include/asm/mem_encrypt.h |    6 +++
 arch/x86/include/asm/pgtable.h     |    3 ++
 arch/x86/kernel/head64.c           |   30 +++++++++++++++--
 arch/x86/kernel/setup.c            |    9 +++++
 arch/x86/mm/kasan_init_64.c        |    2 +
 arch/x86/mm/mem_encrypt.c          |   63 ++++++++++++++++++++++++++++++++++++
 6 files changed, 108 insertions(+), 5 deletions(-)

diff --git a/arch/x86/include/asm/mem_encrypt.h b/arch/x86/include/asm/mem_encrypt.h
index 8baa35b..ab1fe77 100644
--- a/arch/x86/include/asm/mem_encrypt.h
+++ b/arch/x86/include/asm/mem_encrypt.h
@@ -26,6 +26,9 @@ void __init sme_early_encrypt(resource_size_t paddr,
 void __init sme_early_decrypt(resource_size_t paddr,
 			      unsigned long size);
 
+void __init sme_map_bootdata(char *real_mode_data);
+void __init sme_unmap_bootdata(char *real_mode_data);
+
 void __init sme_early_init(void);
 
 void __init sme_encrypt_kernel(void);
@@ -40,6 +43,9 @@ static inline void __init sme_early_encrypt(resource_size_t paddr,
 static inline void __init sme_early_decrypt(resource_size_t paddr,
 					    unsigned long size) { }
 
+static inline void __init sme_map_bootdata(char *real_mode_data) { }
+static inline void __init sme_unmap_bootdata(char *real_mode_data) { }
+
 static inline void __init sme_early_init(void) { }
 
 static inline void __init sme_encrypt_kernel(void) { }
diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h
index c6452cb..bbeae4a 100644
--- a/arch/x86/include/asm/pgtable.h
+++ b/arch/x86/include/asm/pgtable.h
@@ -23,6 +23,9 @@
 #ifndef __ASSEMBLY__
 #include <asm/x86_init.h>
 
+extern pgd_t early_top_pgt[PTRS_PER_PGD];
+int __init __early_make_pgtable(unsigned long address, pmdval_t pmd);
+
 void ptdump_walk_pgd_level(struct seq_file *m, pgd_t *pgd);
 void ptdump_walk_pgd_level_checkwx(void);
 
diff --git a/arch/x86/kernel/head64.c b/arch/x86/kernel/head64.c
index 5cd0b72..0cdb53b 100644
--- a/arch/x86/kernel/head64.c
+++ b/arch/x86/kernel/head64.c
@@ -34,7 +34,6 @@
 /*
  * Manage page tables very early on.
  */
-extern pgd_t early_top_pgt[PTRS_PER_PGD];
 extern pmd_t early_dynamic_pgts[EARLY_DYNAMIC_PAGE_TABLES][PTRS_PER_PMD];
 static unsigned int __initdata next_early_pgt;
 pmdval_t early_pmd_flags = __PAGE_KERNEL_LARGE & ~(_PAGE_GLOBAL | _PAGE_NX);
@@ -181,13 +180,13 @@ static void __init reset_early_page_tables(void)
 }
 
 /* Create a new PMD entry */
-int __init early_make_pgtable(unsigned long address)
+int __init __early_make_pgtable(unsigned long address, pmdval_t pmd)
 {
 	unsigned long physaddr = address - __PAGE_OFFSET;
 	pgdval_t pgd, *pgd_p;
 	p4dval_t p4d, *p4d_p;
 	pudval_t pud, *pud_p;
-	pmdval_t pmd, *pmd_p;
+	pmdval_t *pmd_p;
 
 	/* Invalid address or early pgt is done ?  */
 	if (physaddr >= MAXMEM || read_cr3_pa() != __pa_nodebug(early_top_pgt))
@@ -246,12 +245,21 @@ int __init early_make_pgtable(unsigned long address)
 		memset(pmd_p, 0, sizeof(*pmd_p) * PTRS_PER_PMD);
 		*pud_p = (pudval_t)pmd_p - __START_KERNEL_map + phys_base + _KERNPG_TABLE;
 	}
-	pmd = (physaddr & PMD_MASK) + early_pmd_flags;
 	pmd_p[pmd_index(address)] = pmd;
 
 	return 0;
 }
 
+int __init early_make_pgtable(unsigned long address)
+{
+	unsigned long physaddr = address - __PAGE_OFFSET;
+	pmdval_t pmd;
+
+	pmd = (physaddr & PMD_MASK) + early_pmd_flags;
+
+	return __early_make_pgtable(address, pmd);
+}
+
 /* Don't add a printk in there. printk relies on the PDA which is not initialized 
    yet. */
 static void __init clear_bss(void)
@@ -274,6 +282,12 @@ static void __init copy_bootdata(char *real_mode_data)
 	char * command_line;
 	unsigned long cmd_line_ptr;
 
+	/*
+	 * If SME is active, this will create decrypted mappings of the
+	 * boot data in advance of the copy operations.
+	 */
+	sme_map_bootdata(real_mode_data);
+
 	memcpy(&boot_params, real_mode_data, sizeof boot_params);
 	sanitize_boot_params(&boot_params);
 	cmd_line_ptr = get_cmd_line_ptr();
@@ -281,6 +295,14 @@ static void __init copy_bootdata(char *real_mode_data)
 		command_line = __va(cmd_line_ptr);
 		memcpy(boot_command_line, command_line, COMMAND_LINE_SIZE);
 	}
+
+	/*
+	 * The old boot data is no longer needed and won't be reserved,
+	 * freeing up that memory for use by the system. If SME is active,
+	 * we need to remove the mappings that were created so that the
+	 * memory doesn't remain mapped as decrypted.
+	 */
+	sme_unmap_bootdata(real_mode_data);
 }
 
 asmlinkage __visible void __init x86_64_start_kernel(char * real_mode_data)
diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
index 65622f0..31ae85e 100644
--- a/arch/x86/kernel/setup.c
+++ b/arch/x86/kernel/setup.c
@@ -69,6 +69,7 @@
 #include <linux/crash_dump.h>
 #include <linux/tboot.h>
 #include <linux/jiffies.h>
+#include <linux/mem_encrypt.h>
 
 #include <linux/usb/xhci-dbgp.h>
 #include <video/edid.h>
@@ -374,6 +375,14 @@ static void __init reserve_initrd(void)
 	    !ramdisk_image || !ramdisk_size)
 		return;		/* No initrd provided by bootloader */
 
+	/*
+	 * If SME is active, this memory will be marked encrypted by the
+	 * kernel when it is accessed (including relocation). However, the
+	 * ramdisk image was loaded decrypted by the bootloader, so make
+	 * sure that it is encrypted before accessing it.
+	 */
+	sme_early_encrypt(ramdisk_image, ramdisk_end - ramdisk_image);
+
 	initrd_start = 0;
 
 	mapped_size = memblock_mem_size(max_pfn_mapped);
diff --git a/arch/x86/mm/kasan_init_64.c b/arch/x86/mm/kasan_init_64.c
index d7cc830..1b8791f 100644
--- a/arch/x86/mm/kasan_init_64.c
+++ b/arch/x86/mm/kasan_init_64.c
@@ -11,8 +11,8 @@
 #include <asm/e820/types.h>
 #include <asm/tlbflush.h>
 #include <asm/sections.h>
+#include <asm/pgtable.h>
 
-extern pgd_t early_top_pgt[PTRS_PER_PGD];
 extern struct range pfn_mapped[E820_MAX_ENTRIES];
 
 static int __init map_range(struct range *range)
diff --git a/arch/x86/mm/mem_encrypt.c b/arch/x86/mm/mem_encrypt.c
index 54bb73c..0843d02 100644
--- a/arch/x86/mm/mem_encrypt.c
+++ b/arch/x86/mm/mem_encrypt.c
@@ -16,6 +16,8 @@
 
 #include <asm/tlbflush.h>
 #include <asm/fixmap.h>
+#include <asm/setup.h>
+#include <asm/bootparam.h>
 
 /*
  * Since SME related variables are set early in the boot process they must
@@ -98,6 +100,67 @@ void __init sme_early_decrypt(resource_size_t paddr, unsigned long size)
 	__sme_early_enc_dec(paddr, size, false);
 }
 
+static void __init __sme_early_map_unmap_mem(void *vaddr, unsigned long size,
+					     bool map)
+{
+	unsigned long paddr = (unsigned long)vaddr - __PAGE_OFFSET;
+	pmdval_t pmd_flags, pmd;
+
+	/* Use early_pmd_flags but remove the encryption mask */
+	pmd_flags = __sme_clr(early_pmd_flags);
+
+	do {
+		pmd = map ? (paddr & PMD_MASK) + pmd_flags : 0;
+		__early_make_pgtable((unsigned long)vaddr, pmd);
+
+		vaddr += PMD_SIZE;
+		paddr += PMD_SIZE;
+		size = (size <= PMD_SIZE) ? 0 : size - PMD_SIZE;
+	} while (size);
+
+	__native_flush_tlb();
+}
+
+void __init sme_unmap_bootdata(char *real_mode_data)
+{
+	struct boot_params *boot_data;
+	unsigned long cmdline_paddr;
+
+	if (!sme_active())
+		return;
+
+	/* Get the command line address before unmapping the real_mode_data */
+	boot_data = (struct boot_params *)real_mode_data;
+	cmdline_paddr = boot_data->hdr.cmd_line_ptr | ((u64)boot_data->ext_cmd_line_ptr << 32);
+
+	__sme_early_map_unmap_mem(real_mode_data, sizeof(boot_params), false);
+
+	if (!cmdline_paddr)
+		return;
+
+	__sme_early_map_unmap_mem(__va(cmdline_paddr), COMMAND_LINE_SIZE, false);
+}
+
+void __init sme_map_bootdata(char *real_mode_data)
+{
+	struct boot_params *boot_data;
+	unsigned long cmdline_paddr;
+
+	if (!sme_active())
+		return;
+
+	__sme_early_map_unmap_mem(real_mode_data, sizeof(boot_params), true);
+
+	/* Get the command line address after mapping the real_mode_data */
+	boot_data = (struct boot_params *)real_mode_data;
+	cmdline_paddr = boot_data->hdr.cmd_line_ptr | ((u64)boot_data->ext_cmd_line_ptr << 32);
+
+	if (!cmdline_paddr)
+		return;
+
+	__sme_early_map_unmap_mem(__va(cmdline_paddr), COMMAND_LINE_SIZE, true);
+}
+
 void __init sme_early_init(void)
 {
 	unsigned int i;



From thomas.lendacky at amd.com  Tue Jun 27 07:58:34 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Tue, 27 Jun 2017 09:58:34 -0500
Subject: [PATCH v8 13/38] x86/mm: Add support for early encrypt/decrypt of
 memory
In-Reply-To: <20170627145607.15908.26571.stgit@tlendack-t1.amdoffice.net>
References: <20170627145607.15908.26571.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170627145834.15908.73745.stgit@tlendack-t1.amdoffice.net>

Add support to be able to either encrypt or decrypt data in place during
the early stages of booting the kernel. This does not change the memory
encryption attribute - it is used for ensuring that data present in either
an encrypted or decrypted memory area is in the proper state (for example
the initrd will have been loaded by the boot loader and will not be
encrypted, but the memory that it resides in is marked as encrypted).

Reviewed-by: Borislav Petkov <bp at suse.de>
Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/include/asm/mem_encrypt.h |   10 +++++
 arch/x86/mm/mem_encrypt.c          |   76 ++++++++++++++++++++++++++++++++++++
 2 files changed, 86 insertions(+)

diff --git a/arch/x86/include/asm/mem_encrypt.h b/arch/x86/include/asm/mem_encrypt.h
index dbae7a5..8baa35b 100644
--- a/arch/x86/include/asm/mem_encrypt.h
+++ b/arch/x86/include/asm/mem_encrypt.h
@@ -21,6 +21,11 @@
 
 extern unsigned long sme_me_mask;
 
+void __init sme_early_encrypt(resource_size_t paddr,
+			      unsigned long size);
+void __init sme_early_decrypt(resource_size_t paddr,
+			      unsigned long size);
+
 void __init sme_early_init(void);
 
 void __init sme_encrypt_kernel(void);
@@ -30,6 +35,11 @@
 
 #define sme_me_mask	0UL
 
+static inline void __init sme_early_encrypt(resource_size_t paddr,
+					    unsigned long size) { }
+static inline void __init sme_early_decrypt(resource_size_t paddr,
+					    unsigned long size) { }
+
 static inline void __init sme_early_init(void) { }
 
 static inline void __init sme_encrypt_kernel(void) { }
diff --git a/arch/x86/mm/mem_encrypt.c b/arch/x86/mm/mem_encrypt.c
index f973d3d..54bb73c 100644
--- a/arch/x86/mm/mem_encrypt.c
+++ b/arch/x86/mm/mem_encrypt.c
@@ -14,6 +14,9 @@
 #include <linux/init.h>
 #include <linux/mm.h>
 
+#include <asm/tlbflush.h>
+#include <asm/fixmap.h>
+
 /*
  * Since SME related variables are set early in the boot process they must
  * reside in the .data section so as not to be zeroed out when the .bss
@@ -22,6 +25,79 @@
 unsigned long sme_me_mask __section(.data) = 0;
 EXPORT_SYMBOL_GPL(sme_me_mask);
 
+/* Buffer used for early in-place encryption by BSP, no locking needed */
+static char sme_early_buffer[PAGE_SIZE] __aligned(PAGE_SIZE);
+
+/*
+ * This routine does not change the underlying encryption setting of the
+ * page(s) that map this memory. It assumes that eventually the memory is
+ * meant to be accessed as either encrypted or decrypted but the contents
+ * are currently not in the desired state.
+ *
+ * This routine follows the steps outlined in the AMD64 Architecture
+ * Programmer's Manual Volume 2, Section 7.10.8 Encrypt-in-Place.
+ */
+static void __init __sme_early_enc_dec(resource_size_t paddr,
+				       unsigned long size, bool enc)
+{
+	void *src, *dst;
+	size_t len;
+
+	if (!sme_me_mask)
+		return;
+
+	local_flush_tlb();
+	wbinvd();
+
+	/*
+	 * There are limited number of early mapping slots, so map (at most)
+	 * one page at time.
+	 */
+	while (size) {
+		len = min_t(size_t, sizeof(sme_early_buffer), size);
+
+		/*
+		 * Create mappings for the current and desired format of
+		 * the memory. Use a write-protected mapping for the source.
+		 */
+		src = enc ? early_memremap_decrypted_wp(paddr, len) :
+			    early_memremap_encrypted_wp(paddr, len);
+
+		dst = enc ? early_memremap_encrypted(paddr, len) :
+			    early_memremap_decrypted(paddr, len);
+
+		/*
+		 * If a mapping can't be obtained to perform the operation,
+		 * then eventual access of that area in the desired mode
+		 * will cause a crash.
+		 */
+		BUG_ON(!src || !dst);
+
+		/*
+		 * Use a temporary buffer, of cache-line multiple size, to
+		 * avoid data corruption as documented in the APM.
+		 */
+		memcpy(sme_early_buffer, src, len);
+		memcpy(dst, sme_early_buffer, len);
+
+		early_memunmap(dst, len);
+		early_memunmap(src, len);
+
+		paddr += len;
+		size -= len;
+	}
+}
+
+void __init sme_early_encrypt(resource_size_t paddr, unsigned long size)
+{
+	__sme_early_enc_dec(paddr, size, true);
+}
+
+void __init sme_early_decrypt(resource_size_t paddr, unsigned long size)
+{
+	__sme_early_enc_dec(paddr, size, false);
+}
+
 void __init sme_early_init(void)
 {
 	unsigned int i;



From thomas.lendacky at amd.com  Tue Jun 27 07:58:55 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Tue, 27 Jun 2017 09:58:55 -0500
Subject: [PATCH v8 15/38] x86/boot/e820: Add support to determine the E820
 type of an address
In-Reply-To: <20170627145607.15908.26571.stgit@tlendack-t1.amdoffice.net>
References: <20170627145607.15908.26571.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170627145855.15908.1043.stgit@tlendack-t1.amdoffice.net>

Add a function that will return the E820 type associated with an address
range.

Reviewed-by: Borislav Petkov <bp at suse.de>
Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/include/asm/e820/api.h |    2 ++
 arch/x86/kernel/e820.c          |   26 +++++++++++++++++++++++---
 2 files changed, 25 insertions(+), 3 deletions(-)

diff --git a/arch/x86/include/asm/e820/api.h b/arch/x86/include/asm/e820/api.h
index 8e0f8b8..3641f5f 100644
--- a/arch/x86/include/asm/e820/api.h
+++ b/arch/x86/include/asm/e820/api.h
@@ -38,6 +38,8 @@
 extern void e820__reallocate_tables(void);
 extern void e820__register_nosave_regions(unsigned long limit_pfn);
 
+extern int  e820__get_entry_type(u64 start, u64 end);
+
 /*
  * Returns true iff the specified range [start,end) is completely contained inside
  * the ISA region.
diff --git a/arch/x86/kernel/e820.c b/arch/x86/kernel/e820.c
index d78a586..46c9b65 100644
--- a/arch/x86/kernel/e820.c
+++ b/arch/x86/kernel/e820.c
@@ -84,7 +84,8 @@ bool e820__mapped_any(u64 start, u64 end, enum e820_type type)
  * Note: this function only works correctly once the E820 table is sorted and
  * not-overlapping (at least for the range specified), which is the case normally.
  */
-bool __init e820__mapped_all(u64 start, u64 end, enum e820_type type)
+static struct e820_entry *__e820__mapped_all(u64 start, u64 end,
+					     enum e820_type type)
 {
 	int i;
 
@@ -110,9 +111,28 @@ bool __init e820__mapped_all(u64 start, u64 end, enum e820_type type)
 		 * coverage of the desired range exists:
 		 */
 		if (start >= end)
-			return 1;
+			return entry;
 	}
-	return 0;
+
+	return NULL;
+}
+
+/*
+ * This function checks if the entire range <start,end> is mapped with type.
+ */
+bool __init e820__mapped_all(u64 start, u64 end, enum e820_type type)
+{
+	return __e820__mapped_all(start, end, type);
+}
+
+/*
+ * This function returns the type associated with the range <start,end>.
+ */
+int e820__get_entry_type(u64 start, u64 end)
+{
+	struct e820_entry *entry = __e820__mapped_all(start, end, 0);
+
+	return entry ? entry->type : -EINVAL;
 }
 
 /*



From thomas.lendacky at amd.com  Tue Jun 27 07:59:03 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Tue, 27 Jun 2017 09:59:03 -0500
Subject: [PATCH v8 16/38] efi: Add an EFI table address match function
In-Reply-To: <20170627145607.15908.26571.stgit@tlendack-t1.amdoffice.net>
References: <20170627145607.15908.26571.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170627145903.15908.84218.stgit@tlendack-t1.amdoffice.net>

Add a function that will determine if a supplied physical address matches
the address of an EFI table.

Reviewed-by: Matt Fleming <matt at codeblueprint.co.uk>
Reviewed-by: Borislav Petkov <bp at suse.de>
Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 drivers/firmware/efi/efi.c |   33 +++++++++++++++++++++++++++++++++
 include/linux/efi.h        |    7 +++++++
 2 files changed, 40 insertions(+)

diff --git a/drivers/firmware/efi/efi.c b/drivers/firmware/efi/efi.c
index 045d6d3..69d4d13 100644
--- a/drivers/firmware/efi/efi.c
+++ b/drivers/firmware/efi/efi.c
@@ -55,6 +55,25 @@ struct efi __read_mostly efi = {
 };
 EXPORT_SYMBOL(efi);
 
+static unsigned long *efi_tables[] = {
+	&efi.mps,
+	&efi.acpi,
+	&efi.acpi20,
+	&efi.smbios,
+	&efi.smbios3,
+	&efi.sal_systab,
+	&efi.boot_info,
+	&efi.hcdp,
+	&efi.uga,
+	&efi.uv_systab,
+	&efi.fw_vendor,
+	&efi.runtime,
+	&efi.config_table,
+	&efi.esrt,
+	&efi.properties_table,
+	&efi.mem_attr_table,
+};
+
 static bool disable_runtime;
 static int __init setup_noefi(char *arg)
 {
@@ -855,6 +874,20 @@ int efi_status_to_err(efi_status_t status)
 	return err;
 }
 
+bool efi_is_table_address(unsigned long phys_addr)
+{
+	unsigned int i;
+
+	if (phys_addr == EFI_INVALID_TABLE_ADDR)
+		return false;
+
+	for (i = 0; i < ARRAY_SIZE(efi_tables); i++)
+		if (*(efi_tables[i]) == phys_addr)
+			return true;
+
+	return false;
+}
+
 #ifdef CONFIG_KEXEC
 static int update_efi_random_seed(struct notifier_block *nb,
 				  unsigned long code, void *unused)
diff --git a/include/linux/efi.h b/include/linux/efi.h
index 8269bcb..8e24f09 100644
--- a/include/linux/efi.h
+++ b/include/linux/efi.h
@@ -1091,6 +1091,8 @@ static inline bool efi_enabled(int feature)
 	return test_bit(feature, &efi.flags) != 0;
 }
 extern void efi_reboot(enum reboot_mode reboot_mode, const char *__unused);
+
+extern bool efi_is_table_address(unsigned long phys_addr);
 #else
 static inline bool efi_enabled(int feature)
 {
@@ -1104,6 +1106,11 @@ static inline bool efi_enabled(int feature)
 {
 	return false;
 }
+
+static inline bool efi_is_table_address(unsigned long phys_addr)
+{
+	return false;
+}
 #endif
 
 extern int efi_status_to_err(efi_status_t status);



From thomas.lendacky at amd.com  Tue Jun 27 07:59:35 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Tue, 27 Jun 2017 09:59:35 -0500
Subject: [PATCH v8 19/38] x86/mm: Add support to access boot related data in
 the clear
In-Reply-To: <20170627145607.15908.26571.stgit@tlendack-t1.amdoffice.net>
References: <20170627145607.15908.26571.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170627145935.15908.91473.stgit@tlendack-t1.amdoffice.net>

Boot data (such as EFI related data) is not encrypted when the system is
booted because UEFI/BIOS does not run with SME active. In order to access
this data properly it needs to be mapped decrypted.

Update early_memremap() to provide an arch specific routine to modify the
pagetable protection attributes before they are applied to the new
mapping. This is used to remove the encryption mask for boot related data.

Update memremap() to provide an arch specific routine to determine if RAM
remapping is allowed.  RAM remapping will cause an encrypted mapping to be
generated. By preventing RAM remapping, ioremap_cache() will be used
instead, which will provide a decrypted mapping of the boot related data.

Reviewed-by: Matt Fleming <matt at codeblueprint.co.uk>
Reviewed-by: Borislav Petkov <bp at suse.de>
Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/include/asm/io.h |    5 +
 arch/x86/mm/ioremap.c     |  179 +++++++++++++++++++++++++++++++++++++++++++++
 include/linux/io.h        |    2 +
 kernel/memremap.c         |   20 ++++-
 mm/early_ioremap.c        |   18 ++++-
 5 files changed, 217 insertions(+), 7 deletions(-)

diff --git a/arch/x86/include/asm/io.h b/arch/x86/include/asm/io.h
index 7afb0e2..09c5557 100644
--- a/arch/x86/include/asm/io.h
+++ b/arch/x86/include/asm/io.h
@@ -381,4 +381,9 @@ extern int __must_check arch_phys_wc_add(unsigned long base,
 #define arch_io_reserve_memtype_wc arch_io_reserve_memtype_wc
 #endif
 
+extern bool arch_memremap_can_ram_remap(resource_size_t offset,
+					unsigned long size,
+					unsigned long flags);
+#define arch_memremap_can_ram_remap arch_memremap_can_ram_remap
+
 #endif /* _ASM_X86_IO_H */
diff --git a/arch/x86/mm/ioremap.c b/arch/x86/mm/ioremap.c
index 26db273..ee33838 100644
--- a/arch/x86/mm/ioremap.c
+++ b/arch/x86/mm/ioremap.c
@@ -14,6 +14,7 @@
 #include <linux/vmalloc.h>
 #include <linux/mmiotrace.h>
 #include <linux/mem_encrypt.h>
+#include <linux/efi.h>
 
 #include <asm/set_memory.h>
 #include <asm/e820/api.h>
@@ -22,6 +23,7 @@
 #include <asm/tlbflush.h>
 #include <asm/pgalloc.h>
 #include <asm/pat.h>
+#include <asm/setup.h>
 
 #include "physaddr.h"
 
@@ -414,6 +416,183 @@ void unxlate_dev_mem_ptr(phys_addr_t phys, void *addr)
 	iounmap((void __iomem *)((unsigned long)addr & PAGE_MASK));
 }
 
+/*
+ * Examine the physical address to determine if it is an area of memory
+ * that should be mapped decrypted.  If the memory is not part of the
+ * kernel usable area it was accessed and created decrypted, so these
+ * areas should be mapped decrypted.
+ */
+static bool memremap_should_map_decrypted(resource_size_t phys_addr,
+					  unsigned long size)
+{
+	/* Check if the address is outside kernel usable area */
+	switch (e820__get_entry_type(phys_addr, phys_addr + size - 1)) {
+	case E820_TYPE_RESERVED:
+	case E820_TYPE_ACPI:
+	case E820_TYPE_NVS:
+	case E820_TYPE_UNUSABLE:
+		return true;
+	default:
+		break;
+	}
+
+	return false;
+}
+
+/*
+ * Examine the physical address to determine if it is EFI data. Check
+ * it against the boot params structure and EFI tables and memory types.
+ */
+static bool memremap_is_efi_data(resource_size_t phys_addr,
+				 unsigned long size)
+{
+	u64 paddr;
+
+	/* Check if the address is part of EFI boot/runtime data */
+	if (!efi_enabled(EFI_BOOT))
+		return false;
+
+	paddr = boot_params.efi_info.efi_memmap_hi;
+	paddr <<= 32;
+	paddr |= boot_params.efi_info.efi_memmap;
+	if (phys_addr == paddr)
+		return true;
+
+	paddr = boot_params.efi_info.efi_systab_hi;
+	paddr <<= 32;
+	paddr |= boot_params.efi_info.efi_systab;
+	if (phys_addr == paddr)
+		return true;
+
+	if (efi_is_table_address(phys_addr))
+		return true;
+
+	switch (efi_mem_type(phys_addr)) {
+	case EFI_BOOT_SERVICES_DATA:
+	case EFI_RUNTIME_SERVICES_DATA:
+		return true;
+	default:
+		break;
+	}
+
+	return false;
+}
+
+/*
+ * Examine the physical address to determine if it is boot data by checking
+ * it against the boot params setup_data chain.
+ */
+static bool memremap_is_setup_data(resource_size_t phys_addr,
+				   unsigned long size)
+{
+	struct setup_data *data;
+	u64 paddr, paddr_next;
+
+	paddr = boot_params.hdr.setup_data;
+	while (paddr) {
+		unsigned int len;
+
+		if (phys_addr == paddr)
+			return true;
+
+		data = memremap(paddr, sizeof(*data),
+				MEMREMAP_WB | MEMREMAP_DEC);
+
+		paddr_next = data->next;
+		len = data->len;
+
+		memunmap(data);
+
+		if ((phys_addr > paddr) && (phys_addr < (paddr + len)))
+			return true;
+
+		paddr = paddr_next;
+	}
+
+	return false;
+}
+
+/*
+ * Examine the physical address to determine if it is boot data by checking
+ * it against the boot params setup_data chain (early boot version).
+ */
+static bool __init early_memremap_is_setup_data(resource_size_t phys_addr,
+						unsigned long size)
+{
+	struct setup_data *data;
+	u64 paddr, paddr_next;
+
+	paddr = boot_params.hdr.setup_data;
+	while (paddr) {
+		unsigned int len;
+
+		if (phys_addr == paddr)
+			return true;
+
+		data = early_memremap_decrypted(paddr, sizeof(*data));
+
+		paddr_next = data->next;
+		len = data->len;
+
+		early_memunmap(data, sizeof(*data));
+
+		if ((phys_addr > paddr) && (phys_addr < (paddr + len)))
+			return true;
+
+		paddr = paddr_next;
+	}
+
+	return false;
+}
+
+/*
+ * Architecture function to determine if RAM remap is allowed. By default, a
+ * RAM remap will map the data as encrypted. Determine if a RAM remap should
+ * not be done so that the data will be mapped decrypted.
+ */
+bool arch_memremap_can_ram_remap(resource_size_t phys_addr, unsigned long size,
+				 unsigned long flags)
+{
+	if (!sme_active())
+		return true;
+
+	if (flags & MEMREMAP_ENC)
+		return true;
+
+	if (flags & MEMREMAP_DEC)
+		return false;
+
+	if (memremap_is_setup_data(phys_addr, size) ||
+	    memremap_is_efi_data(phys_addr, size) ||
+	    memremap_should_map_decrypted(phys_addr, size))
+		return false;
+
+	return true;
+}
+
+/*
+ * Architecture override of __weak function to adjust the protection attributes
+ * used when remapping memory. By default, early_memremap() will map the data
+ * as encrypted. Determine if an encrypted mapping should not be done and set
+ * the appropriate protection attributes.
+ */
+pgprot_t __init early_memremap_pgprot_adjust(resource_size_t phys_addr,
+					     unsigned long size,
+					     pgprot_t prot)
+{
+	if (!sme_active())
+		return prot;
+
+	if (early_memremap_is_setup_data(phys_addr, size) ||
+	    memremap_is_efi_data(phys_addr, size) ||
+	    memremap_should_map_decrypted(phys_addr, size))
+		prot = pgprot_decrypted(prot);
+	else
+		prot = pgprot_encrypted(prot);
+
+	return prot;
+}
+
 #ifdef CONFIG_ARCH_USE_MEMREMAP_PROT
 /* Remap memory with encryption */
 void __init *early_memremap_encrypted(resource_size_t phys_addr,
diff --git a/include/linux/io.h b/include/linux/io.h
index 2195d9e..32e30e8 100644
--- a/include/linux/io.h
+++ b/include/linux/io.h
@@ -157,6 +157,8 @@ enum {
 	MEMREMAP_WB = 1 << 0,
 	MEMREMAP_WT = 1 << 1,
 	MEMREMAP_WC = 1 << 2,
+	MEMREMAP_ENC = 1 << 3,
+	MEMREMAP_DEC = 1 << 4,
 };
 
 void *memremap(resource_size_t offset, size_t size, unsigned long flags);
diff --git a/kernel/memremap.c b/kernel/memremap.c
index 23a6483..9b4fbd5 100644
--- a/kernel/memremap.c
+++ b/kernel/memremap.c
@@ -34,13 +34,24 @@ static void *arch_memremap_wb(resource_size_t offset, unsigned long size)
 }
 #endif
 
-static void *try_ram_remap(resource_size_t offset, size_t size)
+#ifndef arch_memremap_can_ram_remap
+static bool arch_memremap_can_ram_remap(resource_size_t offset, size_t size,
+					unsigned long flags)
+{
+	return true;
+}
+#endif
+
+static void *try_ram_remap(resource_size_t offset, size_t size,
+			   unsigned long flags)
 {
 	unsigned long pfn = PHYS_PFN(offset);
 
 	/* In the simple case just return the existing linear address */
-	if (pfn_valid(pfn) && !PageHighMem(pfn_to_page(pfn)))
+	if (pfn_valid(pfn) && !PageHighMem(pfn_to_page(pfn)) &&
+	    arch_memremap_can_ram_remap(offset, size, flags))
 		return __va(offset);
+
 	return NULL; /* fallback to arch_memremap_wb */
 }
 
@@ -48,7 +59,8 @@ static void *try_ram_remap(resource_size_t offset, size_t size)
  * memremap() - remap an iomem_resource as cacheable memory
  * @offset: iomem resource start address
  * @size: size of remap
- * @flags: any of MEMREMAP_WB, MEMREMAP_WT and MEMREMAP_WC
+ * @flags: any of MEMREMAP_WB, MEMREMAP_WT, MEMREMAP_WC,
+ *		  MEMREMAP_ENC, MEMREMAP_DEC
  *
  * memremap() is "ioremap" for cases where it is known that the resource
  * being mapped does not have i/o side effects and the __iomem
@@ -95,7 +107,7 @@ void *memremap(resource_size_t offset, size_t size, unsigned long flags)
 		 * the requested range is potentially in System RAM.
 		 */
 		if (is_ram == REGION_INTERSECTS)
-			addr = try_ram_remap(offset, size);
+			addr = try_ram_remap(offset, size, flags);
 		if (!addr)
 			addr = arch_memremap_wb(offset, size);
 	}
diff --git a/mm/early_ioremap.c b/mm/early_ioremap.c
index d7d30da..b1dd4a9 100644
--- a/mm/early_ioremap.c
+++ b/mm/early_ioremap.c
@@ -30,6 +30,13 @@ static int __init early_ioremap_debug_setup(char *str)
 
 static int after_paging_init __initdata;
 
+pgprot_t __init __weak early_memremap_pgprot_adjust(resource_size_t phys_addr,
+						    unsigned long size,
+						    pgprot_t prot)
+{
+	return prot;
+}
+
 void __init __weak early_ioremap_shutdown(void)
 {
 }
@@ -215,14 +222,19 @@ void __init early_iounmap(void __iomem *addr, unsigned long size)
 void __init *
 early_memremap(resource_size_t phys_addr, unsigned long size)
 {
-	return (__force void *)__early_ioremap(phys_addr, size,
-					       FIXMAP_PAGE_NORMAL);
+	pgprot_t prot = early_memremap_pgprot_adjust(phys_addr, size,
+						     FIXMAP_PAGE_NORMAL);
+
+	return (__force void *)__early_ioremap(phys_addr, size, prot);
 }
 #ifdef FIXMAP_PAGE_RO
 void __init *
 early_memremap_ro(resource_size_t phys_addr, unsigned long size)
 {
-	return (__force void *)__early_ioremap(phys_addr, size, FIXMAP_PAGE_RO);
+	pgprot_t prot = early_memremap_pgprot_adjust(phys_addr, size,
+						     FIXMAP_PAGE_RO);
+
+	return (__force void *)__early_ioremap(phys_addr, size, prot);
 }
 #endif
 



From thomas.lendacky at amd.com  Tue Jun 27 07:59:48 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Tue, 27 Jun 2017 09:59:48 -0500
Subject: [PATCH v8 20/38] x86,
 mpparse: Use memremap to map the mpf and mpc data
In-Reply-To: <20170627145607.15908.26571.stgit@tlendack-t1.amdoffice.net>
References: <20170627145607.15908.26571.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170627145948.15908.24705.stgit@tlendack-t1.amdoffice.net>

The SMP MP-table is built by UEFI and placed in memory in a decrypted
state. These tables are accessed using a mix of early_memremap(),
early_memunmap(), phys_to_virt() and virt_to_phys(). Change all accesses
to use early_memremap()/early_memunmap(). This allows for proper setting
of the encryption mask so that the data can be successfully accessed when
SME is active.

Reviewed-by: Borislav Petkov <bp at suse.de>
Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/kernel/mpparse.c |   98 ++++++++++++++++++++++++++++++++-------------
 1 file changed, 70 insertions(+), 28 deletions(-)

diff --git a/arch/x86/kernel/mpparse.c b/arch/x86/kernel/mpparse.c
index fd37f39..5cbb317 100644
--- a/arch/x86/kernel/mpparse.c
+++ b/arch/x86/kernel/mpparse.c
@@ -429,7 +429,7 @@ static inline void __init construct_default_ISA_mptable(int mpc_default_type)
 	}
 }
 
-static struct mpf_intel *mpf_found;
+static unsigned long mpf_base;
 
 static unsigned long __init get_mpc_size(unsigned long physptr)
 {
@@ -451,6 +451,7 @@ static int __init check_physptr(struct mpf_intel *mpf, unsigned int early)
 
 	size = get_mpc_size(mpf->physptr);
 	mpc = early_memremap(mpf->physptr, size);
+
 	/*
 	 * Read the physical hardware table.  Anything here will
 	 * override the defaults.
@@ -497,12 +498,12 @@ static int __init check_physptr(struct mpf_intel *mpf, unsigned int early)
  */
 void __init default_get_smp_config(unsigned int early)
 {
-	struct mpf_intel *mpf = mpf_found;
+	struct mpf_intel *mpf;
 
 	if (!smp_found_config)
 		return;
 
-	if (!mpf)
+	if (!mpf_base)
 		return;
 
 	if (acpi_lapic && early)
@@ -515,6 +516,12 @@ void __init default_get_smp_config(unsigned int early)
 	if (acpi_lapic && acpi_ioapic)
 		return;
 
+	mpf = early_memremap(mpf_base, sizeof(*mpf));
+	if (!mpf) {
+		pr_err("MPTABLE: error mapping MP table\n");
+		return;
+	}
+
 	pr_info("Intel MultiProcessor Specification v1.%d\n",
 		mpf->specification);
 #if defined(CONFIG_X86_LOCAL_APIC) && defined(CONFIG_X86_32)
@@ -529,7 +536,7 @@ void __init default_get_smp_config(unsigned int early)
 	/*
 	 * Now see if we need to read further.
 	 */
-	if (mpf->feature1 != 0) {
+	if (mpf->feature1) {
 		if (early) {
 			/*
 			 * local APIC has default address
@@ -542,8 +549,10 @@ void __init default_get_smp_config(unsigned int early)
 		construct_default_ISA_mptable(mpf->feature1);
 
 	} else if (mpf->physptr) {
-		if (check_physptr(mpf, early))
+		if (check_physptr(mpf, early)) {
+			early_memunmap(mpf, sizeof(*mpf));
 			return;
+		}
 	} else
 		BUG();
 
@@ -552,6 +561,8 @@ void __init default_get_smp_config(unsigned int early)
 	/*
 	 * Only use the first configuration found.
 	 */
+
+	early_memunmap(mpf, sizeof(*mpf));
 }
 
 static void __init smp_reserve_memory(struct mpf_intel *mpf)
@@ -561,15 +572,16 @@ static void __init smp_reserve_memory(struct mpf_intel *mpf)
 
 static int __init smp_scan_config(unsigned long base, unsigned long length)
 {
-	unsigned int *bp = phys_to_virt(base);
+	unsigned int *bp;
 	struct mpf_intel *mpf;
-	unsigned long mem;
+	int ret = 0;
 
 	apic_printk(APIC_VERBOSE, "Scan for SMP in [mem %#010lx-%#010lx]\n",
 		    base, base + length - 1);
 	BUILD_BUG_ON(sizeof(*mpf) != 16);
 
 	while (length > 0) {
+		bp = early_memremap(base, length);
 		mpf = (struct mpf_intel *)bp;
 		if ((*bp == SMP_MAGIC_IDENT) &&
 		    (mpf->length == 1) &&
@@ -579,24 +591,26 @@ static int __init smp_scan_config(unsigned long base, unsigned long length)
 #ifdef CONFIG_X86_LOCAL_APIC
 			smp_found_config = 1;
 #endif
-			mpf_found = mpf;
+			mpf_base = base;
 
-			pr_info("found SMP MP-table at [mem %#010llx-%#010llx] mapped at [%p]\n",
-				(unsigned long long) virt_to_phys(mpf),
-				(unsigned long long) virt_to_phys(mpf) +
-				sizeof(*mpf) - 1, mpf);
+			pr_info("found SMP MP-table at [mem %#010lx-%#010lx] mapped at [%p]\n",
+				base, base + sizeof(*mpf) - 1, mpf);
 
-			mem = virt_to_phys(mpf);
-			memblock_reserve(mem, sizeof(*mpf));
+			memblock_reserve(base, sizeof(*mpf));
 			if (mpf->physptr)
 				smp_reserve_memory(mpf);
 
-			return 1;
+			ret = 1;
 		}
-		bp += 4;
+		early_memunmap(bp, length);
+
+		if (ret)
+			break;
+
+		base += 16;
 		length -= 16;
 	}
-	return 0;
+	return ret;
 }
 
 void __init default_find_smp_config(void)
@@ -838,29 +852,40 @@ static int __init update_mp_table(void)
 	char oem[10];
 	struct mpf_intel *mpf;
 	struct mpc_table *mpc, *mpc_new;
+	unsigned long size;
 
 	if (!enable_update_mptable)
 		return 0;
 
-	mpf = mpf_found;
-	if (!mpf)
+	if (!mpf_base)
+		return 0;
+
+	mpf = early_memremap(mpf_base, sizeof(*mpf));
+	if (!mpf) {
+		pr_err("MPTABLE: mpf early_memremap() failed\n");
 		return 0;
+	}
 
 	/*
 	 * Now see if we need to go further.
 	 */
-	if (mpf->feature1 != 0)
-		return 0;
+	if (mpf->feature1)
+		goto do_unmap_mpf;
 
 	if (!mpf->physptr)
-		return 0;
+		goto do_unmap_mpf;
 
-	mpc = phys_to_virt(mpf->physptr);
+	size = get_mpc_size(mpf->physptr);
+	mpc = early_memremap(mpf->physptr, size);
+	if (!mpc) {
+		pr_err("MPTABLE: mpc early_memremap() failed\n");
+		goto do_unmap_mpf;
+	}
 
 	if (!smp_check_mpc(mpc, oem, str))
-		return 0;
+		goto do_unmap_mpc;
 
-	pr_info("mpf: %llx\n", (u64)virt_to_phys(mpf));
+	pr_info("mpf: %llx\n", (u64)mpf_base);
 	pr_info("physptr: %x\n", mpf->physptr);
 
 	if (mpc_new_phys && mpc->length > mpc_new_length) {
@@ -878,21 +903,32 @@ static int __init update_mp_table(void)
 		new = mpf_checksum((unsigned char *)mpc, mpc->length);
 		if (old == new) {
 			pr_info("mpc is readonly, please try alloc_mptable instead\n");
-			return 0;
+			goto do_unmap_mpc;
 		}
 		pr_info("use in-position replacing\n");
 	} else {
+		mpc_new = early_memremap(mpc_new_phys, mpc_new_length);
+		if (!mpc_new) {
+			pr_err("MPTABLE: new mpc early_memremap() failed\n");
+			goto do_unmap_mpc;
+		}
 		mpf->physptr = mpc_new_phys;
-		mpc_new = phys_to_virt(mpc_new_phys);
 		memcpy(mpc_new, mpc, mpc->length);
+		early_memunmap(mpc, size);
 		mpc = mpc_new;
+		size = mpc_new_length;
 		/* check if we can modify that */
 		if (mpc_new_phys - mpf->physptr) {
 			struct mpf_intel *mpf_new;
 			/* steal 16 bytes from [0, 1k) */
+			mpf_new = early_memremap(0x400 - 16, sizeof(*mpf_new));
+			if (!mpf_new) {
+				pr_err("MPTABLE: new mpf early_memremap() failed\n");
+				goto do_unmap_mpc;
+			}
 			pr_info("mpf new: %x\n", 0x400 - 16);
-			mpf_new = phys_to_virt(0x400 - 16);
 			memcpy(mpf_new, mpf, 16);
+			early_memunmap(mpf, sizeof(*mpf));
 			mpf = mpf_new;
 			mpf->physptr = mpc_new_phys;
 		}
@@ -909,6 +945,12 @@ static int __init update_mp_table(void)
 	 */
 	replace_intsrc_all(mpc, mpc_new_phys, mpc_new_length);
 
+do_unmap_mpc:
+	early_memunmap(mpc, size);
+
+do_unmap_mpf:
+	early_memunmap(mpf, sizeof(*mpf));
+
 	return 0;
 }
 



From thomas.lendacky at amd.com  Tue Jun 27 07:59:56 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Tue, 27 Jun 2017 09:59:56 -0500
Subject: [PATCH v8 21/38] x86/mm: Add support to access persistent memory in
 the clear
In-Reply-To: <20170627145607.15908.26571.stgit@tlendack-t1.amdoffice.net>
References: <20170627145607.15908.26571.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170627145956.15908.64000.stgit@tlendack-t1.amdoffice.net>

Persistent memory is expected to persist across reboots. The encryption
key used by SME will change across reboots which will result in corrupted
persistent memory.  Persistent memory is handed out by block devices
through memory remapping functions, so be sure not to map this memory as
encrypted.

Reviewed-by: Borislav Petkov <bp at suse.de>
Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/mm/ioremap.c |   31 ++++++++++++++++++++++++++++++-
 1 file changed, 30 insertions(+), 1 deletion(-)

diff --git a/arch/x86/mm/ioremap.c b/arch/x86/mm/ioremap.c
index ee33838..effa529 100644
--- a/arch/x86/mm/ioremap.c
+++ b/arch/x86/mm/ioremap.c
@@ -420,17 +420,46 @@ void unxlate_dev_mem_ptr(phys_addr_t phys, void *addr)
  * Examine the physical address to determine if it is an area of memory
  * that should be mapped decrypted.  If the memory is not part of the
  * kernel usable area it was accessed and created decrypted, so these
- * areas should be mapped decrypted.
+ * areas should be mapped decrypted. And since the encryption key can
+ * change across reboots, persistent memory should also be mapped
+ * decrypted.
  */
 static bool memremap_should_map_decrypted(resource_size_t phys_addr,
 					  unsigned long size)
 {
+	int is_pmem;
+
+	/*
+	 * Check if the address is part of a persistent memory region.
+	 * This check covers areas added by E820, EFI and ACPI.
+	 */
+	is_pmem = region_intersects(phys_addr, size, IORESOURCE_MEM,
+				    IORES_DESC_PERSISTENT_MEMORY);
+	if (is_pmem != REGION_DISJOINT)
+		return true;
+
+	/*
+	 * Check if the non-volatile attribute is set for an EFI
+	 * reserved area.
+	 */
+	if (efi_enabled(EFI_BOOT)) {
+		switch (efi_mem_type(phys_addr)) {
+		case EFI_RESERVED_TYPE:
+			if (efi_mem_attributes(phys_addr) & EFI_MEMORY_NV)
+				return true;
+			break;
+		default:
+			break;
+		}
+	}
+
 	/* Check if the address is outside kernel usable area */
 	switch (e820__get_entry_type(phys_addr, phys_addr + size - 1)) {
 	case E820_TYPE_RESERVED:
 	case E820_TYPE_ACPI:
 	case E820_TYPE_NVS:
 	case E820_TYPE_UNUSABLE:
+	case E820_TYPE_PRAM:
 		return true;
 	default:
 		break;



From thomas.lendacky at amd.com  Tue Jun 27 08:00:05 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Tue, 27 Jun 2017 10:00:05 -0500
Subject: [PATCH v8 22/38] x86/mm: Add support for changing the memory
 encryption attribute
In-Reply-To: <20170627145607.15908.26571.stgit@tlendack-t1.amdoffice.net>
References: <20170627145607.15908.26571.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170627150005.15908.84160.stgit@tlendack-t1.amdoffice.net>

Add support for changing the memory encryption attribute for one or more
memory pages. This will be useful when we have to change the AP trampoline
area to not be encrypted. Or when we need to change the SWIOTLB area to
not be encrypted in support of devices that can't support the encryption
mask range.

Reviewed-by: Borislav Petkov <bp at suse.de>
Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/include/asm/set_memory.h |    3 ++
 arch/x86/mm/pageattr.c            |   62 +++++++++++++++++++++++++++++++++++++
 2 files changed, 65 insertions(+)

diff --git a/arch/x86/include/asm/set_memory.h b/arch/x86/include/asm/set_memory.h
index eaec6c3..cd71273 100644
--- a/arch/x86/include/asm/set_memory.h
+++ b/arch/x86/include/asm/set_memory.h
@@ -11,6 +11,7 @@
  * Executability : eXeutable, NoteXecutable
  * Read/Write    : ReadOnly, ReadWrite
  * Presence      : NotPresent
+ * Encryption    : Encrypted, Decrypted
  *
  * Within a category, the attributes are mutually exclusive.
  *
@@ -42,6 +43,8 @@
 int set_memory_wb(unsigned long addr, int numpages);
 int set_memory_np(unsigned long addr, int numpages);
 int set_memory_4k(unsigned long addr, int numpages);
+int set_memory_encrypted(unsigned long addr, int numpages);
+int set_memory_decrypted(unsigned long addr, int numpages);
 
 int set_memory_array_uc(unsigned long *addr, int addrinarray);
 int set_memory_array_wc(unsigned long *addr, int addrinarray);
diff --git a/arch/x86/mm/pageattr.c b/arch/x86/mm/pageattr.c
index e7d3866..d9e09fb 100644
--- a/arch/x86/mm/pageattr.c
+++ b/arch/x86/mm/pageattr.c
@@ -1769,6 +1769,68 @@ int set_memory_4k(unsigned long addr, int numpages)
 					__pgprot(0), 1, 0, NULL);
 }
 
+static int __set_memory_enc_dec(unsigned long addr, int numpages, bool enc)
+{
+	struct cpa_data cpa;
+	unsigned long start;
+	int ret;
+
+	/* Nothing to do if the SME is not active */
+	if (!sme_active())
+		return 0;
+
+	/* Should not be working on unaligned addresses */
+	if (WARN_ONCE(addr & ~PAGE_MASK, "misaligned address: %#lx\n", addr))
+		addr &= PAGE_MASK;
+
+	start = addr;
+
+	memset(&cpa, 0, sizeof(cpa));
+	cpa.vaddr = &addr;
+	cpa.numpages = numpages;
+	cpa.mask_set = enc ? __pgprot(_PAGE_ENC) : __pgprot(0);
+	cpa.mask_clr = enc ? __pgprot(0) : __pgprot(_PAGE_ENC);
+	cpa.pgd = init_mm.pgd;
+
+	/* Must avoid aliasing mappings in the highmem code */
+	kmap_flush_unused();
+	vm_unmap_aliases();
+
+	/*
+	 * Before changing the encryption attribute, we need to flush caches.
+	 */
+	if (static_cpu_has(X86_FEATURE_CLFLUSH))
+		cpa_flush_range(start, numpages, 1);
+	else
+		cpa_flush_all(1);
+
+	ret = __change_page_attr_set_clr(&cpa, 1);
+
+	/*
+	 * After changing the encryption attribute, we need to flush TLBs
+	 * again in case any speculative TLB caching occurred (but no need
+	 * to flush caches again).  We could just use cpa_flush_all(), but
+	 * in case TLB flushing gets optimized in the cpa_flush_range()
+	 * path use the same logic as above.
+	 */
+	if (static_cpu_has(X86_FEATURE_CLFLUSH))
+		cpa_flush_range(start, numpages, 0);
+	else
+		cpa_flush_all(0);
+
+	return ret;
+}
+
+int set_memory_encrypted(unsigned long addr, int numpages)
+{
+	return __set_memory_enc_dec(addr, numpages, true);
+}
+
+int set_memory_decrypted(unsigned long addr, int numpages)
+{
+	return __set_memory_enc_dec(addr, numpages, false);
+}
+
 int set_pages_uc(struct page *page, int numpages)
 {
 	unsigned long addr = (unsigned long)page_address(page);



From thomas.lendacky at amd.com  Tue Jun 27 08:00:16 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Tue, 27 Jun 2017 10:00:16 -0500
Subject: [PATCH v8 23/38] x86/realmode: Decrypt trampoline area if memory
 encryption is active
In-Reply-To: <20170627145607.15908.26571.stgit@tlendack-t1.amdoffice.net>
References: <20170627145607.15908.26571.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170627150016.15908.37813.stgit@tlendack-t1.amdoffice.net>

When Secure Memory Encryption is enabled, the trampoline area must not
be encrypted. A CPU running in real mode will not be able to decrypt
memory that has been encrypted because it will not be able to use addresses
with the memory encryption mask.

Reviewed-by: Borislav Petkov <bp at suse.de>
Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/realmode/init.c |    8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/arch/x86/realmode/init.c b/arch/x86/realmode/init.c
index cd4be19..d6ddc7e 100644
--- a/arch/x86/realmode/init.c
+++ b/arch/x86/realmode/init.c
@@ -1,6 +1,7 @@
 #include <linux/io.h>
 #include <linux/slab.h>
 #include <linux/memblock.h>
+#include <linux/mem_encrypt.h>
 
 #include <asm/set_memory.h>
 #include <asm/pgtable.h>
@@ -59,6 +60,13 @@ static void __init setup_real_mode(void)
 
 	base = (unsigned char *)real_mode_header;
 
+	/*
+	 * If SME is active, the trampoline area will need to be in
+	 * decrypted memory in order to bring up other processors
+	 * successfully.
+	 */
+	set_memory_decrypted((unsigned long)base, size >> PAGE_SHIFT);
+
 	memcpy(base, real_mode_blob, size);
 
 	phys_base = __pa(base);



From thomas.lendacky at amd.com  Tue Jun 27 08:00:27 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Tue, 27 Jun 2017 10:00:27 -0500
Subject: [PATCH v8 24/38] x86, swiotlb: Add memory encryption support
In-Reply-To: <20170627145607.15908.26571.stgit@tlendack-t1.amdoffice.net>
References: <20170627145607.15908.26571.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170627150027.15908.70421.stgit@tlendack-t1.amdoffice.net>

Since DMA addresses will effectively look like 48-bit addresses when the
memory encryption mask is set, SWIOTLB is needed if the DMA mask of the
device performing the DMA does not support 48-bits. SWIOTLB will be
initialized to create decrypted bounce buffers for use by these devices.

Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/include/asm/dma-mapping.h |    5 ++-
 arch/x86/include/asm/mem_encrypt.h |    5 +++
 arch/x86/kernel/pci-dma.c          |   11 +++++--
 arch/x86/kernel/pci-nommu.c        |    2 +
 arch/x86/kernel/pci-swiotlb.c      |   15 +++++++++-
 arch/x86/mm/mem_encrypt.c          |   22 +++++++++++++++
 include/linux/swiotlb.h            |    1 +
 init/main.c                        |   10 +++++++
 lib/swiotlb.c                      |   54 +++++++++++++++++++++++++++++++-----
 9 files changed, 108 insertions(+), 17 deletions(-)

diff --git a/arch/x86/include/asm/dma-mapping.h b/arch/x86/include/asm/dma-mapping.h
index 08a0838..191f9a5 100644
--- a/arch/x86/include/asm/dma-mapping.h
+++ b/arch/x86/include/asm/dma-mapping.h
@@ -12,6 +12,7 @@
 #include <asm/io.h>
 #include <asm/swiotlb.h>
 #include <linux/dma-contiguous.h>
+#include <linux/mem_encrypt.h>
 
 #ifdef CONFIG_ISA
 # define ISA_DMA_BIT_MASK DMA_BIT_MASK(24)
@@ -62,12 +63,12 @@ static inline bool dma_capable(struct device *dev, dma_addr_t addr, size_t size)
 
 static inline dma_addr_t phys_to_dma(struct device *dev, phys_addr_t paddr)
 {
-	return paddr;
+	return __sme_set(paddr);
 }
 
 static inline phys_addr_t dma_to_phys(struct device *dev, dma_addr_t daddr)
 {
-	return daddr;
+	return __sme_clr(daddr);
 }
 #endif /* CONFIG_X86_DMA_REMAP */
 
diff --git a/arch/x86/include/asm/mem_encrypt.h b/arch/x86/include/asm/mem_encrypt.h
index ab1fe77..70e55f6 100644
--- a/arch/x86/include/asm/mem_encrypt.h
+++ b/arch/x86/include/asm/mem_encrypt.h
@@ -34,6 +34,11 @@ void __init sme_early_decrypt(resource_size_t paddr,
 void __init sme_encrypt_kernel(void);
 void __init sme_enable(void);
 
+/* Architecture __weak replacement functions */
+void __init mem_encrypt_init(void);
+
+void swiotlb_set_mem_attributes(void *vaddr, unsigned long size);
+
 #else	/* !CONFIG_AMD_MEM_ENCRYPT */
 
 #define sme_me_mask	0UL
diff --git a/arch/x86/kernel/pci-dma.c b/arch/x86/kernel/pci-dma.c
index 3a216ec..72d96d4 100644
--- a/arch/x86/kernel/pci-dma.c
+++ b/arch/x86/kernel/pci-dma.c
@@ -93,9 +93,12 @@ void *dma_generic_alloc_coherent(struct device *dev, size_t size,
 	if (gfpflags_allow_blocking(flag)) {
 		page = dma_alloc_from_contiguous(dev, count, get_order(size),
 						 flag);
-		if (page && page_to_phys(page) + size > dma_mask) {
-			dma_release_from_contiguous(dev, page, count);
-			page = NULL;
+		if (page) {
+			addr = phys_to_dma(dev, page_to_phys(page));
+			if (addr + size > dma_mask) {
+				dma_release_from_contiguous(dev, page, count);
+				page = NULL;
+			}
 		}
 	}
 	/* fallback */
@@ -104,7 +107,7 @@ void *dma_generic_alloc_coherent(struct device *dev, size_t size,
 	if (!page)
 		return NULL;
 
-	addr = page_to_phys(page);
+	addr = phys_to_dma(dev, page_to_phys(page));
 	if (addr + size > dma_mask) {
 		__free_pages(page, get_order(size));
 
diff --git a/arch/x86/kernel/pci-nommu.c b/arch/x86/kernel/pci-nommu.c
index a88952e..98b576a 100644
--- a/arch/x86/kernel/pci-nommu.c
+++ b/arch/x86/kernel/pci-nommu.c
@@ -30,7 +30,7 @@ static dma_addr_t nommu_map_page(struct device *dev, struct page *page,
 				 enum dma_data_direction dir,
 				 unsigned long attrs)
 {
-	dma_addr_t bus = page_to_phys(page) + offset;
+	dma_addr_t bus = phys_to_dma(dev, page_to_phys(page)) + offset;
 	WARN_ON(size == 0);
 	if (!check_addr("map_single", dev, bus, size))
 		return DMA_ERROR_CODE;
diff --git a/arch/x86/kernel/pci-swiotlb.c b/arch/x86/kernel/pci-swiotlb.c
index 1e23577..6770775 100644
--- a/arch/x86/kernel/pci-swiotlb.c
+++ b/arch/x86/kernel/pci-swiotlb.c
@@ -6,12 +6,14 @@
 #include <linux/swiotlb.h>
 #include <linux/bootmem.h>
 #include <linux/dma-mapping.h>
+#include <linux/mem_encrypt.h>
 
 #include <asm/iommu.h>
 #include <asm/swiotlb.h>
 #include <asm/dma.h>
 #include <asm/xen/swiotlb-xen.h>
 #include <asm/iommu_table.h>
+
 int swiotlb __read_mostly;
 
 void *x86_swiotlb_alloc_coherent(struct device *hwdev, size_t size,
@@ -79,8 +81,8 @@ int __init pci_swiotlb_detect_override(void)
 		  pci_swiotlb_late_init);
 
 /*
- * if 4GB or more detected (and iommu=off not set) return 1
- * and set swiotlb to 1.
+ * If 4GB or more detected (and iommu=off not set) or if SME is active
+ * then set swiotlb to 1 and return 1.
  */
 int __init pci_swiotlb_detect_4gb(void)
 {
@@ -89,6 +91,15 @@ int __init pci_swiotlb_detect_4gb(void)
 	if (!no_iommu && max_possible_pfn > MAX_DMA32_PFN)
 		swiotlb = 1;
 #endif
+
+	/*
+	 * If SME is active then swiotlb will be set to 1 so that bounce
+	 * buffers are allocated and used for devices that do not support
+	 * the addressing range required for the encryption mask.
+	 */
+	if (sme_active())
+		swiotlb = 1;
+
 	return swiotlb;
 }
 IOMMU_INIT(pci_swiotlb_detect_4gb,
diff --git a/arch/x86/mm/mem_encrypt.c b/arch/x86/mm/mem_encrypt.c
index 0843d02..a7400ec 100644
--- a/arch/x86/mm/mem_encrypt.c
+++ b/arch/x86/mm/mem_encrypt.c
@@ -13,11 +13,14 @@
 #include <linux/linkage.h>
 #include <linux/init.h>
 #include <linux/mm.h>
+#include <linux/dma-mapping.h>
+#include <linux/swiotlb.h>
 
 #include <asm/tlbflush.h>
 #include <asm/fixmap.h>
 #include <asm/setup.h>
 #include <asm/bootparam.h>
+#include <asm/set_memory.h>
 
 /*
  * Since SME related variables are set early in the boot process they must
@@ -177,6 +180,25 @@ void __init sme_early_init(void)
 		protection_map[i] = pgprot_encrypted(protection_map[i]);
 }
 
+/* Architecture __weak replacement functions */
+void __init mem_encrypt_init(void)
+{
+	if (!sme_me_mask)
+		return;
+
+	/* Call into SWIOTLB to update the SWIOTLB DMA buffers */
+	swiotlb_update_mem_attributes();
+}
+
+void swiotlb_set_mem_attributes(void *vaddr, unsigned long size)
+{
+	WARN(PAGE_ALIGN(size) != size,
+	     "size is not page-aligned (%#lx)\n", size);
+
+	/* Make the SWIOTLB buffer area decrypted */
+	set_memory_decrypted((unsigned long)vaddr, size >> PAGE_SHIFT);
+}
+
 void __init sme_encrypt_kernel(void)
 {
 }
diff --git a/include/linux/swiotlb.h b/include/linux/swiotlb.h
index 4ee479f..15e7160 100644
--- a/include/linux/swiotlb.h
+++ b/include/linux/swiotlb.h
@@ -35,6 +35,7 @@ enum swiotlb_force {
 extern unsigned long swiotlb_nr_tbl(void);
 unsigned long swiotlb_size_or_default(void);
 extern int swiotlb_late_init_with_tbl(char *tlb, unsigned long nslabs);
+extern void __init swiotlb_update_mem_attributes(void);
 
 /*
  * Enumeration for sync targets
diff --git a/init/main.c b/init/main.c
index df58a41..5013cbc 100644
--- a/init/main.c
+++ b/init/main.c
@@ -488,6 +488,8 @@ void __init __weak thread_stack_cache_init(void)
 }
 #endif
 
+void __init __weak mem_encrypt_init(void) { }
+
 /*
  * Set up kernel memory allocators
  */
@@ -640,6 +642,14 @@ asmlinkage __visible void __init start_kernel(void)
 	 */
 	locking_selftest();
 
+	/*
+	 * This needs to be called before any devices perform DMA
+	 * operations that might use the SWIOTLB bounce buffers. It will
+	 * mark the bounce buffers as decrypted so that their usage will
+	 * not cause "plain-text" data to be decrypted when accessed.
+	 */
+	mem_encrypt_init();
+
 #ifdef CONFIG_BLK_DEV_INITRD
 	if (initrd_start && !initrd_below_start_ok &&
 	    page_to_pfn(virt_to_page((void *)initrd_start)) < min_low_pfn) {
diff --git a/lib/swiotlb.c b/lib/swiotlb.c
index a8d74a7..04ac91a 100644
--- a/lib/swiotlb.c
+++ b/lib/swiotlb.c
@@ -30,6 +30,7 @@
 #include <linux/highmem.h>
 #include <linux/gfp.h>
 #include <linux/scatterlist.h>
+#include <linux/mem_encrypt.h>
 
 #include <asm/io.h>
 #include <asm/dma.h>
@@ -155,6 +156,15 @@ unsigned long swiotlb_size_or_default(void)
 	return size ? size : (IO_TLB_DEFAULT_SIZE);
 }
 
+void __weak swiotlb_set_mem_attributes(void *vaddr, unsigned long size) { }
+
+/* For swiotlb, clear memory encryption mask from dma addresses */
+static dma_addr_t swiotlb_phys_to_dma(struct device *hwdev,
+				      phys_addr_t address)
+{
+	return __sme_clr(phys_to_dma(hwdev, address));
+}
+
 /* Note that this doesn't work with highmem page */
 static dma_addr_t swiotlb_virt_to_bus(struct device *hwdev,
 				      volatile void *address)
@@ -183,6 +193,31 @@ void swiotlb_print_info(void)
 	       bytes >> 20, vstart, vend - 1);
 }
 
+/*
+ * Early SWIOTLB allocation may be too early to allow an architecture to
+ * perform the desired operations.  This function allows the architecture to
+ * call SWIOTLB when the operations are possible.  It needs to be called
+ * before the SWIOTLB memory is used.
+ */
+void __init swiotlb_update_mem_attributes(void)
+{
+	void *vaddr;
+	unsigned long bytes;
+
+	if (no_iotlb_memory || late_alloc)
+		return;
+
+	vaddr = phys_to_virt(io_tlb_start);
+	bytes = PAGE_ALIGN(io_tlb_nslabs << IO_TLB_SHIFT);
+	swiotlb_set_mem_attributes(vaddr, bytes);
+	memset(vaddr, 0, bytes);
+
+	vaddr = phys_to_virt(io_tlb_overflow_buffer);
+	bytes = PAGE_ALIGN(io_tlb_overflow);
+	swiotlb_set_mem_attributes(vaddr, bytes);
+	memset(vaddr, 0, bytes);
+}
+
 int __init swiotlb_init_with_tbl(char *tlb, unsigned long nslabs, int verbose)
 {
 	void *v_overflow_buffer;
@@ -320,6 +355,7 @@ int __init swiotlb_init_with_tbl(char *tlb, unsigned long nslabs, int verbose)
 	io_tlb_start = virt_to_phys(tlb);
 	io_tlb_end = io_tlb_start + bytes;
 
+	swiotlb_set_mem_attributes(tlb, bytes);
 	memset(tlb, 0, bytes);
 
 	/*
@@ -330,6 +366,8 @@ int __init swiotlb_init_with_tbl(char *tlb, unsigned long nslabs, int verbose)
 	if (!v_overflow_buffer)
 		goto cleanup2;
 
+	swiotlb_set_mem_attributes(v_overflow_buffer, io_tlb_overflow);
+	memset(v_overflow_buffer, 0, io_tlb_overflow);
 	io_tlb_overflow_buffer = virt_to_phys(v_overflow_buffer);
 
 	/*
@@ -581,7 +619,7 @@ phys_addr_t swiotlb_tbl_map_single(struct device *hwdev,
 		return SWIOTLB_MAP_ERROR;
 	}
 
-	start_dma_addr = phys_to_dma(hwdev, io_tlb_start);
+	start_dma_addr = swiotlb_phys_to_dma(hwdev, io_tlb_start);
 	return swiotlb_tbl_map_single(hwdev, start_dma_addr, phys, size,
 				      dir, attrs);
 }
@@ -702,7 +740,7 @@ void swiotlb_tbl_sync_single(struct device *hwdev, phys_addr_t tlb_addr,
 			goto err_warn;
 
 		ret = phys_to_virt(paddr);
-		dev_addr = phys_to_dma(hwdev, paddr);
+		dev_addr = swiotlb_phys_to_dma(hwdev, paddr);
 
 		/* Confirm address can be DMA'd by device */
 		if (dev_addr + size - 1 > dma_mask) {
@@ -812,10 +850,10 @@ dma_addr_t swiotlb_map_page(struct device *dev, struct page *page,
 	map = map_single(dev, phys, size, dir, attrs);
 	if (map == SWIOTLB_MAP_ERROR) {
 		swiotlb_full(dev, size, dir, 1);
-		return phys_to_dma(dev, io_tlb_overflow_buffer);
+		return swiotlb_phys_to_dma(dev, io_tlb_overflow_buffer);
 	}
 
-	dev_addr = phys_to_dma(dev, map);
+	dev_addr = swiotlb_phys_to_dma(dev, map);
 
 	/* Ensure that the address returned is DMA'ble */
 	if (dma_capable(dev, dev_addr, size))
@@ -824,7 +862,7 @@ dma_addr_t swiotlb_map_page(struct device *dev, struct page *page,
 	attrs |= DMA_ATTR_SKIP_CPU_SYNC;
 	swiotlb_tbl_unmap_single(dev, map, size, dir, attrs);
 
-	return phys_to_dma(dev, io_tlb_overflow_buffer);
+	return swiotlb_phys_to_dma(dev, io_tlb_overflow_buffer);
 }
 EXPORT_SYMBOL_GPL(swiotlb_map_page);
 
@@ -958,7 +996,7 @@ void swiotlb_unmap_page(struct device *hwdev, dma_addr_t dev_addr,
 				sg_dma_len(sgl) = 0;
 				return 0;
 			}
-			sg->dma_address = phys_to_dma(hwdev, map);
+			sg->dma_address = swiotlb_phys_to_dma(hwdev, map);
 		} else
 			sg->dma_address = dev_addr;
 		sg_dma_len(sg) = sg->length;
@@ -1026,7 +1064,7 @@ void swiotlb_unmap_page(struct device *hwdev, dma_addr_t dev_addr,
 int
 swiotlb_dma_mapping_error(struct device *hwdev, dma_addr_t dma_addr)
 {
-	return (dma_addr == phys_to_dma(hwdev, io_tlb_overflow_buffer));
+	return (dma_addr == swiotlb_phys_to_dma(hwdev, io_tlb_overflow_buffer));
 }
 EXPORT_SYMBOL(swiotlb_dma_mapping_error);
 
@@ -1039,6 +1077,6 @@ void swiotlb_unmap_page(struct device *hwdev, dma_addr_t dev_addr,
 int
 swiotlb_dma_supported(struct device *hwdev, u64 mask)
 {
-	return phys_to_dma(hwdev, io_tlb_end - 1) <= mask;
+	return swiotlb_phys_to_dma(hwdev, io_tlb_end - 1) <= mask;
 }
 EXPORT_SYMBOL(swiotlb_dma_supported);



From thomas.lendacky at amd.com  Tue Jun 27 08:00:36 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Tue, 27 Jun 2017 10:00:36 -0500
Subject: [PATCH v8 25/38] swiotlb: Add warnings for use of bounce buffers
 with SME
In-Reply-To: <20170627145607.15908.26571.stgit@tlendack-t1.amdoffice.net>
References: <20170627145607.15908.26571.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170627150036.15908.80988.stgit@tlendack-t1.amdoffice.net>

Add warnings to let the user know when bounce buffers are being used for
DMA when SME is active.  Since the bounce buffers are not in encrypted
memory, these notifications are to allow the user to determine some
appropriate action - if necessary.  Actions can range from utilizing an
IOMMU, replacing the device with another device that can support 64-bit
DMA, ignoring the message if the device isn't used much, etc.

Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 include/linux/dma-mapping.h |   13 +++++++++++++
 lib/swiotlb.c               |    3 +++
 2 files changed, 16 insertions(+)

diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h
index 4f3eece..a156c40 100644
--- a/include/linux/dma-mapping.h
+++ b/include/linux/dma-mapping.h
@@ -10,6 +10,7 @@
 #include <linux/scatterlist.h>
 #include <linux/kmemcheck.h>
 #include <linux/bug.h>
+#include <linux/mem_encrypt.h>
 
 /**
  * List of possible attributes associated with a DMA mapping. The semantics
@@ -554,6 +555,12 @@ static inline int dma_mapping_error(struct device *dev, dma_addr_t dma_addr)
 #endif
 }
 
+static inline void dma_check_mask(struct device *dev, u64 mask)
+{
+	if (sme_active() && (mask < (((u64)sme_get_me_mask() << 1) - 1)))
+		dev_warn(dev, "SME is active, device will require DMA bounce buffers\n");
+}
+
 #ifndef HAVE_ARCH_DMA_SUPPORTED
 static inline int dma_supported(struct device *dev, u64 mask)
 {
@@ -577,6 +584,9 @@ static inline int dma_set_mask(struct device *dev, u64 mask)
 
 	if (!dev->dma_mask || !dma_supported(dev, mask))
 		return -EIO;
+
+	dma_check_mask(dev, mask);
+
 	*dev->dma_mask = mask;
 	return 0;
 }
@@ -596,6 +606,9 @@ static inline int dma_set_coherent_mask(struct device *dev, u64 mask)
 {
 	if (!dma_supported(dev, mask))
 		return -EIO;
+
+	dma_check_mask(dev, mask);
+
 	dev->coherent_dma_mask = mask;
 	return 0;
 }
diff --git a/lib/swiotlb.c b/lib/swiotlb.c
index 04ac91a..8c6c83e 100644
--- a/lib/swiotlb.c
+++ b/lib/swiotlb.c
@@ -507,6 +507,9 @@ phys_addr_t swiotlb_tbl_map_single(struct device *hwdev,
 	if (no_iotlb_memory)
 		panic("Can not allocate SWIOTLB buffer earlier and can't now provide you with the DMA bounce buffer");
 
+	if (sme_active())
+		pr_warn_once("SME is active and system is using DMA bounce buffers\n");
+
 	mask = dma_get_seg_boundary(hwdev);
 
 	tbl_dma_addr &= mask;



From thomas.lendacky at amd.com  Tue Jun 27 08:00:45 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Tue, 27 Jun 2017 10:00:45 -0500
Subject: [PATCH v8 26/38] x86/CPU/AMD: Make the microcode level available
 earlier in the boot
In-Reply-To: <20170627145607.15908.26571.stgit@tlendack-t1.amdoffice.net>
References: <20170627145607.15908.26571.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170627150045.15908.23856.stgit@tlendack-t1.amdoffice.net>

Move the setting of the cpuinfo_x86.microcode field from amd_init() to
early_amd_init() so that it is available earlier in the boot process. This
avoids having to read MSR_AMD64_PATCH_LEVEL directly during early boot.

Reviewed-by: Borislav Petkov <bp at suse.de>
Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/kernel/cpu/amd.c |    8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/arch/x86/kernel/cpu/amd.c b/arch/x86/kernel/cpu/amd.c
index 5bdcbd4..fdcf305 100644
--- a/arch/x86/kernel/cpu/amd.c
+++ b/arch/x86/kernel/cpu/amd.c
@@ -547,8 +547,12 @@ static void bsp_init_amd(struct cpuinfo_x86 *c)
 
 static void early_init_amd(struct cpuinfo_x86 *c)
 {
+	u32 dummy;
+
 	early_init_amd_mc(c);
 
+	rdmsr_safe(MSR_AMD64_PATCH_LEVEL, &c->microcode, &dummy);
+
 	/*
 	 * c->x86_power is 8000_0007 edx. Bit 8 is TSC runs at constant rate
 	 * with P/T states and does not stop in deep C-states
@@ -746,8 +750,6 @@ static void init_amd_bd(struct cpuinfo_x86 *c)
 
 static void init_amd(struct cpuinfo_x86 *c)
 {
-	u32 dummy;
-
 	early_init_amd(c);
 
 	/*
@@ -809,8 +811,6 @@ static void init_amd(struct cpuinfo_x86 *c)
 	if (c->x86 > 0x11)
 		set_cpu_cap(c, X86_FEATURE_ARAT);
 
-	rdmsr_safe(MSR_AMD64_PATCH_LEVEL, &c->microcode, &dummy);
-
 	/* 3DNow or LM implies PREFETCHW */
 	if (!cpu_has(c, X86_FEATURE_3DNOWPREFETCH))
 		if (cpu_has(c, X86_FEATURE_3DNOW) || cpu_has(c, X86_FEATURE_LM))



From thomas.lendacky at amd.com  Tue Jun 27 08:01:10 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Tue, 27 Jun 2017 10:01:10 -0500
Subject: [PATCH v8 28/38] x86, realmode: Check for memory encryption on the APs
In-Reply-To: <20170627145607.15908.26571.stgit@tlendack-t1.amdoffice.net>
References: <20170627145607.15908.26571.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170627150110.15908.65796.stgit@tlendack-t1.amdoffice.net>

Add support to check if memory encryption is active in the kernel and that
it has been enabled on the AP. If memory encryption is active in the kernel
but has not been enabled on the AP, then set the memory encryption bit (bit
23) of MSR_K8_SYSCFG to enable memory encryption on that AP and allow the
AP to continue start up.

Reviewed-by: Borislav Petkov <bp at suse.de>
Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/include/asm/realmode.h      |   12 ++++++++++++
 arch/x86/realmode/init.c             |    4 ++++
 arch/x86/realmode/rm/trampoline_64.S |   24 ++++++++++++++++++++++++
 3 files changed, 40 insertions(+)

diff --git a/arch/x86/include/asm/realmode.h b/arch/x86/include/asm/realmode.h
index 230e190..90d9152 100644
--- a/arch/x86/include/asm/realmode.h
+++ b/arch/x86/include/asm/realmode.h
@@ -1,6 +1,15 @@
 #ifndef _ARCH_X86_REALMODE_H
 #define _ARCH_X86_REALMODE_H
 
+/*
+ * Flag bit definitions for use with the flags field of the trampoline header
+ * in the CONFIG_X86_64 variant.
+ */
+#define TH_FLAGS_SME_ACTIVE_BIT		0
+#define TH_FLAGS_SME_ACTIVE		BIT(TH_FLAGS_SME_ACTIVE_BIT)
+
+#ifndef __ASSEMBLY__
+
 #include <linux/types.h>
 #include <asm/io.h>
 
@@ -38,6 +47,7 @@ struct trampoline_header {
 	u64 start;
 	u64 efer;
 	u32 cr4;
+	u32 flags;
 #endif
 };
 
@@ -69,4 +79,6 @@ static inline size_t real_mode_size_needed(void)
 void set_real_mode_mem(phys_addr_t mem, size_t size);
 void reserve_real_mode(void);
 
+#endif /* __ASSEMBLY__ */
+
 #endif /* _ARCH_X86_REALMODE_H */
diff --git a/arch/x86/realmode/init.c b/arch/x86/realmode/init.c
index d6ddc7e..1f71980 100644
--- a/arch/x86/realmode/init.c
+++ b/arch/x86/realmode/init.c
@@ -108,6 +108,10 @@ static void __init setup_real_mode(void)
 	trampoline_cr4_features = &trampoline_header->cr4;
 	*trampoline_cr4_features = mmu_cr4_features;
 
+	trampoline_header->flags = 0;
+	if (sme_active())
+		trampoline_header->flags |= TH_FLAGS_SME_ACTIVE;
+
 	trampoline_pgd = (u64 *) __va(real_mode_header->trampoline_pgd);
 	trampoline_pgd[0] = trampoline_pgd_entry.pgd;
 	trampoline_pgd[511] = init_top_pgt[511].pgd;
diff --git a/arch/x86/realmode/rm/trampoline_64.S b/arch/x86/realmode/rm/trampoline_64.S
index dac7b20..614fd70 100644
--- a/arch/x86/realmode/rm/trampoline_64.S
+++ b/arch/x86/realmode/rm/trampoline_64.S
@@ -30,6 +30,7 @@
 #include <asm/msr.h>
 #include <asm/segment.h>
 #include <asm/processor-flags.h>
+#include <asm/realmode.h>
 #include "realmode.h"
 
 	.text
@@ -92,6 +93,28 @@ ENTRY(startup_32)
 	movl	%edx, %fs
 	movl	%edx, %gs
 
+	/*
+	 * Check for memory encryption support. This is a safety net in
+	 * case BIOS hasn't done the necessary step of setting the bit in
+	 * the MSR for this AP. If SME is active and we've gotten this far
+	 * then it is safe for us to set the MSR bit and continue. If we
+	 * don't we'll eventually crash trying to execute encrypted
+	 * instructions.
+	 */
+	bt	$TH_FLAGS_SME_ACTIVE_BIT, pa_tr_flags
+	jnc	.Ldone
+	movl	$MSR_K8_SYSCFG, %ecx
+	rdmsr
+	bts	$MSR_K8_SYSCFG_MEM_ENCRYPT_BIT, %eax
+	jc	.Ldone
+
+	/*
+	 * Memory encryption is enabled but the SME enable bit for this
+	 * CPU has has not been set.  It is safe to set it, so do so.
+	 */
+	wrmsr
+.Ldone:
+
 	movl	pa_tr_cr4, %eax
 	movl	%eax, %cr4		# Enable PAE mode
 
@@ -147,6 +170,7 @@ GLOBAL(trampoline_header)
 	tr_start:		.space	8
 	GLOBAL(tr_efer)		.space	8
 	GLOBAL(tr_cr4)		.space	4
+	GLOBAL(tr_flags)	.space	4
 END(trampoline_header)
 
 #include "trampoline_common.S"



From thomas.lendacky at amd.com  Tue Jun 27 08:01:20 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Tue, 27 Jun 2017 10:01:20 -0500
Subject: [PATCH v8 29/38] x86, drm,
 fbdev: Do not specify encrypted memory for video mappings
In-Reply-To: <20170627145607.15908.26571.stgit@tlendack-t1.amdoffice.net>
References: <20170627145607.15908.26571.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170627150120.15908.89995.stgit@tlendack-t1.amdoffice.net>

Since video memory needs to be accessed decrypted, be sure that the
memory encryption mask is not set for the video ranges.

Reviewed-by: Borislav Petkov <bp at suse.de>
Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/include/asm/vga.h       |   14 +++++++++++++-
 arch/x86/mm/pageattr.c           |    2 ++
 drivers/gpu/drm/drm_gem.c        |    2 ++
 drivers/gpu/drm/drm_vm.c         |    4 ++++
 drivers/gpu/drm/ttm/ttm_bo_vm.c  |    7 +++++--
 drivers/gpu/drm/udl/udl_fb.c     |    4 ++++
 drivers/video/fbdev/core/fbmem.c |   12 ++++++++++++
 7 files changed, 42 insertions(+), 3 deletions(-)

diff --git a/arch/x86/include/asm/vga.h b/arch/x86/include/asm/vga.h
index c4b9dc2..9f42bee 100644
--- a/arch/x86/include/asm/vga.h
+++ b/arch/x86/include/asm/vga.h
@@ -7,12 +7,24 @@
 #ifndef _ASM_X86_VGA_H
 #define _ASM_X86_VGA_H
 
+#include <asm/set_memory.h>
+
 /*
  *	On the PC, we can just recalculate addresses and then
  *	access the videoram directly without any black magic.
+ *	To support memory encryption however, we need to access
+ *	the videoram as decrypted memory.
  */
 
-#define VGA_MAP_MEM(x, s) (unsigned long)phys_to_virt(x)
+#define VGA_MAP_MEM(x, s)					\
+({								\
+	unsigned long start = (unsigned long)phys_to_virt(x);	\
+								\
+	if (IS_ENABLED(CONFIG_AMD_MEM_ENCRYPT))			\
+		set_memory_decrypted(start, (s) >> PAGE_SHIFT);	\
+								\
+	start;							\
+})
 
 #define vga_readb(x) (*(x))
 #define vga_writeb(x, y) (*(y) = (x))
diff --git a/arch/x86/mm/pageattr.c b/arch/x86/mm/pageattr.c
index d9e09fb..13fc5db 100644
--- a/arch/x86/mm/pageattr.c
+++ b/arch/x86/mm/pageattr.c
@@ -1825,11 +1825,13 @@ int set_memory_encrypted(unsigned long addr, int numpages)
 {
 	return __set_memory_enc_dec(addr, numpages, true);
 }
+EXPORT_SYMBOL_GPL(set_memory_encrypted);
 
 int set_memory_decrypted(unsigned long addr, int numpages)
 {
 	return __set_memory_enc_dec(addr, numpages, false);
 }
+EXPORT_SYMBOL_GPL(set_memory_decrypted);
 
 int set_pages_uc(struct page *page, int numpages)
 {
diff --git a/drivers/gpu/drm/drm_gem.c b/drivers/gpu/drm/drm_gem.c
index b1e28c9..019f48c 100644
--- a/drivers/gpu/drm/drm_gem.c
+++ b/drivers/gpu/drm/drm_gem.c
@@ -36,6 +36,7 @@
 #include <linux/pagemap.h>
 #include <linux/shmem_fs.h>
 #include <linux/dma-buf.h>
+#include <linux/mem_encrypt.h>
 #include <drm/drmP.h>
 #include <drm/drm_vma_manager.h>
 #include <drm/drm_gem.h>
@@ -928,6 +929,7 @@ int drm_gem_mmap_obj(struct drm_gem_object *obj, unsigned long obj_size,
 	vma->vm_ops = dev->driver->gem_vm_ops;
 	vma->vm_private_data = obj;
 	vma->vm_page_prot = pgprot_writecombine(vm_get_page_prot(vma->vm_flags));
+	vma->vm_page_prot = pgprot_decrypted(vma->vm_page_prot);
 
 	/* Take a ref for this mapping of the object, so that the fault
 	 * handler can dereference the mmap offset's pointer to the object.
diff --git a/drivers/gpu/drm/drm_vm.c b/drivers/gpu/drm/drm_vm.c
index 1170b32..ed4bcbf 100644
--- a/drivers/gpu/drm/drm_vm.c
+++ b/drivers/gpu/drm/drm_vm.c
@@ -40,6 +40,7 @@
 #include <linux/efi.h>
 #include <linux/slab.h>
 #endif
+#include <linux/mem_encrypt.h>
 #include <asm/pgtable.h>
 #include "drm_internal.h"
 #include "drm_legacy.h"
@@ -58,6 +59,9 @@ static pgprot_t drm_io_prot(struct drm_local_map *map,
 {
 	pgprot_t tmp = vm_get_page_prot(vma->vm_flags);
 
+	/* We don't want graphics memory to be mapped encrypted */
+	tmp = pgprot_decrypted(tmp);
+
 #if defined(__i386__) || defined(__x86_64__) || defined(__powerpc__)
 	if (map->type == _DRM_REGISTERS && !(map->flags & _DRM_WRITE_COMBINING))
 		tmp = pgprot_noncached(tmp);
diff --git a/drivers/gpu/drm/ttm/ttm_bo_vm.c b/drivers/gpu/drm/ttm/ttm_bo_vm.c
index 9f53df9..622dab6 100644
--- a/drivers/gpu/drm/ttm/ttm_bo_vm.c
+++ b/drivers/gpu/drm/ttm/ttm_bo_vm.c
@@ -39,6 +39,7 @@
 #include <linux/rbtree.h>
 #include <linux/module.h>
 #include <linux/uaccess.h>
+#include <linux/mem_encrypt.h>
 
 #define TTM_BO_VM_NUM_PREFAULT 16
 
@@ -230,9 +231,11 @@ static int ttm_bo_vm_fault(struct vm_fault *vmf)
 	 * first page.
 	 */
 	for (i = 0; i < TTM_BO_VM_NUM_PREFAULT; ++i) {
-		if (bo->mem.bus.is_iomem)
+		if (bo->mem.bus.is_iomem) {
+			/* Iomem should not be marked encrypted */
+			cvma.vm_page_prot = pgprot_decrypted(cvma.vm_page_prot);
 			pfn = bdev->driver->io_mem_pfn(bo, page_offset);
-		else {
+		} else {
 			page = ttm->pages[page_offset];
 			if (unlikely(!page && i == 0)) {
 				retval = VM_FAULT_OOM;
diff --git a/drivers/gpu/drm/udl/udl_fb.c b/drivers/gpu/drm/udl/udl_fb.c
index 4a65003..92e1690 100644
--- a/drivers/gpu/drm/udl/udl_fb.c
+++ b/drivers/gpu/drm/udl/udl_fb.c
@@ -14,6 +14,7 @@
 #include <linux/slab.h>
 #include <linux/fb.h>
 #include <linux/dma-buf.h>
+#include <linux/mem_encrypt.h>
 
 #include <drm/drmP.h>
 #include <drm/drm_crtc.h>
@@ -169,6 +170,9 @@ static int udl_fb_mmap(struct fb_info *info, struct vm_area_struct *vma)
 	pr_notice("mmap() framebuffer addr:%lu size:%lu\n",
 		  pos, size);
 
+	/* We don't want the framebuffer to be mapped encrypted */
+	vma->vm_page_prot = pgprot_decrypted(vma->vm_page_prot);
+
 	while (size > 0) {
 		page = vmalloc_to_pfn((void *)pos);
 		if (remap_pfn_range(vma, start, page, PAGE_SIZE, PAGE_SHARED))
diff --git a/drivers/video/fbdev/core/fbmem.c b/drivers/video/fbdev/core/fbmem.c
index 069fe79..b5e7c33 100644
--- a/drivers/video/fbdev/core/fbmem.c
+++ b/drivers/video/fbdev/core/fbmem.c
@@ -32,6 +32,7 @@
 #include <linux/device.h>
 #include <linux/efi.h>
 #include <linux/fb.h>
+#include <linux/mem_encrypt.h>
 
 #include <asm/fb.h>
 
@@ -1405,6 +1406,12 @@ static long fb_compat_ioctl(struct file *file, unsigned int cmd,
 	mutex_lock(&info->mm_lock);
 	if (fb->fb_mmap) {
 		int res;
+
+		/*
+		 * The framebuffer needs to be accessed decrypted, be sure
+		 * SME protection is removed ahead of the call
+		 */
+		vma->vm_page_prot = pgprot_decrypted(vma->vm_page_prot);
 		res = fb->fb_mmap(info, vma);
 		mutex_unlock(&info->mm_lock);
 		return res;
@@ -1430,6 +1437,11 @@ static long fb_compat_ioctl(struct file *file, unsigned int cmd,
 	mutex_unlock(&info->mm_lock);
 
 	vma->vm_page_prot = vm_get_page_prot(vma->vm_flags);
+	/*
+	 * The framebuffer needs to be accessed decrypted, be sure
+	 * SME protection is removed
+	 */
+	vma->vm_page_prot = pgprot_decrypted(vma->vm_page_prot);
 	fb_pgprotect(file, vma, start);
 
 	return vm_iomap_memory(vma, start, len);



From thomas.lendacky at amd.com  Tue Jun 27 08:01:31 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Tue, 27 Jun 2017 10:01:31 -0500
Subject: [PATCH v8 30/38] kvm: x86: svm: Support Secure Memory Encryption
 within KVM
In-Reply-To: <20170627145607.15908.26571.stgit@tlendack-t1.amdoffice.net>
References: <20170627145607.15908.26571.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170627150131.15908.22634.stgit@tlendack-t1.amdoffice.net>

Update the KVM support to work with SME. The VMCB has a number of fields
where physical addresses are used and these addresses must contain the
memory encryption mask in order to properly access the encrypted memory.
Also, use the memory encryption mask when creating and using the nested
page tables.

Reviewed-by: Borislav Petkov <bp at suse.de>
Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/include/asm/kvm_host.h |    2 +-
 arch/x86/kvm/mmu.c              |   12 ++++++++----
 arch/x86/kvm/mmu.h              |    2 +-
 arch/x86/kvm/svm.c              |   35 ++++++++++++++++++-----------------
 arch/x86/kvm/vmx.c              |    3 ++-
 arch/x86/kvm/x86.c              |    3 ++-
 6 files changed, 32 insertions(+), 25 deletions(-)

diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index 695605e..6d1267f 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -1069,7 +1069,7 @@ struct kvm_arch_async_pf {
 void kvm_mmu_uninit_vm(struct kvm *kvm);
 void kvm_mmu_set_mask_ptes(u64 user_mask, u64 accessed_mask,
 		u64 dirty_mask, u64 nx_mask, u64 x_mask, u64 p_mask,
-		u64 acc_track_mask);
+		u64 acc_track_mask, u64 me_mask);
 
 void kvm_mmu_reset_context(struct kvm_vcpu *vcpu);
 void kvm_mmu_slot_remove_write_access(struct kvm *kvm,
diff --git a/arch/x86/kvm/mmu.c b/arch/x86/kvm/mmu.c
index cb82259..e85888c 100644
--- a/arch/x86/kvm/mmu.c
+++ b/arch/x86/kvm/mmu.c
@@ -107,7 +107,7 @@ enum {
 	(((address) >> PT32_LEVEL_SHIFT(level)) & ((1 << PT32_LEVEL_BITS) - 1))
 
 
-#define PT64_BASE_ADDR_MASK (((1ULL << 52) - 1) & ~(u64)(PAGE_SIZE-1))
+#define PT64_BASE_ADDR_MASK __sme_clr((((1ULL << 52) - 1) & ~(u64)(PAGE_SIZE-1)))
 #define PT64_DIR_BASE_ADDR_MASK \
 	(PT64_BASE_ADDR_MASK & ~((1ULL << (PAGE_SHIFT + PT64_LEVEL_BITS)) - 1))
 #define PT64_LVL_ADDR_MASK(level) \
@@ -125,7 +125,7 @@ enum {
 					    * PT32_LEVEL_BITS))) - 1))
 
 #define PT64_PERM_MASK (PT_PRESENT_MASK | PT_WRITABLE_MASK | shadow_user_mask \
-			| shadow_x_mask | shadow_nx_mask)
+			| shadow_x_mask | shadow_nx_mask | shadow_me_mask)
 
 #define ACC_EXEC_MASK    1
 #define ACC_WRITE_MASK   PT_WRITABLE_MASK
@@ -184,6 +184,7 @@ struct kvm_shadow_walk_iterator {
 static u64 __read_mostly shadow_dirty_mask;
 static u64 __read_mostly shadow_mmio_mask;
 static u64 __read_mostly shadow_present_mask;
+static u64 __read_mostly shadow_me_mask;
 
 /*
  * The mask/value to distinguish a PTE that has been marked not-present for
@@ -317,7 +318,7 @@ static bool check_mmio_spte(struct kvm_vcpu *vcpu, u64 spte)
 
 void kvm_mmu_set_mask_ptes(u64 user_mask, u64 accessed_mask,
 		u64 dirty_mask, u64 nx_mask, u64 x_mask, u64 p_mask,
-		u64 acc_track_mask)
+		u64 acc_track_mask, u64 me_mask)
 {
 	if (acc_track_mask != 0)
 		acc_track_mask |= SPTE_SPECIAL_MASK;
@@ -330,6 +331,7 @@ void kvm_mmu_set_mask_ptes(u64 user_mask, u64 accessed_mask,
 	shadow_present_mask = p_mask;
 	shadow_acc_track_mask = acc_track_mask;
 	WARN_ON(shadow_accessed_mask != 0 && shadow_acc_track_mask != 0);
+	shadow_me_mask = me_mask;
 }
 EXPORT_SYMBOL_GPL(kvm_mmu_set_mask_ptes);
 
@@ -2398,7 +2400,8 @@ static void link_shadow_page(struct kvm_vcpu *vcpu, u64 *sptep,
 	BUILD_BUG_ON(VMX_EPT_WRITABLE_MASK != PT_WRITABLE_MASK);
 
 	spte = __pa(sp->spt) | shadow_present_mask | PT_WRITABLE_MASK |
-	       shadow_user_mask | shadow_x_mask | shadow_accessed_mask;
+	       shadow_user_mask | shadow_x_mask | shadow_accessed_mask |
+	       shadow_me_mask;
 
 	mmu_spte_set(sptep, spte);
 
@@ -2700,6 +2703,7 @@ static int set_spte(struct kvm_vcpu *vcpu, u64 *sptep,
 		pte_access &= ~ACC_WRITE_MASK;
 
 	spte |= (u64)pfn << PAGE_SHIFT;
+	spte |= shadow_me_mask;
 
 	if (pte_access & ACC_WRITE_MASK) {
 
diff --git a/arch/x86/kvm/mmu.h b/arch/x86/kvm/mmu.h
index 330bf3a..08b779d 100644
--- a/arch/x86/kvm/mmu.h
+++ b/arch/x86/kvm/mmu.h
@@ -48,7 +48,7 @@
 
 static inline u64 rsvd_bits(int s, int e)
 {
-	return ((1ULL << (e - s + 1)) - 1) << s;
+	return __sme_clr(((1ULL << (e - s + 1)) - 1) << s);
 }
 
 void kvm_mmu_set_mmio_spte_mask(u64 mmio_mask);
diff --git a/arch/x86/kvm/svm.c b/arch/x86/kvm/svm.c
index ba9891a..d2e9fca 100644
--- a/arch/x86/kvm/svm.c
+++ b/arch/x86/kvm/svm.c
@@ -1138,9 +1138,9 @@ static void avic_init_vmcb(struct vcpu_svm *svm)
 {
 	struct vmcb *vmcb = svm->vmcb;
 	struct kvm_arch *vm_data = &svm->vcpu.kvm->arch;
-	phys_addr_t bpa = page_to_phys(svm->avic_backing_page);
-	phys_addr_t lpa = page_to_phys(vm_data->avic_logical_id_table_page);
-	phys_addr_t ppa = page_to_phys(vm_data->avic_physical_id_table_page);
+	phys_addr_t bpa = __sme_set(page_to_phys(svm->avic_backing_page));
+	phys_addr_t lpa = __sme_set(page_to_phys(vm_data->avic_logical_id_table_page));
+	phys_addr_t ppa = __sme_set(page_to_phys(vm_data->avic_physical_id_table_page));
 
 	vmcb->control.avic_backing_page = bpa & AVIC_HPA_MASK;
 	vmcb->control.avic_logical_id = lpa & AVIC_HPA_MASK;
@@ -1203,8 +1203,8 @@ static void init_vmcb(struct vcpu_svm *svm)
 		set_intercept(svm, INTERCEPT_MWAIT);
 	}
 
-	control->iopm_base_pa = iopm_base;
-	control->msrpm_base_pa = __pa(svm->msrpm);
+	control->iopm_base_pa = __sme_set(iopm_base);
+	control->msrpm_base_pa = __sme_set(__pa(svm->msrpm));
 	control->int_ctl = V_INTR_MASKING_MASK;
 
 	init_seg(&save->es);
@@ -1338,9 +1338,9 @@ static int avic_init_backing_page(struct kvm_vcpu *vcpu)
 		return -EINVAL;
 
 	new_entry = READ_ONCE(*entry);
-	new_entry = (page_to_phys(svm->avic_backing_page) &
-		     AVIC_PHYSICAL_ID_ENTRY_BACKING_PAGE_MASK) |
-		     AVIC_PHYSICAL_ID_ENTRY_VALID_MASK;
+	new_entry = __sme_set((page_to_phys(svm->avic_backing_page) &
+			      AVIC_PHYSICAL_ID_ENTRY_BACKING_PAGE_MASK) |
+			      AVIC_PHYSICAL_ID_ENTRY_VALID_MASK);
 	WRITE_ONCE(*entry, new_entry);
 
 	svm->avic_physical_id_cache = entry;
@@ -1608,7 +1608,7 @@ static struct kvm_vcpu *svm_create_vcpu(struct kvm *kvm, unsigned int id)
 
 	svm->vmcb = page_address(page);
 	clear_page(svm->vmcb);
-	svm->vmcb_pa = page_to_pfn(page) << PAGE_SHIFT;
+	svm->vmcb_pa = __sme_set(page_to_pfn(page) << PAGE_SHIFT);
 	svm->asid_generation = 0;
 	init_vmcb(svm);
 
@@ -1636,7 +1636,7 @@ static void svm_free_vcpu(struct kvm_vcpu *vcpu)
 {
 	struct vcpu_svm *svm = to_svm(vcpu);
 
-	__free_page(pfn_to_page(svm->vmcb_pa >> PAGE_SHIFT));
+	__free_page(pfn_to_page(__sme_clr(svm->vmcb_pa) >> PAGE_SHIFT));
 	__free_pages(virt_to_page(svm->msrpm), MSRPM_ALLOC_ORDER);
 	__free_page(virt_to_page(svm->nested.hsave));
 	__free_pages(virt_to_page(svm->nested.msrpm), MSRPM_ALLOC_ORDER);
@@ -2303,7 +2303,7 @@ static u64 nested_svm_get_tdp_pdptr(struct kvm_vcpu *vcpu, int index)
 	u64 pdpte;
 	int ret;
 
-	ret = kvm_vcpu_read_guest_page(vcpu, gpa_to_gfn(cr3), &pdpte,
+	ret = kvm_vcpu_read_guest_page(vcpu, gpa_to_gfn(__sme_clr(cr3)), &pdpte,
 				       offset_in_page(cr3) + index * 8, 8);
 	if (ret)
 		return 0;
@@ -2315,7 +2315,7 @@ static void nested_svm_set_tdp_cr3(struct kvm_vcpu *vcpu,
 {
 	struct vcpu_svm *svm = to_svm(vcpu);
 
-	svm->vmcb->control.nested_cr3 = root;
+	svm->vmcb->control.nested_cr3 = __sme_set(root);
 	mark_dirty(svm->vmcb, VMCB_NPT);
 	svm_flush_tlb(vcpu);
 }
@@ -2803,7 +2803,7 @@ static bool nested_svm_vmrun_msrpm(struct vcpu_svm *svm)
 		svm->nested.msrpm[p] = svm->msrpm[p] | value;
 	}
 
-	svm->vmcb->control.msrpm_base_pa = __pa(svm->nested.msrpm);
+	svm->vmcb->control.msrpm_base_pa = __sme_set(__pa(svm->nested.msrpm));
 
 	return true;
 }
@@ -4435,7 +4435,7 @@ static int svm_ir_list_add(struct vcpu_svm *svm, struct amd_iommu_pi_data *pi)
 	pr_debug("SVM: %s: use GA mode for irq %u\n", __func__,
 		 irq.vector);
 	*svm = to_svm(vcpu);
-	vcpu_info->pi_desc_addr = page_to_phys((*svm)->avic_backing_page);
+	vcpu_info->pi_desc_addr = __sme_set(page_to_phys((*svm)->avic_backing_page));
 	vcpu_info->vector = irq.vector;
 
 	return 0;
@@ -4486,7 +4486,8 @@ static int svm_update_pi_irte(struct kvm *kvm, unsigned int host_irq,
 			struct amd_iommu_pi_data pi;
 
 			/* Try to enable guest_mode in IRTE */
-			pi.base = page_to_phys(svm->avic_backing_page) & AVIC_HPA_MASK;
+			pi.base = __sme_set(page_to_phys(svm->avic_backing_page) &
+					    AVIC_HPA_MASK);
 			pi.ga_tag = AVIC_GATAG(kvm->arch.avic_vm_id,
 						     svm->vcpu.vcpu_id);
 			pi.is_guest_mode = true;
@@ -4911,7 +4912,7 @@ static void svm_set_cr3(struct kvm_vcpu *vcpu, unsigned long root)
 {
 	struct vcpu_svm *svm = to_svm(vcpu);
 
-	svm->vmcb->save.cr3 = root;
+	svm->vmcb->save.cr3 = __sme_set(root);
 	mark_dirty(svm->vmcb, VMCB_CR);
 	svm_flush_tlb(vcpu);
 }
@@ -4920,7 +4921,7 @@ static void set_tdp_cr3(struct kvm_vcpu *vcpu, unsigned long root)
 {
 	struct vcpu_svm *svm = to_svm(vcpu);
 
-	svm->vmcb->control.nested_cr3 = root;
+	svm->vmcb->control.nested_cr3 = __sme_set(root);
 	mark_dirty(svm->vmcb, VMCB_NPT);
 
 	/* Also sync guest cr3 here in case we live migrate */
diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c
index 7dd53fb..53098cd 100644
--- a/arch/x86/kvm/vmx.c
+++ b/arch/x86/kvm/vmx.c
@@ -6452,7 +6452,8 @@ void vmx_enable_tdp(void)
 		enable_ept_ad_bits ? VMX_EPT_DIRTY_BIT : 0ull,
 		0ull, VMX_EPT_EXECUTABLE_MASK,
 		cpu_has_vmx_ept_execute_only() ? 0ull : VMX_EPT_READABLE_MASK,
-		enable_ept_ad_bits ? 0ull : VMX_EPT_RWX_MASK);
+		enable_ept_ad_bits ? 0ull : VMX_EPT_RWX_MASK,
+		0ull);
 
 	ept_set_mmio_spte_mask();
 	kvm_enable_tdp();
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 87d3cb9..559f710 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -54,6 +54,7 @@
 #include <linux/kvm_irqfd.h>
 #include <linux/irqbypass.h>
 #include <linux/sched/stat.h>
+#include <linux/mem_encrypt.h>
 
 #include <trace/events/kvm.h>
 
@@ -6095,7 +6096,7 @@ int kvm_arch_init(void *opaque)
 
 	kvm_mmu_set_mask_ptes(PT_USER_MASK, PT_ACCESSED_MASK,
 			PT_DIRTY_MASK, PT64_NX_MASK, 0,
-			PT_PRESENT_MASK, 0);
+			PT_PRESENT_MASK, 0, sme_me_mask);
 	kvm_timer_init();
 
 	perf_register_guest_info_callbacks(&kvm_guest_cbs);



From thomas.lendacky at amd.com  Tue Jun 27 07:59:22 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Tue, 27 Jun 2017 09:59:22 -0500
Subject: [PATCH v8 18/38] x86/efi: Update EFI pagetable creation to work
 with SME
In-Reply-To: <20170627145607.15908.26571.stgit@tlendack-t1.amdoffice.net>
References: <20170627145607.15908.26571.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170627145922.15908.65516.stgit@tlendack-t1.amdoffice.net>

When SME is active, pagetable entries created for EFI need to have the
encryption mask set as necessary.

When the new pagetable pages are allocated they are mapped encrypted. So,
update the efi_pgt value that will be used in cr3 to include the encryption
mask so that the PGD table can be read successfully. The pagetable mapping
as well as the kernel are also added to the pagetable mapping as encrypted.
All other EFI mappings are mapped decrypted (tables, etc.).

Reviewed-by: Matt Fleming <matt at codeblueprint.co.uk>
Reviewed-by: Borislav Petkov <bp at suse.de>
Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/platform/efi/efi_64.c |   15 +++++++++++----
 1 file changed, 11 insertions(+), 4 deletions(-)

diff --git a/arch/x86/platform/efi/efi_64.c b/arch/x86/platform/efi/efi_64.c
index 9bf72f5..12e8388 100644
--- a/arch/x86/platform/efi/efi_64.c
+++ b/arch/x86/platform/efi/efi_64.c
@@ -327,7 +327,7 @@ void efi_sync_low_kernel_mappings(void)
 
 int __init efi_setup_page_tables(unsigned long pa_memmap, unsigned num_pages)
 {
-	unsigned long pfn, text;
+	unsigned long pfn, text, pf;
 	struct page *page;
 	unsigned npages;
 	pgd_t *pgd;
@@ -335,7 +335,12 @@ int __init efi_setup_page_tables(unsigned long pa_memmap, unsigned num_pages)
 	if (efi_enabled(EFI_OLD_MEMMAP))
 		return 0;
 
-	efi_scratch.efi_pgt = (pgd_t *)__pa(efi_pgd);
+	/*
+	 * Since the PGD is encrypted, set the encryption mask so that when
+	 * this value is loaded into cr3 the PGD will be decrypted during
+	 * the pagetable walk.
+	 */
+	efi_scratch.efi_pgt = (pgd_t *)__sme_pa(efi_pgd);
 	pgd = efi_pgd;
 
 	/*
@@ -345,7 +350,8 @@ int __init efi_setup_page_tables(unsigned long pa_memmap, unsigned num_pages)
 	 * phys_efi_set_virtual_address_map().
 	 */
 	pfn = pa_memmap >> PAGE_SHIFT;
-	if (kernel_map_pages_in_pgd(pgd, pfn, pa_memmap, num_pages, _PAGE_NX | _PAGE_RW)) {
+	pf = _PAGE_NX | _PAGE_RW | _PAGE_ENC;
+	if (kernel_map_pages_in_pgd(pgd, pfn, pa_memmap, num_pages, pf)) {
 		pr_err("Error ident-mapping new memmap (0x%lx)!\n", pa_memmap);
 		return 1;
 	}
@@ -388,7 +394,8 @@ int __init efi_setup_page_tables(unsigned long pa_memmap, unsigned num_pages)
 	text = __pa(_text);
 	pfn = text >> PAGE_SHIFT;
 
-	if (kernel_map_pages_in_pgd(pgd, pfn, text, npages, _PAGE_RW)) {
+	pf = _PAGE_RW | _PAGE_ENC;
+	if (kernel_map_pages_in_pgd(pgd, pfn, text, npages, pf)) {
 		pr_err("Failed to map kernel text 1:1\n");
 		return 1;
 	}



From thomas.lendacky at amd.com  Tue Jun 27 07:59:12 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Tue, 27 Jun 2017 09:59:12 -0500
Subject: [PATCH v8 17/38] efi: Update efi_mem_type() to return an error
 rather than 0
In-Reply-To: <20170627145607.15908.26571.stgit@tlendack-t1.amdoffice.net>
References: <20170627145607.15908.26571.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170627145912.15908.60861.stgit@tlendack-t1.amdoffice.net>

The efi_mem_type() function currently returns a 0, which maps to
EFI_RESERVED_TYPE, if the function is unable to find a memmap entry for
the supplied physical address. Returning EFI_RESERVED_TYPE implies that
a memmap entry exists, when it doesn't.  Instead of returning 0, change
the function to return a negative error value when no memmap entry is
found.

Reviewed-by: Matt Fleming <matt at codeblueprint.co.uk>
Reviewed-by: Borislav Petkov <bp at suse.de>
Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/ia64/kernel/efi.c      |    4 ++--
 arch/x86/platform/efi/efi.c |    6 +++---
 include/linux/efi.h         |    2 +-
 3 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/arch/ia64/kernel/efi.c b/arch/ia64/kernel/efi.c
index 1212956..8141600 100644
--- a/arch/ia64/kernel/efi.c
+++ b/arch/ia64/kernel/efi.c
@@ -757,14 +757,14 @@ static void __init handle_palo(unsigned long phys_addr)
 	return 0;
 }
 
-u32
+int
 efi_mem_type (unsigned long phys_addr)
 {
 	efi_memory_desc_t *md = efi_memory_descriptor(phys_addr);
 
 	if (md)
 		return md->type;
-	return 0;
+	return -EINVAL;
 }
 
 u64
diff --git a/arch/x86/platform/efi/efi.c b/arch/x86/platform/efi/efi.c
index f084d87..6217b23 100644
--- a/arch/x86/platform/efi/efi.c
+++ b/arch/x86/platform/efi/efi.c
@@ -1035,12 +1035,12 @@ void __init efi_enter_virtual_mode(void)
 /*
  * Convenience functions to obtain memory types and attributes
  */
-u32 efi_mem_type(unsigned long phys_addr)
+int efi_mem_type(unsigned long phys_addr)
 {
 	efi_memory_desc_t *md;
 
 	if (!efi_enabled(EFI_MEMMAP))
-		return 0;
+		return -ENOTSUPP;
 
 	for_each_efi_memory_desc(md) {
 		if ((md->phys_addr <= phys_addr) &&
@@ -1048,7 +1048,7 @@ u32 efi_mem_type(unsigned long phys_addr)
 				  (md->num_pages << EFI_PAGE_SHIFT))))
 			return md->type;
 	}
-	return 0;
+	return -EINVAL;
 }
 
 static int __init arch_parse_efi_cmdline(char *str)
diff --git a/include/linux/efi.h b/include/linux/efi.h
index 8e24f09..4e47f78 100644
--- a/include/linux/efi.h
+++ b/include/linux/efi.h
@@ -985,7 +985,7 @@ static inline void efi_esrt_init(void) { }
 extern int efi_config_parse_tables(void *config_tables, int count, int sz,
 				   efi_config_table_type_t *arch_tables);
 extern u64 efi_get_iobase (void);
-extern u32 efi_mem_type (unsigned long phys_addr);
+extern int efi_mem_type(unsigned long phys_addr);
 extern u64 efi_mem_attributes (unsigned long phys_addr);
 extern u64 efi_mem_attribute (unsigned long phys_addr, unsigned long size);
 extern int __init efi_uart_console_only (void);



From thomas.lendacky at amd.com  Tue Jun 27 08:07:31 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Tue, 27 Jun 2017 10:07:31 -0500
Subject: [PATCH v8 RESEND 01/38] x86: Document AMD Secure Memory Encryption
 (SME)
In-Reply-To: <20170627150718.17428.81813.stgit@tlendack-t1.amdoffice.net>
References: <20170627150718.17428.81813.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170627150731.17428.51715.stgit@tlendack-t1.amdoffice.net>

Create a Documentation entry to describe the AMD Secure Memory
Encryption (SME) feature and add documentation for the mem_encrypt=
kernel parameter.

Reviewed-by: Borislav Petkov <bp at suse.de>
Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 Documentation/admin-guide/kernel-parameters.txt |   11 ++++
 Documentation/x86/amd-memory-encryption.txt     |   68 +++++++++++++++++++++++
 2 files changed, 79 insertions(+)
 create mode 100644 Documentation/x86/amd-memory-encryption.txt

diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index 9b0b3de..51e03ee 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -2197,6 +2197,17 @@
 			memory contents and reserves bad memory
 			regions that are detected.
 
+	mem_encrypt=	[X86-64] AMD Secure Memory Encryption (SME) control
+			Valid arguments: on, off
+			Default (depends on kernel configuration option):
+			  on  (CONFIG_AMD_MEM_ENCRYPT_ACTIVE_BY_DEFAULT=y)
+			  off (CONFIG_AMD_MEM_ENCRYPT_ACTIVE_BY_DEFAULT=n)
+			mem_encrypt=on:		Activate SME
+			mem_encrypt=off:	Do not activate SME
+
+			Refer to Documentation/x86/amd-memory-encryption.txt
+			for details on when memory encryption can be activated.
+
 	mem_sleep_default=	[SUSPEND] Default system suspend mode:
 			s2idle  - Suspend-To-Idle
 			shallow - Power-On Suspend or equivalent (if supported)
diff --git a/Documentation/x86/amd-memory-encryption.txt b/Documentation/x86/amd-memory-encryption.txt
new file mode 100644
index 0000000..f512ab7
--- /dev/null
+++ b/Documentation/x86/amd-memory-encryption.txt
@@ -0,0 +1,68 @@
+Secure Memory Encryption (SME) is a feature found on AMD processors.
+
+SME provides the ability to mark individual pages of memory as encrypted using
+the standard x86 page tables.  A page that is marked encrypted will be
+automatically decrypted when read from DRAM and encrypted when written to
+DRAM.  SME can therefore be used to protect the contents of DRAM from physical
+attacks on the system.
+
+A page is encrypted when a page table entry has the encryption bit set (see
+below on how to determine its position).  The encryption bit can also be
+specified in the cr3 register, allowing the PGD table to be encrypted. Each
+successive level of page tables can also be encrypted by setting the encryption
+bit in the page table entry that points to the next table. This allows the full
+page table hierarchy to be encrypted. Note, this means that just because the
+encryption bit is set in cr3, doesn't imply the full hierarchy is encyrpted.
+Each page table entry in the hierarchy needs to have the encryption bit set to
+achieve that. So, theoretically, you could have the encryption bit set in cr3
+so that the PGD is encrypted, but not set the encryption bit in the PGD entry
+for a PUD which results in the PUD pointed to by that entry to not be
+encrypted.
+
+Support for SME can be determined through the CPUID instruction. The CPUID
+function 0x8000001f reports information related to SME:
+
+	0x8000001f[eax]:
+		Bit[0] indicates support for SME
+	0x8000001f[ebx]:
+		Bits[5:0]  pagetable bit number used to activate memory
+			   encryption
+		Bits[11:6] reduction in physical address space, in bits, when
+			   memory encryption is enabled (this only affects
+			   system physical addresses, not guest physical
+			   addresses)
+
+If support for SME is present, MSR 0xc00100010 (MSR_K8_SYSCFG) can be used to
+determine if SME is enabled and/or to enable memory encryption:
+
+	0xc0010010:
+		Bit[23]   0 = memory encryption features are disabled
+			  1 = memory encryption features are enabled
+
+Linux relies on BIOS to set this bit if BIOS has determined that the reduction
+in the physical address space as a result of enabling memory encryption (see
+CPUID information above) will not conflict with the address space resource
+requirements for the system.  If this bit is not set upon Linux startup then
+Linux itself will not set it and memory encryption will not be possible.
+
+The state of SME in the Linux kernel can be documented as follows:
+	- Supported:
+	  The CPU supports SME (determined through CPUID instruction).
+
+	- Enabled:
+	  Supported and bit 23 of MSR_K8_SYSCFG is set.
+
+	- Active:
+	  Supported, Enabled and the Linux kernel is actively applying
+	  the encryption bit to page table entries (the SME mask in the
+	  kernel is non-zero).
+
+SME can also be enabled and activated in the BIOS. If SME is enabled and
+activated in the BIOS, then all memory accesses will be encrypted and it will
+not be necessary to activate the Linux memory encryption support.  If the BIOS
+merely enables SME (sets bit 23 of the MSR_K8_SYSCFG), then Linux can activate
+memory encryption by default (CONFIG_AMD_MEM_ENCRYPT_ACTIVE_BY_DEFAULT=y) or
+by supplying mem_encrypt=on on the kernel command line.  However, if BIOS does
+not enable SME, then Linux will not be able to activate memory encryption, even
+if configured to do so by default or the mem_encrypt=on command line parameter
+is specified.



From thomas.lendacky at amd.com  Tue Jun 27 08:07:40 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Tue, 27 Jun 2017 10:07:40 -0500
Subject: [PATCH v8 RESEND 02/38] x86/mm/pat: Set write-protect cache mode
 for full PAT support
In-Reply-To: <20170627150718.17428.81813.stgit@tlendack-t1.amdoffice.net>
References: <20170627150718.17428.81813.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170627150740.17428.95661.stgit@tlendack-t1.amdoffice.net>

For processors that support PAT, set the write-protect cache mode
(_PAGE_CACHE_MODE_WP) entry to the actual write-protect value (x05).

Acked-by: Borislav Petkov <bp at suse.de>
Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/mm/pat.c |    6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/arch/x86/mm/pat.c b/arch/x86/mm/pat.c
index 9b78685..6753d9c 100644
--- a/arch/x86/mm/pat.c
+++ b/arch/x86/mm/pat.c
@@ -295,7 +295,7 @@ static void init_cache_modes(void)
  * pat_init - Initialize PAT MSR and PAT table
  *
  * This function initializes PAT MSR and PAT table with an OS-defined value
- * to enable additional cache attributes, WC and WT.
+ * to enable additional cache attributes, WC, WT and WP.
  *
  * This function must be called on all CPUs using the specific sequence of
  * operations defined in Intel SDM. mtrr_rendezvous_handler() provides this
@@ -356,7 +356,7 @@ void pat_init(void)
 		 *      010    2    UC-: _PAGE_CACHE_MODE_UC_MINUS
 		 *      011    3    UC : _PAGE_CACHE_MODE_UC
 		 *      100    4    WB : Reserved
-		 *      101    5    WC : Reserved
+		 *      101    5    WP : _PAGE_CACHE_MODE_WP
 		 *      110    6    UC-: Reserved
 		 *      111    7    WT : _PAGE_CACHE_MODE_WT
 		 *
@@ -364,7 +364,7 @@ void pat_init(void)
 		 * corresponding types in the presence of PAT errata.
 		 */
 		pat = PAT(0, WB) | PAT(1, WC) | PAT(2, UC_MINUS) | PAT(3, UC) |
-		      PAT(4, WB) | PAT(5, WC) | PAT(6, UC_MINUS) | PAT(7, WT);
+		      PAT(4, WB) | PAT(5, WP) | PAT(6, UC_MINUS) | PAT(7, WT);
 	}
 
 	if (!boot_cpu_done) {



From thomas.lendacky at amd.com  Tue Jun 27 08:07:51 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Tue, 27 Jun 2017 10:07:51 -0500
Subject: [PATCH v8 RESEND 03/38] x86, mpparse, x86/acpi, x86/PCI, x86/dmi,
 SFI: Use memremap for RAM mappings
In-Reply-To: <20170627150718.17428.81813.stgit@tlendack-t1.amdoffice.net>
References: <20170627150718.17428.81813.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170627150751.17428.63752.stgit@tlendack-t1.amdoffice.net>

The ioremap() function is intended for mapping MMIO. For RAM, the
memremap() function should be used. Convert calls from ioremap() to
memremap() when re-mapping RAM.

This will be used later by SME to control how the encryption mask is
applied to memory mappings, with certain memory locations being mapped
decrypted vs encrypted.

Reviewed-by: Borislav Petkov <bp at suse.de>
Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/include/asm/dmi.h   |    8 ++++----
 arch/x86/kernel/acpi/boot.c  |    6 +++---
 arch/x86/kernel/kdebugfs.c   |   34 +++++++++++-----------------------
 arch/x86/kernel/ksysfs.c     |   28 ++++++++++++++--------------
 arch/x86/kernel/mpparse.c    |   10 +++++-----
 arch/x86/pci/common.c        |    4 ++--
 drivers/firmware/dmi-sysfs.c |    5 +++--
 drivers/firmware/pcdp.c      |    4 ++--
 drivers/sfi/sfi_core.c       |   22 +++++++++++-----------
 9 files changed, 55 insertions(+), 66 deletions(-)

diff --git a/arch/x86/include/asm/dmi.h b/arch/x86/include/asm/dmi.h
index 3c69fed..a8e15b0 100644
--- a/arch/x86/include/asm/dmi.h
+++ b/arch/x86/include/asm/dmi.h
@@ -13,9 +13,9 @@ static __always_inline __init void *dmi_alloc(unsigned len)
 }
 
 /* Use early IO mappings for DMI because it's initialized early */
-#define dmi_early_remap		early_ioremap
-#define dmi_early_unmap		early_iounmap
-#define dmi_remap		ioremap_cache
-#define dmi_unmap		iounmap
+#define dmi_early_remap		early_memremap
+#define dmi_early_unmap		early_memunmap
+#define dmi_remap(_x, _l)	memremap(_x, _l, MEMREMAP_WB)
+#define dmi_unmap(_x)		memunmap(_x)
 
 #endif /* _ASM_X86_DMI_H */
diff --git a/arch/x86/kernel/acpi/boot.c b/arch/x86/kernel/acpi/boot.c
index 6bb6806..850160a 100644
--- a/arch/x86/kernel/acpi/boot.c
+++ b/arch/x86/kernel/acpi/boot.c
@@ -115,7 +115,7 @@
 #define	ACPI_INVALID_GSI		INT_MIN
 
 /*
- * This is just a simple wrapper around early_ioremap(),
+ * This is just a simple wrapper around early_memremap(),
  * with sanity checks for phys == 0 and size == 0.
  */
 char *__init __acpi_map_table(unsigned long phys, unsigned long size)
@@ -124,7 +124,7 @@ char *__init __acpi_map_table(unsigned long phys, unsigned long size)
 	if (!phys || !size)
 		return NULL;
 
-	return early_ioremap(phys, size);
+	return early_memremap(phys, size);
 }
 
 void __init __acpi_unmap_table(char *map, unsigned long size)
@@ -132,7 +132,7 @@ void __init __acpi_unmap_table(char *map, unsigned long size)
 	if (!map || !size)
 		return;
 
-	early_iounmap(map, size);
+	early_memunmap(map, size);
 }
 
 #ifdef CONFIG_X86_LOCAL_APIC
diff --git a/arch/x86/kernel/kdebugfs.c b/arch/x86/kernel/kdebugfs.c
index 38b6458..fd6f8fb 100644
--- a/arch/x86/kernel/kdebugfs.c
+++ b/arch/x86/kernel/kdebugfs.c
@@ -33,7 +33,6 @@ static ssize_t setup_data_read(struct file *file, char __user *user_buf,
 	struct setup_data_node *node = file->private_data;
 	unsigned long remain;
 	loff_t pos = *ppos;
-	struct page *pg;
 	void *p;
 	u64 pa;
 
@@ -47,18 +46,13 @@ static ssize_t setup_data_read(struct file *file, char __user *user_buf,
 		count = node->len - pos;
 
 	pa = node->paddr + sizeof(struct setup_data) + pos;
-	pg = pfn_to_page((pa + count - 1) >> PAGE_SHIFT);
-	if (PageHighMem(pg)) {
-		p = ioremap_cache(pa, count);
-		if (!p)
-			return -ENXIO;
-	} else
-		p = __va(pa);
+	p = memremap(pa, count, MEMREMAP_WB);
+	if (!p)
+		return -ENOMEM;
 
 	remain = copy_to_user(user_buf, p, count);
 
-	if (PageHighMem(pg))
-		iounmap(p);
+	memunmap(p);
 
 	if (remain)
 		return -EFAULT;
@@ -109,7 +103,6 @@ static int __init create_setup_data_nodes(struct dentry *parent)
 	struct setup_data *data;
 	int error;
 	struct dentry *d;
-	struct page *pg;
 	u64 pa_data;
 	int no = 0;
 
@@ -126,16 +119,12 @@ static int __init create_setup_data_nodes(struct dentry *parent)
 			goto err_dir;
 		}
 
-		pg = pfn_to_page((pa_data+sizeof(*data)-1) >> PAGE_SHIFT);
-		if (PageHighMem(pg)) {
-			data = ioremap_cache(pa_data, sizeof(*data));
-			if (!data) {
-				kfree(node);
-				error = -ENXIO;
-				goto err_dir;
-			}
-		} else
-			data = __va(pa_data);
+		data = memremap(pa_data, sizeof(*data), MEMREMAP_WB);
+		if (!data) {
+			kfree(node);
+			error = -ENOMEM;
+			goto err_dir;
+		}
 
 		node->paddr = pa_data;
 		node->type = data->type;
@@ -143,8 +132,7 @@ static int __init create_setup_data_nodes(struct dentry *parent)
 		error = create_setup_data_node(d, no, node);
 		pa_data = data->next;
 
-		if (PageHighMem(pg))
-			iounmap(data);
+		memunmap(data);
 		if (error)
 			goto err_dir;
 		no++;
diff --git a/arch/x86/kernel/ksysfs.c b/arch/x86/kernel/ksysfs.c
index 4afc67f..ee51db9 100644
--- a/arch/x86/kernel/ksysfs.c
+++ b/arch/x86/kernel/ksysfs.c
@@ -16,8 +16,8 @@
 #include <linux/stat.h>
 #include <linux/slab.h>
 #include <linux/mm.h>
+#include <linux/io.h>
 
-#include <asm/io.h>
 #include <asm/setup.h>
 
 static ssize_t version_show(struct kobject *kobj,
@@ -79,12 +79,12 @@ static int get_setup_data_paddr(int nr, u64 *paddr)
 			*paddr = pa_data;
 			return 0;
 		}
-		data = ioremap_cache(pa_data, sizeof(*data));
+		data = memremap(pa_data, sizeof(*data), MEMREMAP_WB);
 		if (!data)
 			return -ENOMEM;
 
 		pa_data = data->next;
-		iounmap(data);
+		memunmap(data);
 		i++;
 	}
 	return -EINVAL;
@@ -97,17 +97,17 @@ static int __init get_setup_data_size(int nr, size_t *size)
 	u64 pa_data = boot_params.hdr.setup_data;
 
 	while (pa_data) {
-		data = ioremap_cache(pa_data, sizeof(*data));
+		data = memremap(pa_data, sizeof(*data), MEMREMAP_WB);
 		if (!data)
 			return -ENOMEM;
 		if (nr == i) {
 			*size = data->len;
-			iounmap(data);
+			memunmap(data);
 			return 0;
 		}
 
 		pa_data = data->next;
-		iounmap(data);
+		memunmap(data);
 		i++;
 	}
 	return -EINVAL;
@@ -127,12 +127,12 @@ static ssize_t type_show(struct kobject *kobj,
 	ret = get_setup_data_paddr(nr, &paddr);
 	if (ret)
 		return ret;
-	data = ioremap_cache(paddr, sizeof(*data));
+	data = memremap(paddr, sizeof(*data), MEMREMAP_WB);
 	if (!data)
 		return -ENOMEM;
 
 	ret = sprintf(buf, "0x%x\n", data->type);
-	iounmap(data);
+	memunmap(data);
 	return ret;
 }
 
@@ -154,7 +154,7 @@ static ssize_t setup_data_data_read(struct file *fp,
 	ret = get_setup_data_paddr(nr, &paddr);
 	if (ret)
 		return ret;
-	data = ioremap_cache(paddr, sizeof(*data));
+	data = memremap(paddr, sizeof(*data), MEMREMAP_WB);
 	if (!data)
 		return -ENOMEM;
 
@@ -170,15 +170,15 @@ static ssize_t setup_data_data_read(struct file *fp,
 		goto out;
 
 	ret = count;
-	p = ioremap_cache(paddr + sizeof(*data), data->len);
+	p = memremap(paddr + sizeof(*data), data->len, MEMREMAP_WB);
 	if (!p) {
 		ret = -ENOMEM;
 		goto out;
 	}
 	memcpy(buf, p + off, count);
-	iounmap(p);
+	memunmap(p);
 out:
-	iounmap(data);
+	memunmap(data);
 	return ret;
 }
 
@@ -250,13 +250,13 @@ static int __init get_setup_data_total_num(u64 pa_data, int *nr)
 	*nr = 0;
 	while (pa_data) {
 		*nr += 1;
-		data = ioremap_cache(pa_data, sizeof(*data));
+		data = memremap(pa_data, sizeof(*data), MEMREMAP_WB);
 		if (!data) {
 			ret = -ENOMEM;
 			goto out;
 		}
 		pa_data = data->next;
-		iounmap(data);
+		memunmap(data);
 	}
 
 out:
diff --git a/arch/x86/kernel/mpparse.c b/arch/x86/kernel/mpparse.c
index 0d904d7..fd37f39 100644
--- a/arch/x86/kernel/mpparse.c
+++ b/arch/x86/kernel/mpparse.c
@@ -436,9 +436,9 @@ static unsigned long __init get_mpc_size(unsigned long physptr)
 	struct mpc_table *mpc;
 	unsigned long size;
 
-	mpc = early_ioremap(physptr, PAGE_SIZE);
+	mpc = early_memremap(physptr, PAGE_SIZE);
 	size = mpc->length;
-	early_iounmap(mpc, PAGE_SIZE);
+	early_memunmap(mpc, PAGE_SIZE);
 	apic_printk(APIC_VERBOSE, "  mpc: %lx-%lx\n", physptr, physptr + size);
 
 	return size;
@@ -450,7 +450,7 @@ static int __init check_physptr(struct mpf_intel *mpf, unsigned int early)
 	unsigned long size;
 
 	size = get_mpc_size(mpf->physptr);
-	mpc = early_ioremap(mpf->physptr, size);
+	mpc = early_memremap(mpf->physptr, size);
 	/*
 	 * Read the physical hardware table.  Anything here will
 	 * override the defaults.
@@ -461,10 +461,10 @@ static int __init check_physptr(struct mpf_intel *mpf, unsigned int early)
 #endif
 		pr_err("BIOS bug, MP table errors detected!...\n");
 		pr_cont("... disabling SMP support. (tell your hw vendor)\n");
-		early_iounmap(mpc, size);
+		early_memunmap(mpc, size);
 		return -1;
 	}
-	early_iounmap(mpc, size);
+	early_memunmap(mpc, size);
 
 	if (early)
 		return -1;
diff --git a/arch/x86/pci/common.c b/arch/x86/pci/common.c
index 190e718..08cf71c 100644
--- a/arch/x86/pci/common.c
+++ b/arch/x86/pci/common.c
@@ -691,7 +691,7 @@ int pcibios_add_device(struct pci_dev *dev)
 
 	pa_data = boot_params.hdr.setup_data;
 	while (pa_data) {
-		data = ioremap(pa_data, sizeof(*rom));
+		data = memremap(pa_data, sizeof(*rom), MEMREMAP_WB);
 		if (!data)
 			return -ENOMEM;
 
@@ -710,7 +710,7 @@ int pcibios_add_device(struct pci_dev *dev)
 			}
 		}
 		pa_data = data->next;
-		iounmap(data);
+		memunmap(data);
 	}
 	set_dma_domain_ops(dev);
 	set_dev_domain_options(dev);
diff --git a/drivers/firmware/dmi-sysfs.c b/drivers/firmware/dmi-sysfs.c
index ef76e5e..d5de6ee 100644
--- a/drivers/firmware/dmi-sysfs.c
+++ b/drivers/firmware/dmi-sysfs.c
@@ -25,6 +25,7 @@
 #include <linux/slab.h>
 #include <linux/list.h>
 #include <linux/io.h>
+#include <asm/dmi.h>
 
 #define MAX_ENTRY_TYPE 255 /* Most of these aren't used, but we consider
 			      the top entry type is only 8 bits */
@@ -380,7 +381,7 @@ static ssize_t dmi_sel_raw_read_phys32(struct dmi_sysfs_entry *entry,
 	u8 __iomem *mapped;
 	ssize_t wrote = 0;
 
-	mapped = ioremap(sel->access_method_address, sel->area_length);
+	mapped = dmi_remap(sel->access_method_address, sel->area_length);
 	if (!mapped)
 		return -EIO;
 
@@ -390,7 +391,7 @@ static ssize_t dmi_sel_raw_read_phys32(struct dmi_sysfs_entry *entry,
 		wrote++;
 	}
 
-	iounmap(mapped);
+	dmi_unmap(mapped);
 	return wrote;
 }
 
diff --git a/drivers/firmware/pcdp.c b/drivers/firmware/pcdp.c
index 75273a25..e83d6ae 100644
--- a/drivers/firmware/pcdp.c
+++ b/drivers/firmware/pcdp.c
@@ -95,7 +95,7 @@
 	if (efi.hcdp == EFI_INVALID_TABLE_ADDR)
 		return -ENODEV;
 
-	pcdp = early_ioremap(efi.hcdp, 4096);
+	pcdp = early_memremap(efi.hcdp, 4096);
 	printk(KERN_INFO "PCDP: v%d at 0x%lx\n", pcdp->rev, efi.hcdp);
 
 	if (strstr(cmdline, "console=hcdp")) {
@@ -131,6 +131,6 @@
 	}
 
 out:
-	early_iounmap(pcdp, 4096);
+	early_memunmap(pcdp, 4096);
 	return rc;
 }
diff --git a/drivers/sfi/sfi_core.c b/drivers/sfi/sfi_core.c
index 296db7a..d5ce534 100644
--- a/drivers/sfi/sfi_core.c
+++ b/drivers/sfi/sfi_core.c
@@ -86,13 +86,13 @@
 /*
  * FW creates and saves the SFI tables in memory. When these tables get
  * used, they may need to be mapped to virtual address space, and the mapping
- * can happen before or after the ioremap() is ready, so a flag is needed
+ * can happen before or after the memremap() is ready, so a flag is needed
  * to indicating this
  */
-static u32 sfi_use_ioremap __read_mostly;
+static u32 sfi_use_memremap __read_mostly;
 
 /*
- * sfi_un/map_memory calls early_ioremap/iounmap which is a __init function
+ * sfi_un/map_memory calls early_memremap/memunmap which is a __init function
  * and introduces section mismatch. So use __ref to make it calm.
  */
 static void __iomem * __ref sfi_map_memory(u64 phys, u32 size)
@@ -100,10 +100,10 @@ static void __iomem * __ref sfi_map_memory(u64 phys, u32 size)
 	if (!phys || !size)
 		return NULL;
 
-	if (sfi_use_ioremap)
-		return ioremap_cache(phys, size);
+	if (sfi_use_memremap)
+		return memremap(phys, size, MEMREMAP_WB);
 	else
-		return early_ioremap(phys, size);
+		return early_memremap(phys, size);
 }
 
 static void __ref sfi_unmap_memory(void __iomem *virt, u32 size)
@@ -111,10 +111,10 @@ static void __ref sfi_unmap_memory(void __iomem *virt, u32 size)
 	if (!virt || !size)
 		return;
 
-	if (sfi_use_ioremap)
-		iounmap(virt);
+	if (sfi_use_memremap)
+		memunmap(virt);
 	else
-		early_iounmap(virt, size);
+		early_memunmap(virt, size);
 }
 
 static void sfi_print_table_header(unsigned long long pa,
@@ -507,8 +507,8 @@ void __init sfi_init_late(void)
 	length = syst_va->header.len;
 	sfi_unmap_memory(syst_va, sizeof(struct sfi_table_simple));
 
-	/* Use ioremap now after it is ready */
-	sfi_use_ioremap = 1;
+	/* Use memremap now after it is ready */
+	sfi_use_memremap = 1;
 	syst_va = sfi_map_memory(syst_pa, length);
 
 	sfi_acpi_init();



From thomas.lendacky at amd.com  Tue Jun 27 08:08:02 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Tue, 27 Jun 2017 10:08:02 -0500
Subject: [PATCH v8 RESEND 04/38] x86/CPU/AMD: Add the Secure Memory
 Encryption CPU feature
In-Reply-To: <20170627150718.17428.81813.stgit@tlendack-t1.amdoffice.net>
References: <20170627150718.17428.81813.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170627150802.17428.331.stgit@tlendack-t1.amdoffice.net>

Update the CPU features to include identifying and reporting on the
Secure Memory Encryption (SME) feature.  SME is identified by CPUID
0x8000001f, but requires BIOS support to enable it (set bit 23 of
MSR_K8_SYSCFG).  Only show the SME feature as available if reported by
CPUID and enabled by BIOS.

Reviewed-by: Borislav Petkov <bp at suse.de>
Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/include/asm/cpufeatures.h |    1 +
 arch/x86/include/asm/msr-index.h   |    2 ++
 arch/x86/kernel/cpu/amd.c          |   13 +++++++++++++
 arch/x86/kernel/cpu/scattered.c    |    1 +
 4 files changed, 17 insertions(+)

diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h
index 2701e5f..2b692df 100644
--- a/arch/x86/include/asm/cpufeatures.h
+++ b/arch/x86/include/asm/cpufeatures.h
@@ -196,6 +196,7 @@
 
 #define X86_FEATURE_HW_PSTATE	( 7*32+ 8) /* AMD HW-PState */
 #define X86_FEATURE_PROC_FEEDBACK ( 7*32+ 9) /* AMD ProcFeedbackInterface */
+#define X86_FEATURE_SME		( 7*32+10) /* AMD Secure Memory Encryption */
 
 #define X86_FEATURE_INTEL_PPIN	( 7*32+14) /* Intel Processor Inventory Number */
 #define X86_FEATURE_INTEL_PT	( 7*32+15) /* Intel Processor Trace */
diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h
index 18b1623..460ac01 100644
--- a/arch/x86/include/asm/msr-index.h
+++ b/arch/x86/include/asm/msr-index.h
@@ -352,6 +352,8 @@
 #define MSR_K8_TOP_MEM1			0xc001001a
 #define MSR_K8_TOP_MEM2			0xc001001d
 #define MSR_K8_SYSCFG			0xc0010010
+#define MSR_K8_SYSCFG_MEM_ENCRYPT_BIT	23
+#define MSR_K8_SYSCFG_MEM_ENCRYPT	BIT_ULL(MSR_K8_SYSCFG_MEM_ENCRYPT_BIT)
 #define MSR_K8_INT_PENDING_MSG		0xc0010055
 /* C1E active bits in int pending message */
 #define K8_INTP_C1E_ACTIVE_MASK		0x18000000
diff --git a/arch/x86/kernel/cpu/amd.c b/arch/x86/kernel/cpu/amd.c
index bb5abe8..c47ceee 100644
--- a/arch/x86/kernel/cpu/amd.c
+++ b/arch/x86/kernel/cpu/amd.c
@@ -611,6 +611,19 @@ static void early_init_amd(struct cpuinfo_x86 *c)
 	 */
 	if (cpu_has_amd_erratum(c, amd_erratum_400))
 		set_cpu_bug(c, X86_BUG_AMD_E400);
+
+	/*
+	 * BIOS support is required for SME. If BIOS has not enabled SME
+	 * then don't advertise the feature (set in scattered.c)
+	 */
+	if (cpu_has(c, X86_FEATURE_SME)) {
+		u64 msr;
+
+		/* Check if SME is enabled */
+		rdmsrl(MSR_K8_SYSCFG, msr);
+		if (!(msr & MSR_K8_SYSCFG_MEM_ENCRYPT))
+			clear_cpu_cap(c, X86_FEATURE_SME);
+	}
 }
 
 static void init_amd_k8(struct cpuinfo_x86 *c)
diff --git a/arch/x86/kernel/cpu/scattered.c b/arch/x86/kernel/cpu/scattered.c
index 23c2350..05459ad 100644
--- a/arch/x86/kernel/cpu/scattered.c
+++ b/arch/x86/kernel/cpu/scattered.c
@@ -31,6 +31,7 @@ struct cpuid_bit {
 	{ X86_FEATURE_HW_PSTATE,	CPUID_EDX,  7, 0x80000007, 0 },
 	{ X86_FEATURE_CPB,		CPUID_EDX,  9, 0x80000007, 0 },
 	{ X86_FEATURE_PROC_FEEDBACK,    CPUID_EDX, 11, 0x80000007, 0 },
+	{ X86_FEATURE_SME,		CPUID_EAX,  0, 0x8000001f, 0 },
 	{ 0, 0, 0, 0, 0 }
 };
 



From thomas.lendacky at amd.com  Tue Jun 27 08:08:13 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Tue, 27 Jun 2017 10:08:13 -0500
Subject: [PATCH v8 RESEND 05/38] x86/CPU/AMD: Handle SME reduction in
 physical address size
In-Reply-To: <20170627150718.17428.81813.stgit@tlendack-t1.amdoffice.net>
References: <20170627150718.17428.81813.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170627150813.17428.17375.stgit@tlendack-t1.amdoffice.net>

When System Memory Encryption (SME) is enabled, the physical address
space is reduced. Adjust the x86_phys_bits value to reflect this
reduction.

Reviewed-by: Borislav Petkov <bp at suse.de>
Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/kernel/cpu/amd.c |   10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/arch/x86/kernel/cpu/amd.c b/arch/x86/kernel/cpu/amd.c
index c47ceee..5bdcbd4 100644
--- a/arch/x86/kernel/cpu/amd.c
+++ b/arch/x86/kernel/cpu/amd.c
@@ -613,15 +613,19 @@ static void early_init_amd(struct cpuinfo_x86 *c)
 		set_cpu_bug(c, X86_BUG_AMD_E400);
 
 	/*
-	 * BIOS support is required for SME. If BIOS has not enabled SME
-	 * then don't advertise the feature (set in scattered.c)
+	 * BIOS support is required for SME. If BIOS has enabld SME then
+	 * adjust x86_phys_bits by the SME physical address space reduction
+	 * value. If BIOS has not enabled SME then don't advertise the
+	 * feature (set in scattered.c).
 	 */
 	if (cpu_has(c, X86_FEATURE_SME)) {
 		u64 msr;
 
 		/* Check if SME is enabled */
 		rdmsrl(MSR_K8_SYSCFG, msr);
-		if (!(msr & MSR_K8_SYSCFG_MEM_ENCRYPT))
+		if (msr & MSR_K8_SYSCFG_MEM_ENCRYPT)
+			c->x86_phys_bits -= (cpuid_ebx(0x8000001f) >> 6) & 0x3f;
+		else
 			clear_cpu_cap(c, X86_FEATURE_SME);
 	}
 }



From thomas.lendacky at amd.com  Tue Jun 27 08:08:24 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Tue, 27 Jun 2017 10:08:24 -0500
Subject: [PATCH v8 RESEND 06/38] x86/mm: Add Secure Memory Encryption (SME)
 support
In-Reply-To: <20170627150718.17428.81813.stgit@tlendack-t1.amdoffice.net>
References: <20170627150718.17428.81813.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170627150824.17428.98358.stgit@tlendack-t1.amdoffice.net>

Add support for Secure Memory Encryption (SME). This initial support
provides a Kconfig entry to build the SME support into the kernel and
defines the memory encryption mask that will be used in subsequent
patches to mark pages as encrypted.

Reviewed-by: Borislav Petkov <bp at suse.de>
Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/Kconfig                   |   25 +++++++++++++++++++++++++
 arch/x86/include/asm/mem_encrypt.h |   30 ++++++++++++++++++++++++++++++
 arch/x86/mm/Makefile               |    1 +
 arch/x86/mm/mem_encrypt.c          |   21 +++++++++++++++++++++
 include/linux/mem_encrypt.h        |   35 +++++++++++++++++++++++++++++++++++
 5 files changed, 112 insertions(+)
 create mode 100644 arch/x86/include/asm/mem_encrypt.h
 create mode 100644 arch/x86/mm/mem_encrypt.c
 create mode 100644 include/linux/mem_encrypt.h

diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index 72028a1..3a59e9c 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -1409,6 +1409,31 @@ config X86_DIRECT_GBPAGES
 	  supports them), so don't confuse the user by printing
 	  that we have them enabled.
 
+config ARCH_HAS_MEM_ENCRYPT
+	def_bool y
+
+config AMD_MEM_ENCRYPT
+	bool "AMD Secure Memory Encryption (SME) support"
+	depends on X86_64 && CPU_SUP_AMD
+	---help---
+	  Say yes to enable support for the encryption of system memory.
+	  This requires an AMD processor that supports Secure Memory
+	  Encryption (SME).
+
+config AMD_MEM_ENCRYPT_ACTIVE_BY_DEFAULT
+	bool "Activate AMD Secure Memory Encryption (SME) by default"
+	default y
+	depends on AMD_MEM_ENCRYPT
+	---help---
+	  Say yes to have system memory encrypted by default if running on
+	  an AMD processor that supports Secure Memory Encryption (SME).
+
+	  If set to Y, then the encryption of system memory can be
+	  deactivated with the mem_encrypt=off command line option.
+
+	  If set to N, then the encryption of system memory can be
+	  activated with the mem_encrypt=on command line option.
+
 # Common NUMA Features
 config NUMA
 	bool "Numa Memory Allocation and Scheduler Support"
diff --git a/arch/x86/include/asm/mem_encrypt.h b/arch/x86/include/asm/mem_encrypt.h
new file mode 100644
index 0000000..a105796
--- /dev/null
+++ b/arch/x86/include/asm/mem_encrypt.h
@@ -0,0 +1,30 @@
+/*
+ * AMD Memory Encryption Support
+ *
+ * Copyright (C) 2016 Advanced Micro Devices, Inc.
+ *
+ * Author: Tom Lendacky <thomas.lendacky at amd.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#ifndef __X86_MEM_ENCRYPT_H__
+#define __X86_MEM_ENCRYPT_H__
+
+#ifndef __ASSEMBLY__
+
+#ifdef CONFIG_AMD_MEM_ENCRYPT
+
+extern unsigned long sme_me_mask;
+
+#else	/* !CONFIG_AMD_MEM_ENCRYPT */
+
+#define sme_me_mask	0UL
+
+#endif	/* CONFIG_AMD_MEM_ENCRYPT */
+
+#endif	/* __ASSEMBLY__ */
+
+#endif	/* __X86_MEM_ENCRYPT_H__ */
diff --git a/arch/x86/mm/Makefile b/arch/x86/mm/Makefile
index 0fbdcb6..a94a7b6 100644
--- a/arch/x86/mm/Makefile
+++ b/arch/x86/mm/Makefile
@@ -39,3 +39,4 @@ obj-$(CONFIG_X86_INTEL_MPX)	+= mpx.o
 obj-$(CONFIG_X86_INTEL_MEMORY_PROTECTION_KEYS) += pkeys.o
 obj-$(CONFIG_RANDOMIZE_MEMORY) += kaslr.o
 
+obj-$(CONFIG_AMD_MEM_ENCRYPT)	+= mem_encrypt.o
diff --git a/arch/x86/mm/mem_encrypt.c b/arch/x86/mm/mem_encrypt.c
new file mode 100644
index 0000000..b99d469
--- /dev/null
+++ b/arch/x86/mm/mem_encrypt.c
@@ -0,0 +1,21 @@
+/*
+ * AMD Memory Encryption Support
+ *
+ * Copyright (C) 2016 Advanced Micro Devices, Inc.
+ *
+ * Author: Tom Lendacky <thomas.lendacky at amd.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include <linux/linkage.h>
+
+/*
+ * Since SME related variables are set early in the boot process they must
+ * reside in the .data section so as not to be zeroed out when the .bss
+ * section is later cleared.
+ */
+unsigned long sme_me_mask __section(.data) = 0;
+EXPORT_SYMBOL_GPL(sme_me_mask);
diff --git a/include/linux/mem_encrypt.h b/include/linux/mem_encrypt.h
new file mode 100644
index 0000000..59769f7
--- /dev/null
+++ b/include/linux/mem_encrypt.h
@@ -0,0 +1,35 @@
+/*
+ * AMD Memory Encryption Support
+ *
+ * Copyright (C) 2016 Advanced Micro Devices, Inc.
+ *
+ * Author: Tom Lendacky <thomas.lendacky at amd.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#ifndef __MEM_ENCRYPT_H__
+#define __MEM_ENCRYPT_H__
+
+#ifndef __ASSEMBLY__
+
+#ifdef CONFIG_ARCH_HAS_MEM_ENCRYPT
+
+#include <asm/mem_encrypt.h>
+
+#else	/* !CONFIG_ARCH_HAS_MEM_ENCRYPT */
+
+#define sme_me_mask	0UL
+
+#endif	/* CONFIG_ARCH_HAS_MEM_ENCRYPT */
+
+static inline bool sme_active(void)
+{
+	return !!sme_me_mask;
+}
+
+#endif	/* __ASSEMBLY__ */
+
+#endif	/* __MEM_ENCRYPT_H__ */



From thomas.lendacky at amd.com  Tue Jun 27 08:08:35 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Tue, 27 Jun 2017 10:08:35 -0500
Subject: [PATCH v8 RESEND 07/38] x86/mm: Remove phys_to_virt() usage in
 ioremap()
In-Reply-To: <20170627150718.17428.81813.stgit@tlendack-t1.amdoffice.net>
References: <20170627150718.17428.81813.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170627150835.17428.60833.stgit@tlendack-t1.amdoffice.net>

Currently there is a check if the address being mapped is in the ISA
range (is_ISA_range()), and if it is, then phys_to_virt() is used to
perform the mapping. When SME is active, the default is to add pagetable
mappings with the encryption bit set unless specifically overridden. The
resulting pagetable mapping from phys_to_virt() will result in a mapping
that has the encryption bit set. With SME, the use of ioremap() is
intended to generate pagetable mappings that do not have the encryption
bit set through the use of the PAGE_KERNEL_IO protection value.

Rather than special case the SME scenario, remove the ISA range check and
usage of phys_to_virt() and have ISA range mappings continue through the
remaining ioremap() path.

Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/mm/ioremap.c |    7 +------
 1 file changed, 1 insertion(+), 6 deletions(-)

diff --git a/arch/x86/mm/ioremap.c b/arch/x86/mm/ioremap.c
index 4c1b5fd..bfc3e2d 100644
--- a/arch/x86/mm/ioremap.c
+++ b/arch/x86/mm/ioremap.c
@@ -13,6 +13,7 @@
 #include <linux/slab.h>
 #include <linux/vmalloc.h>
 #include <linux/mmiotrace.h>
+#include <linux/mem_encrypt.h>
 
 #include <asm/set_memory.h>
 #include <asm/e820/api.h>
@@ -106,12 +107,6 @@ static void __iomem *__ioremap_caller(resource_size_t phys_addr,
 	}
 
 	/*
-	 * Don't remap the low PCI/ISA area, it's always mapped..
-	 */
-	if (is_ISA_range(phys_addr, last_addr))
-		return (__force void __iomem *)phys_to_virt(phys_addr);
-
-	/*
 	 * Don't allow anybody to remap normal RAM that we're using..
 	 */
 	pfn      = phys_addr >> PAGE_SHIFT;



From thomas.lendacky at amd.com  Tue Jun 27 08:08:43 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Tue, 27 Jun 2017 10:08:43 -0500
Subject: [PATCH v8 RESEND 08/38] x86/mm: Add support to enable SME in early
 boot processing
In-Reply-To: <20170627150718.17428.81813.stgit@tlendack-t1.amdoffice.net>
References: <20170627150718.17428.81813.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170627150843.17428.77939.stgit@tlendack-t1.amdoffice.net>

Add support to the early boot code to use Secure Memory Encryption (SME).
Since the kernel has been loaded into memory in a decrypted state, encrypt
the kernel in place and update the early pagetables with the memory
encryption mask so that new pagetable entries will use memory encryption.

The routines to set the encryption mask and perform the encryption are
stub routines for now with functionality to be added in a later patch.

Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/include/asm/mem_encrypt.h |    8 +++++
 arch/x86/kernel/head64.c           |   53 +++++++++++++++++++++++++++++-------
 arch/x86/kernel/head_64.S          |   20 ++++++++++++--
 arch/x86/mm/mem_encrypt.c          |    9 ++++++
 include/linux/mem_encrypt.h        |    5 +++
 5 files changed, 82 insertions(+), 13 deletions(-)

diff --git a/arch/x86/include/asm/mem_encrypt.h b/arch/x86/include/asm/mem_encrypt.h
index a105796..475e34f 100644
--- a/arch/x86/include/asm/mem_encrypt.h
+++ b/arch/x86/include/asm/mem_encrypt.h
@@ -15,14 +15,22 @@
 
 #ifndef __ASSEMBLY__
 
+#include <linux/init.h>
+
 #ifdef CONFIG_AMD_MEM_ENCRYPT
 
 extern unsigned long sme_me_mask;
 
+void __init sme_encrypt_kernel(void);
+void __init sme_enable(void);
+
 #else	/* !CONFIG_AMD_MEM_ENCRYPT */
 
 #define sme_me_mask	0UL
 
+static inline void __init sme_encrypt_kernel(void) { }
+static inline void __init sme_enable(void) { }
+
 #endif	/* CONFIG_AMD_MEM_ENCRYPT */
 
 #endif	/* __ASSEMBLY__ */
diff --git a/arch/x86/kernel/head64.c b/arch/x86/kernel/head64.c
index 46c3c73..1f0ddcc 100644
--- a/arch/x86/kernel/head64.c
+++ b/arch/x86/kernel/head64.c
@@ -14,6 +14,7 @@
 #include <linux/start_kernel.h>
 #include <linux/io.h>
 #include <linux/memblock.h>
+#include <linux/mem_encrypt.h>
 
 #include <asm/processor.h>
 #include <asm/proto.h>
@@ -45,9 +46,10 @@ static void __head *fixup_pointer(void *ptr, unsigned long physaddr)
 	return ptr - (void *)_text + (void *)physaddr;
 }
 
-void __head __startup_64(unsigned long physaddr)
+unsigned long __head __startup_64(unsigned long physaddr)
 {
 	unsigned long load_delta, *p;
+	unsigned long pgtable_flags;
 	pgdval_t *pgd;
 	p4dval_t *p4d;
 	pudval_t *pud;
@@ -68,6 +70,12 @@ void __head __startup_64(unsigned long physaddr)
 	if (load_delta & ~PMD_PAGE_MASK)
 		for (;;);
 
+	/* Activate Secure Memory Encryption (SME) if supported and enabled */
+	sme_enable();
+
+	/* Include the SME encryption mask in the fixup value */
+	load_delta += sme_get_me_mask();
+
 	/* Fixup the physical addresses in the page table */
 
 	pgd = fixup_pointer(&early_top_pgt, physaddr);
@@ -94,28 +102,30 @@ void __head __startup_64(unsigned long physaddr)
 
 	pud = fixup_pointer(early_dynamic_pgts[next_early_pgt++], physaddr);
 	pmd = fixup_pointer(early_dynamic_pgts[next_early_pgt++], physaddr);
+	pgtable_flags = _KERNPG_TABLE + sme_get_me_mask();
 
 	if (IS_ENABLED(CONFIG_X86_5LEVEL)) {
 		p4d = fixup_pointer(early_dynamic_pgts[next_early_pgt++], physaddr);
 
 		i = (physaddr >> PGDIR_SHIFT) % PTRS_PER_PGD;
-		pgd[i + 0] = (pgdval_t)p4d + _KERNPG_TABLE;
-		pgd[i + 1] = (pgdval_t)p4d + _KERNPG_TABLE;
+		pgd[i + 0] = (pgdval_t)p4d + pgtable_flags;
+		pgd[i + 1] = (pgdval_t)p4d + pgtable_flags;
 
 		i = (physaddr >> P4D_SHIFT) % PTRS_PER_P4D;
-		p4d[i + 0] = (pgdval_t)pud + _KERNPG_TABLE;
-		p4d[i + 1] = (pgdval_t)pud + _KERNPG_TABLE;
+		p4d[i + 0] = (pgdval_t)pud + pgtable_flags;
+		p4d[i + 1] = (pgdval_t)pud + pgtable_flags;
 	} else {
 		i = (physaddr >> PGDIR_SHIFT) % PTRS_PER_PGD;
-		pgd[i + 0] = (pgdval_t)pud + _KERNPG_TABLE;
-		pgd[i + 1] = (pgdval_t)pud + _KERNPG_TABLE;
+		pgd[i + 0] = (pgdval_t)pud + pgtable_flags;
+		pgd[i + 1] = (pgdval_t)pud + pgtable_flags;
 	}
 
 	i = (physaddr >> PUD_SHIFT) % PTRS_PER_PUD;
-	pud[i + 0] = (pudval_t)pmd + _KERNPG_TABLE;
-	pud[i + 1] = (pudval_t)pmd + _KERNPG_TABLE;
+	pud[i + 0] = (pudval_t)pmd + pgtable_flags;
+	pud[i + 1] = (pudval_t)pmd + pgtable_flags;
 
 	pmd_entry = __PAGE_KERNEL_LARGE_EXEC & ~_PAGE_GLOBAL;
+	pmd_entry += sme_get_me_mask();
 	pmd_entry +=  physaddr;
 
 	for (i = 0; i < DIV_ROUND_UP(_end - _text, PMD_SIZE); i++) {
@@ -136,9 +146,30 @@ void __head __startup_64(unsigned long physaddr)
 			pmd[i] += load_delta;
 	}
 
-	/* Fixup phys_base */
+	/*
+	 * Fixup phys_base - remove the memory encryption mask to obtain
+	 * the true physical address.
+	 */
 	p = fixup_pointer(&phys_base, physaddr);
-	*p += load_delta;
+	*p += load_delta - sme_get_me_mask();
+
+	/* Encrypt the kernel (if SME is active) */
+	sme_encrypt_kernel();
+
+	/*
+	 * Return the SME encryption mask (if SME is active) to be used as a
+	 * modifier for the initial pgdir entry programmed into CR3.
+	 */
+	return sme_get_me_mask();
+}
+
+unsigned long __startup_secondary_64(void)
+{
+	/*
+	 * Return the SME encryption mask (if SME is active) to be used as a
+	 * modifier for the initial pgdir entry programmed into CR3.
+	 */
+	return sme_get_me_mask();
 }
 
 /* Wipe all early page tables except for the kernel symbol map */
diff --git a/arch/x86/kernel/head_64.S b/arch/x86/kernel/head_64.S
index 6225550..ec5d5e9 100644
--- a/arch/x86/kernel/head_64.S
+++ b/arch/x86/kernel/head_64.S
@@ -73,12 +73,19 @@ startup_64:
 	/* Sanitize CPU configuration */
 	call verify_cpu
 
+	/*
+	 * Perform pagetable fixups. Additionally, if SME is active, encrypt
+	 * the kernel and retrieve the modifier (SME encryption mask if SME
+	 * is active) to be added to the initial pgdir entry that will be
+	 * programmed into CR3.
+	 */
 	leaq	_text(%rip), %rdi
 	pushq	%rsi
 	call	__startup_64
 	popq	%rsi
 
-	movq	$(early_top_pgt - __START_KERNEL_map), %rax
+	/* Form the CR3 value being sure to include the CR3 modifier */
+	addq	$(early_top_pgt - __START_KERNEL_map), %rax
 	jmp 1f
 ENTRY(secondary_startup_64)
 	/*
@@ -98,7 +105,16 @@ ENTRY(secondary_startup_64)
 	/* Sanitize CPU configuration */
 	call verify_cpu
 
-	movq	$(init_top_pgt - __START_KERNEL_map), %rax
+	/*
+	 * Retrieve the modifier (SME encryption mask if SME is active) to be
+	 * added to the initial pgdir entry that will be programmed into CR3.
+	 */
+	pushq	%rsi
+	call	__startup_secondary_64
+	popq	%rsi
+
+	/* Form the CR3 value being sure to include the CR3 modifier */
+	addq	$(init_top_pgt - __START_KERNEL_map), %rax
 1:
 
 	/* Enable PAE mode, PGE and LA57 */
diff --git a/arch/x86/mm/mem_encrypt.c b/arch/x86/mm/mem_encrypt.c
index b99d469..3ac6f99 100644
--- a/arch/x86/mm/mem_encrypt.c
+++ b/arch/x86/mm/mem_encrypt.c
@@ -11,6 +11,7 @@
  */
 
 #include <linux/linkage.h>
+#include <linux/init.h>
 
 /*
  * Since SME related variables are set early in the boot process they must
@@ -19,3 +20,11 @@
  */
 unsigned long sme_me_mask __section(.data) = 0;
 EXPORT_SYMBOL_GPL(sme_me_mask);
+
+void __init sme_encrypt_kernel(void)
+{
+}
+
+void __init sme_enable(void)
+{
+}
diff --git a/include/linux/mem_encrypt.h b/include/linux/mem_encrypt.h
index 59769f7..570f4fc 100644
--- a/include/linux/mem_encrypt.h
+++ b/include/linux/mem_encrypt.h
@@ -30,6 +30,11 @@ static inline bool sme_active(void)
 	return !!sme_me_mask;
 }
 
+static inline unsigned long sme_get_me_mask(void)
+{
+	return sme_me_mask;
+}
+
 #endif	/* __ASSEMBLY__ */
 
 #endif	/* __MEM_ENCRYPT_H__ */



From thomas.lendacky at amd.com  Tue Jun 27 08:08:55 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Tue, 27 Jun 2017 10:08:55 -0500
Subject: [PATCH v8 RESEND 09/38] x86/mm: Simplify p[g4um]d_page() macros
In-Reply-To: <20170627150718.17428.81813.stgit@tlendack-t1.amdoffice.net>
References: <20170627150718.17428.81813.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170627150855.17428.90364.stgit@tlendack-t1.amdoffice.net>

Create a pgd_pfn() macro similar to the p[4um]d_pfn() macros and then
use the p[g4um]d_pfn() macros in the p[g4um]d_page() macros instead of
duplicating the code.

Reviewed-by: Borislav Petkov <bp at suse.de>
Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/include/asm/pgtable.h |   16 +++++++++-------
 1 file changed, 9 insertions(+), 7 deletions(-)

diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h
index 77037b6..b64ea52 100644
--- a/arch/x86/include/asm/pgtable.h
+++ b/arch/x86/include/asm/pgtable.h
@@ -195,6 +195,11 @@ static inline unsigned long p4d_pfn(p4d_t p4d)
 	return (p4d_val(p4d) & p4d_pfn_mask(p4d)) >> PAGE_SHIFT;
 }
 
+static inline unsigned long pgd_pfn(pgd_t pgd)
+{
+	return (pgd_val(pgd) & PTE_PFN_MASK) >> PAGE_SHIFT;
+}
+
 static inline int p4d_large(p4d_t p4d)
 {
 	/* No 512 GiB pages yet */
@@ -704,8 +709,7 @@ static inline unsigned long pmd_page_vaddr(pmd_t pmd)
  * Currently stuck as a macro due to indirect forward reference to
  * linux/mmzone.h's __section_mem_map_addr() definition:
  */
-#define pmd_page(pmd)		\
-	pfn_to_page((pmd_val(pmd) & pmd_pfn_mask(pmd)) >> PAGE_SHIFT)
+#define pmd_page(pmd)	pfn_to_page(pmd_pfn(pmd))
 
 /*
  * the pmd page can be thought of an array like this: pmd_t[PTRS_PER_PMD]
@@ -773,8 +777,7 @@ static inline unsigned long pud_page_vaddr(pud_t pud)
  * Currently stuck as a macro due to indirect forward reference to
  * linux/mmzone.h's __section_mem_map_addr() definition:
  */
-#define pud_page(pud)		\
-	pfn_to_page((pud_val(pud) & pud_pfn_mask(pud)) >> PAGE_SHIFT)
+#define pud_page(pud)	pfn_to_page(pud_pfn(pud))
 
 /* Find an entry in the second-level page table.. */
 static inline pmd_t *pmd_offset(pud_t *pud, unsigned long address)
@@ -824,8 +827,7 @@ static inline unsigned long p4d_page_vaddr(p4d_t p4d)
  * Currently stuck as a macro due to indirect forward reference to
  * linux/mmzone.h's __section_mem_map_addr() definition:
  */
-#define p4d_page(p4d)		\
-	pfn_to_page((p4d_val(p4d) & p4d_pfn_mask(p4d)) >> PAGE_SHIFT)
+#define p4d_page(p4d)	pfn_to_page(p4d_pfn(p4d))
 
 /* Find an entry in the third-level page table.. */
 static inline pud_t *pud_offset(p4d_t *p4d, unsigned long address)
@@ -859,7 +861,7 @@ static inline unsigned long pgd_page_vaddr(pgd_t pgd)
  * Currently stuck as a macro due to indirect forward reference to
  * linux/mmzone.h's __section_mem_map_addr() definition:
  */
-#define pgd_page(pgd)		pfn_to_page(pgd_val(pgd) >> PAGE_SHIFT)
+#define pgd_page(pgd)	pfn_to_page(pgd_pfn(pgd))
 
 /* to find an entry in a page-table-directory. */
 static inline p4d_t *p4d_offset(pgd_t *pgd, unsigned long address)



From thomas.lendacky at amd.com  Tue Jun 27 08:09:05 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Tue, 27 Jun 2017 10:09:05 -0500
Subject: [PATCH v8 RESEND 10/38] x86/mm: Provide general kernel support for
 memory encryption
In-Reply-To: <20170627150718.17428.81813.stgit@tlendack-t1.amdoffice.net>
References: <20170627150718.17428.81813.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170627150905.17428.72579.stgit@tlendack-t1.amdoffice.net>

Changes to the existing page table macros will allow the SME support to
be enabled in a simple fashion with minimal changes to files that use these
macros.  Since the memory encryption mask will now be part of the regular
pagetable macros, we introduce two new macros (_PAGE_TABLE_NOENC and
_KERNPG_TABLE_NOENC) to allow for early pagetable creation/initialization
without the encryption mask before SME becomes active.  Two new pgprot()
macros are defined to allow setting or clearing the page encryption mask.

The FIXMAP_PAGE_NOCACHE define is introduced for use with MMIO.  SME does
not support encryption for MMIO areas so this define removes the encryption
mask from the page attribute.

Two new macros are introduced (__sme_pa() / __sme_pa_nodebug()) to allow
creating a physical address with the encryption mask.  These are used when
working with the cr3 register so that the PGD can be encrypted. The current
__va() macro is updated so that the virtual address is generated based off
of the physical address without the encryption mask thus allowing the same
virtual address to be generated regardless of whether encryption is enabled
for that physical location or not.

Also, an early initialization function is added for SME.  If SME is active,
this function:
 - Updates the early_pmd_flags so that early page faults create mappings
   with the encryption mask.
 - Updates the __supported_pte_mask to include the encryption mask.
 - Updates the protection_map entries to include the encryption mask so
   that user-space allocations will automatically have the encryption mask
   applied.

Reviewed-by: Borislav Petkov <bp at suse.de>
Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/boot/compressed/pagetable.c |    7 +++++
 arch/x86/include/asm/fixmap.h        |    7 +++++
 arch/x86/include/asm/mem_encrypt.h   |   13 ++++++++++
 arch/x86/include/asm/page_types.h    |    3 ++
 arch/x86/include/asm/pgtable.h       |    9 +++++++
 arch/x86/include/asm/pgtable_types.h |   45 ++++++++++++++++++++++------------
 arch/x86/include/asm/processor.h     |    3 ++
 arch/x86/kernel/espfix_64.c          |    2 +-
 arch/x86/kernel/head64.c             |   11 +++++++-
 arch/x86/kernel/head_64.S            |   20 ++++++++-------
 arch/x86/mm/kasan_init_64.c          |    4 ++-
 arch/x86/mm/mem_encrypt.c            |   17 +++++++++++++
 arch/x86/mm/pageattr.c               |    3 ++
 include/asm-generic/pgtable.h        |   12 +++++++++
 include/linux/mem_encrypt.h          |    8 ++++++
 15 files changed, 131 insertions(+), 33 deletions(-)

diff --git a/arch/x86/boot/compressed/pagetable.c b/arch/x86/boot/compressed/pagetable.c
index 8e69df9..246bf29 100644
--- a/arch/x86/boot/compressed/pagetable.c
+++ b/arch/x86/boot/compressed/pagetable.c
@@ -15,6 +15,13 @@
 #define __pa(x)  ((unsigned long)(x))
 #define __va(x)  ((void *)((unsigned long)(x)))
 
+/*
+ * The pgtable.h and mm/ident_map.c includes make use of the SME related
+ * information which is not used in the compressed image support. Un-define
+ * the SME support to avoid any compile and link errors.
+ */
+#undef CONFIG_AMD_MEM_ENCRYPT
+
 #include "misc.h"
 
 /* These actually do the work of building the kernel identity maps. */
diff --git a/arch/x86/include/asm/fixmap.h b/arch/x86/include/asm/fixmap.h
index b65155c..d9ff226 100644
--- a/arch/x86/include/asm/fixmap.h
+++ b/arch/x86/include/asm/fixmap.h
@@ -157,6 +157,13 @@ static inline void __set_fixmap(enum fixed_addresses idx,
 }
 #endif
 
+/*
+ * FIXMAP_PAGE_NOCACHE is used for MMIO. Memory encryption is not
+ * supported for MMIO addresses, so make sure that the memory encryption
+ * mask is not part of the page attributes.
+ */
+#define FIXMAP_PAGE_NOCACHE PAGE_KERNEL_IO_NOCACHE
+
 #include <asm-generic/fixmap.h>
 
 #define __late_set_fixmap(idx, phys, flags) __set_fixmap(idx, phys, flags)
diff --git a/arch/x86/include/asm/mem_encrypt.h b/arch/x86/include/asm/mem_encrypt.h
index 475e34f..dbae7a5 100644
--- a/arch/x86/include/asm/mem_encrypt.h
+++ b/arch/x86/include/asm/mem_encrypt.h
@@ -21,6 +21,8 @@
 
 extern unsigned long sme_me_mask;
 
+void __init sme_early_init(void);
+
 void __init sme_encrypt_kernel(void);
 void __init sme_enable(void);
 
@@ -28,11 +30,22 @@
 
 #define sme_me_mask	0UL
 
+static inline void __init sme_early_init(void) { }
+
 static inline void __init sme_encrypt_kernel(void) { }
 static inline void __init sme_enable(void) { }
 
 #endif	/* CONFIG_AMD_MEM_ENCRYPT */
 
+/*
+ * The __sme_pa() and __sme_pa_nodebug() macros are meant for use when
+ * writing to or comparing values from the cr3 register.  Having the
+ * encryption mask set in cr3 enables the PGD entry to be encrypted and
+ * avoid special case handling of PGD allocations.
+ */
+#define __sme_pa(x)		(__pa(x) | sme_me_mask)
+#define __sme_pa_nodebug(x)	(__pa_nodebug(x) | sme_me_mask)
+
 #endif	/* __ASSEMBLY__ */
 
 #endif	/* __X86_MEM_ENCRYPT_H__ */
diff --git a/arch/x86/include/asm/page_types.h b/arch/x86/include/asm/page_types.h
index 7bd0099..b98ed9d 100644
--- a/arch/x86/include/asm/page_types.h
+++ b/arch/x86/include/asm/page_types.h
@@ -3,6 +3,7 @@
 
 #include <linux/const.h>
 #include <linux/types.h>
+#include <linux/mem_encrypt.h>
 
 /* PAGE_SHIFT determines the page size */
 #define PAGE_SHIFT		12
@@ -15,7 +16,7 @@
 #define PUD_PAGE_SIZE		(_AC(1, UL) << PUD_SHIFT)
 #define PUD_PAGE_MASK		(~(PUD_PAGE_SIZE-1))
 
-#define __PHYSICAL_MASK		((phys_addr_t)((1ULL << __PHYSICAL_MASK_SHIFT) - 1))
+#define __PHYSICAL_MASK		((phys_addr_t)(__sme_clr((1ULL << __PHYSICAL_MASK_SHIFT) - 1)))
 #define __VIRTUAL_MASK		((1UL << __VIRTUAL_MASK_SHIFT) - 1)
 
 /* Cast *PAGE_MASK to a signed type so that it is sign-extended if
diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h
index b64ea52..c6452cb 100644
--- a/arch/x86/include/asm/pgtable.h
+++ b/arch/x86/include/asm/pgtable.h
@@ -1,6 +1,7 @@
 #ifndef _ASM_X86_PGTABLE_H
 #define _ASM_X86_PGTABLE_H
 
+#include <linux/mem_encrypt.h>
 #include <asm/page.h>
 #include <asm/pgtable_types.h>
 
@@ -13,6 +14,12 @@
 		     cachemode2protval(_PAGE_CACHE_MODE_UC_MINUS)))	\
 	 : (prot))
 
+/*
+ * Macros to add or remove encryption attribute
+ */
+#define pgprot_encrypted(prot)	__pgprot(__sme_set(pgprot_val(prot)))
+#define pgprot_decrypted(prot)	__pgprot(__sme_clr(pgprot_val(prot)))
+
 #ifndef __ASSEMBLY__
 #include <asm/x86_init.h>
 
@@ -38,6 +45,8 @@
 
 extern struct mm_struct *pgd_page_get_mm(struct page *page);
 
+extern pmdval_t early_pmd_flags;
+
 #ifdef CONFIG_PARAVIRT
 #include <asm/paravirt.h>
 #else  /* !CONFIG_PARAVIRT */
diff --git a/arch/x86/include/asm/pgtable_types.h b/arch/x86/include/asm/pgtable_types.h
index bf9638e..de32ca3 100644
--- a/arch/x86/include/asm/pgtable_types.h
+++ b/arch/x86/include/asm/pgtable_types.h
@@ -2,6 +2,8 @@
 #define _ASM_X86_PGTABLE_DEFS_H
 
 #include <linux/const.h>
+#include <linux/mem_encrypt.h>
+
 #include <asm/page_types.h>
 
 #define FIRST_USER_ADDRESS	0UL
@@ -121,10 +123,10 @@
 
 #define _PAGE_PROTNONE	(_AT(pteval_t, 1) << _PAGE_BIT_PROTNONE)
 
-#define _PAGE_TABLE	(_PAGE_PRESENT | _PAGE_RW | _PAGE_USER |	\
-			 _PAGE_ACCESSED | _PAGE_DIRTY)
-#define _KERNPG_TABLE	(_PAGE_PRESENT | _PAGE_RW | _PAGE_ACCESSED |	\
-			 _PAGE_DIRTY)
+#define _PAGE_TABLE_NOENC	(_PAGE_PRESENT | _PAGE_RW | _PAGE_USER |\
+				 _PAGE_ACCESSED | _PAGE_DIRTY)
+#define _KERNPG_TABLE_NOENC	(_PAGE_PRESENT | _PAGE_RW |		\
+				 _PAGE_ACCESSED | _PAGE_DIRTY)
 
 /*
  * Set of bits not changed in pte_modify.  The pte's
@@ -191,18 +193,29 @@ enum page_cache_mode {
 #define __PAGE_KERNEL_IO		(__PAGE_KERNEL)
 #define __PAGE_KERNEL_IO_NOCACHE	(__PAGE_KERNEL_NOCACHE)
 
-#define PAGE_KERNEL			__pgprot(__PAGE_KERNEL)
-#define PAGE_KERNEL_RO			__pgprot(__PAGE_KERNEL_RO)
-#define PAGE_KERNEL_EXEC		__pgprot(__PAGE_KERNEL_EXEC)
-#define PAGE_KERNEL_RX			__pgprot(__PAGE_KERNEL_RX)
-#define PAGE_KERNEL_NOCACHE		__pgprot(__PAGE_KERNEL_NOCACHE)
-#define PAGE_KERNEL_LARGE		__pgprot(__PAGE_KERNEL_LARGE)
-#define PAGE_KERNEL_LARGE_EXEC		__pgprot(__PAGE_KERNEL_LARGE_EXEC)
-#define PAGE_KERNEL_VSYSCALL		__pgprot(__PAGE_KERNEL_VSYSCALL)
-#define PAGE_KERNEL_VVAR		__pgprot(__PAGE_KERNEL_VVAR)
-
-#define PAGE_KERNEL_IO			__pgprot(__PAGE_KERNEL_IO)
-#define PAGE_KERNEL_IO_NOCACHE		__pgprot(__PAGE_KERNEL_IO_NOCACHE)
+#ifndef __ASSEMBLY__
+
+#define _PAGE_ENC	(_AT(pteval_t, sme_me_mask))
+
+#define _PAGE_TABLE	(_PAGE_PRESENT | _PAGE_RW | _PAGE_USER |	\
+			 _PAGE_ACCESSED | _PAGE_DIRTY | _PAGE_ENC)
+#define _KERNPG_TABLE	(_PAGE_PRESENT | _PAGE_RW | _PAGE_ACCESSED |	\
+			 _PAGE_DIRTY | _PAGE_ENC)
+
+#define PAGE_KERNEL		__pgprot(__PAGE_KERNEL | _PAGE_ENC)
+#define PAGE_KERNEL_RO		__pgprot(__PAGE_KERNEL_RO | _PAGE_ENC)
+#define PAGE_KERNEL_EXEC	__pgprot(__PAGE_KERNEL_EXEC | _PAGE_ENC)
+#define PAGE_KERNEL_RX		__pgprot(__PAGE_KERNEL_RX | _PAGE_ENC)
+#define PAGE_KERNEL_NOCACHE	__pgprot(__PAGE_KERNEL_NOCACHE | _PAGE_ENC)
+#define PAGE_KERNEL_LARGE	__pgprot(__PAGE_KERNEL_LARGE | _PAGE_ENC)
+#define PAGE_KERNEL_LARGE_EXEC	__pgprot(__PAGE_KERNEL_LARGE_EXEC | _PAGE_ENC)
+#define PAGE_KERNEL_VSYSCALL	__pgprot(__PAGE_KERNEL_VSYSCALL | _PAGE_ENC)
+#define PAGE_KERNEL_VVAR	__pgprot(__PAGE_KERNEL_VVAR | _PAGE_ENC)
+
+#define PAGE_KERNEL_IO		__pgprot(__PAGE_KERNEL_IO)
+#define PAGE_KERNEL_IO_NOCACHE	__pgprot(__PAGE_KERNEL_IO_NOCACHE)
+
+#endif	/* __ASSEMBLY__ */
 
 /*         xwr */
 #define __P000	PAGE_NONE
diff --git a/arch/x86/include/asm/processor.h b/arch/x86/include/asm/processor.h
index f3b1b27..8010c97 100644
--- a/arch/x86/include/asm/processor.h
+++ b/arch/x86/include/asm/processor.h
@@ -29,6 +29,7 @@
 #include <linux/math64.h>
 #include <linux/err.h>
 #include <linux/irqflags.h>
+#include <linux/mem_encrypt.h>
 
 /*
  * We handle most unaligned accesses in hardware.  On the other hand
@@ -241,7 +242,7 @@ static inline unsigned long read_cr3_pa(void)
 
 static inline void load_cr3(pgd_t *pgdir)
 {
-	write_cr3(__pa(pgdir));
+	write_cr3(__sme_pa(pgdir));
 }
 
 #ifdef CONFIG_X86_32
diff --git a/arch/x86/kernel/espfix_64.c b/arch/x86/kernel/espfix_64.c
index 6b91e2e..9c4e7ba 100644
--- a/arch/x86/kernel/espfix_64.c
+++ b/arch/x86/kernel/espfix_64.c
@@ -195,7 +195,7 @@ void init_espfix_ap(int cpu)
 
 	pte_p = pte_offset_kernel(&pmd, addr);
 	stack_page = page_address(alloc_pages_node(node, GFP_KERNEL, 0));
-	pte = __pte(__pa(stack_page) | (__PAGE_KERNEL_RO & ptemask));
+	pte = __pte(__pa(stack_page) | ((__PAGE_KERNEL_RO | _PAGE_ENC) & ptemask));
 	for (n = 0; n < ESPFIX_PTE_CLONES; n++)
 		set_pte(&pte_p[n*PTE_STRIDE], pte);
 
diff --git a/arch/x86/kernel/head64.c b/arch/x86/kernel/head64.c
index 1f0ddcc..5cd0b72 100644
--- a/arch/x86/kernel/head64.c
+++ b/arch/x86/kernel/head64.c
@@ -102,7 +102,7 @@ unsigned long __head __startup_64(unsigned long physaddr)
 
 	pud = fixup_pointer(early_dynamic_pgts[next_early_pgt++], physaddr);
 	pmd = fixup_pointer(early_dynamic_pgts[next_early_pgt++], physaddr);
-	pgtable_flags = _KERNPG_TABLE + sme_get_me_mask();
+	pgtable_flags = _KERNPG_TABLE_NOENC + sme_get_me_mask();
 
 	if (IS_ENABLED(CONFIG_X86_5LEVEL)) {
 		p4d = fixup_pointer(early_dynamic_pgts[next_early_pgt++], physaddr);
@@ -177,7 +177,7 @@ static void __init reset_early_page_tables(void)
 {
 	memset(early_top_pgt, 0, sizeof(pgd_t)*(PTRS_PER_PGD-1));
 	next_early_pgt = 0;
-	write_cr3(__pa_nodebug(early_top_pgt));
+	write_cr3(__sme_pa_nodebug(early_top_pgt));
 }
 
 /* Create a new PMD entry */
@@ -310,6 +310,13 @@ asmlinkage __visible void __init x86_64_start_kernel(char * real_mode_data)
 
 	clear_page(init_top_pgt);
 
+	/*
+	 * SME support may update early_pmd_flags to include the memory
+	 * encryption mask, so it needs to be called before anything
+	 * that may generate a page fault.
+	 */
+	sme_early_init();
+
 	kasan_early_init();
 
 	for (i = 0; i < NUM_EXCEPTION_VECTORS; i++)
diff --git a/arch/x86/kernel/head_64.S b/arch/x86/kernel/head_64.S
index ec5d5e9..513cbb0 100644
--- a/arch/x86/kernel/head_64.S
+++ b/arch/x86/kernel/head_64.S
@@ -351,9 +351,9 @@ GLOBAL(name)
 NEXT_PAGE(early_top_pgt)
 	.fill	511,8,0
 #ifdef CONFIG_X86_5LEVEL
-	.quad	level4_kernel_pgt - __START_KERNEL_map + _PAGE_TABLE
+	.quad	level4_kernel_pgt - __START_KERNEL_map + _PAGE_TABLE_NOENC
 #else
-	.quad	level3_kernel_pgt - __START_KERNEL_map + _PAGE_TABLE
+	.quad	level3_kernel_pgt - __START_KERNEL_map + _PAGE_TABLE_NOENC
 #endif
 
 NEXT_PAGE(early_dynamic_pgts)
@@ -366,15 +366,15 @@ NEXT_PAGE(init_top_pgt)
 	.fill	512,8,0
 #else
 NEXT_PAGE(init_top_pgt)
-	.quad   level3_ident_pgt - __START_KERNEL_map + _KERNPG_TABLE
+	.quad   level3_ident_pgt - __START_KERNEL_map + _KERNPG_TABLE_NOENC
 	.org    init_top_pgt + PGD_PAGE_OFFSET*8, 0
-	.quad   level3_ident_pgt - __START_KERNEL_map + _KERNPG_TABLE
+	.quad   level3_ident_pgt - __START_KERNEL_map + _KERNPG_TABLE_NOENC
 	.org    init_top_pgt + PGD_START_KERNEL*8, 0
 	/* (2^48-(2*1024*1024*1024))/(2^39) = 511 */
-	.quad   level3_kernel_pgt - __START_KERNEL_map + _PAGE_TABLE
+	.quad   level3_kernel_pgt - __START_KERNEL_map + _PAGE_TABLE_NOENC
 
 NEXT_PAGE(level3_ident_pgt)
-	.quad	level2_ident_pgt - __START_KERNEL_map + _KERNPG_TABLE
+	.quad	level2_ident_pgt - __START_KERNEL_map + _KERNPG_TABLE_NOENC
 	.fill	511, 8, 0
 NEXT_PAGE(level2_ident_pgt)
 	/* Since I easily can, map the first 1G.
@@ -386,14 +386,14 @@ NEXT_PAGE(level2_ident_pgt)
 #ifdef CONFIG_X86_5LEVEL
 NEXT_PAGE(level4_kernel_pgt)
 	.fill	511,8,0
-	.quad	level3_kernel_pgt - __START_KERNEL_map + _PAGE_TABLE
+	.quad	level3_kernel_pgt - __START_KERNEL_map + _PAGE_TABLE_NOENC
 #endif
 
 NEXT_PAGE(level3_kernel_pgt)
 	.fill	L3_START_KERNEL,8,0
 	/* (2^48-(2*1024*1024*1024)-((2^39)*511))/(2^30) = 510 */
-	.quad	level2_kernel_pgt - __START_KERNEL_map + _KERNPG_TABLE
-	.quad	level2_fixmap_pgt - __START_KERNEL_map + _PAGE_TABLE
+	.quad	level2_kernel_pgt - __START_KERNEL_map + _KERNPG_TABLE_NOENC
+	.quad	level2_fixmap_pgt - __START_KERNEL_map + _PAGE_TABLE_NOENC
 
 NEXT_PAGE(level2_kernel_pgt)
 	/*
@@ -411,7 +411,7 @@ NEXT_PAGE(level2_kernel_pgt)
 
 NEXT_PAGE(level2_fixmap_pgt)
 	.fill	506,8,0
-	.quad	level1_fixmap_pgt - __START_KERNEL_map + _PAGE_TABLE
+	.quad	level1_fixmap_pgt - __START_KERNEL_map + _PAGE_TABLE_NOENC
 	/* 8MB reserved for vsyscalls + a 2MB hole = 4 + 1 entries */
 	.fill	5,8,0
 
diff --git a/arch/x86/mm/kasan_init_64.c b/arch/x86/mm/kasan_init_64.c
index 88215ac..d7cc830 100644
--- a/arch/x86/mm/kasan_init_64.c
+++ b/arch/x86/mm/kasan_init_64.c
@@ -92,7 +92,7 @@ static int kasan_die_handler(struct notifier_block *self,
 void __init kasan_early_init(void)
 {
 	int i;
-	pteval_t pte_val = __pa_nodebug(kasan_zero_page) | __PAGE_KERNEL;
+	pteval_t pte_val = __pa_nodebug(kasan_zero_page) | __PAGE_KERNEL | _PAGE_ENC;
 	pmdval_t pmd_val = __pa_nodebug(kasan_zero_pte) | _KERNPG_TABLE;
 	pudval_t pud_val = __pa_nodebug(kasan_zero_pmd) | _KERNPG_TABLE;
 	p4dval_t p4d_val = __pa_nodebug(kasan_zero_pud) | _KERNPG_TABLE;
@@ -158,7 +158,7 @@ void __init kasan_init(void)
 	 */
 	memset(kasan_zero_page, 0, PAGE_SIZE);
 	for (i = 0; i < PTRS_PER_PTE; i++) {
-		pte_t pte = __pte(__pa(kasan_zero_page) | __PAGE_KERNEL_RO);
+		pte_t pte = __pte(__pa(kasan_zero_page) | __PAGE_KERNEL_RO | _PAGE_ENC);
 		set_pte(&kasan_zero_pte[i], pte);
 	}
 	/* Flush TLBs again to be sure that write protection applied. */
diff --git a/arch/x86/mm/mem_encrypt.c b/arch/x86/mm/mem_encrypt.c
index 3ac6f99..f973d3d 100644
--- a/arch/x86/mm/mem_encrypt.c
+++ b/arch/x86/mm/mem_encrypt.c
@@ -12,6 +12,7 @@
 
 #include <linux/linkage.h>
 #include <linux/init.h>
+#include <linux/mm.h>
 
 /*
  * Since SME related variables are set early in the boot process they must
@@ -21,6 +22,22 @@
 unsigned long sme_me_mask __section(.data) = 0;
 EXPORT_SYMBOL_GPL(sme_me_mask);
 
+void __init sme_early_init(void)
+{
+	unsigned int i;
+
+	if (!sme_me_mask)
+		return;
+
+	early_pmd_flags = __sme_set(early_pmd_flags);
+
+	__supported_pte_mask = __sme_set(__supported_pte_mask);
+
+	/* Update the protection map with memory encryption mask */
+	for (i = 0; i < ARRAY_SIZE(protection_map); i++)
+		protection_map[i] = pgprot_encrypted(protection_map[i]);
+}
+
 void __init sme_encrypt_kernel(void)
 {
 }
diff --git a/arch/x86/mm/pageattr.c b/arch/x86/mm/pageattr.c
index c8520b2..e7d3866 100644
--- a/arch/x86/mm/pageattr.c
+++ b/arch/x86/mm/pageattr.c
@@ -2014,6 +2014,9 @@ int kernel_map_pages_in_pgd(pgd_t *pgd, u64 pfn, unsigned long address,
 	if (!(page_flags & _PAGE_RW))
 		cpa.mask_clr = __pgprot(_PAGE_RW);
 
+	if (!(page_flags & _PAGE_ENC))
+		cpa.mask_clr = pgprot_encrypted(cpa.mask_clr);
+
 	cpa.mask_set = __pgprot(_PAGE_PRESENT | page_flags);
 
 	retval = __change_page_attr_set_clr(&cpa, 0);
diff --git a/include/asm-generic/pgtable.h b/include/asm-generic/pgtable.h
index 7dfa767..4d7bb98 100644
--- a/include/asm-generic/pgtable.h
+++ b/include/asm-generic/pgtable.h
@@ -583,6 +583,18 @@ static inline void ptep_modify_prot_commit(struct mm_struct *mm,
 #endif /* CONFIG_MMU */
 
 /*
+ * No-op macros that just return the current protection value. Defined here
+ * because these macros can be used used even if CONFIG_MMU is not defined.
+ */
+#ifndef pgprot_encrypted
+#define pgprot_encrypted(prot)	(prot)
+#endif
+
+#ifndef pgprot_decrypted
+#define pgprot_decrypted(prot)	(prot)
+#endif
+
+/*
  * A facility to provide lazy MMU batching.  This allows PTE updates and
  * page invalidations to be delayed until a call to leave lazy MMU mode
  * is issued.  Some architectures may benefit from doing this, and it is
diff --git a/include/linux/mem_encrypt.h b/include/linux/mem_encrypt.h
index 570f4fc..1255f09 100644
--- a/include/linux/mem_encrypt.h
+++ b/include/linux/mem_encrypt.h
@@ -35,6 +35,14 @@ static inline unsigned long sme_get_me_mask(void)
 	return sme_me_mask;
 }
 
+/*
+ * The __sme_set() and __sme_clr() macros are useful for adding or removing
+ * the encryption mask from a value (e.g. when dealing with pagetable
+ * entries).
+ */
+#define __sme_set(x)		((unsigned long)(x) | sme_me_mask)
+#define __sme_clr(x)		((unsigned long)(x) & ~sme_me_mask)
+
 #endif	/* __ASSEMBLY__ */
 
 #endif	/* __MEM_ENCRYPT_H__ */



From thomas.lendacky at amd.com  Tue Jun 27 08:09:14 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Tue, 27 Jun 2017 10:09:14 -0500
Subject: [PATCH v8 RESEND 11/38] x86/mm: Add SME support for read_cr3_pa()
In-Reply-To: <20170627150718.17428.81813.stgit@tlendack-t1.amdoffice.net>
References: <20170627150718.17428.81813.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170627150914.17428.14700.stgit@tlendack-t1.amdoffice.net>

The cr3 register entry can contain the SME encryption mask that indicates
the PGD is encrypted.  The encryption mask should not be used when
creating a virtual address from the cr3 register, so remove the SME
encryption mask in the read_cr3_pa() function.

During early boot SME will need to use a native version of read_cr3_pa(),
so create native_read_cr3_pa().

Reviewed-by: Borislav Petkov <bp at suse.de>
Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/include/asm/processor-flags.h |    5 +++--
 arch/x86/include/asm/processor.h       |    5 +++++
 2 files changed, 8 insertions(+), 2 deletions(-)

diff --git a/arch/x86/include/asm/processor-flags.h b/arch/x86/include/asm/processor-flags.h
index 79aa2f9..f5d3e50 100644
--- a/arch/x86/include/asm/processor-flags.h
+++ b/arch/x86/include/asm/processor-flags.h
@@ -2,6 +2,7 @@
 #define _ASM_X86_PROCESSOR_FLAGS_H
 
 #include <uapi/asm/processor-flags.h>
+#include <linux/mem_encrypt.h>
 
 #ifdef CONFIG_VM86
 #define X86_VM_MASK	X86_EFLAGS_VM
@@ -32,8 +33,8 @@
  * CR3_ADDR_MASK is the mask used by read_cr3_pa().
  */
 #ifdef CONFIG_X86_64
-/* Mask off the address space ID bits. */
-#define CR3_ADDR_MASK 0x7FFFFFFFFFFFF000ull
+/* Mask off the address space ID and SME encryption bits. */
+#define CR3_ADDR_MASK __sme_clr(0x7FFFFFFFFFFFF000ull)
 #define CR3_PCID_MASK 0xFFFull
 #else
 /*
diff --git a/arch/x86/include/asm/processor.h b/arch/x86/include/asm/processor.h
index 8010c97..ab878bd 100644
--- a/arch/x86/include/asm/processor.h
+++ b/arch/x86/include/asm/processor.h
@@ -240,6 +240,11 @@ static inline unsigned long read_cr3_pa(void)
 	return __read_cr3() & CR3_ADDR_MASK;
 }
 
+static inline unsigned long native_read_cr3_pa(void)
+{
+	return __native_read_cr3() & CR3_ADDR_MASK;
+}
+
 static inline void load_cr3(pgd_t *pgdir)
 {
 	write_cr3(__sme_pa(pgdir));



From thomas.lendacky at amd.com  Tue Jun 27 08:09:39 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Tue, 27 Jun 2017 10:09:39 -0500
Subject: [PATCH v8 RESEND 13/38] x86/mm: Add support for early
 encrypt/decrypt of memory
In-Reply-To: <20170627150718.17428.81813.stgit@tlendack-t1.amdoffice.net>
References: <20170627150718.17428.81813.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170627150939.17428.62679.stgit@tlendack-t1.amdoffice.net>

Add support to be able to either encrypt or decrypt data in place during
the early stages of booting the kernel. This does not change the memory
encryption attribute - it is used for ensuring that data present in either
an encrypted or decrypted memory area is in the proper state (for example
the initrd will have been loaded by the boot loader and will not be
encrypted, but the memory that it resides in is marked as encrypted).

Reviewed-by: Borislav Petkov <bp at suse.de>
Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/include/asm/mem_encrypt.h |   10 +++++
 arch/x86/mm/mem_encrypt.c          |   76 ++++++++++++++++++++++++++++++++++++
 2 files changed, 86 insertions(+)

diff --git a/arch/x86/include/asm/mem_encrypt.h b/arch/x86/include/asm/mem_encrypt.h
index dbae7a5..8baa35b 100644
--- a/arch/x86/include/asm/mem_encrypt.h
+++ b/arch/x86/include/asm/mem_encrypt.h
@@ -21,6 +21,11 @@
 
 extern unsigned long sme_me_mask;
 
+void __init sme_early_encrypt(resource_size_t paddr,
+			      unsigned long size);
+void __init sme_early_decrypt(resource_size_t paddr,
+			      unsigned long size);
+
 void __init sme_early_init(void);
 
 void __init sme_encrypt_kernel(void);
@@ -30,6 +35,11 @@
 
 #define sme_me_mask	0UL
 
+static inline void __init sme_early_encrypt(resource_size_t paddr,
+					    unsigned long size) { }
+static inline void __init sme_early_decrypt(resource_size_t paddr,
+					    unsigned long size) { }
+
 static inline void __init sme_early_init(void) { }
 
 static inline void __init sme_encrypt_kernel(void) { }
diff --git a/arch/x86/mm/mem_encrypt.c b/arch/x86/mm/mem_encrypt.c
index f973d3d..54bb73c 100644
--- a/arch/x86/mm/mem_encrypt.c
+++ b/arch/x86/mm/mem_encrypt.c
@@ -14,6 +14,9 @@
 #include <linux/init.h>
 #include <linux/mm.h>
 
+#include <asm/tlbflush.h>
+#include <asm/fixmap.h>
+
 /*
  * Since SME related variables are set early in the boot process they must
  * reside in the .data section so as not to be zeroed out when the .bss
@@ -22,6 +25,79 @@
 unsigned long sme_me_mask __section(.data) = 0;
 EXPORT_SYMBOL_GPL(sme_me_mask);
 
+/* Buffer used for early in-place encryption by BSP, no locking needed */
+static char sme_early_buffer[PAGE_SIZE] __aligned(PAGE_SIZE);
+
+/*
+ * This routine does not change the underlying encryption setting of the
+ * page(s) that map this memory. It assumes that eventually the memory is
+ * meant to be accessed as either encrypted or decrypted but the contents
+ * are currently not in the desired state.
+ *
+ * This routine follows the steps outlined in the AMD64 Architecture
+ * Programmer's Manual Volume 2, Section 7.10.8 Encrypt-in-Place.
+ */
+static void __init __sme_early_enc_dec(resource_size_t paddr,
+				       unsigned long size, bool enc)
+{
+	void *src, *dst;
+	size_t len;
+
+	if (!sme_me_mask)
+		return;
+
+	local_flush_tlb();
+	wbinvd();
+
+	/*
+	 * There are limited number of early mapping slots, so map (at most)
+	 * one page at time.
+	 */
+	while (size) {
+		len = min_t(size_t, sizeof(sme_early_buffer), size);
+
+		/*
+		 * Create mappings for the current and desired format of
+		 * the memory. Use a write-protected mapping for the source.
+		 */
+		src = enc ? early_memremap_decrypted_wp(paddr, len) :
+			    early_memremap_encrypted_wp(paddr, len);
+
+		dst = enc ? early_memremap_encrypted(paddr, len) :
+			    early_memremap_decrypted(paddr, len);
+
+		/*
+		 * If a mapping can't be obtained to perform the operation,
+		 * then eventual access of that area in the desired mode
+		 * will cause a crash.
+		 */
+		BUG_ON(!src || !dst);
+
+		/*
+		 * Use a temporary buffer, of cache-line multiple size, to
+		 * avoid data corruption as documented in the APM.
+		 */
+		memcpy(sme_early_buffer, src, len);
+		memcpy(dst, sme_early_buffer, len);
+
+		early_memunmap(dst, len);
+		early_memunmap(src, len);
+
+		paddr += len;
+		size -= len;
+	}
+}
+
+void __init sme_early_encrypt(resource_size_t paddr, unsigned long size)
+{
+	__sme_early_enc_dec(paddr, size, true);
+}
+
+void __init sme_early_decrypt(resource_size_t paddr, unsigned long size)
+{
+	__sme_early_enc_dec(paddr, size, false);
+}
+
 void __init sme_early_init(void)
 {
 	unsigned int i;



From thomas.lendacky at amd.com  Tue Jun 27 08:09:48 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Tue, 27 Jun 2017 10:09:48 -0500
Subject: [PATCH v8 RESEND 14/38] x86/mm: Insure that boot memory areas are
 mapped properly
In-Reply-To: <20170627150718.17428.81813.stgit@tlendack-t1.amdoffice.net>
References: <20170627150718.17428.81813.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170627150948.17428.69974.stgit@tlendack-t1.amdoffice.net>

The boot data and command line data are present in memory in a decrypted
state and are copied early in the boot process.  The early page fault
support will map these areas as encrypted, so before attempting to copy
them, add decrypted mappings so the data is accessed properly when copied.

For the initrd, encrypt this data in place. Since the future mapping of
the initrd area will be mapped as encrypted the data will be accessed
properly.

Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/include/asm/mem_encrypt.h |    6 +++
 arch/x86/include/asm/pgtable.h     |    3 ++
 arch/x86/kernel/head64.c           |   30 +++++++++++++++--
 arch/x86/kernel/setup.c            |    9 +++++
 arch/x86/mm/kasan_init_64.c        |    2 +
 arch/x86/mm/mem_encrypt.c          |   63 ++++++++++++++++++++++++++++++++++++
 6 files changed, 108 insertions(+), 5 deletions(-)

diff --git a/arch/x86/include/asm/mem_encrypt.h b/arch/x86/include/asm/mem_encrypt.h
index 8baa35b..ab1fe77 100644
--- a/arch/x86/include/asm/mem_encrypt.h
+++ b/arch/x86/include/asm/mem_encrypt.h
@@ -26,6 +26,9 @@ void __init sme_early_encrypt(resource_size_t paddr,
 void __init sme_early_decrypt(resource_size_t paddr,
 			      unsigned long size);
 
+void __init sme_map_bootdata(char *real_mode_data);
+void __init sme_unmap_bootdata(char *real_mode_data);
+
 void __init sme_early_init(void);
 
 void __init sme_encrypt_kernel(void);
@@ -40,6 +43,9 @@ static inline void __init sme_early_encrypt(resource_size_t paddr,
 static inline void __init sme_early_decrypt(resource_size_t paddr,
 					    unsigned long size) { }
 
+static inline void __init sme_map_bootdata(char *real_mode_data) { }
+static inline void __init sme_unmap_bootdata(char *real_mode_data) { }
+
 static inline void __init sme_early_init(void) { }
 
 static inline void __init sme_encrypt_kernel(void) { }
diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h
index c6452cb..bbeae4a 100644
--- a/arch/x86/include/asm/pgtable.h
+++ b/arch/x86/include/asm/pgtable.h
@@ -23,6 +23,9 @@
 #ifndef __ASSEMBLY__
 #include <asm/x86_init.h>
 
+extern pgd_t early_top_pgt[PTRS_PER_PGD];
+int __init __early_make_pgtable(unsigned long address, pmdval_t pmd);
+
 void ptdump_walk_pgd_level(struct seq_file *m, pgd_t *pgd);
 void ptdump_walk_pgd_level_checkwx(void);
 
diff --git a/arch/x86/kernel/head64.c b/arch/x86/kernel/head64.c
index 5cd0b72..0cdb53b 100644
--- a/arch/x86/kernel/head64.c
+++ b/arch/x86/kernel/head64.c
@@ -34,7 +34,6 @@
 /*
  * Manage page tables very early on.
  */
-extern pgd_t early_top_pgt[PTRS_PER_PGD];
 extern pmd_t early_dynamic_pgts[EARLY_DYNAMIC_PAGE_TABLES][PTRS_PER_PMD];
 static unsigned int __initdata next_early_pgt;
 pmdval_t early_pmd_flags = __PAGE_KERNEL_LARGE & ~(_PAGE_GLOBAL | _PAGE_NX);
@@ -181,13 +180,13 @@ static void __init reset_early_page_tables(void)
 }
 
 /* Create a new PMD entry */
-int __init early_make_pgtable(unsigned long address)
+int __init __early_make_pgtable(unsigned long address, pmdval_t pmd)
 {
 	unsigned long physaddr = address - __PAGE_OFFSET;
 	pgdval_t pgd, *pgd_p;
 	p4dval_t p4d, *p4d_p;
 	pudval_t pud, *pud_p;
-	pmdval_t pmd, *pmd_p;
+	pmdval_t *pmd_p;
 
 	/* Invalid address or early pgt is done ?  */
 	if (physaddr >= MAXMEM || read_cr3_pa() != __pa_nodebug(early_top_pgt))
@@ -246,12 +245,21 @@ int __init early_make_pgtable(unsigned long address)
 		memset(pmd_p, 0, sizeof(*pmd_p) * PTRS_PER_PMD);
 		*pud_p = (pudval_t)pmd_p - __START_KERNEL_map + phys_base + _KERNPG_TABLE;
 	}
-	pmd = (physaddr & PMD_MASK) + early_pmd_flags;
 	pmd_p[pmd_index(address)] = pmd;
 
 	return 0;
 }
 
+int __init early_make_pgtable(unsigned long address)
+{
+	unsigned long physaddr = address - __PAGE_OFFSET;
+	pmdval_t pmd;
+
+	pmd = (physaddr & PMD_MASK) + early_pmd_flags;
+
+	return __early_make_pgtable(address, pmd);
+}
+
 /* Don't add a printk in there. printk relies on the PDA which is not initialized 
    yet. */
 static void __init clear_bss(void)
@@ -274,6 +282,12 @@ static void __init copy_bootdata(char *real_mode_data)
 	char * command_line;
 	unsigned long cmd_line_ptr;
 
+	/*
+	 * If SME is active, this will create decrypted mappings of the
+	 * boot data in advance of the copy operations.
+	 */
+	sme_map_bootdata(real_mode_data);
+
 	memcpy(&boot_params, real_mode_data, sizeof boot_params);
 	sanitize_boot_params(&boot_params);
 	cmd_line_ptr = get_cmd_line_ptr();
@@ -281,6 +295,14 @@ static void __init copy_bootdata(char *real_mode_data)
 		command_line = __va(cmd_line_ptr);
 		memcpy(boot_command_line, command_line, COMMAND_LINE_SIZE);
 	}
+
+	/*
+	 * The old boot data is no longer needed and won't be reserved,
+	 * freeing up that memory for use by the system. If SME is active,
+	 * we need to remove the mappings that were created so that the
+	 * memory doesn't remain mapped as decrypted.
+	 */
+	sme_unmap_bootdata(real_mode_data);
 }
 
 asmlinkage __visible void __init x86_64_start_kernel(char * real_mode_data)
diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
index 65622f0..31ae85e 100644
--- a/arch/x86/kernel/setup.c
+++ b/arch/x86/kernel/setup.c
@@ -69,6 +69,7 @@
 #include <linux/crash_dump.h>
 #include <linux/tboot.h>
 #include <linux/jiffies.h>
+#include <linux/mem_encrypt.h>
 
 #include <linux/usb/xhci-dbgp.h>
 #include <video/edid.h>
@@ -374,6 +375,14 @@ static void __init reserve_initrd(void)
 	    !ramdisk_image || !ramdisk_size)
 		return;		/* No initrd provided by bootloader */
 
+	/*
+	 * If SME is active, this memory will be marked encrypted by the
+	 * kernel when it is accessed (including relocation). However, the
+	 * ramdisk image was loaded decrypted by the bootloader, so make
+	 * sure that it is encrypted before accessing it.
+	 */
+	sme_early_encrypt(ramdisk_image, ramdisk_end - ramdisk_image);
+
 	initrd_start = 0;
 
 	mapped_size = memblock_mem_size(max_pfn_mapped);
diff --git a/arch/x86/mm/kasan_init_64.c b/arch/x86/mm/kasan_init_64.c
index d7cc830..1b8791f 100644
--- a/arch/x86/mm/kasan_init_64.c
+++ b/arch/x86/mm/kasan_init_64.c
@@ -11,8 +11,8 @@
 #include <asm/e820/types.h>
 #include <asm/tlbflush.h>
 #include <asm/sections.h>
+#include <asm/pgtable.h>
 
-extern pgd_t early_top_pgt[PTRS_PER_PGD];
 extern struct range pfn_mapped[E820_MAX_ENTRIES];
 
 static int __init map_range(struct range *range)
diff --git a/arch/x86/mm/mem_encrypt.c b/arch/x86/mm/mem_encrypt.c
index 54bb73c..0843d02 100644
--- a/arch/x86/mm/mem_encrypt.c
+++ b/arch/x86/mm/mem_encrypt.c
@@ -16,6 +16,8 @@
 
 #include <asm/tlbflush.h>
 #include <asm/fixmap.h>
+#include <asm/setup.h>
+#include <asm/bootparam.h>
 
 /*
  * Since SME related variables are set early in the boot process they must
@@ -98,6 +100,67 @@ void __init sme_early_decrypt(resource_size_t paddr, unsigned long size)
 	__sme_early_enc_dec(paddr, size, false);
 }
 
+static void __init __sme_early_map_unmap_mem(void *vaddr, unsigned long size,
+					     bool map)
+{
+	unsigned long paddr = (unsigned long)vaddr - __PAGE_OFFSET;
+	pmdval_t pmd_flags, pmd;
+
+	/* Use early_pmd_flags but remove the encryption mask */
+	pmd_flags = __sme_clr(early_pmd_flags);
+
+	do {
+		pmd = map ? (paddr & PMD_MASK) + pmd_flags : 0;
+		__early_make_pgtable((unsigned long)vaddr, pmd);
+
+		vaddr += PMD_SIZE;
+		paddr += PMD_SIZE;
+		size = (size <= PMD_SIZE) ? 0 : size - PMD_SIZE;
+	} while (size);
+
+	__native_flush_tlb();
+}
+
+void __init sme_unmap_bootdata(char *real_mode_data)
+{
+	struct boot_params *boot_data;
+	unsigned long cmdline_paddr;
+
+	if (!sme_active())
+		return;
+
+	/* Get the command line address before unmapping the real_mode_data */
+	boot_data = (struct boot_params *)real_mode_data;
+	cmdline_paddr = boot_data->hdr.cmd_line_ptr | ((u64)boot_data->ext_cmd_line_ptr << 32);
+
+	__sme_early_map_unmap_mem(real_mode_data, sizeof(boot_params), false);
+
+	if (!cmdline_paddr)
+		return;
+
+	__sme_early_map_unmap_mem(__va(cmdline_paddr), COMMAND_LINE_SIZE, false);
+}
+
+void __init sme_map_bootdata(char *real_mode_data)
+{
+	struct boot_params *boot_data;
+	unsigned long cmdline_paddr;
+
+	if (!sme_active())
+		return;
+
+	__sme_early_map_unmap_mem(real_mode_data, sizeof(boot_params), true);
+
+	/* Get the command line address after mapping the real_mode_data */
+	boot_data = (struct boot_params *)real_mode_data;
+	cmdline_paddr = boot_data->hdr.cmd_line_ptr | ((u64)boot_data->ext_cmd_line_ptr << 32);
+
+	if (!cmdline_paddr)
+		return;
+
+	__sme_early_map_unmap_mem(__va(cmdline_paddr), COMMAND_LINE_SIZE, true);
+}
+
 void __init sme_early_init(void)
 {
 	unsigned int i;



From thomas.lendacky at amd.com  Tue Jun 27 08:09:59 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Tue, 27 Jun 2017 10:09:59 -0500
Subject: [PATCH v8 RESEND 15/38] x86/boot/e820: Add support to determine the
 E820 type of an address
In-Reply-To: <20170627150718.17428.81813.stgit@tlendack-t1.amdoffice.net>
References: <20170627150718.17428.81813.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170627150959.17428.15703.stgit@tlendack-t1.amdoffice.net>

Add a function that will return the E820 type associated with an address
range.

Reviewed-by: Borislav Petkov <bp at suse.de>
Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/include/asm/e820/api.h |    2 ++
 arch/x86/kernel/e820.c          |   26 +++++++++++++++++++++++---
 2 files changed, 25 insertions(+), 3 deletions(-)

diff --git a/arch/x86/include/asm/e820/api.h b/arch/x86/include/asm/e820/api.h
index 8e0f8b8..3641f5f 100644
--- a/arch/x86/include/asm/e820/api.h
+++ b/arch/x86/include/asm/e820/api.h
@@ -38,6 +38,8 @@
 extern void e820__reallocate_tables(void);
 extern void e820__register_nosave_regions(unsigned long limit_pfn);
 
+extern int  e820__get_entry_type(u64 start, u64 end);
+
 /*
  * Returns true iff the specified range [start,end) is completely contained inside
  * the ISA region.
diff --git a/arch/x86/kernel/e820.c b/arch/x86/kernel/e820.c
index d78a586..46c9b65 100644
--- a/arch/x86/kernel/e820.c
+++ b/arch/x86/kernel/e820.c
@@ -84,7 +84,8 @@ bool e820__mapped_any(u64 start, u64 end, enum e820_type type)
  * Note: this function only works correctly once the E820 table is sorted and
  * not-overlapping (at least for the range specified), which is the case normally.
  */
-bool __init e820__mapped_all(u64 start, u64 end, enum e820_type type)
+static struct e820_entry *__e820__mapped_all(u64 start, u64 end,
+					     enum e820_type type)
 {
 	int i;
 
@@ -110,9 +111,28 @@ bool __init e820__mapped_all(u64 start, u64 end, enum e820_type type)
 		 * coverage of the desired range exists:
 		 */
 		if (start >= end)
-			return 1;
+			return entry;
 	}
-	return 0;
+
+	return NULL;
+}
+
+/*
+ * This function checks if the entire range <start,end> is mapped with type.
+ */
+bool __init e820__mapped_all(u64 start, u64 end, enum e820_type type)
+{
+	return __e820__mapped_all(start, end, type);
+}
+
+/*
+ * This function returns the type associated with the range <start,end>.
+ */
+int e820__get_entry_type(u64 start, u64 end)
+{
+	struct e820_entry *entry = __e820__mapped_all(start, end, 0);
+
+	return entry ? entry->type : -EINVAL;
 }
 
 /*



From thomas.lendacky at amd.com  Tue Jun 27 08:10:28 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Tue, 27 Jun 2017 10:10:28 -0500
Subject: [PATCH v8 RESEND 17/38] efi: Update efi_mem_type() to return an
 error rather than 0
In-Reply-To: <20170627150718.17428.81813.stgit@tlendack-t1.amdoffice.net>
References: <20170627150718.17428.81813.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170627151028.17428.65111.stgit@tlendack-t1.amdoffice.net>

The efi_mem_type() function currently returns a 0, which maps to
EFI_RESERVED_TYPE, if the function is unable to find a memmap entry for
the supplied physical address. Returning EFI_RESERVED_TYPE implies that
a memmap entry exists, when it doesn't.  Instead of returning 0, change
the function to return a negative error value when no memmap entry is
found.

Reviewed-by: Matt Fleming <matt at codeblueprint.co.uk>
Reviewed-by: Borislav Petkov <bp at suse.de>
Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/ia64/kernel/efi.c      |    4 ++--
 arch/x86/platform/efi/efi.c |    6 +++---
 include/linux/efi.h         |    2 +-
 3 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/arch/ia64/kernel/efi.c b/arch/ia64/kernel/efi.c
index 1212956..8141600 100644
--- a/arch/ia64/kernel/efi.c
+++ b/arch/ia64/kernel/efi.c
@@ -757,14 +757,14 @@ static void __init handle_palo(unsigned long phys_addr)
 	return 0;
 }
 
-u32
+int
 efi_mem_type (unsigned long phys_addr)
 {
 	efi_memory_desc_t *md = efi_memory_descriptor(phys_addr);
 
 	if (md)
 		return md->type;
-	return 0;
+	return -EINVAL;
 }
 
 u64
diff --git a/arch/x86/platform/efi/efi.c b/arch/x86/platform/efi/efi.c
index f084d87..6217b23 100644
--- a/arch/x86/platform/efi/efi.c
+++ b/arch/x86/platform/efi/efi.c
@@ -1035,12 +1035,12 @@ void __init efi_enter_virtual_mode(void)
 /*
  * Convenience functions to obtain memory types and attributes
  */
-u32 efi_mem_type(unsigned long phys_addr)
+int efi_mem_type(unsigned long phys_addr)
 {
 	efi_memory_desc_t *md;
 
 	if (!efi_enabled(EFI_MEMMAP))
-		return 0;
+		return -ENOTSUPP;
 
 	for_each_efi_memory_desc(md) {
 		if ((md->phys_addr <= phys_addr) &&
@@ -1048,7 +1048,7 @@ u32 efi_mem_type(unsigned long phys_addr)
 				  (md->num_pages << EFI_PAGE_SHIFT))))
 			return md->type;
 	}
-	return 0;
+	return -EINVAL;
 }
 
 static int __init arch_parse_efi_cmdline(char *str)
diff --git a/include/linux/efi.h b/include/linux/efi.h
index 8e24f09..4e47f78 100644
--- a/include/linux/efi.h
+++ b/include/linux/efi.h
@@ -985,7 +985,7 @@ static inline void efi_esrt_init(void) { }
 extern int efi_config_parse_tables(void *config_tables, int count, int sz,
 				   efi_config_table_type_t *arch_tables);
 extern u64 efi_get_iobase (void);
-extern u32 efi_mem_type (unsigned long phys_addr);
+extern int efi_mem_type(unsigned long phys_addr);
 extern u64 efi_mem_attributes (unsigned long phys_addr);
 extern u64 efi_mem_attribute (unsigned long phys_addr, unsigned long size);
 extern int __init efi_uart_console_only (void);



From thomas.lendacky at amd.com  Tue Jun 27 08:10:36 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Tue, 27 Jun 2017 10:10:36 -0500
Subject: [PATCH v8 RESEND 18/38] x86/efi: Update EFI pagetable creation to
 work with SME
In-Reply-To: <20170627150718.17428.81813.stgit@tlendack-t1.amdoffice.net>
References: <20170627150718.17428.81813.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170627151036.17428.14824.stgit@tlendack-t1.amdoffice.net>

When SME is active, pagetable entries created for EFI need to have the
encryption mask set as necessary.

When the new pagetable pages are allocated they are mapped encrypted. So,
update the efi_pgt value that will be used in cr3 to include the encryption
mask so that the PGD table can be read successfully. The pagetable mapping
as well as the kernel are also added to the pagetable mapping as encrypted.
All other EFI mappings are mapped decrypted (tables, etc.).

Reviewed-by: Matt Fleming <matt at codeblueprint.co.uk>
Reviewed-by: Borislav Petkov <bp at suse.de>
Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/platform/efi/efi_64.c |   15 +++++++++++----
 1 file changed, 11 insertions(+), 4 deletions(-)

diff --git a/arch/x86/platform/efi/efi_64.c b/arch/x86/platform/efi/efi_64.c
index 9bf72f5..12e8388 100644
--- a/arch/x86/platform/efi/efi_64.c
+++ b/arch/x86/platform/efi/efi_64.c
@@ -327,7 +327,7 @@ void efi_sync_low_kernel_mappings(void)
 
 int __init efi_setup_page_tables(unsigned long pa_memmap, unsigned num_pages)
 {
-	unsigned long pfn, text;
+	unsigned long pfn, text, pf;
 	struct page *page;
 	unsigned npages;
 	pgd_t *pgd;
@@ -335,7 +335,12 @@ int __init efi_setup_page_tables(unsigned long pa_memmap, unsigned num_pages)
 	if (efi_enabled(EFI_OLD_MEMMAP))
 		return 0;
 
-	efi_scratch.efi_pgt = (pgd_t *)__pa(efi_pgd);
+	/*
+	 * Since the PGD is encrypted, set the encryption mask so that when
+	 * this value is loaded into cr3 the PGD will be decrypted during
+	 * the pagetable walk.
+	 */
+	efi_scratch.efi_pgt = (pgd_t *)__sme_pa(efi_pgd);
 	pgd = efi_pgd;
 
 	/*
@@ -345,7 +350,8 @@ int __init efi_setup_page_tables(unsigned long pa_memmap, unsigned num_pages)
 	 * phys_efi_set_virtual_address_map().
 	 */
 	pfn = pa_memmap >> PAGE_SHIFT;
-	if (kernel_map_pages_in_pgd(pgd, pfn, pa_memmap, num_pages, _PAGE_NX | _PAGE_RW)) {
+	pf = _PAGE_NX | _PAGE_RW | _PAGE_ENC;
+	if (kernel_map_pages_in_pgd(pgd, pfn, pa_memmap, num_pages, pf)) {
 		pr_err("Error ident-mapping new memmap (0x%lx)!\n", pa_memmap);
 		return 1;
 	}
@@ -388,7 +394,8 @@ int __init efi_setup_page_tables(unsigned long pa_memmap, unsigned num_pages)
 	text = __pa(_text);
 	pfn = text >> PAGE_SHIFT;
 
-	if (kernel_map_pages_in_pgd(pgd, pfn, text, npages, _PAGE_RW)) {
+	pf = _PAGE_RW | _PAGE_ENC;
+	if (kernel_map_pages_in_pgd(pgd, pfn, text, npages, pf)) {
 		pr_err("Failed to map kernel text 1:1\n");
 		return 1;
 	}



From thomas.lendacky at amd.com  Tue Jun 27 08:10:48 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Tue, 27 Jun 2017 10:10:48 -0500
Subject: [PATCH v8 RESEND 19/38] x86/mm: Add support to access boot related
 data in the clear
In-Reply-To: <20170627150718.17428.81813.stgit@tlendack-t1.amdoffice.net>
References: <20170627150718.17428.81813.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170627151048.17428.56847.stgit@tlendack-t1.amdoffice.net>

Boot data (such as EFI related data) is not encrypted when the system is
booted because UEFI/BIOS does not run with SME active. In order to access
this data properly it needs to be mapped decrypted.

Update early_memremap() to provide an arch specific routine to modify the
pagetable protection attributes before they are applied to the new
mapping. This is used to remove the encryption mask for boot related data.

Update memremap() to provide an arch specific routine to determine if RAM
remapping is allowed.  RAM remapping will cause an encrypted mapping to be
generated. By preventing RAM remapping, ioremap_cache() will be used
instead, which will provide a decrypted mapping of the boot related data.

Reviewed-by: Matt Fleming <matt at codeblueprint.co.uk>
Reviewed-by: Borislav Petkov <bp at suse.de>
Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/include/asm/io.h |    5 +
 arch/x86/mm/ioremap.c     |  179 +++++++++++++++++++++++++++++++++++++++++++++
 include/linux/io.h        |    2 +
 kernel/memremap.c         |   20 ++++-
 mm/early_ioremap.c        |   18 ++++-
 5 files changed, 217 insertions(+), 7 deletions(-)

diff --git a/arch/x86/include/asm/io.h b/arch/x86/include/asm/io.h
index 7afb0e2..09c5557 100644
--- a/arch/x86/include/asm/io.h
+++ b/arch/x86/include/asm/io.h
@@ -381,4 +381,9 @@ extern int __must_check arch_phys_wc_add(unsigned long base,
 #define arch_io_reserve_memtype_wc arch_io_reserve_memtype_wc
 #endif
 
+extern bool arch_memremap_can_ram_remap(resource_size_t offset,
+					unsigned long size,
+					unsigned long flags);
+#define arch_memremap_can_ram_remap arch_memremap_can_ram_remap
+
 #endif /* _ASM_X86_IO_H */
diff --git a/arch/x86/mm/ioremap.c b/arch/x86/mm/ioremap.c
index 26db273..ee33838 100644
--- a/arch/x86/mm/ioremap.c
+++ b/arch/x86/mm/ioremap.c
@@ -14,6 +14,7 @@
 #include <linux/vmalloc.h>
 #include <linux/mmiotrace.h>
 #include <linux/mem_encrypt.h>
+#include <linux/efi.h>
 
 #include <asm/set_memory.h>
 #include <asm/e820/api.h>
@@ -22,6 +23,7 @@
 #include <asm/tlbflush.h>
 #include <asm/pgalloc.h>
 #include <asm/pat.h>
+#include <asm/setup.h>
 
 #include "physaddr.h"
 
@@ -414,6 +416,183 @@ void unxlate_dev_mem_ptr(phys_addr_t phys, void *addr)
 	iounmap((void __iomem *)((unsigned long)addr & PAGE_MASK));
 }
 
+/*
+ * Examine the physical address to determine if it is an area of memory
+ * that should be mapped decrypted.  If the memory is not part of the
+ * kernel usable area it was accessed and created decrypted, so these
+ * areas should be mapped decrypted.
+ */
+static bool memremap_should_map_decrypted(resource_size_t phys_addr,
+					  unsigned long size)
+{
+	/* Check if the address is outside kernel usable area */
+	switch (e820__get_entry_type(phys_addr, phys_addr + size - 1)) {
+	case E820_TYPE_RESERVED:
+	case E820_TYPE_ACPI:
+	case E820_TYPE_NVS:
+	case E820_TYPE_UNUSABLE:
+		return true;
+	default:
+		break;
+	}
+
+	return false;
+}
+
+/*
+ * Examine the physical address to determine if it is EFI data. Check
+ * it against the boot params structure and EFI tables and memory types.
+ */
+static bool memremap_is_efi_data(resource_size_t phys_addr,
+				 unsigned long size)
+{
+	u64 paddr;
+
+	/* Check if the address is part of EFI boot/runtime data */
+	if (!efi_enabled(EFI_BOOT))
+		return false;
+
+	paddr = boot_params.efi_info.efi_memmap_hi;
+	paddr <<= 32;
+	paddr |= boot_params.efi_info.efi_memmap;
+	if (phys_addr == paddr)
+		return true;
+
+	paddr = boot_params.efi_info.efi_systab_hi;
+	paddr <<= 32;
+	paddr |= boot_params.efi_info.efi_systab;
+	if (phys_addr == paddr)
+		return true;
+
+	if (efi_is_table_address(phys_addr))
+		return true;
+
+	switch (efi_mem_type(phys_addr)) {
+	case EFI_BOOT_SERVICES_DATA:
+	case EFI_RUNTIME_SERVICES_DATA:
+		return true;
+	default:
+		break;
+	}
+
+	return false;
+}
+
+/*
+ * Examine the physical address to determine if it is boot data by checking
+ * it against the boot params setup_data chain.
+ */
+static bool memremap_is_setup_data(resource_size_t phys_addr,
+				   unsigned long size)
+{
+	struct setup_data *data;
+	u64 paddr, paddr_next;
+
+	paddr = boot_params.hdr.setup_data;
+	while (paddr) {
+		unsigned int len;
+
+		if (phys_addr == paddr)
+			return true;
+
+		data = memremap(paddr, sizeof(*data),
+				MEMREMAP_WB | MEMREMAP_DEC);
+
+		paddr_next = data->next;
+		len = data->len;
+
+		memunmap(data);
+
+		if ((phys_addr > paddr) && (phys_addr < (paddr + len)))
+			return true;
+
+		paddr = paddr_next;
+	}
+
+	return false;
+}
+
+/*
+ * Examine the physical address to determine if it is boot data by checking
+ * it against the boot params setup_data chain (early boot version).
+ */
+static bool __init early_memremap_is_setup_data(resource_size_t phys_addr,
+						unsigned long size)
+{
+	struct setup_data *data;
+	u64 paddr, paddr_next;
+
+	paddr = boot_params.hdr.setup_data;
+	while (paddr) {
+		unsigned int len;
+
+		if (phys_addr == paddr)
+			return true;
+
+		data = early_memremap_decrypted(paddr, sizeof(*data));
+
+		paddr_next = data->next;
+		len = data->len;
+
+		early_memunmap(data, sizeof(*data));
+
+		if ((phys_addr > paddr) && (phys_addr < (paddr + len)))
+			return true;
+
+		paddr = paddr_next;
+	}
+
+	return false;
+}
+
+/*
+ * Architecture function to determine if RAM remap is allowed. By default, a
+ * RAM remap will map the data as encrypted. Determine if a RAM remap should
+ * not be done so that the data will be mapped decrypted.
+ */
+bool arch_memremap_can_ram_remap(resource_size_t phys_addr, unsigned long size,
+				 unsigned long flags)
+{
+	if (!sme_active())
+		return true;
+
+	if (flags & MEMREMAP_ENC)
+		return true;
+
+	if (flags & MEMREMAP_DEC)
+		return false;
+
+	if (memremap_is_setup_data(phys_addr, size) ||
+	    memremap_is_efi_data(phys_addr, size) ||
+	    memremap_should_map_decrypted(phys_addr, size))
+		return false;
+
+	return true;
+}
+
+/*
+ * Architecture override of __weak function to adjust the protection attributes
+ * used when remapping memory. By default, early_memremap() will map the data
+ * as encrypted. Determine if an encrypted mapping should not be done and set
+ * the appropriate protection attributes.
+ */
+pgprot_t __init early_memremap_pgprot_adjust(resource_size_t phys_addr,
+					     unsigned long size,
+					     pgprot_t prot)
+{
+	if (!sme_active())
+		return prot;
+
+	if (early_memremap_is_setup_data(phys_addr, size) ||
+	    memremap_is_efi_data(phys_addr, size) ||
+	    memremap_should_map_decrypted(phys_addr, size))
+		prot = pgprot_decrypted(prot);
+	else
+		prot = pgprot_encrypted(prot);
+
+	return prot;
+}
+
 #ifdef CONFIG_ARCH_USE_MEMREMAP_PROT
 /* Remap memory with encryption */
 void __init *early_memremap_encrypted(resource_size_t phys_addr,
diff --git a/include/linux/io.h b/include/linux/io.h
index 2195d9e..32e30e8 100644
--- a/include/linux/io.h
+++ b/include/linux/io.h
@@ -157,6 +157,8 @@ enum {
 	MEMREMAP_WB = 1 << 0,
 	MEMREMAP_WT = 1 << 1,
 	MEMREMAP_WC = 1 << 2,
+	MEMREMAP_ENC = 1 << 3,
+	MEMREMAP_DEC = 1 << 4,
 };
 
 void *memremap(resource_size_t offset, size_t size, unsigned long flags);
diff --git a/kernel/memremap.c b/kernel/memremap.c
index 23a6483..9b4fbd5 100644
--- a/kernel/memremap.c
+++ b/kernel/memremap.c
@@ -34,13 +34,24 @@ static void *arch_memremap_wb(resource_size_t offset, unsigned long size)
 }
 #endif
 
-static void *try_ram_remap(resource_size_t offset, size_t size)
+#ifndef arch_memremap_can_ram_remap
+static bool arch_memremap_can_ram_remap(resource_size_t offset, size_t size,
+					unsigned long flags)
+{
+	return true;
+}
+#endif
+
+static void *try_ram_remap(resource_size_t offset, size_t size,
+			   unsigned long flags)
 {
 	unsigned long pfn = PHYS_PFN(offset);
 
 	/* In the simple case just return the existing linear address */
-	if (pfn_valid(pfn) && !PageHighMem(pfn_to_page(pfn)))
+	if (pfn_valid(pfn) && !PageHighMem(pfn_to_page(pfn)) &&
+	    arch_memremap_can_ram_remap(offset, size, flags))
 		return __va(offset);
+
 	return NULL; /* fallback to arch_memremap_wb */
 }
 
@@ -48,7 +59,8 @@ static void *try_ram_remap(resource_size_t offset, size_t size)
  * memremap() - remap an iomem_resource as cacheable memory
  * @offset: iomem resource start address
  * @size: size of remap
- * @flags: any of MEMREMAP_WB, MEMREMAP_WT and MEMREMAP_WC
+ * @flags: any of MEMREMAP_WB, MEMREMAP_WT, MEMREMAP_WC,
+ *		  MEMREMAP_ENC, MEMREMAP_DEC
  *
  * memremap() is "ioremap" for cases where it is known that the resource
  * being mapped does not have i/o side effects and the __iomem
@@ -95,7 +107,7 @@ void *memremap(resource_size_t offset, size_t size, unsigned long flags)
 		 * the requested range is potentially in System RAM.
 		 */
 		if (is_ram == REGION_INTERSECTS)
-			addr = try_ram_remap(offset, size);
+			addr = try_ram_remap(offset, size, flags);
 		if (!addr)
 			addr = arch_memremap_wb(offset, size);
 	}
diff --git a/mm/early_ioremap.c b/mm/early_ioremap.c
index d7d30da..b1dd4a9 100644
--- a/mm/early_ioremap.c
+++ b/mm/early_ioremap.c
@@ -30,6 +30,13 @@ static int __init early_ioremap_debug_setup(char *str)
 
 static int after_paging_init __initdata;
 
+pgprot_t __init __weak early_memremap_pgprot_adjust(resource_size_t phys_addr,
+						    unsigned long size,
+						    pgprot_t prot)
+{
+	return prot;
+}
+
 void __init __weak early_ioremap_shutdown(void)
 {
 }
@@ -215,14 +222,19 @@ void __init early_iounmap(void __iomem *addr, unsigned long size)
 void __init *
 early_memremap(resource_size_t phys_addr, unsigned long size)
 {
-	return (__force void *)__early_ioremap(phys_addr, size,
-					       FIXMAP_PAGE_NORMAL);
+	pgprot_t prot = early_memremap_pgprot_adjust(phys_addr, size,
+						     FIXMAP_PAGE_NORMAL);
+
+	return (__force void *)__early_ioremap(phys_addr, size, prot);
 }
 #ifdef FIXMAP_PAGE_RO
 void __init *
 early_memremap_ro(resource_size_t phys_addr, unsigned long size)
 {
-	return (__force void *)__early_ioremap(phys_addr, size, FIXMAP_PAGE_RO);
+	pgprot_t prot = early_memremap_pgprot_adjust(phys_addr, size,
+						     FIXMAP_PAGE_RO);
+
+	return (__force void *)__early_ioremap(phys_addr, size, prot);
 }
 #endif
 



From thomas.lendacky at amd.com  Tue Jun 27 08:11:01 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Tue, 27 Jun 2017 10:11:01 -0500
Subject: [PATCH v8 RESEND 20/38] x86,
 mpparse: Use memremap to map the mpf and mpc data
In-Reply-To: <20170627150718.17428.81813.stgit@tlendack-t1.amdoffice.net>
References: <20170627150718.17428.81813.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170627151101.17428.86533.stgit@tlendack-t1.amdoffice.net>

The SMP MP-table is built by UEFI and placed in memory in a decrypted
state. These tables are accessed using a mix of early_memremap(),
early_memunmap(), phys_to_virt() and virt_to_phys(). Change all accesses
to use early_memremap()/early_memunmap(). This allows for proper setting
of the encryption mask so that the data can be successfully accessed when
SME is active.

Reviewed-by: Borislav Petkov <bp at suse.de>
Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/kernel/mpparse.c |   98 ++++++++++++++++++++++++++++++++-------------
 1 file changed, 70 insertions(+), 28 deletions(-)

diff --git a/arch/x86/kernel/mpparse.c b/arch/x86/kernel/mpparse.c
index fd37f39..5cbb317 100644
--- a/arch/x86/kernel/mpparse.c
+++ b/arch/x86/kernel/mpparse.c
@@ -429,7 +429,7 @@ static inline void __init construct_default_ISA_mptable(int mpc_default_type)
 	}
 }
 
-static struct mpf_intel *mpf_found;
+static unsigned long mpf_base;
 
 static unsigned long __init get_mpc_size(unsigned long physptr)
 {
@@ -451,6 +451,7 @@ static int __init check_physptr(struct mpf_intel *mpf, unsigned int early)
 
 	size = get_mpc_size(mpf->physptr);
 	mpc = early_memremap(mpf->physptr, size);
+
 	/*
 	 * Read the physical hardware table.  Anything here will
 	 * override the defaults.
@@ -497,12 +498,12 @@ static int __init check_physptr(struct mpf_intel *mpf, unsigned int early)
  */
 void __init default_get_smp_config(unsigned int early)
 {
-	struct mpf_intel *mpf = mpf_found;
+	struct mpf_intel *mpf;
 
 	if (!smp_found_config)
 		return;
 
-	if (!mpf)
+	if (!mpf_base)
 		return;
 
 	if (acpi_lapic && early)
@@ -515,6 +516,12 @@ void __init default_get_smp_config(unsigned int early)
 	if (acpi_lapic && acpi_ioapic)
 		return;
 
+	mpf = early_memremap(mpf_base, sizeof(*mpf));
+	if (!mpf) {
+		pr_err("MPTABLE: error mapping MP table\n");
+		return;
+	}
+
 	pr_info("Intel MultiProcessor Specification v1.%d\n",
 		mpf->specification);
 #if defined(CONFIG_X86_LOCAL_APIC) && defined(CONFIG_X86_32)
@@ -529,7 +536,7 @@ void __init default_get_smp_config(unsigned int early)
 	/*
 	 * Now see if we need to read further.
 	 */
-	if (mpf->feature1 != 0) {
+	if (mpf->feature1) {
 		if (early) {
 			/*
 			 * local APIC has default address
@@ -542,8 +549,10 @@ void __init default_get_smp_config(unsigned int early)
 		construct_default_ISA_mptable(mpf->feature1);
 
 	} else if (mpf->physptr) {
-		if (check_physptr(mpf, early))
+		if (check_physptr(mpf, early)) {
+			early_memunmap(mpf, sizeof(*mpf));
 			return;
+		}
 	} else
 		BUG();
 
@@ -552,6 +561,8 @@ void __init default_get_smp_config(unsigned int early)
 	/*
 	 * Only use the first configuration found.
 	 */
+
+	early_memunmap(mpf, sizeof(*mpf));
 }
 
 static void __init smp_reserve_memory(struct mpf_intel *mpf)
@@ -561,15 +572,16 @@ static void __init smp_reserve_memory(struct mpf_intel *mpf)
 
 static int __init smp_scan_config(unsigned long base, unsigned long length)
 {
-	unsigned int *bp = phys_to_virt(base);
+	unsigned int *bp;
 	struct mpf_intel *mpf;
-	unsigned long mem;
+	int ret = 0;
 
 	apic_printk(APIC_VERBOSE, "Scan for SMP in [mem %#010lx-%#010lx]\n",
 		    base, base + length - 1);
 	BUILD_BUG_ON(sizeof(*mpf) != 16);
 
 	while (length > 0) {
+		bp = early_memremap(base, length);
 		mpf = (struct mpf_intel *)bp;
 		if ((*bp == SMP_MAGIC_IDENT) &&
 		    (mpf->length == 1) &&
@@ -579,24 +591,26 @@ static int __init smp_scan_config(unsigned long base, unsigned long length)
 #ifdef CONFIG_X86_LOCAL_APIC
 			smp_found_config = 1;
 #endif
-			mpf_found = mpf;
+			mpf_base = base;
 
-			pr_info("found SMP MP-table at [mem %#010llx-%#010llx] mapped at [%p]\n",
-				(unsigned long long) virt_to_phys(mpf),
-				(unsigned long long) virt_to_phys(mpf) +
-				sizeof(*mpf) - 1, mpf);
+			pr_info("found SMP MP-table at [mem %#010lx-%#010lx] mapped at [%p]\n",
+				base, base + sizeof(*mpf) - 1, mpf);
 
-			mem = virt_to_phys(mpf);
-			memblock_reserve(mem, sizeof(*mpf));
+			memblock_reserve(base, sizeof(*mpf));
 			if (mpf->physptr)
 				smp_reserve_memory(mpf);
 
-			return 1;
+			ret = 1;
 		}
-		bp += 4;
+		early_memunmap(bp, length);
+
+		if (ret)
+			break;
+
+		base += 16;
 		length -= 16;
 	}
-	return 0;
+	return ret;
 }
 
 void __init default_find_smp_config(void)
@@ -838,29 +852,40 @@ static int __init update_mp_table(void)
 	char oem[10];
 	struct mpf_intel *mpf;
 	struct mpc_table *mpc, *mpc_new;
+	unsigned long size;
 
 	if (!enable_update_mptable)
 		return 0;
 
-	mpf = mpf_found;
-	if (!mpf)
+	if (!mpf_base)
+		return 0;
+
+	mpf = early_memremap(mpf_base, sizeof(*mpf));
+	if (!mpf) {
+		pr_err("MPTABLE: mpf early_memremap() failed\n");
 		return 0;
+	}
 
 	/*
 	 * Now see if we need to go further.
 	 */
-	if (mpf->feature1 != 0)
-		return 0;
+	if (mpf->feature1)
+		goto do_unmap_mpf;
 
 	if (!mpf->physptr)
-		return 0;
+		goto do_unmap_mpf;
 
-	mpc = phys_to_virt(mpf->physptr);
+	size = get_mpc_size(mpf->physptr);
+	mpc = early_memremap(mpf->physptr, size);
+	if (!mpc) {
+		pr_err("MPTABLE: mpc early_memremap() failed\n");
+		goto do_unmap_mpf;
+	}
 
 	if (!smp_check_mpc(mpc, oem, str))
-		return 0;
+		goto do_unmap_mpc;
 
-	pr_info("mpf: %llx\n", (u64)virt_to_phys(mpf));
+	pr_info("mpf: %llx\n", (u64)mpf_base);
 	pr_info("physptr: %x\n", mpf->physptr);
 
 	if (mpc_new_phys && mpc->length > mpc_new_length) {
@@ -878,21 +903,32 @@ static int __init update_mp_table(void)
 		new = mpf_checksum((unsigned char *)mpc, mpc->length);
 		if (old == new) {
 			pr_info("mpc is readonly, please try alloc_mptable instead\n");
-			return 0;
+			goto do_unmap_mpc;
 		}
 		pr_info("use in-position replacing\n");
 	} else {
+		mpc_new = early_memremap(mpc_new_phys, mpc_new_length);
+		if (!mpc_new) {
+			pr_err("MPTABLE: new mpc early_memremap() failed\n");
+			goto do_unmap_mpc;
+		}
 		mpf->physptr = mpc_new_phys;
-		mpc_new = phys_to_virt(mpc_new_phys);
 		memcpy(mpc_new, mpc, mpc->length);
+		early_memunmap(mpc, size);
 		mpc = mpc_new;
+		size = mpc_new_length;
 		/* check if we can modify that */
 		if (mpc_new_phys - mpf->physptr) {
 			struct mpf_intel *mpf_new;
 			/* steal 16 bytes from [0, 1k) */
+			mpf_new = early_memremap(0x400 - 16, sizeof(*mpf_new));
+			if (!mpf_new) {
+				pr_err("MPTABLE: new mpf early_memremap() failed\n");
+				goto do_unmap_mpc;
+			}
 			pr_info("mpf new: %x\n", 0x400 - 16);
-			mpf_new = phys_to_virt(0x400 - 16);
 			memcpy(mpf_new, mpf, 16);
+			early_memunmap(mpf, sizeof(*mpf));
 			mpf = mpf_new;
 			mpf->physptr = mpc_new_phys;
 		}
@@ -909,6 +945,12 @@ static int __init update_mp_table(void)
 	 */
 	replace_intsrc_all(mpc, mpc_new_phys, mpc_new_length);
 
+do_unmap_mpc:
+	early_memunmap(mpc, size);
+
+do_unmap_mpf:
+	early_memunmap(mpf, sizeof(*mpf));
+
 	return 0;
 }
 



From thomas.lendacky at amd.com  Tue Jun 27 08:11:11 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Tue, 27 Jun 2017 10:11:11 -0500
Subject: [PATCH v8 RESEND 21/38] x86/mm: Add support to access persistent
 memory in the clear
In-Reply-To: <20170627150718.17428.81813.stgit@tlendack-t1.amdoffice.net>
References: <20170627150718.17428.81813.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170627151111.17428.69076.stgit@tlendack-t1.amdoffice.net>

Persistent memory is expected to persist across reboots. The encryption
key used by SME will change across reboots which will result in corrupted
persistent memory.  Persistent memory is handed out by block devices
through memory remapping functions, so be sure not to map this memory as
encrypted.

Reviewed-by: Borislav Petkov <bp at suse.de>
Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/mm/ioremap.c |   31 ++++++++++++++++++++++++++++++-
 1 file changed, 30 insertions(+), 1 deletion(-)

diff --git a/arch/x86/mm/ioremap.c b/arch/x86/mm/ioremap.c
index ee33838..effa529 100644
--- a/arch/x86/mm/ioremap.c
+++ b/arch/x86/mm/ioremap.c
@@ -420,17 +420,46 @@ void unxlate_dev_mem_ptr(phys_addr_t phys, void *addr)
  * Examine the physical address to determine if it is an area of memory
  * that should be mapped decrypted.  If the memory is not part of the
  * kernel usable area it was accessed and created decrypted, so these
- * areas should be mapped decrypted.
+ * areas should be mapped decrypted. And since the encryption key can
+ * change across reboots, persistent memory should also be mapped
+ * decrypted.
  */
 static bool memremap_should_map_decrypted(resource_size_t phys_addr,
 					  unsigned long size)
 {
+	int is_pmem;
+
+	/*
+	 * Check if the address is part of a persistent memory region.
+	 * This check covers areas added by E820, EFI and ACPI.
+	 */
+	is_pmem = region_intersects(phys_addr, size, IORESOURCE_MEM,
+				    IORES_DESC_PERSISTENT_MEMORY);
+	if (is_pmem != REGION_DISJOINT)
+		return true;
+
+	/*
+	 * Check if the non-volatile attribute is set for an EFI
+	 * reserved area.
+	 */
+	if (efi_enabled(EFI_BOOT)) {
+		switch (efi_mem_type(phys_addr)) {
+		case EFI_RESERVED_TYPE:
+			if (efi_mem_attributes(phys_addr) & EFI_MEMORY_NV)
+				return true;
+			break;
+		default:
+			break;
+		}
+	}
+
 	/* Check if the address is outside kernel usable area */
 	switch (e820__get_entry_type(phys_addr, phys_addr + size - 1)) {
 	case E820_TYPE_RESERVED:
 	case E820_TYPE_ACPI:
 	case E820_TYPE_NVS:
 	case E820_TYPE_UNUSABLE:
+	case E820_TYPE_PRAM:
 		return true;
 	default:
 		break;



From thomas.lendacky at amd.com  Tue Jun 27 08:11:24 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Tue, 27 Jun 2017 10:11:24 -0500
Subject: [PATCH v8 RESEND 22/38] x86/mm: Add support for changing the memory
 encryption attribute
In-Reply-To: <20170627150718.17428.81813.stgit@tlendack-t1.amdoffice.net>
References: <20170627150718.17428.81813.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170627151124.17428.69701.stgit@tlendack-t1.amdoffice.net>

Add support for changing the memory encryption attribute for one or more
memory pages. This will be useful when we have to change the AP trampoline
area to not be encrypted. Or when we need to change the SWIOTLB area to
not be encrypted in support of devices that can't support the encryption
mask range.

Reviewed-by: Borislav Petkov <bp at suse.de>
Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/include/asm/set_memory.h |    3 ++
 arch/x86/mm/pageattr.c            |   62 +++++++++++++++++++++++++++++++++++++
 2 files changed, 65 insertions(+)

diff --git a/arch/x86/include/asm/set_memory.h b/arch/x86/include/asm/set_memory.h
index eaec6c3..cd71273 100644
--- a/arch/x86/include/asm/set_memory.h
+++ b/arch/x86/include/asm/set_memory.h
@@ -11,6 +11,7 @@
  * Executability : eXeutable, NoteXecutable
  * Read/Write    : ReadOnly, ReadWrite
  * Presence      : NotPresent
+ * Encryption    : Encrypted, Decrypted
  *
  * Within a category, the attributes are mutually exclusive.
  *
@@ -42,6 +43,8 @@
 int set_memory_wb(unsigned long addr, int numpages);
 int set_memory_np(unsigned long addr, int numpages);
 int set_memory_4k(unsigned long addr, int numpages);
+int set_memory_encrypted(unsigned long addr, int numpages);
+int set_memory_decrypted(unsigned long addr, int numpages);
 
 int set_memory_array_uc(unsigned long *addr, int addrinarray);
 int set_memory_array_wc(unsigned long *addr, int addrinarray);
diff --git a/arch/x86/mm/pageattr.c b/arch/x86/mm/pageattr.c
index e7d3866..d9e09fb 100644
--- a/arch/x86/mm/pageattr.c
+++ b/arch/x86/mm/pageattr.c
@@ -1769,6 +1769,68 @@ int set_memory_4k(unsigned long addr, int numpages)
 					__pgprot(0), 1, 0, NULL);
 }
 
+static int __set_memory_enc_dec(unsigned long addr, int numpages, bool enc)
+{
+	struct cpa_data cpa;
+	unsigned long start;
+	int ret;
+
+	/* Nothing to do if the SME is not active */
+	if (!sme_active())
+		return 0;
+
+	/* Should not be working on unaligned addresses */
+	if (WARN_ONCE(addr & ~PAGE_MASK, "misaligned address: %#lx\n", addr))
+		addr &= PAGE_MASK;
+
+	start = addr;
+
+	memset(&cpa, 0, sizeof(cpa));
+	cpa.vaddr = &addr;
+	cpa.numpages = numpages;
+	cpa.mask_set = enc ? __pgprot(_PAGE_ENC) : __pgprot(0);
+	cpa.mask_clr = enc ? __pgprot(0) : __pgprot(_PAGE_ENC);
+	cpa.pgd = init_mm.pgd;
+
+	/* Must avoid aliasing mappings in the highmem code */
+	kmap_flush_unused();
+	vm_unmap_aliases();
+
+	/*
+	 * Before changing the encryption attribute, we need to flush caches.
+	 */
+	if (static_cpu_has(X86_FEATURE_CLFLUSH))
+		cpa_flush_range(start, numpages, 1);
+	else
+		cpa_flush_all(1);
+
+	ret = __change_page_attr_set_clr(&cpa, 1);
+
+	/*
+	 * After changing the encryption attribute, we need to flush TLBs
+	 * again in case any speculative TLB caching occurred (but no need
+	 * to flush caches again).  We could just use cpa_flush_all(), but
+	 * in case TLB flushing gets optimized in the cpa_flush_range()
+	 * path use the same logic as above.
+	 */
+	if (static_cpu_has(X86_FEATURE_CLFLUSH))
+		cpa_flush_range(start, numpages, 0);
+	else
+		cpa_flush_all(0);
+
+	return ret;
+}
+
+int set_memory_encrypted(unsigned long addr, int numpages)
+{
+	return __set_memory_enc_dec(addr, numpages, true);
+}
+
+int set_memory_decrypted(unsigned long addr, int numpages)
+{
+	return __set_memory_enc_dec(addr, numpages, false);
+}
+
 int set_pages_uc(struct page *page, int numpages)
 {
 	unsigned long addr = (unsigned long)page_address(page);



From thomas.lendacky at amd.com  Tue Jun 27 08:11:38 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Tue, 27 Jun 2017 10:11:38 -0500
Subject: [PATCH v8 RESEND 23/38] x86/realmode: Decrypt trampoline area if
 memory encryption is active
In-Reply-To: <20170627150718.17428.81813.stgit@tlendack-t1.amdoffice.net>
References: <20170627150718.17428.81813.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170627151138.17428.15995.stgit@tlendack-t1.amdoffice.net>

When Secure Memory Encryption is enabled, the trampoline area must not
be encrypted. A CPU running in real mode will not be able to decrypt
memory that has been encrypted because it will not be able to use addresses
with the memory encryption mask.

Reviewed-by: Borislav Petkov <bp at suse.de>
Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/realmode/init.c |    8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/arch/x86/realmode/init.c b/arch/x86/realmode/init.c
index cd4be19..d6ddc7e 100644
--- a/arch/x86/realmode/init.c
+++ b/arch/x86/realmode/init.c
@@ -1,6 +1,7 @@
 #include <linux/io.h>
 #include <linux/slab.h>
 #include <linux/memblock.h>
+#include <linux/mem_encrypt.h>
 
 #include <asm/set_memory.h>
 #include <asm/pgtable.h>
@@ -59,6 +60,13 @@ static void __init setup_real_mode(void)
 
 	base = (unsigned char *)real_mode_header;
 
+	/*
+	 * If SME is active, the trampoline area will need to be in
+	 * decrypted memory in order to bring up other processors
+	 * successfully.
+	 */
+	set_memory_decrypted((unsigned long)base, size >> PAGE_SHIFT);
+
 	memcpy(base, real_mode_blob, size);
 
 	phys_base = __pa(base);



From thomas.lendacky at amd.com  Tue Jun 27 08:11:53 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Tue, 27 Jun 2017 10:11:53 -0500
Subject: [PATCH v8 RESEND 24/38] x86, swiotlb: Add memory encryption support
In-Reply-To: <20170627150718.17428.81813.stgit@tlendack-t1.amdoffice.net>
References: <20170627150718.17428.81813.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170627151152.17428.46131.stgit@tlendack-t1.amdoffice.net>

Since DMA addresses will effectively look like 48-bit addresses when the
memory encryption mask is set, SWIOTLB is needed if the DMA mask of the
device performing the DMA does not support 48-bits. SWIOTLB will be
initialized to create decrypted bounce buffers for use by these devices.

Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/include/asm/dma-mapping.h |    5 ++-
 arch/x86/include/asm/mem_encrypt.h |    5 +++
 arch/x86/kernel/pci-dma.c          |   11 +++++--
 arch/x86/kernel/pci-nommu.c        |    2 +
 arch/x86/kernel/pci-swiotlb.c      |   15 +++++++++-
 arch/x86/mm/mem_encrypt.c          |   22 +++++++++++++++
 include/linux/swiotlb.h            |    1 +
 init/main.c                        |   10 +++++++
 lib/swiotlb.c                      |   54 +++++++++++++++++++++++++++++++-----
 9 files changed, 108 insertions(+), 17 deletions(-)

diff --git a/arch/x86/include/asm/dma-mapping.h b/arch/x86/include/asm/dma-mapping.h
index 08a0838..191f9a5 100644
--- a/arch/x86/include/asm/dma-mapping.h
+++ b/arch/x86/include/asm/dma-mapping.h
@@ -12,6 +12,7 @@
 #include <asm/io.h>
 #include <asm/swiotlb.h>
 #include <linux/dma-contiguous.h>
+#include <linux/mem_encrypt.h>
 
 #ifdef CONFIG_ISA
 # define ISA_DMA_BIT_MASK DMA_BIT_MASK(24)
@@ -62,12 +63,12 @@ static inline bool dma_capable(struct device *dev, dma_addr_t addr, size_t size)
 
 static inline dma_addr_t phys_to_dma(struct device *dev, phys_addr_t paddr)
 {
-	return paddr;
+	return __sme_set(paddr);
 }
 
 static inline phys_addr_t dma_to_phys(struct device *dev, dma_addr_t daddr)
 {
-	return daddr;
+	return __sme_clr(daddr);
 }
 #endif /* CONFIG_X86_DMA_REMAP */
 
diff --git a/arch/x86/include/asm/mem_encrypt.h b/arch/x86/include/asm/mem_encrypt.h
index ab1fe77..70e55f6 100644
--- a/arch/x86/include/asm/mem_encrypt.h
+++ b/arch/x86/include/asm/mem_encrypt.h
@@ -34,6 +34,11 @@ void __init sme_early_decrypt(resource_size_t paddr,
 void __init sme_encrypt_kernel(void);
 void __init sme_enable(void);
 
+/* Architecture __weak replacement functions */
+void __init mem_encrypt_init(void);
+
+void swiotlb_set_mem_attributes(void *vaddr, unsigned long size);
+
 #else	/* !CONFIG_AMD_MEM_ENCRYPT */
 
 #define sme_me_mask	0UL
diff --git a/arch/x86/kernel/pci-dma.c b/arch/x86/kernel/pci-dma.c
index 3a216ec..72d96d4 100644
--- a/arch/x86/kernel/pci-dma.c
+++ b/arch/x86/kernel/pci-dma.c
@@ -93,9 +93,12 @@ void *dma_generic_alloc_coherent(struct device *dev, size_t size,
 	if (gfpflags_allow_blocking(flag)) {
 		page = dma_alloc_from_contiguous(dev, count, get_order(size),
 						 flag);
-		if (page && page_to_phys(page) + size > dma_mask) {
-			dma_release_from_contiguous(dev, page, count);
-			page = NULL;
+		if (page) {
+			addr = phys_to_dma(dev, page_to_phys(page));
+			if (addr + size > dma_mask) {
+				dma_release_from_contiguous(dev, page, count);
+				page = NULL;
+			}
 		}
 	}
 	/* fallback */
@@ -104,7 +107,7 @@ void *dma_generic_alloc_coherent(struct device *dev, size_t size,
 	if (!page)
 		return NULL;
 
-	addr = page_to_phys(page);
+	addr = phys_to_dma(dev, page_to_phys(page));
 	if (addr + size > dma_mask) {
 		__free_pages(page, get_order(size));
 
diff --git a/arch/x86/kernel/pci-nommu.c b/arch/x86/kernel/pci-nommu.c
index a88952e..98b576a 100644
--- a/arch/x86/kernel/pci-nommu.c
+++ b/arch/x86/kernel/pci-nommu.c
@@ -30,7 +30,7 @@ static dma_addr_t nommu_map_page(struct device *dev, struct page *page,
 				 enum dma_data_direction dir,
 				 unsigned long attrs)
 {
-	dma_addr_t bus = page_to_phys(page) + offset;
+	dma_addr_t bus = phys_to_dma(dev, page_to_phys(page)) + offset;
 	WARN_ON(size == 0);
 	if (!check_addr("map_single", dev, bus, size))
 		return DMA_ERROR_CODE;
diff --git a/arch/x86/kernel/pci-swiotlb.c b/arch/x86/kernel/pci-swiotlb.c
index 1e23577..6770775 100644
--- a/arch/x86/kernel/pci-swiotlb.c
+++ b/arch/x86/kernel/pci-swiotlb.c
@@ -6,12 +6,14 @@
 #include <linux/swiotlb.h>
 #include <linux/bootmem.h>
 #include <linux/dma-mapping.h>
+#include <linux/mem_encrypt.h>
 
 #include <asm/iommu.h>
 #include <asm/swiotlb.h>
 #include <asm/dma.h>
 #include <asm/xen/swiotlb-xen.h>
 #include <asm/iommu_table.h>
+
 int swiotlb __read_mostly;
 
 void *x86_swiotlb_alloc_coherent(struct device *hwdev, size_t size,
@@ -79,8 +81,8 @@ int __init pci_swiotlb_detect_override(void)
 		  pci_swiotlb_late_init);
 
 /*
- * if 4GB or more detected (and iommu=off not set) return 1
- * and set swiotlb to 1.
+ * If 4GB or more detected (and iommu=off not set) or if SME is active
+ * then set swiotlb to 1 and return 1.
  */
 int __init pci_swiotlb_detect_4gb(void)
 {
@@ -89,6 +91,15 @@ int __init pci_swiotlb_detect_4gb(void)
 	if (!no_iommu && max_possible_pfn > MAX_DMA32_PFN)
 		swiotlb = 1;
 #endif
+
+	/*
+	 * If SME is active then swiotlb will be set to 1 so that bounce
+	 * buffers are allocated and used for devices that do not support
+	 * the addressing range required for the encryption mask.
+	 */
+	if (sme_active())
+		swiotlb = 1;
+
 	return swiotlb;
 }
 IOMMU_INIT(pci_swiotlb_detect_4gb,
diff --git a/arch/x86/mm/mem_encrypt.c b/arch/x86/mm/mem_encrypt.c
index 0843d02..a7400ec 100644
--- a/arch/x86/mm/mem_encrypt.c
+++ b/arch/x86/mm/mem_encrypt.c
@@ -13,11 +13,14 @@
 #include <linux/linkage.h>
 #include <linux/init.h>
 #include <linux/mm.h>
+#include <linux/dma-mapping.h>
+#include <linux/swiotlb.h>
 
 #include <asm/tlbflush.h>
 #include <asm/fixmap.h>
 #include <asm/setup.h>
 #include <asm/bootparam.h>
+#include <asm/set_memory.h>
 
 /*
  * Since SME related variables are set early in the boot process they must
@@ -177,6 +180,25 @@ void __init sme_early_init(void)
 		protection_map[i] = pgprot_encrypted(protection_map[i]);
 }
 
+/* Architecture __weak replacement functions */
+void __init mem_encrypt_init(void)
+{
+	if (!sme_me_mask)
+		return;
+
+	/* Call into SWIOTLB to update the SWIOTLB DMA buffers */
+	swiotlb_update_mem_attributes();
+}
+
+void swiotlb_set_mem_attributes(void *vaddr, unsigned long size)
+{
+	WARN(PAGE_ALIGN(size) != size,
+	     "size is not page-aligned (%#lx)\n", size);
+
+	/* Make the SWIOTLB buffer area decrypted */
+	set_memory_decrypted((unsigned long)vaddr, size >> PAGE_SHIFT);
+}
+
 void __init sme_encrypt_kernel(void)
 {
 }
diff --git a/include/linux/swiotlb.h b/include/linux/swiotlb.h
index 4ee479f..15e7160 100644
--- a/include/linux/swiotlb.h
+++ b/include/linux/swiotlb.h
@@ -35,6 +35,7 @@ enum swiotlb_force {
 extern unsigned long swiotlb_nr_tbl(void);
 unsigned long swiotlb_size_or_default(void);
 extern int swiotlb_late_init_with_tbl(char *tlb, unsigned long nslabs);
+extern void __init swiotlb_update_mem_attributes(void);
 
 /*
  * Enumeration for sync targets
diff --git a/init/main.c b/init/main.c
index df58a41..5013cbc 100644
--- a/init/main.c
+++ b/init/main.c
@@ -488,6 +488,8 @@ void __init __weak thread_stack_cache_init(void)
 }
 #endif
 
+void __init __weak mem_encrypt_init(void) { }
+
 /*
  * Set up kernel memory allocators
  */
@@ -640,6 +642,14 @@ asmlinkage __visible void __init start_kernel(void)
 	 */
 	locking_selftest();
 
+	/*
+	 * This needs to be called before any devices perform DMA
+	 * operations that might use the SWIOTLB bounce buffers. It will
+	 * mark the bounce buffers as decrypted so that their usage will
+	 * not cause "plain-text" data to be decrypted when accessed.
+	 */
+	mem_encrypt_init();
+
 #ifdef CONFIG_BLK_DEV_INITRD
 	if (initrd_start && !initrd_below_start_ok &&
 	    page_to_pfn(virt_to_page((void *)initrd_start)) < min_low_pfn) {
diff --git a/lib/swiotlb.c b/lib/swiotlb.c
index a8d74a7..04ac91a 100644
--- a/lib/swiotlb.c
+++ b/lib/swiotlb.c
@@ -30,6 +30,7 @@
 #include <linux/highmem.h>
 #include <linux/gfp.h>
 #include <linux/scatterlist.h>
+#include <linux/mem_encrypt.h>
 
 #include <asm/io.h>
 #include <asm/dma.h>
@@ -155,6 +156,15 @@ unsigned long swiotlb_size_or_default(void)
 	return size ? size : (IO_TLB_DEFAULT_SIZE);
 }
 
+void __weak swiotlb_set_mem_attributes(void *vaddr, unsigned long size) { }
+
+/* For swiotlb, clear memory encryption mask from dma addresses */
+static dma_addr_t swiotlb_phys_to_dma(struct device *hwdev,
+				      phys_addr_t address)
+{
+	return __sme_clr(phys_to_dma(hwdev, address));
+}
+
 /* Note that this doesn't work with highmem page */
 static dma_addr_t swiotlb_virt_to_bus(struct device *hwdev,
 				      volatile void *address)
@@ -183,6 +193,31 @@ void swiotlb_print_info(void)
 	       bytes >> 20, vstart, vend - 1);
 }
 
+/*
+ * Early SWIOTLB allocation may be too early to allow an architecture to
+ * perform the desired operations.  This function allows the architecture to
+ * call SWIOTLB when the operations are possible.  It needs to be called
+ * before the SWIOTLB memory is used.
+ */
+void __init swiotlb_update_mem_attributes(void)
+{
+	void *vaddr;
+	unsigned long bytes;
+
+	if (no_iotlb_memory || late_alloc)
+		return;
+
+	vaddr = phys_to_virt(io_tlb_start);
+	bytes = PAGE_ALIGN(io_tlb_nslabs << IO_TLB_SHIFT);
+	swiotlb_set_mem_attributes(vaddr, bytes);
+	memset(vaddr, 0, bytes);
+
+	vaddr = phys_to_virt(io_tlb_overflow_buffer);
+	bytes = PAGE_ALIGN(io_tlb_overflow);
+	swiotlb_set_mem_attributes(vaddr, bytes);
+	memset(vaddr, 0, bytes);
+}
+
 int __init swiotlb_init_with_tbl(char *tlb, unsigned long nslabs, int verbose)
 {
 	void *v_overflow_buffer;
@@ -320,6 +355,7 @@ int __init swiotlb_init_with_tbl(char *tlb, unsigned long nslabs, int verbose)
 	io_tlb_start = virt_to_phys(tlb);
 	io_tlb_end = io_tlb_start + bytes;
 
+	swiotlb_set_mem_attributes(tlb, bytes);
 	memset(tlb, 0, bytes);
 
 	/*
@@ -330,6 +366,8 @@ int __init swiotlb_init_with_tbl(char *tlb, unsigned long nslabs, int verbose)
 	if (!v_overflow_buffer)
 		goto cleanup2;
 
+	swiotlb_set_mem_attributes(v_overflow_buffer, io_tlb_overflow);
+	memset(v_overflow_buffer, 0, io_tlb_overflow);
 	io_tlb_overflow_buffer = virt_to_phys(v_overflow_buffer);
 
 	/*
@@ -581,7 +619,7 @@ phys_addr_t swiotlb_tbl_map_single(struct device *hwdev,
 		return SWIOTLB_MAP_ERROR;
 	}
 
-	start_dma_addr = phys_to_dma(hwdev, io_tlb_start);
+	start_dma_addr = swiotlb_phys_to_dma(hwdev, io_tlb_start);
 	return swiotlb_tbl_map_single(hwdev, start_dma_addr, phys, size,
 				      dir, attrs);
 }
@@ -702,7 +740,7 @@ void swiotlb_tbl_sync_single(struct device *hwdev, phys_addr_t tlb_addr,
 			goto err_warn;
 
 		ret = phys_to_virt(paddr);
-		dev_addr = phys_to_dma(hwdev, paddr);
+		dev_addr = swiotlb_phys_to_dma(hwdev, paddr);
 
 		/* Confirm address can be DMA'd by device */
 		if (dev_addr + size - 1 > dma_mask) {
@@ -812,10 +850,10 @@ dma_addr_t swiotlb_map_page(struct device *dev, struct page *page,
 	map = map_single(dev, phys, size, dir, attrs);
 	if (map == SWIOTLB_MAP_ERROR) {
 		swiotlb_full(dev, size, dir, 1);
-		return phys_to_dma(dev, io_tlb_overflow_buffer);
+		return swiotlb_phys_to_dma(dev, io_tlb_overflow_buffer);
 	}
 
-	dev_addr = phys_to_dma(dev, map);
+	dev_addr = swiotlb_phys_to_dma(dev, map);
 
 	/* Ensure that the address returned is DMA'ble */
 	if (dma_capable(dev, dev_addr, size))
@@ -824,7 +862,7 @@ dma_addr_t swiotlb_map_page(struct device *dev, struct page *page,
 	attrs |= DMA_ATTR_SKIP_CPU_SYNC;
 	swiotlb_tbl_unmap_single(dev, map, size, dir, attrs);
 
-	return phys_to_dma(dev, io_tlb_overflow_buffer);
+	return swiotlb_phys_to_dma(dev, io_tlb_overflow_buffer);
 }
 EXPORT_SYMBOL_GPL(swiotlb_map_page);
 
@@ -958,7 +996,7 @@ void swiotlb_unmap_page(struct device *hwdev, dma_addr_t dev_addr,
 				sg_dma_len(sgl) = 0;
 				return 0;
 			}
-			sg->dma_address = phys_to_dma(hwdev, map);
+			sg->dma_address = swiotlb_phys_to_dma(hwdev, map);
 		} else
 			sg->dma_address = dev_addr;
 		sg_dma_len(sg) = sg->length;
@@ -1026,7 +1064,7 @@ void swiotlb_unmap_page(struct device *hwdev, dma_addr_t dev_addr,
 int
 swiotlb_dma_mapping_error(struct device *hwdev, dma_addr_t dma_addr)
 {
-	return (dma_addr == phys_to_dma(hwdev, io_tlb_overflow_buffer));
+	return (dma_addr == swiotlb_phys_to_dma(hwdev, io_tlb_overflow_buffer));
 }
 EXPORT_SYMBOL(swiotlb_dma_mapping_error);
 
@@ -1039,6 +1077,6 @@ void swiotlb_unmap_page(struct device *hwdev, dma_addr_t dev_addr,
 int
 swiotlb_dma_supported(struct device *hwdev, u64 mask)
 {
-	return phys_to_dma(hwdev, io_tlb_end - 1) <= mask;
+	return swiotlb_phys_to_dma(hwdev, io_tlb_end - 1) <= mask;
 }
 EXPORT_SYMBOL(swiotlb_dma_supported);



From thomas.lendacky at amd.com  Tue Jun 27 08:12:06 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Tue, 27 Jun 2017 10:12:06 -0500
Subject: [PATCH v8 RESEND 25/38] swiotlb: Add warnings for use of bounce
 buffers with SME
In-Reply-To: <20170627150718.17428.81813.stgit@tlendack-t1.amdoffice.net>
References: <20170627150718.17428.81813.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170627151206.17428.59944.stgit@tlendack-t1.amdoffice.net>

Add warnings to let the user know when bounce buffers are being used for
DMA when SME is active.  Since the bounce buffers are not in encrypted
memory, these notifications are to allow the user to determine some
appropriate action - if necessary.  Actions can range from utilizing an
IOMMU, replacing the device with another device that can support 64-bit
DMA, ignoring the message if the device isn't used much, etc.

Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 include/linux/dma-mapping.h |   13 +++++++++++++
 lib/swiotlb.c               |    3 +++
 2 files changed, 16 insertions(+)

diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h
index 4f3eece..a156c40 100644
--- a/include/linux/dma-mapping.h
+++ b/include/linux/dma-mapping.h
@@ -10,6 +10,7 @@
 #include <linux/scatterlist.h>
 #include <linux/kmemcheck.h>
 #include <linux/bug.h>
+#include <linux/mem_encrypt.h>
 
 /**
  * List of possible attributes associated with a DMA mapping. The semantics
@@ -554,6 +555,12 @@ static inline int dma_mapping_error(struct device *dev, dma_addr_t dma_addr)
 #endif
 }
 
+static inline void dma_check_mask(struct device *dev, u64 mask)
+{
+	if (sme_active() && (mask < (((u64)sme_get_me_mask() << 1) - 1)))
+		dev_warn(dev, "SME is active, device will require DMA bounce buffers\n");
+}
+
 #ifndef HAVE_ARCH_DMA_SUPPORTED
 static inline int dma_supported(struct device *dev, u64 mask)
 {
@@ -577,6 +584,9 @@ static inline int dma_set_mask(struct device *dev, u64 mask)
 
 	if (!dev->dma_mask || !dma_supported(dev, mask))
 		return -EIO;
+
+	dma_check_mask(dev, mask);
+
 	*dev->dma_mask = mask;
 	return 0;
 }
@@ -596,6 +606,9 @@ static inline int dma_set_coherent_mask(struct device *dev, u64 mask)
 {
 	if (!dma_supported(dev, mask))
 		return -EIO;
+
+	dma_check_mask(dev, mask);
+
 	dev->coherent_dma_mask = mask;
 	return 0;
 }
diff --git a/lib/swiotlb.c b/lib/swiotlb.c
index 04ac91a..8c6c83e 100644
--- a/lib/swiotlb.c
+++ b/lib/swiotlb.c
@@ -507,6 +507,9 @@ phys_addr_t swiotlb_tbl_map_single(struct device *hwdev,
 	if (no_iotlb_memory)
 		panic("Can not allocate SWIOTLB buffer earlier and can't now provide you with the DMA bounce buffer");
 
+	if (sme_active())
+		pr_warn_once("SME is active and system is using DMA bounce buffers\n");
+
 	mask = dma_get_seg_boundary(hwdev);
 
 	tbl_dma_addr &= mask;



From thomas.lendacky at amd.com  Tue Jun 27 08:12:41 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Tue, 27 Jun 2017 10:12:41 -0500
Subject: [PATCH v8 RESEND 28/38] x86,
 realmode: Check for memory encryption on the APs
In-Reply-To: <20170627150718.17428.81813.stgit@tlendack-t1.amdoffice.net>
References: <20170627150718.17428.81813.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170627151241.17428.99489.stgit@tlendack-t1.amdoffice.net>

Add support to check if memory encryption is active in the kernel and that
it has been enabled on the AP. If memory encryption is active in the kernel
but has not been enabled on the AP, then set the memory encryption bit (bit
23) of MSR_K8_SYSCFG to enable memory encryption on that AP and allow the
AP to continue start up.

Reviewed-by: Borislav Petkov <bp at suse.de>
Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/include/asm/realmode.h      |   12 ++++++++++++
 arch/x86/realmode/init.c             |    4 ++++
 arch/x86/realmode/rm/trampoline_64.S |   24 ++++++++++++++++++++++++
 3 files changed, 40 insertions(+)

diff --git a/arch/x86/include/asm/realmode.h b/arch/x86/include/asm/realmode.h
index 230e190..90d9152 100644
--- a/arch/x86/include/asm/realmode.h
+++ b/arch/x86/include/asm/realmode.h
@@ -1,6 +1,15 @@
 #ifndef _ARCH_X86_REALMODE_H
 #define _ARCH_X86_REALMODE_H
 
+/*
+ * Flag bit definitions for use with the flags field of the trampoline header
+ * in the CONFIG_X86_64 variant.
+ */
+#define TH_FLAGS_SME_ACTIVE_BIT		0
+#define TH_FLAGS_SME_ACTIVE		BIT(TH_FLAGS_SME_ACTIVE_BIT)
+
+#ifndef __ASSEMBLY__
+
 #include <linux/types.h>
 #include <asm/io.h>
 
@@ -38,6 +47,7 @@ struct trampoline_header {
 	u64 start;
 	u64 efer;
 	u32 cr4;
+	u32 flags;
 #endif
 };
 
@@ -69,4 +79,6 @@ static inline size_t real_mode_size_needed(void)
 void set_real_mode_mem(phys_addr_t mem, size_t size);
 void reserve_real_mode(void);
 
+#endif /* __ASSEMBLY__ */
+
 #endif /* _ARCH_X86_REALMODE_H */
diff --git a/arch/x86/realmode/init.c b/arch/x86/realmode/init.c
index d6ddc7e..1f71980 100644
--- a/arch/x86/realmode/init.c
+++ b/arch/x86/realmode/init.c
@@ -108,6 +108,10 @@ static void __init setup_real_mode(void)
 	trampoline_cr4_features = &trampoline_header->cr4;
 	*trampoline_cr4_features = mmu_cr4_features;
 
+	trampoline_header->flags = 0;
+	if (sme_active())
+		trampoline_header->flags |= TH_FLAGS_SME_ACTIVE;
+
 	trampoline_pgd = (u64 *) __va(real_mode_header->trampoline_pgd);
 	trampoline_pgd[0] = trampoline_pgd_entry.pgd;
 	trampoline_pgd[511] = init_top_pgt[511].pgd;
diff --git a/arch/x86/realmode/rm/trampoline_64.S b/arch/x86/realmode/rm/trampoline_64.S
index dac7b20..614fd70 100644
--- a/arch/x86/realmode/rm/trampoline_64.S
+++ b/arch/x86/realmode/rm/trampoline_64.S
@@ -30,6 +30,7 @@
 #include <asm/msr.h>
 #include <asm/segment.h>
 #include <asm/processor-flags.h>
+#include <asm/realmode.h>
 #include "realmode.h"
 
 	.text
@@ -92,6 +93,28 @@ ENTRY(startup_32)
 	movl	%edx, %fs
 	movl	%edx, %gs
 
+	/*
+	 * Check for memory encryption support. This is a safety net in
+	 * case BIOS hasn't done the necessary step of setting the bit in
+	 * the MSR for this AP. If SME is active and we've gotten this far
+	 * then it is safe for us to set the MSR bit and continue. If we
+	 * don't we'll eventually crash trying to execute encrypted
+	 * instructions.
+	 */
+	bt	$TH_FLAGS_SME_ACTIVE_BIT, pa_tr_flags
+	jnc	.Ldone
+	movl	$MSR_K8_SYSCFG, %ecx
+	rdmsr
+	bts	$MSR_K8_SYSCFG_MEM_ENCRYPT_BIT, %eax
+	jc	.Ldone
+
+	/*
+	 * Memory encryption is enabled but the SME enable bit for this
+	 * CPU has has not been set.  It is safe to set it, so do so.
+	 */
+	wrmsr
+.Ldone:
+
 	movl	pa_tr_cr4, %eax
 	movl	%eax, %cr4		# Enable PAE mode
 
@@ -147,6 +170,7 @@ GLOBAL(trampoline_header)
 	tr_start:		.space	8
 	GLOBAL(tr_efer)		.space	8
 	GLOBAL(tr_cr4)		.space	4
+	GLOBAL(tr_flags)	.space	4
 END(trampoline_header)
 
 #include "trampoline_common.S"



From thomas.lendacky at amd.com  Tue Jun 27 08:12:53 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Tue, 27 Jun 2017 10:12:53 -0500
Subject: [PATCH v8 RESEND 29/38] x86, drm,
 fbdev: Do not specify encrypted memory for video mappings
In-Reply-To: <20170627150718.17428.81813.stgit@tlendack-t1.amdoffice.net>
References: <20170627150718.17428.81813.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170627151253.17428.76893.stgit@tlendack-t1.amdoffice.net>

Since video memory needs to be accessed decrypted, be sure that the
memory encryption mask is not set for the video ranges.

Reviewed-by: Borislav Petkov <bp at suse.de>
Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/include/asm/vga.h       |   14 +++++++++++++-
 arch/x86/mm/pageattr.c           |    2 ++
 drivers/gpu/drm/drm_gem.c        |    2 ++
 drivers/gpu/drm/drm_vm.c         |    4 ++++
 drivers/gpu/drm/ttm/ttm_bo_vm.c  |    7 +++++--
 drivers/gpu/drm/udl/udl_fb.c     |    4 ++++
 drivers/video/fbdev/core/fbmem.c |   12 ++++++++++++
 7 files changed, 42 insertions(+), 3 deletions(-)

diff --git a/arch/x86/include/asm/vga.h b/arch/x86/include/asm/vga.h
index c4b9dc2..9f42bee 100644
--- a/arch/x86/include/asm/vga.h
+++ b/arch/x86/include/asm/vga.h
@@ -7,12 +7,24 @@
 #ifndef _ASM_X86_VGA_H
 #define _ASM_X86_VGA_H
 
+#include <asm/set_memory.h>
+
 /*
  *	On the PC, we can just recalculate addresses and then
  *	access the videoram directly without any black magic.
+ *	To support memory encryption however, we need to access
+ *	the videoram as decrypted memory.
  */
 
-#define VGA_MAP_MEM(x, s) (unsigned long)phys_to_virt(x)
+#define VGA_MAP_MEM(x, s)					\
+({								\
+	unsigned long start = (unsigned long)phys_to_virt(x);	\
+								\
+	if (IS_ENABLED(CONFIG_AMD_MEM_ENCRYPT))			\
+		set_memory_decrypted(start, (s) >> PAGE_SHIFT);	\
+								\
+	start;							\
+})
 
 #define vga_readb(x) (*(x))
 #define vga_writeb(x, y) (*(y) = (x))
diff --git a/arch/x86/mm/pageattr.c b/arch/x86/mm/pageattr.c
index d9e09fb..13fc5db 100644
--- a/arch/x86/mm/pageattr.c
+++ b/arch/x86/mm/pageattr.c
@@ -1825,11 +1825,13 @@ int set_memory_encrypted(unsigned long addr, int numpages)
 {
 	return __set_memory_enc_dec(addr, numpages, true);
 }
+EXPORT_SYMBOL_GPL(set_memory_encrypted);
 
 int set_memory_decrypted(unsigned long addr, int numpages)
 {
 	return __set_memory_enc_dec(addr, numpages, false);
 }
+EXPORT_SYMBOL_GPL(set_memory_decrypted);
 
 int set_pages_uc(struct page *page, int numpages)
 {
diff --git a/drivers/gpu/drm/drm_gem.c b/drivers/gpu/drm/drm_gem.c
index b1e28c9..019f48c 100644
--- a/drivers/gpu/drm/drm_gem.c
+++ b/drivers/gpu/drm/drm_gem.c
@@ -36,6 +36,7 @@
 #include <linux/pagemap.h>
 #include <linux/shmem_fs.h>
 #include <linux/dma-buf.h>
+#include <linux/mem_encrypt.h>
 #include <drm/drmP.h>
 #include <drm/drm_vma_manager.h>
 #include <drm/drm_gem.h>
@@ -928,6 +929,7 @@ int drm_gem_mmap_obj(struct drm_gem_object *obj, unsigned long obj_size,
 	vma->vm_ops = dev->driver->gem_vm_ops;
 	vma->vm_private_data = obj;
 	vma->vm_page_prot = pgprot_writecombine(vm_get_page_prot(vma->vm_flags));
+	vma->vm_page_prot = pgprot_decrypted(vma->vm_page_prot);
 
 	/* Take a ref for this mapping of the object, so that the fault
 	 * handler can dereference the mmap offset's pointer to the object.
diff --git a/drivers/gpu/drm/drm_vm.c b/drivers/gpu/drm/drm_vm.c
index 1170b32..ed4bcbf 100644
--- a/drivers/gpu/drm/drm_vm.c
+++ b/drivers/gpu/drm/drm_vm.c
@@ -40,6 +40,7 @@
 #include <linux/efi.h>
 #include <linux/slab.h>
 #endif
+#include <linux/mem_encrypt.h>
 #include <asm/pgtable.h>
 #include "drm_internal.h"
 #include "drm_legacy.h"
@@ -58,6 +59,9 @@ static pgprot_t drm_io_prot(struct drm_local_map *map,
 {
 	pgprot_t tmp = vm_get_page_prot(vma->vm_flags);
 
+	/* We don't want graphics memory to be mapped encrypted */
+	tmp = pgprot_decrypted(tmp);
+
 #if defined(__i386__) || defined(__x86_64__) || defined(__powerpc__)
 	if (map->type == _DRM_REGISTERS && !(map->flags & _DRM_WRITE_COMBINING))
 		tmp = pgprot_noncached(tmp);
diff --git a/drivers/gpu/drm/ttm/ttm_bo_vm.c b/drivers/gpu/drm/ttm/ttm_bo_vm.c
index 9f53df9..622dab6 100644
--- a/drivers/gpu/drm/ttm/ttm_bo_vm.c
+++ b/drivers/gpu/drm/ttm/ttm_bo_vm.c
@@ -39,6 +39,7 @@
 #include <linux/rbtree.h>
 #include <linux/module.h>
 #include <linux/uaccess.h>
+#include <linux/mem_encrypt.h>
 
 #define TTM_BO_VM_NUM_PREFAULT 16
 
@@ -230,9 +231,11 @@ static int ttm_bo_vm_fault(struct vm_fault *vmf)
 	 * first page.
 	 */
 	for (i = 0; i < TTM_BO_VM_NUM_PREFAULT; ++i) {
-		if (bo->mem.bus.is_iomem)
+		if (bo->mem.bus.is_iomem) {
+			/* Iomem should not be marked encrypted */
+			cvma.vm_page_prot = pgprot_decrypted(cvma.vm_page_prot);
 			pfn = bdev->driver->io_mem_pfn(bo, page_offset);
-		else {
+		} else {
 			page = ttm->pages[page_offset];
 			if (unlikely(!page && i == 0)) {
 				retval = VM_FAULT_OOM;
diff --git a/drivers/gpu/drm/udl/udl_fb.c b/drivers/gpu/drm/udl/udl_fb.c
index 4a65003..92e1690 100644
--- a/drivers/gpu/drm/udl/udl_fb.c
+++ b/drivers/gpu/drm/udl/udl_fb.c
@@ -14,6 +14,7 @@
 #include <linux/slab.h>
 #include <linux/fb.h>
 #include <linux/dma-buf.h>
+#include <linux/mem_encrypt.h>
 
 #include <drm/drmP.h>
 #include <drm/drm_crtc.h>
@@ -169,6 +170,9 @@ static int udl_fb_mmap(struct fb_info *info, struct vm_area_struct *vma)
 	pr_notice("mmap() framebuffer addr:%lu size:%lu\n",
 		  pos, size);
 
+	/* We don't want the framebuffer to be mapped encrypted */
+	vma->vm_page_prot = pgprot_decrypted(vma->vm_page_prot);
+
 	while (size > 0) {
 		page = vmalloc_to_pfn((void *)pos);
 		if (remap_pfn_range(vma, start, page, PAGE_SIZE, PAGE_SHARED))
diff --git a/drivers/video/fbdev/core/fbmem.c b/drivers/video/fbdev/core/fbmem.c
index 069fe79..b5e7c33 100644
--- a/drivers/video/fbdev/core/fbmem.c
+++ b/drivers/video/fbdev/core/fbmem.c
@@ -32,6 +32,7 @@
 #include <linux/device.h>
 #include <linux/efi.h>
 #include <linux/fb.h>
+#include <linux/mem_encrypt.h>
 
 #include <asm/fb.h>
 
@@ -1405,6 +1406,12 @@ static long fb_compat_ioctl(struct file *file, unsigned int cmd,
 	mutex_lock(&info->mm_lock);
 	if (fb->fb_mmap) {
 		int res;
+
+		/*
+		 * The framebuffer needs to be accessed decrypted, be sure
+		 * SME protection is removed ahead of the call
+		 */
+		vma->vm_page_prot = pgprot_decrypted(vma->vm_page_prot);
 		res = fb->fb_mmap(info, vma);
 		mutex_unlock(&info->mm_lock);
 		return res;
@@ -1430,6 +1437,11 @@ static long fb_compat_ioctl(struct file *file, unsigned int cmd,
 	mutex_unlock(&info->mm_lock);
 
 	vma->vm_page_prot = vm_get_page_prot(vma->vm_flags);
+	/*
+	 * The framebuffer needs to be accessed decrypted, be sure
+	 * SME protection is removed
+	 */
+	vma->vm_page_prot = pgprot_decrypted(vma->vm_page_prot);
 	fb_pgprotect(file, vma, start);
 
 	return vm_iomap_memory(vma, start, len);



From thomas.lendacky at amd.com  Tue Jun 27 08:13:04 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Tue, 27 Jun 2017 10:13:04 -0500
Subject: [PATCH v8 RESEND 30/38] kvm: x86: svm: Support Secure Memory
 Encryption within KVM
In-Reply-To: <20170627150718.17428.81813.stgit@tlendack-t1.amdoffice.net>
References: <20170627150718.17428.81813.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170627151304.17428.80457.stgit@tlendack-t1.amdoffice.net>

Update the KVM support to work with SME. The VMCB has a number of fields
where physical addresses are used and these addresses must contain the
memory encryption mask in order to properly access the encrypted memory.
Also, use the memory encryption mask when creating and using the nested
page tables.

Reviewed-by: Borislav Petkov <bp at suse.de>
Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/include/asm/kvm_host.h |    2 +-
 arch/x86/kvm/mmu.c              |   12 ++++++++----
 arch/x86/kvm/mmu.h              |    2 +-
 arch/x86/kvm/svm.c              |   35 ++++++++++++++++++-----------------
 arch/x86/kvm/vmx.c              |    3 ++-
 arch/x86/kvm/x86.c              |    3 ++-
 6 files changed, 32 insertions(+), 25 deletions(-)

diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index 695605e..6d1267f 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -1069,7 +1069,7 @@ struct kvm_arch_async_pf {
 void kvm_mmu_uninit_vm(struct kvm *kvm);
 void kvm_mmu_set_mask_ptes(u64 user_mask, u64 accessed_mask,
 		u64 dirty_mask, u64 nx_mask, u64 x_mask, u64 p_mask,
-		u64 acc_track_mask);
+		u64 acc_track_mask, u64 me_mask);
 
 void kvm_mmu_reset_context(struct kvm_vcpu *vcpu);
 void kvm_mmu_slot_remove_write_access(struct kvm *kvm,
diff --git a/arch/x86/kvm/mmu.c b/arch/x86/kvm/mmu.c
index cb82259..e85888c 100644
--- a/arch/x86/kvm/mmu.c
+++ b/arch/x86/kvm/mmu.c
@@ -107,7 +107,7 @@ enum {
 	(((address) >> PT32_LEVEL_SHIFT(level)) & ((1 << PT32_LEVEL_BITS) - 1))
 
 
-#define PT64_BASE_ADDR_MASK (((1ULL << 52) - 1) & ~(u64)(PAGE_SIZE-1))
+#define PT64_BASE_ADDR_MASK __sme_clr((((1ULL << 52) - 1) & ~(u64)(PAGE_SIZE-1)))
 #define PT64_DIR_BASE_ADDR_MASK \
 	(PT64_BASE_ADDR_MASK & ~((1ULL << (PAGE_SHIFT + PT64_LEVEL_BITS)) - 1))
 #define PT64_LVL_ADDR_MASK(level) \
@@ -125,7 +125,7 @@ enum {
 					    * PT32_LEVEL_BITS))) - 1))
 
 #define PT64_PERM_MASK (PT_PRESENT_MASK | PT_WRITABLE_MASK | shadow_user_mask \
-			| shadow_x_mask | shadow_nx_mask)
+			| shadow_x_mask | shadow_nx_mask | shadow_me_mask)
 
 #define ACC_EXEC_MASK    1
 #define ACC_WRITE_MASK   PT_WRITABLE_MASK
@@ -184,6 +184,7 @@ struct kvm_shadow_walk_iterator {
 static u64 __read_mostly shadow_dirty_mask;
 static u64 __read_mostly shadow_mmio_mask;
 static u64 __read_mostly shadow_present_mask;
+static u64 __read_mostly shadow_me_mask;
 
 /*
  * The mask/value to distinguish a PTE that has been marked not-present for
@@ -317,7 +318,7 @@ static bool check_mmio_spte(struct kvm_vcpu *vcpu, u64 spte)
 
 void kvm_mmu_set_mask_ptes(u64 user_mask, u64 accessed_mask,
 		u64 dirty_mask, u64 nx_mask, u64 x_mask, u64 p_mask,
-		u64 acc_track_mask)
+		u64 acc_track_mask, u64 me_mask)
 {
 	if (acc_track_mask != 0)
 		acc_track_mask |= SPTE_SPECIAL_MASK;
@@ -330,6 +331,7 @@ void kvm_mmu_set_mask_ptes(u64 user_mask, u64 accessed_mask,
 	shadow_present_mask = p_mask;
 	shadow_acc_track_mask = acc_track_mask;
 	WARN_ON(shadow_accessed_mask != 0 && shadow_acc_track_mask != 0);
+	shadow_me_mask = me_mask;
 }
 EXPORT_SYMBOL_GPL(kvm_mmu_set_mask_ptes);
 
@@ -2398,7 +2400,8 @@ static void link_shadow_page(struct kvm_vcpu *vcpu, u64 *sptep,
 	BUILD_BUG_ON(VMX_EPT_WRITABLE_MASK != PT_WRITABLE_MASK);
 
 	spte = __pa(sp->spt) | shadow_present_mask | PT_WRITABLE_MASK |
-	       shadow_user_mask | shadow_x_mask | shadow_accessed_mask;
+	       shadow_user_mask | shadow_x_mask | shadow_accessed_mask |
+	       shadow_me_mask;
 
 	mmu_spte_set(sptep, spte);
 
@@ -2700,6 +2703,7 @@ static int set_spte(struct kvm_vcpu *vcpu, u64 *sptep,
 		pte_access &= ~ACC_WRITE_MASK;
 
 	spte |= (u64)pfn << PAGE_SHIFT;
+	spte |= shadow_me_mask;
 
 	if (pte_access & ACC_WRITE_MASK) {
 
diff --git a/arch/x86/kvm/mmu.h b/arch/x86/kvm/mmu.h
index 330bf3a..08b779d 100644
--- a/arch/x86/kvm/mmu.h
+++ b/arch/x86/kvm/mmu.h
@@ -48,7 +48,7 @@
 
 static inline u64 rsvd_bits(int s, int e)
 {
-	return ((1ULL << (e - s + 1)) - 1) << s;
+	return __sme_clr(((1ULL << (e - s + 1)) - 1) << s);
 }
 
 void kvm_mmu_set_mmio_spte_mask(u64 mmio_mask);
diff --git a/arch/x86/kvm/svm.c b/arch/x86/kvm/svm.c
index ba9891a..d2e9fca 100644
--- a/arch/x86/kvm/svm.c
+++ b/arch/x86/kvm/svm.c
@@ -1138,9 +1138,9 @@ static void avic_init_vmcb(struct vcpu_svm *svm)
 {
 	struct vmcb *vmcb = svm->vmcb;
 	struct kvm_arch *vm_data = &svm->vcpu.kvm->arch;
-	phys_addr_t bpa = page_to_phys(svm->avic_backing_page);
-	phys_addr_t lpa = page_to_phys(vm_data->avic_logical_id_table_page);
-	phys_addr_t ppa = page_to_phys(vm_data->avic_physical_id_table_page);
+	phys_addr_t bpa = __sme_set(page_to_phys(svm->avic_backing_page));
+	phys_addr_t lpa = __sme_set(page_to_phys(vm_data->avic_logical_id_table_page));
+	phys_addr_t ppa = __sme_set(page_to_phys(vm_data->avic_physical_id_table_page));
 
 	vmcb->control.avic_backing_page = bpa & AVIC_HPA_MASK;
 	vmcb->control.avic_logical_id = lpa & AVIC_HPA_MASK;
@@ -1203,8 +1203,8 @@ static void init_vmcb(struct vcpu_svm *svm)
 		set_intercept(svm, INTERCEPT_MWAIT);
 	}
 
-	control->iopm_base_pa = iopm_base;
-	control->msrpm_base_pa = __pa(svm->msrpm);
+	control->iopm_base_pa = __sme_set(iopm_base);
+	control->msrpm_base_pa = __sme_set(__pa(svm->msrpm));
 	control->int_ctl = V_INTR_MASKING_MASK;
 
 	init_seg(&save->es);
@@ -1338,9 +1338,9 @@ static int avic_init_backing_page(struct kvm_vcpu *vcpu)
 		return -EINVAL;
 
 	new_entry = READ_ONCE(*entry);
-	new_entry = (page_to_phys(svm->avic_backing_page) &
-		     AVIC_PHYSICAL_ID_ENTRY_BACKING_PAGE_MASK) |
-		     AVIC_PHYSICAL_ID_ENTRY_VALID_MASK;
+	new_entry = __sme_set((page_to_phys(svm->avic_backing_page) &
+			      AVIC_PHYSICAL_ID_ENTRY_BACKING_PAGE_MASK) |
+			      AVIC_PHYSICAL_ID_ENTRY_VALID_MASK);
 	WRITE_ONCE(*entry, new_entry);
 
 	svm->avic_physical_id_cache = entry;
@@ -1608,7 +1608,7 @@ static struct kvm_vcpu *svm_create_vcpu(struct kvm *kvm, unsigned int id)
 
 	svm->vmcb = page_address(page);
 	clear_page(svm->vmcb);
-	svm->vmcb_pa = page_to_pfn(page) << PAGE_SHIFT;
+	svm->vmcb_pa = __sme_set(page_to_pfn(page) << PAGE_SHIFT);
 	svm->asid_generation = 0;
 	init_vmcb(svm);
 
@@ -1636,7 +1636,7 @@ static void svm_free_vcpu(struct kvm_vcpu *vcpu)
 {
 	struct vcpu_svm *svm = to_svm(vcpu);
 
-	__free_page(pfn_to_page(svm->vmcb_pa >> PAGE_SHIFT));
+	__free_page(pfn_to_page(__sme_clr(svm->vmcb_pa) >> PAGE_SHIFT));
 	__free_pages(virt_to_page(svm->msrpm), MSRPM_ALLOC_ORDER);
 	__free_page(virt_to_page(svm->nested.hsave));
 	__free_pages(virt_to_page(svm->nested.msrpm), MSRPM_ALLOC_ORDER);
@@ -2303,7 +2303,7 @@ static u64 nested_svm_get_tdp_pdptr(struct kvm_vcpu *vcpu, int index)
 	u64 pdpte;
 	int ret;
 
-	ret = kvm_vcpu_read_guest_page(vcpu, gpa_to_gfn(cr3), &pdpte,
+	ret = kvm_vcpu_read_guest_page(vcpu, gpa_to_gfn(__sme_clr(cr3)), &pdpte,
 				       offset_in_page(cr3) + index * 8, 8);
 	if (ret)
 		return 0;
@@ -2315,7 +2315,7 @@ static void nested_svm_set_tdp_cr3(struct kvm_vcpu *vcpu,
 {
 	struct vcpu_svm *svm = to_svm(vcpu);
 
-	svm->vmcb->control.nested_cr3 = root;
+	svm->vmcb->control.nested_cr3 = __sme_set(root);
 	mark_dirty(svm->vmcb, VMCB_NPT);
 	svm_flush_tlb(vcpu);
 }
@@ -2803,7 +2803,7 @@ static bool nested_svm_vmrun_msrpm(struct vcpu_svm *svm)
 		svm->nested.msrpm[p] = svm->msrpm[p] | value;
 	}
 
-	svm->vmcb->control.msrpm_base_pa = __pa(svm->nested.msrpm);
+	svm->vmcb->control.msrpm_base_pa = __sme_set(__pa(svm->nested.msrpm));
 
 	return true;
 }
@@ -4435,7 +4435,7 @@ static int svm_ir_list_add(struct vcpu_svm *svm, struct amd_iommu_pi_data *pi)
 	pr_debug("SVM: %s: use GA mode for irq %u\n", __func__,
 		 irq.vector);
 	*svm = to_svm(vcpu);
-	vcpu_info->pi_desc_addr = page_to_phys((*svm)->avic_backing_page);
+	vcpu_info->pi_desc_addr = __sme_set(page_to_phys((*svm)->avic_backing_page));
 	vcpu_info->vector = irq.vector;
 
 	return 0;
@@ -4486,7 +4486,8 @@ static int svm_update_pi_irte(struct kvm *kvm, unsigned int host_irq,
 			struct amd_iommu_pi_data pi;
 
 			/* Try to enable guest_mode in IRTE */
-			pi.base = page_to_phys(svm->avic_backing_page) & AVIC_HPA_MASK;
+			pi.base = __sme_set(page_to_phys(svm->avic_backing_page) &
+					    AVIC_HPA_MASK);
 			pi.ga_tag = AVIC_GATAG(kvm->arch.avic_vm_id,
 						     svm->vcpu.vcpu_id);
 			pi.is_guest_mode = true;
@@ -4911,7 +4912,7 @@ static void svm_set_cr3(struct kvm_vcpu *vcpu, unsigned long root)
 {
 	struct vcpu_svm *svm = to_svm(vcpu);
 
-	svm->vmcb->save.cr3 = root;
+	svm->vmcb->save.cr3 = __sme_set(root);
 	mark_dirty(svm->vmcb, VMCB_CR);
 	svm_flush_tlb(vcpu);
 }
@@ -4920,7 +4921,7 @@ static void set_tdp_cr3(struct kvm_vcpu *vcpu, unsigned long root)
 {
 	struct vcpu_svm *svm = to_svm(vcpu);
 
-	svm->vmcb->control.nested_cr3 = root;
+	svm->vmcb->control.nested_cr3 = __sme_set(root);
 	mark_dirty(svm->vmcb, VMCB_NPT);
 
 	/* Also sync guest cr3 here in case we live migrate */
diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c
index 7dd53fb..53098cd 100644
--- a/arch/x86/kvm/vmx.c
+++ b/arch/x86/kvm/vmx.c
@@ -6452,7 +6452,8 @@ void vmx_enable_tdp(void)
 		enable_ept_ad_bits ? VMX_EPT_DIRTY_BIT : 0ull,
 		0ull, VMX_EPT_EXECUTABLE_MASK,
 		cpu_has_vmx_ept_execute_only() ? 0ull : VMX_EPT_READABLE_MASK,
-		enable_ept_ad_bits ? 0ull : VMX_EPT_RWX_MASK);
+		enable_ept_ad_bits ? 0ull : VMX_EPT_RWX_MASK,
+		0ull);
 
 	ept_set_mmio_spte_mask();
 	kvm_enable_tdp();
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 87d3cb9..559f710 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -54,6 +54,7 @@
 #include <linux/kvm_irqfd.h>
 #include <linux/irqbypass.h>
 #include <linux/sched/stat.h>
+#include <linux/mem_encrypt.h>
 
 #include <trace/events/kvm.h>
 
@@ -6095,7 +6096,7 @@ int kvm_arch_init(void *opaque)
 
 	kvm_mmu_set_mask_ptes(PT_USER_MASK, PT_ACCESSED_MASK,
 			PT_DIRTY_MASK, PT64_NX_MASK, 0,
-			PT_PRESENT_MASK, 0);
+			PT_PRESENT_MASK, 0, sme_me_mask);
 	kvm_timer_init();
 
 	perf_register_guest_info_callbacks(&kvm_guest_cbs);



From thomas.lendacky at amd.com  Tue Jun 27 08:13:14 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Tue, 27 Jun 2017 10:13:14 -0500
Subject: [PATCH v8 RESEND 31/38] x86/mm, kexec: Allow kexec to be used with SME
In-Reply-To: <20170627150718.17428.81813.stgit@tlendack-t1.amdoffice.net>
References: <20170627150718.17428.81813.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170627151314.17428.39391.stgit@tlendack-t1.amdoffice.net>

Provide support so that kexec can be used to boot a kernel when SME is
enabled.

Support is needed to allocate pages for kexec without encryption.  This
is needed in order to be able to reboot in the kernel in the same manner
as originally booted.

Additionally, when shutting down all of the CPUs we need to be sure to
flush the caches and then halt. This is needed when booting from a state
where SME was not active into a state where SME is active (or vice-versa).
Without these steps, it is possible for cache lines to exist for the same
physical location but tagged both with and without the encryption bit. This
can cause random memory corruption when caches are flushed depending on
which cacheline is written last.

Reviewed-by: Borislav Petkov <bp at suse.de>
Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/include/asm/init.h          |    1 +
 arch/x86/include/asm/kexec.h         |    8 ++++++++
 arch/x86/include/asm/pgtable_types.h |    1 +
 arch/x86/kernel/machine_kexec_64.c   |   22 +++++++++++++++++++++-
 arch/x86/kernel/process.c            |   17 +++++++++++++++--
 arch/x86/mm/ident_map.c              |   12 ++++++++----
 include/linux/kexec.h                |    8 ++++++++
 kernel/kexec_core.c                  |   12 +++++++++++-
 8 files changed, 73 insertions(+), 8 deletions(-)

diff --git a/arch/x86/include/asm/init.h b/arch/x86/include/asm/init.h
index 474eb8c..05c4aa0 100644
--- a/arch/x86/include/asm/init.h
+++ b/arch/x86/include/asm/init.h
@@ -7,6 +7,7 @@ struct x86_mapping_info {
 	unsigned long page_flag;	 /* page flag for PMD or PUD entry */
 	unsigned long offset;		 /* ident mapping offset */
 	bool direct_gbpages;		 /* PUD level 1GB page support */
+	unsigned long kernpg_flag;	 /* kernel pagetable flag override */
 };
 
 int kernel_ident_mapping_init(struct x86_mapping_info *info, pgd_t *pgd_page,
diff --git a/arch/x86/include/asm/kexec.h b/arch/x86/include/asm/kexec.h
index 70ef205..e8183ac 100644
--- a/arch/x86/include/asm/kexec.h
+++ b/arch/x86/include/asm/kexec.h
@@ -207,6 +207,14 @@ struct kexec_entry64_regs {
 	uint64_t r15;
 	uint64_t rip;
 };
+
+extern int arch_kexec_post_alloc_pages(void *vaddr, unsigned int pages,
+				       gfp_t gfp);
+#define arch_kexec_post_alloc_pages arch_kexec_post_alloc_pages
+
+extern void arch_kexec_pre_free_pages(void *vaddr, unsigned int pages);
+#define arch_kexec_pre_free_pages arch_kexec_pre_free_pages
+
 #endif
 
 typedef void crash_vmclear_fn(void);
diff --git a/arch/x86/include/asm/pgtable_types.h b/arch/x86/include/asm/pgtable_types.h
index 32095af..830992f 100644
--- a/arch/x86/include/asm/pgtable_types.h
+++ b/arch/x86/include/asm/pgtable_types.h
@@ -213,6 +213,7 @@ enum page_cache_mode {
 #define PAGE_KERNEL		__pgprot(__PAGE_KERNEL | _PAGE_ENC)
 #define PAGE_KERNEL_RO		__pgprot(__PAGE_KERNEL_RO | _PAGE_ENC)
 #define PAGE_KERNEL_EXEC	__pgprot(__PAGE_KERNEL_EXEC | _PAGE_ENC)
+#define PAGE_KERNEL_EXEC_NOENC	__pgprot(__PAGE_KERNEL_EXEC)
 #define PAGE_KERNEL_RX		__pgprot(__PAGE_KERNEL_RX | _PAGE_ENC)
 #define PAGE_KERNEL_NOCACHE	__pgprot(__PAGE_KERNEL_NOCACHE | _PAGE_ENC)
 #define PAGE_KERNEL_LARGE	__pgprot(__PAGE_KERNEL_LARGE | _PAGE_ENC)
diff --git a/arch/x86/kernel/machine_kexec_64.c b/arch/x86/kernel/machine_kexec_64.c
index cb0a304..9cf8daa 100644
--- a/arch/x86/kernel/machine_kexec_64.c
+++ b/arch/x86/kernel/machine_kexec_64.c
@@ -87,7 +87,7 @@ static int init_transition_pgtable(struct kimage *image, pgd_t *pgd)
 		set_pmd(pmd, __pmd(__pa(pte) | _KERNPG_TABLE));
 	}
 	pte = pte_offset_kernel(pmd, vaddr);
-	set_pte(pte, pfn_pte(paddr >> PAGE_SHIFT, PAGE_KERNEL_EXEC));
+	set_pte(pte, pfn_pte(paddr >> PAGE_SHIFT, PAGE_KERNEL_EXEC_NOENC));
 	return 0;
 err:
 	free_transition_pgtable(image);
@@ -115,6 +115,7 @@ static int init_pgtable(struct kimage *image, unsigned long start_pgtable)
 		.alloc_pgt_page	= alloc_pgt_page,
 		.context	= image,
 		.page_flag	= __PAGE_KERNEL_LARGE_EXEC,
+		.kernpg_flag	= _KERNPG_TABLE_NOENC,
 	};
 	unsigned long mstart, mend;
 	pgd_t *level4p;
@@ -602,3 +603,22 @@ void arch_kexec_unprotect_crashkres(void)
 {
 	kexec_mark_crashkres(false);
 }
+
+int arch_kexec_post_alloc_pages(void *vaddr, unsigned int pages, gfp_t gfp)
+{
+	/*
+	 * If SME is active we need to be sure that kexec pages are
+	 * not encrypted because when we boot to the new kernel the
+	 * pages won't be accessed encrypted (initially).
+	 */
+	return set_memory_decrypted((unsigned long)vaddr, pages);
+}
+
+void arch_kexec_pre_free_pages(void *vaddr, unsigned int pages)
+{
+	/*
+	 * If SME is active we need to reset the pages back to being
+	 * an encrypted mapping before freeing them.
+	 */
+	set_memory_encrypted((unsigned long)vaddr, pages);
+}
diff --git a/arch/x86/kernel/process.c b/arch/x86/kernel/process.c
index 0bb8842..fdad0fb 100644
--- a/arch/x86/kernel/process.c
+++ b/arch/x86/kernel/process.c
@@ -355,6 +355,7 @@ bool xen_set_default_idle(void)
 	return ret;
 }
 #endif
+
 void stop_this_cpu(void *dummy)
 {
 	local_irq_disable();
@@ -365,8 +366,20 @@ void stop_this_cpu(void *dummy)
 	disable_local_APIC();
 	mcheck_cpu_clear(this_cpu_ptr(&cpu_info));
 
-	for (;;)
-		halt();
+	for (;;) {
+		/*
+		 * Use wbinvd followed by hlt to stop the processor. This
+		 * provides support for kexec on a processor that supports
+		 * SME. With kexec, going from SME inactive to SME active
+		 * requires clearing cache entries so that addresses without
+		 * the encryption bit set don't corrupt the same physical
+		 * address that has the encryption bit set when caches are
+		 * flushed. To achieve this a wbinvd is performed followed by
+		 * a hlt. Even if the processor is not in the kexec/SME
+		 * scenario this only adds a wbinvd to a halting processor.
+		 */
+		asm volatile("wbinvd; hlt" : : : "memory");
+	}
 }
 
 /*
diff --git a/arch/x86/mm/ident_map.c b/arch/x86/mm/ident_map.c
index adab159..31cea98 100644
--- a/arch/x86/mm/ident_map.c
+++ b/arch/x86/mm/ident_map.c
@@ -51,7 +51,7 @@ static int ident_pud_init(struct x86_mapping_info *info, pud_t *pud_page,
 		if (!pmd)
 			return -ENOMEM;
 		ident_pmd_init(info, pmd, addr, next);
-		set_pud(pud, __pud(__pa(pmd) | _KERNPG_TABLE));
+		set_pud(pud, __pud(__pa(pmd) | info->kernpg_flag));
 	}
 
 	return 0;
@@ -79,7 +79,7 @@ static int ident_p4d_init(struct x86_mapping_info *info, p4d_t *p4d_page,
 		if (!pud)
 			return -ENOMEM;
 		ident_pud_init(info, pud, addr, next);
-		set_p4d(p4d, __p4d(__pa(pud) | _KERNPG_TABLE));
+		set_p4d(p4d, __p4d(__pa(pud) | info->kernpg_flag));
 	}
 
 	return 0;
@@ -93,6 +93,10 @@ int kernel_ident_mapping_init(struct x86_mapping_info *info, pgd_t *pgd_page,
 	unsigned long next;
 	int result;
 
+	/* Set the default pagetable flags if not supplied */
+	if (!info->kernpg_flag)
+		info->kernpg_flag = _KERNPG_TABLE;
+
 	for (; addr < end; addr = next) {
 		pgd_t *pgd = pgd_page + pgd_index(addr);
 		p4d_t *p4d;
@@ -116,14 +120,14 @@ int kernel_ident_mapping_init(struct x86_mapping_info *info, pgd_t *pgd_page,
 		if (result)
 			return result;
 		if (IS_ENABLED(CONFIG_X86_5LEVEL)) {
-			set_pgd(pgd, __pgd(__pa(p4d) | _KERNPG_TABLE));
+			set_pgd(pgd, __pgd(__pa(p4d) | info->kernpg_flag));
 		} else {
 			/*
 			 * With p4d folded, pgd is equal to p4d.
 			 * The pgd entry has to point to the pud page table in this case.
 			 */
 			pud_t *pud = pud_offset(p4d, 0);
-			set_pgd(pgd, __pgd(__pa(pud) | _KERNPG_TABLE));
+			set_pgd(pgd, __pgd(__pa(pud) | info->kernpg_flag));
 		}
 	}
 
diff --git a/include/linux/kexec.h b/include/linux/kexec.h
index c9481eb..bd7af8b 100644
--- a/include/linux/kexec.h
+++ b/include/linux/kexec.h
@@ -334,6 +334,14 @@ static inline void *boot_phys_to_virt(unsigned long entry)
 	return phys_to_virt(boot_phys_to_phys(entry));
 }
 
+#ifndef arch_kexec_post_alloc_pages
+static inline int arch_kexec_post_alloc_pages(void *vaddr, unsigned int pages, gfp_t gfp) { return 0; }
+#endif
+
+#ifndef arch_kexec_pre_free_pages
+static inline void arch_kexec_pre_free_pages(void *vaddr, unsigned int pages) { }
+#endif
+
 #else /* !CONFIG_KEXEC_CORE */
 struct pt_regs;
 struct task_struct;
diff --git a/kernel/kexec_core.c b/kernel/kexec_core.c
index ae1a3ba..2df1c39 100644
--- a/kernel/kexec_core.c
+++ b/kernel/kexec_core.c
@@ -300,7 +300,7 @@ static struct page *kimage_alloc_pages(gfp_t gfp_mask, unsigned int order)
 {
 	struct page *pages;
 
-	pages = alloc_pages(gfp_mask, order);
+	pages = alloc_pages(gfp_mask & ~__GFP_ZERO, order);
 	if (pages) {
 		unsigned int count, i;
 
@@ -309,6 +309,13 @@ static struct page *kimage_alloc_pages(gfp_t gfp_mask, unsigned int order)
 		count = 1 << order;
 		for (i = 0; i < count; i++)
 			SetPageReserved(pages + i);
+
+		arch_kexec_post_alloc_pages(page_address(pages), count,
+					    gfp_mask);
+
+		if (gfp_mask & __GFP_ZERO)
+			for (i = 0; i < count; i++)
+				clear_highpage(pages + i);
 	}
 
 	return pages;
@@ -320,6 +327,9 @@ static void kimage_free_pages(struct page *page)
 
 	order = page_private(page);
 	count = 1 << order;
+
+	arch_kexec_pre_free_pages(page_address(page), count);
+
 	for (i = 0; i < count; i++)
 		ClearPageReserved(page + i);
 	__free_pages(page, order);



From thomas.lendacky at amd.com  Tue Jun 27 08:13:27 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Tue, 27 Jun 2017 10:13:27 -0500
Subject: [PATCH v8 RESEND 32/38] xen/x86: Remove SME feature in PV guests
In-Reply-To: <20170627150718.17428.81813.stgit@tlendack-t1.amdoffice.net>
References: <20170627150718.17428.81813.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170627151327.17428.11746.stgit@tlendack-t1.amdoffice.net>

Xen does not currently support SME for PV guests. Clear the SME CPU
capability in order to avoid any ambiguity.

Reviewed-by: Borislav Petkov <bp at suse.de>
Reviewed-by: Juergen Gross <jgross at suse.com>
Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/xen/enlighten_pv.c |    1 +
 1 file changed, 1 insertion(+)

diff --git a/arch/x86/xen/enlighten_pv.c b/arch/x86/xen/enlighten_pv.c
index f33eef4..e6ecf42 100644
--- a/arch/x86/xen/enlighten_pv.c
+++ b/arch/x86/xen/enlighten_pv.c
@@ -294,6 +294,7 @@ static void __init xen_init_capabilities(void)
 	setup_clear_cpu_cap(X86_FEATURE_MTRR);
 	setup_clear_cpu_cap(X86_FEATURE_ACC);
 	setup_clear_cpu_cap(X86_FEATURE_X2APIC);
+	setup_clear_cpu_cap(X86_FEATURE_SME);
 
 	if (!xen_initial_domain())
 		setup_clear_cpu_cap(X86_FEATURE_ACPI);



From thomas.lendacky at amd.com  Tue Jun 27 08:13:38 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Tue, 27 Jun 2017 10:13:38 -0500
Subject: [PATCH v8 RESEND 33/38] x86/mm: Use proper encryption attributes
 with /dev/mem
In-Reply-To: <20170627150718.17428.81813.stgit@tlendack-t1.amdoffice.net>
References: <20170627150718.17428.81813.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170627151338.17428.47553.stgit@tlendack-t1.amdoffice.net>

When accessing memory using /dev/mem (or /dev/kmem) use the proper
encryption attributes when mapping the memory.

To insure the proper attributes are applied when reading or writing
/dev/mem, update the xlate_dev_mem_ptr() function to use memremap()
which will essentially perform the same steps of applying __va for
RAM or using ioremap() if not RAM.

To insure the proper attributes are applied when mmapping /dev/mem,
update the phys_mem_access_prot() to call phys_mem_access_encrypted(),
a new function which will check if the memory should be mapped encrypted
or not. If it is not to be mapped encrypted then the VMA protection
value is updated to remove the encryption bit.

Reviewed-by: Borislav Petkov <bp at suse.de>
Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/include/asm/io.h |    3 +++
 arch/x86/mm/ioremap.c     |   18 +++++++++---------
 arch/x86/mm/pat.c         |    3 +++
 3 files changed, 15 insertions(+), 9 deletions(-)

diff --git a/arch/x86/include/asm/io.h b/arch/x86/include/asm/io.h
index 09c5557..e080a39 100644
--- a/arch/x86/include/asm/io.h
+++ b/arch/x86/include/asm/io.h
@@ -386,4 +386,7 @@ extern bool arch_memremap_can_ram_remap(resource_size_t offset,
 					unsigned long flags);
 #define arch_memremap_can_ram_remap arch_memremap_can_ram_remap
 
+extern bool phys_mem_access_encrypted(unsigned long phys_addr,
+				      unsigned long size);
+
 #endif /* _ASM_X86_IO_H */
diff --git a/arch/x86/mm/ioremap.c b/arch/x86/mm/ioremap.c
index effa529..71d4ca7 100644
--- a/arch/x86/mm/ioremap.c
+++ b/arch/x86/mm/ioremap.c
@@ -396,12 +396,10 @@ void *xlate_dev_mem_ptr(phys_addr_t phys)
 	unsigned long offset = phys & ~PAGE_MASK;
 	void *vaddr;
 
-	/* If page is RAM, we can use __va. Otherwise ioremap and unmap. */
-	if (page_is_ram(start >> PAGE_SHIFT))
-		return __va(phys);
+	/* memremap() maps if RAM, otherwise falls back to ioremap() */
+	vaddr = memremap(start, PAGE_SIZE, MEMREMAP_WB);
 
-	vaddr = ioremap_cache(start, PAGE_SIZE);
-	/* Only add the offset on success and return NULL if the ioremap() failed: */
+	/* Only add the offset on success and return NULL if memremap() failed */
 	if (vaddr)
 		vaddr += offset;
 
@@ -410,10 +408,7 @@ void *xlate_dev_mem_ptr(phys_addr_t phys)
 
 void unxlate_dev_mem_ptr(phys_addr_t phys, void *addr)
 {
-	if (page_is_ram(phys >> PAGE_SHIFT))
-		return;
-
-	iounmap((void __iomem *)((unsigned long)addr & PAGE_MASK));
+	memunmap((void *)((unsigned long)addr & PAGE_MASK));
 }
 
 /*
@@ -622,6 +617,11 @@ pgprot_t __init early_memremap_pgprot_adjust(resource_size_t phys_addr,
 	return prot;
 }
 
+bool phys_mem_access_encrypted(unsigned long phys_addr, unsigned long size)
+{
+	return arch_memremap_can_ram_remap(phys_addr, size, 0);
+}
+
 #ifdef CONFIG_ARCH_USE_MEMREMAP_PROT
 /* Remap memory with encryption */
 void __init *early_memremap_encrypted(resource_size_t phys_addr,
diff --git a/arch/x86/mm/pat.c b/arch/x86/mm/pat.c
index 6753d9c..b970c95 100644
--- a/arch/x86/mm/pat.c
+++ b/arch/x86/mm/pat.c
@@ -748,6 +748,9 @@ void arch_io_free_memtype_wc(resource_size_t start, resource_size_t size)
 pgprot_t phys_mem_access_prot(struct file *file, unsigned long pfn,
 				unsigned long size, pgprot_t vma_prot)
 {
+	if (!phys_mem_access_encrypted(pfn << PAGE_SHIFT, size))
+		vma_prot = pgprot_decrypted(vma_prot);
+
 	return vma_prot;
 }
 



From thomas.lendacky at amd.com  Tue Jun 27 08:13:48 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Tue, 27 Jun 2017 10:13:48 -0500
Subject: [PATCH v8 RESEND 34/38] x86/mm: Create native_make_p4d() for
 PGTABLE_LEVELS <= 4
In-Reply-To: <20170627150718.17428.81813.stgit@tlendack-t1.amdoffice.net>
References: <20170627150718.17428.81813.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170627151348.17428.43372.stgit@tlendack-t1.amdoffice.net>

Currently, native_make_p4d() is only defined when CONFIG_PGTABLE_LEVELS
is greater than 4. Create a macro that will allow for defining and using
native_make_p4d() when CONFIG_PGTABLES_LEVELS is not greater than 4.

Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/include/asm/pgtable_types.h |    5 +++++
 1 file changed, 5 insertions(+)

diff --git a/arch/x86/include/asm/pgtable_types.h b/arch/x86/include/asm/pgtable_types.h
index 830992f..6c55973 100644
--- a/arch/x86/include/asm/pgtable_types.h
+++ b/arch/x86/include/asm/pgtable_types.h
@@ -309,6 +309,11 @@ static inline p4dval_t native_p4d_val(p4d_t p4d)
 #else
 #include <asm-generic/pgtable-nop4d.h>
 
+static inline p4d_t native_make_p4d(pudval_t val)
+{
+	return (p4d_t) { .pgd = native_make_pgd((pgdval_t)val) };
+}
+
 static inline p4dval_t native_p4d_val(p4d_t p4d)
 {
 	return native_pgd_val(p4d.pgd);



From thomas.lendacky at amd.com  Tue Jun 27 08:14:16 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Tue, 27 Jun 2017 10:14:16 -0500
Subject: [PATCH v8 RESEND 35/38] x86/mm: Add support to encrypt the kernel
 in-place
In-Reply-To: <20170627150718.17428.81813.stgit@tlendack-t1.amdoffice.net>
References: <20170627150718.17428.81813.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170627151415.17428.76460.stgit@tlendack-t1.amdoffice.net>

Add the support to encrypt the kernel in-place. This is done by creating
new page mappings for the kernel - a decrypted write-protected mapping
and an encrypted mapping. The kernel is encrypted by copying it through
a temporary buffer.

Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/include/asm/mem_encrypt.h |    6 +
 arch/x86/mm/Makefile               |    1 
 arch/x86/mm/mem_encrypt.c          |  310 ++++++++++++++++++++++++++++++++++++
 arch/x86/mm/mem_encrypt_boot.S     |  150 +++++++++++++++++
 4 files changed, 467 insertions(+)
 create mode 100644 arch/x86/mm/mem_encrypt_boot.S

diff --git a/arch/x86/include/asm/mem_encrypt.h b/arch/x86/include/asm/mem_encrypt.h
index 70e55f6..7122c36 100644
--- a/arch/x86/include/asm/mem_encrypt.h
+++ b/arch/x86/include/asm/mem_encrypt.h
@@ -21,6 +21,12 @@
 
 extern unsigned long sme_me_mask;
 
+void sme_encrypt_execute(unsigned long encrypted_kernel_vaddr,
+			 unsigned long decrypted_kernel_vaddr,
+			 unsigned long kernel_len,
+			 unsigned long encryption_wa,
+			 unsigned long encryption_pgd);
+
 void __init sme_early_encrypt(resource_size_t paddr,
 			      unsigned long size);
 void __init sme_early_decrypt(resource_size_t paddr,
diff --git a/arch/x86/mm/Makefile b/arch/x86/mm/Makefile
index a94a7b6..72bf8c0 100644
--- a/arch/x86/mm/Makefile
+++ b/arch/x86/mm/Makefile
@@ -40,3 +40,4 @@ obj-$(CONFIG_X86_INTEL_MEMORY_PROTECTION_KEYS) += pkeys.o
 obj-$(CONFIG_RANDOMIZE_MEMORY) += kaslr.o
 
 obj-$(CONFIG_AMD_MEM_ENCRYPT)	+= mem_encrypt.o
+obj-$(CONFIG_AMD_MEM_ENCRYPT)	+= mem_encrypt_boot.o
diff --git a/arch/x86/mm/mem_encrypt.c b/arch/x86/mm/mem_encrypt.c
index a7400ec..e5d5439 100644
--- a/arch/x86/mm/mem_encrypt.c
+++ b/arch/x86/mm/mem_encrypt.c
@@ -21,6 +21,8 @@
 #include <asm/setup.h>
 #include <asm/bootparam.h>
 #include <asm/set_memory.h>
+#include <asm/cacheflush.h>
+#include <asm/sections.h>
 
 /*
  * Since SME related variables are set early in the boot process they must
@@ -199,8 +201,316 @@ void swiotlb_set_mem_attributes(void *vaddr, unsigned long size)
 	set_memory_decrypted((unsigned long)vaddr, size >> PAGE_SHIFT);
 }
 
+static void __init sme_clear_pgd(pgd_t *pgd_base, unsigned long start,
+				 unsigned long end)
+{
+	unsigned long pgd_start, pgd_end, pgd_size;
+	pgd_t *pgd_p;
+
+	pgd_start = start & PGDIR_MASK;
+	pgd_end = end & PGDIR_MASK;
+
+	pgd_size = (((pgd_end - pgd_start) / PGDIR_SIZE) + 1);
+	pgd_size *= sizeof(pgd_t);
+
+	pgd_p = pgd_base + pgd_index(start);
+
+	memset(pgd_p, 0, pgd_size);
+}
+
+#define PGD_FLAGS	_KERNPG_TABLE_NOENC
+#define P4D_FLAGS	_KERNPG_TABLE_NOENC
+#define PUD_FLAGS	_KERNPG_TABLE_NOENC
+#define PMD_FLAGS	(__PAGE_KERNEL_LARGE_EXEC & ~_PAGE_GLOBAL)
+
+static void __init *sme_populate_pgd(pgd_t *pgd_base, void *pgtable_area,
+				     unsigned long vaddr, pmdval_t pmd_val)
+{
+	pgd_t *pgd_p;
+	p4d_t *p4d_p;
+	pud_t *pud_p;
+	pmd_t *pmd_p;
+
+	pgd_p = pgd_base + pgd_index(vaddr);
+	if (native_pgd_val(*pgd_p)) {
+		if (IS_ENABLED(CONFIG_X86_5LEVEL))
+			p4d_p = (p4d_t *)(native_pgd_val(*pgd_p) & ~PTE_FLAGS_MASK);
+		else
+			pud_p = (pud_t *)(native_pgd_val(*pgd_p) & ~PTE_FLAGS_MASK);
+	} else {
+		pgd_t pgd;
+
+		if (IS_ENABLED(CONFIG_X86_5LEVEL)) {
+			p4d_p = pgtable_area;
+			memset(p4d_p, 0, sizeof(*p4d_p) * PTRS_PER_P4D);
+			pgtable_area += sizeof(*p4d_p) * PTRS_PER_P4D;
+
+			pgd = native_make_pgd((pgdval_t)p4d_p + PGD_FLAGS);
+		} else {
+			pud_p = pgtable_area;
+			memset(pud_p, 0, sizeof(*pud_p) * PTRS_PER_PUD);
+			pgtable_area += sizeof(*pud_p) * PTRS_PER_PUD;
+
+			pgd = native_make_pgd((pgdval_t)pud_p + PGD_FLAGS);
+		}
+		native_set_pgd(pgd_p, pgd);
+	}
+
+	if (IS_ENABLED(CONFIG_X86_5LEVEL)) {
+		p4d_p += p4d_index(vaddr);
+		if (native_p4d_val(*p4d_p)) {
+			pud_p = (pud_t *)(native_p4d_val(*p4d_p) & ~PTE_FLAGS_MASK);
+		} else {
+			p4d_t p4d;
+
+			pud_p = pgtable_area;
+			memset(pud_p, 0, sizeof(*pud_p) * PTRS_PER_PUD);
+			pgtable_area += sizeof(*pud_p) * PTRS_PER_PUD;
+
+			p4d = native_make_p4d((pudval_t)pud_p + P4D_FLAGS);
+			native_set_p4d(p4d_p, p4d);
+		}
+	}
+
+	pud_p += pud_index(vaddr);
+	if (native_pud_val(*pud_p)) {
+		if (native_pud_val(*pud_p) & _PAGE_PSE)
+			goto out;
+
+		pmd_p = (pmd_t *)(native_pud_val(*pud_p) & ~PTE_FLAGS_MASK);
+	} else {
+		pud_t pud;
+
+		pmd_p = pgtable_area;
+		memset(pmd_p, 0, sizeof(*pmd_p) * PTRS_PER_PMD);
+		pgtable_area += sizeof(*pmd_p) * PTRS_PER_PMD;
+
+		pud = native_make_pud((pmdval_t)pmd_p + PUD_FLAGS);
+		native_set_pud(pud_p, pud);
+	}
+
+	pmd_p += pmd_index(vaddr);
+	if (!native_pmd_val(*pmd_p) || !(native_pmd_val(*pmd_p) & _PAGE_PSE))
+		native_set_pmd(pmd_p, native_make_pmd(pmd_val));
+
+out:
+	return pgtable_area;
+}
+
+static unsigned long __init sme_pgtable_calc(unsigned long len)
+{
+	unsigned long p4d_size, pud_size, pmd_size;
+	unsigned long total;
+
+	/*
+	 * Perform a relatively simplistic calculation of the pagetable
+	 * entries that are needed. That mappings will be covered by 2MB
+	 * PMD entries so we can conservatively calculate the required
+	 * number of P4D, PUD and PMD structures needed to perform the
+	 * mappings. Incrementing the count for each covers the case where
+	 * the addresses cross entries.
+	 */
+	if (IS_ENABLED(CONFIG_X86_5LEVEL)) {
+		p4d_size = (ALIGN(len, PGDIR_SIZE) / PGDIR_SIZE) + 1;
+		p4d_size *= sizeof(p4d_t) * PTRS_PER_P4D;
+		pud_size = (ALIGN(len, P4D_SIZE) / P4D_SIZE) + 1;
+		pud_size *= sizeof(pud_t) * PTRS_PER_PUD;
+	} else {
+		p4d_size = 0;
+		pud_size = (ALIGN(len, PGDIR_SIZE) / PGDIR_SIZE) + 1;
+		pud_size *= sizeof(pud_t) * PTRS_PER_PUD;
+	}
+	pmd_size = (ALIGN(len, PUD_SIZE) / PUD_SIZE) + 1;
+	pmd_size *= sizeof(pmd_t) * PTRS_PER_PMD;
+
+	total = p4d_size + pud_size + pmd_size;
+
+	/*
+	 * Now calculate the added pagetable structures needed to populate
+	 * the new pagetables.
+	 */
+	if (IS_ENABLED(CONFIG_X86_5LEVEL)) {
+		p4d_size = ALIGN(total, PGDIR_SIZE) / PGDIR_SIZE;
+		p4d_size *= sizeof(p4d_t) * PTRS_PER_P4D;
+		pud_size = ALIGN(total, P4D_SIZE) / P4D_SIZE;
+		pud_size *= sizeof(pud_t) * PTRS_PER_PUD;
+	} else {
+		p4d_size = 0;
+		pud_size = ALIGN(total, PGDIR_SIZE) / PGDIR_SIZE;
+		pud_size *= sizeof(pud_t) * PTRS_PER_PUD;
+	}
+	pmd_size = ALIGN(total, PUD_SIZE) / PUD_SIZE;
+	pmd_size *= sizeof(pmd_t) * PTRS_PER_PMD;
+
+	total += p4d_size + pud_size + pmd_size;
+
+	return total;
+}
+
 void __init sme_encrypt_kernel(void)
 {
+	unsigned long workarea_start, workarea_end, workarea_len;
+	unsigned long execute_start, execute_end, execute_len;
+	unsigned long kernel_start, kernel_end, kernel_len;
+	unsigned long pgtable_area_len;
+	unsigned long paddr, pmd_flags;
+	unsigned long decrypted_base;
+	void *pgtable_area;
+	pgd_t *pgd;
+
+	if (!sme_active())
+		return;
+
+	/*
+	 * Prepare for encrypting the kernel by building new pagetables with
+	 * the necessary attributes needed to encrypt the kernel in place.
+	 *
+	 *   One range of virtual addresses will map the memory occupied
+	 *   by the kernel as encrypted.
+	 *
+	 *   Another range of virtual addresses will map the memory occupied
+	 *   by the kernel as decrypted and write-protected.
+	 *
+	 *     The use of write-protect attribute will prevent any of the
+	 *     memory from being cached.
+	 */
+
+	/* Physical addresses gives us the identity mapped virtual addresses */
+	kernel_start = __pa_symbol(_text);
+	kernel_end = ALIGN(__pa_symbol(_end), PMD_PAGE_SIZE);
+	kernel_len = kernel_end - kernel_start;
+
+	/* Set the encryption workarea to be immediately after the kernel */
+	workarea_start = kernel_end;
+
+	/*
+	 * Calculate required number of workarea bytes needed:
+	 *   executable encryption area size:
+	 *     stack page (PAGE_SIZE)
+	 *     encryption routine page (PAGE_SIZE)
+	 *     intermediate copy buffer (PMD_PAGE_SIZE)
+	 *   pagetable structures for the encryption of the kernel
+	 *   pagetable structures for workarea (in case not currently mapped)
+	 */
+	execute_start = workarea_start;
+	execute_end = execute_start + (PAGE_SIZE * 2) + PMD_PAGE_SIZE;
+	execute_len = execute_end - execute_start;
+
+	/*
+	 * One PGD for both encrypted and decrypted mappings and a set of
+	 * PUDs and PMDs for each of the encrypted and decrypted mappings.
+	 */
+	pgtable_area_len = sizeof(pgd_t) * PTRS_PER_PGD;
+	pgtable_area_len += sme_pgtable_calc(execute_end - kernel_start) * 2;
+
+	/* PUDs and PMDs needed in the current pagetables for the workarea */
+	pgtable_area_len += sme_pgtable_calc(execute_len + pgtable_area_len);
+
+	/*
+	 * The total workarea includes the executable encryption area and
+	 * the pagetable area.
+	 */
+	workarea_len = execute_len + pgtable_area_len;
+	workarea_end = workarea_start + workarea_len;
+
+	/*
+	 * Set the address to the start of where newly created pagetable
+	 * structures (PGDs, PUDs and PMDs) will be allocated. New pagetable
+	 * structures are created when the workarea is added to the current
+	 * pagetables and when the new encrypted and decrypted kernel
+	 * mappings are populated.
+	 */
+	pgtable_area = (void *)execute_end;
+
+	/*
+	 * Make sure the current pagetable structure has entries for
+	 * addressing the workarea.
+	 */
+	pgd = (pgd_t *)native_read_cr3_pa();
+	paddr = workarea_start;
+	while (paddr < workarea_end) {
+		pgtable_area = sme_populate_pgd(pgd, pgtable_area,
+						paddr,
+						paddr + PMD_FLAGS);
+
+		paddr += PMD_PAGE_SIZE;
+	}
+
+	/* Flush the TLB - no globals so cr3 is enough */
+	native_write_cr3(__native_read_cr3());
+
+	/*
+	 * A new pagetable structure is being built to allow for the kernel
+	 * to be encrypted. It starts with an empty PGD that will then be
+	 * populated with new PUDs and PMDs as the encrypted and decrypted
+	 * kernel mappings are created.
+	 */
+	pgd = pgtable_area;
+	memset(pgd, 0, sizeof(*pgd) * PTRS_PER_PGD);
+	pgtable_area += sizeof(*pgd) * PTRS_PER_PGD;
+
+	/* Add encrypted kernel (identity) mappings */
+	pmd_flags = PMD_FLAGS | _PAGE_ENC;
+	paddr = kernel_start;
+	while (paddr < kernel_end) {
+		pgtable_area = sme_populate_pgd(pgd, pgtable_area,
+						paddr,
+						paddr + pmd_flags);
+
+		paddr += PMD_PAGE_SIZE;
+	}
+
+	/*
+	 * A different PGD index/entry must be used to get different
+	 * pagetable entries for the decrypted mapping. Choose the next
+	 * PGD index and convert it to a virtual address to be used as
+	 * the base of the mapping.
+	 */
+	decrypted_base = (pgd_index(workarea_end) + 1) & (PTRS_PER_PGD - 1);
+	decrypted_base <<= PGDIR_SHIFT;
+
+	/* Add decrypted, write-protected kernel (non-identity) mappings */
+	pmd_flags = (PMD_FLAGS & ~_PAGE_CACHE_MASK) | (_PAGE_PAT | _PAGE_PWT);
+	paddr = kernel_start;
+	while (paddr < kernel_end) {
+		pgtable_area = sme_populate_pgd(pgd, pgtable_area,
+						paddr + decrypted_base,
+						paddr + pmd_flags);
+
+		paddr += PMD_PAGE_SIZE;
+	}
+
+	/* Add decrypted workarea mappings to both kernel mappings */
+	paddr = workarea_start;
+	while (paddr < workarea_end) {
+		pgtable_area = sme_populate_pgd(pgd, pgtable_area,
+						paddr,
+						paddr + PMD_FLAGS);
+
+		pgtable_area = sme_populate_pgd(pgd, pgtable_area,
+						paddr + decrypted_base,
+						paddr + PMD_FLAGS);
+
+		paddr += PMD_PAGE_SIZE;
+	}
+
+	/* Perform the encryption */
+	sme_encrypt_execute(kernel_start, kernel_start + decrypted_base,
+			    kernel_len, workarea_start, (unsigned long)pgd);
+
+	/*
+	 * At this point we are running encrypted.  Remove the mappings for
+	 * the decrypted areas - all that is needed for this is to remove
+	 * the PGD entry/entries.
+	 */
+	sme_clear_pgd(pgd, kernel_start + decrypted_base,
+		      kernel_end + decrypted_base);
+
+	sme_clear_pgd(pgd, workarea_start + decrypted_base,
+		      workarea_end + decrypted_base);
+
+	/* Flush the TLB - no globals so cr3 is enough */
+	native_write_cr3(__native_read_cr3());
 }
 
 void __init sme_enable(void)
diff --git a/arch/x86/mm/mem_encrypt_boot.S b/arch/x86/mm/mem_encrypt_boot.S
new file mode 100644
index 0000000..7720b00
--- /dev/null
+++ b/arch/x86/mm/mem_encrypt_boot.S
@@ -0,0 +1,150 @@
+/*
+ * AMD Memory Encryption Support
+ *
+ * Copyright (C) 2016 Advanced Micro Devices, Inc.
+ *
+ * Author: Tom Lendacky <thomas.lendacky at amd.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include <linux/linkage.h>
+#include <asm/pgtable.h>
+#include <asm/page.h>
+#include <asm/processor-flags.h>
+#include <asm/msr-index.h>
+
+	.text
+	.code64
+ENTRY(sme_encrypt_execute)
+
+	/*
+	 * Entry parameters:
+	 *   RDI - virtual address for the encrypted kernel mapping
+	 *   RSI - virtual address for the decrypted kernel mapping
+	 *   RDX - length of kernel
+	 *   RCX - virtual address of the encryption workarea, including:
+	 *     - stack page (PAGE_SIZE)
+	 *     - encryption routine page (PAGE_SIZE)
+	 *     - intermediate copy buffer (PMD_PAGE_SIZE)
+	 *    R8 - physcial address of the pagetables to use for encryption
+	 */
+
+	push	%rbp
+	push	%r12
+
+	/* Set up a one page stack in the non-encrypted memory area */
+	movq	%rsp, %rbp		/* Save current stack pointer */
+	movq	%rcx, %rax		/* Workarea stack page */
+	movq	%rax, %rsp		/* Set new stack pointer */
+	addq	$PAGE_SIZE, %rsp	/* Stack grows from the bottom */
+	addq	$PAGE_SIZE, %rax	/* Workarea encryption routine */
+
+	movq	%rdi, %r10		/* Encrypted kernel */
+	movq	%rsi, %r11		/* Decrypted kernel */
+	movq	%rdx, %r12		/* Kernel length */
+
+	/* Copy encryption routine into the workarea */
+	movq	%rax, %rdi				/* Workarea encryption routine */
+	leaq	__enc_copy(%rip), %rsi			/* Encryption routine */
+	movq	$(.L__enc_copy_end - __enc_copy), %rcx	/* Encryption routine length */
+	rep	movsb
+
+	/* Setup registers for call */
+	movq	%r10, %rdi		/* Encrypted kernel */
+	movq	%r11, %rsi		/* Decrypted kernel */
+	movq	%r8, %rdx		/* Pagetables used for encryption */
+	movq	%r12, %rcx		/* Kernel length */
+	movq	%rax, %r8		/* Workarea encryption routine */
+	addq	$PAGE_SIZE, %r8		/* Workarea intermediate copy buffer */
+
+	call	*%rax			/* Call the encryption routine */
+
+	movq	%rbp, %rsp		/* Restore original stack pointer */
+
+	pop	%r12
+	pop	%rbp
+
+	ret
+ENDPROC(sme_encrypt_execute)
+
+ENTRY(__enc_copy)
+/*
+ * Routine used to encrypt kernel.
+ *   This routine must be run outside of the kernel proper since
+ *   the kernel will be encrypted during the process. So this
+ *   routine is defined here and then copied to an area outside
+ *   of the kernel where it will remain and run decrypted
+ *   during execution.
+ *
+ *   On entry the registers must be:
+ *     RDI - virtual address for the encrypted kernel mapping
+ *     RSI - virtual address for the decrypted kernel mapping
+ *     RDX - address of the pagetables to use for encryption
+ *     RCX - length of kernel
+ *      R8 - intermediate copy buffer
+ *
+ *     RAX - points to this routine
+ *
+ * The kernel will be encrypted by copying from the non-encrypted
+ * kernel space to an intermediate buffer and then copying from the
+ * intermediate buffer back to the encrypted kernel space. The physical
+ * addresses of the two kernel space mappings are the same which
+ * results in the kernel being encrypted "in place".
+ */
+	/* Enable the new page tables */
+	mov	%rdx, %cr3
+
+	/* Flush any global TLBs */
+	mov	%cr4, %rdx
+	andq	$~X86_CR4_PGE, %rdx
+	mov	%rdx, %cr4
+	orq	$X86_CR4_PGE, %rdx
+	mov	%rdx, %cr4
+
+	/* Set the PAT register PA5 entry to write-protect */
+	push	%rcx
+	movl	$MSR_IA32_CR_PAT, %ecx
+	rdmsr
+	push	%rdx			/* Save original PAT value */
+	andl	$0xffff00ff, %edx	/* Clear PA5 */
+	orl	$0x00000500, %edx	/* Set PA5 to WP */
+	wrmsr
+	pop	%rdx			/* RDX contains original PAT value */
+	pop	%rcx
+
+	movq	%rcx, %r9		/* Save kernel length */
+	movq	%rdi, %r10		/* Save encrypted kernel address */
+	movq	%rsi, %r11		/* Save decrypted kernel address */
+
+	wbinvd				/* Invalidate any cache entries */
+
+	/* Copy/encrypt 2MB at a time */
+1:
+	movq	%r11, %rsi		/* Source - decrypted kernel */
+	movq	%r8, %rdi		/* Dest   - intermediate copy buffer */
+	movq	$PMD_PAGE_SIZE, %rcx	/* 2MB length */
+	rep	movsb
+
+	movq	%r8, %rsi		/* Source - intermediate copy buffer */
+	movq	%r10, %rdi		/* Dest   - encrypted kernel */
+	movq	$PMD_PAGE_SIZE, %rcx	/* 2MB length */
+	rep	movsb
+
+	addq	$PMD_PAGE_SIZE, %r11
+	addq	$PMD_PAGE_SIZE, %r10
+	subq	$PMD_PAGE_SIZE, %r9	/* Kernel length decrement */
+	jnz	1b			/* Kernel length not zero? */
+
+	/* Restore PAT register */
+	push	%rdx			/* Save original PAT value */
+	movl	$MSR_IA32_CR_PAT, %ecx
+	rdmsr
+	pop	%rdx			/* Restore original PAT value */
+	wrmsr
+
+	ret
+.L__enc_copy_end:
+ENDPROC(__enc_copy)



From thomas.lendacky at amd.com  Tue Jun 27 08:14:24 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Tue, 27 Jun 2017 10:14:24 -0500
Subject: [PATCH v8 RESEND 36/38] x86/boot: Add early cmdline parsing for
 options with arguments
In-Reply-To: <20170627150718.17428.81813.stgit@tlendack-t1.amdoffice.net>
References: <20170627150718.17428.81813.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170627151424.17428.16088.stgit@tlendack-t1.amdoffice.net>

Add a cmdline_find_option() function to look for cmdline options that
take arguments. The argument is returned in a supplied buffer and the
argument length (regardless of whether it fits in the supplied buffer)
is returned, with -1 indicating not found.

Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/include/asm/cmdline.h |    2 +
 arch/x86/lib/cmdline.c         |  105 ++++++++++++++++++++++++++++++++++++++++
 2 files changed, 107 insertions(+)

diff --git a/arch/x86/include/asm/cmdline.h b/arch/x86/include/asm/cmdline.h
index e01f7f7..84ae170 100644
--- a/arch/x86/include/asm/cmdline.h
+++ b/arch/x86/include/asm/cmdline.h
@@ -2,5 +2,7 @@
 #define _ASM_X86_CMDLINE_H
 
 int cmdline_find_option_bool(const char *cmdline_ptr, const char *option);
+int cmdline_find_option(const char *cmdline_ptr, const char *option,
+			char *buffer, int bufsize);
 
 #endif /* _ASM_X86_CMDLINE_H */
diff --git a/arch/x86/lib/cmdline.c b/arch/x86/lib/cmdline.c
index 5cc78bf..3261abb 100644
--- a/arch/x86/lib/cmdline.c
+++ b/arch/x86/lib/cmdline.c
@@ -104,7 +104,112 @@ static inline int myisspace(u8 c)
 	return 0;	/* Buffer overrun */
 }
 
+/*
+ * Find a non-boolean option (i.e. option=argument). In accordance with
+ * standard Linux practice, if this option is repeated, this returns the
+ * last instance on the command line.
+ *
+ * @cmdline: the cmdline string
+ * @max_cmdline_size: the maximum size of cmdline
+ * @option: option string to look for
+ * @buffer: memory buffer to return the option argument
+ * @bufsize: size of the supplied memory buffer
+ *
+ * Returns the length of the argument (regardless of if it was
+ * truncated to fit in the buffer), or -1 on not found.
+ */
+static int
+__cmdline_find_option(const char *cmdline, int max_cmdline_size,
+		      const char *option, char *buffer, int bufsize)
+{
+	char c;
+	int pos = 0, len = -1;
+	const char *opptr = NULL;
+	char *bufptr = buffer;
+	enum {
+		st_wordstart = 0,	/* Start of word/after whitespace */
+		st_wordcmp,	/* Comparing this word */
+		st_wordskip,	/* Miscompare, skip */
+		st_bufcpy,	/* Copying this to buffer */
+	} state = st_wordstart;
+
+	if (!cmdline)
+		return -1;      /* No command line */
+
+	/*
+	 * This 'pos' check ensures we do not overrun
+	 * a non-NULL-terminated 'cmdline'
+	 */
+	while (pos++ < max_cmdline_size) {
+		c = *(char *)cmdline++;
+		if (!c)
+			break;
+
+		switch (state) {
+		case st_wordstart:
+			if (myisspace(c))
+				break;
+
+			state = st_wordcmp;
+			opptr = option;
+			/* fall through */
+
+		case st_wordcmp:
+			if ((c == '=') && !*opptr) {
+				/*
+				 * We matched all the way to the end of the
+				 * option we were looking for, prepare to
+				 * copy the argument.
+				 */
+				len = 0;
+				bufptr = buffer;
+				state = st_bufcpy;
+				break;
+			} else if (c == *opptr++) {
+				/*
+				 * We are currently matching, so continue
+				 * to the next character on the cmdline.
+				 */
+				break;
+			}
+			state = st_wordskip;
+			/* fall through */
+
+		case st_wordskip:
+			if (myisspace(c))
+				state = st_wordstart;
+			break;
+
+		case st_bufcpy:
+			if (myisspace(c)) {
+				state = st_wordstart;
+			} else {
+				/*
+				 * Increment len, but don't overrun the
+				 * supplied buffer and leave room for the
+				 * NULL terminator.
+				 */
+				if (++len < bufsize)
+					*bufptr++ = c;
+			}
+			break;
+		}
+	}
+
+	if (bufsize)
+		*bufptr = '\0';
+
+	return len;
+}
+
 int cmdline_find_option_bool(const char *cmdline, const char *option)
 {
 	return __cmdline_find_option_bool(cmdline, COMMAND_LINE_SIZE, option);
 }
+
+int cmdline_find_option(const char *cmdline, const char *option, char *buffer,
+			int bufsize)
+{
+	return __cmdline_find_option(cmdline, COMMAND_LINE_SIZE, option,
+				     buffer, bufsize);
+}



From thomas.lendacky at amd.com  Tue Jun 27 08:14:38 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Tue, 27 Jun 2017 10:14:38 -0500
Subject: [PATCH v8 RESEND 37/38] compiler-gcc.h: Introduce __nostackp
 function attribute
In-Reply-To: <20170627150718.17428.81813.stgit@tlendack-t1.amdoffice.net>
References: <20170627150718.17428.81813.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170627151438.17428.63531.stgit@tlendack-t1.amdoffice.net>

Create a new function attribute, __nostackp, that can used to turn off
stack protection on a per function basis.

Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 include/linux/compiler-gcc.h |    2 ++
 include/linux/compiler.h     |    4 ++++
 2 files changed, 6 insertions(+)

diff --git a/include/linux/compiler-gcc.h b/include/linux/compiler-gcc.h
index 0efef9c..de48b32 100644
--- a/include/linux/compiler-gcc.h
+++ b/include/linux/compiler-gcc.h
@@ -162,6 +162,8 @@
 
 #if GCC_VERSION >= 40100
 # define __compiletime_object_size(obj) __builtin_object_size(obj, 0)
+
+#define __nostackp	__attribute__((__optimize__("no-stack-protector")))
 #endif
 
 #if GCC_VERSION >= 40300
diff --git a/include/linux/compiler.h b/include/linux/compiler.h
index 707242f..615d50d 100644
--- a/include/linux/compiler.h
+++ b/include/linux/compiler.h
@@ -458,6 +458,10 @@ static __always_inline void __write_once_size(volatile void *p, void *res, int s
 #define __visible
 #endif
 
+#ifndef __nostackp
+#define __nostackp
+#endif
+
 /*
  * Assume alignment of return value.
  */



From thomas.lendacky at amd.com  Tue Jun 27 08:14:48 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Tue, 27 Jun 2017 10:14:48 -0500
Subject: [PATCH v8 RESEND 38/38] x86/mm: Add support to make use of Secure
 Memory Encryption
In-Reply-To: <20170627150718.17428.81813.stgit@tlendack-t1.amdoffice.net>
References: <20170627150718.17428.81813.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170627151447.17428.56826.stgit@tlendack-t1.amdoffice.net>

Add support to check if SME has been enabled and if memory encryption
should be activated (checking of command line option based on the
configuration of the default state).  If memory encryption is to be
activated, then the encryption mask is set and the kernel is encrypted
"in place."

Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/include/asm/mem_encrypt.h |    6 ++-
 arch/x86/kernel/head64.c           |    5 +-
 arch/x86/mm/mem_encrypt.c          |   77 ++++++++++++++++++++++++++++++++++++
 3 files changed, 83 insertions(+), 5 deletions(-)

diff --git a/arch/x86/include/asm/mem_encrypt.h b/arch/x86/include/asm/mem_encrypt.h
index 7122c36..8e618fc 100644
--- a/arch/x86/include/asm/mem_encrypt.h
+++ b/arch/x86/include/asm/mem_encrypt.h
@@ -17,6 +17,8 @@
 
 #include <linux/init.h>
 
+#include <asm/bootparam.h>
+
 #ifdef CONFIG_AMD_MEM_ENCRYPT
 
 extern unsigned long sme_me_mask;
@@ -38,7 +40,7 @@ void __init sme_early_decrypt(resource_size_t paddr,
 void __init sme_early_init(void);
 
 void __init sme_encrypt_kernel(void);
-void __init sme_enable(void);
+void __init sme_enable(struct boot_params *bp);
 
 /* Architecture __weak replacement functions */
 void __init mem_encrypt_init(void);
@@ -60,7 +62,7 @@ static inline void __init sme_unmap_bootdata(char *real_mode_data) { }
 static inline void __init sme_early_init(void) { }
 
 static inline void __init sme_encrypt_kernel(void) { }
-static inline void __init sme_enable(void) { }
+static inline void __init sme_enable(struct boot_params *bp) { }
 
 #endif	/* CONFIG_AMD_MEM_ENCRYPT */
 
diff --git a/arch/x86/kernel/head64.c b/arch/x86/kernel/head64.c
index 0cdb53b..925b292 100644
--- a/arch/x86/kernel/head64.c
+++ b/arch/x86/kernel/head64.c
@@ -45,7 +45,8 @@ static void __head *fixup_pointer(void *ptr, unsigned long physaddr)
 	return ptr - (void *)_text + (void *)physaddr;
 }
 
-unsigned long __head __startup_64(unsigned long physaddr)
+unsigned long __head __startup_64(unsigned long physaddr,
+				  struct boot_params *bp)
 {
 	unsigned long load_delta, *p;
 	unsigned long pgtable_flags;
@@ -70,7 +71,7 @@ unsigned long __head __startup_64(unsigned long physaddr)
 		for (;;);
 
 	/* Activate Secure Memory Encryption (SME) if supported and enabled */
-	sme_enable();
+	sme_enable(bp);
 
 	/* Include the SME encryption mask in the fixup value */
 	load_delta += sme_get_me_mask();
diff --git a/arch/x86/mm/mem_encrypt.c b/arch/x86/mm/mem_encrypt.c
index e5d5439..053d540 100644
--- a/arch/x86/mm/mem_encrypt.c
+++ b/arch/x86/mm/mem_encrypt.c
@@ -15,6 +15,7 @@
 #include <linux/mm.h>
 #include <linux/dma-mapping.h>
 #include <linux/swiotlb.h>
+#include <linux/mem_encrypt.h>
 
 #include <asm/tlbflush.h>
 #include <asm/fixmap.h>
@@ -23,6 +24,13 @@
 #include <asm/set_memory.h>
 #include <asm/cacheflush.h>
 #include <asm/sections.h>
+#include <asm/processor-flags.h>
+#include <asm/msr.h>
+#include <asm/cmdline.h>
+
+static char sme_cmdline_arg[] __initdata = "mem_encrypt";
+static char sme_cmdline_on[]  __initdata = "on";
+static char sme_cmdline_off[] __initdata = "off";
 
 /*
  * Since SME related variables are set early in the boot process they must
@@ -190,6 +198,8 @@ void __init mem_encrypt_init(void)
 
 	/* Call into SWIOTLB to update the SWIOTLB DMA buffers */
 	swiotlb_update_mem_attributes();
+
+	pr_info("AMD Secure Memory Encryption (SME) active\n");
 }
 
 void swiotlb_set_mem_attributes(void *vaddr, unsigned long size)
@@ -513,6 +523,71 @@ void __init sme_encrypt_kernel(void)
 	native_write_cr3(__native_read_cr3());
 }
 
-void __init sme_enable(void)
+void __init __nostackp sme_enable(struct boot_params *bp)
 {
+	const char *cmdline_ptr, *cmdline_arg, *cmdline_on, *cmdline_off;
+	unsigned int eax, ebx, ecx, edx;
+	bool active_by_default;
+	unsigned long me_mask;
+	char buffer[16];
+	u64 msr;
+
+	/* Check for the SME support leaf */
+	eax = 0x80000000;
+	ecx = 0;
+	native_cpuid(&eax, &ebx, &ecx, &edx);
+	if (eax < 0x8000001f)
+		return;
+
+	/*
+	 * Check for the SME feature:
+	 *   CPUID Fn8000_001F[EAX] - Bit 0
+	 *     Secure Memory Encryption support
+	 *   CPUID Fn8000_001F[EBX] - Bits 5:0
+	 *     Pagetable bit position used to indicate encryption
+	 */
+	eax = 0x8000001f;
+	ecx = 0;
+	native_cpuid(&eax, &ebx, &ecx, &edx);
+	if (!(eax & 1))
+		return;
+
+	me_mask = 1UL << (ebx & 0x3f);
+
+	/* Check if SME is enabled */
+	msr = __rdmsr(MSR_K8_SYSCFG);
+	if (!(msr & MSR_K8_SYSCFG_MEM_ENCRYPT))
+		return;
+
+	/*
+	 * Fixups have not been applied to phys_base yet and we're running
+	 * identity mapped, so we must obtain the address to the SME command
+	 * line argument data using rip-relative addressing.
+	 */
+	asm ("lea sme_cmdline_arg(%%rip), %0"
+	     : "=r" (cmdline_arg)
+	     : "p" (sme_cmdline_arg));
+	asm ("lea sme_cmdline_on(%%rip), %0"
+	     : "=r" (cmdline_on)
+	     : "p" (sme_cmdline_on));
+	asm ("lea sme_cmdline_off(%%rip), %0"
+	     : "=r" (cmdline_off)
+	     : "p" (sme_cmdline_off));
+
+	if (IS_ENABLED(CONFIG_AMD_MEM_ENCRYPT_ACTIVE_BY_DEFAULT))
+		active_by_default = true;
+	else
+		active_by_default = false;
+
+	cmdline_ptr = (const char *)((u64)bp->hdr.cmd_line_ptr |
+				     ((u64)bp->ext_cmd_line_ptr << 32));
+
+	cmdline_find_option(cmdline_ptr, cmdline_arg, buffer, sizeof(buffer));
+
+	if (!strncmp(buffer, cmdline_on, sizeof(buffer)))
+		sme_me_mask = me_mask;
+	else if (!strncmp(buffer, cmdline_off, sizeof(buffer)))
+		sme_me_mask = 0;
+	else
+		sme_me_mask = active_by_default ? me_mask : 0;
 }



From thomas.lendacky at amd.com  Tue Jun 27 08:12:30 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Tue, 27 Jun 2017 10:12:30 -0500
Subject: [PATCH v8 RESEND 27/38] iommu/amd: Allow the AMD IOMMU to work with
 memory encryption
In-Reply-To: <20170627150718.17428.81813.stgit@tlendack-t1.amdoffice.net>
References: <20170627150718.17428.81813.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170627151230.17428.75281.stgit@tlendack-t1.amdoffice.net>

The IOMMU is programmed with physical addresses for the various tables
and buffers that are used to communicate between the device and the
driver. When the driver allocates this memory it is encrypted. In order
for the IOMMU to access the memory as encrypted the encryption mask needs
to be included in these physical addresses during configuration.

The PTE entries created by the IOMMU should also include the encryption
mask so that when the device behind the IOMMU performs a DMA, the DMA
will be performed to encrypted memory.

Reviewed-by: Borislav Petkov <bp at suse.de>
Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 drivers/iommu/amd_iommu.c       |   30 ++++++++++++++++--------------
 drivers/iommu/amd_iommu_init.c  |   34 ++++++++++++++++++++++++++++------
 drivers/iommu/amd_iommu_proto.h |   10 ++++++++++
 drivers/iommu/amd_iommu_types.h |    2 +-
 4 files changed, 55 insertions(+), 21 deletions(-)

diff --git a/drivers/iommu/amd_iommu.c b/drivers/iommu/amd_iommu.c
index 503849d..16cc54b 100644
--- a/drivers/iommu/amd_iommu.c
+++ b/drivers/iommu/amd_iommu.c
@@ -544,7 +544,7 @@ static void dump_dte_entry(u16 devid)
 
 static void dump_command(unsigned long phys_addr)
 {
-	struct iommu_cmd *cmd = phys_to_virt(phys_addr);
+	struct iommu_cmd *cmd = iommu_phys_to_virt(phys_addr);
 	int i;
 
 	for (i = 0; i < 4; ++i)
@@ -865,11 +865,13 @@ static void copy_cmd_to_buffer(struct amd_iommu *iommu,
 
 static void build_completion_wait(struct iommu_cmd *cmd, u64 address)
 {
+	u64 paddr = iommu_virt_to_phys((void *)address);
+
 	WARN_ON(address & 0x7ULL);
 
 	memset(cmd, 0, sizeof(*cmd));
-	cmd->data[0] = lower_32_bits(__pa(address)) | CMD_COMPL_WAIT_STORE_MASK;
-	cmd->data[1] = upper_32_bits(__pa(address));
+	cmd->data[0] = lower_32_bits(paddr) | CMD_COMPL_WAIT_STORE_MASK;
+	cmd->data[1] = upper_32_bits(paddr);
 	cmd->data[2] = 1;
 	CMD_SET_TYPE(cmd, CMD_COMPL_WAIT);
 }
@@ -1328,7 +1330,7 @@ static bool increase_address_space(struct protection_domain *domain,
 		return false;
 
 	*pte             = PM_LEVEL_PDE(domain->mode,
-					virt_to_phys(domain->pt_root));
+					iommu_virt_to_phys(domain->pt_root));
 	domain->pt_root  = pte;
 	domain->mode    += 1;
 	domain->updated  = true;
@@ -1365,7 +1367,7 @@ static u64 *alloc_pte(struct protection_domain *domain,
 			if (!page)
 				return NULL;
 
-			__npte = PM_LEVEL_PDE(level, virt_to_phys(page));
+			__npte = PM_LEVEL_PDE(level, iommu_virt_to_phys(page));
 
 			/* pte could have been changed somewhere. */
 			if (cmpxchg64(pte, __pte, __npte) != __pte) {
@@ -1481,10 +1483,10 @@ static int iommu_map_page(struct protection_domain *dom,
 			return -EBUSY;
 
 	if (count > 1) {
-		__pte = PAGE_SIZE_PTE(phys_addr, page_size);
+		__pte = PAGE_SIZE_PTE(__sme_set(phys_addr), page_size);
 		__pte |= PM_LEVEL_ENC(7) | IOMMU_PTE_P | IOMMU_PTE_FC;
 	} else
-		__pte = phys_addr | IOMMU_PTE_P | IOMMU_PTE_FC;
+		__pte = __sme_set(phys_addr) | IOMMU_PTE_P | IOMMU_PTE_FC;
 
 	if (prot & IOMMU_PROT_IR)
 		__pte |= IOMMU_PTE_IR;
@@ -1700,7 +1702,7 @@ static void free_gcr3_tbl_level1(u64 *tbl)
 		if (!(tbl[i] & GCR3_VALID))
 			continue;
 
-		ptr = __va(tbl[i] & PAGE_MASK);
+		ptr = iommu_phys_to_virt(tbl[i] & PAGE_MASK);
 
 		free_page((unsigned long)ptr);
 	}
@@ -1715,7 +1717,7 @@ static void free_gcr3_tbl_level2(u64 *tbl)
 		if (!(tbl[i] & GCR3_VALID))
 			continue;
 
-		ptr = __va(tbl[i] & PAGE_MASK);
+		ptr = iommu_phys_to_virt(tbl[i] & PAGE_MASK);
 
 		free_gcr3_tbl_level1(ptr);
 	}
@@ -1807,7 +1809,7 @@ static void set_dte_entry(u16 devid, struct protection_domain *domain, bool ats)
 	u64 flags = 0;
 
 	if (domain->mode != PAGE_MODE_NONE)
-		pte_root = virt_to_phys(domain->pt_root);
+		pte_root = iommu_virt_to_phys(domain->pt_root);
 
 	pte_root |= (domain->mode & DEV_ENTRY_MODE_MASK)
 		    << DEV_ENTRY_MODE_SHIFT;
@@ -1819,7 +1821,7 @@ static void set_dte_entry(u16 devid, struct protection_domain *domain, bool ats)
 		flags |= DTE_FLAG_IOTLB;
 
 	if (domain->flags & PD_IOMMUV2_MASK) {
-		u64 gcr3 = __pa(domain->gcr3_tbl);
+		u64 gcr3 = iommu_virt_to_phys(domain->gcr3_tbl);
 		u64 glx  = domain->glx;
 		u64 tmp;
 
@@ -3470,10 +3472,10 @@ static u64 *__get_gcr3_pte(u64 *root, int level, int pasid, bool alloc)
 			if (root == NULL)
 				return NULL;
 
-			*pte = __pa(root) | GCR3_VALID;
+			*pte = iommu_virt_to_phys(root) | GCR3_VALID;
 		}
 
-		root = __va(*pte & PAGE_MASK);
+		root = iommu_phys_to_virt(*pte & PAGE_MASK);
 
 		level -= 1;
 	}
@@ -3652,7 +3654,7 @@ static void set_dte_irq_entry(u16 devid, struct irq_remap_table *table)
 
 	dte	= amd_iommu_dev_table[devid].data[2];
 	dte	&= ~DTE_IRQ_PHYS_ADDR_MASK;
-	dte	|= virt_to_phys(table->table);
+	dte	|= iommu_virt_to_phys(table->table);
 	dte	|= DTE_IRQ_REMAP_INTCTL;
 	dte	|= DTE_IRQ_TABLE_LEN;
 	dte	|= DTE_IRQ_REMAP_ENABLE;
diff --git a/drivers/iommu/amd_iommu_init.c b/drivers/iommu/amd_iommu_init.c
index 5a11328..26862f1 100644
--- a/drivers/iommu/amd_iommu_init.c
+++ b/drivers/iommu/amd_iommu_init.c
@@ -29,6 +29,7 @@
 #include <linux/export.h>
 #include <linux/iommu.h>
 #include <linux/kmemleak.h>
+#include <linux/mem_encrypt.h>
 #include <asm/pci-direct.h>
 #include <asm/iommu.h>
 #include <asm/gart.h>
@@ -346,7 +347,7 @@ static void iommu_set_device_table(struct amd_iommu *iommu)
 
 	BUG_ON(iommu->mmio_base == NULL);
 
-	entry = virt_to_phys(amd_iommu_dev_table);
+	entry = iommu_virt_to_phys(amd_iommu_dev_table);
 	entry |= (dev_table_size >> 12) - 1;
 	memcpy_toio(iommu->mmio_base + MMIO_DEV_TABLE_OFFSET,
 			&entry, sizeof(entry));
@@ -602,7 +603,7 @@ static void iommu_enable_command_buffer(struct amd_iommu *iommu)
 
 	BUG_ON(iommu->cmd_buf == NULL);
 
-	entry = (u64)virt_to_phys(iommu->cmd_buf);
+	entry = iommu_virt_to_phys(iommu->cmd_buf);
 	entry |= MMIO_CMD_SIZE_512;
 
 	memcpy_toio(iommu->mmio_base + MMIO_CMD_BUF_OFFSET,
@@ -631,7 +632,7 @@ static void iommu_enable_event_buffer(struct amd_iommu *iommu)
 
 	BUG_ON(iommu->evt_buf == NULL);
 
-	entry = (u64)virt_to_phys(iommu->evt_buf) | EVT_LEN_MASK;
+	entry = iommu_virt_to_phys(iommu->evt_buf) | EVT_LEN_MASK;
 
 	memcpy_toio(iommu->mmio_base + MMIO_EVT_BUF_OFFSET,
 		    &entry, sizeof(entry));
@@ -664,7 +665,7 @@ static void iommu_enable_ppr_log(struct amd_iommu *iommu)
 	if (iommu->ppr_log == NULL)
 		return;
 
-	entry = (u64)virt_to_phys(iommu->ppr_log) | PPR_LOG_SIZE_512;
+	entry = iommu_virt_to_phys(iommu->ppr_log) | PPR_LOG_SIZE_512;
 
 	memcpy_toio(iommu->mmio_base + MMIO_PPR_LOG_OFFSET,
 		    &entry, sizeof(entry));
@@ -744,10 +745,10 @@ static int iommu_init_ga_log(struct amd_iommu *iommu)
 	if (!iommu->ga_log_tail)
 		goto err_out;
 
-	entry = (u64)virt_to_phys(iommu->ga_log) | GA_LOG_SIZE_512;
+	entry = iommu_virt_to_phys(iommu->ga_log) | GA_LOG_SIZE_512;
 	memcpy_toio(iommu->mmio_base + MMIO_GA_LOG_BASE_OFFSET,
 		    &entry, sizeof(entry));
-	entry = ((u64)virt_to_phys(iommu->ga_log) & 0xFFFFFFFFFFFFFULL) & ~7ULL;
+	entry = (iommu_virt_to_phys(iommu->ga_log) & 0xFFFFFFFFFFFFFULL) & ~7ULL;
 	memcpy_toio(iommu->mmio_base + MMIO_GA_LOG_TAIL_OFFSET,
 		    &entry, sizeof(entry));
 	writel(0x00, iommu->mmio_base + MMIO_GA_HEAD_OFFSET);
@@ -2535,6 +2536,24 @@ static int __init amd_iommu_init(void)
 	return ret;
 }
 
+static bool amd_iommu_supports_sme(void)
+{
+	if (!sme_active() || (boot_cpu_data.x86 != 0x17))
+		return true;
+
+	/* For Fam17h, a specific level of support is required */
+	if (boot_cpu_data.microcode >= 0x08001205)
+		return true;
+
+	if ((boot_cpu_data.microcode >= 0x08001126) &&
+	    (boot_cpu_data.microcode <= 0x080011ff))
+		return true;
+
+	pr_notice("AMD-Vi: IOMMU not currently supported when SME is active\n");
+
+	return false;
+}
+
 /****************************************************************************
  *
  * Early detect code. This code runs at IOMMU detection time in the DMA
@@ -2552,6 +2571,9 @@ int __init amd_iommu_detect(void)
 	if (amd_iommu_disabled)
 		return -ENODEV;
 
+	if (!amd_iommu_supports_sme())
+		return -ENODEV;
+
 	ret = iommu_go_to_state(IOMMU_IVRS_DETECTED);
 	if (ret)
 		return ret;
diff --git a/drivers/iommu/amd_iommu_proto.h b/drivers/iommu/amd_iommu_proto.h
index 466260f..3f12fb2 100644
--- a/drivers/iommu/amd_iommu_proto.h
+++ b/drivers/iommu/amd_iommu_proto.h
@@ -87,4 +87,14 @@ static inline bool iommu_feature(struct amd_iommu *iommu, u64 f)
 	return !!(iommu->features & f);
 }
 
+static inline u64 iommu_virt_to_phys(void *vaddr)
+{
+	return (u64)__sme_set(virt_to_phys(vaddr));
+}
+
+static inline void *iommu_phys_to_virt(unsigned long paddr)
+{
+	return phys_to_virt(__sme_clr(paddr));
+}
+
 #endif /* _ASM_X86_AMD_IOMMU_PROTO_H  */
diff --git a/drivers/iommu/amd_iommu_types.h b/drivers/iommu/amd_iommu_types.h
index 4de8f41..3ce587d 100644
--- a/drivers/iommu/amd_iommu_types.h
+++ b/drivers/iommu/amd_iommu_types.h
@@ -343,7 +343,7 @@
 
 #define IOMMU_PAGE_MASK (((1ULL << 52) - 1) & ~0xfffULL)
 #define IOMMU_PTE_PRESENT(pte) ((pte) & IOMMU_PTE_P)
-#define IOMMU_PTE_PAGE(pte) (phys_to_virt((pte) & IOMMU_PAGE_MASK))
+#define IOMMU_PTE_PAGE(pte) (iommu_phys_to_virt((pte) & IOMMU_PAGE_MASK))
 #define IOMMU_PTE_MODE(pte) (((pte) >> 9) & 0x07)
 
 #define IOMMU_PROT_MASK 0x03



From thomas.lendacky at amd.com  Tue Jun 27 08:12:18 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Tue, 27 Jun 2017 10:12:18 -0500
Subject: [PATCH v8 RESEND 26/38] x86/CPU/AMD: Make the microcode level
 available earlier in the boot
In-Reply-To: <20170627150718.17428.81813.stgit@tlendack-t1.amdoffice.net>
References: <20170627150718.17428.81813.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170627151218.17428.25499.stgit@tlendack-t1.amdoffice.net>

Move the setting of the cpuinfo_x86.microcode field from amd_init() to
early_amd_init() so that it is available earlier in the boot process. This
avoids having to read MSR_AMD64_PATCH_LEVEL directly during early boot.

Reviewed-by: Borislav Petkov <bp at suse.de>
Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/kernel/cpu/amd.c |    8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/arch/x86/kernel/cpu/amd.c b/arch/x86/kernel/cpu/amd.c
index 5bdcbd4..fdcf305 100644
--- a/arch/x86/kernel/cpu/amd.c
+++ b/arch/x86/kernel/cpu/amd.c
@@ -547,8 +547,12 @@ static void bsp_init_amd(struct cpuinfo_x86 *c)
 
 static void early_init_amd(struct cpuinfo_x86 *c)
 {
+	u32 dummy;
+
 	early_init_amd_mc(c);
 
+	rdmsr_safe(MSR_AMD64_PATCH_LEVEL, &c->microcode, &dummy);
+
 	/*
 	 * c->x86_power is 8000_0007 edx. Bit 8 is TSC runs at constant rate
 	 * with P/T states and does not stop in deep C-states
@@ -746,8 +750,6 @@ static void init_amd_bd(struct cpuinfo_x86 *c)
 
 static void init_amd(struct cpuinfo_x86 *c)
 {
-	u32 dummy;
-
 	early_init_amd(c);
 
 	/*
@@ -809,8 +811,6 @@ static void init_amd(struct cpuinfo_x86 *c)
 	if (c->x86 > 0x11)
 		set_cpu_cap(c, X86_FEATURE_ARAT);
 
-	rdmsr_safe(MSR_AMD64_PATCH_LEVEL, &c->microcode, &dummy);
-
 	/* 3DNow or LM implies PREFETCHW */
 	if (!cpu_has(c, X86_FEATURE_3DNOWPREFETCH))
 		if (cpu_has(c, X86_FEATURE_3DNOW) || cpu_has(c, X86_FEATURE_LM))



From thomas.lendacky at amd.com  Tue Jun 27 08:10:17 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Tue, 27 Jun 2017 10:10:17 -0500
Subject: [PATCH v8 RESEND 16/38] efi: Add an EFI table address match function
In-Reply-To: <20170627150718.17428.81813.stgit@tlendack-t1.amdoffice.net>
References: <20170627150718.17428.81813.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170627151017.17428.9460.stgit@tlendack-t1.amdoffice.net>

Add a function that will determine if a supplied physical address matches
the address of an EFI table.

Reviewed-by: Matt Fleming <matt at codeblueprint.co.uk>
Reviewed-by: Borislav Petkov <bp at suse.de>
Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 drivers/firmware/efi/efi.c |   33 +++++++++++++++++++++++++++++++++
 include/linux/efi.h        |    7 +++++++
 2 files changed, 40 insertions(+)

diff --git a/drivers/firmware/efi/efi.c b/drivers/firmware/efi/efi.c
index 045d6d3..69d4d13 100644
--- a/drivers/firmware/efi/efi.c
+++ b/drivers/firmware/efi/efi.c
@@ -55,6 +55,25 @@ struct efi __read_mostly efi = {
 };
 EXPORT_SYMBOL(efi);
 
+static unsigned long *efi_tables[] = {
+	&efi.mps,
+	&efi.acpi,
+	&efi.acpi20,
+	&efi.smbios,
+	&efi.smbios3,
+	&efi.sal_systab,
+	&efi.boot_info,
+	&efi.hcdp,
+	&efi.uga,
+	&efi.uv_systab,
+	&efi.fw_vendor,
+	&efi.runtime,
+	&efi.config_table,
+	&efi.esrt,
+	&efi.properties_table,
+	&efi.mem_attr_table,
+};
+
 static bool disable_runtime;
 static int __init setup_noefi(char *arg)
 {
@@ -855,6 +874,20 @@ int efi_status_to_err(efi_status_t status)
 	return err;
 }
 
+bool efi_is_table_address(unsigned long phys_addr)
+{
+	unsigned int i;
+
+	if (phys_addr == EFI_INVALID_TABLE_ADDR)
+		return false;
+
+	for (i = 0; i < ARRAY_SIZE(efi_tables); i++)
+		if (*(efi_tables[i]) == phys_addr)
+			return true;
+
+	return false;
+}
+
 #ifdef CONFIG_KEXEC
 static int update_efi_random_seed(struct notifier_block *nb,
 				  unsigned long code, void *unused)
diff --git a/include/linux/efi.h b/include/linux/efi.h
index 8269bcb..8e24f09 100644
--- a/include/linux/efi.h
+++ b/include/linux/efi.h
@@ -1091,6 +1091,8 @@ static inline bool efi_enabled(int feature)
 	return test_bit(feature, &efi.flags) != 0;
 }
 extern void efi_reboot(enum reboot_mode reboot_mode, const char *__unused);
+
+extern bool efi_is_table_address(unsigned long phys_addr);
 #else
 static inline bool efi_enabled(int feature)
 {
@@ -1104,6 +1106,11 @@ static inline bool efi_enabled(int feature)
 {
 	return false;
 }
+
+static inline bool efi_is_table_address(unsigned long phys_addr)
+{
+	return false;
+}
 #endif
 
 extern int efi_status_to_err(efi_status_t status);



From thomas.lendacky at amd.com  Tue Jun 27 08:00:56 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Tue, 27 Jun 2017 10:00:56 -0500
Subject: [PATCH v8 27/38] iommu/amd: Allow the AMD IOMMU to work with memory
 encryption
In-Reply-To: <20170627145607.15908.26571.stgit@tlendack-t1.amdoffice.net>
References: <20170627145607.15908.26571.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170627150056.15908.79730.stgit@tlendack-t1.amdoffice.net>

The IOMMU is programmed with physical addresses for the various tables
and buffers that are used to communicate between the device and the
driver. When the driver allocates this memory it is encrypted. In order
for the IOMMU to access the memory as encrypted the encryption mask needs
to be included in these physical addresses during configuration.

The PTE entries created by the IOMMU should also include the encryption
mask so that when the device behind the IOMMU performs a DMA, the DMA
will be performed to encrypted memory.

Reviewed-by: Borislav Petkov <bp at suse.de>
Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 drivers/iommu/amd_iommu.c       |   30 ++++++++++++++++--------------
 drivers/iommu/amd_iommu_init.c  |   34 ++++++++++++++++++++++++++++------
 drivers/iommu/amd_iommu_proto.h |   10 ++++++++++
 drivers/iommu/amd_iommu_types.h |    2 +-
 4 files changed, 55 insertions(+), 21 deletions(-)

diff --git a/drivers/iommu/amd_iommu.c b/drivers/iommu/amd_iommu.c
index 503849d..16cc54b 100644
--- a/drivers/iommu/amd_iommu.c
+++ b/drivers/iommu/amd_iommu.c
@@ -544,7 +544,7 @@ static void dump_dte_entry(u16 devid)
 
 static void dump_command(unsigned long phys_addr)
 {
-	struct iommu_cmd *cmd = phys_to_virt(phys_addr);
+	struct iommu_cmd *cmd = iommu_phys_to_virt(phys_addr);
 	int i;
 
 	for (i = 0; i < 4; ++i)
@@ -865,11 +865,13 @@ static void copy_cmd_to_buffer(struct amd_iommu *iommu,
 
 static void build_completion_wait(struct iommu_cmd *cmd, u64 address)
 {
+	u64 paddr = iommu_virt_to_phys((void *)address);
+
 	WARN_ON(address & 0x7ULL);
 
 	memset(cmd, 0, sizeof(*cmd));
-	cmd->data[0] = lower_32_bits(__pa(address)) | CMD_COMPL_WAIT_STORE_MASK;
-	cmd->data[1] = upper_32_bits(__pa(address));
+	cmd->data[0] = lower_32_bits(paddr) | CMD_COMPL_WAIT_STORE_MASK;
+	cmd->data[1] = upper_32_bits(paddr);
 	cmd->data[2] = 1;
 	CMD_SET_TYPE(cmd, CMD_COMPL_WAIT);
 }
@@ -1328,7 +1330,7 @@ static bool increase_address_space(struct protection_domain *domain,
 		return false;
 
 	*pte             = PM_LEVEL_PDE(domain->mode,
-					virt_to_phys(domain->pt_root));
+					iommu_virt_to_phys(domain->pt_root));
 	domain->pt_root  = pte;
 	domain->mode    += 1;
 	domain->updated  = true;
@@ -1365,7 +1367,7 @@ static u64 *alloc_pte(struct protection_domain *domain,
 			if (!page)
 				return NULL;
 
-			__npte = PM_LEVEL_PDE(level, virt_to_phys(page));
+			__npte = PM_LEVEL_PDE(level, iommu_virt_to_phys(page));
 
 			/* pte could have been changed somewhere. */
 			if (cmpxchg64(pte, __pte, __npte) != __pte) {
@@ -1481,10 +1483,10 @@ static int iommu_map_page(struct protection_domain *dom,
 			return -EBUSY;
 
 	if (count > 1) {
-		__pte = PAGE_SIZE_PTE(phys_addr, page_size);
+		__pte = PAGE_SIZE_PTE(__sme_set(phys_addr), page_size);
 		__pte |= PM_LEVEL_ENC(7) | IOMMU_PTE_P | IOMMU_PTE_FC;
 	} else
-		__pte = phys_addr | IOMMU_PTE_P | IOMMU_PTE_FC;
+		__pte = __sme_set(phys_addr) | IOMMU_PTE_P | IOMMU_PTE_FC;
 
 	if (prot & IOMMU_PROT_IR)
 		__pte |= IOMMU_PTE_IR;
@@ -1700,7 +1702,7 @@ static void free_gcr3_tbl_level1(u64 *tbl)
 		if (!(tbl[i] & GCR3_VALID))
 			continue;
 
-		ptr = __va(tbl[i] & PAGE_MASK);
+		ptr = iommu_phys_to_virt(tbl[i] & PAGE_MASK);
 
 		free_page((unsigned long)ptr);
 	}
@@ -1715,7 +1717,7 @@ static void free_gcr3_tbl_level2(u64 *tbl)
 		if (!(tbl[i] & GCR3_VALID))
 			continue;
 
-		ptr = __va(tbl[i] & PAGE_MASK);
+		ptr = iommu_phys_to_virt(tbl[i] & PAGE_MASK);
 
 		free_gcr3_tbl_level1(ptr);
 	}
@@ -1807,7 +1809,7 @@ static void set_dte_entry(u16 devid, struct protection_domain *domain, bool ats)
 	u64 flags = 0;
 
 	if (domain->mode != PAGE_MODE_NONE)
-		pte_root = virt_to_phys(domain->pt_root);
+		pte_root = iommu_virt_to_phys(domain->pt_root);
 
 	pte_root |= (domain->mode & DEV_ENTRY_MODE_MASK)
 		    << DEV_ENTRY_MODE_SHIFT;
@@ -1819,7 +1821,7 @@ static void set_dte_entry(u16 devid, struct protection_domain *domain, bool ats)
 		flags |= DTE_FLAG_IOTLB;
 
 	if (domain->flags & PD_IOMMUV2_MASK) {
-		u64 gcr3 = __pa(domain->gcr3_tbl);
+		u64 gcr3 = iommu_virt_to_phys(domain->gcr3_tbl);
 		u64 glx  = domain->glx;
 		u64 tmp;
 
@@ -3470,10 +3472,10 @@ static u64 *__get_gcr3_pte(u64 *root, int level, int pasid, bool alloc)
 			if (root == NULL)
 				return NULL;
 
-			*pte = __pa(root) | GCR3_VALID;
+			*pte = iommu_virt_to_phys(root) | GCR3_VALID;
 		}
 
-		root = __va(*pte & PAGE_MASK);
+		root = iommu_phys_to_virt(*pte & PAGE_MASK);
 
 		level -= 1;
 	}
@@ -3652,7 +3654,7 @@ static void set_dte_irq_entry(u16 devid, struct irq_remap_table *table)
 
 	dte	= amd_iommu_dev_table[devid].data[2];
 	dte	&= ~DTE_IRQ_PHYS_ADDR_MASK;
-	dte	|= virt_to_phys(table->table);
+	dte	|= iommu_virt_to_phys(table->table);
 	dte	|= DTE_IRQ_REMAP_INTCTL;
 	dte	|= DTE_IRQ_TABLE_LEN;
 	dte	|= DTE_IRQ_REMAP_ENABLE;
diff --git a/drivers/iommu/amd_iommu_init.c b/drivers/iommu/amd_iommu_init.c
index 5a11328..26862f1 100644
--- a/drivers/iommu/amd_iommu_init.c
+++ b/drivers/iommu/amd_iommu_init.c
@@ -29,6 +29,7 @@
 #include <linux/export.h>
 #include <linux/iommu.h>
 #include <linux/kmemleak.h>
+#include <linux/mem_encrypt.h>
 #include <asm/pci-direct.h>
 #include <asm/iommu.h>
 #include <asm/gart.h>
@@ -346,7 +347,7 @@ static void iommu_set_device_table(struct amd_iommu *iommu)
 
 	BUG_ON(iommu->mmio_base == NULL);
 
-	entry = virt_to_phys(amd_iommu_dev_table);
+	entry = iommu_virt_to_phys(amd_iommu_dev_table);
 	entry |= (dev_table_size >> 12) - 1;
 	memcpy_toio(iommu->mmio_base + MMIO_DEV_TABLE_OFFSET,
 			&entry, sizeof(entry));
@@ -602,7 +603,7 @@ static void iommu_enable_command_buffer(struct amd_iommu *iommu)
 
 	BUG_ON(iommu->cmd_buf == NULL);
 
-	entry = (u64)virt_to_phys(iommu->cmd_buf);
+	entry = iommu_virt_to_phys(iommu->cmd_buf);
 	entry |= MMIO_CMD_SIZE_512;
 
 	memcpy_toio(iommu->mmio_base + MMIO_CMD_BUF_OFFSET,
@@ -631,7 +632,7 @@ static void iommu_enable_event_buffer(struct amd_iommu *iommu)
 
 	BUG_ON(iommu->evt_buf == NULL);
 
-	entry = (u64)virt_to_phys(iommu->evt_buf) | EVT_LEN_MASK;
+	entry = iommu_virt_to_phys(iommu->evt_buf) | EVT_LEN_MASK;
 
 	memcpy_toio(iommu->mmio_base + MMIO_EVT_BUF_OFFSET,
 		    &entry, sizeof(entry));
@@ -664,7 +665,7 @@ static void iommu_enable_ppr_log(struct amd_iommu *iommu)
 	if (iommu->ppr_log == NULL)
 		return;
 
-	entry = (u64)virt_to_phys(iommu->ppr_log) | PPR_LOG_SIZE_512;
+	entry = iommu_virt_to_phys(iommu->ppr_log) | PPR_LOG_SIZE_512;
 
 	memcpy_toio(iommu->mmio_base + MMIO_PPR_LOG_OFFSET,
 		    &entry, sizeof(entry));
@@ -744,10 +745,10 @@ static int iommu_init_ga_log(struct amd_iommu *iommu)
 	if (!iommu->ga_log_tail)
 		goto err_out;
 
-	entry = (u64)virt_to_phys(iommu->ga_log) | GA_LOG_SIZE_512;
+	entry = iommu_virt_to_phys(iommu->ga_log) | GA_LOG_SIZE_512;
 	memcpy_toio(iommu->mmio_base + MMIO_GA_LOG_BASE_OFFSET,
 		    &entry, sizeof(entry));
-	entry = ((u64)virt_to_phys(iommu->ga_log) & 0xFFFFFFFFFFFFFULL) & ~7ULL;
+	entry = (iommu_virt_to_phys(iommu->ga_log) & 0xFFFFFFFFFFFFFULL) & ~7ULL;
 	memcpy_toio(iommu->mmio_base + MMIO_GA_LOG_TAIL_OFFSET,
 		    &entry, sizeof(entry));
 	writel(0x00, iommu->mmio_base + MMIO_GA_HEAD_OFFSET);
@@ -2535,6 +2536,24 @@ static int __init amd_iommu_init(void)
 	return ret;
 }
 
+static bool amd_iommu_supports_sme(void)
+{
+	if (!sme_active() || (boot_cpu_data.x86 != 0x17))
+		return true;
+
+	/* For Fam17h, a specific level of support is required */
+	if (boot_cpu_data.microcode >= 0x08001205)
+		return true;
+
+	if ((boot_cpu_data.microcode >= 0x08001126) &&
+	    (boot_cpu_data.microcode <= 0x080011ff))
+		return true;
+
+	pr_notice("AMD-Vi: IOMMU not currently supported when SME is active\n");
+
+	return false;
+}
+
 /****************************************************************************
  *
  * Early detect code. This code runs at IOMMU detection time in the DMA
@@ -2552,6 +2571,9 @@ int __init amd_iommu_detect(void)
 	if (amd_iommu_disabled)
 		return -ENODEV;
 
+	if (!amd_iommu_supports_sme())
+		return -ENODEV;
+
 	ret = iommu_go_to_state(IOMMU_IVRS_DETECTED);
 	if (ret)
 		return ret;
diff --git a/drivers/iommu/amd_iommu_proto.h b/drivers/iommu/amd_iommu_proto.h
index 466260f..3f12fb2 100644
--- a/drivers/iommu/amd_iommu_proto.h
+++ b/drivers/iommu/amd_iommu_proto.h
@@ -87,4 +87,14 @@ static inline bool iommu_feature(struct amd_iommu *iommu, u64 f)
 	return !!(iommu->features & f);
 }
 
+static inline u64 iommu_virt_to_phys(void *vaddr)
+{
+	return (u64)__sme_set(virt_to_phys(vaddr));
+}
+
+static inline void *iommu_phys_to_virt(unsigned long paddr)
+{
+	return phys_to_virt(__sme_clr(paddr));
+}
+
 #endif /* _ASM_X86_AMD_IOMMU_PROTO_H  */
diff --git a/drivers/iommu/amd_iommu_types.h b/drivers/iommu/amd_iommu_types.h
index 4de8f41..3ce587d 100644
--- a/drivers/iommu/amd_iommu_types.h
+++ b/drivers/iommu/amd_iommu_types.h
@@ -343,7 +343,7 @@
 
 #define IOMMU_PAGE_MASK (((1ULL << 52) - 1) & ~0xfffULL)
 #define IOMMU_PTE_PRESENT(pte) ((pte) & IOMMU_PTE_P)
-#define IOMMU_PTE_PAGE(pte) (phys_to_virt((pte) & IOMMU_PAGE_MASK))
+#define IOMMU_PTE_PAGE(pte) (iommu_phys_to_virt((pte) & IOMMU_PAGE_MASK))
 #define IOMMU_PTE_MODE(pte) (((pte) >> 9) & 0x07)
 
 #define IOMMU_PROT_MASK 0x03



From thomas.lendacky at amd.com  Tue Jun 27 08:09:29 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Tue, 27 Jun 2017 10:09:29 -0500
Subject: [PATCH v8 RESEND 12/38] x86/mm: Extend early_memremap() support
 with additional attrs
In-Reply-To: <20170627150718.17428.81813.stgit@tlendack-t1.amdoffice.net>
References: <20170627150718.17428.81813.stgit@tlendack-t1.amdoffice.net>
Message-ID: <20170627150929.17428.31589.stgit@tlendack-t1.amdoffice.net>

Add early_memremap() support to be able to specify encrypted and
decrypted mappings with and without write-protection. The use of
write-protection is necessary when encrypting data "in place". The
write-protect attribute is considered cacheable for loads, but not
stores. This implies that the hardware will never give the core a
dirty line with this memtype.

Reviewed-by: Borislav Petkov <bp at suse.de>
Signed-off-by: Tom Lendacky <thomas.lendacky at amd.com>
---
 arch/x86/Kconfig                     |    4 +++
 arch/x86/include/asm/fixmap.h        |   13 ++++++++++
 arch/x86/include/asm/pgtable_types.h |    8 ++++++
 arch/x86/mm/ioremap.c                |   44 ++++++++++++++++++++++++++++++++++
 include/asm-generic/early_ioremap.h  |    2 ++
 mm/early_ioremap.c                   |   10 ++++++++
 6 files changed, 81 insertions(+)

diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index 3a59e9c..a04081ce 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -1434,6 +1434,10 @@ config AMD_MEM_ENCRYPT_ACTIVE_BY_DEFAULT
 	  If set to N, then the encryption of system memory can be
 	  activated with the mem_encrypt=on command line option.
 
+config ARCH_USE_MEMREMAP_PROT
+	def_bool y
+	depends on AMD_MEM_ENCRYPT
+
 # Common NUMA Features
 config NUMA
 	bool "Numa Memory Allocation and Scheduler Support"
diff --git a/arch/x86/include/asm/fixmap.h b/arch/x86/include/asm/fixmap.h
index d9ff226..dcd9fb5 100644
--- a/arch/x86/include/asm/fixmap.h
+++ b/arch/x86/include/asm/fixmap.h
@@ -164,6 +164,19 @@ static inline void __set_fixmap(enum fixed_addresses idx,
  */
 #define FIXMAP_PAGE_NOCACHE PAGE_KERNEL_IO_NOCACHE
 
+/*
+ * Early memremap routines used for in-place encryption. The mappings created
+ * by these routines are intended to be used as temporary mappings.
+ */
+void __init *early_memremap_encrypted(resource_size_t phys_addr,
+				      unsigned long size);
+void __init *early_memremap_encrypted_wp(resource_size_t phys_addr,
+					 unsigned long size);
+void __init *early_memremap_decrypted(resource_size_t phys_addr,
+				      unsigned long size);
+void __init *early_memremap_decrypted_wp(resource_size_t phys_addr,
+					 unsigned long size);
+
 #include <asm-generic/fixmap.h>
 
 #define __late_set_fixmap(idx, phys, flags) __set_fixmap(idx, phys, flags)
diff --git a/arch/x86/include/asm/pgtable_types.h b/arch/x86/include/asm/pgtable_types.h
index de32ca3..32095af 100644
--- a/arch/x86/include/asm/pgtable_types.h
+++ b/arch/x86/include/asm/pgtable_types.h
@@ -161,6 +161,7 @@ enum page_cache_mode {
 
 #define _PAGE_CACHE_MASK	(_PAGE_PAT | _PAGE_PCD | _PAGE_PWT)
 #define _PAGE_NOCACHE		(cachemode2protval(_PAGE_CACHE_MODE_UC))
+#define _PAGE_CACHE_WP		(cachemode2protval(_PAGE_CACHE_MODE_WP))
 
 #define PAGE_NONE	__pgprot(_PAGE_PROTNONE | _PAGE_ACCESSED)
 #define PAGE_SHARED	__pgprot(_PAGE_PRESENT | _PAGE_RW | _PAGE_USER | \
@@ -189,6 +190,7 @@ enum page_cache_mode {
 #define __PAGE_KERNEL_VVAR		(__PAGE_KERNEL_RO | _PAGE_USER)
 #define __PAGE_KERNEL_LARGE		(__PAGE_KERNEL | _PAGE_PSE)
 #define __PAGE_KERNEL_LARGE_EXEC	(__PAGE_KERNEL_EXEC | _PAGE_PSE)
+#define __PAGE_KERNEL_WP		(__PAGE_KERNEL | _PAGE_CACHE_WP)
 
 #define __PAGE_KERNEL_IO		(__PAGE_KERNEL)
 #define __PAGE_KERNEL_IO_NOCACHE	(__PAGE_KERNEL_NOCACHE)
@@ -202,6 +204,12 @@ enum page_cache_mode {
 #define _KERNPG_TABLE	(_PAGE_PRESENT | _PAGE_RW | _PAGE_ACCESSED |	\
 			 _PAGE_DIRTY | _PAGE_ENC)
 
+#define __PAGE_KERNEL_ENC	(__PAGE_KERNEL | _PAGE_ENC)
+#define __PAGE_KERNEL_ENC_WP	(__PAGE_KERNEL_WP | _PAGE_ENC)
+
+#define __PAGE_KERNEL_NOENC	(__PAGE_KERNEL)
+#define __PAGE_KERNEL_NOENC_WP	(__PAGE_KERNEL_WP)
+
 #define PAGE_KERNEL		__pgprot(__PAGE_KERNEL | _PAGE_ENC)
 #define PAGE_KERNEL_RO		__pgprot(__PAGE_KERNEL_RO | _PAGE_ENC)
 #define PAGE_KERNEL_EXEC	__pgprot(__PAGE_KERNEL_EXEC | _PAGE_ENC)
diff --git a/arch/x86/mm/ioremap.c b/arch/x86/mm/ioremap.c
index bfc3e2d..26db273 100644
--- a/arch/x86/mm/ioremap.c
+++ b/arch/x86/mm/ioremap.c
@@ -414,6 +414,50 @@ void unxlate_dev_mem_ptr(phys_addr_t phys, void *addr)
 	iounmap((void __iomem *)((unsigned long)addr & PAGE_MASK));
 }
 
+#ifdef CONFIG_ARCH_USE_MEMREMAP_PROT
+/* Remap memory with encryption */
+void __init *early_memremap_encrypted(resource_size_t phys_addr,
+				      unsigned long size)
+{
+	return early_memremap_prot(phys_addr, size, __PAGE_KERNEL_ENC);
+}
+
+/*
+ * Remap memory with encryption and write-protected - cannot be called
+ * before pat_init() is called
+ */
+void __init *early_memremap_encrypted_wp(resource_size_t phys_addr,
+					 unsigned long size)
+{
+	/* Be sure the write-protect PAT entry is set for write-protect */
+	if (__pte2cachemode_tbl[_PAGE_CACHE_MODE_WP] != _PAGE_CACHE_MODE_WP)
+		return NULL;
+
+	return early_memremap_prot(phys_addr, size, __PAGE_KERNEL_ENC_WP);
+}
+
+/* Remap memory without encryption */
+void __init *early_memremap_decrypted(resource_size_t phys_addr,
+				      unsigned long size)
+{
+	return early_memremap_prot(phys_addr, size, __PAGE_KERNEL_NOENC);
+}
+
+/*
+ * Remap memory without encryption and write-protected - cannot be called
+ * before pat_init() is called
+ */
+void __init *early_memremap_decrypted_wp(resource_size_t phys_addr,
+					 unsigned long size)
+{
+	/* Be sure the write-protect PAT entry is set for write-protect */
+	if (__pte2cachemode_tbl[_PAGE_CACHE_MODE_WP] != _PAGE_CACHE_MODE_WP)
+		return NULL;
+
+	return early_memremap_prot(phys_addr, size, __PAGE_KERNEL_NOENC_WP);
+}
+#endif	/* CONFIG_ARCH_USE_MEMREMAP_PROT */
+
 static pte_t bm_pte[PAGE_SIZE/sizeof(pte_t)] __page_aligned_bss;
 
 static inline pmd_t * __init early_ioremap_pmd(unsigned long addr)
diff --git a/include/asm-generic/early_ioremap.h b/include/asm-generic/early_ioremap.h
index 734ad4d..2edef8d 100644
--- a/include/asm-generic/early_ioremap.h
+++ b/include/asm-generic/early_ioremap.h
@@ -13,6 +13,8 @@ extern void *early_memremap(resource_size_t phys_addr,
 			    unsigned long size);
 extern void *early_memremap_ro(resource_size_t phys_addr,
 			       unsigned long size);
+extern void *early_memremap_prot(resource_size_t phys_addr,
+				 unsigned long size, unsigned long prot_val);
 extern void early_iounmap(void __iomem *addr, unsigned long size);
 extern void early_memunmap(void *addr, unsigned long size);
 
diff --git a/mm/early_ioremap.c b/mm/early_ioremap.c
index 6d5717b..d7d30da 100644
--- a/mm/early_ioremap.c
+++ b/mm/early_ioremap.c
@@ -226,6 +226,16 @@ void __init early_iounmap(void __iomem *addr, unsigned long size)
 }
 #endif
 
+#ifdef CONFIG_ARCH_USE_MEMREMAP_PROT
+void __init *
+early_memremap_prot(resource_size_t phys_addr, unsigned long size,
+		    unsigned long prot_val)
+{
+	return (__force void *)__early_ioremap(phys_addr, size,
+					       __pgprot(prot_val));
+}
+#endif
+
 #define MAX_MAP_CHUNK	(NR_FIX_BTMAPS << PAGE_SHIFT)
 
 void __init copy_from_early_mem(void *dest, phys_addr_t src, unsigned long size)



From thomas.lendacky at amd.com  Tue Jun 27 08:07:19 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Tue, 27 Jun 2017 10:07:19 -0500
Subject: [PATCH v8 RESEND 00/38] x86: Secure Memory Encryption (AMD)
Message-ID: <20170627150718.17428.81813.stgit@tlendack-t1.amdoffice.net>

RESENDING - Mail Server Issues

This patch series provides support for AMD's new Secure Memory Encryption (SME)
feature.

SME can be used to mark individual pages of memory as encrypted through the
page tables. A page of memory that is marked encrypted will be automatically
decrypted when read from DRAM and will be automatically encrypted when
written to DRAM. Details on SME can found in the links below.

The SME feature is identified through a CPUID function and enabled through
the SYSCFG MSR. Once enabled, page table entries will determine how the
memory is accessed. If a page table entry has the memory encryption mask set,
then that memory will be accessed as encrypted memory. The memory encryption
mask (as well as other related information) is determined from settings
returned through the same CPUID function that identifies the presence of the
feature.

The approach that this patch series takes is to encrypt everything possible
starting early in the boot where the kernel is encrypted. Using the page
table macros the encryption mask can be incorporated into all page table
entries and page allocations. By updating the protection map, userspace
allocations are also marked encrypted. Certain data must be accounted for
as having been placed in memory before SME was enabled (EFI, initrd, etc.)
and accessed accordingly.

This patch series is a pre-cursor to another AMD processor feature called
Secure Encrypted Virtualization (SEV). The support for SEV will build upon
the SME support and will be submitted later. Details on SEV can be found
in the links below.

The following links provide additional detail:

AMD Memory Encryption whitepaper:
   http://amd-dev.wpengine.netdna-cdn.com/wordpress/media/2013/12/AMD_Memory_Encryption_Whitepaper_v7-Public.pdf

AMD64 Architecture Programmer's Manual:
   http://support.amd.com/TechDocs/24593.pdf
   SME is section 7.10
   SEV is section 15.34

---

This patch series is based off of the master branch of tip:
  https://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git master

  Commit 6ab5af989579 ("Merge branch 'irq/core'")

Source code is also available at https://github.com/codomania/tip/tree/sme-v8


Still to do:
- Kdump support, including using memremap() instead of ioremap_cache()

Changes since v7:
- Fixed kbuild test robot failure related to pgprot_decrypted() macro
  usage for some non-x86 archs
- Moved calls to encrypt the kernel and retrieve the encryption mask
  from assembler (head_64.S) into C (head64.c)
- Removed use of phys_to_virt() in __ioremap_caller() when address is in
  the ISA range. Now regular ioremap() processing occurs.
- Two new, small patches:
  - Introduced a native_make_p4d() for use when CONFIG_PGTABLE_LEVELS is
    not greater than 4
  - Introduced __nostackp GCC option to turn off stack protection on a
    per function basis
- General code cleanup based on feedback

Changes since v6:
- Fixed the asm include file issue that caused build errors on other archs
- Rebased the CR3 register changes on top of Andy Lutomirski's patch
- Added a patch to clear the SME cpu feature if running as a PV guest under
  Xen
- Added a patch to obtain the AMD microcode level earlier in the boot
  instead of directly reading the MSR
- Refactor patch #8 ("x86/mm: Add support to enable SME in early boot
  processing") because the 5-level paging support moved the code into the
  new C-function __startup_64()
- Removed need to decrypt trampoline area in-place (set memory attributes
  before copying the trampoline code)
- General code cleanup based on feedback

Changes since v5:
- Added support for 5-level paging
- Added IOMMU support
- Created a generic asm/mem_encrypt.h in order to remove a bunch of
  #ifndef/#define entries
- Removed changes to the __va() macro and defined a function to return
  the true physical address in cr3
- Removed sysfs support as it was determined not to be needed
- General code cleanup based on feedback
- General cleanup of patch subjects and descriptions

Changes since v4:
- Re-worked mapping of setup data to not use a fixed list. Rather, check
  dynamically whether the requested early_memremap()/memremap() call
  needs to be mapped decrypted.
- Moved SME cpu feature into scattered features
- Moved some declarations into header files
- Cleared the encryption mask from the __PHYSICAL_MASK so that users
  of macros such as pmd_pfn_mask() don't have to worry/know about the
  encryption mask
- Updated some return types and values related to EFI and e820 functions
  so that an error could be returned
- During cpu shutdown, removed cache disabling and added a check for kexec
  in progress to use wbinvd followed immediately by halt in order to avoid
  any memory corruption
- Update how persistent memory is identified
- Added a function to find command line arguments and their values
- Added sysfs support
- General code cleanup based on feedback
- General cleanup of patch subjects and descriptions


Changes since v3:
- Broke out some of the patches into smaller individual patches
- Updated Documentation
- Added a message to indicate why the IOMMU was disabled
- Updated CPU feature support for SME by taking into account whether
  BIOS has enabled SME
- Eliminated redundant functions
- Added some warning messages for DMA usage of bounce buffers when SME
  is active
- Added support for persistent memory
- Added support to determine when setup data is being mapped and be sure
  to map it un-encrypted
- Added CONFIG support to set the default action of whether to activate
  SME if it is supported/enabled
- Added support for (re)booting with kexec

Changes since v2:
- Updated Documentation
- Make the encryption mask available outside of arch/x86 through a
  standard include file
- Conversion of assembler routines to C where possible (not everything
  could be converted, e.g. the routine that does the actual encryption
  needs to be copied into a safe location and it is difficult to
  determine the actual length of the function in order to copy it)
- Fix SME feature use of scattered CPUID feature
- Creation of SME specific functions for things like encrypting
  the setup data, ramdisk, etc.
- New take on early_memremap / memremap encryption support
- Additional support for accessing video buffers (fbdev/gpu) as
  un-encrypted
- Disable IOMMU for now - need to investigate further in relation to
  how it needs to be programmed relative to accessing physical memory

Changes since v1:
- Added Documentation.
- Removed AMD vendor check for setting the PAT write protect mode
- Updated naming of trampoline flag for SME as well as moving of the
  SME check to before paging is enabled.
- Change to early_memremap to identify the data being mapped as either
  boot data or kernel data.  The idea being that boot data will have
  been placed in memory as un-encrypted data and would need to be accessed
  as such.
- Updated debugfs support for the bootparams to access the data properly.
- Do not set the SYSCFG[MEME] bit, only check it.  The setting of the
  MemEncryptionModeEn bit results in a reduction of physical address size
  of the processor.  It is possible that BIOS could have configured resources
  resources into a range that will now not be addressable.  To prevent this,
  rely on BIOS to set the SYSCFG[MEME] bit and only then enable memory
  encryption support in the kernel.

Tom Lendacky (38):
      x86: Document AMD Secure Memory Encryption (SME)
      x86/mm/pat: Set write-protect cache mode for full PAT support
      x86, mpparse, x86/acpi, x86/PCI, x86/dmi, SFI: Use memremap for RAM mappings
      x86/CPU/AMD: Add the Secure Memory Encryption CPU feature
      x86/CPU/AMD: Handle SME reduction in physical address size
      x86/mm: Add Secure Memory Encryption (SME) support
      x86/mm: Remove phys_to_virt() usage in ioremap()
      x86/mm: Add support to enable SME in early boot processing
      x86/mm: Simplify p[g4um]d_page() macros
      x86/mm: Provide general kernel support for memory encryption
      x86/mm: Add SME support for read_cr3_pa()
      x86/mm: Extend early_memremap() support with additional attrs
      x86/mm: Add support for early encrypt/decrypt of memory
      x86/mm: Insure that boot memory areas are mapped properly
      x86/boot/e820: Add support to determine the E820 type of an address
      efi: Add an EFI table address match function
      efi: Update efi_mem_type() to return an error rather than 0
      x86/efi: Update EFI pagetable creation to work with SME
      x86/mm: Add support to access boot related data in the clear
      x86, mpparse: Use memremap to map the mpf and mpc data
      x86/mm: Add support to access persistent memory in the clear
      x86/mm: Add support for changing the memory encryption attribute
      x86/realmode: Decrypt trampoline area if memory encryption is active
      x86, swiotlb: Add memory encryption support
      swiotlb: Add warnings for use of bounce buffers with SME
      x86/CPU/AMD: Make the microcode level available earlier in the boot
      iommu/amd: Allow the AMD IOMMU to work with memory encryption
      x86, realmode: Check for memory encryption on the APs
      x86, drm, fbdev: Do not specify encrypted memory for video mappings
      kvm: x86: svm: Support Secure Memory Encryption within KVM
      x86/mm, kexec: Allow kexec to be used with SME
      xen/x86: Remove SME feature in PV guests
      x86/mm: Use proper encryption attributes with /dev/mem
      x86/mm: Create native_make_p4d() for PGTABLE_LEVELS <= 4
      x86/mm: Add support to encrypt the kernel in-place
      x86/boot: Add early cmdline parsing for options with arguments
      compiler-gcc.h: Introduce __nostackp function attribute
      x86/mm: Add support to make use of Secure Memory Encryption


 Documentation/admin-guide/kernel-parameters.txt |   11 
 Documentation/x86/amd-memory-encryption.txt     |   68 +++
 arch/ia64/kernel/efi.c                          |    4 
 arch/x86/Kconfig                                |   29 +
 arch/x86/boot/compressed/pagetable.c            |    7 
 arch/x86/include/asm/cmdline.h                  |    2 
 arch/x86/include/asm/cpufeatures.h              |    1 
 arch/x86/include/asm/dma-mapping.h              |    5 
 arch/x86/include/asm/dmi.h                      |    8 
 arch/x86/include/asm/e820/api.h                 |    2 
 arch/x86/include/asm/fixmap.h                   |   20 +
 arch/x86/include/asm/init.h                     |    1 
 arch/x86/include/asm/io.h                       |    8 
 arch/x86/include/asm/kexec.h                    |    8 
 arch/x86/include/asm/kvm_host.h                 |    2 
 arch/x86/include/asm/mem_encrypt.h              |   80 +++
 arch/x86/include/asm/msr-index.h                |    2 
 arch/x86/include/asm/page_types.h               |    3 
 arch/x86/include/asm/pgtable.h                  |   28 +
 arch/x86/include/asm/pgtable_types.h            |   57 ++
 arch/x86/include/asm/processor-flags.h          |    5 
 arch/x86/include/asm/processor.h                |    8 
 arch/x86/include/asm/realmode.h                 |   12 
 arch/x86/include/asm/set_memory.h               |    3 
 arch/x86/include/asm/vga.h                      |   14 +
 arch/x86/kernel/acpi/boot.c                     |    6 
 arch/x86/kernel/cpu/amd.c                       |   25 +
 arch/x86/kernel/cpu/scattered.c                 |    1 
 arch/x86/kernel/e820.c                          |   26 +
 arch/x86/kernel/espfix_64.c                     |    2 
 arch/x86/kernel/head64.c                        |   93 +++-
 arch/x86/kernel/head_64.S                       |   40 +-
 arch/x86/kernel/kdebugfs.c                      |   34 -
 arch/x86/kernel/ksysfs.c                        |   28 +
 arch/x86/kernel/machine_kexec_64.c              |   22 +
 arch/x86/kernel/mpparse.c                       |  108 +++-
 arch/x86/kernel/pci-dma.c                       |   11 
 arch/x86/kernel/pci-nommu.c                     |    2 
 arch/x86/kernel/pci-swiotlb.c                   |   15 +
 arch/x86/kernel/process.c                       |   17 +
 arch/x86/kernel/setup.c                         |    9 
 arch/x86/kvm/mmu.c                              |   12 
 arch/x86/kvm/mmu.h                              |    2 
 arch/x86/kvm/svm.c                              |   35 +
 arch/x86/kvm/vmx.c                              |    3 
 arch/x86/kvm/x86.c                              |    3 
 arch/x86/lib/cmdline.c                          |  105 ++++
 arch/x86/mm/Makefile                            |    2 
 arch/x86/mm/ident_map.c                         |   12 
 arch/x86/mm/ioremap.c                           |  275 ++++++++++-
 arch/x86/mm/kasan_init_64.c                     |    6 
 arch/x86/mm/mem_encrypt.c                       |  593 +++++++++++++++++++++++
 arch/x86/mm/mem_encrypt_boot.S                  |  150 ++++++
 arch/x86/mm/pageattr.c                          |   67 +++
 arch/x86/mm/pat.c                               |    9 
 arch/x86/pci/common.c                           |    4 
 arch/x86/platform/efi/efi.c                     |    6 
 arch/x86/platform/efi/efi_64.c                  |   15 -
 arch/x86/realmode/init.c                        |   12 
 arch/x86/realmode/rm/trampoline_64.S            |   24 +
 arch/x86/xen/enlighten_pv.c                     |    1 
 drivers/firmware/dmi-sysfs.c                    |    5 
 drivers/firmware/efi/efi.c                      |   33 +
 drivers/firmware/pcdp.c                         |    4 
 drivers/gpu/drm/drm_gem.c                       |    2 
 drivers/gpu/drm/drm_vm.c                        |    4 
 drivers/gpu/drm/ttm/ttm_bo_vm.c                 |    7 
 drivers/gpu/drm/udl/udl_fb.c                    |    4 
 drivers/iommu/amd_iommu.c                       |   30 +
 drivers/iommu/amd_iommu_init.c                  |   34 +
 drivers/iommu/amd_iommu_proto.h                 |   10 
 drivers/iommu/amd_iommu_types.h                 |    2 
 drivers/sfi/sfi_core.c                          |   22 -
 drivers/video/fbdev/core/fbmem.c                |   12 
 include/asm-generic/early_ioremap.h             |    2 
 include/asm-generic/pgtable.h                   |   12 
 include/linux/compiler-gcc.h                    |    2 
 include/linux/compiler.h                        |    4 
 include/linux/dma-mapping.h                     |   13 +
 include/linux/efi.h                             |    9 
 include/linux/io.h                              |    2 
 include/linux/kexec.h                           |    8 
 include/linux/mem_encrypt.h                     |   48 ++
 include/linux/swiotlb.h                         |    1 
 init/main.c                                     |   10 
 kernel/kexec_core.c                             |   12 
 kernel/memremap.c                               |   20 +
 lib/swiotlb.c                                   |   57 ++
 mm/early_ioremap.c                              |   28 +
 89 files changed, 2293 insertions(+), 267 deletions(-)
 create mode 100644 Documentation/x86/amd-memory-encryption.txt
 create mode 100644 arch/x86/include/asm/mem_encrypt.h
 create mode 100644 arch/x86/mm/mem_encrypt.c
 create mode 100644 arch/x86/mm/mem_encrypt_boot.S
 create mode 100644 include/linux/mem_encrypt.h

-- 
Tom Lendacky


From xiyou.wangcong at gmail.com  Tue Jun 27 09:43:33 2017
From: xiyou.wangcong at gmail.com (Cong Wang)
Date: Tue, 27 Jun 2017 09:43:33 -0700
Subject: [PATCH] vmcore-dmesg: avoid allocating large memory chunk for log
 buf
In-Reply-To: <1497910994-15129-1-git-send-email-xiyou.wangcong@gmail.com>
References: <1497910994-15129-1-git-send-email-xiyou.wangcong@gmail.com>
Message-ID: <CAM_iQpW0w-VAd8ng2E_wTJPw7qx8g6JJzfv0+cmV2_3U2uBH3Q@mail.gmail.com>

Ping... Any review?

On Mon, Jun 19, 2017 at 3:23 PM, Cong Wang <xiyou.wangcong at gmail.com> wrote:
> In kdump kernel, we got:
>
>     Saving vmcore-dmesg.txt
>     Failed to malloc 67108864 bytes for the logbuf: Cannot allocate memory
>
> Apparently 64M is too luxury for a kdump kernel which only has 128M
> in total.
>
> We can avoid allocating such a large memory chunk, instead just allocate
> a smaller chunk inside the loop each time.
>
> Verify the result by comparing the vmcore-dmesg output with and without
> this patch.
>
> Cc: Simon Horman <horms at verge.net.au>
> Signed-off-by: Cong Wang <xiyou.wangcong at gmail.com>
> ---
>  vmcore-dmesg/vmcore-dmesg.c | 101 ++++++++++++++++++--------------------------
>  1 file changed, 41 insertions(+), 60 deletions(-)
>
> diff --git a/vmcore-dmesg/vmcore-dmesg.c b/vmcore-dmesg/vmcore-dmesg.c
> index a8f56df..593557c 100644
> --- a/vmcore-dmesg/vmcore-dmesg.c
> +++ b/vmcore-dmesg/vmcore-dmesg.c
> @@ -565,53 +565,12 @@ static inline uint64_t struct_val_u64(char *ptr, unsigned int offset)
>         return(file64_to_cpu(*(uint64_t *)(ptr + offset)));
>  }
>
> -/* human readable text of the record */
> -static char *log_text(char *msg)
> -{
> -       return msg + log_sz;
> -}
> -
> -/* get record by index; idx must point to valid msg */
> -static char *log_from_idx(char *log_buf, uint32_t idx)
> -{
> -       char *msg = log_buf + idx;
> -
> -       /*
> -        * A length == 0 record is the end of buffer marker. Wrap around and
> -        * read the message at the start of the buffer.
> -        */
> -       if (!struct_val_u16(msg, log_offset_len))
> -               return log_buf;
> -       return msg;
> -}
> -
> -/* get next record; idx must point to valid msg */
> -static uint32_t log_next(char *log_buf, uint32_t idx)
> -{
> -       char *msg = log_buf + idx;
> -       uint16_t len;
> -
> -       /* length == 0 indicates the end of the buffer; wrap */
> -       /*
> -        * A length == 0 record is the end of buffer marker. Wrap around and
> -        * read the message at the start of the buffer as *this* one, and
> -        * return the one after that.
> -        */
> -       len = struct_val_u16(msg, log_offset_len);
> -       if (!len) {
> -               msg = log_buf;
> -               return struct_val_u16(msg, log_offset_len);
> -       }
> -       return idx + len;
> -}
> -
>  /* Read headers of log records and dump accordingly */
>  static void dump_dmesg_structured(int fd)
>  {
>  #define OUT_BUF_SIZE   4096
>         uint64_t log_buf, log_buf_offset, ts_nsec;
>         uint32_t log_first_idx, log_next_idx, current_idx, len = 0, i;
> -       int log_buf_len;
>         char *buf, out_buf[OUT_BUF_SIZE];
>         ssize_t ret;
>         char *msg;
> @@ -659,34 +618,33 @@ static void dump_dmesg_structured(int fd)
>         }
>
>         log_buf = read_file_pointer(fd, vaddr_to_offset(log_buf_vaddr));
> -       log_buf_len = read_file_s32(fd, vaddr_to_offset(log_buf_len_vaddr));
>
>         log_first_idx = read_file_u32(fd, vaddr_to_offset(log_first_idx_vaddr));
>         log_next_idx = read_file_u32(fd, vaddr_to_offset(log_next_idx_vaddr));
>
>         log_buf_offset = vaddr_to_offset(log_buf);
>
> -       buf = calloc(1, log_buf_len);
> +       buf = calloc(1, log_sz);
>         if (!buf) {
> -               fprintf(stderr, "Failed to malloc %d bytes for the logbuf:"
> -                               " %s\n", log_buf_len, strerror(errno));
> +               fprintf(stderr, "Failed to malloc %" PRId64 " bytes for the log:"
> +                               " %s\n", log_sz, strerror(errno));
>                 exit(64);
>         }
>
> -       ret = pread(fd, buf, log_buf_len, log_buf_offset);
> -       if (ret != log_buf_len) {
> -               fprintf(stderr, "Failed to read log buffer of size %d bytes:"
> -                       " %s\n", log_buf_len, strerror(errno));
> -               exit(65);
> -       }
> -
>         /* Parse records and write out data at standard output */
>
>         current_idx = log_first_idx;
>         len = 0;
>         while (current_idx != log_next_idx) {
> -               msg = log_from_idx(buf, current_idx);
> -               ts_nsec = struct_val_u64(msg, log_offset_ts_nsec);
> +               uint16_t loglen;
> +
> +               ret = pread(fd, buf, log_sz, log_buf_offset + current_idx);
> +               if (ret != log_sz) {
> +                       fprintf(stderr, "Failed to read log of size %" PRId64 " bytes:"
> +                               " %s\n", log_sz, strerror(errno));
> +                       exit(65);
> +               }
> +               ts_nsec = struct_val_u64(buf, log_offset_ts_nsec);
>                 imaxdiv_sec = imaxdiv(ts_nsec, 1000000000);
>                 imaxdiv_usec = imaxdiv(imaxdiv_sec.rem, 1000);
>
> @@ -695,9 +653,22 @@ static void dump_dmesg_structured(int fd)
>                         (long long unsigned int)imaxdiv_usec.quot);
>
>                 /* escape non-printable characters */
> -               text_len = struct_val_u16(msg, log_offset_text_len);
> +               text_len = struct_val_u16(buf, log_offset_text_len);
> +               msg = calloc(1, text_len);
> +               if (!msg) {
> +                       fprintf(stderr, "Failed to malloc %u bytes for log text:"
> +                               " %s\n", text_len, strerror(errno));
> +                       exit(64);
> +               }
> +
> +               ret = pread(fd, msg, text_len, log_buf_offset + current_idx + log_sz);
> +               if (ret != text_len) {
> +                       fprintf(stderr, "Failed to read log text of size %u bytes:"
> +                               " %s\n", text_len, strerror(errno));
> +                       exit(65);
> +               }
>                 for (i = 0; i < text_len; i++) {
> -                       unsigned char c = log_text(msg)[i];
> +                       unsigned char c = msg[i];
>
>                         if (!isprint(c) && !isspace(c))
>                                 len += sprintf(out_buf + len, "\\x%02x", c);
> @@ -711,11 +682,21 @@ static void dump_dmesg_structured(int fd)
>                 }
>
>                 out_buf[len++] = '\n';
> -
> -               /* Move to next record */
> -               current_idx = log_next(buf, current_idx);
> +               free(msg);
> +               /* length == 0 indicates the end of the buffer; wrap */
> +               /*
> +                * A length == 0 record is the end of buffer marker. Wrap around and
> +                * read the message at the start of the buffer as *this* one, and
> +                * return the one after that.
> +                */
> +               loglen = struct_val_u16(buf, log_offset_len);
> +               if (!loglen)
> +                       current_idx = log_first_idx;
> +               else
> +                       /* Move to next record */
> +                       current_idx += loglen;
>         }
> -
> +       free(buf);
>         if (len)
>                 write_to_stdout(out_buf, len);
>  }
> --
> 2.5.5
>


From panand at redhat.com  Tue Jun 27 20:45:51 2017
From: panand at redhat.com (Pratyush Anand)
Date: Wed, 28 Jun 2017 09:15:51 +0530
Subject: [Makedumpfile PATCH V3] elf_info: fix file_size if segment is
 excluded
In-Reply-To: <0910DD04CBD6DE4193FCF86B9C00BE9701EDC3E3@BPXM01GP.gisp.nec.co.jp>
References: <648cd9280e333c1b78584afb3f0843434640fe14.1494390678.git.panand@redhat.com>
 <0910DD04CBD6DE4193FCF86B9C00BE9701ED845A@BPXM01GP.gisp.nec.co.jp>
 <8991bcd8-32ed-f7f1-64f4-86858ebb76f7@redhat.com>
 <0910DD04CBD6DE4193FCF86B9C00BE9701EDC3E3@BPXM01GP.gisp.nec.co.jp>
Message-ID: <dbaed487-b5cf-bf09-bcc0-4e638194f6a7@redhat.com>

Hi Atsushi,

On Wednesday 31 May 2017 06:19 AM, Atsushi Kumagai wrote:
>> Hi Atsushi,
>>
>> On Wednesday 17 May 2017 01:15 PM, Atsushi Kumagai wrote:
>>> Hello Pratyush,
>>>
>>> Sorry for late reply.
>>> I have no further comment for now, but the current --mem-usage
>>> seems have some problems. I can't decide that this patch is insufficient or
>>> that's quite a different matter now, so please let me defer acknowledgment
>>> for a while.
>>>
>>
>> Any decision on this patch.
>
> The problem I met is still unresolved, but it occurs even without this patch.
> So I accept this patch for now.
>

This patch is missing in your devel branch.

Just a gentle reminder, so that it does not miss 1.6.2.

Thanks


Pratyush



From dyoung at redhat.com  Tue Jun 27 22:19:02 2017
From: dyoung at redhat.com (Dave Young)
Date: Wed, 28 Jun 2017 13:19:02 +0800
Subject: [PATCH] vmcore-dmesg: avoid allocating large memory chunk for
 log buf
In-Reply-To: <1497910994-15129-1-git-send-email-xiyou.wangcong@gmail.com>
References: <1497910994-15129-1-git-send-email-xiyou.wangcong@gmail.com>
Message-ID: <20170628051902.GA1719@dhcp-128-65.nay.redhat.com>

On 06/19/17 at 03:23pm, Cong Wang wrote:
> In kdump kernel, we got:
> 
>     Saving vmcore-dmesg.txt
>     Failed to malloc 67108864 bytes for the logbuf: Cannot allocate memory
> 
> Apparently 64M is too luxury for a kdump kernel which only has 128M
> in total.
> 
> We can avoid allocating such a large memory chunk, instead just allocate
> a smaller chunk inside the loop each time.
> 
> Verify the result by comparing the vmcore-dmesg output with and without
> this patch.
> 
> Cc: Simon Horman <horms at verge.net.au>
> Signed-off-by: Cong Wang <xiyou.wangcong at gmail.com>
> ---
>  vmcore-dmesg/vmcore-dmesg.c | 101 ++++++++++++++++++--------------------------
>  1 file changed, 41 insertions(+), 60 deletions(-)
> 
> diff --git a/vmcore-dmesg/vmcore-dmesg.c b/vmcore-dmesg/vmcore-dmesg.c
> index a8f56df..593557c 100644
> --- a/vmcore-dmesg/vmcore-dmesg.c
> +++ b/vmcore-dmesg/vmcore-dmesg.c
> @@ -565,53 +565,12 @@ static inline uint64_t struct_val_u64(char *ptr, unsigned int offset)
>  	return(file64_to_cpu(*(uint64_t *)(ptr + offset)));
>  }
>  
> -/* human readable text of the record */
> -static char *log_text(char *msg)
> -{
> -	return msg + log_sz;
> -}
> -
> -/* get record by index; idx must point to valid msg */
> -static char *log_from_idx(char *log_buf, uint32_t idx)
> -{
> -	char *msg = log_buf + idx;
> -
> -	/*
> -	 * A length == 0 record is the end of buffer marker. Wrap around and
> -	 * read the message at the start of the buffer.
> -	 */
> -	if (!struct_val_u16(msg, log_offset_len))
> -		return log_buf;
> -	return msg;
> -}
> -
> -/* get next record; idx must point to valid msg */
> -static uint32_t log_next(char *log_buf, uint32_t idx)
> -{
> -	char *msg = log_buf + idx;
> -	uint16_t len;
> -
> -	/* length == 0 indicates the end of the buffer; wrap */
> -	/*
> -	 * A length == 0 record is the end of buffer marker. Wrap around and
> -	 * read the message at the start of the buffer as *this* one, and
> -	 * return the one after that.
> -	 */
> -	len = struct_val_u16(msg, log_offset_len);
> -	if (!len) {
> -		msg = log_buf;
> -		return struct_val_u16(msg, log_offset_len);
> -	}
> -	return idx + len;
> -}
> -
>  /* Read headers of log records and dump accordingly */
>  static void dump_dmesg_structured(int fd)
>  {
>  #define OUT_BUF_SIZE	4096
>  	uint64_t log_buf, log_buf_offset, ts_nsec;
>  	uint32_t log_first_idx, log_next_idx, current_idx, len = 0, i;
> -	int log_buf_len;
>  	char *buf, out_buf[OUT_BUF_SIZE];
>  	ssize_t ret;
>  	char *msg;
> @@ -659,34 +618,33 @@ static void dump_dmesg_structured(int fd)
>  	}
>  
>  	log_buf = read_file_pointer(fd, vaddr_to_offset(log_buf_vaddr));
> -	log_buf_len = read_file_s32(fd, vaddr_to_offset(log_buf_len_vaddr));
>  
>  	log_first_idx = read_file_u32(fd, vaddr_to_offset(log_first_idx_vaddr));
>  	log_next_idx = read_file_u32(fd, vaddr_to_offset(log_next_idx_vaddr));
>  
>  	log_buf_offset = vaddr_to_offset(log_buf);
>  
> -	buf = calloc(1, log_buf_len);
> +	buf = calloc(1, log_sz);
>  	if (!buf) {
> -		fprintf(stderr, "Failed to malloc %d bytes for the logbuf:"
> -				" %s\n", log_buf_len, strerror(errno));
> +		fprintf(stderr, "Failed to malloc %" PRId64 " bytes for the log:"
> +				" %s\n", log_sz, strerror(errno));
>  		exit(64);
>  	}
>  
> -	ret = pread(fd, buf, log_buf_len, log_buf_offset);
> -	if (ret != log_buf_len) {
> -		fprintf(stderr, "Failed to read log buffer of size %d bytes:"
> -			" %s\n", log_buf_len, strerror(errno));
> -		exit(65);
> -	}
> -
>  	/* Parse records and write out data at standard output */
>  
>  	current_idx = log_first_idx;
>  	len = 0;
>  	while (current_idx != log_next_idx) {
> -		msg = log_from_idx(buf, current_idx);
> -		ts_nsec = struct_val_u64(msg, log_offset_ts_nsec);
> +		uint16_t loglen;
> +
> +		ret = pread(fd, buf, log_sz, log_buf_offset + current_idx);
> +		if (ret != log_sz) {
> +			fprintf(stderr, "Failed to read log of size %" PRId64 " bytes:"
> +				" %s\n", log_sz, strerror(errno));
> +			exit(65);
> +		}
> +		ts_nsec = struct_val_u64(buf, log_offset_ts_nsec);
>  		imaxdiv_sec = imaxdiv(ts_nsec, 1000000000);
>  		imaxdiv_usec = imaxdiv(imaxdiv_sec.rem, 1000);
>  
> @@ -695,9 +653,22 @@ static void dump_dmesg_structured(int fd)
>  			(long long unsigned int)imaxdiv_usec.quot);
>  
>  		/* escape non-printable characters */
> -		text_len = struct_val_u16(msg, log_offset_text_len);
> +		text_len = struct_val_u16(buf, log_offset_text_len);
> +		msg = calloc(1, text_len);
> +		if (!msg) {
> +			fprintf(stderr, "Failed to malloc %u bytes for log text:"
> +				" %s\n", text_len, strerror(errno));
> +			exit(64);
> +		}
> +
> +		ret = pread(fd, msg, text_len, log_buf_offset + current_idx + log_sz);
> +		if (ret != text_len) {
> +			fprintf(stderr, "Failed to read log text of size %u bytes:"
> +				" %s\n", text_len, strerror(errno));
> +			exit(65);
> +		}
>  		for (i = 0; i < text_len; i++) {
> -			unsigned char c = log_text(msg)[i];
> +			unsigned char c = msg[i];
>  
>  			if (!isprint(c) && !isspace(c))
>  				len += sprintf(out_buf + len, "\\x%02x", c);
> @@ -711,11 +682,21 @@ static void dump_dmesg_structured(int fd)
>  		}
>  
>  		out_buf[len++] = '\n';
> -
> -		/* Move to next record */
> -		current_idx = log_next(buf, current_idx);
> +		free(msg);
> +		/* length == 0 indicates the end of the buffer; wrap */

Since the code is restructured, above line can be dropped, other wise:

Reviewed-by: Dave Young <dyoung at redhat.com>

> +		/*
> +		 * A length == 0 record is the end of buffer marker. Wrap around and
> +		 * read the message at the start of the buffer as *this* one, and
> +		 * return the one after that.
> +		 */
> +		loglen = struct_val_u16(buf, log_offset_len);
> +		if (!loglen)
> +			current_idx = log_first_idx;
> +		else
> +			/* Move to next record */
> +			current_idx += loglen;
>  	}
> -
> +	free(buf);
>  	if (len)
>  		write_to_stdout(out_buf, len);
>  }
> -- 
> 2.5.5
> 
> 
> _______________________________________________
> kexec mailing list
> kexec at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/kexec

Thanks
Dave
> 
> 


From panand at redhat.com  Tue Jun 27 22:27:58 2017
From: panand at redhat.com (Pratyush Anand)
Date: Wed, 28 Jun 2017 10:57:58 +0530
Subject: [PATCH] vmcore-dmesg: avoid allocating large memory chunk for log
 buf
In-Reply-To: <1497910994-15129-1-git-send-email-xiyou.wangcong@gmail.com>
References: <1497910994-15129-1-git-send-email-xiyou.wangcong@gmail.com>
Message-ID: <2550dd3e-4473-aef8-41cf-4bfe1bb930df@redhat.com>



On Tuesday 20 June 2017 03:53 AM, Cong Wang wrote:
> In kdump kernel, we got:
>
>     Saving vmcore-dmesg.txt
>     Failed to malloc 67108864 bytes for the logbuf: Cannot allocate memory
>
> Apparently 64M is too luxury for a kdump kernel which only has 128M
> in total.
>
> We can avoid allocating such a large memory chunk, instead just allocate
> a smaller chunk inside the loop each time.

Nice improvement.

>
> Verify the result by comparing the vmcore-dmesg output with and without
> this patch.

Yes, it works.

Modification looks fine to me.

Reviewed-by: Pratyush Anand <panand at redhat.com>




From thomas.lendacky at amd.com  Wed Jun 28 07:05:44 2017
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Wed, 28 Jun 2017 09:05:44 -0500
Subject: [PATCH v8 RESEND 27/38] iommu/amd: Allow the AMD IOMMU to work
 with memory encryption
In-Reply-To: <20170628093627.GD14532@8bytes.org>
References: <20170627150718.17428.81813.stgit@tlendack-t1.amdoffice.net>
 <20170627151230.17428.75281.stgit@tlendack-t1.amdoffice.net>
 <20170628093627.GD14532@8bytes.org>
Message-ID: <03a9d0f5-1f0e-ca0c-78a4-c3a7242f6935@amd.com>

On 6/28/2017 4:36 AM, Joerg Roedel wrote:
> Hi Tom,

Hi Joerg,

> 
> On Tue, Jun 27, 2017 at 10:12:30AM -0500, Tom Lendacky wrote:
>> ---
>>   drivers/iommu/amd_iommu.c       |   30 ++++++++++++++++--------------
>>   drivers/iommu/amd_iommu_init.c  |   34 ++++++++++++++++++++++++++++------
>>   drivers/iommu/amd_iommu_proto.h |   10 ++++++++++
>>   drivers/iommu/amd_iommu_types.h |    2 +-
>>   4 files changed, 55 insertions(+), 21 deletions(-)
> 
> Looks like a straightforward change. Just one nit below.
> 
>> +static bool amd_iommu_supports_sme(void)
>> +{
>> +	if (!sme_active() || (boot_cpu_data.x86 != 0x17))
>> +		return true;
>> +
>> +	/* For Fam17h, a specific level of support is required */
>> +	if (boot_cpu_data.microcode >= 0x08001205)
>> +		return true;
>> +
>> +	if ((boot_cpu_data.microcode >= 0x08001126) &&
>> +	    (boot_cpu_data.microcode <= 0x080011ff))
>> +		return true;
>> +
>> +	pr_notice("AMD-Vi: IOMMU not currently supported when SME is active\n");
>> +
>> +	return false;
>> +}
> 
> The name of the function is misleading. It checks whether the IOMMU can
> be enabled when SME is active. But the name suggests that it checks
> whether the iommu hardware supports SME.
> 
> How about renaming it to amd_iommu_sme_check()?

Can do.

Thanks,
Tom

> 
> With that change the patch is:
> 
> 	Acked-by: Joerg Roedel <jroedel at suse.de>
> 


From ats-kumagai at wm.jp.nec.com  Wed Jun 28 17:13:52 2017
From: ats-kumagai at wm.jp.nec.com (Atsushi Kumagai)
Date: Thu, 29 Jun 2017 00:13:52 +0000
Subject: [Makedumpfile PATCH V3] elf_info: fix file_size if segment is
 excluded
In-Reply-To: <dbaed487-b5cf-bf09-bcc0-4e638194f6a7@redhat.com>
References: <648cd9280e333c1b78584afb3f0843434640fe14.1494390678.git.panand@redhat.com>
 <0910DD04CBD6DE4193FCF86B9C00BE9701ED845A@BPXM01GP.gisp.nec.co.jp>
 <8991bcd8-32ed-f7f1-64f4-86858ebb76f7@redhat.com>
 <0910DD04CBD6DE4193FCF86B9C00BE9701EDC3E3@BPXM01GP.gisp.nec.co.jp>
 <dbaed487-b5cf-bf09-bcc0-4e638194f6a7@redhat.com>
Message-ID: <0910DD04CBD6DE4193FCF86B9C00BE9701EEC174@BPXM01GP.gisp.nec.co.jp>

>Hi Atsushi,
>
>On Wednesday 31 May 2017 06:19 AM, Atsushi Kumagai wrote:
>>> Hi Atsushi,
>>>
>>> On Wednesday 17 May 2017 01:15 PM, Atsushi Kumagai wrote:
>>>> Hello Pratyush,
>>>>
>>>> Sorry for late reply.
>>>> I have no further comment for now, but the current --mem-usage
>>>> seems have some problems. I can't decide that this patch is insufficient or
>>>> that's quite a different matter now, so please let me defer acknowledgment
>>>> for a while.
>>>>
>>>
>>> Any decision on this patch.
>>
>> The problem I met is still unresolved, but it occurs even without this patch.
>> So I accept this patch for now.
>>
>
>This patch is missing in your devel branch.
>
>Just a gentle reminder, so that it does not miss 1.6.2.

Thank you for informing me, I pushed it.

Thanks,
Atsushi Kumagai



From abrs612 at gmail.com  Fri Jun 30 08:51:38 2017
From: abrs612 at gmail.com (Abhishek Shah)
Date: Fri, 30 Jun 2017 21:21:38 +0530
Subject: makedumpfile: support for newer kernels [v4.9 onwards]
In-Reply-To: <CAEzMDiBU98VzS9Yye6UrCVM5-f9F64ZUqCOJPpnd+5ufdY5OsA@mail.gmail.com>
References: <CAEzMDiD-w91Q+Fjy_KgXDcwo9YGasa9ggqW2dEhF4ffV7RstqQ@mail.gmail.com>
 <0bd0a35a-88cd-3f68-846f-2a2ad1e7afcb@redhat.com>
 <CAEzMDiBKdA15Gn2kgs=BgzCM=bzaBXCQP_tM_Vmfc4uxGL0SRg@mail.gmail.com>
 <0ae1e066-821e-3b11-ee44-b615245cb1ea@redhat.com>
 <3b72a75c-012e-2779-74a4-d36ca1ef9e56@redhat.com>
 <CAEzMDiARnp-FMqUcoVdCBSSApaRbenvrk7cawgyo8kJJvy0jXg@mail.gmail.com>
 <364d6133-158b-0f28-0278-dc0da33b2916@redhat.com>
 <CAEzMDiAq1U3SRkvwh5nqETDBoZs8DojhmMTDSG7B-Uzbh1a3Nw@mail.gmail.com>
 <148b498a-cd13-bcb2-de04-d74b1c2a0f65@redhat.com>
 <CAEzMDiBU98VzS9Yye6UrCVM5-f9F64ZUqCOJPpnd+5ufdY5OsA@mail.gmail.com>
Message-ID: <CAEzMDiAJwLXHRqF9whvRoXw3Va47X-2M1=v0cTK6RqvMZLb91Q@mail.gmail.com>

Hi All,

I was finally able to root cause the issue that I was facing in the
kdump compressed output
generated by makedumpfile.

While debugging "write_kdump_pages_and_bitmap_cyclic"; I came to know that
somehow "info->max_mapnr" was modified/ reduced.

Seeing further, I could see that the "get_mem_map" function at the end
[if (!is_xen_memory())]
is attempting to change "info->max_mapnr" value based on "mem=" option.
-- in particular commit ebe2fa3a5c1e2bfac3778c0fbeca4287a934cc58.
And in my kernel command line, "mem=2G" was present, as I intended to
take dump of 2G RAM,
resulting into reduced "info->max_mapnr".

So, I commented out this portion and confirmed that "info->max_mapnr"
remains same
throughout. And I was able to get kdump compressed vmcore process-able
by crash utility.

Just to summarize:
Following was the problem statement: crash was giving "seek error"
while makedumpfile generated kdump-compressed vmcore was getting analyzed.
Details:
> The output generated by makedumpfile gets processed by crash utility
> if I use following command:
> makedumpfile -E /proc/vmcore vmcore
>
> But, the output generated by following command (which I used while
> reporting the issue) gives the "seek error" while being processed by
> crash.
> makedumpfile /proc/vmcore vmcore
>
> (As a result?), output from following command gives "seek error" as
> well while being processed by crash:
> makedumpfile -c /proc/vmcore vmcore

Following is my setup details:
>>> Kernel config:
>>> ARM64_VA_BITS_48 [=y]
>>> ARM64_PAGE_SHIFT [=12]
>>> RANDOMIZE_BASE [=n]
>>> ARM64_4K_PAGES [=y]

> kernel: 4.12- rc5
> yocto based rootfs: Pyro[2.3] which has kexec 2.0.14 and makedumpfile 1.6.1.
> crash: 7.1.9++ [Head: 4d517ad28acd845fe6e91360e645cf0446a4757b]


Thanks,
Abhishek


