large pages

[Shelwien]

On win10, Large Pages only work when
1) Program executable is x64
2) Runs under admin
3) Policy allows large pages (“Computer Configuration” , “Windows Settings”, “Security Settings”, “Local Policies” , “User Rights Assignment”, “Lock Pages in memory”)
4) Unfragmented 2MB pages physically exist in memory manager (ie test is done soon after reboot)

z:\021>2mpages.exe

OpenProcessToken: <The operation completed successfully.  >
LookupPrivilegeValue: <The operation completed successfully.  >
AdjustTokenPrivileges: <The operation completed successfully.  >
LPM.size=200000: 
VirtualAlloc flags=20001000: <The operation completed successfully.  >
p=00C00000 Flags=20001000

Z:\021>timetest 7z a -mx1 -md27 -slp -mmt=1 1 "D:\000\enwik8"

7-Zip 19.02 alpha (x64) : Copyright (c) 1999-2019 Igor Pavlov : 2019-09-05
Archive size: 31739530 bytes (31 MiB)
Tested program has wasted 12.563s

Z:\021>timetest 7z a -mx1 -md27 -slp- -mmt=1 2 "D:\000\enwik8"

7-Zip 19.02 alpha (x64) : Copyright (c) 1999-2019 Igor Pavlov : 2019-09-05
Archive size: 31739530 bytes (31 MiB)
Tested program has wasted 15.547s

(15.547/12.563-1)*100 = 23.75%

[necros]

so what exactly does your utility do, sets OS to use 2mb pages?

[Shelwien]

Yes, its tricky enough on windows - there's source in archive.

[necros]

I`m not strong in coding - does app sets it until next reboot system wide?

[Shelwien]

> does app sets it until next reboot system wide

That's impossible - well, Igor Pavlov had a project that attempted to do that - http://sevenmax.sourceforge.net/

Normally 4k and 2M (and 1GB) virtual memory pages are managed separately, and you need a specific allocation method in each case.

[Shelwien]

Here's a linux example.
On linux though, its also necessary to force the system to actually allocate the pages (as root):

echo 16 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages
./2Mlin
echo 0 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages

#include <stdio.h>

#include <string.h>



#include <linux/mman.h>

#include <sys/mman.h>

#include <errno.h>



int main( void ) {



  void* p = 0;

  int N = 16<<20;



  p = 0; errno=0;

  p = mmap( 0, N, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_HUGETLB|MAP_HUGE_2MB, 0, 0);

  printf( "!p = %p  errno=%i <%s>!\n", p, errno, strerror(errno) );



  p = 0; errno=0;

  p = mmap( 0, N, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, 0, 0);

  printf( "!p = %p  errno=%i <%s>!\n", p, errno, strerror(errno) );



}

[necros]

Sorry, still see no answer to what this utility does- 1. just shows info 2. sets OS to use 2mb pages 3. sets OS to use 2mb pages until reboot?

[Shelwien]

Current PC virtual memory architecture supports different page sizes - 4K,2M,1G; https://en.wikipedia.org/wiki/Page_(...age_table_size
Large pages were added in hardware long after the memory management in popular operating systems was already established,
so large pages are managed separately, and are not allocated automatically by default, eg. just because requested memory block is large enough.

> 1. just shows info

2mpages.cpp attempts to acquire permissions and allocate a memory block using large pages.
It does print error messages for called winapi functions, so it can be said that it "shows info".

2Mlin.cpp is just an example of memory allocation using large pages.

> 2. sets OS to use 2mb pages

That's currently impossible without using non-default OS memory manager.

Linux example shows how to reserve large pages via device FS, so they'd stay reserved until next change.
But even then linux won't automatically use them.

> 3. sets OS to use 2mb pages until reboot?

No, these pages have to be allocated in a special way, so I posted examples on how to do it.