Transparent Hugepages

Transparent hugepages are enabled on the test platforms by doing these commands as root:

echo always > /sys/kernel/mm/transparent_hugepage/enabled
echo always > /sys/kernel/mm/transparent_hugepage/defrag

To show that the search arrays are actually using hugepages, the following shell script is run after loading dist files into the solver:

ps aux | grep chrome | awk '{print $2}' |
while read pid; do
  grep AnonH /proc/$pid/smaps 2>/dev/null |
  awk '{if ($2 > 4096) print}'
done

An example of the output listing:

/proc/250780/smaps:AnonHugePages:     55296 kB
/proc/250780/smaps:AnonHugePages:     51200 kB
/proc/250780/smaps:AnonHugePages:     40960 kB
/proc/250780/smaps:AnonHugePages:   1867776 kB
/proc/250780/smaps:AnonHugePages:    851968 kB

The above shows that the allocations match the dist file sizes below (from du ‑k) after adjusting as shown.

    File                 Hugepage Allocation
--------------  ------------------------------------
Dist1_09F.dat   floor(58548/2048-1)*2048 = 55296
Dist2_09F.dat   floor(54800/2048-1)*2048 = 51200
Dist5_11F.dat   floor(43084/2048-1)*2048 = 40960
Dist4_11F.dat   floor(1870576/2048-1)*2048 = 1867776
Dist3_10FQ.dat  floor(855124/2048-1)*2048 = 851968

If any of the hugepage allocations are less than the adjusted file size then the following commands can be used (as root) after closing the browser to make more hugepages available.

sync
echo 3 > /proc/sys/vm/drop_caches

More info about that command is here. To clarify, that does not remove the caches it just clears them and they continue to function normally. There is a discussion of that here.

Another command to make more hugepages available is:

echo 1 > /proc/sys/vm/compact_memory

There are many temporary allocations that occur while the dist files are loading, those will be cleared by the JavaScript garbage collector as a normal course of operation. This means that the above shell script may show more allocations than just the search arrays but they should clear eventually.