Redis memory issue (Kali Linux 2021.2)

[m-1-k-3]

New Kali Linux is available. We need to test everything on it

https://www.kali.org/get-kali/#kali-platforms

[m-1-k-3]

In my first installation I had some issues with the cve-database. It needs a complete drop and repopulation:

└─$ sudo ./external/cve-search/sbin/db_updater.py -f

afterwards it was running as expected. Could someone verify this?

I have included this update to our installer: 2c26077

[Humberto-Davila]

It last just 20 minutes , after crashing multiple times.

redis-server has a maxmemory value of 0, also is not auto started.

'echo madvise > /sys/kernel/mm/transparent_hugepage/enabled'
'To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf / reboot

supervised by systemd - you MUST set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit.

[p4cx]

Thank you for your feedback and your solution. Can you please give us some information about your setup? I also had issues with Redis once and solved it by increasing the available storage for my Kali VM from 4GB to 8GB. But I don't know if it's the same problem, so I need additional information from you.

[Humberto-Davila]

Thank you for your feedback and your solution. Can you please give us some information about your setup? I also had issues with Redis once and solved it by increasing the available storage for my Kali VM from 4GB to 8GB. But I don't know if it's the same problem, so I need additional information from you.

The issue is that redis-server crashes all other process because of the huge memory pages, you cannot limit it because you will broke a dependency method, I tried 5, 8 , 12 GB ram and it is worst 2GB Ram is enough, just set transparent_hugepage to
'madvise' or better to 'never' , do not give permission to make a giant RAM DRIVE to redis.

last time I used transparent_hugepage was in red hat 5

Kali Rolling (2021.2) x64 2021-05-31 on WorkStation 16.x VM

Just disable permanently by creating a service.

$sudo nano /etc/systemd/system/disable-thp.service

[Unit]
Description=Disable Transparent Huge Pages (THP)

[Service]
Type=simple
ExecStart=/bin/sh -c "echo 'never' > /sys/kernel/mm/transparent_hugepage/enabled && echo 'never' > /sys/kernel/mm/transparent_hugepage/defrag"

[Install]
WantedBy=multi-user.target

$ sudo systemctl daemon-reload
$ sudo systemctl start disable-thp
$ sudo systemctl enable disable-thp

Both Mongo and Redis has the issue with hugepages. so be sure that DBs start after disabling the pages and its defrag.

Kind Regards.

[m-1-k-3]

As far as I can see the redis server is started during the installation process (see

emba/installer.sh

Line 530 in 849eba6

/etc/init.d/redis-server start

). Additionally the update script is also able to start the redis server (see

emba/config/cve_database_updater.init

Line 8 in 849eba6

/etc/init.d/redis-server start

). During normal operation redis server is not needed and does not need to be started:

[m-1-k-3]

2GB of memory is not a lot for emba. Probably we should add a wiki section with hardware requirements.

[Humberto-Davila]

pmd_set_huge: Cannot satisfy [mem 0xf0000000-0xf0200000] with a huge-page mapping due to MTRR override. Out of memory: Killed process 1032607 (cwmpClient) total-vm:2513456kB, anon-rss:871180kB, file-rss:4kB, shmem-rss:0kB, UID:0 pgtables:1828kB oom_score_adj:0 Out of memory: Killed process 640 (mongod) total-vm:2275988kB, ****/ redis-server invoked oom-killer: gfp_mask=0x100cca(GFP_HIGHUSER_MOVABLE) /***** *oom*-*killer* is a process that the kernel will call when a system is over committed on memory. This will only be *invoked* if your system has exhausted both physical and swap memory. Xorg invoked oom-killer Out of memory: Killed process 1035790.... and so on....... after the service install is running smooth [image: image.png] Kind regards

…

On Thu, 8 Jul 2021 at 02:37, Michael Messner ***@***.***> wrote: As far as I can see the redis server is started during the installation process (see https://github.com/e-m-b-a/emba/blob/849eba667a52db7e769d978658b66aa4e75c203c/installer.sh#L530). Additionally the update script is also able to start the redis server (see https://github.com/e-m-b-a/emba/blob/849eba667a52db7e769d978658b66aa4e75c203c/config/cve_database_updater.init#L8). During normal operation redis server is not needed and does not need to be started: [image: image] <https://user-images.githubusercontent.com/497520/124881707-f49baf00-dfcf-11eb-9903-adc52f0131a8.png> — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#118 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AUILWT2I3INWIKWX7FFQIDDTWVITFANCNFSM46FY5HCQ> .

[Humberto-Davila]

I know , I was procrastinating about doing the GUI to manage binwalk and some tools, virtualization in my experience must be done according to the binaries to emulate, 32/64 bit, also realtek has in the latest Kernel virtualization and emulation detection and the process starts crashing and dumping memory, so it creates a crash on host. I did this work manually and just can say that the report contains the data I found and of course dozens more. Thank you for your work on this. If you need a hand or are stuck on something just drop an email. Regards La informacion contenida en este correo electronico y cualquier documento adjunto es confidencial y puede tener derechos reservados o ser propiedad intelectual. Si usted no es el destinatario usted no esta autorizado a usar o discutir esta informacion, por lo tanto se le pide notificarnos via correo electronico o por telefono y borrar el mensaje de su sistema de correo. The information contained in this email and any attachments is confidential and may be subject to copyright or other intellectual property protection. If you are not the intended recipient, you are not authorized to use or disclose this information, and we request that you notify us by reply mail or telephone and delete the original message from your mail system.

…

On Thu, 8 Jul 2021 at 02:38, Michael Messner ***@***.***> wrote: 2GB of memory is not a lot for emba. Probably we should add a wiki section with hardware requirements. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#118 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AUILWT4PV6NFUY7USYWSYWTTWVIXLANCNFSM46FY5HCQ> .

[m-1-k-3]

I have added a quick prerequisits section to our wiki: https://github.com/e-m-b-a/emba/wiki/Installation#prerequisites