[Mod] free_lj_mem 1.4 (31.03.2019)

[Festus1965]

[Mod] free_lj_mem

Version: 1.4
new: 2nd status with settings at start, 2 more var to remember highest/lowest lua memory / setting for loop-time

last update: 31.03.2019

no dependencies

tested on mts 5.0.0, but there is nothing used, should also run under 0.4.x

Licence: LGPL v3.0

Download / Source at: https://github.com/Minetest-One/free_lj_mem

What does it:
* automatic collectgarbage() every x time (default set to 3600 = 1 hours)

* depend on minetest.conf debug_log_level (warning, action, info) reports on reduce memory usage, until warning message to admin as of name, or minetest.log

* depend on minetest.conf debug_log_level (warning, action, info) print to log / terminal / player when a lua maximal memory (value to set) is reached even after a collectgarbage().
- report that actual memory usage is higher than set max memory
- setting of maximum lua memory depends on lua 2 GB or luajit 1 GB and also on other unknown facts
- set maxluamem can start with max of lua 2048000 or 1024000 , and after got an OOM should be set about 10% lower, so admin might get aware, OOM getting close

* detect kind and version of used lua / luajit, and post report at loading

* give status at loading depend on minetest.conf debug_log_level that loaded (+ amount of memory is used after this have been loaded)

start on test-server (5.0.0)

Code: Select all

[Mod] free_lj_mem detected: LuaJIT 2.1.0-beta3
[Mod] free_lj_mem settings: 60 sec / warn: 2000 KB
[Mod] free_lj_mem (1.40 31.03.2019) - loaded - end mem : 1016 KB

start on main server (5.0.0)

Code: Select all

free_lj_mem detected LuaJIT 2.0.5
[Mod] free_lj_mem settings: 1800 sec / warn: 150000 KB
[Mod] free_lj_mem (1.4 - 31.03.2019) - loaded - end mem : 56413 KB

and test when running, depends on destination

Code: Select all

[MOD] free_lj_mem: cleaned 2157 to 2088 (max: 2590 / min: 2088) KB

or the warning in minetest.log

Code: Select all

2019-03-29 06:31:50: ACTION[Server]: [MOD] free_lj_mem detected luamem warning: 10490 > 10000 !! - near OOM

or at terminal / chat to player

Code: Select all

[MOD] free_lj_mem warning: 2088 actual more memory used then set max: 2000 KB ! possible OOM

... this is my first mod, report issues here

* as this should sole the problems we faced yet with [urlhttps://forum.minetest.net/viewtopic.php?f=6&t=21934]OOM[/url]
* the advantage to [Mod] stop_lj_oom is less lag, even report and/or warnings on several channels (I think I will fork that mod also, just make is less time check memory, as it does now)

[Festus1965]

further explanations how to set or calculate

1.4 - 2nd status with settings at start, 2 more var to remember highest/lowest lua memory / setting for loop-time
1.3 - including minetest.after and debug_log_level used
1.2 - including minetest.setting_get("name"), and fix missing file, fix wrong name
1.1 - after I was sure

the version is running on my server "Asia Thailand Minetest.one" since 12 hours, tested on 2nd server, and I am reporting there about its work yet.

[sofar]

the file `lua.ini` should be named `init.lua` instead.

LGPL-3.0 is just fine!

Make a mod.conf file as well...

Don't use a globalstep for something that runs once every hour. Instead, use minetest.after() instead with a timeout value of 1 hour.

Don't hardcode `Admin` name in code. Instead, just use minetest.log() or scan online players that have the `server` priv.

Maybe it's interesting to run the code that looks at memory usage more often so an admin can get an idea about memory usage more often.

Try and use minetest settings instead of hardcoded values, this way everyone can change them.

(Not bad for a first mod!)

[sofar]

Why do you need to depend on `default`? If you don't need it, just write `no dependencies` instead and remove depends.txt.

[Festus1965]

Why do you need to depend on `default`? If you don't need it, just write `no dependencies` instead and remove depends.txt.

oh, I see - not good coding with too less sleep, but cant get rid of this problem with lua mem
* init.lua fixed - shame on me about THIS
* mod.conf created, have to look what was to put in now 5.x
* hardcore might be more safe first, but will look the scan option and learn further
* timestep 1 hour, just testing now 30 minutes on my server, it seams to be good, and can be changed - will depend on first experiences - as I never read about how long servers needed to OOM, jsut I know at me was 8-10 hours
* minetest.settings, ok, get some automatic like name and log, but new in 5.x
* dependencies fixed
* globalstep, better minetest.after reading the wiki = got it, need call itself inside

would so be interesting how much servers use luajit ?
and I hope for replay when tried it out, maybe more can go luajit as so far I see that is most problem

add:
this minetest.after seams to be often, when I remember checking just start code when was editing for the report of lua memory usage ... most I see "minetest.register_globalstep(function(dtime) ... )" with its shown in "profiler save" then

add: got it, Version 1.3 is running

[sofar]

would so be interesting how much servers use luajit ?

since most of the builds we do use luajit, I expect a lot of them.

this minetest.after seams to be often, when I remember checking just start code when was editing for the report of lua memory usage ... most I see "minetest.register_globalstep(function(dtime) ... )" with its shown in "profiler save" then

It's used by many mods and therefore shows up a lot, even though it's not the `after` code itself that is the cause.

[Festus1965]

since most of the builds we do use luajit, I expect a lot of them.

hmm, for that there are 230 servers listed, I see no new posts about OOM.

Also I think it depends on the version of lua, luajit also a bit ... ok we will see, I did my work, and until now on my server with fast luajit it never exceeded 120.000, and I expect OOM again at 500.000 to 850.000 as logged.

But this is so far strange then, as what [Mod]s I took of, be responsible for so much more ... even moreblocks it back true.

[Festus1965]

in work: [Mod] free_lj_mem 1.5 (01.04.2019)
as aware during testing on my main server Asia Thailand Minetest.one after a view hours working I thought about restart to make longer/shorter setting as situation forces an admin and change warning memory size,
so mean, this [Mod] needed to be able to change some settings during run server.

It works on test

* two parameter are also set global (as needed)
-- and read there value out of minetest.conf (if set) also
-- or keep default set value before in [Mod] init.lua
-- but can be changed with /privs server via commend /set name value

* parameters used
-- looptime (how often should be checked - default 1800)
-- maxluamem (limit above warning message start)

test on 3rd server worked fine
will test on main server first also