Openelec Build for OPI PC and 2 now with HW decoding

hojnikb

Well, i've formatted to ext3 but issue still persists. Although i didn't test with movies playing...

Could kswapd be disabled entirely ? I mean, do we really need swap on openelec ?

chaszim

All the Linux PC systems I've used in the past, had a swap file equal to 2 times the system ram. So, I didn't know we had an actual swap file in the system.
This page defines what KSWAPD is suppose to do in a standard PC, in reguards  to the writing to the swap file http://www.science.unitn.it/~fiorella/guidelinux/tlk/node39.html
A little above my pay grade, but, it's talking about what the Damon does when cached pages value, get close to low.Only thing I can think of is someone (more knowledgeable) should open KSWAPD up, and see what calls it's making, and make a decission from there.  

I'm adding as I read up on the problem,
so here's another explaination that might fit.
https://lwn.net/Articles/550410/
patch-fix-kswapd-mod-timer-race-condition
http://ck.vds.kolivas.narkive.co ... imer-race-condition
in this patch example, Kswapd once awakened doesn't know when to quit. The patch has a lot info embeded
http://ftp.dei.uc.pt/pub/linux/k ... 9/free_pages-vmstat
throttling kswapd
https://www.kernel.org/pub/linux ... ottling-fixes.patch
linux kswapd linux-ext4 patch
http://lists.openwall.net/linux-ext4/2011/05/18/78
mm-fix-lost-kswapd-wakeup-in-kswapd_stop.patch
https://github.com/Angstrom-dist ... n-kswapd_stop.patch
shrink high memory pages on kswapd pages
https://gitlab.com/k2wl/g2_kerne ... 499e1ca9ce653d701ea
and here's what IBM has to say about it.
http://www-01.ibm.com/support/docview.wss?uid=swg21665911


So I hope there's enough information in these links to find the right settings for kswapd and the programs that call it.

lymon

chaszim replied at 2016-1-2 06:35
All the Linux PC systems I've used in the past, had a swap file equal to 2 times the system ram. So ...

It is not advised to have/use a swap-file on a system that runs of a SD/SSD as it will wear out your SD/SSD rather fast.

Regarding the kswapd bug , I'm not sure if there's a lot of interest to patch up an already criple kernel, also because there seems to be some actual progress in mainline support for the H3
(http://www.orangepi.org/orangepibbsen/forum.php?mod=viewthread&tid=119&extra=page%3D1&page=7)

Also Openelec doesn't really seem to suffer from the current kswapd bug taking out one core.
In the rare cases where Kswapd seems to run normal there isn't a  performance boost or something...you don't even notice it.


> patch-fix-kswapd-mod-timer-race-condition
>in this patch example, Kswapd once awakened doesn't know when to quit. The patch has a lot info embeded
> throttling kswapd

these were patches for the even older kernel 2.6.x

> linux kswapd linux-ext4 patch

I already tried this one earlier, but unfortunately it doesn't make any difference or work for us.

> mm-fix-lost-kswapd-wakeup-in-kswapd_stop.patch

This one is already present in "our" v3.4 vmscan.c  (...but obviously doesn't work for us)

> shrink high memory pages on kswapd pages

this one looked promising and interesting, so I tried it ...but again...no difference and it is not fixing the problem for us

> and here's what IBM has to say about it.

their proposed "fix" of dropping caches doesn't  do anything

a "fix" that "kinda" works ...most of the time (for me at least) is converting the storage partition to EXT3 and adding this to autostart:

1. /usr/sbin/swapoff -a
   
2. echo 1 >/proc/sys/vm/swappiness
   
3. echo 1 >/proc/sys/vm/vfs_cache_pressure

Copy code

Another "fix" that "kinda" works ...most of the time.. is to restart kodi (via ssh)

1. systemctl restart kodi

Copy code

runnerway

Hi jernej! Wishing you a good year

Did you make any progress on proper renderer?

I watched a lot of movies without big problems.
I confirm that kswapd using 100% of one core is not a big deal for performance.
Maybe when it will be solved 100% we will get a bit less hot CPU but again, i don't think it's a big problem as of now.

I think i noticed some very little freezes in some movies..Probably they are related to threads and proper renderer with V-Sync.

jernej

runnerway replied at 2016-1-4 19:49
Hi jernej! Wishing you a good year

Did you make any progress on proper renderer?

Happy new year to you too!

Nope, proper renderer needs a lot of research, for which I currently don't have time.

On related note, I was going through ion initialization code and found weird stuff around that in arch/arm/mach-sunxi/sun8i.c. For me, it seems very likely that kswapd bug is not bug in vmscan but in memory initialization code. What I currently found out is that reserved memory by ion is never reported to system as reserved. If I lowered amount of reserved memory by ion to 160MB, kswapd behaved much  better. However, in that case CedarX run out of memory for 4K videos.

Can someone check if disabling CONFIG_ION for some other distribution helps? If someone would like to play with amount of memory reserved by ion, CONFIG_ION_SUNXI_RESERVE_LIST must be changed. Be aware that only second value will have effect, e.g. CONFIG_ION_SUNXI_RESERVE_LIST="160M@0,256M@0,130M@1,200M@1" <- 256MB of memory reserved

chaszim

lymon replied at 2016-1-3 15:18
It is not advised to have/use a swap-file on a system that runs of a SD/SSD as it will wear out you ...

@lymon
I never suggusted we add a swap file into the mix, I simply said, all linux systems I seen and used, had one. There's a guy on Facebook says, "Kswapd's cure, is to use the higher quality SD card, by Sandisk (the gray and red one)".  

jernej

Currently, swap is not enabled and I don't have a plan to enable it.

lymon

jernej replied at 2016-1-5 22:08
Happy new year to you too!

Nope, proper renderer needs a lot of research, for which I currently d ...

can not get the kernel to compile correctly when disabling the config_ion/config_sunxi_ion in the kernel-config.
there seem to be too many dependencies (eg. graphics + hdmi drivers)

Also experimented a bit with the  amount of reserved memory for ion, but with anything over 160 Mb kswapd starts acting up again.
However I don't experience any issues with 160 Mb reserved memory and playing 4K videos (or any other type of videos for that matter) ?

> What I currently found out is that reserved memory by ion is never reported to system as reserved.

I noticed there's a small difference in sun8i.c on the allwinner github and the sun8i.c used in the linux-3.4 sources for the OPI :
https://github.com/allwinner-zh/linux-3.4-sunxi/blob/master/arch/arm/mach-sunxi/sun8i.c
check the "ion_reserve_common" function
Maybe a step in the right direction ?

patch :  https://github.com/lymon66/ORANGEPI-PC/blob/master/Openelec/linux-04-sun8i.patch


..I wonder if there are any other changes

jernej

lymon replied at 2016-1-6 13:38
can not get the kernel to compile correctly when disabling the config_ion/config_sunxi_ion in the k ...

That difference doesn't change anything important, just parsing memory string. There is a lot of spaghetti code around ion so there is no easy way to disable it. I will try to do some hack just to check if there would be a difference to use CMA allocated contiguous memory instead of carveout memory (reserved at boot in sun8i.c). Naturally, CedarX's libMemAdapter.so would need a patch if everything works. Patching is not a big deal, but there is no guarantee that libMemAdapter will easily work that way either.

Can you please try this video https://app.box.com/s/qkhfruhz1xv9xq7dqfcfv6x262wkogw9 with 160m? Even though it says 120 fps it is 30 fps (check with ffmpeg, vlc or similar).

Is there anyone who knows something about CMA and/or ION's carveout? Should be memory reserved for those two mechanisms always threated as used, e.g. when checking with command "free"?

EDIT: current findings:
- Carveout memory region cannot be easily disabled because of kernel freeze (currently unknown reason)
- it can be minimized at least to 24MB (16MB doesn't work anymore), kswapd is not seen in "top" command
- CMA can be extended to 256MB, but not sure if really works with such amount of memory
- found some comment in arch/arm/mach-sunxi/include/mach/memory.h which is contradictory to the current status (carveout memory region is last 256MB of memory):

1. /*
   
2. * NOTE: CMA reserved area: (CONFIG_CMA_RESERVE_BASE ~ CONFIG_CMA_RESERVE_BASE + CONFIG_CMA_SIZE_MBYTES),
   
3. *   which is reserved in the function of dma_contiguous_reserve, drivers/base/dma-contiguous.c.
   
4. *   Please DO NOT conflict with it when you reserved your own areas.
   
5. *
   
6. *   We need to restrict CMA area in the front 256M, because VE only support these address.
   
7. */

Copy code

- sunxi-libvdpau uses CMA for memory allocation, so in theory should work (?)

lymon

tried your video and that one won't play with 160Mb and after trying to play your video it won't play any of my other 4K videos and I have to restart kodi
..Could it be there's something wrong/corrupt with your 4K video ?...as it won't play (at all or extremely laggy) on any of the other mediaplayers I have ?


edit: redownloaded the video and now it's working on other mediaplayers..so must have been a corrupt download or something.
anyway...on the OPI it still doesn't play with 160MB reserved mem.
I tried a couple of settings and now have the following reserve_list: 64M@0,164M@0,130M@1,200M@1 (not exactly sure what the first value in the array is for, but it definately seems to make a difference)
With these settings your video works, but kswapd starts acting up again after playing your or a couple of other 4K videos (but not always).
However when playing lower resolution videos I can't reproduce the kswapd issue and it stays away...so it only seems to happen with some UHD/4K videos.

...so maybe a temp fix for now ? ... 64/164Mb reserved memory and have the kswapd issue fixed for 95% (most of the time and for most videos)
Although I am not sure if this works for others also, so maybe others can test this as well ?

PS: do you know what the other values in the CONFIG_ION_SUNXI_RESERVE_LIST array are used for ?
I can't find any info or documentation about it