mpdPup - Simplified MPD Music Server/Jukebox - v0.9.3

For talk and support relating specifically to Puppy derivatives
Message
Author
DenisP
Posts: 56
Joined: Mon 25 Mar 2013, 22:17
Location: Croatia

Squeezebox server possible?

#441 Post by DenisP »

As a sidenote, since I'm tinkering with various music "appliances", I wondered if it would be possible to compile any kind of squeezebox server, either for mpdPup or any other Puppy variant?
I checked, and there were several questions over the past few years, but it seems no one was able to compile it for Puppy (Perl issues?).

I also saw that Idolse once asked about it in another part of the forum a couple of years back, so I wondered if anything came of it?

I know I can run squeezebox server under some other linux variants, but I would much prefer to use any puppy variant for such a server.

I tried the squeezelite player (http://code.google.com/p/squeezelite/) in mpdpup, and it runs without problems :) It appears as another player in the squeeze network, and I can direct my music to it from the main squeezebox server. So, the player works :)

However, the server is another issue....

Could it be compiled for Puppy at all (this exceeds my abilities, though...)?

Sorry for the digression, this does not relate directly to mpdPup, but since we're already tinkering with music hardware/software, I thought I might as well ask here :)

boulogne75
Posts: 15
Joined: Mon 17 Dec 2012, 13:34

Re: where to put ulimit

#442 Post by boulogne75 »

wlowes wrote:I too have been reading and experimenting where to put ulimit command at startup.

I tried at the end of my startup script which is last in the init.d list. No luck.
Then I tried in the 20.mpd startup right after the renice. Also no luck.

The thing that surprises me most is I leave my system on 7x24. Every evening when I come to listen, ulimit -s & -l are back to the default. I set them to unlimited and the next day they are back to the default.

I can't imagine I get a power failure everyday. Is it possible that logging on to root via ssh or connecting to mpd via a client somehow involkes a script that sets the default?
ulimit xxx has effect only on the current login session and for the logged user. That is why it is reseted every time you ssh.
I can't imagine any positive impact on sound unless you ssh under the user executing mpd.

To make these changes apply on boot and for all users or a specific user you shoul use:
/etc/security/limits.conf

boulogne75
Posts: 15
Joined: Mon 17 Dec 2012, 13:34

#443 Post by boulogne75 »

Hi Idolse,

Do you have an easy way to test a real time kernel?
What is actually the kernel shipped with mpdPup (uname -a). I can't find it and don't have mpdPup installed right now :oops:

Did you make progress with nfs support?

Thanks in advance.
Last edited by boulogne75 on Fri 05 Jul 2013, 20:21, edited 1 time in total.

Dynobot
Posts: 44
Joined: Wed 05 Dec 2012, 12:04

#444 Post by Dynobot »

MPDPup ships with kernel 2.6.39

# uname -r
2.6.39
#

StudioPuppy has a real time kernel but only supports an old version of MPD 15.4

Puppy Linux be default only has one user...root.

Every shell is started by root and PuppyLinux does not have a security/limits.conf file nor a security directory in /etc. For this reason the only way to change limts is via shell or by finding the file that invokes the limits in the first place.

The Realtime kernel has benefits of lower latency [however widely disputed as being for record only]. However the realtime kernel allows scheduling and FIFO preferences that the normal kernel does not.

Ideally repositories would be up to date with the latest versions of products and dependencies will be kept up to date and available as well. I think Idolse went through a monumental effort to get MPDPup up to the level its currently all things considered. Installing even rev 16.x of MPD on a normal distro of Puppy Linux is all but impossible for the avg user.

boulogne75
Posts: 15
Joined: Mon 17 Dec 2012, 13:34

#445 Post by boulogne75 »

Thanks for the reply Dynobot,
Dynobot wrote: MPDPup ships with kernel 2.6.39
# uname -r
2.6.39
#
What about uname -a? That is it give any clue if it is a preempt kernel?
Dynobot wrote: StudioPuppy has a real time kernel but only supports an old version of MPD 15.4
How is it?
Dynobot wrote: Puppy Linux be default only has one user...root.
Every shell is started by root and PuppyLinux does not have a security/limits.conf file nor a security directory in /etc. For this reason the only way to change limts is via shell or by finding the file that invokes the limits in the first place.
Ok. Did I a quick search and all that seems to be related. Only one user. Looks like it is the reason pam is not installed by default.
Can you confirm that there is not any /etc/pam or /etc/pam.d directories? If this is the case is it installable via repositories?
Which is the user running mpd, root?
Dynobot wrote: The Realtime kernel has benefits of lower latency [however widely disputed as being for record only]. However the realtime kernel allows scheduling and FIFO preferences that the normal kernel does not.
Scheduling and FIFO are actually possible with any preempt kernel.
RT kernel only adds true real time preemption for all the kernel, that is even lower latency.
Here are interesting links:
https://rt.wiki.kernel.org/index.php/CO ... T_RT_Patch
https://www.osadl.org/Realtime-Linux.pr ... nux.0.html
http://www.artist-embedded.org/docs/Eve ... riques.pdf

The interesting thing here was to test mpdPup with a preempt kernel, threadirqs enabled and rtirq script. I thought it was more easy :lol:
In my case/setup (arch linux, alix, rtirq, nrpacks=1, no mpd tweaks), I found significant improvements when switching:
stock kernel (PREEMPT,CONFIG_HZ=300) -> custom kernel (PREEMPT,CONFIG_HZ=1000) -> rt kernel (FULL PREEMPT,CONFIG_HZ=1000)
Dynobot wrote: Ideally repositories would be up to date with the latest versions of products and dependencies will be kept up to date and available as well. I think Idolse went through a monumental effort to get MPDPup up to the level its currently all things considered. Installing even rev 16.x of MPD on a normal distro of Puppy Linux is all but impossible for the avg user.
The first time I tried mpdPup I was very surprised with its sound quality and praised ldolse. The main and only reason I moved away from mpdPup was the lack of support for nfs (I don't have cifs anymore).
I greatly appreciate his effort, that is why I regularly keep an eye on this thread.

Dynobot, not sure if there was some misunderstanding in my last post. I hope no, I am just trying to help :wink:

Dynobot
Posts: 44
Joined: Wed 05 Dec 2012, 12:04

#446 Post by Dynobot »

Hi no worries boulogje75, didn't mean to come off the wrong way either.

I appreciate your insights :D

boulogne75
Posts: 15
Joined: Mon 17 Dec 2012, 13:34

#447 Post by boulogne75 »

Dynobot wrote:Hi no worries boulogje75, didn't mean to come off the wrong way either.

I appreciate your insights :D
Great, just wanted to make it clear :D

Dynobot
Posts: 44
Joined: Wed 05 Dec 2012, 12:04

#448 Post by Dynobot »

@boulogje75

With regards to studio4 and MPD ver 15.4. To my ears it sounds just a tad bit thick compared to MpdPup. I tried installing newer versions to no avail.

Linux has a way of sucking you in....in that you start out with a seemingly simple task and 12 hours later you are still at square one. As frustration builds you [..I...] tend to become more determined and exhaust all resources along with myself in the process.

ldolse
Posts: 367
Joined: Fri 23 Oct 2009, 16:33

#449 Post by ldolse »

Regarding the discussion on ulimit - this is starting to sound like a major research project, I'll see what I can find out though - wouldn't be surprised if the defaults are set at compile time for the kernel. I'm pretty sure I don't want to install PAM or any extra background processes to make limits.conf work.

ldolse
Posts: 367
Joined: Fri 23 Oct 2009, 16:33

#450 Post by ldolse »

@DenisP - compiling Squeeze Server is possible and I've done it successfully while exploring how to package it. The real problem is the packaging - it relies heavily on Perl and numerous CPAN packages. Trying to package dozens of Perl dependencies together is a nightmare for which I've found no solution.

DenisP
Posts: 56
Joined: Mon 25 Mar 2013, 22:17
Location: Croatia

#451 Post by DenisP »

ldolse wrote:@DenisP - compiling Squeeze Server is possible and I've done it successfully while exploring how to package it. The real problem is the packaging - it relies heavily on Perl and numerous CPAN packages.
OK, I understand - but I don't need no packaging :)
I just want to install it on ANY Puppy version in any way possible, so I'd appreciate ANY kind of help - e.g. which version of LMS server to use and how to compile and install everything.
Installing it on mpdPup would be awesome - that way I could choose either mpd or LMS as a server upon startup or something.... Add squeezelite as a player, and I could have a standalone squeezebox ecosystem-in-a-box (if desired for connecting to other players on the network - where other "players" could be thin clients with mpdPup installed, but using "squeezelite" as a player in this case).

I've checked the possibility of using mpdPup (i.e. mpd) with several "players" on the network, but it is not possible - at least not in the way LMS (squeezebox server) does it... :(

boulogne75
Posts: 15
Joined: Mon 17 Dec 2012, 13:34

#452 Post by boulogne75 »

ldolse wrote:Regarding the discussion on ulimit - this is starting to sound like a major research project, I'll see what I can find out though - wouldn't be surprised if the defaults are set at compile time for the kernel. I'm pretty sure I don't want to install PAM or any extra background processes to make limits.conf work.
That is how I understand it:
The values set at kernel level define system wide (for all users) limits which can be changed on the fly via sysctl commands. ulimits allow to change them for a user context. So, I am not sure it is good idea to set them in the kernel.

May be you can try this, assuming mpd is running under mpd user:
(it won't work with pam installed but may be without it...)

# su - mpd
# ulimit -a --> confirm 64
# ulimit -l unlimited
# ulimit -a --> confirm unlimited
# exit

Close ssh session and open it again:

# su - mpd
# ulimit -a --> confirm unlimited

If it works you should be able to add it to your rc script.

About PAM, I don't think it runs any background process. IIRC, It is only accessed whith a user login or when a shell is started.

ldolse
Posts: 367
Joined: Fri 23 Oct 2009, 16:33

#453 Post by ldolse »

Was having issues posting responses yesterday, a couple more:

Regarding the questions on the kernel - the specific kernel is 2.6.39-4, and it has the BFS scheduler patches. I'm not sure whether this scheduler contributes to any sound quality perception discussions, but theoretically it sounds like a better deal for audio than the stock scheduler.

@boulogne75 - NFS support still isn't in the wizards, but the system has always supported NFS to the best of my knowledge. I've recently picked up a Synology NAS which supports NFS, so I may dig back into it.

Thanks for the extra info on ulimits.

@DenisP, unfortunately I don't remember exactly what I did to get SqueezeServer up and running - it involved installing CPAN and tons of dependencies (using CPAN) - once all the dependencies were installed I was able to compile/install the SqueezeServer software. I think the dependencies were tracked down by installation failure errors. I just grabbed the latest SqueezeServer package from logitech at that time - this was about 18 months ago.

Dynobot
Posts: 44
Joined: Wed 05 Dec 2012, 12:04

#454 Post by Dynobot »

To achieve lowest latency with BFS

Configure your kernel with 1000Hz, preempt ON and disable dynamic ticks.


With the BFS patch schetool cound be used to give MPD top priority.

From the mouth of the creator of BFS...
"Run your audio and video apps SCHED_ISO:
schedtool -I -e amarok
This will run amarok as an unprivileged real-time task. Note that if you start
an application that tries to get real-time scheduling (eg jackd) and you are
not starting it as root, BFS will automatically elevate it to SCHED_ISO for
you to give it the next best thing."


With BFS, once MPD is given unprivileged real-time priority it will be auto-magically run in a 'core' that give best performance.


From the mouth of the creator of BFS.
"Because BFS uses one global runqueue
for all tasks on the system, when a realtime task wants CPU time, BFS will
find the most suitable CPU anywhere in the system to run it on, by kicking
off the lower priority task running anywhere."



http://ck.kolivas.org/patches/bfs/bfs-faq.txt


Looks like this one command [schedtool -I -e mpd] after schedtool is installed will increase performance.


Update:

Yes schedtool does seem to work as advertised. It can be added to the start up script.

Two birds down via one stone. :D[quote][/quote]

Dynobot
Posts: 44
Joined: Wed 05 Dec 2012, 12:04

#455 Post by Dynobot »

Bug in MPD with wav file for version 17.x

I remember this awhile back and I bought it up to the MPD folks...I found the fix for it back then and will work with MpdPup.

Indeed wav files did not play on my machine with rev 17.x installed. The fix is, you have to insert this line in the mpd.conf file.

decoder {
plugin "sndfile"
enabled "no"
}

Dynobot
Posts: 44
Joined: Wed 05 Dec 2012, 12:04

#456 Post by Dynobot »

Looking at some Kernel tuning from the IBM and Oracle web sites.

I remember earlier we were trying SHIMAX and SHIMIN etc. tuning. The original poster was just throwing numbers in and listening for differences. I'd like to logically tune the kernel to run at optimal per IBM and Oracle's suggestions based on what the actual perimeters do vs. random values.

This may or may not be fruitful but you never know until you try... :)

lugili
Posts: 16
Joined: Sun 24 Feb 2013, 21:07

#457 Post by lugili »

Dynobot,

sounds quite interesting what you are trying here.
I would like to try schedtool. How to install schedtool in mpdpup? I saw that there is also a nice parameter in schedtool, do you think this will make a difference to change that?

Thanks,
lugili

boulogne75
Posts: 15
Joined: Mon 17 Dec 2012, 13:34

#458 Post by boulogne75 »

Dynobot wrote:To achieve lowest latency with BFS

Configure your kernel with 1000Hz, preempt ON and disable dynamic ticks.


With the BFS patch schetool cound be used to give MPD top priority.

From the mouth of the creator of BFS...
"Run your audio and video apps SCHED_ISO:
schedtool -I -e amarok
This will run amarok as an unprivileged real-time task. Note that if you start
an application that tries to get real-time scheduling (eg jackd) and you are
not starting it as root, BFS will automatically elevate it to SCHED_ISO for
you to give it the next best thing."


With BFS, once MPD is given unprivileged real-time priority it will be auto-magically run in a 'core' that give best performance.


From the mouth of the creator of BFS.
"Because BFS uses one global runqueue
for all tasks on the system, when a realtime task wants CPU time, BFS will
find the most suitable CPU anywhere in the system to run it on, by kicking
off the lower priority task running anywhere."



http://ck.kolivas.org/patches/bfs/bfs-faq.txt


Looks like this one command [schedtool -I -e mpd] after schedtool is installed will increase performance.


Update:

Yes schedtool does seem to work as advertised. It can be added to the start up script.

Two birds down via one stone. :D
Hi Dynobot,

"schedtool -I -e mpd" should start a new mpd process with SCHED_ISO class. To change the class of the running mpd process try:
schedtool -I $(pidof mpd)

Anyway, I am not sure schedtool can change the scheduling class of the 4 mpd's sub-threads.
May be you can give a try to this:
chrt -a -xxx -p $(pidof mpd) where xxx is the sched_class

In case SCHED_ISO is not available in chrt, may be you can compare:
"schedtool -I $(pidof mpd)" and "chrt -a -r -p 80 $(pidof mpd)"

("ulimit -a" --> real-time priority will give you the max priority you can assign to RR_SCHED and FIFO_SCHED)

Dynobot
Posts: 44
Joined: Wed 05 Dec 2012, 12:04

#459 Post by Dynobot »

boulogne75 wrote:
Dynobot wrote:To achieve lowest latency with BFS

Configure your kernel with 1000Hz, preempt ON and disable dynamic ticks.


With the BFS patch schetool cound be used to give MPD top priority.

From the mouth of the creator of BFS...
"Run your audio and video apps SCHED_ISO:
schedtool -I -e amarok
This will run amarok as an unprivileged real-time task. Note that if you start
an application that tries to get real-time scheduling (eg jackd) and you are
not starting it as root, BFS will automatically elevate it to SCHED_ISO for
you to give it the next best thing."


With BFS, once MPD is given unprivileged real-time priority it will be auto-magically run in a 'core' that give best performance.


From the mouth of the creator of BFS.
"Because BFS uses one global runqueue
for all tasks on the system, when a realtime task wants CPU time, BFS will
find the most suitable CPU anywhere in the system to run it on, by kicking
off the lower priority task running anywhere."



http://ck.kolivas.org/patches/bfs/bfs-faq.txt


Looks like this one command [schedtool -I -e mpd] after schedtool is installed will increase performance.


Update:

Yes schedtool does seem to work as advertised. It can be added to the start up script.

Two birds down via one stone. :D
Hi Dynobot,

"schedtool -I -e mpd" should start a new mpd process with SCHED_ISO class. To change the class of the running mpd process try:
schedtool -I $(pidof mpd)

Anyway, I am not sure schedtool can change the scheduling class of the 4 mpd's sub-threads.
May be you can give a try to this:
chrt -a -xxx -p $(pidof mpd) where xxx is the sched_class

In case SCHED_ISO is not available in chrt, may be you can compare:
"schedtool -I $(pidof mpd)" and "chrt -a -r -p 80 $(pidof mpd)"

("ulimit -a" --> real-time priority will give you the max priority you can assign to RR_SCHED and FIFO_SCHED)
Hi boulogne75

Personally I like FIFO vs ISO, esp for audio

Here is a link about scheduling

http://ck.wikia.com/wiki/SchedulingPolicies

About the other sub audio threads ksoftirqd/0 through 3. These each run on their own core...hence the 0,1,2,3. Don't think you want to ISO those.

boulogne75
Posts: 15
Joined: Mon 17 Dec 2012, 13:34

#460 Post by boulogne75 »

Dynobot,

I agree with you FIFO should be a better choice. I personally use FIFO for IRQs threads and RR for mpd.
I read the link (and others) some time ago and that is why I use RR. I am almost sure mpd is not coded for pure realtime. The FIFO class require some special coding for resources (threads, cpu, memory. etc...) management (claim, release, use) where the RR class don't.

That is what is running in my htpc (my alix is not connected right now) ordered by effective priority:

Code: Select all

  PID CLS RTPRIO PRI  NI CMD
    8  FF     99 139   - [posixcputmr/0]
    9  FF     99 139   - [migration/0]
   14  FF     99 139   - [posixcputmr/1]
   15  FF     99 139   - [migration/1]
   39  FF     95 135   - [irq/8-rtc0]
  750  RR     90 130   - /usr/bin/mpd --no-daemon
  750  RR     90 130   - /usr/bin/mpd --no-daemon
  750  RR     90 130   - /usr/bin/mpd --no-daemon
  750  RR     90 130   - /usr/bin/mpd --no-daemon
  750  RR     90 130   - /usr/bin/mpd --no-daemon
  189  FF     85 125   - [irq/18-snd_oxyg]
   67  FF     75 115   - [irq/23-ehci_hcd]
   80  FF     74 114   - [irq/23-ehci_hcd]
  217  FF     65 105   - [irq/45-enp3s0]
   24  FF     50  90   - [irq/9-acpi]
   69  FF     50  90   - [irq/43-ahci]
   34  FF     50  90   - [irq/40-PCIe PME]
   35  FF     50  90   - [irq/41-PCIe PME]
   36  FF     50  90   - [irq/42-PCIe PME]
  149  FF     50  90   - [irq/44-mei]
   37  FF     50  90   - [irq/1-i8042]
    3  FF      1  41   - [ksoftirqd/0]
   13  FF      1  41   - [ksoftirqd/1]
    7  TS      -  39 -20 [kworker/u:0H]
    5  TS      -  39 -20 [kworker/0:0H]
   17  TS      -  39 -20 [kworker/1:0H]
   82  TS      -  39 -20 [kworker/0:1H]
   83  TS      -  39 -20 [kworker/1:1H]
   18  TS      -  39 -20 [cpuset]
   19  TS      -  39 -20 [khelper]
   21  TS      -  39 -20 [netns]
   23  TS      -  39 -20 [kblockd]
   29  TS      -  39 -20 [crypto]
   33  TS      -  39 -20 [kthrotld]
   41  TS      -  39 -20 [deferwq]
   68  TS      -  39 -20 [ata_sff]
   92  TS      -  39 -20 [ext4-dio-unwrit]
  108  TS      -  39 -20 [rpciod]
  111  TS      -  39 -20 [nfsiod]
  155  TS      -  39 -20 [kvm-irqfd-clean]
  183  TS      -  39 -20 [ext4-dio-unwrit]
  205  TS      -  19   0 /usr/lib/systemd/systemd-logind
  107  TS      -  19   0 /usr/lib/systemd/systemd-journald
  134  TS      -  19   0 /usr/lib/systemd/systemd-udevd
   20  TS      -  19   0 [kdevtmpfs]
   66  TS      -  19   0 [khubd]
   10  TS      -  19   0 [rcu_preempt]
   11  TS      -  19   0 [rcu_bh]
   12  TS      -  19   0 [rcu_sched]
   91  TS      -  19   0 [jbd2/sda2-8]
  182  TS      -  19   0 [jbd2/sda3-8]
   26  TS      -  19   0 [kswapd0]
   28  TS      -  19   0 [fsnotify_mark]
   22  TS      -  19   0 [bdi-default]
  218  TS      -  19   0 /usr/bin/dbus-daemon --system --address=systemd: --nofork --nopidfile --systemd-activation
  230  TS      -  19   0 /sbin/agetty --noclear tty1 38400 linux
  749  TS      -  19   0 [kworker/1:3]
  753  TS      -  19   0 /usr/bin/lighttpd-angel -D -f /etc/lighttpd/lighttpd.conf
  754  TS      -  19   0 /usr/bin/lighttpd -D -f /etc/lighttpd/lighttpd.conf
  757  TS      -  19   0 /usr/bin/ntpd -g -u ntp:ntp
  761  TS      -  19   0 [nfsv4.0-svc]
  762  TS      -  19   0 [flush-8:0]
  780  TS      -  19   0 [kworker/0:1]
  782  TS      -  19   0 [kworker/1:0]
   70  TS      -  19   0 [scsi_eh_0]
   71  TS      -  19   0 [scsi_eh_1]
   72  TS      -  19   0 [scsi_eh_2]
   73  TS      -  19   0 [scsi_eh_3]
   74  TS      -  19   0 [scsi_eh_4]
   75  TS      -  19   0 [scsi_eh_5]
   40  TS      -  19   0 [mce-notify]
   38  TS      -  19   0 [kworker/0:2]
   76  TS      -  19   0 [kworker/u:2]
   77  TS      -  19   0 [kworker/u:3]
  783  TS      -  19   0 sshd: maverick [priv]
  786  TS      -  19   0 sshd: maverick@pts/0
  788  TS      -  19   0 -bash
  791  TS      -  19   0 su
  792  TS      -  19   0 bash
  795  TS      -  19   0 [kworker/0:0]
  798  TS      -  19   0 ps -eLo pid,cls,rtprio,pri,nice,cmd
    1  TS      -  19   0 /sbin/init threadirqs
    2  TS      -  19   0 [kthreadd]
   27  TS      -  14   5 [ksmd]
I have a Xonar Essence ST (snd-oxygen), an M2Tech Hiface2 (ehci-hcd), an M2tech Young (ehci-hcd) and the music is on a remote server (enp3s0). That should explain how are my irqs configured.


The ksoftirqd is not an audio process, it's a kernel daemon that manage software irqs (as you say, one per cpu). There should be no need to prioritze this.
I was talking about the 4 threads (audio decoding, player...) that are created by the mpd process. If you use htop you can see them if you use tree view (F5). Be sure in setup->display options->hide userland threads is not checked.

Idolse,

If you don't already know, the kernel release 3.11 will ship with 32 new audio drivers iirc (hiface included) and many alsa/drivers improvements.

Post Reply