Startup lag with big playlist

Submitted by yavuzileri on Sun, 2020-04-12 09:46

Forums:

MOC version: 
2.6alpha3

Hi, thanks for such a good audio player.

I've got two problems and want to ask what I can do about them. Let me give you the details:

Problem-1
========
At first, I had a disturbing delay at shutdown, but was able to solve it by Setting "ShowTime = yes" and increasing TagCacheSize enough to hold all tags possibly read from the playlist entries, as suggested by the forum post [ moc.daper.net/node/345 ], titled "MOC takes long to exit (after q or Q command) with big playlist (hangs on "Reading tags...")".

Now, when I start moc with the mocp command, the following information messages are displayed: "(0) Playlist loaded", and then "(0) The playlist was cleared", and the following status message is displayed: "File: <counting up>". Navigating up and down in the playlist is very laggy during the counting of files, which takes about 5-6 seconds for ~15000 entries in the playlist. Saving the playlist and setting 'MusicDir = "/home/yi/Music/all.m3u"' in the config, or starting moc with the "mocp ~/Music/all.m3u" command doesn't prevent the counting of the files in playlist.

So, is there any way I can prevent this lag?

I have the below lines in ~/.moc/config:

MusicDir = "/home/yi/Music"
StartInMusicDir = yes
FormatString = "%(a:%a - :)%(t:%t:)"
SoundDriver = ALSA:JACK:OSS
ShowFormat = no
ShowTime = yes
Theme = yi_theme
Keymap = keymap
SeekTime = 2
TagsCacheSize = 200000
Layout1 = playlist(0,0,100%,100%)
Layout2 =
Layout3 =
FollowPlayedFile = no
PlaylistFullPaths = no
MessageLingerTime = 0
TiMidity_Config = /etc/timidity.cfg

Problem-2
========
Another minor problem is that when I leave the TiMidity_Config option unset in my user-specific config (~/.moc/config), I get the below error while trying to run moc:

FATAL_ERROR: TiMidity-Plugin: Error processing TiMidity-Configuration!
Configuration file is: <default>

The default config (/usr/share/doc/moc/config.example) says: "Leave it unset to use library defaults (/etc/timidity.cfg mostly)", but even though /etc/timidity.cfg exists, I still get the error unless I set TiMidity_Config to either yes or no.

jcf

Sun, 2020-04-12 10:47

That's a bit strange as both the default and "yes" value for TiMidity_Config are treated the same. Specificly setting it to "yes" should make no difference.

There is some suggestion here that some distributions place the default TiMidity configuration file in /etc/timidity/timidity.cfg, but the default or "yes" setting of TiMidity_Config should not make a difference.

yavuzileri

Sun, 2020-04-12 11:01

That's a mistake I did while writing the post, here's the correction: The default config (/usr/share/doc/moc/config.example) says: "Leave it unset to use library defaults (/etc/timidity.cfg mostly)", but even though /etc/timidity.cfg exists, I still get the error unless I set TiMidity_Config either explicitly to "/etc/timidity.cfg" or to "no".

jcf

Sun, 2020-04-12 22:19

Which version of TiMidity are you using?

What happens if you create a /etc/timidity directory and copy (or symlink) /etc/timidity.cfg into it?

It's increasingly seems to me that your TiMidity is looking for the default timidity.cfg somewhere other than /etc. If you know how to use strace(1), you can determine exactly where it is looking.

yavuzileri

Mon, 2020-04-13 05:13

timidity was not installed at all. On my system (Fedora 31), the file /etc/timidity.cfg was provided by the package fluid-soundfont-lite-patches-3.1-21.fc31.noarch, which is a dependency of moc due do that single file (i.e. timidity.cfg) and wastes 195 M if you don't use it at all:

[ forums.fedoraforum.org/showthread.php?262744-Any-ideas-about-fluid-soundfont-lite-patches ]: "I forget which package, but one of the updates added a dependency on /etc/timidity.conf,
which is apparently satisfied by that large, otherwise unnecessary, soundfont package."

The binary /usr/bin/timidity is provided by the package timidity++-2.14.0-19.fc31.x86_64, which installs the fluid-soundfont-gm-3.1-21.fc31.noarch package as a dependency, which comes with /etc/timidity++.cfg. I installed it just to test.

This is what (?) the mocp command does with the timidity.cfg file:

With TiMidity_Config = yes in ~/.moc/config:

$ strace mocp |& grep timidity openat(AT_FDCWD, "/usr/lib64/moc/decoder_plugins/libtimidity_decoder.la", O_RDONLY) = -1 ENOENT (No such file or directory) openat(AT_FDCWD, "/usr/lib64/moc/decoder_plugins/libtimidity_decoder.so", O_RDONLY|O_CLOEXEC) = 3 openat(AT_FDCWD, "/lib64/libtimidity.so.2", O_RDONLY|O_CLOEXEC) = 3 openat(AT_FDCWD, "timidity.cfg", O_RDONLY) = -1 ENOENT (No such file or directory) $ mocp FATAL_ERROR: TiMidity-Plugin: Error processing TiMidity-Configuration! Configuration file is: <default>


With TiMidity_Config = no in ~/.moc/config:

$ strace mocp |& grep timidity openat(AT_FDCWD, "/usr/lib64/moc/decoder_plugins/libtimidity_decoder.la", O_RDONLY) = -1 ENOENT (No such file or directory) openat(AT_FDCWD, "/usr/lib64/moc/decoder_plugins/libtimidity_decoder.so", O_RDONLY|O_CLOEXEC) = 3 openat(AT_FDCWD, "/lib64/libtimidity.so.2", O_RDONLY|O_CLOEXEC) = 3


With TiMidity_Config = /etc/timidity.cfg in ~/.moc/config:

$ strace mocp |& grep timidity openat(AT_FDCWD, "/usr/lib64/moc/decoder_plugins/libtimidity_decoder.la", O_RDONLY) = -1 ENOENT (No such file or directory) openat(AT_FDCWD, "/usr/lib64/moc/decoder_plugins/libtimidity_decoder.so", O_RDONLY|O_CLOEXEC) = 3 openat(AT_FDCWD, "/lib64/libtimidity.so.2", O_RDONLY|O_CLOEXEC) = 3 openat(AT_FDCWD, "/etc/timidity.cfg", O_RDONLY) = 3


With TiMidity_Config = /etc/timidity/timidity.cfg in ~/.moc/config:

$ strace mocp |& grep timidity openat(AT_FDCWD, "/usr/lib64/moc/decoder_plugins/libtimidity_decoder.la", O_RDONLY) = -1 ENOENT (No such file or directory) openat(AT_FDCWD, "/usr/lib64/moc/decoder_plugins/libtimidity_decoder.so", O_RDONLY|O_CLOEXEC) = 3 openat(AT_FDCWD, "/lib64/libtimidity.so.2", O_RDONLY|O_CLOEXEC) = 3 openat(AT_FDCWD, "/etc/timidity/timidity.cfg", O_RDONLY) = 3


With TiMidity_Config = /etc/timidity++.cfg in ~/.moc/config:

$ strace mocp |& grep timidity openat(AT_FDCWD, "/usr/lib64/moc/decoder_plugins/libtimidity_decoder.la", O_RDONLY) = -1 ENOENT (No such file or directory) openat(AT_FDCWD, "/usr/lib64/moc/decoder_plugins/libtimidity_decoder.so", O_RDONLY|O_CLOEXEC) = 3 openat(AT_FDCWD, "/lib64/libtimidity.so.2", O_RDONLY|O_CLOEXEC) = 3 openat(AT_FDCWD, "/etc/timidity++.cfg", O_RDONLY) = 3 Configuration file is: /etc/timidity++.cfg $ mocp FATAL_ERROR: TiMidity-Plugin: Error processing TiMidity-Configuration! Configuration file is: /etc/timidity++.cfg

jcf

Mon, 2020-04-13 07:59

With this additional information, I can see a couple of issues here now.

Firstly, there is a Fedora packaging deficiency for MOC. You should not have to jump through those hoops to get a workable application. MOC does not require any particular decoder to be present (even if configured with it), so the way to package it (so it doesn't drag in unnecessary dependancies) is to do what Debian have done with the FFmpeg decoder and create a separate optional package which identifies the necessary dependancies for the decoder rather than MOC as a whole. But I don't know Fedora's packaging policies or capabilities, so it may not be a workable approach in this case and they will have to work out an approach which satisfies them.

There has also been a change in the TiMidity code for which the commit message of 679a1b6e says:

Make the library to not add common timidity.cfg system locations to its search path any longer: callers of mid_init() is responsible for it from now on.

So, they appear to have broken backwards compatibility without any warning of which I was aware. The workaround is to always specify the fully qualified filename for timidity.cfg in TiMidity_Config.

yavuzileri

Mon, 2020-04-13 08:45

Thanks for the efforts, I really have no problem at all with including TiMidity_Config = no or TiMidity_Config = /etc/timidity.cfg in ~/.moc/config. I wasn't even gonna mention this minor problem.

My real issue is the startup lag due to counting the files in the playlist, even if the playlist is saved and the tag cache is set to a sufficiently large number to hold all tags. There isn't any config option that can cause the loading of the playlist, then clearing the playlist, and then counting all entries again, at each startup, so I think there is a problem in the current alpha version. Even if moc doesn't use a library, if I save the playlist to disk, the metadata is stored too I think. So, setting a large tag cache resolved the lag due to "Reading tags..." on exit, but setting the ShowTime option and saving the playlist to disk still doesn't prevent the startup lag.

jcf

Mon, 2020-04-13 10:08

I think I understand what is happening, now. Could you try setting StartInMusicDir to "no" and launching MOC in /home/yi. Leave TagsCacheSize at 200000 and ShowTime at "yes". Also ensure there is already a populated playlist.m3u in ~/.moc.

yavuzileri

Mon, 2020-04-13 10:38

Negative. Same thing in ~ and ~/Music: playlist loaded and playlist cleared as consequent info messages, and "Files: <rapidly counting up>" as the status message, still taking about 5-6 seconds.

yavuzileri

Mon, 2020-04-13 12:24

With SyncPlaylist = no, the startup lag disappears, i.e. playlist is not cleared and counted after being loaded. BUT the "Reading tags..." lag on quit returns this time, and I have to hit ^c to interrupt saving the playlist, which was already saved! There is a problem in the logic.

yavuzileri

Mon, 2020-04-13 13:24

For anybody who is having a lag on startup or exit, I had both and the following setup solved both of the issues:

I have the below lines in ~/.moc/config:

MusicDir = "/home/yi/Music" StartInMusicDir = yes FormatString = "%(a:%a - :)%(t:%t:)" SoundDriver = ALSA:JACK:OSS ShowFormat = no ShowTime = yes Theme = yi_theme SyncPlaylist = no Keymap = keymap SeekTime = 2 TagsCacheSize = 200000 Layout1 = playlist(0,0,100%,100%) Layout2 = Layout3 = FollowPlayedFile = no PlaylistFullPaths = no MessageLingerTime = 0 TiMidity_Config = no

And I start moc with the saved playlist as the argument, so I have the following alias in my .bashrc:
alias mocp='mocp ~/Music/yi_all.m3u'

I think the startup playlist counting thing shouldn't occur if there is only one instance, so the SyncPlaylist option can be set to yes without causing the playlist to be cleared and counted again when the first client is started.

Thanks again.

jcf

Tue, 2020-04-14 05:11

I did not realise you are running multiple clients, or connecting a single new client to an already-running server. That changes the playing field.

yavuzileri

Tue, 2020-04-14 05:21

Not running multiple clients, or connecting to an already running server, those were never the cases. That's why I think moc shouldn't do the playlist resting thing if the user is NOT doing those things, because it seems like it does assume the user is running multiple clients even if this isn't the case.

jcf

Tue, 2020-04-14 04:56

I've been trying to figure out a way of handling this abdication of responsibility by TiMidity for the location of its configuration file without propagating the breakage to MOC users. I haven't come up with a clean solution as yet.

Complicating the task is that the version of the library is not available to the client application with sufficient resolution to determine which behaviour it is dealing with. Also, the library may be configured with the location (and name) of the configuration file and there is no way for the client to determine this either.

So, my current thinking is to upgrade the TiMidity library requirement to version 2 and introduce any MOC configuration file option changes in MOC 2.7 (or later). MOC 2.7 (or later) will see a number of well-flagged, backwardly-incompatible configuration file changes anyway.

In the meantime, a comment in the current configuration file to make users aware of the workaround would be appropriate.

jcf

Sun, 2020-04-12 11:00

For some audio formats (MP3, I'm looking at you) MOC has to read the entire audio file to extract the tags (because they can be placed anywhere within the file), so that's going to be slow and particularly so if you are reading from a remote filesystem.

If you don't specificly need the duration of unplayed files, you could try setting the ShowTime option to IfAvailable. I'd also like to check on why the "The playlist was cleared" message is being issued immediately after it was loaded.

yavuzileri

Sun, 2020-04-12 11:06

The same messages are displayed with "ShowTime = IfAvailable": "(0) Playlist loaded", and then "(0) The playlist was cleared", and "File: <counting up>" as the status message. But now, the lag on exit returns with the "Reading tags..." message.

jcf

Sun, 2020-04-12 22:24

Exactly what command (verbatim) are you using to start MOC?

yavuzileri

Mon, 2020-04-13 05:14

$ mocp