MOC is getting stuck (stuttering) after hours of playing

Submitted by GenghisKhan on Sat, 2010-09-04 17:17

Forums:

1. MOC is getting stuck after hours (I think more than six) of playing of a streaming audio data.
1.1. MOC itself is responding very slow.
1.2. The same sounds of a certain fraction of a second is repeating over and over.
1.2.1. A funny thing is that it keeps downloading the stream but it is, as mentioned, no longer playing the new collected data.

Version: MOC Version 2.4.0
Distro: SalixOS 13.1.1 (Slackware based)
http://www.salixos.org/wiki/index.php/Home

GenghisKhan

Thu, 2010-09-09 05:23

moc 2.5.0-alpha4 Build: Sep 8 2010 10:54:08
Compiled with: OSS ALSA internet streams

Still the same problem, what could cause that?
_______________
Updated version: http://slackbuilds.org/repository/13.1/audio/moc/

jcf

Sun, 2012-07-08 02:59

I have today committed to SVN r2346 which possibly resolves this problem.

GenghisKhan

Sun, 2012-11-18 09:05

After ~470 minutes the sound got stuck again.

Using development release 2.5.0-beta1

GenghisKhan

Sun, 2012-11-18 09:11

streaming audio data = playing audio stream from HTTP.

GenghisKhan

Thu, 2013-07-25 09:21

See http://moc.daper.net/node/903

GenghisKhan

Thu, 2013-07-25 14:18

Usually I get a hiccup/strutter (broken record syndrome) with played streams of audio/mpeg format after roughly 10 hours.

I am now playing an audio/aacp streams.
I am, still, not experiencing any strutter.

<br /> $ echo ; date ; echo ; mocp -i</p> <p>Thu Jul 25 12:39:29 IDT 2013</p> <p>State: PLAY<br /> File: http://gaia02.radiogaia.com.br:8010/<br /> Title: JAMES HORNER - APOLLO 13 END TITLES<br /> Artist:<br /> SongTitle: JAMES HORNER - APOLLO 13 END TITLES<br /> Album:<br /> CurrentTime: 915m<br /> CurrentSec: 54949<br /> Bitrate: 56kbps<br /> AvgBitrate: 0kbps<br /> Rate: 44kHz<br />

Edit: Over 17 hours of playing with no strutter, yet.

<br /> $ echo ; date ; echo ; mocp -i</p> <p>Thu Jul 25 14:32:42 IDT 2013</p> <p>State: PLAY<br /> File: http://gaia02.radiogaia.com.br:8010/<br /> Title: WORLD OF NEW AGE - PERFECT HARMONY<br /> Artist:<br /> SongTitle: WORLD OF NEW AGE - PERFECT HARMONY<br /> Album:<br /> CurrentTime: 1030m<br /> CurrentSec: 61843<br /> Bitrate: 50kbps<br /> AvgBitrate: 0kbps<br /> Rate: 44kHz<br />

Edit: Play stopped with no strutter at 62081 seconds (around 17 hours and 15 minutes). Either my end has terminated the connection (damn ISP) with the stream or it was the other end (stream server).

<br /> $ echo ; date ; echo ; mocp -i</p> <p>Thu Jul 25 14:36:46 IDT 2013</p> <p>State: PLAY<br /> File: http://gaia02.radiogaia.com.br:8010/<br /> Title: WORLD OF NEW AGE - PERFECT HARMONY<br /> Artist:<br /> SongTitle: WORLD OF NEW AGE - PERFECT HARMONY<br /> Album:<br /> CurrentTime: 1034m<br /> CurrentSec: 62081<br /> Bitrate: 55kbps<br /> AvgBitrate: 0kbps<br /> Rate: 44kHz<br />

<br /> $ echo ; date ; echo ; mocp -i</p> <p>Thu Jul 25 14:44:21 IDT 2013</p> <p>State: PLAY<br /> File: http://gaia02.radiogaia.com.br:8010/<br /> Title: WORLD OF NEW AGE - PERFECT HARMONY<br /> Artist:<br /> SongTitle: WORLD OF NEW AGE - PERFECT HARMONY<br /> Album:<br /> CurrentTime: 1034m<br /> CurrentSec: 62081<br /> Bitrate: 55kbps<br /> AvgBitrate: 0kbps<br /> Rate: 44kHz<br />

jcf

Mon, 2013-10-28 21:22

After a remote debugging marathon I unfortunately have to report that we are no closer to establishing the cause of this problem, though we have eliminated many possibilities along the way. We could not even determine whether it is internal or external to MOC code, or what the triggering event (if any) might be. It's quite possible that it is specific to a particular system's hardware and/or software configuration.

I do regret that the only other person to encounter this problem chose not to participate in any way. If he had, it would have doubled the amount of information available for debugging.

At the other extreme, I cannot thank user "GenghisKhan" enough for the huge amount of time he has put in running lengthy tests and comprehensively reporting the results.

There seems little more we can do now, except note it and move on. But if anyone is able to offer any further clues they will be gratefully received.

tokapix

Sun, 2015-11-08 10:23

Good day all,

I use Music On Console for a few years now. Back in the days of Debian 6, I listened some of my playlists for hours on a laptop using an amd64 processor architecture. I currently run these sessions on a netbook with an i386 architecture. The behaviour mentionned in this bug tracker corresponds exactly to what I hear after hours of playing. This behaviour appeared on Ubuntu 12.04 (shipped with the netbook), Debian 8, and the current Debian testing.

Here is what I can tell of my installation and the symptoms I observed:

  • The versions of mocp I used were always shipped with the system, not compiled myself from source.
  • If this is an architecture dependent bug, it may happen on AMD64 processor after a few centuries... :-)
  • It happens also when listening local music, not only HTTP audio streams. (should I open a new thread in this case ?)
  • Changing the state of the player (pausing then playing for example) seems to reset the counter and you get another 6 hours to wait before the stuck happens.
  • Mainly using Alsa, though other compenents are installed for other specific use cases
  • Pulseaudio not installed, as my way of using my machine makes its presence irrelevant and sometimes source of problems.

Machine currently available for testing:

  • Debian Testing dated from 2015-11-08 with i386 processor
  • Debian Testing with about the same level and amd64 processor, not had the occasion to test
  • mocp version relevant information:
    Version : 2.5.0
    Revision : 2668
    Built : Jul 20 2015 20:39:23
    Compiled with : OSS ALSA JACK DEBUG Network streams resample
    Running on : Linux 4.2.0-1-686-pae i686
  • mocp config file relevant information
    SoundDriver = ALSA:JACK:OSS
    ALSADevice = default
    ALSAMixer1 = Master
    ALSAMixer2 = PCM
    ResampleMethod = SincMediumQuality

    Please note I also had the problem without changing the ResampleMethod.

I hope this helps,
Best Regards.

jcf

Tue, 2015-11-10 02:57

It's been a while since I've worked on this particular problem, so all the details are no longer in my necktop computer's memory. :-)


* If this is an architecture dependent bug, it may happen on AMD64
processor after a few centuries... :-)

Your thinking is similar to mine, here. Indeed, the timing does seem to roughly correspond to 32-bit overflow of sample count... but I've tested (with GenghisKhan's help) and checked the MOC codepath but nothing was apparant and suggests the problem is within ALSA (but I'm not prepared to blame it just yet). (And I forget just now the architecture GenghisKhan is using.)


* Changing the state of the player (pausing then playing for example)
seems to reset the counter and you get another 6 hours to wait
before the stuck happens.

Now that's interesting!


* Debian Testing dated from 2015-11-08 with i386 processor
* Debian Testing with about the same level and amd64 processor, not
had the occasion to test

If you could possibly find the time to test (out to about 12 hours) on the 64-bit machine then I'd appreciate it.


I hope this helps,

You've certainly given me some useful clues. Thanks, tokapix, for such a good report!

jcf

Sun, 2016-11-13 06:58

After pursuing this bug for six years across three continents, I'm delighted and relieved to be able to say that it has finally met its well deserved end in commits r2921 (MOC 2.5) and r2922.

At the heart of the problem lies an integer overflow bug during the resampling process of ALSA's dmix component. This bug has been apparently reported on the ALSA Project's Bug Tracker and the associated Wiki. Unfortunately, those services are defunct and there seems no way to access the information they held (even using the Wayback Machine); the links to their Bug Tracker on the ALSA Project's web pages remain broken.

A good technical write up of the ALSA bug and a proposed fix was provided by Andrew Church, but as at this time his solution has not been implemented. The MOC commit log provides further MOC-specific technical details.

The solution has been to provide a new ALSAStutterDefeat configuration option which can be turned on to circumvent ALSA's dmix component, and therefore the ALSA stutter bug. There are several trade-offs to take into account when using this option, so the reading of the option's comments in the example configuration file is recommended.

The pursuit of this bug was extremely frustrating. Firstly, mocmaint could never reproduce it (because it only emerged in reasonable time on a 32-bit machine) and so was reliant on remote debugging by e-mail and the good will and availablity of those who reported it. Secondly, the long time to failure meant many of the usual tools required too much HDD space for logging and it was too long for hands-on monitoring. Interactive debugging was too disruptive to the playing process to be viable. The timings involved suggested an integer overflow, but it was nowhere apparant that it was actually occurring in MOC code despite many hours of desk checking and code diving.

Many of the test results were contradictory. In retrospect, it's obvious now why. The manifestation of the ALSA stutter bug depends on multiple factors and the relationship between them was not obvious; it was impossible to control every variable remotely (even if they had all been known at the time).

Thanks must go to user "vectis" for providing several hardware devices which allowed for the narrowing of the search space sufficiently to finally home in on the cause. Also to user "tokapix" who provided additional proving testing. But by far the biggest contibution came from the original reporter user "GenghisKhan" who stuck with the problem throughout its six year lifetime and invested many hours in testing and debugging runs; we are all indebted to him.

It must be said that this problem would probably have been solved much earlier if the ALSA Project had maintained its Bug Tracker.