Tonic Trouble Soundtrack

Talk about the Rayman music website.

Moderators: Droolie, English moderators

Forum rules
Please keep the forum rules and guidelines in mind when creating or replying to a topic.
deton24
Garatta Ciatik
Posts: 1141
Joined: Fri Jan 14, 2011 1:22 am
Location: Poland
Tings: 8652

Re: Tonic Trouble Soundtrack

Post by deton24 »

Man, I can't believe that somebody is still so deep in such things this days either ^^
It would be cool to get this music finally in decent quality.

If you look for some people able to answer at some of your questions, maybe go to xentax.com, but Drolpiraat already started a topic with a view to a find a solution for unpacking beta music files, and no success.
TT's beta/special edition 44kHz sample-rate MPEG-based codec decoded.
Droolie sad that these files are rather not 44kHz.
to get the APMs converted into a common ADPCM standard (no conversion, just making it compliant with compatible players like FFMPEG)

the MPX version definitely has a higher frequency range

beta/SE/MPX version
I think that is a dead end. And when you have compared these two versions of R2 recordings from the game, it fits into the puzzle that decoder shouldn't be responsible so much for deterioration of quality.
At least, achieved difference shouldn't be crucial. And I don't think it is even achievable to decode these files in other way. Or it would be hard eventually.
I'd be willing to set up a retro built PC with an SPDIF output equipped card for another run at it the old fashioned way.
I was thinking about that since you started to speaking out on quality of the rip.
I have tried to remaster it. And I noticed downsides of this particular compression and overall quality, which somehow could remind remastering PC files of Rayman 2. So results wasn't enough reassuring to draw my kind of special attention to finish the job. But it not means it is impossible.

And recently I was thinking about, if Special Edition may be better source for remastering, or just for listening, it may be good solution to capture music on fly. But while I was working more seriously on R2 files, it turned out that line-in rip files, though sounded better than converted files directly from the game, are worse to remastering.

If you have an opportunity to capture music via SPDIF digitally round-trip, so without analog distortions, it can be even better source to listen than we have know, and also possibly, for remastering. And I afraid it will be faster than looking for some people from all over the world to unpack this Special Edition music, or much less, decode PC files in other way with the aim of achieving better quality. Because finding such people may be difficult.

And I'm taken-aback that somebody noticed some depending on decoder in the game, and differences to unpacked/converted files from the game. Even if differences are not worth to canvassing, or even possible to repairing somehow.
It reminds me that I tried to push that argument before xd
Last edited by deton24 on Mon Oct 16, 2017 5:45 pm, edited 1 time in total.
kooz
Moussu
Posts: 39
Joined: Mon Aug 15, 2011 2:00 pm
Tings: 1195

Re: Tonic Trouble Soundtrack

Post by kooz »

I spent a good chunk of time this morning reading through the thread you speak of, and I have no intentions on repeating history here. Quality, in terms of frequency range, in this instance is a far less controversial matter. The spectrum analysis of my analog line-in DC Rayman 2 rip revealed a clear frequency cut off at 11kHz, along with line noise visible in the higher end of the spectrum, separated by a wide, dead frequency gap. This proved that the DC source material was indeed still the limiting factor, disproving any possible "magic" the console could be doing to reproduce lost high frequencies. In that instance, we already had concrete proof (through bit rate and format specifications) that the line-in DC source could not be anything but worse than those same files converted (or played back natively) on a PC.

In this case, we have a mystery codec which leaves many questions unanswered. One thing is certain, however: the MPX files sound significantly different than the APM files. The spectrum analysis shows that MPX files are not limited by a 22kHz sample rate, however the other downsides of the format may make this benefit irrelevant.

Based on that difference in audio quality, as well as the almost unanimous level of interest in decoding those files, I believe there is merit in providing a rip of the SE version, either by means of perfect digital conversion or otherwise.

I want to express that I am not trying to "sell" one version over the other in terms of quality - I think they each have unique advantages and disadvantages that warrant further analysis and comparison by the community. The rip I posted was put together 15 years ago, and I made it for no one other than myself at that time. I've uploaded it simply for the sake of contributing to a more complete pool of source material for an optimal Tonic Trouble soundtrack.
deton24
Garatta Ciatik
Posts: 1141
Joined: Fri Jan 14, 2011 1:22 am
Location: Poland
Tings: 8652

Re: Tonic Trouble Soundtrack

Post by deton24 »

he spectrum analysis of my analog line-in DC Rayman 2 rip revealed a clear frequency cut off at 11kHz, along with line noise visible in the higher end of the spectrum, separated by a wide, dead frequency gap.
Ok. Might be. But your rip sounds better for me than Revolution extracted from the game.
Sketchy attempts to remaster DC brought better results then, but finally on better software I've achieved really, really good, and much better results while remastering Revolution flac files.
Probably extracted DC files would bring even better results, only if I would work on these files then.
I just forgot about this source, but it's not that bad.
More agonizing on R2 music formats here:
http://raymanpc.com/forum/viewtopic.php ... 0#p1124787
Actually, everything what I have said.
Droolie
Mr Sax
Posts: 5162
Joined: Wed Apr 20, 2005 7:31 pm
Tings: 61830

Re: Tonic Trouble Soundtrack

Post by Droolie »

kooz wrote:Based on that difference in audio quality, as well as the almost unanimous level of interest in decoding those files, I believe there is merit in providing a rip of the SE version, either by means of perfect digital conversion or otherwise.
There is definitely merit in providing a rip of the special edition and after listening to your rip, I think that version does sound a lot better. There are some MPEG artifacts, but they are much less audible than the frequency cut-off and especially the aliasing from the final version. It would be great if we could replace the current rip by this one, but so many segments are out of order or missing just because this was played in-game instead of playing the files in the correct order. You'd really need to convert them... and that's pretty much why I made those threads over at XeNTaX and HCS, but you know what came of that. :P

So at this point your SE rip is a good alternative for the current rip. I could provide a link on RayTunes but I can't really replace it. (though with your permission I can and will replace the staff roll track, since your Monkey's Audio rip is clearly a better quality source than the one I used. :) I'll replace other full tracks such as the intro, outro and ski slope while I'm at it.)

The only possibility I see to get a proper recording is to replace the Ski Slope music with each of the segments, one by one (and I don't mean each of the MPX files, but each of the segments starting with "2RUS" inside) and record the results in-game. The Ski Slope music is a single looping segment, so replacing it would make the segment loop, meaning you'd have to cut the segment out properly too. Then we'd have the segments converted, allowing us to assemble the segments like with the current rip... but this type of "conversion" would literally take ages. :/
deton24
Garatta Ciatik
Posts: 1141
Joined: Fri Jan 14, 2011 1:22 am
Location: Poland
Tings: 8652

Re: Tonic Trouble Soundtrack

Post by deton24 »

As I said in other topic before, Foobar may be easily used to order segments - files can be put into a playlist, ordered, and then, marked and merged.

Kooz, your 1st link is just compressed lossless files of 2nd link which is a line-in rip of PC edition?
Interesting.
The first rip in this thread is Special Edition line-in rip at 128.
http://www.corbac.com/page22.html

edit. No, sorry, it is rather usual PC version. My mistake.
kooz
Moussu
Posts: 39
Joined: Mon Aug 15, 2011 2:00 pm
Tings: 1195

Re: Tonic Trouble Soundtrack

Post by kooz »

Droolie,

Of course you have my permission to use any of the files I've provided. :D

I also had that same idea of replacing the ski slope track and going through each and every one of them that way. It would certainly be tedious, but at this point, unless we can get some expert help in decoding those files outside the game, it may be our best bet.

Deton24, yes. The Monkey's Audio is the lossless version of the 192kbps CBR rip I posted earlier. To the best of my recollection, this was done via line-in. Though a small part of me believes I may have been recording "What U Hear" / "Stereo Mix" directly from the PC that was playing the game. I can't really remember any more.
deton24
Garatta Ciatik
Posts: 1141
Joined: Fri Jan 14, 2011 1:22 am
Location: Poland
Tings: 8652

Re: Tonic Trouble Soundtrack

Post by deton24 »

So if you'll not record it via SPDIF in some closer future, I'll consider to do this my self. Still, I don't have any necessary equipment yet.
But now I can see at least how capable to remaster are your files ^^ Maybe recording won't that necessary. Still, I'm not a pro, and some guys here burned me at the stake because of my convictions of doing such a things.
kooz
Moussu
Posts: 39
Joined: Mon Aug 15, 2011 2:00 pm
Tings: 1195

Re: Tonic Trouble Soundtrack

Post by kooz »

I've created a script which splits MPX files into their individual segments. It uses snakebite from VGMToolbox. It will create a subfolder named "split" with a whopping total of 1,585 smaller files.

I played around with running TT in a VM this morning. Replacing SKI.MPX does work... but with this many files, it is going to be incredibly tedious.
Attachments
split.7z
(9.94 KiB) Downloaded 217 times
Last edited by kooz on Mon Jun 06, 2016 8:35 am, edited 1 time in total.
deton24
Garatta Ciatik
Posts: 1141
Joined: Fri Jan 14, 2011 1:22 am
Location: Poland
Tings: 8652

Re: Tonic Trouble Soundtrack

Post by deton24 »

I'm not really familiar with things that you are trying to do, but foobar has vgmstream plugin. Maybe it'll read these files, and it will be easier to arrange tracks in playlist, and then merge. If it's possible.
Or if any conversion to eg. Wav is not possible in some way.

edit.
Kooz
U R A GENIUS. Your rip rocks!!
It sounds way, way, way better than every other version.
And it not especially abridged (I don't even notice a difference, so it can be put on Raytunes without any problem, now it is not linked yet to this topic).
Thanks a lot
deton24
Garatta Ciatik
Posts: 1141
Joined: Fri Jan 14, 2011 1:22 am
Location: Poland
Tings: 8652

Re: Tonic Trouble Soundtrack

Post by deton24 »

How do you like it now?
https://drive.google.com/folderview?id= ... sp=sharing
Press CTRL+A, or mark all files, then by right click pick download. It'll download all files at a time.
Although I got error then, unable to compress...

30 band EQ, and also Dynamics Compressor for highs, and Advanced Limiter.
I didn't touch freqs bellow 315Hz, so it could be even better.

It wasn't even so requiring source to use Magix. But maybe in future. Maybe.
I thinks it's better now.

There is one downside of it, massive cutoff one freq in the middle.
But it sounds the best among others so far imho.
Last edited by deton24 on Tue Aug 02, 2016 10:16 am, edited 2 times in total.
kooz
Moussu
Posts: 39
Joined: Mon Aug 15, 2011 2:00 pm
Tings: 1195

Re: Tonic Trouble Soundtrack

Post by kooz »

First off, Deton, thank you for your kind words. I'm glad you are enjoying my old rip =)

I think we are basically all in universal agreement that the Special Edition soundtrack sounds the best. And with that said, I am excited to bring you all some very big news. After many days of hard work experimenting, learning, discovering, scripting, etc, I have managed to accomplish what we had all hoped would be someday possible.

The fruits of my labor are linked below.

After listening to the result, it is amazing how much of the music on the disc was left out of the game!

Droolie, if you are interested in collaborating on this project with me, please let me know. I have already spent many, MANY hours getting to the result you see here, and I would love some help in cleaning up/arranging this.

The preliminary work is done, folks!

Full write-up on how I made this possible to follow (tomorrow, probably). I am too tired to explain it all right now. =D


EDIT (7/4/16): updated link to completed segmented rip
https://mega.nz/#!gNFF1SKR!Zm6Sb0Zm6pqc ... 8FYTG-DJDA
Last edited by kooz on Mon Jul 04, 2016 8:45 pm, edited 1 time in total.
deton24
Garatta Ciatik
Posts: 1141
Joined: Fri Jan 14, 2011 1:22 am
Location: Poland
Tings: 8652

Re: Tonic Trouble Soundtrack

Post by deton24 »

Wait, wait, you did it? ;D
You've extracted segments from the game!
There's no denying that you put hell of a lot of good work... wow..
Good news is that filenames are preserved. That gives us hope that it'll not take us ages for arranging.

I recommend Foobar for it. We just putting files in order, and add the same files for the playlist if needed, then mark files, right click, convert, ...., destination, merge all files.

We can share among each other by part of files to arrange, to not do everything by one person, Droolie is also welcome, but personally I will have more time in a week and a half to look at it. But it will be done.

Thank you very much for your substantial contribution!
kooz
Moussu
Posts: 39
Joined: Mon Aug 15, 2011 2:00 pm
Tings: 1195

Re: Tonic Trouble Soundtrack

Post by kooz »

Yeah, I listened to them last night using Foobar and they align pretty well. Original filenames are not actually preserved, but the segments are numbered in the order they exist in the original files, so it plays through properly. There are occasionally clicks between files which will need to be edited out prior to joining everything together. I just wanted to get these files out to you guys as I was pretty excited by the results.

This took a TON of work.

My first experimentation started with finding a reliable way of actually running TT. It doesn't run under Windows 7, but is pretty easy to get going in XP. I started by using a WinXP VM for general testing.

My initial goal was to try to figure out how to use the game's DLLs to write a standalone decoder, however my knowledge in this field is extremely limited. After playing around with OllyDbg and IDA, I managed to get some some pseudo-code for what I believed to be the decode function of MPGMXBVR.DLL. I posted my findings to Xentax, once again hoping to get some help from their experts, however the reply I got was rather discouraging. Long story short, I only had a small piece of the puzzle, and it would take way more effort than anyone there was willing to spend.

After that, I came to the decision that I would try to use the game itself to do the decoding for me. I ran into several hurdles here. First, I managed to get an entire bigfile to play (albeit glitchy) by switching out one of them for a known game-triggered playback (i.e. SKIAMB1.MPX or SKI.MPX) and then replacing the header with SKI.MPX's. This presented a few problems, however, as when each segment played, it would produce a glitch. I was able to minimize this by removing each subsequent segment's headers and rejoining them back into one file, but this left us with no individual segments, and still would produce an occasional skip between them.

After deciding that playing individual segments on their own was the best course of action, I ran into another problem. They were simply not being played by the game engine. It turned out that each file needed a certain amount of data for the game to even play it properly. I was able to verify this by simply copy/pasting an individual segment over and over (after the header) and, sure enough, I was able to get that segment to repeat properly. Convinced that the problem was due to filesize, I managed to get a segment to play ONCE by simply padding the end of the segment with 10k of 00's.

So, ultimately, to prep each segment, I wrote a script which replaced the first 8 bytes with those from SKI.MPX (the longest playing single segment MPX), and then padded the end of the file with zeros.

Next, I needed a clean way of capturing the audio. My first attempt at this was to build a Windows XP machine with a sound card capable of S/PDIF output. The computer was an Athlon XP 2500 with a Radeon 128MB AGP video card. It had digital output on both the built-in sound card, as well as on a SB Live PCI card. This ultimately failed, as I was unable to get any card to output at 44.1kHz, 16 bit stereo. One of them would output at 48kHz, but I knew that sample rate conversion would be occurring, and that would result in loss of quality. The other refused to output audio completely.

I then decided to install XP on my current main PC. This is an i7 @ 4GHz with a GTX 480 and an Audigy4 Pro. I disconnected all of my hard drives and created a new installation on a 20GB drive connected via eSATA.

The game ran on this machine, but with severe graphics issues. Nevertheless, the game audio was working. This time with a far more capable PC at my disposal, I decided to try and capture the audio directly. This also encountered issues, as the Audigy4 Pro appears to do some sample-rate conversion natively. I was able to get Audition to record the audio, but when I analyzed the audio and magnified several times vertically, I saw that recorded silence was not TRUE silence. Frustrated, I started looking at further software solutions, and finally came up with a combination of Virtual Audio Cable and Total Recorder. These combined allowed for bit-perfect recording of game audio, which I verified with a hex editor as 00 data for recorded silence.

I now needed the game to immediately and automatically play whichever segment I was recording. I accomplished this by editing TT's game.dsc file to skip the main menu, as well as load a level which plays theme music at full volume immediately upon loading the level. SKIAMB1.MPX is a good candidate, as it plays during the level load screen, and continues until you press a button to begin Ed's descent down the slope. Unfortunately, it is not played at full volume (127), so this would not be a bit-perfect option. I eventually settled on using the level "carota," which plays the music file REPAITH8.MPX after loading the level. I removed all ambience MPX files to ensure the only thing being played was the theme music.

With a reliable recording method established, the last step was figuring out a way to automate the recording process. Individually recording and naming 900+ segments of audio is just not a feasible option. Fortunately, as I am rather adept at writing scripts, I managed to write a batch file which did this for me. Essentially the batch uses a FOR loop to do the following:

- Copy current file in *.MPX to the SOUND directory and overwrite REPAITH8.MPX
- Start Total Recorder and record for a set period of time (for 99% of the segments, this was about 25 seconds), using current file name
- Run Tonic Trouble
- Force kill Tonic Trouble after Total Recorder finishes recording
- Wait for a few seconds to ensure Tonic Trouble has been terminated
- Repeat

Recording concluded after several hours and I was left with a folder with WAV files whose filenames matched their corresponding MPX files.

The last step was to use a command-line utility named SoX to remove silence from the beginning and end of all the resulting WAV files. This was automated by using another FOR loop.

Now... I do realize there appear to be some segments missing. Total Recorder does not produce an output file if no audio is detected. This means that those segments either have no audio at all, or were not playing for one reason or another. During this write-up, it has dawned on me that these segments may have been too short, and will require more than 10k of padding. I will experiment with this tomorrow (as I am at work right now), and if that is the case I will upload a new rip set when I have them all recorded. I may also attempt to automate the removal of any end-of-file glitches beforehand.
Last edited by kooz on Mon Jul 04, 2016 3:51 am, edited 2 times in total.
Droolie
Mr Sax
Posts: 5162
Joined: Wed Apr 20, 2005 7:31 pm
Tings: 61830

Re: Tonic Trouble Soundtrack

Post by Droolie »

This is all really amazing, kooz! Glad to see there are still some completely crazy people in the world who would spend all that time and effort to crack one of the worst audio formats ever, just for better quality music! :P Sadly, I don't really have much time to work with you on this as my thesis deadline is in little more than a month. I wish you good luck though, and instead of replacing a few tracks, I'll replace the full soundtrack on RayTunes when you're done. :)

Also, I had personally already tried both reverse-engineering the DLL with IDA Pro and getting the DLL to work with my own code back in 2014, and I even got some response out of it, but it proved way too difficult in the end. I then looked into playing all the segments in order by modifying the BNM files - they're basically CSB files in byte format and recording it that way, but I had a lot of deadlines for uni back then and stopped. I think this would produce the same skips you heard though.

However, I just had another idea after reading your post: what if you made an in-game playlist (e.g. in a BNM or just by combining the segments into one big MPX... which probably won't work just like that) with all of the segments padded with some 00 bytes before and after each segment? Wouldn't you be able to record everything in one go, remove the skips programmatically and extract the segments from the recording? Though it would probably require extra work, and the current recording process is already fine as it is, as long as you know how long to record for each segment. :)
kooz
Moussu
Posts: 39
Joined: Mon Aug 15, 2011 2:00 pm
Tings: 1195

Re: Tonic Trouble Soundtrack

Post by kooz »

Nice to hear from you again, Droolie!

Actually, combining all the individual segments back into one big MPX works fine, as long as you remove the subsequent segment headers, and provide a long enough looplength (uncompressed sample size) for the first 4 bytes of the file.

Regarding the segment record length, I took all but the 6 biggest segments (which were drastically larger than the rest) and just set a stopwatch for the largest of that group. It came out to ~22 seconds or something, so I just rounded up to 25 to be safe. I recorded the 6 larger segments using manual timing.

Now for some news:
In my excitement to get an announcement to you guys I neglected to QC the result of SoX's silence-trimming, which, as it turns out, had been cutting off more than just null bytes at both ends of the file. I have addressed that today as well. Despite numerous attempts, I was unable to coerce SoX to remove only true, digital silence. I did find another solution, which goes by the name of Noisetime Silence Remover. It has an option to specifically only remove digital zero silence, which I verified to be sample-accurate by comparing an uncut file to the result.

Also, my suspicions in my previous post have been validated... the missing recordings were due to extremely short segments not having enough padding, and therefore failing to playback in TT. I am pleased to say this has been remedied, and all missing segments are now recorded and accounted for.

With those two obstacles tackled, I now have a 100% complete, bit-perfect rip of each MPX segment.

BUT...

The problem is, each individual segment is supposed to be played for a very specific duration, which does not always perfectly coincide with the end of the segment itself. I believe this is due to frame-based encoding of MPEG audio, and not being able to end each segment with sample-accurate timing. Therefore, each segment actually contains a small (and varying) amount of content longer than what is actually intended to be played.

While I would not be completely opposed to fixing every single alignment issue in those resulting files by ear/hand, I decided it would be best if I could automate the process while also preserving the accuracy of sample length that was coded into the game.

So far I have finished scripting the extraction of the sample length value from each segment. That alone was a pain, as I needed to reverse the byte order and perform a hex-to-decimal conversion, all from within the command prompt. I really should take the time to learn a tool like QuickBMS... Anyway, I ran out of time before work so that's where I am now, writing this.

Just need to apply those sample length values to a file-cropping script, and we should be all set. =)

Edit: Man... I am just now getting around to listening to the retail/final version rip for more than just a quality comparison. Having never played that version, it is really disappointing to realize how much content is missing from the beta/SE version. I guess we have another "DC vs. PSX" quantity vs. quality trade-off on our hands. :roll:
kooz
Moussu
Posts: 39
Joined: Mon Aug 15, 2011 2:00 pm
Tings: 1195

Re: Tonic Trouble Soundtrack

Post by kooz »

And now, I present to you the final "preliminary" rip. All segments are here intact, and I have included a script which truncates them to their original headers' sample-size value. The resulting files will give you near-perfect gapless playback in foobar, etc.

Unfortunately for me, I will never be satisfied with near-perfection. I plan on manually joining these together from the untrimmed complete segments. I have noticed that they occasionally overlap each other, and I aim to ensure each splice is correctly aligned, both visibly and audibly.

So... expect a complete and polished soundtrack from me in the somewhat near-future. In the meantime, here is the complete segmented rip.


https://mega.nz/#!gNFF1SKR!Zm6Sb0Zm6pqc ... 8FYTG-DJDA
kooz
Moussu
Posts: 39
Joined: Mon Aug 15, 2011 2:00 pm
Tings: 1195

Re: Tonic Trouble Soundtrack

Post by kooz »

It's not simply a matter of detecting silence. Some segments have extra audio which continues slightly past where the next segment starts. Joining them after eliminating only silence cannot fix that. It causes a stutter/skip/pop. What I'd like is to find an editor that lets me visually overlay one clip on top of another. That way I'd be able to verify with certainty that I'm cutting at the correct location.

If I can't find something like that, no big deal. I've done this before, obviously. I joined the first 3 or 4 bigfiles a few hours ago, and they are perfect. It will just take a little more time than I would've liked. Especially after all that awesome automation I set up to get to this point :mryellow:

Have you checked out the new set yet, deton?

After running the trim batch file and loading the files up in foobar, I think you'll be pleasantly surprised at how good it is already. I'm just going the extra mile. 8)
deton24
Garatta Ciatik
Posts: 1141
Joined: Fri Jan 14, 2011 1:22 am
Location: Poland
Tings: 8652

Re: Tonic Trouble Soundtrack

Post by deton24 »

If understood well, Audacity allows to use two segments at the same time, in order to join/play them together, and fit the position of the second in matter of the first, in order to make a good transition. Of course everything with waveform.

Sorry, I didn't check yet, but I will.
Post Reply