randy's Recent Posts

An update of Aalto and Kaivo to 1.8.5 is out, fixing a possible crashing bug introduced with 1.8.4 and optimizing file handling. Because the bug only manifested for certain users, it was an interesting one to fix and I wrote a little about it, afterwards:

Six days ago, I was feeling happy and content at the end of a work day. I'd just deployed my 1.8.4 release that added a useful expert feature and a handful of bug fixes. I'd tested in all the usual hosts on both MacOS and Windows, and both plugins were making the right sounds and not eating too much CPU and most importantly, not crashing. So with my celebratory bourbon in hand... you already know where this is going.

I got a couple reports back right away that Aalto was crashing immediately on being scanned. One person reported in that Kaivo was crashing too, but most people were not having problems with it. In short, a whole rotten gift basket of ugly issues was collected in the field, but nothing was reproducible here. This was one to sleep on.

The next morning I upgraded one of my Macs to Mojave to match a customer's software and hardware setup exactly. Still no problems here. I spent the day reviewing code I'd just written and looking for bugs. I'd just done some fairly low-level work on timing, changing a lot of objects to use the C++ std::chrono library for their clocks. (C++ is what I use to write all the plugins, a sprawling mess of a language that can make very fast programs, used by most all audio software developers.) I found a couple of suspicious areas and spent a day on that, making the code more clear and obviously correct. Sent out an update to a few folks—same results. Guessing about what might be crashing and fixing anything suspicious is not a great way to track down bugs, but sometimes it's all I can productively do until a better course of action becomes clear.

One of the crashes seemed to be in code that was making lists of files. What if a particular preset was causing it? So, I asked a nice Aalto user to send me their preset library. And boom! Right away the crash.

I looked at the offending C++ code, which began:

void Path::parsePathString(const char* pathStr)
{
  const int pathStrBytes = strlen(pathStr); 
  SmallStackBuffer<char, kShortFragmentSizeInChars> buf(pathStrBytes);
  ...

What this parsePathString does is read a raw sequence of characters in memory and turn it into a compact Path object that can refer to an object in a collection, like DSP objects in a graph, or files on disk.

The strlen function is from the old C standard library that, for better or worse, is available as part of any modern C++ toolset. It counts the number of characters in a null-terminated string—that's some piece of text defined by where it starts in memory and containing all the bytes in the following memory until a zero is reached. Sensibly, (?) the zero is not counted as part of the length.

The text SmallStackBuffer<char, kShortFragmentSizeInChars> buf(pathStrBytes) is an description of an object to be made. The presence of the <> brackets tells us the object is defined by a C++ template. Inside the brackets are the arguments to the template, and these describe a kind of variation on the theme of the basic object. In this case, SmallStackBuffer is one of my own memory utilities that holds some data in a different place depending on how much data there is. Without getting too deeply into it, if there's a small amount of data it can go into a commonly used area called the stack, ensuring that other memory is not allocated, an operation that may lead to audio glitches. kShortFragmentSizeInChars is the constant that determines how much data will go onto the stack.

Back to the scene of the crash. The file name "Xenos Soundworks - Forbidden Experiments/MA Bass Engine (Use MW)" was being read at the time. (Check out their Soundbanks for Aalto and many other synths.) Calling the strlen function on this string returns: 64. That's a power of two, which might be a clue, and sure enough, it turns out that 64 was the value of kShortFragmentSizeInChars. The crash starts to look like a crime scene. Time to look at what happens next, very carefully.

The next thing that happens to a 64-character file name is, it gets written to the SmallStackBuffer object—including the final null terminator which goes to location 65 out of 64, overflowing the object's memory on the stack. The null might be written to part of another object, which could lead to the program crashing right away. Or, depending on when that other object is accessed, it might not crash for a while, which is why memory corruption like this can be so hard to diagnose.

How do we fix this? The easiest fix is very easy. Just change strlen(pathStr) to strlen(pathStr) + 1. Then there will be room for a 63-character file name and its terminator byte. Post update, go upstairs, back to celebratory bourbon. But hold up. To improve stability and efficiency over time I treat each bug as an opportunity to make all the code better, to fix not just this one instance of something having gone wrong but rather a whole class of things with the potential to go wrong. What can I learn from this mistake? Instead of just adding a +1 I'm going to look at all of my code and tests, and I've got four tasks ahead of me.

The first task is to look at my use of strlen across all of the code for all my plugins. If I messed up this way once, I'm likely to have done it elsewhere. Thankfully I have not used strlen many times, because of the potential for problems like this very one. The only good place to use it is in a higher-level object or function that will abstract away the confusion. Searching, I find there are only five other places in my own code where I've used strlen, and I can quickly verify that I'm doing the right thing in all of them.

Secondly, I'm going to look at all the code for parsePathString. Zoom out a bit. Could I have been doing this whole operation in a different way that didn't open the door to the problem? Yes, it turns out. The stack was getting corrupted because I was writing the path string to a temporary buffer in the process of decoding it to make symbols. That was the easiest way to use the Unicode text library I'm relying on. It should be possible to do the same thing just reading the path and not writing it, which will make the program a little faster, but how to do that wasn't clear the first time around. Digging into the library a little more deeply, I understood how to do it, and made a faster version of parsePathString that never has to allocate any heap memory at all.

Third, I'm going to add tests that could have caught this problem before it happened to anyone else. The Path object is a part of my core library madronalib and it really should be tested automatically as part of each release. Testing it with a string of length 64 is the edge case that would have caught this, so I'll add that as well as the neighboring lengths and some weird strings like empty and badly-formed ones.

Finally, there's a simple way I could have figured this out sooner: by having had the magic preset file in my collection to begin with! I usually have on hand only the "factory patches" I've made here, because when I make music I'm always making patches from scratch. But lots of people have made Aalto patches over the years and they are a bigger collection of data that might help shed light on other aspects of the code in the future. So I'll be grabbing some sample banks to have on hand.

Every programmer I know has days like the past few, and could empathize if I'd just said "I got stuck on a stack corrupting off-by-one bug" instead of the above. I wrote this more for myself, to cement good practices, and for the non-programmers out there, to share some feeling of what it's like to do this work. Some days it's even more fun.

No worries... Audio CPU use should be more or less the same in 1.8.5. I've changed a lot of interface timer code so UI stuff may feel snappier, which I don't have a good way of quantifying but it does feel better to me.

It's the "master_tune" parameter. I wrote more about it with the 1.8.4 release. https://www.madronalabs.com/topics/7588-kaivo-and-aalto-updated-to-1-8-4

An update of Aalto and Kaivo to version 1.8.4 is available now on the product pages. This fixes a few compatibility issues that showed up in recent versions of Live, and a bug introduced in version 1.8.3 where Kaivo would stop making reasonable sound.

Aalto and Kaivo changes:

  • removed debug code that was causing a crash in certain hosts
  • streamlined OSC services
  • add master_tune program parameter, accessible via program text
  • fixed t3d input in unison mode

Kaivo-only changes:

  • fixed a filter bug that could lead to runaway noise
  • improve interpolation filters in Body

The master_tune parameter addresses the need of some players to move the reference pitch of A to some value other than 440 Hz. This parameter is saved with each patch. To edit it, first select "copy to clipboard" to get the patch as text then paste into a text editor. Change the value after "master_tune", and then paste the text back into the plugin using "paste from clipboard." I'll be adding a UI for this feature in the future but for now, this works and lets you tune Aalto to your old piano or whatever.

The interpolation filter change to Kaivo should not be too obvious, but may be noticeable as a reduced harshness when sending bright tones to the body.

OK, the 1.8.5 release for Mac and Windows is posted.

I figured this out. Update today or tomorrow.

[EDIT: womp womp]

@datolar to go back to older versions, you may have to delete the newer DLLs manually before running the installer.

@datolar If you want to go back to an older version, you must remove the Aalto DLL files manually before running the installer.

Sorry for the trouble, I'm working on this.

Meanwhile 1.8.3 is still available:
Mac: http://madronalabs.com/media/aalto/Aalto1.8.3.pkg
Win: http://madronalabs.com/media/aalto/AaltoInstaller1.8.3.exe

Since they are newer you will have to delete your 1.8.4 plugins manually first, then install 1.8.3.

I still can't reproduce this crash. All my tests are passing.

Happy to rule out some VST3 weirdness.

It seems like all the Mac crashes are on High Sierra, so I'm going to try installing that.

Meanwhile 1.8.3 is still available:
Mac: http://madronalabs.com/media/aalto/Aalto1.8.3.pkg
Win: http://madronalabs.com/media/aalto/AaltoInstaller1.8.3.exe

Since they are newer you will have to delete your 1.8.4 plugins manually first, then install 1.8.3.

Also it looks like there's no released version of Live that supports VST3. If you are trying to use a Live 10.1 beta please let me know. I can't support beta versions of hosts, but I'll be willing to try running it to track down any issues.

All the updates are available from the main product pages as usual.

I'm trying in Windows 10 as well as Mac OS and have experienced no crashes. Sorry for the difficulty. I have no idea what is going on but I'll hopefully find out soon with your help.

When you report an issue, please please send also all the version numbers of your OS and host! Not just the names like "10 Suite" or "High Sierra" but the numbers e.g. "Live 10.0.6" and "Mac OS version 10.13.1".

If you got a crash please send Ableton or MacOS crash logs to: support@madronalabs.com

I do not make a VST3 version of either Aalto or Kaivo. If you are seeing that a VST is present, where do you see this information?

If you have Live 10 but less than 10.0.6, please upgrade to the latest version of Live because there are known issues with plugins in 10.0.3, -4.

It's working here for me in Bidule, Windows 10 x64.

I would make sure you don't have any additional copies of Aalto around. I'm not sure yet but it's possible the new Aalto may be causing a crash in this situation where the old one handled it OK.

Also please update to the latest Bidule (0.9761) if you can, so we rule out a difference there—I tried to make Bidule scan a duplicate Aalto VST but it is smart and won't allow it, so I can't duplicate this problem yet.

I'm going to make this thread sticky. It should be a good place to find and share Aalto patches. I'll try to post one every day or two for a while.

It would be cool if we could embed Soundcloud links here, but setting that up will take some time.

A place for Kaivo patches! Just select copy to clipboard from the patch menu, and paste here to share.

I've deployed the update to the main product pages now.

I've now deployed the release to the main product pages.

There have been some crashes that revealed themselves in more recent versions of Ableton Live and it sounds like you are experiencing that. Sorry for the frustration.

I'm just about ready to release an upgrade to both plugins. Currently squashing some final issues with Kaivo. Aalto should be fixed. You can try the Aalto installer:
http://madronalabs.com/media/aalto/Aalto1.8.4.pkg (Mac)
http://madronalabs.com/media/aalto/AaltoInstaller1.8.4.exe (Win)

You are not hearing the triggers because you have 'x vel' on ENV1 turned on. This multiplies the output of the envelope by the MIDI note velocity. Maybe not intuitively, it also applies when the trigger is coming from the sequencer! So without any MIDI input you won't get audible envelopes.

I'm working on the Windows update and will release ASAP.

Yes it will. Sorry the beta is only on Mac this time because that's where I'm developing. thanks for your patience.

Yes it will. Sorry the beta is only on Mac this time because that's where I'm developing. thanks for your patience.

Great Journey, thanks for sharing! I like the feel of this stuff many people would do with sequencers but played live.

I don't think putting videos here is possible and I guess that's a feature.

Checked out AE modular a bit and that looks like a fun project.

...

Full support! I look forward to its development.

A bump so the beta is more visible. Kaivo and Aalto releases coming soon.

Good news! I'll get the full release out including a Windows version ASAP.

I think I have fixed this. A beta for Mac OS is available at http://madronalabs.com/media/kaivo/Kaivo1.8.4b2.pkg .

Yup, I've been thinking the same thing. Not only did computers get faster, I did a lot of optimization since little Aalto 1.0 came out. So 8 voices is no problem. I feel like that's the sweet spot because the scope-dials get very cluttered with more indicators in them six or so (one per voice).

In any case more voices is a definite plan for Aalto 2 along with a big list of other improvements I have in mind. On the inevitable question of timing I'll only say "hopefully later this year."

I know Roger Linn has done some demos with it, which I was happy to see. So there must be others...