Realm Blog

Realm Cocoa 0.91.5

We just pushed a Realm Objective-C update to this website and to CocoaPods. This release fixes a bug which could lead to data loss when removing indexes from properties. This bug was introduced in Realm Cocoa 0.91.2.

If you are using Realm Cocoa 0.91.2 or 0.91.3, you should upgrade to 0.91.5 right away.

Users of Realm Cocoa 0.91.4 are not affected because that version was specifically released to disable the faulty functionality. Users of Realm Java were never affected by this issue.

Chain of events:

  • The bug was originally reported on Friday, April 24 at 10:15 PM Pacific, and investigated by the Realm team through the weekend.
  • Root cause identified on Monday April 27 at 2:36PM Pacific.
  • Issue patched on Monday April 27 at 3:17 PM Pacific, with the release of 0.91.4.
  • Functionality resumed on Tuesday April 28 at 10:26 AM Pacific with the release of 0.91.5.

Please accept our apologies for the issue, as well as the following post-mortem. If you have any questions or concerns, we’re always happy to chat at [email protected].

The bug

Realm internally stores the data for your objects in tables, with one column per persisted property. In addition, there are extra hidden columns for storing things like the data for search index, and what objects link to a given object (used to set the links to nil when the destination object is deleted, and for things like linkingObjectsOfClass:). For search indexes, the index data is always stored in the column immediately after the data for the indexed property. For example, given the following data model:

@interface Person : RLMObject
@property NSString *pk;
@property int age;
@property NSString *name;
@end

@implementation Person
+ (NSString *)primaryKey {
    return @"pk"
}

+ (NSArray *)indexedProperties {
    return @[@"name"]
}
@end

You get a table like the following:

Column 1 2 3 4 5
Name pk (hidden) age name (hidden)
Type String String Index Int String String Index

When removing the index for a property, one of the steps is to remove the hidden column with the index data. Unfortunately, the code for this chose the column to remove based on only the visible columns used for properties, rather than adjusting for hidden columns. For example, if you make pk no longer a primary key, it’ll remove the column after column #1, which is correct. However, if you instead remove @"name" from indexedProperties, it should remove column #5, but instead it would remove the column after column #3 (since it’s the third property being deindexed), which is the actual data for the property.

Here were the conditions necessary to trigger this bug:

  1. Have an app using Realm Cocoa 0.91.2 or 0.91.3.
  2. Remove a property name from the array returned by an implementation of indexedProperties in one of your RLMObject subclasses.
  3. Run a migration on an existing dataset.

If you did not meet these conditions, you would not have been affected by this bug.

Why did it happen?

Our primary mechanisms for finding and preventing bugs are having multiple developers review every change to the code, and extensive automated tests for both the core library and each of the language bindings which are run on a variety of platforms and devices for every pull request and push to master. There were several problematic factors which lead to this bug being hard to catch:

  1. It worked correctly in simple realistic cases. One thing that we have many tests for is adding/removing/changing the primary key of a class, which implicitly results in a property having its index removed. However, primary keys are typically the first property in a class, and we didn’t have any tests which had it in a different location and also had another indexed property.

  2. It worked correctly in simple artificial cases. For example, we had a test which added indexes to all columns of a table with several properties, and then removed them all, with verifications done between each step. Unfortunately, removing them was doing in ascending order, so we were always removing the first index in the table, and it worked correctly.

  3. Realm does not modify any existing data on disk during a write transaction until you commit the write transaction. This ensures that reading data from another thread always sees a consistent view of the data, and if your application crashes during a write transaction your changes are not left partially applied. However, this means that reading from a column which has been erased will happen to work fine until the write transaction is committed. This means that a nice focused unit test which doesn’t exercise a bunch of seemingly unrelated code is unable to catch this sort of bug, and only a more complex integration test can.

  4. The problematic bit of code, in isolation, looked “obviously correct” despite actually being wrong.

What we are doing to prevent bugs like this in the future

We consider bugs of this sort completely unacceptable. Our quality processes had so far prevented them, but we will now explore additional steps we can take to improve our testing.

Our primary quality processes will continue to include:

  • peer code review,
  • unit testing of all code changes in our C++ core as well as in Cocoa & Java,
  • automated testing on all platforms with Jenkins, both in simulator and on devices.

Going forward we plan to add:

  • additional dedicated QA resources (for which we have been hiring)
  • formalize code reviews in critical code areas by: requesting additional reviews as well as periodical reviews from members outside the primary team
  • we are also considering maintaining separate “stable” releases containing code that has been vetted in regular releases for a certain period of time.

Although new bugs are always likely to occur, we will continue to find better ways to avoid them, and resolve them as fast as possible and in full transparency. We encourage all bug to be reported in the open on GitHub where others can see them, and where you can see our team’s response to them.

No matter how much testing we do, we still very much need your feedback on bugs & other problems you find in Realm. Please accept our apologies for this issue, and thank you for your continued support in creating Realm. (If you have any questions or concerns, we’re always happy to chat at [email protected].)


Realm Team

At Realm, our mission is to help developers build better apps faster. We provide a unique set of tools and platform technologies designed to make it easy for developers to build apps with sophisticated, powerful features — things like realtime collaboration, augmented reality, live data synchronization, offline experiences, messaging, and more.

Everything we build is developed with an eye toward enabling developers for what we believe the mobile internet evolves into — an open network of billions of users and trillions of devices, and realtime interactivity across them all.

Get more development news like this