Realm Blog

Realm ObjC & Swift 2.3: Sync Progress Notifications, Improved Sharing & Backup Recovery!

by /

We’re releasing version 2.3 of Realm Objective‑C and Realm Swift today, which includes several improvements to the Realm Mobile Platform features, such as sync progress notifications, backup recovery, and more flexible sharing mechanisms. Read on to learn more!

Sync Progress Notifications

Realm Mobile Platform offers a true offline-first experience, where changes can be applied immediately to the local Realm irrelevant of the network conditions. In the background, the synchronization happens automatically, and when the network is available, changes quickly propagate across devices and the server. This enables the realtime, collaborative experiences unique to Realm.

There are times, however, when it’s useful to know how much data is left to transfer. Perhaps on the first launch of your app, you’d like to make sure a sizable amount of the data is available before presenting your user interface. At other times, your app would simply benefit from showing when data is synchronizing, by presenting a progress bar or activity indicator.

This is why we’ve added APIs to monitor sync progress by registering notification blocks on SyncSession. Simply, specify the transfer direction (.upload/.download) and mode (.reportIndefinitely/.forCurrentlyOutstandingWork) to monitor.

For example, to show an upload progress bar after writing a large image to your Realm and waiting for it to upload, you might write this:

let session = SyncUser.current.session(for: realmURL)!
self.token = session.addProgressNotification(for: .upload,
                                             mode: .forCurrentlyOutstandingWork) { progress in
  self.updateProgressBar(fraction: progress.fractionTransferred)
  if progress.isTransferComplete {
    self.hideProgressBar()
    self.token.stop()
  }
}

Or to show an activity indicator for the entire lifetime of your app, you might write this:

let session = SyncUser.current.session(for: realmURL)!
self.token = session.addProgressNotification(for: .download,
                                             mode: .reportIndefinitely) { progress in
  if progress.isTransferComplete {
    self.hideActivityIndicator()
  } else {
    self.showActivityIndicator()
  }
}

Flexible Sharing Mechanism

Realtime collaboration and data sharing is an important pillar of the Realm Mobile Platform. To enable this, we made it possible for mobile developers to share Realms between users when we introduced the PermissionChange APIs in November.

Today, we’re making it even easier to share Realms between users, allowing extremely flexible sharing and permission management, all controlled from the Realm client APIs and without the need to write any server code! Specifically, we’re adding the SyncPermissionOffer and SyncPermissionOfferResponse classes to allow creating and accepting permission change events for synchronized Realms between different users.

These APIs once again demonstrate the power of the Realm Mobile Platform’s objects-as-APIs approach. Making and receiving permission offers is very similar to the existing PermissionChange process. Simply create the object in the current user’s management Realm and observe it to know when it was synchronized and processed by the Realm Object Server.

Sharing a synchronized Realm is as easy as following these steps:

  1. Create a SyncPermissionOffer object in the user’s management Realm.
  2. Wait for the offer to be synced and processed by the server.
  3. Once the token property of the object is populated, send it to another user however you’d like: email, iMessage, Action Controller, carrier pigeon, whatever you like!
  4. The receiving user then creates a SyncPermissionOfferResponse object in his/her management Realm.
  5. The receiving user then waits for the response to be synced and processed by the server.
  6. Once the response has been processed, the receiving user can now access the Realm at the response’s realmUrl property.

For example:

////////////////
// Sender
////////////////

// Create offer with full permissions
let shareOffer = SyncPermissionOffer(realmURL: realmURL, expiresAt: nil,
                                     mayRead: true, mayWrite: true, mayManage: true)
// Add to management Realm to sync with ROS
try managementRealm.write {
  managementRealm.add(shareOffer)
}
// Wait for server to process
let offerResults = managementRealm.objects(SyncPermissionOffer.self).filter("id = %@", shareOffer.id)
shareOfferNotificationToken = offerResults.addNotificationBlock { _ in
  guard case let offer = offerResults.first,
             offer.status == .success,
             let token = offer.token else {
    return
  }
  // Send token via UIActivityViewController
  let url = "realmtasks://" + token.replacingOccurrences(of: ":", with: "/")
  let activityViewController = UIActivityViewController(activityItems: [url], applicationActivities: nil)
  self.present(activityViewController, animated: true, completion: nil)
}

////////////////
// Receiver
////////////////

// Create response with received token
let response = SyncPermissionOfferResponse(token: token)
try managementRealm.write {
  managementRealm.add(response)
}
// Wait for server to process
let responseResults = managementRealm.objects(SyncPermissionOfferResponse.self).filter("id = %@", response.id)
acceptShareNotificationToken = responseResults.addNotificationBlock { _ in
  guard case let response = responseResults.first,
             response.status == .success,
             let realmURL = response.realmUrl else {
    return
  }
  // User can now access Realm at realmURL 🎉
}

We have a Proof-Of-Concept branch towards RealmTasks that demonstrates how sharing lists between users could be built using this mechanism.

Backup Recovery

When your servers go down, you need a plan to recover. That’s why the Realm Mobile Platform has offered the ability to back up your data for a few months now. Today we’re announcing our Continuous Backup solution to automate keeping up-to-date backups.

Under normal conditions, Realm’s synchronization engine works by transferring just the specific operations. When the Realm Object Server confirms the receipt of new operations, the local logs are cleaned up. This helps keep your app’s disk usage small and Realm blazing fast ⚡️.

Furthermore, because Realm is an offline-first database, if your Realm Object Server is down for whatever reason, all your local data stays available.

However, if you need to recover from a backup on your server, your clients will receive a “client reset” error via the global error handler. Note that you may continue to use the local Realm as you normally would, but that any subsequent changes, or changes made after the last backup point, will be lost.

Once you receive a “client reset” error, you could inform the user about the situation, stop accessing the Realm and redownload the Realm from the server at the latest backed up version. The NSError’s userInfo property contains a block that you may call to clean up the previous local version of the Realm.

If you choose to not handle the client reset error immediately, the next time your app launches, the previous local version of the Realm will be deleted and redownloaded from the server at the latest backed up version automatically on first access.

Bug Fixes

  • Fix a call to commitWrite(withoutNotifying:) committing a transaction that would not have triggered a notification incorrectly skipping the next notification.
  • Fix incorrect results and crashes when conflicting object insertions are merged by the synchronization mechanism when there is a collection notification registered for that object type.

Legacy Swift Version Support

We’d like to remind you that we will continue to support Xcode 7.3.1 and Swift 2.x as long as we can, but encourage all our users to migrate to Swift 3 as soon as possible.


Thanks for reading. Now go forth and build amazing apps with Realm! As always, we’re around on Stack Overflow, GitHub, or Twitter.

Read more

Realm Java 2.3: Improved Sharing, Backup Recovery and Wildcard Queries

by /

We just released a new version of Realm Java to this website and to Bintray. It contains a new wildcard query syntax as well as several improvements to the Realm Mobile Platform such as backup recovery and a more flexible data sharing and permission management system. Read on to learn more.

Flexible Data Sharing

Realtime collaboration and data sharing is an important pillar of the Realm Mobile Platform. To enable this, we made it possible for mobile developers to share Realms between users when we introduced the PermissionChange APIs in November.

Today, we’re making it even easier to share Realms between users, allowing extremely flexible sharing and permission management, all controlled from the Realm client APIs and without the need to write any server code! Specifically, we’re adding the PermissionOffer and PermissionOfferResponse classes to allow creating and accepting permission change events for synchronized Realms between different users.

These APIs once again demonstrate the power of the Realm Mobile Platform’s objects-as-APIs approach. Making and receiving permission offers is very similar to the existing PermissionChange process. Simply create the object in the current user’s management Realm and observe it to know when it was synchronized and processed by the Realm Object Server.

Sharing a synchronized Realm is as easy as following these steps:

  1. Create a PermissionOffer object in the user’s management Realm.
  2. Wait for a notification that the token of the object is populated by the server. Once the token property of the object is populated, send it to another user however you’d like: push notification, email, carrier pigeon!
  3. The receiving user then creates a PermissionOfferResponse object in his/her management Realm.
  4. The receiving user then waits for the response to be synced and processed by the server.
  5. Once the response has been processed, the receiving user can now access the Realm at the response’s realmUrl property.

For example:

/**
 * Sender
 */
SyncUser user = getUser("user");
Realm managementRealm = user.getManagementRealm();
String sharedRealmUrl = "realm://my.server/~/my-realm";
boolean mayRead = true;
boolean mayWrite = true;
boolean mayManage = true;
Date expiresAt = null; // Offer never expires;
final PermissionOffer offer = new PermissionOffer(sharedRealmUrl, mayRead, mayWrite, mayManage, expiresAt);
String offerId = offer.getId();
managementRealm.executeTransaction(new Realm.Transaction() {
    @Override
    public void execute(Realm realm) {
        realm.insert(offer);
    }
});

// Wait for server to handle the offer
RealmResults<PermissionOffer> offers = managementRealm.where(PermissionOffer.class)
        .equalTo("id", offerId)
        .equalTo("statusCode", 0)
        .findAll();
offers.addChangeListener(new RealmChangeListener<RealmResults<PermissionOffer>>() {
    @Override
    public void onChange(RealmResults<PermissionOffer> offers) {
        PermissionOffer offer = offers.first();
        String token = offer.getToken();
        // Offer is ready, send token to the other user
        sendTokenToOtherUser(token);
    }
});

/**
 * Receiver
 */
final SyncUser user2 = getUser("user2");
Realm managementRealm = user.getManagementRealm();

// Accept the offer
String offerToken = getToken();
final PermissionOfferResponse offerResponse = new PermissionOfferResponse(offerToken);
managementRealm.executeTransaction(new Realm.Transaction() {
    @Override
    public void execute(Realm realm) {
        realm.insert(offerResponse);
    }
});

We have a Proof-Of-Concept branch towards RealmTasks that demonstrates how sharing lists between users could be built using this mechanism.

Backup Recovery

When your servers goes down, you need a plan to recover. That’s why the Realm Mobile Platform has offered the ability to back up your data for a few months now. Today we’re announcing our Continuous Backup solution to automate keeping up-to-date backups.

Under normal conditions, Realm’s synchronization engine works by transferring just the specific operations. When the Realm Object Server confirms the receipt of new operations, the local logs are cleaned up. This helps keep your app’s disk usage small and Realm blazing fast ⚡️.

Furthermore, because Realm is an offline-first database, if your Realm Object Server is down for whatever reason, all your local data stays available.

However, if you need to recover from a backup on your server, your clients will receive a “client reset” error via the global error handler. Note that you may continue to use the local Realm as you normally would, but that any subsequent changes, or changes made after the last backup point, will be lost.

The next time your app launches, the previous local version of the Realm will be deleted and redownloaded from the server at the latest backed up version automatically on first access.

Wildcard Queries

Until now, the string query predicates have been limited to substring searches. With this release, we introduce the like predicate which can do simple pattern matching. ? will match a single-character while * matches zero or more characters. For example

String[] names = {"Jim", "Simon", "Jimmy", "Tommy", "Jamie"};

realm.beginTransaction();
for (String name : names) {
    Person person = realm.createObject(Person.class);
    person.setName(name);
}
realm.commitTransaction();
RealmResults<Person> persons1 = realm.where(Person.class).like("name", "J?m*").findAll(); // => Jim, Jimmy, Jamie
RealmResults<Person> persons2 = realm.where(Person.class).like("name", "?im*").findAll(); // => Jim, Simon, Jimmy

Please read the API documentation to get the full picture on how to use like.

Bug Fixes

  • Fixed native memory leak setting the value of a primary key.
  • Fixed “too many open files” issue.
  • Activated Realm’s annotation processor on connectedTest when the project is using kapt.

Thanks for reading. Now go forth and build amazing apps with Realm! As always, we’re around on Stack Overflow, GitHub, or Twitter.

Read more

JP Simard on Realm & Open Source on the Consult Podcast

by /

Consult is a monthly interview podcast hosted by David Kopec. It focuses on software developers who work on Apple platforms to create client products.

JP Simard, iOS lead at Realm, joins the show to discuss the Realm Mobile Database and the new Realm Mobile Platform. JP digs into the technical details that make Realm a great choice for developers. He also discusses his prodigious open source work and his prior life as lead of an iOS consulting firm.

Listen to the full interview here, or check out the episode’s transcript below.


Introduction (0:00)

Hi everyone. Welcome to the November 2016 episode of Consult. I do want to reach out to the community, so if you know somebody who you think would be great on Consult, let me know. I’m @davekopec on Twitter.

My interview this month is with JP Simard of Realm. You all know Realm, the popular, open source mobile data base that’s really starting to take on Core Data. We really dig into the details of Realm and we also get into JP’s previous career as an iOS consultant.

A lot of you probably know JP from the open source community. He’s also the iOS lead at Realm.

Background (1:58)

Q: Tell us a little bit about your background. Go way back. How did you first get into computing? I know you had a consulting company for a number of years, tell us about that too.

JP: Going way back for me doesn’t actually go back all that far. My first intro into programming was in university. I was taking electrical engineering and I was honestly pretty fed up with the very long project cycles. You send something over to fab, or you do simulations. Essentially, every project either takes something on the scale of months or years, if you look at commercial projects. That just wasn’t very exciting for me.

I ended up building this technology that was linked to electrical engineering. Then I wanted to build an app around it. I stared programming, trying to figure out how you actually do that and then I realized that I had to pay the bills. I went around a few different festivals in my hometown in Ottawa, Canada. I tried to pitch them on making an app, even though I had never done it before. Eventually one of them stuck, and ended up having to build an app for them. This was in 2011, right when I was getting started.

I got by right by the skin of my teeth. I managed to patch together just enough of an app and interface builder to satisfy the contract and move on. I continued with that process, going door to door, asking friends, acquaintances, trying to figure out who needed an app, saying that I could build it without ever having anything close to it. Over time I ended up actually fulfilling those contracts, then having more contracts and growing that team.

Essentially, that became a business that was initially spawned by this idea of just scratching my own itch, making my own app. Ultimately, I never ended up building that app, but did grow that agency, something called Magnetic Bear Studios into a small team. It consisted of a few developers, a few designers and making apps for all sorts of companies across North America. At that point I was really hooked into programming. There was no escape.

iOS in Canada (4:29)

Q: Very interesting. Tell us a little bit about the iOS scene in Canada, in Ottawa more specifically. I actually lived for Ottawa for a year when I was a little kid, but I know nothing about the iOS scene there. What’s it like?

JP: I love it because it’s a very tight knit community. Now living in Silicon Valley, in San Francisco, there are definitely a lot of people in the community here. You tend to go to meet-ups. There are tons of them to start off with, and they’re actually competing in some sense; competing for the market share of local developers. Not only that, you tend to have such large groups or groups with members that pop in and out every now and again. There’s a really big difference between that, and say specifically, the Apple developer community in a small-ish city like Ottawa where you have people who are really ingrained in the community.

It’s a very small, tight knit group, a group of maybe 20 or 30 really hardcore people who have been doing Mac programming since the 90s, for some of them. People who have really been ingrained in the industry. There are so few of them there, relatively speaking, compared to a major hub like New York City, London, or San Francisco, that you really do have to stick together. It’s very regular meetings for groups like Cocoa Heads, for example, and it’s people who are keen on sharing ideas and sharing what they learn on a very regular basis and keeping in touch.

It’s an interesting dynamic, and I certainly miss it in a lot of ways. I like being nostalgic every now and again, thinking back to those days. In the last few years, there’s this conference called NSNorth that’s run by Philippe Casgrain and Dan Byers. They’re able to attract people from all around the world to come to this conference because they’re so passionate about it and because they’ve been doing it for decades at this point, and the community is really small but strong.

Clients (7:00)

Q: I’ve always heard tremendous things about NSNorth and what a great experience it is for the attendees. When you were at Magnetic Bear Studios and running that, did you have mostly local clients or were the clients from all across Canada and the United States?

JP: For the most part they were local clients. We did have a handful of clients when we were growing and making a name for ourselves that were everywhere from Silicon Valley to both coasts of Canada. Ultimately, what lead to the end of the Magnetic Bear Studios agency was really my own limitations. At that point, I was running this business that I had no idea how to run. It was mostly focused on hustling and doing sales, doing business development, doing everything from accounting to everything that falls into managing small business.

Ultimately, not only was I not very good at that, I also didn’t really like it. All of that lead to the conclusion that towards the end of that three and a half year stint I was really looking for something a little bit more technical to sink my teeth into and that’s when I joined Realm.

Building a Team (8:24)

Q: I want to get more into Realm, but still talking about Magnetic Bear Studios, what was it like building out that team in Canada? Was there enough iOS local talent there? What’s the supply and demand like for iOS labor in Canada?

JP: Answering that in the present tense is really different than the landscape as it was in 2011-2013 when I was really involved. Back then, people were still riding the mobile wave as the hot new thing. Today you look at buzz words and they’ve shifted. Even though mobile is now a concrete part of a lot of businesses’ strategies, back in 2011 and 2012 people knew it was going to be big, it was already established at that point. The iPhone had been out for three or four years at that point.

It was already established, but companies still hadn’t fully caught up and created their own in-house teams. We were really at a good point, and got really lucky to ride the wave, honestly. That’s probably why the company survived at all.

If I was trying to do the same thing today, starting from scratch with no knowledge of the industry, it would be a lot harder to replicate that; at this point, a lot of the companies that used to hire agencies like Magnetic Bear Studios, or small agencies in general, now tend to have their own in-house group of developers and designers who can lead their mobile effort.

It’s not longer a fad, it’s no longer hype, it’s just like the web was five years ago. It’s expected that if you have some sort of electronic presence that you definitely need a mobile strategy of some sorts. The landscape has certainly shifted.

Consultancy Challenges (10:27)

Q: Right, that makes absolute sense. What was your key takeaway in terms of challenges of running an iOS consultancy during those three and a half years?

JP: Trying to put myself back in those shoes from a few years ago, which is becoming increasingly harder to do, I recall some of the challenges or at least some of the things that I used to complain about, at least internally. Even in 2011 and 2012, I thought I was too late to ride the hype and wave for mobile.

There were a handful of agencies that had popped up around the world, and I was finding myself competing with the more established smaller sized agencies that had done what I was doing. That was pretty tough, and I can only imagine someone trying to do the same thing today would have a much harder job of actually trying to pull that off. In retrospect, although I was doing that not at the very earliest time and maybe not at the perfect opportunistic time, the conditions were still really quite good to pull that off.

You look at the landscape now and not just in agencies but just in mobile development in general and the race to the bottom is really won. Very few people can make a living off of apps on the App Store and the strategies haves completely shifted towards either subscription based or in app purchases. Ultimately, we were already starting to see that trend three or four years ago, and that’s one of the reasons why I essentially grew a company as a contractor and as an agency rather than as a company developing apps directly for the App Store.

In a sense, a lot of these companies that were hiring us knew they needed a mobile presence, but didn’t know exactly how to monetize it and probably wouldn’t have been able to in a very good way anyway. Really the only way to bootstrap a company from the ground up was to get people who were willing to invest because end users of applications weren’t necessarily ready even at that point.

Starting at Realm (13:09)

Q: Tell us how you ended up at Realm and tell us a little bit about your role there.

JP: In 2013 and 2014, we were starting to see the limits of Magnetic Bear Studios and growing that as an agency. I was really lucky to have some very talented people, but ultimately, all the people in my team still had somewhat of an entrepreneurial spirit and building apps as an agency tends to wear you down. It’s not exactly for everyone, even though I do look back fondly on those days where we could have a wide variety of projects on an ongoing basis. It really kept you on your toes and kept you interested.

Ultimately, towards the end like I mentioned, I was doing more of the business side of things and I A) wasn’t very good at it and B) I didn’t enjoy it. I was looking to get more into a technical role and that’s where I saw that this stealth company in Silicon Valley was hiring. If you know me at all, especially in those days, those kinds of words were really a big turn off: “VC funded stealth company.” Back then, I was really all about bootstrapping a company, creating value right from day one rather than the promise of value for an investor.

Ultimately, the closer I looked at this company who didn’t even broadcast their name at that point, the more I appreciated that different types of companies and products require different types of business models and even bootstrapping processes. You look at a database that needs to be built from bottom up, and your minimum viable product there is a fully ACID compliant database. You can’t really just build half a database and release it out to the world.

If you’re going to disrupt a technological space like that that needs several years and a small team at least of upfront investment before you can actually start to monetize a product out of it, then venture capital or investment based strategies is really the only way unless you take more of a side project approach to building something like that. To be honest, I don’t really think that that would have worked with such a core fundamental technology as a database. I really appreciated and respected the approach that Realm took at the time.

I was chatting with the founders and at that point they had been building Realm’s underlying storage engine for the past three years. Starting summer 2011 and going through Y Combinator, that’s where they were really starting out. They had this concept of modernizing database engines, especially for embedded databases, in a way that really didn’t make sense 15 years ago when SQLite was developed. If anyone’s been in the mobile development industry here, even in the embedded space in the last decade and a half, they know that SQLite is a very robust type of software, but it didn’t always start off that way.

It needed to grow into that level of stability. Also, it was designed at a time when we didn’t have the smartphone revolution as we know it today. We didn’t exactly have iPhones or Android phones or even this concept of model oriented or object oriented, object relational mapping libraries. You look at Core Data, I mean you had web objects and stuff, but not so much in the embedded space.

You look at Core Data coming along and the genesis there really had the right ideas, where it was like, “Let’s build something that’s object oriented because that’s how we want people to write software.” Not this kind of table and row, kind of snapshot based system where you essentially query some external thing to ask what its state is and then you go and translate that. If you actually had a system like an ORM that can keep that in sync with the models and the objects that are really integral to object oriented programming, then you’re really burring the lines and at that point persistence and more database type concepts, like ACID compliance, then becomes more of an implementation detail.

You look at the progression of technology in the 90s, the early 2000s, even the late 2000s and people really built amazing things on top of the fundamental technologies that they had. Then you get to 2010, 2011 and you really reach a tipping point where you’re starting to stretch the useful space in which those technologies designed ten years ago really start to apply and some of these concepts start to break down. You start to have arm based chips that are really prolific. You have SSD’s, you have vectorized operations and CPU’s, and these are all things that if you were designing a low level database today, you would really want to take advantage of those things.

It’s hard to take this older system that’s been battle tested, made very robust over the course of a long set of years, five, ten years, and then say, “Okay, well let’s re-architect this.” No matter what, at that point, anything you touch has an impact on the memory layout, has an impact on small things. That when you consider the scope of how wide it’s distribution is, you really start to realize that it’ll have very strong knockout effects. It’s very likely that it’ll break something somewhere.

You’re put in this impossible situation where you can’t innovate without breaking things, but people rely on these foundational technologies to be robust before anything else. Robustness over feature completeness, really. That’s where throwing out that concept, basically starting from scratch, really comes in handy. Where you can consider some of the research that was done in modern times about multi-version concurrency control, about copy on write semantics, about making the most of SSD’s and modern components, and modern computing architecture and really build something from scratch.

That’s what Realm did and that’s what ultimately drew me over to them. This promise of starting fresh. If we were to build something, a database for the smartphone revolution era. That’s what really got me in.

Working at Realm today (20:41)

Q: What’s your role at Realm today?

JP: My role today is leading the Apple software teams. We have two core products and these are user facing products. Those are Realm Objective-C and Realm Swift. I lead the team that’s responsible for building those, but ultimately the kinds of work that we do, that our team does, which is about seven or eight people at this point, spans everything from writing cross platform low level functionality, even at the Core Database level that can be used for other products, like Realm Java or Reactnative.net.

It spans from that to doing specifically Objective-C and Swift stuff, or even the Realm browser which is a Mac app that we have, to doing documentation tests, sample apps, tutorials, and community outreach as well. We don’t have any pure Realm evangelists in the company. We just have the engineers that are working on the product.

Why work at Realm? (22:03)

Q: Give us the elevator pitch for Realm. If I’m an iOS developer and I’m looking at my options in terms of data store, why should I pick Realm?

JP: The main thing that we strive for there is simplicity. Simplicity on so many levels. You look at a system, like a traditional ORM, and there’s so much intermediate state that needs to be maintained to keep the object side and the relational side in sync. That’s where you start to introduce faults, where you’re accessing something on one thread and another thread was mutating it, but because of the shared global mutable state that’s in common there, halfway through an operation you hit a fault and say, “Well, another thread deleted an object in a completely different context and I’m suffering the consequences there.”

Simplicity at that level. Everything from the most foundational layers up. Where if you lose this object relational mapping it’s just objects the whole way day. That simplicity translates into an easier way to reason about what’s happening under the hood. It translates to performance as well because you’re doing less work. It translates to safety because when the very lowest levels are built with that concurrency in mind, then you don’t need to have multiple layers where you need to ensure that that safety is preserved if it’s kept as simple as possible.

There’s simplicity on that end of the foundational technologies, and there’s also simplicity in terms of usage and the API. One of the things that we really strive for, more than feature completeness or even performance, is ease of use and maintainability. Initially when we released Realm Objective-C, which was our first public product, we had one idea of what the developer community was looking for and then they responded with something, not exactly completely different, but it did surprise us.

Initially we thought that speed was a very important factor and after talking to a lot of developers, a lot of them, if it’s fast enough, that’s all that matters. As long as they can have their UI table view that swirls at 60 frames a second, they’re happy. It doesn’t really matter if you can perform this complex object graph type query where you’re traversing many object graphs in a fraction of the time that it would take you with an ORM, that doesn’t matter as much for the vast majority of developers.

Another thing that we found is that a lot of the ping points like performance that people tend to experience with traditional solutions, people tend to internalize that. Where it’s part of normal development. You don’t exactly remember exactly how much effort it was to get things working within a reasonable time frame. We found out that performance as such wasn’t an end goal for most people, that it was actually more simplicity, maintainability, and the ability to have code that lasts for a long time, that doesn’t need to be constantly updated.

Core Data is complicated (25:41)

Q: That’s very interesting. A complaint of many new iOS devs is, “Why is Core Data so complicated?” Is it fair to say that Realm is competing with Core Data?

JP: Absolutely. It depends from what angle you look at it. Realm is definitely an object data layer and you can consider Core Data to be very similar in that sense. The types of things that you would do with Core Data, Realm is really a suitable replacement. There are design decisions that we’ve done to make sure that people migrating from Core Data to Realm have an easy path there. I’d say, especially in recent weeks, ever since Realm launched the Realm Mobile Platform, Core Data starts to really be more of kind of a subset in competition to Realm rather than an exact kind of competitor, if you will.

The Realm Mobile Platform, you can look at it as sync. It’s data synchronization between devices, between servers, but really if you peal back the layers, it’s a lot more than that. You look at apps today and a lot of the complexity goes into things like parsing JSON, doing network requests, doing caching, knowing when to pull, when to push, when to cache, when to invalidate a cache, what parts of that cache to invalidate, et cetera.

You look at all of that cruft and if there was a way to cut down on a lot of that complexity, app developers would really be able to focus a lot more on the value that they want to add in their app, on the kinds of functionality that really sets them apartment from the competition, compared to doing all this boiler plate stuff that has such a high investment cost and also so many ways in which things can go wrong.

You consider a traditional application that uses rest API’s and JSON serialization, caching layer, and you look at a round trip between keeping data that’s on the server, which is typically the canonical source of truth, in sync with at least what the application presents the user. There’s so much cruft that needs to go into that there if there was a way to cut down on that, you would really drastically simplify the way that people build apps today.

In a sense, Realm isn’t exactly competing with any one thing, we’re really trying to upset the status quo so that you can drastically simplify your apps, make them a lot more robust, while adding things like real time functionality with very minimal effort. That’s really kind of the approach that we’re taking.

Realm Performance Relative to Core Data (28:42)

Q: While performance is not your number one goal, simplicity is, how does Realm perform relative to Core Data? Are there any areas where Core Data outperforms it?

JP: There certainly are. There are fewer and fewer. Our mantra is to be within the same ball park as SQLite, for the most part. When it comes to benchmarks, one of the reasons why we’ve stopped producing benchmarks, at least externally, is because you can always make them look good for you, or for your use case. If you set the right size, the right transaction, or the right amount of concurrency competition that’s happening, you can make one product look much better than the other. We can tweak those numbers to make any product clip better than the rest of its competition.

We really do take performance very seriously and we have a number of performance tests that we will at least make sure that we don’t regress on a lot of key metrics or key operations. I’d really leave that to the end user to determine for themselves if for their use case Core Data, SQLite, or Realm is much faster than anything else. What I will say is that we really do strive to hit a sweet spot where we can be as performant, if not more than SQLite or Core Data, for the vast majority of types of architectures that mobile developers tend to use these days.

But I’d stop short from saying that any one product is considerably faster than anything else. You even look at Core Data and some of the things that it does is actually faster than Ross SQLite, depending on the type of operation you’re doing. I’d really leave that up to end users to determine for themselves.

Back end Realm usage (30:46)

Q: Realm is on several different platforms. Is Realm ever used as a web back end as well?

JP: It certainly is in several ways. One is in the same way as MySQL, or PostgreSQL, even SQLite can be used in server circumstances, where you need a transactional ACID database. Realm fits the bill there. There’s another angle to this, and this is probably the more common one, and that’s with the Realm Mobile Platform. Having a database on the server side that can communicate and coordinate with remote clients to facilitate things like synchronization and even things like event handling, that’s a much stronger use case.

That being said, Realm on the server is still in its early days, at least from a product standpoint. I wouldn’t say from a technological standpoint, but from a product standpoint, for sure. The only official language SDK that we provide really for the service is JS, is JavaScript. You look at the rest of our product line and you’ve got C sharp for .NET, you’re got Java for Android, you have Object C and Swift for Apple platforms, so that’s iOS, Mac OS, TV OS and Watch OS. None of those are designed to run say in a Linux infrastructure. The one product that we’ve specifically honed for that is the no JS SDK.

Q: The reason I asked the question is because Realm started as an embedded database for mobile devices. It seems like over time the database is spreading its wings and becoming a more general purpose database, is that a fair characterization?

JP: You could say that. Another way to put it is that Realm really started with the fundamental building blocks of building a very robust distributed database. That’s where the whole transaction log comes into play where we preserve the semantic intent of all the operations. This comes in nicely when you have distributed systems, especially when you need really high partitioning tolerance where devices can be offline at any time and you don’t want the performance of the database to take a hit for that.

If you’re in an intermittent connectivity area, which is basically all of mobile devices, all of mobile networks, because even as you’re walking through a building, even as you’re walking down the street switching between cell towers, you have intermittent connection, the last thing that you want is for your on device performance to suffer. Since the very beginning, since the genesis of the project, we’ve built everything in preparation for having a really robust distributed system. I’m really excited that we can finally offer that as a product now.

Realm Mobile Platform (34:13)

Q: Tell us more about that product. The big announcement from Realm in the last couple months was the Realm Mobile Platform. What is the Realm Mobile Platform?

JP: The Realm Mobile Platform is essentially exactly what you said. It’s taking Realm on device and making it spread its wings. The mobile platform lets you do things like device-to-device synchronization, it lets you do things like real time collaboration, data back up and collaboration.

If you wanted to build something similar to Google Docs where you have multiple users that have different permission levels that are concurrently accessing a document making changes and having it all resolve automatically, those are really some of the main use cases that we’ve honed the Realm Mobile Platform to support.

There’s a number of other things that you can do. Earlier I was saying if you wanted to cut down on the complexity of your local stack, you remove the necessity to do JSON serialization, network requests, cache logic for invalidation and cache checking, if you wanted to do all of that, you could move a lot of the functionality from your app into the server and then at that point all the server really needs to do is to write objects to its local realm. Because of the way the Realm Mobile Platform works, that data is automatically available on the devices that have access to that realm.

Imagine you had an app that took a picture of a wine label and automatically gave you information about that wine bottle. What you could do is you could write a local object in your local embedded database, local embedded realm on your device that has the picture for that. Because of the way sync works, the server would then very quickly have access to that information, it could listen to changes that are happening, see, “Oh, this device just added a picture. Let me do all of the back end processing that I need to do, talk to several different servers, some of them maybe online, offline, some of them may have high latency,” whatever.

For the most part because it’s doing this work on a back end infrastructure, it typically has a much better internet connection so it can do a lot of this work and then once the response comes back from whatever this server that was listening to this change did, it can just write the response to its local realm object and then the device would then see that change locally in its local data store. You can really cut down on a lot of the complexity by sharing, say you have a cross platform app, you can share all of this business logic on the server where you don’t need to duplicate it and rewrite it in Java or Swift, or whatever, while still maintaining all of the functionality.

Q: Is it fair to say that the Realm Mobile Platform is in the same category as something like CloudKit, or what Parse was, or Firebase?

JP: There’s a lot of overlap when you mention products like those. There’s a lot of really key differences. You’re looking at CloudKit and for the bulk of what it offers, it really is for the most part a key value store. You have things like iCloud document syncing, but that’s really done more of on a very course grain document level. Then you look at something more like Firebase and it doesn’t really have much in terms of persistence or database type operations. You still need a solution for that locally on device.

Then you get into other differences, such as the merge algorithm for one, how do conflicting changes get resolved? This is something that Core Data with Core Data Sync tried to do with its ubiquity store. Ultimately there were a few architectural challenges with getting that to work whereby building that at an ORM level rather than at a database level, you lose the transaction log of the database. You lose all of the fine grain operations that are done.

Whereas with Realm, if you actually build synchronization and merge conflict resolution from the very lowest levels up, you maintain that intent, that semantic intent of operation. When you have things like a move or a re-name or sub-string operations, if you were doing that at a high level without access to a transaction log, you have no idea if a change happened because of a move, because of a modification.

You can apply heuristics and try to guess what actually happened and what the intent of the user was, but as soon as you have a moderately complex use case, and you really don’t need to look very far to get that, that system starts to break down. That’s where you get things like incorrect merges. That’s really why Realm went and essentially reinvented the concept of an end-to-end database was to provide the fundamental tools to be able to do this efficiently and easily.

Q: Would it be fair to say that Realm is more robust than those other two solutions because it’s an end-to-end solution?

JP: I would certainly say so, but like you said, that’s generalizing certainly. Depending on your use case, you may not necessarily think so. I’m hard pressed to think of anything specific right now, but the last thing that I want to say is that this hammer that we’ve built, which is Realm, is a tool for all problems out there. The old adage, “When you have a hammer, everything looks like a nail.”

Sure you can do a lot with Realm. One of the reasons why we focused on a fundamental low level side of things was that it’s easier to build flexible solutions on top of robust primitives, rather than say have a really easy to use but super high level wrapper that’s a black box and that we don’t expose anything below the very surface. That’s what I say.

Q: Can developers install the Realm Mobile Platform back end server on their own servers or do they need to go through Realm?

JP: No, they actually have to install it on their own servers. We don’t offer a hosted solution right now. This is both a good thing and a bad thing because if you’re looking to own your data, if you’re looking to have full flexibility in terms of how it’s deployed, how it’s exposed, how it’s maintained, then the way that we currently offer the Realm Mobile Platform is really ideal for you.

This tends to be what most larges businesses tend to look for. That’s really who we’ve aimed to target from the very beginning. If you’re looking for a hosted solutions, we don’t offer anything right now and we are keeping our finger on the pulse in terms of how enterprises are using us in their own deployments to be able to figure out, “Okay, well is there a way that we can learn from this to maybe offer a fully hosted solution?” If that comes, it’ll come later.

Q: It sounds like you’re doing it in the right order. A lot of people felt very burned by what happened with Parse and that lead to a lot of consternation about having the data store itself be on someone else’s hardware. It sounds like you’re going about this in the right direction.

JP: We think so too, but obviously that’s more of a matter of opinion. If you’re a single person developer, managing your own infrastructure on top of your app might not necessarily be the answer that you’re looking for. We completely understand that, but on the other hand that’s exactly why we chose to fully open source the local embedded database. If you’re running Realm on your device, if you’re not using Sync, you’re using an entirely open source solution.

At that point there are no shackles, there is no lock in. You can do absolutely whatever you like. You can fork it and modify it to your heart’s content. We really hope that individual developers aren’t taken aback by our lack of hosted solution offering. On the other hand, it also helps us really focus on delivering the absolute best experience for the people are willing to run this on their own infrastructure for now. We can then apply those lessons maybe later on when we’re ready to offer maybe a different type of product. The future will tell.

Q: Okay, so a bit of a cliff hanger. Maybe there will be a hosted solution in the future but that’s not yet determined, is that what we’re hearing?

JP: We’re not offering anything right now and we are focusing on self-hosted solution at this point.

Realm and the Open Source Community (43:57)

Q: Realm, the main database, is open source. Realm is very involved in the open source community. Tell us a little bit about that.

JP: One of the reasons why I wanted to join Realm in the first is really this ethos of contributing and being a participating member of the community. You’ll see a lot of companies that are in the developer tools space who might sponsor a lot of conferences, or they might have booths at trade shows and things like that. Although we do that from time to time, we aim to be a contributing member of the community. Not only does that help us to better keep in touch with our user base, but it also helps users because they can interact with us in a more fluid way.

That’s why we’ve been running the Swift Language User Group and hosting those videos on our site. Whenever possible with conferences, we’ll provide video recording, hosting, and transcription services for free, rather than just asking them to put our logo on a banner and giving them money for it. We want to be an active participant. That’s more on the community engagement side.

On the open source side, the high level SDKs have been open sourced since day one. Realm Objective-C, Realm Swift, Realm Java, JavaScript, .NET; all of that’s been open sourced. The core engine, however, was closed source for the first two years. There’s a handful of reasons there, but ultimately we want to be very transparent in terms of what our business model is and where Realm fits in, not just as a community member, but as a company.

You mentioned some other companies earlier and how people got burned. If you’re not fully clear about what your business model is, even though some people might not exactly be fully aligned with those priorities, then you’re really not being transparent. That really hurts people who are trying to build a business of off our products and our offerings. In that sense, keeping the core closed sources was an important message there, since we certainly did need to clean it up before it was available to the public, but also that’s where our intellectual property was. Being transparent about that was way more important than anything else.

Q: That makes complete sense. I want to also talk a little bit about some of your open source projects, like PeerKit and SourceKit. Tell us a little bit about them.

JP: PeerKit and SourceKit are really personal projects of mine. There are a handful of non-database related company projects that we have open sourced, like Jazzy, SwiftLint, and the Realm Browser.

Jazzy is an Objective-C and Swift documentation generator. That really was birthed out of a necessity for the company to have documentation for our own product.

Two years ago, at WWDC 2014, Swift was announced. This was funny timing because this was right around the time that we were gearing up to release Realm Objective-C. We said, “You know what, let’s just wait a week and see what happens at WWDC.”

Sure enough, Apple announced this new language and blindsided everyone. We essentially gave ourselves a week to support the language, and then released our product. Building a product that’s birthed into this new language, you need documentation for that language. That’s where Jazzy came about’ it was for our own business need to document our product, and it was a really fun project.

I loved hacking on that, reverse engineering Xcode, reverse engineering SourceKit, back when it was closed source and trying to figure out how to build a robust documentation generator. Another option would have been to build a parser for Swift, but if you look at Swift’s tumultuous history in the last two years, it would not have been fun at all.

Swift is a much more complex language than Objective-C, where in Objective-C you have very simple syntax. I’m not saying that it’s easier to learn at all, but from a parsing standpoint it’s a lot easier to par Objective-C than Swift. For one, you don’t have things like struts and language, sure there’s C++ struts and things like that. Another point is that you only ever have one level of nesting in Objective-C. You have classes and they can have properties, but you can’t have a class within class within property, or a closure within a function within a class.

Whereas with Swift, you have so much flexibility and so much complexity that follows that building a parser would have been a nightmare. The investment of reverse engineering Xcode and SourceKit seemed like a smarter choice and in retrospect I think it was.

There’s Jazzy and then there’s SwiftLint which is more of a linter for Swift that was really done more out of my own frustration that people hadn’t built one yet. I built SwiftLint in a weekend in May 2015 because I was giving a talk on fixing the Swift tooling problem at UIKonf in 2015. My point at that talk was really to say, “Sure, Swift is a new language and therefore has a lot of room for improvement, especially around tooling,” and also that, “We shouldn’t necessarily have to rely on Apple to build that tooling.”

I’ve been saying for a while that using something like SourceKit that you could pretty easily built a linter because Apple has gone through the efforts of exposing a way for people to build tooling for Swift. I’d been saying that for over a year at that point saying, “Hey, someone should really just built a linter on top of SourceKit.” As I was writing the talk, as anyone who’s given a talk before, you really want to procrastinate and do anything but. I ended up writing that on the plan just to show, to kind of make my point from the talk that, “Hey, we don’t have to wait for Apple to build tooling, we can do it ourselves.”

I built it then and it really exploded from there, mostly because no one else had really done it. Since then, there’s some terrific formatters and linters that have come out, but I think the community really latched on to SwiftLint and now it’s pretty actively used project.

Conclusion (52:04)

Q: Jazzy and SwiftLint are really fantastic projects and I recommend everyone listing check them out. Is there anything about Realm that we didn’t cover that you really want developers to know about it?

JP: That’s a good question. I think we really touched on a lot, especially around the Mobile Platform. This is really on us at Realm to do a better job of talking about this, but there are a handful of limitations in Realm that albeit annoying, are actually there for a very good reason. It’s not to say that they can’t be lifted, eventually, just that there’s a reason why they’re there.

One thing that comes to mind is thread confinement. Earlier you may have heard me say that Realm was a threadsafe database. What I mean by that is that transactions are isolated and any changes that are being done by one thread, or a different process, or even a different device that’s being synced, won’t effect your current consistent view of the database. We do this via a number of different ways: multi-version concurrency control, snapshots and copy-on-write.

One of the side effects of this is because Realm is an object graph database and that one thread may be accessing a different snapshot of the database than another, passing an object across a thread is actually very dangerous. What could happen is that as you’re passing this object, for a person that has a child property, a relationship or a link, what happens when the two different snapshots between those two threads have different values for that relationship? Or even different values between scalar primitive properties, like a String or an Integer?

You get into a situation where you don’t want to have a later snapshot try to access an object from an earlier snapshot. You get into this dependency resolution state in which it’s impossible to have a general solution. That’s why for safety we say, “Every object is confined to its own thread.” That’s not necessarily done out of our inability to let you access the object from another thread. It’s done so that people don’t accidentally write unsafe code that expects thing to be a certain way and then there data just changes from under them.

That being said, there are ways that we can improve ways to work around this, but you’re never going to be able to really pass data between threads or freely write data outside of a write transaction and that’s do to wanting transactional mutations. So if you need to change two things that if your app crashes half way through, then you don’t end up in this inconsistent state.

There’s a bunch of these limitations that people love to complain about and they are annoying (I’m the first to admit that), but the solution isn’t as simple as giving people the first solution that would come to mind. We’re working on solutions for a lot of these things and I really just appreciate everyone’s patience in understanding the complexity of the issues.

Q: I really recommend everyone check out Realm. How can our listeners get in touch with you, and where can they follow you?

JP: People can get in touch with Realm via Twitter, @realm or Realm.io on the interwebs. Email us [email protected] if you have any questions or issues. For me personally, I’m @SimJP on Twitter and then “JPSim” on everything else, like GitHub.

Q: JP, it’s been fantastic having you on the show. Thank you so much for taking the time, and I hope that all of our listeners learned a lot about Realm.

JP: Dave, thank you for having me, and I hope this was helpful!

Read more

Realm Objective-C & Swift 2.2: Objects across threads, sort over relationships & more!

by /

One of Realm’s earliest design goals was to have a consistent and straightforward threading model. Today, we’re releasing version 2.2 of Realm Objective‑C and Realm Swift to improve our threading model by introducing a mechanism to safely pass objects across threads. We’re also releasing the ability to sort by relationship properties, some sync improvements and a few bug fixes. Read on to learn more!

Following the Thread

Today, Realm makes it easy and safe to work with your objects in multiple threads. Our current multi-threading support is the result of years of steady effort and deliberate design decisions (because threading is hard).

Our previous Threading Deep Dive article lays out many of the philosophies behind the way Realm safely handles concurrency without requiring users to think about locks or resource coordination, all while providing a consistent view of the entire Realm and its object graph. Notably, this design architects away the concept of ‘faults’ present in other ORMs and data frameworks 💥.

Our documentation on Threading is also a must-read resource to truly understand how to best use Realm in concurrent environments. With these resources, you can make extremely productive use of multi-threading and Realm. But until now, the objects you use haven’t been able to cross threads.

Thread Confinement

If Realm’s so thread-safe, why do I get exceptions when I try to pass my object between threads?!

In order to provide the consistency and safety Realm guarantees, a simple constraint is imposed: all Realms, objects, results, and lists are confined to the thread on which they created.

It’s important to understand that thread-confinement isn’t a temporary limitation based on Realm’s internals, or an artificial constraint, but rather a crucial part of the design that makes it easier to write correct code.

In fact, freely passing Realm objects across threads would be very easy for us to implement, but would have tradeoffs that would render it very dangerous and unpredictable to use.

Realm is a transactional database (think the scope of a write transaction) that allows an app to crash halfway through a write without having partially valid data written to disk.

To have isolation (the ‘I’ in ACID), changes in different transactions aren’t reflected in other transactions until those transactions advance to the latest version. Otherwise, you’d get ‘faults’, which Realm doesn’t have thanks to this design.

To aid in this isolation, Realms on a given thread are only ever at the same version (under the hood, there’s only a single Realm per-thread). Passing a Realm, or Realm objects/queries/etc freely across threads would mean that you’re mixing different versions of the database together, which would lead to very surprising results. For example, passing an object to a thread that has since deleted it could cause a crash, or it could change values as it crosses the thread boundary, or an object graph could have different relationships in either transaction version.

The Old Way to Pass Data Across Threads

Up until now, in order to pass data across threads, you’d need to pass data that wasn’t backed by a Realm.

This was commonly done by passing unmanaged instances of Realm objects, or reading data out of Realm-backed properties such as a primary key value.

let realm = try! Realm()
let person = Person(name: "Jane", primaryKey: 123)
let pk = person.primaryKey
try! realm.write {
  realm.add(person)
}
DispatchQueue(label: "com.example.myApp.bg").async {
  let realm = try! Realm()
  guard let person = realm.object(ofType: Person.self,
                                  forPrimaryKey: pk) else {
    return // person was deleted
  }
  try! realm.write {
    person.name = "Jane Doe"
  }
}

However, this doesn’t work at all when objects don’t have a primary key, and may result in you operating on stale data. Passing something other than a Realm Object, such as a List, Results or LinkingObjects, also can’t easily be done with this approach.

Using Thread-Safe References

Now, you can create thread-safe references for all types that were previously thread-confined within a Realm, and passing objects between threads becomes a simple, three-step process:

  1. Initialize a ThreadSafeReference with the thread-confined object.
  2. Pass that ThreadSafeReference to a destination thread or queue.
  3. Resolve this reference on the target Realm by calling Realm.resolve(_:). Use the returned object as you normally would.

For example:

let realm = try! Realm()
let person = Person(name: "Jane") // no primary key required
try! realm.write {
  realm.add(person)
}
let personRef = ThreadSafeReference(to: person)
DispatchQueue(label: "com.example.myApp.bg").async {
  let realm = try! Realm()
  guard let person = realm.resolve(personRef) else {
    return // person was deleted
  }
  try! realm.write {
    person.name = "Jane Doe"
  }
}

Real World Example 🌏

Thanks to RealmTasks being open-source, the pull requests replacing existing thread passing code with thread-safe references can be viewed on GitHub: iOS PR #374.

Here’s the relevant code, where we automatically deduplicate a List property, performing the write transaction on a background thread:

realm.addNotificationBlock { _, realm in
  let items = realm.objects(TaskListList.self).first!.items
  guard items.count > 1 && !realm.isInWriteTransaction else { return }
  let itemsReference = ThreadSafeReference(to: items)
  DispatchQueue(label: "io.realm.RealmTasks.bg").async {
    let realm = try! Realm()
    guard let items = realm.resolve(itemsReference), items.count > 1 else {
      return
    }
    realm.beginWrite()
    let listReferenceIDs = NSCountedSet(array: items.map { $0.id })
    for id in listReferenceIDs where listReferenceIDs.count(for: id) > 1 {
      let id = id as! String
      let indexesToRemove = items.enumerated().flatMap { index, element in
        return element.id == id ? index : nil
      }
      indexesToRemove.dropFirst().reversed().forEach(items.remove(objectAtIndex:))
    }
    try! realm.commitWrite()
  }
}

Sorting By Relationship Properties

Until now, Realm Collections could only be sorted by “direct” properties.

Starting with Realm 2.2, you can now sort collections by properties on the other side of an object’s to-one relationships.

For example, to sort a collection of Persons based on their dog property’s age value, you could call dogOwners.sorted(byKeyPath: "dog.age"):

class Person: Object {
  dynamic var name = ""
  dynamic var dog: Dog?
}
class Dog: Object {
  dynamic var name = ""
  dynamic var age = 0
}

realm.beginWrite()

let lucy = realm.create(Dog.self, value: ["Lucy", 7])
let freyja = realm.create(Dog.self, value: ["Freyja", 6])
let ziggy = realm.create(Dog.self, value: ["Ziggy", 9])

let mark = realm.create(Person.self, value: ["Mark", freyja])
let diane = realm.create(Person.self, value: ["Diane", lucy])
let hannah = realm.create(Person.self, value: ["Hannah"])
let don = realm.create(Person.self, value: ["Don", ziggy])
let diane_sr = realm.create(Person.self, value: ["Diane Sr", ziggy])

let dogOwners = realm.objects(Person.self)
print(dogOwners.sorted(byKeyPath: "dog.age").map({ $0.name }))
// Prints: ["Mark", "Diane", "Don", "Diane Sr", "Hannah"]

To accomplish this with previous Realm versions, it was necessary to store the dog.age property on the Person object, or to sort outside Realm, losing all advantages of Results.

This is an API breaking change: instead of referring to ‘properties’, we’ll use the more general ‘key paths’ term.

  • The following Objective-C APIs have been deprecated in favor of newer or preferred versions:
Deprecated API New API
-[RLMArray sortedResultsUsingProperty:] -[RLMArray sortedResultsUsingKeyPath:]
-[RLMCollection sortedResultsUsingProperty:] -[RLMCollection sortedResultsUsingKeyPath:]
-[RLMResults sortedResultsUsingProperty:] -[RLMResults sortedResultsUsingKeyPath:]
+[RLMSortDescriptor sortDescriptorWithProperty:​ascending] +[RLMSortDescriptor sortDescriptorWithKeyPath:​ascending:]
RLMSortDescriptor​.property RLMSortDescriptor​.keyPath
  • The following Swift APIs have been deprecated in favor of newer or preferred versions:
Deprecated API New API
LinkingObjects​.sorted(byProperty:​ascending:) LinkingObjects​.sorted(byKeyPath:​ascending:)
List.sorted(byProperty:​ascending:) List.sorted(byKeyPath:​ascending:)
RealmCollection.sorted(byProperty:​ascending:) RealmCollection.sorted(byKeyPath:​ascending:)
Results.sorted(byProperty:​ascending:) Results.sorted(byKeyPath:​ascending:)
SortDescriptor(property:​ascending:) SortDescriptor(keyPath:​ascending:)
SortDescriptor​.property SortDescriptor​.keyPath

Realm’s commitment to SemVer means that these deprecated methods will continue to be supported until Realm hits 3.x.

Other Changes

Sync Breaking Changes (In Beta)

  • Underlying sync engine upgraded to version BETA-6.5.
  • Sync-related error reporting behavior has been changed. Errors not related to a particular user or session are only reported if they are classed as ‘fatal’ by the underlying sync engine.

Bug Fixes

  • Setting deleteRealmIfMigrationNeeded now also deletes the Realm if a file format migration is required, such as when moving from a file last accessed with Realm 0.x to 1.x, or 1.x to 2.x.
  • Fix queries containing nested SUBQUERY expressions.
  • Fix spurious incorrect thread exceptions when a thread id happens to be reused while an RLMRealm instance from the old thread still exists.

Legacy Swift Version Support

We’d like to remind you that we will continue to support Xcode 7.3.1 and Swift 2.x as long as we can, but encourage all our users to migrate to Swift 3 as soon as possible.


Thanks for reading. Now go forth and build amazing apps with Realm! As always, we’re around on Stack Overflow, GitHub, or Twitter.

Read more

Introducing the Realm Mobile Platform

by /

 

Introduction

If you’ve been following Realm for a while, you probably had an idea that the Realm Mobile Platform was coming. It took some pretty hardcore research getting it to actually work, as it’s a project that has been underway for at least two years.

Realm’s goal has always been to make it possible for people to build better apps: ones that have amazing live and interactive features, that are more easily and quickly buildable, since those kind of apps are currently far too hard to build.

Realm Mobile Database

Realm has quite a long history, and today we’re doing great! We have offices in San Francisco and Copenhagen (some of us have a Danish background). We have around 50 employees now, and we’ve raised money for ourselves. But it didn’t start off like that! It’s funny that it all started with just me and my co-founder, who had this crazy idea while sitting in Denmark that we wanted to build a mobile database.

We applied to Y Combinator. I know there are people who say that if you apply to Y Combinator, you need to know all the right people and fit a certain profile, etc, but we were two people sitting totally alone in Denmark. We didn’t know anybody, and we had no credentials other than our skills and the fact that we’d worked at Nokia doing mobile stuff for a long time. We were total nobodies, and very naive.

Still, we sent our application video, and it was amazing that it was well received! It’s a long story from there, but we tried harder and got everything up and running. We got started and created the mobile database. It was the core engine of what we wanted to do.

Being young, when we started we thought, “How long time can it possibly take?” We both had lots of experience in this field, so we imagined that it would take around half a year before we would have something we could sell. Here we are, several years later, and we actually have a decent product! We launched in 2014, almost three years after starting the company. It takes a long time to build an embedded database, especially if it’s a good one. It’s probably a good thing we didn’t know that when we started, because we may not have started at all. Now that we’re here, it’s really cool!

Now we have this technology in place, developed, fully open source, and it’s pretty awesome. We promised from the beginning that we would open source it, and now you can do whatever you want with it. It’s always a pleasure to give things to the world.

What is most satisfying for me is how much people love it. We get tons of tweets and feedback where people talk about their experience with how cool Realm is. We’re also used pretty much everywhere, which is amazing. We’ve changed so many apps!

The Realm Mobile Platform

After releasing Realm Mobile Database, we needed a next step. It’s great to make a cool database and give it to everybody for free, but that’s not how you create a sustainable company. We needed the Realm Mobile Platform.

One anecdote that most people won’t know is that the original inspiration for Realm came because I was building a text editor at my own company. It was a collaborative text editor with git built in (before git was invented), and I wanted to have real-time collaboration. No database was fast enough to have the type of real-time collaboration I was looking for, so I wanted to make it for myself. I started building that, but the whole idea was to build an ideal database that allows people to share data in real-time. It turns out that it takes a long time, because you actually have to build a database first!

We got a lot of requests for synchronization, and a lot of people wanted to know when it was coming. Today, you want real-time stuff that just works, where you can update and have everything be reactive. It’s currently difficult to make things real-time in your app; you should be able to just connect your UI directly to the data and it should just update. With our platform, you can have these features with hardly any code at all, and it’s easy to do.

You want a reactive setup, where you can just create all your objects, connect them to your UI, and it should work. From my original vision, it should be collaborative. Everybody is getting mobile phones, and we want people to work together. Every app should be collaborative. It should help people to create, so we’re not just sharing cat pictures, but we should have quality apps that actually help people do things. We want everybody to have a seamless experience that just works, online and offline. That was the vision, and it’s what we’ve heard from users.

Realm Mobile Platform is the counterpart to the Database, because you can only do offline for real if you have a real database on the device. That’s key, otherwise it always falls through and the experience is not good. The magic of this is that you can have a database, but the database is just a webpage. Instead of opening the database with a filepath on your disc, you just open it with the URL. You open the shared Realm, and you can open one, ten, or hundreds of them. Anybody who opens the same URL will have the same data, at the same time, in real-time. It’s a very simple concept, but it’s pretty amazing when you actually work with it.

The idea is that the database is offline-first by nature, because it’s real object database. It’s not a SQL database with OM layers, copying data back and forth with inconsistent data. It’s a real database that’s relational, so you can have links to everything. You also have chain notifications that allow you to do reactive things. Now you can totally abstract away the network handling. Instead of thinking about networking, you simply open the Realm at the URL, and it’ll naturally synchronize with whoever else just opened the same Realm.

Other cool things include encryption, which you definitely want today, in both the database and the network. It does a real-time sync, and we actually have event handling, so you can run code on the server and see responsive changes.

Of course, the real magic is in the automatic conflict resolution: the fact that you can adjust the changes. Why automatic conflict resolution instead of manual? The problem is if you do manual conflict resolution, you can never really do real-time, because things just stop. You really want to be able to do it at a real-time speed, so it has to be automatic.

You can actually change the scheme on the fly, you can push data to devices, and with the event frame you can introduce to an existing infrastructure. You have all these devices that all connect to the server. The server has a sync engine that makes all the automatic conflict resolution work. It has an event framework to run code on the server. There’s a dashboard, of course, so you can see what’s happening. There’s authentication, access control, and then you have all the Realms. Every single device can have its own Realm, and they can have Realms that are shared between all of them. You can have as many Realms as you want! That allows you to scale up so you can have millions of Realms, and you’d basically spread out your data around them. We can also connect to other servers, so if you have backend databases or APIs, you can actually make connectors to those. It will integrate into your existing infrastructure.

Live Objects

The key thing that enables this is the idea that everything is just objects all the way down. It’s objects, and the objects are live. If you use Realm already, you have this idea that even if you’re on multiple threads, you can change through multiple threads and you’re still safe, and you’ll see the updates in the other threads. We take that one step further and say that also happens with the network. You can have multiple devices and they all update the objects, but it’s always the same thing: they’re just live objects, they change. The objects are the data model, which really simplifies the mental model. It makes it so much easier to build and maintain apps. The idea is that you can define objects exactly in the same syntax you’d usually do it in.

You create these objects just like regular objects, but suddenly it’s kind of like supercharging them. You get the ability to do transactions and queries on them, you can do persistence, and you get all that for free. It all just works. Plus, it’s still full as a database, and it’s observable. With these features, this is the foundation that allows you to build pretty much anything you want in an app!

The Realm Mobile Platform: it’s available now, so download it! There’s a totally free version for developers: you can run it on your own hardware, and build anything you want with it.

We also have a commercial counterpart (the company has to live!) that solves enterprise needs and has more integration into your backend and more server-side handling, and custom authentication. But the free version is super powerful too! You can build full apps on it with no rate limits. You can use it for anything, as much as you want, for as long as you want.

Read more

SwiftLint 0.14 is out: CocoaPods installs, Linux support, and more!

by /

We released SwiftLint 0.14 over the weekend, and it’s probably our biggest release yet! We added 14 new rules, and shipped CocoaPods, SwiftPM, and Linux support. We’ll dive into the rule changes first, and then explore the optimizations we made to keep the SwiftLint platform useful as Swift continues to grow and change.

We’ve marked the locations of errors and warnings with arrows () so you can see without having to run these snippets yourself!

New Rules

Dynamic Inline (Daniel Duan)

It’s always nice to see rules added to SwiftLint that aren’t just stylistic, but prevent real bugs from creeping into your code.

This rule will trigger if you mark a declaration with both the dynamic and @inline(__always) attributes, since that combination leads to undefined dispatch behavior.

class C {
  @inline(__always) dynamic func f() {}
}

Number Separator (Marcelo Fabri)

This rule encourages the use of a number separator when writing large number literals, preferring 10_000_000 over 10000000. I think you’ll agree this makes for more readable code. It’ll also catch misleading separations, like 10_000_00!

var foo = 10_0
foo = 1000
foo = 1__000
foo = 1.0001
foo = 1_000_000.000000_1
foo = 1000000.000000_1

File Header (Marcelo Fabri)

This opt-in, configurable rule let you specify a format that all Swift file headers should or shouldn’t conform to.

For example, all SwiftLint files should use the following template:

//
//  FileHeaderRule.swift
//  SwiftLint
//
//  Created by Marcelo Fabri on 27/11/16.
//  Copyright © 2016 Realm. All rights reserved.
//

So we include a regular expression to validate in our .swiftlint.yml file.

You can configure this rule to require or forbid certain strings or regular expressions. Some users already use this to prevent Copyright from being included in their headers. To each their own. ¯\_(ツ)_/¯

Redundant String Enum (Marcelo Fabri)

Given that Swift implicitly assigns raw enum values, it’s a bit redundant to duplicate each String enum member’s name as its associated value. This rule will trigger when all explicitly assigned values are character-for-character equal to the member names. If even one of the names differs, this rule won’t trigger so you can stay consistent within your enum and keep your sanity.

enum Numbers: String {
  case one = "one", case two = "two"
  // and
  case one = "one", two = "two"
  // and even
  case one, two = "two"
}

Attributes (Marcelo Fabri)

This opt-in & configurable rule validates if an attribute (@objc, @IBOutlet, @discardableResult, etc) is in the “correct” position.

The correct position in this case being what’s been established as convention by the Swift community:

  • If the attribute is @IBAction or @NSManaged, it should always be on the same line as the declaration.
  • If the attribute has parameters, it should always be on the line above the declaration.
  • Otherwise:
    • If the attribute is applied to a variable, it should be on the same line.
    • If it’s applied to a type or function, it should be on the line above.
    • If it’s applied to an import (the only option is @testable import), it should be on the same line.

You can also configure what attributes should be always on a new line or on the same line as the declaration with the always_on_same_line and always_on_line_above keys.

Empty Parentheses with Trailing Closure (Marcelo Fabri)

In Swift, both [1, 2].map { $0 + 1 } and [1, 2].map() { $0 + 1 } are valid syntaxes! Using these interchangeably is at best inconsistent and at worst confusing. This rule will trigger when empty parentheses are used:

[1, 2].map( ) { $0 + 1 }

Closure Parameter Position (Marcelo Fabri)

Closure parameters should be on the same line as opening brace. For example:

// Prefer
[1, 2].map { number in
  number + 1
}
// Over
[1, 2].map {
  number in
  number + 1
}

Prohibited Super (Aaron McTavish)

Some of Apple’s methods should not call super when overridden, according to their documentation. This opt-in rule will trigger when one of these methods does call super in its implementation. For example:

class VC: UIViewController {
  override func loadView() {
    super.loadView()
  }
}

Empty Parameters & Void Return (Marcelo Fabri & JP Simard)

Apple and the Swift community has settled on this style:

FWIW, we’ve recently decided to standardize on () -> Void (generally, () for parameters and Void for return types) across all of our documentation.

This rule will trigger (and automatically correct) incorrect usages:

// Prefer
let abc: () -> Void = {}
// Over
let abc: Void -> Void = {}
let abc: () -> ()

Operator Usage Whitespace (Marcelo Fabri)

This opt-in rule will trigger if an operator isn’t surrounded by a single whitespace on either side when used:

// Prefer
let foo = 1 + 2
let v8 = 1 << 6
// Over
let foo = 1+2
let foo=1 + 2
let v8 = 1 <<  (6)

Unused Closure Parameter (Marcelo Fabri & JP Simard)

Here’s another rule that can help prevent bugs, or at least make your code mode self-documenting. This rule will trigger (and automatically correct!) any unused parameter in a closure by replacing them with _. For example:

[1, 2].map { number in
  return 3
}
[1, 2].something { number, idx in
  return number
}

Unused Enumerated (Marcelo Fabri)

Here’s another rule that can help prevent bugs, or at least make your code mode self-documenting. This rule will trigger if the indexes aren’t used iterating over the results of an .enumerated() call. For example:

for (_, foo) in bar.enumerated() {}
// or
for (_, foo) in abc.something().enumerated() {}

Platform Changes

Port to Swift 3

We designed SwiftLint to work with any Swift version you’ll throw at it. That means our very first releases from a year ago work out of the box with Swift 3, and that you can use our very latest releases with older Swift versions.

Learn more about how SwiftLint finds which toolchain to lint with here.

Thanks to this design choice, we weren’t in a rush to rewrite all of SwiftLint in Swift 3.

However, this release is the first after a large effort to port it to Swift 3. This should make it easier to update dependencies regularly, and should lower the barrier to entry to contributors since they won’t need to install older Xcode versions.

Swift Package Manager & Linux Support

Because the latest releases of the Swift Package Manager and the Swift Linux Distributions require Swift 3, we couldn’t reasonably offer support for these installation methods until we ported all of SwiftLint and its dependencies to Swift 3. This took a few months, but now the wait is over!

You can now use SwiftLintFramework in your SwiftPM packages.

Moving to Swift 3 also meant that we could pull in the SourceKitten work that happened from June to October to support Linux.

So SwiftLint now builds and passes most tests on Linux using the Swift Package Manager with Swift 3!

We hope this will make it easier to include SwiftLint in your infrastructure without having to maintain Mac servers.

This requires libsourcekitdInProc.so to be built and located in /usr/lib, or in another location specified by the LINUX_SOURCEKIT_LIB_PATH environment variable.

Unfortunately, SourceKit isn’t included in Swift Linux distributions by default (yet) but a preconfigured Docker image is available on Docker Hub by the ID of norionomura/sourcekit:302.

CocoaPods Support

We’ve added support for CocoaPods for both SwiftLintFramework and the swiftlint CLI executable.

Add pod 'SwiftLintFramework' to your Podfile if you’re building a macOS app and wish to incorporate SwiftLint’s functionality via the framework APIs. Or spec.dependency 'SwiftLintFramework to your podspec if you’re building your own pod.

Add pod 'SwiftLint', '= 0.14' to install a specific version of the SwiftLint CLI executable in your Pods/ directory. This follows the pattern introduced by SwiftGen to install specific and pinned versions of SwiftLint, so your CI builds will never break again because SwiftLint released a new version with new rules!

Other Enhancements

Denis Lebedev added FunctionParameterCountRule now ignores initializers (in #544), Michał Kałużny added EmojiReporter, and Marcelo Fabri made sure thattype_name rule now validates both typealias and associatedtype (in #49 and #956).

Bug Fixes

Finally, there are quite a few bug fixes in this release, including some that deal with multi-byte characters in strings. Marcelo Fabri improved our unit tests to prevent bugs like these from creeping back in. To see the full list of the fixes our contributors worked on, check out our changelog. Our thanks go out to them!


This release was made possible by a large number of community contributions. A big thanks to Aaron McTavish, Daniel Duan, Denis Lebedev, Fabian Ehrentraud, JP Simard, Marcelo Fabri, Michał Kałużny, Norio Nomura and Phil Webster! 👏

We hope you’ll contribute next!

Read more

A Reactive Platform for Mobile Apps: Introducing Realm Professional Edition

by /

At Realm, our mission is to help developers build better apps faster. We have a vision for a much more powerful data layer for the mobile internet, where developers can easily build native mobile apps using live objects, where realtime data synchronization just happens automatically, and where server-side logic can be easily triggered by data changes, enabling more sophisticated features and integrations. And we’ve been hard at work making that a reality by constantly improving the Realm Mobile Database as well as the components and features of the Realm Mobile Platform.

We’re excited to share two pieces of news that advance that vision, and that make it easier for more developers to use Realm in their apps. Today we are announcing the new Realm Mobile Platform Professional Edition, a product that brings the power of the Realm platform to a larger group of developers at a new price point. We are also very happy to unveil the results of an ongoing collaboration with IBM: Scanner, a cool new Professional Edition demo app that integrates Realm with IBM Watson and provides developers with an open-source template for building reactive mobile apps that incorporate sophisticated image processing.

Realm Mobile Platform Professional Edition

Professional Edition is a new version of the Realm Mobile Platform designed for mid-sized apps and businesses. It shares all the capabilities of the free Developer Edition—notably, automatic realtime data sync—but adds key features like event handling and server-side data access (more on these below). We’ve built Professional Edition in response to customers who need these features to build the realtime apps they want to build, but who don’t yet need the high scale and other large company features of the Enterprise Edition. You can download a copy of PE today for a free 60-day trial.

The launch of Professional Edition establishes a pattern we plan to follow for a very long time: continually moving Realm features and functionality downstream, and inventing new pricing and packaging to allow greater access to more developers. We now have three versions of Realm Mobile Platform: Developer Edition, which is entirely free and available to anyone; Professional Edition, more robustly featured, aimed at medium-sized businesses, and capable of handling tens of thousands of concurrent users, starting at $1500 per month; and the fully-featured Enterprise Edition, which can handle even the largest number of users and most sophisticated apps, and is priced based on scale of use case. We work very closely with every potential customer to understand their needs, so if you have questions about editions and pricing, please reach out.

Introducing Scanner: Our Collaboration with IBM

When we shared an early version of Professional Edition with some of our contacts at IBM, they got very creative, very quickly. The result was Scanner, a Realm demo app that allows users to take a picture of anything and then quickly get a result based on sophisticated server-side image processing, including image classification, face detection, and text recognition. The app relies on the key features of Professional Edition–realtime data synchronization and event processing–as well as IBM’s Watson Visual Recognition API. You could imagine using code like this for building a variety of realtime features and apps: location identification for VR applications, face recognition for security, categorizing images for industrial and scientific applications, reading labels in retail applications, and more. But don’t spend too much time imagining, get to hacking: The demo application and all of its code (Swift and Android) are available as a free open-source project; you’ll also find there a tutorial that serves as an ideal starting point. The IBM team has also blogged about our collaboration, and has included a video of the app in action.

The pattern behind Scanner is a very common one amongst Realm Mobile Platform customers; we call it API mobilization (it’s sometimes also referred to as “API bridging”). It’s utilized when you need a highly responsive, great UX mobile app, but that app also needs to connect to and stay fully synchronized with existing APIs–could be a modern service like IBM Watson, but it’s also very often a legacy system inside the organization. The challenge for the mobile developer is that all network requests are asynchronous and can fail. We encounter many organizations with complex business logic requiring multiple API calls in a specific order. Managing the network requires additional code to orchestrate these requests, including handling the many failure scenarios. While this is a challenge on its own, each network request requires converting data into a new format, such as JSON. Again, further code is required to deal with the pain of serialization/deserialization, and bugs can easily appear when the data or schema changes.

The API mobilization pattern utilizes the Realm Platform as middleware. The mobile application uses the Realm Database (iOS or Android) on the client side, which is automatically kept in sync with the server-side data thanks to the Realm Platform. On the server side, the Realm Platform uses event handling to trigger the Watson API any time it notices a data change from the app (i.e., a new image is added); the returned data from Watson after processing the image is pushed back into Realm on the server side, and immediately synchronized with the client side. This removes the need for networking or data conversion code in your application. Mobile developers can work directly with objects in their native language relying on the Realm Database locally while the Realm Platform handles all the complexity of transferring the data.

Get Started Now

As always, we are most excited about what developers can build with Realm, so we invite you to download and install a free trial of Professional Edition, set it up with Scanner or your own app, and let us know what you think!

Read more

Realtime Collaboration, Now For Every Mobile Developer

by /

Since the release of the Realm Mobile Platform, we’ve seen how much more powerful an app becomes when developers add truly collaborative features, with multiple users sharing data with one another in realtime. The key element needed for this is client-defined access control, a feature that had been limited to paying Enterprise Edition customers. Today, we’re bringing access control to all editions of the Realm Mobile Platform, so that every mobile developer can bring realtime, multiuser collaboration to their apps – even with the free Developer Edition of the platform.

To make this all work, we’re launching new functionality in our Cocoa and Android client SDKs to expose the access control permissions on a Realm managed by Realm Object Server, so that your users can take control over who has access to their data.

It’s simple: a user who owns a Realm can share it with other users by creating a PermissionChange object, and specifying a user and a new permission for him (such as mayRead or mayWrite). After the PermissionChange object is created, that object gets saved to a special management Realm, and synced to your Realm Object Server, which will then apply the requested changes to the specified user.

Client Permission diagram

By using a dedicated synchronized Realm along with the PermissionChange object to communicate with Realm Object Server, this naturally takes advantage of Realm Mobile Platform’s objects-as-APIs approach. The PermissionChange will be synchronized automatically and Realm Object Server will react to the change and update the object with the status. Objects become the new API, making requests offline-first and immune to networking failures, and all you need to do is add some client-side code to make it work.

For more information regarding how to use the new APIs, check out the Realm Object Server documentation and relevant sections in the Cocoa and Android documentation. Then download the free Dev Edition, build something cool, and tell us about it – we’re eager to see what you can create.

Read more

Realm Objective-C & Swift 2.1 – Improved Collection Notifications!

by /

We’re releasing version 2.1 of Realm Objective‑C and Realm Swift today, which includes several improvements and bug fixes to fine-grained collection notifications. Read on to learn more!

Background (on) Notifications

The Realm SDKs have always provided a reactive, Objects-as-APIs approach to dealing with data, and with the recent launch of the Realm Mobile Platform, apps built on Realm can now easily respond to changes no matter where they originate from: other threads, processes and even other devices from half-way around the world 🌏!

With collection notifications, change information is continuously processed and delivered as operations representing what was inserted, deleted, moved and changed. Unlike ORMs that rely on diffing the old and new values, Realm’s change notifications leverage the semantic intent of operations to produce a much more accurate result, leading to nicer animations and easier code.

These notifications are always delivered asynchronously so your app never stutters or block the main UI thread. However, there are situations when changes need to be done synchronously, on the main thread, and reflected in the UI instantly. We refer to these transactions as interface-driven writes.

Interface-Driven Writes

We’ll refer to writes done on the UI thread and explicitly mirrored in the UI’s state as Interface-Driven Writes.

For example, say a user adds an item to a table view. The UI should ideally animate this operation and start this process as soon as the user initiates the action.

However, when the Realm change notification for this insertion is delivered a little later, it will indicate that an object was added to the collection backing the table view and we will once again attempt to insert a new row in the UI. This double insertion leads to inconsistent state between the UI and the backing data, which in turn will crash your app! 💥NSInternalInconsistencyException💥

Writes Skipping Notifications

To resolve this, we’re introducing a mechanism that allows for write transactions that are immediately reflected in the UI to avoid triggering a change notification for the changes that occurred during that transaction.

We’re calling these types of writes “writes skipping notifications”.

This feature especially useful when using fine-grained collection notifications with a synchronized Realm, because many of the workarounds to previously account for interface-driven writes rely on controlling the full state of when the app can perform changes. With synchronized Realms, changes are applied whenever they’re synchronized, which can happen at any point in the app’s lifetime.

Code Sample

// Add fine-grained notification block
self.notificationToken = self.collection.addNotificationBlock { [unowned self] changes in
  switch changes {
  case .Initial:
    self.tableView.reloadData()
  case .Update(_, let deletions, let insertions, let modifications):
    // Query results have changed, so apply them to the UITableView
    self.tableView.beginUpdates()
    self.tableView.insertRowsAtIndexPaths(insertions.map { NSIndexPath(forRow: $0, inSection: 0) }, withRowAnimation: .Automatic)
    self.tableView.deleteRowsAtIndexPaths(deletions.map { NSIndexPath(forRow: $0, inSection: 0) }, withRowAnimation: .Automatic)
    self.tableView.reloadRowsAtIndexPaths(modifications.map { NSIndexPath(forRow: $0, inSection: 0) }, withRowAnimation: .None)
    self.tableView.endUpdates()
  case .Error(let error):
    // handle error
  }
}

func insertItem() throws {
  // Perform an interface-driven write on the main thread:
  self.collection.realm!.beginWrite()
  self.collection.insert(Item(), atIndex: 0)
  // And mirror it instantly in the UI
  self.tableView.insertRowsAtIndexPaths([NSIndexPath(forRow: 0, inSection: 0)], withRowAnimation: .Automatic)
  // Making sure the change notification doesn't apply the change a second time
  try self.collection.realm!.commitWrite(withoutNotifying: [self.notificationToken])
}

Demo

RealmTasks Fine-Grained Notifications

Thanks to RealmTasks being open-source, the pull requests adding fine-grained notifications and writes skipping notifications can be viewed on GitHub: iOS PR #352 and macOS PR #355.

Bug Fixes

As we were building this functionality, we took the opportunity to fix a number of issues with collection notifications, so this release should make them more reliable than ever before!

  • Deliver collection notifications when beginning a write transaction which advances the read version of a Realm (previously only Realm-level notifications were sent).
  • Fix some scenarios which would lead to inconsistent states when using collection notifications.
  • Fix several race conditions in the notification functionality.
  • Don’t send Realm change notifications when canceling a write transaction.

Legacy Swift Version Support

We’d like to remind you that we will continue to support Xcode 7.3.1 and Swift 2.2 as long as we can, but encourage all our users to migrate to Xcode 8 as soon as possible.


Thanks for reading. Now go forth and build amazing apps with Realm! As always, we’re around on Stack Overflow, GitHub, or Twitter.

Read more

The First Object Database for Node: Introducing Realm Node.js

by /

At Realm, we’re very focused on mobile developers, and have built and open-sourced Realm Mobile Database versions for Swift, Objective-C, Java, Xamarin, and React Native. But today, we’re announcing something completely different: Realm Node.js. We’re pretty sure this is the first real object database for Node, and is available today on NPM as a free and fully open source repo – just run npm install --save realm.

Over the years, we’ve had tons of requests from mobile developers for a version of Realm that worked on the server, so it’s long been on our backlog, though not highly prioritized. But our priorities changed when we released the new Realm Mobile Platform in September, and potential customers started asking us for a Node interface for the platform. We pretty quickly made Node part of the platform product strategy (more on that in a few days), and that led to the creation of Realm Node.js.

What is it, and what is it for?

Realm is an object database. It means that you simply work with objects, as you’re used to, except that these objects persist to disk easily and performantly in a Realm. You don’t have to serialize them into JSON, and you don’t have to send them through an ORM to store them in tables. You stick with the principles that object-oriented development encourages, and you make the thing you set out to make. It works on all the mobile client side platforms – and now Node.js on the server side as well.

For mobile developers, Realm Node.js means you can create and send pre-populated Realms to clients, which is very handy for a number of reasons, including getting around pesky app size limits. That’s the primary reason mobile dev have asked us for something like this over the years.

But things got even more interesting when we showed some early versions to backend developers and learned that it might be a lot more broadly useful to the Node community than we originally imagined. For example, we realized that it is useful for sharing live data between multiple Node instances. To illustrate this, we threw together a quick demo to show you how you can use Realm to get two Node processes on the same machine to work together, without the burden of serializing and deserializing the objects with which both need to work.

One of the unique features of Realm is its reactive architecture. Underneath the hood, Realm uses Multiversion Concurrency Control to provide concurrent access to the database across threads and processes. This means that readers are never blocked against a writer and each has a consistent view of the database–ensuring Realm’s ACID compliance. To coordinate this, Realm uses a notification system to update accessors when a write transaction completes. Developers can take advantage of this through Realm’s notification APIs to react to changes. This example demonstrates how a Node.js developer can use this functionality to coordinate two independent services by using Realm for interprocess communication.

In the example, we use two popular Node.js libraries: Express, a web framework, and Winston, a logging library. We will use Express to create HTTP endpoints and Winston to log information about the requests:

var express = require('express'),
    util = require('util'),
    winston = require('winston');
    RealmWinston = require('./winston-realm').Realm;

var app = express();

// Use custom Winston transport: RealmWinston
// Writes log data to winston.realm
winston.add(RealmWinston, {});

app.get('/', function (req, res) {
  res.send('Hello World!');
  winston.info('Handled Hello World');
});

app.use(function (req, res, next) {
  res.status(404).send('Sorry can not find that!');
  winston.error('404 Error at: ' + req.url);
})

app.listen(3000, function () {
  console.log('Example app listening on port 3000!');
});

To take advantage of Realm, we created a custom transport for Winston that saves the log data to a Realm file. At the base path, we log an info message confirming the “Hello World” response. For all other paths, we respond with a 404 error and log the corresponding path in an error message. By saving the log data in Realm, we can then start another Node process and register a Realm listener to react to changes:

'use strict';

var Realm = require('realm');

let winstonRealm = new Realm({
  path: 'winston.realm'
});

// Register listener to print out log messages at error level
winstonRealm.objects('Log').filtered('level = "error"').addListener((logs, changes) => {
  changes.insertions.forEach((index) => {
    let log = logs[index];
    console.log(log.message);
  })
});

The listener uses Realm’s support for collection notifications, which passes the specific changes as indexes corresponding to the inserted, deleted, or modified objects. The example adds a listener to a query of all logs at the error level and then prints the log message to the console.

While this example is simple, this setup could be used in more advanced scenarios to coordinate data across microservice architectures. Check out the complete demo here. When you’re ready to use Realm in your own Node.js projects, you can find the answers you need in our React Native documentation and our API docs (since the codebase is shared between all our JavaScript platforms).

What else can you use Realm Node.js for? Work with datasets bigger than local memory. Use as queryable and observable data structures. Build reactive architectures across multiple Node instances. Or perhaps for communication between Docker images? We’re eager to hear what you do with it!

Read more

Introducing the Realm Mobile Platform: Realtime Sync Plus Fully Open Source Database

by /

This post announced the launch of Realm Mobile Platform in Sept. 2016. Please keep in mind that some pieces may be out of date, so check out the latest here.

When we launched Realm in 2014, our goal was to help mobile developers build better apps faster by giving them a powerful alternative to SQLite and Core Data. Two and a half years later, we’ve reached more than a hundred thousand active developers and more than a billion worldwide installs of apps that use Realm. We’re extremely grateful to the developer community for the adoption, support, and feedback that has gotten us to where we are today — but we have much more to do.

Today, we’re launching the Realm Mobile Platform, a new offering that integrates our fully open-source (see below!) client-side database for iOS and Android with new server-side technology providing realtime synchronization, conflict resolution, and reactive event handling. The new platform makes it easy for mobile developers to create apps with difficult-to-build features like realtime collaboration, messaging, offline-first experiences, and more.

 

The new platform is available as a beta today, in two versions (read about our third, Pro Edition, launched in December 2016). The Developer Edition is free forever for all kinds of use cases, from hobbyists to small commercial applications, and the Enterprise Edition is available as a time-limited trial and fully supported product designed for the largest production use cases. The full details are covered below. Today the platform supports Java, Objective-C, and Swift, but React Native and Xamarin support are not far behind.

We’re also making another big change: We’re proud to be open-sourcing Realm Core, making the Realm Mobile Database fully open-source as of today — you can find the repos here. The Mobile Database is critically important to us, both as a component to the new platform and as a standalone database option for mobile developers, and we’ll continue to have engineering teams dedicated to maintaining and improving Realm Java, Realm Objective-C, Realm React Native, Realm Swift, and Realm Xamarin.

The Realm Mobile Platform

Objects All the Way Down

Even long-time Realm users may not realize that we’re an embedded object database at the core level, but it is one of our key differentiators. The ease and speed of use that developers love about Realm is a direct result of this. There is no complicated object relational mapping to wrestle with, there are only objects — so your database is your data model.

We have extended that aspect of our design philosophy to the new Object Server, and an important part of the magic behind our platform is something we call “live objects.” Live objects mean that the data objects on the device are always kept fully synchronized with the data objects on the server. Synchronization happens automatically and seamlessly — there’s no networking code to write — and is very efficient because only the changes are transmitted, not entire objects. The Realm platform also handles conflict resolution, so competing changes to shared live objects are resolved deterministically. It does this automatically by default with preset rules, and this mode will handle most use cases, but you can also easily create custom conflict resolution rules.

With Realm, you never need to think about networking. You can simply build your app with the confidence that it is working with live objects. That means all kinds of otherwise difficult-to-build features, like realtime collaboration or messaging, are a lot easier. Think, for example, about allowing users on different devices to collaborate on a shared whiteboard. The hardest parts of making that work are ensuring that changes are reflected in each user’s UI immediately, and that any conflicts — users drawing or erasing similar parts of the drawing simultaneously — are handled elegantly. The Realm platform gives you both out of the box.

Building Offline-first Experiences

The Realm platform is also a great solution for offline-first apps, where delivering a seamless user experience despite lost or intermittent connectivity is a priority. Using the Realm Mobile Database by itself gives you a very efficient, embedded database on the device, so even if the connection is dropped, your app will maintain a good user experience. It’s always been an important benefit of using the Realm Mobile Database.

But when you use the whole Realm platform — Realm Database plus the Object Server — you get much more. Automatic two-way sync just happens in the background, you can be certain that the data objects on the device are fully up-to-date as of the moment before the connection was lost. When the connection is restored, sync happens automatically again, and any conflicts that may have occurred are automatically resolved — the live objects are again fully up to date. Your users can just keep on working as if nothing happened.

Event Handling

The professional and enterprise versions of the Realm platform include another key feature: a server-side event handling framework that provides the plumbing required to easily trigger server-side logic whenever an object changes. You can configure this framework to listen for object changes from the device side and to then instantly run server code in response. Perhaps a user enters a coupon code while filling out an order form. This is captured locally as a change to the device-side coupon object, which syncs to the server instantly. That specific change causes the event handling framework to trigger a check of the coupon code against valid codes, updating the server-side coupon object with a response about the validity. This response is then synced back down to the device-side coupon object triggering a notification that updates the UI.

Event handling

You can use event handling to build integrations to existing APIs and infrastructure, allowing the Realm Object Server to serve as an API bridge or mobilization middleware. It also enables you to design your app around “serverless” computing principles — you can write discrete functions for each specific trigger.

Get Started Today

The Realm Mobile Platform is available in three versions (as of December 2016). You can download the free Developer Edition, start a free trial of Professional Edition, which includes event handling, or request a demo of Enterprise Edition. Compare features and editions here.

Read more

Realm Objective-C & Swift 2.0 – Mobile Platform Support

by /

We’re releasing version 2.0 of Realm Objective‑C and Realm Swift today, adding support for the Realm Mobile Platform. Also included in this release is one source breaking change for Swift 3 and bug fixes. Read on to learn more!

Realm Mobile Platform Extensions

If you haven’t read the Realm Mobile Platform launch post yet, please do that now so we can get back to the technical cocoa bits! (err, nibs? 😄)

Let’s start with a quick look at what it takes to configure a synchronized Realm:

// Authenticating the User
User.authenticate(with: .google("google token"),
                  server: URL(string: "https://realm.example.com:9443")) { user, error in
  if let user = user {
    // Opening a remote Realm
    let realmURL = URL(string: "realm://realm.example.com:9080/~/userRealm")!
    let config = Realm.Configuration(syncConfiguration: (user, realmURL))
    let realm = try! Realm(configuration: config)
    // Any changes made to this Realm will be synced across all devices!
  } else if let error = error {
    // handle error
  }
}

That’s it! Once a Realm is opened like this, any changes made to it will be automatically synchronized across all other devices for that user.

Back to technical bits. To support the Mobile Platform, we’ve added some new classes and APIs:

Sync User & Credential

Sync Users are a central concept when dealing with synchronized Realms. Every Realm is associated with a user. The user can be authenticated to a Realm via a username/password credential or through a number of additional authentication methods (Google, Facebook, iCloud, etc.). See docs on users for details.

Sync Configuration

To configure a synchronized Realm, you must set the syncConfiguration property of RLMRealmConfiguration/Realm.Configuration with the user and the remote realm URL (e.g. realm://realm.example.com:9080/~/userRealm). See our docs for more details.

Sync Manager & Session

The sync manager (RLMSyncManager/SyncManager) is a singleton (for the time being) that manages all synchronized Realms. You’ll mostly interact with it to set the sync engine’s log level, or to specify a global error handler.

Each synchronized Realm has a corresponding session (RMLSyncSession/SyncSession) which can provide further information about its state. We intend to attach more information to this in upcoming releases.

Platform Extensions are in Beta

Although the underlying technology for Realm synchronization has been built, refined and stabilized over the last 5 years, we want to make sure we get the APIs right before committing to them. So we’re releasing the platform extensions as a beta. These APIs will change in future releases. We’re doing this to make sure we end up with the best possible product.

If your app doesn’t use the platform extensions, you can continue to count on the reliability of Realm just like you always have. The platform extensions have been designed in a way that clearly differentiates from using Realm entirely locally or using a synchronized Realm, so this should be easy to identify if your app relies on any of the beta components.

A (mostly) non-breaking 2.0 release

This is a major version bump for our frameworks to reflect the magnitude of the new features and capabilities included in this release. However, we’ve chosen to not introduce API breaking changes, because we realize this release comes just a few months after our 1.0 release and we value the time it takes for you, our users, to update your apps for API changes. With one exception… In Realm 1.1.0 we introduced official Swift 3 APIs and the most common feedback we got there was how it wasn’t ideal that RealmSwift.Error conflicted with the standard library Swift.Error. So by popular demand, we’ve nested our Error into a Realm extension, and this is the only source-breaking change in this release.

Note that Realm Objective-C and Swift 2.0 introduces a new file format that allows us to gain performance improvements and fix some bugs. Files written by Realm 2.0 cannot be read by 1.x or earlier versions. Old files can still be opened.

Removing some extra files

The .log, .log_a and .log_b files no longer exist and the state tracked in them has been moved to the main Realm file. This reduces the number of open files needed by Realm, improves performance of both opening and writing to Realms, and eliminates a small window where committing write transactions would prevent other processes from opening the file.

Bug Fixes

  • Fix an assertion failure when sorting by zero properties.
  • Fix a mid-commit crash in one process also crashing all other processes with the same Realm open.
  • Properly initialize new nullable float and double properties added to existing objects to null rather than 0.
  • Fix a stack overflow when objects with indexed string properties had very long common prefixes.
  • Fix a race condition which could lead to crashes when using async queries or collection notifications.
  • Fix a bug which could lead to incorrect state when an object which links to itself is deleted from the Realm.

Q: Can I use Realm with its newly open sourced core storage engine?

Yes! You can build and use Realm as an embedded database built 100% from open source components. Although the installation steps are a bit more involved than pod install, but you can follow the instructions listed in the READMEs for both realm-core and realm-cocoa.

We’re working on adding a way to install entirely from source using CocoaPods in the next few weeks. Stay tuned!

We’d like to remind you that we will continue to support Xcode 7.3.1 and Swift 2.2 as long as we can, but encourage all our users to migrate to Xcode 8 as soon as possible.


Thanks for reading. Now go forth and build amazing apps with Realm! As always, we’re around on Stack Overflow, GitHub, or Twitter.

Read more

Realm Java 2.0 - Mobile Platform Support

by /

We’re releasing version 2.0 of Realm Java today, adding support for the Realm Mobile Platform as well as some other minor changes and bug fixes.

Realm Mobile Platform Extensions

If you didn’t read the Realm Mobile Platform launch post yet, we suggest you do that now in order to make more sense of the technical details that will soon follow.

First you need to enable the Mobile Platform Extensions by adding the following to your app’s build.gradle file:

realm {
  syncEnabled = true
}  

Then, let’s take a quick look at what it takes to configure a synchronized Realm:

// Login a user
Credentials credentials = Credentials.usernamePassword(username, password, true);
User.loginAsync(credentials, "https://realm.example.com:9443", new User.Callback() {
  @Override
  public void onSuccess(User user) {
    // Opening a synchronized Realm
    // All changes to this Realm will be synchronized with all devices
    SyncConfiguration config = new SyncConfiguration.Builder(user, "realm://realm.example.com/~/userRealm").build();
    Realm realm = Realm.getInstance(config);
  }
  
  @Override
  public void onError(ObjectServerError error) {
    // Handle error
  }
});

That’s it! Once a Realm is opened like this, any changes made to it will be automatically synchronized across all other devices for that user.

Back to the technical bits. To support the Realm Mobile Platform, we’ve added some new classes and APIs:

User & Credential

Users are a central concept when dealing with synchronized Realms. Every Realm is associated with a user. The user can be authenticated to a Realm via a username/password credential or through a number of additional authentication methods (Google, Facebook, iCloud, etc.). See docs on users for details.

Sync Configuration

To configure a synchronized Realm, you must create a SyncConfiguration tied to the user and the remote Realm URL (e.g., realm://realm.example.com/~/userRealm). See our docs for more details.

Sync Manager & Session

The SyncManager is a singleton that manages all synchronized Realms. Currently, you will most likely interact with it to specify a global error handler or how users are stored.

Each synchronized Realm has a corresponding Session which can provide additional information about its state. We intend to attach more information to the session in upcoming releases.

Platform Extensions are in Beta

Although the underlying technology for Realm synchronization has been built, refined and stabilized over the last 5 years, we want to make sure we get the APIs right before committing to them. So we’re releasing the platform extensions as a beta. These APIs will change in future releases. We’re doing this to make sure we end up with the best possible product. Feedback on the APIs is appreciated.

If your app doesn’t use the platform extensions, you can continue to count on the reliability of Realm just like you always have. The platform extensions have been designed in a way that clearly differentiates from using Realm entirely locally or using a synchronized Realm. This should make easy to identify if your app relies on any of the beta components.

All APIs that are in beta are marked with a @Beta annotation and tag in respectively code and Javadoc.

Other 2.0 changes

This is a major version bump for our frameworks to reflect the magnitude of the new features and capabilities included in this release. At the same time we have streamlined a few things as well as made a few adjustments to bring the Realm Java more in line with Realm Swift and Realm Objective-C.

Note that Realm Java introduces a new file format that allows us to gain performance improvements and fixes some bugs. Files written by Realm 2.0 cannot be read by 1.x or earlier versions. Old files can still be opened and they will be migrated to the new format automatically. NOTE: If you are shipping prebuild Realm files with your app, you have to upgrade those to the new version. You can use Realm Browser to upgrade to the latest file format.

Global initializer

The Context reference was sneaking into more and more of our APIs, causing unnecessary friction. In the end we took a step back and introduced a global initializer function

Realm.init(context);

This also made it possible to remove the context from other parts of the APIs which means that:

  • Creating a RealmConfiguration.Builder() no longer requires a context.
  • RealmConfiguration.Builder.assetFile() no longer requires a context.
  • Realm.getDefaultInstance() now works without setting a default first.
  • More failsafe loading of native code.

A natural place for calling this method is in an Application subclass:

public class MyApplication extends Application {
    @Override
    public void onCreate() {
        super.onCreate();
        Realm.init(this);
        // Other initializers
    }
}

Default Values

With Realm Java 2.0 we support default values when creating objects. This implies that it is possible to define default values for fields and in the default constructor.

public class Person extends RealmObject {
  private String name = "John";
  private int age;
  
  public Person() {
    this.age = 42;
  }
}

Person p = realm.createObject(Person.class);
p.getName(); // John
p.getAge(); // 42

Default values will also be applied when creating objects from JSON data, but not when creating objects using the dynamic API.

RealmLog

A new class has been added to the public API called RealmLog. It makes it possible to customize how much is logged from Realm and how. All events from the Realm Mobile Platform are also routed through this class.

RealmLog.clear(); // Remove all loggers
RealmLog.add(new AndroidLogger(Log.DEBUG)); // Add custom logger

Objects with Primary Keys

In order to support merging objects from other devices, two new restrictions are imposed on objects with primary keys:

1) A primary key must be provided when the object is created. 2) Once a primary key is set it cannot be changed.

First restriction should only be visible if you use realm.createObject(Class) to create objects. In that case it is required to set the value of the primary key by using realm.createObject(Class, Object).

Using realm.copyToRealm() and realm.createObjectFromJson() will continue to work as before.

If you, for some reason, need to change the primary key, it is necessary to create a new object and copy all fields over. copyFromRealm()/copyToRealm() can be used to make that process easier.

Bug fixes and other changes

For a full list of bug fixes and other changes, see the Changelog.


Thanks for reading. Now go forth and build amazing apps with Realm! As always, we’re around on Stack Overflow, GitHub, or Twitter.

Read more

Realm Objective-C & Swift 1.1.0 – Official Swift 3 APIs & Xcode 8 Support

by /

We will continue to support Xcode 7.3.1 and Swift 2.2 as long as we can, but encourage all our users to migrate to Xcode 8 as soon as possible.

We’re releasing version 1.1.0 of Realm Objective‑C and Realm Swift today, adding support for Xcode 8, Swift 2.3, Swift 3.0, with frameworks linked against iOS 10, macOS 10.12, tvOS 10 and watchOS 3. Also included in this release are some minor non-breaking API improvements, performance improvements and bug fixes. Read on to learn more!

Swift 3 API Refinements

Those of you keeping up with changes to Swift already know how big a jump Swift took from versions 2.x to 3.0!

We’ve audited Realm Swift’s public API in preparation for this and made some adjustments to reflect changes to Swift’s syntax and API design guidelines. We strived to strike a balance between conforming to the API guidelines without drastically changing ours APIs at their call site (i.e., in your code), so changes to your code should be minimal…

Here are some examples of how Realm Swift looks in Swift 2.2 and Swift 3.0:

// Swift 2.2/2.3
let config = Realm.Configuration(
  schemaVersion: 1,
  migrationBlock: { migration, oldSchemaVersion in
    migration.renamePropertyForClass(Person.className(), oldName: "yearsSinceBirth", newName: "age")
    migration.enumerate(Person.className()) { oldObject, newObject in
      // Migrate Person
    }
  }
)

let realm = try! Realm(configuration: config)
let sortedDogs = realm.objects(Dog.self).filter("color = 'tan'").sorted("name")
let jim = Person()
try! realm.write {
  jim.dogs.appendContentsOf(sortedDogs)
}

// Swift 3.0
let config = Realm.Configuration(
  schemaVersion: 1,
  migrationBlock: { migration, oldSchemaVersion in
    migration.renameProperty(onType: Person.className(), from: "yearsSinceBirth", to: "age")
    migration.enumerateObjects(ofType: Person.className()) { oldObject, newObject in
      // Migrate Person
    }
  }
)

let realm = try! Realm(configuration: config)
let sortedDogs = realm.objects(Dog.self).filter("color = 'tan'").sorted(byProperty: "name")
let jim = Person()
try! realm.write {
  jim.dogs.append(objectsIn: sortedDogs)
}

For simplicity, our documentation throughout the site has been updated to use Swift 3 exclusively. That includes code samples, long form docs and API docs.

API breaking changes

  • Deprecate migrateRealm: in favor of a new performMigrationForConfiguration:error: method that follows Cocoa’s NSError conventions.
  • Fix an issue where RLMResults used id instead of its generic type as the return type of subscript.

Enhancements

  • Improve error message when using NSNumber incorrectly in Swift models.
  • Further reduce the download size of the prebuilt static libraries.
  • Improve sort performance, especially on non-nullable columns.
  • Allow partial initialization of an object by initWithValue:, deferring required property checks until the object is added to a Realm.

Bug Fixes

  • Fix incorrect truncation of the constant value for queries of the form column < value for float and double columns.
  • Fix a crash when an aggregate is accessed as an Int8, Int16, Int32, or Int64.
  • Fix a race condition that could lead to a crash if an RLMArray or List was deallocated on a different thread than it was created on.
  • Fix a crash when the last reference to an observed object is released from within the observation.
  • Fix a crash when initWithValue: is used to create a nested object for a class with an uninitialized schema.
  • Enforce uniqueness for RealmOptional primary keys when using the value setter.

Thanks for reading. Now go forth and build amazing apps with Realm! As always, we’re around on Stack Overflow, GitHub, or Twitter.

Read more

Realm Java 1.1.0 — New and faster Insert API!

by /

We just released a new version of Realm Java to this website and to Bintray. It contains a new insert API as well as a number of bug fixes.

Insert API

Realm’s storage engine has always been faster than SQLite when it comes to inserting data, but due to a number of design decisions in the Realm Java API, that advantage was mostly lost. This meant that Realm was generally as fast as SQLite for a small number of items (<1000), but started getting slower the more items you added in a batch.

In v1.1.0 we came up with some optimizations and now offer 4 new methods:

  • void Realm.insert(RealmModel obj)
  • void Realm.insert(Collection collection)
  • void Realm.insertOrUpdate(RealmModel obj)
  • void Realm.insertOrUpdate(Collection collection)
Realm realm = Realm.getDefaultInstance();
realm.beginTransaction();
realm.insertOrUpdate(restApi.getPersons());
realm.commitTransaction();

These methods differ mostly from the standard Realm.copyToRealm() in the sense that they do not return any objects. This has allowed us to reduce memory allocations to almost zero as well as remove many checks that were pure overhead.

Doing this, we went from being ~40% slower than an optimized SQLite implementation to being ~70% faster for 100K objects.

Realm Batch Insert Benchmark

We have released our benchmarks repo so you can verify these numbers yourself. Find it here. The numbers above were produced on a Nexus 6P using stock Android 6.0.1.

Note that due to Realm’s auto-update features it is possible to query for data before they are even saved, and you will get notified when the data is available. This can be a huge benefit for those of you that want to separate displaying data from saving it:

final PersonApi api = new PersonApi();
Realm realm = Realm.getDefaultInstance();
RealmResults<Person> persons = realm.where(Person.class).findAllAsync();
person.addChangeListner(new RealmChangeListener() {
    @Override
    public void onChange(RealmResults<Person> persons) {
      if (!persons.isEmpty()) {
	    // Callback when data is available
      }
  }
});

realm.executeTransactionAsync(new Realm.Transaction() {
    @Override
    public void execute(Realm realm) {
      realm.insertOrUpdate(api.getPersons());
    }
});

Bug fixes

  • Combining async transactions and async queries on the UI thread could result in the transaction onSuccess callback being invoked before the data is available on the UI thread.

  • The RealmOptionalAPITransformer introduced in 1.0.1 does not work with DexGuard and has been disabled for now.

See the full changelog for all the details.


Thanks for reading. Now go forth and build amazing apps with Realm! As always, we’re around on Stack Overflow, GitHub, and Twitter.

Read more