Realm Blog

React Native Radio Ep. 24: Realm React Native

by Tim Anglade /

On the latest episode of Devchat.tv’s React Native Radio, hosts Peter Piekarczyk and Lee Johnson sat down with Scott Kyle and Tim Anglade to talk about the recently released Realm React Native. They discuss everything from the design paradigms behind the database to the company’s business model, and get in-depth about encryption, object types, async, and more. Have you been wondering why Realm decided to directly support React Native rather than asking devs to bridge to iOS or Android? Tim and Scott reveal the thinking and product design going on behind the scenes, as well as what goes on in our underlying storage engine to make Realm so fast and easy to use. Be sure to check out the episode on Devchat.tv and subscribe on iTunes.


Play the episode below


Play on Devchat.tv

Introduction

Peter Piekarczyk: Hello and welcome to episode 24 of React Native Radio. I’m your host, Peter Piekarczyk, and we also have Lee Johnson, Scott Kyle, and Tim Anglade.

So, Scott and Tim, would you guys like to start by introducing yourselves and letting us know what you do?

Scott Kyle: I’m Scott, and I have been working on the Realm React Native binding for maybe the past 6 months or so. Before that, I co-founded a startup and we did software for WiFi routers, so a whole different thing. I was with Apple prior to that, so I’ve been bouncing around, doing lots of different stuff and enjoying every minute of it.

Tim Anglade: I’m Tim, I’m overseeing product at Realm and I’ve been with them for about 2 1/2 years, through some dark ages when we were called something else, doing something else as a startup prior to being called Realm, launching for Objective-C, Swift, Java, Android, and most recently React Native. Before that, I did a bunch of different things around NoSQL distributive systems. I came from that background and started on web originally. I’m coming back to JavaScript after a long hiatus. So that’s my back story.

Lee Johnson: All right so you piqued my interest. I didn’t realize you guys were around that long, with the dark ages comment. I’d love to hear the story of how that all transpired.

Tim Anglade: We used to be called TightDB, which was a bad name on a number of levels. We were doing something very similar. The technology hasn’t changed all too much but we were doing more of an embedded database, like a pound for pound type of replacement for SQLite. As I said, the technology hasn’t changed all that much but we’re much more focused on disk space and raw speed and a bunch of advanced features, something you couldn’t quite do with SQLite if you were into embedded systems and their devices or low level programming.

The company was really founded by a bunch of C++ dudes and was like a developers’ database type of ideal to it. We decided around early 2014 to focus more specifically on mobile, which was the founders’ first dream anyway. They both came out of Nokia and they both had worked for a long time on old mobile and new mobile. They started building a database for mobile way back when they founded the company in the late 2000s, early 2010s. A lot of the investors told them that nobody needed a mobile database so they had gone into different direction.

We took it back to the start, really focus it on what mobile was today, in the iPhone and Android world, and really focused on the Realm you see today, which is this really easy to use mobile database that has a first party integration into your language of choice, so Objective-C, Swift, Java, and specifically now JavaScript and React Native. That is the detour that we took towards building what it is today and how you always end up doing what you thought of in the first place anyway.

Realm’s Features & Use Cases

Lee Johnson: So it’d be nice if you could give us an overview of Realm’s features and use cases.

What’s cool about it is React Native is new and Objective-C and Xcode projects obviously have Core Data and Android has their database. So a database for React Native is a unique thing. It’s specific to that environment and we don’t have a lot of options for that now. You guys are pretty new to the scene for an actual database in React Native, and that makes it easier to use than writing bridges to Core Data or something crazy like that.

Tim Anglade: Realm is the umbrella term – it’s the name of the company, it’s also the name of the product that we have for each platform. In its raw form Realm is this easy to use, really fast replacement for your data model layer, including persistence, relationships, querying, and everywhere you would use a model in your app, you use a Realm object.

The basic principle of it is to build a modern data model if we were trying to rebuild SQLite today. It’s easy to use, which is what we think most developers need at this point – they really need something that doesn’t take a lot of time. When you’re developing your application and trying to maintain it and find a bug, we’re trying to do something really, really fast because you don’t want to be slowed down by an ORM or database queries and having to optimize that.

We try to maintain a really advanced feature set, like encryptions built in, graph queries built in, easy migrations, and a nice reactive notification system, and we try to do so in the open so we develop everything we can on GitHub. We support developers directly on GitHub, Stack Overflow, Twitter, or by email. We’ve tried to build a large, trusted community around it, and we are now running on half a billion devices around the world, which is pretty nice. We’re used in some really large apps on the Objective-C, Swift, Java side. We just launched JavaScript but we’ve seen some good adoption. On the older products, we’re already used by Google, Amazon, Starbucks, eBay, Budweiser, BBC, McDonald’s, Wal Mart, a bunch of global companies, which you can find on our website. That’s Realm in general.

Realm for React Native is our reinvention of the product specifically for the platform. We don’t really do bindings in the strict sense of “Let’s just take this thing we built and expose the API directly into whatever language we have to deal with.” We really try to start every new product thinking “What is that product trying to do? What is the platform trying to do? What is the framework behind it trying to do? What are its design principles and how can we fit in there in the best possible way?” We spent an enormous amount of time launching every product specifically for that community. You should look at our Swift product, it’s drastically different from our Android product in terms of the API it exposes, the query language it adds, a lot of things like that. If you look at our Swift products versus our Objective-C products, they’re also very different even though they both target the iOS platform.

Our React Native and JavaScript product is no different. We’re trying to build something that is very first party, that feels native to JavaScript, that feels like you’re interacting with a standard component of the React Native ecosystem. The jury’s still out as to whether or not we succeeded in that, so we’re always looking for feedback on everything we’re doing wrong and making improvements. It’s the application of our basic design principles around data models, around concurrency, around consistency and having different types of data you’re offloading on your app, but also applying all these things to what’s idiomatic to JavaScript and React Native. We came out with Realm for React Native very recently, and actually launched at React Conf this year. We’re lucky to have some users like TaskRabbit already using us in production for two months actually, since before we even launched publicly, and adoption’s been pretty good so far. We’re still in the early days, but we’re in the hundreds of users already after a few weeks, and we’re growing about fifteen to twenty percent every week, so we’re seeing nice exponential growth and nice pick-up in the React Native community. I’m pretty excited about that.

Peter Piekarczyk: Do you have a particular client or customer or app type that you think would most benefit from Realm? Or is it anything?

Tim Anglade: We really tried to design this for any app and for if you need a data model that would basically feel like you should be able to use Realm. Specifically on a relatively new ecosystem like the React Native ecosystem that doesn’t have a very strong integration with SQLite, like Android does, or doesn’t have a very strong framework like Core Data on iOS, we feel like there are very few reasons why you shouldn’t consider Realm for React Native. I say this because there’s a lot of trust and safety built into using the first party option in every ecosystem, so definitely there are a lot of reasons why if you’re on iOS you might still want to stick with Core Data or if you’re doing Android that you may want to use the built in SQLite functions. For React Native, there’s not the same nice, clean, first party persistence solution that has the semantics that we do and has the performance that we do. Unless you’re really happy with key-value storage, we think that you should probably give Realm a look. And again, if you don’t think we stay true to that promise, then we really would like your feedback to see how we can do that.

One of the specific use cases that tend to trigger people using us are if you need to use data offline. If you need to do complex querying, complex relationship between your objects, that’s another reason to think about it. If you need a lot of first party support from a company and not just community answers or open source projects, that’s another. There’s a bunch of different reasons to consider Realm but in terms of what apps, we really tried to invent something that should be usable by any app, whether you need offline or not, whether you need complex queries or not, no matter what business or application you’re building.

The Business Model

Peter Piekarczyk: If you don’t mind me asking, how are you guys going to make money? Because Realm is free, which is amazing, but what are your plans for the future? Offering an enterprise package?

Tim Anglade: Yeah. We’ve already generated revenue selling both services and also products around licenses to large company that need it. Our basic business model isn’t dissimilar to say, GitHub, where 90% plus of the product ends up being given away for free but it helps us build a community, build a standard on which we can up-sell a small portion of the population, to specifically ensure a sort of product that a large corporation would need. We’re looking to be a bit more public about what those offerings are in the next few months. We’re actually still learning a lot as to what people want to do on top of Realm and what specifically big corporations want out of Realm. It’s taking us a long time to refine that understanding into a nice packaged product with nice names and nice messages around them. Doing that while also taking care of launching all these new platforms and doing all support around that has been a bit of a slow ordeal but we’re going to have more announcement on that front in the next few months.

I always assume people have read the website, but in case they haven’t, the basic thing is we are free to use. All of our high level code, all of our JavaScript code in the case of React Native, is free and open sourced, Apache 2.0 on GitHub. We’ve committed to keeping that open source and to maintaining it, so it’s not going to be the case that what you’re using today is going to become paid all of a sudden. Everything that is open on GitHub will stay open on GitHub and we’ll also be actively contributing to expand it. That’s the basic promise of the business model.

Under the Hood

Lee Johnson: There are definitely a couple key features that I really love that I want to dig into shortly. But first, I just have a question about the code base for the JavaScript versus the OS side for React Native or Android side. Is it a SQLite under the covers with a bridge, or did you guys rewrite something completely different? How is that orchestrated, and could this ever develop into a node package?

Tim Anglade: The basic thing about Realm is that we don’t use SQLite. We’re not an ORM and we’re not building stuff with SQLite. So that’s different from virtually every other persistence option out there. We’re not building on LevelDB or any other key value store either. We’re built on our own underlying storage engine, which is a part of Realm that’s not open source right now, in full disclaimer. It’s something that we’re looking to open source, but it just keeps getting backlogged with everything else.

So we’re built on our own underlying storage system right now, that’s gonna go Apache 2 in the next few months. That is not strictly speaking a relational database, it’s a mix between a relational database and an object database. That powers a lot of the high-level feature that you see.

The reason why we’re able to give you a native JavaScript interface with really nice objects, and nice relationship model, and fast queries and everything else, is because we have our own storage underneath. It allows us to do a lot of really nice things, like an MVCC type storage underneath. like Git, but for your data objects. It allows us to do a lot of zero-copy things, it gives us a really nice threading model, which isn’t terribly useful of course in the case of React Native, but we hope that might change in other platforms. It’s a replacement for SQLite underneath, and at the higher-level is a replacement for an ORM, but it’s not exactly what’s happening.

There are some major differences, for example an ORM will take a copy of your data out of the database. That’s really bad for consistency, that’s really bad for resource usage, that’s really bad for speed. Whereas Realm has those two separate layers as well, like a SQLite layer, and like an ORM layer, but Realm’s layers are actually integrated to work together. We don’t copy data in and out, we give you complete consistency, everything is memory mapped all the way down. We get huge performance savings, huge ease of use improvements, and huge consistency improvements by having those two layers really tightly integrated together as opposed to the SQLite and ORM combo.

Scott Kyle: A huge portion of the Realm language bindings are shared between all the platforms within C++. Both that core storage layer that Tim was talking about and then another layer on top of that provides many of the facilities that all the language bindings need. Then we just specialize on top of that for each language. So with Swift, Objective-C, Java, they all require their own custom integration points, and we have this intermediary layer where we can make that really easy. We’re constantly improving that as well, and we do that obviously for Realm React Native as well.

Also like what Tim said, the objects that you pull out of the database, they’re not copied out. You’re just always looking at accessors to the data that’s in the database. So we can do lots of different kinds of operations very quickly that are sometimes very slow with ORMs because they have to make many queries through the database. Whereas we don’t really need to make many queries. We can do it all in one shot when the user needs it.

Tim Anglade: Yeah and there’s a specific example to give people a concrete example. If you come from a SQLite or ORM background, you have this idea of if you do a query, and that’s just a representation or projection of your data. If you want to edit the object you have to do a separate thing, and then find the object and edit it. With Realm, you don’t have to worry about that. Any time you get a query, all you get is the actual object and you can just modify them in line, for example. Again, any time you access any value in Realm, you’re not accessing some sort of derivative, or projection, or copy of it. You’re accessing the raw data itself, with a minor asterisk for consistency and refresh loops and stuff like that. This is a bit of a simplified view but you have access to everything.

Scott Kyle: We still have transactions, so when you do make modifications to your database you do that inside of a transaction. If anything does go wrong, like an exception is thrown, or even if the app itself crashed for some reason, your data should remain consistent because that transaction would not have been committed. Besides for making all your mutations inside a transaction, we try to keep the objects like their native object, like with JavaScript. Most of the Realm objects that you use, they just act like native JavaScript objects, or arrays, or whatever. So it’s really easy to use and integrate, and even pass around Realm objects into other libraries that are not even expecting a Realm object to be able to act on them.

Tim Anglade: Another quick thing to drop in there as an example of the interesting aspect of building our own core and integration thereof, is that we get to really apply a lot of reactive patterns directly into the design. Not in a very functional sense, we’re not doing streams and that stuff, but we have a really nice ability to just let you work very asynchronously, get notified of changes that are happening, and we’re doing more and more work there.

This is especially raw on the React Native side, where we still have to build in async inquiries to start with. The basic promise of the underlying engine, that you can see in the other products that we’re developing for Swift and Objective-C, and that’s on its way to React Native, is the ability to follow in a lot of really nice reactive patterns. Where you just subscribe to updates and are able to modify your UI when the underlying data changes very easily. Many people have tried and it’s very hard to build on top of SQLite or key value store, so it’s one of the nice things of being able to start fresh and build on your own foundation.

Auto-Updating Results & Listviews

Lee Johnson: That actually brings up one of the features I wanted to talk about, which is the auto-updating results which you even have for the React Native client. It’s pretty awesome especially considering the JavaScript, and iOS with its restrictions on the WebSockets and other things. For example, in iOS you’re restricted on how you can do those notifications for anything. How does that work, and could you explain the auto-updating a little bit and how it ties in? You also have a custom listview for React Native, how could those maybe be used together to show auto-updating of a listview?

Scott Kyle: Like we were talking about when you make a query into a Realm, you’re getting a view on that query that you made. So when you try to access objects inside those results, that’s when the actual query is made, and if something in the Realm changes that would affect that query. Then that query would get automatically updated so when you try to access objects out of those results again, it would give you a live representation of what the state of the Realm is at that moment.

We have change notifications but unfortunately right now they’re not super granular. So if you make a change to your Realm you’ll get a notification, but soon you’ll be able to get a change directly on a query that you made. For instance, if you have a listview – We have our own custom listview that would tie in with this very nicely. You can get a notification that something in that results changed so that you can re-render the listview. Right now we would re-render it on any change for now. It’s made to do that very efficiently because nothing has changed.

Our custom listview is not actually necessary to use to integrate Realm with a listview, it just exists to make it extra fast. For our own test purposes, we have a little app with a listview with about one million items in it, and you can scroll through it and delete them, and see them be deleted immediately. Using the stock listview, you could still use Realm with it but it would be a bit slower because of certain assumptions that it makes. So that’s why we built that, to make it extra easy and extra fast to use Realm queries, or Realm lists, inside a listview in your app.

Tim Anglade: The basic promise of what we’re building, that’s coming to React Native really quickly, is, let’s say you have a part of a UI that depends on a counter (the number of items that match a query and just one dynamic tooltip, or some sort of counter badge). You can subscribe to whether or not the results of this query have changed in any way, and then in the handler for that notification, you just recompute and redraw your UI. You don’t need to rerun the query. You don’t need to do anything because the count itself will have data automatically.

It’s super straightforward and painless to do a lot of basic UI changes. You don’t need to set up a lot of boilerplate code and you don’t need to think about the consistency of your data. It feels pretty native and fluid, with the JavaScript intact. We’re pretty excited about it – it’s pretty raw, but you can take a look at the other bindings and see what’s gonna drop in React Native very soon, as soon as we get done fixing some of these early bugs we’ve found, and support issues from the community.

Scott Kyle: This ties back in with what Tim was talking about earlier, how we have certain advantages. We’re not just building on top of SQLite, and one of the advantages of that is the deep integration from the core storage layer. When we do integrate fine-grained notifications, we won’t just get a message that something changed, we’ll be able to get a message as to exactly what changed. When updating UI components, knowing exactly what changed means you have to do a lot less recomputing diff’s and trying to figure out for yourself what changed. Again, it would just let you build an app that’s a little bit more performant.

Tim Anglade: It’s important to understand the general design patterns we have at the end of this. We fully bind to reactive patterns. I remember talking about that with Alexander, one of our two founders, and how he was pushing that when I first met him in 2013. Back then not very many people were super eager to talk about that, but it’s been driving the design of what we’ve been doing, at TightDB before even Realm, and definitely today at Realm. Whenever you’re wondering about what database we’re building, what framework we’re building, you can expect us to follow reactive patterns in everything we do.

Async

Peter Piekarczyk: How do you handle async states, loading indicators, and anything similar to that?

Scott Kyle: Right now when you query a Realm in Realm React Native, we only support synchronous queries right now, but async is coming pretty soon. Generally speaking, one of the reasons why async hasn’t been a priority for us is that we don’t generally see a performance impact unless you’re around 100,000+ items, and also asking them to be sorted in a certain way. For your regular use, standard to-do lists for normal people who don’t have 100,000 to-do’s, you don’t see much of a performance impact at all by doing live synchronous queries. That’s why we haven’t done that yet, but we are working on it, so for large databases you will be able to make asynchronous queries.

Tim Anglade: We haven’t agreed on the design specifically for that yet, but in other products that we have for Java and Swift, we let you determine what state your query is in and whether you’re still computing results, or whether you have finally computed the results, which makes it easy to update your UI accordingly.

As Scott said, in practice this is not a performance bottleneck, which I know flies in the face of a lot of “best practices” for app development, where people tell you to do all the I/O asynchronously. A lot of those best practices are designed with stuff like SQLite and ORMs in mind, and the performance improvement you get out of Realm is actually very major to the point where it’s really much closer to accessing objects in memory than it is to performing old school I/O like you’re used to with SQLite. We understand some queries still will need to be done asynchronously, but we recommend people to just enjoy the fact that they don’t have to worry about the state of their query and whether it has finished completing for most cases, and can just do a synchronous query.

Encryption

Lee Johnson: One thing I definitely want to discuss, because it’s one of the features that you guys have that really excited me, is the encryption. A lot of times, especially with SQLite for example because that’s what most people are familiar with, it’s a pain to get the SQLite cipher and get it set up. You have data on a device and devices are getting hacked all the time, and it’s in the news, now, with Apple fighting with the government over encryption.

We deal with data, but we don’t put any student data on mobile devices for that very reason. It’s just not worth the risk. But for health care applications for example or anything sensitive, can you walk through the use case and your encryption?

Tim Anglade: Encryption is one of the things that we actually had at launch for Android because we knew it was a big limiting factor on that platform. iOS already has a lot of built in encryption of files and all that stuff, so it was pretty good. For Android it was really nice to be able to just do it first party. So now we have encryption rolled out across platforms on both IOS and Android and accessible from React Native whether you’re targeting iOS or Android. To be clear though we don’t implement our own crypto, we piggy-back on either OpenSL or common crypto and we use standard AES 256 algorithm.

It’s really just like accessing any Realm you have in your application but instead of just opening it, you just have to pass the key to encrypt it or decrypt it. The only tricky part is where you store that key. You have to make sure you either get that key from somewhere secure, from a third party service, or you have to store it securely on the device. I believe there’s a library that lets you alias that across different OS’s. If you’re comfortable storing that key or getting that key from somewhere, Realm makes the encryption part very easy and uses best practice implementations from trusted 3rd party libraries. It’s definitely been mentioned as something that people like quite a bit. Getting the same level of encryption from other frameworks, especially on React Native, is really cumbersome otherwise.

Lee Johnson: Yeah, there are a lot of hoops to jump through to get any of that, but with Realm it’s literally just setting the key, right? You have a method to create the hash based on your key and then you pass it – is that how it works?

Tim Anglade: Yeah, it’s just a 64 byte key and as long as it’s correct we’ll let you open the file. I don’t have specific stats on usage but I’m always amazed because I hear about even non-health care applications or e-commerce applications using this feature, because it’s much easier than relying on platform features or other integrations. We’re really glad that we put that in. It was a bit of a complex implementation task as you can imagine to wire all that in, and we weren’t quite sure if it was worth the time we spent, but it seems to be very well-received.

Object Types

Lee Johnson: You have different object types in your models, with lists being one of those types. Can you go through the object types that you can have in your models and your schema and how they’re set up, and some of those list types, object types, and different ways you can connect data?

Scott Kyle: When you define a schema for an object, for instance, and you have a data model of a person for example, you can have your basic primitive types as properties on that person, such as string for their name and a number for their age, or a date for their birthday. We also support relationships so you can have, for instance, partner could be a pointer to another person, for instance. Or we refer to them as references to other persons. You can have lists too. You could have a list of friends and inside that list there are references to other persons, or to other objects, like pets for instance.

You define all that in your schema and then when you create those person objects or you get them out of the Realm when you query for them, you get objects that will basically behave just like native Javascript objects. You access a property like a name, and you get back a string, or you access a property called “friends” and you get back a list that basically looks and acts as much like an array as we really can make it. It has all your standard array methods, both for iterating through them and also for slicing them or pushing new objects onto them. Generally speaking it will behave as much like an array as we could make it behave. That’s list and that’s why we don’t call it array because we don’t want to mislead anyone into thinking it’s exactly an array, but generally speaking it will behave just like an array.

Tim Anglade: That name varies a little bit from platform to platform because we call it an array when we do Objective-C and we call it a list when we do it in Java. It really emulates arrays pretty closely as Scott mentioned. We think about it as “What if you just had to model your data with regular Javascript objects? How would you make that happen?” We try to give you Realm models that are as close to that as possible, that you can very intuitively understand. For example you’re going to declare your own standard properties, you’re going to link one object to another just by referencing them, and when an object references multiple other objects like a person having many cats then you’re going to use an array to do that. We wanted that syntax to be very intuitive. We don’t want you to have to worry about schemas and abstract types that are not native to your language or any other complexities, but rather you can just chain things as if they were just regular objects.

Lee Johnson: You mentioned the pointer to the actual object. There are document stores like MongoDB, where things are nested and it gets really difficult to normalize, then you have relational databases like anything Postgres, where the relationships can get really complex and can be overkill for mobile applications. Is Realm some blend between the two?

Tim Anglade: It’s more of a mix between a relational database and an object database. This is the part where it gets tricky, and I’m simplifying drastically, but the short of it is underneath, your data is in tables with rows and columns. The columns have types, and everything is neatly stored in this platform-independent format. So if you write that Realm data from React Native you can open it from Swift concurrently, for example. In both languages you’d be able to read and write the same file.

That data is always automatically mapped to an object, so when you reference another object from an object it’s just going to point you to that other table that contains the row that has the columns for the object that you want. It’s a property that lets you do a very expressive thing, like a document model, but without the overhead of every document with nested sub-documents becoming too large and complex to index or fetch in and out of memory well, because we always have independent properties accessible in memory.

It allows us to do an immense amount of lazy loading. If, for example, you’re loading an object that is graphed sequentially to every other object in your database, and it goes all the way down: object linked to an object, linked to another object, etc. Just because you load that top level object, it won’t load the rest of your graph in memory. You won’t even load any of their properties until you try to access one of those properties, because everything is just referenced. It’s a nice hybrid storage system that gives you a high level expressive object model but with the performance of a traditional table-oriented database. It’s not strictly speaking relational, because we don’t use real relational algebra and joined tables. It’s a table that consists of rows and columns, so you can isolate individual properties and values very easily and you have maximum performance there.

Read more

Realm React Native 0.11.0

by /

First off, a word of thanks for the warm reception you all gave to our newest product. Hundreds of you are already using Realm in React Native apps, and the community is growing every week. Thanks to your feedback, we were able to release version 0.11.0 of Realm React Native. We fixed a ton of bugs since our initial release, added support for indexed fields and encryption, and you can now install Realm with just one line in your terminal!

Easier setup and installation

Realm React Native is now packaged as a static library for iOS and can be setup completely from the command line using rnpm. To use the new version after updating your package.json:

  • Remove the reference to RealmJS.xcodeproj from your Xcode project (under the Libraries group)
  • Make sure rnpm is installed and up-to-date: npm install -g rnpm
  • Run rnpm link realm from your app’s root directory

Major additions

  • Support for encrypted Realms
  • List and Results are now iterable (e.g. supports for...of loops)
  • Common iteration and other methods from Array have been added to Results and List
  • Case insensitive queries are supported by adding [c] after operators
  • Support for indexed bool, string, and int properties
  • Added Realm.schemaVersion method, to check whether unopened Realms should be migrated

Major bugfixes

  • Fix for crash on Android when initializing the Realm module
  • Fix for using Chrome debug mode from a device

For the full details, check our the full release notes.


Thanks for reading. Now go forth and build amazing apps with Realm! As always, we’re around on Stack Overflow, GitHub, or Twitter.

Read more

Introducing Realm Cocoa Converter

by /

Today, we’re launching a new open source library to complement Realm Objective-C and Realm Swift called Realm Cocoa Converter!

We’ve heard from many of our users that there are times in their development where they would like to be able to move large batches of data in and out of Realm files more easily. The goal of Realm Cocoa Converter is to provide a common library to enable importing data from other file formats into Realm, and vice-versa, exporting data from Realm.

The library is written in Swift, and is very easy to work with. For exporting a Realm file, it only takes a single class. For example, exporting to CSV:

let path = ... // Absolute path to the target Realm file
let destinationFolder = ... // Absolute path to the folder in which to save the CSV files

let exporter = CSVDataImporter(realmFilePath: path)
exporter.exportToFolderAtPath(outputFolderPath: destinationFolder)

When importing files into Realm, the process is a little more complicated. This is because Realm needs to generate a schema for each type of object in the input file before the data can be imported.

Realm Cocoa Converter handles this by having a separate class for intelligently generating the schema, which can then be either saved, or modified by the user before the import.

This schema is then supplied when the import operation takes place.

var filePaths = [String]() // An array of paths to each file representing a Realm table

// Create an instance of the schema generator and analyze the contents of those data files
let generator =  ImportSchemaGenerator(files: filePaths)
let schema = try! generator.generate()

let destinationRealmPath = ... // The destination folder for this Realm file

// Create an importer object, and import those files using the previously generated schema
let dataImporter = CSVDataImporter(files: filePaths)
try! dataImporter.importToPath(String(destinationRealmPath), schema: schema)

While Realm Cocoa Converter itself is implemented in Swift, it has also been designed to properly integrate with Objective-C and its code conventions as well.

NSString *path = ... // Absolute path to the target Realm file
NSString *destinationFolder = ... // Absolute path to the folder in which to save the CSV files

RLMCSVDataExporter *exporter = [[RLMCSVDataExporter alloc] initWithRealmFileAtPath:realmFilePath];
[exporter exportToFolderAtPath:destinationFolder withError:nil];

Currently, the library contains support for generating Realm files from groups of CSV and XLSX files, and the ability to export to CSV files. We’re planning on adding more formats in the future, including JSON and SQLite.

Realm Cocoa Converter is still very much in its infancy, and we’d love to have you help in its development! If you have any specific use-cases in your app for it, or additional formats you think it should support, please let us know!


Thanks for reading. Now go forth and build amazing apps with Realm! As always, we’re around on Stack Overflow, GitHub, or Twitter.

Read more

Realm Java 0.88 — Better Objects!

by /

We just released a new version of Realm Java to this website and to Maven. This release includes many exciting new features.

Better Objects!

With this release, Realm for Android gives developers full control over their Realm objects. You will now be able to:

  • Add custom methods to your objects.
  • Implement interfaces.
  • Add custom logic to the getters and setters.
  • Name the accessors in any way you prefer.
  • Skip accessors altogether and make the fields public.
  • Write your own toString(), equals(), and hashCode() methods.
  • Combine Realm with Lombok.

This means that from now on, dealing with Realm objects will feel a lot more like handling normal POJOs.

In order to make this happen, we had to make Realm a Gradle plugin and not simply a library. As mentioned in a previous post this means that from now on you will have to change the way you associate Realm to your project in the build.gradle file.

buildscript {
 repositories {
    jcenter()
 }

 dependencies {
    classpath 'io.realm:realm-gradle-plugin:0.88.0'
 }
}

apply plugin: 'com.android.application'
apply plugin: 'realm-android'

This also means that from now on we will not support other build systems other than Gradle. If this seems unreasonable to you, we’ve already prepared two issues for you to vote on to re-introduce support for Maven and/or Ant.

Your opinion will lead our decision on whether and when we will implement Ant and Maven plugins.

Note that a few restrictions still apply to Realm objects:

  • Subclassing from anything but RealmObject is not allowed.
  • Fields are not allowed to be final, volatile, or transient.

Other Improvements

This release also brings a number of other improvements:

Breaking Changes 🚨

  • All notifications now use the Looper queue. Previously, when committing a transaction, all RealmChangeListeners on the same thread were triggered immediately. In order to make change listeners more predictable and consistent, they are now being delayed to the next Looper message.

  • All RxJava Observables now hold a reference to the Realm instance that created them. This means that a Realm file is now only fully closed when all Observables have been unsubscribed from. It also means that it is now possible to close a Realm instance using doOnUnsubscribe.

See the full changelog for all the details.


Thanks for reading. Now go forth and build amazing apps with Realm! As always, we’re around on Stack Overflow, GitHub, and Twitter.

Read more

Introducing Realm React Native

by /

This post announced the launch of Realm React Native in Feb. 2016 – Since then we’ve released new features and made numerous updates. Check out the latest here in Docs.

Today at Facebook’s React.js Conference, we’re launching a new Realm mobile database built specifically for React Native. It offers easy object persistence and full query capabilities, with a performance profile that’s usually 2–10x faster than existing options.

Like the other editions of Realm, it is designed from the ground up to enable reactive app development, with live objects, change events, and support for unidirectional data flows.

Here’s how it looks:

const Realm = require('realm');

class Person {}
Person.schema = {
    name: 'Person',
    primaryKey: 'name',
    properties: {
        name: 'string',
        age: {type: 'int', default: 0},
    },
};

const realm = new Realm({schema: [Person]});

// Query
let people = realm.objects('Person', 'age >= 17');
people.length // => 0

// Write
realm.write(() => {
    savedPerson = realm.create('Person', {
        name: 'Hal Incandenza',
        age: 17,
    });
});

// Queries are updated in real-time
people.length // => 1

Realm is a database built from the ground up for the unique challenges of mobile app development, that runs directly inside phones, tablets or wearables. We launched for Java, Objective-C, & Swift in 2014, and are now used on hundreds of millions of devices today by appmakers including Starbucks, Cisco, Walmart, Google, Amazon, & eBay, plus many many others.

Today, we’re launching for React Native, Facebook’s JavaScript framework that lets developers write JS code that becomes native iOS & Android apps. (Note: Realm supports React Native, not the web framework React. We’re all in on mobile!)

Realm React Native brings the modern design & simplicity you expect from Realm, and will allow you to target both iOS and Android with the same codebase. Despite only launching to the public today, Realm React Native has already been used in production for over 2 months by TaskRabbit!

What is Realm

Realm is not an ORM, and is not built on top of SQLite. Instead we’ve built a full database for mobile app developers, one that uses native JavaScript objects that are dynamically mapped to a full, custom database engine (not just a key-value store). This allows us to provide a simple API while preserving performance. With Realm, you can model complex data, link objects in a graph, and compose advanced queries.

class Dog {}
Dog.schema = {
    name: 'Dog',
    properties: {
        name: 'string',
        age: 'int',
    }
};

let realm = new Realm({schema: [Dog]});

realm.write(() => {
    realm.create('Dog', { name: 'Rex', age: 3 });
});
// Basic query
let r = realm.objects('Dog').filtered('age < 8');

// Queries are chainable
let r2 = r.filtered('name contains "Rex"');
r2.length // => 1

realm.write(() => {
    realm.create('Dog', { name: 'Rex Maximus', age: 4 });
});

// Queries are updated in real-time
r2.length // => 2
class Person {}
Person.schema = {
    name: 'string',
    dogs: {type: 'list', objectType: 'Dog'},
};

let realm = new Realm({schema: [Dog, Person]});

realm.write(() => {
    let person = realm.create('Person', {
        name: 'Tim',
        dogs: [{name: 'rex', age: 3}],
    });
});

You can see more examples of how to use these APIs in the ReactExample app and in the JS test files.

Why use Realm?

Easy

Realm’s primary focus has always been ease of use, and as you can see from the samples above, Realm React Native is no different. After that, we’ve been working on some of the same advantages our other products are known for…

Fast

Realm’s ease of use doesn’t come at a performance cost. Because of its memory mapping, lazy loading, and custom storage engine, Realm is usually faster than SQLite or AsyncStorage despite offering a rich object-based API. Although we always recommend everyone test their own use-cases, we usually see huge speedups when porting code to Realm. See benchmark results below.

Cross-platform

Realm React Native’s API allows you to write your app once in JavaScript and target both iOS & Android, but the Realm file format is also completely cross-platform, allowing data to be shared across iOS & Android easily. For debugging, .realm files can be opened via the Realm Browser.

Advanced

Realm objects are always up-to-date with the underlying data, making it trivial to follow reactive patterns or a unidirectional data flow. You can link Realm objects in graphs, query any property combination via an expressive query language, and even easily integrate Realm data in React Native ListViews.

Trusted

Realm React Native is built on top of the same core as Realm Java, Objective-C, & Swift, which is trusted by hundreds of millions of people around the world, and used by e-commerce applications, banks, healthcare providers, and even governments. Realm React Native itself has been used by TaskRabbit in production since December 2015.

Community-driven

Realm React Native is built in the open on GitHub. Features are prioritized based on customer requests and we welcome contributions.

Supported

Realm prioritizes support & bugfixes above all else. You can get answers about your database directly from the people that build & maintain it, via Stack Overflow, GitHub or Twitter.

Tests conducted on February 19, 2016, using the latest versions available of Realm, React Native SQLite Storage for SQLite, and React Native Store for AsyncStorage. Measurements taken on an iPhone 6s running iOS 9.2.1 and a Nexus 9 running Android 5.0.1. Source.

What’s Next

Today’s release of Realm React Native is slightly less advanced than what Realm Java, Objective-C, & Swift currently offer, but it’s already trusted in production by appmakers like TaskRabbit. We’d love your feedback on what we can improve, and we’re particularly interested in bug reports or feature requests on our GitHub repository. Expect the API to improve significantly over the next few weeks, especially as we polish advanced features like migrations and queries.

If you are a fan of JavaScript but React Native is not your platform, note that we plan on adding support for Cordova/PhoneGap/Ionic, and Node.js (V8) compatibility in the future.

We can’t wait to see what you will build with Realm!

We’d like to thank TaskRabbit & the React Native team for their support. Realm React Native is much better because of their input, & we are honored to launch Realm React Native at React Conf 2016!

Read more

Realm featured on the Fragmented podcast

by /

Christian Melchior, of our Java team, was a guest on the latest episode of the Fragmented podcast to talk about the latest at Realm. Listen to the whole episode for more!

Realm was featured on the Fragmented podcast.

Thanks, Donn, for having Christian on!

Read more

Realm Objective-C & Swift 0.98

by /

We just released version 0.98 of Realm Objective-C and Realm Swift.

This release adds support for collection notifications, background queries, subqueries, indexing & performance improvements, bug fixes and more!

Collection Notifications

It’s now possible to register for notification blocks to be invoked whenever Realm collections are updated by calling their addNotificationBlock method.

Collection instances asynchronously invoke the block with the initial collection when registered, and then invoke it again after each write transaction which changes the collection or any items contained within it.

// Observe Results Notifications
let token = realm.objects(Person).filter("age > 5").addNotificationBlock { results, error in
    // results is identical to 'realm.objects(Person).filter("age > 5")'
    viewController.updateUI()
}

// later
token.stop()
// Observe RLMResults Notifications
self.token = [[Person objectsWhere:@"age > 5"] addNotificationBlock:^(RLMResults *results, NSError *error) {
    // results is identical to '[Person objectsWhere:@"age > 5"]'
    [myViewController updateUI];
}];

// later
[self.token stop];

See our docs on Notifications and the following API documentation for details:

Swift Objective-C
Realm.addNotificationBlock(_:) -[RLMRealm addNotificationBlock:]
AnyRealmCollection.​addNotificationBlock(_:) -[RLMCollection addNotificationBlock:]
Results.addNotificationBlock(_:) -[RLMResults addNotificationBlock:]
List.addNotificationBlock(_:) -[RLMArray addNotificationBlock:]
NotificationToken.stop() -[RLMNotificationToken stop]

This continues laying the foundation that was started with KVO support for more granular notifications. Expect more improvements in this area in the near future.

Background Queries

Working on collection notifications allowed us to improve the mechanism that kept results always “live”.

Once the query has been executed, or a notification block has been added, the Results is kept up to date with changes made in the Realm, with the query execution performed on a background thread when possible.

This change should automatically reduce work done on the main thread for apps using Results to directly back table views, which is a recommended design pattern.

Subqueries

Subqueries are now supported, enabling a whole new class of queries that were previously either impossible or inefficient to implement with Realm.

For example, here’s how you could previously model a query for companies with part time employees over 30 years old:

let partTimeOverThirty = realm.objects(Employee).filter("age > 30 AND fulltime = NO")
let companies = realm.objects(Company).filter("ANY employee IN %@", partTimeOverThirty)
RLMResults<Employee *> *partTimeOverThirty = [Employee objectsWhere:@"age > 30 AND fulltime = NO"];
RLMResults<Company *> *companies = [Company objectsWhere:@"ANY employee IN %@", partTimeOverThirty];

But this query is fairly expensive to compute and doesn’t correctly auto-update. It would also be much more complex to compare count against any value other than zero.

And now with subqueries:

let companies = realm.objects(Company).filter(
  "SUBQUERY(employees, $employee, $employee.age > 30 AND $employee.fulltime = NO)[email protected] > 0"
)
NSString *predicate = @"SUBQUERY(employees, $employee, $employee.age > 30 AND $employee.fulltime = NO)[email protected] > 0";
RLMResults<Company *> *companies = [Company objectsWhere:predicate];

However, the following limitations apply to subqueries in Realm:

  • @count is the only operator that may be applied to the SUBQUERY expression.
  • The SUBQUERY(…)[email protected] expression must be compared with a constant.
  • Correlated subqueries are not yet supported.

Indexing & Performance Improvements

It’s now possible to index BOOL/Bool and NSDate properties, as well as optional properties in Realm Swift.

Performance when deleting objects with one or more indexed properties was also greatly improved.

Breaking Changes

  • Realm Swift no longer supports Swift 1.2. The last release to support Swift 1.2 was 0.97.1.
  • +[RLMRealm realmWithPath:]/Realm.init(path:) now inherits from the default configuration.

Other Enhancements

Includes changes shipped in 0.97.1.

  • Swift: Added Error enum allowing to catch errors e.g. thrown on initializing RLMRealm/Realm instances.
  • Fail with RLMErrorFileNotFound instead of the more generic RLMErrorFileAccess, if no file was found when a realm was opened as read-only or if the directory part of the specified path was not found when a copy should be written.

Bug Fixes

Includes changes shipped in 0.97.1.

  • Fix incorrect results or crashes when using -[RLMResults setValue:forKey:] on an RLMResults which was filtered on the key being set.
  • Fix crashes when an RLMRealm is deallocated from the wrong thread.
  • Fix incorrect results from aggregate methods on Results/RLMResults after objects which were previously in the results are deleted.
  • Fix a crash when adding a new property to an existing class with over a million objects in the Realm.
  • Fix errors when opening encrypted Realm files created with writeCopyToPath.
  • Fix crashes or incorrect results for queries that use relationship equality in cases where the RLMResults is kept alive and instances of the target class of the relationship are deleted.

Thanks for reading. Now go forth and build amazing apps with Realm! As always, we’re around on Stack Overflow, GitHub, or Twitter.

Read more

Installing Realm for Android: Why is it Changing?

by /

Limitations of Realm

In the year since we launched Realm for Android, we have been honored by the popularity of our library despite its limitations. In particular, we have heard many complaints about the requirement to extend classes from RealmObject, which in turns causes the following constraints:

  • No public fields
  • Only standard accessor names
  • No added logic to accessors
  • No custom methods
  • No implementing of interfaces

All these limitations come from a common reason: Realm is using proxy classes to perform zero-copy operations on the underlying data store. Let us go through all the limitations one by one.

No public fields

Java does not allow proxies to access fields, so Realm proxy classes cannot intercept when such fields are accessed. Also, fields are by definition stored in the Java heap. This means that data would have to be copied from Realm to Java memory, which is against our zero-copy practice.

Only standard accessor names

Annotations processors are extremely powerful tools, but they have some limitations. One of those is the inability to inspect the actual code of the class being proxied: only fields and methods names are available. This means that we had to rely on convention to make Realm proxy classes behave as expected.

No added logic to accessors

This is a direct consequence of the previous limitation. Since the implementation of the accessors cannot be accessed by the annotation processor, it is impossible to replicate such logic in the proxy classes. Of course we could simply call super.getMyField();, but that operates on the in-memory data, which is not being used in the proxy class.

No custom methods

Again, since the annotation processor cannot inspect code, it’s impossible to guess what a custom method actually does. Hence, Realm cannot allow such methods in order to guarantee correctness.

No implementing of interfaces

Since custom methods are not allowed, interfaces that require the implementation of methods are not allowed either.

So, How to Fix This?

The only way to overcome such limitations is with bytecode weaving. This means that after compilation is over, some modifications are applied to the .class files. We will go through the details of such operation in a future blog post. What’s important here is that the only way to cleanly perform bytecode weaving on Gradle builds is with a plugin. This is also what the popular libraries RetroLambda and Hugo are doing, so we know we are in good company!

Your build.gradle file

Until now, you installed Realm like this:

repositories {
    jcenter()
}

dependencies {
    compile 'io.realm:realm-android:<version>'
}

With the Gradle plugin, it looks like this:

buildscript {
    repositories {
        jcenter()
    }
    dependencies {
        classpath "io.realm:realm-gradle-plugin:<version>"
    }
}

apply plugin: 'realm-android'

The first code sample should already be in your build.gradle file, so all you need to add are the lines staring with classpath and apply plugin: 'realm-android'.

Why Now?

Why do we provide the plugin now, even before the bytecode weaving step is released? There are actually some other reasons!

Smaller APKs!

Using the plugin we can now ship Realm as an AAR, as opposed to a JAR. We can also avoid including the annotation processor in the library and make it a standalone package. This means your final APKs will not have to include the annotation processor, shaving a few kilobytes off your app.

Easy ABI splits!

Another consequence of shipping an AAR is that now you are able to cleanly perform ABI splits without having to use the workaround described in our previous blog post. This means that you won’t have to ship a single APK containing native libraries for all CPU architectures.

What About Eclipse?

As you probably already know, Google is deprecating the ADT plugin for Eclipse and is ending its support at the end of the year. As far as we can tell, the vast majority of Android developers have already switched over to Android Studio, so that’s where we are focusing our efforts. That being said, our priorities are dictated by your feedback, and we will consider writing an Ant or Maven plugin if enough people ask for it.

Happy Coding!

We hope you will find this change useful! Please don’t hesitate to contact us. As always, we’re around on Stack Overflow, GitHub, and Twitter.

Read more

Year in Review: 2015

by /

2015 was a pretty busy year for Realm. From talks to releases, here are some of our favorite moments!

Releases

We had 24 releases this year, which took us from Realm Java 0.76 and Realm Cocoa 0.89 (remember when we only had Objective-C?) to Realm Java 0.87 and Realm Objective‑C & Swift 0.97.

Realm Java picked up better sorting, encryption, ARM64 & x8664 support, RealmModules, configurations, in-memory Realms, null support, an async API, a new migration API, a dynamic API _(Ed. note: How many APIs do you people need?), and most recently RxJava support.

Realm Cocoa of course had its biggest change in its Swift-first API, with all the associated Swift-y improvements. Other than that, all it got was encryption, sharing Realms between processes, Objective‑C generic support, support for watchOS & tvOS (Ed. note: How many OSs do you people need?), configurations, KVC collection operators, KVO, class subsets, and bitcode… so basically nothing else.

We also spent some time on a few side projects:

  • SwiftLint - Our Swift linter, which just launched this year!
  • jazzy - The Swift documentation generator, which went from lowly v0.0.19 to v0.4.1, adding support for Objective-C along the way, and is being used all over the web.
  • Realm Browser - Our OS X browser for Realm files, which is now on the App Store.

Blog posts

We published almost 200 articles to our blog this year. PHEW! We’re so happy that so many of you found them interesting! In case you missed some, here are a few of our favorites that are worth catching up on:

2016

With the way 2015 went down we feel like we’ve got a lot to live up to in 2016. We’re setting our sights even higher (25 releases!), and we’ve got big plans in the works. I hear you saying “Where’s my bulleted list of those?”. All in good time, friends, all in good time. For now, relax, enjoy your holidays, and Happy New Year, from all of us at Realm!

Read more

Realm Objective-C & Swift 0.97

by /

This is the last version of Realm Swift to support Swift 1.2.

We just released version 0.97 of Realm Objective-C and Realm Swift.

This release contains official support for tvOS, better Carthage support, more Objective-C API generics, performance & query improvements, deprecation removals and more!

tvOS 📺

You can now build Realm apps for the Apple TV! Although some enterprising users have been building & shipping apps for months already using our unofficial Apple TV development branch, it’s now a platform we officially support! 🎉

Because writing to the Documents directory is prohibited on tvOS, the default Realm path is set to NSCachesDirectory. However, please be mindful that tvOS can purge files in the Caches directory at any point, so we encourage you to rely on Realm as a rebuildable cache rather than storing important user data.

Building Realm Swift From Source Using Carthage

We’ve drastically simplified our build system to use a single Xcode project for both Realm Objective-C and Realm Swift, which means that Realm Swift can now be built using carthage --no-use-binaries.

Debugging Encrypted Realms

Until now, the mechanism used by our native encryption functionality relied heavily on Mach exception handling, which interfered with the operation of LLDB.

We’ve reworked the implementation of encryption so that it no longer relies on Mach exceptions, which means that it’s now possible to use LLDB to debug a process using an encrypted Realm. This also means that encryption no longer interferes with the behavior of crash reporters, which typically also rely on Mach exception handling. Finally, avoiding Mach exceptions means that platforms like tvOS & watchOS (which mark Mach APIs as unavailable) can now support encryption! Although we have yet to enable this for watchOS & tvOS pending further testing.

Generic Type Annotations in Realm Objective-C

Why let Swift have all the fun? Our Objective-C APIs have been updated to use generic type annotations wherever possible.

Breaking Changes

  • All functionality deprecated in previous releases has been removed entirely.
  • Adding a Realm notification block on a thread not currently running from within a run loop throws an exception rather than silently never calling the notification block.

Minor Enhancements

  • The block parameter of -[RLMRealm transactionWithBlock:]/Realm.write(_:) is now marked as __attribute__((noescape))/@noescape.
  • Many forms of queries with key paths on both sides of the comparison operator are now supported.
  • Add support for KVC collection operators in RLMResults and RLMArray.
  • Fail instead of deadlocking in +[RLMRealm sharedSchema], if a Swift property is initialized to a computed value, which attempts to open a Realm on its own.

Bug Fixes

  • Fix poor performance when calling -[RLMRealm deleteObjects:] on an RLMResults which filtered the objects when there are other classes linking to the type of the deleted objects.
  • An exception is now thrown when defining Object properties of an unsupported type.

Thanks for reading. Now go forth and build amazing apps with Realm! As always, we’re around on Stack Overflow, GitHub, or Twitter.

Read more

Realm Java 0.87 — with RxJava support!

by /

We just released a new version of Realm Java to this website and to Maven. This release includes first-class API support for RxJava.

RxJava

RxJava is a Reactive Extensions library from Netflix that extends the Observer pattern. It makes it possible to work with changes to Realm data using composable sequences of operators. All Realm classes except RealmQuery and RealmList can now be exposed as an Observable(https://github.com/ReactiveX/RxJava/wiki/Observable).

Realm Observables are built on top of our standard RealmChangeListener, so any Observable will emit objects at the same rate as the change listener would have been notified.

Realm realm = Realm.getDefaultInstance();
RealmResults<Person> persons = realm.where(Person.class).findAll();
Person person = persons.first();

Observable<Realm> realmObservable = realm.asObservable();
Observable<RealmResults<Person>> resultsObservable = persons.asObservable();
Observable<Person> objectObservable = person.asObservable();

This also works with Realm’s Async API.

realm.where(Person.class).equalTo("name", "John").findAllAsync().asObservable()
  .filter(new Func1<RealmResults<Person>, Boolean>() {
      @Override
      public Boolean call(RealmResults<Person> persons) {
          // Ignore unloaded results
          return persons.isLoaded();
      }
  })
  .subscribe(new Action1<RealmResults<Person>>() {
      @Override
      public void call(RealmResults<Person> persons) {
          // Show persons...
      }
  });

See the RxJava example project for more examples.

Configuration

Although RxJava is now a first-class citizen in our API, it is still an opt-in dependency. This means you have to provide RxJava yourself in build.gradle. This has the benefits of letting you decide which version of RxJava you want to use and preventing method count bloat for projects that don’t use RxJava.

dependencies {
  compile 'io.realm:realm-android:0.87.0'
  compile 'io.reactivex:rxjava:1.1.0'
}

We also added a new interface called RxObservableFactory for people that want to customize how the Observables are created. A custom factory can be configured using RealmConfiguration, but Realm comes with a RealmObservableFactory class which is used as the default factory. It supports RxJava <= 1.1.*.

RealmConfiguration config = new RealmConfiguration.Builder(context)
  .rxFactory(new MyFactory())
  .build()

Work-in-progress

While we are shipping initial support for RxJava now, we still lack Observable support for RealmQuery and RealmList. These will be added in a future update.

It is also important to remember that RealmObject and RealmResults are still thread confined. This means that is not possible to load Realm objects on one thread and use them on another.

// Default pattern for loading data on a background thread
Observable.defer(new Func0<Observable<RealmResults<Person>>>() {
    @Override
    public Observable<RealmResults<Person>> call() {
        return realm.where(Person.class).findAll().asObservable();
    }
})
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.subscribe(new Action1<RealmResults<Person>>() {
    @Override
    public void call(RealmResults<Person> persons) {
        // Accessing persons here would crash.
    }
});

// Instead, use Realms Async API for the same effect
realm.where(Person.class).findAllAsync().asObservable()
  .subscribe(new Action1<RealmResults<Person>>() {
      @Override
      public void call(RealmResults<Person> persons) {
          // Show data in the UI
      }
  });

RealmObjects and RealmResults are also live objects, which means that they are continuously updated with the latest data from Realm. This is great for consistency, but doesn’t work well with RxJava’s model that favor immutable thread‑safe objects. Therefore, we have added Realm.copyFromRealm() which makes it possible to copy Realm data into normal Java objects, effectively detaching them from Realm. This has a cost in terms of both speed and memory usage but can make it easier to work with operators such as buffer().

// Get different versions of a person
realm.where(Person.class).findFirst().<Person>asObservable()
        .map(new Func1<Person, Person>() {
            @Override
            public Person call(Person person) {
                // Convert live object to a static copy
                return realm.copyFromRealm(person);
            }
        })
        .buffer(2)
        .subscribe(new Action1<List<Person>>() {
            @Override
            public void call(List<Person> persons) {
                // Without `map` and `copyFromRealm`, v1 and v2 would be the same
                Person v1 = persons.get(0);
                Person v2 = persons.get(1);
            }
        });

Solutions for avoiding this work-around will be added in a future update.

See the full changelog for all the details.


Thanks for reading. Now go forth and build amazing apps with Realm! As always, we’re around on Stack Overflow, GitHub, and Twitter.

Read more

How 60Hz Went 100% Crash‑Free With Realm

by /

Today’s blogpost is by Suneth Mendis, cofounder at M2D2, who wrote to us about the M2D2 app 60Hz. Thanks Suneth!


60Hz is a simple app that helps users to keep track of their favourite TV shows. The app is in its 4th incarnation since its launch back in 2011.

The app churns through lot of data while operating. It has a large number of asynchronous background data-fetching tasks to keep the content in sync with the server. In v3.0, we introduced an NSCoding and keyed archiving technique to persist data in the app. This technique gave us the flexibility and stability we were after, but it carried a major flaw: accessing the persisted archive caused the device to heat up and eventually drain the battery, especially with large user libraries.

As a result, we re-wrote the entire application using Core Data in v4.0. First couple of releases of the app worked relatively well, with fewer crashes. However, the more changes we made to the Core Data schema, the more crashes seemed to come through. We worked really hard to isolate and fix these mysterious crashes, but by v4.1 things turned nasty. Our crash-free user rate was down to 60%, and our Twitter and Facebook feeds were filled with angry comments from our loyal users.

As a desperate measure, we turned to Realm. We had experimented with Realm for another project previously, but abandoned due to the lack of null properties. We realized Realm had matured since our last look and took the plunge to migrate from Core Data to Realm. The migration took few weeks to implement, but the results were stunning. Using Realm significantly reduced the complexity of the app, not to mention the neat and easy-to-understand data accessors. The current implementation of the app uses a combination of in-memory and persisted Realms to achieve the functionality. The crashes are no longer mysterious, thanks to very meaningful error messaging in Realm.

Since the launch of v4.2 few weeks ago, the crash-free user rate has gone up to 98% mark (sitting at 100% on most days). More than 120 DAUs compare to less than 80 previously.

Thank you Realm! I am sleeping well knowing our app is stable.


We’re glad we could help, Suneth! If you have a Realm success story, let us know!

Read more

Realm Java 0.86 — with new Migration API!

by /

We just released a new version of Realm Java to this website and to Maven. This release includes a replacement for the old Migration API, as well as a new Dynamic API to complement it.

Migration API

Migrations are still specified using a migration block in RealmConfiguration, but the interface has changed. Migrations now operate on a new Realm type called a DynamicRealm. DynamicRealm is a special variant of a conventional Realm that allows you to interact and change the schema while still having access to all of the ordinary object creation and query capabilities.

RealmMigration migration = new RealmMigration() {
  @Override
  public void migrate(DynamicRealm realm, long oldVersion, long newVersion) {

     // DynamicRealm exposes an editable schema
     RealmSchema schema = realm.getSchema(); 

     if (oldVersion == 0) {
        // Migrate to version 1
        oldVersion++;
     }

     if (oldVersion == 1) {
        // Migrate to version 2
        oldVersion++;
     }
  }
}

// Configure the migration as normal
RealmConfiguration realmConfig = new RealmConfiguration.Builder(context)
  .schemaVersion(2)
  .migration(migration)
  .build();

Classes and fields are now referenced using their names and can be created and modified in a fluent manner.

RealmSchema schema = realm.getSchema();

// Create Person class with two fields: name and age
schema.create("Person")
    .addField("name", String.class)
    .addField("age", int.class);

// Fields with special properties
schema.get("Person")
    .addField("id", long.class, FieldAttribute.PRIMARY_KEY);

// References to other Realm objects
schema.get("Person")
    .addRealmObjectField("favoriteDog", schema.get("Dog"))
    .addRealmListField("dogs", schema.get("Dog"));

// Creating new data during migrations
DynamicRealmObject person = realm.createObject("Person");
person.setString("name", "John");

// Delete a class and all its data
schema.remove("Person");

It is also possible to run custom transformations on all RealmObjects of a specific type. This can be useful if converting between types, or merging or splitting fields.

schema.get("Person")
  .transform(new RealmObjectSchema.Function() {
    @Override
    public void apply(DynamicRealmObject obj) {
      obj.setInt("age", obj.getInt("age") + 1); // Add 1 to all ages
    }
  });

See RealmSchema and RealmObjectSchema for all options, and take a look at our updated migration example.

Dynamic API

Migrations make use of the new Dynamic API, but it is also available outside migrations. It can be useful in scenarios where you are working with data where the model is not known at compile time, like importing data from CSV files.

Dynamic Realms are opened using the same configuration you use for the conventional Realms, but they ignore any schema, migration, and schema version.

RealmConfiguration realmConfig = new RealmConfiguration.Builder(context).build();
DynamicRealm realm = DynamicRealm.getInstance(realmConfig);

// In a DynamicRealm all objects are DynamicRealmObjects
DynamicRealmObject person = realm.createObject("Person");

// All fields are accessed using strings
String name = person.getString("name");
int age = person.getInt("age");

// An underlying schema still exists, so accessing a field that does not exist will throw an exception
person.getString("I don't exist");

// Queries still work normally
RealmResults<DynamicRealmObject> persons = realm.where("Person").equalTo("name", "John").findAll();

A DynamicRealm trades type safety and performance for flexibility, so only use it in cases where you really need the flexibility.

#enumsmatter

We have updated our query API so it now uses enums instead of booleans when specifying Sort and Case options. This should provide a more readable and type-safe API.

realm.where(Person.class)
  .beginsWith("name", "John", Case.INSENSITIVE)
  .findAllSorted("name", Sort.ASCENDING)

See the full changelog for all the details.


Thanks for reading. Now go forth and build amazing apps with Realm! As always, we’re around on Stack Overflow, GitHub, and Twitter.

Read more

Realm named a startup to “bet your career on”

by /

We’re stoked to be included on Business Insider’s list of 50 enterprise startups to bet your career on in 2016.

Business Insider selects Realm as a startup to "bet your career on in 2016."

We hope you’ll take a bet on us and join the team! See our jobs page for our current available positions.

Thanks, Julie, for including us!

Read more

A Year of Using Realm in iComics

by /

Hi everyone!

As of today, I’ve now officially been using Realm in my current little side project app, shipping on the App Store for a whole year. To mark this milestone, I thought I’d write a quick piece on what that was like, including why I decided to go with Realm, what it was like converting an existing app to it, and what the resulting feedback from users was like.

The Epic Backstory

Since 2012, I’ve been building a DRM-free comic reader for iOS named iComics. The function of the app is very simple: take a ZIP file full of JPEG images, crunch some numbers, and present the images on-screen in the format of a comic book. The goal of the app is to enable users to read comics from online sources that don’t have their own dedicated app (such as Humble Bundle Books).

With elements like caching pre-processed file information, and saving/restoring the user’s reading progress, a proper data framework was required from the start. Initially, I toyed with the idea of using raw SQLite, but eventually chose instead to use this opportunity to try out Core Data for the first time. A lot of my friends had recommended Core Data to me over SQLite, so I was eager to see what it was like.

I won’t go into the specifics too deeply, but suffice it to say, working with Core Data was…a ‘challenge.’ The learning curve was huge, accessing data between threads was often unreliable, and if an error did occur, due to its very Apple-esque ‘black box model’, it sometimes took hours of trial and error to work out what was actually going wrong.

The straw that broke the camel’s back was in October 2014, when I had put the finishing touches on the update that would introduce iOS 7’s new design language. When I uploaded the build to TestFlight, literally every user reported that the app crashed on launch and woudn’t run. After doing some digging, I discovered that installing the update over the previous version was triggering a Core Data migration, which would reliably fail, and corrupt the database! What an absolute showstopper! I hadn’t noticed this at all during development since I was working off a fresh install, so no migrations were occurring.

So, it was at that point where I said a bunch of naughty words and decided to choose a new data storage framework. I was considering biting the bullet and moving to SQLite, when one of my colleagues pointed me at this ‘Realm’ thing and suggested I give it a try. I remember thinking to myself, ‘What the hell. It’s worth a shot. There’s absolutely no way it could make things worse than Core Data did.’

Literally three hours later, there was no longer any trace of Core Data in my codebase; just Realm.

And it just worked.

Straight away. I remember sitting on my couch in utter shock and disbelief for a few minutes simply repeating, “That was it? Really? No way. I’m missing something. It can’t have been that easy. Impossible. Something’s going to explode in my face down the line.”

But no! One TestFlight build release later, and it was clear from the unanimous response from the testers that Realm had instantly made all of Core Data’s problems go away. It felt like magic.

Converting From Core Data to Realm

To be completely honest, I’d actually heard of Realm a few months before adopting it (possibly right after it was announced!). My colleague had showed me the news article talking about it, and I went as far as to download the sample apps and had a play with it. My impression was, ‘Yeah, that’s definitely cool. But surely it would be a lot of effort to switch over from Core Data.’

With many, MANY custom UIView subclasses, and a large number of third-party libraries, the iComics codebase is around 160,000 lines of code (I’m sure that’s not something to be proud of!). As such, I was very wary that doing anything so fundamental as completely replacing the data layer would take a VERY long time to implement and, given the architecture of the code, maybe even impossible. It was only after Core Data catastrophically failed, to the point where the app was rendered un-shippable, that I was motivated to give Realm a try.

Like I said earlier, converting an app of a relatively non-trivial size from Core Data to Realm took mere hours. While Realm doesn’t technically follow the proper conventions of object-relational mapping (ORM) technologies, its API design is sufficiently similar in that it’s very easy to map out its logic in a way that matches (but doesn’t emulate) Core Data. It was simply a matter of converting my NSManagedObject subclasses to RLMObject subclasses, converting any NSNumber properties to straight-up NSInteger values, and then changing any portions of the code related to querying/saving to the database to the equivalent Realm statements.

What amazed me the most about Realm, as compared to Core Data, was the sheer lack of code and auxiliary files that were required to achieve the same thing. No .xcdatamodeld files; the schema is derived from the model classes themselves with no boilerplate initialization code; simply two or three lines of code to configure the default Realm file’s settings.

As a sidenote, deleting all of that boilerplate Core Data code felt good. Really really good.

One Year Later

A whole year later, and Realm really has exceeded my expectations. The number of thread-related data crashes/corruptions has dropped to zero (as far as my analytics have told me!), and many users have written that the app somehow ‘feels’ snappier than the previous Core Data version.

Additionally, as Realm has matured and more features have been added in these past months, I’ve been able to refine its integration in the app. Since originally adding Realm to iComics, additional features have been added:

  • Being able to configure the default Realm file (such as its location on disk)
  • Accessing Realm files from different processes
  • Key-value observing on model properties
  • Most recently, null property support

Each and every one of these features have meant I could go back and remove much of the original custom code I’d written to ‘simulate’ these features in my app and now rely directly on that functionality in Realm. This has directly lowered my own technical debt as well as made the app more architecturally sound in the process.

Suffice it to say, there has never been a moment in the past 365 days when I regretted ditching Core Data for Realm.

Where to From Here?

While Realm is working incredibly well for me, I’m acutely aware that my app was originally architected for Core Data, and that Realm was (somewhat awkwardly) ‘shoe‑horned’ into Core Data’s place.

Core Data and Realm work in fundamentally different ways. Core Data tries to be clever about batch-loading and then keeping objects in memory, whereas Realm eschews that in favour of lazy-loading data on cue, but in an ultimately faster way.

As a result of that, a lot of my app has been designed to potentially try and cope with concepts like paging groups of objects, and offloading tonnes of work to background threads when it really isn’t necessary to do so with Realm. As a result, I’m planning to rip out the data-heavy components of my app and rewrite them from scratch; this time with a Realm-centric mindset as the goal. Hopefully this would result in less code required to maintain, and even faster, more efficient operation of the app.

Conclusion

So…yeah. Realm is good. Very good. I’m sold.

When I was looking to jump ship from Core Data, my other options were either raw SQLite or JSON. In hindsight, going with either of those would have been an absolute disaster. SQLite would have required a hilarious amount of additional bridging code (e.g., converting SQL queries into objects) and JSON would have been incredibly inefficient.

Suffice it to say, I definitely feel like I made the right choice with Realm, and I’m looking forward to many more years of integrating it into my apps.

Read more