React Native Radio Ep. 24: Realm React Native
On the latest episode of Devchat.tv’s React Native Radio, hosts Peter Piekarczyk and Lee Johnson sat down with Scott Kyle and Tim Anglade to talk about the recently released Realm React Native. They discuss everything from the design paradigms behind the database to the company’s business model, and get in-depth about encryption, object types, async, and more. Have you been wondering why Realm decided to directly support React Native rather than asking devs to bridge to iOS or Android? Tim and Scott reveal the thinking and product design going on behind the scenes, as well as what goes on in our underlying storage engine to make Realm so fast and easy to use. Be sure to check out the episode on Devchat.tv and subscribe on iTunes.
Play the episode below
Peter Piekarczyk: Hello and welcome to episode 24 of React Native Radio. I’m your host, Peter Piekarczyk, and we also have Lee Johnson, Scott Kyle, and Tim Anglade.
So, Scott and Tim, would you guys like to start by introducing yourselves and letting us know what you do?
Scott Kyle: I’m Scott, and I have been working on the Realm React Native binding for maybe the past 6 months or so. Before that, I co-founded a startup and we did software for WiFi routers, so a whole different thing. I was with Apple prior to that, so I’ve been bouncing around, doing lots of different stuff and enjoying every minute of it.
Lee Johnson: All right so you piqued my interest. I didn’t realize you guys were around that long, with the dark ages comment. I’d love to hear the story of how that all transpired.
Tim Anglade: We used to be called TightDB, which was a bad name on a number of levels. We were doing something very similar. The technology hasn’t changed all too much but we were doing more of an embedded database, like a pound for pound type of replacement for SQLite. As I said, the technology hasn’t changed all that much but we’re much more focused on disk space and raw speed and a bunch of advanced features, something you couldn’t quite do with SQLite if you were into embedded systems and their devices or low level programming.
The company was really founded by a bunch of C++ dudes and was like a developers’ database type of ideal to it. We decided around early 2014 to focus more specifically on mobile, which was the founders’ first dream anyway. They both came out of Nokia and they both had worked for a long time on old mobile and new mobile. They started building a database for mobile way back when they founded the company in the late 2000s, early 2010s. A lot of the investors told them that nobody needed a mobile database so they had gone into different direction.
Realm’s Features & Use Cases
Lee Johnson: So it’d be nice if you could give us an overview of Realm’s features and use cases.
What’s cool about it is React Native is new and Objective-C and Xcode projects obviously have Core Data and Android has their database. So a database for React Native is a unique thing. It’s specific to that environment and we don’t have a lot of options for that now. You guys are pretty new to the scene for an actual database in React Native, and that makes it easier to use than writing bridges to Core Data or something crazy like that.
Tim Anglade: Realm is the umbrella term – it’s the name of the company, it’s also the name of the product that we have for each platform. In its raw form Realm is this easy to use, really fast replacement for your data model layer, including persistence, relationships, querying, and everywhere you would use a model in your app, you use a Realm object.
The basic principle of it is to build a modern data model if we were trying to rebuild SQLite today. It’s easy to use, which is what we think most developers need at this point – they really need something that doesn’t take a lot of time. When you’re developing your application and trying to maintain it and find a bug, we’re trying to do something really, really fast because you don’t want to be slowed down by an ORM or database queries and having to optimize that.
Realm for React Native is our reinvention of the product specifically for the platform. We don’t really do bindings in the strict sense of “Let’s just take this thing we built and expose the API directly into whatever language we have to deal with.” We really try to start every new product thinking “What is that product trying to do? What is the platform trying to do? What is the framework behind it trying to do? What are its design principles and how can we fit in there in the best possible way?” We spent an enormous amount of time launching every product specifically for that community. You should look at our Swift product, it’s drastically different from our Android product in terms of the API it exposes, the query language it adds, a lot of things like that. If you look at our Swift products versus our Objective-C products, they’re also very different even though they both target the iOS platform.
Peter Piekarczyk: Do you have a particular client or customer or app type that you think would most benefit from Realm? Or is it anything?
Tim Anglade: We really tried to design this for any app and for if you need a data model that would basically feel like you should be able to use Realm. Specifically on a relatively new ecosystem like the React Native ecosystem that doesn’t have a very strong integration with SQLite, like Android does, or doesn’t have a very strong framework like Core Data on iOS, we feel like there are very few reasons why you shouldn’t consider Realm for React Native. I say this because there’s a lot of trust and safety built into using the first party option in every ecosystem, so definitely there are a lot of reasons why if you’re on iOS you might still want to stick with Core Data or if you’re doing Android that you may want to use the built in SQLite functions. For React Native, there’s not the same nice, clean, first party persistence solution that has the semantics that we do and has the performance that we do. Unless you’re really happy with key-value storage, we think that you should probably give Realm a look. And again, if you don’t think we stay true to that promise, then we really would like your feedback to see how we can do that.
One of the specific use cases that tend to trigger people using us are if you need to use data offline. If you need to do complex querying, complex relationship between your objects, that’s another reason to think about it. If you need a lot of first party support from a company and not just community answers or open source projects, that’s another. There’s a bunch of different reasons to consider Realm but in terms of what apps, we really tried to invent something that should be usable by any app, whether you need offline or not, whether you need complex queries or not, no matter what business or application you’re building.
The Business Model
Peter Piekarczyk: If you don’t mind me asking, how are you guys going to make money? Because Realm is free, which is amazing, but what are your plans for the future? Offering an enterprise package?
Tim Anglade: Yeah. We’ve already generated revenue selling both services and also products around licenses to large company that need it. Our basic business model isn’t dissimilar to say, GitHub, where 90% plus of the product ends up being given away for free but it helps us build a community, build a standard on which we can up-sell a small portion of the population, to specifically ensure a sort of product that a large corporation would need. We’re looking to be a bit more public about what those offerings are in the next few months. We’re actually still learning a lot as to what people want to do on top of Realm and what specifically big corporations want out of Realm. It’s taking us a long time to refine that understanding into a nice packaged product with nice names and nice messages around them. Doing that while also taking care of launching all these new platforms and doing all support around that has been a bit of a slow ordeal but we’re going to have more announcement on that front in the next few months.
Under the Hood
Tim Anglade: The basic thing about Realm is that we don’t use SQLite. We’re not an ORM and we’re not building stuff with SQLite. So that’s different from virtually every other persistence option out there. We’re not building on LevelDB or any other key value store either. We’re built on our own underlying storage engine, which is a part of Realm that’s not open source right now, in full disclaimer. It’s something that we’re looking to open source, but it just keeps getting backlogged with everything else.
So we’re built on our own underlying storage system right now, that’s gonna go Apache 2 in the next few months. That is not strictly speaking a relational database, it’s a mix between a relational database and an object database. That powers a lot of the high-level feature that you see.
There are some major differences, for example an ORM will take a copy of your data out of the database. That’s really bad for consistency, that’s really bad for resource usage, that’s really bad for speed. Whereas Realm has those two separate layers as well, like a SQLite layer, and like an ORM layer, but Realm’s layers are actually integrated to work together. We don’t copy data in and out, we give you complete consistency, everything is memory mapped all the way down. We get huge performance savings, huge ease of use improvements, and huge consistency improvements by having those two layers really tightly integrated together as opposed to the SQLite and ORM combo.
Scott Kyle: A huge portion of the Realm language bindings are shared between all the platforms within C++. Both that core storage layer that Tim was talking about and then another layer on top of that provides many of the facilities that all the language bindings need. Then we just specialize on top of that for each language. So with Swift, Objective-C, Java, they all require their own custom integration points, and we have this intermediary layer where we can make that really easy. We’re constantly improving that as well, and we do that obviously for Realm React Native as well.
Also like what Tim said, the objects that you pull out of the database, they’re not copied out. You’re just always looking at accessors to the data that’s in the database. So we can do lots of different kinds of operations very quickly that are sometimes very slow with ORMs because they have to make many queries through the database. Whereas we don’t really need to make many queries. We can do it all in one shot when the user needs it.
Tim Anglade: Yeah and there’s a specific example to give people a concrete example. If you come from a SQLite or ORM background, you have this idea of if you do a query, and that’s just a representation or projection of your data. If you want to edit the object you have to do a separate thing, and then find the object and edit it. With Realm, you don’t have to worry about that. Any time you get a query, all you get is the actual object and you can just modify them in line, for example. Again, any time you access any value in Realm, you’re not accessing some sort of derivative, or projection, or copy of it. You’re accessing the raw data itself, with a minor asterisk for consistency and refresh loops and stuff like that. This is a bit of a simplified view but you have access to everything.
Tim Anglade: Another quick thing to drop in there as an example of the interesting aspect of building our own core and integration thereof, is that we get to really apply a lot of reactive patterns directly into the design. Not in a very functional sense, we’re not doing streams and that stuff, but we have a really nice ability to just let you work very asynchronously, get notified of changes that are happening, and we’re doing more and more work there.
This is especially raw on the React Native side, where we still have to build in async inquiries to start with. The basic promise of the underlying engine, that you can see in the other products that we’re developing for Swift and Objective-C, and that’s on its way to React Native, is the ability to follow in a lot of really nice reactive patterns. Where you just subscribe to updates and are able to modify your UI when the underlying data changes very easily. Many people have tried and it’s very hard to build on top of SQLite or key value store, so it’s one of the nice things of being able to start fresh and build on your own foundation.
Auto-Updating Results & Listviews
Scott Kyle: Like we were talking about when you make a query into a Realm, you’re getting a view on that query that you made. So when you try to access objects inside those results, that’s when the actual query is made, and if something in the Realm changes that would affect that query. Then that query would get automatically updated so when you try to access objects out of those results again, it would give you a live representation of what the state of the Realm is at that moment.
We have change notifications but unfortunately right now they’re not super granular. So if you make a change to your Realm you’ll get a notification, but soon you’ll be able to get a change directly on a query that you made. For instance, if you have a listview – We have our own custom listview that would tie in with this very nicely. You can get a notification that something in that results changed so that you can re-render the listview. Right now we would re-render it on any change for now. It’s made to do that very efficiently because nothing has changed.
Our custom listview is not actually necessary to use to integrate Realm with a listview, it just exists to make it extra fast. For our own test purposes, we have a little app with a listview with about one million items in it, and you can scroll through it and delete them, and see them be deleted immediately. Using the stock listview, you could still use Realm with it but it would be a bit slower because of certain assumptions that it makes. So that’s why we built that, to make it extra easy and extra fast to use Realm queries, or Realm lists, inside a listview in your app.
Tim Anglade: The basic promise of what we’re building, that’s coming to React Native really quickly, is, let’s say you have a part of a UI that depends on a counter (the number of items that match a query and just one dynamic tooltip, or some sort of counter badge). You can subscribe to whether or not the results of this query have changed in any way, and then in the handler for that notification, you just recompute and redraw your UI. You don’t need to rerun the query. You don’t need to do anything because the count itself will have data automatically.
Scott Kyle: This ties back in with what Tim was talking about earlier, how we have certain advantages. We’re not just building on top of SQLite, and one of the advantages of that is the deep integration from the core storage layer. When we do integrate fine-grained notifications, we won’t just get a message that something changed, we’ll be able to get a message as to exactly what changed. When updating UI components, knowing exactly what changed means you have to do a lot less recomputing diff’s and trying to figure out for yourself what changed. Again, it would just let you build an app that’s a little bit more performant.
Tim Anglade: It’s important to understand the general design patterns we have at the end of this. We fully bind to reactive patterns. I remember talking about that with Alexander, one of our two founders, and how he was pushing that when I first met him in 2013. Back then not very many people were super eager to talk about that, but it’s been driving the design of what we’ve been doing, at TightDB before even Realm, and definitely today at Realm. Whenever you’re wondering about what database we’re building, what framework we’re building, you can expect us to follow reactive patterns in everything we do.
Peter Piekarczyk: How do you handle async states, loading indicators, and anything similar to that?
Scott Kyle: Right now when you query a Realm in Realm React Native, we only support synchronous queries right now, but async is coming pretty soon. Generally speaking, one of the reasons why async hasn’t been a priority for us is that we don’t generally see a performance impact unless you’re around 100,000+ items, and also asking them to be sorted in a certain way. For your regular use, standard to-do lists for normal people who don’t have 100,000 to-do’s, you don’t see much of a performance impact at all by doing live synchronous queries. That’s why we haven’t done that yet, but we are working on it, so for large databases you will be able to make asynchronous queries.
Tim Anglade: We haven’t agreed on the design specifically for that yet, but in other products that we have for Java and Swift, we let you determine what state your query is in and whether you’re still computing results, or whether you have finally computed the results, which makes it easy to update your UI accordingly.
As Scott said, in practice this is not a performance bottleneck, which I know flies in the face of a lot of “best practices” for app development, where people tell you to do all the I/O asynchronously. A lot of those best practices are designed with stuff like SQLite and ORMs in mind, and the performance improvement you get out of Realm is actually very major to the point where it’s really much closer to accessing objects in memory than it is to performing old school I/O like you’re used to with SQLite. We understand some queries still will need to be done asynchronously, but we recommend people to just enjoy the fact that they don’t have to worry about the state of their query and whether it has finished completing for most cases, and can just do a synchronous query.
Lee Johnson: One thing I definitely want to discuss, because it’s one of the features that you guys have that really excited me, is the encryption. A lot of times, especially with SQLite for example because that’s what most people are familiar with, it’s a pain to get the SQLite cipher and get it set up. You have data on a device and devices are getting hacked all the time, and it’s in the news, now, with Apple fighting with the government over encryption.
We deal with data, but we don’t put any student data on mobile devices for that very reason. It’s just not worth the risk. But for health care applications for example or anything sensitive, can you walk through the use case and your encryption?
Tim Anglade: Encryption is one of the things that we actually had at launch for Android because we knew it was a big limiting factor on that platform. iOS already has a lot of built in encryption of files and all that stuff, so it was pretty good. For Android it was really nice to be able to just do it first party. So now we have encryption rolled out across platforms on both IOS and Android and accessible from React Native whether you’re targeting iOS or Android. To be clear though we don’t implement our own crypto, we piggy-back on either OpenSL or common crypto and we use standard AES 256 algorithm.
It’s really just like accessing any Realm you have in your application but instead of just opening it, you just have to pass the key to encrypt it or decrypt it. The only tricky part is where you store that key. You have to make sure you either get that key from somewhere secure, from a third party service, or you have to store it securely on the device. I believe there’s a library that lets you alias that across different OS’s. If you’re comfortable storing that key or getting that key from somewhere, Realm makes the encryption part very easy and uses best practice implementations from trusted 3rd party libraries. It’s definitely been mentioned as something that people like quite a bit. Getting the same level of encryption from other frameworks, especially on React Native, is really cumbersome otherwise.
Lee Johnson: Yeah, there are a lot of hoops to jump through to get any of that, but with Realm it’s literally just setting the key, right? You have a method to create the hash based on your key and then you pass it – is that how it works?
Tim Anglade: Yeah, it’s just a 64 byte key and as long as it’s correct we’ll let you open the file. I don’t have specific stats on usage but I’m always amazed because I hear about even non-health care applications or e-commerce applications using this feature, because it’s much easier than relying on platform features or other integrations. We’re really glad that we put that in. It was a bit of a complex implementation task as you can imagine to wire all that in, and we weren’t quite sure if it was worth the time we spent, but it seems to be very well-received.
Lee Johnson: You have different object types in your models, with lists being one of those types. Can you go through the object types that you can have in your models and your schema and how they’re set up, and some of those list types, object types, and different ways you can connect data?
Scott Kyle: When you define a schema for an object, for instance, and you have a data model of a person for example, you can have your basic primitive types as properties on that person, such as string for their name and a number for their age, or a date for their birthday. We also support relationships so you can have, for instance, partner could be a pointer to another person, for instance. Or we refer to them as references to other persons. You can have lists too. You could have a list of friends and inside that list there are references to other persons, or to other objects, like pets for instance.
Lee Johnson: You mentioned the pointer to the actual object. There are document stores like MongoDB, where things are nested and it gets really difficult to normalize, then you have relational databases like anything Postgres, where the relationships can get really complex and can be overkill for mobile applications. Is Realm some blend between the two?
Tim Anglade: It’s more of a mix between a relational database and an object database. This is the part where it gets tricky, and I’m simplifying drastically, but the short of it is underneath, your data is in tables with rows and columns. The columns have types, and everything is neatly stored in this platform-independent format. So if you write that Realm data from React Native you can open it from Swift concurrently, for example. In both languages you’d be able to read and write the same file.
That data is always automatically mapped to an object, so when you reference another object from an object it’s just going to point you to that other table that contains the row that has the columns for the object that you want. It’s a property that lets you do a very expressive thing, like a document model, but without the overhead of every document with nested sub-documents becoming too large and complex to index or fetch in and out of memory well, because we always have independent properties accessible in memory.
It allows us to do an immense amount of lazy loading. If, for example, you’re loading an object that is graphed sequentially to every other object in your database, and it goes all the way down: object linked to an object, linked to another object, etc. Just because you load that top level object, it won’t load the rest of your graph in memory. You won’t even load any of their properties until you try to access one of those properties, because everything is just referenced. It’s a nice hybrid storage system that gives you a high level expressive object model but with the performance of a traditional table-oriented database. It’s not strictly speaking relational, because we don’t use real relational algebra and joined tables. It’s a table that consists of rows and columns, so you can isolate individual properties and values very easily and you have maximum performance there.