This documentation applies to 2.x editions of Realm Object Server (ROS).

Realm Mobile Platform Architecture

The Realm Object Server synchronizes Realms between devices, provides authentication and access control services for Realms, and offers backend integrations and server-side event handling.

Install Realm Object Server

The Realm Platform runs on any modern x64 platform that can support Node.js including:

  • Linux: RedHat Enterprise Linux, CentOS, Ubuntu 16.04 or higher
  • MacOS: 10.12/Sierra or higher
  • Windows: coming soon!

For Linux and Windows distributions this also includes running on top of platforms such as Digital Ocean, Amazon Web Services (AWS) and Microsoft Azure where these distributions are supported.

It’s a prerequisite that Node.js (6 or later) is installed on your system. If you don’t have that, click below to see how to do that.

To get started, please ensure you have Node installed on your machine. Instructions are available here and we recommend using NVM.

Ubuntu

16.04 (64 bit; 32-bit is not supported)

It is recommended that you install the server as a normal user.

sudo apt-get update

sudo apt-get install build-essential libssl-dev

sudo apt-get install python

curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.33.4/install.sh | bash

// Force current session to know changes or logout and log back in
source ~/.profile

nvm install --lts

node -v
// Output
// v6.11.4
npm install -g node-gyp

macOS

curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.33.4/install.sh | bash

nvm install --lts

node -v
// Output
// v6.11.4
npm install -g node-gyp

Installation

npm install -g realm-object-server

This will install the server globally which is the easiest way to try it out since it includes a CLI. For Linux, if you are installing as root (not recommended), you may need to add --unsafe-perm to the npm install commands.

Start the server in the default configuration:

ros start

To start the server for production-use, the CLI can create a JS/Typescript project via ros init. Follow the directions in the CLI documentation.

Finding the Admin Token

The Admin Token is required for a number of advanced features including event handling and data connectors.

On Linux, you’ll find the token at the folllowing location:

/etc/realm/data/keys/admin.json

On macOS, the token is stored within your realm-object-server folder. You’ll find it at the following location:

realm-object-server/data/keys/admin.json

Realm Studio

Realm Studio is Realm’s GUI utility for managing and inspecting both local and synchronized Realms. In addition, you can connect to a running Realm Object Server to view logs and manage users.

Note: Realm Studio is only compatible with ROS 2.0 or higher.

To get started with Realm Studio download the latest release:

Running the Server

The easiest way to start the server is through the included commandline utility.

CLI

In addition to running Realm Object Server, the ROS commandline utility provides additional functionality as well as a means to configure your server:

$ ros --help
  Usage: ros [options] [command]


  Options:

    -V, --version  output the version number
    -h, --help     output usage information


  Commands:

    init <projectName>      Creates a new ROS project
    start                   Start Realm Object Server
    migrate [options]       Migrate data from Realm Object Server 1.x format to 2.0 format. Existing 1.x data is preserved.

  Help for individual commands:

    ros init --help
    ros start --help
    ros migrate --help
init

The ros init command will create a Javascript or Typescript project that includes Realm Object Server for you. This is the recommended way to start Realm Object Server for production use-cases.

Let’s say you want to create a project named MyServer. You can then run:

ros init MyServer

You’ll now have a project directory structure that looks like this:

MyServer
│   package.json
└───src
│   │   index.ts

With TypeScript 2.5.3, you’ll have a few more scripts that you can run:

  • npm build which will build all the src/**/*.ts files into equivalent dist/**/*.js files
  • npm clean which will clean all the js artifacts from dist/
  • npm start which will build the .ts files and run node dist/index.js

To start MyServer:

  1. go into the working directory by cd MyServer
  2. run npm start

To run this in a production enviroment see the “Going Into Production” documentation.

You can configure the server by editing the index.ts file which inclues comments on the available configuration options:

import { BasicServer } from 'realm-object-server'
import * as path from 'path'

const server = new BasicServer()

server.start({
        // This is the location where ROS will store its runtime data
        dataPath: path.join(__dirname, '../data')

        // The address on which to listen for connections
        // Default: 0.0.0.0
        // address?: string

        // The port on which to listen for connections
        // Default: 9080
        // port?: number

        // Override the default list of authentication providers
        // authProviders?: IAuthProvider[]

        // Autogenerate public and private keys on startup
        // autoKeyGen?: boolean

        // Specify an alternative path to the private key. Otherwise, it is expected to be under the data path.
        // privateKeyPath?: string 

        // Specify an alternative path to the public key. Otherwise, it is expected to be under the data path.
        // publicKeyPath?: string

        // The desired logging threshold. Can be one of: all, trace, debug, detail, info, warn, error, fatal, off)
        // Default: info
        // logLevel?: string

        // Enable the HTTPS Server.
        // https?: boolean

        // The port on which to listen for HTTPS connections.
        // Default: 0.0.0.0
        // httpsAddress?: string

        // The address on which to listen for HTTPS connections.
        // Default: 9443
        // httpsPort?: number

        // The path to your HTTPS private key in PEM format. Required if HTTPS is enabled.
        // httpsKeyPath?: string

        // The path to your HTTPS certificate chain in PEM format. Required if HTTPS is enabled.
        // httpsCertChainPath?: string

        // Specify the length of time (in seconds) in which access tokens are valid.
        // Default: 600 (ten minutes)
        // accessTokenTtl?: number

        // Specify the length of time (in seconds) in which refresh tokens are valid.
        // Default: 3153600000 (ten years)
        // refreshTokenTtl?: number
    })
    .then(() => {
        console.log(`Your server is started `, server.address)
    })
    .catch(err => {
        console.error(`There was an error starting your file`)
    })

If you prefer to use Javascript then you can create a JS ROS template with:

ros init MyServer --template js

You’ll now have a project directory structure that looks like this:

MyServer
│   package.json
└───src
│   │   index.js

To start MyServer:

  1. go into the working directory by running cd MyServer
  2. run npm start

To run this in a production enviroment see the “Going Into Production” documentation.

You can configure the server by editing the index.js file which inclues comments on the available configuration options:

const BasicServer = require('realm-object-server').BasicServer
const path = require('path')
const server = new BasicServer()

server.start({
        // This is the location where ROS will store its runtime data
        dataPath: path.join(__dirname, '../data')

        // The address on which to listen for connections
        // Default: 0.0.0.0
        // address?: string

        // The port on which to listen for connections
        // Default: 9080
        // port?: number

        // Override the default list of authentication providers
        // authProviders?: IAuthProvider[]

        // Autogenerate public and private keys on startup
        // autoKeyGen?: boolean

        // Specify an alternative path to the private key. Otherwise, it is expected to be under the data path.
        // privateKeyPath?: string 

        // Specify an alternative path to the public key. Otherwise, it is expected to be under the data path.
        // publicKeyPath?: string

        // The desired logging threshold. Can be one of: all, trace, debug, detail, info, warn, error, fatal, off)
        // Default: info
        // logLevel?: string

        // Enable the HTTPS Server.
        // https?: boolean

        // The port on which to listen for HTTPS connections.
        // Default: 0.0.0.0
        // httpsAddress?: string

        // The address on which to listen for HTTPS connections.
        // Default: 9443
        // httpsPort?: number

        // The path to your HTTPS private key in PEM format. Required if HTTPS is enabled.
        // httpsKeyPath?: string

        // The path to your HTTPS certificate chain in PEM format. Required if HTTPS is enabled.
        // httpsCertChainPath?: string

        // Specify the length of time (in seconds) in which access tokens are valid.
        // Default: 600 (ten minutes)
        // accessTokenTtl?: number

        // Specify the length of time (in seconds) in which refresh tokens are valid.
        // Default: 3153600000 (ten years)
        // refreshTokenTtl?: number
    })
    .then(() => {
        console.log(`Your server is started `, server.address)
    })
    .catch(err => {
        console.error(`There was an error starting your file`)
    })

##### start

The start command will start the Realm Object Server and listen for connections. This mode is perfect for demoing or quickly starting the server. However, it will be tied to your terminal window and isn’t recommended for production (use ros init). By default, no arguments are needed. However, if you would like to adjust some behavior of ROS, you can provide some configuration options:

CLI Option Default Description
--data <path> ./data This is the location where ROS will store its runtime data
--address <address> 0.0.0.0 The address on which to listen for connections
--port <port> 9080 The port on which to listen for connections
--loglevel <level> info The desired logging threshold. Can be one of: all, trace, debug, detail, info, warn, error, fatal, off)
--logfile <path>   Log to the specified file instead of the console
--no-auto-keygen   Do not autogenerate public and private keys on startup
--private-key <path>   Specify an alternative path to the private key. Otherwise, it is expected to be under the data path.
--public-key <path>   Specify an alternative path to the public key. Otherwise, it is expected to be under the data path.
--refresh-token-ttl <seconds> 3153600000 (ten years) Specify the length of time (in seconds) in which refresh tokens are valid.
--access-token-ttl <seconds> 600 (ten minutes) Specify the length of time (in seconds) in which access tokens are valid.
--auth <provider1[,provider2 ...]> password Override the default list of authentication providers. Multiple auth providers may be specified by separating them with a comma.
--https   Enable the HTTPS Server.
--https-key <path>   The path to your HTTPS private key in PEM format. Required if HTTPS is enabled.
--https-cert <path>   The path to your HTTPS certificate chain in PEM format. Required if HTTPS is enabled.
--https-address <address> 0.0.0.0 The address on which to listen for HTTPS connections.
--https-port <port> 9443 The port on which to listen for HTTPS connections.

Realm Studio

Realm Studio is Realm’s GUI console utility for managing and inspecting both local and synchronized Realms. In addition, you can connect to a running Realm Object Server to view logs and manage users.

To get started with Realm Studio download the latest release:

Authentication

Custom Authentication

Realm Object Server supports authenticating users to establish a user identity. By default, it includes a pre-built authentication provider for username and password. In addition, several third-party providers are included demonstrating how the server can be customized for any authentication scheme.

Adding Your Own Authentication

Integrating your own authentication into Realm Object Server is very easy. To do so, simply extend the AuthProvider class. You must override the name property and the method authenticateOrCreateUser() which passes in the JSON body submitted by the mobile client SDK. This JSON can include any relevant information needed to authenticate the user. The example below shows the simplest possible authentication provider:

const RealmObjectServer = require('realm-object-server');
const path = require('path');

const server = new RealmObjectServer.BasicServer();

class MyAuthProvider extends RealmObjectServer.auth.AuthProvider {
    constructor() {
        super();
        this.name = 'myAuthProvider';
    }

    authenticateOrCreateUser(body) {
        const userId = body.userId;
        return this.service.createOrUpdateUser(
            userId,
            "myAuthProvider",
            false,
            null
        );
    }
}

server.start({
    dataPath: path.join(__dirname, '../data'),
    authProviders: [ new MyAuthProvider() ],
}).catch((err) => {
    console.error("There was an error starting your custom ROS Server", err);
});

In the authenticateOrCreateUser() method, we get the requested userId sent from the mobile client and then we return a User object which consists of:

{
    "providerId": "<Requested UserId>",
    "provider": "myAuthProvider", // The name for your provider
    "isAdmin": true/false, // Whether this user should be an admin in ROS
    
    // You can choose to set the `userID` in ROS, only if the ID is guaranteed to be unique!
    //
    // If you cannot guarantee this, just leave it `null` and ROS will create a unique ID
    //
    // All pre-built providers use `null`
    "userId": "<Requested UserId>" or null
}

Username/Password

Realm Object Server provides a built-in username and password authentication provider. This is great to quickly get started with but isn’t designed to be a full-featured authentication system (such as providing email capabilities).

When you start Realm Object Server via:

ros start

The username/password provider is included and a default admin is created with the following credentials:

username = 'realm-admin'
password = '' // Empty string

To customize the username/password provider, create a ROS project via ros init:

const RealmObjectServer = require('realm-object-server');
const path = require('path');

const server = new RealmObjectServer.BasicServer();

// Customize username/password provider
let usernamePasswordProvider = new RealmObjectServer.auth.PasswordAuthProvider({
    autoCreateAdminUser: true,
    saltLength: 32,
    iterations: 10000,
    keyLength: 512,
    digest: "sha512"
});

server.start({
  dataPath: path.join(__dirname, '../data'),
  authProviders: [ usernamePasswordProvider ],
}).catch((err) => {
  console.error("There was an error starting your custom ROS Server", err);
});
Resetting a User's Password

A user's password can be reset by the user in question or by an admin. To do this, you'll need the hex-based User ID, the user/admin refresh token, and the desired password.

Here's an example for resetting the password via an HTTP request. The token is supplied through an Authorization header.

curl -X PUT -H "Content-Type: application/json" -H "Authorization:<user-or-admin-refresh-token>" -d '{"user_id": <user-id-to-change-password>, "data": {"new_password": <new-password>}}' http://localhost:9080/auth/password

The JSON payload is:

{
  "user_id": "5d0fbc03f4d7ab1dcc127aace224270c", // optional if user is changing his/her own password
  "data": {
      "new_password": "new-password"
  }
}

On a successful reset, you will receive a response code of 200.

The refresh token can be retrieved by calling user.token after logging in via Realm.Sync.User

Azure Authentication

The Realm Object Server package includes a pre-built provider for Azure Active Directory. Clients authenticate through Azure SDK/API and sends the access token to ROS. This provider handles validating the JWT access token and then authenticating in Realm.

To include or customize the Azure provider, create a ROS project via ros init:

const RealmObjectServer = require('realm-object-server');
const path = require('path');

const server = new RealmObjectServer.BasicServer();

// Add your Directory ID from the Azure Portal (under "Properties")
let azureProvider = new AzureAuthProvider(
  {
    tenant_id: '81560d038272f7ffae5724220b9e9ea75d6e3f18'
  }
)

server.start({
    dataPath: path.join(__dirname, '../data'),
    authProviders: [ azureProvider ],
}).catch((err) => {
    console.error("There was an error starting your custom ROS Server", err);
});

CloudKit Authentication

You will need to create a public key for the Realm Object Server to access CloudKit. The steps are slightly different for Linux and macOS. Note that iCloud client support is only available on Apple platforms: iOS, macOS, tvOS, and watchOS.

  1. Open a terminal and cd to the Realm Object Server directory.

  2. Generate a private key:

    openssl ecparam -name prime256v1 -genkey -noout -out cloudkit_eckey.pem

  3. Generate a public key to be submitted to the CloudKit Dashboard:

    openssl ec -in cloudkit_eckey.pem -pubout

  1. Generate a private key:

    openssl ecparam -name prime256v1 -genkey -noout -out cloudkit_eckey.pem

  2. Generate a public key to be submitted to the CloudKit Dashboard:

    openssl ec -in cloudkit_eckey.pem -pubout

Generating an access key with the CloudKit Dashboard

Log in to Apple’s CloudKit Dashboard and select your application. In the left-hand side of the dashboard, select “API Access”, then select “Server-to-Server Keys”. Select “Add Server-to-Server Key”. Give the new key a name and paste in the public key generated above. Click “Save.” After a few seconds, a key will be generated and displayed in the “Key ID” section at the top of the page.

Security note: Create a new private key for each application you plan on using with Realm CloudKit authentication. Reusing private keys can compromise all Realms using the shared key if the private key itself becomes compromised or needs to be revoked.

To include or customize the CloudKit provider, create a ROS project via ros init:

const RealmObjectServer = require('realm-object-server');
const path = require('path');

const server = new RealmObjectServer.BasicServer();

// Add the Key ID, private key path, container ID and environment information
const cloudKitProvider = new RealmObjectServer.auth.CloudkitAuthProvider({
    container: 'iCloud.io.realm.exampleApp.ios',
    // For production deployment on the App Store, you must specify 'production'
    environment: 'development',
    keyId: '0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef',
    privateKeyPath:  'cloudkit_eckey.pem'
});

server.start({
    dataPath: path.join(__dirname, '../data'),
    authProviders: [ cloudKitProvider ],
}).catch((err) => {
    console.error("There was an error starting your custom ROS Server", err);
});

Please ensure the path to the privateKeyPath is correct. If you followed the steps above, on Linux the path will be cloudkit_eckey.pem.

Facebook Authentication

The Realm Object Server package includes a pre-built provider for Facebook. Clients authenticate through Facebook SDK/API and sends the access token to ROS. This provider handles validating the access token with Facebook’s API, and then authenticating in Realm.

To include the Facebook provider, create a ROS project via ros init:

const RealmObjectServer = require('realm-object-server');
const path = require('path');

const server = new RealmObjectServer.BasicServer();
const facebookProvider = new FacebookAuthProvider()

server.start({
    dataPath: path.join(__dirname, '../data'),
    authProviders: [ facebookProvider ],
}).catch((err) => {
    console.error("There was an error starting your custom ROS Server", err);
});

Google Authentication

The Realm Object Server package includes a pre-built provider for Google. Clients authenticate through Google SDK/API and sends the access token to ROS. This provider handles validating the access token with Google’s API, and then authenticating in Realm. To setup, you must obtain a client Id described in this guide: https://developers.google.com/identity/protocols/OAuth2InstalledApp

To include the Google provider, create a ROS project via ros init:

const RealmObjectServer = require('realm-object-server');
const path = require('path');

const server = new RealmObjectServer.BasicServer();
const googleProvider = new GoogleAuthProvider({
    clientId: '012345678901-abcdefghijklmnopqrstvuvwxyz01234.apps.googleusercontent.com'
})

server.start({
    dataPath: path.join(__dirname, '../data'),
    authProviders: [ googleProvider ],
}).catch((err) => {
    console.error("There was an error starting your custom ROS Server", err);
});

Extending Functionality

Opening up a Synced Realm

If you ever need to open up a realm in your index.ts file, you can easily request the server instance with a call to server.openRealm

server.openRealm('/products')
  .then(realm => {
    console.log('My products realm!', realm)
  })

A schema is not required to open up a realm. However if you’d like to get a realm with specific schemas. Then you can run:

const ProductSchema: Realm.ObjectSchema = {
  name: 'Product',
  primaryKey: 'productId',
  properties: {
    productId: 'int',
    name: 'string',
    price: 'float'
  }
}

const CompanySchema: Realm.ObjectSchema = {
  name: 'Company',
  primaryKey: 'companyId',
  properties: {
    companyId: 'int',
    name: 'string',
    address: 'string'
  }
}

server.openRealm('/products', [ProductSchema, CompanySchema])
  .then(realm => {
    console.log('My products realm!', realm)
  })

The realm is always opened up with admin privileges.

Adding a Custom Service

ROS 2.x is also a fully featured web framework! Adding functionality is simple, modular, and service oriented. A ROS custom service is just a class that you can add to a Server instance.

  • You can expose methods as http endpoints using @Get, @Post, @Put, @Delete etc…
  • You can read parameters using the @Param('carId') or just @Param decorator
  • Services don’t need http endpoints at all

Writing a Service in TypeScript

When writing a custom service in TypeScript, we highly recommend that you ensure that your tsconfig.json’s compiler options has

"emitDecoratorMetadata": true,
"experimentalDecorators": true

It should look relatively similar to:

{
  "compilerOptions": {
    "target": "es6",
    "module": "commonjs",
    "moduleResolution": "node",
    "emitDecoratorMetadata": true,
    "experimentalDecorators": true
    ... other options
  }
}

If you are using ros init. You should have no problem with the generated tsconfig.json file in your project.

Desiging your Service with GET, PUT, POST, DELETE

Remember, a service is just an object. You can easily write a CarService like so:

class CarService { 

}

You can then add the service to the server:

server.addService(new CarService())

However this service doesn’t really do anything. Let’s give it some REST behavior! First import some of the HTTP decorators from the realm-object-server package:

import { 
  Get, 
  Post, 
  Put, 
  Delete, 
  BaseRoute
} from 'realm-object-server'

Now you can decorate your CarService class with an expressive set of functions. In order for the service to have HTTP capabilities the service must have a @BaseRoute decorator. In this example we will have a CarService that returns JSON objects with an HTTP GET request

@BaseRoute('/cars')
class CarService {

  cars = [{
      carId: 1,
      name: 'toyota',
    }, {
      carId: 2,
      name: 'ford',
    }]

  @Get('/')
  getAllCars(){
    return this.cars
  }
}

You can now make a GET request to return your the cars JSON at http://localhost:9080/cars. Services can return normal objects (which will be serialized into JSON) or a Promise<Object>, which will be serialized after the promise is resolved.

Thus the getAllCars() method could also look like this:

@Get('/')
  getAllCars(){
    return new Promise((resolve, reject) => {
      setTimeout(() => {
        resolve(this.cars)
      }, 2000)
    })
  }

You can even add URL parameters (using @Params('param') and query string (@Query('querykey')) decorators to methods. Parameters can be defined with a /:someparam. Multiple params can be chained with /:someparamone/:someparamtwo

@Get('/:carId')
  getCarById(@Params('carId') carId: string){
    return this.cars.find((c) => c.carId === carId))
  }

An example with multiple params:

@Get('/:carId/:name')
  getCarById(@Params('carId') carId: string, @Params('name') name: string){
    return this.cars.find((c) => c.carId === carId && c.name === name))
  }

An example with parameters from the query string:

@Get('/:carId')
  getCarById(@Params('carId') carId: string, @Query('limit') limit: string){
    let limitNumber: number = parseInt(limit)
    return this.cars.filter((c) => c.carId === carId && c.name === name))
      .limit(limitNumber)
  }

All returned values will have an HTTP status code of 200 and will be serialized with a JSON body. If you need fine grained control over the message that you are returning you can pass in the raw express request and response with @Request and @Response:

@Get('/:carId')
  getCarById(@Request req, @Response res){
    res.status(203).send({
      carId: 'something',
      name: 'somethingelse'
    })
  }
import { 
  Get, 
  Post, 
  Put, 
  Delete, 
  BaseRoute, 
  Body, 
  Request, 
  Response 
} from 'realm-object-server'

@BaseRoute('/cars')
class CarService {

  cars = [{
      carId: 1,
      name: 'toyota',
    }, {
      carId: 2,
      name: 'ford',
    }]

  @Get('/all')
  getAllCars(){
    return this.cars
  }

  @Post('/')
  async addNewCar(@Body body: any) {
    this.cars.push(body.car)
    return body
  }

  @Put('/:carId')
  async updateCar(@Body body: any) {
    this.cars.push(body.car)
    return body
  }

  @Delete('/:carId')
  async deleteCar(@Param carId: string) {
    for(let car of this.cars) {
      if (car.carId === carId) {
        this.cars.splice(this.cars.indexOf(car), 1)
      }
    }
  }
}

Service Lifecycle

When designing services, it’s important to know the lifecycle of the service. Services have 3 events

  1. an optional async start method that can be decorated by @Start
    • If you have isolated asynchronous setup code use this method.
    • You shouldn’t be calling other services.
    • The server object is passed as the only argument, where you can grab the logger, discovery, or any other global configuration needed by the service.
    • Prefer not to block the execution of this method.
  2. an optional async serverStarted method that can be decorated by @ServerStarted
    • At this stage, the server has successfully started all services that were added.
  3. an optional async stop method that can be decorated by @Stop
    • This is called when the shutdown command is called on the server.
    • Use this method to quickly dispose or close any connections.

Note, all these decorators are completely optional.

class SampleService {

    @Start
    async start(server: Server) {
        // called when the server is about to startup
    }

    @ServerStarted
    async serverStarted(server: Server) {
        // called when all services have started and the underlying httpServer is started
    }

    @Stop
    async stop(){
        // do your service cleanup here
        // called when the server is attempting to shutdown
    }
}

Custom Middlewares

Realm Object Server 2.0 is built on top of express-js. Thus, we enable the ability for you to add middlewares to both the server and the service instances

import * as cors from 'cors'
server.useMiddleware(cors())

IP Filtering Middleware

There are many scenarios where you’d like to limit IPs. The common cases are:

  1. I’d like to limit specific IP addresses from accessing ROS entirely
  2. I’d like to limit specific IP address from accessing a service entirely

Since ROS allows you to use express middlewares, you are fully capable of using express-ipfilter or ip-regex on your entire server instance, an entire service instance, and/or just a service route.

The following examples will use express-ip-filter. You can read more about its custom settings here.

Limiting the IP Address for your entire Server
import { IpFilter } from 'express-ipfilter'
import { Server } from 'realm-object-server'

const server = new Server()

const ips = ['127.0.0.1'];
server.useMiddleware(ipfilter(ips, {mode: 'allow'}));
server.start()
    .then(() => {
        console.log('Only localhost (aka 127.0.0.1) can connect!')
    })
Limiting the IP Address for just a service

There are instances where you’d only like to limit IPs for a certain service. You can use the MiddlewaresBefore on a single route. In the following snippet we have limited whitelisting of ips for the entire CarsService.

import { IpFilter } from 'express-ipfilter'
import { MiddlewaresBefore, Server, BaseRoute, Get } from 'realm-object-server'

const ips = ['127.0.0.1'];
const middleware = ipfilter(ips, {mode: 'allow'})

@BaseRoute('/cars')
@MiddlewaresBefore(middleware)
class CarsService {

    @Get('/teslas')
    getTeslas(){
        // only whitelisted ips will enter in here
        return someJson
    }
}

const server = new Server()
server.start()
    .then(() => {
        console.log('Only localhost (aka 127.0.0.1) can get cars')
    })
Limiting the IP Address for just a service route

If you just want to limit the IP address of a single service route but not the entire service, you can use the MiddlewaresBefore on a single route. In the following snippet we have limited whitelisting of ips only for the getTeslas method on CarsService.

import { IpFilter } from 'express-ipfilter'
import { MiddlewaresBefore, Server, BaseRoute, Get } from 'realm-object-server'

const ips = ['127.0.0.1'];
const middleware = ipfilter(ips, {mode: 'allow'})

@BaseRoute('/cars')
class CarsService {
    @Get('/teslas')
    @MiddlewaresBefore(middleware)
    getTeslas(){
        // only whitelisted ips will enter in here
        return someJson
    }
}

const server = new Server()
server.start()
    .then(() => {
        console.log('Only localhost (aka 127.0.0.1) can get teslas')
    })

Just like with any middleware, they will be processed in the following order:

  1. server.userMiddleware
  2. @MiddlewaresBefore on a Service class
  3. @MiddlewaresBefore on a Service class method

Serving Static Files

You can serve static files from a service by using a @BaseRoute with a @ServeStatic decorator. For example, to serve a directory or path, you could use the following snippet:

import { BaseRoute, ServeStatic } from 'realm-object-server'
import * as path from 'path'
@BaseRoute('some')
@ServeStatic(path.join(__dirname, 'assets/index.html'))
class SomeClass {

}

You can now reach the index.html file at: http://localhost:9080/some

Going Into Production

We recommend using pm2 to maintain instances of Realm Object Server in production. PM2 is a battle tested process manager that allows your applications to stay alive forever and helps with mission critical administration during downtime.

You can install pm2 in your machine using npm with:

npm install pm2 -g

You can then start ROS with:

pm2 start path/to/myserver/dist/index.js

Now your ROS instance is daemonized, monitored and kept alive forever.

You can supply a name to your started instance with the --name option:

pm2 start path/to/myserver/dist/index.js --name my-ros

Stopping the ROS instance:

pm2 stop my-ros

Restarting the ROS instance:

pm2 restart my-ros

When you are no longer interested in registering a daemonized and monitored instance you can use delete:

pm2 delete my-ros

Monitoring and Log Management

You can use pm2 to monitor the usage of ros instances by:

pm2 monit

This will print a GUI that will display both logs, CPU usage, and Memory usage.

Backend Integration

Data Access

The Professional and Enterprise Editions of the Realm Object Server allow you to access and change any shared Realm server-side using the administration token generated when you start the server. This allows access to all data in that Realm.

To retrieve the admin token:

const adminTokenUser = require(path.resolve(server.config.dataPath, 'keys/admin.json')).ADMIN_TOKEN

This can be used to synchronously construct aRealm.Sync.Userobject which can be passed into theRealmconstructor to open a connection to any Realm on the server side.

// Open a Realm using the admin user
const adminTokenUser = require(path.resolve(server.config.dataPath, 'keys/admin.json')).ADMIN_TOKEN
var adminUser = Realm.Sync.User.adminUser(adminTokenUser, SERVER_URL);
var realm = new Realm({
  sync: {
    user: adminUser,
    url: 'realm://object-server-url:9080/my-realm',
  },
  schema: [{...}
  }]
});

Note that when used with an admin user, the Realm URL does _not _include the ~character that resolves into the user ID for non-admin authentication; the admin user does not have a user ID. For non-admin authentication, read the Authentication section in the Object Server Access Control documentation.

Event Handling

An exclusive feature of the Enterprise and Professional Editions of Realm Object Server is event handling capabilities. This functionality is provided in the server-side Node.js and .NET SDKs via a global event listener API which hooks into the Realm Object Server, allowing you to observe changes across Realms. This could mean listening to every Realm for changes, or Realms that match a specific pattern. For example, if your app architecture separated user settings data into a Realm unique for each user where the virtual path was /~/settings, then a listener could be setup to react to changes to any user’s settings Realm.

Whenever a change is synchronized to the server, it triggers a notification which allows you to run custom server-side logic in response to the change. The notification will inform you about the virtual path of the updated Realm and provide the Realm object and fine-grained information on which objects changed. The change set provides the object indexes broken down by class name for any inserted, deleted, or modified object in the last synchronized transaction.

Creating an Event Handler

To use Realm Event Handling, you’ll need to create a small Node.js application.

Create a directory to place the server files, then create a file named package.json. This JSON file is used by Node.js and npm, its package manager, to describe an application and specify external dependencies.

You can create this file interactively by using npm init. You can also fill in a simple skeleton file yourself using your text editor:

{
    "name": "MyApp",
    "version": "0.0.1",
    "main": "index.js",
    "author": "Your Name",
    "description": "My Cool Realm App",
    "dependencies": {
        "realm": "^1.8.0"
    }
}

We specify the Realm-JS as version 1.8 or higher. If you have other dependencies for your application, go ahead and specify them in the dependencies section of this file.

After the package.json file is configured properly, type:

npm install

to download, unpack and configure all the modules and their dependencies.

Your event handler will need to access the Object Server with administrative privileges, so you’ll need to get the Object Server’s admin token.

const adminTokenUser = require(path.resolve(server.config.dataPath, 'keys/admin.json')).ADMIN_TOKEN

A sample index.js file might look something like this. This example listens for changes to a user-specific private Realm at the virtual path /~/private. It will look for updated Coupon objects in these Realms, verify their coupon code if it wasn’t verified yet, and write the result of the verification into the isValid property of the Coupon object.

var Realm = require('realm');

var FEATURE_TOKEN = "eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9...";

// Unlock Professional Edition APIs
Realm.Sync.setFeatureToken(FEATURE_TOKEN); 

// Insert the Realm admin token here
var ADMIN_TOKEN = require(path.resolve(server.config.dataPath, 'keys/admin.json')).ADMIN_TOKEN;

// the URL to the Realm Object Server
var SERVER_URL = 'realm://127.0.0.1:9080';

// The regular expression you provide restricts the observed Realm files to only the subset you
// are actually interested in. This is done in a separate step to avoid the cost
// of computing the fine-grained change set if it's not necessary.
var NOTIFIER_PATH = '^/.*/private$';

// The handleChange callback is called for every observed Realm file whenever it
// has changes. It is called with a change event which contains the path, the Realm,
// a version of the Realm from before the change, and indexes indication all objects
// which were added, deleted, or modified in this change
function handleChange(changeEvent) {
  // Extract the user ID from the virtual path, assuming that we're using
  // a filter which only subscribes us to updates of user-scoped Realms.
  var matches = changeEvent.path.match(/^\/([0-9a-f]+)\/private$/);
  var userId = matches[1];

  var realm = changeEvent.realm;
  var coupons = realm.objects('Coupon');
  var couponIndexes = changeEvent.changes.Coupon.insertions;

  for (let couponIndex of couponIndexes) {
    var coupon = coupons[couponIndex];
    if (coupon.isValid !== undefined) {
      var isValid = verifyCouponForUser(coupon, userId);
      // Attention: Writes here will trigger a subsequent notification.
      // Take care that this doesn't cause infinite changes!
      realm.write(function() {
        coupon.isValid = isValid;
      });
    }
  }
}

// create the admin user
var adminUser = Realm.Sync.User.adminUser(adminToken, SERVER_URL);

// register the event handler callback
Realm.Sync.addListener(SERVER_URL, adminUser, NOTIFIER_PATH, 'change', handleChange);

console.log('Listening for Realm changes');

The heart of the event handler is thehandleChange()function, which is passed achangeEventobject. This object has four keys:

  • path: The path of the changed Realm (used above with match to extract the user ID)
  • realm: the changed Realm itself
  • oldRealm: the changed Realm in its old state, before the changes were applied
  • changes: an object containing a hash map of the Realm’s changed objects

The changes object itself has a more complicated structure: it’s a series of key/value pairs, the keys of which are the names of objects (i.e., Coupon in the above code), and the values of which are another object with key/value pairs listing insertions, deletions, and modifications to those objects. The values of those keys are index values into the Realm. Here’s the overall structure of thechangeEventobject:

{
  path: "realms://server/user/realm",
  realm: <realm object>,
  oldRealm: <realm object>,
  changes: {
    objectType1: {
      insertions: [ a, b, c, ...],
      deletions: [ a, b, c, ...],
      modifications: [ a, b, c, ...]
    },
    objectType2: {
      insertions: [ a, b, c, ...],
      deletions: [ a, b, c, ...],
      modifications: [ a, b, c, ...]
    }
  }
}

In the example above, we get the Coupons and the indexes of the newly inserted coupons with this:

var realm = changeEvent.realm;
var coupons = realm.objects('Coupon');
var couponIndexes = changeEvent.changes.Coupon.insertions;

Then, we use for (let couponIndex of couponIndexes) to loop through the indexes and to get each changed coupon.

To use Realm Event Handling, you’ll need to create a .NET application. It can be a console app or Asp.NET app and can run on all flavours of Linux, macOS, or Windows that the .NET SDK supports.

Create a new project or open your existing one and add the Realm.Server NuGet package.

You’ll need to create a class that implements the INotificationHandler interface. It has two methods - ShouldHandle and HandleChangesAsync.

To tell the notifier that your handler is interested in observing changes for a particular path, return true in the ShouldHandle callback. As a general principle, ShouldHandle should always return stable result every time it is invoked with the same path and should return as quickly as possible to avoid blocking observing of notifications. We’ve provided an abstract implementation of the INotificationHandler interface in a Regex​Notification​Handler class that implements ShouldHandle in terms of regex matching the string that is passed to its constructor. For example, if you want to observe changes to the user-specific private Realm at virtual path /~/private, you would provide the following implementation:

class PrivateHandler : RegexNotificationHandler
{
    public PrivateHandler() : base("^/.*/private$")
    {
    }

    public override async Task HandleChangeAsync(IChangeDetails details)
    {
        // TODO: handle change notifications
    }
}

The HandleChangesAsync method is the heart of the event handler - it gets invoked whenever a Realm at an observed path (ShouldHandle returned true) changes and is passed an IChangeDetails object that contains detailed information about the change that occurred. It has the following properties and methods:

  • PreviousRealm and CurrentRealm are readonly snapshots of the state of the Realm just before and just after the change has occurred. PreviousRealm may be null if the notification was received just after creating the Realm. CurrentRealm can never be null.
  • RealmPath is the virtual path of the Realm that has changed, e.g. /some-user-id/private.
  • GetRealmForWriting() can be invoked to get a writeable instance of the Realm that has changed in case you need to write some data in response to the change notification. Since writing to any Realm automatically advances it to the latest version, this Realm instance may contain slightly newer data than CurrentRealm if new changes have been received while handling the notification. Unlike CurrentRealm and PreviousRealm, the lifetime of this instance is not managed by the notifier, so make sure to place it inside a using statement or manually dispose it as soon as you’re done with it.
  • Changes is a dictionary of class names and IChangeSetDetails objects. For each object type that has changed, you’ll get a corresponding IChangeSetDetails object containing Insertions, Deletions, and Modifications collections. Each of those contains IModification​Details instances with the following properties:
    • PreviousIndex indicates the index of the changed object in the PreviousRealm view. It will be -1 if the object was inserted.
    • PreviousObject is the state of the object just before it has changed. It will be null if the object was inserted.
    • CurrentIndex indicates the index of the changed object in the CurrentRealm view. It will be -1 if the object was deleted.
    • CurrentObject is the state of the object just after it has changed. It will be null if the object was deleted.

Here’s what a sample event handling application might look like:

public class Program
{
    private const string FeatureToken = "eyhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9...";
    private const string AdminUsername = "[email protected]";
    private const string AdminPassword = "super-secure-password";
    private const string ServerUrl = "127.0.0.1:9080";

    public static void Main(string[] args) => MainAsync().Wait();

    public static async Task MainAsync()
    {
        // Unlock Professional Edition APIs
        SyncConfiguration.SetFeatureToken(FeatureToken);

        // Login the admin user
        var credentials = Credentials.UsernamePassword(AdminUsername, AdminPassword, createUser: false);
        var admin = await User.LoginAsync(credentials, new Uri($"http://{ServerUrl}"));
    
        var config = new NotifierConfiguration(admin)
        {
            // Add all handlers that this notifier will invoke
            Handlers = { new CouponHandler() }
        };

        // Start the notifier. Your handlers will be invoked for as
        // long as the notifier is not disposed.
        using (var notifier = await Notifier.StartAsync(config))
        {
            do
            {
                Console.WriteLine("Type in 'exit' to quit the app.");
            }
            while (Console.ReadLine() != "exit");
        }
    }

    class CouponHandler : RegexNotificationHandler
    {
        // The regular expression you provide restricts the observed Realm files
        // to only the subset you are actually interested in. This is done to 
        // avoid the cost of computing the fine-grained change set if it's not
        // necessary.
        public CouponHandler() : base($"^/.*/private$")
        {
        }

        // The HandleChangeAsync method is called for every observed Realm file 
        // whenever it has changes. It is called with a change event which contains 
        // a version of the Realm from before and after the change, as well as
        // collections of all objects which were added, deleted, or modified in this change
        public override async Task HandleChangeAsync(IChangeDetails details)
        {
            if (details.TryGetValue("Coupon", out var changeSetDetails) &&
                changeSetDetails.Insertions.Length > 0)
            {
                // Extract the user ID from the virtual path, assuming that we're using
                // a filter which only subscribes us to updates of user-scoped Realms.
                var userId = details.RealmPath.Split(new[] { '/' }, StringSplitOptions.RemoveEmptyEntries)[0];

                using (var realm = details.GetRealmForWriting())
                {
                    foreach (var coupon in changeSetDetails.Insertions.Select(c => c.CurrentObject))
                    {
                        var isValid = await CouponVerifier.VerifyAsync(coupon, userId);

                        // Create a ThreadSafeReference of the coupon. While both
                        // details.CurrentRealm and details.GetRealmForWriting() are open 
                        // on the same thread, they are at different versions, so you need
                        // to pass the Coupon between them either via ThreadSafeReference
                        // or by its PrimaryKey.
                        var writeableCoupon = realm.ResolveReference(ThreadSafeReference.Create(coupon));

                        // It may be null if the coupon was deleted by the time we get here
                        if (writeableCoupon != null)
                        {
                            realm.Write(() => writeableCoupon.IsValid = isValid);
                        }
                    }
                }
            }
        }
    }
}

Notes

  • Multiple Notifiers may be started (via Notifier.StartAsync) but they need to have a different WorkingDirectory specified to avoid errors.
  • HandleChangesAsync will be invoked in parallel for different Realms but sequentially for changes on a single Realm. This means that if your code takes a lot of time to return from the function, a queue of notifications may build up for that particular Realm. Make sure to design your architecture with that in mind.
  • Asynchronous calls inside HandleChangesAsync must not use ConfigureAwait(false) as that will dispatch the continuation on a different thread making all Realm and RealmObject instances (that were open prior to the async operation) inaccessible from that thread.

    </div>

Integrating With Another Service

For a complete example of integrating the Event Handling framework with another service (in this case, IBM Watson’s Bluemix), read the tutorial for the Scanner App.

Data Connector

This feature is limited to our Enterprise editions. Learn more.

The Enterprise Edition of the Realm Object Server offers a Node.js-based adapter API that allows you to access all low-level Object Server operations and data. This can be used to let a synced Realm interact with an existing legacy database such as PostgreSQL: the Realm will also be kept in sync with the external database in real time. Client applications can use the Realm Database API and get the benefits of working with real-time, native objects.

The Adapter API is set up in a very similar fashion to the Event Handler API described above. Create a small Node.js application by creating a directory to place the server files, then create a package.json for npm dependencies or use npm init to create it interactively.

{
    "name": "MyApp",
    "version": "0.0.1",
    "main": "index.js",
    "author": "Your Name",
    "description": "My Cool Realm App",
    "dependencies": {
        "realm": "^1.8.0"
    }
}

After your other dependencies are specified, use npm install to download, unpack and configure the modules.

As with the event handler API, you’ll need to access the Object Server with administrative privileges, and will need to get the Object Server’s admin token. Under Linux, view the token with:

cat /etc/realm/admin_token.base64

On macOS, the token is stored in the realm-object-server folder, inside the Realm Platform folder. Navigate to that folder and view the token:

cd path-to/realm-mobile-platform
cat realm-object-server/admin_token.base64

To use the Adapter API, the Node.js application you’re creating will act as a translator, receiving instructions from the Object Server and calling your external database’s API to read and write to it. A sample application might look like this:

var Realm = require('realm');

var FEATURE_TOKEN = "eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9...";

// Unlock Enterprise Edition APIs
Realm.Sync.setFeatureToken(FEATURE_TOKEN); 

var adapterConfig = {
  // Insert the Realm admin token here
  //   Linux:  cat /etc/realm/admin_token.base64
  //   macOS:  cat realm-object-server/admin_token.base64
  admin_token: 'ADMIN_TOKEN',

  // the URL to the Realm Object Server
  server_url: 'realms://127.0.0.1:9080',

  // local path for the Adapter API file
  local_path: './adapter',

  // regular expression to limit which Realms will be observed
  realm_path_regex: '/^\/([0-9a-f]+)\/private$/'
};

new CustomAdapter(adapterConfig);

class CustomAdapter {
  constructor(config) {
    this.adapter = new Realm.Sync.Adapter(
      config.local_path,
      config.server_url,
      Realm.Sync.User.adminUser(config.admin_token, config.server_url),
      config.realm_path_regex,

      // This callback is called any time a new transaction is available for
      // processing for the given path. The argument is the path to the Realm
      // for which changes are available. This will be called for all Realms
      // which match realm_path_regex.
      (realm_path) => {
        var current_instructions = this.adapter.current(realm_path);
        while (current_instructions) {
          // if defined, process the current array of instructions
          this.process_instructions(current_instructions);

          // call advance to progress to the next transaction
          this.adapter.advance(realm_path);
          current_instructions = this.adapter.current(realm_path);
        }
      }
    )
  }

  // This method is passed the list of instructions returned from
  // Adapter.current(path)
  process_instructions(instructions) {
    instructions.forEach((instruction) => {
        // perform an operation for each type of instruction
        switch (instruction.type) {
          case 'INSERT':
            insert_object(instruction.object_type, instruction.identity, instruction.values);
            break;
          case 'DELETE':
            delete_object(instruction.object_type, instruction.identity);
            break;
          // ... add handlers for all other relevant instruction types
          default:
            break;
        }
      })
  }
}

Each instruction object returned by Adapter.current has a type property which is one of the following strings. Two or more other properties containing data for the instruction processing will also be set.

  • INSERT: insert a new object
    • object_type: type of the object being inserted
    • identity: primary key value or row index for the object
    • values: map of property names and property values for the object to insert
  • SET: set property values for an existing object
    • object_type: type of the object
    • identity: primary key value or row index for the object
    • values: map of property names and property values to update for the object
  • DELETE: delete an existing object
    • object_type: type of the object
    • identity: primary key value or row index for the object
  • CLEAR: delete all objects of a given type
    • object_type: type of the object
  • LIST_SET: set the object at a given list index to an object
    • object_type: type of the object
    • identity: primary key for the object
    • property: property name for the list property to mutate
    • list_index: list index to set
    • object_identity: primary key or row number of the object being set
  • LIST_INSERT: insert an object in the list at the given index
    • object_type: type of the object
    • identity: primary key for the object
    • property: property name for the list property to mutate
    • list_index: list index at which to insert
    • object_identity: primary key or row number of the object to insert
  • LIST_ERASE: erase an object in the list at the given index: this removes the object
  • from the list but the object will still exist in the Realm
    • object_type: type of the object
    • identity: primary key for the object
    • property: property name for the list property to mutate
    • list_index: list index which should be erased
  • LIST_CLEAR: clear a list removing all objects: objects are not deleted from the Realm
    • object_type: type of the object
    • identity: primary key for the object
    • property: property name for the list property to clear
  • ADD_TYPE: add a new type
    • object_type: name of the type
    • primary_key: name of primary key property for this type
    • properties: Property map as described in Realm.ObjectSchema
  • ADD_PROPERTIES: add properties to an existing type
    • object_type: name of the type
    • properties: Property map as described in Realm.ObjectSchema
  • CHANGE_IDENTITY: change the row index for an existing object: not called for objects with primary keys
    • object_type: type of the object
    • identity: old row value for the object
    • new_identity: new row value for the object

For full details, including all the instruction types and the data passed to them, consult the API reference for the Realm.Sync.Adapter class.

A PostgreSQL data connector is already implemented, and more are on the way, including MongoDB and Microsoft SQL Server. Any data connector can be customized to your application’s specific needs. If you’re a Realm Enterprise customer, contact your representative for more information.

Advanced Features

High Availability

With High Availability we aim to reduce reduce downtime due to failures. Realm Object Server achieves this by eliminating single points of failure, based on three main ideas.

  1. Stateless services All internal services in Realm Object Server (except the Sync service) are stateless, in the sense that they don’t keep the state themselves, using the Sync service instead. Stateless Services This allows spawning multiple Auth or Proxy services. The only thing that has a state is the Sync service, and the next two ideas solve that problem.

  2. Synchronous replication The fact that the backup process is synchronous ensures that the committed data is replicated at any given point in time. Synchronous Replication

Whenever a server fails, there is a full copy on another one.

  1. Service discovery Reliable service discovery is what allows Realm Object Server to notice and react to failures. Service Discovery

Realm Object Server chooses the new master replica carefully, it will not start an incomplete copy (which would be a disaster for the data).

All three at once make Realm Object Server highly available.

Consul

Realm Object Server depends on Consul for service discovery.

The Auth and Proxy services ask Consul for the current Sync master. The Proxy service connects the clients to the master, and it drops all current connections in case a new master is elected.

The Sync services use Consul to elect a master and a slave and expose themselves to other services.

Deployment

One of the possible deployment schemes is shown in the picture: Failover Deployment

It is recommended to run a Consul agent on each server in your cluster. In addition, it is recommanded that you maintain at least three Consul server agents. Refer to the Consul Documentation for more information on best practices for running Consul in production.

Since the sync workers and the proxy are supposed to be started individually on separate servers, the CLI provides new commands.

These examples assume that you’ve installed the Realm Object Server Enterprise NPM package:

npm install -g realm-object-server-enterprise

Starting the Proxy and Auth Services

These two services are currently united under one command. This will probably split into separate commands in a future release.

### set environment variables
CONSUL_HOST=<host>:<port>
FEATURE_TOKEN=<base64-encoded-and-signed> # provided by Realm
PRIVATE_KEY_PATH=<path>
PUBLIC_KEY_PATH=<path>
# don't forget to export all of those

### run the proxy and auth services
ros-enterprise run auth -p 9080 -a 0.0.0.0

Starting the Replicated Sync Services

In order to start the Replicated Sync services, repeat this on all servers where you want to run them.

### set environment variables
CONSUL_HOST=<host>:<port>
FEATURE_TOKEN=<base64-encoded-and-signed> # provided by Realm
PRIVATE_KEY_PATH=<path> # use the same key pair
PUBLIC_KEY_PATH=<path> # for all your services
SYNC_ADDRESS=<host>
SYNC_PORT=<port>
SYNC_ID=<string>
SYNC_LABEL=<string>
# don't forget to export all of those

### run one of the replicated sync services
ros-enterprise run resync -p 9080 -a host1

SYNC_ADDRESS is used by the proxy and other workers to connect to the worker, so this cannot be 127.0.0.1 or 0.0.0.0, use the real address at which the worker is to be found on the network.

SYNC_LABEL defines the sync worker group which serves as a single shard for load balancing. Should be default in the unbalanced case.

SYNC_ID uniquely identifies a sync worker among its worker group.

The services can use different values for CONSUL_HOST, if you run (and you should) multiple consul servers.

WARNING: A worker at a particular data directory should always be started with the same SYNC_ID and SYNC_LABEL, or else you lose your data.

WARNING: Always start at least three workers per label, because it needs at least two to successfully serve clients.

Backup

The Realm Object Server provides one or two backup systems, depending on your Edition.

  • The manual backup system is included in all versions of the Realm Platform via a command line utility. It can be triggered during server operations to create a copy of the data in the Object Server.
  • The synchronous backup system, available only in the Enterprise Edition, is a backup slave running alongside the master Realm Object Server. The backup slave continuously receives changes across all Realms.

Both systems create a data directory from which the Realm Object Server can be restarted after a server crash. The backed up data consists of the keys, the user Realms, user account information, and all other metadata used by the Realm Object Server. Backup can be made from a running Realm Object Server without taking it offline.

Manual backup

The manual backup is a console command that backs up a running instance of the Realm Object Server. This command can be executed on a server without shutting it down, any number of times, in order to create an “at rest” backup that can be stored on long-term storage for safekeeping. In order to be as agnostic as possible regarding the way the backup will be persisted, the command simply creates a directory structure containing everything the server needs to get started again in the event of a disk failure.

Because Realms can be modified during the backup process, the backup command uses Realm’s transaction features to take a consistent snapshot of each Realm. However, since the server is running continuously, the backed up Realms do not represent the state of the server at one particular instant in time. A Realm added to the server while a backup is in progress might be completely left out of the backup. Such a Realm will be included in the next backup.

To run realm-backup at the command line, type:

realm-backup SOURCE TARGET
  • SOURCE is the data directory of the Realm Object Server.
  • TARGET is the directory where the backup files will be placed. This directory must be empty or absent when the backup starts for safety reasons.

After the backup command completes, TARGET will be a directory containing all keys and Realms needed for backup.

Server Recovery From A Backup

If the data of a Realm Object Server is lost or otherwise corrupted, a new Realm Object Server can be restarted with the backed up data. This is done simply by stopping the server, if needed, and placing the latest backup directory into the Realm Object Server’s data directory. After the backup has been fully placed in the Realm Object Server’s data directory, the server may be restarted.

Client Recovery From A Backup

Any data added to the Object Server after its most recent backup will be lost when the server is restored from that backup. Since Realm Database clients communicate with the Realm Object Server continuously, though, it’s possible for clients to have been updated with that newer data that’s no longer on the server.

In the case of such an inconsistency, the Realm Object Server will send an error message to the client, and refuse to synchronize data.

The client side APIs will emit a “client reset” exception or error and give the app the possibility to reset the Realm and download the data from the server.

If client reset happens, the client will lose any data that was not included in the backup data. The best way to minimize this potential data loss is to use the synchronous backup system. If you can’t do that, run the manual backup system frequently.

Monitoring

Realm Object Server workers support sending metrics to statsd, which is assumed to be listening at localhost:8125. You can then forward these metrics to graphite or similar systems for monitoring.

All metrics keys start with a prefix of realm.<hostname>:

realm.example.com.connection.online
realm.example.com.connection.failed
realm.example.com.realms.open
realm.example.com.protocol.violated
realm.example.com.protocol.bytes.received
realm.example.com.authentication.failed

Metrics

Name Type Description
<prefix>.client.unsyncable counter Triggered every time a client fails to initiate synchronization of a realm because of messed up history. Such clients need their realm file deleted and then recovered from the server. This might happen if the server crashes and is recovered from a backup.
     
<prefix>.session.started counter Triggered every time a session is started. A session is considered started even before the authentication.
<prefix>.session.online gauge The total number of sessions currently being served.
<prefix>.session.failed counter Triggered every time there is a session-level error.
<prefix>.session.terminated counter Triggered every time a session terminates.
     
<prefix>.connection.started counter Triggered every time a client opens a connection.
<prefix>.connection.online gauge The total number of connections currently open. Multiple sessions may be served through a connection.
<prefix>.connection.failed counter Low-level errors on connections. Triggered every time a failure happens during accept(), read() or write().
<prefix>.connection.terminated counter Triggered every time a connection terminates. This includes the failed ones.
     
<prefix>.realms.open gauge The number of currently open realms.
     
<prefix>.authentication.failed counter Triggered on authentication failures, e.g. token invalid or expired. Should not normally happen.
<prefix>.permission.denied counter Triggered on permission failures, e.g. trying to access a realm with a token for another realm or trying to upload with a download-only token. Should not normally happen.
<prefix>.protocol.<version>.used counter Triggered every time a connection over protocol version <version> is established. Use this to track how many connections of each protocol version are initiated, and choose a better time to update the server and app.
<prefix>.protocol.violated counter Triggered every time the sync protocol is violated. May mean that the application is too old or badly written.
     
<prefix>.protocol.bytes.received counter Triggered every time an upload message is received by the server.
<prefix>.protocol.bytes.sent counter Triggered every time a download message is sent by the server.
     
<prefix>.protocol.bytes.received gauge The number of bytes received since the start.
<prefix>.protocol.bytes.sent gauge The number of bytes sent since the start.

Upgrading

Migrating from ROS 1.x

There is a built-in migration tool in the CLI which converts the old ROS (1.x) metadata into the new schema and (optionally) copies the user Realms to the new location. The user Realms are not converted by this tool, because ROS itself performs the necessary low-level conversion of the Realms when it starts.

Migration consists of the following steps.

In the examples we assume that the old server is running in a root dir /srv/old-root and is using keys /srv/keys/auth.key and /srv/keys/auth.pub .

  1. Stop ROS 1.x

  2. Run the migration tool

    mkdir -p /srv/new-root # should be an empty dir
    
    ros migrate --from /srv/old-root --to /srv/new-root/ --copyrealms

    If you use custom paths in the ROS 1.x configuration then omit --copyrealmsand copy the user Realms manually since the CLI doesn’t support configuring the custom path.

  3. Copy the keys

    Skip this if you intend to store the keys in a different place and point to them with --private-key and --public-key flags.

    KEYDIR=/srv/new-ros/keys # ROS 2.x uses 'keys' subdir by default
    mkdir -p /srv/new-ros/keys
    cp /srv/keys/auth.{key,pub} /srv/new-ros/keys/
  4. Start ROS 2.x

    ros start -d /srv/new-ros

Compatibility With 1.x

The 2.0 server is not backwards compatible with 1.x-compatible client SDKs. This means that if you upgrade your server in production, apps running 1.x-compatible SDKs will stop syncing until they are updated to 2.0-compatible SDKs.

What is the cause of the backwards compatibility?

The unique capability of Realm Platform is our ability to sync data changes that occurred offline seamlessly. The algorithms powering this functionality are novel and we identified a way to drastically improve the performance when merging out-of-order operations. These optimizations required that we add a unique and stable identifier to every object. This identifier is internal (we might expose in the future given that applications could utilize it), so it doesn’t change the outward experience at all, but at a protocol-level this is where the lack of backwards compatibility comes from.

Upgrading Deployed 1.x app

These instructions are subject to change but are provided for reference.

The general steps include:

  • Deploy a new version of the app using a 1.x-compatible SDK that has logic to react to an error callback triggered when the app connects to a 2.0 server. This logic should display UI to inform the user to upgrade the app.
  • Allow the user base to upgrade to this new version.
  • Submit a new version of the app using a 2.0-compatible SDK to the App Store, but prevent it from immediate public release.
  • Once the 2.0-compatible version of the app is available for public release, plan the upgrade of the Realm Object Server to 2.0 alongside the public release of the 2.0 compatible app in the App Store.

The resulting user experience should be that a user on the 1.x version of the app will connect to the newly upgraded 2.0 server and receive visual instructions in the app to go to the App Store to update to continue syncing. Because the 2.0 app in the App Store was made available publicly alongside the upgrade, the user should be able to immediate update and connect to the new server.

For apps deployed through enterprise App Stores that can control when the app gets updated, this process will be simpler. The complicated nature is due to the fact that developers of apps on the public App Stores have less control over their users’ upgrade process.

Finally, we can provide custom workarounds to enable running two Realm Object Servers in parallel to further ease the upgrade experience for end-users for paid customers.

Conflict Resolution

One of the defining features of mobile is the fact that you can never count on being online. Loss of connectivity is a fact of life, and so are slow networks and choppy connections. But people still expect their apps to work!

This means that you may end up having two or more users making changes to the same piece of data independently, creating conflicts. Note that this can happen even with perfect connectivity as the latency of communicating between the phone and the server may be slow enough that they can end up creating conflicting changes at the same time.

What Realm does in this case is that it merges the changes after the application of specific rules that ensure that both sides always end up converging to the same result even, though they may have applied the changes in different order.

This means that you no longer have the kind of perfect consistency that you could have in a traditional database, what you have now is rather what is termed “strong eventual consistency”. The tradeoff is that you have to be aware of the rules to ensure the consistent result you want, but the upside is that by following a few rules you can have devices working entirely offline and still converging on meaningful results when they meet.

At a very high level the rules are as follows:

  • Deletes always win. If one side deletes an object it will always stay deleted, even if the other side has made changes to it later on.

  • Last update wins. If two sides update the same property, the value will end up as the last updated.

  • Inserts in lists are ordered by time. If two items are inserted at the same position, the item that was inserted first will end up before the other item. This means that if both sides append items to the end of a list they will end up in order of insertion time.

Primary Keys

A primary key is a property whose value uniquely identifies an object in a Realm (just like a primary key in a conventional relational database is a field that uniquely identifies a row in a table). Primary keys aren’t required by Realm, but you can enable them on any object type. Add a property to a Realm model class that you’d like to use as the primary key (such as id), and then let Realm know that property is the primary key. The method for doing that is dependent on the language you’re using; in Cocoa, you override the primaryKey() class method, whereas Java and .NET use annotations. (Consult the documentation for your language SDK for more details.)

Once a Realm model class has a primary key, Realm will ensure that no other object can be added to the Realm with the same key value. You can update the existing object; in fact, you can update only a subset of properties on a specified object without fetching a copy of the object from the Realm. Again, consult the documentation for your language SDK for specifics.

For more information, read the Primary Keys Tutorial.

Strings

Note: Strings are not exposed in client APIs yet.

Strings are special in that you can see them both as scalar values and as lists of characters. This means that you can set the string to a new string (replacing the entire string) or you can edit the string. If multiple users are editing the same string, you want conflicts to be handled at the character level (similar to the experience you would have in something like Google docs).

Counters

Note: Counters are not exposed in client APIs yet.

Using integers for counting is also a special case. The way that most programming languages would implement an increment operation (like v += 1), is to read the value, increment the result, and then store it back. This will obviously not work if you have multiple parties doing it simultaneously (they may both read 10, increment it to 11, and when it merges you would get the result of 11 rather than the intended 12).

To support this common use case we offer a way to express the intent that you are incrementing (or decrementing) the value, giving enough hints to the merge that it can reach the correct result. Just as with the strings above, it gives you the choice of updating the entire value, or editing it in a way that conveys more meaning, and allow you to get more precise control of the conflict resolution.

Custom Conflict Resolution

The standard way to do custom conflict resolution is to change the value into a list. Then each side can add its updates to the list and apply any conflict resolution rules it wants directly in the data model.

You can use this technique to implement max, min, first write wins, last write wins and pretty much any kind of resolution you can think of.


These features are limited to the Realm Platform Professional and Enterprise editions. Learn more about them and start a free trial.

Note: This documentation section only describes features specific to the Enterprise and Professional Editions. For other topics, including configuration, general usage, and troubleshooting, read the main Object Server documentation.

Troubleshooting

Version Compatibilities

It is important to make sure that you have the correct version of client SDKs when interfacing with the Realm Object Server. Compatibilities are listed below.

Version Table for Realm Object Server 2.0
Browser Program Node.js (Required for Install) Realm JS Realm Swift + Objective-C Realm Java Realm .Net Realm Data Adapter
Realm Studio >= 6.0   >= 2.0  >= 3.0  >= 4.0  >= 2.0  >= 1.0.0

Verify Port Access

Certain factors may impact external (non-localhost) access to the Realm Object Server’s synchronization facility, and the Realm Dashboard. In order to enable access, it may be necessary to open port 9080 on your server(s).

Using the standard Linux tools, these commands will open access to the port:

sudo iptables -A INPUT -p tcp -m tcp --dport 9080 -j ACCEPT
sudo service iptables save

Please refer to the CentOS 6 Documentation for more information regarding how to configure your firewall.

sudo firewall-cmd --get-active-zones
sudo firewall-cmd --zone=public --add-port=9080/tcp --permanent
sudo firewall-cmd --reload
sudo ufw allow 9080/tcp
sudo ufw reload

If your environment does not use your distribution’s standard firewall, you will have to modify these instructions to fit your environment.

Operational Errors

Occasionally it can be useful for debugging purposes to check the Realm Object Server logs - which are in the terminal window on macOS or /var/log/realm-object-server.log on Linux systems. Realm Object Server produces two specific classes of error and warning diagnostics that may be useful to system admins and developers.

Session Specific Errors

  • 204 “Illegal Realm path (BIND)” Indicates that the Realm path is not valid for the user.

  • 207 “Bad server file identifier (IDENT)” Indicates that the local Realm specifies a link to a server-side Realm that does not exist. This is most likely because the server state has been completely reset.

  • 211 “Diverging histories (IDENT)” Indicates that the local Realm specifies a server version that does not exists. This is most likely because the server state has been partially reset (for example because a backup was restored).

Client Level Errors

  • 105 “Wrong protocol version (CLIENT)” The client and the server use different versions of the sync protocol due to a mismatch in upgrading.

  • 108 “Client file bound in other session (IDENT)” Indicates that multiple sync sessions for the same client-side Realm file overlap in time.

  • 203 “Bad user authentication (BIND, REFRESH)” Indicates that the server has produced a bad token, or that the SDK has done something wrong.

  • 206 “Permission denied (BIND, REFRESH)” Indicates that the user does not have permission to access the Realm at the given path.