Utilities

A bidirectional cache has the following features:

• stores pairs
• supports a single value per key
• supports a single key per value
• efficient mapping from key to value
• efficient mapping from value to key
• optional strict cache size
• eviction based on least-recently-used

The cache maintains two dictionaries internally. One mapping from key to value. And the other mapping from value to key. So a lookup based on the either the key or value can be performed in O(1).

Caching:

When the countLimit is non-zero, this class operates as a cache, enforcing the designed limit, and using eviction when the limit is exceeded. When the countLimit is zero, this class operates as a generic container (with no limit, and no automatic eviction).

Eviction depends entirely on usage. The cache maintains a doubly linked-list of tuples ordered by access. The most recently accessed item is at the front of the linked-list, and the least recently accessed item is at the back. So it’s very quick and efficient to evict items based on recent usage.

YapCache implements a simple strict cache.

It is very similar to NSCache and shares a similar API. However, YapCache implements a strict countLimit and monitors usage so eviction is properly ordered.

For example: If you set a countLimit of 4, then when you add the 5th item to the cache, another item is automatically evicted. It doesn’t happen at a later time as with NSCache. It happens atomically during the addition of the 5th item.

Which item gets evicted depends entirely on usage. YapCache maintains a doubly linked-list of keys ordered by access. The most recently accessed key is at the front of the linked-list, and the least recently accessed key is at the back. So it’s very quick and efficient to evict items based on recent usage.

YapCache is considerably faster than NSCache. The project even comes with a benchmarking tool for comparing the speed of YapCache vs NSCache.

However, YapCache is NOT thread-safe. (Whereas NSCache is.) This is because YapCache was designed specifically for performance. Thus it’s NOT recommended you use YapCache unless you’re always going to be using it from the same thread, or from within the same serial dispatch queue. The various YapDatabase classes which use it inherently serialize access to the cache via their own internal serial queue.

Also, YapCache does NOT automatically purge itself in the even of a low memory condition. (Whereas NSCache does.) This also has to do with YapCache not being thread-safe. And thus performing this action (if desired) is up to you. The various YapDatabase classes which use it do this themselves.

An efficient collection/key tuple class.

Combines collection & key into a single object, and provides the proper methods to use the object in various classes (such as NSDictionary, NSSet, etc).

Allows you to configure the default values for new connections.

When you create a connection via [YapDatabase newConnection], that new connection will inherit its initial configuration via these connectionDefaults. Of course, the connection may then override these default configuration values, and configure itself as needed.

Changing the connectionDefault values only affects future connections that will be created. It does not affect connections that have already been created.

The connection pool class was designed to help you optimize background read-only transactions. As a reminder:

• You’re encouraged to use a dedicated read-only connection for the main thread. https://github.com/yapstudios/YapDatabase/wiki/Performance-Primer#readonly_vs_readwrite_transactions

• You’re encouraged to share the dedicated read-only main-thread connection between your viewControllers: https://github.com/yapstudios/YapDatabase/wiki/Performance-Pro#sharing_the_ui_databaseconnection

• You’re encouraged to create a dedicated read-write connection for read-write transactions: (Because there can only be a single read-write transaction per database at any one time.) https://github.com/yapstudios/YapDatabase/wiki/Performance-Primer

This leaves only non-main-thread read-only transactions. What’s the recommendation for them? You could create a single read-only connection that will be shared by all background tasks. However, since all transactions are serialized via the shared connection, this implies that background taskA may have to wait for background taskB to finish its read-only transaction before background taskA can execute its transaction. And for background tasks, this is NOT the preferred scenario.

The connection pool was designed to increase the performance in these scenarios. It will create connections on demand, up to (but not over) the connectionLimit. And it will vend connections using a simple load balancer that’s based on the number of pending transactions that each connection has. (So you’ll be handed the connection with the smallest queue of pending work.)

This allows for increased parallelization amongst your background tasks.

A proxy connection is a trade-off in terms of the ACID guarantees of the database.

Under normal operations, you must execute a read-write transaction in order to modify the datatbase. If the transaction completes, then all data from the transaction has been written to the database. And further, the transaction is durable even in the event of an application or system crash. In other words, you’re guaranteed the data will be there when the app re-launches.

A proxy connection allows you to relax these constraints, which may be useful for certain subsets of your data.

Here’s how it works:

• you write collection/key/value rows to a proxy connection instance
• the value(s) are immediately readable via the proxy connection instance
• the proxy will attempt to write the changes (in batches) at some point in the near future

Thus you can read & write values as if the proxy is an in-memory dictionary. However, the proxy will transparently write the changes to the database (but without any guarantees).

** When should I use a proxy ?

You can use a proxy when:

• it’s not important that you manage an individual transaction
• it’s not important if the values don’t make it to disk

Example #1 : A download manager

Applications sometimes have download logic encapsulated in a manager class. The manager class typically handles things such as:

• downloaing resources on demand
• parsing the results
• providing getter methods to fetch particular items
• automatically checking expiration dates & refreshing items as needed
• automatically deleting unused expired items

You’ll notice the manager class is the ONLY class that handles reading/writing certain values in the database. And it doesn’t matter if a value doesn’t get written to disk, as it can simply be re-downloaed.

Example #2 : NSUserDefaults replacement

NSUserDefaults works similarly to a connection proxy. From the documentation for NSUserDefaults:

The synchronize method, which is automatically invoked at periodic intervals, keeps the in-memory cache in sync with a user’s defaults database.

So NSUserDefaults will eventually write changes to disk. Unless you invoke the synchronize method, which involves waiting for disk IO to complete.

Thus a connection proxy can easily replace your NSUserDefaults system. It’s worth pointing out that a proxy connection doesn’t have the equivalent of a synchronize method because you don’t need one. A connection proxy always begins an asyncReadWrite transaction once it becomes dirty.

In case you’re wondering, Why would I use YapDatabase over NSUserDefaults ?

1. NSUserDefaults is not encrypted. And you can easily encrypt a YapDatabase.
2. NSUserDefaults writes ALL values to disk everytime because it uses a plist to store the values on disk. YapDatabase uses a database, so only changed values need to be re-written. Thus, YapDatabase has the potential to be faster and involve less disk IO.
3. YapDatabase has a notification system that tells you exactly which key/value pairs were changed.
4. YapDatabase makes it easier to sync your data using a wide variety of cloud services.

** Caveats

A connection proxy instance expects to own a subset of the database. That is, it expects to be the only thing reading & writing a subset of rows in the database. If you violate this, you may get unexpected results.

Example #1 :

• You write a value using a proxy.
• You then attempt to read that value using a regular database connection.
• If the proxy hasn’t written the value to disk yet, then the regualar database connection won’t see the proper value.

Example #2 :

• You write a value using the proxy.
• You write a different value (for the same collection/key) using the regular database connection.
• The proxy later performs a readWrite, and writes its value for the collection/key, overwriting the regular database connection.

A YapDatabaseQuery is used to pass SQL style queries into various extension classes. The query is generally a subset of a full SQL query, as the system can handle various details automatically.

For example:

 query = [YapDatabaseQuery queryWithFormat:@"WHERE department = ? AND salary >= ?", deptStr, @(minSalary)];
 [secondaryIndex enumerateKeysAndObjectsMatchingQuery:query
                                           usingBlock:^(NSString *collection, NSString *key, id object, BOOL *stop){
     ...
 }];

YapDatabaseQuery supports the following types as query parameters:

• NSNumber
• NSDate (automatically converted to double via timeIntervalSinceReferenceDate)
• NSString
• NSData
• NSArray (of any regular type above)

Array example:

 NSArray *departments = [self engineeringDepartments];
 query = [YapDatabaseQuery queryWithFormat:@"WHERE title = ? AND department IN (?)", @"manager", departments];

YapProxyObject acts as a proxy for a real object in order to lazily load the object on demand.

Generally, a YapProxyObject will be passed via a block parameter. The underlying object that the proxy represents may or may not be loaded in memory. If not, the proxy is configured to automatically load the underlying object (using the current transaction) on demand.

This class provides a standardized way to create a sort of whitelist / blacklist. It is used often within the options of extensions to create the set of allowedCollections.