VoodooPad's WebDAV Syncing Tech
(Or, how VoodooPad does its page syncing)
Misc Info You Gotta Know First
- Every document in VoodooPad has a uuid associated with it. And every page in that document also has a uuid associated with it.
- A "client" is just a VoodooPad document.
- A "sync id" is a unique identifier for each VP document. It's based on the volume uuid that the document in sitting on, along with "file system number" of the document. This allows you to have multiple clients for the same sync data on the same machine.
- This document was written with VoodooPad 4.0.1 in mind. Changes might have occurred after that release.
The Document Structure
When you setup VoodooPad to sync it's pages over a webdav server, it'll create a directory structure with the following hierarchy:
VoodooPadSync/ - This is the top level dir that documents are synchronized to. Inside it are only document folders.
VoodooPadSync/<doc uuid>/ - This directory represents a document, and contains a couple of files and a pages directory. The uuid is of the first document to sync to this location.
VoodooPadSync/<doc uuid>/Info.plist - This contains information about the document, which is retrieved when you pair up a document for a first sync. It's got the name of the document, and some other tidbits about it.
VoodooPadSync/<doc uuid>/syncinfo.plist.gz - This contains information on all the pages in the document, a list of clients (documents), and pages to be deleted.
VoodooPadSync/<doc uuid>/pages/<page(s) uuid.plist.gz> - this is a collection of the pages, which are gzip'd binary plists.
VoodooPadSync/<doc uuid>/Lock/ - This directory is created when a document syncs to the server. All updated pages, as well as an updated syncinfo file are written in this directory. When everything is safely written to the server, VP moves the page.plist.gz files from the Lock directory to the pages/ directory, and the moves the syncinfo file to the document base directory. It then deletes the Lock dir.
If the Lock directory is already present when VoodooPad tries to sync, then it just spins in the background waiting for it to clear. However, VP does peek at the Lock/ directory contents and if it's been over 8 minutes (by default) since anything has been written to it, it assumes that the last client must have crashed. It then deletes the Lock/ folder, recreates it, and moves along with it's pull and push.
The Lock directory is created even when there is nothing to push up. If we're reading data, then we don't want another client changing it when we're pulling it down.
A flaw- since nothing is written to the Lock directory when we read, then if it takes over 8 minutes and another client is waiting on the lock to clear... well, our lock will be deleted. A fix would be to write some piece of data in there when reading, and update it every 7 minutes or so, to keep other clients from deleting the Lock. (This will happen in a version of VP after 4.0.1)
Push/Pull
When syncing, VoodooPad first looks on the server and then pulls down any changes. If there's a page name conflict, VP renames the local page and assumes the one on the server is the "right" one. Once everything has been pulled down and deleted pages have been removed, VP then will push up the local changes.
One other trick VP does- it makes a hash of the pages, and compares the server version along with the local version. If the server version is newer, but the data hash is the same VP won't bother pulling it down. The same thing for pushing.
Other Stuff
And here's what a typical VoodooPadSync folder would look like:
VoodooPadSync/
8a513e9e-da86-4fc7-9c2a-6da29c2f091d/
Info.plist
syncinfo.plist.gz
pages/
./05cb8826-b9e6-4151-9ec1-46187a92f1ef.plist.gz
./0e7c7775-4f5d-47d7-8379-21eff0c8530a.plist.gz
...
./3d357f41-2d9f-4d3a-be16-87ee943186af.plist.gz
fca0e46f-f8f3-43c1-829d-95aad0cfdb25/
Info.plist
syncinfo.plist.gz
pages/
./d7e9db5e-e5a0-4631-b9a3-084fcb147ccc.plist.gz
./dcb06838-99e8-4990-b0e6-2b1545a977da.plist.gz
...
./f9813210-3ae0-4dd3-b3d7-c71f7bd8275b.plist.gz
If you're a cocoa developer and you'd like to investigate using WebDAV for your syncing needs, make sure to check check out my FMWebDAVRequest class: http://code.google.com/p/flycode/source/browse/trunk/davtest/ . It does do everything, but it does what I need and it's what I use in VoodooPad.
Fin.
