Quicken Online is designed to compete with websites like Mint, but is not a replacement for desktop finance apps. It’s a single portal to view all of your bank transactions… and that’s about it. I don’t see any significant features that Mint doesn’t already have.

When you sign up for a Quicken Online account, it asks you to enter the usernames and passwords for your bank accounts. Intuit claims that the sign-on data is “encrypted and stored on our firewall-protected servers,” but as a software developer, I don’t find that particularly reassuring. I’d rather avoid giving my sign-on information to third parties altogether. That said, Intuit is a large company with many years of experience in storing financial data, so I do have some faith that they know how to handle it safely.

Once you enter your bank accounts, you can view transactions in those accounts from the Quicken Online home page. One nice feature is that you can enter upcoming transactions before they’re posted by your bank, so you can get an estimate of your upcoming cash flow. But you can’t enter any accounts that don’t have sync capabilities with Intuit.

Reconciliation

Unlike many people, I still keep track of my transactions separately from my banks, and reconcile bills and statements when they arrive. It’s a good way to avoid unauthorized charges and keep track of my spending. I’d like to “bring that to the 21st century” by checking off transactions as they post from my bank, rather than having to enter every transaction. Well, Quicken Online doesn’t have any such features. Here’s how they justify this:

Reconciling is useful for matching up your paper checkbook register with the transactions on your bank statement. But who keeps a paper checkbook register these days?

I find this justification highly suspect, coming from a company that sells millions of copies of an electronic check register that does just that. Of course I don’t want to reconcile a paper checkbook register. I do want to reconcile credit card receipts and (gasp) paper checks. I want to differentiate transactions I’ve already seen from those that are brand new. Reconciliation is a critical process of staying within a budget and avoiding fraudulent charges. I can’t tell whether Intuit’s online division’s management truly believes that reconciliation has gone the way of the dodo, or whether they’re just trying to avoid cannibalizing Quicken software sales.

At any rate, I can’t find any significant reasons to recommend Quicken Online. Intuit is a more established company than Mint and Wesabe, so I trust them a little more to store my bank account login safely. But Mint has a much “fresher” design, a nice iPhone app, and a more active user community.

In a few days I’ll review Wesabe as well and see how it matches up to these two.

First, I sent out about 150 online Christmas cards. For the last few years I used Yahoo Greetings, but this time I decided I wanted something a little more personal. I included a Flickr slideshow and a short holiday letter to the people I haven’t talked to recently. Plus I got to play around with some cool new technology like sIFR and the Amazon CloudFront CDN. If you didn’t receive a card but you’d like one, let me know in the comments.

Second, I developed a dashboard to monitor the health of my web server. I wrote a small script which captures statistics every minute, and then built the dashboard to go along with it. I’m using Flot, a JavaScript graphing library, to generate the graphs. It probably would have been wise to use an existing component to capture the data, but this works fine for now.

Geeky/cool indeed.

I’ve been worried about Internet security myself recently, so I started trying to set up my own VPN using the free OpenVPN software. My goal was simply to encrypt the traffic between my laptop and a computer I run on a wired network. This wouldn’t encrypt all my communications on the Internet backbone, but at least it would prevent snooping on wireless networks.

OpenVPN is designed to handle an incredibly wide variety of networks, and as a result it’s very difficult to configure to do something “simple” like this. I spent an hour reading instructions and generating encryption keys, but when I first tried to run the OpenVPN software on my MacBook, it crashed the computer. I quickly decided this wasn’t for me.

Instead I tried PureVPN, which was a low-cost VPN service open to the public. PureVPN is a pay-as-you-go service and offers a variety of inexpensive service plans. I paid $2.50 and received ten hours of VPN use. This would be a great deal—if the service worked as promised.

PureVPN doesn’t require any software beyond what’s built into Mac OS X or Windows. It was very easy to set up and when I tested it from home, it seemed to work fine. I confirmed that all of my Internet traffic was sent over the encrypted VPN, which ensured that I’d be protected from nosy neighbors. I tried it from a coffee shop once and it worked fine from there as well.

However, the real test occurred when I went on vacation in Las Vegas, Nevada. Away from home for a week, I wanted to use PureVPN over many insecure wireless networks—at hotels, at cafés, and at at my sister’s house. But when I got to Las Vegas, I found that PureVPN was down! It was down the entire week that I was gone, and only came back a few days after I returned home.

I hadn’t invested much money in my PureVPN subscription, so I haven’t contacted them about the downtime. At $2.50 I figure “you get what you pay for.” But unfortunately I can’t recommend PureVPN to anyone else, simply because I don’t trust I can rely on them when I need them.

I’ve been a long-time user of Microsoft Money to keep track of all my bank accounts: checking, credit cards, loans, and investments. I enjoy the comfort of manually entering all my transactions, and reconciling the bank statement at the end of the month. This process lets me keep an eye on my accounts and watch out for suspicious activity. But in ten years of banking, I don’t think I’ve ever found a truly unauthorized transaction.

Microsoft Money made data entry painless and provided many simple reports about where my money is going. But I’ve had trouble finding a similar solution after I switched to a Mac in 2007. Both Quicken Personal Finance for Mac and iBank have atrocious data entry processes that require constant movement between the keyboard and the mouse. It’s also very difficult to generate simple reports in either program. I’ve been using iBank for a year and I still can’t figure out how to see a list of all transactions in a category.

Frustrated with the lack of Mac options, I finally decided to try some online finance apps instead. Mint has developed a very good reputation online so it was the first site I tried. Mint is a web application that actually connects to your banks every night and downloads all of your transactions. It presents you with a summary of your recent transactions and your current balances. You can easily drill down and find more information about your accounts and transactions. The site is beautiful, and reports are easy to generate and customize.

Security was my overwhelming concern—they have your online banking password, so they could access all of the money in your accounts! But they say that they’ve developed reasonable safeguards, and they work with a third-party intermediary to secure your data.

Some more things I like about Mint:

1. It was incredibly easy to add most of my bank and investment accounts to the site. Mint was pre-configured to work with nearly all of my financial institutions.
2. The site automatically imports new transactions every time I login. This works well and it’s easy to customize the way different transactions are handled. Mint will send me a text message when large transactions clear or when my balances are low, which is a great service.
3. The forums are fantastic and there’s already a strong user community sharing tips and tricks about saving money with Mint.
4. Mint is free, and they make money by suggesting new financial services. But they only recommend services when it’s likely in your best interest. For example, right now Mint says I could save money by moving my checking account to HSBC. This is an unusual business model because most referral programs are difficult to set up and don’t pay well. But financial referrals usually do pay well—think of all the gimmicks to entice you to sign up for a credit card. I suspect Mint will become profitable quickly.

What I don’t like:

1. I’m still not eager to give my bank account credentials to Mint. It’s impossible to know exactly how Mint stores this data or how secure their service providers are. I’m not sure that there is a good solution to this problem in any environment where the data is outside my immediate control.
2. If your bank isn’t supported by Mint, there’s no way to track an account manually. This is a significant problem for me. I have a health savings account (HSA) which Mint can’t recognize. Either I need to keep using Quicken for this one account—which is silly—or I need to stop recording transactions from my HSA. Neither option is satisfactory. I will not be able to rely on Mint alone unless I can enter data manually and have my HSA treated equal to all my other accounts.

Next week I’m going to try Wesabe, another site similar to Mint, and see how it stacks up. I also might try Quicken Online, which Intuit is now offering for free. I’ll post reviews of those applications once I’ve had a chance to try them.

But the hype around these new databases is growing frantic. This morning I read an article by Todd Hoff which fawned over SimpleDB’s unconventional rules to such an extent that I thought it might be satire. There are some significant drawbacks to developing in this new database paradigm. In fact, many of Mr. Hoff’s supposed advantages are actually serious disadvantages to the paradigm. Before designing your architecture around a database engine like SimpleDB, it’s important to consider the reasons not to do so.

Most of my points are directed at the Amazon SimpleDB service, but many also apply to other databases like CouchDB and the Google Datastore.

1. Data integrity is not guaranteed.

Data stores like SimpleDB don’t support the same rigorous constraints that RDBMSes do. Some of these databases support single-row constraints, like requiring data in certain fields, but it is nearly impossible for these systems to enforce UNIQUE constraints and foreign keys.

Programmers can work around this by issuing extra queries to confirm an update is valid, but this requires a lot of extra work. This will never be perfectly accurate–it may be impossible to avoid race conditions when two clients simultaneously attempt conflicting updates. And it’s especially difficult with SimpleDB because SimpleDB doesn’t guarantee that a client sees all the recent updates to the data.

2. Inconsistency will provide a terrible user experience.

Speaking of inconsistency, it’s critical to shield users from
this property of SimpleDB.

SimpleDB is optimized for fast writes. Your API calls return as soon as the data is written to the SimpleDB service, but before it’s replicated across all of the SimpleDB servers. If you issue any queries before the data is propagated, you won’t necessarily see your most recent change.

When I save my changes in your web application, I expect that your system will show me a consistent view of those changes. If you show me the data that’s in SimpleDB, my changes might not appear, and I’ll probably get confused. In fact, I will probably freak out, thinking that you lost my data. You can try to inform me about how this works (“It will take a few minutes for your changes to be visible…”) but that’s not easy for users to grasp.

3. Aggregate operations will require more coding.

SimpleDB does not support aggregate operations like joins, GROUP BY, SUM/AVERAGE functions, and sorting. You will need to implement these yourself.

Todd Hoff argues that this “suckiness” is a fair tradeoff:

SimpleDB shifts work out of the database and onto programmers which is why the SimpleDB programming model sucks: it requires a lot more programming to do simple things. I’ll argue however that this is the kind of suckiness programmers like. Programmers like problems they can solve with more programming. We don’t even care how twisted and inelegant the code is because we can make it work. And as long as we can make it work we are happy.

I disagree. More boilerplate code distracts you from actually solving real users’ needs. Why reinvent the GROUP BY wheel when MySQL, PostgreSQL and Oracle have already perfected it?

4. Complicated reports, and ad hoc queries, will require a lot more coding.

In my experience, database use falls into three broad patterns: (1) standard queries and updates performed by your application’s users; (2) more complicated reports for users and internal staff; and (3) ad hoc queries for troubleshooting and system monitoring. SimpleDB may be optimized for category 1, but categories 2 and 3 will be much more difficult without SQL.

Complicated reports are probably the best application of the SQL language. Because SQL is a declarative language, it’s incredibly easy to generate aggregate information about your data. In my previous jobs, our reports often required hundreds of lines of SQL to get the right information out of the database. This is a lot of code, but it was required to generate the data for our customers. Without access to SQL, your programmers will need to implement reports through imperative statements, which will exponentially increase the development time.

Ad hoc queries are even worse: they’re usually simpler, but they’re always changing. An RDBMS expert can often write an ad hoc SQL query as fast as the marketing department can explain what they need. Using an imperative programming language to write these queries would destroy your developers’ productivity.

5. Aggregate operations will be much slower if you don’t use an RDBMS.

RDBMSes are highly optimized for performing aggregate operations across huge volumes of data. Fast algorithms like the hash join, merge join, and indexed binary search have been around for 20 years or more. SimpleDB and the Google Datastore return datasets which are more like objects than traditional database rows. It’s unlikely that you’ll be able to process this data with anything other than nested loops, especially if your programmers aren’t database algorithm experts. Nested loop algorithms are considerably slower than the others.

Even if you’re the 31337est database expert and enjoy writing these operations in your business objects, there’s another performance factor to consider. In order for your application server to handle aggregate operations, you will need a copy of all the relevant data on the application server. Rather than downloading a single SUM function result from the database, your application server will need to download all the data required to calculate the sum. This extra data transfer will add considerable latency when you’re dealing with thousands or millions of records.

6. Data import, export, and backup will be slow and difficult.

Oracle, MySQL and other RDBMSes include advanced tools to perform large-scale data import and export operations. These tools have also been refined for 20 years or so, and can process millions of rows per minute. There are no such tools for key-value data stores, because these products are so new.

When you’re processing millions of records, network latency makes a big impact. Most of these services perform a remote procedure call for each record inserted; some even limit you to querying one record per remote call. On the Internet, round-trip latency is usually 20-40ms, which may slow you down to fewer than 2,000 rows per minute. (You can process more quickly via multi-threading, but again, that requires you to write a lot more infrastructure code.)

7. SimpleDB isn’t that fast.

Todd Hoff’s article referenced a SimpleDB performance test which found that 10 record IDs could be retrieved in 141ms from a 1,000-record table; in 266ms from a 100,000-record table; and in 433ms from a 1,000,000-record table.

Compared to relational databases, this is pretty slow.

If you want your web application to be responsive, you need your database queries to operate much faster than this. 20ms responses would be more in line with conventional databases. If you perform 3 SimpleDB queries in series, your web app will take about 1.5 seconds for that operation, and users will notice when the app is that slow. Many web applications actually make dozens of queries per request.

Further, tables with a million records aren’t large enough to need significant scalability. A million-record table is probably small enough to fit entirely in RAM; surely its indexes could fit in RAM. The real test of SimpleDB scalability is its performance on a table with 100 million or 1 billion records.

8. Relational databases are scalable, even with massive data sets.

The world’s largest companies all use giant relational databases, and they’ve been able to make this work. The world’s largest websites use relational databases, and they’ve also been able to scale successfully. Facebook and LiveJournal use MySQL; MySpace uses Microsoft SQL Server; Salesforce.com uses Oracle. When websites like Friendster have scalability issues, it’s not usually because of the RDBMS.

We all expect Oracle to scale if we pay them enough money, but even free databases have made significant advances to prevent the database server from becoming a bottleneck. The first line of defense is caching–eliminating repetitive queries can offload massive amount of processing. Beyond caching, there are free clustering engines which let you balance your database requests around a few servers in a cluster.

Without a complicated clustering setup, your data can usually be partitioned across multiple servers to eliminate the single-server bottleneck. Lest you think I’m ragging on Todd Hoff, he’s written a nice overview of sharding, one way of designing a federated database to get around the bottleneck.

9. Super-scalability is overrated. Slowing the pace of your product development is even worse.

Time-to-market is a critical factor for most software products. If you’re writing internal software for a business, budgetary concerns are equally critical. You can workaround most of the drawbacks I’ve identified above, but it will cost you time and money.

More importantly, all these technical workarounds distract you from addressing the real needs of your customers. If you don’t focus on making something people want, it doesn’t matter how scalable your database is, because you won’t have any customers to fill up the database.

The hype around the new data stores seems to be a case of premature optimization, yet we all know Donald Knuth’s famous quote, “Premature optimization is the root of all evil.” Why not wait and address super-scalability once you’ve created a super product and have generated super cash flow?

10. SimpleDB is useful, but only in certain contexts.

Everyone’s assuming that SimpleDB was designed to be a general-purpose replacement for OLTP database servers. I don’t think it was ever intended for that purpose. SimpleDB’s architecture is similar to Dynamo, Amazon’s internal “highly-available key-value store.” One of its main distinguishing features is the flexible schema: the ability to add custom fields to individual records, and to store multiple values in each field.

If you’re working with “semi-structured” data, then this is actually incredibly useful. For example, it’s an awesome way to persist web application sessions. You can avoid the overhead of marshaling the object-oriented session data into columns and rows, and many of the drawbacks above don’t apply because you don’t generally query sessions like you query more typical relational data.

Amazon SimpleDB, Apache CouchDB, and the Google Datastore API aren’t bad products. But we do them a disservice when we construe them to be replacements for general-purpose databases. Used carefully, they can help your organization. But used indiscriminately, you’ll create a lot more work for your programmers and you’ll make your application perform even worse.

With that out of the way, I just needed a way to display the calendar on my website. I considered writhing my own iCalendar display engine, but then remembered that PHP iCalendar would do this for me. PHP iCalendar gives me a lot of flexibility with the display, so I was able to remove all of the material except for the calendar itself.

I’ve embedded it into my website using an iframe. This isn’t the most technologically advanced approach, but it was a lot easier than doing anything fancier. Even if I did use a more advanced approach to integrate the calendar into my website template, it wouldn’t improve the user experience at all. So, no need to bother.

What do you think of the calendar? Make sure to send me those meeting requests and party invitations!

Yesterday I registered for WordCamp 2007, a conference dedicated to the software that runs this blog. It sounds pretty interesting: the first day is going to feature presentations about blogging with WordPress, and the second day focuses on WordPress development. It’s coming up next weekend here in San Francisco. If you see me there, say hi!