Off-heap cache: ehcache vs Derby DB

I had my test like so:
- 1M properties where
- key varied between 30 and 70 characters, with average of 50
- value varied from 100 to 300 characters with average of 200
- size of file on disk 240MB

I chose to go through JMeter and its HTTP samplers, I wrote a simple JSP, so the overhead of both JMeter and Tomcat are sizable (run on same machine) and completely obfuscate meaningful values. 60 threads with constant throughput were used.

ehcache:
- configured to keep all values eternally, allowed to overflow to disk

Test 1: cache 5000 properties in memory
Test 2: cache 1000 properties -//-
Test 3: cache 10000 properties -//-
Test 4: cache 15000 properties -//-

Throughput achieved: 180 req/sec
CPU used: Tomcat 5%, JMeter 42%.
In all tests heap usage was around 225M (with max steadily rising to 300-380M), regardless of number of entries to cache... BAD!!!
Sampler time average: 1ms, max: over 3000ms (yep, GC is a bitch). But note the caveat above, these values are not to be trusted as a measure of performance.
Size of overflow file on disk: 512MB

Derby
- simple table, with PK constraint and index on key column
- used in embedded mode
- DB size 410MB (308MB table, 102MB index) on disk

Test 1: cache 10000 pages
Throughput achieved: 180 req/sec
CPU used: Tomcat 17%, JMeter 41%.
Heap usage: 72MB after full GC and went up to 128 if unhindered Not bad!!!
Sampler time average: 1 ms, max: 500 ms. But note the caveat above, these values are not to be trusted.

Test 2: cache 5000 pages
Throughput achieved: 180 req/sec
CPU used: Tomcat 17%, JMeter 41%.
Heap usage: 38MB after full GC and went up to 74 if unhindered Awesome!!!
Sampler time average: 1 ms, max: 180 ms.

Test 3: cache 1000 pages
Throughput achieved: 180 req/sec
CPU used: Tomcat 17%, JMeter 41%.
Heap usage: 14MB after full GC and went up to 27 if unhindered. W00t!!!
Sampler time average: 1 ms, max: 170 ms.

Test 4: default number of pages to cache
Throughput achieved: 180 req/sec
Heap usage: 14MB after full GC and went up to 27 if unhindered.
Sampler time average: 1 ms, max: 170 ms.
Test 4 proves that default is 1000 pages to cache... I had a doubt about it for a while there.

So, there. Derby is extremely good at keeping strict SLA (low pauses), but it does burn 3x CPU as compared to ehcache. In real-world scenario, there are ways when ehcache may further benefit: if small keys are used (int/long values), or when complex objects are stored (which would have required joining several tables). Caching Java properties may not be the best scenario for ehcache. However others were successful in combining the two. I might try doing the same at some point, but maybe not, since I got what I wanted from this test, the "feel" of these approaches.