Large Scale High Performance OpenLDAP Deployments

by Wolfgang Hummel

 

The deployment of a large scale OpenLDAP deployment with high performance requirements is a complex task.

Most data available in the Internet only has comparably small deployments as a topic of discussion. No one reports about details of a large scale real production world setup.

This is a report about a deployment of OpenLDAP with 2 DBs each with 35 Mio. subscriber profile entries at a tier 1 mobile telephony network operator environment and of a test wrt the limits of OpenLDAP at a mixed production-like read/write load. The effect of log levels on performance was also tested.

The deployment started initially with a setup of 2 LDAP Masters and 10 LDAP Caches. During the rollout based on experience of a growing LDAP DB and overload on the Masters in disaster recovery situations where syncrepl generated full DB scans this was modified to a setup with 2 Masters and 2 Caches only - with the option to move to a pure 2 LDAP Master setup in the near future.

The deployed setup requires the following performance figures:

  • read transactions< 20 msec avg response time
  • write transactins< 100 msec avg response time
  • load profile 1:
    • 1.200 reads /sec in parallel with 2 writes / sec on DB1
    • 400 reads / sec in parallel with 4 writes / sec on DB2
  • load profile 2:
    • 10 reads /sec in parallel with 10 writes / sec on DB1
    • 60 reads / sec in parallel with 30 writes / sec on DB2

The "test the limit" setup used the following scenario:

  • 3 DBs,
    • DB1 = 22 Mio. entries,
    • DB2 = 35 Mio. entries
    • DB3 = 40 Mio. entries
  • 4.000 reads / sec in combination with 700 writes / sec on DB3 running on a single server (DB1 and DB2 are on different servers)
  • 10.000 reads / sec in combination with 1.000 writes / sec distributed evenly over 3 DBs all hosted on a single server.

The test setup consisted of 2 servers HP DL380 G7, each with 2 Intel hexcore CPUs 3.33 GHZ, 192 GB memory, 16 x 15k rpm SAS disks. Both servers were acting as a LDAP Master / LDAP Read Cache configuration and as load drivers in parallel.

It required quite some fine tuning to achieve the requested results. OpenLDAP did a good job here and managed to accomplish or over-accomplish all requested performance figures as long as loglevels did not exceed a certain threshold.

 

Back to the program