In this article, I'm going to explain the difference between optimistic and pessimistic locking and when to use each approach in general.
If you are a developer you might have faced challenges in data access in distributed systems. A typical problem would be updating data from multiple threads ensuring that only one process or thread accesses a shared resource at a given time.
To deal with this type of problems, Distributed locks come into the picture.
So, What is Distributed locks?
Distributed locks are synchronization mechanisms used in distributed systems to ensure that only one process or thread can access a shared resource at any given time, even when the resource is being spread across multiple machines or nodes.
By using distributed locks, business systems can be improved in the following two aspects
Operational Excellence - To avoid tasks being executed multiple times, each executor grabs a lock when the tasks start which can avoid duplication of work in most cases.
Data Integrity - Only one task executor can obtain the task execution, ensuring that lock failure does not occur in any circumstances. If the lock fails, it may bring serious problems such as data consistency, data loss and so on.
Basically what is lock?
Locks are a technique for protecting shared resources from being modified by concurrent operations. Say when a process wants to access a shared resource, it must first acquire a lock. If another process already holds the lock, the process must wait until the lock is released.
How it solves the problem - It simply introduces a coordination mechanism between processes running on multiple nodes.
Note
Distributed locks can be implemented in different ways
Database locks - MySQL (GET_LOCK), PostgreSQL (pg_advisory_lock) block other operations until a transaction completes.
Cache-based locks - Redis (Redisson, Redlock) and Hazelcast (ILock/FencedLock) acquire exclusive locks to ensure that only one process can modify a resource.
Coordination tools - ZooKeeper ( Curator Framework or ZKClient), etcd (via the Lease API), and Consul (session or key-value locks) coordinate locks across distributed systems using consensus protocols.
Job schedulers like ShedLock and Quartz further ensure that jobs do not run concurrently across multiple nodes.
Pessimistic locking
With pessimistic locking, when a transaction begins, the record being updated (row or table based on your design choice) is immediately locked. This means no other transaction can modify that record until the lock is released, which usually typically ends when the transaction ends or is rolled back.
How It Works
With pessimistic locking, when a transaction starts, the record being updated is locked immediately. This means that no other transaction can modify that record until the lock is released (typically when the transaction ends or is rolled back).
The lock is session-based, so the record remains locked for the duration of the session.
Example
Imagine a banking application where both a deposit and a withdrawal are initiated on the same account at the same time.
If the deposit transaction starts first, it locks the account record and withdrawal transaction must then wait until the deposit transaction completes and the lock is released.
This approach ensures that every operation works on the most current data, thereby reducing the risk of errors. However, one drawback is that pessimistic locking is that it can sometimes lead to deadlocks when multiple transactions are waiting on each other. More details on the deadlock explained below
Scenario
The bank account has a balance of $1000.
Deposit Transaction Begins:
The Deposit Process requests a lock on the account (id = 123).
The Database grants the lock to the deposit transaction.
The Deposit Process retrieves the current balance from the database:
SELECT balance FROM account WHERE id = 123;
The database returns $1000.
Concurrent Withdrawal Attempt:
The Withdrawal Process attempts to acquire a lock on the same account (id = 123).
The Database does not grant the lock immediately because the deposit transaction is still in progress. The withdrawal request remains in a waiting state.
Deposit Transaction Execution:
The Deposit Process adds $500 to the balance and updates the account:
UPDATE account
SET balance = balance + 500
WHERE id = 123;
The deposit transaction is committed and the lock is released.
The Database confirms the transaction is successful, and the new balance is $1500.
Withdrawal Transaction Begins:
Now that the lock is released, the Withdrawal Process acquires the lock.It retrieves the updated balance:
SELECT balance FROM account WHERE id = 123;
The database returns $1500.
Withdrawal Transaction Execution:
The Withdrawal Process deducts $200 from the balance:
UPDATE account
SET balance = balance - 200
WHERE id = 123;
The withdrawal transaction is committed, and the final balance is $1300.
Transaction Completion
Both transactions were executed sequentially, ensuring data consistency.
The withdrawal had to wait for the deposit to complete before proceeding.
The pessimistic locking mechanism prevented race conditions and guaranteed that both transactions operated on the most up-to-date data.
Optimistic Locking
As the same suggests, is “optimistic” and says that multiple transactions can occur without affecting each other and no lock on any data.
In simple terms, Instead of locking, it checks whether the data to be updated is stale or not by verifying the value in a column such as version number or last updated timestamp. If the value has changed, indicating that another transaction has updated the record in the meantime, the transaction is withdrawn.
Example
Consider an order management where orders can have statuses like "Processing," "Shipped," or "Cancelled."Each order record includes a last_updated_datetime field that tracks the last modification time.
Scenario
Initial Read:
Two sales users (or processes) access the same order record, which currently has the status "Processing" and a last_updated_datetime of, say, 2025-02-10 10:00:00.Concurrent Updates
User A decides to update the order status from "Processing" to "Shipped."
User B simultaneously decides to update the order status from "Processing" to "Cancelled."
Update Process:
When User A submits the update, the system performs a query like this:
UPDATE orders
SET status = 'Shipped', last_updated_datetime = CURRENT_TIMESTAMP
WHERE order_id = :orderId AND last_updated_datetime = '2025-02-10 10:00:00';
If the order's last_updated_datetime is still 2025-02-10 10:00:00, the update succeeds. The last_updated_datetime is then updated to the current time (for example, 2025-02-10 10:05:00).
Conflict Detection:
When User B attempts their update, a similar query is executed:
UPDATE orders
SET status = 'Cancelled', last_updated_datetime = CURRENT_TIMESTAMP
WHERE order_id = :orderId AND last_updated_datetime = '2025-02-10 10:00:00';
However, since the last_updated_datetime no longer matches (it has been updated to 2025-02-10 10:05:00 by User A's successful update), this query affects zero rows. The system then detects that the record was modified after User B read it, indicating that the data is now stale.
Handling the Conflict:
At this point, the system can return an error message like:
"The order you attempted to update was modified by another user after you retrieved its details."User B's application can then prompt for a refresh or provide options to resolve the conflict.
Challenges of Distributed Locking:
While distributed locking solves critical issues in distributed systems, it also introduces its own set of challenges.
Deadlocks: A process holding a lock may crash before releasing the lock, then all the other processes waiting to acquire the lock will be blocked forever. To solve this problem, a common approach is to set a timeout for the lock.
Lock granularity issue: Lock granularity refers to the smallest unit that can be selected when locking a shared resource in a database. It is nothing but the amount of data occupied by a locked object. There are two types of locks
Coarse grained - Locks the entire table, uses a strategy to lock the entire table for a transaction. Though it has the advantage of simple implementation and low management overhead. If a transaction acquires the lock, all the other transactions cannot access the table, resulting in poor concurrency performance.
Fine grained - Locks specific rows, uses a strategy to lock the entire row for a transaction needing to access any data in a table. Though it can greatly improve concurrency performance because different transactions can access different rows in the table at the same time. However, resulting in large management overhead because a lock needs to be maintained for each row in a table.
To implement distributed locking effectively, developers must consider lock timeouts, heartbeat checks, and quorum-based lock acquisition to prevent system bottlenecks.
Bottom Line
Optimistic Locking: "Don't lock the table while reading."
Pessimistic Locking: "Lock the table while reading.”
Conclusion:
Distributed locks are an effective mechanism for coordination of concurrent processes running in a distributed environment. They ensure that only one process modifies a resource at a time, avoiding issues like race conditions and data inconsistencies. The right choice between pessimistic and optimistic locking depends on your use case and system architecture.