Designing a multi service architecture might give you some headaches when it comes to concurrency. In this post I will describe the most famous technical concurrency models and propose a different view on them, so we can design better our systems.
Follow bellow these concurrency models, their benefits and challenges.
This concurrency model proposes that no one can change an object while it is being held by a process, meaning that all subsequent transactions will wait on the line until they get to their job.
This approach in some cases is bad because you hold your data, cannot scale, and sometimes happens that a given process decides to go on holidays while holding the line for other processes, forcing someone else manually solving the issue by killing the transaction.
This model proposes that given two or more transactions on the same object, either the first one wins or the last one wins. There is a down side on this model that in any cases somebody will lose data.
There are two strategies for the optimistic concurrency model:
- First one wins, meaning that the second one gets the data rejected. Unless you have a retry strategy (mentioned here), the second operation loose data.
- Last one wins, by default the data is overwritten by the second one. This means that the first had its data lost. A difference here is that the first operation doesn’t know that the data was lost. This is problematic because you are not able to decide on what to do in such cases.
All of the above concurrency models have its own challenges, but we need a model focused on the business that deals with both worlds: optimism and pessimism.
In order to you understand this proposal, let me first show you a transaction example:
1: Begin Transaction
2: Read Entity A
3: Read Entity B
4: Update Entity C with data from A and B
5: Commit Transaction
Looking at this example we see that once we read data, there is no way we can guarantee, in a scalable system, that the state of C will be fully consistent with A and B, because A and B might be in a total different state that doesn’t align with C at the end of the transaction.
The remaining question is: Does this inconsistency matters to the business ?
Most of the times we assume that if we put transactions all around, then everything will be consistent. This is given to the fact that as ORM’s were hugely introduced in our softwares, developers started saying:
Ah! that’s fine, I don’t need to worry about the database, the ORM will take care of it for me. Let me focus on the business!
Which is not absolutely true. In fact, we need to worry about the business, and if you are in a collaborative domain, where somebody can change the state of the entity that you are about to use, then you need to think in concurrency, which is part of your business and very much related to how your database works.
So, in order to have a business driven concurrency model, first rule:
- Consciously choose your technical concurrency model (optimistic or pessimistic).
But this is not enough, you still need to code in such way that you save your business from the issues that every choice might give to you.
And the way to address this issue you have to split your transaction in little pieces of transactions, doing one thing at a time with one piece of data at a time.
1: Receive a message (command/event)
1: Get a Domain Object
2: Change the state of the Domain Object
3: Commit your transaction (possibly sending domain events)
It is really important to not touch multiple domain objects, because every time you access another domain object within a single transaction, more probability to have concurrency issues is added. If you need to change more state across your domain objects, you might want to have a look at domain events.
In essence: Relationship between your domain objects, will give you concurrency issues. So, really think carefully on your service boundaries, because they will move apart unnecessary domain object relationships and by design, most of your challenges will be gone.
This approach save us in several cases, but in a higher level there is another case in which we need to think further: Sometimes the business requires that in a given process only one transaction wins, but not both. Here is an example:
In a given collaborative domain where a customer placed an order, and your business want to be proactive on shipping by having multiple shipping providers in case of delay. The following racing condition can happen:
1: User places an order
2: Request shipping to provider #1
4: Provider #1 is taking too long, request shipping to provider #2
5: Provider #2 answer right away that order is shipped
6: Request cancel shipping to provider #1
7: At the same time, provider #1 answers that the order is shipped
In this scenario you have a racing condition and both providers will delivery this order. There is no way you can run from this concurrency issue without finding a business reason and address it.
There are a few possible solutions that you might take:
- Deliver both and assume the loss, considering that your business wants to work proactively on shipping.
- Ask the consumer to return one of the products, by giving an incentive in future purchases, reducing the costs of the issue.
- And this list could go on forever…
Considering the list above, you can notice that we were talking about a technical racing condition between requests, and now that we tried to find a business reason, we started talking about: returning policy. Based on this, the domain expert is able to decide what rule to follow in such scenarios.
If I’d give you a suggestion, not everything is solved by a technical decision. Sometimes you need to find a business reason for the issue and address it as part of your business.
If you are in a single user system, you don’t need to worry about it, but must scenarios, this is not the case.
Finally, don’t underestimate concurrency on you software, because there’s business hidden on it.