Max Indelicato, a software development director and former chief software architect, recently published an article on how to design scalable web applications. He proposed to choose the right deployment and storage solutions, choose scalable data storage and patterns, and use the abstraction layer.
Tools suitable for work
Indelicato's first suggestion is "choose the right tool for work". To achieve this goal, you should choose one of the following architectural solutions:
- Deploy the solution using the cloud
- Use scalable data storage solutions, such as MongoDB, CouchDB, Cassandra, or Redis.
- Add a cache layer, such as Memcached.
He believes that these solutions are not necessary when developing applications, but it is wise to choose a scalable data storage solution at the beginning, because it will avoid switching later. Deploying applications to the cloud will bring us some benefits, especially for start-ups, because they cannot accurately determine how many people will use their applications after they are enabled. After the application is deployed to the cloud, when the demand increases, the application can be scaled in an elegant way. Many software architects have talked about the event that they have to extend the application, in which they will introduce the cache layer, which will solve most problems. But we should consider the corresponding solutions at the design stage. This will be easy to implement later.
Scalable data storage
Next, Indelicato recommends that you choose to support partition, replication and elastic data storage, including MongoDB Cassandra、Redis、Tokyo Cabinet、Project Voldemort， Or select MySQL as the relational database. This is necessary because partitioning is necessary in the life cycle of the application anyway. Partitioning is not required for scalability, but is required for "ensuring high availability". Flexibility allows us to quickly add more nodes. This may be when there is a traffic peak, or it may be "when the node needs to be maintained due to hardware failure or upgrade, large-scale scaling mode changes, or any need to offline the node."
Scalable data model
Indelicato suggests creating a pattern so that we can easily perform data sharding. He also gives the following examples of temporary components, User and UserFeedEntry:
Collection (or Table, or Entries, etc) User
{
UserId : guid, unique, key
Username : string
PasswordHash : string
LastModified : timestamp
Created : timestamp
}
Collection (or Table, or Entries, etc) UserFeedEntry
{
UserFeedEntryId : guid, unique, key
UserId : guid, unique, foreign key
Body : string
LastModified : timestamp
Created : timestamp
}
Then he suggested partitioning according to UserId:
By partitioning the User collection and UserFeedEntry collection according to the UserId field, we will place the two related data blocks on the same node. All UserFeedEntry data with UserId of xxx xxx xxx xxx and User data with UserId of xxx xxx xxx xxx xxx will be included in the same data fragment.
Why is this scalable? Because our requirements for this application are completely aimed at data distribution. When each visitor visits the user's information page, the system will send a request to the data fragment to obtain the user's details displayed in the User column, and then send a request to the same data fragment to obtain the user's UserFeedEntries. In these two requests, one will get one piece of data, and the other will get multiple pieces of data, which are all contained in the same data segment. Assuming that most users' information is accessed the same number of times in a day, we have designed a scalable model that will support the needs of our web applications.
Use abstraction layer
Indelicato's last suggestion is to use one of the following abstraction layers, but not limited to: Repository, cache and service. When creating the metabase layer, he suggested:
Don't name methods in a way that is specific to your abstract data store. For example, if you abstract a relational database, we generally define the Select(), Insert(), Delete(), and Update() functions to execute SQL queries and commands. Don't do that. Instead, you should make your function name less specialized. You can use Fetch(), Put(), Delete(), and Replace(). This will ensure that you are better able to follow the metabase schema, and when you need to switch the underlying database, the work will be easier.
If possible, use interfaces (or abstract classes, etc.) to pass these interfaces to higher levels in the application, so that you will never directly reference the specific inherent implementation of the metabase. This is also great for building and unit testing, because you can write other built-in implementations that come with data related to test cases in advance.
Encapsulate all special code for storage into a class (or module, etc.), and the real metabase will reference or inherit it. Only the details necessary for the access function (query statements, etc.) are placed in each function.
Keep in mind that not all metadatabases need to abstract the same data storage solution. If you want, you can store the User in MySQL and the UserFeedEntries in MongoDB. Metadatabases should be implemented in this way. They support this without paying too much. The previous three suggestions also indirectly help us to achieve this.
Indelicato said that for the cache layer, he often used "simple page (or view, etc.) level cache or service layer cache at the beginning, because these are two areas where state changes do not occur frequently."
Indelicato believes that the service layer needs to be sufficiently abstracted, so that when the demand increases, we can easily switch from the internal implementation of the service to the implementation outside the process.
Some people think that scalability should not be considered when building applications, because it will be emphasized when necessary. But if we want to consider scalability from the beginning, what good advice do you have?
