Bulky Data Transfer with Single Hit vs Light Data Transfer with Multiple Hits

This is a classic dilemma. What is a better model for browser based clients and back-end services to interact? Should there be as little dialogue as possible with maximum data transfer or as much dialogue as possible with minimum data transfer in each exchange of request-response. The problem is not as simple to solve as the problem statement makes it sound.

There are various parameters that need to be considered for answering this question. And the solution varies as per the requirements.

Let’s start with considering a revolutionary example: Gmail!

As we are aware, Google employed the AJAX philosophy and created the first ever AJAX based email application that changed the paradigm of web mails, by increasing the performance of reading, composing and sending emails over the internet. The browsers were same as before, the bandwidth was the same but the application architecture had changed drastically, making more possible with the same set of resources.

However, it was not just about using a new technology but identifying the appropriate problem that this technology could solve. In an email type web application, a user generally performs units of tasks with every click.

  • Open a mail for reading,
  • Compose a mail/reply,
  • Add / Remove the attachments and
  • Send the mail.

It never happens that a user is reading a mail that is being modified by the sender simultaneously. So, there is no real time change in an email’s content.

If at all there is a modification required to the content of an email that was already sent/delivered, one needs to re-send the mail with the modifications. In this scenario, fetching each mail’s content as and when the header of the mail is clicked makes sense, and doing it asynchronously (i.e. without refreshing the whole page) makes the usability far more intuitive and elegant.

On the other hand, is an application that shows data that is highly likely to be modified at the same time when a user is viewing the information, such a model of conversation may not really work.

Say, you have an application that displays data in the grid where rows and columns are collapsible and contain levels of information (something like a hierarchical data set). See the example.

The grid’s cells don’t enjoy absolute independence from the other rows and columns of data surrounding it. In other words, each cell of information is not just an atomic information in itself, but also forms a part of the information that is at a level higher than it. For instance, Physical Supply of 100 could mean there is a PO of 50, POK of 30 and TO of 20, which are all displayed as the child levels of Physical Supply. Now, in case there was a new PO of 20 created by some user, while the data mentioned above is on display in the SCP, by showing an additional 20 for PO i.e. PO of 70 would not be enough to represent the actual state of Supply Chain because, the sum of quantities for different Documents (70+30+20 = 120) would be incosistent with the total being shown for the Physical Supply (50+30+20 = 100).

With this problem at hand the requirement would be to update all the cells in the row and column that contain the modified cell, as part of calculating the summary for that row and column respectively. This requirement in turn requires us to keep a track of all the cells in the grid because the probability of any cell getting modified at any given time is almost equal, therefore any row and column could be required to be refreshed with the latest data.

Essentially, each cell update leads to an update of almost all the cells in that section (sections here refer to the parent rows viz. Demand, Suppply, Recommendations etc.) We could therefore limit the updates of a cell leading to the update of just the section in which it lies. This in turn would lead to a different kind of inconsistency. For instnace, a Demand of 30 and a Supply or 20 leads to a Recommendation of 10 and a Projected Inventory of -10. This means that even the sections within the SCP grid are interdependent therefore any cell that has to show updated data, would require all the other cells in the grid to update the data, directly or indirectly.

So, it is pretty clear that the data in the entire grid has to be treated as a whole in order to view a consistent information in the Supply Chain Profile at any given time. The rendering of data, however, doesn’t need to happen in a single shot because like viewing a section or a particular row or column or a cell in particular, is quite similar to checking one’s inbox for a specific email. And once that email is located, all one cares about is reading the information within. Similarly, a user will only follow a single path for drilling down information about the SCP at any given time. This in turn leads to a simple requirement of rendering data (cells, rows, columns) only for the particular paths.

The conclusion or the solution for our problem is to fetch the entire data in a Single Hit to the server, and then use selective and efficient rendering of the data based on where user clicks on the UI. We can and have tried to make the cell specific information (which is shown in the pop-ups) asynchronous, but that leads to holding the state of the data object that was transported to the client earlier. This leads to additional memory and state management requirements on the back end. Therefore, we chose to send the information in a bulk to the front end. This might be slightly slower but it represents the state of SCP accurately and consistently always and everytime.