One of my student is dealing with a particularly big data set and the program behaves weird when the connection is unstable - Probably because the readRecords() is not finished due to the absolute massive heap of data. She’s wondering if there’s a way for readRecords() to only request specific column of a data table so that the program would require less bandwidth and time to load.
So… requesting the entire list then sorting it locally is not what she’s looking for - as the goal is to reduce traffic.
readRecords() returns JSON objects. You cannot get part of a JSON object from the database.
What you can do is only request a few objects at a time. The second parameter to readRecords() is an object that filters by exact match. So let’s say you have a database with a color column. You could request a subset of records by passing in {color: “red”}. Then you can work with a subset of the data at one time. The more specific you can be about which records you need the smaller the dataset. You cannot of course do that with numeric data only enumerations.
Another technique that might be good for numeric data is with really large datasets there is usually a summary that is more usable. So as you are putting data into your database you can create or update a few summary records that you can recall and use later instead of trying to work with the detailed dataset.
Working with large data with limited computing power is a current topic that your student can think about and research online.
Thanks you so much! I will forward this along 