Before the start
After read the paper, each of us has selected a possible research topic. After discussion, we finally decide research on predictive techniques.We know that it can increase the access speed of database by saving data in memory.
The author’s idea
But the traditional H-Store architecture has limited on data recovery and parallel processing. The author of the paper put forward a new point of view. When the data capacity is larger than the available memory space, save the data to The hard disk. It can extracts data from the disk and merges it with the memory data when needed. His results prove that his view is perfect.
Our improvement and implement method
Our implementation will be based on the paper, memory will used by main storage device, when the memory is full, enable the hard disk to record data.But we will try to reduce the memory usage. When the memory capacity is enough, we adopt the same method as the author. When the memory capacity is not enough, we can reduce the volume of the data by predicting the user’s behaviors, so that more data can be saved on the memory.
Details
We specify three states of the system based on the remaining capacity of the memory. When the memory usage lower than 50%, all data can stored in the memory,I called that state 「normal」.
II When the memory usage is between 50% and 80%, 「Crowded」 state is implemented. The strategy of this state:
—1. When the user accesses a table more than 10% of all accesses behaviors, we called the table “warm”
—2. When the user accesses a table more than 30% of all accesses behaviors, we refer to this table as a “hot”
—3. When the user accesses a table more than 50% of all accesses behaviors, we refer to this table as a “burning”
III When the memory usage is above 80%, special strategies are implemented to lower the operating threshold.
—1. When the user accesses a table more than 10% of all accesses behaviors, we refer to this table as a “hot”
—2. When the user accesses a table more than 20% of all accesses behaviors, we refer to this table as a “burning”
One example
For each stage of the table, we try to find the rules of user』s behaviors.Most query or update statements are usually fixed. (E.g: password query from specific username.)
For example, we found that a lot of operations on the table user are:
Select password from 『user』 where username=』Tom』;
Select password from 『user』 where username=』Jack』;
Select password from 『user』 where username=』Lucy』;
We can define a general model:
Select password from 『user』 where username=』***』;
Hypothesis there are six tuples in the user table:
id,username,password,address,phonenumber,habit
A complete data can be defined as
01,Tom,123,No.23 JiLong RD,12345,reading
Since most of the time we use the username&password ,other elements is fewer to use, saving them in memory is obviously a waste of time.We save 40% of the data of warm table. The hot table and the of hot and burning table is 70% and 100%.
When we start compressing space,building the following data table and save it to your momery:
Key Result ResultId
Tom 123 #08787
Save the following table on your harddisk:
ResultAdd Result
#08787 Tom,123,No.23 JiLong RD,12345,reading
We avoid the waste of memory space by prediction instead of all data must being stored in the memory in the traditional H-Store method. Once the memory hot table is established, the whole table will be expelled to the hard disk. If we need to query more detail, just connect through the ResultId .
We need to design a software to implement our proposal.It must based on memory, can automatically optimize storage structure to improving the usability and effectiveness.Ensure maximum use of memory space.