Multi-metric data-based hierarchical storage strategy
The storage method used for hierarchical storage and the selected storage device are based on the importance of data, frequency of access, and other indicators. The multi-index data information grading strategy refers to grading the value of data according to a plurality of parameters based on the life cycle of the data, the last access time, the size, and the relevance of the data information.
If the data can be predicted as soon as it is created, and then given the appropriate level, it will reduce unnecessary migration bumps. Because the data grading changes means that the data should be migrated between different levels of storage devices to ensure that the right data is stored at the right storage level at the right time.
In practical applications, the classification based on the static features of the data and the dynamic characteristics of the access can be used to obtain better results. For example, in the case of file classification, first, the static characteristics of the file system, such as the distribution of the size of the file; second, the macro access rules of the file system, such as the distribution of the number of visits to the size of the file; third, the individual file access mode, such as Whether it has access locality; Fourth, the access-association feature between files, such as when a file in the same job is accessed and when another file is accessed.
Based on the characteristics of these files and the classification of the storage devices, the triggering conditions for changing the file classification criteria and the file classification are determined, so that the appropriate files can be stored at an appropriate storage level at a suitable time.
Consistency guarantee technology in online migration
In a hierarchical storage system, data migration is inevitable on different storage devices. Data migration can be divided into upgrade migration and downgrade migration. Upgrade migration means that data is migrated from a slow storage device and a low-level storage device to a fast storage device or a higher-level storage device. The degraded migration is just the opposite. However, due to the different purposes of migration, the two migrations have different characteristics.
For degraded migrations, it is very likely that no I/O requests will occur during the migration process; but for upgrade migrations, migration occurs almost at the time of the most intensive I/O. How to ensure that during the migration process, Minimizing the impact of the migration process on foreground I/O is one of the issues that tiered storage systems need to address.
Currently, read-write locks can be used to ensure data consistency, and data blocks are used as scheduling granularities to reduce the impact on foreground I/O performance. The migration process applies read-write locks for the current data block to ensure data consistency between the migration process and the write process.
Automatic data migration storage technology
Data in hierarchical storage needs to be migrated online. This requires consideration of the performance impact of data movement on foreground I/O load. Automatic data migration technology refers to minimizing the impact of data migration actions on the I/O performance of the compute nodes and is transparent to the front end. It adjusts the data migration rate according to the change in front-end I/O load to make data migration actions possible. The impact on the QoS of the storage system itself is very small, and at the same time, the data migration task can be completed as soon as possible. The main technologies involved in automatic data migration and storage are: rate control and scheduling of data migration, latency hiding of data migration to applications, prediction of file access block position sequence, and the like.
In practical applications, when the data information reaches the trigger condition for migration, the data migration node automatically migrates the data, thereby realizing the degraded or upgraded storage of the data information. For example, the data is migrated and migrated, and the user's delayed hiding technology prevents the application from experiencing data upgrade. The delay in migration and the performance of the entire storage system is comparable to that of the highest performance storage devices.
Currently, hierarchical storage management solves data storage problems from the perspective of reducing costs, not affecting data application efficiency, and improving efficiency. With the decline in the cost of a single disk.
Industry experts predict that in the near future, near-line storage technology will replace data migration technology, and users will save historical data in a more secure and reliable disk media that simulates massive volumes of space. The mature and perfect data migration technology will be better integrated into near-line storage devices and play a key role in promoting the development of tiered storage.
The storage method used for hierarchical storage and the selected storage device are based on the importance of data, frequency of access, and other indicators. The multi-index data information grading strategy refers to grading the value of data according to a plurality of parameters based on the life cycle of the data, the last access time, the size, and the relevance of the data information.
If the data can be predicted as soon as it is created, and then given the appropriate level, it will reduce unnecessary migration bumps. Because the data grading changes means that the data should be migrated between different levels of storage devices to ensure that the right data is stored at the right storage level at the right time.
In practical applications, the classification based on the static features of the data and the dynamic characteristics of the access can be used to obtain better results. For example, in the case of file classification, first, the static characteristics of the file system, such as the distribution of the size of the file; second, the macro access rules of the file system, such as the distribution of the number of visits to the size of the file; third, the individual file access mode, such as Whether it has access locality; Fourth, the access-association feature between files, such as when a file in the same job is accessed and when another file is accessed.
Based on the characteristics of these files and the classification of the storage devices, the triggering conditions for changing the file classification criteria and the file classification are determined, so that the appropriate files can be stored at an appropriate storage level at a suitable time.
Consistency guarantee technology in online migration
In a hierarchical storage system, data migration is inevitable on different storage devices. Data migration can be divided into upgrade migration and downgrade migration. Upgrade migration means that data is migrated from a slow storage device and a low-level storage device to a fast storage device or a higher-level storage device. The degraded migration is just the opposite. However, due to the different purposes of migration, the two migrations have different characteristics.
For degraded migrations, it is very likely that no I/O requests will occur during the migration process; but for upgrade migrations, migration occurs almost at the time of the most intensive I/O. How to ensure that during the migration process, Minimizing the impact of the migration process on foreground I/O is one of the issues that tiered storage systems need to address.
Currently, read-write locks can be used to ensure data consistency, and data blocks are used as scheduling granularities to reduce the impact on foreground I/O performance. The migration process applies read-write locks for the current data block to ensure data consistency between the migration process and the write process.
Automatic data migration storage technology
Data in hierarchical storage needs to be migrated online. This requires consideration of the performance impact of data movement on foreground I/O load. Automatic data migration technology refers to minimizing the impact of data migration actions on the I/O performance of the compute nodes and is transparent to the front end. It adjusts the data migration rate according to the change in front-end I/O load to make data migration actions possible. The impact on the QoS of the storage system itself is very small, and at the same time, the data migration task can be completed as soon as possible. The main technologies involved in automatic data migration and storage are: rate control and scheduling of data migration, latency hiding of data migration to applications, prediction of file access block position sequence, and the like.
In practical applications, when the data information reaches the trigger condition for migration, the data migration node automatically migrates the data, thereby realizing the degraded or upgraded storage of the data information. For example, the data is migrated and migrated, and the user's delayed hiding technology prevents the application from experiencing data upgrade. The delay in migration and the performance of the entire storage system is comparable to that of the highest performance storage devices.
Currently, hierarchical storage management solves data storage problems from the perspective of reducing costs, not affecting data application efficiency, and improving efficiency. With the decline in the cost of a single disk.
Industry experts predict that in the near future, near-line storage technology will replace data migration technology, and users will save historical data in a more secure and reliable disk media that simulates massive volumes of space. The mature and perfect data migration technology will be better integrated into near-line storage devices and play a key role in promoting the development of tiered storage.
linear pendant lighting,linear light fixture,linear wall sconce,recessed linear light
HSONG LIGHTING CO,. LTD , https://www.wallwasherlighting.com