The original implementation of queries in OBKV-HBase uses a full-range scan strategy: it first scans the data of all column qualifiers (that is, column names) and then filters the results. This approach causes significant redundant scanning and processing overhead on the server side, particularly when a query on a wide row or a multi-version row involves only a few columns, in which case latency issues are especially pronounced.
This topic describes the query optimization feature in OBKV-HBase (specifically, optimization for GET operations), which helps improve query performance in scenarios involving wide rows or multi-version rows.
Feature description
This feature provides a parameter that enables query optimization for GET operations. When the parameter is enabled, queries are optimized on the server side by using OBKV-HBase's intelligent scanning technology, which quickly skips over wide or multi-version rows, thereby reducing query latency.
This feature is mainly intended for wide-row or hot key (many versions) scenarios. The optimization is most noticeable when the number of distinct qualifiers (that is, column names) plus the total number of versions of those qualifiers exceeds 1,000 in an HBase wide row.
Notice
After this parameter is enabled, the client only sends optimization suggestions to the server. The server evaluates whether to perform the optimization based on the actual situation. Therefore, the optimization may not take effect in all scenarios.
Limitations
This optimization feature takes effect only when all of the following conditions are met:
Limitation |
Description |
|---|---|
| Applies only to GET operations | SCAN operations are not supported. |
| MaxVersions = 1 | MaxVersions=1 is set at the table level, or get.setMaxVersions(1) is configured for the GET operation. |
| Non-time-series table | Time-series tables are not supported. |
| No filter specified | A filter cannot be specified in the query. |
| Reverse scan not supported | Reverse scan cannot be used in the query. |
| Local read | The server must perform local reads. In distributed scenarios, batch GET operations may be executed remotely, in which case the optimization does not take effect. |
This feature can be configured at the statement level or the global level. The two configuration methods support different data types, as described below:
- Statement-level configuration: You must set the parameter by using the
Get.setAttribute()method. The parameter type isbyte[]. Pass the byte array of the string"true"or"false"(for example,"true".getBytes()), and the server automatically parses it as a boolean value. - Global-level configuration: The value must be of the boolean type or a string that can be parsed as a boolean value (
"true"/"false").
Performance impact:
- This optimization is mainly intended for wide-row and multi-version scenarios. In other scenarios, the performance improvement may not be significant.
Parameters
This feature can be configured at the statement level or the global level:
Configuration level |
Constant name |
Configuration value |
Type |
Default value |
|---|---|---|---|---|
| Statement level | HBASE_HTABLE_HOTKEY_GET_OPTIMIZE_ENABLE |
hbase.htable.hotkey.get.optimize.enable |
byte[] (set by using Get.setAttribute()) |
false |
| Global level | HBASE_HTABLE_HOTKEY_GET_OPTIMIZE_ENABLE_GLOBAL |
hbase.htable.hotkey.get.optimize.enable.global |
boolean (set by using Configuration) |
false |
The priority rules are as follows:
- Statement-level priority: If a statement-level configuration is set by using
Get.setAttribute(), that value is used. - Global configuration as fallback: If no statement-level configuration is set, the global configuration is used.
- Default value: If neither level is configured, the default value
falseis used.
Usage
You can use the Get.setAttribute() method to set the optimization switch for an individual GET operation. This is suitable for scenarios that require fine-grained control.
The following is an example of a single GET operation:
Get get = new Get(Bytes.toBytes("rowkey"));
get.addColumn(family.getBytes(), column.getBytes());
// Enable the hot key GET optimization.
get.setAttribute(HBASE_HTABLE_HOTKEY_GET_OPTIMIZE_ENABLE, "true".getBytes());
// Or disable the optimization.
// get.setAttribute(HBASE_HTABLE_HOTKEY_GET_OPTIMIZE_ENABLE, "false".getBytes());
Result result = hTable.get(get);
The following is an example of a batch GET operation:
List<Get> gets = new ArrayList<>();
for (String key : keys) {
Get get = new Get(Bytes.toBytes(key));
get.addColumn(family.getBytes(), column.getBytes());
get.setAttribute(HBASE_HTABLE_HOTKEY_GET_OPTIMIZE_ENABLE, "true".getBytes());
gets.add(get);
}
Result[] results = hTable.get(gets);
You can use the Configuration object to set the global optimization switch. This is suitable for scenarios that require unified control.
Configuration configuration = ObHTableTestUtil.newConfiguration();
// Enable GET query optimization globally.
configuration.setBoolean(HBASE_HTABLE_HOTKEY_GET_OPTIMIZE_ENABLE_GLOBAL, true);
OHTable hTable = new OHTable(configuration, "table_name");
// At this point, all GET and SCAN operations without a statement-level configuration use the global setting.
Get get = new Get(Bytes.toBytes("rowkey"));
Result result = hTable.get(get); // The globally configured value true is automatically used.
A statement-level configuration overrides the global configuration. This is suitable for scenarios where most operations require optimization but a few need optimization disabled.
Configuration configuration = ObHTableTestUtil.newConfiguration();
// Enable the optimization globally.
configuration.setBoolean(HBASE_HTABLE_HOTKEY_GET_OPTIMIZE_ENABLE_GLOBAL, true);
OHTable hTable = new OHTable(configuration, "table_name");
// Get1: No statement-level configuration is set, so the global configuration (true) is used.
Get get1 = new Get(Bytes.toBytes("rowkey1"));
Result result1 = hTable.get(get1);
// Get2: The statement-level configuration is set to false, which overrides the global configuration.
Get get2 = new Get(Bytes.toBytes("rowkey2"));
get2.setAttribute(HBASE_HTABLE_HOTKEY_GET_OPTIMIZE_ENABLE, "false".getBytes());
Result result2 = hTable.get(get2); // The value false is used instead of the global value true.