The OBKV-HBase client implements the Filter interface. This topic describes the compatibility of OBKV-HBase with open-source HBase filters.
Filter compatibility and examples
ColumnPrefixFilter
ColumnPrefixFilter is a column name prefix filter.
It returns all columns whose names start with the specified string. You must specify the prefix string to use for filtering.
Example:
Find all data whose column names start with abc.
byte[] prefix = Bytes.toBytes("abc");
ColumnPrefixFilter f = new ColumnPrefixFilter(prefix);
scan.setFilter(f);
ColumnPaginationFilter
ColumnPaginationFilter is a filter that limits the number of returned columns.
It returns the specified number of columns (cells of the latest version) starting from a specified column, for column pagination. You need to set the offset or specify the column (columnOffset) and the number to return (limit).
Example:
Return 3 columns of data starting from column q.
byte[] col = Bytes.toBytes("q");
ColumnPaginationFilter f = new ColumnPaginationFilter(3, col);
scan.setFilter(f);
FirstKeyOnlyFilter
FirstKeyOnlyFilter returns the first data cell of each row.
Example:
Return the first data cell of each row.
FirstKeyOnlyFilter f = new FirstKeyOnlyFilter();
scan.setFilter(f);
KeyOnlyFilter
KeyOnlyFilter returns all data except the values.
You can choose to specify one parameter or no parameter. By default, only the key of each row is returned, and the value is not returned. You can specify the lenAsVal parameter, which is a boolean value. When it is set to true, the length of the value is returned instead of the value.
Example:
Return the length of the value instead of the value.
KeyOnlyFilter f = new KeyOnlyFilter(true);
scan.setFilter(f);
RandomRowFilter
RandomRowFilter is a row filter that randomly selects a portion of the rows. It can be used to implement random sampling by determining whether to retain a row based on a specified probability.
Example:
Randomly filter data with a probability of 0.3.
RandomRowFilter f = new RandomRowFilter(0.3);
scan.setFilter(f);
TimestampsFilter
TimestampsFilter is a timestamp filter. It filters data by the given timestamps and retains only the data that matches.
It accepts two parameters. The first is a required list of timestamps, which provides the timestamps of the data to retain. The second is an optional boolean value, whose default value is false, which indicates whether SEEK_NEXT_USING_HINT can be used.
Example:
Return data with the timestamps 3 and 4.
ArrayList<Long> timestamps = new ArrayList<>();
timestamps.add(new Long(3));
timestamps.add(new Long(4));
TimestampsFilter filter = new TimestampsFilter(timestamps);
scan.setFilter(filter);
DependentColumnFilter
DependentColumnFilter is a dependent column filter used to filter columns that contain a specific value.
It requires at least two parameters: a column family and a column name. It finds the corresponding data cell in each row and returns all key-value pairs whose timestamps are the same as that of the data cell. You can also specify a third parameter, dropDependentColumn, which defaults to false. If it is set to true, the specified data cell is skipped and only other key-value pairs with the same timestamp are returned. In addition, you can specify the fourth and fifth parameters, CompareOperator and ByteArrayComparable, to compare and filter data cells.
Example:
Return all data cells whose column family is family_group, column name is column1, and value is value2, together with all other data cells with the same timestamp.
DependentColumnFilter filter = new DependentColumnFilter(
Bytes.toBytes("family_group"),
Bytes.toBytes("column1"),
false,
CompareFilter.CompareOp.EQUAL,
new BinaryComparator(toBytes("value2"))
);
scan.setFilter(filter);
SingleColumnValueExcludeFilter
SingleColumnValueExcludeFilter is a column value filter used to filter columns that contain a specific value.
Example:
Prepare the test dataset as follows:
RowKey: key1||||ColumnFamily: cf||||Column Qualifier: column1||||Value: value1||||Timestamp: 1723797655011
RowKey: key1||||ColumnFamily: cf||||Column Qualifier: column2||||Value: value2||||Timestamp: 1723797655270
RowKey: key2||||ColumnFamily: cf||||Column Qualifier: column2||||Value: value2||||Timestamp: 1723797655404
RowKey: row1||||ColumnFamily: cf||||Column Qualifier: column1||||Value: v1||||Timestamp: 1723537364927
RowKey: row2||||ColumnFamily: cf||||Column Qualifier: column1||||Value: ||||Timestamp: 1723537384827
Set the filter to SingleColumnValueFilter:
SingleColumnValueFilter filter = new SingleColumnValueFilter( "cf".getBytes(), "column2".getBytes(), CompareFilter.CompareOp.EQUAL, new BinaryComparator(toBytes("value2")) ); scan.setFilter(filter);The returned result is as follows:
RowKey: key1||||ColumnFamily: cf||||Column Qualifier: column1||||Value: value1||||Timestamp: 1723797632274 RowKey: key1||||ColumnFamily: cf||||Column Qualifier: column2||||Value: value2||||Timestamp: 1723797632504 RowKey: key2||||ColumnFamily: cf||||Column Qualifier: column2||||Value: value2||||Timestamp: 1723797632622 RowKey: row1||||ColumnFamily: cf||||Column Qualifier: column1||||Value: v1||||Timestamp: 1723537364927 RowKey: row2||||ColumnFamily: cf||||Column Qualifier: column1||||Value: ||||Timestamp: 1723537384827The result meets expectations.
Reset the filter to SingleColumnValueExcludeFilter:
SingleColumnValueExcludeFilter filter = new SingleColumnValueExcludeFilter( "cf".getBytes(), "column2".getBytes(), CompareFilter.CompareOp.EQUAL, new BinaryComparator(toBytes("value2")) ); scan.setFilter(filter);The returned result is as follows:
RowKey: key1||||ColumnFamily: cf||||Column Qualifier: column1||||Value: value1||||Timestamp: 1723797613418 RowKey: row1||||ColumnFamily: cf||||Column Qualifier: column1||||Value: v1||||Timestamp: 1723537364927 RowKey: row2||||ColumnFamily: cf||||Column Qualifier: column1||||Value: ||||Timestamp: 1723537384827The result still includes all the rows above but excludes the columns that match the specified value or comparison, which is as expected.
ColumnRangeFilter
ColumnRangeFilter is a column range filter that retains only the columns in a specified range.
The parameters specify the lower and upper bounds of the range and whether the bounds are inclusive or exclusive. If the lower or upper bound is NULL or an empty string, it represents infinity (that is, there is no lower or upper bound).
Example:
Return data in the column family cf with column names from column1 to column2, including column1 but excluding column2.
byte[] startColumn = Bytes.toBytes("column1");
byte[] endColumn = Bytes.toBytes("column2");
Filter f = new ColumnRangeFilter(startColumn, true, endColumn, false);
MultipleColumnPrefixFilter
MultipleColumnPrefixFilter is a multi-column prefix filter used to filter columns that have a specified prefix.
It can specify multiple column name prefixes, and the number of parameters is flexible.
Example:
Matches columns with the prefix column1 or column2.
byte[][] prefixes = new byte[][] {Bytes.toBytes("column1"), Bytes.toBytes("column2")};
Filter f = new MultipleColumnPrefixFilter(prefixes);
MultiRowRangeFilter
MultiRowRangeFilter is a multi-row range filter used to filter the rows in a specified range.
The input parameter for MultiRowRangeFilter is a list containing multiple RowRange objects. Each RowRange object defines an interval, including its upper and lower bounds, and whether the interval is open or closed. Alternatively, you can directly provide multiple prefixes as parameters; the HBase client will automatically convert these prefixes into RowRange objects for storage. Note that MultiRowRangeFilter requires at least one RowRange object as a parameter.
The following example shows how to use these filters:
Return data between row1 and row2 and between row3 and row4, including row1 but excluding the other rows.
List<MultiRowRangeFilter.RowRange> ranges = new ArrayList<MultiRowRangeFilter.RowRange>();
ranges.add(new MultiRowRangeFilter.RowRange("row3", false, "row4", false));
ranges.add(new MultiRowRangeFilter.RowRange("row1", true, "row2", false));
MultiRowRangeFilter f = new MultiRowRangeFilter(ranges);
InclusiveStopFilter
InclusiveStopFilter is a stop filter that returns all rows up to and including the specified row.
The following example shows how to use it:
Return all rows up to and including row1.
Filter f = new InclusiveStopFilter(Bytes.toBytes("row1"));
FirstKeyValueMatchingQualifiersFilter
FirstKeyValueMatchingQualifiersFilter is a filter that matches data cells with specific column names (qualifiers). It returns all data cells until it finds the first one that matches a specified column name. You can specify multiple column names as match criteria.
The following example shows how to use it:
Return each row of data until column0 or column1 is found.
TreeSet<byte []> set = new TreeSet<>(Bytes.BYTES_COMPARATOR);
set.add(Bytes.toBytes("column0"));
set.add(Bytes.toBytes("column1"));
Filter f = new FirstKeyValueMatchingQualifiersFilter(set);
FamilyFilter
FamilyFilter is a column family filter that filters data cells by a specific column family.
FamilyFilter filters data based on the column family name. You can specify a comparison operator (<, <=, =, >, >=, !=) and a comparator (the default is a binary comparator) to compare the column family names.
The following example shows how to use it:
Use FamilyFilter to filter out data whose column family name is cf1.
FamilyFilter f = new FamilyFilter(CompareOperator.NOT_EQUAL,
new BinaryComparator(Bytes.toBytes("cf1")));
FuzzyRowFilter
FuzzyRowFilter is a fuzzy row filter that filters data cells by a specific row key.
FuzzyRowFilter is a fuzzy match filter based on the row key. You can customize one or more match rules of the form (rowkey key, fuzzy info) to filter row keys. Here, fuzzy info is a byte array of the same length as the row key. For each byte in fuzzy info:
- If a byte in
fuzzy infois0, the corresponding byte of the row key must be the same as the given byte. - If a byte in
fuzzy infois1, the corresponding byte of the row key can be any value.
The following example shows how to use it:
Assume the row key format is userId_actionId_year_month, where userId occupies 4 bytes, actionId occupies 2 bytes, year occupies 4 bytes, and month occupies 2 bytes. To query the users who performed action 99 in January, use the following query:
List<Pair<byte[], byte[]>> data = new ArrayList<>();
byte[] fuzzyKey = "????_99_????_01".getBytes();
byte[] fuzzyInfo = new byte[] {1, 1, 1, 1, 0, 0, 0, 0, 1, 1, 1, 1, 0, 0, 0};
data.add(new Pair<byte[], byte[]>(fuzzyKey, fuzzyInfo));
FuzzyRowFilter f = new FuzzyRowFilter(data);
ColumnValueFilter
ColumnValueFilter is a column value filter used to filter columns that contain a specific value.
It filters the content of a specified column. It filters data cells by the comparison type and content that you specify and returns the cells that meet the criteria. The parameters specify the column family to filter, the column name, the comparison type, and the comparison content.
Here is an example:
Retrieve data with the column family name cf, column name column, and value value.
ColumnValueFilter filter = new ColumnValueFilter(
cf,
column,
CompareOperator.EQUAL,
Bytes.toBytes("value")
);
scan.setFilter(filter);