首页 » OceanBase » Oceanbase Memstore 使用分析

Oceanbase Memstore 使用分析

在业务租户内存结构中有个内存区叫做memstore, 主要用于保存数据库增量数据,对应MemTable属性,众所周知OB是基于LSM-tree存储引擎,数据分为磁盘上的静态数据(SSTable)、内存的增量数据(MEMTable)、以及记录事务的日志. 内存中的数据需要制定一些策略,控制从内存刷到磁盘,如同oracle的checkpoint, 在OB中叫做转储, 转储可能会产生I/O的较大影响,当然不希望业务高峰期做转储操作,这样就需要对MEMSTOR使用做监控,OB甚至可以控制Memstore中Flower占用memstore的比列,决定是否给leader副本使用,有memstore_percent属性决定。 这里记录memstore使用高时的分析方法。

memstore 统计信息使用查询
相关视图

  • oceanbase.gv$server_memstore
  • oceanbase.__all_virtual_server_memory_info
字段名称 类型 是否可以为 NULL 描述
IP varchar(32) NO 服务器的 IP。
PORT bigint(20) NO 服务器的端口。
ACTIVE bigint(20) NO 当前活跃 MEMStore 所占内存。
TOTAL bigint(20) NO MEMStore 一共使用了多少内存。
FREEZE_TRIGGER bigint(20) NO 触发冻结的阈值。
MEM_LIMIT bigint(20) NO MEMStore 的内存限制。

memstore 租户级信息查询
相关视图

    • oceanbase.gv$memstore
    • oceanbase.__all_virtual_tenant_memstore_info
    • GV$OB_MEMSTORE (v4)
select a.IP,a.tenant_id,b.tenant_name,round(active/1024/1024/1024,2) active,
round(total/1024/1024/1024,2) total,
round(freeze_trigger/1024/1024/1024,2) freeze_trigger,
round(total/freeze_trigger,2) "total/freeze_trigger",
round(mem_limit/1024/1024/1024,2) mem_limit,FREEZE_CNT
from oceanbase.gv$memstore a,oceanbase.__all_tenant b
 where a.tenant_id>1000 and a.tenant_id=b.tenant_id 
order by 1,2,3;

+---------------+-----------+-------------+--------+-------+----------------+----------------------+-----------+------------+
| IP            | tenant_id | tenant_name | active | total | freeze_trigger | total/freeze_trigger | mem_limit | FREEZE_CNT |
+---------------+-----------+-------------+--------+-------+----------------+----------------------+-----------+------------+
| 192.168.0.102 |      1002 | bj000001    |   3.16 |  3.16 |          12.50 |                 0.25 |     25.00 |          0 |
| 192.168.0.102 |      1003 | sh0001      |   8.41 |  8.41 |          47.50 |                 0.18 |     95.00 |          3 |
| 192.168.0.104 |      1002 | bj000001    |   3.16 |  3.16 |          12.50 |                 0.25 |     25.00 |          0 |
| 192.168.0.104 |      1003 | sh0001      |   1.51 | 33.14 |          47.50 |                 0.70 |     95.00 |          3 |
| 192.168.0.109 |      1002 | bj000001    |   3.39 |  3.39 |          12.50 |                 0.27 |     25.00 |          0 |
| 192.168.0.109 |      1003 | sh0001      |  46.81 | 46.82 |          47.50 |                 0.99 |     95.00 |          2 |
+---------------+-----------+-------------+--------+-------+----------------+----------------------+-----------+------------+
6 rows in set (0.020 sec)

memstore 上限由参数 memstore_limit_percentage 控制,它表示租户 memStore 最多占租户总内存上限的百分比。freeze_trigger_percentage 参数用于控制转储时机,它表示当 memstore 内存达到 memstore 上限的某个百分比时触发转储,调优参数 freeze_trigger_percentage 和 转储内部并发数,可以将租户的内存水位控制在一个合理的安全的范围内.

统计memstore中对象级信息

相关视图

  • oceanbase.gv$memstore_info
  • oceanbase.__all_virtual_memstore_info
  • GV$OB_MEMSTORE_INFO(V4)
MySQL [oceanbase]> select * from gv$memstore_info order by BTREE_ITEMS desc limit 10;
+-----------+---------------+------+------------------+--------------+---------+------------------+---------------------+---------------------+-----------+-------------+------------+-------------+
| TENANT_ID | IP            | PORT | TABLE_ID         | PARTITION_ID | VERSION | BASE_VERSION     | MULTI_VERSION_START | SNAPSHOT_VERSION    | IS_ACTIVE | USED        | HASH_ITEMS | BTREE_ITEMS |
+-----------+---------------+------+------------------+--------------+---------+------------------+---------------------+---------------------+-----------+-------------+------------+-------------+
|      1003 | 192.168.0.134 | 2882 | 1102810162725393 |            0 | 0-0-0   | 1715046633887238 |    1715046633887238 | 9223372036854775807 |         1 | 32526827520 |   91814016 |    92040035 |
|      1003 | 192.168.0.139 | 2882 | 1102810162725393 |            0 | 0-0-0   | 1715046684008873 |    1715046684008873 | 9223372036854775807 |         1 | 31998345216 |   90422400 |    90558651 |
|      1003 | 192.168.0.134 | 2882 | 1102810162863026 |            0 | 0-0-0   | 1715018411193594 |    1715018411193594 | 9223372036854775807 |         1 |  9646899200 |   19494784 |    19357086 |
|      1003 | 192.168.0.139 | 2882 | 1102810162863026 |            0 | 0-0-0   | 1715018411193594 |    1715018411193594 | 9223372036854775807 |         1 |  9655287808 |   19494784 |    19357086 |
...

使用table_id,partition_id关连具体的表或分区, 当转储时触发minor freeze,memtable会被冻结(is_active=0), 并生成新的memtable(is_active=1)。

识别对象名

select 
  (select tenant_name from __all_tenant where tenant_id=t.tenant_id) tenant_name,
  (select database_name from __all_virtual_database where tenant_id=t.tenant_id and database_id=t.database_id) database_name,
  TABLE_NAME,Table_type,index_type
 from __all_virtual_table t 
 where table_id=xxxx;

Table_type 的定义

  • 0:SYSTEM_TABLE
  • 1:SYSTEM_VIEW
  • 2:VIRTUAL_TABLE
  • 3:USER_TABLE
  • 4:USER_VIEW
  • 5:USER_INDEX
  • 6:TMP_TABLE // MySQL 模式的临时表
  • 7:MATERIALIZED_VIEW // 3.x 版本暂时不支持 MQT
  • 8:TMP_TABLE_ORA_SESS // Oracle 模式的临时表, session level
  • 9:TMP_TABLE_ORA_TRX // Oracle 模式的临时表, transaction level

查看表的转储记录

相关视图

  • oceanbase.gv$merge_info
  • oceanbase.__all_virtual_partition_sstable_merge_info
  • __all_merge_info
  • gv$minor_merge_info(租户级转储)
select * from __all_virtual_partition_sstable_merge_info
 where table_id = xxx and partition_id = xxx and svr_ip = "xxx" 
order by merge_finish_time desc limit 10; 

MySQL [oceanbase]> select * from gv$merge_info
 where table_id=xxx and svr_ip='192.168.0.102'
 order by START_TIME;

+---------------+----------+-----------+------------------+--------------+-------+-------------+------------------+----------------------------+----------------------------+-------------------+-----------+-----------------+
| SVR_IP        | SVR_PORT | TENANT_ID | TABLE_ID         | PARTITION_ID | TYPE  | ACTION      | VERSION          | START_TIME                 | END_TIME                   | MACRO_BLOCK_COUNT | REUSE_PCT | PARALLEL_DEGREE |
+---------------+----------+-----------+------------------+--------------+-------+-------------+------------------+----------------------------+----------------------------+-------------------+-----------+-----------------+
| 192.168.0.102 |     2882 |      1003 | 1102810162725393 |            0 | major | major merge |              319 | 2024-05-07 02:02:04.349306 | 2024-05-07 02:03:52.687201 |              8866 |     98.34 |              64 |
| 192.168.0.102 |     2882 |      1003 | 1102810162725393 |            0 | minor | mini merge  | 1715046621332603 | 2024-05-07 09:50:46.034553 | 2024-05-07 09:53:52.629192 |              1090 |      0.00 |               1 |
...

merge_percentage 列表示某分区的转储进度.

— over —

打赏

,

对不起,这篇文章暂时关闭评论。