74 Star 358 Fork 217

GVPopenEuler / A-Tune

 / 详情

examples下mariadb调优失败

已完成
缺陷
创建于  
2024-03-19 09:41

【标题描述】能够简要描述问题:说明什么场景下,做了什么操作,出现什么问题(尽量使用正向表达方式)
examples下mariadb调优失败,
一、缺陷信息
2.start to tuning the system...... 之后一直没有后续显示
内核信息:

缺陷归属组件:

缺陷归属的版本:

缺陷简述:

【环境信息】
硬件信息
vmware上搭建的21.10系统

atuned状态

评论 (3)

dongchao 创建了缺陷

问题可以复现。
1) 针对需要密码的问题
在 mariadb/my.cnf 文件末尾新增一行。 skip-grant-tables 可以跳过密码部分
执行:

cp my.cnf /etc/
systemctl restart mariadb
systemctl restart atuned
systemctl restart atune-engine
systemctl restart atune-rest

2) 调优命令: atune-adm tuning --project mariadb --detail mariadb_client.yaml, 这个命令执行时间比较长。需要足够的时间等待。
可以通过一下命令快速参看是否能获取得到对应的 tpmc
2.1)tpcc_start -h 127.0.0.1 -d tpcctest -u root -w 10 -c 5 -r 10 -l 60 >> tpcc.log
tpcc.log 如果最后的 tpmc 非 nan 就是对的, 有可能是 nan
输入图片说明

2.2) 如果 tpmc 为 nan
1) 检查一下磁盘空间是否已满 df -h 看看 “/” 占用了多少
2) 进去 mariadb/tpcc-mysql, 删除 “rm *.out” 所有 out 文件。
3) 重新生成一遍 sh prepare.sh "xx" 然后继续去 mariadb/tpcc-mysql 看数据是否加载成功。
loading data, please waiting ... 如果一直卡着 retring, 那就是磁盘空间已满
ERROR 1698 (28000): Access denied for user 'root'@'localhost' 这个是 mysql 有默认密码, 需要清空密码。

最后
问题修复情况如下图
输入图片说明

关于卡死再 random 轮次后的问题, 这里提供一个可行的配置。具体配置可以根据自己的机器进行适当调整
更改 mariadb_server.yaml
问题原因是: 生成的参数空间太多, 导致机器内存 OOM, 进程卡死。
代码挂死位置:(原因还是因为参数量大, 步长太短, 生成的数据太多, 这里算法又抽样了,导致的内存超限)
输入图片说明
这里面有一些参数需要根据自己的机器进行适当的调整。
这里提供一份可行的 mariadb_server.yaml 参数配置, 比如:

project: "mariadb"
maxiterations: 100
startworkload: "systemctl start mariadb"
stopworkload: "systemctl stop mariadb"
object : 
  -
    name : "mariadb.key_buffer_size"
    info :
        desc : "Index parameters of the myisam storage engine"
        get : "cat /etc/my.cnf | grep key_buffer_size | awk -F '=' '{print $2}' | awk '$1=$1'"
        set : "sed -i 's/key_buffer_size.*/key_buffer_size = $value/g' /etc/my.cnf"
        needrestart : "true"
        type : "discrete"
        scope :
          - 1048576
          - 536870912
        step : 1048576
        items : 
        dtype : "int"
  -
    name : "mariadb.max_allowed_packet"
    info :
        desc : "Maximum number of received packets"
        get : "cat /etc/my.cnf | grep max_allowed_packet | awk -F '=' '{print $2}' | awk '$1=$1'"
        set : "sed -i 's/max_allowed_packet.*/max_allowed_packet = $value/g' /etc/my.cnf"
        needrestart : "true"
        type : "discrete"
        scope :
          - 1048576
          - 104857600
        step : 1048576
        items : 
        dtype : "int"
  -
    name : "mariadb.table_open_cache"
    info :
        desc : "Table cache for storing data"
        get : "cat /etc/my.cnf | grep table_open_cache | awk -F '=' '{print $2}' | awk '$1=$1'"
        set : "sed -i 's/table_open_cache.*/table_open_cache = $value/g' /etc/my.cnf"
        needrestart : "true"
        type : "discrete"
        scope :
          - 16
          - 1000000
        step : 2
        items : 
        dtype : "int"
  -
    name : "mariadb.back_log"
    info :
        desc : "The number of new requests stored in the stack"
        get : "cat /etc/my.cnf | grep back_log | awk -F '=' '{print $2}' | awk '$1=$1'"
        set : "sed -i 's/back_log.*/back_log = $value/g' /etc/my.cnf"
        needrestart : "true"
        type : "continuous"
        scope :
          - 16
          - 65536
        dtype : "int"
  -
    name : "mariadb.sort_buffer_size"
    info :
        desc : "Cache used for sorting"
        get : "cat /etc/my.cnf | grep sort_buffer_size | awk -F '=' '{print $2}' | awk '$1=$1'"
        set : "sed -i 's/sort_buffer_size.*/sort_buffer_size = $value/g' /etc/my.cnf"
        needrestart : "true"
        type : "discrete"
        scope :
          - 256
          - 104857600
        step : 1024
        items : 
        dtype : "int"
  -
    name : "mariadb.read_buffer_size"
    info :
        desc : "the buffer allocated to each thread during sequential table scanning."
        get : "cat /etc/my.cnf | grep read_buffer_size | awk -F '=' '{print $2}' | awk '$1=$1'"
        set : "sed -i 's/read_buffer_size.*/read_buffer_size = $value/g' /etc/my.cnf"
        needrestart : "true"
        type : "discrete"
        scope :
          - 1024
          - 104857600
        step : 1024
        items : 
        dtype : "int"
  -
    name : "mariadb.read_rnd_buffer_size"
    info :
        desc : "the buffer allocated to each thread when the table is read randomly"
        get : "cat /etc/my.cnf | grep read_rnd_buffer_size | awk -F '=' '{print $2}' | awk '$1=$1'"
        set : "sed -i 's/read_rnd_buffer_size.*/read_rnd_buffer_size = $value/g' /etc/my.cnf"
        needrestart : "true"
        type : "discrete"
        scope :
          - 1024
          - 104857600
        step : 1024
        items : 
        dtype : "int"
  -
    name : "mariadb.myisam_sort_buffer_size"
    info :
        desc : "the buffer required for reordering when the MyISAM table changes"
        get : "cat /etc/my.cnf | grep myisam_sort_buffer_size | awk -F '=' '{print $2}' | awk '$1=$1'"
        set : "sed -i 's/myisam_sort_buffer_size.*/myisam_sort_buffer_size = $value/g' /etc/my.cnf"
        needrestart : "true"
        type : "discrete"
        scope :
          - 1024
          - 104857600
        step : 1024
        items : 
        dtype : "int"
  -
    name : "mariadb.thread_cache_size"
    info :
        desc : "Number of threads saved in the cache that are reused"
        get : "cat /etc/my.cnf | grep thread_cache_size | awk -F '=' '{print $2}' | awk '$1=$1'"
        set : "sed -i 's/thread_cache_size.*/thread_cache_size = $value/g' /etc/my.cnf"
        needrestart : "true"
        type : "continuous"
        scope :
          - 8
          - 1000
        dtype : "int"
  -
    name : "mariadb.max_connections"
    info :
        desc : "the max number of connections"
        get : "cat /etc/my.cnf | grep max_connections | awk -F '=' '{print $2}' | awk '$1=$1'"
        set : "sed -i 's/max_connections.*/max_connections = $value/g' /etc/my.cnf"
        needrestart : "true"
        type : "continuous"
        scope :
          - 10
          - 65536
        dtype : "int"
  -
    name : "mariadb.max_heap_table_size"
    info :
        desc : "size of a memory table that can be created"
        get : "cat /etc/my.cnf | grep max_heap_table_size | awk -F '=' '{print $2}' | awk '$1=$1'"
        set : "sed -i 's/max_heap_table_size.*/max_heap_table_size = $value/g' /etc/my.cnf"
        needrestart : "true"
        type : "discrete"
        scope :
          - 1024
          - 104857600
        step : 1024
        items : 
        dtype : "int"
  -
    name : "mariadb.innodb_buffer_pool_size"
    info :
        desc : "size of innodb buffer pool"
        get : "cat /etc/my.cnf | grep innodb_buffer_pool_size | awk -F '=' '{print $2}' | awk '$1=$1'"
        set : "sed -i 's/innodb_buffer_pool_size.*/innodb_buffer_pool_size = $value/g' /etc/my.cnf"
        needrestart : "true"
        type : "discrete"
        scope :
          - 1024
          - 1.37E+11
        step : 1024
        items : 
        dtype : "int"
  -
    name : "mariadb.innodb_log_buffer_size"
    info :
        desc : "size of innodb log buffer"
        get : "cat /etc/my.cnf | grep innodb_log_buffer_size | awk -F '=' '{print $2}' | awk '$1=$1'"
        set : "sed -i 's/innodb_log_buffer_size.*/innodb_log_buffer_size = $value/g' /etc/my.cnf"
        needrestart : "true"
        type : "discrete"
        scope :
          - 1048576
          - 104857600
        step : 1048576
        items : 
        dtype : "int"
  -
    name : "vm.swappiness"
    info :
        desc : "A larger value indicates that the swap partition is used more actively. A smaller value indicates that the memory is used more actively."
        get : "sysctl -n vm.swappiness"
        set : "sysctl -w vm.swappiness=$value"
        needrestart : "false"
        type : "discrete"
        scope :
          - 0
          - 100
        step : 1
        items : 
        dtype : "int"

mariadb_server.yaml 参考配置如下:

project: "mariadb"
maxiterations: 100
startworkload: "systemctl start mariadb"
stopworkload: "systemctl stop mariadb"
object :
  -
    name : "mariadb.key_buffer_size"
    info :
        desc : "Index parameters of the myisam storage engine"
        get : "cat /etc/my.cnf | grep key_buffer_size | awk -F '=' '{print $2}' | awk '$1=$1'"
        set : "sed -i 's/key_buffer_size.*/key_buffer_size = $value/g' /etc/my.cnf"
        needrestart : "true"
        type : "discrete"
        scope :
          - 1048576
          - 536870912
        step : 1048576
        items :
        dtype : "int"
  -
    name : "mariadb.max_allowed_packet"
    info :
        desc : "Maximum number of received packets"
        get : "cat /etc/my.cnf | grep max_allowed_packet | awk -F '=' '{print $2}' | awk '$1=$1'"
        set : "sed -i 's/max_allowed_packet.*/max_allowed_packet = $value/g' /etc/my.cnf"
        needrestart : "true"
        type : "discrete"
        scope :
          - 1048576
          - 104857600
        step : 1048576
        items :
        dtype : "int"
  -
    name : "mariadb.back_log"
    info :
        desc : "The number of new requests stored in the stack"
        get : "cat /etc/my.cnf | grep back_log | awk -F '=' '{print $2}' | awk '$1=$1'"
        set : "sed -i 's/back_log.*/back_log = $value/g' /etc/my.cnf"
        needrestart : "true"
        type : "continuous"
        scope :
          - 16
          - 65536
        dtype : "int"
  -
    name : "mariadb.sort_buffer_size"
    info :
        desc : "Cache used for sorting"
        get : "cat /etc/my.cnf | grep sort_buffer_size | awk -F '=' '{print $2}' | awk '$1=$1'"
        set : "sed -i 's/sort_buffer_size.*/sort_buffer_size = $value/g' /etc/my.cnf"
        needrestart : "true"
        type : "discrete"
        scope :
          - 256
          - 104857600
        step : 10240
        items :
        dtype : "int"
  -
    name : "mariadb.read_buffer_size"
    info :
        desc : "the buffer allocated to each thread during sequential table scanning."
        get : "cat /etc/my.cnf | grep read_buffer_size | awk -F '=' '{print $2}' | awk '$1=$1'"
        set : "sed -i 's/read_buffer_size.*/read_buffer_size = $value/g' /etc/my.cnf"
        needrestart : "true"
        type : "discrete"
        scope :
          - 1024
          - 104857600
        step : 10240
        items :
        dtype : "int"
  - name: "vm.swappiness"
    info:
      desc: "A larger value indicates that the swap partition is used more actively. A smaller value indicates that the memory is used more actively."
      get: "sysctl -n vm.swappiness"
      set: "sysctl -w vm.swappiness=$value"
      needrestart: "false"
      type: "discrete"
      scope:
        - 0
        - 100
      step: 1
      items:
      dtype: "int"
  -
    name : "mariadb.max_connections"
    info :
        desc : "the max number of connections"
        get : "cat /etc/my.cnf | grep max_connections | awk -F '=' '{print $2}' | awk '$1=$1'"
        set : "sed -i 's/max_connections.*/max_connections = $value/g' /etc/my.cnf"
        needrestart : "true"
        type : "continuous"
        scope :
          - 10
          - 65536
        dtype : "int"
  -
    name : "mariadb.innodb_log_buffer_size"
    info :
        desc : "size of innodb log buffer"
        get : "cat /etc/my.cnf | grep innodb_log_buffer_size | awk -F '=' '{print $2}' | awk '$1=$1'"
        set : "sed -i 's/innodb_log_buffer_size.*/innodb_log_buffer_size = $value/g' /etc/my.cnf"
        needrestart : "true"
        type : "discrete"
        scope :
          - 1048576
          - 104857600
        step : 1048576
        items :
        dtype : "int"
  -
    name : "mariadb.thread_cache_size"
    info :
        desc : "Number of threads saved in the cache that are reused"
        get : "cat /etc/my.cnf | grep thread_cache_size | awk -F '=' '{print $2}' | awk '$1=$1'"
        set : "sed -i 's/thread_cache_size.*/thread_cache_size = $value/g' /etc/my.cnf"
        needrestart : "true"
        type : "continuous"
        scope :
          - 8
          - 1000
        dtype : "int"
  -
    name : "mariadb.innodb_buffer_pool_size"
    info :
        desc : "size of innodb buffer pool"
        get : "cat /etc/my.cnf | grep innodb_buffer_pool_size | awk -F '=' '{print $2}' | awk '$1=$1'"
        set : "sed -i 's/innodb_buffer_pool_size.*/innodb_buffer_pool_size = $value/g' /etc/my.cnf"
        needrestart : "true"
        type : "discrete"
        scope :
          - 1024
          - 1.37E+11
        step: 27454908
        items :
        dtype : "int"
  -
    name : "mariadb.max_heap_table_size"
    info :
        desc : "size of a memory table that can be created"
        get : "cat /etc/my.cnf | grep max_heap_table_size | awk -F '=' '{print $2}' | awk '$1=$1'"
        set : "sed -i 's/max_heap_table_size.*/max_heap_table_size = $value/g' /etc/my.cnf"
        needrestart : "true"
        type : "discrete"
        scope :
          - 1024
          - 104857600
        step : 200000
        items :
        dtype : "int"
  -
    name : "mariadb.read_rnd_buffer_size"
    info :
        desc : "the buffer allocated to each thread when the table is read randomly"
        get : "cat /etc/my.cnf | grep read_rnd_buffer_size | awk -F '=' '{print $2}' | awk '$1=$1'"
        set : "sed -i 's/read_rnd_buffer_size.*/read_rnd_buffer_size = $value/g' /etc/my.cnf"
        needrestart : "true"
        type : "discrete"
        scope :
          - 1024
          - 104857600
        step : 200000
        items :
        dtype : "int"
  -
    name : "mariadb.myisam_sort_buffer_size"
    info :
        desc : "the buffer required for reordering when the MyISAM table changes"
        get : "cat /etc/my.cnf | grep myisam_sort_buffer_size | awk -F '=' '{print $2}' | awk '$1=$1'"
        set : "sed -i 's/myisam_sort_buffer_size.*/myisam_sort_buffer_size = $value/g' /etc/my.cnf"
        needrestart : "true"
        type : "discrete"
        scope :
          - 1024
          - 104857600
        step : 200000
        items :
        dtype : "int"
  -
    name : "mariadb.table_open_cache"
    info :
        desc : "Table cache for storing data"
        get : "cat /etc/my.cnf | grep table_open_cache | awk -F '=' '{print $2}' | awk '$1=$1'"
        set : "sed -i 's/table_open_cache.*/table_open_cache = $value/g' /etc/my.cnf"
        needrestart : "true"
        type : "discrete"
        scope :
          - 16
          - 100000
        step : 100
        items :
        dtype : "int"

yangyongguang 任务状态待办的 修改为已完成

登录 后才可以发表评论

状态
负责人
项目
里程碑
Pull Requests
关联的 Pull Requests 被合并后可能会关闭此 issue
分支
开始日期   -   截止日期
-
置顶选项
优先级
预计工期 (小时)
参与者(3)
Go
1
https://gitee.com/openeuler/A-Tune.git
git@gitee.com:openeuler/A-Tune.git
openeuler
A-Tune
A-Tune

搜索帮助