3 Star 12 Fork 8

leon / dataroad-all

加入 Gitee
与超过 1200万 开发者一起发现、参与优秀开源项目,私有仓库也完全免费 :)
免费加入
该仓库未声明开源许可证文件(LICENSE),使用请关注具体项目描述及其代码上游依赖。
克隆/下载
mysql-writer.md 5.86 KB
一键复制 编辑 原始数据 按行查看 历史
leon 提交于 2022-03-18 14:12 . 提交MD文档

MySQL Writer

一、插件名称

类型:reader
名称:mysqlWriter

二、支持的数据源版本

MySQL 5.X

三、参数说明

  • jdbcUrl
    • 描述:针对关系型数据库的jdbc连接字符串
    • 必选:是
    • 字段类型:String
    • 默认值:无

  • username
    • 描述:数据源的用户名
    • 必选:是
    • 字段类型:String
    • 默认值:无

  • password
    • 描述:数据源指定用户名的密码
    • 必选:是
    • 字段类型:String
    • 默认值:无

  • table
    • 描述:目的表的表名称。
    • 必选:是
    • 字段类型:String
    • 默认值:无

  • column
    • 描述:目的表需要写入数据的字段,字段之间用英文逗号分隔。例如: "column": ["id","name","age"]
    • 必选:是
    • 字段类型:Array
    • 默认值:无

  • fullcolumn
    • 描述:目的表中的所有字段,字段之间用英文逗号分隔。例如: "column": ["id","name","age","hobby"],如果不配置,将在系统表中获取
    • 必选:否
    • 字段类型:Array
    • 默认值:无

  • preSql
    • 描述:写入数据到目的表前,会先执行这里的一组标准语句。例如:["update student t1 set t1.status='0'"]
    • 必选:否
    • 字段类型:Array
    • 默认值:无

  • postSql
    • 描述:写入数据到目的表后,会执行这里的一组标准语句。例如:["update student t1 set t1.status='0'"]
    • 必选:否
    • 字段类型:Array
    • 默认值:无

  • writeMode
    • 描述:控制写入数据到目标表采用 insert into 或者 replace into 或者 ON DUPLICATE KEY UPDATE 语句。其中当写入模式为update和replace时,默认采用主键作为唯一索引
    • 必选:是
    • 所有选项:INSERT/REPLACE/UPDATE
    • 字段类型:String
    • 默认值:INSERT

  • batchSize
    • 描述:一次性批量提交的记录数大小,该值可以极大减少与数据库的网络交互次数,并提升整体吞吐量。但是该值设置过大可能会造成运行进程OOM情况
    • 必选:否
    • 字段类型:int
    • 默认值:1024

四、配置示例

1、insert

{
  "job" : {
    "content" : [ {
      "mysqlReader1" : {
        "type" : "reader",
        "pluginName" : "mysqlReader",
        "parameter" : {
          "jdbcUrl" : "jdbc:mysql://0.0.0.1:3306/database?useSSL=false",
          "username" : "username",
          "password" : "password",
          "table" : "student",
          "column": [
            {
              "name": "id",
              "type": "id"
            },
            {
              "name": "name",
              "type": "varchar"
            },
            {
              "name": "sex",
              "type": "int"
            }
          ]
        }
      },
      "mysqlWriter2" : {
        "type" : "writer",
        "pluginName" : "mysqlWriter",
        "parameter" : {
          "jdbcUrl" : "jdbc:mysql://0.0.0.1:3306/database?useSSL=false",
          "username" : "username",
          "password" : "password",
          "table" : "student1",
          "column" : [ "id", "name", "sex" ],
          "writeMode" : "INSERT"
        }
      }
    } ],
    "setting": {
      "name": "myJob",
      "speed": {
        "channel": 1
      },
      "restore": {
        "isRestore": false
      }
    }
  }
}

2、update

{
  "job" : {
    "content" : [ {
      "mysqlReader1" : {
        "type" : "reader",
        "pluginName" : "mysqlReader",
        "parameter" : {
          "jdbcUrl" : "jdbc:mysql://0.0.0.1:3306/database?useSSL=false",
          "username" : "username",
          "password" : "password",
          "table" : "student",
          "column": [
            {
              "name": "id",
              "type": "id"
            },
            {
              "name": "name",
              "type": "varchar"
            },
            {
              "name": "sex",
              "type": "int"
            }
          ]
        }
      },
      "mysqlWriter2" : {
        "type" : "writer",
        "pluginName" : "mysqlWriter",
        "parameter" : {
          "jdbcUrl" : "jdbc:mysql://0.0.0.1:3306/database?useSSL=false",
          "username" : "username",
          "password" : "password",
          "table" : "student1",
          "column" : [ "id", "name", "sex" ],
          "writeMode" : "UPDATE"
        }
      }
    } ],
    "setting": {
      "name": "myJob",
      "speed": {
        "channel": 1
      },
      "restore": {
        "isRestore": false,
        "restoreColumnName": "",
        "restoreColumnIndex": 0
      }
    }
  }
}

3、replace

{
  "job" : {
    "content" : [ {
      "mysqlReader1" : {
        "type" : "reader",
        "pluginName" : "mysqlReader",
        "parameter" : {
          "jdbcUrl" : "jdbc:mysql://0.0.0.1:3306/database?useSSL=false",
          "username" : "username",
          "password" : "password",
          "table" : "student",
          "column": [
            {
              "name": "id",
              "type": "id"
            },
            {
              "name": "name",
              "type": "varchar"
            },
            {
              "name": "sex",
              "type": "int"
            }
          ]
        }
      },
      "mysqlWriter2" : {
        "type" : "writer",
        "pluginName" : "mysqlWriter",
        "parameter" : {
          "jdbcUrl" : "jdbc:mysql://0.0.0.1:3306/database?useSSL=false",
          "username" : "username",
          "password" : "password",
          "table" : "student1",
          "column" : [ "id", "name", "sex" ],
          "writeMode" : "REPLACE"
        }
      }
    } ],
    "setting": {
      "name": "myJob",
      "speed": {
        "channel": 1
      },
      "restore": {
        "isRestore": false
      }
    }
  }
}
JavaScript
1
https://gitee.com/leonside/dataroad-all.git
git@gitee.com:leonside/dataroad-all.git
leonside
dataroad-all
dataroad-all
master

搜索帮助