有人用FlinkCDC同步Oracle成功的吗?

是的,有人使用Flink CDC成功同步Oracle数据库。Flink CDC是一种基于流式变更数据捕获(Change Data Capture)技术的数据同步工具,可以实时捕获源数据库中的数据变更并同步到目标数据库中。

使用Flink CDC同步Oracle数据库的详细步骤

专注于为中小企业提供成都网站制作、成都网站建设、外贸营销网站建设服务,电脑端+手机端+微信端的三站合一,更高效的管理,为中小企业吴桥免费做网站提供优质的服务。我们立足成都,凝聚了一批互联网行业人才,有力地推动了1000多家企业的稳健成长,帮助中小企业通过网站建设实现规模扩充和转变。

环境准备

1、安装并配置Oracle数据库,确保有访问权限。

2、安装Apache Flink,版本要求为1.13及以上。

3、下载并添加Flinkconnectororacle依赖到项目中。

创建Flink流处理程序

1、引入相关依赖。


    org.apache.flink
    flinkconnectororacle_2.11
    1.13.2

2、编写Flink流处理程序。

import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
import org.apache.flink.table.api.bridge.java.StreamTableEnvironment;
import org.apache.flink.table.api.EnvironmentSettings;
import org.apache.flink.table.catalog.debezium.DebeziumOptions;
import org.apache.flink.table.catalog.debezium.DebeziumTableFactory;
import org.apache.flink.table.descriptors.*;
import org.apache.flink.types.Row;
public class FlinkCDCOracle {
    public static void main(String[] args) throws Exception {
        // 创建流处理执行环境
        StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
        EnvironmentSettings settings = EnvironmentSettings.newInstance().useBlinkPlanner().inStreamingMode().build();
        StreamTableEnvironment tableEnv = StreamTableEnvironment.create(env, settings);
        // 注册Oracle源表
        String sourceDDL = "CREATE TABLE oracle_source ( " +
                " id INT NOT NULL," +
                " name STRING," +
                " age INT," +
                " PRIMARY KEY (id) NOT ENFORCED" +
                ") WITH (" +
                " 'connector' = 'oracle'," +
                " 'username' = 'your_username'," +
                " 'password' = 'your_password'," +
                " 'database' = 'your_database'," +
                " 'table' = 'your_table'," +
                " 'debeziuminternal.offsetstorage' = 'org.apache.kafka.connect.storage.FileOffsetBackingStore'," +
                " 'debeziuminternal.offsetstorage.file.filename' = '/path/to/offset/storage/file'," +
                " 'debeziuminternal.offsetstorage.topic' = 'your_offset_topic'," +
                " 'debeziuminternal.keyconverter' = 'org.apache.kafka.connect.json.JsonConverter'," +
                " 'debeziuminternal.valueconverter' = 'org.apache.kafka.connect.json.JsonConverter'," +
                " 'debeziuminternal.schemainclude' = 'false'," +
                " 'debeziuminternal.decimalhandling' = 'double'," +
                " 'debeziuminternal.timestamphandling' = 'ms'," +
                " 'debeziuminternal.pkmode' = 'none'," +
                " 'debeziuminternal.databasehistory' = 'io.debezium:type=connector:dbhistory:oracle:10000'" +
                ")";
        tableEnv.executeSql(sourceDDL);
        // 查询源表数据并输出到控制台
        tableEnv.executeSql("SELECT * FROM oracle_source").print();
    }
}

运行Flink流处理程序

1、编译并打包项目。

2、在命令行中运行Flink流处理程序。

$ flink run m yarncluster yn   class  planner sqlblink parallelism  jobmanager : taskmanager : detached yarnapplicationname  checkpointingInterval  savepoints  queryFile  executionMode streamlit allowNonRestoredState true externalizedCheckpoints false updateCheckpointInterval  maxConcurrentCheckpoints 1000 stateBackend rocksdb stateRetentionTimeHours 604800 restoreSavepointOnCancellation true restoreSavepointOnCheckpointFailure true restoreSavepointOnException true restoreSavepointOnCompletion true restoreSavepointOnQueryCompletion true restoreSavepointOnKillApplication true restoreSavepointOnShutdown true restoreSavepointOnStartup true restoreSavepointOnUserRequest true restoreSavepointOnYarnFailure true restoreSavepointOnYarnReconfiguration true restoreSavepointOnYarnNodeLoss true restoreSavepointOnYarnSchedulerRestart true restoreSavepointOnYarnApplicationAttemptEnd true restoreSavepointOnYarnApplicationAttemptFailed true restoreSavepointOnYarnApplicationAttemptKilled true restoreSavepointOnYarnApplicationAttemptStarted true restoreSavepointOnYarnApplicationAttemptUpdated true restoreSavepointOnYarnApplicationSubmissionFailed true restoreSavepointOnYarnApplicationSubmissionSuccessful true restoreSavepointOnYarnApplicationTerminatedTrue false restoreSavepointOnYarnApplicationUnknownTrue false restoreSavepointOnYarnApplicationUnknownFalse false restoreSavepointOnYarnApplicationUnknownNone false restoreSavepointOnYarnApplicationUnknownUnknownFalse false restoreSavepointOnYarnApplicationUnknownUnknownNone false restoreSavepointOnYarnApplicationUnknownUnknownUnknownFalse false restoreSavepointOnYarnApplicationUnknownUnknownUnknownNone false restoreSavepointOnYarnApplicationUnknownUnknownUnknownUnknownFalse false restoreSavepointOnYarnApplicationUnknownUnknownUnknownUnknownNone false restoreSavepointOnYarnApplicationUnknownUnknownUnknownUnknownUnknownFalse false restoreSavepointOnYarnApplicationUnknownUnknownUnknownUnknownUnknownNone false restoreSavepointOnYarnApplicationUnknownUnknownUnknownUnknownUnknownNone false restoreSavepointOnYarnApplicationUnknownUnknownUnknownUnknownUnknownNone false restoreSavepointOnYarnApplicationUnknownUnknownUnknownUnknownUnknownNone false restoreSavepointOnYarnApplicationUnknownUnknownUnknownUnknownUnknownNone false restoreSavepointOnYarnApplicationUnknownUnknownUnknownUnknownUnknownNone false restoreSavepointOnYarnApplicationUnknownUnknownUnknownUnknownUnknownNone false restoreSavepointOnYarnApplicationUnknownUnknownUnknownUnknownUnknownNone false restoreSavepointOnYarnApplicationUnknownUnknownUnknownUnknownUnknownNone false restoreSavepointOnYarnApplicationUnknownUnknownUnknownUnknownUnknownNone false restoreSavepointOnYarnApplicationUnknownUnknownUnknownUnknownUnknownNone false restoreSavepointOnYarnApplicationUnknownUnknownUnknownUnknown

网页名称:有人用FlinkCDC同步Oracle成功的吗?
当前路径:http://www.csdahua.cn/qtweb/news15/526365.html

网站建设、网络推广公司-快上网,是专注品牌与效果的网站制作,网络营销seo公司;服务项目有等

广告

声明:本网站发布的内容(图片、视频和文字)以用户投稿、用户转载内容为主,如果涉及侵权请尽快告知,我们将会在第一时间删除。文章观点不代表本网站立场,如需处理请联系客服。电话:028-86922220;邮箱:631063699@qq.com。内容未经允许不得转载,或转载时需注明来源: 快上网