前面文章已经介绍过ogg的安装部署,这里不做赘述,只针对kafka集群是kerberos认证模式的情况下,挖掘,传输,和应用进程的配置做介绍,其他安装步骤可参考前面的文档。
挖掘进程配置(DG端挖掘)
table oggtest.test1;
传输进程配置
重点区别在于目标端应用进程的相关配置,如下:
kafka_r2kafka.props配置:
gg.handlerlist=kafkahandler
gg.handler.kafkahandler.type=kafka
gg.handler.kafkahandler.KafkaProducerConfigFile= kafka_handler/kafka_producer.properties
gg.handler.kafkahandler.topicMappingTemplate=oggtest_${tableName}
gg.handler.kafkahandler.keyMappingTemplate=${primaryKeys}
gg.handler.kafkahandler.format=json
gg.handler.kafkahandler.format.includePrimaryKeys=true
gg.handler.kafkahandler.mode=op
gg.handler.kafkahandler.BlockingSend=false
gg.classpath=dirprm/:/usr/local/TDH-Client/kafka/libs*:/ogg:/ogg/lib/*
jvm.bootoptions=-Xmx512m -Xms512m -Djava.class.path=./ggjava/ggjava.jar -Dlog4j.configuration=log4j.properties - Djava.security.auth.login.config=/etc/eventstore1/conf/jaas.conf -Djava.security.krb5.conf=/etc/krb5.conf
/etc/eventstore1/conf/jaas.con f 配置:
KafkaClient {
com.sun.security.auth.module.Krb5LoginModule required
useKeyTab=true
keyTab="/etc/eventstore1/conf/kafka.keytab"
storeKey=true
useTicketCache=false
principal="kafka@TDH";
};
kafka_producer.properties 配置:
bootstrap.servers=tdh02:9092, tdh03:9092, tdh04:9092, tdh05:9092, tdh07:9092, tdh08:9092, tdh09:9092, tdh10:9092
acks=-1
compression.type=lz4
buffer.memory=134217728
reconnect.backoff.ms=20000
retries=10
retry.backoff.ms=20000
max.in.flight.requests.per.connection=1
value.serializer=org.apache.kafka.common.serialization.ByteArraySerializer
key.serializer=org.apache.kafka.common.serialization.ByteArraySerializer
batch.size=524288
linger.ms=1000
request.timeout.ms=60000
send.buffer.bytes=5242880
security.protocol=SASL_PLAINTEXT
sasl.kerberos.service.name=kafka
sasl.mechanism=GSSAPI
标红部分为该模式下需要重点关注和配置的。
来自 “ ITPUB博客 ” ,链接:https://blog.itpub.net/29956245/viewspace-3109817/ ,如需转载,请注明出处,否则将追究法律责任。