{"id":88726,"date":"2024-12-18T03:36:04","date_gmt":"2024-12-18T00:06:04","guid":{"rendered":"https:\/\/nabfollower.com\/blog\/building-a-real-time-data-pipeline-app-with-change-data-capture-tools-debezium-kafka-and-nifi-6ig\/"},"modified":"2024-12-18T03:36:04","modified_gmt":"2024-12-18T00:06:04","slug":"building-a-real-time-data-pipeline-app-with-change-data-capture-tools-debezium-kafka-and-nifi-6ig","status":"publish","type":"post","link":"https:\/\/nabfollower.com\/blog\/building-a-real-time-data-pipeline-app-with-change-data-capture-tools-debezium-kafka-and-nifi-6ig\/","title":{"rendered":"\u0633\u0627\u062e\u062a\u0646 \u06cc\u06a9 \u0628\u0631\u0646\u0627\u0645\u0647 \u062e\u0637 \u0644\u0648\u0644\u0647 \u062f\u0627\u062f\u0647 \u0628\u0644\u0627\u062f\u0631\u0646\u06af \u0628\u0627 \u0627\u0628\u0632\u0627\u0631\u0647\u0627\u06cc Change Data Capture: Debezium\u060c Kafka \u0648 NiFi"},"content":{"rendered":"

Summarize this content to 400 words in Persian Lang
\n Change Data Capture (CDC) \u0628\u0647 \u06cc\u06a9 \u062a\u06a9\u0646\u06cc\u06a9 \u062d\u06cc\u0627\u062a\u06cc \u0628\u0631\u0627\u06cc \u06cc\u06a9\u067e\u0627\u0631\u0686\u0647 \u0633\u0627\u0632\u06cc \u062f\u0627\u062f\u0647 \u0647\u0627\u06cc \u0645\u062f\u0631\u0646 \u062a\u0628\u062f\u06cc\u0644 \u0634\u062f\u0647 \u0627\u0633\u062a \u06a9\u0647 \u0628\u0647 \u0633\u0627\u0632\u0645\u0627\u0646 \u0647\u0627 \u0627\u062c\u0627\u0632\u0647 \u0645\u06cc \u062f\u0647\u062f \u062a\u063a\u06cc\u06cc\u0631\u0627\u062a \u062f\u0627\u062f\u0647 \u0647\u0627 \u0631\u0627 \u062f\u0631 \u0633\u06cc\u0633\u062a\u0645 \u0647\u0627\u06cc \u0645\u062e\u062a\u0644\u0641 \u062f\u0631 \u0632\u0645\u0627\u0646 \u0648\u0627\u0642\u0639\u06cc \u0631\u062f\u06cc\u0627\u0628\u06cc \u0648 \u0627\u0646\u062a\u0634\u0627\u0631 \u062f\u0647\u0646\u062f. \u062f\u0631 \u0627\u06cc\u0646 \u0645\u0642\u0627\u0644\u0647\u060c \u0646\u062d\u0648\u0647 \u0633\u0627\u062e\u062a \u06cc\u06a9 \u0631\u0627\u0647 \u062d\u0644 \u062c\u0627\u0645\u0639 CDC \u0628\u0627 \u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u0627\u0632 \u0627\u0628\u0632\u0627\u0631\u0647\u0627\u06cc \u0645\u0646\u0628\u0639 \u0628\u0627\u0632 \u0642\u062f\u0631\u062a\u0645\u0646\u062f \u0645\u0627\u0646\u0646\u062f Debezium\u060c Apache Kafka \u0648 Apache NiFi \u0631\u0627 \u0628\u0631\u0631\u0633\u06cc \u062e\u0648\u0627\u0647\u06cc\u0645 \u06a9\u0631\u062f.<\/p>\n

\u0641\u0646 \u0622\u0648\u0631\u06cc \u0647\u0627\u06cc \u06a9\u0644\u06cc\u062f\u06cc \u062f\u0631 \u067e\u0634\u062a\u0647 CDC \u0645\u0627<\/p>\n

Debezium: \u06cc\u06a9 \u067e\u0644\u062a\u0641\u0631\u0645 \u0645\u0646\u0628\u0639 \u0628\u0627\u0632 \u0628\u0631\u0627\u06cc \u062a\u063a\u06cc\u06cc\u0631 \u062f\u0627\u062f\u0647 \u0647\u0627 \u06a9\u0647 \u0627\u0632 \u0686\u0646\u062f\u06cc\u0646 \u0645\u0646\u0628\u0639 \u067e\u0627\u06cc\u06af\u0627\u0647 \u062f\u0627\u062f\u0647 \u067e\u0634\u062a\u06cc\u0628\u0627\u0646\u06cc \u0645\u06cc \u06a9\u0646\u062f.
\n\u0622\u067e\u0627\u0686\u06cc \u06a9\u0627\u0641\u06a9\u0627: \u06cc\u06a9 \u067e\u0644\u062a \u0641\u0631\u0645 \u067e\u062e\u0634 \u062a\u0648\u0632\u06cc\u0639 \u0634\u062f\u0647 \u06a9\u0647 \u0628\u0647 \u0639\u0646\u0648\u0627\u0646 \u0633\u06cc\u0633\u062a\u0645 \u0639\u0635\u0628\u06cc \u0645\u0631\u06a9\u0632\u06cc \u0628\u0631\u0627\u06cc \u062e\u0637 \u0644\u0648\u0644\u0647 \u062f\u0627\u062f\u0647 \u0645\u0627 \u0639\u0645\u0644 \u0645\u06cc \u06a9\u0646\u062f.
\nApache NiFi: \u0627\u0628\u0632\u0627\u0631\u06cc \u0628\u0631\u0627\u06cc \u0645\u062f\u06cc\u0631\u06cc\u062a \u062c\u0631\u06cc\u0627\u0646 \u062f\u0627\u062f\u0647 \u06a9\u0647 \u0628\u0647 \u0645\u0627 \u06a9\u0645\u06a9 \u0645\u06cc \u06a9\u0646\u062f \u062c\u0631\u06cc\u0627\u0646 \u0647\u0627\u06cc \u062f\u0627\u062f\u0647 \u0631\u0627 \u0645\u0633\u06cc\u0631\u06cc\u0627\u0628\u06cc\u060c \u062a\u0628\u062f\u06cc\u0644 \u0648 \u067e\u0631\u062f\u0627\u0632\u0634 \u06a9\u0646\u06cc\u0645.<\/p>\n

\u0646\u0645\u0627\u06cc \u06a9\u0644\u06cc \u0645\u0639\u0645\u0627\u0631\u06cc\u0645\u0639\u0645\u0627\u0631\u06cc \u067e\u06cc\u0634\u0646\u0647\u0627\u062f\u06cc \u0645\u0627 \u0627\u06cc\u0646 \u0645\u0631\u0627\u062d\u0644 \u06a9\u0644\u06cc\u062f\u06cc \u0631\u0627 \u062f\u0646\u0628\u0627\u0644 \u0645\u06cc \u06a9\u0646\u062f:<\/p>\n

\u0628\u0627 \u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u0627\u0632 Debezium \u062a\u063a\u06cc\u06cc\u0631\u0627\u062a \u067e\u0627\u06cc\u06af\u0627\u0647 \u062f\u0627\u062f\u0647 \u0631\u0627 \u0636\u0628\u0637 \u06a9\u0646\u06cc\u062f
\n\u062c\u0631\u06cc\u0627\u0646 \u0627\u0632 \u0637\u0631\u06cc\u0642 \u06a9\u0627\u0641\u06a9\u0627 \u062a\u063a\u06cc\u06cc\u0631 \u0645\u06cc \u06a9\u0646\u062f
\n\u067e\u0631\u062f\u0627\u0632\u0634 \u0648 \u0645\u0633\u06cc\u0631\u06cc\u0627\u0628\u06cc \u062f\u0627\u062f\u0647 \u0647\u0627 \u0628\u0627 \u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u0627\u0632 NiFi
\n\u0630\u062e\u06cc\u0631\u0647 \u06cc\u0627 \u067e\u0631\u062f\u0627\u0632\u0634 \u0628\u06cc\u0634\u062a\u0631 \u062f\u0627\u062f\u0647 \u0647\u0627\u06cc \u062a\u0628\u062f\u06cc\u0644 \u0634\u062f\u0647<\/p>\n

\u0646\u0645\u0648\u0646\u0647 \u0631\u0648\u06cc\u06a9\u0631\u062f \u067e\u06cc\u0627\u062f\u0647 \u0633\u0627\u0632\u06cc<\/p>\n

from confluent_kafka import Consumer, Producer
\nimport json
\nimport debezium<\/p>\n

class CDCDataPipeline:
\n def __init__(self, source_db, kafka_bootstrap_servers):
\n “””
\n Initialize CDC pipeline with database source and Kafka configuration<\/p>\n

:param source_db: Source database connection details
\n :param kafka_bootstrap_servers: Kafka broker addresses
\n “””
\n self.source_db = source_db
\n self.kafka_servers = kafka_bootstrap_servers<\/p>\n

# Debezium connector configuration
\n self.debezium_config = {
\n ‘connector.class’: ‘io.debezium.connector.mysql.MySqlConnector’,
\n ‘tasks.max’: ‘1’,
\n ‘database.hostname’: source_db[‘host’],
\n ‘database.port’: source_db[‘port’],
\n ‘database.user’: source_db[‘username’],
\n ‘database.password’: source_db[‘password’],
\n ‘database.server.name’: ‘my-source-database’,
\n ‘database.include.list’: source_db[‘database’]\n }<\/p>\n

def start_capture(self):
\n “””
\n Start change data capture process
\n “””
\n # Configure Kafka producer for streaming changes
\n producer = Producer({
\n ‘bootstrap.servers’: self.kafka_servers,
\n ‘client.id’: ‘cdc-change-producer’
\n })<\/p>\n

# Set up Debezium connector
\n def handle_record(record):
\n “””
\n Process each captured change record
\n “””
\n # Transform record and publish to Kafka
\n change_event = {
\n ‘source’: record.source(),
\n ‘operation’: record.operation(),
\n ‘data’: record.after()
\n }<\/p>\n

producer.produce(
\n topic=”database-changes”,
\n value=json.dumps(change_event)
\n )<\/p>\n

# Start Debezium connector
\n debezium.start_connector(
\n config=self.debezium_config,
\n record_handler=handle_record
\n )<\/p>\n

# Example usage
\nsource_database = {
\n ‘host’: ‘localhost’,
\n ‘port’: 3306,
\n ‘username’: ‘cdc_user’,
\n ‘password’: ‘secure_password’,
\n ‘database’: ‘customer_db’
\n}<\/p>\n

pipeline = CDCDataPipeline(
\n source_database,
\n kafka_bootstrap_servers=”localhost:9092″
\n)
\npipeline.start_capture()<\/p>\n

\u0648\u0627\u0631\u062f \u062d\u0627\u0644\u062a \u062a\u0645\u0627\u0645 \u0635\u0641\u062d\u0647 \u0634\u0648\u06cc\u062f<\/p>\n

\u0627\u0632 \u062d\u0627\u0644\u062a \u062a\u0645\u0627\u0645 \u0635\u0641\u062d\u0647 \u062e\u0627\u0631\u062c \u0634\u0648\u06cc\u062f<\/p>\n

\u0645\u0631\u0627\u062d\u0644 \u0627\u062c\u0631\u0627\u06cc \u062f\u0642\u06cc\u0642<\/p>\n

\u067e\u06cc\u06a9\u0631\u0628\u0646\u062f\u06cc \u0645\u0646\u0628\u0639 \u067e\u0627\u06cc\u06af\u0627\u0647 \u062f\u0627\u062f\u0647 \u0627\u0648\u0644\u06cc\u0646 \u0645\u0631\u062d\u0644\u0647 \u0634\u0627\u0645\u0644 \u067e\u06cc\u06a9\u0631\u0628\u0646\u062f\u06cc Debezium \u0628\u0631\u0627\u06cc \u0627\u062a\u0635\u0627\u0644 \u0628\u0647 \u067e\u0627\u06cc\u06af\u0627\u0647 \u062f\u0627\u062f\u0647 \u0645\u0646\u0628\u0639 \u0634\u0645\u0627 \u0627\u0633\u062a. \u0627\u06cc\u0646 \u0645\u0633\u062a\u0644\u0632\u0645:<\/p>\n

\u0645\u062c\u0648\u0632\u0647\u0627\u06cc \u06a9\u0627\u0631\u0628\u0631 \u067e\u0627\u06cc\u06af\u0627\u0647 \u062f\u0627\u062f\u0647 \u0645\u0646\u0627\u0633\u0628
\n\u0627\u062a\u0635\u0627\u0644 \u0628\u0647 \u0634\u0628\u06a9\u0647
\n\u0641\u0639\u0627\u0644 \u06a9\u0631\u062f\u0646 \u062b\u0628\u062a \u0628\u0627\u06cc\u0646\u0631\u06cc (\u0628\u0631\u0627\u06cc MySQL)<\/p>\n

\u06a9\u0627\u0641\u06a9\u0627 \u0628\u0647 \u0639\u0646\u0648\u0627\u0646 \u06cc\u06a9 \u067e\u0644\u062a\u0641\u0631\u0645 \u062c\u0631\u06cc\u0627\u0646 \u0622\u067e\u0627\u0686\u06cc \u06a9\u0627\u0641\u06a9\u0627 \u0628\u0647 \u0639\u0646\u0648\u0627\u0646 \u06cc\u06a9 \u0648\u0627\u0633\u0637\u0647 \u067e\u06cc\u0627\u0645 \u0645\u0631\u06a9\u0632\u06cc \u0639\u0645\u0644 \u0645\u06cc \u06a9\u0646\u062f \u0648 \u0631\u0648\u06cc\u062f\u0627\u062f\u0647\u0627\u06cc \u062a\u063a\u06cc\u06cc\u0631 \u0631\u0627 \u062b\u0628\u062a \u0648 \u0630\u062e\u06cc\u0631\u0647 \u0645\u06cc \u06a9\u0646\u062f. \u0645\u0644\u0627\u062d\u0638\u0627\u062a \u06a9\u0644\u06cc\u062f\u06cc \u0639\u0628\u0627\u0631\u062a\u0646\u062f \u0627\u0632:<\/p>\n

\u067e\u06cc\u06a9\u0631\u0628\u0646\u062f\u06cc \u067e\u0627\u0631\u062a\u06cc\u0634\u0646 \u0647\u0627\u06cc \u0645\u0648\u0636\u0648\u0639\u06cc
\n\u062a\u0646\u0638\u06cc\u0645 \u0633\u06cc\u0627\u0633\u062a \u0647\u0627\u06cc \u0646\u06af\u0647\u062f\u0627\u0631\u06cc \u0645\u0646\u0627\u0633\u0628
\n\u067e\u06cc\u0627\u062f\u0647 \u0633\u0627\u0632\u06cc \u0645\u0639\u0646\u0627\u06cc\u06cc \u067e\u0631\u062f\u0627\u0632\u0634 \u062f\u0642\u06cc\u0642\u0627\u064b \u06cc\u06a9 \u0628\u0627\u0631<\/p>\n

\u062a\u0628\u062f\u06cc\u0644 \u062f\u0627\u062f\u0647 \u0628\u0627 NiFi Apache NiFi \u0642\u0627\u0628\u0644\u06cc\u062a \u0647\u0627\u06cc \u0642\u062f\u0631\u062a\u0645\u0646\u062f \u0645\u0633\u06cc\u0631\u06cc\u0627\u0628\u06cc \u0648 \u062a\u0628\u062f\u06cc\u0644 \u062f\u0627\u062f\u0647 \u0631\u0627 \u0627\u0631\u0627\u0626\u0647 \u0645\u06cc \u062f\u0647\u062f:<\/p>\n

\u0631\u0648\u06cc\u062f\u0627\u062f\u0647\u0627\u06cc \u062a\u063a\u06cc\u06cc\u0631 \u0645\u0633\u06cc\u0631 \u0648 \u0641\u06cc\u0644\u062a\u0631
\n\u063a\u0646\u06cc \u0633\u0627\u0632\u06cc \u062f\u0627\u062f\u0647 \u0647\u0627 \u0631\u0627 \u0627\u0639\u0645\u0627\u0644 \u06a9\u0646\u06cc\u062f
\n\u0645\u0646\u0637\u0642 \u062a\u062d\u0648\u0644 \u067e\u06cc\u0686\u06cc\u062f\u0647 \u0631\u0627 \u0645\u062f\u06cc\u0631\u06cc\u062a \u06a9\u0646\u06cc\u062f<\/p>\n

\u0686\u0627\u0644\u0634 \u0647\u0627 \u0648 \u0628\u0647\u062a\u0631\u06cc\u0646 \u0634\u06cc\u0648\u0647 \u0647\u0627<\/p>\n

\u0645\u062f\u06cc\u0631\u06cc\u062a \u062a\u063a\u06cc\u06cc\u0631\u0627\u062a \u0637\u0631\u062d\u0648\u0627\u0631\u0647: \u0627\u0633\u062a\u0631\u0627\u062a\u0698\u06cc \u0647\u0627\u06cc \u0642\u0648\u06cc \u062a\u06a9\u0627\u0645\u0644 \u0637\u0631\u062d\u0648\u0627\u0631\u0647 \u0631\u0627 \u0627\u062c\u0631\u0627 \u06a9\u0646\u06cc\u062f
\n\u0628\u0647\u06cc\u0646\u0647 \u0633\u0627\u0632\u06cc \u0639\u0645\u0644\u06a9\u0631\u062f: \u0627\u0632 \u067e\u0627\u0631\u062a\u06cc\u0634\u0646 \u0628\u0646\u062f\u06cc \u0648 \u0641\u0634\u0631\u062f\u0647 \u0633\u0627\u0632\u06cc \u0645\u0646\u0627\u0633\u0628 \u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u06a9\u0646\u06cc\u062f
\n\u0645\u062f\u06cc\u0631\u06cc\u062a \u062e\u0637\u0627: \u0645\u06a9\u0627\u0646\u06cc\u0632\u0645 \u0647\u0627\u06cc \u062c\u0627\u0645\u0639 \u0631\u062f\u06cc\u0627\u0628\u06cc \u062e\u0637\u0627 \u0648 \u062a\u0644\u0627\u0634 \u0645\u062c\u062f\u062f \u0631\u0627 \u067e\u06cc\u0627\u062f\u0647 \u0633\u0627\u0632\u06cc \u06a9\u0646\u06cc\u062f<\/p>\n

\u0645\u062e\u0632\u0646 GitHub<\/p>\n

\u0645\u0646 \u06cc\u06a9 \u0646\u0645\u0648\u0646\u0647 \u067e\u06cc\u0627\u062f\u0647 \u0633\u0627\u0632\u06cc \u0627\u06cc\u062c\u0627\u062f \u06a9\u0631\u062f\u0647 \u0627\u0645 \u06a9\u0647 \u0645\u06cc \u062a\u0648\u0627\u0646\u06cc\u062f \u0622\u0646 \u0631\u0627 \u0628\u0631\u0631\u0633\u06cc \u06a9\u0631\u062f\u0647 \u0648 \u0628\u0647 \u0639\u0646\u0648\u0627\u0646 \u0645\u0631\u062c\u0639 \u0627\u0632 \u0622\u0646 \u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u06a9\u0646\u06cc\u062f. \u06a9\u062f \u06a9\u0627\u0645\u0644 \u0648 \u0627\u0633\u0646\u0627\u062f \u0627\u0636\u0627\u0641\u06cc \u0631\u0627 \u0645\u06cc \u062a\u0648\u0627\u0646\u06cc\u062f \u062f\u0631 \u0622\u062f\u0631\u0633 \u0632\u06cc\u0631 \u0628\u06cc\u0627\u0628\u06cc\u062f:\u0645\u062e\u0632\u0646 GitHub: https:\/\/github.com\/Angelica-R\/cdc-data-pipeline<\/p>\n

\u0646\u062a\u06cc\u062c\u0647 \u06af\u06cc\u0631\u06cc\u0627\u06cc\u062c\u0627\u062f \u0631\u0627\u0647 \u062d\u0644 Change Data Capture \u0646\u06cc\u0627\u0632\u0645\u0646\u062f \u0637\u0631\u0627\u062d\u06cc \u062f\u0642\u06cc\u0642 \u0645\u0639\u0645\u0627\u0631\u06cc \u0648 \u0627\u0646\u062a\u062e\u0627\u0628 \u0627\u0628\u0632\u0627\u0631 \u0645\u0646\u0627\u0633\u0628 \u0627\u0633\u062a. \u0628\u0627 \u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u0627\u0632 Debezium\u060c Kafka \u0648 NiFi\u060c \u0645\u06cc\u200c\u062a\u0648\u0627\u0646\u06cc\u062f \u06cc\u06a9 \u067e\u0644\u062a\u0641\u0631\u0645 \u06cc\u06a9\u067e\u0627\u0631\u0686\u0647\u200c\u0633\u0627\u0632\u06cc \u062f\u0627\u062f\u0647\u200c\u0647\u0627\u06cc \u0645\u0642\u06cc\u0627\u0633\u200c\u067e\u0630\u06cc\u0631 \u0648 \u0642\u0648\u06cc \u0627\u06cc\u062c\u0627\u062f \u06a9\u0646\u06cc\u062f \u06a9\u0647 \u0628\u06cc\u0646\u0634\u200c\u0647\u0627\u06cc \u0628\u06cc\u200c\u062f\u0631\u0646\u06af \u062f\u0631\u0628\u0627\u0631\u0647 \u062a\u063a\u06cc\u06cc\u0631\u0627\u062a \u062f\u0627\u062f\u0647\u200c\u0647\u0627\u06cc \u0634\u0645\u0627 \u0627\u0631\u0627\u0626\u0647 \u0645\u06cc\u200c\u062f\u0647\u062f.<\/p>\n

\n

Change Data Capture (CDC) \u0628\u0647 \u06cc\u06a9 \u062a\u06a9\u0646\u06cc\u06a9 \u062d\u06cc\u0627\u062a\u06cc \u0628\u0631\u0627\u06cc \u06cc\u06a9\u067e\u0627\u0631\u0686\u0647 \u0633\u0627\u0632\u06cc \u062f\u0627\u062f\u0647 \u0647\u0627\u06cc \u0645\u062f\u0631\u0646 \u062a\u0628\u062f\u06cc\u0644 \u0634\u062f\u0647 \u0627\u0633\u062a \u06a9\u0647 \u0628\u0647 \u0633\u0627\u0632\u0645\u0627\u0646 \u0647\u0627 \u0627\u062c\u0627\u0632\u0647 \u0645\u06cc \u062f\u0647\u062f \u062a\u063a\u06cc\u06cc\u0631\u0627\u062a \u062f\u0627\u062f\u0647 \u0647\u0627 \u0631\u0627 \u062f\u0631 \u0633\u06cc\u0633\u062a\u0645 \u0647\u0627\u06cc \u0645\u062e\u062a\u0644\u0641 \u062f\u0631 \u0632\u0645\u0627\u0646 \u0648\u0627\u0642\u0639\u06cc \u0631\u062f\u06cc\u0627\u0628\u06cc \u0648 \u0627\u0646\u062a\u0634\u0627\u0631 \u062f\u0647\u0646\u062f. \u062f\u0631 \u0627\u06cc\u0646 \u0645\u0642\u0627\u0644\u0647\u060c \u0646\u062d\u0648\u0647 \u0633\u0627\u062e\u062a \u06cc\u06a9 \u0631\u0627\u0647 \u062d\u0644 \u062c\u0627\u0645\u0639 CDC \u0628\u0627 \u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u0627\u0632 \u0627\u0628\u0632\u0627\u0631\u0647\u0627\u06cc \u0645\u0646\u0628\u0639 \u0628\u0627\u0632 \u0642\u062f\u0631\u062a\u0645\u0646\u062f \u0645\u0627\u0646\u0646\u062f Debezium\u060c Apache Kafka \u0648 Apache NiFi \u0631\u0627 \u0628\u0631\u0631\u0633\u06cc \u062e\u0648\u0627\u0647\u06cc\u0645 \u06a9\u0631\u062f.<\/p>\n

\u0641\u0646 \u0622\u0648\u0631\u06cc \u0647\u0627\u06cc \u06a9\u0644\u06cc\u062f\u06cc \u062f\u0631 \u067e\u0634\u062a\u0647 CDC \u0645\u0627<\/strong><\/p>\n

    \n
  1. Debezium: \u06cc\u06a9 \u067e\u0644\u062a\u0641\u0631\u0645 \u0645\u0646\u0628\u0639 \u0628\u0627\u0632 \u0628\u0631\u0627\u06cc \u062a\u063a\u06cc\u06cc\u0631 \u062f\u0627\u062f\u0647 \u0647\u0627 \u06a9\u0647 \u0627\u0632 \u0686\u0646\u062f\u06cc\u0646 \u0645\u0646\u0628\u0639 \u067e\u0627\u06cc\u06af\u0627\u0647 \u062f\u0627\u062f\u0647 \u067e\u0634\u062a\u06cc\u0628\u0627\u0646\u06cc \u0645\u06cc \u06a9\u0646\u062f.<\/li>\n
  2. \u0622\u067e\u0627\u0686\u06cc \u06a9\u0627\u0641\u06a9\u0627: \u06cc\u06a9 \u067e\u0644\u062a \u0641\u0631\u0645 \u067e\u062e\u0634 \u062a\u0648\u0632\u06cc\u0639 \u0634\u062f\u0647 \u06a9\u0647 \u0628\u0647 \u0639\u0646\u0648\u0627\u0646 \u0633\u06cc\u0633\u062a\u0645 \u0639\u0635\u0628\u06cc \u0645\u0631\u06a9\u0632\u06cc \u0628\u0631\u0627\u06cc \u062e\u0637 \u0644\u0648\u0644\u0647 \u062f\u0627\u062f\u0647 \u0645\u0627 \u0639\u0645\u0644 \u0645\u06cc \u06a9\u0646\u062f.<\/li>\n
  3. Apache NiFi: \u0627\u0628\u0632\u0627\u0631\u06cc \u0628\u0631\u0627\u06cc \u0645\u062f\u06cc\u0631\u06cc\u062a \u062c\u0631\u06cc\u0627\u0646 \u062f\u0627\u062f\u0647 \u06a9\u0647 \u0628\u0647 \u0645\u0627 \u06a9\u0645\u06a9 \u0645\u06cc \u06a9\u0646\u062f \u062c\u0631\u06cc\u0627\u0646 \u0647\u0627\u06cc \u062f\u0627\u062f\u0647 \u0631\u0627 \u0645\u0633\u06cc\u0631\u06cc\u0627\u0628\u06cc\u060c \u062a\u0628\u062f\u06cc\u0644 \u0648 \u067e\u0631\u062f\u0627\u0632\u0634 \u06a9\u0646\u06cc\u0645.<\/li>\n<\/ol>\n

    \u0646\u0645\u0627\u06cc \u06a9\u0644\u06cc \u0645\u0639\u0645\u0627\u0631\u06cc<\/strong>
    \u0645\u0639\u0645\u0627\u0631\u06cc \u067e\u06cc\u0634\u0646\u0647\u0627\u062f\u06cc \u0645\u0627 \u0627\u06cc\u0646 \u0645\u0631\u0627\u062d\u0644 \u06a9\u0644\u06cc\u062f\u06cc \u0631\u0627 \u062f\u0646\u0628\u0627\u0644 \u0645\u06cc \u06a9\u0646\u062f:<\/p>\n

      \n
    • \u0628\u0627 \u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u0627\u0632 Debezium \u062a\u063a\u06cc\u06cc\u0631\u0627\u062a \u067e\u0627\u06cc\u06af\u0627\u0647 \u062f\u0627\u062f\u0647 \u0631\u0627 \u0636\u0628\u0637 \u06a9\u0646\u06cc\u062f<\/li>\n
    • \u062c\u0631\u06cc\u0627\u0646 \u0627\u0632 \u0637\u0631\u06cc\u0642 \u06a9\u0627\u0641\u06a9\u0627 \u062a\u063a\u06cc\u06cc\u0631 \u0645\u06cc \u06a9\u0646\u062f<\/li>\n
    • \u067e\u0631\u062f\u0627\u0632\u0634 \u0648 \u0645\u0633\u06cc\u0631\u06cc\u0627\u0628\u06cc \u062f\u0627\u062f\u0647 \u0647\u0627 \u0628\u0627 \u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u0627\u0632 NiFi<\/li>\n
    • \u0630\u062e\u06cc\u0631\u0647 \u06cc\u0627 \u067e\u0631\u062f\u0627\u0632\u0634 \u0628\u06cc\u0634\u062a\u0631 \u062f\u0627\u062f\u0647 \u0647\u0627\u06cc \u062a\u0628\u062f\u06cc\u0644 \u0634\u062f\u0647<\/li>\n<\/ul>\n

      \u0646\u0645\u0648\u0646\u0647 \u0631\u0648\u06cc\u06a9\u0631\u062f \u067e\u06cc\u0627\u062f\u0647 \u0633\u0627\u0632\u06cc<\/strong><\/p>\n

      \n
      from confluent_kafka import Consumer, Producer\nimport json\nimport debezium\n\nclass CDCDataPipeline:\n    def __init__(self, source_db, kafka_bootstrap_servers):\n        \"\"\"\n        Initialize CDC pipeline with database source and Kafka configuration\n\n        :param source_db: Source database connection details\n        :param kafka_bootstrap_servers: Kafka broker addresses\n        \"\"\"\n        self.source_db = source_db\n        self.kafka_servers = kafka_bootstrap_servers\n\n        # Debezium connector configuration\n        self.debezium_config = {\n            'connector.class': 'io.debezium.connector.mysql.MySqlConnector',\n            'tasks.max': '1',\n            'database.hostname': source_db['host'],\n            'database.port': source_db['port'],\n            'database.user': source_db['username'],\n            'database.password': source_db['password'],\n            'database.server.name': 'my-source-database',\n            'database.include.list': source_db['database']\n        }\n\n    def start_capture(self):\n        \"\"\"\n        Start change data capture process\n        \"\"\"\n        # Configure Kafka producer for streaming changes\n        producer = Producer({\n            'bootstrap.servers': self.kafka_servers,\n            'client.id': 'cdc-change-producer'\n        })\n\n        # Set up Debezium connector\n        def handle_record(record):\n            \"\"\"\n            Process each captured change record\n            \"\"\"\n            # Transform record and publish to Kafka\n            change_event = {\n                'source': record.source(),\n                'operation': record.operation(),\n                'data': record.after()\n            }\n\n            producer.produce(\n                topic=\"database-changes\", \n                value=json.dumps(change_event)\n            )\n\n        # Start Debezium connector\n        debezium.start_connector(\n            config=self.debezium_config,\n            record_handler=handle_record\n        )\n\n# Example usage\nsource_database = {\n    'host': 'localhost',\n    'port': 3306,\n    'username': 'cdc_user',\n    'password': 'secure_password',\n    'database': 'customer_db'\n}\n\npipeline = CDCDataPipeline(\n    source_database, \n    kafka_bootstrap_servers=\"localhost:9092\"\n)\npipeline.start_capture()\n<\/code><\/pre>\n
      \n
      \n    \u0648\u0627\u0631\u062f \u062d\u0627\u0644\u062a \u062a\u0645\u0627\u0645 \u0635\u0641\u062d\u0647 \u0634\u0648\u06cc\u062f<\/title>\n    \n<\/svg><\/p>\n
          \u0627\u0632 \u062d\u0627\u0644\u062a \u062a\u0645\u0627\u0645 \u0635\u0641\u062d\u0647 \u062e\u0627\u0631\u062c \u0634\u0648\u06cc\u062f<\/title>\n    \n<\/svg><\/p>\n<\/div>\n<\/div>\n<\/div>\n
      \u0645\u0631\u0627\u062d\u0644 \u0627\u062c\u0631\u0627\u06cc \u062f\u0642\u06cc\u0642<\/strong><\/p>\n
        \n
      1. \u067e\u06cc\u06a9\u0631\u0628\u0646\u062f\u06cc \u0645\u0646\u0628\u0639 \u067e\u0627\u06cc\u06af\u0627\u0647 \u062f\u0627\u062f\u0647 \u0627\u0648\u0644\u06cc\u0646 \u0645\u0631\u062d\u0644\u0647 \u0634\u0627\u0645\u0644 \u067e\u06cc\u06a9\u0631\u0628\u0646\u062f\u06cc Debezium \u0628\u0631\u0627\u06cc \u0627\u062a\u0635\u0627\u0644 \u0628\u0647 \u067e\u0627\u06cc\u06af\u0627\u0647 \u062f\u0627\u062f\u0647 \u0645\u0646\u0628\u0639 \u0634\u0645\u0627 \u0627\u0633\u062a. \u0627\u06cc\u0646 \u0645\u0633\u062a\u0644\u0632\u0645:<\/li>\n<\/ol>\n
          \n
        • \u0645\u062c\u0648\u0632\u0647\u0627\u06cc \u06a9\u0627\u0631\u0628\u0631 \u067e\u0627\u06cc\u06af\u0627\u0647 \u062f\u0627\u062f\u0647 \u0645\u0646\u0627\u0633\u0628<\/li>\n
        • \u0627\u062a\u0635\u0627\u0644 \u0628\u0647 \u0634\u0628\u06a9\u0647<\/li>\n
        • \u0641\u0639\u0627\u0644 \u06a9\u0631\u062f\u0646 \u062b\u0628\u062a \u0628\u0627\u06cc\u0646\u0631\u06cc (\u0628\u0631\u0627\u06cc MySQL)<\/li>\n<\/ul>\n
            \n
          1. \u06a9\u0627\u0641\u06a9\u0627 \u0628\u0647 \u0639\u0646\u0648\u0627\u0646 \u06cc\u06a9 \u067e\u0644\u062a\u0641\u0631\u0645 \u062c\u0631\u06cc\u0627\u0646 \u0622\u067e\u0627\u0686\u06cc \u06a9\u0627\u0641\u06a9\u0627 \u0628\u0647 \u0639\u0646\u0648\u0627\u0646 \u06cc\u06a9 \u0648\u0627\u0633\u0637\u0647 \u067e\u06cc\u0627\u0645 \u0645\u0631\u06a9\u0632\u06cc \u0639\u0645\u0644 \u0645\u06cc \u06a9\u0646\u062f \u0648 \u0631\u0648\u06cc\u062f\u0627\u062f\u0647\u0627\u06cc \u062a\u063a\u06cc\u06cc\u0631 \u0631\u0627 \u062b\u0628\u062a \u0648 \u0630\u062e\u06cc\u0631\u0647 \u0645\u06cc \u06a9\u0646\u062f. \u0645\u0644\u0627\u062d\u0638\u0627\u062a \u06a9\u0644\u06cc\u062f\u06cc \u0639\u0628\u0627\u0631\u062a\u0646\u062f \u0627\u0632:<\/li>\n<\/ol>\n
              \n
            • \u067e\u06cc\u06a9\u0631\u0628\u0646\u062f\u06cc \u067e\u0627\u0631\u062a\u06cc\u0634\u0646 \u0647\u0627\u06cc \u0645\u0648\u0636\u0648\u0639\u06cc<\/li>\n
            • \u062a\u0646\u0638\u06cc\u0645 \u0633\u06cc\u0627\u0633\u062a \u0647\u0627\u06cc \u0646\u06af\u0647\u062f\u0627\u0631\u06cc \u0645\u0646\u0627\u0633\u0628<\/li>\n
            • \u067e\u06cc\u0627\u062f\u0647 \u0633\u0627\u0632\u06cc \u0645\u0639\u0646\u0627\u06cc\u06cc \u067e\u0631\u062f\u0627\u0632\u0634 \u062f\u0642\u06cc\u0642\u0627\u064b \u06cc\u06a9 \u0628\u0627\u0631<\/li>\n<\/ul>\n
                \n
              1. \u062a\u0628\u062f\u06cc\u0644 \u062f\u0627\u062f\u0647 \u0628\u0627 NiFi Apache NiFi \u0642\u0627\u0628\u0644\u06cc\u062a \u0647\u0627\u06cc \u0642\u062f\u0631\u062a\u0645\u0646\u062f \u0645\u0633\u06cc\u0631\u06cc\u0627\u0628\u06cc \u0648 \u062a\u0628\u062f\u06cc\u0644 \u062f\u0627\u062f\u0647 \u0631\u0627 \u0627\u0631\u0627\u0626\u0647 \u0645\u06cc \u062f\u0647\u062f:<\/li>\n<\/ol>\n
                  \n
                • \u0631\u0648\u06cc\u062f\u0627\u062f\u0647\u0627\u06cc \u062a\u063a\u06cc\u06cc\u0631 \u0645\u0633\u06cc\u0631 \u0648 \u0641\u06cc\u0644\u062a\u0631<\/li>\n
                • \u063a\u0646\u06cc \u0633\u0627\u0632\u06cc \u062f\u0627\u062f\u0647 \u0647\u0627 \u0631\u0627 \u0627\u0639\u0645\u0627\u0644 \u06a9\u0646\u06cc\u062f<\/li>\n
                • \u0645\u0646\u0637\u0642 \u062a\u062d\u0648\u0644 \u067e\u06cc\u0686\u06cc\u062f\u0647 \u0631\u0627 \u0645\u062f\u06cc\u0631\u06cc\u062a \u06a9\u0646\u06cc\u062f<\/li>\n<\/ul>\n
                  \u0686\u0627\u0644\u0634 \u0647\u0627 \u0648 \u0628\u0647\u062a\u0631\u06cc\u0646 \u0634\u06cc\u0648\u0647 \u0647\u0627<\/strong><\/p>\n
                    \n
                  1. \u0645\u062f\u06cc\u0631\u06cc\u062a \u062a\u063a\u06cc\u06cc\u0631\u0627\u062a \u0637\u0631\u062d\u0648\u0627\u0631\u0647: \u0627\u0633\u062a\u0631\u0627\u062a\u0698\u06cc \u0647\u0627\u06cc \u0642\u0648\u06cc \u062a\u06a9\u0627\u0645\u0644 \u0637\u0631\u062d\u0648\u0627\u0631\u0647 \u0631\u0627 \u0627\u062c\u0631\u0627 \u06a9\u0646\u06cc\u062f<\/li>\n
                  2. \u0628\u0647\u06cc\u0646\u0647 \u0633\u0627\u0632\u06cc \u0639\u0645\u0644\u06a9\u0631\u062f: \u0627\u0632 \u067e\u0627\u0631\u062a\u06cc\u0634\u0646 \u0628\u0646\u062f\u06cc \u0648 \u0641\u0634\u0631\u062f\u0647 \u0633\u0627\u0632\u06cc \u0645\u0646\u0627\u0633\u0628 \u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u06a9\u0646\u06cc\u062f<\/li>\n
                  3. \u0645\u062f\u06cc\u0631\u06cc\u062a \u062e\u0637\u0627: \u0645\u06a9\u0627\u0646\u06cc\u0632\u0645 \u0647\u0627\u06cc \u062c\u0627\u0645\u0639 \u0631\u062f\u06cc\u0627\u0628\u06cc \u062e\u0637\u0627 \u0648 \u062a\u0644\u0627\u0634 \u0645\u062c\u062f\u062f \u0631\u0627 \u067e\u06cc\u0627\u062f\u0647 \u0633\u0627\u0632\u06cc \u06a9\u0646\u06cc\u062f<\/li>\n<\/ol>\n
                    \u0645\u062e\u0632\u0646 GitHub<\/strong><\/p>\n
                    \u0645\u0646 \u06cc\u06a9 \u0646\u0645\u0648\u0646\u0647 \u067e\u06cc\u0627\u062f\u0647 \u0633\u0627\u0632\u06cc \u0627\u06cc\u062c\u0627\u062f \u06a9\u0631\u062f\u0647 \u0627\u0645 \u06a9\u0647 \u0645\u06cc \u062a\u0648\u0627\u0646\u06cc\u062f \u0622\u0646 \u0631\u0627 \u0628\u0631\u0631\u0633\u06cc \u06a9\u0631\u062f\u0647 \u0648 \u0628\u0647 \u0639\u0646\u0648\u0627\u0646 \u0645\u0631\u062c\u0639 \u0627\u0632 \u0622\u0646 \u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u06a9\u0646\u06cc\u062f. \u06a9\u062f \u06a9\u0627\u0645\u0644 \u0648 \u0627\u0633\u0646\u0627\u062f \u0627\u0636\u0627\u0641\u06cc \u0631\u0627 \u0645\u06cc \u062a\u0648\u0627\u0646\u06cc\u062f \u062f\u0631 \u0622\u062f\u0631\u0633 \u0632\u06cc\u0631 \u0628\u06cc\u0627\u0628\u06cc\u062f:
                    \u0645\u062e\u0632\u0646 GitHub: https:\/\/github.com\/Angelica-R\/cdc-data-pipeline<\/p>\n
                    \u0646\u062a\u06cc\u062c\u0647 \u06af\u06cc\u0631\u06cc<\/strong>
                    \u0627\u06cc\u062c\u0627\u062f \u0631\u0627\u0647 \u062d\u0644 Change Data Capture \u0646\u06cc\u0627\u0632\u0645\u0646\u062f \u0637\u0631\u0627\u062d\u06cc \u062f\u0642\u06cc\u0642 \u0645\u0639\u0645\u0627\u0631\u06cc \u0648 \u0627\u0646\u062a\u062e\u0627\u0628 \u0627\u0628\u0632\u0627\u0631 \u0645\u0646\u0627\u0633\u0628 \u0627\u0633\u062a. \u0628\u0627 \u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u0627\u0632 Debezium\u060c Kafka \u0648 NiFi\u060c \u0645\u06cc\u200c\u062a\u0648\u0627\u0646\u06cc\u062f \u06cc\u06a9 \u067e\u0644\u062a\u0641\u0631\u0645 \u06cc\u06a9\u067e\u0627\u0631\u0686\u0647\u200c\u0633\u0627\u0632\u06cc \u062f\u0627\u062f\u0647\u200c\u0647\u0627\u06cc \u0645\u0642\u06cc\u0627\u0633\u200c\u067e\u0630\u06cc\u0631 \u0648 \u0642\u0648\u06cc \u0627\u06cc\u062c\u0627\u062f \u06a9\u0646\u06cc\u062f \u06a9\u0647 \u0628\u06cc\u0646\u0634\u200c\u0647\u0627\u06cc \u0628\u06cc\u200c\u062f\u0631\u0646\u06af \u062f\u0631\u0628\u0627\u0631\u0647 \u062a\u063a\u06cc\u06cc\u0631\u0627\u062a \u062f\u0627\u062f\u0647\u200c\u0647\u0627\u06cc \u0634\u0645\u0627 \u0627\u0631\u0627\u0626\u0647 \u0645\u06cc\u200c\u062f\u0647\u062f.<\/p>\n<\/p><\/div>\n","protected":false},"excerpt":{"rendered":"
                    Summarize this content to 400 words in Persian Lang Change Data Capture (CDC) \u0628\u0647 \u06cc\u06a9 \u062a\u06a9\u0646\u06cc\u06a9 \u062d\u06cc\u0627\u062a\u06cc \u0628\u0631\u0627\u06cc \u06cc\u06a9\u067e\u0627\u0631\u0686\u0647 \u0633\u0627\u0632\u06cc \u062f\u0627\u062f\u0647 \u0647\u0627\u06cc \u0645\u062f\u0631\u0646 \u062a\u0628\u062f\u06cc\u0644 \u0634\u062f\u0647 \u0627\u0633\u062a \u06a9\u0647 \u0628\u0647 \u0633\u0627\u0632\u0645\u0627\u0646 \u0647\u0627 \u0627\u062c\u0627\u0632\u0647 \u0645\u06cc \u062f\u0647\u062f \u062a\u063a\u06cc\u06cc\u0631\u0627\u062a \u062f\u0627\u062f\u0647 \u0647\u0627 \u0631\u0627 \u062f\u0631 \u0633\u06cc\u0633\u062a\u0645 \u0647\u0627\u06cc \u0645\u062e\u062a\u0644\u0641 \u062f\u0631 \u0632\u0645\u0627\u0646 \u0648\u0627\u0642\u0639\u06cc \u0631\u062f\u06cc\u0627\u0628\u06cc \u0648 \u0627\u0646\u062a\u0634\u0627\u0631 \u062f\u0647\u0646\u062f. \u062f\u0631 \u0627\u06cc\u0646 \u0645\u0642\u0627\u0644\u0647\u060c \u0646\u062d\u0648\u0647 \u0633\u0627\u062e\u062a \u06cc\u06a9 \u0631\u0627\u0647 …<\/p>\n","protected":false},"author":2,"featured_media":88727,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"fifu_image_url":"","fifu_image_alt":"","footnotes":""},"categories":[339],"tags":[],"class_list":["post-88726","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-dev"],"_links":{"self":[{"href":"https:\/\/nabfollower.com\/blog\/wp-json\/wp\/v2\/posts\/88726","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/nabfollower.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/nabfollower.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/nabfollower.com\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/nabfollower.com\/blog\/wp-json\/wp\/v2\/comments?post=88726"}],"version-history":[{"count":0,"href":"https:\/\/nabfollower.com\/blog\/wp-json\/wp\/v2\/posts\/88726\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/nabfollower.com\/blog\/wp-json\/wp\/v2\/media\/88727"}],"wp:attachment":[{"href":"https:\/\/nabfollower.com\/blog\/wp-json\/wp\/v2\/media?parent=88726"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/nabfollower.com\/blog\/wp-json\/wp\/v2\/categories?post=88726"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/nabfollower.com\/blog\/wp-json\/wp\/v2\/tags?post=88726"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}