Skip to content
This repository was archived by the owner on Dec 21, 2023. It is now read-only.
This repository was archived by the owner on Dec 21, 2023. It is now read-only.

Kafka service goes down and never up again. #138

@superpep

Description

@superpep

This is happening to me since today. I have all my services running and can access control center without problem. After 10-15 minutes, every service is up (ZooKeeper, ksqlDB, etc) but Kafka is down.
I run confluent local services kafka start and it doesn't start. It says: Error: Kafka failed to start.

The only thing that solves it is running confluent local destroy but this obviously deletes everything so it doesn't really solve anything.

Log output:

[2021-03-23 12:13:46,022] INFO AuditLogConfig values:
        confluent.security.event.logger.authentication.enable = false
        confluent.security.event.logger.cloudevent.codec = structured
        confluent.security.event.logger.enable = true
        confluent.security.event.logger.exporter.class = class io.confluent.security.audit.telemetry.exporter.NonBlockingKafkaExporter
        confluent.security.event.logger.exporter.kafka.topic.create = true
        confluent.security.event.logger.exporter.kafka.topic.partitions = 12
        confluent.security.event.logger.exporter.kafka.topic.replicas = 1
        confluent.security.event.logger.exporter.kafka.topic.retention.bytes = -1
        confluent.security.event.logger.exporter.kafka.topic.retention.ms = 2592000000
        confluent.security.event.logger.exporter.kafka.topic.roll.ms = 14400000
        confluent.security.event.router.cache.entries = 10000
        confluent.security.event.router.config =
 (io.confluent.security.audit.AuditLogConfig)
[2021-03-23 12:13:46,022] INFO AuditLogConfig values:
        confluent.security.event.logger.authentication.enable = false
        confluent.security.event.logger.cloudevent.codec = structured
        confluent.security.event.logger.enable = true
        confluent.security.event.logger.exporter.class = class io.confluent.security.audit.telemetry.exporter.NonBlockingKafkaExporter
        confluent.security.event.logger.exporter.kafka.topic.create = true
        confluent.security.event.logger.exporter.kafka.topic.partitions = 12
        confluent.security.event.logger.exporter.kafka.topic.replicas = 1
        confluent.security.event.logger.exporter.kafka.topic.retention.bytes = -1
        confluent.security.event.logger.exporter.kafka.topic.retention.ms = 2592000000
        confluent.security.event.logger.exporter.kafka.topic.roll.ms = 14400000
        confluent.security.event.router.cache.entries = 10000
        confluent.security.event.router.config =
 (io.confluent.security.audit.AuditLogConfig)
[2021-03-23 12:13:46,458] INFO CrnAuthorityConfig values:
        confluent.authorizer.authority.cache.entries = 10000
        confluent.authorizer.authority.name =
 (io.confluent.crn.CrnAuthorityConfig)
[2021-03-23 12:13:46,461] INFO CrnAuthorityConfig values:
        confluent.authorizer.authority.cache.entries = 10000
        confluent.authorizer.authority.name =
 (io.confluent.crn.CrnAuthorityConfig)
[2021-03-23 12:13:46,808] INFO Awaiting socket connections on 0.0.0.0:9092. (kafka.network.Acceptor)
[2021-03-23 12:13:46,871] INFO [SocketServer brokerId=1001] Created data-plane acceptor and processors for endpoint : ListenerName(PLAINTEXT) (kafka.network.SocketServer)
[2021-03-23 12:13:46,907] INFO [ExpirationReaper-1001-topic]: Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2021-03-23 12:13:46,942] INFO [ExpirationReaper-1001-DeleteRecords]: Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2021-03-23 12:13:46,942] INFO [ExpirationReaper-1001-ElectLeader]: Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2021-03-23 12:13:46,942] INFO [ExpirationReaper-1001-Fetch]: Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2021-03-23 12:13:46,942] INFO [ExpirationReaper-1001-Produce]: Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2021-03-23 12:13:46,943] INFO [ExpirationReaper-1001-ListOffsets]: Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2021-03-23 12:13:46,969] INFO [LogDirFailureHandler]: Starting (kafka.server.ReplicaManager$LogDirFailureHandler)
[2021-03-23 12:13:46,970] WARN [ReplicaManager broker=1001] Stopping serving replicas in dir /tmp/confluent.092905/kafka/data (kafka.server.ReplicaManager)
[2021-03-23 12:13:46,989] WARN [ReplicaManager broker=1001] Broker 1001 stopped fetcher for partitions  and stopped moving logs for partitions  because they are in the failed log directory /tmp/confluent.092905/kafka/data. (kafka.server.ReplicaManager)
[2021-03-23 12:13:46,989] WARN Stopping serving logs in dir /tmp/confluent.092905/kafka/data (kafka.log.LogManager)
[2021-03-23 12:13:46,993] ERROR Shutdown broker because all log dirs in /tmp/confluent.092905/kafka/data have failed (kafka.log.LogManager)
[2021-03-23 12:15:22,251] INFO Registered kafka:type=kafka.Log4jController MBean (kafka.utils.Log4jControllerRegistration$)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions