Skip to content

ConfChange silently converted to no-op without error notification #354

@SherlockShemol

Description

@SherlockShemol

What happened?

When a ConfChange proposal is rejected (e.g., due to an existing pending config change), the proposal is silently converted to a no-op entry without returning any error. This silent failure can lead to security issues where administrators believe a configuration change succeeded when it was actually ignored.

What did you expect to happen?

The caller should receive an error notification when their ConfChange is rejected, allowing them to:

  • Detect when their ConfChange was rejected
  • Take appropriate action (e.g., wait and retry)
  • Log warnings or alerts

How can we reproduce it (as minimally and precisely as possible)?

r := newTestRaft(1, 10, 1, newTestMemoryStorage(withPeers(1, 2)))
r.becomeCandidate()
r.becomeLeader()

// First ConfChange - accepted
r.Step(pb.Message{From: 1, To: 1, Type: pb.MsgProp, 
    Entries: []pb.Entry{{Type: pb.EntryConfChange}}})

// Second ConfChange - silently converted to no-op, err == nil!
err := r.Step(pb.Message{From: 1, To: 1, Type: pb.MsgProp, 
    Entries: []pb.Entry{{Type: pb.EntryConfChange}}})

// BUG: err is nil, but ConfChange was actually rejected!
fmt.Println(err) // <nil>

Security Impact

  1. Compromised Node Cannot Be Removed: Admin tries to remove a compromised node, operation "succeeds" (no error), but node remains in cluster
  2. Configuration Change DoS: Attacker can block all legitimate config changes by keeping one pending
  3. Quorum Confusion: Admin thinks cluster configuration is X, but it's actually Y

Anything else we need to know?

The relevant code is in raft.go around line 1331:

if failedCheck != "" && !r.disableConfChangeValidation {
    r.logger.Infof("%x ignoring conf change %v at config %s: %s", ...)
    m.Entries[i] = pb.Entry{Type: pb.EntryNormal}  // Silent conversion!
    // No error returned!
}

etcd version

main branch (latest)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions