Add support for HA and multishard functionality in import APIs

[shivaji-kharse]

Description

Please explain the changes you made here.

Checklist

• Code compiles correctly and linting passes locally
• For all code changes, an entry added to the CHANGELOG.md file describing and linking to
  this PR
• Tests added for new functionality, or regression tests for bug fixes added as applicable
• For public APIs, new features, etc., PR on
  docs repo staged and linked here

Instructions

• The PR title should follow the Conventional Commits
  syntax, leading with fix:, feat:, chore:, ci:, etc.
• The description should briefly explain what the PR is about. In the case of a bugfix, describe or
  link to the bug.
• In the checklist section, check the boxes in that are applicable, using [x] syntax.
  • If not applicable, remove the entire line. Only leave the box unchecked if you intend to come
    back and check the box later.
• Delete the Instructions line and everything below it, to indicate you have read and are
  following these instructions. 🙂

Thank you for your contribution to Dgraph!

[Copilot]

Pull Request Overview

This PR adds support for high availability and multishard functionality in the import APIs by introducing a new streaming mechanism for propagating partition directory (P dir) updates across nodes. Key changes include:

• Adding new stream processing functions and a helper for managing ongoing P dir tasks.
• Introducing a new RPC (ReqPDirStream) and related message and client/server implementations.
• Updating tests and client code to leverage the new streaming workflow.

Reviewed Changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
worker/import.go	Adds streamProcessorSender support, a new stream task check, and updates the acknowledgment flow for stream processing.
worker/draft.go	Introduces a new opStreamPDir constant and string conversion for task identification.
protos/pb/pb_grpc.pb.go	Implements the new ReqPDirStream RPC handler methods and client stream handling.
protos/pb.proto	Defines the new ReqPDirStreamRequest message and RPC in the Worker service.
edgraph/server.go	Removes legacy drain mode invocation in favor of the new stream-based approach.
dgraph/cmd/dgraphimport/import_test.go	Updates test parameters and introduces a longer fixed sleep period post-import.
dgraph/cmd/dgraphimport/import_client.go	Refactors stream data logic to use the new RunBadgerStream method from the worker package.

Comments suppressed due to low confidence (1)

worker/import.go:319

• The removal of sending the acknowledgment using stream.SendAndClose in this function appears to be intentional given the updated flow in InStream. Please verify that the ACK signal is being sent exactly once during stream processing to avoid protocol inconsistencies.

return nil

[trunk-io]

     

View Full Report ↗︎ ⋅ Docs

[trunk-io]

Running Code Quality on PRs by uploading data to Trunk will soon be removed. You can still run checks on your PRs using trunk-action - see the migration guide for more information.