Skip to content

Test all major features #96

@cereallarceny

Description

@cereallarceny

Description

We want to test all major features of the federated learning process in this repo. This includes:

  • Mocking PyGrid in order to run the test suite without the use of another server running (these endpoints will likely be what you want to mock)
  • Testing a flow with no authentication
  • Testing a flow with basic JWT authentication (failing)
  • Testing a flow with basic JWT authentication (passing)
  • Testing the acceptance of a worker in an FL cycle request
  • Testing the timeout for the re-application of an FL cycle request
  • Testing the graceful handling of an error whereby a plan or model does not exist in PyGrid from what is being requested
  • Testing the creation of one job
  • Testing the creation of multiple jobs
  • Testing that a job can be completed successfully
  • Testing that multiple jobs can be completed successfully
  • Testing that errors in a job can be caught by the onError handler and handled appropriately
  • Testing that we can successfully report a model diff to PyGrid
  • Any other related tests, or permutations of the above tests, that are required to pass for a developer to successfully use this library... in other words, be thorough and test everything you can think of that a developer might do.

Here's a rough list of the units to test:

  • JobStatusSubscriber
  • SocketClient
  • WebRTC Client
  • WebSocketClient
  • DeviceMonitor
  • Plan
  • SyftJob
  • ResponseRequestType

Type of Test

  • Unit test (e.g. checking a loop, method, or function is working as intended)
  • Integration test (e.g. checking if a certain group or set of functionality is working as intended)
  • Regression test (e.g. checking if by adding or removing a module of code allows other systems to continue to function as intended)
  • Stress test (e.g. checking to see how well a system performs under various situations, including heavy usage)
  • Performance test (e.g. checking to see how efficient a system is as performing the intended task)
  • Other...

Expected Behavior

We expect all the above tests to pass.

Additional Context

None

Metadata

Metadata

Assignees

Labels

Priority: 2 - High 😰Should be fixed as quickly as possible, ideally within the current or following sprintSeverity: 2 - High 😰Does not cause a failure, but causes the system to be incorrect, incomplete, or inconsistentStatus: Available 👋Available for assignment, who wants it?Type: Testing 🧪Add testing or improving existing testing of a file, feature, or codebase

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions