NP licensing add functional tests by mshustov · Pull Request #53002 · elastic/kibana

mshustov · 2019-12-13T16:19:19Z

Summary

This PR:

Adds core_provider_plugin, providing access to the platform API in the browser.
Updates platform test to use core_provider_plugin instead of extending a global window object.
Adds an integration test for the client-side licensing plugin. This test is placed in functional tests because we need a way to control the elasticsearch state.
Separates integration tests for the licensing plugin & legacy xpack plugin to simplify further cleanup.
Creates 3 different configs for each test suit. Since elasticsearch supports downgrading license and running a trial only once.

I used the LP plugin to have access to all plugin contracts. It will stop working when we drop LP support. We will need to find a way to test a plugin + a combination of plugins in the integration tests. Requires a separate issue

Checklist

Use ~~strikethroughs~~ to remove checklist items you don't feel are applicable to this PR.

~~[ ] This was checked for cross-browser compatibility, including a check against IE11~~
~~[ ] Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support~~
~~[ ] Documentation was added for features that require explanation or tutorials~~
Unit or functional tests were updated or added to match the most common scenarios
~~[ ] This was checked for keyboard-only and screenreader accessibility~~

For maintainers

This was checked for breaking API changes and was labeled appropriately
This includes a feature addition or change that requires a release note and was labeled appropriately

elasticmachine · 2019-12-16T12:09:23Z

Pinging @elastic/kibana-platform (Team:Platform)

pgayvallet

We will need to find a way to test a plugin + a combination of plugins in the integration tests

A test plugin per scenario that have the plugin it needs to test/interact with as dependencies would not work?

test/plugin_functional/config.js

test/plugin_functional/plugins/core_provider_plugin/types.ts

pgayvallet · 2019-12-16T17:00:32Z

x-pack/test/licensing_plugin/legacy/index.ts

+    // MUST BE LAST! CHANGES LICENSE TYPE!
+    loadTestFile(require.resolve('./updates'));


🙈 Wonder what we could actually do without changing the whole test runner mechanism...

test/plugin_functional/test_suites/core_plugins/ui_plugins.ts

pgayvallet · 2019-12-16T18:52:25Z

x-pack/test/licensing_plugin/public/updates.ts

+          // this call enforces signature check to detect license update
+          // and causes license re-fetch
+          await window.np.setup.core.http.get('/');
+          await window.np.testUtils.delay(100);


Question: What is the upside of exposing that kind of static utils from the globally exposed np instead of static import?

Do you mean this?

import { delay } from '...'; await browser.executeAsync(async (cb: Function) => { await window.np.setup.core.http.get('/'); await delay(100); ... });

Selenium serializes a function passed into browser.executeAsync(fn) to run it in a browser. Any closures will be lost.

Oh yea right. That's indeed an excellent answer.

x-pack/test/licensing_plugin/public/updates.ts

x-pack/test/licensing_plugin/scenario.ts

mshustov · 2019-12-17T10:48:53Z

A test plugin per scenario that have the plugin it needs to test/interact with as dependencies would not work?

It would work unless we need to change the elasticsearch state. In theory, we can call elasticsearch API from a plugin as well if we make the security model available in the browser as well. https://github.com/elastic/kibana/pull/53002/files#diff-286f825e99754fbdf390bfeedaff15f6R21

mshustov · 2019-12-17T13:21:58Z

@pgayvallet updated

pgayvallet · 2019-12-17T14:22:51Z

test/plugin_functional/test_suites/core_plugins/ui_plugins.ts

+          await browser.execute(() => {
+            const { setup } = ((window as unknown) as CoreProvider).__coreProvider;
+            return setup.plugins.core_plugin_b.sayHi();


I feel bad about this now that I know that vscode doesn't behave as IDEA regarding declare statement. This is indeed less elegant that the previous solution. We should decide with the team if we want to 'allow' these declare statements for extensions that are not necessarily accessible from everywhere in the code.

ok, I will revert this change

elasticmachine · 2019-12-19T09:58:04Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request
Commit: d06e46d

History

💚 Build #16066 succeeded 7a8de50
💚 Build #15719 succeeded 942d8c9
💚 Build #15669 succeeded f3ffacf
💚 Build #15475 succeeded d19e67a
💔 Build #15441 failed 9483684

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

* fix comment * introduce core provider plugin for integration tests * platform functional tests use core_provider_plugin for testing * add 3 scenario for licensing plugins: server, client, legacy * remove unused code * run all licensing_plugin tests on CI * remove duplicated config * address comments * declare global type for core provider

This reverts commit a4c5b13.

This reverts commit f464641.

* fix comment * introduce core provider plugin for integration tests * platform functional tests use core_provider_plugin for testing * add 3 scenario for licensing plugins: server, client, legacy * remove unused code * run all licensing_plugin tests on CI * remove duplicated config * address comments * declare global type for core provider

…f github.com:mbondyra/kibana into IS-51910_share-lens-change-index-pattern-in-discover * 'IS-51910_share-lens-change-index-pattern-in-discover' of github.com:mbondyra/kibana: [Discover] Remove angular field filter template code (elastic#53513) [APM] Improve table and other panel loading states (elastic#53459) Security/Spaces - cleanup react warnings (elastic#53287) Revert "NP licensing add functional tests (elastic#53002)" (elastic#53577) NP licensing add functional tests (elastic#53002) fix onLicenseInfoChange callback to be called on update (elastic#53559) Document how to extend request handler context (elastic#53271)

* NP licensing add functional tests (#53002) * fix comment * introduce core provider plugin for integration tests * platform functional tests use core_provider_plugin for testing * add 3 scenario for licensing plugins: server, client, legacy * remove unused code * run all licensing_plugin tests on CI * remove duplicated config * address comments * declare global type for core provider * remove potentially dangerous operation Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

* NP licensing add functional tests (elastic#53002) * fix comment * introduce core provider plugin for integration tests * platform functional tests use core_provider_plugin for testing * add 3 scenario for licensing plugins: server, client, legacy * remove unused code * run all licensing_plugin tests on CI * remove duplicated config * address comments * declare global type for core provider * remove potentially dangerous operation Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

* NP licensing add functional tests (#53002) * fix comment * introduce core provider plugin for integration tests * platform functional tests use core_provider_plugin for testing * add 3 scenario for licensing plugins: server, client, legacy * remove unused code * run all licensing_plugin tests on CI * remove duplicated config * address comments * declare global type for core provider * remove potentially dangerous operation Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

* NP licensing add functional tests (elastic#53002) * fix comment * introduce core provider plugin for integration tests * platform functional tests use core_provider_plugin for testing * add 3 scenario for licensing plugins: server, client, legacy * remove unused code * run all licensing_plugin tests on CI * remove duplicated config * address comments * declare global type for core provider * remove potentially dangerous operation Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

mshustov added 9 commits December 13, 2019 17:14

fix comment

5d74ce6

introduce core provider plugin for integration tests

89f3449

platform functional tests use core_provider_plugin for testing

b74a8c8

add 3 scenario for licensing plugins: server, client, legacy

9483684

Merge branch 'master' into np-licensing-functional-tests

4f5c072

remove unused code

d19e67a

Merge branch 'master' into np-licensing-functional-tests

05a97fc

run all licensing_plugin tests on CI

f3ffacf

remove duplicated config

942d8c9

mshustov added chore Feature:New Platform release_note:skip Skip the PR/issue when compiling release notes Team:Core Platform Core services: plugins, logging, config, saved objects, http, ES client, i18n, etc t// v7.6.0 labels Dec 16, 2019

mshustov changed the title ~~Np licensing functional tests~~ NP licensing add functional tests Dec 16, 2019

mshustov marked this pull request as ready for review December 16, 2019 12:10

mshustov requested a review from a team December 16, 2019 12:10

elasticmachine mentioned this pull request Dec 16, 2019

License plugin #18356

Closed

12 tasks

pgayvallet reviewed Dec 16, 2019

View reviewed changes

mshustov added 2 commits December 17, 2019 11:49

Merge branch 'master' into np-licensing-functional-tests

70ff9df

address comments

7a8de50

mshustov requested a review from pgayvallet December 17, 2019 13:22

pgayvallet approved these changes Dec 17, 2019

View reviewed changes

mshustov added 3 commits December 19, 2019 08:31

Merge branch 'master' into np-licensing-functional-tests

7c7617c

declare global type for core provider

cc4943c

Merge branch 'master' into np-licensing-functional-tests

d06e46d

mshustov merged commit a4c5b13 into elastic:master Dec 19, 2019

mshustov deleted the np-licensing-functional-tests branch December 19, 2019 09:59

mshustov mentioned this pull request Dec 19, 2019

[7.x] NP licensing add functional tests (#53002) #53569

Merged

mshustov added the backported label Dec 19, 2019

mshustov added a commit that referenced this pull request Dec 19, 2019

Revert "NP licensing add functional tests (#53002)"

257161d

This reverts commit a4c5b13.

mshustov mentioned this pull request Dec 19, 2019

Revert "NP licensing add functional tests" #53577

Merged

mshustov added a commit that referenced this pull request Dec 19, 2019

Revert "NP licensing add functional tests (#53002)" (#53577)

4ea2599

This reverts commit a4c5b13.

mshustov added a commit that referenced this pull request Dec 19, 2019

Revert "NP licensing add functional tests (#53002) (#53569)"

d1fb7e4

This reverts commit f464641.

mshustov added a commit that referenced this pull request Dec 19, 2019

Revert "NP licensing add functional tests (#53002) (#53569)" (#53579)

5931dce

This reverts commit f464641.

This was referenced Dec 19, 2019

Licensing plugin functional tests #53580

Merged

Failing test: License plugin API Integration Tests.x-pack/test/licensing_plugin/public/updates·ts - Licensing plugin public client changes in license types provides changes in license types #53575

Closed

		// MUST BE LAST! CHANGES LICENSE TYPE!
		loadTestFile(require.resolve('./updates'));

Conversation

mshustov commented Dec 13, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Checklist

For maintainers

Uh oh!

elasticmachine commented Dec 16, 2019

Uh oh!

pgayvallet left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

pgayvallet Dec 16, 2019

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

pgayvallet Dec 16, 2019

Choose a reason for hiding this comment

Uh oh!

mshustov Dec 17, 2019

Choose a reason for hiding this comment

Uh oh!

pgayvallet Dec 17, 2019

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

mshustov commented Dec 17, 2019

Uh oh!

mshustov commented Dec 17, 2019

Uh oh!

pgayvallet Dec 17, 2019

Choose a reason for hiding this comment

Uh oh!

mshustov Dec 19, 2019

Choose a reason for hiding this comment

Uh oh!

elasticmachine commented Dec 19, 2019

💚 Build Succeeded

History

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

mshustov commented Dec 13, 2019 •

edited

Loading