Skip to content
This repository was archived by the owner on Mar 4, 2020. It is now read-only.

chore: add visual-test route#106

Closed
levithomason wants to merge 5 commits intomasterfrom
chore/reduce-visual-tests
Closed

chore: add visual-test route#106
levithomason wants to merge 5 commits intomasterfrom
chore/reduce-visual-tests

Conversation

@levithomason
Copy link
Member

@levithomason levithomason commented Aug 17, 2018

We currently separately test every example for every component. We have consumed over 50% of our 20,000 images per month in just a few days.

This PR introduces a new route visual-test that renders all the examples for a single component on one page. It also renders an RTL version.

The markup is very minimal, a couple divs and headers to indicate which examples are being rendered. Now, we can just take a single image of this route.


http://localhost:8080/visual-test/button

image

@codecov
Copy link

codecov bot commented Aug 17, 2018

Codecov Report

Merging #106 into master will not change coverage.
The diff coverage is n/a.

Impacted file tree graph

@@           Coverage Diff           @@
##           master     #106   +/-   ##
=======================================
  Coverage   88.37%   88.37%           
=======================================
  Files          45       45           
  Lines         757      757           
  Branches      109      109           
=======================================
  Hits          669      669           
  Misses         85       85           
  Partials        3        3

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 62ac7b5...e096aeb. Read the comment docs.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug fix for mergeThemes, see additional tests.

@levithomason
Copy link
Member Author

/cc @miroslavstastny @kuzhelov

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are still looping over all examples here! That means that if a component has 10 examples, you will trigger the test for the whole component 10 times and end up having 10 same screenshots in screener.io

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed, thanks!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Before this change mergeComponentVariables(undefined, undefined) returned and empty object, now it returns undefined - as a result if component has no variables, it receives variables=undefined in it's componentStyles(). Not sure this is the best practice.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This still returns an object because the "initial" value is an object, which is the initial acc value.

I've added a test that asserts this just be sure:

  test('always returns an object', () => {
    expect(mergeThemes(undefined, undefined)).toMatchObject({})
    expect(mergeThemes(null, undefined)).toMatchObject({})
    expect(mergeThemes(undefined, null)).toMatchObject({})
    expect(mergeThemes(null, null)).toMatchObject({})

    expect(mergeThemes({}, undefined)).toMatchObject({})
    expect(mergeThemes(undefined, {})).toMatchObject({})
    expect(mergeThemes({}, {})).toMatchObject({})

    expect(mergeThemes({}, null)).toMatchObject({})
    expect(mergeThemes(null, {})).toMatchObject({})
    expect(mergeThemes({}, {})).toMatchObject({})
  })

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's true for mergeThemes(), but not for mergeComponentVariables() which is called directly in renderComponent():

expect(mergeComponentVariables(undefined, undefined)()).toMatchObject({})

received value must be an object.
Received: undefined

For that reason, Header component fails to render in the branch. (The fact that it uses a variable and doesn't have any headerVariables is another story, will talk to @kuzhelov about that).

@miroslavstastny
Copy link
Member

I like it. The only problem I see is the screener.io diff if anything in the middle changes it's size.
In the screenshot bellow, I removed ButtonExampleContentAndIcon.tsx. This change is correctly detected by screener, but bellow this change, screener detects several layout changes. This can make reviewing changes difficult.

image

Source: https://screener.io/v2/states/rkGqtccHX.stardust-ui-react/(default)/1024x768/Chrome/button-1-1-0

@levithomason
Copy link
Member Author

The only problem I see is the screener.io diff if anything in the middle changes it's size.

I've fixed the height of each example to 100vh to prevent the offset jitter issues.

@levithomason levithomason force-pushed the chore/reduce-visual-tests branch from f8ae8c4 to c5e7636 Compare August 21, 2018 00:06
@levithomason levithomason force-pushed the chore/reduce-visual-tests branch from c5e7636 to e096aeb Compare August 21, 2018 18:22
@levithomason
Copy link
Member Author

Even with the updates, we will still have lots of breaking changes that are hard to review to due added/removed/renamed example files.

We cannot sort examples by created date without considerable updates to the build. This information would be needed at render time where we only have the Webpack require.context() information. In order to resolve created times, we'd need to pre-build this information or pass it into the view somehow.

For now, screener is still running despite having maxed out our quota. Also, I have saved many test cycles by changing the CI config to skip testing redundant and stale commits.

At this point, I think it is worth it to keep our current setup and assess the cost of the service instead of degrading our ability to rely on it.

@kuzhelov kuzhelov added needs reviewer feedback question Further information is requested, concerns that require additional thought are raised and removed needs reviewer feedback labels Aug 22, 2018
@levithomason
Copy link
Member Author

No longer needed. We now have unlimited image comparisons on Screener.io 🎉

@levithomason levithomason deleted the chore/reduce-visual-tests branch September 14, 2018 04:29
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

question Further information is requested, concerns that require additional thought are raised

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants